Aplikuj teraz

GenAI Azure Data Engineer

Pragmile

Warszawa, Ochota
32000 zł/mth.
Zdalna
🐍 Python
SQL
Terraform
PySpark
CI/CD
☁️ Azure Services
🌐 Zdalna

Requirements

Expected technologies

Python

SQL

Terraform

PySpark

CI/CD

Azure Services

Our requirements

  • Core Azure Expertise
  • Advanced SQL, PySpark, and Python.
  • Infrastructure-as-Code (Terraform, ARM/Bicep).
  • Performance tuning (partitioning, indexing, query optimization).
  • Azure Services:
  • Data Factory (ADF), Synapse, Databricks, Cosmos DB.
  • DevOps (Azure Pipelines, Repos), Monitor, and Security Center.
  • GCP Proficiency (nice to have):
  • BigQuery (partitioned tables, materialized views).
  • Cloud Storage, Dataflow, Pub/Sub.
  • IAM and VPC networking.
  • Looker, Dataproc, Composer (Airflow).

Optional

  • Certification:
  • Azure: DP-203 (Data Engineer), AZ-400 (DevOps).
  • GCP: Professional Data Engineer (preferred).
  • Education:
  • Bachelors/Masters in CS/IT or equivalent.

Your responsibilities

  • Azure-Centric Data Engineering
  • Design and deploy Azure data platforms:
  • Data Lakes: Azure Data Lake Storage (ADLS Gen2) with Delta Lake optimizations.
  • ETL/ELT: Azure Data Factory, Synapse Analytics, and Databricks workflows.
  • Data Warehousing: Synapse Dedicated Pools, Azure SQL DB.
  • Optimize Spark workloads (e.g. Azure Databricks) for performance tuning (partitioning, caching).
  • Implement real-time pipelines: Azure Event Hubs, Stream Analytics, and IoT Hub integrations.
  • Ensure data governance:
  • Enforce data quality with Azure Purview (lineage tracking, sensitivity labeling).
  • Implement RBAC and Azure Active Directory (AAD) integration for secure access.
  • Build real-time analytics pipelines:
  • Process streaming data via Azure Event Hubs/Stream Analytics with windowing and watermarking strategies.
  • Operational Excellence
  • Automate CI/CD pipelines.
  • Lead disaster recovery (DR)
  • Manage data lifecycle
Wyświetlenia: 8
Opublikowanaokoło miesiąc temu
Wygasaza 23 dni
Tryb pracyZdalna
Źródło
Logo

Podobne oferty, które mogą Cię zainteresować

Na podstawie "GenAI Azure Data Engineer"