Data Engineer with Azure

Introduction & Summary: The objective of this engagement is to provide Data Engineers to execute the technical implementation of the analytical platform. This role focuses on covering key aspects such as data ingestion, transformation, and orchestration processes that align with the designed architecture and business requirements. The ideal candidate will have a minimum of 3 years of experience in data inflow and transformation processes in a cloud environment, with practical exposure to advanced technologies.Main Responsibilities:Ingesting data to the bronze layerData anonymizationCreating orchestration flowData platform initial setupCreating and testing ETL/ELT processes in the data platformGenerating data transformations between the bronze and silver layers (including data validation, duplicate removal, and merging data from different sources)Implementing monitoring and alerting for data pipelinesOptimizing data storage and retrieval for cost and performanceDocumenting all processes and configurations for maintainabilityRecommending necessary changes to the production system to ensure high-quality data for analytics.Key Requirements:Minimum 3 years of experience in creating data inflow and transformation processes (pipelines) in a cloud environmentPractical experience with technologies:Azure Stack (ADLS, Azure Blob Storage)Databricks, Delta Lakehouse, DBT, GITKnowledge of orchestration tools: Apache Airflow or similar solutionsKnowledge of PySparkVery good knowledge of ETL/ELT processesAdvanced knowledge of SQL, especially in query optimization and performanceB2 English levelNice to Have:Experience with big data technologiesFamiliarity with data security practicesUnderstanding of data governance principles