Requirements

Expected technologies

Python

SQL

PySpark

Azure Data Lake Gen2

Azure Data Factory

Databricks

GitHub VCS

GitHub Actions

Azure DevOps Pipelines

Our requirements

• Bachelor/Master’s degree in Computer Science, Engineering, or a related field.
• 5+ years of working experience in the field of Data & Analytics, preferably in the CPG industry
• 5+ years of proficient coding experience with Python for data engineering, including SQL and PySpark (DataFrame API, Spark SQL, MLlib), with hands-on experience in various databases (SQL/NoSQL), key libraries (e.g., pandas, SQLAlchemy), parallel processing, and advanced data transformation and performance optimization techniques.
• Ability to develop, and maintain data engineering applications using Python classes and PySpark, ensuring code modularity, reusability, and maintainability.
• Excellent data engineering & technology knowledge (Azure Data Lake Gen2, Azure Data Factory and Databricks as well as data management knowhow (data cataloguing, data quality management)
• Solid understanding of data modeling, ETL processing and lakehouse concepts.
• Profound knowledge of CI/CD processes and tools (GitHub VCS, GitHub Actions, Azure DevOps Pipelines))
• Profound data content knowledge of market data (Sell-out, Sell-through) for the CPG or health care industry which is licensable from data providers like IQVIA, Nielsen and IRI.
• Advanced understanding of data governance processes to support mapping of data from multiple sources.
• Experience in leading developer teams and breaking down work into smaller work packages, experience in Agile methodologies (Scrum, Kanban) in order to manage the work.
• Strong problem solving and analytical skills. Excellent interpersonal and communication skills, team collaboration, active listening, consulting, challenging in a constructive way.
• Fluent in English, both written & spoken, intercultural awareness and willingness to travel.

Your responsibilities

• Implement efficient data pipelines that integrate data from different sources and business domains to develop globally harmonized data models and KPI calculations
• Contribute to define data management and data quality standards. Ensure that data is well-managed to build stable, reusable and quality assured data assets.
• Collaborate with other IT functions (enabling functions data asset teams, analytics teams, platform product managers & integration architects) to ensure the aforementioned activities are executed effectively.
• Ensure that data products adhere to the data protection and compliance standards. Implement effective data access management policies, data privacy policies and secure data provisioning based on corporate guidelines.
• Continuously enhance implementation frameworks based on the needs of the analytics products consuming the data assets in your responsibility.
• Guide other data engineers in your team (internal or external engineers) and ensure that all engineers apply same design principles and assure code quality. Break down bigger work packages into manageable tasks.
• Together with the assigned data architect, ensure that cost and time estimations are accurate, quality of delivery is assured, and deliverables are properly tested, documented and handed over to the operations team.