What you will do

We are looking for an experienced Data Engineer to join the Shelf Analytics project – a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights.

As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.

Design, develop, and maintain data pipelines and workflows using Databricks and PySpark

Read, understand, and extend existing codebases; independently develop new components for Databricks workflows

Implement object-oriented Python solutions (classes, inheritance, reusable modules)

Develop and maintain unit tests to ensure code quality and reliability

Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries

Create and manage Databricks workflows, clusters, databases, and tables

Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions

Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions

Support continuous improvement of data engineering standards, performance, and scalability

Requirements

MUST

Strong programming skills in Python and PySpark

Hands-on experience with Databricks (workflows, clusters, tables, databases)

Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio

Experience with pandas, dbx, and unit testing frameworks

Practical experience working with Azure Storage (ADLS) and access control (ACLs)

Proficiency with GitHub, including CI/CD pipelines and GitHub Actions

Ability to work independently, analyze existing solutions, and propose improvements

NICE TO HAVE

Experience with retail, CPG, or shelf analytics–related solutions

Familiarity with large-scale data processing and analytics platforms

Strong communication skills and a proactive, problem-solving mindset