Job description

About the Role:

We are looking for an experienced Data Engineer to join our delivery team working for a leading financial client in the UK. This is an exciting opportunity to contribute to the development of a scalable and secure data platform leveraging Databricks, Apache Spark, and Azure. The ideal candidate will be comfortable working in regulated environments and building robust pipelines that support critical analytics and reporting needs.

Key Responsibilities:

• Build, optimize, and maintain data pipelines using Apache Spark and Databricks

• Develop and deploy ETL workflows using PySpark and Python

• Write clean, optimized SQL queries for complex data transformations

• Work closely with data analysts, data scientists, and business users to deliver well-structured, reliable datasets

• Implement data modeling best practices for both batch and streaming data

• Leverage Azure services to manage data storage, orchestration, and compute workloads

• Ensure data quality, lineage, and compliance within a financial services context

• Contribute to solution architecture and participate in code reviews and Agile ceremonies

Requirements:

• Strong experience with Apache Spark (Advanced)

• Proficient in Databricks, PySpark, Python, and SQL (Regular)

• Hands-on experience working with Azure cloud services

• Solid understanding of data modeling and data warehousing principles

• Experience working with financial or highly regulated datasets (preferred)

• Strong communication and stakeholder collaboration skills

Nice to Have:

• Experience with Delta Lake, Unity Catalog, or Azure Synapse

• Familiarity with data governance, security, and compliance standards (e.g., GDPR, FCA)

• Exposure to CI/CD, Git, or workflow/orchestration tools (e.g., Azure Data Factory)

• Knowledge of data quality frameworks (e.g., Great Expectations)

What We Offer:

• Opportunity to work on a high-impact project in the financial domain

• Access to cutting-edge technologies and cloud infrastructure

• Collaborative, diverse, and inclusive team environment

• Flexible working model (remote)