Data Engineer / Senior Data Engineer with Azure, Databricks and Microsoft Fabric

Technology stack

Azure Data Factory, Azure Databricks, Microsoft Fabric, Azure Data Lake Storage Gen2, Delta Lake, Apache Spark, PySpark, Python, SQL Server, Azure DevOps, Git, Power BI, Microsoft Purview, GitHub Copilot

ClientOur client is a UK financial services group operating in a highly regulated environment.

Project overview

This project focuses on building a modern enterprise data and analytics platform on Microsoft Azure as part of a large scale modernization program. A legacy policy administration system is being replaced with a SaaS solution, while reporting and analytics capabilities are being migrated from a traditional SQL Server environment to Databricks and Microsoft Fabric. The platform will support governed data products, advanced analytics, and AI assisted engineering practices.

Position overviewWe are looking for a Senior Data Engineer with strong experience in Azure, Databricks, and Microsoft Fabric to help design and build a modern cloud data platform. You will contribute to the ingestion, transformation, and curation layers of the platform while collaborating with technical and business stakeholders. In this role, you will also support engineering standards, platform design decisions, and the delivery of scalable data solutions.

Responsibilities

Design and develop data ingestion pipelines from business systems into a governed cloud data platform.

Build scalable data engineering solutions using Azure Databricks and Microsoft Fabric.

Implement medallion architecture patterns and reusable engineering frameworks.

Develop curated data products and analytical datasets for business reporting and decision making.

Apply AI assisted development practices while ensuring quality, governance, and auditability.

Contribute to data modelling, semantic models, metadata management, and analytical solutions.

Support the migration of reporting and analytics workloads from legacy platforms.

Collaborate with stakeholders to translate business requirements into technical solutions.

Provide design guidance, establish engineering standards, and participate in code reviews.

Requirements6+ years of experience in Data Engineering.

Hands on experience with Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2.

Strong programming experience with Python, SQL, and PySpark.

Experience implementing Delta Lake and production lakehouse or medallion architectures.

Experience with Git and Azure DevOps, including CI/CD pipelines for data solutions.

Experience working with SQL Server environments.

Practical experience using AI coding assistants within software delivery projects.

Ability to communicate effectively in English with technical and business stakeholders.

Experience taking ownership of architecture or platform design decisions.

Experience reviewing code and establishing engineering standards.

Experience gathering requirements and collaborating directly with business stakeholders.

Nice to have

Experience implementing Microsoft Fabric in production environments.

Microsoft certifications including DP 203, DP 600, DP 700, AZ 900, or DP 900.

Databricks Data Engineer Associate or Databricks Data Engineer Professional certification.

Experience with dimensional modelling, including star schema, snowflake schema, or Data Vault.

Experience with semantic modelling in Power BI or Microsoft Fabric.

Knowledge of Microsoft Purview, data lineage, and metadata management.

Experience working in regulated financial services environments.

What We Offer:

Vacation days: Up to 26 business days per year.10 illness/special days off per year (fully paid, no medical papers needed) for all contract types

Health and life insurance (Luxmed)

MyBenefit platform with Multisport option

Internal psychological support service

English language classes from the first working dayAccess to external learning platforms: O’

Reilly, LinkedIn Learning, Udemy, and a wide catalog of diverse internal training

• Flexible workplace: work from the office, from home, or choose a hybrid optionTech Skills Mentoring Program
• Opportunities to develop as a public speaker, mentor, or technical interviewer
• Fully paid idle (bench) when not involved in a project
• Certification reimbursement (AWS, GCP, Microsoft, etc.)