Senior Databricks Engineer - Governance

Technologies we use

About the project

Your responsibilities

• Design and implement Unity Catalog for clients: metastore and catalog structure, workspace binding, external locations and storage credentials, identity federation (SCIM, service principals), ownership model.
• Build access control that reflects the governance design: privileges and groups, row filters and column masks, attribute-based access control with tags, data classification; test it and document it so that auditors can read it.
• Produce governance evidence from the platform: lineage, audit logs and billing from system tables, access reviews, dashboards for data owners and compliance teams; integrate with enterprise catalogs (Microsoft Purview, Collibra) where the client has them.
• Implement data quality in pipelines: expectations in Lakeflow pipelines, Lakehouse Monitoring, quality metrics published to the catalog and to the governance team.
• Govern AI assets on the platform with the AI governance team: models and functions registered in Unity Catalog, MLflow approval workflows, AI Gateway policies for LLM endpoints, logging for traceability.
• Implement sovereignty and residency controls with the sovereignty team: region placement, customer-managed keys, private connectivity, Delta Sharing and Lakehouse Federation configured so that data stays where it must.
• Migrate clients from Hive metastore and legacy permission models to Unity Catalog, and clean up what governance cannot see.
• Build reusable assets: Unity Catalog reference setups as code (Terraform, Databricks Asset Bundles), access policy patterns, evidence dashboards; contribute to the governance-by-design reference architecture.

Our requirements

• 5+ years of hands-on Databricks engineering in production, including at least two years designing and administering Unity Catalog at enterprise scale (several workspaces, many teams).
• Deep knowledge of Unity Catalog: privilege model, row and column security, tags and attribute-based policies, system tables, lineage, Delta Sharing, Lakehouse Federation; migration from Hive metastore.
• Strong Spark, SQL and Python; Delta Lake; Lakeflow pipelines and jobs; infrastructure as code with Terraform and Databricks Asset Bundles; CI/CD.
• Working knowledge of cloud security on Azure or AWS as it applies to Databricks: identity, storage access, private networking, key management.
• Databricks Certified Data Engineer Professional or Associate; Platform Administrator accreditation is a plus.
• Interest in data governance and regulation; experience with a governance or catalog tool (Purview, Collibra) or with GDPR, EU AI Act or sector-regulated data is a plus.
• Ability to explain an access model or a lineage graph to a non-technical data owner; fluent English.
• Experience in financial services, energy, pharma or public sector is a plus.

What we offer

• Practical benefits: yearly financial bonus, private medical care with Medicover with additional packages (e.g., dental, senior care, oncology) available on preferential terms, life insurance and access to NAIS benefit platform.
• Access to over 70 training tracks with certification opportunities (e.g., GenAI, Excel, Business Analysis, Project Management) on our NEXT platform. Dive into a world of knowledge with free access to Education First languages platform, TED Talks and Udemy Business materials and trainings.
• Cutting-Edge Technology: Position yourself at the forefront of IT innovation, working with the latest technologies and platforms. Capgemini partners with top global enterprises, including 145 Fortune 500 companies.
• Enjoy hybrid working model that fits your life - after completing onboarding, connect work from a modern office with ergonomic work from home, thanks to home office package (including laptop, monitor, and chair). Ask your recruiter about the details.

Benefits