Senior Python Engineer

SummaryThe Senior Python Engineer role is focused on building a modern, scalable Data Platform using Microsoft Azure and Fabric. The primary objective is to design reusable libraries, define standards, lead a team, and ensure long-term support for a high-impact data platform.Main Responsibilities:Design and build reusable Python + SQL libraries/frameworks for data ingestion and transformation.Strong hands-on experience with Microsoft Azure cloud services.Architect event-driven ingestion pipelines using Kafka and Azure Functions.Implement robust data transformation and loading frameworks (batch and near real-time).Apply strong data warehousing concepts (fact/dimension modeling, SCDs, incremental loads, partitioning, indexing).Develop scalable data loading strategies into warehouse/lake environments.Build and manage APIs for data exposure and integration.Lead CI/CD implementation using GitHub (Actions), including automated testing, versioning, and deployments.Ensure production readiness: monitoring, logging, alerting, and operational support.Guide architecture decisions focused on performance, scalability, and cost optimization on Azure.Mentor team members and lead code reviews and best practices adoption.Key Requirements:10+ years of experience in Python.Strong hands-on expertise in Python (advanced) and SQL (advanced).Understanding of Data Warehousing concepts.Experience with Kafka in production environments.Hands-on experience with Azure services (Azure Functions, ADLS, Storage, App Services, etc.).Good understanding of Azure architecture and cost analysis/optimization.Experience building and deploying APIs and applications in Azure.Strong experience with GitHub and CI/CD pipelines.Experience leading small teams or owning modules end-to-end.Nice to Have:Experience with DuckDB (preferred but NOT mandatory).Experience with modern lakehouse technologies.Containerization, Infrastructure as Code (Terraform/Bicep).Observability tools (App Insights, monitoring frameworks).Exposure to AI/ML data preparation pipelines.Other Details:Location: Pune Hybrid modelTeam Structure: Collaborative work with Data Engineering and AI teams.Reporting Line: Reports to the Head of Data Engineering.Project Context: Focus on building a modern enterprise data platform.