Senior Data Engineer – AWS/Databricks/PythonPrimary Responsibilities: Design, develop and maintain data pipelines to extract data from a variety of sources and populate data lake and data warehouse Develop the various data transformation rules and data modelling capabilities Collaborate with Product Analyst, Data Scientists, and Machine Learning Engineers to identify and transform data to make data understandable. Experience in implementing data quality checks, maintain data catalogues, use orchestration, logging, and monitoring tools, data analysis, information architecture concept and implementation, Git for version control and understand various branching strategies

Use test driven development methodology when building ELT/ETL pipelines

Required Experience and Skills: min 5 yrs of relevant experience Solid experience with AWS services (S3, IAM, Redshift, Sagemaker, Glue, Lambda, Step Functions, CloudWatch) Experience with platforms like Databricks, Dataiku Proficient in Python / Java, SQL – Redshift preferred, Jenkins, CloudFormation, Terraform, Git, Docker, 2-3 years of Spark – PySpark Good communication and SDLC documentation skills.