Senior Databricks Engineer

Key ResponsibilitiesDesign, develop, and maintain scalable and robust data pipelines that collect, process, and store large volumes of structured and unstructured data.Architect and implement data warehouses, data lakes, and other storage solutions that support analytics and reporting needs.Optimize data architectures and workflows for performance, scalability, and cost-efficiency.Collaborate with data scientists and analysts to understand data requirements and ensure that data systems meet their needs.Ensure data quality, integrity, and security by implementing best practices in data governance and management.Develop and maintain documentation for data systems, including data models, flow diagrams, and operational procedures.Mentor and guide junior data engineers, providing technical leadership and support.Stay current with emerging technologies and trends in data engineering, and apply them to improve existing systems.QualificationsBachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.5+ years of experience in data engineering or related fields, with a focus on large-scale data systems3+ years of experience with DatabricksProficiency in SQL and experience with databases such as MySQL, PostgreSQL, or NoSQL databasesStrong programming skills in languages such as Python, Java, or Scala.Experience with ETL tools and frameworks and data pipeline orchestration.Familiarity with big data technologies (e.g., Hadoop, Spark, Kafka) and cloud platforms (AWS, GCP, Azure).Strong understanding of data warehousing concepts, data modeling, and schema design.Excellent problem-solving skills and attention to detail.Strong communication skills and ability to work effectively in a team-oriented environment