Data Engineer (AWS)

Technologies we use

About the project

Your responsibilities

• Build reusable, metadata-driven data pipelines.
• Automate and optimize data platform processes.
• Develop integrations with data sources and consumers.
• Extend shared ETL libraries with transformation methods.
• Write unit tests.
• Create monitoring solutions for the Databricks platform.
• Proactively address ETL performance and quality issues.
• Collaborate with infrastructure teams on cloud resources.
• Update data platform wiki and documentation.
• Conduct code reviews to ensure quality.
• Initiate and implement architecture improvements.

Our requirements

• Proficiency in Python/PySpark and SQL
• Strong experience building robust data pipelines using Apache Spark (e.g., on AWS EMR)
• Proven track record handling large, complex datasets
• Expertise in developing reusable data transformation libraries (Python packages)
• Deep knowledge of data lake optimization techniques (e.g., partitioning, compaction, file sizing, performance tuning)
• Hands-on experience with CI/CD pipeline development
• Experience with AWS services such as S3, Glue, EMR, and Lambda
• Skilled in event streaming integration using Kafka or Amazon Kinesis
• Solid understanding of fundamental networking concepts

This is how we organize our work

This is how we work

Team members

This is how we work on a project

What we offer

• 100 Remote work model.
• Superior co-working and personal development experience in an international setting.