Data Engineer | Krakow

Work mode:hybrid 3 days/week in the office in Krakow/Wroclaw.

Job SummaryWe are looking for an experienced Azure Data Engineer with strong hands-on expertise in PySpark, Python, and Azure Data Factory (ADF) to design, develop, and maintain scalable data processing and ETL/ELT solutions.

The ideal candidate will have solid experience working with large-scale data processing, performance optimization, and modern data engineering architectures. Hands-on experience with Azure Databricks (ADB) is a significant advantage.

You will collaborate closely with Data Scientists, Data Analysts, Solution Architects, and other stakeholders to build reliable, high-performance data solutions that meet business and technical requirements.

Key Responsibilities

Design, develop, and maintain scalable and reliable data pipelines using PySpark and Python.

Develop and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.

Build efficient, reusable, maintainable, and high-performance data engineering solutions.

Develop, orchestrate, schedule, and monitor data workflows using Azure Data Factory (ADF).

Process and integrate data from multiple sources and ensure consistency across data platforms.

Apply data engineering best practices for performance, scalability, reliability, maintainability, and code quality.

Optimize Spark workloads through effective use of partitioning, caching, joins, query optimization, and resource utilization.

Identify and troubleshoot performance bottlenecks, pipeline failures, and data processing issues.

Ensure data quality, accuracy, reliability, and integrity across data platforms.

Collaborate with Data Scientists, Data Analysts, Solution Architects, and business stakeholders to understand requirements and deliver effective data solutions.

Contribute to data architecture and solution design discussions.

Participate in continuous improvement initiatives to enhance data platforms, pipelines, and engineering practices.

Follow established standards for documentation, testing, version control, and deployment.

Required Skills & Experience6–10 years of overall IT experience, with relevant experience in data engineering.4–5 years of hands-on experience with PySpark and Python.

Strong understanding of data engineering principles, distributed computing, and modern data pipeline architectures.

Strong hands-on experience with:

PySpark, including Spark SQL, DataFrames, and performance tuning.

Python scripting and application development.

Azure Data Factory (ADF), including pipeline development, orchestration, scheduling, and monitoring.

Strong knowledge of data processing and optimization techniques, including:

Data partitioning

• Caching
• Join optimization
• Query optimization
• Spark performance tuning
• Experience working with large-scale distributed data processing systems.

Strong understanding of SQL, relational databases, database concepts, and data modeling.

Strong analytical, troubleshooting, and problem-solving skills.

Ability to write clean, reusable, scalable, and maintainable code.

Good communication and collaboration skills, with the ability to work effectively across technical and business teams.

Nice-to-Have Skills

Hands-on experience with Azure Databricks (ADB).

Experience with other Microsoft Azure data services.

Exposure to CI/CD pipelines and DevOps practices for data engineering.

Experience with Data Lake and Lakehouse architectures.

Familiarity with modern data governance, data quality, monitoring, and observability practices.

Experience with version control systems such as Git.

Exposure to automated testing and deployment of data pipelines.