Lead Data Platform Engineer – Apache Iceberg & PySpark

Technologies we use

About the project

This is how we organize our work

This is how we work

Your responsibilities

• Lead the design, development, and implementation of data pipelines and table management workflows using PySpark and Apache Iceberg.
• Drive performance and cost optimization for Apache Iceberg by refining partitioning strategies, file sizing, and query efficiency.
• Manage and improve Apache Iceberg maintenance operations, including snapshot management, compaction, and metadata cleanup.
• Troubleshoot platform issues by applying deep understanding of Apache Iceberg internals such as metadata layout, schemas, and schema evolution.
• Promote engineering excellence through code reviews, automation, and best practices in CI/CD and observability.
• Support operational reliability by participating in on-call duties, incident response, and root cause analysis.
• Collaborate with cross-functional teams, influence architectural decisions, and mentor less experienced engineers.
• Partner with stakeholders to ensure solutions are secure, scalable, and compliant.

Our requirements

• 7+ years of experience in data engineering or related roles.
• Strong hands-on expertise in PySpark, including DataFrame APIs, performance tuning, and job optimization.
• Practical experience with Apache Iceberg, especially in optimizing file layout, partitioning, and understanding internal architecture.
• Proven ability to automate maintenance operations and operate production data workloads independently.
• Solid software engineering fundamentals with a focus on reliable, maintainable systems.
• Excellent communication skills in English and the ability to influence across geographies.
• Ownership mindset with a focus on end-to-end delivery, discipline, and initiative.

Optional

• Certifications or additional skills in Big Data, Cloud platforms, or related fields.

What we offer

• Stable and long-term cooperation with very good conditions
• Enhance your skills and develop your expertise in the financial industry
• Work on the most strategic projects available in the market
• Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years
• Participate in Social Events, training, and work in an international environment
• Access to attractive Medical Package
• Access to Multisport Program
• Access to Pluralsight
• Flexible hours

Benefits

#GETREADY to meet with us!

ITDS’s Whistleblower Procedure