Czym będziesz się zajmować? Design, develop, and maintain robust data pipelines using PySpark in a scalable Big Data environment.Contribute to system architecture and design discussions.Participate in Agile ceremonies, including sprint planning, reviews, and retrospectives.Collaborate with Business Analysts to ensure accurate interpretation and implementation of requirements.Implement tools and processes that ensure performance, scalability, availability, and accuracy.Promote best practices through code reviews, mentoring, and knowledge sharing within the team.Provide production support, including troubleshooting and resolving data pipeline issues.Maintain clear documentation and participate in planning and status meetings

Kogo poszukujemy? 5+ years of experience in software design and PySpark/Scala development.Strong experience in developing and optimizing data pipelines using Hadoop components (Hive, Spark, Spark SQL).Hands-on experience with scheduling tools like Apache Airflow.Proficiency with Big Data technologies: Apache Spark, YARN, Hive, Python, SQL, RESTful APIs, MapReduce.Solid understanding of working in Unix/Linux environments.Experience with version control and CI/CD tools: Git/GitHub, Ansible, Jenkins.Familiarity with Agile project management tools such as JIRA.Strong understanding of both relational and non-relational data modeling techniques.Proven debugging and code analysis skills with the ability to clearly communicate findings to the development team.