About the project

• Correct Context is looking for a Java/Spark Big Data Developer for Comscore in Poland and around.
• This project is a 6-month engagement with the possibility of extending the cooperation upon its completion
• Comscore is a global leader in media analytics, revolutionizing insights into consumer behavior, media consumption, and digital engagement.
• Comscore leads in measuring and analyzing audiences across diverse digital platforms. Thrive on using cutting-edge technology, play a vital role as a trusted partner delivering accurate data to global businesses, and collaborate with industry leaders like Facebook, Disney, and Amazon. Contribute to empowering businesses in the digital era across media, advertising, e-commerce, and technology sectors.

Your responsibilities

• Taking care of our high-volume (tens of billions of events daily), multi-tenant, and distributed data processing pipelines
• Develop and maintain Java-based and Spark-based Big Data processing pipelines, ensuring scalability and performance
• Work with distributed systems, contributing to the design and implementation of backend solutions
• Collaborate with cross-functional teams to integrate different platforms and ensure smooth data flow
• Troubleshoot and debug issues to improve overall system stability
• Participate in code reviews and offer guidance on best practices
• Debug java/aws/airflow pipelines
• Help team to produce better solutions (teaching/code reviews/showing directions)
• Design high efficient processing pipelines

Our requirements

• Very good understanding of Linux internals (processes/threads, service management, resource allocation)
• Linux as a daily driver of operations
• Good understanding of Spark internals and mechanics
• Knowledge of Iceberg
• 4+ years of commercial experience with Java
• Proven ability to debug JVM/Java (hopefully in Spark ecosystem)
• Ability to design high performance processing pipelines
• Hands-on, proactive approach
• Familiarity with CI/CD pipelines and DevOps practices
• Proficiency in English enables effective communication in an international environment

Technologies we use

This is how we work on a project

• code review

• DevOps

• issue tracking tools

• integration tests

What we offer

• Real big data projects, petabytes scale 🚀

• An international team 🌎

• A small, independent team working environment 🧑‍💻

• Flexible work time ⏰

• Fully remote or in-office work in Wroclaw, Poland 🏢

• 18,000 - 25,000 PLN net/month B2B 💰

• Multikafeteria 🍽️

• Private healthcare 🏥

• Free parking 🚗