Senior Data Engineer

Information about the project:Industry: bankingRate: up to 155 pln/h net + VAT, B2BLocation: Warsaw/Tricity/Łódź - hybridSummaryThe role involves a senior data engineer or IT developer working in the Client, primarily focusing on implementing data requirements for the Group Financial Crime Prevention unit. The primary objective is to develop and maintain data processing pipelines that support advanced analytics and ensure the integrity and transformation of large datasets.Main Responsibilities:Building distributed and highly parallelized Big Data processing pipelines for massive data processing in near real-time.Leveraging Spark for data enrichment and transformation aimed at facilitating data visualization and advanced analytics.Collaborating closely with DevOps, QA, and Product Management teams within a Continuous Delivery environment.Key Requirements:Minimum 5 years of experience in Scala and Spark.Knowledge of the Big Data Ecosystem (Hadoop stack: YARN, EMR, Sqoop, Hive, etc.).Basic experience with Kafka.Experience with Impala.Familiarity with Jenkins, JIRA, Bitbucket, and Git.Nice to Have:Strong knowledge of Unit testing.Understanding of code versioning and branching strategies.Familiarity with Agile/Safe frameworks.Experience with Snowflake in an AWS environment.Experience implementing data security measures like encryption and anonymization.Excellent communication skills and experience working with distributed global teams.Data-driven thinking capabilities.Prior experience in Financial Services is a plus