Requirements

Expected technologies

SQLPython

Java

Scala

BigQuery

Teradata

Google Cloud Platform

Optional technologies

Spark

AWS

Kafka

Kubernetes

Operating system

Windows

Our requirements

• 2+ years of relevant Data Engineering experience
• Classical Data Warehouse, Data structures, Data marked, Data profiling
• Experience writing advanced SQL
• Experience with one of Python, Java or Scala
• ETL
• Experience with GCP BigQuery or Teradata
• Basic experience with BigData

Optional

• Experience with Spark
• Other cloud computing Experience (e.g AWS)
• Knowledge of Kafka for event streaming and real-time data pipelines
• Familiarity with Kubernetes for container orchestration

Your responsibilities

• Data Platform Development: Design, develop, and maintain data pipelines and workflows on distributed data platforms such as BigQuery, Hadoop/EMR/DataProc, or Teradata.
• Cloud Integration: Build and optimize cloud-based solutions using AWS or GCP to process and store large-scale datasets.
• Workflow Orchestration: Design and manage workflows and data pipelines using Apache Airflow to ensure scalability, reliability, and maintainability.
• Containerization and Orchestration: Deploy and manage containerized applications using Kubernetes for efficient scalability and resource management.
• Event Streaming: Work with Kafka to implement reliable and scalable event streaming systems for real-time data processing.
• Programming and Automation: Write clean, efficient, and maintainable code in Python and SQL to automate data processing, transformation, and analytics tasks.
• Database Management: Design and optimize relational and non-relational databases to support high-performance querying and analytics.
• System Monitoring & Troubleshooting: Participate in the on-call rotation to monitor systems, address incidents, and ensure the reliability of production environments.
• Collaboration: Work closely with cross-functional teams, including data scientists, analysts, and product managers, to understand data requirements and deliver solutions that meet business objectives.
• Participate in code reviews, technical discussions, and team collaboration to deliver high-quality software solutions