Data Engineer - Site Reliability

Digital Hub Warsaw at Bayer

Warszawa, Ochota
15 000–21 000 zł brutto / mies.
Praca hybrydowa
Umowa o pracę
Umowa o pracę
🏠 Praca hybrydowa
Pełny etat
🐍 Python
Golang
PostgreSQL
PostGIS
🔍 Google Cloud Vertex AI
☁️ AWS SageMaker
☁️ AWS
🚢 Kubernetes
Terraform
GCDM
CloudFormation
🐳 Docker compose
SQL

About the project

  • In our Location 360 team we believe that the location of things, and the relationships between them in time and space, are of fundamental importance to creating transformational digital products. We are passionate about enabling teams to seamlessly incorporate spatial and location data into their applications, analyses, operations, and models. We do this by ingesting and stewarding much of the location related data for Bayer Crop Science, by integrating models that enrich that data, and by building platforms and interfaces that make it easy for teams to integrate that data into their work. Our Imagery squad is looking for an experienced and innovative SRE/DevOps Engineer to join us.
  • As an SRE you will partner with Imagery squad engineers to bridge software engineering and system administration with a focus on ensuring the reliability, availability, and performance of software systems operating on common cloud infrastructures. You will proactively monitor cloud-enabled imagery pipelines and drive continuous improvement initiatives to gain efficiencies and minimize system down time. You’ll work closely with data engineers, cloud engineers, data stewards, software developers, data scientists and domain experts to build observable and scalable cloud-based infrastructures using modern cloud-native and open-source Infrastructure as Code (IaC) technologies.

Your responsibilities

  • Drive continual improvement in deployment, observability, monitoring, and scalability
  • Automate tasks to improve efficiency, manage infrastructure, and ensure performance of Loc360 assets that meet or exceed stated SLOs
  • Assist developers implementing scalable data pipelines in Python and Go for ingestion, transformation, and delivery of structured and unstructured geospatial data.
  • Write and review code, develop documentation, and debug complex problems between systems and components.
  • Optimize event-driven data processing solutions using Kafka, AWS SQS, and Google Cloud Pub/Sub to orchestrate multi-stage spatial workflows.
  • Integrate and manage data flows across cloud platforms such as AWS and GCP, databases such as PostgreSQL/PostGIS and BigQuery, and cloud storage such as AWS S3 and Google Cloud Storage
  • Leverage Kubernetes (K8s) for deploying and managing containerized applications and workflows.
  • Work closely with cloud engineers and platform engineers to integrate cloud infrastructure with CI/CD pipelines and standardize deployment patterns.
  • Collaborate with data engineers to optimize and monitor data pipelines and services for performance, reliability, scalability, and cost-effectiveness
  • Provide technical support, including incident response, troubleshooting and resolution for production issues in data pipelines and API services
  • Ensure compliance with company and industry standards and best practices for data security and regulatory requirements
  • Stay updated on emerging data engineering technologies and data infrastructures; evaluate their potential impact and application in our systems and processes
  • Provide technical leadership and mentorship to junior data engineers. Forster a culture of knowledge sharing and continuous learning
  • Participate in an on-call rotation (but rarely paged/alerted) and be able to occasionally work with flexible hours

Our requirements

  • Bachelor's degree in Computer Science, Engineering, or relevant job experience with 5+ years
  • Strong proficiency in Infrastructure as Code (IaC) principles and tools (i.e. Terraform, GCDM, CloudFormation, Docker compose, etc.)
  • Strong understanding and hands-on experience with CI/CD tools such as GitHub Actions, Terraform, Google Cloud Build, ArgoCD
  • Strong proficiency in Python and/or Go programming languages
  • Hands-on experience working with Kubernetes(K8s) for orchestrating and managing containerized data services and workflows.
  • Proven experience with SQL databases and/or BigQuery, including schema design and query optimization
  • Hands-on experience with event-driven and streaming data architectures using platform and services such as Kafka, AWS SQS, and Google Cloud Pub/Sub
  • Ability to work in a highly collaborative environment and to communicate effectively with internal and external partners
  • Experience working with and testing high-volume RESTful APIs deployed on common cloud Kubernetes infrastructures
  • Experience with utilizing Docker to build and deploy containerized applications
  • Experience with telemetry data flows and monitoring and logging tools such as Grafana, Prometheus, ELK stack, or equivalent.
  • Experience in cloud platforms such as GCP and AWS, including native data and compute services such as Bigquery/Aurora, GCS/S3, GKE/EKS, GCE/EC2, Cloud Functions/Lambda
  • Experience with code versioning and dependency management systems such as GitHub
  • Excellent problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.
  • Strong communication skills and the ability to articulate technical concepts to non-technical stakeholders.
  • Preferred:
  • Highly proficient (5+ years) in Python or Golang with a strong track record of maintaining production data pipelines and backend systems
  • Experience with PostgreSQL/PostGIS enabled databases
  • Experience scaling GRPC services, working with protobuf schemas and tooling such as BSR
  • Experience with object-oriented design, coding and testing patterns, and implementing complex data projects in a large-scale data infrastructure
  • Understanding of geospatial data concepts. Experience with data processing and analysis using geospatial libraries and tools
  • Familiarity with cloud-based machine learning services and platforms such as Google Cloud Vertex AI or AWS SageMaker. Experience with deploying invoking model endpoints.
  • Solid understanding of networking concepts, security principles, and best practices for cloud environments
  • Experience working with customers and developers to deliver full-stack development solutions; the ability to translate customer requirements into technical requirements in an Agile environment
  • You feel you do not meet all criteria we are looking for? That doesn’t mean you aren’t the right fit for the role. Apply with confidence, we value potential over perfection!

Technologies we use

What we offer

  • A flexible, hybrid work model

  • Great workplace in a new modern office in Warsaw

  • Career development, 360° Feedback & Mentoring programme

  • Wide access to professional development tools, trainings, & conferences

  • Company Bonus & Reward Structure

  • Increased tax-deductible costs for authors of copyrighted works

  • VIP Medical Care Package (including Dental & Mental health)

  • Life & Travel Insurance

  • Pension plan

  • Co-financed sport card - FitProfit

  • Meals Subsidy in Office

  • Budget for Home Office Setup & Maintenance

  • Access to Company Game Room equipped with table tennis, soccer table, Sony PlayStation 5 and Xbox Series X consoles setup with premium game passes, and massage chairs

  • Tailored-made support in relocation to Warsaw when needed

Aufrufe: 5
Veröffentlichtvor 2 Tagen
Läuft abin 28 Tagen
Art des VertragsUmowa o pracę
ArbeitsmodusPraca hybrydowa
Quelle
Logo
Logo

Ähnliche Jobs, die für Sie von Interesse sein könnten

Basierend auf "Data Engineer - Site Reliability"