Senior Data Engineer

We are looking for a Senior Data Engineer to join our Data Exploration Service (DES) team, a platform that enables internal users to explore, process, analyze, and leverage large-scale data using JupyterLab, Apache Spark, and modern data engineering technologies. In this role, you will contribute to the development and operation of a cloud-native data platform running on Kubernetes-based on-premises infrastructure, while also supporting initiatives in AWS, Snowflake, and cloud-based data exploration environments. You will have opportunities to contribute to AI and GenAI-related initiatives, including the development and integration of AI-powered assistants and agents.

Responsibilities

Developing and maintaining data platform capabilities and services; Supporting Spark-based data processing and analytics workloads; Working with Kubernetes, containerized applications, and platform automation; Improving platform reliability, scalability, and user experience; Collaborating with data scientists, engineers, and platform teams; Contributing to cloud data initiatives involving AWS and Snowflake; Supporting AI/GenAI use cases, agent development, and data platform integrations for AI workloads

Requirements

Strong experience with Python and Apache Spark; Experience with Kubernetes, Docker, and Linux environments; Understanding of data engineering principles and distributed systems; Familiarity with cloud platforms, preferably AWS; Interest in AI, machine learning, or Generative AI technologies; Experience working with modern data platforms and analytics ecosystems