Senior Data Engineer (Databricks)

We are looking for a Senior Data Engineer with strong expertise in Databricks and modern data platforms. The ideal candidate will be responsible for building scalable data pipelines, optimizing data workflows, and enabling data-driven decision-making across the organization.

Responsibilities

Design, develop, and maintain scalable data pipelines using Databricks and Python; Build and optimize ETL workflows for structured and unstructured data; Work with Databricks Lakehouse architecture and implement best practices; Manage and implement data governance using Unity Catalog; Integrate AWS data services such as S3, IAM, VPC, and other relevant services; Collaborate with cross-functional teams to deliver high-quality data solutions; Build dashboards and support data visualization requirements; Ensure data quality, reliability, and performance optimization

Requirements

8+ years of experience; Strong experience with Databricks platform; Proficiency in Python for data engineering and ETL pipelines; Hands-on experience with Unity Catalog; Good knowledge of AWS services (S3, IAM, VPC, Glue/Lambda – preferred); Solid understanding of Data Lake / Lakehouse architecture; Experience in building dashboards and reporting solutions

Nice to have

Experience with REST API development (Flask, FastAPI, etc.); Knowledge of authentication/authorization (OAuth, API keys, IAM roles); Strong query optimization and performance tuning skills; Experience with PySpark optimization; Exposure to ML/AI pipelines; Familiarity with Databricks AI/BI capabilities; Strong problem-solving and analytical skills; Excellent communication and collaboration abilities