Senior Data Engineer

We are seeking an experienced Senior Data Engineer with 8–10+ years of experience in designing, developing, and managing modern data platforms, data pipelines, and cloud-based analytics solutions. The ideal candidate will have strong expertise in Azure Data Services, large-scale data processing, data warehousing, ETL/ELT frameworks, and cloud-native data architectures. The role requires hands-on experience in building scalable, secure, and high-performance data solutions that support enterprise analytics, reporting, AI, and business intelligence initiatives.

Responsibilities

Design, develop, and maintain scalable data platforms and data pipelines on Microsoft Azure; Build and optimize batch and real-time data ingestion frameworks from multiple structured and unstructured data sources; Design and implement data lake, data warehouse, and lakehouse architectures to support analytics and reporting workloads; Develop and manage ETL/ELT processes using modern cloud-native data engineering practices; Implement data transformation, cleansing, validation, and quality frameworks to ensure data accuracy and reliability; Collaborate with business stakeholders, data analysts, data scientists, and application teams to understand data requirements and deliver scalable solutions; Optimize data storage, processing, and query performance across enterprise data platforms; Implement security, governance, monitoring, and compliance best practices across Azure environments; Support integration of data platforms with AI/ML, business intelligence, and enterprise applications; Participate in architecture reviews, code reviews, troubleshooting, and technical mentoring activities; Ensure high availability, scalability, and operational excellence of data platforms and pipelines

Requirements

8–10+ years of experience in Data Engineering, Data Warehousing, and Enterprise Data Platform development; Strong hands-on experience with Microsoft Azure Data Services; Expertise in Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), and Azure SQL Database; Experience building and managing large-scale ETL/ELT pipelines and data integration solutions; Strong proficiency in SQL, query optimization, and database performance tuning; Hands-on experience with PySpark, Apache Spark, and distributed data processing frameworks; Strong programming skills in Python, Scala, or Java; Experience with dimensional modeling, data warehousing concepts, and modern lakehouse architectures; Experience working with structured, semi-structured, and unstructured data; Strong understanding of data governance, data quality, metadata management, and security best practices; Experience with REST APIs, data integration patterns, and enterprise system connectivity; Hands-on experience with Git, CI/CD pipelines, and DevOps practices; Strong analytical, problem-solving, and communication skills

Nice to have

Experience with Microsoft Fabric, OneLake, Dataflows, and Fabric Data Engineering workloads; Experience with Databricks, Delta Lake, and lakehouse implementations; Knowledge of real-time streaming technologies such as Azure Event Hubs, Apache Kafka, or Azure Stream Analytics; Experience supporting AI/ML and advanced analytics workloads through enterprise data platforms; Familiarity with Power BI datasets, semantic models, and enterprise reporting architectures; Experience with data governance tools such as Microsoft Purview; Microsoft Azure Data Engineering certifications are highly preferred; Experience working in Agile/Scrum environments; Experience with Microsoft Fabric Data Engineering and Analytics solutions; Exposure to MLOps and DataOps practices; Knowledge of containerization technologies such as Docker and Kubernetes; Experience with Infrastructure as Code (Terraform, ARM Templates, or Bicep); Familiarity with Snowflake, BigQuery, or other cloud data warehouse platforms; Experience with enterprise-scale data migration and modernization projects; Understanding of Generative AI, Vector Databases, and data platforms supporting AI workloads