Lead / Senior Data Engineer

Diggibyte is looking for an experienced Lead / Senior Data Engineer to design, build, and optimize modern data platforms that power business intelligence, analytics, and AI-driven solutions. The ideal candidate will have strong expertise in cloud-based data engineering, scalable ETL/ELT pipelines, and the Databricks Lakehouse Platform. If you are passionate about solving complex data challenges and delivering high-performance solutions, we’d love to hear from you.

Responsibilities

Lead the design, development, and maintenance of scalable data engineering solutions on the Databricks Lakehouse Platform; Develop, optimize, and maintain high-performance data pipelines using Python and PySpark; Build reliable data transformation workflows and analytical models using DBT; Design scalable ETL/ELT pipelines to process structured and unstructured data efficiently; Work extensively with Databricks, PySpark, and SQL to deliver robust data solutions; Leverage cloud platforms including AWS, Microsoft Azure, or Google Cloud Platform (GCP) for modern data architectures; Implement data governance, metadata management, and security using Unity Catalog; Collaborate with data scientists, analysts, architects, and business teams to understand data requirements; Ensure data quality, performance optimization, and adherence to best engineering practices; Mentor junior engineers and contribute to technical leadership within the team

Requirements

Databricks Lakehouse Platform; Python; PySpark; SQL; DBT (Data Build Tool); AWS / Microsoft Azure / Google Cloud Platform; Unity Catalog; Data Warehousing & ETL/ELT; Data Modeling; Version Control (Git); 4–12 years of professional experience in Data Engineering

Nice to have

Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field; Strong understanding of cloud-native data platforms and distributed data processing; Experience with performance tuning and optimization of large-scale data pipelines; Excellent analytical, problem-solving, and communication skills; Databricks Certified Data Engineer Associate or Professional certification; Experience with Databricks Auto Loader; Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines; Exposure to CI/CD practices for data engineering projects; Familiarity with data governance and security best practices

Benefits

Work on cutting-edge cloud and data engineering projects; Collaborate with experienced technology professionals; Opportunity to work with modern data platforms and AI-ready architectures; Continuous learning and professional development; Competitive salary and excellent career growth opportunities; Flexible and collaborative work environment