Data Engineer Lead (OT Data) (Oil & Gas)

At Codvo, we are committed to building scalable, future-ready data platforms that power business impact. The Data Engineer will be a foundational architect of our delivery strategy, responsible for building and maintaining the data ecosystem required for our scoped-out use cases. This involves bridging operational technology (OT) and information technology (IT) in the oil and gas sector, building high-performance data pipelines, and creating reliable, scalable, and secure data products that enable AI models and analytics dashboards.

Responsibilities

Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines; Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians (like OSIsoft PI, Aspen InfoPlus.21), DCS, PLC, and IoT sensors; Cloud Data Platform Development: Implement and manage data solutions on the Microsoft Azure cloud platform, leveraging Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics for real-time ingestion and processing of OT data; Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets; Enable Advanced AI: Build data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data; Champion Master Data Management (MDM): Design and implement MDM strategies and solutions for critical data domains such as wells, equipment, and assets; Ensure Data Quality & Governance: Implement robust data quality checks, validation rules, and monitoring; Embrace Industry Standards: Champion and implement industry-specific data standards such as the OSDU™ Data Platform; Collaborate & Innovate: Work closely with geoscientists, drilling engineers, data scientists, and business analysts; Automate & Optimize: Identify opportunities for process automation and infrastructure optimization; Security First: Implement and maintain security best practices

Requirements

Bachelor's in engineering, Information Systems, or a related quantitative field; 6+ years of experience in a data engineering role; 5+ years of proven experience in a data engineering role; Demonstrable experience building and operationalizing large-scale data pipelines and applications; Expert-level proficiency in SQL and Python for data manipulation and pipeline development; Hands-on experience with distributed computing frameworks like Apache Spark (PySpark); Deep experience with Microsoft Azure (Azure Data Lake Storage, Azure Data Factory, Azure Databricks, Azure Synapse); Proven experience with modern data platforms Databricks Delta Lake; Understanding of machine learning lifecycles and the data requirements for training and deploying AI/ML models; Experience with workflow orchestration tools like Airflow, Dagster, or Azure Data Factory; Strong understanding of both relational (e.g., PostgreSQL, SQL Server) and NoSQL databases; Proficiency with Git and CI/CD best practices

Nice to have

Experience within oil and gas industry is highly preferred; Experience with streaming technologies like Kafka; Experience with graph databases (e.g., Neo4j) and vector databases is highly desirable; Familiarity with historian systems (e.g., OSIsoft PI System) and their data structures; Hands-on experience with the OSDU™ Data Platform; Experience working with industrial communication protocols (e.g., OPC UA, Modbus TCP/IP); Understanding of cybersecurity considerations for OT environments and data segregation; Experience integrating data from ERP systems like SAP; Professional certifications in Azure or Databricks; Advanced SQL skills, including query optimization and performance tuning; Knowledge of containerization technologies (Docker, Kubernetes); Experience of constructing and maintaining enterprise knowledge graphs