Sr. Data Platform Engineering

This is a Cactus Wellhead position and is located in Houston, TX. Onsite 4 days in office 1 working from home. No fully remote options. Job Summary: The Data Platform Lead is responsible for implementing, configuring, and governing the technical foundation of the company's enterprise data Lakehouse on Azure Databricks. This role combines hands-on data platform ownership with data engineering leadership, ensuring that data from ERP systems, product SaaS PostgreSQL databases, APIs, and other enterprise sources is ingested, modeled, governed, secured, and prepared for analytics, reporting, automation, and AI use cases. This position serves as the internal technical owner for Databricks platform standards, Bronze/Silver/Gold data architecture, Unity Catalog governance, pipeline design patterns, source-to-target mapping standards, data quality implementation, vendor technical review, and production readiness. The role is hands-on, with responsibility for platform configuration, environment setup, access controls, catalog/schema structure, compute standards, and operational readiness, while also providing technical directions to data engineers, contractors, vendors, and future internal team members through strong standards, practical architecture, and disciplined delivery.

Responsibilities

Own the technical architecture for the Azure Databricks Lakehouse; Design, build, and oversee production-grade data pipelines using Databricks, Spark, Python/PySpark, SQL, Delta Lake, and approved orchestration patterns; Define and maintain bronze, silver, and gold layer standards; Implement and govern Unity Catalog standards; Implement practical data quality controls; Support data governance by ensuring datasets have clear owners, stewards, classifications, lineage, refresh frequency, and required documentation before go-live; Enable certified silver and gold datasets; Own technical incident response for pipeline failures, data refresh issues, root cause analysis, and corrective actions; Define enterprise data architecture standards, reference architectures, and long-term roadmap; Act as the internal technical authority for Databricks platform and data engineering decisions; Provide direction to vendors, contractors, and future internal data engineers; Partner with Cactus IT Leadership team on execution, prioritization, architecture decisions, production risk, and vendor acceptance; Collaborate with analytics, business applications, ERP, product engineering, cybersecurity, infrastructure, and business stakeholders; Promote engineering discipline, documentation quality, reusable patterns, and operational excellence across the data function

Requirements

7+ years of experience in data engineering, data architecture, cloud data platforms, or enterprise data integration; Hands-on experience designing and supporting production data pipelines in cloud environments; Experience with modern Lakehouse architecture, preferably Azure Databricks and Delta Lake; Experience working with enterprise source systems such as ERP, CRM, SaaS applications, PostgreSQL, SQL Server, Oracle, or similar relational databases; Experience reviewing vendors or contractor technical deliverables and enforcing engineering standards; Strong expertise with Azure Databricks, Apache Spark, Python/PySpark, advanced SQL, Delta Lake, and Lakehouse design patterns; Working knowledge of Unity Catalog, RBAC, lineage, data classification, metadata, and access governance; Experience with Azure Data Lake Storage, GitHub or Azure DevOps, CI/CD, secrets management, and cloud integration patterns; Strong understanding of data modeling, dimensional modeling, normalized models, medallion architecture, data quality, and reconciliation; Ability to design ingestion patterns for ERP data, application databases, APIs, files, and incremental source changes; Ability to operate as both a hands-on technical lead and a manager of delivery standards; Strong judgment to challenge designs that are not secure, scalable, documented, or production ready; Strong communication skills with technical teams, business stakeholders, vendors, and leadership; Ability to translate business data needs into scalable platform and engineering solutions; Strong ownership mindset, documentation discipline, and ability to work in a growing data organization with evolving standards

Nice to have

Experience in manufacturing, oil & gas, field services, industrial operations, or ERP-heavy environments; Exposure to Power BI, Tableau, semantic models, reporting migration, or analytics product delivery; Exposure to ML/AI pipelines, feature engineering, GenAI use cases, automation, or AI-ready data product development; Experience with infrastructure-as-code, automated testing, data observability, or enterprise data catalog tools; Databricks, Azure Data Engineer, Azure Solutions Architect, or related cloud/data certifications