Principal Data Engineer

CodaMetrix is revolutionizing Revenue Cycle Management with its AI-powered autonomous coding solution, a multi-specialty AI-platform that translates clinical information into accurate sets of medical codes. The Principal Data Engineer is a member of the Data Platform team, reporting to the Director of Machine Learning Engineering and Data. The Data Platform team is responsible for executing the data strategy for the organization. As a Principal Data Engineer (L4), you will serve as a key technical leader for CodaMetrix’s Databricks-based data platform, supporting streaming and batch workloads across 30+ healthcare customers.

Responsibilities

Own the technical strategy and roadmap for the CodaMetrix data platform; Design, build, and maintain scalable streaming and batch data platforms using Databricks, PySpark, Unity Catalog, Delta Lake, and Structured Streaming; Build Jenkins CI/CD workflows for automated testing, tagging, and deployments; Implement and evolve least-privilege access controls across Databricks; Ensure HIPAA and SOC 2 compliance through PHI masking, audit logging, and data segmentation; Drive platform cost reduction and monitor Databricks/AWS spend; Resolve production incidents and maintain runbooks; Enable ML, BI, Analytics, DevOps, and customer success teams; Mentor engineers through code and design reviews

Requirements

A degree in Computer Science or a related field or equivalent combination of education and demonstrable professional experience; 8+ years of data engineering experience with progressive responsibility; 5+ years hands-on experience with the Databricks platform (Unity Catalog, Delta Lake, Structured Streaming, Spark SQL, cluster management, platform administration); Expert proficiency in PySpark and Python and working knowledge of Scala; Expert experience with Terraform for infrastructure-as-code; Expert experience with Apache Kafka / AWS MSK; Deep understanding of medallion/lakehouse architecture patterns; Proven track record building and maintaining CI/CD pipelines (Jenkins, GitHub Actions) for data platform deployments; Expert SQL skills including complex CTEs, window functions, and performance optimization across petabyte-scale datasets; Strong AWS experience (S3, IAM, Secrets Manager, VPC/Private Link, cross-region replication); Demonstrated ability to operate in a HIPAA-regulated environment with PHI handling requirements; Experience with disaster recovery architecture; Experience using AI tools and agentic workflows to augment design, development, and testing processes

Nice to have

Prior experience with healthcare data such as medical claims, electronic health records, or billing code systems (ICD-10, CPT); Familiarity with healthcare data standards like HL7 or FHIR; Track record of influencing technical direction beyond immediate team; Strategic ability to make build-vs-buy decisions and evaluate emerging technologies

Benefits

Health insurance covering 80% of medical and dental plus vision; 401(k) plan; Generous Paid Time Off; Annual performance evaluations; Annual company awards; Modern open plan workspace in Back Bay; Employer-paid life insurance and short-term and long-term disability insurance