Data Platform Engineer

Lantern's data organization is moving from a centralized model toward one where domain teams build and own their own data products, and the platform team enables that. This hands-on administration and enablement role administers the Databricks platform and the Lakebase (managed Postgres) estate, holds elevated privileges, and enforces standards through platform controls, automation, and review across Azure infrastructure and IAM, CI/CD, observability, and governance. The role also serves as the platform's front door for domain teams. Location: Hybrid, at least 3 days/week in the Dallas, TX offices.

Responsibilities

Administer the Azure Databricks and Unity Catalog platform against architecture standards, including workspace and metastore configuration, catalog and schema structure, cluster policies, and compute cost management with chargeback reporting; Build self-serve infrastructure (provisioning workflows, templates, access patterns) so domain teams can manage their own data products; Act as DBA for the Lakebase Postgres estate, including provisioning, user and role management, performance tuning, version upgrades, backup and point-in-time recovery, and incident response; Build and maintain Terraform Infrastructure-as-Code for Azure resources, networking, Entra ID, and Key Vault; Implement and enforce access model across lakehouse and Postgres (Unity Catalog grants, OBO, ABAC, row-level security, data masking); Build audit and lineage layer, including audit logging, lineage capture, and sensitivity classification tagging; Operate platform observability across Databricks, Lakebase, and Azure services; Execute infrastructure lifecycle, including provisioning, decommissioning, backup and restore validation, and disaster recovery testing; Occasional availability outside core hours for scheduled deployments and major incidents (no overnight on-call rotation); Support CI/CD for data workloads and maintain GitHub / GitHub Actions integration; Work with Cloud Platform, Azure SQL DBA, and Security teams on shared foundations; Maintain liaison with Security, InfoSec, and Core Apps and ensure HIPAA and SOC 2 compliance; Engage early in domain-team project kickoffs; Serve as escalation point for domain engineers; Produce platform KPI reporting and contribute to the platform roadmap; Document reusable patterns in a shared knowledge base

Requirements

A minimum of 4 years in a platform or infrastructure engineering role supporting a data or analytics organization; Deep hands-on experience with Databricks (workspace administration, Unity Catalog, Delta Lake) and Azure (Entra ID, networking, Key Vault, Infrastructure-as-Code via Terraform or Bicep); Hands-on PostgreSQL administration experience (user and role management, connection configuration, query and index performance tuning, backup and restore, point-in-time recovery, HA/DR); Real experience with identity and access architecture, including service principals vs. user-authorization (OBO) patterns and their tradeoffs; Practical experience building and maintaining CI/CD pipelines with GitHub Actions and/or Azure DevOps; Proficiency with Python or another scripting language for platform automation and tooling; Experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to accelerate engineering work; Comfort owning the infrastructure lifecycle: provisioning, decommissioning, and disaster recovery; Experience working in a regulated environment, ideally healthcare, with HIPAA and/or SOC 2 requirements; Comfort operating with elevated privileges in a regulated, PHI-bearing environment, and the judgment to escalate rather than grant exceptions unilaterally; Ability to hold a standard without owning it, enforcing architecture and security standards through influence and automation; Strong problem-solving and troubleshooting ability, including under incident conditions; Excellent communication and collaboration skills

Nice to have

Exposure to data mesh or domain-oriented data architecture; Hands-on experience with Databricks Lakebase or another cloud-native Postgres service (Azure Database for PostgreSQL, Neon, Aurora); Familiarity with Postgres tooling such as psql, pgAdmin, and migration frameworks; Enough familiarity with Azure SQL Database to coordinate with the DBA team; Experience integrating OLTP systems with a lakehouse; Healthcare data environment experience (HIPAA-driven access controls, PHI segregation); Experience with observability tooling such as Datadog or OpenTelemetry-based pipelines; Experience with Databricks Asset Bundles, DLT, or dbt in a multi-team deployment model; Relevant certifications (e.g., Databricks Data Engineer Professional, Azure Administrator)

Benefits

Medical Insurance; Dental Insurance; Vision Insurance; Short & Long Term Disability; Life Insurance; 401k with company match; Flexible Time Off; Paid Parental Leave