Data Platform Engineer

Lantern is the specialty care platform connecting people with the best care when they need it most. Lantern’s data organization is moving from a centralized model toward one where domain teams (Data Engineering, Data Ops, Analytics Engineering) build and own their own data products, and the platform team’s job is to make that possible. This is a hands-on administration and enablement role. This role administers the Databricks platform and the Lakebase (managed Postgres) estate, holds the elevated privileges that go with that, and enforces standards through platform controls, automation, and review across Azure infrastructure and IAM, CI/CD, observability, and the governance layer of access control, audit, and lineage. It is also the platform’s front door for domain teams.

Responsibilities

Administer the Azure Databricks and Unity Catalog platform against architecture standards, including workspace and metastore configuration, catalog and schema structure, cluster policies, and compute cost management with spend-by-team visibility and chargeback reporting; Build the self-serve infrastructure layer, including provisioning workflows, templates, and access patterns for domain teams; Act as DBA for the Postgres (Lakebase) estate, including provisioning and decommissioning, user and role management, performance tuning, version upgrades, backup and point-in-time recovery, and incident response; Build and maintain Terraform Infrastructure-as-Code for Azure resources; Implement and enforce the approved access model across the lakehouse and Postgres layer (Unity Catalog grants, OBO authentication, ABAC, row-level security, data masking); Build the audit and lineage layer, including audit logging, lineage capture, and sensitivity classification tagging; Operate platform observability across Databricks, Lakebase, and Azure services; Execute infrastructure lifecycle tasks, including backup and restore validation and disaster recovery testing; Support CI/CD for data workloads across domain teams; Maintain the integration between the platform and GitHub / GitHub Actions; Collaborate with Cloud Platform, Azure SQL DBA, Security, InfoSec, and Core Apps teams; Engage early in domain-team project kickoffs to advise on platform capabilities and standards; Serve as the escalation point for domain engineers on cluster policies, permissions, database performance, and CI steps; Produce platform KPI reporting and contribute to the platform roadmap and backlog; Capture reusable patterns in a shared knowledge base

Requirements

A minimum of 4 years in a platform or infrastructure engineering role supporting a data or analytics organization; Deep hands-on experience with Databricks (workspace administration, Unity Catalog, Delta Lake) and Azure (Entra ID, networking, Key Vault, Infrastructure-as-Code via Terraform or Bicep); Hands-on PostgreSQL administration experience, including user and role management, connection configuration, query and index performance tuning, backup and restore, point-in-time recovery, and HA/DR; Real experience with identity and access architecture, including service principals vs. user-authorization (OBO) patterns and their tradeoffs; Practical experience building and maintaining CI/CD pipelines with GitHub Actions and/or Azure DevOps; Proficiency with Python or another scripting language for platform automation and tooling; Experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to accelerate engineering work; Comfort owning the infrastructure lifecycle: provisioning, decommissioning, and disaster recovery; Experience working in a regulated environment, ideally healthcare, with HIPAA and/or SOC 2 requirements; Comfort operating with elevated privileges in a regulated, PHI-bearing environment, and the judgment to escalate rather than grant exceptions unilaterally; Ability to hold a standard without owning it, enforcing architecture and security standards set elsewhere through influence and automation; Strong problem-solving and troubleshooting ability, including under incident conditions; Excellent communication and collaboration skills, and the ability to work effectively with cross-functional engineering teams

Nice to have

Exposure to data mesh or domain-oriented data architecture; Hands-on experience with Databricks Lakebase or another cloud-native Postgres service (Azure Database for PostgreSQL, Neon, Aurora); Experience with Postgres tooling such as psql, pgAdmin, and migration frameworks; Familiarity with Azure SQL Database and experience integrating OLTP systems with a lakehouse; Healthcare data environment experience (HIPAA-driven access controls, PHI segregation); Ability to learn new business models and technologies; Experience with observability tooling such as Datadog or OpenTelemetry-based pipelines; Experience with Databricks Asset Bundles, DLT, or dbt in a multi-team deployment model; Relevant certifications (e.g., Databricks Data Engineer Professional, Azure Administrator)

Benefits

Medical Insurance; Dental Insurance; Vision Insurance; Short & Long Term Disability; Life Insurance; 401k with company match; Flexible Time Off; Paid Parental Leave