jobs.pl/Oferty pracy/IT - Rozwój oprogramowania/Databricks Practice Lead / Engineering Manager
Źródło: CrawlJobs

Databricks Practice Lead / Engineering Manager

Scicom Infrastructure Services Atlanta, United States
Remote work

Publication date: 04/11/2026

Scicom Infrastructure Services is seeking an experienced Databricks Practice Lead / Engineering Manager to provide hands-on technical leadership while managing a team of data engineers, architects, and consultants supporting complex enterprise and government programs. This role requires a senior Databricks expert who can design and oversee modern data platforms, establish technical standards, guide delivery teams, and remain actively involved in architecture, troubleshooting, code reviews, and client-facing solution development. The successful candidate will balance deep technical expertise with strong people leadership, delivery management, and stakeholder communication skills.

Responsibilities

Serve as the organization's subject-matter expert for the Databricks Lakehouse Platform; Design scalable, secure, and highly available data architectures using Databricks, Apache Spark, Delta Lake, and cloud-native technologies; Lead the implementation of batch, streaming, ETL, ELT, analytics, machine-learning, and AI-enabled data solutions; Define architectural standards for medallion architectures, data modeling, ingestion, transformation, orchestration, and data consumption; Establish governance frameworks using Unity Catalog; Guide Databricks workspace design, cluster configuration, serverless computing, workload isolation, performance tuning, and cost optimization; Oversee integration between Databricks and cloud platforms such as Microsoft Azure, AWS, or Google Cloud; Develop or review solutions involving PySpark, Spark SQL, Python, Delta Live Tables, Structured Streaming, Auto Loader, MLflow, and Databricks Workflows; Lead platform migrations and modernization efforts from legacy databases, data warehouses, Hadoop environments, and traditional ETL platforms; Establish development standards for source control, automated testing, CI/CD, infrastructure as code, monitoring, and production support; Conduct architecture reviews, code reviews, technical assessments, and root-cause analyses; Evaluate emerging Databricks capabilities and recommend adoption strategies; Manage, mentor, and develop a team of Databricks engineers, data engineers, architects, and technical consultants; Assign resources and responsibilities based on project needs, employee strengths, availability, and technical complexity; Establish measurable goals, performance expectations, development plans, and technical competency standards; Conduct regular one-on-one meetings, performance reviews, coaching sessions, and technical development activities; Support recruiting, interviewing, candidate evaluation, onboarding, and workforce planning; Identify technical or performance gaps and coordinate training, mentoring, or corrective action; Promote collaboration, accountability, documentation, knowledge sharing, and continuous improvement; Develop reusable accelerators, reference architectures, templates, and delivery playbooks; Build and maintain a strong Databricks practice capable of supporting multiple concurrent client engagements; Provide delivery oversight for Databricks and data-engineering projects from planning through operational support; Translate business, functional, security, and contractual requirements into technical plans and deliverables; Develop project estimates, staffing plans, delivery schedules, milestones, and risk-mitigation strategies; Monitor project scope, schedule, quality, budget, resource utilization, dependencies, and technical risks; Ensure deliverables meet client requirements, internal quality standards, security controls, and contractual commitments; Coordinate work across engineering, cloud, cybersecurity, data governance, analytics, project-management, and client teams; Track delivery metrics and provide clear status reports to internal leadership, clients, and program stakeholders; Lead technical escalations and ensure issues are resolved promptly and appropriately documented; Support statements of work, technical proposals, solution estimates, presentations, and client demonstrations; Participate in client meetings as the technical and delivery authority for Databricks-related work

Requirements

Bachelor's degree in computer science, information technology, data engineering, engineering, or a related discipline; At least 10 years of experience in data engineering, data architecture, analytics engineering, or related technology roles; At least 5 years of hands-on experience designing and implementing solutions using Databricks; At least 3 years of experience managing or formally leading technical engineering teams; Advanced experience with Databricks Lakehouse Platform; Advanced experience with Apache Spark and PySpark; Advanced experience with Spark SQL and advanced SQL development; Advanced experience with Delta Lake and medallion architecture; Advanced experience with Unity Catalog and enterprise data governance; Advanced experience with ETL and ELT pipeline architecture; Advanced experience with batch and real-time data processing; Advanced experience with data modeling and data warehousing; Advanced experience with Python-based data engineering; Advanced experience with Databricks Workflows, Jobs, and cluster management; Experience deploying Databricks solutions in Azure, AWS, or Google Cloud; Experience with CI/CD, Git-based development, automated testing, and infrastructure as code; Demonstrated ability to optimize Spark workloads, cluster configurations, query performance, reliability, and cloud costs; Experience managing technical delivery, resource assignments, risks, schedules, and client expectations; Strong written, verbal, presentation, documentation, and stakeholder-management skills; Ability to explain complex technical concepts to executives, business stakeholders, and nontechnical audiences

Nice to have

Databricks Certified Data Engineer Professional, Databricks Certified Data Engineer Associate, or Databricks Certified Machine Learning Professional; Databricks Certified Data Architect or comparable advanced architecture credentials; Microsoft Azure, AWS, or Google Cloud professional-level certification; Experience working in a consulting, professional-services, systems-integration, or managed-services environment; Experience supporting federal, state, or local government clients; Experience working with major consulting or systems-integration partners; Knowledge of federal security, privacy, governance, and compliance requirements; Experience with Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, AWS Glue, Amazon S3, Snowflake, dbt, Kafka, Airflow, or Terraform; Experience with MLflow, MLOps, generative AI, Databricks Mosaic AI, vector search, or machine-learning deployment; Familiarity with data standards, metadata frameworks, data catalogs, data-sharing protocols, and open-data environments; Experience managing geographically distributed or remote technical teams; Experience contributing to proposals, technical responses, statements of work, and project estimates
Apply with employer

Podobne oferty

Wszystkie →

Dostosuj zgody

Każdy cel jest opcjonalny i domyślnie wyłączony. Zgodę możesz wycofać w dowolnym momencie.

Pokazujemy oferty najbliżej Ciebie