Lead Architect, Data & AI Platforms

We are hiring a Lead Architect to own the technical design of Azure-first data and AI platforms. You will define the reusable patterns, reference architectures and end-to-end solution concepts that our delivery teams build from, and prove them in code rather than in slideware. This is an engineering-first, developer-architect role: roughly 70–80% designing solutions, building and packaging reusable patterns, and prototyping new concepts in code, and 20–30% client-facing advisory.

Responsibilities

Lead Azure-First Platform Architecture: Architect and implement Azure data platforms for clients, including landing zones, networking, identity, and data platform services (Azure Data Lake Storage, Synapse, Databricks), extending patterns to AWS or GCP where a client estate requires it; Design & Build Data Pipelines: Design, Implement key data ingress patterns across data platforms. Key focus would be Azure Data Factory pipelines and data flows for ingestion, transformation, and orchestration; establish patterns for parameterization, retries, and error handling to meet pipeline SLAs, then work with the Data Engineering team to scale those patterns into production delivery; Implement Modern IaC & CI/CD: Design, implement key IaC patterns. Deliver infrastructure-as-code using Terraform; create reusable modules and policy-as-code guardrails. Establish repo strategies, branching, pull requests, multi-stage pipelines, approvals, and automated promotion gates; Package Reusable Solutions: Turn proven designs into packaged, deployable assets (Terraform modules, pipeline templates, reference implementations and accelerators) with the documentation and worked examples delivery teams need to reuse them across clients; Architect for AI & DataOps: Introduce DataOps practices (automated unit/integration testing, data quality checks) and build the AI layer on top of it: vector search, RAG workflows and agentic capabilities; Prototype & Showcase New Concepts: Build working prototypes of new data and AI concepts (RAG, agentic workflows, streaming and real-time patterns) and demonstrate them to clients to shape demand, test feasibility and de-risk delivery before a full team is committed; Client Advisory & Technical Leadership: Own the technical relationship with client and agency stakeholders. Run discovery and design workshops, present at design authorities, and translate commercial objectives into phased roadmaps, costs and trade-offs that a non-technical audience can act on; Ship Secure-by-Default Controls: Codify private endpoints, managed identities, Key Vault-backed secrets, RBAC, network isolation, data encryption and Purview cataloging/lineage as reusable modules, so security ships with the platform rather than being applied afterwards; Optimize Performance & Cost (FinOps): Optimize compute and storage across Spark clusters, autoscaling, partitioning, and serverless options; track, report, and drive cloud cost savings; Observability & Reliability: Build monitoring into pipelines and platforms using Azure Monitor and Log Analytics, and define the SLOs and alerting the owning engineering team runs against; Guide Event-Driven Architectures: Lead streaming and micro-batch patterns (Event Hubs, Kafka on Azure, Structured Streaming) where required by client use cases; Enable the Data Engineering Team: Mentor engineers, review pull requests/IaC, and hand over patterns and accelerators in a state the Data Engineering team can run with, so larger builds are delivered by the team rather than bottlenecked on you.

Requirements

Hands-on Developer-Architect experience: Proven track record in cloud platform and data engineering, with deep expertise in Microsoft Azure and hands-on exposure to AWS and/or Google Cloud Platform (GCP); Production IaC Fluency: Strong proficiency in Terraform applied with a software engineering mindset (reusable modules, testing, modular code structures); Data Platform Core: Hands-on experience with Azure Data Factory, Synapse Analytics, and Azure Databricks (Spark); CI/CD Mastery: Strong CI/CD experience using GitHub Actions or Azure DevOps for both IaC and data workloads; Software Engineering Fundamentals: Solid coding capability in Python and SQL; performance tuning for Spark/Databricks; data modelling for analytical and AI workloads; Evidence of Shipped Work: Able to walk through data platform code, Terraform modules or pipelines you have personally authored; Patterns Adopted by Others: Examples of reference architectures, reusable modules or accelerators you have designed, packaged and had adopted by teams other than your own; Client-Facing Advisory Skills: Demonstrated ability to consult directly with senior client and agency stakeholders, including non-technical business audiences, present pragmatic technical solutions, and lead architectural workshops unaided; Security as Code: Ability to implement VNets, private endpoints, managed identities, Key Vault, RBAC and governance patterns as Terraform, rather than through portal configuration or change-request processes.

Nice to have

Production experience shipping RAG or agentic AI workloads to live users (vector databases, LLM orchestration frameworks); Event-driven architectures with Event Hubs, Kafka, Stream Analytics, or Functions; CDC patterns; Lakehouse/Delta patterns on Databricks; MLflow and feature store awareness; Data quality tooling and frameworks (Great Expectations, Deequ) and dbt for transformation governance; Data Governance: Experience tracking data lineage, defining data quality policies and procedures, and enforcing them at runtime; API Management, Logic Apps, or Azure Functions integration experience; Experience with Azure Enterprise-Scale Landing Zones and Azure Policy; Relevant certifications: Microsoft Certified Azure Solutions Architect Expert, Azure Data Engineer Associate, or AWS / GCP equivalents.

Benefits

competitive medical; group retirement plans; vision insurance; dental insurance; significant paid time off; preferential partner discounts; employee mental health awareness days; access to global WPP Media & WPP networks; employee resource groups; frequent in-office events