Database Internals Engineer

Join the team that built the original Teradata engine, and work on what comes next. Inside the Office of the CTO, our Advanced Research Team is exploring a new generation of massively parallel, decentralized compute — engine architecture where parallelism is a first principle rather than a later refinement. This is a rare opportunity to work at the frontier of database internals, learn from the people who helped define the field, and leave your mark on a new engine as it takes shape.

Responsibilities

Build components across the engine spine: SQL front end (tokenizer, parser, binder), logical/physical plan layer, rule- and cost-based optimizer, and operators (joins, aggregates, sort, scan); Build the storage substrate: Arrow in-memory format, slotted-page on-disk format with checksums, buffer pool, and a B-tree or LSM access method; Implement transactions and recovery: lock/latch management, MVCC/snapshot isolation, WAL/ARIES, checkpoints, and crash recovery; Add parallelism and distribution along the correct axis — exchange-based parallel execution for query work, consensus/replication/atomic-commit for data correctness — without conflating the two; Write design specifications before coding (schemas; null/empty/duplicate semantics; memory budget and spill behavior; cost characteristics), write tests first, implement behind the established operator interface, verify safety then speed, and report the benchmark delta

Requirements

Deep expertise in distributed query optimization: cascading optimizers, Abstract Syntax Tree (AST) binding, logical and physical plan distribution; Experience designing parallel execution pipelines where distribution is a first-class concern, not an afterthought; Familiarity with systems like Apache DataFusion or similar distributed execution frameworks; Strong hands-on background with analytical/embedded engines (e.g., DuckDB, DataFusion) and their internals; Deep knowledge of pipeline execution, vectorized processing, file system I/O, and open table formats (Iceberg, Delta); Experience with transaction management, lock managers, cache management, or OS-level scheduling; Systems programming in Rust or modern C/C++ with memory-safety discipline; Relational foundations end to end: relational algebra and the algebraic laws behind query rewrites, logical→physical lowering, and cost-based optimization; Demonstrated depth in at least one area of the engine spine (front end, plan/optimizer, operators, storage engine, transactions/recovery, or scale-out) with working literacy across its neighbors; Vectorized execution over a columnar (Arrow) representation: batch-at-a-time operators, the pull/iterator model, validity-bitmap-correct null handling, zero-copy buffer sharing; Cost reasoning at the metal: cache behavior, alignment, SIMD, allocation patterns, false sharing in parallel accumulators; Test- and benchmark-gated engineering: golden/sqllogictest, property and fuzz testing, deterministic simulation testing for concurrent/distributed paths, and microbenchmarks (Criterion/Google Benchmark) plus a TPC-H subset; Sophisticated, hands-on use of AI coding agents directed against a reference-grade spec; Communicates clearly; Doer over theorist; Intellectually curious and self-directed; Collaborative with peers; Comfortable with evolving requirements; High personal bar for quality

Benefits

People-first culture; Flexible work model; Focus on well-being; Anti-racist company and commitment to Diversity, Equity, and Inclusion; Healthcare; Life and disability insurance; Retirement savings / 401(k)-retirement savings plan; Time-off programs; Annual incentive plans based on company financial attainment and individual performance