Runtime Engineer

Normal's hardware runs the most memory-bound parts of large-model inference inside memory itself, rather than shuttling weights to a processor. The software that makes this programmable doesn't exist yet in any standard form. The runtime is the layer the rest of that software runs through: it is how compiled programs get loaded, scheduled, and executed on our hardware. You will own its design and much of its implementation, and set its technical direction. This is a senior, hands-on role. Our chip is pre-silicon, so you will build the runtime against our simulators and FPGA prototypes first, behind one interface, so that software above the runtime doesn't change when silicon arrives.

Responsibilities

Runtime Architecture: Own the design of the runtime: device management, memory allocation and host-device transfer, executable loading, and command submission; Hardware Abstraction: Hide the differences between simulators, FPGA prototypes, silicon, and later hardware generations behind that interface; Device State: Manage the state that lives on our hardware between requests, such as resident model weights; Performance: Keep runtime overhead off the critical path for LLM inference, measured by end-to-end latency and throughput; Cross-Team Interfaces: Define the boundary with the compiler above the runtime, and with the driver and firmware below it; Technical Leadership: Set the runtime's technical direction, review designs that depend on it, and work with compiler and serving engineers so the interface serves them

Requirements

Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware; Strong systems programming in C++ or Rust: memory management, concurrency, and low-level performance; Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams; Understanding of LLM inference workloads and what they demand of a runtime; Comfort building software for hardware that is still evolving, from simulation through bring-up

Nice to have

Experience with open runtime designs such as IREE's HAL, or with CUDA or ROCm runtime internals; Experience with in-memory compute or other non-von-Neumann hardware; Experience integrating a runtime with an inference server such as vLLM; Contributions to open-source runtime or systems infrastructure