Skip to content

Normal Computing

Runtime Engineer

Posted Updated

Description

  • Normal’s hardware runs the most memory-bound parts of large-model inference inside memory itself, rather than shuttling weights to a processor
  • The software that makes this programmable doesn’t exist yet in any standard form
  • The runtime is the layer the rest of that software runs through: it is how compiled programs get loaded, scheduled, and executed on our hardware
  • You will own its design and much of its implementation, and set its technical direction
  • This is a senior, hands-on role
  • Our chip is pre-silicon, so you will build the runtime against our simulators and FPGA prototypes first, behind one interface, so that software above the runtime doesn’t change when silicon arrives
  • Runtime Architecture: Own the design of the runtime: device management, memory allocation and host-device transfer, executable loading, and command submission. Success means the compiler and serving layers program our hardware through one stable interface
  • Hardware Abstraction: Hide the differences between simulators, FPGA prototypes, silicon, and later hardware generations behind that interface
  • Device State: Manage the state that lives on our hardware between requests, such as resident model weights, so the serving layer can rely on it
  • Performance: Keep runtime overhead off the critical path for LLM inference, measured by end-to-end latency and throughput
  • Cross-Team Interfaces: Define the boundary with the compiler above the runtime, and with the driver and firmware below it, which our hardware team owns
  • Technical Leadership: Set the runtime’s technical direction, review designs that depend on it, and work with compiler and serving engineers so the interface serves them
  • Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware
  • Understanding of LLM inference workloads and what they demand of a runtime
  • Track record of owning a complex systems component end to end, from design through production, and driving technical decisions across teams
  • Strong systems programming in C++ or Rust: memory management, concurrency, and low-level performance
  • Comfort building software for hardware that is still evolving, from simulation through bring-up
  • Contributions to open-source runtime or systems infrastructure
  • Experience integrating a runtime with an inference server such as vLLM
  • Experience with in-memory compute or other non-von-Neumann hardware
  • Experience with open runtime designs such as IREE’s HAL, or with CUDA or ROCm runtime internals

Skills

Apply

Apply to Runtime Engineer at Normal Computing

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also