AI Framework Engineer
Posted Updated
Description
- At Modular, we are building a next generation AI platform to power modern applications and facilitate access to cutting-edge hardware
- The MAX Framework is our developer-facing layer: it defines the APIs developers use to express models, integrate custom kernels, orchestrate execution, iterate on quality and ship systems into production
- As an Senior AI Framework Engineer for MAX, you will own and evolve core APIs and developer experience for inference and training of AI models
- You will work at the intersection of API design, systems engineering, and modern AI frameworks
- Your output will be the specifications, abstractions, and reference implementations that make MAX feel coherent, powerful, and intuitive to use — while preserving performance, portability, maintainability and TCO
- Influence the design of the API surface for MAX: namespaces, core abstractions, extension points, programming model, compatibility guarantees and developer experience
- Design inference APIs that support real-world serving needs: model loading, distributed inference, quantization, tokenization/pipelines, configuration surfaces, batching/streaming, and deployment-oriented ergonomics
- Design training APIs that scale from single device to distributed execution, with clear primitives for device placement, parallelism, checkpointing, and observability
- Create a coherent programming model across Python and Mojo-adjacent surfaces: align naming, types, and conventions; avoid leaky abstractions; define the “pit of success”
- Drive RFCs and technical specs: write and socialize proposals; gather feedback from internal model engineers and external users; iterate towards consensus
- Partner cross-functionally with compiler/runtime, kernels, cloud/serving, and documentation/DevRel teams to ensure APIs map cleanly to underlying capabilities
- Build reference implementations and exemplar code: golden-path examples, architecture templates, and best-practice patterns that teams can copy
- Set quality bars for APIs: versioning policy, deprecation strategy, test strategy, and documentation requirements
Benefits
- A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
- A 401k plan with up to 5% match
- Free tax advice on Carta
- Generous work-from-home stipend of $1500 to help you improve your home office
- Unlimited paid time off and flexible work hours
- PhD in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 4+ years of Software Engineering or related work experience
- Bachelor’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 6+ years of Software Engineering or related work experience. OR
- 3+ years of work experience with Programming Language such as Python, Mojo, Rust, C++, etc
- Master’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 5+ years of Software Engineering or related work experience. OR
- Experience with inference and training systems (model execution graphs, compilation, runtime scheduling, distributed execution, checkpointing, performance tuning)
- Strong understanding of modern AI frameworks and their design tradeoffs (e.g., PyTorch, JAX, TensorFlow, vLLM, XLA/MLIR-adjacent ecosystems)
- Strong written communication: you can write specs that engineers can implement without ambiguity
- Significant experience designing and evolving developer-facing APIs (SDKs, frameworks, or platforms)
- High engineering standards, pragmatism, and a bias towards incremental development without compromising long term design
- Fluency in one or more systems / performance languages (C++, Rust, Go) and one or more user-facing languages (Python; familiarity with Mojo is a plus)
- Excellent taste for API ergonomics: naming, composability, types, error handling, configurability, and clarity
- For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply, as we may have openings at lower or higher levels than the one advertised
- Onboarding for new hires is conducted in-person in our Los Altos, CA office
Skills
Sign in to read and join the discussion.