Skip to content

Modular

NewBe an early applicant

AI Framework Engineer

Posted Updated

Description

  • At Modular, we are building a next generation AI platform to power modern applications and facilitate access to cutting-edge hardware
  • The MAX Framework is our developer-facing layer: it defines the APIs developers use to express models, integrate custom kernels, orchestrate execution, iterate on quality and ship systems into production
  • As an Senior AI Framework Engineer for MAX, you will own and evolve core APIs and developer experience for inference and training of AI models
  • You will work at the intersection of API design, systems engineering, and modern AI frameworks
  • Your output will be the specifications, abstractions, and reference implementations that make MAX feel coherent, powerful, and intuitive to use — while preserving performance, portability, maintainability and TCO
  • Influence the design of the API surface for MAX: namespaces, core abstractions, extension points, programming model, compatibility guarantees and developer experience
  • Design inference APIs that support real-world serving needs: model loading, distributed inference, quantization, tokenization/pipelines, configuration surfaces, batching/streaming, and deployment-oriented ergonomics
  • Design training APIs that scale from single device to distributed execution, with clear primitives for device placement, parallelism, checkpointing, and observability
  • Create a coherent programming model across Python and Mojo-adjacent surfaces: align naming, types, and conventions; avoid leaky abstractions; define the “pit of success”
  • Drive RFCs and technical specs: write and socialize proposals; gather feedback from internal model engineers and external users; iterate towards consensus
  • Partner cross-functionally with compiler/runtime, kernels, cloud/serving, and documentation/DevRel teams to ensure APIs map cleanly to underlying capabilities
  • Build reference implementations and exemplar code: golden-path examples, architecture templates, and best-practice patterns that teams can copy
  • Set quality bars for APIs: versioning policy, deprecation strategy, test strategy, and documentation requirements

Benefits

  • A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
  • A 401k plan with up to 5% match
  • Free tax advice on Carta
  • Generous work-from-home stipend of $1500 to help you improve your home office
  • Unlimited paid time off and flexible work hours
  • PhD in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 4+ years of Software Engineering or related work experience
  • Bachelor’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 6+ years of Software Engineering or related work experience. OR
  • 3+ years of work experience with Programming Language such as Python, Mojo, Rust, C++, etc
  • Master’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 5+ years of Software Engineering or related work experience. OR
  • Experience with inference and training systems (model execution graphs, compilation, runtime scheduling, distributed execution, checkpointing, performance tuning)
  • Strong understanding of modern AI frameworks and their design tradeoffs (e.g., PyTorch, JAX, TensorFlow, vLLM, XLA/MLIR-adjacent ecosystems)
  • Strong written communication: you can write specs that engineers can implement without ambiguity
  • Significant experience designing and evolving developer-facing APIs (SDKs, frameworks, or platforms)
  • High engineering standards, pragmatism, and a bias towards incremental development without compromising long term design
  • Fluency in one or more systems / performance languages (C++, Rust, Go) and one or more user-facing languages (Python; familiarity with Mojo is a plus)
  • Excellent taste for API ergonomics: naming, composability, types, error handling, configurability, and clarity
  • For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply, as we may have openings at lower or higher levels than the one advertised
  • Onboarding for new hires is conducted in-person in our Los Altos, CA office

Skills

Apply

Apply to AI Framework Engineer at Modular

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also