Skip to content

Together AI

Machine Learning Engineer (Inference)

Posted Updated

Description

  • Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems
  • This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale
  • This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solution
  • Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale
  • Develop and optimize runtime inference services for large-scale AI applications
  • Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world
  • Conduct design and code reviews to ensure high standards of quality
  • Create services, tools, and developer documentation to support the inference engine
  • Implement robust and fault-tolerant systems for data ingestion and processing

Benefits

  • Competitive health insurance plans
  • Pre-tax flexible spending accounts
  • Income protection & retirement
  • Dental and vision insurance
  • Mental health support and services
  • 401(k) plan
  • STD & LTD insurance
  • AD&D insurance
  • Life insurance
  • Team-driven celebrations and events
  • Monthly team lunches
  • Flexible time off policy
  • Monthly commuting stipend + pre-tax bene
  • If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you
  • Demonstrated experience in building high performance libraries and tooling
  • 3+ years of experience writing high-performance, well-tested, production-quality code
  • Proficiency with Python and PyTorch
  • Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale
  • Preferred: Knowledge of AI inference techniques such as speculative decoding
  • Nice to have: Knowledge of Rust, Cython and compilers
  • Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
  • Preferred: Knowledge of CUDA/Triton programming

Skills

Apply

Apply to Machine Learning Engineer (Inference) at Together AI

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

ML / AI jobs by country — openings, pay and top skills →