Machine Learning Engineer (Inference)
Posted Updated
Description
- Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems
- This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale
- This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solution
- Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale
- Develop and optimize runtime inference services for large-scale AI applications
- Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world
- Conduct design and code reviews to ensure high standards of quality
- Create services, tools, and developer documentation to support the inference engine
- Implement robust and fault-tolerant systems for data ingestion and processing
Benefits
- Competitive health insurance plans
- Pre-tax flexible spending accounts
- Income protection & retirement
- Dental and vision insurance
- Mental health support and services
- 401(k) plan
- STD & LTD insurance
- AD&D insurance
- Life insurance
- Team-driven celebrations and events
- Monthly team lunches
- Flexible time off policy
- Monthly commuting stipend + pre-tax bene
- If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you
- Demonstrated experience in building high performance libraries and tooling
- 3+ years of experience writing high-performance, well-tested, production-quality code
- Proficiency with Python and PyTorch
- Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale
- Preferred: Knowledge of AI inference techniques such as speculative decoding
- Nice to have: Knowledge of Rust, Cython and compilers
- Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
- Preferred: Knowledge of CUDA/Triton programming
Skills
Sign in to read and join the discussion.