Software Engineer (LLM Infrastructure)
Posted Updated
Description
- As a Software Engineer on our AI Infrastructure team, you will help design the core systems that power Fireworks AI’s generative AI platform
- You will help build infrastructure and tools that ensure the reliability, performance, quality, and availability of our AI system
- Our mission is to make Fireworks AI the most reliable and user friendly generative AI platform in the world
- You will partner closely with our cloud infrastructure team, product team, and performance team to deliver infrastructure that bridges the gap between our customer and the ultra-performant proprietary Fireworks inference engine
- Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
- Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
- Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
- Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
- Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
- Participate in code reviews, technical discussions, and continuous integration and deployment processes
- Solve Hard Problems: Tackle challenges at the forefront of AI infrastructure, from low-latency inference to scalable model serving
- Build What’s Next: Work with bleeding-edge technology that impacts how businesses and developers harness AI globally
- Ownership & Impact: Join a fast-growing, passionate team where your work directly shapes the future of AI—no bureaucracy, just results
- Learn from the Best: Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation
- Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)
- Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)
- Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
- Strong programming skills in Python, Go, or a similar language
- 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
- 5+ years of experience in software engineering, with a focus on infrastructure or machine learning systems
- Experience with open source inference engine like vLLM, Sglang, or TRT-LLM
- Experience in building large scale ML/MLOps infrastructure
- Contributions to open-source infrastructure or ML projects