Forward Deployed Engineer (Inference)
Posted Updated
Description
- We are looking for a Staff Forward Deployed Engineer (FDE) to join our team
- The team is responsible for taking our large language models and making them work reliably, securely, and at peak performance within our customers’ custom infrastructure and data perimeters
- In this role, you will act as the principal technical translation layer between our core engineering teams and our strategic partners
- You will own the architecture for highly ambiguous, customer-specific inference challenges—ranging from multi-tenant model serving to multi-datacenter disaster recovery and custom hardware validation
- You will lead technical adoption end-to-end, ensuring our models achieve exceptionally low latency, high throughput, and seamless resilience in the field
- Embed directly with customer engineering teams to architect the “connective tissue” between our AI models and their live, complex production environments
- Design advanced model serving topologies, implementing multi-tenant architectures, intelligent model routing, and cross-datacenter failover strategies for multi-cluster environments
- Optimize “AI-in-a-box” and private deployments, balancing localized SRE constraints like Kubernetes multi-cluster scale with raw model performance
- Conduct rigorous benchmarking and profiling of customer LLM workloads, root-causing performance degradation (e.g., slow generation) via metrics, dashboards, and distributed logs
- Accelerate hardware validation, collaborating with clients to run and support Cohere models on next-generation accelerators (GPUs, TPUs, AMD, and custom ASICs)
- Translate complex field friction into platform upgrades, delivering prescriptive feedback and contributing directly back to core product features
- Ensure Day-1 production reliability, assisting customers in establishing sound deployment patterns, SLOs, and observability frameworks
Benefits
- Six weeks’ paid vacation
- Equity / stock options
- RRSP, 401(k), and Pension Scheme contributions
- Coverage for 100% of your insurance premiums across health, dental, vision, and travel
- Additional coverage for accessing mental health providers/services
- Six months of fully paid parental leave, including adoption and surrogacy
- Financial support for egg freezing and IVF in Canada and the UK
- A monthly fitness and wellness allowance
- Globally dispersed company that supports a remote work culture
- A $2,000 annual education benefit for professional development
- A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
- A monthly arts and culture allowance
- A monthly quality time allowance
- Are you energized by bridging the gap between frontier AI capability and complex customer reality
- Do you want to architect high-performance, resilient machine learning systems directly inside strategic enterprise environments
- Familiarity with distributed serving techniques, such as prefill/decode disaggregated serving, distributed KV caching, and multi-GPU workloads
- Exceptional customer-facing technical leadership, with a proven track record of scoping highly ambiguous problems and steering executive technical stakeholders
- Deep technical depth in production Kubernetes, including orchestration across multi-cluster and hybrid/multi-cloud ecosystems (GCP, Azure, AWS, OCI)
- Strong understanding of LLM architecture and inference dynamics, specifically how attention mechanisms, compute-bound prefill, and memory-bound decode phases dictate live performance
- Strong programming foundations in Python, Go, or C++, comfortable moving fluidly between backend data pipelines and infrastructure layer tuning
- The high agency, grit, and adaptability to write same-day production code while managing client-side deployment politics or blockers
- 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software in complex environments
- Hands-on experience with modern inference engines and optimization stacks (e.g., vLLM, TensorRT-LLM, lmd, or Dynamo)
Skills
Sign in to read and join the discussion.