Skip to content

Cohere

NewBe an early applicant

Forward Deployed Engineer (Inference)

Posted Updated

Description

  • We are looking for a Staff Forward Deployed Engineer (FDE) to join our team
  • The team is responsible for taking our large language models and making them work reliably, securely, and at peak performance within our customers’ custom infrastructure and data perimeters
  • In this role, you will act as the principal technical translation layer between our core engineering teams and our strategic partners
  • You will own the architecture for highly ambiguous, customer-specific inference challenges—ranging from multi-tenant model serving to multi-datacenter disaster recovery and custom hardware validation
  • You will lead technical adoption end-to-end, ensuring our models achieve exceptionally low latency, high throughput, and seamless resilience in the field
  • Embed directly with customer engineering teams to architect the “connective tissue” between our AI models and their live, complex production environments
  • Design advanced model serving topologies, implementing multi-tenant architectures, intelligent model routing, and cross-datacenter failover strategies for multi-cluster environments
  • Optimize “AI-in-a-box” and private deployments, balancing localized SRE constraints like Kubernetes multi-cluster scale with raw model performance
  • Conduct rigorous benchmarking and profiling of customer LLM workloads, root-causing performance degradation (e.g., slow generation) via metrics, dashboards, and distributed logs
  • Accelerate hardware validation, collaborating with clients to run and support Cohere models on next-generation accelerators (GPUs, TPUs, AMD, and custom ASICs)
  • Translate complex field friction into platform upgrades, delivering prescriptive feedback and contributing directly back to core product features
  • Ensure Day-1 production reliability, assisting customers in establishing sound deployment patterns, SLOs, and observability frameworks

Benefits

  • Six weeks’ paid vacation
  • Equity / stock options
  • RRSP, 401(k), and Pension Scheme contributions
  • Coverage for 100% of your insurance premiums across health, dental, vision, and travel
  • Additional coverage for accessing mental health providers/services
  • Six months of fully paid parental leave, including adoption and surrogacy
  • Financial support for egg freezing and IVF in Canada and the UK
  • A monthly fitness and wellness allowance
  • Globally dispersed company that supports a remote work culture
  • A $2,000 annual education benefit for professional development
  • A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
  • A monthly arts and culture allowance
  • A monthly quality time allowance
  • Are you energized by bridging the gap between frontier AI capability and complex customer reality
  • Do you want to architect high-performance, resilient machine learning systems directly inside strategic enterprise environments
  • Familiarity with distributed serving techniques, such as prefill/decode disaggregated serving, distributed KV caching, and multi-GPU workloads
  • Exceptional customer-facing technical leadership, with a proven track record of scoping highly ambiguous problems and steering executive technical stakeholders
  • Deep technical depth in production Kubernetes, including orchestration across multi-cluster and hybrid/multi-cloud ecosystems (GCP, Azure, AWS, OCI)
  • Strong understanding of LLM architecture and inference dynamics, specifically how attention mechanisms, compute-bound prefill, and memory-bound decode phases dictate live performance
  • Strong programming foundations in Python, Go, or C++, comfortable moving fluidly between backend data pipelines and infrastructure layer tuning
  • The high agency, grit, and adaptability to write same-day production code while managing client-side deployment politics or blockers
  • 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software in complex environments
  • Hands-on experience with modern inference engines and optimization stacks (e.g., vLLM, TensorRT-LLM, lmd, or Dynamo)

Skills

Apply

Apply to Forward Deployed Engineer (Inference) at Cohere

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

Solutions Engineering jobs by country — openings, pay and top skills →