Skip to content

d-Matrix

Principal System Software Engineer (AI Inference Execution)

Posted Updated

Description

  • The role: AI/ML System Software Engineer, Senior Staff
  • The role requires you to be part of the team that helps productize the SW stack for our AI compute engine
  • As part of the software team, you will be responsible for the development, enhancement, and maintenance of the next-generation AI deployment software
  • You have had past experience working across all aspects of the full-stack toolchain and understand the nuances of what it takes to optimize and trade-off various aspects of hardware-software co-design
  • You are able to build and scale software deliverables in a tight development window
  • You will work with a team of system software experts to build out the deployment infrastructure, working closely with other software (ML, compilers) and hardware experts in the company

Benefits

  • Equity
  • Health care
  • Flexible time-off
  • Paid paternity leave
  • 401k retirement plan
  • Remote & hybrid working model
  • Work with world-leading engineers and researchers
  • Team-building activities
  • On-site fitness center
  • Free food and snacks
  • Happy hours
  • Peer coaching & development
  • BS in Computer Science, Engineering, Math, Physics, or related degree with 12+ years of industry software development experience and MS in Computer Science, Engineering, Math, Physics, or related degree preferred with 6+ years
  • Experience with distributed, high-performance software design and implementation
  • Self-motivated team player with a strong sense of ownership and leadership
  • Proficient in C/C++/Python development in Linux environment and using standard development tools
  • Strong grasp of system software, data structures, computer architecture, and machine learning fundamentals
  • MS or PhD in Computer Science, Electrical Engineering, or related fields
  • Experience with inference servers/model serving frameworks (such as TensorRT-LLM, vLLM, SGLang, etc.)
  • Experience with deep learning frameworks (such as PyTorch and TensorFlow)
  • Experience with deep learning runtimes (such as ONNX Runtime, TensorRT, etc.)
  • Experience with software testing fundamentals
  • Experience deploying ML workloads (LLMs, VLMs, NLP, etc.) on distributed systems
  • Experience with distributed systems collectives such as NCCL and OpenMPI
  • Experience with Kubernetes, Ray, or other MLOps tools and techniques used from definition to deployment
  • Prior startup, small team, or incubation experience
  • Work experience at a cloud provider or AI compute/subsystem company

Skills

What Principal Software Engineering jobs ask for — and how much of it you have →
Apply

Apply to Principal System Software Engineer (AI Inference Execution) at d-Matrix

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

Software Engineering jobs by country — openings, pay and top skills →