Senior Software Engineer - LLM Ops & Evals

Every AI call in DataSnipper goes through us. Product teams do not talk to model providers directly, they talk to our gateway. We are responsible for how inference is routed, how it fails over, what it costs, and how anyone can tell whether the output is any good.

The second half of the job is evaluation. We are building the platform teams use to measure AI quality: versioned datasets, experiment tracking, and evaluation runs they can act on. It is a hard problem and largely an open one, so you will have real influence over how we solve it.

This is a small team with a large blast radius. You will own real production systems, set the standards other teams build against, and see your work in front of hundreds of thousands of users in audit and finance.

About DataSnipper

The hardest AI problems aren't about getting an answer fast. They're about getting one people can trust.

We believe trust is essential to how society works. Every day, we rely on information that's been independently verified, from financial statements to the products we buy, to the sustainability claims companies make. Behind that trust are audit and finance professionals who examine evidence, verify information and hold organizations accountable.

At DataSnipper, we build AI solutions that empower those professionals. By making evidence easier to trace and verify, we help them focus on the judgment and expertise that make their work so important.

For our product and engineering team, this is where things get exciting. In audit, a plausible answer isn't good enough. Every output has to be explainable and traceable back to its source. We've just launched a brand new agentic platform, Alwin by DataSnipper. AI agents running in the cloud across hundreds of documents at once, reasoning across PDF, Word and Excel, with a full audit trail behind every step. There are lots of greenfield problems ahead, and real room to shape what we build and how we build it.

And we build at scale. More than 600,000 professionals across 3,000+ organizations in 175 countries rely on DataSnipper, including Fortune 500 companies, government agencies and all Big Four audit firms. In 2025 alone, the DataSnipper platform delivered more than $1.4 billion in productivity savings to is customers. Backed by a $100 million Series B led by Index Ventures, at a $1 billion valuation, and as proud members of 1% for the Planet, we care about how we grow, not just how fast.

If you want to build AI that professionals trust with work that really matters, this is your chance to help shape the future of audit and finance and do the best work of your career along the way.

What you will do

Technical Delivery

• Own the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers

• Deploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code

• Build the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run

• Own the infrastructure for async and long-running AI workloads

Reliability, Security & On-Call

• Own observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case

• Take part in the on-call rotation, run incidents, and close the follow-ups

• Implement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO

Collaboration & Impact

• Define and maintain clean integration contracts between the platform and the teams that consume it

• Partner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue

What you will bring

Must-Have

• 5+ years in backend or platform engineering, with strong production Python

• Experience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution

• Experience with cloud at the infrastructure level and infrastructure as code (we run across Azure and GCP with Terraform)

• Experience running a shared service in production: on-call, incidents, postmortems, SLOs

• Hands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts

• Comfort with privacy and compliance work: PII handling, anonymisation, retention, access control

• Experience building platform or shared-service capabilities consumed by multiple internal teams

Nice-to-Have

• Experience with LLM or agent evaluation

• Temporal or another durable workflow engine

• Self-hosted inference, capacity planning, load testing

• Synthetic data or document anonymisation pipelines

• Document AI: VLMs, OCR, structured extraction and the metrics that go with it

• Domain experience in audit, accounting or fintech

What we expect

• Ownership: You own work end-to-end, anticipate issues, and ensure high-quality delivery without close supervision

• Growth Mindset: You encourage open feedback exchange and provide clear, balanced feedback that helps others grow

• Collaboration: You build strong cross-functional relationships and influence peers through expertise, data, and empathy

• Adaptability: You navigate ambiguity calmly, model positive behavior, and help peers adjust through clear communication

• Judgment: You exercise sound judgment in ambiguous situations, balance speed and accuracy, and adjust priorities proactively

Recruitment steps

• Recruiter screen

• Hiring Manager interview

• Peer programming session

• System design interview

• Final interviews with Engineering leadership

At DataSnipper, we believe great ideas come from different perspectives, experiences, and backgrounds. We are committed to building an inclusive workplace and providing every candidate with a fair and equitable hiring experience.

Our hiring decisions are merit-based and grounded in the skills, experience, potential, and capabilities relevant to the role.

We welcome applicants regardless of age, race, ethnicity, nationality, gender, gender identity or expression, sexual orientation, religion, disability, or any other characteristic that makes you, you.

What matters to us is what you can bring to the role, the team, and DataSnipper. 🚀