Senior Engineer (Applied AI)

• Ro is building a team focused on shipping LLM-powered products across the patient experience, clinical operations, and internal tooling
• We’re hiring a Senior Engineer, Applied AI to join a new team we’re building from the ground up alongside a Senior AI Lead and an Engineering Manager
• You’ll own AI features from prototype through production and spend as much time improving workflows with clinical operators as writing code
• Most of your work will involve building orchestration layers, prompts, evaluations, retrieval systems, and tooling that make LLMs reliable enough for real-world healthcare operations
• Design, build, and ship production LLM powered features end-to-end
• Build the AI application layer, including prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and structured outputs
• Develop evaluation infrastructure, including regression suites, LLM-as-a-judge evaluations, synthetic datasets, human review workflows, and quality metrics that catch regressions before users do
• Design observability systems that measure accuracy, latency, cost, hallucination rates, and model behavior in production
• Build safety and reliability guardrails that ensure AI systems meet quality, compliance, and operational requirements
• Evaluate and integrate third-party tools, including orchestration frameworks, evaluation platforms, vector databases, and model providers; and make pragmatic build-versus-buy decisions
• Raise engineering standards across the team through technical leadership, code reviews, mentorship, and thoughtful system design
• Sit with EHR operators, Patient Advocates, and clinical teams to understand the workflows you’re automating
• Prototype quickly, ship early, measure outcomes, and iterate based on real-world usage

Benefits

• Medical, dental, and vision membership
• Equity for all team members
• Flexible time off
• Paid parental leave, and flexible work arrangement
• Well-stocked office with snacks and bi-weekly free lunches
• Gym, learning, and development stipends

• 5+ years building production software, with at least 1–2 years building and deploying LLM-powered applications in production (or equivalent depth through substantial side projects or open-source work)
• You’ve shipped AI features that served real users, and you can clearly articulate how you measured quality, reliability, latency, cost, and business impact
• Comfortable using AI-assisted development tools throughout the software development lifecycle
• You have strong product instincts and thrive in ambiguous environments, translating operational problems into technical solutions without relying on detailed specifications
• Strong Python engineer with experience building backend services and integrating modern AI APIs and frameworks
• Bonus: Experience with orchestration frameworks (LangChain, LangGraph, AgentCore), RAG architectures, vector databases, prompt/version management, and evaluation or observability platforms (e.g., LangSmith, Braintrust, Arize Phoenix)