Staff Software Engineer in Test (Modeling and Simulation)
Posted Updated
Description
- Code Metal is strengthening its Quality Engineering capabilities to make software quality more measurable, repeatable, and actionable
- We are seeking a founding Staff Quality Engineer (SDET) for Modeling & Simulation, within the centralized Quality Engineering function
- Our Modeling & Simulation team builds wargaming and decision-support platform that helps the U.S. Department of War (DoW) evaluate mission- and campaign-level force-design decisions
- The system combines backend APIs, asynchronous workers, geospatial and sensing models, authenticated multi-team play (including fog of war), external simulation integrations, and emerging AI-assisted capabilities
- The platform already has automated unit and integration coverage, but its growing combination of distributed services, stateful simulation, external integrations, probabilistic behavior, and high-consequence user workflows requires a more systematic validation architecture
- This role will connect and extend that foundation into reproducible quality evidence across services, models, and end-to-end workflows
- You will design and build the platform’s automated validation system, defining which behaviors require deterministic verification, statistical or model-based evaluation, or human review, and how limitations and residual risk are communicated as release evidence
- This is a hands-on software engineering role: you will build foundational test infrastructure, write critical tests, and help engineers build quality into their work
- Translate Quality Engineering strategy into a risk-based validation approach for Modeling & Simulation across unit, integration, contract, end-to-end, performance, resilience, and security-critical workflows
- Build reliable test infrastructure, including deterministic fixtures, isolated test data, service and event-contract tests, integration harnesses, versioned reference scenarios, deterministic replay, and focused browser automation for critical user journeys
- Improve CI feedback through effective test partitioning, parallel execution, caching, failure attribution, and flaky-test control, in partnership with shared platform owners
- Write behavior-focused tests for high-risk planning, sensing, simulation, authorization, state-transition, and recovery workflows
- Partner with simulation and model owners to turn intended behavior, assumptions, and known limitations into reusable reference scenarios, regression baselines, characterization or differential tests, and reproducible quality evidence
- Define validation and release criteria for simulation and emerging AI-assisted workflows, combining deterministic assertions, contracts, reference checks, statistical or model-based evaluation, failure-recovery testing, and appropriate human oversight
- Establish repeatable performance and reliability validation with measurable objectives and clear release evidence
- Participate in design and code reviews, identify testability risks early, and mentor engineers in testing complex simulation, distributed-system, and AI-assisted changes without becoming an approval bottleneck
- Turn lessons from Modeling & Simulation into reusable quality bars, reporting practices, testing standards, and validation infrastructure for the centralized Quality Engineering function
Benefits
- Relocation assistance for qualifying employees
- Flexible hybrid work arrangement
- Retirement Plan (401k, IRA)
- Health Care Plan (Medical, Dental & Vision)
- Paid Time Off (Uncapped Vacation, plus Sick & Public Holidays)
- Demonstrated experience designing risk-based test strategies across multiple layers of a distributed system
- Significant experience as an SDET, test-infrastructure engineer, quality architect, or software engineer responsible for automated validation of production systems
- Experience validating complex systems with partially nondeterministic behavior, such as simulation, statistical models, ML, or AI-assisted workflows, and sound judgment about deterministic checks, reference evidence, statistical evaluation, and human oversight
- Deep experience building Python test infrastructure using pytest or comparable frameworks, including asynchronous testing, fixtures and factories, parallel execution, and CI integration
- Strong production programming skills in Python
- Experience building and improving CI workflows with GitHub Actions or a comparable platform, including test isolation, failure diagnosis, and flaky-test control
- Ability to set technical direction, write design documents, mentor engineers, and lead multi-team initiatives through influence
- Hands-on experience testing Docker-based or comparable containerized, multi-service applications, including APIs, databases, messaging, authentication, authorization, and data isolation
- Hands-on experience evaluating AI-assisted or agentic systems, including tool-calling or multi-step workflows, and understanding risks such as nondeterminism, unsupported outputs, prompt or context manipulation, unsafe tool use, hidden state, and silent failure
- Experience with performance testing, generated test data, and API or schema validation
- Experience validating simulation, geospatial, event-driven, or other stateful systems, including reproducible deterministic and probabilistic behavior
- Experience testing TypeScript and React applications with Playwright or equivalent tools
- Experience testing cloud-based, containerized systems using logs, metrics, traces, and retained test artifacts
- Equivalent depth of experience is valued over a rigid year count
- Typically 8+ years of relevant software engineering, test infrastructure, or quality engineering experience, with demonstrated Staff-level scope and impact
- Familiarity with constructive simulation, distributed simulation standards, or mission-analysis software
- You have turned quality strategy into test architecture for a product area, led that work through design docs and influence (not people management), and shipped the infrastructure and tests yourself
- Familiarity with testing or debugging C++ simulation or systems code
- Experience in a regulated, high-assurance, or mission-critical domain
- Experience using AI coding assistants or agents throughout the software development lifecycle and defining quality controls for AI-assisted changes, such as scoped workflows, independently verifiable tests, human code review, CI checks, and risk-based release evidence
- Eligible to obtain and maintain a U.S. Secret security clearance
Skills
Sign in to read and join the discussion.