Sr. Applied Scientist, Applied AI & Data Architecture
Posted Updated
Description
Application deadline: Nov 15, 2026
Do you want to define how enterprises customize their own models ? Do you enjoy work that spans data curation, post-training, evaluation, and production deployment of customized models?
As a Senior Applied Scientist in Amazon Web Services (AWS), you will build that path from customization to trusted evaluation. You will prove it with customers on their own data. You will help customers decide when and how to customize foundation models, using techniques that range from parameter-efficient fine-tuning and distillation to preference optimization and reinforcement fine-tuning. You will turn that work into reusable recipes, evaluation methods, and open-source accelerators that let builders customize models reliably and cost-effectively.
The role splits roughly 50/50 between direct customer engagements and building leverage through reusable science assets, open source, and thought leadership.
- Customer work: You will partner with customer science and engineering teams on their hardest customization problems. You'll frame the problem, choose the right approach, design the evaluation, and work hands-on to get customized models into production.
- Building time : You will develop reference implementations and training recipes, contribute to open-source projects, and publish findings that raise the bar for the broader community. You will work closely with AWS service teams to close the loop between what customers need and what AWS builds.
You will be the team's scientific bar-raiser. You will set evaluation standards, review modeling choices, and publish what you learn, internally and externally. You will work closely with the service teams so that the patterns we prove in the field become native features.
This is a global role within the Solutions Architect organization. You will maintain deep relationships with AI service and science teams and collaborate with specialists and builder customers across regions to stay current on the state of the art and influence roadmaps.
Key job responsibilities
- Own the evaluation methodology. Design the evaluation methodology behind every model-selection and customization decision. Score quality, cost, latency, and safety in a way that is reproducible, statistically sound, and calibrated to each customer's quality bar.
- Lead customization engagements end to end. Drive work from data curation and synthetic data generation through fine-tuning, distillation, continued pre-training, and reinforcement learning. Re-score every change against the same harness so customers see exactly what they gained.
- Invent where no textbook solution exists. Develop new techniques such as reward design for agentic tasks, simulation-based evaluation, and efficient serving of small decision models, then package them as reusable code any Solutions Architect can run.
- Turn engagement learnings into reusable assets. Build reference architectures, open-source accelerators, workshops, enablement materials, blogs, and papers that form a repeatable path from evaluation through deployment.
- Advise field teams and customers. Act as the scientific advisor on trade-offs among prompting, retrieval, customization, and frontier models, including knowing when not to customize.
- Mentor and influence across teams. Mentor engineers and Solutions Architects in ML methodology. Turn customer evidence into feature requests and proposals for AWS service teams that make field-proven patterns native product capabilities.
A day in the life
You start the morning reviewing a customer's distillation results, where a smaller model is closing the gap with a much larger one. The gains cluster in one task slice, so you flag possible data leakage before the design review. Afterward, you add a held-out evaluation set and a capability-regression check to the training recipe. In the afternoon, you share engagement findings with a product team to shape how evaluation fits into the customization workflow. You wrap up by refining an open-source evaluation harness you'll release alongside a customer workshop.
About the team
Applied AI and Data Architects (AIDA) is a team of specialized AI prototypers with a product mindset. The team builds a portfolio of domain-specific Tech Field Products in areas such as semantics, data engineering, ML, AI security/governance/agent-ops, and DevOps. Model customization and evaluation is one of the team's focus areas. Its goal is to make foundation and open weight model customization repeatable, measurable, and accessible to builders. Each product is grounded in customer reality. The team earns the right to guide by building alongside customers.
Do you want to define how enterprises customize their own models ? Do you enjoy work that spans data curation, post-training, evaluation, and production deployment of customized models?
As a Senior Applied Scientist in Amazon Web Services (AWS), you will build that path from customization to trusted evaluation. You will prove it with customers on their own data. You will help customers decide when and how to customize foundation models, using techniques that range from parameter-efficient fine-tuning and distillation to preference optimization and reinforcement fine-tuning. You will turn that work into reusable recipes, evaluation methods, and open-source accelerators that let builders customize models reliably and cost-effectively.
The role splits roughly 50/50 between direct customer engagements and building leverage through reusable science assets, open source, and thought leadership.
- Customer work: You will partner with customer science and engineering teams on their hardest customization problems. You'll frame the problem, choose the right approach, design the evaluation, and work hands-on to get customized models into production.
- Building time : You will develop reference implementations and training recipes, contribute to open-source projects, and publish findings that raise the bar for the broader community. You will work closely with AWS service teams to close the loop between what customers need and what AWS builds.
You will be the team's scientific bar-raiser. You will set evaluation standards, review modeling choices, and publish what you learn, internally and externally. You will work closely with the service teams so that the patterns we prove in the field become native features.
This is a global role within the Solutions Architect organization. You will maintain deep relationships with AI service and science teams and collaborate with specialists and builder customers across regions to stay current on the state of the art and influence roadmaps.
Key job responsibilities
- Own the evaluation methodology. Design the evaluation methodology behind every model-selection and customization decision. Score quality, cost, latency, and safety in a way that is reproducible, statistically sound, and calibrated to each customer's quality bar.
- Lead customization engagements end to end. Drive work from data curation and synthetic data generation through fine-tuning, distillation, continued pre-training, and reinforcement learning. Re-score every change against the same harness so customers see exactly what they gained.
- Invent where no textbook solution exists. Develop new techniques such as reward design for agentic tasks, simulation-based evaluation, and efficient serving of small decision models, then package them as reusable code any Solutions Architect can run.
- Turn engagement learnings into reusable assets. Build reference architectures, open-source accelerators, workshops, enablement materials, blogs, and papers that form a repeatable path from evaluation through deployment.
- Advise field teams and customers. Act as the scientific advisor on trade-offs among prompting, retrieval, customization, and frontier models, including knowing when not to customize.
- Mentor and influence across teams. Mentor engineers and Solutions Architects in ML methodology. Turn customer evidence into feature requests and proposals for AWS service teams that make field-proven patterns native product capabilities.
A day in the life
You start the morning reviewing a customer's distillation results, where a smaller model is closing the gap with a much larger one. The gains cluster in one task slice, so you flag possible data leakage before the design review. Afterward, you add a held-out evaluation set and a capability-regression check to the training recipe. In the afternoon, you share engagement findings with a product team to shape how evaluation fits into the customization workflow. You wrap up by refining an open-source evaluation harness you'll release alongside a customer workshop.
About the team
Applied AI and Data Architects (AIDA) is a team of specialized AI prototypers with a product mindset. The team builds a portfolio of domain-specific Tech Field Products in areas such as semantics, data engineering, ML, AI security/governance/agent-ops, and DevOps. Model customization and evaluation is one of the team's focus areas. Its goal is to make foundation and open weight model customization repeatable, measurable, and accessible to builders. Each product is grounded in customer reality. The team earns the right to guide by building alongside customers.