Software Engineer (AI Evaluation)

• AI is at the heart of what Board Intelligence builds. Boards and senior leaders rely on our AI to support their most important decisions, so the quality of its output has to be close to perfect
• We are looking for a Software Engineer to join our AI team as its dedicated engineer. The team includes prompt engineers and subject matter experts, and you will build the systems they need. Your first priority is our AI evaluation suite: the tools we use to measure and improve the quality of AI output across our products. Over time, you may move on to building one of our AI-powered products
• You do not need deep AI or machine learning experience; the team will help you get up to speed quickly. What matters is that you are a strong full-stack engineer who is comfortable in ambiguity and happy to build largely independently, in a field where good practice is still being defined
• Design and build our AI evaluation suite, the systems we use to measure and improve the quality of AI output
• Build across the full stack yourself, from data pipelines and backend services to the front-end tools our experts use
• Work closely with prompt engineers and subject matter experts, turning their knowledge into tests, metrics and tooling
• Help define how we measure AI quality, in a field where practices are still forming
• Make evaluation processes and results automated as part of our SDLC, so product teams can ship with confidence
• Over time, contribute to building our AI-powered products
• What good looks like in this role
• After three months: the first version of the evaluation suite is running and our engineering teams are using it
• After six months: evaluations cover all of our AI products and are integrated into the SDLC
• After twelve months: evaluation is a standard part of how we ship AI, and you are a trusted voice on AI quality across engineering

Benefits

• 26 days holiday each calendar year
• Bupa health & dental cover (including Babylon Virtual GP)
• Group life insurance (4x annual salary)
• Employee Assistance Program and Bereavement Counselling & Probate Helpline
• Daily breakfast, drinks, fruit and snacks
• Regular training & development, mini MBA series, lunch & learns
• AIG Smart Health (virtual GP app)
• Season Ticket Travel Card Loan and Cycle to Work Scheme
• Eye test vouchers and Flu Jabs
• Gym membership discounts
• Regular company socials. Friday drinks
• Commission plan & pension scheme

• Experience collaborating with non-engineers and turning their expertise into working software
• Fluent written and spoken English
• Experience building AI powered products or tooling
• Upper mid-level to senior professional software engineering experience
• We are looking for a strong all-round engineer who is comfortable working independently, enjoys ambiguity and wants to work at the forefront of AI quality
• Comfort working independently and in ambiguity, setting your own direction
• Full-stack experience across front end and back end
• A strong focus on quality, testing and attention to detail
• Experience using AI coding tools in your day-to-day work
• Experience at a startup or in a small team where you built things end to end
• Experience building internal tools, testing frameworks or data pipelines
• Experience with Python and TypeScript
• Experience with building evaluation suites for AI products