About the role
A small software development company in Canada is hiring for this role. You will own quality for AI agents, building the evaluation frameworks and release gates that show whether an agent is reliable and safe to ship.
What you'll do
- Build and maintain automated evaluation frameworks, test suites and simulators for AI agents
- Set up CI/CD quality gates so only agents that meet the bar get released
- Run hands-on exploratory, regression and adversarial testing
- Define quality metrics and release criteria, and manage evaluation datasets
- Investigate failures and work with engineering and deployment teams to improve agent reliability and safety
What you bring
- Senior-level experience as a software development engineer in test or in QA automation
- Experience building automated test suites and CI/CD quality gates
- A practical approach to testing AI systems, including adversarial and regression testing
- Comfort defining metrics and release criteria and working with evaluation datasets
- Strong collaboration skills with engineering and deployment teams
Details
- Location: Canada
- Type: Full-time
Apply below and a Traverse recruiter will be in touch.