Quality Assurance Analyst
New Yesterday
Join to apply for the Quality Assurance Analyst role at Clue Software
Join to apply for the Quality Assurance Analyst role at Clue Software
Get AI-powered advice on this job and more exclusive features.
This range is provided by Clue Software. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.
Base pay range
Direct message the job poster from Clue Software
- 25 days annual leave, plus flexible bank holidays and the opportunity to buy additional days
- Enhanced workplace Pension scheme - opt-in salary sacrifice scheme
- Life Insurance (3x annual salary)
- Employee Assistance Programme (EAP) and workplace wellbeing initiatives
- Flexible working hours and location, open to part-time/ condensed hours
- Flexible benefits, such as: Cycle to Work, volunteer days/ opportunities and charity events
- Enhanced parental leave packages and enhanced sick pay
- Training and development opportunities
- Engagement and celebration activities– anniversaries, birthdays, team building, company-wide events
Role specifics
- Salary range: Market rate
- Reporting to: Test Lead
- Key stakeholders: Tech and Product teams,
- Organisational Framework Level: 2
About You / Job Summary
We're looking for an experienced and curious QA Analyst to lead the testing of our AI and GenAI-powered features. This role goes beyond traditional SaaS QA — it requires a mindset for dealing with uncertainty, edge cases, model behaviour, and testing for quality when outcomes are not always predictable. If you're passionate about both automation and understanding how intelligent systems behave in the wild, we want to hear from you.
You’ll be embedded in a cross-functional scrum team, working closely with engineers, data scientists, and product managers to define and execute test strategies for AI features. You'll bring an automation-first mindset but be comfortable with exploratory and manual testing when required — especially when testing model-driven features. Your focus will include not just functional correctness, but also trust, safety, performance, and the overall quality of the AI experience.
Key Accountabilities
- Design, develop, and maintain test plans and automated test suites for AI/GenAI features across frontend and backend.
- Test complex AI-driven systems where responses may vary — validate logic, relevance, completeness, tone, and safety.
- Build automation frameworks using Playwright and Cucumber with a focus on maintainability and scalability.
- Develop tools/scripts to validate model outputs and compare results across model versions (e.g. regression drift, hallucination checks).
- Define and execute non-functional tests (e.g. performance under load, prompt injection resilience, fairness/bias testing).
- Log and manage defects effectively; collaborate in root cause analysis and prevention.
- Actively contribute to CI/CD testing pipelines to ensure fast, reliable releases.
- Stay current on trends in AI/ML testing and suggest improvements to our QA practices.
Key role measures
- Automation Coverage & Test Strategy Execution - Delivering and maintaining high levels of automated test coverage across AI/GenAI features, with a focus on scalable and maintainable test suites using Playwright and Cucumber.
- Model Output Quality & Safety Validation - Proactively identifying and validating issues in AI model outputs - including hallucinations, bias, and prompt injection vulnerabilities - to ensure safe, relevant, and trustworthy user experiences.
- CI/CD Integration & Test Reliability - Contributing to fast, stable releases by integrating tests into CI/CD pipelines and maintaining a low rate of flaky or failing tests.
- Exploratory Testing & Edge Case Discovery - Uncovering subtle bugs and edge cases through thoughtful exploratory testing, especially in areas where AI behaviour is less predictable - making a meaningful impact from early in the role.
Experience and skills
Our ideal candidate would have experience in the following areas:
- Proven experience testing AI/ML-powered systems, ideally including GenAI features (e.g. LLMs, chatbots, classifiers).
- Strong proficiency with Playwright, Cucumber, and automation-first test strategies.
- Understanding of common GenAI testing challenges: non-determinism, bias, hallucinations, injection attacks.
- Experience with prompt testing, output validation, or model comparison tools.
- Familiarity with REST APIs, JSON, and integrating model endpoints into test frameworks.
- Inquisitive and detail-oriented mindset — able to challenge assumptions and uncover subtle issues.
- Experience in Agile/Scrum environments and collaborative cross-functional teams.
- Familiarity with CI/CD tooling (e.g. Azure DevOps, GitHub Actions, or similar).
- Strong written and verbal communication skills.
Nice to Have:
- Background in machine learning or data science.
- Experience testing ethical/guardrail functionality in AI systems.
- Knowledge of data labelling, synthetic data generation, or test data management in AI contexts.
- Familiarity with tools like OpenAI’s evals, LangChain testing modules, or similar frameworks.
Diversity, Equity and Inclusion
If you’re excited about this role but your experience doesn’t align perfectly, we encourage you to apply anyway and tell us more about yourself. You may be just the right candidate for this or other roles.
We believe that seeing the world from all sorts of angles makes life better for all. We want you to know that the things that make you an individual, like your identity, age, ethnicity, religion, ability and background, are things that we choose to celebrate and support.
We are a scale-up company, and as we continue to grow, we are passionate that having a diverse, inclusive and authentic workplace will remain at our core. We are creating an inclusive environment where our people can thrive.
Our values are aligned and at the heart of everything we do. We are respectful, united, rigorous, relentless and ethical.
Seniority level
Seniority level
Associate
Employment type
Employment type
Full-time
Job function
Job function
Engineering and AnalystIndustries
Software Development
Referrals increase your chances of interviewing at Clue Software by 2x
Sign in to set job alerts for “Quality Assurance Analyst” roles.
Bristol, England, United Kingdom 5 days ago
Bristol, England, United Kingdom 2 weeks ago
Greater Bristol Area, United Kingdom 6 days ago
Bristol, England, United Kingdom 3 weeks ago
Bristol, England, United Kingdom 1 week ago
Process Assurance Engineer - Software Quality
Bristol, England, United Kingdom 1 month ago
Bristol, England, United Kingdom 2 weeks ago
Process Assurance Engineer - Software Quality
Bristol, England, United Kingdom 4 days ago
Greater Bristol Area, United Kingdom 2 weeks ago
Bristol, England, United Kingdom 4 days ago
Bristol, England, United Kingdom 1 week ago
Bristol, England, United Kingdom 1 week ago
Bristol, England, United Kingdom 1 month ago
Senior Finance Analyst (Finance Business Partnering)
Bristol, England, United Kingdom 2 weeks ago
Bristol, England, United Kingdom 6 days ago
Bristol, England, United Kingdom 6 days ago
Bristol, England, United Kingdom 13 hours ago
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr- Location:
- Bristol, England, United Kingdom
- Salary:
- £60,000 - £80,000
- Job Type:
- PartTime
- Category:
- Management & Operations
We found some similar jobs based on your search
-
New Yesterday
Quality Assurance Analyst
-
Bristol, England, United Kingdom
-
£60,000 - £80,000
- Management & Operations
Join to apply for the Quality Assurance Analyst role at Clue Software Join to apply for the Quality Assurance Analyst role at Clue Software Get AI-powered advice on this job and more exclusive features. This range is provided by Clue Software. ...
More Details -
-
New Yesterday
Quality Assurance ESG Analyst
-
Belfast, Northern Ireland, United Kingdom
-
£60,000 - £80,000
- Management & Operations
Inspired PLC Inspired PLC is a UK market leading technology enabled provider of energy and sustainability advisory services. Since 2000, it has grown in scale and capability and is servicing over 3,400 UK and Irish clients across all sectors. To enab...
More Details -
-
New Yesterday
PMO Quality Assurance Analyst
-
England, United Kingdom
-
£60,000 - £80,000
- Management & Operations
Join to apply for the PMO Quality Assurance Analyst role at BAM UK & Ireland Join to apply for the PMO Quality Assurance Analyst role at BAM UK & Ireland Direct message the job poster from BAM UK & Ireland Recruitment Manager @ BAM UK & Ireland...
More Details -
-
New Yesterday
Quality Assurance ESG Analyst
-
West Midlands Combined Authority, England, United Kingdom
-
£60,000 - £80,000
- Management & Operations
Inspired PLC Inspired PLC is a UK market leading technology enabled provider of energy and sustainability advisory services. Since 2000, it has grown in scale and capability and is servicing over 3,400 UK and Irish clients across all sectors. To enab...
More Details -
-
3 Days Old
IT Quality Assurance and Compliance Analyst
-
Wolverhampton, England, United Kingdom
-
£60,000 - £80,000
- Management & Operations
IT Quality Assurance and Compliance Analyst Full-time Eurofins Scientific is an international life sciences company, providing analytical testing services to ensure the safety and authenticity of products across various industries. We are committed...
More Details -
-
6 Days Old
Quality Assurance Test Analyst
-
Exeter, England, United Kingdom
-
£60,000 - £80,000
- Management & Operations
QA Test Analyst Exeter, Devon (Hybrid-working - minimum of x2 days in the office, you must be based locally) Up to £40,000 per year The Opportunity: My client based in the sustainability sector are currently on the lookout for a QA Test Analyst where...
More Details -