Job Description

Design and execute test strategies for Gen AI models including LLMs image generators and multimodal systems




Create and maintain test cases and prompt libraries to evaluate model output quality eg relevance coherence creativity factual accuracy




Test model behaviour across different scenarios languages and user intents eg hallucinations bias toxicity safety risks




Perform manual and automated testing of Gen AI APIs chatbots and content generation tools




Validate input output behaviour prompt testing response evaluation token level analysis




Collaborate with engineering teams to implement prompt evaluation frameworks log monitoring and output scoring systems




Track model performance metrics BLEU ROUGE perplexity toxicity score etc and suggest areas for improvement




Conduct fairness bias and safety testing to ensure compliance with ethical and regulatory guidelines




Assist in evaluating AB experiments and fine tuning model behaviours




Document issues clearly and contribute to root cause analysis with dev teams.

About Virtusa

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth -- one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.


Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.


Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.