Job Description

Functional Tester - Agentic AI Solution (Banking & Wealth Management)
Location: Noida
Experience Level: 2-4 years
Reports To: QA Lead / Product Manager
We are looking for an experienced Functional Tester to validate the business workflows, compliance logic, and customer-facing features of an Agentic AI solution built for the banking and wealth management domain. This solution leverages LLMs, retrieval-augmented generation (RAG), and domain ontologies to provide intelligent data retrieval, client interaction, and regulatory compliance automation.
Key Responsibilities
Functional Testing

• Understand functional and non-functional requirements for Agentic AI features (e.g., document Q&A, investment summarization, client onboarding, compliance tracking)

• Validate end-to-end user journeys across banking and wealth advisory use cases

• Test conversational workflows and agentic logic flows (multi-step LLM tasks)

• Verify outputs from LLMs for relevance, consistency, and context-awareness

• Ensure correct role-based access across advisors, relationship managers, and compliance officers

Test Planning & Execution

• Prepare detailed test plans, test cases, and traceability matrices from EPICs, features, and user stories

• Execute manual testing for user interfaces, APIs, and chat/agent interactions

• Perform regression testing across AI pipelines after prompt or model updates

• Log, track, and retest defects using tools like Azure DevOps, Jira, or TestRail

AI Model & RAG Testing

• Validate grounding of LLM responses from vector databases (e.g., Azure AI Search, Pinecone)

• Check performance of RAG pipelines including hallucination mitigation, context window boundaries, and fallback logic

• Test interaction of agent orchestration flows (LangChain/Autogen/semantic kernel)

• Verify prompt-based test cases for accuracy and coverage

Integration & Data Testing

• Validate PDF/statement extraction logic from custodians (e.g., Morgan Stanley, Pershing)

• Check structured data output (JSON, tables) from Azure Form Recognizer or OpenAI pipelines

• Ensure data integrity and lineage from ingestion ? processing ? user response

Must-Have Skills & Tools
Category
Tools/Tech
Test Management
Azure / AWA DevOps, Jira, TestRail, Azure Board
Test Types
Functional, UI, Integration, AI behavior
Domain Knowledge
Wealth Mgmt workflows, KYC, portfolio views, regulatory rules
AI Concepts
LLM output validation, RAG testing, prompt engineering basics
Data
JSON validation, SQL for test data setup
Tools (Optional)
Postman, Python (for test automation scripts), Excel-based test mapping
Requirements
Preferred Qualifications

• Bachelor's degree in Engineering, Finance, or Computer Science

• Experience testing AI/ML solutions or conversational AI products

• Familiarity with Azure AI stack, OpenAI, or AWS Bedrock

• Exposure to compliance-heavy domains like banking, wealth, or insurance

• Ability to work with cross-functional Agile teams (product, engineering, AI/ML, compliance)

Nice-to-Have Skills

• Experience writing prompt-based test cases for LLMs

• Familiarity with agentic workflows (e.g., task chaining, autonomy testing)

• Understanding of financial statement formats (portfolio summaries, trade confirms)

• Basic scripting to validate data output (e.g.,Python, bash)

Skills Required