Job Description

We are looking for a highly skilled Senior QA Engineer/Lead with strong experience in testing Generative AI and Agentic AI solutions across enterprise platforms. In this role, you will ensure the quality, reliability, and performance of AI-powered capabilities such as autonomous agents, conversational experiences, RAG-based intelligence, and workflow automation. You will work closely with cross-functional teams to define testing strategies, build AI-aware automation frameworks, and validate implementations at scale across OCI, ServiceNow, Oracle Fusion, and custom enterprise applications
Responsibilities:

• Design and execute comprehensive test strategies to ensure the quality, reliability, and effectiveness of Generative AI and Agentic AI capabilities across enterprise use cases.
• Validate AI-driven workflows across ServiceNow ITSM modules, Oracle Fusion processes, and other enterprise applications, leveraging automation and conversational interactions.
• Test RAG-based intelligence for reliable context grounding and consistent retrieval performance
• Test Agentic AI behaviors to ensure appropriate task execution, autonomous decision quality, and safe workflow orchestration, and evaluate performance, scalability, and reliability of AI agents integrated across cloud and enterprise environments
• Build and maintain automation frameworks for API, functional, web, and backend testing. Integrate automation into CI/CD pipelines (Azure DevOps / GitHub / Jenkins)
• Create and maintain QA documentation, including test plans, test cases, traceability matrices, and execution reports
• Conduct performance and reliability testing for AI endpoints and automations across workloads
• Diagnose failures, analyze patterns, and drive issue resolution by collaborating with platform and engineering teams
• Drive continuous improvement by adopting advanced AI testing practices and metrics suitable for nondeterministic systems
• Stay current with advancements in AI testing tools, automation accelerators, and emerging testing standards

Requirements:

• 8+ years of experience in QA with 2+ years of testing Generative AI / Agentic AI / ML-based systems
• Hands-on experience validating AI workflows in one or more enterprise ecosystems such as ServiceNow, Oracle Fusion, or cloud-native platforms
• Strong scripting and automation skills (Python preferred) with experience in frameworks like Selenium, Playwright, REST Assured, or PyTest
• Knowledge of LLM architectures, NLP behavior, RAG workflows, and vector retrieval validation
• Proficiency with Git, CI/CD pipelines, Docker/Kubernetes for enterprise automation support
• Expert in QA including test design, defect lifecycle, and quality metrics
• Knowledge of Responsible AI, bias/fairness testing, and governance practices
• Strong communication skills to explain testing results and quality improvements clearly
• Ability to guide and support team members on testing best practices

Who We are...
SmarTek21, founded in 2006, is an innovative digital transformation leader dedicated to empowering organizations through design-led, data-centric solutions. We believe that exceptional user experiences and intelligent data-driven insights are pivotal in shaping the future of business. At the heart of our approach is Design-Led Engineering, where we combine strategic design principles with cutting-edge technology to create tailored solutions that deliver significant business outcomes. Our commitment to accelerating digital transformation includes specialized expertise in Agile DevOps, Data Engineering and analytics, Testing Automation and support, and comprehensive Managed Application and Infrastructure Services. We've evolved from our consulting and development roots into a full-spectrum service provider, integrating advanced technologies like Generative AI (GenAI) to facilitate our clients' seamless, transformative digital journeys. A passion unites our dynamic team of technology visionaries and business strategists for innovation and excellence. At SmarTek21, you'll join a collaborative environment dedicated to shaping the digital landscape and driving impactful results for organizations worldwide.
SmarTek21 is committed to fostering a diverse and inclusive environment. We encourage applications from individuals of all backgrounds and experiences.
Powered by JazzHR

Skills Required