Job Description

This role is for one of the Weekday's clients
Min Experience: 8 years
Location: Bengaluru
JobType: full-time
As a QA Automation Engineer, you will own end-to-end quality assurance for complex, distributed, and AI-powered systems. You will build robust automation frameworks, strengthen test coverage, and partner closely with engineering, data science, and product teams to ensure reliable, secure, and high-performing releases. Your expertise in Selenium, Rest Assured, Python, and modern testing methodologies will be central to accelerating development cycles while maintaining the highest standards of quality.
In addition, your experience with AI Agentic workflows, LLM evaluation, Retrieval-Augmented Generation (RAG) architectures, and PyTorch will help validate emerging AI capabilities, safeguard model behavior, and ensure deterministic, trustworthy outcomes across the platform.
Requirements
Key Responsibilities

• Design, develop, and maintain automation frameworks using Python, Selenium (preferred), Rest Assured, and Pytest/JUnit.
• Build and scale automated test suites for web applications, APIs, microservices, and AI/ML pipelines.
• Create effective test strategies, test plans, and robust regression suites aligned with product and release goals.
• Work closely with developers to define clear testability requirements and ensure high-quality code delivery.
• Develop automation for model validation, LLM behavior testing, RAG pipeline consistency, and agentic workflow scenarios.
• Implement data-driven testing approaches for AI/ML systems, incorporating tools such as PyTorch for environment setup and model handling.
• Perform end-to-end functional, integration, performance, and reliability testing for both traditional software components and new AI-driven features.
• Collaborate with Data Science and ML Engineering teams to design validation processes for model response correctness, drift checks, hallucination detection, and guardrail verification.
• Conduct root cause analysis on quality issues and drive corrective actions across engineering teams.
• Continuously enhance automation frameworks and CI/CD integrations to optimize execution times and stability.
• Ensure documentation, reporting, and coverage metrics are thorough, accurate, and continuously improved.

Required Skills & Experience

• 8-12 years of proven experience in software QA with a strong focus on automation.
• Expert-level hands-on experience in Python for building automation frameworks and tools.
• Strong experience with Selenium for web automation (preferred) and Rest Assured for API automation.
• Hands-on proficiency with test frameworks such as Pytest or JUnit.
• Working knowledge of PyTorch for interacting with models or ML pipelines.
• Demonstrated experience in testing AI Agentic systems, LLM-based features, and RAG architectures.
• Strong understanding of software testing principles, CI/CD pipelines, Git-based workflows, and Agile methodologies.
• Excellent problem-solving, debugging, and analytical skills with the ability to break down complex systems.
• Strong communication skills to collaborate effectively across teams and influence quality processes.

Skills Required