Senior QA (Gen AI - Must have) Exp: 5+ Years

Role: Senior QA (Gen AI - Must Have)

Mandatory Skills: Gen AI, Python, Backend/API test automation

Experience:

5+ Years

Location: Bengaluru or Pune Preferred

Detailed JD :

• Design, automate, and execute API test suites (functional, contract, performance, resilience)
• Define LLM evaluation strategies (performance testing, golden sets, guardrails, hallucination/consistency checks, relevance & latency KPIs); build automated eval pipelines tied to releases.
• Create PDLM deliverables: verification plans/protocols, test reports, risk-based coverage, V&V trace matrices to requirements and use cases.
• Integrate test infrastructure with Azure DevOps (CI/CD, test orchestration, quality gates, dashboards); enforce build-time gates for API and LLM quality.
• Partner with Product, Clinical, and R&D to refine requirements and acceptance criteria; ensure summaries render correctly for different user roles and workflows.

What is required from candidate?

• 5+ years in backend/API test automation (REST, JSON, auth), performance testing, and CI/CD.
• Hands-on with Python/TypeScript, Postman/Newman or REST-assured, PyTest/Jest, k6/JMeter, Docker.
• Experience validating ML/LLM systems (eval datasets, metrics, prompt/response testing, safety/guardrails).
• Strong documentation and traceability discipline for regulated product lifecycles (PDLM/QMS).
• Bonus: Azure DevOps pipelines, FHIR/HL7 familiarity, healthcare data privacy/security.
• Bonus: Frontend testing capabilities are preferred, but not part of the PDLM requirements.

#LI-MA1