AI Agentic Tester
Denver, MO (Remote)
**
**
Must-Have Skills
-
AI Agentic Testing
-
Functional Testing
-
Generative AI Testing
-
Large Language Models (LLMs)
-
AI Agents & Agentic AI
-
Retrieval-Augmented Generation (RAG)
-
Prompt Engineering Validation
-
AI Model Validation & Evaluation
-
API Testing (Postman, REST APIs, Swagger)
-
Python
-
Test Automation (Selenium / Playwright / PyTest)
-
AI Evaluation Metrics (Accuracy, Hallucination, Relevance, Consistency)
-
End-to-End Workflow Testing
-
Multi-Agent Testing
-
Azure OpenAI / AWS Bedrock / Gemini
Core Responsibilities
-
Perform functional testing of AI agent workflows and end-to-end AI applications.
-
Validate LLM responses for accuracy, relevance, consistency, and hallucination rates.
-
Test RAG pipelines, prompt engineering, and AI agent orchestration.
-
Execute API testing using Postman, Swagger, and REST APIs.
-
Develop and maintain automation scripts using Python, Selenium, Playwright, or PyTest.
-
Validate multi-agent interactions, tool calling, and workflow execution.
-
Apply AI testing methodologies and evaluation metrics to ensure response quality.
-
Work with Azure OpenAI, AWS Bedrock, Gemini, or similar AI platforms to validate AI solutions.