We are seeking a Data Science focused QA engineer to develop next-generation Security Analytics products. You will work closely with Data scientists,engineersand product managers to design andoptimizeAI driven security solutions.
AsQAengineer, the ideal candidate has a strong background in Backend engineering, system integrations,ML,AIand data pipelines.
Responsibilities (QA Engineer – Data Science / ML)
- Establish QA best practices for Traditional ML and Generative AI workflows, including:
Establish QA best practices for Traditional ML and Generative AI workflows, including:
- Functional and regression testing of ML pipelines usingpytestand Airflow/Dagstertest utilitiesand API testing tools (e.g., Postman,pytest-httpx).
Functional and regression testing of ML pipelines usingpytestand Airflow/Dagstertest utilitiesand API testing tools (e.g., Postman,pytest-httpx).
- Validate data contracts, schemas, and API compatibility across services usingPandera, and custom validation rules.
Validate data contracts, schemas, and API compatibility across services usingPandera, and custom validation rules.
- Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions
Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions
- Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests.
Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests.
- Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).
Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).
- Implement LLM-specific testing, including:
Implement LLM-specific testing, including:
- Prompt and response validation, determinism checks, and regression testing usingLangSmith.
Prompt and response validation, determinism checks, and regression testing usingLangSmith.
- Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and/orrule-based checks.
Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and/orrule-based checks.
- Cost, token usage, and timeout monitoring for GenAI workflows
Cost, token usage, and timeout monitoring for GenAI workflows
- Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.
Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.
- BS or MSinComputer Science or a related field.
in .
- 2-5 yearsof experience in Dataor MachineLearningprojects.
of experience in Dataor Machine projects.
-
Familiarity andexperienceof GenAI applicationsand tools -PyTorch,LangChain,vLLMetc.
-
Demonstratesa commitment to continuous learning in this rapidly evolving field.
Demonstratesa commitment to continuous learning in this rapidly evolving field.
- Tools listed inthe responsibilitiessection.
Tools listed inthe responsibilitiessection.