Senior QA Engineer

About the company

Joist AI is a technology company revolutionising the way professionals in the architecture, engineering, and construction (AEC) industry manage marketing and revenue operations. Our AI-powered software streamlines workflows, making it easier for teams to collaborate, innovate, and succeed.

About the Role

Senior QA Engineer

You won’t just be finding bugs; you will be architecting the testing frameworks that ensure our AI's "hit rate" and compliance checks are flawless for our enterprise customers. Reporting to the Engineering Manager, you will lead the QA effort to ensure Joist AI remains the best-in-class platform for the built world.

What You’ll Do

• QA Strategy & Leadership:Define and implement the end-to-end testing strategy for Joist AI, focusing on theProposal Processing PipelineandKnowledge Graphbuilding.
• AI/LLM Validation:Develop novel ways to test and monitor LLM-generated narratives and chat responses for accuracy, hallucination reduction, and compliance.
• Pipeline Oversight:Own the quality of our extraction services (Adobe API), object/image detection, and captioning services.
• Automation Architecture:Lead the development of automated test suites usingPlaywright(browser) andAPI automation(FastAPI) integrated into our CI/CD pipelines.
• Defect Intelligence:UseLinearto track defects through the lifecycle and collaborate with leads (Bala, Sharat, Chandan) to prioritize fixes based on customer impact.
• Cross-Functional Collaboration:Partner with Data Science teams to validate image processing (face/label detection) and with the GTM team to understand real-world proposal workflows.

QA Strategy & Leadership: Proposal Processing Pipeline Knowledge Graph

AI/LLM Validation:

Pipeline Oversight:

Automation Architecture: Playwright API automation

Defect Intelligence: Linear

Cross-Functional Collaboration:

What You’ll Bring

• Experience:6+ years of professional QA experience, with at least 2 years in a startup or high-growth AI/Data-heavy environment.
• Technical Mastery:Deep proficiency inSQLfor validating complex RDS and OpenSearch data structures.Advanced automation skills inPythonorJava.Expertise inPostman/Swaggerfor API testing within a microservices (AWS Lambda/FastAPI) architecture.
• Deep proficiency inSQLfor validating complex RDS and OpenSearch data structures.
• Advanced automation skills inPythonorJava.
• Expertise inPostman/Swaggerfor API testing within a microservices (AWS Lambda/FastAPI) architecture.
• AI Knowledge:Familiarity with theSTLCas it applies to non-deterministic systems, including RAG (Retrieval-Augmented Generation) and Agentic workflows.
• Tools:Mastery ofLinear(issue tracking),PostHog(user event tracking), andAWSenvironments (Cloudwatch, S3).
• Attributes:You are an "Everyday Innovator" who thinks outside the box to solve the unique challenges of testing AI for the built world.

Experience:

Technical Mastery:

SQL

Python Java

Postman/Swagger

AI Knowledge: STLC

Tools: Linear PostHog AWS

Attributes:

Experience We’d Be Particularly Excited About

• AEC Industry Knowledge:Familiarity with the SMPS proposal process or RFP workflows.
• AI Tooling:Experience withLangchainor evaluating LLM performance metrics.
• DevOps Leanings:Interest in compliance or improving Developer Experience (DevEx).

AEC Industry Knowledge:

AI Tooling: Langchain

DevOps Leanings:

What to expect

We conduct a rigorous interview process based on integrity, talent, and drive. We trust our teammates from day one and move quickly to evaluate whether you are fit for the role. The entire interview process typically takes two weeks. Here's what to expect:

• A 30 minute Zoom meeting to talk about Joist AI, your background, and answer any questions about the role.
• A 30 minute Zoom meeting with another one of our team members to hear more about your experience and how you'd approach working in the role.
• A take home project to assess your functional expertise for the role you're applying for.
• A 60 minute Zoom call to review your project and answer any outstanding questions.

A 30 minute Zoom meeting to talk about Joist AI, your background, and answer any questions about the role.

A 30 minute Zoom meeting with another one of our team members to hear more about your experience and how you'd approach working in the role.

A take home project to assess your functional expertise for the role you're applying for.

A 60 minute Zoom call to review your project and answer any outstanding questions.