Job Description

About the company

---------------------


Joist AI is a technology company revolutionising the way professionals in the architecture, engineering, and construction (AEC) industry manage marketing and revenue operations. Our AI-powered software streamlines workflows, making it easier for teams to collaborate, innovate, and succeed.

About the Role

------------------


Joist AI is looking for a strategic and highly technical

Senior QA Engineer

to own the quality roadmap for our AI-powered RevOps platform. Unlike traditional software, our product relies on complex document processing pipelines, multi-model LLM architectures (OpenAI, Anthropic, Gemini), and intricate knowledge graphs.


You won't just be finding bugs; you will be architecting the testing frameworks that ensure our AI's "hit rate" and compliance checks are flawless for our enterprise customers. Reporting to the Engineering Manager, you will lead the QA effort to ensure Joist AI remains the best-in-class platform for the built world.

What You'll Do

------------------

QA Strategy & Leadership:

Define and implement the end-to-end testing strategy for Joist AI, focusing on the

Proposal Processing Pipeline

and

Knowledge Graph

building.

AI/LLM Validation:

Develop novel ways to test and monitor LLM-generated narratives and chat responses for accuracy, hallucination reduction, and compliance.

Pipeline Oversight:

Own the quality of our extraction services (Adobe API), object/image detection, and captioning services.

Automation Architecture:

Lead the development of automated test suites using

Playwright

(browser) and

API automation

(FastAPI) integrated into our CI/CD pipelines.

Defect Intelligence:

Use

Linear

to track defects through the lifecycle and collaborate with leads (Bala, Sharat, Chandan) to prioritize fixes based on customer impact.

Cross-Functional Collaboration:

Partner with Data Science teams to validate image processing (face/label detection) and with the GTM team to understand real-world proposal workflows.

What You'll Bring

---------------------

Experience:

6+ years of professional QA experience, with at least 2 years in a startup or high-growth AI/Data-heavy environment.

Technical Mastery:

+ Deep proficiency in

SQL

for validating complex RDS and OpenSearch data structures.
+ Advanced automation skills in

Python

or

Java

.
+ Expertise in

Postman/Swagger

for API testing within a microservices (AWS Lambda/FastAPI) architecture.

AI Knowledge:

Familiarity with the

STLC

as it applies to non-deterministic systems, including RAG (Retrieval-Augmented Generation) and Agentic workflows.

Tools:

Mastery of

Linear

(issue tracking),

PostHog

(user event tracking), and

AWS

environments (Cloudwatch, S3).

Attributes:

You are an "Everyday Innovator" who thinks outside the box to solve the unique challenges of testing AI for the built world.

Experience We'd Be Particularly Excited About

-------------------------------------------------

AEC Industry Knowledge:

Familiarity with the SMPS proposal process or RFP workflows.

AI Tooling:

Experience with

Langchain

or evaluating LLM performance metrics.

DevOps Leanings:

Interest in compliance or improving Developer Experience (DevEx).

What to expect

------------------


We conduct a rigorous interview process based on integrity, talent, and drive. We trust our teammates from day one and move quickly to evaluate whether you are fit for the role. The entire interview process typically takes two weeks. Here's what to expect:

A 30 minute Zoom meeting to talk about Joist AI, your background, and answer any questions about the role. A 30 minute Zoom meeting with another one of our team members to hear more about your experience and how you'd approach working in the role. A take home project to assess your functional expertise for the role you're applying for. * A 60 minute Zoom call to review your project and answer any outstanding questions.