QA Tester, AI Development
About This Role
The QA Tester is the quality gatekeeper for Insight's AI Development team, responsible for defining and executing test strategies across AI-powered applications. This role goes beyond traditional functional testing — you will validate the reasoning, accuracy, and safety of AI agents and LLM-powered features, while also maintaining robust automation frameworks for the full application stack. You will lead QA efforts across multiple projects and mentor team members on testing best practices for AI applications.
Key Responsibilities
Define and implement test strategies for AI-powered applications, including functional, integration, regression, and AI-specific testing (prompt evaluation, hallucination detection, guardrail validation).
Build and maintain automated test frameworks using Python (Pytest) and UI automation tools (Playwright, Cypress, or Selenium) for React front-ends and API back-ends.
Validate AI model outputs for accuracy, safety, and adherence to expected behaviors across agentic workflows and conversational interfaces.
Integrate automated test suites into CI/CD pipelines to ensure rapid feedback on code changes.
Lead defect management: identify, document, reproduce, and track bugs; work closely with developers to verify fixes.
Collaborate with AI Developers, Full Stack Developers, and DevOps Engineers to ensure quality is embedded throughout the development lifecycle.
Establish QA metrics and reporting to communicate quality status and risk to stakeholders.
Qualifications
Experience: Minimum 3 years of experience as a QA Engineer or SDET, with experience testing complex web applications and API integrations.
Automation: Proficient in Python for writing automated test scripts (Pytest preferred); experience with UI automation tools (Playwright, Cypress, or Selenium).
API Testing: Strong understanding of REST APIs; experience validating API responses and JSON schemas.
AI/ML Testing: Familiarity with testing LLM-powered applications, including non-deterministic output validation. Experience with evaluation frameworks (RAGAS, LangSmith, or similar) is a plus.
Cloud: Working knowledge of Microsoft Azure; ability to navigate cloud logs and debug deployed services.
CI/CD: Experience integrating test suites into CI/CD pipelines (Azure DevOps, GitHub Actions).
Communication: Strong written and verbal English communication skills; ability to articulate risk and quality metrics to stakeholders.
Nice to Have: Experience with performance testing, security testing, or accessibility testing.
The position described above provides a summary of some the job duties required and what it would be like to work at Insight. For a comprehensive list of physical demands and work environment for this position, click here .
About Insight
Today, every business is a technology business. Insight Enterprises, Inc. empowers organizations of all sizes with Insight Intelligent Technology Solutions™ and services to maximize the business value of IT. As a Fortune 500-ranked global provider of digital innovation, cloud/data center transformation, connected workforce, and supply chain optimization solutions and services, we help clients successfully manage their IT today while transforming for tomorrow. From IT strategy and design to implementation and management, our 11,000 teammates help clients innovate and optimize their operations to run smarter. Discover more at insight.com
Ashley | The Wellsville Groups, Johnstown, PA location is looking for a Retail Operations Coordinator. As a Retail Operations Coordinator you are responsible for ensuring operational excellence within the store while supporting retail management and the sales team....
...Certified Nursing Assistant (CNA) Day Shift Full Time About Us At Norris Post-Acute and Rehabilitation Center in Andersonville, TN, we are: New grad friendly Family oriented Teamwork driven Unique! Join our growing team at Norris Post-Acute...
...We're seeking Bilingual Virtual Administrative Assistants (Remote) People residing in Venezuela, Colombia, Argentina, Ecuador, Peru, Nicaragua... ...clients provide even more PTO days). ~ Monthly stipend for medical insurance (after an induction period). ~ Birthday/...
...trucks and railcars - Move coils and bundles of steel to production lines and designated storage areas - Utilize wireless crane remotes and proper lifting techniques - Accurately identify materials and complete inventory-related paperwork - Read and interpret...
...Maintain cleanliness and organization of the kitchen, including floors, walls, and workstations. Assist culinary staff with basic food preparation tasks as needed. Dispose of trash and recyclables in accordance with health and safety regulations. Ensure all...