JobCubby JobCubby
Sign up
Intercontinental Exchange Holdings, Inc.

QA Engineer

Intercontinental Exchange Holdings, Inc. Atlanta, Kansas, United States

Financial Services · 10,001+ employees

10 h ago
qa Mid (2-5 yrs) Full-time United States
Create a free account to apply — email only, no card. You can also save this posting or score it against your profile with AI.

Applying here? Try the free cover letter tool — paste this posting and your résumé, no account needed.

About the role

You will implement and maintain automated test frameworks for LLM-based applications while utilizing AI to verify the correctness of generated information. Additionally, you will participate in designing and refining prompts to improve the performance and accuracy of these applications.

What they look for

QA Automation Python LLM Testing Agentic AI Test Frameworks Prompt Engineering Problem Solving Attention To Detail Communication Teamwork Financial Instruments Fixed Income Markets

Requirements

The role requires a bachelor's degree in a STEM field and at least 3 years of experience in QA and QA automation. Proficiency in Python and experience with Agentic AI tools are essential for success in this position.

Full description

Overview

Job Purpose

We are seeking a QA Engineer with experience in testing LLM based software. As a member of the QA team, you will have the opportunity to participate in various projects where LLMs are used to solve complex engineering problems. You will participate in creating and executing automated tests for such applications. You will also participate in the creation and improvement of prompts driving these applications.

Responsibilities

  • Implement automated test frameworks and take ownership of existing frameworks for LLM based applications.
  • Implement automated tests that use AI for reasoning on the correctness of LLM generated information.
  • Improve testing methodologies for LLM based applications.
  • Participate and designing and improving prompts used in LLM based applications.

Knowledge and Experience

  • Bachelor’s degree in STEM fields.
  • 3+ years in QA and QA automation.
  • Experience using Agentic AI tools to build test automation tools.
  • Proficiency in Python.
  • Ability to learn quickly and adapt to new technologies and processes, with a focus on keeping up to date with the latest developments in LLM related technologies.
  • Ability to independently learn new complex business domains.
  • Strong problem-solving skills and attention to detail.
  • Excellent communication and teamwork skills.
  • Ability to communicate fluently and professionally in complex technical and business contexts.

Preferred Knowledge and Experience

  • Knowledge of fixed income markets and financial instruments.

#LI-JM1

Similar roles