AI Automation Engineer

At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.”

– be empathetic and help create a place where everyone belongs.

Be inspired by our higher callingofimproving lives.

act with integrity, be real and empower others.

It’swhywe’reso driven to connect passion with purpose.Alight helps clients gaina benefitsadvantage while building a healthy and financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, absencemanagementand navigation.

With a comprehensive total rewards package, continuing education and training, and tremendous potential with a growing global organization, Alight is the perfect place to put your passion to work.

Join our team if you Champion People, want to Grow with Purpose through acting with integrity and if you embody the meaning of Be Alight.

Learn more at .

As the AI Automation Engineer in the AICoE, you will be instrumental in standing up theinitialautomation framework for AI Quality.You’llwork closely with the AI Quality Lead and developers to build, run, and scale automated test suites for AI models and agents.

RESPONSIBILITIES

• Develop andmaintainautomated test frameworks for LLM, RAG, agentic, and classic ML systems

Develop andmaintainautomated test frameworks for LLM, RAG, agentic, and classic ML systems

• Implement golden/adversarial test cases (functional, safety, bias, prompt injection, refusal correctness, tool use)

Implement golden/adversarial test cases (functional, safety, bias, prompt injection, refusal correctness, tool use)

• Integrate evaluation tools (promptfoo, Ragas) into CI/CD pipelines and support batch quality assessments

Integrate evaluation tools (promptfoo, Ragas) into CI/CD pipelines and support batch quality assessments

• Author and validateOpenAPI/JSON Schema contracts for model inputs/outputs; enforce schema checks in pipelines

Author and validateOpenAPI/JSON Schema contracts for model inputs/outputs; enforce schema checks in pipelines

• Instrument observability metrics (latency, success rate, cost, drift) usingOpenTelemetryand export to dashboards

Instrument observability metrics (latency, success rate, cost, drift) usingOpenTelemetryand export to dashboards

• Support data QA, lineage tracking, and drift monitoring (Evidently,OpenLineage)

Support data QA, lineage tracking, and drift monitoring (Evidently,OpenLineage)

• Collaborate with AI developers, SDETs, and SRE to troubleshoot,optimize, and document automation processes

Collaborate with AI developers, SDETs, and SRE to troubleshoot,optimize, and document automation processes

• Contribute to red-team playbooks and securitytestbooksmapped to OWASP/ATLAS risks

Contribute to red-team playbooks and securitytestbooksmapped to OWASP/ATLAS risks

REQUIREMENTS

• Experience in test automation, software engineering, or quality assurance (AI/ML experience is a plus, but notrequired)

Experience in test automation, software engineering, or quality assurance (AI/ML experience is a plus, but notrequired)

• Proficiencyin Python, JavaScript/Node.js, or similar programming languages

Proficiencyin Python, JavaScript/Node.js, or similar programming languages

• Familiarity with CI/CD, version control, and cloud-based deployment environments

Familiarity with CI/CD, version control, and cloud-based deployment environments

• Experience with test automation tools (promptfoo, Ragas, Postman)

Experience with test automation tools (promptfoo, Ragas, Postman)

• Understanding of AI model evaluation metrics (faithfulness, answer relevancy, hallucination, refusal correctness)

Understanding of AI model evaluation metrics (faithfulness, answer relevancy, hallucination, refusal correctness)

• Basic knowledge of fairness, safety, and security testing for AI systems

Basic knowledge of fairness, safety, and security testing for AI systems

• Experience with AI infrastructure, orchestration, data integrity, deployments, and model registry practices

Experience with AI infrastructure, orchestration, data integrity, deployments, and model registry practices

• Interest in Responsible AI, bias mitigation, and regulatory compliance

Interest in Responsible AI, bias mitigation, and regulatory compliance

• Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent practical experience

Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent practical experience

So that you canbeyour best at work and home, we consider flexible working arrangements wherever possible.Alight ha s been a leader in the flexible workspace and “Top 100 Company for Remote Jobs”5 yearsin a row.

We offer programs andplansfor a healthy mind, body,walletand life becauseit’simportantour benefits care for the whole person. Options include a variety of health coverage options, wellbeing and support programs, retirement, vacation and sick leave, maternity, paternity & adoption leave, continuing education and training as well asseveralvoluntary benefit options.

By applying for a position with Alight, you understand that, should you bemadean offer, it will be contingent on your undergoing and successfully completing a background check consistent with Alight’s employment policies. Background checks may include some or all the following based on the nature of the positio n: You will b e notified during the hiring process which checks are required by the position.

Wecelebrate differences and believe in fostering an environment where everyone feels valued, respected, and supported. We know that diverse teams are stronger, more innovative, and more successful.

At Alight, we welcome and embrace all individuals, regardless of their background, and are dedicated to creating a culture that enables every employee to thrive. Join us in building a brighter, more inclusive future.

As part of this commitment, Alight will ensure that persons with disabilities areprovidedreasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact .

Alight is an Equal Employment Opportunity employer and does not discriminate against anyone based on sex, race, color, religion, creed, national origin, ancestry, age, physical or mental disability, medical condition, pregnancy, marital or domestic partner status, citizenship, military or veteran status, sexual orientation, gender, gender identity or expression, genetic information, or any other legally protected characteristics or conduct covered by federal, state, or local law.   In addition, we take affirmative action to employ disabled persons, disabledveteransand other covered veterans.

Alight provides reasonable accommodations to the known limitations of otherwise qualified employees and applicants for employment with disabilities andsincerely heldreligious beliefs,practicesand observances, unless doing so would result in undue hardship. Applicants for employment may requesta reasonableaccommodation/modificationby contacting their recruiter.

Applicants for employment in the country in which they are applying (Employing Country) must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the Employing Country and with Alight.

Note, this job description does not restrict management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.

We offer you a competitive total rewards package, continuing education & training, and tremendous potential with a growing worldwide organization.

Salary Pay Range

Minimum :

Maximum :

Pay Transparency Statement: Alight considers a variety of factors in determining whether to extend an offer of employment and in setting the appropriate compensation level, including, but not limited to, a candidate’s experience, education, certification/credentials, market data, internal equity, and geography. Alight makes these decisions on an individualized, non-discriminatory basis. Bonus and/or incentive eligibility are determined by role and level. Alight also offers a comprehensive benefits package; for specific details on our benefits package, please visit:

DISCLAIMER:

Nothing in this job description restricts management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.