AI Evaluation Engineer
virtual vocationsDenver, CO
AI Evaluation Engineer
L5
virtual vocationsDenver, COtoday
Industries
Other Scientific and Technical Consulting ServicesCustom Computer Programming ServicesComputer Systems Design ServicesBuilding high-quality AI evaluation systems, the full-time AI Evaluation Infrastructure Engineer will develop infrastructure and tooling to ensure reliable and efficient evaluations, enabling product teams to make data-driven decisions quickly and confidently in a remote work environment.
Key responsibilities:
Build an execution engine to score candidate versions against task sets rapidly
Create tooling for task validation and human reviewer calibration
Develop systems to compare offline evaluation scores with online outcomes and improve evaluation sets
Required qualifications:
Experience in building production platforms or infrastructure, including data pipelines
Strong statistical literacy, including knowledge of confidence intervals and variance
Experience evaluating LLM or ML systems or deep systems engineering experience with a focus on AI evaluation
Product instinct for internal tools that drive team usage and engagement
Ability to collaborate effectively across product, data, and engineering teams
Apply now (opens in a new tab)
Level
SeniorL5
Location
Denver, CO
Occupation
Validation Engineers
Industry
Other Scientific and Technical Consulting Services
Posted
today
To get sharper similar jobs, create your profile using the link below.