Expert Talent Pool, Agents and Reinforcement Learning

About the Role

Join an expert network supporting AI human-data projects focused on agents and reinforcement learning. You will design training environments and evaluation systems, and assess agent behavior as projects become available.

What You'll Do

• Design reinforcement learning environments, including Python and Docker setups, state spaces, tool integrations, and scoring mechanisms.

• Build robust verifiers and reward functions that resist agent gaming.

• Translate real-world business workflows in areas such as ERP, CRM, finance, and operations into measurable scenarios.

• Integrate tools and REST APIs, and build test harnesses using pytest or similar tools.

• Lead agent demonstration and computer-use programs, setting standards for trace collection quality.

• Grade and repair agent trajectories, and mentor contributors on evaluation practices.

What We're Looking For

• At least 5 years of experience in software engineering, QA automation, machine learning, reinforcement learning, or business systems and operations paired with hands-on agent or RL data work.

• Strong Python skills and experience building tools or services used by others.

• Experience designing environments, verifiers, reward functions, or test harnesses that evaluate software or agent behavior, using tools such as pytest, Playwright, Selenium, or Gymnasium-style environments.

• Hands-on experience with agent or RL data, such as demonstrations, trajectory grading, trace collection, or computer-use demos.

• Experience with REST APIs and Docker.

• Experience leading or reviewing data, evaluation, or demonstration work is valuable, as is expertise modeling workflows in business systems such as Salesforce or ERP platforms.

• A degree in computer science, engineering, or a related field is relevant.

Compensation & Benefits

Compensation is $80 to $200 per hour, depending on project, field, and level. Pay is based on hours worked.

Location

Remote. Projects typically run from one week to several months.