General
AI Agent Evaluator
- Posted
- Location
- Global
- Openings
- 50
Role Title: AI Agent Evaluator
Role Type: Contractor
Location: U.S based
Opportunity Snapshot
Pay: $25-45/hour
Time per task: ~1.5 hours per run (evidence and scoring included)
Project kickoff: Monday, October 12
Onboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th
You’ll need: An active personal popular AI account (Free, Go, or Plus; not Pro), an active personal and a smartphone that can screen record
Your tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results
About the role
AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.
You’ll test two AI agents side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.
What you’ll do
- Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
- Approve or stop the agent at key steps
- Screen record every attempt
- Score each run on completion, quality, control issues, satisfaction, and time
- Share clear written feedback on the agent experience
Who we’re looking for
- US-based, with a Bachelor’s degree
- Someone who already uses AI in daily life
- 2-3 years of experience, preferably in engineering, business management, or office/operations
- Detail-oriented, great at following instructions and documenting each step
- Clear written communication
- Generalists welcome, domain experts preferred
- No coding or AI background needed
Why you’ll like it
- Get paid to use the AI tools you already use
- Work entirely from your phone, on your schedule
- Have a direct say in how next-generation AI agents get built and improved
How the selection process works
- Apply
- Complete the screening questions
- Interview: 15-30 minutes max
- Profile review: if everything looks good, we present you to the client
- Onboarding: our Talent Ops team will contact you to get you started
Skills
- Attention to detail
- AI Agents
- Following instructions
- ChatGPT
- Muse
- Screen Recording
This role at a glance
At $25–$45/hr, the AI Agent Evaluator role comes to about $2,170–$3,900 a month for 20 hours a week, or $4,330–$7,800 a month full time — roughly $52,000–$94,000 over a full year. These are estimates before tax; contract roles are paid for the hours you work. It was posted 7 days ago. General roles listed on RemoteJobsOne have stayed open a median of 21 days before closing, counted over the 9 that have closed so far. 6 people applied to this role through RemoteJobsOne in the last 30 days.
20 hours a week
$2,170–$3,900
a month, before tax
Full time
$4,330–$7,800
a month at 40 hours a week
A full year
$52,000–$94,000
full time, 52 weeks
Open for
7 days
General roles here close after about 21 days, counted over 9 that have closed. Apply while it is open.
How this pay compares
This role pays $25–$45/hr. Across the 16 hourly general roles open on RemoteJobsOne right now, the median runs from $28 to $43 an hour, and published rates range from $14 to $130. Its top rate of $45 sits above the category's median ceiling.
This role
$25–$45/hr
General median
$28–$43/hr · full spread $14 to $130
What these skills pay
Hourly pay across every open role on RemoteJobsOne that asks for the same skills as this one.
- Attention to detail$25–$60/hr median
53 open roles · up to $400/hr
Questions about this role
How much does the AI Agent Evaluator role pay?
It pays $25–$45/hr, which is about $4,330–$7,800 a month full time before tax. The rate is set by the employer and shown exactly as published.
Compare pay across remote rolesIs this AI Agent Evaluator job fully remote?
Yes. It is listed as a fully remote role, with the posting naming Global. You work from home, on your own equipment.
Is it full-time, part-time or contract work?
It is a contract role: you are paid for the hours you work, and you choose how many to take on within what the employer needs.
How remote contract work worksWhat skills does the AI Agent Evaluator role ask for?
The posting asks for Attention to detail, AI Agents, Following instructions, ChatGPT, Muse and Screen Recording. Naming them clearly in your application helps.
How do I apply for AI Agent Evaluator?
Press Apply on this page and enter your name and email. We email you your personal application link straight away, and you can continue at once to the employer's hiring platform to finish. Most roles there ask for a short AI interview afterwards — voice or video, in your browser. Then watch your email: the next steps arrive there.
How to prepare for the AI interview
Before you apply
Something wrong with this listing? Report it