AI Agent Tester – Real-World Tasks - Remote

AI Agent Tester – Real-World Tasks - Remote

Role Overview

• Employment Type: Contract

• Location: Remote — United States

• Compensation: $25–$45 per hour

• Estimated Time per Task: Approximately 1.5 hours, including evidence collection and scoring

About the Role

We are seeking detail-oriented individuals to test AI agents on everyday tasks and evaluate their performance from a real user's perspective. You will compare two AI agents using your own accounts, assess their ability to complete practical activities, and document the quality, reliability, and usability of their results.

Tasks may involve shopping, travel bookings, bill payments, email follow-ups, returns, family logistics, and completing online forms. No coding or formal AI background is required, but familiarity with AI assistants and everyday digital workflows is important.

Key Responsibilities

• Execute realistic scenarios involving errands, purchases, bookings, communications, and other everyday activities using two AI agents.

• Approve, pause, or stop agents at key decision points when required.

• Record each testing attempt using smartphone screen-recording capabilities.

• Evaluate task completion, output quality, user control, satisfaction, and time taken.

• Document errors, unexpected behavior, incomplete actions, and other usability issues.

• Provide clear written feedback explaining what worked, what failed, and how the experience could improve.

• Follow testing instructions and maintain accurate evidence and scoring for each run.

Required Qualifications

• Be based in the United States and hold a bachelor's degree.

• Have 2–3 years of professional experience, preferably in engineering, business management, office administration, or operations.

• Use AI assistants regularly in everyday activities.

• Have strong attention to detail and the ability to follow detailed instructions.

• Demonstrate clear written communication and accurate documentation skills.

• Have an active personal AI assistant account with Plus-tier access.

• Have active access to Muse and Dots.

• Own a smartphone capable of recording the screen.

• Have a payment card available for test purchases, with eligible purchases reimbursed according to project terms.

Preferred Qualifications

• Experience in a technical, business, or operational domain relevant to the scenarios being tested.

• Strong familiarity with AI agents and their use in practical workflows.

• Experience evaluating digital products, documenting user experiences, or identifying process failures.

Work Arrangement and Compensation

• Hourly rate: $25–$45.

• Schedule: Flexible, remote work performed using your phone.

• Task duration: Approximately 1.5 hours per run, including evidence collection and scoring.

• Test purchases: Eligible purchases are reimbursed according to project terms.

Selection Process

1. Submit an application and complete the screening questions.

2. Attend an interview lasting approximately 15–30 minutes.

3. Complete a profile review.

4. Complete onboarding before beginning assigned tasks.