AI Session Annotator
Recruitment Room →AI Session Annotator at Recruitment Room is a contract role based in Remote · American Samoa-4 (remote). The listing states pay of $49,920–$62,398/mo and asks for 0–25 yrs of experience. Listed skills: Annotation. It was published on 27 August 2026 and was open at last check.
| Role | AI Session Annotator |
|---|---|
| Company | Recruitment Room |
| Location | Remote · American Samoa-4 (remote) |
| Employment type | Contract |
| Pay (as listed) | $49,920–$62,398/mo |
| Experience asked | 0–25 yrs |
| Skills listed | Annotation |
| Published | 27 August 2026 |
| Status | Open at last check |
AI Session Annotator
Remote | Independent Contractor | $49,920–$62,400 annualized ($24–$30/hour)
About the Role
Help improve the next generation of voice-and-camera AI assistants by evaluating how accurately they understand and respond to real-world visual information.
As an AI Session Annotator, you’ll review recorded interactions between users and an AI assistant. You’ll compare the assistant’s responses with what was actually visible in the video, evaluate its performance against a detailed rubric, and provide precise written feedback.
Your comments and evaluations will directly help identify issues such as visual hallucinations, inaccurate interpretations, poor timing, and inadequate safety warnings.
This is a judgment-focused role where accuracy, consistency, and quality of feedback matter more than review speed.
What You’ll Do
- Review recorded sessions where users interact with an AI assistant through voice and camera.
- Compare the assistant’s responses with the visual information actually available in the video.
- Identify fabricated, hallucinated, or inaccurate visual details.
- Evaluate each session for: Visual interpretation
- Accuracy
- Relevance
- Response timing
- Appropriate use of spatial and directional language
- Assess whether the assistant provides appropriate and timely warnings when responses could affect a user's health, physical safety, or financial security.
- Write detailed comments supporting each score with specific evidence from the session.
- Document conversations turn by turn using the provided intake form.
- Apply the evaluation rubric consistently without introducing personal scoring criteria.
What You Bring
- Native or native-level English proficiency, both spoken and written.
- Excellent written English and the ability to communicate observations clearly and precisely.
- Strong attention to detail and analytical judgment.
- Ability to follow a detailed evaluation rubric consistently.
- Ability to distinguish between what is actually visible in a video and what an AI assistant claims to see.
- An Android smartphone with a working camera and microphone — mandatory for this project.
Preferred Qualifications
Experience in any of the following is a plus:
- Data annotation
- Linguistic quality assurance
- Content moderation
- AI model evaluation
- Linguistics
- Translation or interpreting
- Conversational AI evaluation
An ear for regional English usage and the ability to recognize unnatural or non-native phrasing is also valuable.
Content & Project Details
- Start Date: Immediate
- High-Volume Period: August 26–31
- Compensation: $24–$30/hour
- Annualized Equivalent: $49,920–$62,400
- Work Arrangement: Fully remote
- Engagement: Independent contractor
- Schedule: Flexible, based on project requirements
Annualized compensation is based on 2,080 hours per year for comparison purposes only. Actual earnings depend on the number of hours and projects completed.
Content Considerations
Some sessions involve scenarios where an incorrect AI response could have meaningful real-world consequences. Examples may include identifying medication, determining whether food has spoiled, navigating a street-crossing situation, or interpreting bank card information.
There is no graphic or violent content, but some sessions involve safety-sensitive situations. You’ll be expected to evaluate whether the AI assistant recognized potential risks and provided an appropriate warning when necessary.
Application Process
- Submit your resume or relevant professional background.
- If selected, you may be asked to complete a short calibration exercise using sample sessions.
- The calibration focuses on scoring consistency and the quality of your written comments—not speed.
Why Join?
- Help improve the accuracy and reliability of advanced multimodal AI systems.
- Apply your language and analytical skills to real AI evaluation work.
- Work remotely with a flexible schedule.
- Contribute directly to identifying and correcting AI visual reasoning failures.
- Develop hands-on experience in AI model evaluation and data quality.
Contract & Payment Terms
- You will be engaged as an independent contractor.
- Work is fully remote and can be completed on your own schedule.
- Projects may be extended, shortened, or concluded early depending on project needs and performance.
- Your work will not require access to confidential or proprietary information belonging to any employer, client, or institution.
- Payments are made weekly via Stripe or Wise based on services rendered.
- H-1B and STEM OPT candidates cannot be supported at this time.
Equal Opportunity
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Create a free OnJob profile to apply and see your AI match score before you apply. · Posted 27 Aug 2026.
AI Session Annotator at Recruitment Room — questions answered
What does the AI Session Annotator role at Recruitment Room pay?
The AI Session Annotator at Recruitment Room listing states pay of $49,920–$62,398/mo for the Remote · American Samoa-4 (remote) position. That figure is taken directly from the employer's own posting as published, not estimated or averaged from other roles, and the same listing asks for 0–25 yrs of experience.
Is the AI Session Annotator role at Recruitment Room remote?
Yes — Recruitment Room advertises this AI Session Annotator role as remote, tied to Remote · American Samoa-4, and it is listed as contract work. Remote terms come from the employer's own posting, so confirm the expected working hours, timezone and any on-site days with Recruitment Room before you apply.
What skills does the AI Session Annotator role at Recruitment Room require?
The AI Session Annotator at Recruitment Room listing names Annotation. Those are the skills the employer put on the posting itself, so they are the ones worth matching in your profile and covering first in an interview.
How much experience do you need for the AI Session Annotator role at Recruitment Room?
Recruitment Room asks for 0–25 yrs of experience on this AI Session Annotator posting, alongside Annotation. Employers commonly consider candidates slightly under a stated band when the listed skills line up.
Is the AI Session Annotator role at Recruitment Room still open?
The AI Session Annotator posting at Recruitment Room was open at OnJob's last check of the employer's careers page, having been published on 27 August 2026. OnJob re-checks source listings on each build and marks a role closed once it disappears, but listings can close without notice, so the employer's own page is the final word.
How do you apply for the AI Session Annotator role at Recruitment Room?
Apply to the AI Session Annotator at Recruitment Room role through OnJob with a free profile: OnJob scores your fit against the listing, shows the skills lowering that score, and submits an ATS-ready profile. Creating a profile is free and needs no card.
Explore more on OnJob
Hiring for a role like this?
Post a job on OnJob and reach AI-matched candidates.
