LLM Business Analyst/Evaluation (3 YOE)

Job description:

Key Responsibilities

• Perform side-by-side (SxS) evaluations of LLM outputs
• Assess quality, accuracy, relevance, and factual correctness
• Conduct fact-checking and identify inconsistencies
• Provide clear, structured justifications for evaluations
• Follow evaluation guidelines and quality standards
• Use browser-based tools and basic command-line instructions
• Document findings and share insights when required

Requirements

• Strong analytical and critical thinking skills
• High attention to detail with clear decision justification
• Comfortable with evaluation frameworks and structured analysis
• Good written English communication
• Ability to use online tools, browser extensions, and basic CLI commands

Nice to Have

• Experience in evaluation, annotation, QA, or audit work
• Familiarity with data quality or model/output assessments
• Background in business analysis, operations, or process improvement

Pay: ₹500,000.00 - ₹800,000.00 per year

Benefits:

• Work from home

Application Question(s):

• Do you have experience with evaluation, annotation, QA, or audit work ?
• Are you fine with this Short term contract(2-3 Months)?

Experience:

• Business analysis: 3 years (Required)
• LLM: 2 years (Required)

Work Location: Remote