Software Test Engineer AI (LLM / RAG Focus)

As a Quality Assurance Engineer for AI systems, your role will involve testing and validating LLM-based and Generative AI systems. You will be responsible for evaluating AI outputs for relevance, accuracy, consistency, and reliability. Using Retrieval Augmented Generation (RAG) applications, you will test and analyze AI outputs with Python. It will be essential for you to identify, document, and report any issues in AI model behavior. Collaborating with engineering teams, you will contribute to understanding AI workflows and suggesting improvements to ensure the overall quality of AI-driven features and systems.

Key Responsibilities:

• Test and validate LLM-based and Generative AI systems
• Evaluate AI outputs for relevance, accuracy, consistency, and reliability
• Test applications using Retrieval Augmented Generation (RAG)
• Work with Python to validate and analyze AI outputs
• Identify, document, and report issues in AI model behavior
• Collaborate with engineering teams to understand AI workflows and improvements
• Ensure overall quality of AI-driven features and systems

Qualifications Required:

• 2 to 3 years of experience in software testing or quality engineering
• Hands-on experience with AI, NLP, or Generative AI systems
• Strong Python programming skills
• Experience testing data-driven or AI-based applications
• Understanding of AI output evaluation and validation concepts
• Good analytical and problem-solving skills

Good to Have:

• Exposure to LLM or RAG-based applications
• Experience with AI or ML frameworks
• Knowledge of cloud platforms
• Experience working in fast-paced engineering environments

This role is based in Pune, Maharashtra, and offers a hybrid remote work option. As a full-time employee, you will also benefit from health insurance coverage. As a Quality Assurance Engineer for AI systems, your role will involve testing and validating LLM-based and Generative AI systems. You will be responsible for evaluating AI outputs for relevance, accuracy, consistency, and reliability. Using Retrieval Augmented Generation (RAG) applications, you will test and analyze AI outputs with Python. It will be essential for you to identify, document, and report any issues in AI model behavior. Collaborating with engineering teams, you will contribute to understanding AI workflows and suggesting improvements to ensure the overall quality of AI-driven features and systems.

Key Responsibilities:

• Test and validate LLM-based and Generative AI systems
• Evaluate AI outputs for relevance, accuracy, consistency, and reliability
• Test applications using Retrieval Augmented Generation (RAG)
• Work with Python to validate and analyze AI outputs
• Identify, document, and report issues in AI model behavior
• Collaborate with engineering teams to understand AI workflows and improvements
• Ensure overall quality of AI-driven features and systems

Qualifications Required:

• 2 to 3 years of experience in software testing or quality engineering
• Hands-on experience with AI, NLP, or Generative AI systems
• Strong Python programming skills
• Experience testing data-driven or AI-based applications
• Understanding of AI output evaluation and validation concepts
• Good analytical and problem-solving skills

Good to Have:

• Exposure to LLM or RAG-based applications
• Experience with AI or ML frameworks
• Knowledge of cloud platforms
• Experience working in fast-paced engineering environments

This role is based in Pune, Maharashtra, and offers a hybrid remote work option. As a full-time employee, you will also benefit from health insurance coverage.