Generative AI Tester

Role : Gen AI Tester Location : Bengaluru, Karnataka, India Job Summary : Talent Worx is seeking a highly motivated and detail-oriented Gen AI Tester to join our growing team. In this role, you will be crucial in ensuring the quality, functionality, and usability of our cutting-edge artificial intelligence models, including natural language processing (NLP), machine learning (ML) systems, and conversational AI applications. You will collaborate closely with data scientists, AI engineers, and developers to assess the performance of AI systems, identify potential issues, and drive product improvements. Key Responsibilities : - Conduct comprehensive testing of AI systems, encompassing generative models, chatbot platforms, and machine learning algorithms, to evaluate their effectiveness, accuracy, and usability. - Develop and execute test plans and test cases tailored to the unique challenges of AI testing. - Detect, report, and meticulously document system defects, bugs, and anomalous AI behavior. - Test for AI response consistency, context understanding, and the ability to generate relevant and coherent outputs. - Assess the performance of AI models against predefined benchmarks and real-world scenarios. - Ensure the model's accuracy, robustness, and generalization capabilities. - Perform continuous regression testing on AI system updates and improvements to prevent regression issues and performance degradation after model retraining or new version releases. - Collaborate with AI researchers, data scientists, and development teams to understand AI model behavior, provide constructive feedback, and ensure quality control throughout the development lifecycle. - Maintain detailed documentation of test plans, test cases, test results, and identified issues. - Provide clear, actionable insights and reports to the development team. - Evaluate the user interfaces (UI) of AI applications, including chatbot dialogues and virtual assistant platforms, to ensure intuitive and seamless user experiences. - Assist in training and fine-tuning AI models based on real-world user feedback and test results. - Provide validation to ensure adherence to ethical AI standards. - Stay updated with the latest advancements in AI technology, machine learning, and testing methodologies to continuously improve testing processes. Skills and Qualifications : - Education : degree in Computer Science, Engineering, Data Science, or a related field. - Strong knowledge of AI technologies, machine learning frameworks, and NLP. - Experience in testing software applications, including automated and manual testing practices. - Familiarity with AI tools and platforms (TensorFlow, PyTorch, OpenAI, etc.). - Proficiency in programming languages like Python, Java, or similar. - Ability to troubleshoot, debug, and identify issues in AI models and algorithms. - Strong analytical skills and the ability to work with complex systems and data. - Excellent attention to detail and critical thinking skills. - Effective communication skills to collaborate with cross-functional teams. - Prior experience with Generative AI models (GPT, DALL-E, etc.) is a plus. Preferred Qualifications : - Experience with AI model validation techniques and testing frameworks. - Familiarity with cloud-based AI platforms (AWS, Google Cloud, Azure). - Knowledge of ethical AI principles and bias detection in AI models. - Prior experience in user acceptance testing (UAT) for AI systems. Technical Skills : - Python, Java, or similar. - TensorFlow, PyTorch, Scikit-learn. - NLTK, spaCy, Hugging Face Transformers. - Postman, Selenium, JUnit, PyTest. - API Testing : REST API testing. - Generative AI Models : GPT, DALL-E, etc. - Prompt Engineering : Understanding of prompt engineering best practices. (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 5/1/2025