- AI4Bharat and Josh Talks AI announced the launch of Voice of India, the country’s first independent multimodal AI evaluation platform dedicated to assessing AI systems under real-world Indian conditions.
- The Voice of India platform aims to become the country’s trusted reference point for AI evaluation, enabling governments, enterprises, researchers, and AI developers to independently assess AI performance across Indian languages, accents, and dialects.
- As part of the launch, a suite of benchmarks was introduced, including a finance-focused benchmark for assessing AI performance across key banking workflows.
- The platform plans to expand evaluations across voice AI, large language models (LLMs), OCR, safety and cultural assessment, and Indian legal reasoning over time.
- Voice of India is designed as an open, neutral, and scientifically rigorous evaluation platform that enables AI developers to benchmark their systems while helping buyers make better-informed deployment decisions.
- The platform introduces a three-stage evaluation framework comprising constituent model evaluation, end-to-end deployment evaluation under real operating conditions, and periodic post-deployment audits.
- Professor Mitesh Khapra of AI4Bharat stated that rigorous evaluation is as important as model development itself, emphasizing the need for evaluations that reflect India’s linguistic diversity.
- The platform targets governments, enterprises, researchers, and AI developers to independently assess how AI models perform in India rather than relying only on vendor-reported benchmarks.
AI4Bharat and Josh Talks AI have launched Voice of India, the first independent multimodal AI evaluation platform in India, aimed at assessing AI systems under real-world conditions. This initiative is crucial for ensuring that AI technologies are evaluated based on India's unique linguistic and cultural diversity.125
The platform provides a three-stage evaluation framework that includes constituent model evaluation, end-to-end deployment evaluation, and periodic post-deployment audits. This comprehensive approach allows organizations to understand AI performance not just in controlled settings but also in real-world scenarios.6
Professor Mitesh Khapra from AI4Bharat emphasized the importance of rigorous evaluation, stating, “Evaluation determines what gets built. With AI systems becoming more deeply integrated into society, rigorous evaluation becomes as important as model development itself.”
The platform is designed to serve a wide range of stakeholders, including governments, enterprises, and researchers, enabling them to make informed decisions based on scientifically robust evaluations rather than vendor-reported benchmarks. Supriya Paul, co-founder of Josh Talks AI, noted, “India has a once-in-a-generation opportunity to contribute something unique to global AI — not necessarily the biggest models, but the evaluation infrastructure that makes AI trustworthy for the world.”8
Voice of India will also introduce a suite of benchmarks, including a finance-focused benchmark for assessing AI performance in banking workflows, with plans to expand evaluations across various AI applications, including voice AI and large language models.34
“Professor Mitesh Khapra said evaluation determines what gets built, while Supriya Paul called it India's once-in-a-generation opportunity to build trusted AI evaluation infrastructure. The platform's three-stage framework includes model evaluation, end-to-end deployment checks, and periodic post-deployment audits.”
