H2O.ai sets the world record in GAIA Agentic AI benchmark with h2oGPTe
H2O.ai beats Microsoft and Google researchers by more than 15 points on GAIA — widely hailed as the ultimate test for real-world intelligence.
MOUNTAIN VIEW, Calif.--(BUSINESS WIRE)--H2O.ai, the leader in open-source Generative AI and the most accurate Predictive AI platforms, today announced that h2oGPTe Agent has secured the #1 position on the GAIA (General AI Assistants) benchmark leaderboard with an unprecedented score of 65% — outperforming Google’s Langfun Agent (49%), Microsoft Research (38%), and Hugging Face (33%) leading entries. This remarkable achievement underscores H2O.ai's dominance in the emerging domain of general-purpose AI agents, setting a new gold standard for the industry.
Why GAIA Matters
The GAIA benchmark measures how useful AI systems are in solving real-world tasks that require a lot of time, thought and effort for skilled humans. It consists of hundreds of challenges that require laborious research, data analysis, document handling and reasoning. Degree-holding human respondents achieve a score of 92% and require several human-days to solve all 300 test set problems.
h2oGPTe Agent outpaced competitors by delivering consistent robustness, accuracy and efficiency, highlighting its readiness for enterprise use cases that depend heavily on skilled human assistants.
Enterprise h2oGPTe Agent: A Landmark Achievement
This achievement solidifies H2O.ai’s leadership in the global race to build intelligent, adaptable AI assistants capable of transforming businesses.
Sri Ambati, Founder and CEO of H2O.ai, shared his enthusiasm:
“Today we are announcing that AI is only 30% away from matching human-level general intelligence on the GAIA benchmark. Open-ended questions in GAIA are a better measure of intelligence than MMLU, which relies on multiple choice. To share how exciting this is: the entire Gen AI ecosystem was barely able to pass a tenth in accuracy on one of the toughest AGI benchmarks merely a year ago."
“Makers at H2O.ai built h2oGPTe Agentic AI wielding the best models in the world for reasoning, multi-modal image, video, language understanding, code generation and execution to ace the GAIA benchmark with a stunning 15% accuracy leap over the previous record set by researchers from Google Deepmind using the same Claude-3.5-Sonnet. h2oGPTe Agent also beat Microsoft Research’s agent Magentic-1 that used OpenAI’s o1 model by 27%."
“Agentic AI is eating SaaS and with h2oGPTe Agentic AI now being generally available, all our enterprise customers can solve a wide range of sophisticated business and research problems.”
H2O.ai's success on GAIA underscores its philosophy of simplicity and adaptability:
- Advanced reasoning and planning for solving complex, real-world tasks
- Multimodal comprehension across text, images, and audio for seamless context understanding
- Integration of enterprise tools like Python execution and DriverlessAI for predictive analytics and decision-making
H2O.ai’s win reaffirms its leadership in AI innovation, particularly in agentic systems poised to reshape business workflows.
Enterprise h2oGPTe 1.6 includes the Agent feature and is available on all public clouds, virtual private clouds and for on-premise deployments: https://h2o.ai/platform/enterprise-h2ogpte/
Read technical blog https://h2o.ai/blog/2024/h2o-ai-tops-gaia-leaderboard/
H2O.ai created the industry's trusted Sovereign AI Platform because organizations should own their AI: their agents, models, data, and results, run anywhere from cloud to air-gapped.
Super Agent orchestrates predictive, generative, and agentic AI in one system, with observability and governance, empowering regulated and public sector organizations to get value from AI. Moving from pilot to production takes expertise, from top-ranked data scientists, including Kaggle Grandmasters, to forward-deployed engineers embedded with customer teams.
Built on a decade of open source, H2O.ai is trusted by 20,000+ organizations, half the Fortune 500. AT&T, Commonwealth Bank of Australia, Wells Fargo, Bank of America, Workday, Progressive Insurance, and NIH rely on H2O.ai to cut cost and risk and own their AI future.
Through its AI for Good program, H2O.ai applies technology and investment to non-profits and academia solving hard problems in health, education, and the environment.
Learn more at h2o.ai.