Vals Raises $40M to Build Private, Real-World AI Benchmarks
The Andreessen Horowitz-backed startup is expanding confidential model tests across industry tasks and high-risk domains while pursuing federal work.
Share
Rankings & Benchmarks
9 posts
Samsung’s TRUEBench Tests AI on Real Office Work
TRUEBench evaluates language models across multilingual, context-rich workplace tasks instead of relying on academic question-and-answer tests.
Share
MLPerf Training v1.0 Measured Full-System AI Progress
MLCommons’ latest training results compared complete hardware and software systems and added speech recognition and 3D medical imaging workloads.
Share
Five AI Companies Showed the Breadth of the 2021 Market
A 2021 watchlist spanning emotion recognition, MLOps, conversational systems and industrial AI showed how quickly the market was diversifying.
Share
A100 Set Another MLPerf Inference Record
Nvidia’s A100 led another round of MLPerf inference testing across vision and translation, highlighting both accelerator gains and benchmark limits.
Share
The First MLPerf Inference Results Put Nvidia in the Lead
MLPerf’s first inference results compared hundreds of measurements, with Nvidia leading available processors across server and offline scenarios.
Share
Google Assistant Led a Virtual-Assistant IQ Test
Loup Ventures tested 800 questions and found Google Assistant led overall accuracy while Siri and Alexa recorded substantial annual gains.
Share
A New NLP Benchmark Sought to Make Language AI Easier to Compare
NLP consortium's work on language benchmarks showed how the topic was shaping the AI landscape.
Share
Microsoft Led the AI Patent Race Going Into 2019
Microsoft's work on AI patents showed how the topic was shaping the AI landscape.
Share