All news

Palihapitiya: "Model Benchmarks Are Worthless"

Who Said What · X · · AI-summarised by FinFlash

Chamath Palihapitiya questioned AI model benchmarks in a post on X, citing scores of 24% and 59% on the same 113 coding tasks when the surrounding evaluation harness changes. He said he increasingly ignores recent model leaderboards. His criticism concerns how AI capabilities are measured, rather than a specific company's results, and could influence perceptions of model comparisons, enterprise purchasing criteria, and AI developer claims. Could affect: AI model developers (Mixed), Enterprise AI software (Mixed).

Read original source

Understand the context

Published information may change. Summaries can contain errors; check the original source. Educational information only, not investment advice.