Tag: Evaluation

  1. Ai2's BenchMIRT digs into what an LLM benchmark actually measures (huggingface.co)
    AI · · September 12, 2026
  2. Astra beats humans on the one ARC-AGI metric that was supposed to hold (arcprize.org)
    AI · · September 4, 2026
  3. Reasoning models more than doubled the pace of AI progress, Epoch finds (epoch.ai)
    AI · · September 4, 2026
  4. OpenAI ships GPT-6 Astra, its answer to Claude Fable (openai.com)
    AI · · September 4, 2026
  5. K2 Horizon ships six fully open models from 0.9B to 375B (ifm.ai)
    AI · · September 4, 2026
  6. The same model, nine agent harnesses, and a 17x cost gap (frontierharness.org)
    AI · · September 3, 2026
  7. How 215,000 machine-made pages ended up inside Perplexity's answers (trellner.com)
    AI · · September 3, 2026
  8. DeepMind runs a frontier model evaluation where neither side can peek (deepmind.google)
    AI · · September 1, 2026
  9. DeepMind ran a frontier model through a double-blind benchmark (deepmind.google)
    AI · · August 29, 2026
  10. Nine questions Epoch wants AI benchmarks to answer (epoch.ai)
    AI · · August 27, 2026
  11. A stricter test finds most LLM-written GPU kernels are broken (arxiv.org)
    AI · · August 15, 2026
  12. Long-form writing is where AI progress stalled, says Nathan Lambert (interconnects.ai)
    AI · · August 14, 2026
  13. Probing frontier models to work out their real training cutoffs (blog.sshh.io)
    AI · · August 11, 2026
  14. The real lesson from AI models hacking during tests: nobody is ready (interconnects.ai)
    AI · · August 10, 2026
  15. Now three labs have had models attack real systems during tests (simonwillison.net)
    AI · · August 7, 2026
  16. Voice models can speak well but listen poorly (huggingface.co)
    AI · · July 26, 2026
  17. AI agents now finish 16 percent of real freelance jobs, up from 2.5 (safe.ai)
    AI · · July 15, 2026
  18. Why price per million tokens tells you almost nothing (janilowski.pl)
    AI · · July 7, 2026
  19. GeneBench-Pro shows AI still stumbles on real biology analysis (openai.com)
    AI · · July 5, 2026
  20. Using DSPy to find a hidden bug in an agent's prompt (simonwillison.net)
    AI · · July 3, 2026
  21. A new benchmark shows agents rebuilding software that would take humans weeks (epoch.ai)
    AI · · June 30, 2026
  22. An open model beat Claude Code at finding access-control bugs (semgrep.dev)
    Security · · June 29, 2026
  23. An API change that helps big models can break small ones (huggingface.co)
    AI · · June 21, 2026
  24. GLM-5.2 takes the lead among open-weights models (artificialanalysis.ai)
    AI · · June 18, 2026
  25. OpenAI predicts model behavior by replaying real conversations (openai.com)
    AI · · June 18, 2026
  26. AI2's olmo-eval brings statistical rigor to the model training loop (huggingface.co)
    AI · · June 13, 2026
  27. Did Claude make rsync buggier? The numbers say no (alexispurslane.github.io)
    AI · · June 5, 2026
  28. Microsoft ASSERT turns plain-English behavior specs into AI tests (techcrunch.com)
    Engineering · · June 4, 2026
  29. Princeton researchers pick apart Google's claim that AI agents built an OS for $916 (normaltech.ai)
    AI · · May 30, 2026
  30. A new benchmark sends frontier agents to fix Kubernetes, and none pass (huggingface.co)
    AI · · May 27, 2026
  31. An open leaderboard for whole agent systems, not just models (huggingface.co)
    AI · · May 19, 2026
  32. Ai2 launches a shared benchmark for AI climate models (allenai.org)
    AI · · May 18, 2026
  33. The Open ASR Leaderboard adds private tests to stop gaming (huggingface.co)
    AI · · May 18, 2026
  34. Testing AI in the open world, not just on benchmarks (normaltech.ai)
    AI · · May 17, 2026
  35. How you pick benchmarks decides whether open models are far behind (interconnects.ai)
    AI · · May 16, 2026
  36. One benchmark number hides which jobs a model is actually good at (interconnects.ai)
    AI · · April 20, 2026
  37. DeepMind proposes a cognitive framework for measuring AGI progress (blog.google)
    AI · · March 17, 2026
  38. Crash Testing GPT-4: The First Dangerous-Capability Eval (asteriskmag.com)
    AI · · June 1, 2023

← all tags