Tag: Evaluation

  1. What the Astra weight-reading demo actually showed (honnibal.dev)
    AI · · November 4, 2026
  2. Turning a database benchmark into reinforcement learning rewards (huggingface.co)
    AI · · October 31, 2026
  3. LLMs are shaky judges of whether a research idea is new (huggingface.co)
    AI · · October 30, 2026
  4. Nathan Lambert on what a leading AI benchmark can and cannot tell you (interconnects.ai)
    AI · · October 29, 2026
  5. Forced to pick a candidate, Grok leans Republican and GPT leans Democrat (techcrunch.com)
    AI · · October 26, 2026
  6. A 10-year-old CNN beat frontier LLMs at reading skin cancer (lynchlab.net)
    AI · · October 26, 2026
  7. Ai2's FairStress finds models bend their fairness under pressure (allenai.org)
    AI · · October 23, 2026
  8. Open models are about six months behind on cyber, by Epoch's count (epoch.ai)
    AI · · October 18, 2026
  9. Most working hours fall outside what AI benchmarks measure (mercor.com)
    AI · · October 17, 2026
  10. Anthropic cuts live internet off from all its internal evals (anthropic.com)
    AI · · October 11, 2026
  11. Epoch tested AI on its own jobs, and it couldn't do them (epoch.ai)
    AI · · October 9, 2026
  12. ThinkingBox grades AI agents on the database, not the transcript (huggingface.co)
    AI · · October 9, 2026
  13. Frontier models still can't run AI research on their own (epoch.ai)
    AI · · October 8, 2026
  14. A new leaderboard scores open text-to-speech models on objective metrics (huggingface.co)
    AI · · October 4, 2026
  15. Allen AI's AstaBrief writes cited research reports from a small open model (allenai.org)
    AI · · October 3, 2026
  16. The phrases that give away AI writing, measured across models (techcrunch.com)
    AI · · October 2, 2026
  17. A professor's retrospective on flagging 207 students for AI-written code (turkeyland.net)
    AI · · October 1, 2026
  18. Your agent passed the test. IBM asks whether it will pass five times (huggingface.co)
    AI · · September 21, 2026
  19. Physics benchmarks were underrating the models (arxiv.org)
    AI · · September 17, 2026
  20. Why self-tuning research agents don't overfit (amazon.science)
    AI · · September 15, 2026
  21. Ai2's BenchMIRT digs into what an LLM benchmark actually measures (huggingface.co)
    AI · · September 12, 2026
  22. Astra beats humans on the one ARC-AGI metric that was supposed to hold (arcprize.org)
    AI · · September 4, 2026
  23. Reasoning models more than doubled the pace of AI progress, Epoch finds (epoch.ai)
    AI · · September 4, 2026
  24. OpenAI ships GPT-6 Astra, its answer to Claude Fable (openai.com)
    AI · · September 4, 2026
  25. K2 Horizon ships six fully open models from 0.9B to 375B (ifm.ai)
    AI · · September 4, 2026
  26. The same model, nine agent harnesses, and a 17x cost gap (frontierharness.org)
    AI · · September 3, 2026
  27. How 215,000 machine-made pages ended up inside Perplexity's answers (trellner.com)
    AI · · September 3, 2026
  28. DeepMind runs a frontier model evaluation where neither side can peek (deepmind.google)
    AI · · September 1, 2026
  29. DeepMind ran a frontier model through a double-blind benchmark (deepmind.google)
    AI · · August 29, 2026
  30. Nine questions Epoch wants AI benchmarks to answer (epoch.ai)
    AI · · August 27, 2026
  31. A stricter test finds most LLM-written GPU kernels are broken (arxiv.org)
    AI · · August 15, 2026
  32. Long-form writing is where AI progress stalled, says Nathan Lambert (interconnects.ai)
    AI · · August 14, 2026
  33. Probing frontier models to work out their real training cutoffs (blog.sshh.io)
    AI · · August 11, 2026
  34. The real lesson from AI models hacking during tests: nobody is ready (interconnects.ai)
    AI · · August 10, 2026
  35. Now three labs have had models attack real systems during tests (simonwillison.net)
    AI · · August 7, 2026
  36. Voice models can speak well but listen poorly (huggingface.co)
    AI · · July 26, 2026
  37. AI agents now finish 16 percent of real freelance jobs, up from 2.5 (safe.ai)
    AI · · July 15, 2026
  38. Why price per million tokens tells you almost nothing (janilowski.pl)
    AI · · July 7, 2026
  39. GeneBench-Pro shows AI still stumbles on real biology analysis (openai.com)
    AI · · July 5, 2026
  40. Using DSPy to find a hidden bug in an agent's prompt (simonwillison.net)
    AI · · July 3, 2026
  41. A new benchmark shows agents rebuilding software that would take humans weeks (epoch.ai)
    AI · · June 30, 2026
  42. An open model beat Claude Code at finding access-control bugs (semgrep.dev)
    Security · · June 29, 2026
  43. An API change that helps big models can break small ones (huggingface.co)
    AI · · June 21, 2026
  44. GLM-5.2 takes the lead among open-weights models (artificialanalysis.ai)
    AI · · June 18, 2026
  45. OpenAI predicts model behavior by replaying real conversations (openai.com)
    AI · · June 18, 2026
  46. AI2's olmo-eval brings statistical rigor to the model training loop (huggingface.co)
    AI · · June 13, 2026
  47. Did Claude make rsync buggier? The numbers say no (alexispurslane.github.io)
    AI · · June 5, 2026
  48. Microsoft ASSERT turns plain-English behavior specs into AI tests (techcrunch.com)
    Engineering · · June 4, 2026
  49. Princeton researchers pick apart Google's claim that AI agents built an OS for $916 (normaltech.ai)
    AI · · May 30, 2026
  50. A new benchmark sends frontier agents to fix Kubernetes, and none pass (huggingface.co)
    AI · · May 27, 2026
  51. An open leaderboard for whole agent systems, not just models (huggingface.co)
    AI · · May 19, 2026
  52. Ai2 launches a shared benchmark for AI climate models (allenai.org)
    AI · · May 18, 2026
  53. The Open ASR Leaderboard adds private tests to stop gaming (huggingface.co)
    AI · · May 18, 2026
  54. Testing AI in the open world, not just on benchmarks (normaltech.ai)
    AI · · May 17, 2026
  55. How you pick benchmarks decides whether open models are far behind (interconnects.ai)
    AI · · May 16, 2026
  56. One benchmark number hides which jobs a model is actually good at (interconnects.ai)
    AI · · April 20, 2026
  57. DeepMind proposes a cognitive framework for measuring AGI progress (blog.google)
    AI · · March 17, 2026
  58. Crash Testing GPT-4: The First Dangerous-Capability Eval (asteriskmag.com)
    AI · · June 1, 2023

← all tags