Categories: AI/ML News

DeepMind’s Michelangelo benchmark reveals limitations of long-context LLMs

LLMs can retrieve disparate facts from their context windows, but when it comes to reasoning over their context, they struggle badly.Read More

QwenLong-L1 solves long-context reasoning challenge that stumps current LLMs

May 31, 2025

In "AI/ML News"

Evaluating Long Range Dependency Handling in Code Generation LLMs

As language models support larger and larger context sizes, evaluating their ability to make effective use of that context becomes increasingly important. We analyze the ability of several code generation models to handle long range dependencies using a suite of multi-step key retrieval tasks in context windows up to 8k…

June 28, 2025

In "FAANG"

Do LLMs Estimate Uncertainty Well in Instruction-Following?

Large language models (LLMs) could be valuable personal AI agents across various domains, provided they can precisely follow user instructions. However, recent studies have shown significant limitations in LLMs’ instruction-following capabilities, raising concerns about their reliability in high-stakes applications. Accurately estimating LLMs’ uncertainty in adhering to instructions is critical to…

April 9, 2025

In "FAANG"

AI Generated Robotic Content