Categories: FAANG

Evaluating Long Range Dependency Handling in Code Generation LLMs

As language models support larger and larger context sizes, evaluating their ability to make
effective use of that context becomes increasingly important. We analyze the ability of
several code generation models to handle long range dependencies using a suite of multi-step
key retrieval tasks in context windows up to 8k tokens in length. The tasks progressively
increase in difficulty and allow more nuanced evaluation of model capabilities than tests like
the popular needle-in-the-haystack test. We find that performance degrades significantly for
many models (up to 2x) when a function…

Vertex AI at I/O: Bringing new Gemini and Gemma models to Google Cloud customers

Vertex AI is Google Cloud’s fully-managed, unified development platform for leveraging models at scale, with a selection of over 150 first-party, open, and third-party foundation models; for customizing models with enterprise-ready tuning, grounding, monitoring, and deployment capabilities; and for building AI agents. Customers such as ADT, IHG Hotels & Resorts,…

May 15, 2024

In "FAANG"

Stable Diffusion Models are Secretly Good at Visual In-Context Learning

Large language models (LLM) in natural language processing (NLP) have demonstrated great potential for in-context learning (ICL) — the ability to leverage a few sets of example prompts to adapt to various tasks without having to explicitly update the model weights. ICL has recently been explored for computer vision tasks…

October 8, 2025

In "FAANG"

Can Large Language Models Understand Context?

Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent. However, though the evaluation of LLMs encompasses various domains within the realm of Natural Language Processing, limited attention has been paid to probing their linguistic…

April 22, 2026

In "FAANG"

AI Generated Robotic Content