7 Chunking Strategies That Decide Whether Your RAG Works
Day 100 in production isn’t really about chunking strategies anymore.
Taming Outlier Tokens in Diffusion Transformers
We study outlier tokens in Diffusion Transformers (DiTs) for image generation. Prior work has shown that Vision Transformers (ViTs) can produce a small number of high-norm tokens that attract disproportionate attention while carrying limited local information, but their role in generative models remains underexplored. We show that this phenomenon appears in both the encoder and …
AI Sovereignty is Your Alpha: How to Avoid Transferring Your Alpha to a Hosted Model Provider
Use of third party AI model services poses significant risk to your alpha. Without sovereign control over how your data is processed by those services (either the AI Labs or the Hyperscalers, collectively referred to as “Hosted Model Providers”), Hosted Model Providers may extract your alpha (your unique institutional knowledge and tradecraft embodied in data …
Modeling Device Capabilities for Analytics
by Aarti Laddha, Richard Diaz-Cool, Rishika Idnani, Venkatesh Selveraj Netflix supports a vast and evolving set of features and content types, ranging from 4K streaming and immersive audio to live streaming and cloud gaming, across a diverse ecosystem of devices. However, not all devices are created equal. Hardware limitations such as available RAM, CPU cores, …
How LendingTree built a multi-agent mortgage assistant on Amazon Bedrock
Buying a home is one of the biggest financial decisions most people face, and LendingTree built a multi-agent mortgage assistant on Amazon Bedrock to make the process more straightforward. The assistant educates borrowers, understands their situation, and provides tailored options in a natural conversation. Borrowers must weigh purchase or refinance, conventional or government-backed, 15-year or …
Read more “How LendingTree built a multi-agent mortgage assistant on Amazon Bedrock”
OpenAI Didn’t Notice Its AI Agents Using a Message Board to Plan Their Hacking Spree
At the Black Hat security conference, the AI giant revealed new details about how its agents went rogue, hacked several other companies—and did it all right under the company’s nose.
AI models nearly erase female characters when they write kids stories about animals
Last year, Melanie Walsh, a University of Washington assistant professor in the Information School, wrote an article examining how 300 popular children’s books gendered their animal characters. Of the 13 most common animals, most were male—unless they happened to be cats, ducks or birds, which trended slightly more female. But a frog, a wolf? More …
Read more “AI models nearly erase female characters when they write kids stories about animals”
Measuring Performance of Transformer Inference
This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most common inference metrics are: • …
Static vs. Dynamic vs. Continuous Batching in LLM Inference
In this article, you will learn how static, dynamic, and continuous batching work in LLM inference, and why the differences between them matter at production…
Introducing Web Search on Amazon Bedrock for foundation model grounding
When a foundation model needs to answer a question about last week’s earnings call, yesterday’s regulatory change, or this morning’s weather forecast, it needs knowledge it was never trained on. Grounding the model in current web knowledge closes that gap – whether it’s powering chatbots, coding assistants, CLI tools, or enterprise applications, grounding helps answer …
Read more “Introducing Web Search on Amazon Bedrock for foundation model grounding”