Categories: FAANG

The Super Weight in Large Language Models

Recent works have shown a surprising result: a small fraction of Large Language Model (LLM) parameter outliers are disproportionately important to the quality of the model. LLMs contain billions of parameters, so these small fractions, such as 0.01%, translate to hundreds of thousands of parameters. In this work, we present an even more surprising finding: Pruning as few as a single parameter can destroy an LLM’s ability to generate text — increasing perplexity by 3 orders of magnitude and reducing zero-shot accuracy to guessing. We propose a data-free method for identifying such parameters…
AI Generated Robotic Content

Recent Posts

When she says she only likes open source dudes

submitted by /u/Jack_Fryy [link] [comments]

6 hours ago

SceneScout: Towards AI Agent-driven Access to Street View Imagery for Blind Users

People who are blind or have low vision (BLV) may hesitate to travel independently in…

6 hours ago

Why We Serve: Palantirians Reflect on Duty, Honor & Innovation

In honor of Independence Day, Palantir Veterans and Intelligence Community (IC) alums offer reflections on…

6 hours ago

Transforming network operations with AI: How Swisscom built a network assistant using Amazon Bedrock

In the telecommunications industry, managing complex network infrastructures requires processing vast amounts of data from…

6 hours ago

How to build a simple multi-agentic system using Google’s ADK

Agents are top of mind for enterprises, but often we find customers building one “super”…

6 hours ago

Sakana AI’s TreeQuest: Deploy multi-model teams that outperform individual LLMs by 30%

Sakana AI's new inference-time scaling technique uses Monte-Carlo Tree Search to orchestrate multiple LLMs to…

7 hours ago