As AI models increase in sophistication, there’s increasingly large model data needed to serve them. Loading the models and weights along with necessary frameworks to serve them for inference can add seconds or even minutes of scaling delay, impacting both costs and the end-user’s experience. For example, inference servers such as Triton, Text Generation Inference …
Read more “Data loading best practices for AI/ML inference on GKE”
To provide high-quality medical care to its population — around 30% of whom are 65 or older — Japan is pursuing sovereign AI initiatives supporting nearly every aspect of healthcare. AI tools trained on country-specific data and local compute infrastructure are supercharging the abilities of Japan’s clinicians and researchers so they can care for patients, …
A smart combination of quantization and sparsity allows BitNet LLMs to become even faster and more compute/memory efficientRead More
Alan Filion, believed to have operated under the handle “Torswats,” admitted to making more than 375 fake threats against schools, places of worship, and government buildings around the United States.
We introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience. What does it mean for large language models (LLMs) to be trained on massive text corpora, collectively produced by millions and billions of distinctive human authors? …
Read more “Virtual Personas for Language Models via an Anthology of Backstories”
This paper was accepted at the Efficient Natural Language and Speech Processing (ENLSP) Workshop at NeurIPS 2024. The pre-training phase of language models often begins with randomly initialized parameters. With the current trends in scaling models, training their large number of parameters can be extremely slow and costly. In contrast, small language models are less …
How Palantir Enables a Secure, Rapid Software Development Environment (Software Supply Chain Security, #1) Editor’s Note: This is the first post in a series that shares insights from our journey to enhance our software supply chain security story at Palantir. This post provides background on why and how we initiated our Software Supply Chain Security …
Read more “How Palantir Enables a Secure, Rapid Software Development Environment”
By: Rajiv Shringi, Oleksii Tkachuk, Kartik Sathyanarayanan Introduction In our previous blog post, we introduced Netflix’s TimeSeries Abstraction, a distributed service designed to store and query large volumes of temporal event data with low millisecond latencies. Today, we’re excited to present the Distributed Counter Abstraction. This counting service, built on top of the TimeSeries Abstraction, …
This post is co-written with Etzik Bega from Agmatix. Agmatix is an Agtech company pioneering data-driven solutions for the agriculture industry that harnesses advanced AI technologies, including generative AI, to expedite R&D processes, enhance crop yields, and advance sustainable agriculture. Focused on addressing the challenge of agricultural data standardization, Agmatix has developed proprietary patented technology …
Read more “Generative AI for agriculture: How Agmatix is improving agriculture with Amazon Bedrock”
Have you heard of the monkey and the pedestal? Astro Teller, the head of Google’s X “moonshot factory,” likes to use this metaphor to describe tackling the biggest challenge first, despite being tempted by the endorphin boost of completing more familiar tasks. It’s a challenge startups know well. When you’re re-inventing the industry standard, it’s …
Read more “Efficiency engine: How three startups deliver results faster with Vertex AI”