Cut Your Losses in Large-Vocabulary Language Models

As language models grow ever larger, so do their vocabularies. This has shifted the memory footprint of LLMs during training disproportionately to one single layer: the cross-entropy in the loss computation. Cross-entropy builds up a logit matrix with entries for each pair of input tokens and vocabulary items and, for small models, consumes an order …

12ARe5mfk0ia1ItwRGH52INwA

Requirements for AI in Production in Insurance Underwriting

Introduction Large language models present both massive opportunities and significant complexities for the insurance industry. Insurers can use AI to increase operational efficiency, improve the accuracy of underwriting decisions, enhance customer experience, and more effectively coordinate with partners. Yet in a heavily-regulated industry like insurance, ensuring objectivity and the appropriate level of human oversight in …

arch diagram 02072025 1

Governing the ML lifecycle at scale, Part 4: Scaling MLOps with security and governance controls

Data science teams often face challenges when transitioning models from the development environment to production. These include difficulties integrating data science team’s models into the IT team’s production environment, the need to retrofit data science code to meet enterprise security and governance standards, gaining access to production grade data, and maintaining repeatability and reproducibility in …

News you can use: What we announced in AI this month

2025 is off to a racing start. From announcing strides in the new Gemini 2.0 model family to retailers accelerating with Cloud AI, we spent January investing in our partner ecosystem, open-source, and ways to make AI more useful. We’ve heard from people everywhere, from developers to CMOs, about the pressure to adapt the latest …