Categories: AI/ML Research

Measuring Performance of Transformer Inference

This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most common inference metrics are: • Latency: How long a request takes from start to finish.

How to evaluate the quality of the synthetic data – measuring from the perspective of fidelity, utility, and privacy

December 17, 2022

In "FAANG"

EHR-Safe: Generating High-Fidelity and Privacy-Preserving Synthetic Electronic Health Records

December 22, 2022

In "FAANG"

ProText: A Benchmark Dataset for Measuring (Mis)gendering in Long-Form Texts

We introduce ProText, a dataset for measuring gendering and misgendering in stylistically diverse long-form English texts. ProText spans three dimensions: Theme nouns (names, occupations, titles, kinship terms), Theme category (stereotypically male, stereotypically female, gender-neutral/non-gendered), and Pronoun category (masculine, feminine, gender-neutral, none). The dataset is designed to probe (mis)gendering in text…

April 1, 2026

In "FAANG"

AI Generated Robotic Content