Skip to main content

Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

AWS AI/MLOfficial

Why Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips matters

Teams running large-scale embedding workloads can consider Neuron-based inference and custom kernel tuning as a way to cut per-document indexing cost, though the gains reflect Exa's specific workload.

Summary

Exa, the neural search company, describes how it improved embedding inference throughput by about 50% on AWS Neuron hardware (Inferentia2 and Trainium2) using custom NKI kernels. The optimization yields a 47% gain in performance and cost per million documents, which matters because Exa embeds tens of billions of web documents for its in-house vector database.

Read the full article at AWS AI/ML

CoFabrix summarises and comments on this story. The original reporting belongs to AWS AI/ML.

More in Infrastructure

Browse the full AI Pulse feed

Seeing AI Disruption in Your Industry?

Find out where your organization stands -- and what to do about it.