SambaNova unveils new AI inference cloud

Chip and artificial intelligence (AI) model developer SambaNova Systems has launched an AI cloud offering using the SN40L AI chip. The SambaNova cloud can be accessed by developers via an API and create their own generative AI applications using the open-source Llama 3.1 405B and Llama 3.1 70B models.

“SambaNova Cloud is the fastest API service for developers. We deliver world record speed and in full 16-bit precision — all enabled by the world’s fastest AI chip,” said Rodrigo Liang, CEO of SambaNova Systems. “SambaNova Cloud is bringing the most accurate open source models to the vast developer community at speeds they have never experienced before.”

“Competitors are not offering the 405B model to developers today because of their inefficient chips. Providers running on Nvidia GPUs are reducing the precision of this model, hurting its accuracy, and running it at almost unusably slow speeds,” continued Liang. “Only SambaNova is running 405B — the best open-source model created - at full precision and at well over 100 tokens per second.”