Google: Google launches Nvidia H100 GPUs and TPUs, adding generative AI cloud services

Google unveiled several cloud-based tools and services centered on artificial intelligence. Cloud TPU v5e, the company's newest Tensor Processing Unit, is now available for preview, according to the company. Google claims that, compared to TPU v4, which was released in 2021, the chip has up to two times quicker training results per dollar and up to 2.5 times faster inference performance per dollar for huge language models as well as generative AI models.

Within a single slice, the new TPU will be available in eight distinct virtual machine configurations, ranging from one TPU chip to over 250 TPU chips. Multislice, a method to sell models to tens of thousands of TPU processors, is being rolled out by the company for those who require more computing power.

Google also announced that A3 virtual machines (VMs) with eight Nvidia H100 GPUs, dual 4th Generation Intel Xeon Scalable processors, and two terabytes of memory will be generally available next month. The instances were initially announced in May and have the capacity to expand to 26,000 Nvidia H100 Hopper GPUs; however, it is unknown how many H100s Google will have, given the ongoing GPU shortage.

According to the cloud provider, generative AI startup Anthropic was an early adopter of the TPU v5e and A3 VMs. Google invested $300 million in the venture, but it is also a vocal user of Amazon Web Services.