Oracle adds Nvidia L40S GPU to OCI

Oracle Cloud Infrastructure (OCI) has added Nvidia L40S GPU bare-metal instances to its cloud infrastructure, launching them alongside plans for a new virtual machine accelerated by a single Nvidia H100 Tensor Core GPU. OCI will offer the L40S GPUs in its BM.GPU.L40S.4 bare-metal compute offering which has four L40S GPUs, each with 48GB of GDDR6 memory. It also includes local NVMe drives with 7.38TB capacity, fourth generation Intel Xeon CPUs with 112 cores, and 1TB of system memory.

According to Nvidia, one L40S GPU (FP8) can generate up to 1.4x more tokens per second than a single Nvidia A100 Tensor Core GPU (FP16) for Llama 3 8B with Nvidia TensorRT-LLM at an input and output sequence length of 128. It has fourth-generation tensor cores and can support the FP8 data format. OCI will offer the L40S GPUs in its BM.GPU.L40S.4 bare-metal compute offering which has four L40S GPUs, each with 48GB of GDDR6 memory.

“We chose OCI AI infrastructure with bare-metal instances and Nvidia L40S GPUs for 30 percent more efficient video encoding,” said Sharon Carmel, CEO of Beamr Cloud. “Videos processed with Beamr Cloud on OCI will have up to 50 percent reduced storage and network bandwidth consumption, speeding up file transfers by 2x and increasing productivity for end users. Beamr will provide OCI customers with video AI workflows, preparing them for the future of video.”