Optical CXL for disaggregated compute architectures

Ron Swartzentruber, Director of Engineering, Lightelligence
Optical CXL will fundamentally change the way datacenters are architected for AI and confront the memory bandwidth needs of LLM processing. Models with hundreds of billions of parameters need to be coherently connected to large arrays of compute in a disaggregated architecture.
With CXL-capable processors, accelerators, switches, and memories, massive systems can be built to connect compute arrays to large amounts of memory. Due to the sheer size of these systems, resources must span across multiple racks in the datacenter. Memory bandwidth and latency are critical factors impacting the time to train large AI models. CXL over optics solves the bandwidth, latency, and distance challenges demanded by LLM applications.
This presentation will illustrate the latency and performance improvements that can be achieved with an optical CXL fabric and the benefits of memory pooling. The distance advantages and decode throughput results will be examined from a LLM inference application.