Evading Datacenter Tax Using MangoBoost’s Customizable FPGA Data Processing Units [Invited]

Speaker: Eriko Nurvitadhi, Mangoboost
Modern datacenter servers rely on an increasing number of devices to improve efficiency in data-centric tasks, such as data storage (SSDs), movement (NICs), and processing (GPU, NPU, etc). Moreover, they offer advanced infrastructure for users to access resources easily/flexibly (e.g., via virtual machines, containers) and deploy/scale applications (e.g., via web/microservices). Managing such servers with a rich set of devices, while running sophisticated infrastructure software, impose a growing burden on CPUs and introduce substantial system overheads, often known as “Datacenter Tax.” MangoBoost’s novel patent-pending technologies offer comprehensive accelerator building blocks to offload a myriad of datacenter taxes, with a customizable framework to produce specialized data processing units (DPUs) to dramatically improve server systems performance, scalability, and cost. These custom DPUs are optimized for desired system targets and readily deployable on production-qualified off-the-shelf FPGAs.
This talk will discuss datacenter trends, the problem with datacenter tax, and MangoBoost’s DPU solutions to address this problem. To highlight the effectiveness of MangoBoost’s solutions, the talk will present case studies in networked storage and AI systems. For example, applying MangoBoost DPU to Samsung’s Petabyte SSD storage system enables 400GbE NVMe/TCP remote storage access at 90%+ of 400GbE line rate, achieving 3x higher throughput over existing solutions (which fall significantly short of line rate), while reducing host CPU usage by up to 95%, resulting in estimated of 20% total cost of ownership (TCO) savings. For AI, applying MangoBoost DPU to an AI training system that accesses datasets via remote storage leads to a major improvement in system performance for MLPerf Storage Benchmark. Finally, the talk will discuss opportunities for researchers to collaborate and use MangoBoost DPUs for research.
Speaker Bio: Dr. Nurvitadhi is a co-founder and the Chief Product Officer of MangoBoost, Inc. that offers novel customizable data processing unit (DPU) solutions to boost server systems performance and efficiency. MangoBoost, Inc. is a well-funded start-up which has received 65M$ seed and Series-A funds to grow (and actively hiring). Dr. Nurvitadhi’s interests are in hardware accelerator architectures, systems, and software for key application domains (e.g., AI, analytics). Previously, he was a Principal Engineer at Intel, focused on FPGAs, accelerators, and AI technologies. He has 70+ peer-reviewed publications, 120+ patents granted/pending, with H-index of 33. In 2020, he was recognized as a top 30 inventor by Intel Patent Group and received a Mahboob Khan Outstanding Liaison Award from SRC. He has served on program committees of IEEE/ACM conferences, and as the Technical Program Chair for FCCM 2022. He received a PhD in ECE from Carnegie Mellon University, and an MBA from Oregon State University.
----------------------------------------------------------
For more videos subscribe to the RUclips channel / @crossroadsfpga !
For more information visit www.crossroads...
----------------------------------------------------------
Website: www.crossroads...
The Intel/VMware Crossroads 3D-FPGA Academic Research Center is jointly supported by Intel and VMware. The center is committed to public and free dissemination of its research outcome.
----------------------------------------------------------
Chapters
00:00