tinyML Asia 2023 - Kyuwoong Hwang: The Future of AI is “On-Device”

The Future of AI is “On-Device”
Kyuwoong HWANG, Senior Director, Qualcomm AI Research, Korea
Welcome to the era of Generative AI. While it originated in the cloud, it is now making its way to devices. In this introduction, we’ll explore the economic rationale for bringing AI to devices, achieving personalization through user and environmental interaction, and seamless integration with the cloud. Generative AI demands substantial computational power, and processing everything in the cloud can be cost-prohibitive. Running AI algorithms on devices offers fast responses, heightened security, and constant availability. The latest mobile chipsets, like Qualcomm’s Snapdragon, can handle the computational demands of Generative AI, enabling tasks such as image generation, natural language processing, video understanding, speech recognition, and synthesis. Devices, with their predefined users and restricted environments, can achieve impressive performance even with smaller models. A seamless hybrid model that combines device and cloud will provide users with a cloud-like experience without significantly increasing computational costs.