Research talk: Approximate nearest neighbor search systems at scale

Speaker: Harsha Simhadri, Principal Researcher, Microsoft Research India
Building deep learning-based search and recommendation systems at internet scale requires a complete redesign of the search index. Key to this redesign is a fast, accurate, and cost-efficient indexing system for approximate nearest neighbor search. In this talk, we’ll present our recent advances in this space, including the DiskANN and FreshDiskANN systems and the underlying algorithms. These algorithms present an order-of-magnitude improvement in scale and cost-of-operation over the state of the art and are a first of their kind at effectively using solid-state drives (SSDs) to serve at interactive (milliseconds) latencies. In addition, they provide faster in-memory search than other graph indices, like HNSW, and support real-time concurrent insertions and deletions to SSD-resident indices without losing recall. We’ll provide an overview their applicability to various product scenarios and highlight directions for further research.
Learn more about the 2021 Microsoft Research Summit: Aka.ms/researchsummit

[@billykotsos4642]

ANN systems are going to blow up in the coming years. Deep learning ftw !

[@vaibhav_haibatrao_patil]

Excellent !

[@billykotsos4642]

So many problems would be solved we just had 1K GB of RAM