it is a vector quantization technique, flat represents no quantization, and PQ16 represents qunatizing the vector of d dimension to 16byte codes. So once you are searching the cell, using quantization can actually speed up the disctance calculation of a query vector from neighbouring search vectors in a cell. However the PQ codes have to be generated per cell.
Mark - top notch post ! I have been following your duckdb videos, and the quality of your videos just keeps improving ! !
Thanks, I'm glad you like them!
Thanks man. Simple and easy to understand.
thank you, great content!
Nice.
Hi, what is IVF_FLAT? it sounds same to the concept u explained in the video
I find that in the docs for Milvus (a vector db) - milvus.io/docs/index.md - and you're right, it does sound similar if not the same.
it is a vector quantization technique, flat represents no quantization, and PQ16 represents qunatizing the vector of d dimension to 16byte codes. So once you are searching the cell, using quantization can actually speed up the disctance calculation of a query vector from neighbouring search vectors in a cell. However the PQ codes have to be generated per cell.
Please, don't fast-forward the coding part. I have to watch you video with 2.0x slow motion. Then only I can understand.
Thanks for the advice - lemme see how I can do that in the future videos
@@learndatawithmark Thanks for considering