Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Lucene and ES implement a shortcut for filters that are restrictive enough. Since it's already optimized for figuring out if something falls into your filter set, you first determine the size of that. You traverse the HNSW normally, then if you have traversed more nodes than your filter set's cardinality, you just switch to brute forcing your filter set distance comparisons. So worst case scenario is you do 2x your filter set size vector distance operations. Quite neat.


Oh that's nice! Any references on this shortcut? How do you activate that behavior? I was playing around with ES, but the only suggestion I found was to use `count` on filters before deciding (manually) which path to take.


Here you go https://github.com/apache/lucene/pull/656 - no need to do anything from the user side to trigger it as far as I know.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: