4-bit Rotational Quantization
By Steven Van ·
4-bit RQ shrinks the index further than 8-bit, works with the hnsw index only, and can be enabled on new or existing uncompressed collections.
Weaviate's rotational quantization (RQ) now has a Weaviate 4-bit setting, which stores each dimension in 4 bits. The index is smaller than with 8-bit RQ, at some cost in accuracy. In the release note, Weaviate says 4-bit RQ gives similar recall with a 45% heap reduction.
Enabling 4-bit RQ
For a new collection, set bits to 4 in the RQ quantizer when you create the collection. An optional rescoreLimit sets how many candidates are fetched before rescoring. The docs show examples for Python, JavaScript/TypeScript, Go, Java and C#.
An existing collection that is not yet compressed can be switched by updating its collection definition. Weaviate re-encodes the existing vectors in the background. The docs show this for Python, Go, Java and C#.
Index and parameter limits
- 4-bit RQ requires the hnsw index. A dynamic index uses it only after converting to HNSW.
- The flat index accepts only 8-bit and 1-bit RQ, and hfresh accepts only 1-bit.
- The bits value is fixed once RQ is enabled and cannot be changed later.
- For hnsw, rescoreLimit defaults to 20 for 4-bit and 8-bit RQ. It can be changed at any time.
Faster rotation and encoding
The release also adds SIMD support for the Fast Walsh-Hadamard Transform that RQ uses to rotate vectors. Weaviate reports speedups of 3.1× to 6.1× on the transform itself, depending on size and CPU. On an Intel Xeon 8581C, 8-bit RQ encoding went from 27.3 µs to 7.11 µs (3.8×). 1-bit RQ encoding went from 15.2 µs to 6.84 µs (2.2×).