Skip to content

Search articles, repositories, topics and pages.

RAG

Three Things I Learned About Embeddings

Short lessons from building retrieval with embeddings — chunk size, normalisation and why retrieval needs its own evaluation.

TIAN NEWS AI

Published Updated 1 min read

Documents split into focused chunks, normalized into vectors and compared with a normalized query in a vector index

Chunk size matters more than the model

Switching embedding models moved my retrieval quality a little. Changing chunks from whole pages to a few paragraphs with a short overlap moved it a lot. Small, focused chunks match focused questions.

Normalise before comparing

Cosine similarity compares directions rather than lengths. L2-normalise both the vectors you store and every query vector; the dot product between two normalised vectors is then equal to their cosine similarity.

python
import numpy as np

def normalise(vector: np.ndarray) -> np.ndarray:
    norm = np.linalg.norm(vector)
    if norm == 0:
        raise ValueError("cannot normalise a zero vector")
    return vector / norm

Evaluate retrieval on its own

When a RAG answer is wrong, first check whether the right chunk was retrieved at all. Keep a small list of questions with the chunk that should answer them, and measure how often it appears in the top results.

Key takeaway

If retrieval is wrong, no prompt will fix the answer.

  1. scikit-learn cosine similarity — cosine similarity as a normalized dot product.