5.14 The find_near_duplicates tool ¶

Read-only diagnostic for memory cleanup. Returns clusters of active memories whose embeddings are at least threshold similar to each other. Clusters are formed by union-find over qualifying pairs (so transitively similar rows a~b~c land in the same group) and sorted strongest-first by the max pair-similarity within each cluster. Soft-deleted and expired rows are excluded.

Pair candidates are looked up via pgvector’s HNSW index using a fixed top-10 per row, which is more than enough at the cutoffs typical for cleanup (threshold >= 0.85).

• Inputs
• Output
• Errors

{
  "ok": true,
  "threshold": 0.92,
  "groups": [
    {
      "ids": [4, 5],
      "max_similarity": 0.984,
      "pair_count": 1,
      "memories": [
        { "id": 4, "text": "...", "tags": ["family"], ... },
        { "id": 5, "text": "...", "tags": ["family"], ... }
      ]
    }
  ],
  "summary": { "groups": 5, "pairs": 8 }
}