feat(benchmark): add software CAM retrieval benchmark

Add software-based CAM retrieval benchmark to compare retrieval quality
and speed against hardware simulation. Includes experiment documentation
with noise sweep analysis on CIFAR-10/100 datasets.

- Add sw_retrieval_benchmark.py for software Hamming distance Top-K retrieval
- Add test_sw_retrieval_benchmark.py with unit tests for dataset loading and metrics
- Add experiment doc (sw_hw_cam_retrieval_benchmark.md) comparing software vs hardware
- Document noise sweep impact on retrieval quality at various WRITE_NOISE_RATE values

docs/exps/sw_hw_cam_retrieval_benchmark.md

@@ -0,0 +1,160 @@
+# 软件/硬件 CAM 检索基准实验总结
+
+**日期**：2026-05-27  
+**工作区**：`.workspace/feat_sw_retrieval_benchmark`  
+**目标**：对比同一组 CAM 检索数据在硬件仿真与软件汉明距离检索下的检索质量，并记录软件检索速度基线。
+
+## 1. 实验配置
+
+### 数据集与哈希配置
+
+| 数据集 | 行数 | 查询数 | 类别数 | 哈希宽度 | Top-K |
+|---|---:|---:|---:|---:|---:|
+| CIFAR-10 | 512 | 128 | 10 | 512 bit | 5 |
+| CIFAR-100 | 512 | 128 | 100 | 512 bit | 5 |
+
+数据文件来自：
+
+- `outputs/cam_retrieval_benchmark/datasets/cifar10_hash512_rows512_queries128.npz`
+- `outputs/cam_retrieval_benchmark/datasets/cifar100_hash512_rows512_queries128.npz`
+
+`.npz` 内部字段为：
+
+- `rows_words`
+- `row_labels`
+- `queries_words`
+- `query_labels`
+
+软件基准直接复用硬件基准数据格式，将 little-endian `uint64` words 转为 Python `int` 后执行汉明距离 / XNOR-popcount Top-K 检索，避免通过软件 CAM 时序模拟带来额外开销。
+
+### 运行命令
+
+软件检索速度基准：
+
+```bash
+just remote "python scripts/sw_retrieval_benchmark.py --dataset outputs/cam_retrieval_benchmark/datasets/cifar10_hash512_rows512_queries128.npz --hash-bits 512 --topk-k 5 --run-id sw_cifar10_hash512_rows512_queries128 && python scripts/sw_retrieval_benchmark.py --dataset outputs/cam_retrieval_benchmark/datasets/cifar100_hash512_rows512_queries128.npz --hash-bits 512 --topk-k 5 --run-id sw_cifar100_hash512_rows512_queries128"
+```
+
+硬件噪声扫描数据来自远端已有 Cocotb/Verilator 输出与 `docs/exps/cam_retrieval_noise_sweep_*.md`。
+
+## 2. 软件检索速度与质量
+
+软件路径只计时 Top-K 匹配阶段，不包含 `.npz` 加载、指标聚合和结果写盘。
+
+| 数据集 | Hit@1 | Precision@1 | Std-Recall@1 | Hit@5 | Precision@5 | Std-Recall@5 | Golden Match@5 | ns/query | queries/s |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| CIFAR-10 | 1.000000 | 1.000000 | 0.019531 | 1.000000 | 1.000000 | 0.097656 | 1.000000 | 205835.281 | 4858.254 |
+| CIFAR-100 | 0.695312 | 0.695312 | 0.134635 | 0.867188 | 0.462500 | 0.445052 | 1.000000 | 205868.953 | 4857.459 |
+
+观察：
+
+- 两个数据集的软件吞吐都约为 **4.86k queries/s**。
+- CIFAR-10 的 Hit@1/Hit@5 均为 1.0，说明在该 512-row 子集上 Top-K 中总能命中同类样本。
+- CIFAR-100 的类别更多、每类样本更少，Hit@1 降至 0.695312，Hit@5 为 0.867188。
+- `Golden Match@K = 1.0` 表示软件实现与 `hw/sim/model/ref_model.py::match_topk` 的排序结果一致。
+
+## 3. 无噪声硬件仿真与软件基线一致性
+
+硬件无噪声结果来自 `WRITE_NOISE_EN=0` 的 Cocotb/Verilator 检索基准。该结果用于确认硬件 Top-K 输出与参考模型一致。
+
+| 数据集 | 模式 | Hit@1 | Hit@5 | Precision@5 | Std-Recall@5 | Golden Match@1 | Golden Match@5 |
+|---|---|---:|---:|---:|---:|---:|---:|
+| CIFAR-10 | software-hamming | 1.000000 | 1.000000 | 1.000000 | 0.097656 | 1.000000 | 1.000000 |
+| CIFAR-10 | hardware no-noise | 1.000000 | 1.000000 | 1.000000 | 0.097656 | 1.000000 | 1.000000 |
+| CIFAR-100 | software-hamming | 0.695312 | 0.867188 | 0.462500 | 0.445052 | 1.000000 | 1.000000 |
+| CIFAR-100 | hardware no-noise | 0.695312 | 0.867188 | 0.462500 | 0.445052 | 1.000000 | 1.000000 |
+
+结论：
+
+- 无噪声硬件仿真与软件汉明距离基线在两个数据集上质量指标一致。
+- 这说明 `.npz` 数据加载、little-endian word 转换、硬件 CAM 匹配和软件参考模型在当前配置下是对齐的。
+
+## 4. 写噪声对硬件检索质量的影响
+
+硬件噪声实验使用 `WRITE_NOISE_EN=1`，按 10% 步长扫描 `WRITE_NOISE_RATE_NUM / 100`。以下表格保留主要指标 Hit@1、Hit@5 与 Golden Match@K。
+
+### CIFAR-10 噪声扫描
+
+| 写噪声率 | Hit@1 | Hit@5 | Golden Match@1 | Golden Match@5 |
+|---:|---:|---:|---:|---:|
+| 0% | 1.000000 | 1.000000 | 1.000000 | 1.000000 |
+| 10% | 1.000000 | 1.000000 | 0.507812 | 0.000000 |
+| 20% | 1.000000 | 1.000000 | 0.234375 | 0.000000 |
+| 30% | 0.992188 | 1.000000 | 0.164062 | 0.000000 |
+| 40% | 0.984375 | 1.000000 | 0.093750 | 0.000000 |
+| 50% | 0.257812 | 0.750000 | 0.023438 | 0.000000 |
+| 60% | 0.000000 | 0.015625 | 0.000000 | 0.000000 |
+| 70% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+| 80% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+| 90% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+| 100% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+
+观察：
+
+- CIFAR-10 在 0%–40% 写噪声下 Hit@5 仍保持 1.0。
+- Golden Match@5 从 10% 噪声开始降为 0，说明 Top-5 的精确排序对噪声非常敏感。
+- 50% 噪声是明显拐点：Hit@1 降至 0.257812，Hit@5 降至 0.75。
+- 60% 以后检索基本失效。
+
+### CIFAR-100 噪声扫描
+
+| 写噪声率 | Hit@1 | Hit@5 | Golden Match@1 | Golden Match@5 |
+|---:|---:|---:|---:|---:|
+| 0% | 0.695312 | 0.867188 | 1.000000 | 1.000000 |
+| 10% | 0.585938 | 0.812500 | 0.593750 | 0.023438 |
+| 20% | 0.562500 | 0.742188 | 0.460938 | 0.000000 |
+| 30% | 0.460938 | 0.640625 | 0.304688 | 0.000000 |
+| 40% | 0.234375 | 0.460938 | 0.101562 | 0.000000 |
+| 50% | 0.000000 | 0.062500 | 0.000000 | 0.000000 |
+| 60% | 0.000000 | 0.007812 | 0.000000 | 0.000000 |
+| 70% | 0.000000 | 0.007812 | 0.000000 | 0.000000 |
+| 80% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+| 90% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+| 100% | 0.000000 | 0.000000 | 0.000000 | 0.000000 |
+
+观察：
+
+- CIFAR-100 对噪声更敏感：10% 噪声时 Hit@1 从 0.695312 降至 0.585938，Hit@5 从 0.867188 降至 0.812500。
+- 40% 噪声时 Hit@5 已降至 0.460938。
+- 50% 噪声后检索基本不可用。
+- 与 CIFAR-10 相同，Golden Match@5 在低噪声下也迅速接近 0，说明精确排序比类别命中率更脆弱。
+
+## 5. 指标解释
+
+| 指标 | 含义 |
+|---|---|
+| Hit@K / `recall@k` | 每个 query 的 Top-K 中是否至少出现一个同类样本，然后对 query 取平均。 |
+| Precision@K / `macro_precision` | 每个 query 的 Top-K 中同类样本比例，即 `tp / k`，再对 query 取平均。 |
+| Std-Recall@K / `retrieval_recall` | 每个 query 检出的同类样本数占数据库中所有同类样本数的比例，即 `tp / |relevant|`，再取平均。 |
+| Std-F1@K / `macro_f1` | 使用 Precision@K 与 Std-Recall@K 计算的 F1。 |
+| Golden Match@K / `exact_match_rate` | Top-K row index 列表是否与参考模型完全一致。该指标比 Hit@K 更严格。 |
+| `ns_per_query` | 软件 Top-K 匹配阶段平均耗时；不含加载和写盘。 |
+| `queries_per_second` | 软件 Top-K 匹配阶段吞吐率。 |
+
+## 6. 当前结论
+
+1. **软件汉明距离基线已可作为硬件 CAM 检索的功能参考。**  
+   在无噪声条件下，硬件仿真和软件基线的质量指标及 Golden Match 均一致。
+
+2. **当前软件基线速度约为 4.86k queries/s。**  
+   该结果来自 Python integer brute-force Hamming scan，数据规模为 512 rows × 128 queries × 512 bits。
+
+3. **硬件检索质量基准目前主要报告质量指标，不报告 cycles/query。**  
+   现有硬件性能基准中已有 cycle 统计逻辑，但尚未和真实 retrieval dataset 的 Top-K benchmark 合并。因此本文不报告硬件 query/s 或 cycles/query。
+
+4. **写噪声对精确排序影响显著。**  
+   即使 Hit@K 保持较高，Golden Match@K 也会快速下降，说明噪声首先破坏精确排序，再进一步破坏类别命中。
+
+5. **CIFAR-100 比 CIFAR-10 更能体现检索难度。**  
+   在无噪声下 CIFAR-100 的 Hit@1/Hit@5 分别为 0.695312/0.867188，明显低于 CIFAR-10 的 1.0/1.0，更适合作为后续检索质量对比主数据集。
+
+## 7. 后续建议
+
+1. 将 `hw/sim/tests/perf/test_cam_perf.py::query_once_with_latency` 的周期测量逻辑合并到 retrieval benchmark，记录真实数据集上的 `cycles/query`。
+2. 在 `docs/exps` 中继续维护：
+   - 软件检索速度表；
+   - 硬件无噪声一致性表；
+   - 硬件噪声鲁棒性表；
+   - 后续硬件 `cycles/query` 表。
+3. 对软件基线补充 NumPy/PyTorch vectorized Hamming scan，以区分“朴素 Python baseline”和“优化软件 baseline”。
+4. 增加 `NUM_ROWS` sweep：例如 512、1024、2048、4096 rows，观察软件 brute-force scan 的线性增长趋势。

scripts/sw_retrieval_benchmark.py

@@ -0,0 +1,329 @@
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+import time
+from dataclasses import dataclass
+from datetime import datetime
+from pathlib import Path
+from typing import Callable, Iterable, Sequence
+
+import numpy as np
+
+
+BENCHMARK_KS = (1, 5)
+PROJECT_ROOT = Path(__file__).resolve().parents[1]
+HW_SIM_DIR = PROJECT_ROOT / "hw" / "sim"
+if str(HW_SIM_DIR) not in sys.path:
+    sys.path.insert(0, str(HW_SIM_DIR))
+
+from model.ref_model import match_topk as ref_match_topk  # noqa: E402
+
+
+@dataclass(frozen=True)
+class RetrievalDataset:
+    rows: list[int]
+    row_labels: list[int]
+    queries: list[int]
+    query_labels: list[int]
+    hash_bits: int
+    num_classes: int
+    positives_per_class: int = 0
+    queries_per_class: int = 0
+    seed: int = 0
+
+
+@dataclass(frozen=True)
+class MetricAccumulator:
+    precision_sum: float = 0.0
+    recall_sum: float = 0.0
+    f1_sum: float = 0.0
+    exact_matches: int = 0
+    label_hits: int = 0
+    count: int = 0
+
+    def add(self, precision: float, recall: float, f1: float, label_hit: bool, exact: bool) -> "MetricAccumulator":
+        return MetricAccumulator(
+            precision_sum=self.precision_sum + precision,
+            recall_sum=self.recall_sum + recall,
+            f1_sum=self.f1_sum + f1,
+            exact_matches=self.exact_matches + int(exact),
+            label_hits=self.label_hits + int(label_hit),
+            count=self.count + 1,
+        )
+
+    def as_dict(self) -> dict[str, float]:
+        if self.count == 0:
+            return {
+                "macro_precision": 0.0,
+                "retrieval_recall": 0.0,
+                "macro_f1": 0.0,
+                "exact_match_rate": 0.0,
+                "recall@k": 0.0,
+            }
+        return {
+            "macro_precision": self.precision_sum / self.count,
+            "retrieval_recall": self.recall_sum / self.count,
+            "macro_f1": self.f1_sum / self.count,
+            "exact_match_rate": self.exact_matches / self.count,
+            "recall@k": self.label_hits / self.count,
+        }
+
+
+def project_root() -> Path:
+    return PROJECT_ROOT
+
+
+def words_le_to_int(words: np.ndarray) -> int:
+    value = 0
+    for idx, word in enumerate(words.tolist()):
+        value |= int(word) << (64 * idx)
+    return value
+
+
+def load_retrieval_dataset_npz(path: str | Path) -> RetrievalDataset:
+    dataset_path = Path(path)
+    if not dataset_path.is_absolute():
+        dataset_path = project_root() / dataset_path
+    if not dataset_path.exists():
+        raise FileNotFoundError(f"retrieval dataset not found: {dataset_path}")
+
+    loaded = np.load(dataset_path)
+    rows_words = loaded["rows_words"]
+    queries_words = loaded["queries_words"]
+    if rows_words.ndim != 2 or queries_words.ndim != 2:
+        raise ValueError("rows_words and queries_words must have shape [N, words]")
+    if rows_words.shape[1] != queries_words.shape[1]:
+        raise ValueError("rows_words and queries_words must use the same word width")
+
+    rows = [words_le_to_int(words) for words in rows_words]
+    queries = [words_le_to_int(words) for words in queries_words]
+    row_labels = [int(x) for x in loaded["row_labels"].tolist()]
+    query_labels = [int(x) for x in loaded["query_labels"].tolist()]
+    return RetrievalDataset(
+        rows=rows,
+        row_labels=row_labels,
+        queries=queries,
+        query_labels=query_labels,
+        hash_bits=int(rows_words.shape[1] * 64),
+        num_classes=len(set(row_labels)),
+    )
+
+
+def hamming_similarity_score(query_row: int, stored_row: int, *, width: int) -> int:
+    if width <= 0:
+        raise ValueError("width must be greater than 0")
+    mask = (1 << width) - 1
+    distance = ((int(query_row) ^ int(stored_row)) & mask).bit_count()
+    return int(width - distance)
+
+
+def match_topk_hamming(query: int, rows: Sequence[int], *, width: int, k: int) -> list[int]:
+    if k <= 0:
+        raise ValueError("k must be greater than 0")
+    scored = [
+        (-hamming_similarity_score(query, row, width=width), row_index)
+        for row_index, row in enumerate(rows)
+    ]
+    scored.sort()
+    return [row_index for _, row_index in scored[: min(k, len(scored))]]
+
+
+def compute_metrics(topk_indices: list[int], row_labels: list[int], query_label: int, k: int) -> tuple[float, float, float]:
+    retrieved = topk_indices[:k]
+    relevant = {idx for idx, label in enumerate(row_labels) if label == query_label}
+    tp = len(set(retrieved) & relevant)
+    precision = tp / float(k)
+    recall = tp / float(len(relevant)) if relevant else 0.0
+    f1 = 0.0 if precision + recall == 0 else (2.0 * precision * recall) / (precision + recall)
+    return precision, recall, f1
+
+
+def _normalized_topk_values(topk_values: Iterable[int]) -> tuple[int, ...]:
+    values = tuple(sorted({int(k) for k in topk_values}))
+    if not values or values[0] <= 0:
+        raise ValueError("topk_values must contain positive integers")
+    return values
+
+
+def run_benchmark(
+    dataset_path: str | Path,
+    *,
+    hash_bits: int = 512,
+    topk_values: Iterable[int] = BENCHMARK_KS,
+    run_id: str | None = None,
+    timer_ns: Callable[[], int] = time.perf_counter_ns,
+) -> dict:
+    dataset = load_retrieval_dataset_npz(dataset_path)
+    if not dataset.rows:
+        raise ValueError("cannot benchmark an empty row set")
+    if hash_bits != dataset.hash_bits:
+        raise ValueError(
+            f"hash_bits={hash_bits} does not match dataset width {dataset.hash_bits}"
+        )
+    if len(dataset.queries) != len(dataset.query_labels):
+        raise ValueError("queries and query_labels must have the same length")
+
+    ks = _normalized_topk_values(topk_values)
+    max_k = max(ks)
+    if max_k > len(dataset.rows):
+        raise ValueError("topk_values cannot exceed the number of dataset rows")
+
+    start_ns = timer_ns()
+    all_topk = [
+        match_topk_hamming(query, dataset.rows, width=hash_bits, k=max_k)
+        for query in dataset.queries
+    ]
+    end_ns = timer_ns()
+
+    golden_topk = [
+        ref_match_topk(query, dataset.rows, width=hash_bits, k=max_k)[0]
+        for query in dataset.queries
+    ]
+
+    accumulators = {k: MetricAccumulator() for k in ks}
+    for topk_indices, golden_indices, query_label in zip(all_topk, golden_topk, dataset.query_labels):
+        for k in ks:
+            precision, recall, f1 = compute_metrics(topk_indices, dataset.row_labels, query_label, k)
+            retrieved_labels = [dataset.row_labels[idx] for idx in topk_indices[:k]]
+            label_hit = query_label in retrieved_labels
+            exact = topk_indices[:k] == golden_indices[:k]
+            accumulators[k] = accumulators[k].add(precision, recall, f1, label_hit, exact)
+
+    metrics = {str(k): accumulators[k].as_dict() for k in ks}
+
+    elapsed_ns = max(0, int(end_ns - start_ns))
+    num_queries = len(dataset.queries)
+    ns_per_query = (elapsed_ns / float(num_queries)) if num_queries else 0.0
+    qps = (1_000_000_000.0 / ns_per_query) if ns_per_query > 0 else 0.0
+    resolved_run_id = run_id or f"{datetime.now().strftime('%Y-%m-%d-%H%M%S')}-software-hamming"
+
+    return {
+        "run_id": resolved_run_id,
+        "mode": "software-hamming",
+        "status": "pass",
+        "params": {
+            "num_rows": len(dataset.rows),
+            "hash_bits": int(hash_bits),
+            "topk_k": max_k,
+        },
+        "dataset": {
+            "num_classes": dataset.num_classes,
+            "positives_per_class": dataset.positives_per_class,
+            "queries_per_class": dataset.queries_per_class,
+            "num_queries": num_queries,
+            "seed": dataset.seed,
+        },
+        "metrics": metrics,
+        "performance": {
+            "total_elapsed_ns": elapsed_ns,
+            "total_elapsed_sec": elapsed_ns / 1_000_000_000.0,
+            "ns_per_query": ns_per_query,
+            "queries_per_second": qps,
+        },
+    }
+
+
+def write_outputs(out_dir: Path, result: dict) -> None:
+    out_dir.mkdir(parents=True, exist_ok=True)
+    (out_dir / "metrics.json").write_text(json.dumps(result, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+
+    fieldnames = [
+        "run_id", "mode", "num_rows", "hash_bits", "topk_k", "num_queries", "k",
+        "macro_precision", "retrieval_recall", "macro_f1", "recall@k", "exact_match_rate",
+        "total_elapsed_ns", "ns_per_query", "queries_per_second", "status",
+    ]
+    with (out_dir / "metrics.csv").open("w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        for k, metrics in result["metrics"].items():
+            writer.writerow({
+                "run_id": result["run_id"],
+                "mode": result["mode"],
+                "num_rows": result["params"]["num_rows"],
+                "hash_bits": result["params"]["hash_bits"],
+                "topk_k": result["params"]["topk_k"],
+                "num_queries": result["dataset"]["num_queries"],
+                "k": int(k),
+                "macro_precision": metrics["macro_precision"],
+                "retrieval_recall": metrics["retrieval_recall"],
+                "macro_f1": metrics["macro_f1"],
+                "recall@k": metrics["recall@k"],
+                "exact_match_rate": metrics["exact_match_rate"],
+                "total_elapsed_ns": result["performance"]["total_elapsed_ns"],
+                "ns_per_query": result["performance"]["ns_per_query"],
+                "queries_per_second": result["performance"]["queries_per_second"],
+                "status": result["status"],
+            })
+
+    lines = [
+        "# Software CAM Retrieval Benchmark Summary",
+        "",
+        f"- run_id: `{result['run_id']}`",
+        f"- mode: `{result['mode']}`",
+        f"- status: `{result['status']}`",
+        f"- num_queries: `{result['dataset']['num_queries']}`",
+        f"- ns_per_query: `{result['performance']['ns_per_query']}`",
+        f"- queries_per_second: `{result['performance']['queries_per_second']}`",
+        "",
+        "| k | macro_precision | retrieval_recall | macro_f1 | recall@k | exact_match_rate |",
+        "|---:|---:|---:|---:|---:|---:|",
+    ]
+    for k, metrics in result["metrics"].items():
+        lines.append(
+            f"| {k} | {metrics['macro_precision']:.6f} | {metrics['retrieval_recall']:.6f} | "
+            f"{metrics['macro_f1']:.6f} | {metrics['recall@k']:.6f} | {metrics['exact_match_rate']:.6f} |"
+        )
+    lines.extend([
+        "",
+        "说明：软件路径直接对 `.npz` 中的 CAM 行整数执行汉明距离 / XNOR-popcount 检索，不使用软件 CAM 时序仿真。",
+    ])
+    (out_dir / "summary.md").write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+
+def output_dir_for(run_id: str, output_root: Path) -> Path:
+    return output_root / run_id
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Run software Hamming CAM retrieval benchmark.")
+    parser.add_argument("--dataset", required=True, help="Prepared CAM retrieval .npz artifact")
+    parser.add_argument("--hash-bits", type=int, default=512, help="Hash width in bits")
+    parser.add_argument("--topk-k", type=int, default=5, help="Maximum Top-K to report; reports k=1 and this value")
+    parser.add_argument("--run-id", default=None, help="Output run id")
+    parser.add_argument(
+        "--output-root",
+        type=Path,
+        default=Path("outputs/sw_retrieval_benchmark"),
+        help="Directory under which the run directory is written",
+    )
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    topk_values = (1,) if args.topk_k == 1 else (1, args.topk_k)
+    result = run_benchmark(
+        args.dataset,
+        hash_bits=args.hash_bits,
+        topk_values=topk_values,
+        run_id=args.run_id,
+    )
+    out_dir = output_dir_for(result["run_id"], args.output_root)
+    write_outputs(out_dir, result)
+    print(
+        "SW_RETRIEVAL_RESULT "
+        f"run_id={result['run_id']} "
+        f"num_rows={result['params']['num_rows']} "
+        f"hash_bits={result['params']['hash_bits']} "
+        f"num_queries={result['dataset']['num_queries']} "
+        f"ns_per_query={result['performance']['ns_per_query']:.3f} "
+        f"queries_per_second={result['performance']['queries_per_second']:.3f} "
+        f"output_dir={out_dir}"
+    )
+
+
+if __name__ == "__main__":
+    main()

tests/test_sw_retrieval_benchmark.py

@@ -0,0 +1,152 @@
+from __future__ import annotations
+
+import importlib.util
+import json
+import sys
+from pathlib import Path
+
+import numpy as np
+import pytest
+
+
+def load_sw_benchmark():
+    path = Path(__file__).resolve().parent.parent / "scripts" / "sw_retrieval_benchmark.py"
+    spec = importlib.util.spec_from_file_location("sw_retrieval_benchmark", path)
+    assert spec is not None, f"Could not find spec for {path}"
+    assert spec.loader is not None, f"No loader available for {path}"
+    module = importlib.util.module_from_spec(spec)
+    sys.modules[spec.name] = module
+    spec.loader.exec_module(module)
+    return module
+
+
+def _write_dataset(path: Path) -> None:
+    np.savez_compressed(
+        path,
+        rows_words=np.array(
+            [
+                [0b11110000],
+                [0b11100000],
+                [0b00001111],
+            ],
+            dtype=np.uint64,
+        ),
+        row_labels=np.array([0, 0, 1], dtype=np.int64),
+        queries_words=np.array(
+            [
+                [0b11110000],
+                [0b00001111],
+            ],
+            dtype=np.uint64,
+        ),
+        query_labels=np.array([0, 1], dtype=np.int64),
+    )
+
+
+def test_load_dataset_preserves_little_endian_words(tmp_path):
+    bench = load_sw_benchmark()
+    dataset_path = tmp_path / "dataset.npz"
+    np.savez_compressed(
+        dataset_path,
+        rows_words=np.array([[0x2, 0x1]], dtype=np.uint64),
+        row_labels=np.array([7], dtype=np.int64),
+        queries_words=np.array([[0x4, 0x3]], dtype=np.uint64),
+        query_labels=np.array([8], dtype=np.int64),
+    )
+
+    dataset = bench.load_retrieval_dataset_npz(dataset_path)
+
+    assert dataset.rows == [0x1_0000_0000_0000_0002]
+    assert dataset.queries == [0x3_0000_0000_0000_0004]
+    assert dataset.row_labels == [7]
+    assert dataset.query_labels == [8]
+
+
+def test_run_benchmark_reports_quality_and_query_speed(tmp_path):
+    bench = load_sw_benchmark()
+    dataset_path = tmp_path / "dataset.npz"
+    _write_dataset(dataset_path)
+    timer_values = iter([1_000, 3_000])
+
+    result = bench.run_benchmark(
+        dataset_path,
+        hash_bits=64,
+        topk_values=(1, 2),
+        run_id="unit-test",
+        timer_ns=lambda: next(timer_values),
+    )
+
+    assert result["mode"] == "software-hamming"
+    assert result["status"] == "pass"
+    assert result["dataset"]["num_queries"] == 2
+    assert result["params"]["num_rows"] == 3
+    assert result["params"]["topk_k"] == 2
+    assert result["metrics"]["1"]["exact_match_rate"] == 1.0
+    assert result["metrics"]["1"]["recall@k"] == 1.0
+    assert result["performance"]["total_elapsed_ns"] == 2_000
+    assert result["performance"]["ns_per_query"] == 1_000.0
+    assert result["performance"]["queries_per_second"] == 1_000_000.0
+
+
+def test_run_benchmark_exact_match_compares_against_reference(tmp_path, monkeypatch):
+    bench = load_sw_benchmark()
+    dataset_path = tmp_path / "dataset.npz"
+    _write_dataset(dataset_path)
+
+    monkeypatch.setattr(bench, "ref_match_topk", lambda query, rows, width, k: ([1, 0], []))
+
+    result = bench.run_benchmark(
+        dataset_path,
+        hash_bits=64,
+        topk_values=(1,),
+        timer_ns=lambda: 0,
+    )
+
+    assert result["metrics"]["1"]["exact_match_rate"] == 0.0
+
+
+def test_run_benchmark_rejects_hash_bits_that_do_not_match_npz_width(tmp_path):
+    bench = load_sw_benchmark()
+    dataset_path = tmp_path / "dataset.npz"
+    _write_dataset(dataset_path)
+
+    with pytest.raises(ValueError, match="hash_bits"):
+        bench.run_benchmark(dataset_path, hash_bits=8)
+
+
+def test_write_outputs_includes_quality_and_performance_csv(tmp_path):
+    bench = load_sw_benchmark()
+    result = {
+        "run_id": "unit-test",
+        "mode": "software-hamming",
+        "status": "pass",
+        "params": {"num_rows": 3, "hash_bits": 64, "topk_k": 2},
+        "dataset": {"num_classes": 2, "num_queries": 2},
+        "metrics": {
+            "1": {
+                "macro_precision": 1.0,
+                "retrieval_recall": 0.75,
+                "macro_f1": 0.85,
+                "exact_match_rate": 1.0,
+                "recall@k": 1.0,
+            }
+        },
+        "performance": {
+            "total_elapsed_ns": 2_000,
+            "total_elapsed_sec": 0.000002,
+            "ns_per_query": 1_000.0,
+            "queries_per_second": 1_000_000.0,
+        },
+    }
+
+    bench.write_outputs(tmp_path, result)
+
+    metrics_json = json.loads((tmp_path / "metrics.json").read_text(encoding="utf-8"))
+    metrics_csv = (tmp_path / "metrics.csv").read_text(encoding="utf-8")
+    summary_md = (tmp_path / "summary.md").read_text(encoding="utf-8")
+
+    assert metrics_json["performance"]["queries_per_second"] == 1_000_000.0
+    assert "queries_per_second" in metrics_csv
+    assert "1000000.0" in metrics_csv
+    assert "Software CAM Retrieval Benchmark Summary" in summary_md
+    assert "queries_per_second" in summary_md