Commit Graph

28 Commits

Author SHA1 Message Date
9eb52f8cef feat(benchmark): support multi-dataset evaluation with configurable top-k list
- Evaluate multiple datasets in a single run (CIFAR10 + CIFAR100)
- Report Recall@K for a list of K values from one underlying search
- Save results to disk: summary.md, metrics.csv, per-dataset predictions.csv, confusion_matrix.csv
- Richer evaluation output: query_labels, topk_ids, topk_labels for downstream analysis
- Add --dataset and --top-k CLI overrides for benchmark command
- Update config schema: dataset→datasets, top_k→top_k_list
2026-05-31 18:58:01 +08:00
7cb6257531 feat(benchmarks): add noise injection experiment support to CAM retrieval benchmark
- Remove hard assertion blocking WRITE_NOISE_EN=1 in retrieval benchmark tests
- Add conditional exact_match assertion: enforces 100% when noise=off, skips when noise=on
- New script run_retrieval_noise_sweep.py: sweeps noise 0–100% (step 10%) and produces markdown summary
- Add just recipes: cam-benchmark-retrieval-sweep, cam-benchmark-sweep-cifar10, cam-benchmark-sweep-cifar100
- Add rsync-based remote sync commands for outputs and docs
2026-05-27 19:04:25 +08:00
d6e1b9d8ba refactor(cam): rename read-noise path to read-pass-through and reorganize test module structure
- Rename `read_noise` scenarios and noise_mode to `read_pass_through` across
  run_cam_correctness.py, test_run_cam_correctness.py, and Makefiles
- Update RTL comment in match_engine_pipeline.sv to reflect pass-through behavior
- Move unit-level cocotb tests from `tests.test_*` flat namespace to
  `tests.modules.*` and `tests.top.*` subdirectory layout, matching actual
  Makefile paths (hw/sim/tests/modules/..., hw/sim/tests/top/...)
- Remove redundant dual-noise subtarget from read_noise/Makefile
- Update help text and docs to reflect read-path pass-through semantics
- Add .codegraph to .gitignore
2026-05-27 16:34:34 +08:00
8b4d4c1b57 refactor(cam): remove read noise from noise architecture (Phase 2)
- Make cam_read_noise a pass-through module, removing all noise injection logic
- Switch write noise to use noise_mask_bernoulli instead of noise_mask_grouped
- Add state machine to cam_write_noise for mask generation timing
- Remove noise_mask_grouped.sv (no longer needed)
- Remove read noise parameters from cam_noisy and cam_top
- Update simulation and benchmark code to reflect read noise removal
- Sync documentation to reflect Phase 2 architecture
2026-05-26 23:45:52 +08:00
1ff9a5f18b feat(retrieval-benchmark): add support for external pre-prepared CAM retrieval datasets with recall@k metric
- Add just recipes for preparing CIFAR10/100 hash artifacts and running benchmarks
- Add CAM_RETRIEVAL_DATASET env var support in Makefile
- Add load_retrieval_dataset_npz() to load pre-prepared retrieval datasets
- Add label_hits counter and recall@k metric for retrieval evaluation
- Rename macro_recall to retrieval_recall to clarify semantics
2026-05-22 21:07:10 +08:00
e1bed00cc4 feat(retrieval): add CAM retrieval benchmark with topk scoring and read noise support
- Add cocotb benchmark infrastructure under hw/sim/benchmarks/retrieval/ with Makefile
- Implement test_retrieval_benchmark.py supporting configurable topk-k, read/write noise
- Add cluster-based synthetic dataset generator with configurable bit-flip rates
- Add reference model functions: match_topk, match_topk_from_scores, score_rows_with_read_noise
- Add .justfile shortcuts: cam-test-retrieval-no-noise, cam-test-retrieval-read-noise
- Add TOPK_K to Verilator EXTRA_ARGS via cocotb-common.mk
- Add unit tests for topk sorting logic and stateful read-noise scoring
2026-05-22 19:04:54 +08:00
29f4cc91f6 fix(pipeline): add S_WAIT_READ_RESP state to fix read-noise PRNG timing
- Emit rd_valid_o for exactly one combinational cycle before waiting for
  read response, ensuring the read-noise PRNG advances once per batch issue
- Fix query handshake: wait for query_ready before asserting query_valid
  to avoid valid&&ready handshake drops on clock edges
- Add dynamic timeout estimation in test utilities based on DUT parameters
- Update test-top Makefile to run all noise configurations by default
- Remove uv run prefix from cocotb-config Makefile invocation
2026-05-22 19:04:50 +08:00
706d148a0b feat(hw): add CAM top-level synthesis infrastructure and fix RTL synthesis compatibility
- Add hw/syn/Makefile with hier/flat/full synth targets and artifact mirroring
- Add synth_cam_top_hier.ys for hierarchy-preserving resource estimation on Xilinx 7-series
- Add synth_cam_top_flat.ys for flattened Xilinx 7-series synthesis
- Add cam-synth just target for convenient invocation
- Guard runtime assertions (NUM_ROWS/LANES checks, noise seed checks, NOISE_BITS checks)
  behind SYNTHESIS guard in cam_core_banked, cam_read_noise, cam_write_noise, and noise_mask_grouped
- Fix shadowed 'return' variable in random128 xorshift128 function
2026-05-18 15:40:04 +08:00
4ea567adba feat(compressors): add JSONL training metrics logging with CLI controls
- Add write_training_metrics() in new compressors/training_metrics.py
  for appending epoch/step/lr/component rows as JSON Lines
- Wire --metrics-path and --log-every CLI options into train.py, passing
  them to the training loop so metrics rows are written every N steps
- Accept absolute metrics paths or paths relative to output directory
- Add quantization component to loss log alongside existing distill/contrastive
- Replace inline torch.device() with get_device() utility
- Add test_hash_training_metrics.py covering multi-row JSONL append

Infrastructure:
- Pin torch 2.7.1 + CUDA 12.8 index for Linux/Windows in pyproject.toml
- Add .justfile rsync upload recipe with .stignore exclusion
- Exclude **/__marimo__ from rsync in .stignore

Dependencies updated: numpy 2.4.5, pandas 3.0.3, black 26.5.0,
click 8.4.0, contourpy, etc.
2026-05-17 14:57:10 +08:00
e8c890a69f feat(sim): enable read noise by default and improve test infrastructure
- Set READ_NOISE_EN=1 as default in cam_top.sv
- Wire READ_NOISE_* parameters to NOISE_* makefile variables in cocotb-common.mk
- Add PYTHON env variable support in Makefile for test invocation
- Update .gitignore with glob patterns (**/sim_build/, **/results.xml, *.fst)
- Add justfile recipe for remote cam-test-top execution
2026-05-17 14:11:59 +08:00
e7765cdb76 refactor(build): consolidate CAM test commands under new make targets
- .envrc: add conda compiler flags (CFLAGS, CXXFLAGS, LDFLAGS) for zlib support
- .justfile: replace inline make commands with reusable targets (test-top, test-all, test-model, test-module)
- environment.yml: reformat YAML, add zlib/gtkwave, rename withbullet → with_bullet
2026-05-16 19:57:34 +08:00
c41e64d1c6 feat(dev): add CAM test recipes and update ignore files
- Add `.opencode/` and `**/sim_build` to ignore lists
- Fix justfile syntax: `set dotenv-required := true` (explicit assignment)
- Update SSH tunnel port 8384 → 42705 for local port forwarding
- Add CAM verification recipes: cam-test-all, cam-test-py, cam-test-module, cam-test
2026-05-13 18:22:14 +08:00
2da17e101b feat(rtl): migrate CAM interface to handshake protocol with integrated noise generation
BREAKING CHANGE: CAM write and query interface replaced with standard valid/ready
handshake. wr_en/wr_row/wr_hash → wr_valid/wr_ready/wr_addr/write_hash.
External noise_mask_lanes_flat removed; noise generation now handled internally
by cam_noisy module with configurable rate via parameters.

- cam_top: add parameters (NOISE_EN, NOISE_RATE_NUM/DEN, NOISE_GEN/SAMPLE_BITS, NOISE_SEED)
- cam_top: replace cam_core with cam_noisy (integrated noise generation)
- match_engine: remove external noise_mask_lanes_flat input
- hw/sim: update Makefile with noise parameters and compile args
- hw/sim/model: add generate_write_flip_mask() and xorshift64() matching RTL behavior
- hw/sim/tests: adapt testbench to new handshake protocol
2026-05-04 18:03:06 +08:00
0ae6d757dc refactor(scripts): extract shared utilities into common module
- Move load_env_file, parse_timeout_seconds, build_remote_script, and
  build_ssh_command to scripts/common.py
- Update remote_docker_run.py to import from common module
- Improves code organization and reusability
2026-05-03 19:49:45 +08:00
7450505a86 feat(remote): unify remote execution workflow with env-based config
- Replace hardcoded SSH/docker targets with environment variables
- Add remote-check, remote-dry, remote-test, remote-test-one recipes
- Add remote cmd for arbitrary command execution in docker container
2026-05-03 15:32:32 +08:00
e3cdc1c6e7 chore(project): update .envrc and justfile for server, and update marimo 2026-04-18 21:10:22 +08:00
8a56c9649d refactor(verification): batch pipeline inference with progress tracking
- Increase verification params: image_size 512→768, rooms 4→5, views 6→12
- Refactor single-batch inference to chunked batch processing with mo.progress_bar
- Extract debug_meta and hash_batches from output for clearer variable flow
- Add progress bars to room-view snapshot saving and scene graph building
- Add .ruff_cache/, .pytest_cache/, .sisyphus/ to .justfile upload excludes
2026-04-11 17:09:39 +08:00
eb016385f8 refactor(sync): improve topdown rendering and make justfile adapt to windows 2026-04-06 19:55:52 +08:00
5f41cf5794 feat(compressors): add OWLv2 bbox crop to HashPipeline and refactor image utilities
- Add crop_batch method to HashPipeline for cropping images using OWLv2 detection boxes
- Integrate crop_batch into pipeline forward pass (extract_hash and extract_features)
- Move extract_masked_region from compressors/proposal/utils.py to utils/image.py
- Add crop_image_by_bbox utility function in utils/image.py
- Update type annotations to use dict[str, Any] instead of bare dict
- Update .justfile to add memory server command
- Update marimo dependency to >=0.22.0
- Update nvidia CUDA package markers for platform compatibility
2026-04-03 19:43:43 +08:00
eaf02cc97a chore: update dev environment and sync workflow
- Add nil LSP package to devenv.nix
- Add datasets and rich.progress imports to feature_retrieval.py
- Update pyproject.toml ty environment config
- Expand .stignore with cache directories
- Update uv.lock dependency lock
- Update AGENTS.md, adding the content of development environment11
2026-04-02 15:02:51 +08:00
e544a7e84f docs(agents): update agent guidelines and remove memorix references 2026-03-30 21:31:46 +08:00
9e6339e580 chore(deps): add sam2 dependency and update docker config 2026-03-26 19:44:47 +08:00
55750f1087 chore(project): update configuration files and dependencies 2026-03-24 19:49:06 +08:00
f60669cd53 chore(dev): update environment and tooling configuration 2026-03-24 14:19:40 +08:00
2985b68f9a fix(env): environment setup correctly and pass habitat test 2026-03-18 21:08:38 +08:00
5885702bed chore: use micromamba and uv to manage python environment 2026-03-18 20:14:30 +08:00
c8dc5f9301 docs: update project documentation and configuration 2026-03-07 15:45:28 +08:00
f9a359fc28 Init python and env 2026-01-30 13:28:59 +08:00