Buck work graph (BXL planners)
SearchBench exposes an internal work-graph layer: Buck2/BXL under buck/bxl/ that prints versioned JSON documents. This is for contributors, meta-harness experiments, and future optimizer traces—not required for running rounds.
Document kinds
kind | Purpose |
|---|---|
searchbench.bxl_target_summary.v1 | Smoke: label + coarse target class |
searchbench.backend_resolution.v1 | Descriptor path, launcher, validator metadata |
searchbench.backend_inventory.v1 | Known optimizable backend targets |
searchbench.proof_plan.v1 | Minimal / acceptable / fallback / too-live proof targets |
searchbench.affected_plan.v1 | Heuristic impact from changed paths |
searchbench.comparison_matrix.v1 | Rounds × backends comparison matrix (legacy_kind: searchbench.evaluation_matrix.v1) |
searchbench.repo_policy.v1 | Repo shape / policy heuristic scan |
searchbench.release_gate.v1 | Ordered release-plane steps (#102) |
searchbench.cost_plan.v1 | Planned live matrix run + paths for Go estimate_cost; targets are derived from searchbench_round_op nodes (same manifest_dir) plus CLI --matches vs each matrix op's configured dataset_max_items |
Schemas: buck/bxl/schemas/*.schema.json.
Proof tiers
Proof plans classify Buck targets for future proof distance metrics (how far an agent’s chosen checks were from the minimal correct set):
- minimal — smallest known validation for a round (e.g.
//configs/rounds/live-ic-vs-jcodemunch:validatefor optimize-ic today). - acceptable — minimal plus repo/round checks (
//:check,//src/searchbench-go:check, bundle validate). - fallback — broad gate (
//:check_full). - too_live_for_default_gate — live smoke, full run, provider-backed evaluate targets.
Split of responsibilities
Buck targets → stable graph-addressed capabilities
BXL (buck/bxl) → JSON plans over those capabilities
Go SearchBench → lifecycle, evaluation, scoring, bundles
Meta-harness → branches, worktrees, retries (out of repo)BXL does not replace Pkl manifests, Go round runners, or public CLI workflows.
Running planners
From repo root inside nix develop:
buck2 bxl //buck/bxl/searchbench.bxl:list_backendsFull command list: buck/bxl/README.md.
Graph traversal (not a static registry)
Planners query the Buck graph via BXL uquery and configured target attributes:
- Backends:
attrfilter(name, optimizable_backend, //...) - Rounds with Buck ops:
kind(searchbench_round_op, //...)+manifest/manifest_dirattrs - Pkl-only rounds:
//:searchbench_go_test_resourcesconfiguredsrcs(expanded artifacts) - Affected proof targets:
owner/targets_in_buildfileseeds →rdepson gatetest_suitetargets;go_testresourcesscanning for filegroup deps
rdeps universes are limited to gate suites because traversing arbitrary go_test / searchbench_round_op deps can fail on toolchains//:cxx_no_default_deps in this workspace.
Research context: bxl-meta-harness.md.