Skip to content

Buck work graph (BXL planners)

SearchBench exposes an internal work-graph layer: Buck2/BXL under buck/bxl/ that prints versioned JSON documents. This is for contributors, meta-harness experiments, and future optimizer traces—not required for running rounds.

Document kinds

kindPurpose
searchbench.bxl_target_summary.v1Smoke: label + coarse target class
searchbench.backend_resolution.v1Descriptor path, launcher, validator metadata
searchbench.backend_inventory.v1Known optimizable backend targets
searchbench.proof_plan.v1Minimal / acceptable / fallback / too-live proof targets
searchbench.affected_plan.v1Heuristic impact from changed paths
searchbench.comparison_matrix.v1Rounds × backends comparison matrix (legacy_kind: searchbench.evaluation_matrix.v1)
searchbench.repo_policy.v1Repo shape / policy heuristic scan
searchbench.release_gate.v1Ordered release-plane steps (#102)
searchbench.cost_plan.v1Planned live matrix run + paths for Go estimate_cost; targets are derived from searchbench_round_op nodes (same manifest_dir) plus CLI --matches vs each matrix op's configured dataset_max_items

Schemas: buck/bxl/schemas/*.schema.json.

Proof tiers

Proof plans classify Buck targets for future proof distance metrics (how far an agent’s chosen checks were from the minimal correct set):

  • minimal — smallest known validation for a round (e.g. //configs/rounds/live-ic-vs-jcodemunch:validate for optimize-ic today).
  • acceptable — minimal plus repo/round checks (//:check, //src/searchbench-go:check, bundle validate).
  • fallback — broad gate (//:check_full).
  • too_live_for_default_gate — live smoke, full run, provider-backed evaluate targets.

Split of responsibilities

text
Buck targets       → stable graph-addressed capabilities
BXL (buck/bxl)      → JSON plans over those capabilities
Go SearchBench     → lifecycle, evaluation, scoring, bundles
Meta-harness       → branches, worktrees, retries (out of repo)

BXL does not replace Pkl manifests, Go round runners, or public CLI workflows.

Running planners

From repo root inside nix develop:

bash
buck2 bxl //buck/bxl/searchbench.bxl:list_backends

Full command list: buck/bxl/README.md.

Graph traversal (not a static registry)

Planners query the Buck graph via BXL uquery and configured target attributes:

  • Backends: attrfilter(name, optimizable_backend, //...)
  • Rounds with Buck ops: kind(searchbench_round_op, //...) + manifest / manifest_dir attrs
  • Pkl-only rounds: //:searchbench_go_test_resources configured srcs (expanded artifacts)
  • Affected proof targets: owner / targets_in_buildfile seeds → rdeps on gate test_suite targets; go_test resources scanning for filegroup deps

rdeps universes are limited to gate suites because traversing arbitrary go_test / searchbench_round_op deps can fail on toolchains//:cxx_no_default_deps in this workspace.

Research context: bxl-meta-harness.md.