The labs dashboard

Scoreboard: the newest row per model, lowest LPIPS first

Negative control 17:31 UTC: nc-noise-bicubic LPIPS 0.8875, beats bicubic: False; nc-noise-white LPIPS 0.7840, beats bicubic: False; nc-shuffled LPIPS 0.7394, beats bicubic: False. Held-out registry SHA-256 e3743f08605f2f11….

Receipts fetched by URL at build time: measure-20260902T172415Z-77645bc8ec58.json · measure-20260902T173233Z-fdb97b77c83f.json · measure-20260902T184007Z-f7a0cf643988.json · negative-control-20260902T172258Z-77645bc8ec58.json · negative-control-20260902T173148Z-fdb97b77c83f.json

Did not answer: measure-20260902T192013Z-d22b86cf9397.json (GET https://store-dev.agentarmies.io/labs/lab-earth-compiler/measure/measure-20260902T192013Z-d22b86cf9397.json: HTTP 404)

Any-coordinate registries, frozen before scoring and excluded from training

Kill criterion (2026-09-30, or earlier if the metric moves)

Keep only if all three hold on input=real-s2:

• LPIPS on the Travis strip at most 0.85 × the best of bicubic, classical and sr-small.
• FID on the Travis strip below every baseline's.
• Eval's blind swipe on the Austin block prefers the model over classical at two to one or better. Only eval's verdict gates; the rows diagnose. A model given noise in place of conditioning must not beat bicubic; if it does, the metric is broken and no row counts until it is fixed.

Ledger receipts: setup PR #1, merged · grind PR #2 · scoreboard row, 18:44 UTC · criterion decision, 19:05 UTC · anycoord.v2 frozen, 20:3x UTC

Full hypothesis

A conditional generative pass over imperfect Earth sources emits the layers the tile compiler wants (ortho, materials, heights, tree instances, facade classes) with `authored` provenance and a confidence, trained without labels by degrading places that have good data (RESEARCH.md §4). v0 tests the load-bearing half of that claim: that a generative model conditioned on coarse public sources produces an ortho that is clearly closer to reality than interpolation plus classical processing of the same inputs.

Full ledger status

open — setup merged; scoreboard on the frozen Travis strip (n=1018, input=real-s2, diagnostic): sr-small 0.6286 / 233.66, classical-worldcover-mean 0.6635 / 404.31, classical-worldcover-colour 0.6713 / 363.40, bicubic 0.6854 / 416.79, classical-lanczos-unsharp 0.6886 / 397.98; the SDXL LoRA (rank-64 + a 98 M control network, 29-channel conditioning) trains in lock windows (lab1-grind-v0-4); any-coordinate registries frozen before scoring: anycoord.v1 (256 world land cells, 14 with truth) and anycoord.v2 (2,000, 112 with truth), training excludes both; no generative row yet

labels.v1: 20 tiles over austin, 60 files, 1,709,929 bytes; freeze digest 2b683c32303641ad5ea165230392c262952d838c4e5cbda4b3072abc52160aef; evidence copy aa-gpu-1:evidence/lab-satellite-vision/labels.v1.freeze.json.

The oracle's ceiling, measured on a sample of one tile labelled twice: pixel agreement 66.9 %, mean class IoU 29.8 % (LAB-2 comment 20:56 UTC). Any IoU against labels.v1 carries this ceiling in the same row; shrub has no polygons in the set.

Spend recorded in docs/spend.md: 0 USD.

Kill criterion

Kill on 2026-09-30 unless the metric moves first. Kill earlier if, by 2026-09-16, no v0 class beats its baseline IoU on Austin, or if eval's blind swipe of the compiled block with our layers does not beat the block with the current layers. A negative result is a result; the card's Result section gets it.

Ledger receipts: setup and pull PR #1, merged · pilot checkpoint and decisions, 18:20 UTC · labels.v1 frozen, 20:56 UTC

Full hypothesis

Segmentation and classification models over NAIP (0.3 to 0.6 m) and Sentinel-2 (10 m), helped by lidar where it exists, produce compiler-grade layers (land cover, roofs and roof material, tree crowns; later species and road surface) for the seventeen portal places faster and cheaper than generation. This is the proven path beside the dark horse (LAB-1, `lab-earth-compiler`); our classical layers are also LAB-1's kill baseline ("bicubic plus classical"), so they must exist before LAB-1 can be judged.

Full ledger status

open — setup merged; labels.v1 (Austin, 20 tiles, 1,084 land-cover polygons, 250 roofs, 827 tree points) frozen on R2 at labs/lab-satellite-vision/labels.v1/, digest 2b683c32…, verified by a fresh client; NOT committed to the lab's fixture until a blind second labeller re-labels 3 to 5 tiles and the per-class agreement is published beside it (one-tile sample: 66.9 % pixel agreement, mean class IoU 29.8 %; no shrub polygons); Santa Barbara unlabelled; labelling cost measured at 9 to 15 minutes per canvas block

Frozen ground truth (groundtruth/frozen.json at origin/main, 728fef1)

Frozen ground truth per place: R2 key under labs/lab-llm-authored/groundtruth/, SHA-256, the labeller's run id and date. Regenerated whole, never edited; a correction is a new version with a new digest. facade_material and furniture are unknown for every unit in v1: no cleared source sees facades or street furniture (assessor records carry no exterior wall; no street imagery is cleared)…

B0 tags-only runs and their shuffled controls, as named on the card

measure/rows.csv at origin/main: 0 rows.

Kill and graduation

The precision bar passes when at least one arm beats both baselines on at least four building fields on both places. The blind-margin bar passes when eval's blind margin with the authored layers is above the margin without them. This is the complete decision table on 2026-09-30:

At the kill date, failure of either bar kills the lab. Before the kill date, continue while both bars remain attainable and have not both passed; graduate as soon as both pass, and kill early only when the remaining planned evidence cannot make a failed bar pass. Graduation is a port into earth-data as an enrichment pass (NORMALIZE stage three), schema and confidence intact.

Ledger receipts: setup PR #1, merged · B0 receipts on the store, corrected 22:3x UTC · review fixes at 82cd86d · ground truth and the spend rule, 21:49 UTC

Full hypothesis

Chains of existing models (a frontier vision-language model over ortho, street imagery and tags; open detectors and captioners; a reasoning pass that reconciles sources) can author the semantic fields real data lacks (what a building is, what a street feels like, which props belong) as `authored` fields with a calibrated confidence, at a precision that beats the tags alone and a single zero-shot pass, and eval's blind swipe can tell that authoring from noise. Both the precision bar and the blind-margin bar defined in the decision table must pass; failure of either at the kill date falsifies the hypothesis.

Full ledger status

open — setup merged (PR #1, 21:37 UTC, nine review findings fixed); B0 rows and their shuffled controls on R2 (uploaded and verified 22:3x UTC; the card's earlier paths never answered); ground truth frozen for Austin (1,156 records over 197 units) and Seattle (1,082 over 217) under labs/lab-llm-authored/groundtruth/<place>.v1/ with digests in groundtruth/frozen.json; facade_material and furniture unscorable until a street-imagery source is cleared (DATA-6); paid arms A1 and A2 allowed from 21:56 UTC (Joel), keys on the rig by name; OpenRouter out (INB-11)

Austin, 9 cells: violations by class before and after iteration 2

Any-coordinate v1 (32 world cells drawn with a seeded uniform draw over land, lab4-measure-anycoord-overture.service): footprints on 8, roads on 8, water on 6; quarry column absent; fixture SHA-256 420fdc628405b361….

Negative control on real data (austin-negative-control-20260902T1753Z.log): every applicable class must fire

Receipts copied from the evidence store into this edition: lab-geometry-solves-austin-20260902T175400Z-summary.json · lab-geometry-solves-anycoord-20260902T201216Z-summary.json · lab-geometry-solves-austin-iter2-comparison-20260902T201016Z.json. The lab has no store token yet, so these copies are the public receipts.

Kill criterion

Kill on 2026-09-30, or earlier, if the total violation count on the seventeen places has not fallen week over week for two consecutive weeks against the raw baseline, or if eval's blind swipe does not prefer the solved Austin block over the quarry's by then. Kill also if the solver only lowers counts by raising terrain_edit_excess: that is flattening, not solving.

Ledger receipts: setup PR #1, merged · Austin rows, 17:59 UTC · iteration 2 and the any-coordinate row, PR #3 merged · rows and digests, 20:18 UTC · exact containment and the seventeen-place row, PR #4 merged as 1d83154 · seventeen-place row, 22:29 UTC · lead's reading, 22:4x UTC

Full hypothesis

A constraint solver over footprints, heights, roads, terrain and water, run once per S2 level-16 block (a cell and its 3×3 halo) in one local tangent frame, produces a single consistent plan in which buildings meet the ground, roads meet each other and buildings, and water is level; the compiler projects that plan to render and sim products; and every failure the solver cannot resolve is reported as a named, located violation rather than shipped silently. Corollary under test: solving bodies once by feature id removes the cross-cell steps that per-cell solving produces.

Full ledger status

open — first result on the card's metric, diagnostic: the seventeen places (153 level-16 cells, one recipe: OSM via Overpass, pinned Overture 2026-08-19.0 footprints, Copernicus GLO-30) 3300 raw → 2478 after v0 on 2026-09-02, eleven places improve, four get worse (london, miami-south-pointe, honolulu, san-francisco) and those four carry 199 of the 310 terrain_edit_excess violations v0 introduces (0 raw); the lead reads it as approach A's ordered-ladder limit on 30 m terrain, not the kill clause (which needs the total to fall *only* by terrain edits), and the next iteration targets that class back to 0 with the total still falling; Austin lidar home row 351 → 116 → 90 (separate series); any-coordinate row (32 frozen world cells) 7 → 6 raw after exact S2 containment, 2 v0; cell assignment is exact S2 containment since 1d83154 (tile-compiler's CMP-7 ask); negative control passed on real data (16 of 17 places, kilimanjaro not applicable)

Toolchain proofs on aa-gpu-1 (docs/arm-toolchains.md)

Arms and kill criteria (AST-2 §3)

Every arm produces every item; every output goes through the same validate and the same judge (EVAL-5). Arms interleave in judged windows so host load never confounds a result.

Ledger receipts: setup PR #2, merged · proof table · state and rail date, 17:52 UTC · legal rows, orchestrator PR 6 · rules and asks, 22:0x UTC

Full ledger status

waiting on rails — setup merged; toolchain proofs on aa-gpu-1: Blender Cycles on the GPU (1.9 s, 1.6 GiB), Hunyuan3D 2.1 image-to-3D fits (140 s, 8.3 GiB, shape only), TRELLIS.2 blocked on a gated DINOv3 download; legal rows logged under Joel's acquire-and-log rule (Hunyuan3D permission-pending: its licence excludes the EU and the node is in Helsinki, question on INB-8; A3's ten free Fab items not yet acquired); Tripo and Meshy callable with a batch estimate; judging waits on EVAL-5 (first blind pair 2026-09-09 at the earliest)

Ledger row

baseline: B3, the quarry's facade band-line and material-factory pipeline ported with a MIGRATION row

budget: three weeks; no separate spend line on the card

card: LAB-7

hypothesis: Twenty Austin buildings across four classes through four arms (grammar, image-conditioned diffusion, the quarry pipeline ported, generative 3D on the massing) under pilot A's judge

kill date: three weeks from pilot A's judge being proven

metric: Pilot A's metrics and constraints, per class, with one held-out building per class

realised gain: not yet

receipt: pilot-A rail receipt

slug: lab-forge-facade-bakeoff

status: waiting on rails behind pilot A