The labs dashboard

Scoreboard: the newest row per model, lowest LPIPS first

Negative control 07:19 UTC: nc-noise-bicubic LPIPS 0.9216, beats bicubic: False; nc-noise-white LPIPS 0.7437, beats bicubic: False; nc-shuffled LPIPS 0.7953, beats bicubic: False. Held-out registry SHA-256 e3743f08605f2f11….

Receipts fetched by URL at build time: measure-20260902T172415Z-77645bc8ec58.json · measure-20260902T173233Z-fdb97b77c83f.json · measure-20260902T184007Z-f7a0cf643988.json · measure-20260902T192013Z-d22b86cf9397.json · measure-20260902T235003Z-77d7f458c8c5.json · measure-20260902T235123Z-77d7f458c8c5.json · measure-20260903T004249Z-77d7f458c8c5.json · measure-20260903T023933Z-77d7f458c8c5.json · measure-20260903T024052Z-77d7f458c8c5.json · measure-20260903T024113Z-77d7f458c8c5.json · measure-20260903T065949Z-79833aa5436d.json · measure-20260903T070150Z-79833aa5436d.json · measure-20260903T071659Z-24e2f24f7d8e.json · measure-20260903T071859Z-24e2f24f7d8e.json · measure-20260903T071934Z-24e2f24f7d8e.json · negative-control-20260902T172258Z-77645bc8ec58.json · negative-control-20260902T173148Z-fdb97b77c83f.json · negative-control-20260902T235115Z-77d7f458c8c5.json · negative-control-20260902T235129Z-77d7f458c8c5.json · negative-control-20260903T004301Z-77d7f458c8c5.json · negative-control-20260903T024045Z-77d7f458c8c5.json · negative-control-20260903T024059Z-77d7f458c8c5.json · negative-control-20260903T024125Z-77d7f458c8c5.json · negative-control-20260903T070138Z-79833aa5436d.json · negative-control-20260903T070200Z-79833aa5436d.json · negative-control-20260903T071847Z-24e2f24f7d8e.json · negative-control-20260903T071909Z-24e2f24f7d8e.json · negative-control-20260903T071951Z-24e2f24f7d8e.json

Any-coordinate registries, frozen before scoring and excluded from training

Kill criterion (2026-09-30, or earlier if the metric moves)

Keep only if all three hold on input=real-s2:

• LPIPS on the Travis strip at most 0.85 × the best of bicubic, classical and sr-small.
• FID on the Travis strip below every baseline's.
• Eval's blind swipe on the Austin block prefers the model over classical at two to one or better. Only eval's verdict gates; the rows diagnose. A model given noise in place of conditioning must not beat bicubic; if it does, the metric is broken and no row counts until it is fixed.

Ledger receipts: setup PR #1, merged · grind PR #2 · scoreboard row, 18:44 UTC · criterion decision, 19:05 UTC · anycoord.v2 frozen, 20:3x UTC · first generative row, 23:52 UTC · the control failure is the metric, 00:06 UTC · lead's re-derivation and decisions, 00:3x UTC · PRs #5, #6, #7 merged: Overpass, 512 px control, the pin survives the pruner · final row at target, re-derived from six receipts, 2026-09-03 02:47 UTC · Travis measure receipt · Travis control receipt · EVAL-11's answer, 2026-09-03 01:37 UTC

Full hypothesis

A conditional generative pass over imperfect Earth sources emits the layers the tile compiler wants (ortho, materials, heights, tree instances, facade classes) with `authored` provenance and a confidence, trained without labels by degrading places that have good data (RESEARCH.md §4). v0 tests the load-bearing half of that claim: that a generative model conditioned on coarse public sources produces an ortho that is clearly closer to reality than interpolation plus classical processing of the same inputs.

Full ledger status

open — the row at target is in, the lab's claim and a diagnostic until eval judges it (EVAL-11): sdxl-control-lora (rank-64 LoRA + a 98 M control network, 29-channel conditioning) trained to 14,404 GPU s on the frozen Travis strip, one pinned checkpoint step-00002642.pt, clean tree at 77d7f458, six receipts of 2026-09-03 02:39 to 02:41 UTC fetched through the public host and equal by SHA-256 to the rig's. Travis, input=real-s2, n=1018: LPIPS 0.5263 / FID 61.87 / KID 0.0164 (from 0.5331 / 73.64 / 0.0235 at 7,061 GPU s on 2026-09-02); the bar (LPIPS ≤ 0.5343 and FID < 233.66) holds, LPIPS by 0.0080; VAE ceiling 0.0590 / 9.82; baselines reproduced to the digit in the same receipt (worldcover-mean 0.6635 / 404.31, worldcover-colour 0.6713 / 363.40, bicubic 0.6854 / 416.79, lanczos-unsharp 0.6886 / 397.98; sr-small's 0.6286 / 233.66 stands from its earlier receipt); negative control passed, margin 0.054 (nc-shuffled 0.7394 the nearest to bicubic). anycoord.v2 (112 truth cells): 0.6146 against bicubic 0.7247 and worldcover-mean 0.7151, its own control passed by 0.019 (white noise 0.7437), so it counts as the lab's claim with its sentence on every worldwide row: on wild high-frequency content noise beats bicubic on about 46 % of cells against Travis's 12 % and LPIPS-VGG stops discriminating there (EVAL-11 999df8c2: a measured fact about the metric, never a gate; DISTS and the per-cell control pass rate go beside it); from 0.6133 at 7,061 GPU s the worldwide row did not move while every Travis number improved: the model learns Travis, not the world, so the next iteration changes what the model sees rather than training longer. anycoord.v1 (14 cells): 0.6096, its own control failed (white noise 0.7427 beats bicubic 0.7610), a diagnostic

labels.v1: 20 tiles over austin, 60 files, 1,709,929 bytes; freeze digest 2b683c32303641ad5ea165230392c262952d838c4e5cbda4b3072abc52160aef; evidence copy gpu-node:evidence/lab-satellite-vision/labels.v1.freeze.json.

The oracle's ceiling, measured on a sample of one tile labelled twice: pixel agreement 66.9 %, mean class IoU 29.8 % (LAB-2 comment 20:56 UTC). Any IoU against labels.v1 carries this ceiling in the same row; shrub has no polygons in the set.

Spend recorded in docs/spend.md: 0 USD.

Kill criterion

Kill on 2026-09-30 unless the metric moves first. Kill earlier if, by 2026-09-16, no v0 class beats its baseline IoU on Austin, or if eval's blind swipe of the compiled block with our layers does not beat the block with the current layers. A negative result is a result; the card's Result section gets it.

Ledger receipts: setup and pull PR #1, merged · pilot checkpoint and decisions, 18:20 UTC · labels.v1 frozen, 20:56 UTC

Full hypothesis

Segmentation and classification models over NAIP (0.3 to 0.6 m) and Sentinel-2 (10 m), helped by lidar where it exists, produce compiler-grade layers (land cover, roofs and roof material, tree crowns; later species and road surface) for the seventeen portal places faster and cheaper than generation. This is the proven path beside the dark horse (LAB-1, `lab-earth-compiler`); our classical layers are also LAB-1's kill baseline ("bicubic plus classical"), so they must exist before LAB-1 can be judged.

Full ledger status

open — setup merged; labels.v1 (Austin, 20 tiles, 1,084 land-cover polygons, 250 roofs, 827 tree points) frozen on R2 at labs/lab-satellite-vision/labels.v1/, digest 2b683c32…, verified by a fresh client; NOT committed to the lab's fixture until a blind second labeller re-labels 3 to 5 tiles and the per-class agreement is published beside it (one-tile sample: 66.9 % pixel agreement, mean class IoU 29.8 %; no shrub polygons); Santa Barbara unlabelled; labelling cost measured at 9 to 15 minutes per canvas block

Frozen ground truth (groundtruth/frozen.json at origin/main, f19ae04)

Frozen ground truth per place: R2 key under labs/lab-llm-authored/groundtruth/, SHA-256, the labeller's run id and date. Regenerated whole, never edited; a correction is a new version with a new digest. facade_material and furniture are unknown for every unit in v1: no cleared source sees facades or street furniture (assessor records carry no exterior wall; no street imagery is cleared)…

B0 tags-only runs and their shuffled controls, as named on the card

measure/rows.csv at origin/main: 112 rows.

Kill and graduation

The precision bar passes when at least one arm beats both baselines on at least four building fields on both places. A row whose control is failed or none is a diagnostic and is outside this table: it is never counted towards beating a baseline (docs/adr/0003). The blind-margin bar passes when eval's blind margin with the authored layers is above the margin without them. This is the complete decision table on 2026-09-30:

At the kill date, failure of either bar kills the lab. Before the kill date, continue while both bars remain attainable and have not both passed; graduate as soon as both pass, and kill early only when the remaining planned evidence cannot make a failed bar pass. Graduation is a port into earth-data as an enrichment pass (NORMALIZE stage three), schema and confidence intact.

Ledger receipts: https://linear.app/agentarmies/issue/LAB-3/existing-models-chained-together-with-llms-authoring-data-semantic#comment-a03adbb6

Full hypothesis

Chains of existing models (a frontier vision-language model over ortho, street imagery and tags; open detectors and captioners; a reasoning pass that reconciles sources) can author the semantic fields real data lacks (what a building is, what a street feels like, which props belong) as `authored` fields with a calibrated confidence, at a precision that beats the tags alone and a single zero-shot pass, and eval's blind swipe can tell that authoring from noise. Both the precision bar and the blind-margin bar defined in the decision table must pass; failure of either at the kill date falsifies the hypothesis.

Full ledger status

killed 2026-09-03

Austin, 9 cells: violations by class before and after iteration 2

Any-coordinate v1 (32 world cells drawn with a seeded uniform draw over land, lab4-measure-anycoord-overture.service): footprints on 8, roads on 8, water on 6; quarry column absent; fixture SHA-256 420fdc628405b361….

Negative control on real data (austin-negative-control-20260902T1753Z.log): every applicable class must fire

Receipts copied from the evidence store into this edition: lab-geometry-solves-austin-20260902T175400Z-summary.json · lab-geometry-solves-anycoord-20260902T201216Z-summary.json · lab-geometry-solves-austin-iter2-comparison-20260902T201016Z.json. The lab has no store token yet, so these copies are the public receipts.

Kill criterion

Kill on 2026-09-30, or earlier, if the total violation count on the seventeen places has not fallen week over week for two consecutive weeks against the raw baseline, or if eval's blind swipe does not prefer the solved Austin block over the quarry's by then. Kill also if the solver only lowers counts by raising terrain_edit_excess: that is flattening, not solving.

Ledger receipts: setup PR #1, merged · Austin rows, 17:59 UTC · iteration 2 and the any-coordinate row, PR #3 merged · rows and digests, 20:18 UTC · exact containment and the seventeen-place row, PR #4 merged as 1d83154 · seventeen-place row, 22:29 UTC · lead's reading, 22:4x UTC · iteration 3: terrain edits to zero, PR #5 merged as ae5c05f5 · scoreboard row, 23:33 UTC · lead's re-derivation, 23:3x UTC

Full hypothesis

A constraint solver over footprints, heights, roads, terrain and water, run once per S2 level-16 block (a cell and its 3×3 halo) in one local tangent frame, produces a single consistent plan in which buildings meet the ground, roads meet each other and buildings, and water is level; the compiler projects that plan to render and sim products; and every failure the solver cannot resolve is reported as a named, located violation rather than shipped silently. Corollary under test: solving bodies once by feature id removes the cross-cell steps that per-cell solving produces.

Full ledger status

open — first result on the card's metric, diagnostic: the seventeen places (153 level-16 cells, one recipe: OSM via Overpass, pinned Overture 2026-08-19.0 footprints, Copernicus GLO-30) 3300 raw → 2169 after v0 on 2026-09-02 (iteration 3, PR #5 merged as ae5c05f5): terrain_edit_excess 0, building_floats 568, building_buried 362, road_grade_kink 343, road_grade_excess 177, road_hits_building 664 and footprint_overlap 19 untouched (approach C's), no place worse than its previous v0 row and nine better; the 310 terrain-edit violations of the 22:29 UTC row (2478) were float32 rounding at the solver's own 3.00 m clamp, counted by the checker's strict bound and removed by a 1 cm margin in the solver with no tolerance, metric or checker change, so the earlier reading of them as approach A's flattening limit is withdrawn and the kill clause is not near; Austin lidar home row 351 → 116 → 90 and Austin under the uniform recipe 204 → 99 (separate series); any-coordinate row (32 frozen world cells) 6 raw, 2 v0, terrain edits 0; cell assignment is exact S2 containment since 1d83154 (tile-compiler's CMP-7 ask); negative control passed on real data (16 of 17 places, kilimanjaro not applicable)

Toolchain proofs on gpu-node (docs/arm-toolchains.md)

Arms and kill criteria (AST-2 §3)

Every arm produces every item; every output goes through the same validate and the same judge (EVAL-5). Arms interleave in judged windows so host load never confounds a result.

Ledger receipts: setup PR #2, merged · proof table · state and rail date, 17:52 UTC · legal rows, orchestrator PR 6 · rules and asks, 22:0x UTC

Full ledger status

waiting on rails — setup merged; toolchain proofs on gpu-node: Blender Cycles on the GPU (1.9 s, 1.6 GiB), Hunyuan3D 2.1 image-to-3D fits (140 s, 8.3 GiB, shape only), TRELLIS.2 blocked on a gated DINOv3 download; legal rows logged under Joel's acquire-and-log rule (Hunyuan3D permission-pending: its licence excludes the EU and the node is in Helsinki, question on INB-8; A3's ten free Fab items not yet acquired); Tripo and Meshy callable with a batch estimate; judging waits on EVAL-5 (first blind pair 2026-09-09 at the earliest)

Ledger row

baseline: B3, the quarry's facade band-line and material-factory pipeline ported with a MIGRATION row

budget: three weeks; no separate spend line on the card

card: LAB-7

hypothesis: Twenty Austin buildings across four classes through four arms (grammar, image-conditioned diffusion, the quarry pipeline ported, generative 3D on the massing) under pilot A's judge

kill date: three weeks from pilot A's judge being proven

metric: Pilot A's metrics and constraints, per class, with one held-out building per class

realised gain: not yet

receipt: pilot-A rail receipt

slug: lab-forge-facade-bakeoff

status: waiting on rails behind pilot A