# Fable 5.1 Rebuilt Union Square and Left Its Misses in Git **Summary:** Fable 5.1 rebuilt Union Square with 453 OSM footprints, then published QA scores and known defects. I find the audit trail more convincing than the walkthrough. - Canonical: https://markhuang.ai/news/fable-51-union-square-misses-in-git - Language: en - Author: [Mark Huang](https://markhuang.ai/about) - Published: 2026-09-02 - Section: News - Tags: Claude Fable 5.1, AI Agents, Digital Twins, Three.js, Software Verification - Source: [PhiloLabs on GitHub](https://github.com/PhiloLabs/fable51-worlds) - License: https://creativecommons.org/licenses/by-nc/4.0/ --- ![A reconstructed Union Square shifts between finished city blocks, wireframe geometry, point clouds, and camera inspection cones](https://cdn.markhuang.ai/news/fable-51-union-square-misses-in-git/hero.webp) *Union Square looks finished at the center. Around it, wireframes and camera angles expose how the reconstruction is being checked.* The [fable51-worlds repository](https://github.com/PhiloLabs/fable51-worlds) presents a browser reconstruction of San Francisco's Union Square that it says was researched, modeled, and checked end to end by Claude Fable 5.1 agent swarms. Its first world loads 453 OpenStreetMap building footprints, places 220 simulated pedestrians, and runs as a plain Three.js application. I became more interested when I left the 59-second walkthrough and opened the audit files. The repo includes 34 camera-matched viewpoints, 147 comparison sheets, and nine reviewer-agent reports. Its own final report gives visual fidelity, street-level detail, and materials 6/10. Those admissions give me more to work with than a polished autonomous-agent demo. The scores are not independent proof that the reconstruction is accurate. What the repository offers is narrower and more credible: a large result with its inputs, checks, and unresolved defects available for inspection. ## The evidence is bigger than the walkthrough The project covers roughly 800 by 720 meters around Union Square. Its README lists 75 authored facade specifications, 129 identified storefronts, 109 vehicles, two explorable interiors, and 23 interactive objects. The runtime assembles the scene from data and generated assets rather than shipping proprietary 3D tiles. Those counts make an easy model-capability story. Anthropic positions [Fable 5.1](https://www.anthropic.com/claude/fable) for long-running coding projects, multi-day autonomous sessions, self-written tests, and visual checking. This repository puts that pitch into one messy job: research a real place, turn mixed sources into code and assets, build an interactive world, then compare the result with photographs. The QA machinery sits beside the output. Playwright captures fixed views. Another script places renders next to reference photographs and produces 50 percent overlays. The reports keep the remaining gaps. A reader can see why the team gave an attractive scene only 6/10 in several categories. ## Reviewer agents are still inside the experiment The README calls the nine reviewer agents independent and assigns roles such as architect, geographer, technical artist, and interaction reviewer. Separate roles can catch different defects. The reports say those reviews found inward-wound facade walls, a transit-station outline extruded into a 28-meter block, mirrored street markings, and reference cameras placed inside buildings or roadways. The project says those problems were fixed and rechecked with screenshots. I still put a boundary around the word "independent." The repository describes reviewer agents within the same project workflow; it does not present the scores as a third-party audit. Role prompts can diversify attention, but they do not automatically diversify training assumptions or prove that the scoring rubric matches how a San Francisco architect, accessibility specialist, or daily pedestrian would judge the place. In my earlier piece on [multi-model ensemble intelligence](https://markhuang.ai/blog/cross-family-multi-ai-science-of-ensemble-intelligence), I argued that several agents can share the same blind spots even when they appear to debate one another. Fable51-worlds partly compensates by comparing renders with external photographs and geospatial data. Outside review still has a different job. ## Open data provides coordinates, not a complete city The input choices are sensible and inspectable. The project derives geometry from OpenStreetMap and elevation from USGS 3DEP, then uses public records and free-licensed photographs for storefronts and visual references. [OpenStreetMap's license page](https://www.openstreetmap.org/copyright) explains that its community data is available under the ODbL with attribution and share-alike conditions. The repo records photograph provenance without redistributing the reference images. Those sources can anchor a street grid and building footprint. They cannot settle every facade, shop tenant, interior, or street behavior. The project's own [discrepancy ledger](https://github.com/PhiloLabs/fable51-worlds/blob/main/union-square-sf/qa/discrepancies.md) says some building details are approximated and procedural textures repeat at close range. Pedestrians do not enter stores, and buses do not stop at shelters. Several storefronts remain unresolved and use blank fascias instead of invented identities. I like that last choice. A neutral blank is less cinematic than a plausible fake, but it keeps missing evidence separate from generated detail. A reconstruction of a real place needs that boundary. > **Info:** > > The [final QA report](https://github.com/PhiloLabs/fable51-worlds/blob/main/union-square-sf/FINAL_QA_REPORT.md) says its category scores are unadjusted reviewer-agent results, not scores raised to meet the project's target bar. They are useful as the project's own acceptance record, not as an external benchmark of Fable 5.1. ## The failure ledger is the part I would copy Most model showcases compress the work into a prompt and a finished screen. The output looks magical, and the mistakes become hard to price. This repo exposes the research, confidence levels, capture scripts, comparison sheets, reviewer reports, performance measurements, and known gaps. A developer can disagree with the result without first reverse-engineering the experiment. I would still want a human domain review before calling this a dependable digital twin. I would also want costs, elapsed agent time, interventions, failed branches, and the exact model and harness used for each reviewer role. Those omissions make it hard to compare this build with a skilled human team or another agent system. The project has already made one decision I would copy: it stores uncertainty with the artifact. If agent swarms are going to build days-long visual systems, the deliverable should say what was checked, what failed, what remains approximate, and which claims an outsider can reproduce. Fable 5.1's Union Square is ambitious. I trust its narrower claims more because the repo leaves the misses in Git.