Refactor documentation and diagrams for web/pipeline. (#272)

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> for assistance with diagram creation and documentation prose updates.
This commit is contained in:
Aaron Po
2026-09-02 01:58:40 -04:00
committed by GitHub
parent 1fe35a2ea6
commit 30317326ce
83 changed files with 4291 additions and 4497 deletions

View File

@@ -1,4 +1,13 @@
# Ethics, bias, and known issues
---
title: Pipeline ethics, bias, and known issues
last-updated: 2026-08-31
tags:
- pipeline
- ethics
- bias
- known-issues
- ai-generated
---
This document covers the ethical context of the Biergarten Pipeline's output,
the model's biases, and known issues including hallucinated brewing science and
@@ -47,7 +56,7 @@ as a source of truth for brewing techniques or cultural representation.**
**Natural language processing is a powerful tool for data analysis and
generation, but its output should be treated with scrutiny. Human language is
not simply data points to be analyzed — it carries cultural and human meaning
not simply data points to be analyzed: it carries cultural and human meaning
that artificial intelligence cannot capture.**
---
@@ -102,7 +111,7 @@ vendored verbatim, unmodified, from
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
release assets), released under **CC0** (public domain). That dataset's own
forename/surname lists are pulled from Wikipedia's "Lists of most common
surnames" and "List of most popular given names" as of the week of 2023-07-08 —
surnames" and "List of most popular given names" as of the week of 2023-07-08;
see that project's README for full provenance. Names are not LLM-generated; this
is curated fixture data per ROADMAP.md §2. Per-forename gender from the source
data is preserved through to the sampled `Name` (rather than discarded during
@@ -113,7 +122,7 @@ for surnames) rather than trimmed to `locations.json`'s current country list, so
it doesn't need re-sourcing if more countries are added later. `SampleName()` (a
free helper in `generate_users.cc`) returns no result for a country present in
neither file; of the countries in `locations.json`, that's currently `KE`, `SE`,
`SG`, `TH`, `VN`, and `ZA` — `GenerateUsers` skips cities in those countries the
`SG`, `TH`, `VN`, and `ZA`; `GenerateUsers` skips cities in those countries the
same way brewery generation skips cities whose enrichment lookup fails.
---
@@ -290,11 +299,20 @@ Each brewery and user address carries a longitude/latitude pair, sampled
uniformly at random within a 5 km disc centred on the city's curated
`latitude`/`longitude` from `locations.json`.
The resulting point is **plausible by construction** — it always falls within
the city's vicinity — but it is not looked up from any real address registry.
It does not correspond to a real address, business, or resident, and should
not be treated as accurate geocoding data. This is consistent with the rest of
the dataset: fixture data for a proof-of-concept, not a source of truth.
The coordinate pair itself is **plausible by construction**, always falling
within the city's vicinity, but does not correspond to a real business or
resident, and should not be treated as accurate geocoding data. This is
consistent with the rest of the dataset: fixture data for a proof-of-concept,
not a source of truth.
For brewery addresses specifically (not user addresses, which carry no street
address at all), `address_line1`/`postal_code` _are_ looked up from a real
address registry — `NominatimAddressService` reverse-geocodes the synthetic
coordinate pair against the live OpenStreetMap/Nominatim database, so the street
address is a genuine nearby address, just not the address of an actual brewery.
This lookup only happens for non-`--mocked` runs; `--mocked` runs use
`MockAddressService`'s fixed placeholder address instead, which is not looked up
from any real registry.
---