Compare commits

..

33 Commits

Author SHA1 Message Date
Aaron Po
3154c2feed Formatted src/web_client 2026-07-19 06:50:56 -04:00
Aaron Po
3df657dc54 Formatted src/services/xeger 2026-07-19 06:50:56 -04:00
Aaron Po
a238db1d37 Formatted src/services/sqlite/helpers 2026-07-19 06:50:56 -04:00
Aaron Po
5d48d6757a Formatted src/services/sqlite 2026-07-19 06:50:56 -04:00
Aaron Po
34a036d7b3 Formatted src/services/postal_code 2026-07-19 06:50:56 -04:00
Aaron Po
59053fcddd Formatted src/services/logging 2026-07-19 06:50:56 -04:00
Aaron Po
d4c7260127 Formatted src/services/enrichment/wikipedia 2026-07-19 06:50:56 -04:00
Aaron Po
be90522eae Formatted src/services/curated_data 2026-07-19 06:50:56 -04:00
Aaron Po
d055a634c2 Formatted src/services 2026-07-19 06:50:56 -04:00
Aaron Po
933b728135 Formatted src/data_generation/prompt_formatting 2026-07-19 06:50:56 -04:00
Aaron Po
ac98a2a22e Formatted src/data_generation/mock 2026-07-19 06:50:56 -04:00
Aaron Po
d723f53cba Formatted src/data_generation/llama 2026-07-19 06:50:56 -04:00
Aaron Po
1885d9859f Formatted src/biergarten_pipeline_orchestrator 2026-07-19 06:50:56 -04:00
Aaron Po
03467cbb65 Formatted src/application_options 2026-07-19 06:50:56 -04:00
Aaron Po
1f25d25dce Formatted src 2026-07-19 06:50:56 -04:00
Aaron Po
67bc5bfc2a Formatted includes/web_client 2026-07-19 06:50:56 -04:00
Aaron Po
eb32a55831 Formatted includes/services/xeger 2026-07-19 06:50:56 -04:00
Aaron Po
fad1611685 Formatted includes/services/prompting 2026-07-19 06:50:56 -04:00
Aaron Po
74f848d015 Formatted includes/services/postal_code 2026-07-19 06:50:56 -04:00
Aaron Po
86db1c2827 Formatted includes/services/logging 2026-07-19 06:50:56 -04:00
Aaron Po
9bcf4c6879 Formatted includes/services/enrichment 2026-07-19 06:50:56 -04:00
Aaron Po
d9b10df436 Formatted includes/services/datetime 2026-07-19 06:50:56 -04:00
Aaron Po
4f0f56d7d7 Formatted includes/services/database 2026-07-19 06:50:56 -04:00
Aaron Po
6c07edf845 Formatted includes/services/curated_data 2026-07-19 06:50:56 -04:00
Aaron Po
969125e3a5 Formatted includes/json_handling 2026-07-19 06:50:56 -04:00
Aaron Po
ffabd85b6d Formatted includes/data_model 2026-07-19 06:50:56 -04:00
Aaron Po
4fae5ead08 Formatted includes/data_generation/prompt_formatting 2026-07-19 06:50:56 -04:00
Aaron Po
efd0d4f2aa Formatted includes/data_generation 2026-07-19 06:50:56 -04:00
Aaron Po
522381357a Formatted includes/concurrency 2026-07-19 06:50:56 -04:00
Aaron Po
5f3706bb93 Formatted includes 2026-07-19 06:50:56 -04:00
Aaron Po
a52b14c49a update clang format 2026-07-19 06:50:56 -04:00
Aaron Po
4ebd28c06b format docs 2026-07-19 06:50:56 -04:00
Aaron Po
9593a07c13 Format curated data and prompts 2026-07-19 06:50:56 -04:00
78 changed files with 13388 additions and 32143 deletions

View File

@@ -102,21 +102,19 @@ vendored verbatim, unmodified, from
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
release assets), released under **CC0** (public domain). That dataset's own
forename/surname lists are pulled from Wikipedia's "Lists of most common
surnames" and "List of most popular given names" as of the week of
2023-07-08 — see that project's README for full provenance. Names are not
LLM-generated; this is curated fixture data per ROADMAP.md §2. Per-forename
gender from the source data is preserved through to the sampled `Name`
(rather than discarded during loading) so it's available for gender-aware
persona/bio generation later.
surnames" and "List of most popular given names" as of the week of 2023-07-08 —
see that project's README for full provenance. Names are not LLM-generated; this
is curated fixture data per ROADMAP.md §2. Per-forename gender from the source
data is preserved through to the sampled `Name` (rather than discarded during
loading) so it's available for gender-aware persona/bio generation later.
The full multinational dataset is kept as-is (106 countries for forenames,
75 for surnames) rather than trimmed to `locations.json`'s current country
list, so it doesn't need re-sourcing if more countries are added later.
`SampleName()` (a free helper in `generate_users.cc`) returns no result for
a country present in neither file; of the countries in `locations.json`,
that's currently `KE`, `SE`, `SG`, `TH`, `VN`, and `ZA` — `GenerateUsers`
skips cities in those countries the same way brewery generation skips
cities whose enrichment lookup fails.
The full multinational dataset is kept as-is (106 countries for forenames, 75
for surnames) rather than trimmed to `locations.json`'s current country list, so
it doesn't need re-sourcing if more countries are added later. `SampleName()` (a
free helper in `generate_users.cc`) returns no result for a country present in
neither file; of the countries in `locations.json`, that's currently `KE`, `SE`,
`SG`, `TH`, `VN`, and `ZA` — `GenerateUsers` skips cities in those countries the
same way brewery generation skips cities whose enrichment lookup fails.
---
@@ -273,8 +271,7 @@ For languages such as Welsh (Wales), Māori (Aotearoa/New Zealand), or Sicilian
but is semantically incoherent. This comes from limited training-data coverage
rather than prompt engineering.
Output sample:
[./french-cities.example](french-cities.example)
Output sample: [./french-cities.example](french-cities.example)
#### Proposed Mitigations
@@ -298,9 +295,9 @@ curated real-world examples.
The generated code is **format-conformant by construction** — it always
satisfies the regex it was generated from — but it is not looked up from any
real postal registry. It does not correspond to a real address, business, or
resident, and should not be treated as accurate geocoding or delivery data.
This is consistent with the rest of the dataset: fixture data for a
proof-of-concept, not a source of truth.
resident, and should not be treated as accurate geocoding or delivery data. This
is consistent with the rest of the dataset: fixture data for a proof-of-concept,
not a source of truth.
---

View File

@@ -1,9 +1,9 @@
# Biergarten Pipeline
A C++20 command-line pipeline that samples city records from local JSON,
enriches each with Wikipedia context, and generates bilingual brewery names
and descriptions plus locale-grounded user profiles via a local GGUF model or
a deterministic mock.
enriches each with Wikipedia context, and generates bilingual brewery names and
descriptions plus locale-grounded user profiles via a local GGUF model or a
deterministic mock.
> **This pipeline produces AI-generated data.** It is not a source of truth for
> brewing techniques, cultural representation, or local-language accuracy. See
@@ -41,7 +41,7 @@ produces seed records the app imports at startup or during a dedicated seed
step.
| Planned app area | Pipeline contribution |
| -------------------------------- | ------------------------------------------------------------------ |
| -------------------------------- | ------------------------------------------------------------------------------------------------- |
| Brewery discovery and management | Sampled city records, localized names, long-form descriptions |
| Beer reviews and ratings | Stable brewery fixtures with enough context to anchor review pages |
| Social follow relationships | Repeatable brewery entities for feeds, follows, and saved lists |
@@ -65,10 +65,9 @@ cmake --build build
CMake automatically detects whether a compatible llama.cpp installation is
present on the system (`libllama`, `libggml`, `libggml-base`, and `llama.h`
visible on the default search paths). If found, it links against those
libraries and skips the FetchContent build. If not found, it fetches and builds
llama.cpp from source at tag `b9012`. No additional flags are required in
either case.
visible on the default search paths). If found, it links against those libraries
and skips the FetchContent build. If not found, it fetches and builds llama.cpp
from source at tag `b9012`. No additional flags are required in either case.
Metal is enabled automatically on Apple Silicon. CUDA or HIP/ROCm is detected
automatically on Linux when the relevant toolkit is present.
@@ -138,17 +137,18 @@ NVIDIA GPU.
The container uses a two-stage build. The builder stage installs CMake/Ninja,
clones the matching llama.cpp release tag for its headers only (installed into
`/usr/local/include`), and copies prebuilt shared libraries (`libllama`,
`libggml`, and CUDA/CPU backend plugins) from `ghcr.io/ggml-org/llama.cpp:full-cuda`
into `/usr/local/lib`. With both headers and libraries present, CMake's
system-library detection (see [Build](#build) above) finds them and skips the
FetchContent source build, keeping image build times short.
`libggml`, and CUDA/CPU backend plugins) from
`ghcr.io/ggml-org/llama.cpp:full-cuda` into `/usr/local/lib`. With both headers
and libraries present, CMake's system-library detection (see [Build](#build)
above) finds them and skips the FetchContent source build, keeping image build
times short.
The runtime stage copies the compiled binary, the same prebuilt shared
libraries, and config/prompt assets into a slim CUDA runtime image. It sets
`LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH` so the dynamic linker
resolves `libllama`/`libggml` at startup, and also co-locates
`libggml-cuda.so` and the CPU backend plugins next to the binary for
`ggml_backend_load_all()`'s `dlopen` scan.
`LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH` so the dynamic linker resolves
`libllama`/`libggml` at startup, and also co-locates `libggml-cuda.so` and the
CPU backend plugins next to the binary for `ggml_backend_load_all()`'s `dlopen`
scan.
### Build the image
@@ -177,8 +177,8 @@ output to confirm the fast path was taken.
The container always runs the model-backed path; there is no `--mocked`
container mode (use a native build for that — see [Quick Start](#quick-start)).
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if
it is not already present at the configured path.
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if it
is not already present at the configured path.
```bash
docker run --rm \
@@ -189,14 +189,14 @@ docker run --rm \
biergarten-pipeline:latest
```
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to
`./models/` on first run if it isn't already there. To use a pre-downloaded
model, place it at that path first — see [Model](#model) above.
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to `./models/` on
first run if it isn't already there. To use a pre-downloaded model, place it at
that path first — see [Model](#model) above.
#### Environment variables
| Variable | Purpose |
| ------------------------- | ----------------------------------------------------------------- |
| ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `BIERGARTEN_MODEL_PATH` | GGUF model path. Default: `/workspace/models/google_gemma-4-E4B-it-Q6_K.gguf`. |
| `BIERGARTEN_OUTPUT_DIR` | SQLite output directory. Default: `/workspace/output`. |
| `BIERGARTEN_LOG_PATH` | Log file path. Default: `/workspace/logs/pipeline.log`. |
@@ -221,7 +221,7 @@ environment variables listed above to match your run.
### Pipeline Stages
| Stage | Implementation |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Load | `ICuratedDataService` (`CuratedJsonDataService`) reads `locations.json`, `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json` (paths supplied via a `CuratedDataFilePaths` DTO at construction) into typed records, caching each after its first load. `--mocked` runs use `MockCuratedDataService`'s fixed in-memory dataset instead. |
| Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). |
| Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. |
@@ -240,33 +240,32 @@ skipped and the pipeline continues. `GenerateUsers()` runs before
- `CuratedJsonDataService` — implements `ICuratedDataService`; takes a
`CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in
its constructor, then parses and validates curated location, persona, and
forename/surname JSON, memoizing each result after its first load on a
given instance. `MockCuratedDataService` is the in-memory substitute (4
fixed locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in
forename/surname JSON, memoizing each result after its first load on a given
instance. `MockCuratedDataService` is the in-memory substitute (4 fixed
locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in
`--mocked` runs.
- `WikipediaEnrichmentService` — queries Wikipedia extracts, caches results,
returns empty context on failure. `MockEnrichmentService` is the no-op
substitute used in `--mocked` runs.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for
both `GenerateBrewery` and `GenerateUser`, retries malformed responses up
to three times with corrective feedback in the retry prompt. The token
budget is fixed across attempts; it is not raised automatically on
truncation.
- `MockGenerator` — stable hash-based output so the same city/persona/name
input always produces the same brewery or user.
- `SqliteExportService` — creates a dated SQLite file per run and persists
each successful user and brewery into normalized tables.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for both
`GenerateBrewery` and `GenerateUser`, retries malformed responses up to three
times with corrective feedback in the retry prompt. The token budget is fixed
across attempts; it is not raised automatically on truncation.
- `MockGenerator` — stable hash-based output so the same city/persona/name input
always produces the same brewery or user.
- `SqliteExportService` — creates a dated SQLite file per run and persists each
successful user and brewery into normalized tables.
- Brewery payloads include English and local-language name and description
fields. User payloads carry a sampled first/last name and gender, an
LLM-generated username/bio/activity weight, and a programmatically
generated (not LLM-authored) unique email and date of birth.
LLM-generated username/bio/activity weight, and a programmatically generated
(not LLM-authored) unique email and date of birth.
### Runtime Behaviour
`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a
generic "brewing" extract and a "beer in `{country}`" extract. It does not
currently query a city- or region-specific page. Each query string is cached
after its first successful (or empty) lookup.
`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a generic
"brewing" extract and a "beer in `{country}`" extract. It does not currently
query a city- or region-specific page. Each query string is cached after its
first successful (or empty) lookup.
`GetLocationContext()` returns an empty string when the web client is
unavailable or when lookup/parsing fails.
@@ -281,15 +280,15 @@ runs.
`CuratedJsonDataService` memoizes each of `LoadLocations()`, `LoadPersonas()`,
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the
first time each is called, since `BiergartenPipelineOrchestrator` owns a
single `ICuratedDataService` instance for the whole run — later calls return
the cached result instead of re-parsing.
first time each is called, since `BiergartenPipelineOrchestrator` owns a single
`ICuratedDataService` instance for the whole run — later calls return the cached
result instead of re-parsing.
`GenerateUsers()` samples a forename/surname pair per city via `SampleName()`,
keyed by the city's ISO 3166-1 code. Countries present in `locations.json`
but absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`,
`ZA`) are skipped, the same way a failed enrichment or generation call skips
a city — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
keyed by the city's ISO 3166-1 code. Countries present in `locations.json` but
absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`, `ZA`)
are skipped, the same way a failed enrichment or generation call skips a city —
see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
### Process Flow - Activity Diagram
@@ -303,10 +302,10 @@ a city — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
## Generated Output
Each successful run stores a `BreweryRecord` pair with the source location
and a `BreweryResult` payload, and a `UserRecord` pair with the source
location and a `UserResult` payload. The same generated records are also
written to a fresh SQLite export file named with the current UTC timestamp.
Each successful run stores a `BreweryRecord` pair with the source location and a
`BreweryResult` payload, and a `UserRecord` pair with the source location and a
`UserResult` payload. The same generated records are also written to a fresh
SQLite export file named with the current UTC timestamp.
| Field | Meaning |
| ------------------- | ------------------------------------------ |
@@ -316,7 +315,7 @@ written to a fresh SQLite export file named with the current UTC timestamp.
| `description_local` | Brewery description in the local language. |
| Field | Meaning |
| ----------------- | ----------------------------------------------------------------- |
| ----------------- | -------------------------------------------------------------------------------------- |
| `first_name` | Sampled forename, copied from the curated name data (not LLM-invented). |
| `last_name` | Sampled surname, copied from the curated name data (not LLM-invented). |
| `gender` | Gender associated with the sampled forename in the source dataset. |
@@ -358,11 +357,9 @@ The build fetches Boost.DI, spdlog, and SQLite via CMake. llama.cpp is fetched
only when a system installation is not detected. Metal is enabled on Apple
Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
> **Code Style:** Modern C++20 throughout — RAII for ownership,
> `std::unique_ptr` for injected dependencies, `std::optional` for parse
> outcomes, `std::span` for read-only views over generated city data, structured
> bindings in pipeline loops. Formatting follows the Google C++ Style Guide via
> `.clang-format` with a narrow column limit and two-space indentation.
> **Code Style:** Code style generally follows the Google Style Guide and
> features modern C++20 throughout. Formatting follows the Google C++ Style
> Guide, with the exception of indentation width (3 spaces vs. Google's 2).
---
@@ -404,24 +401,23 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
## Fixture Strategy
- `--mocked` for stable fixtures, repeatable screenshots, and Storybook runs.
`MockCuratedDataService` swaps in for `CuratedJsonDataService`, so no
fixture files need to be present on disk.
`MockCuratedDataService` swaps in for `CuratedJsonDataService`, so no fixture
files need to be present on disk.
- `--model` when geographically grounded content matters for demos.
- Keep `locations.json` structured enough to support discovery and future
filtering.
- `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` are curated/vendored fixture data, not
LLM-generated — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset
section for provenance.
- Treat SQLite output as seed material for the app's brewery and user
domains, not production data.
- `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json`
are curated/vendored fixture data, not LLM-generated — see
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for provenance.
- Treat SQLite output as seed material for the app's brewery and user domains,
not production data.
---
## Repo Layout
| Path | Purpose |
| -------------------------------------- | -------------------------------------------------- |
| -------------------------------------------- | ----------------------------------------------------------- |
| `tooling/pipeline/includes/` | Public headers and shared models. |
| `tooling/pipeline/src/` | Implementation files. |
| `tooling/pipeline/locations.json` | Curated city input copied into the build tree. |
@@ -456,12 +452,11 @@ Paths below are relative to `tooling/pipeline/`.
## Next Steps
The pipeline currently produces city-aware brewery and user records and
dated SQLite exports. The next passes add additional fixture types so the
app can exercise the full brewery and social domains without live data. For
the detailed engineering breakdown of what's needed to reach the
architecture in [`diagrams/planned/`](./diagrams/planned/), see
[ROADMAP.md](./ROADMAP.md).
The pipeline currently produces city-aware brewery and user records and dated
SQLite exports. The next passes add additional fixture types so the app can
exercise the full brewery and social domains without live data. For the detailed
engineering breakdown of what's needed to reach the architecture in
[`diagrams/planned/`](./diagrams/planned/), see [ROADMAP.md](./ROADMAP.md).
### Testing — Very High Priority

View File

@@ -3,13 +3,12 @@
This is the engineering breakdown for closing the gap between the **current**
pipeline and the **planned** architecture documented in
[`diagrams/planned/class.puml`](./diagrams/planned/class.puml) and
[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing
in `diagrams/planned/` is implemented yet — this file tracks what it would
take to get there. For the current implementation, see
[README.md](./README.md).
[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing in
`diagrams/planned/` is implemented yet — this file tracks what it would take to
get there. For the current implementation, see [README.md](./README.md).
Items are grouped by layer and roughly ordered by dependency: later groups
build on types and services introduced earlier.
Items are grouped by layer and roughly ordered by dependency: later groups build
on types and services introduced earlier.
The only concurrency in the current pipeline is the log dispatcher thread
(`main.cc` spawns it to drain a `BoundedChannel<LogEntry>` into spdlog for the
@@ -25,31 +24,30 @@ architecture needs.
`tooling/pipeline/includes/data_model/generated_models.h` and `models.h`.
- [ ] Add `Completeness` enum (`Full`, `Partial`, `Absent`) and a
`LocationContext` struct (`text`, `completeness`, `char_count`).
Replace `EnrichedCity::region_context` (currently a plain
`std::string`) with `context : LocationContext`.
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`,
`min_ibu`, `max_ibu`).
`LocationContext` struct (`text`, `completeness`, `char_count`). Replace
`EnrichedCity::region_context` (currently a plain `std::string`) with
`context : LocationContext`.
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`, `min_ibu`,
`max_ibu`).
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads.
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
`context_provided`, `generated_with`).
- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to
`UserResult` (currently just `username`, `bio`). `first_name`/
`last_name`/`gender` are copied from the sampled `Name` (see below),
not LLM-invented.
`last_name`/`gender` are copied from the sampled `Name` (see below), not
LLM-invented.
- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result
handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`,
`gender`). Landed as a flatter shape than originally planned here: no
`NamesByCountry` wrapper class. `curated_data_service.h` instead
declares `ForenameList = std::vector<ForenameEntry>` and
`SurnameList = std::vector<std::string>`, and
`ICuratedDataService` exposes two maps keyed by ISO 3166-1 code —
`NamesByCountry` wrapper class. `curated_data_service.h` instead declares
`ForenameList = std::vector<ForenameEntry>` and
`SurnameList = std::vector<std::string>`, and `ICuratedDataService`
exposes two maps keyed by ISO 3166-1 code —
`ForenamesByCountryMap = unordered_map<string, ForenameList>` and
`SurnamesByCountryMap = unordered_map<string, SurnameList>` —
directly (see §2). Sampling is
a free function, `SampleName(forenames_by_country,
surnames_by_country, iso3166_1, rng)`, in
`generate_users.cc`'s anonymous namespace, not a method on a class.
`SurnamesByCountryMap = unordered_map<string, SurnameList>` — directly
(see §2). Sampling is a free function,
`SampleName(forenames_by_country, surnames_by_country, iso3166_1, rng)`,
in `generate_users.cc`'s anonymous namespace, not a method on a class.
Pairing a forename with a surname happens at sample time so gender
(present per-forename in the source data) is never discarded the way a
pre-flattened `{first_name, last_name}` list would lose it; see §2.
@@ -58,14 +56,14 @@ architecture needs.
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
`GeneratedFollow` aggregate structs.
- [x] Add `UserRecord` (`location`, `user : UserResult`, `email`,
`date_of_birth`) as the current stand-in for the planned
`GeneratedUser` — `email` and `date_of_birth` are programmatically
generated by the orchestrator, never LLM-authored, so a downstream
auth-account seeding consumer can register real accounts from the
pipeline's SQLite export. No `password`, `user_id`, or `metadata`
field yet, matching `BreweryRecord`'s equally pre-`metadata` shape.
Already wired into `IExportService`/`SqliteExportService`: a `users`
table exists and `GenerateUsers()` exports each successful record via
`date_of_birth`) as the current stand-in for the planned `GeneratedUser` —
`email` and `date_of_birth` are programmatically generated by the
orchestrator, never LLM-authored, so a downstream auth-account seeding
consumer can register real accounts from the pipeline's SQLite export. No
`password`, `user_id`, or `metadata` field yet, matching `BreweryRecord`'s
equally pre-`metadata` shape. Already wired into
`IExportService`/`SqliteExportService`: a `users` table exists and
`GenerateUsers()` exports each successful record via
`ProcessRecord(const UserRecord&)` (see §5) in addition to logging and
holding `generated_users_` in memory. Still missing vs. the planned
`GeneratedUser`: `user_id`, `password`, and `metadata`.
@@ -75,66 +73,62 @@ architecture needs.
`tooling/pipeline/includes/services/curated_data/`, fixture files.
- [x] Extract an interface; have `CuratedJsonDataService` implement it.
Landed as `ICuratedDataService`
(`services/curated_data/curated_data_service.h`), not `DataPreloader`
— `LoadLocations()`, `LoadPersonas()`, `LoadForenamesByCountry()`, and
`LoadSurnamesByCountry()` are all virtual methods returning `const&`
and take no arguments. `CuratedJsonDataService`
(`services/curated_data/curated_json_data_service.h`, originally
landed as `JsonLoader` under `json_handling/` before being renamed and
moved) takes a `CuratedDataFilePaths` DTO (`locations_path`,
`personas_path`, `forenames_path`, `surnames_path`) in its
constructor rather than a path per `Load*()` call, and memoizes each
result in a private `cache` struct (`locations`, `personas`,
`forenames_by_country`, `surnames_by_country`), so a second call on
the same instance returns the cached result instead of re-parsing —
safe because `CuratedJsonDataService` outlives every call site (owned
by `BiergartenPipelineOrchestrator` via
- [x] Extract an interface; have `CuratedJsonDataService` implement it. Landed
as `ICuratedDataService` (`services/curated_data/curated_data_service.h`),
not `DataPreloader` — `LoadLocations()`, `LoadPersonas()`,
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` are all virtual
methods returning `const&` and take no arguments. `CuratedJsonDataService`
(`services/curated_data/curated_json_data_service.h`, originally landed as
`JsonLoader` under `json_handling/` before being renamed and moved) takes
a `CuratedDataFilePaths` DTO (`locations_path`, `personas_path`,
`forenames_path`, `surnames_path`) in its constructor rather than a path
per `Load*()` call, and memoizes each result in a private `cache` struct
(`locations`, `personas`, `forenames_by_country`, `surnames_by_country`),
so a second call on the same instance returns the cached result instead of
re-parsing — safe because `CuratedJsonDataService` outlives every call
site (owned by `BiergartenPipelineOrchestrator` via
`unique_ptr<ICuratedDataService>` for the whole run). `main.cc`'s DI
injector also gained a `MockCuratedDataService` (fixed in-memory
data: 4 locations across `US`/`DE`/`FR`/`BE`, 3 personas, and
matching forename/surname sets for those four countries), bound in
place of `CuratedJsonDataService` under `--mocked`, mirroring the
existing `MockEnrichmentService`/`MockGenerator` pattern.
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo
and is already copied into the Docker image
(`runpod/Dockerfile`), but no loader reads it yet, and the native
CMake build doesn't copy it into `build/` at all — `CMakeLists.txt`'s
"Runtime Assets" step only copies `locations.json` and `prompts/`.
injector also gained a `MockCuratedDataService` (fixed in-memory data: 4
locations across `US`/`DE`/`FR`/`BE`, 3 personas, and matching
forename/surname sets for those four countries), bound in place of
`CuratedJsonDataService` under `--mocked`, mirroring the existing
`MockEnrichmentService`/`MockGenerator` pattern.
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo and
is already copied into the Docker image (`runpod/Dockerfile`), but no
loader reads it yet, and the native CMake build doesn't copy it into
`build/` at all — `CMakeLists.txt`'s "Runtime Assets" step only copies
`locations.json` and `prompts/`.
- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
- [x] Add name-by-country loading, parsing
`tooling/pipeline/forenames-by-country.json` and
`surnames-by-country.json`. Landed as two separate methods —
`LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each
returning a flat `ForenamesByCountryMap` / `SurnamesByCountryMap`
(aliases for `unordered_map<string, ForenameList>` /
`LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each returning
a flat `ForenamesByCountryMap` / `SurnamesByCountryMap` (aliases for
`unordered_map<string, ForenameList>` /
`unordered_map<string, SurnameList>`) keyed by ISO 3166-1 code, rather
than one combined `LoadNamesByCountry() : NamesByCountry` call. Both
files are vendored verbatim (unmodified,
full multinational coverage) from
than one combined `LoadNamesByCountry() : NamesByCountry` call. Both files
are vendored verbatim (unmodified, full multinational coverage) from
`sigpwned/popular-names-by-country-dataset` (CC0) — see
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for
provenance. Deliberately not pre-paired or filtered down to just the
countries in `locations.json`: keeping the source shape (including
per-forename gender) intact means the loader can support more
countries later, or gender-aware persona/bio generation, without
re-sourcing data.
per-forename gender) intact means the loader can support more countries
later, or gender-aware persona/bio generation, without re-sourcing data.
## 3. Policy / Strategy Layer
Entirely new — no `includes/policy/` (or equivalent) directory exists today.
- [ ] `ContextStrategy` interface + `BreweryContextStrategy` /
`BeerContextStrategy`. Today `WikipediaEnrichmentService::GetLocationContext`
hardcodes a generic `"brewing"` query and a `"beer in {country}"`
query directly — no per-phase strategy selection.
- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing
the inline `std::ranges::sample(...)` call in
`BeerContextStrategy`. Today
`WikipediaEnrichmentService::GetLocationContext` hardcodes a generic
`"brewing"` query and a `"beer in {country}"` query directly — no
per-phase strategy selection.
- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing the
inline `std::ranges::sample(...)` call in
`BiergartenPipelineOrchestrator::QueryCitiesWithCountries()`.
- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to
pick styles per brewery from the `BeerStyle` palette (depends on §1
and §2).
- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to pick
styles per brewery from the `BeerStyle` palette (depends on §1 and §2).
- [ ] `CheckinDistributionStrategy` interface + `JCurveCheckinStrategy` /
`RandomCheckinStrategy` — this is the "Check-In System" item already
called out in README.md's Next Steps, made concrete.
@@ -150,88 +144,87 @@ Entirely new — no `includes/policy/` (or equivalent) directory exists today.
`GenerateUser`).
- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop
mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3
attempts with corrective feedback) and a new
`prompts/USER_GENERATION.md`. Changed the `DataGenerator::GenerateUser`
signature to
attempts with corrective feedback) and a new `prompts/USER_GENERATION.md`.
Changed the `DataGenerator::GenerateUser` signature to
`(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`,
matching what the activity diagram actually passes. `MockGenerator::GenerateUser`
updated to match the new signature too.
matching what the activity diagram actually passes.
`MockGenerator::GenerateUser` updated to match the new signature too.
- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` /
`GenerateRating`.
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
`LlamaGenerator::ValidateModelArchitecture()` so loading a GGUF that
doesn't match the configured chat template fails fast instead of
silently producing degraded output.
- [ ] Add prompt template files for beer/checkin/rating generation next to
the existing `prompts/BREWERY_GENERATION.md`.
doesn't match the configured chat template fails fast instead of silently
producing degraded output.
- [ ] Add prompt template files for beer/checkin/rating generation next to the
existing `prompts/BREWERY_GENERATION.md`.
## 5. Export Service
`tooling/pipeline/includes/services/database/`, `src/services/sqlite/`.
- [x] `IExportService::ProcessRecord(const UserRecord&)` and
`SqliteExportService`'s `users` table (see
`kCreateUsersTableSql` / `kInsertUserSql` in
`includes/services/database/sqlite_statement_helpers.h`) are
implemented — `GenerateUsers()` exports every successful user
alongside its resolved `location_id`.
`SqliteExportService`'s `users` table (see `kCreateUsersTableSql` /
`kInsertUserSql` in
`includes/services/database/sqlite_statement_helpers.h`) are implemented —
`GenerateUsers()` exports every successful user alongside its resolved
`location_id`.
- [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`,
`ProcessRating`, `ProcessFollow` (today `ProcessRecord(const
BreweryRecord&)` and `ProcessRecord(const UserRecord&)` exist).
`ProcessRating`, `ProcessFollow` (today
`ProcessRecord(const BreweryRecord&)` and
`ProcessRecord(const UserRecord&)` exist).
- [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to
`SqliteExportService::InitializeSchema()` (`locations`, `breweries`,
and `users` already exist) — see `kCreateLocationsTableSql` /
`SqliteExportService::InitializeSchema()` (`locations`, `breweries`, and
`users` already exist) — see `kCreateLocationsTableSql` /
`kCreateBreweriesTableSql` / `kCreateUsersTableSql` in
`includes/services/database/sqlite_statement_helpers.h` for the
existing pattern to follow.
- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and
move from one open transaction for the whole run to per-phase batched
commits (`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as
shown in the planned activity diagram.
`includes/services/database/sqlite_statement_helpers.h` for the existing
pattern to follow.
- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and move
from one open transaction for the whole run to per-phase batched commits
(`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as shown in the
planned activity diagram.
## 6. Concurrency & Orchestration
`tooling/pipeline/includes/concurrency/`,
`src/biergarten_pipeline_orchestrator/`.
- [ ] `BoundedChannel<T>` already exists and is production-tested — but
it's only wired to the single log channel today. Stand up the
per-phase producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the
planned activity diagram describes, each with a dedicated LLM worker
thread and SQLite worker thread.
- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous
pass over `generated_breweries_` into phased methods —
`RunUserPhase` → `RunBreweryAndBeerPhase` (with a `RunBeerPhase`
sub-step once `brewery_pool_` is populated) → `RunCheckinPhase` /
`RunFollowPhase` (these two can run in parallel) → `RunRatingPhase` —
backed by `user_pool_`, `brewery_pool_`, `beer_pool_`,
`checkin_pool_`, `follow_pool_`.
- [ ] `BoundedChannel<T>` already exists and is production-tested — but it's
only wired to the single log channel today. Stand up the per-phase
producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the planned activity
diagram describes, each with a dedicated LLM worker thread and SQLite
worker thread.
- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous pass
over `generated_breweries_` into phased methods — `RunUserPhase` →
`RunBreweryAndBeerPhase` (with a `RunBeerPhase` sub-step once
`brewery_pool_` is populated) → `RunCheckinPhase` / `RunFollowPhase`
(these two can run in parallel) → `RunRatingPhase` — backed by
`user_pool_`, `brewery_pool_`, `beer_pool_`, `checkin_pool_`,
`follow_pool_`.
- [ ] Honor the structural-concurrency requirement already called out in a
comment on `BiergartenPipelineOrchestrator::Run()` in
`biergarten_pipeline_orchestrator.h`: once real worker threads exist,
they must be structurally joined (e.g. via `std::jthread`) before
`Run()` returns, so no worker logs to a closed channel during
teardown.
`biergarten_pipeline_orchestrator.h`: once real worker threads exist, they
must be structurally joined (e.g. via `std::jthread`) before `Run()`
returns, so no worker logs to a closed channel during teardown.
## 7. Enrichment
- [ ] Decide whether to restore the city/region-specific Wikipedia query
that's currently commented out in
`src/services/enrichment/wikipedia/get_summary.cc`
(`GetLocationContext`). The `ContextStrategy` work in §3 is a natural
place to reintroduce it via `BreweryContextStrategy::QueriesFor()`.
- [ ] Decide whether to restore the city/region-specific Wikipedia query that's
currently commented out in
`src/services/enrichment/wikipedia/get_summary.cc` (`GetLocationContext`).
The `ContextStrategy` work in §3 is a natural place to reintroduce it via
`BreweryContextStrategy::QueriesFor()`.
- [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`,
`PreWarmLocationCache` in the planned activity diagram) instead of
fetching lazily per record, so the streaming phases never block on a
cold cache mid-run.
fetching lazily per record, so the streaming phases never block on a cold
cache mid-run.
## 8. Fixtures / Build
- [x] Author `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` (see §2).
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime
Assets" `configure_file` step in `CMakeLists.txt` so native builds and
the Docker image agree on what's available at runtime.
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime Assets"
`configure_file` step in `CMakeLists.txt` so native builds and the Docker
image agree on what's available at runtime.
---
@@ -240,13 +233,12 @@ Entirely new — no `includes/policy/` (or equivalent) directory exists today.
The planned activity diagram's own phase notes ("brewery*pool* is now fully
populated. Phase 1b may begin.", etc.) imply a dependency order. Roughly:
1. Domain models (§1) and data preloading (§2) — nothing else compiles
without these.
2. Export schema (§5) — so every later generation phase has somewhere to
land its output.
1. Domain models (§1) and data preloading (§2) — nothing else compiles without
these.
2. Export schema (§5) — so every later generation phase has somewhere to land
its output.
3. Policy/strategy layer (§3) and generator interface (§4).
4. Concurrency/orchestration rewrite (§6), which is the only piece that
actually wires §1–§5 into the phased, parallel pipeline shown in the
planned diagrams.
5. Enrichment cache pre-warming and the city/region query decision (§7) —
useful at any point, but most valuable once phases run concurrently.
4. Concurrency/orchestration rewrite (§6), which is the only piece that actually
wires §1–§5 into the phased, parallel pipeline shown in the planned diagrams.
5. Enrichment cache pre-warming and the city/region query decision (§7) — useful
at any point, but most valuable once phases run concurrently.

View File

@@ -1,5 +1,5 @@
---
BasedOnStyle: Google
ColumnLimit: 80
IndentWidth: 2
IndentWidth: 3
...

View File

@@ -1,12 +1,6 @@
Checks: >
-*,
bugprone-*,
google-*,
modernize-*,
readability-*,
cppcoreguidelines-*,
-modernize-use-trailing-return-type,
-google-runtime-references
-*, bugprone-*, google-*, modernize-*, readability-*, cppcoreguidelines-*,
-modernize-use-trailing-return-type, -google-runtime-references
CheckOptions:
# Enforce Google Naming Conventions with valid clang-tidy strings

File diff suppressed because it is too large Load Diff

View File

@@ -37,7 +37,8 @@ class LlamaGenerator final : public DataGenerator {
* @param prompt_directory Directory service for loading named prompt files.
*/
LlamaGenerator(const ApplicationOptions& options,
const std::string& model_path, std::shared_ptr<ILogger> logger,
const std::string& model_path,
std::shared_ptr<ILogger> logger,
std::unique_ptr<IPromptFormatter> prompt_formatter,
std::unique_ptr<IPromptDirectory> prompt_directory);
@@ -103,7 +104,8 @@ class LlamaGenerator final : public DataGenerator {
* @param grammar Optional GBNF grammar constraining generated output.
* @return Generated text.
*/
std::string Infer(const std::string& system_prompt, const std::string& prompt,
std::string Infer(const std::string& system_prompt,
const std::string& prompt,
int max_tokens = kDefaultMaxTokens,
std::string_view grammar = {});

View File

@@ -52,7 +52,8 @@ class MockGenerator final : public DataGenerator {
* @return Deterministic hash value.
*/
static size_t DeterministicHash(const City& location,
const UserPersona& persona, const Name& name);
const UserPersona& persona,
const Name& name);
// Hash stride constants for deterministic distribution across fixed-size
// arrays. These coprime strides spread hash values uniformly without

View File

@@ -11,7 +11,8 @@ class Gemma4JinjaPromptFormatter final : public IPromptFormatter {
Gemma4JinjaPromptFormatter() = default;
~Gemma4JinjaPromptFormatter() override = default;
[[nodiscard]] std::string Format(std::string_view system_prompt,
[[nodiscard]] std::string Format(
std::string_view system_prompt,
std::string_view user_prompt) const override;
};

View File

@@ -39,7 +39,8 @@ inline void PrettyPrint(std::ostream& outstream,
if (!obj.empty()) {
const auto* iter = obj.begin();
for (;;) {
outstream << *indent << boost::json::serialize(iter->key()) << " : ";
outstream << *indent << boost::json::serialize(iter->key())
<< " : ";
PrettyPrint(outstream, iter->value(), indent);
iter = std::next(iter);
if (iter == obj.end()) {

View File

@@ -15,8 +15,6 @@
*/
class MockEnrichmentService final : public IEnrichmentService {
public:
std::string GetLocationContext(const City& /*loc*/) override {
return {};
}
std::string GetLocationContext(const City& /*loc*/) override { return {}; }
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_MOCK_ENRICHMENT_H_

View File

@@ -67,7 +67,8 @@ class MockPostalCodeService final : public IPostalCodeService {
if (!spec.country_format_regex.empty()) {
has_pattern = true;
if (std::regex_match(candidate, std::regex(spec.country_format_regex))) {
if (std::regex_match(candidate,
std::regex(spec.country_format_regex))) {
return true;
}
}

View File

@@ -2,7 +2,12 @@
{
"name": "Hophead Explorer",
"description": "Chases the newest hop varieties and double-digit IBUs, tracking hazy releases and limited tap takeovers across town.",
"style_affinities": ["American IPA", "Double IPA", "New England IPA", "Black IPA"]
"style_affinities": [
"American IPA",
"Double IPA",
"New England IPA",
"Black IPA"
]
},
{
"name": "Malt & Tradition Loyalist",
@@ -12,7 +17,12 @@
{
"name": "Dark & Roasty Devotee",
"description": "Orders the darkest beer on the board year-round, drawn to roasted malt, coffee, and chocolate notes over carbonated lightness.",
"style_affinities": ["Imperial Stout", "Robust Porter", "Baltic Porter", "Oatmeal Stout"]
"style_affinities": [
"Imperial Stout",
"Robust Porter",
"Baltic Porter",
"Oatmeal Stout"
]
},
{
"name": "Sour & Wild Forager",
@@ -22,21 +32,41 @@
{
"name": "Session Sipper",
"description": "Values an easy-drinking pint that holds up over a long afternoon at the taproom more than raw intensity or novelty.",
"style_affinities": ["Session IPA", "Ordinary Bitter", "Cream Ale", "Blonde Ale"]
"style_affinities": [
"Session IPA",
"Ordinary Bitter",
"Cream Ale",
"Blonde Ale"
]
},
{
"name": "Belgian Abbey Pilgrim",
"description": "Collects abbey and trappist releases, drawn to the dried-fruit esters and warming strength of classic Belgian ales.",
"style_affinities": ["Belgian Tripel", "Belgian Dubbel", "Belgian Quadrupel", "Trappist Single"]
"style_affinities": [
"Belgian Tripel",
"Belgian Dubbel",
"Belgian Quadrupel",
"Trappist Single"
]
},
{
"name": "Wheat & Citrus Casual",
"description": "Reaches for something light, cloudy, and refreshing, usually with a citrus or coriander note, especially on a patio in summer.",
"style_affinities": ["Hefeweizen", "Witbier", "American Wheat Beer", "Berliner Weisse"]
"style_affinities": [
"Hefeweizen",
"Witbier",
"American Wheat Beer",
"Berliner Weisse"
]
},
{
"name": "Big & Boozy Connoisseur",
"description": "Sips slowly from a snifter, seeking out high-ABV releases meant for aging and savoring rather than quick refreshment.",
"style_affinities": ["English Barleywine", "Wee Heavy", "Doppelbock", "Old Ale"]
"style_affinities": [
"English Barleywine",
"Wee Heavy",
"Doppelbock",
"Old Ale"
]
}
]

View File

@@ -41,11 +41,11 @@ You will receive the inputs like this:
ABSOLUTELY NO MARKDOWN FORMATTING. Do NOT wrap your response in json or ```
blocks.
Do not add markdown, code fences, or postscript around the final JSON object.
Do not say "Here is the JSON" or "Enjoy!".
Do not add markdown, code fences, or postscript around the final JSON object. Do
not say "Here is the JSON" or "Enjoy!".
The JSON must contain exactly three keys ("username", "bio",
"activity_weight") in that order. Do not rename or add any other keys.
The JSON must contain exactly three keys ("username", "bio", "activity_weight")
in that order. Do not rename or add any other keys.
ESCAPE ALL QUOTES inside the bio field using \", or use single quotes (' ')
instead.
@@ -53,15 +53,15 @@ instead.
DO NOT use actual line breaks (\n) inside any string. Keep the bio as one
continuous string.
The bio must be between 25 and 60 words, written in first person, and must
read as something this specific person would write about themselves -- not a
generic profile blurb.
The bio must be between 25 and 60 words, written in first person, and must read
as something this specific person would write about themselves -- not a generic
profile blurb.
`activity_weight` is a number between 0 and 1 (inclusive) representing how
often this persona checks in at breweries relative to other users. Persona
archetypes implying frequent, habitual brewery visits should skew higher;
casual or occasional-visitor personas should skew lower. Do not default to
0.5 -- vary it meaningfully based on the persona description.
`activity_weight` is a number between 0 and 1 (inclusive) representing how often
this persona checks in at breweries relative to other users. Persona archetypes
implying frequent, habitual brewery visits should skew higher; casual or
occasional-visitor personas should skew lower. Do not default to 0.5 -- vary it
meaningfully based on the persona description.
Expected JSON format:
@@ -78,8 +78,8 @@ Expected JSON format:
### THE USERNAME:
Derive the username from the given first and/or last name (e.g. lowercase,
abbreviated, or combined with a short word or number tied to the persona's
style affinities). Do not invent an unrelated handle. Do not include spaces.
abbreviated, or combined with a short word or number tied to the persona's style
affinities). Do not invent an unrelated handle. Do not include spaces.
### THE BIO:
@@ -90,14 +90,14 @@ city or country naturally if it fits, but do not force it.
### VOICE & PERSPECTIVE:
Write in the first person ("I"/"my"). The tone should match the persona:
an enthusiastic explorer sounds different from a relaxed, easy-drinking
regular. Do not write in second or third person.
Write in the first person ("I"/"my"). The tone should match the persona: an
enthusiastic explorer sounds different from a relaxed, easy-drinking regular. Do
not write in second or third person.
### THE BLOCKLIST (FORBIDDEN CONCEPTS):
You absolutely cannot use the following words and phrases. Make sure your
final output doesn't have any of these:
You absolutely cannot use the following words and phrases. Make sure your final
output doesn't have any of these:
- "hidden gem"
- "passion"
@@ -108,14 +108,14 @@ final output doesn't have any of these:
#### FORBIDDEN WRITING PATTERNS
The following patterns are common AI writing pitfalls and must not appear in
the bio:
The following patterns are common AI writing pitfalls and must not appear in the
bio:
- Negative parallelism constructions: "It's not X, it's Y"
- Inflated significance phrases: "stands as a testament," "plays a vital
role," "deeply rooted"
- Superficial trailing analyses: sentences ending in -ing words that add
opinion without content
- Inflated significance phrases: "stands as a testament," "plays a vital role,"
"deeply rooted"
- Superficial trailing analyses: sentences ending in -ing words that add opinion
without content
- Promotional travel-copy tone: "breathtaking," "must-visit," "vibrant"
- Overused conjunctive transitions used as sentence openers: "Moreover,"
"Furthermore," "In addition"

View File

@@ -18,7 +18,8 @@ std::optional<ApplicationOptions> ParseArguments(
auto add_sampling_options = [&]() -> void {
const SamplingOptions sampling_defaults{};
opt("temperature",
prog_opts::value<float>()->default_value(sampling_defaults.temperature),
prog_opts::value<float>()->default_value(
sampling_defaults.temperature),
"Sampling temperature (higher = more random)");
opt("top-p",
prog_opts::value<float>()->default_value(sampling_defaults.top_p),
@@ -29,7 +30,8 @@ std::optional<ApplicationOptions> ParseArguments(
opt("n-ctx",
prog_opts::value<uint32_t>()->default_value(sampling_defaults.n_ctx),
"Context window size in tokens");
opt("seed", prog_opts::value<int>()->default_value(sampling_defaults.seed),
opt("seed",
prog_opts::value<int>()->default_value(sampling_defaults.seed),
"Sampler seed: -1 for random, otherwise non-negative integer");
opt("n-gpu-layers", prog_opts::value<int>()->default_value(0),
"Number of layers to offload to GPU");
@@ -82,7 +84,8 @@ std::optional<ApplicationOptions> ParseArguments(
try {
prog_opts::variables_map var_map;
prog_opts::store(prog_opts::parse_command_line(argc, argv, desc), var_map);
prog_opts::store(prog_opts::parse_command_line(argc, argv, desc),
var_map);
prog_opts::notify(var_map);
if (var_map.contains("help")) {
@@ -101,7 +104,8 @@ std::optional<ApplicationOptions> ParseArguments(
options.pipeline.output_path = var_map["output"].as<std::string>();
options.pipeline.log_path = var_map["log-path"].as<std::string>();
options.pipeline.prompt_dir = var_map["prompt-dir"].as<std::string>();
options.pipeline.location_count = var_map["location-count"].as<uint32_t>();
options.pipeline.location_count =
var_map["location-count"].as<uint32_t>();
const bool use_mocked = var_map["mocked"].as<bool>();
const std::string model_path = var_map["model"].as<std::string>();
@@ -161,9 +165,10 @@ std::optional<ApplicationOptions> ParseArguments(
// back to its own SamplingOptions{} defaults, keeping the two paths
// consistent without redundant copies.
const bool user_provided_sampling =
!var_map["temperature"].defaulted() || !var_map["top-p"].defaulted() ||
!var_map["top-k"].defaulted() || !var_map["n-ctx"].defaulted() ||
!var_map["seed"].defaulted() || !var_map["n_gpu_layers"].defaulted();
!var_map["temperature"].defaulted() ||
!var_map["top-p"].defaulted() || !var_map["top-k"].defaulted() ||
!var_map["n-ctx"].defaulted() || !var_map["seed"].defaulted() ||
!var_map["n_gpu_layers"].defaulted();
if (user_provided_sampling) {
// Warn but do not fail — the run is still valid, the flags are just

View File

@@ -24,7 +24,8 @@ void BiergartenPipelineOrchestrator::GenerateBreweries(
[this, &skipped_count](
const EnrichedCity& enriched_city) -> std::optional<BreweryRecord> {
try {
const BreweryResult brewery = generator_->GenerateBrewery(enriched_city);
const BreweryResult brewery =
generator_->GenerateBrewery(enriched_city);
const std::string postal_code =
postal_code_service_->GeneratePostalCode(enriched_city.location);
return BreweryRecord{
@@ -34,10 +35,10 @@ void BiergartenPipelineOrchestrator::GenerateBreweries(
} catch (const std::exception& e) {
++skipped_count;
logger_->Log(
{.level = LogLevel::Warn,
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("[Pipeline] Skipping city '{}' ({}): brewery "
.message = std::format(
"[Pipeline] Skipping city '{}' ({}): brewery "
"generation failed: {}",
enriched_city.location.city,
enriched_city.location.country, e.what())});
@@ -45,13 +46,14 @@ void BiergartenPipelineOrchestrator::GenerateBreweries(
}
};
const auto export_record = [this, &export_failed_count](
const BreweryRecord& record) {
const auto export_record =
[this, &export_failed_count](const BreweryRecord& record) {
try {
exporter_->ProcessRecord(record);
} catch (const std::exception& export_exception) {
++export_failed_count;
logger_->Log({.level = LogLevel::Warn,
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"[Pipeline] Generated brewery for '{}' ({}) "
@@ -62,7 +64,8 @@ void BiergartenPipelineOrchestrator::GenerateBreweries(
};
for (const EnrichedCity& enriched_city : cities) {
const std::optional<BreweryRecord> record = generate_record(enriched_city);
const std::optional<BreweryRecord> record =
generate_record(enriched_city);
if (!record.has_value()) {
continue;
}

View File

@@ -33,8 +33,8 @@ std::string Sanitize(std::string_view value) {
std::string BuildEmail(const Name& name,
std::unordered_set<std::string>& used_local_parts) {
const std::string base =
std::format("{}.{}", Sanitize(name.first_name), Sanitize(name.last_name));
const std::string base = std::format("{}.{}", Sanitize(name.first_name),
Sanitize(name.last_name));
std::string local_part = base;
uint32_t suffix = 1;
@@ -147,12 +147,13 @@ void BiergartenPipelineOrchestrator::GenerateUsers(
};
} catch (const std::exception& e) {
++skipped_count;
logger_->Log({.level = LogLevel::Warn,
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"[Pipeline] Skipping city '{}' ({}): "
.message = std::format("[Pipeline] Skipping city '{}' ({}): "
"user generation failed: {}",
city.location.city, city.location.country, e.what())});
city.location.city, city.location.country,
e.what())});
return std::nullopt;
}
};
@@ -164,7 +165,8 @@ void BiergartenPipelineOrchestrator::GenerateUsers(
} catch (const std::exception& export_exception) {
++export_failed_count;
logger_->Log({.level = LogLevel::Warn,
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"[Pipeline] Generated user for '{}' ({}) but "

View File

@@ -14,8 +14,7 @@
#include "services/curated_data/curated_json_data_service.h"
#include "services/logging/logger.h"
std::vector<City>
BiergartenPipelineOrchestrator::QueryLocations() {
std::vector<City> BiergartenPipelineOrchestrator::QueryLocations() {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "=== GEOGRAPHIC DATA OVERVIEW ==="});

View File

@@ -21,7 +21,8 @@ bool BiergartenPipelineOrchestrator::Run() {
size_t skipped_count = 0;
for (auto& city : cities) {
try {
std::string region_context = context_service_->GetLocationContext(city);
std::string region_context =
context_service_->GetLocationContext(city);
enriched.push_back(
EnrichedCity{.location = std::move(city),

View File

@@ -89,10 +89,10 @@ BreweryResult LlamaGenerator::GenerateBrewery(
raw = this->Infer(system_prompt, user_prompt, max_tokens,
kBreweryJsonGrammar);
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
logger_->Log({.level = LogLevel::Debug,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: raw output (attempt {}): {}",
.message = std::format(
"LlamaGenerator: raw output (attempt {}): {}",
attempt + 1, raw)});
}
@@ -106,10 +106,10 @@ BreweryResult LlamaGenerator::GenerateBrewery(
// Success: return parsed brewery data
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: successfully generated "
.message = std::format(
"LlamaGenerator: successfully generated "
"brewery data on attempt {}",
attempt + 1)});
}

View File

@@ -42,8 +42,8 @@ UserResult LlamaGenerator::GenerateUser(const EnrichedCity& city,
style_affinities);
const std::string retry_context = std::format(
"Name: {} {}\nCity: {}, {}\nPersona: {}", name.first_name, name.last_name,
city.location.city, city.location.country, persona.name);
"Name: {} {}\nCity: {}, {}\nPersona: {}", name.first_name,
name.last_name, city.location.city, city.location.country, persona.name);
constexpr int max_attempts = 3;
std::string raw;
@@ -51,12 +51,13 @@ UserResult LlamaGenerator::GenerateUser(const EnrichedCity& city,
int max_tokens = kUserInitialMaxTokens;
for (int attempt = 0; attempt < max_attempts; ++attempt) {
raw = this->Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
raw =
this->Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
logger_->Log({.level = LogLevel::Debug,
.phase = PipelinePhase::UserGeneration,
.message = std::format("LlamaGenerator: raw output (attempt {}): {}",
.message = std::format(
"LlamaGenerator: raw output (attempt {}): {}",
attempt + 1, raw)});
}
@@ -82,7 +83,8 @@ UserResult LlamaGenerator::GenerateUser(const EnrichedCity& city,
last_error = *validation_error;
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"LlamaGenerator: malformed user JSON (attempt {}): {}",

View File

@@ -4,6 +4,8 @@
* parsing, token decoding, and JSON validation helpers for Llama modules.
*/
#include <llama.h>
#include <algorithm>
#include <array>
#include <boost/json.hpp>
@@ -15,8 +17,6 @@
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator_helpers.h"
namespace {
@@ -82,8 +82,8 @@ bool ReadRequiredTrimmedStringField(const boost::json::object& obj,
bool HasSchemaPlaceholder(std::span<std::string* const> values) {
for (const std::string* value : values) {
std::string lowered = *value;
std::ranges::transform(lowered, lowered.begin(),
[](const unsigned char character) {
std::ranges::transform(
lowered, lowered.begin(), [](const unsigned char character) {
return static_cast<char>(std::tolower(character));
});
@@ -128,8 +128,8 @@ void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
auto buffer_too_small = [](int32_t result) -> bool { return result < 0; };
int32_t bytes =
llama_token_to_piece(vocab, token, buffer.data(), buffer.size(), 0, true);
int32_t bytes = llama_token_to_piece(vocab, token, buffer.data(),
buffer.size(), 0, true);
if (!buffer_too_small(bytes)) {
// Append the decoded bytes from the stack buffer.
@@ -190,8 +190,8 @@ std::optional<std::string> ValidateBreweryJson(const std::string& raw,
return validation_error;
}
if (!ReadRequiredTrimmedStringField(obj, "name_local", brewery_out.name_local,
&validation_error)) {
if (!ReadRequiredTrimmedStringField(
obj, "name_local", brewery_out.name_local, &validation_error)) {
return validation_error;
}
@@ -248,7 +248,8 @@ std::optional<std::string> ValidateUserJson(const std::string& raw,
const boost::json::value* activity_weight_field =
obj.if_contains("activity_weight");
if (activity_weight_field == nullptr || !activity_weight_field->is_number()) {
if (activity_weight_field == nullptr ||
!activity_weight_field->is_number()) {
return "Missing or invalid numeric field: activity_weight";
}

View File

@@ -5,6 +5,8 @@
* output tokens back to text for system+user chat prompts.
*/
#include <llama.h>
#include <algorithm>
#include <chrono>
#include <format>
@@ -14,8 +16,6 @@
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
@@ -58,7 +58,8 @@ SamplerHandle MakeSamplerChain(const llama_vocab* vocab,
if (!grammar.empty()) {
const std::string grammar_text(grammar);
add_sampler(llama_sampler_init_grammar(vocab, grammar_text.c_str(), "root"),
add_sampler(
llama_sampler_init_grammar(vocab, grammar_text.c_str(), "root"),
"LlamaGenerator: failed to initialize grammar sampler");
}
@@ -173,7 +174,8 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
prompt_tokens.resize(static_cast<size_t>(token_count));
if (token_count > prompt_budget) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: prompt too long ({} tokens), "

View File

@@ -5,14 +5,14 @@
#include "data_generation/llama_generator.h"
#include <llama.h>
#include <filesystem>
#include <memory>
#include <random>
#include <stdexcept>
#include <string>
#include <llama.h>
#include "data_model/models.h"
static constexpr uint32_t kMaxContextSize = 32768U;

View File

@@ -4,15 +4,15 @@
* context, and resets prior resources during model initialization.
*/
#include <ggml-backend.h>
#include <llama.h>
#include <algorithm>
#include <chrono>
#include <stdexcept>
#include <string>
#include <utility>
#include <ggml-backend.h>
#include <llama.h>
#include "data_generation/llama_generator.h"
// Maximum batch size for decode operations. Capping the batch prevents

View File

@@ -72,7 +72,8 @@ int main(const int argc, char** argv) {
prompt_directory = std::make_unique<PromptDirectory>(
options.pipeline.prompt_dir, log_producer);
} catch (const std::exception& dir_error) {
log_producer->Log({.level = LogLevel::Error,
log_producer->Log(
{.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = std::format("Invalid --prompt-dir: {}",
dir_error.what())});
@@ -87,7 +88,8 @@ int main(const int argc, char** argv) {
di::bind<std::string>().to(model_path),
di::bind<IExportService>().to<SqliteExportService>(),
di::bind<IPostalCodeService>().to(
[options, &log_producer]() -> std::unique_ptr<IPostalCodeService> {
[options,
&log_producer]() -> std::unique_ptr<IPostalCodeService> {
if (options.generator.use_mocked) {
log_producer->Log(
{.level = LogLevel::Info,
@@ -100,12 +102,14 @@ int main(const int argc, char** argv) {
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Postal code: Xeger (regex-based generation)"});
.message =
"Postal code: Xeger (regex-based generation)"});
return std::make_unique<XegerPostalCodeService>();
}),
di::bind<ICuratedDataService>().to(
[options, &log_producer]() -> std::unique_ptr<ICuratedDataService> {
[options,
&log_producer]() -> std::unique_ptr<ICuratedDataService> {
if (options.generator.use_mocked) {
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
@@ -140,7 +144,8 @@ int main(const int argc, char** argv) {
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Prompt formatter: Gemma4JinjaPromptFormatter"});
.message =
"Prompt formatter: Gemma4JinjaPromptFormatter"});
}
return std::unique_ptr<IPromptFormatter>(
std::make_unique<Gemma4JinjaPromptFormatter>());
@@ -148,7 +153,8 @@ int main(const int argc, char** argv) {
di::bind<WebClient>().to([options, log_producer] {
if (options.generator.use_mocked) {
{
log_producer->Log({.level = LogLevel::Info,
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Web client: none (mock mode)"});
}
@@ -183,7 +189,8 @@ int main(const int argc, char** argv) {
}),
di::bind<DataGenerator>().to(
[&options, &model_path, &sampling, &prompt_directory,
&log_producer](const auto& inj) -> std::unique_ptr<DataGenerator> {
&log_producer](
const auto& inj) -> std::unique_ptr<DataGenerator> {
if (options.generator.use_mocked) {
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,

View File

@@ -179,7 +179,8 @@ const PersonasList& CuratedJsonDataService::LoadPersonas() {
personas.push_back(UserPersona{
.name = ReadRequiredString(object, "name"),
.description = ReadRequiredString(object, "description"),
.style_affinities = ReadRequiredStringArray(object, "style_affinities"),
.style_affinities =
ReadRequiredStringArray(object, "style_affinities"),
});
}
@@ -211,7 +212,8 @@ const ForenamesByCountryMap& CuratedJsonDataService::LoadForenamesByCountry() {
if (!region.is_object()) {
continue;
}
const boost::json::value* names = region.as_object().if_contains("names");
const boost::json::value* names =
region.as_object().if_contains("names");
if (names == nullptr || !names->is_array()) {
continue;
}
@@ -221,8 +223,8 @@ const ForenamesByCountryMap& CuratedJsonDataService::LoadForenamesByCountry() {
}
const auto& name_object = name_value.as_object();
entries.emplace_back(ForenameEntry{
.name =
ReadFirstOfStringArray(name_object, "romanized", "localized"),
.name = ReadFirstOfStringArray(name_object, "romanized",
"localized"),
.gender = ReadRequiredString(name_object, "gender"),
});
}
@@ -257,8 +259,8 @@ const SurnamesByCountryMap& CuratedJsonDataService::LoadSurnamesByCountry() {
if (!name_value.is_object()) {
continue;
}
surnames.emplace_back(ReadFirstOfStringArray(name_value.as_object(),
"romanized", "localized"));
surnames.emplace_back(ReadFirstOfStringArray(
name_value.as_object(), "romanized", "localized"));
}
surnames_by_country.emplace(country_code, std::move(surnames));
}

View File

@@ -21,10 +21,10 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if (const auto cache_it = this->extract_cache_.find(cache_key);
cache_it != this->extract_cache_.end()) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
logger_->Log({.level = LogLevel::Debug,
.phase = PipelinePhase::Enrichment,
.message = std::format("Wikipedia: Cache hit for {}!", cache_key)});
.message = std::format("Wikipedia: Cache hit for {}!",
cache_key)});
}
return cache_it->second;
}
@@ -121,10 +121,10 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
// Handle 404/Missing status
if (page.contains("missing")) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Warn,
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format("WikipediaService: Page '{}' does not exist",
.message = std::format(
"WikipediaService: Page '{}' does not exist",
std::string(query))});
}
this->extract_cache_.emplace(cache_key, "");

View File

@@ -10,8 +10,7 @@
#include "services/enrichment/wikipedia_service.h"
std::string WikipediaEnrichmentService::GetLocationContext(
const City& loc) {
std::string WikipediaEnrichmentService::GetLocationContext(const City& loc) {
using namespace std::literals::chrono_literals;
if (!this->client_) {
if (logger_) {
@@ -53,10 +52,10 @@ std::string WikipediaEnrichmentService::GetLocationContext(
} catch (const std::runtime_error& e) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
logger_->Log({.level = LogLevel::Debug,
.phase = PipelinePhase::Enrichment,
.message = std::format("WikipediaService lookup failed for '{}': {}",
.message = std::format(
"WikipediaService lookup failed for '{}': {}",
location_query, e.what())});
}
}

View File

@@ -19,8 +19,8 @@ void SqliteStatementDeleter::operator()(
}
void ThrowSqliteError(sqlite3* db_handle, std::string_view action) {
const std::string message =
db_handle != nullptr ? sqlite3_errmsg(db_handle) : "unknown SQLite error";
const std::string message = db_handle != nullptr ? sqlite3_errmsg(db_handle)
: "unknown SQLite error";
throw std::runtime_error(std::format("{}: {}", action, message));
}

View File

@@ -108,7 +108,8 @@ uint64_t SqliteExportService::ProcessRecord(const BreweryRecord& brewery) {
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kBreweryEnglishNameBindIndex,
.index =
sqlite_export_service_internal::kBreweryEnglishNameBindIndex,
.value = brewery.brewery.name_en,
.action = "Failed to bind SQLite brewery English name"});
@@ -130,8 +131,8 @@ uint64_t SqliteExportService::ProcessRecord(const BreweryRecord& brewery) {
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index =
sqlite_export_service_internal::kBreweryLocalDescriptionBindIndex,
.index = sqlite_export_service_internal::
kBreweryLocalDescriptionBindIndex,
.value = brewery.brewery.description_local,
.action = "Failed to bind SQLite brewery local description"});
@@ -146,8 +147,8 @@ uint64_t SqliteExportService::ProcessRecord(const BreweryRecord& brewery) {
sqlite_export_service_internal::Bind(
insert_brewery_address_stmt_,
sqlite_export_service_internal::BoundParam<sqlite3_int64>{
.index =
sqlite_export_service_internal::kBreweryAddressBreweryIdBindIndex,
.index = sqlite_export_service_internal::
kBreweryAddressBreweryIdBindIndex,
.value = brewery_id,
.action = "Failed to bind SQLite brewery address brewery id"});
sqlite_export_service_internal::Bind(

View File

@@ -48,7 +48,8 @@ uint64_t SqliteExportService::ProcessRecord(const UserRecord& user) {
sqlite_export_service_internal::Bind(
insert_user_stmt_,
sqlite_export_service_internal::BoundParam<double>{
.index = sqlite_export_service_internal::kUserActivityWeightBindIndex,
.index =
sqlite_export_service_internal::kUserActivityWeightBindIndex,
.value = static_cast<double>(user.user.activity_weight),
.action = "Failed to bind SQLite user activity weight"});
sqlite_export_service_internal::Bind(

View File

@@ -3,14 +3,14 @@
* @brief Visitor that walks a regex AST and emits a random matching string.
*/
#include "services/xeger/xeger_internal.h"
#include <cassert>
#include <limits>
#include <random>
#include <stdexcept>
#include <string>
#include "services/xeger/xeger_internal.h"
namespace xeger_internal {
XegerGenerator::XegerGenerator(std::mt19937& rng, const int unbounded_cap)

View File

@@ -3,8 +3,6 @@
* @brief Recursive-descent parser that turns a regex pattern into an AST.
*/
#include "services/xeger/xeger_internal.h"
#include <cctype>
#include <format>
#include <memory>
@@ -14,6 +12,8 @@
#include <utility>
#include <vector>
#include "services/xeger/xeger_internal.h"
namespace xeger_internal {
Parser::Parser(std::string_view pattern) : src_(StripAnchors(pattern)) {}
@@ -168,7 +168,8 @@ NodePtr Parser::ParseAtom() {
// here (nothing back-references a group), so it's treated exactly
// like a plain group once the "?:" marker is consumed.
if (!AtEnd() && Peek() == kNonCapturingMarker) {
if (pos_ + 1 < src_.size() && src_[pos_ + 1] == kNonCapturingColon) {
if (pos_ + 1 < src_.size() &&
src_[pos_ + 1] == kNonCapturingColon) {
pos_ += 2;
} else {
throw std::runtime_error(std::format(
@@ -231,11 +232,11 @@ NodePtr Parser::ParseEscape() {
return MakeClass({{'a', 'z'}, {'A', 'Z'}, {'0', '9'}, {'_', '_'}},
true);
case 's':
return MakeClass({{' ', ' '}, {'\t', '\t'}, {'\n', '\n'}, {'\r', '\r'}},
false);
return MakeClass(
{{' ', ' '}, {'\t', '\t'}, {'\n', '\n'}, {'\r', '\r'}}, false);
case 'S':
return MakeClass({{' ', ' '}, {'\t', '\t'}, {'\n', '\n'}, {'\r', '\r'}},
true);
return MakeClass(
{{' ', ' '}, {'\t', '\t'}, {'\n', '\n'}, {'\r', '\r'}}, true);
case 'n':
return std::make_unique<Literal>('\n');
case 't':

View File

@@ -60,8 +60,8 @@ std::string HttpWebClient::Get(const std::string& url) {
.message = std::format(
"[HttpWebClient] Request failed for URL: {}", url)});
}
throw std::runtime_error(std::format("[HttpWebClient] HTTP {} for URL: {}",
result->status, url));
throw std::runtime_error(std::format(
"[HttpWebClient] HTTP {} for URL: {}", result->status, url));
}
return result->body;

File diff suppressed because it is too large Load Diff