mirror of
https://github.com/aaronpo97/the-biergarten-app.git
synced 2026-07-16 17:47:22 +00:00
Add user generation feature
This commit is contained in:
@@ -13,6 +13,7 @@ low-resource language failures.
|
||||
- [Model Bias and Language Quality](#model-bias-and-language-quality)
|
||||
- [Western and Eurocentric Lens](#western-and-eurocentric-lens)
|
||||
- [Wikipedia Enrichment](#wikipedia-enrichment)
|
||||
- [Names-by-Country Dataset](#names-by-country-dataset)
|
||||
- [The "Avoid AI Phrases" Prompt Instruction](#the-avoid-ai-phrases-prompt-instruction)
|
||||
- [Known Issues](#known-issues)
|
||||
- [Hallucinated Brewing Techniques](#hallucinated-brewing-techniques)
|
||||
@@ -91,6 +92,33 @@ generated descriptions.
|
||||
|
||||
---
|
||||
|
||||
## Names-by-Country Dataset
|
||||
|
||||
`tooling/pipeline/forenames-by-country.json` and `surnames-by-country.json`
|
||||
(used to sample a `Name` per ISO 3166-1 country code for user generation) are
|
||||
vendored verbatim, unmodified, from
|
||||
[sigpwned/popular-names-by-country-dataset](https://github.com/sigpwned/popular-names-by-country-dataset)
|
||||
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
|
||||
release assets), released under **CC0** (public domain). That dataset's own
|
||||
forename/surname lists are pulled from Wikipedia's "Lists of most common
|
||||
surnames" and "List of most popular given names" as of the week of
|
||||
2023-07-08 — see that project's README for full provenance. Names are not
|
||||
LLM-generated; this is curated fixture data per ROADMAP.md §2. Per-forename
|
||||
gender from the source data is preserved through to the sampled `Name`
|
||||
(rather than discarded during loading) so it's available for gender-aware
|
||||
persona/bio generation later.
|
||||
|
||||
The full multinational dataset is kept as-is (106 countries for forenames,
|
||||
75 for surnames) rather than trimmed to `locations.json`'s current country
|
||||
list, so it doesn't need re-sourcing if more countries are added later.
|
||||
`NamesByCountry::SampleName()` returns no result for a country present in
|
||||
neither file; of the countries in `locations.json`, that's currently `KE`,
|
||||
`SE`, `SG`, `TH`, `VN`, and `ZA` — `GenerateUsers` skips cities in those
|
||||
countries the same way brewery generation skips cities whose enrichment
|
||||
lookup fails.
|
||||
|
||||
---
|
||||
|
||||
## The "Avoid AI Phrases" Prompt Instruction
|
||||
|
||||
The system prompt instructs the model to avoid common AI-generated phrasing
|
||||
|
||||
@@ -33,28 +33,60 @@ architecture needs.
|
||||
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads.
|
||||
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
|
||||
`context_provided`, `generated_with`).
|
||||
- [ ] Add `activity_weight` to `UserResult` (currently just `username`,
|
||||
`bio`).
|
||||
- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to
|
||||
`UserResult` (currently just `username`, `bio`). `first_name`/
|
||||
`last_name`/`gender` are copied from the sampled `Name` (see below),
|
||||
not LLM-invented.
|
||||
- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result
|
||||
handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`,
|
||||
`gender`) and a `NamesByCountry` class (not a bare typedef) that owns
|
||||
two maps — `std::unordered_map<std::string, std::vector<ForenameEntry>>`
|
||||
and `std::unordered_map<std::string, std::vector<std::string>>`, both
|
||||
keyed by ISO 3166-1 code — and exposes `SampleName(iso3166_1, rng)`.
|
||||
Pairing a forename with a surname happens at sample time so gender
|
||||
(present per-forename in the source data) is never discarded the way a
|
||||
pre-flattened `{first_name, last_name}` list would lose it; see
|
||||
`LoadNamesByCountry()` below.
|
||||
- [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`,
|
||||
`metadata` (currently just `{location, brewery}`).
|
||||
- [ ] Add `GeneratedBeer`, `GeneratedUser`, `GeneratedCheckin`,
|
||||
`GeneratedRating`, `GeneratedFollow` aggregate structs.
|
||||
- [ ] Add `UserPersona` (`name`, `description`, `style_affinities`).
|
||||
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
|
||||
`GeneratedFollow` aggregate structs.
|
||||
- [x] Add `GeneratedUser` aggregate struct. Carries `email`, `date_of_birth`,
|
||||
and `password` — programmatically generated by the orchestrator, never
|
||||
LLM-authored — so a downstream auth-account seeding consumer (outside
|
||||
this repo tree's docs) can register real accounts from the pipeline's
|
||||
SQLite export. Skips `user_id`/`metadata` for now, matching
|
||||
`GeneratedBrewery`'s current (pre-`brewery_id`/`metadata`) shape. Not
|
||||
yet wired into `IExportService`/`SqliteExportService` — `GenerateUsers`
|
||||
currently only logs and holds `generated_users_` in memory; no `users`
|
||||
table exists in the SQLite export yet (see §5).
|
||||
- [x] Add `UserPersona` (`name`, `description`, `style_affinities`).
|
||||
|
||||
## 2. Data Preloading
|
||||
|
||||
`tooling/pipeline/includes/json_handling/`, fixture files.
|
||||
|
||||
- [ ] Extract a `DataPreloader` interface; have `JsonLoader` implement it
|
||||
instead of exposing only a static `LoadLocations()`.
|
||||
- [ ] Extract a `DataPreloader` interface; have `JsonLoader` implement it.
|
||||
`JsonLoader` now also exposes static `LoadPersonas()` and
|
||||
`LoadNamesByCountry()` (see below), but as additional static methods,
|
||||
not yet behind a formal interface.
|
||||
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo
|
||||
and is already copied into the Docker image
|
||||
(`runpod/Dockerfile`), but no loader reads it yet, and the native
|
||||
CMake build doesn't copy it into `build/` at all — `CMakeLists.txt`'s
|
||||
"Runtime Assets" step only copies `locations.json` and `prompts/`.
|
||||
- [ ] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
|
||||
- [ ] Add `LoadNamesByCountry()` and author `names-by-country.json` (doesn't
|
||||
exist yet).
|
||||
- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
|
||||
- [x] Add `LoadNamesByCountry(forenames_filepath, surnames_filepath)`,
|
||||
parsing `tooling/pipeline/forenames-by-country.json` and
|
||||
`surnames-by-country.json` into a `NamesByCountry`. Both files are
|
||||
vendored verbatim (unmodified, full multinational coverage) from
|
||||
`sigpwned/popular-names-by-country-dataset` (CC0) — see
|
||||
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for
|
||||
provenance. Deliberately not pre-paired or filtered down to just the
|
||||
countries in `locations.json`: keeping the source shape (including
|
||||
per-forename gender) intact means the loader can support more
|
||||
countries later, or gender-aware persona/bio generation, without
|
||||
re-sourcing data.
|
||||
|
||||
## 3. Policy / Strategy Layer
|
||||
|
||||
@@ -83,10 +115,14 @@ Entirely new — no `includes/policy/` (or equivalent) directory exists today.
|
||||
- [ ] Extend the `DataGenerator` interface with `GenerateBeer`,
|
||||
`GenerateCheckin`, `GenerateRating` (today: only `GenerateBrewery` and
|
||||
`GenerateUser`).
|
||||
- [ ] Implement `LlamaGenerator::GenerateUser` for real. It currently
|
||||
returns a hardcoded `{"test_user", "This is a test user profile from
|
||||
{locale}."}` regardless of input — see the `// TODO` at the top of
|
||||
`src/data_generation/llama/generate_user.cc`.
|
||||
- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop
|
||||
mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3
|
||||
attempts with corrective feedback) and a new
|
||||
`prompts/USER_GENERATION.md`. Changed the `DataGenerator::GenerateUser`
|
||||
signature to
|
||||
`(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`,
|
||||
matching what the activity diagram actually passes. `MockGenerator::GenerateUser`
|
||||
updated to match the new signature too.
|
||||
- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` /
|
||||
`GenerateRating`.
|
||||
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
|
||||
|
||||
@@ -43,7 +43,7 @@ fork again
|
||||
:EnrichmentService::PreWarmLocationCache(sampled_locations);
|
||||
end fork
|
||||
fork
|
||||
:JsonLoader::LoadNamesByCountry("names-by-country.json");
|
||||
:JsonLoader::LoadNamesByCountry(\n "forenames-by-country.json",\n "surnames-by-country.json");
|
||||
fork again
|
||||
:JsonLoader::LoadPersonas("personas.json");
|
||||
end fork
|
||||
|
||||
@@ -67,11 +67,47 @@ package "Domain: Models" {
|
||||
}
|
||||
|
||||
class UserResult {
|
||||
+ first_name : std::string
|
||||
+ last_name : std::string
|
||||
+ gender : std::string
|
||||
+ username : std::string
|
||||
+ bio : std::string
|
||||
+ activity_weight : float
|
||||
}
|
||||
|
||||
class Name {
|
||||
+ first_name : std::string
|
||||
+ last_name : std::string
|
||||
+ gender : std::string
|
||||
}
|
||||
|
||||
class ForenameEntry {
|
||||
+ name : std::string
|
||||
+ gender : std::string
|
||||
}
|
||||
|
||||
class NamesByCountry {
|
||||
- forenames_by_country_ : std::unordered_map<std::string, std::vector<ForenameEntry>>
|
||||
- surnames_by_country_ : std::unordered_map<std::string, std::vector<std::string>>
|
||||
+ SampleName(iso3166_1 : const std::string&,\n rng : std::mt19937&) : std::optional<Name>
|
||||
}
|
||||
|
||||
note right of NamesByCountry
|
||||
Backed by two source-shaped maps,
|
||||
keyed by ISO 3166-1 country code,
|
||||
sourced from popular-names-by-country-dataset
|
||||
(CC0) -- see ETHICS-AND-KNOWN-ISSUES.md.
|
||||
Gender is preserved per forename so it can
|
||||
be threaded through to persona/bio generation
|
||||
later; surnames carry no gender in the source
|
||||
data. Pairing happens at sample time, not at
|
||||
fixture-authoring time, so no information from
|
||||
the source dataset is discarded up front.
|
||||
end note
|
||||
|
||||
NamesByCountry *-- "0..*" ForenameEntry
|
||||
NamesByCountry ..> Name : produces
|
||||
|
||||
class CheckinResult {
|
||||
+ checked_in_at : std::string
|
||||
+ note : std::string
|
||||
@@ -110,9 +146,20 @@ package "Domain: Models" {
|
||||
+ user_id : uint64_t
|
||||
+ location : Location
|
||||
+ user : UserResult
|
||||
+ email : std::string
|
||||
+ date_of_birth : std::string
|
||||
+ password : std::string
|
||||
+ metadata : GenerationMetadata
|
||||
}
|
||||
|
||||
note right of GeneratedUser
|
||||
email, date_of_birth, and password are
|
||||
programmatically generated by the orchestrator
|
||||
(not LLM-authored) so a downstream auth-account
|
||||
seeding consumer can register real accounts from
|
||||
this export. See ROADMAP.md §1.
|
||||
end note
|
||||
|
||||
class GeneratedCheckin {
|
||||
+ checkin_id : uint64_t
|
||||
+ user_id : uint64_t
|
||||
@@ -321,15 +368,15 @@ package "Infrastructure: Data Preloading" {
|
||||
interface DataPreloader <<interface>> {
|
||||
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
|
||||
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
|
||||
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
|
||||
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
|
||||
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<UserPersona>
|
||||
+ LoadNamesByCountry(forenames_filepath : const std::filesystem::path&,\n surnames_filepath : const std::filesystem::path&) : NamesByCountry
|
||||
}
|
||||
|
||||
class JsonLoader {
|
||||
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
|
||||
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
|
||||
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
|
||||
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
|
||||
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<UserPersona>
|
||||
+ LoadNamesByCountry(forenames_filepath : const std::filesystem::path&,\n surnames_filepath : const std::filesystem::path&) : NamesByCountry
|
||||
}
|
||||
|
||||
}
|
||||
@@ -380,7 +427,7 @@ package "Infrastructure: Data Generation" {
|
||||
interface DataGenerator <<interface>> {
|
||||
+ GenerateBrewery(location : const Location&,\n context : const LocationContext&) : BreweryResult
|
||||
+ GenerateBeer(brewery_id : uint64_t,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult
|
||||
+ GenerateUser(location : const Location&) : UserResult
|
||||
+ GenerateUser(city : const EnrichedCity&,\n persona : const UserPersona&,\n name : const Name&) : UserResult
|
||||
+ GenerateCheckin(user : const GeneratedUser&,\n brewery : const GeneratedBrewery&,\n timestamp : const std::string&) : CheckinResult
|
||||
+ GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : uint64_t) : RatingResult
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 106 KiB After Width: | Height: | Size: 139 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 184 KiB After Width: | Height: | Size: 234 KiB |
Reference in New Issue
Block a user