The Biergarten Data Pipeline (Current — Synchronous Data Path)The Biergarten Data Pipeline (Current — Synchronous Data Path)Create BoundedChannel<LogEntry> log_channelThe only concurrency in the current pipeline:a dedicated thread drains log_channel andforwards entries to spdlog for the entire run.Joined during shutdown after log_channel closes.Spawn log dispatcher thread(LogDispatcher::Run())ParseArguments(argc, argv)Log error / usage infonoAre arguments valid?yesInit OpenSSL global state & LlamaBackendStateBinds ILogger, IEnrichmentService, DataGenerator,IExportService, IPromptFormatter, WebClient basedon --mocked vs. model-backed mode.di::make_injector(...)injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>()Run() itself is synchronous — sampling, enrichment,generation, and export all happen on this thread.BiergartenPipelineOrchestrator::Run()Close log_channelJoin log dispatcher threadReturn 0Initialize SQLite exportQueryLocations()Lookup failed?yesnoLog warning, skip cityStore EnrichedCity{Location, region_context}Remaining citiesFor each sampled Location?DoneGenerateUsers(enriched_cities)SampleName(forenames_by_country,surnames_by_country, location.iso3166_1)Names available for country?noyesLog warning, skip citySample UserPersona (uniform random)Generation successful?yesnoGetRandomCoordsWithinRange(city_centre, 5km, rng)BuildEmail(name, used_email_local_parts)email and date_of_birth are generatedprogrammatically, never LLM-authored.Generation + coordinate jitter run insideone try/catch per record. UserRecord.addressnow carries {city, longitude, latitude}.GenerateDateOfBirth(rng)Log warning "SQLite export failed"Log warning "Generation failed, skipping..."Remaining citiesFor each EnrichedCity?DoneGenerateBreweries(enriched_cities)Generation successful?yesnoGeneration + coordinate jitter run inside onetry/catch per record. BreweryRecord.addressnow carries {city, longitude, latitude}.GetRandomCoordsWithinRange(city_centre, 5km, rng)Data loss is prevented per-record.The pipeline continues running.Log warning "SQLite export failed"Log warning "Generation failed, skipping..."GetUtcTimestamp() from SystemDateTimeProviderBuilds a fresh biergarten_seed_<UTC datetime>.sqlite filenameAppends a numeric suffix if the timestamp already existsOpens DB ConnectionExecutes Schema DDLBegins TransactionInitialize()ProcessRecord(UserRecord)Location in cache?yesnoReuse location_idInsert Location & Cache IDInsert User (FK: location_id)yesException caught during insert?noProcessRecord(GeneratedBrewery)Location in cache?yesnoReuse location_idInsert Location & Cache IDInsert Brewery (FK: location_id)yesException caught during insert?noCommits TransactionCloses Database ConnectionFinalize()JsonLoader::LoadLocations("locations.json")--location-count defaults to 10.std::ranges::sample(all_locations, --location-count)LoadPersonas("personas.json")LoadForenamesByCountry("forenames-by-country.json")Each call is cached after its first parse.MockCuratedDataService (--mocked) returns afixed in-memory dataset instead and skipsthese three JSON files entirely.LoadSurnamesByCountry("surnames-by-country.json")WikipediaEnrichmentService fetches a generic"brewing" extract and a "beer in {country}" extract(not a city/region-specific page). MockEnrichmentService(--mocked) returns an empty string instead.GetLocationContext(loc)MockGeneratorGenerator ModeDeterministicHash & FormatLlamaGeneratorGenerator ModeOpenAIGeneratorResolves to USER_GENERATION.md inside --prompt-dir.prompt_directory_->Load("USER_GENERATION")Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar)ValidateUserJson(raw, user)Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 3?yesSame prompt file as LlamaGenerator - contentis backend-agnostic.prompt_directory_->Load("USER_GENERATION")CallChatCompletionsApi(system_prompt, user_prompt,kUserJsonSchema, max_tokens)ValidateUserJson(raw, user)Structured Outputs (response_format:json_schema, strict: true) alreadyguarantees schema shape, so only 2attempts are needed - retries here covertransient failures / placeholder text,not malformed JSON.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 2?yesMockGeneratorGenerator ModeDeterministicHash & FormatLlamaGeneratorGenerator ModeOpenAIGeneratorPrepareRegionContextResolves to BREWERY_GENERATION.md inside --prompt-dir.prompt_directory_->Load("BREWERY_GENERATION")Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar)ValidateBreweryJson(raw, brewery)max_tokens is fixed across retries —it is not raised on truncation.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 3?yesPrepareRegionContextSame prompt file as LlamaGenerator - contentis backend-agnostic.prompt_directory_->Load("BREWERY_GENERATION")CallChatCompletionsApi(system_prompt, user_prompt,kBreweryJsonSchema, max_tokens)ValidateBreweryJson(raw, brewery)Structured Outputs already guarantees schemashape, so only 2 attempts are needed - retrieshere cover transient failures / placeholdertext, not malformed JSON.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 2?yesRemaining citiesFor each EnrichedCity?Donemain.ccBiergartenPipelineOrchestratorSqliteExportServiceJsonLoaderIEnrichmentServiceDataGenerator