6 Commits

Author SHA1 Message Date
Aaron Po
c58e4c1986 updates 2026-05-15 14:11:34 -04:00
Aaron Po
e6a20324e4 update logging to use logger channel 2026-05-15 01:13:53 -04:00
Aaron Po
a4968eb043 update doxygen comments for concurrent logger 2026-05-14 21:21:28 -04:00
Aaron Po
74f11b57e2 Implement pipeline logging with bounded channels and orchestrator integration 2026-05-14 21:16:29 -04:00
Aaron Po
f93b14897b Add bounded channels 2026-05-14 19:50:08 -04:00
Aaron Po
688c28f85d update class diagram 2026-05-14 19:49:14 -04:00
325 changed files with 14506 additions and 73821 deletions

View File

@@ -41,7 +41,7 @@ Data generation pipeline (C++):
Active areas in the repository:
- .NET 10 backend (vertical-slice architecture with MediatR) + SQL Server
- .NET 10 backend (layered architecture) + SQL Server
- React 19 website (React Router 7 + Vite)
- Shared Biergarten theme system + Storybook coverage
- Auth flows and account/email integration (local Mailpit in dev compose)
@@ -105,8 +105,7 @@ npm run test:storybook:playwright
```text
web/
backend/ .NET API: API.Core (host), Features.* (vertical slices),
Shared.*, Domain.*, Infrastructure.*, Database.* projects
backend/ .NET API + domain/service/infrastructure + DB projects
frontend/ React Router website + Storybook + Playwright/Vitest
tooling/

View File

@@ -7,65 +7,61 @@ This document describes the active architecture of The Biergarten App.
The Biergarten App is a monorepo with a clear split between the backend and the
active website:
- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature
slices with MediatR
- **Frontend**: React 19 + React Router 7 website in `web/frontend`
- **Architecture Style**: Vertical-slice backend plus server-rendered React
frontend
- **Backend**: .NET 10 Web API with SQL Server and a layered architecture
- **Frontend**: React 19 + React Router 7 website in `src/Website`
- **Architecture Style**: Layered backend plus server-rendered React frontend
The legacy Next.js frontend has been retained in `archive/next-js-web-app/`
for reference only.
The legacy Next.js frontend has been retained in `src/Website-v1` for reference
only and is documented in
[archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Diagrams
For visual representations, see:
- [architecture.svg](website/diagrams-out/architecture.svg) - Vertical-slice
architecture diagram
- [deployment.svg](website/diagrams-out/deployment.svg) - Docker deployment
- [architecture.svg](diagrams-out/architecture.svg) - Layered architecture
diagram
- [authentication-flow.svg](website/diagrams-out/authentication-flow.svg) -
- [deployment.svg](diagrams-out/deployment.svg) - Docker deployment diagram
- [authentication-flow.svg](diagrams-out/authentication-flow.svg) -
Authentication workflow
- [database-schema.svg](website/diagrams-out/database-schema.svg) - Database
- [database-schema.svg](diagrams-out/database-schema.svg) - Database
relationships
## Backend Architecture
### Vertical Slice Architecture Pattern
### Layered Architecture Pattern
The backend organizes business capabilities as feature slices instead of
technical layers. Each feature (`Features.Auth`, `Features.Breweries`,
`Features.UserManagement`, `Features.Emails`) is a single project that owns
its own controller, MediatR commands/queries/handlers, validators, and
repository, end to end:
The backend follows a strict layered architecture:
```
┌───────────────────────────────────────────────────────────────────────
API.Core (thin host)
│ - Program.cs wiring: MediatR + AddApplicationPart per slice
│ - Swagger/OpenAPI, JWT auth middleware, global exception filter
└───────────────────────────────────────────────────────────────────────┘
↓ discovers controllers via AddApplicationPart
┌───────────────┬───────────────┬───────────────────┬───────────────────┐
│ Features.Auth │Features. │ Features. │ Features.Emails │
│Breweries │ UserManagement │ (no controller,
Controller │ Controller │ Controller │ internal only)
Commands/ │ Commands/ │ Commands/ │ Commands/
Queries + │ Queries + │ Queries + │ Handlers
│ Handlers │ Handlers │ Handlers │ │
│ Repository │ Repository │ Repository │ EmailDispatcher │
└───────────────┴───────────────┴──────────────────────────────────────┘
↓ each slice depends only on shared/domain/infra, never on another slice
┌─────────────────────────┬─────────────────────────┬────────────────────┐
Shared.Contracts │ Shared.Application │ Domain.Entities /
(ResponseBody envelope) │ (ValidationBehavior, │ Domain.Exceptions
│ cross-slice email cmds) │
└─────────────────────────┴─────────────────────────┴────────────────────┘
┌─────────────────────────────────────────────────────────────────────────
Infrastructure.Sql, Infrastructure.Jwt, Infrastructure.PasswordHashing,
Infrastructure.Email, Infrastructure.Email.Templates
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────┐
API Layer (Controllers)
│ - HTTP Endpoints
│ - Request/Response mapping
│ - Swagger/OpenAPI │
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
Service Layer (Business Logic)
- Authentication logic
- User management
- Validation & orchestration
└─────────────────────────────────────┘
─────────────────────────────────────
Infrastructure Layer (Tools) │
│ - JWT token generation │
- Password hashing (Argon2id)
- Email services
- Repository implementations
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
Domain Layer (Entities)
- UserAccount, UserCredential
│ - Pure POCO classes │
│ - No external dependencies │
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│ Database (SQL Server) │
@@ -74,123 +70,75 @@ repository, end to end:
└─────────────────────────────────────┘
```
Slices never reference each other's project. The one cross-slice
interaction (`Features.Auth` triggering a confirmation email handled by
`Features.Emails`) goes through a MediatR command (`SendRegistrationEmailCommand`)
whose contract lives in `Shared.Application`, so neither slice takes a
project reference on the other.
### Layer Responsibilities
#### API Layer (`API.Core`)
**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of
its own
**Purpose**: HTTP interface and request handling
**Components**:
- `Program.cs`: registers MediatR (scanning every `Features.*` assembly),
FluentValidation, and uses `AddApplicationPart` so each slice's controllers
are discovered by MVC
- `GlobalException.cs`: global exception filter (host-level cross-cutting
concern)
- `Authentication/JwtAuthenticationHandler.cs`: JWT auth scheme middleware
- Swagger/OpenAPI documentation, health check endpoints
- Controllers (`AuthController`, `UserController`)
- Middleware for error handling
- Swagger/OpenAPI documentation
- Health check endpoints
**Dependencies**:
- Every `Features.*` project (for controller/MediatR discovery)
- `Shared.Contracts`, `Shared.Application`
- `Infrastructure.Jwt` (for the auth middleware)
- Service layer
- ASP.NET Core framework
**Rules**:
- No controllers, no business logic, no feature-specific contracts
- Exists purely to host and wire up the feature slices
- No business logic
- Only request/response transformation
- Delegates all work to Service layer
#### Feature Slices (`Features.Auth`, `Features.Breweries`, `Features.UserManagement`, `Features.Emails`)
#### Service Layer (`Service.Auth`, `Service.UserManagement`)
**Purpose**: Each slice is the complete vertical for one business capability
**Components** (per slice):
- `Controllers/`: HTTP endpoints, binding directly to Command/Query types
as the request contract
- `Commands/<Operation>/` and `Queries/<Operation>/`: one folder per
operation, each containing the Command/Query record, its
`IRequestHandler`, and (for commands) a FluentValidation validator
- `Repository/`: the slice's own stored-procedure repository
implementation
- `Dtos/`: response shapes returned by query handlers (never the raw
domain entity)
- `DependencyInjection/`: an `AddFeaturesX()` extension method registering
the slice's repository/services
`Features.Emails` has no `Controllers/` folder. It's invoked only via
MediatR commands sent from other slices, never over HTTP.
**Dependencies**:
- `Domain.Entities`, `Domain.Exceptions`
- `Infrastructure.Sql` (generic ADO.NET plumbing) plus whichever
infrastructure project the slice needs (`Infrastructure.Jwt`/
`Infrastructure.PasswordHashing` for Auth, `Infrastructure.Email`/
`Infrastructure.Email.Templates` for Emails)
- `Shared.Contracts`, `Shared.Application`
- **Never** another `Features.*` project
**Rules**:
- All business logic for that feature lives in its command/query handlers
- No direct controller-to-repository calls; everything flows through
MediatR
- Read endpoints return a dedicated `Dto`, never the domain entity directly
#### Shared Projects (`Shared.Contracts`, `Shared.Application`)
**Purpose**: The minimum cross-slice surface area required because every
slice needs it, or because duplicating it four times would be worse than
sharing it
**Purpose**: Business logic and orchestration
**Components**:
- `Shared.Contracts`: `ResponseBody<T>`/`ResponseBody`, the API response
envelope every controller returns
- `Shared.Application`: `ValidationBehavior<TRequest,TResponse>` (the
MediatR pipeline behavior that runs FluentValidation before a handler
executes) and the cross-slice email commands
(`SendRegistrationEmailCommand`, `SendResendConfirmationEmailCommand`)
- Authentication services (login, registration)
- User management services
- Business rule validation
- Transaction coordination
**Dependencies**:
- Infrastructure layer (repositories, JWT, password hashing)
- Domain entities
**Rules**:
- Kept deliberately small: this is the exception to "no slice depends on
another slice," not a general-purpose dumping ground
- Contains all business logic
- Coordinates multiple infrastructure components
- No direct database access (uses repositories)
- Returns domain models, not DTOs
#### Infrastructure Layer
**Purpose**: Technical capabilities and external integrations, shared by
whichever slices need them
**Purpose**: Technical capabilities and external integrations
**Components**:
- **Infrastructure.Sql**: generic ADO.NET connection/command plumbing
(`ISqlConnectionFactory`, the abstract `Repository<T>` base class), not
domain-specific; each slice's own `Repository/` folder builds on this
- **Infrastructure.Repository**: Data access via stored procedures
- **Infrastructure.Jwt**: JWT token generation and validation
- **Infrastructure.PasswordHashing**: Argon2id password hashing
- **Infrastructure.Email**: Email sending capabilities (SMTP/MailKit)
- **Infrastructure.Email.Templates**: Email template rendering (Razor
components)
- **Infrastructure.Email**: Email sending capabilities
- **Infrastructure.Email.Templates**: Email template rendering
**Dependencies**:
- Domain entities
- External libraries (ADO.NET, JWT, Argon2, MailKit, etc.)
- External libraries (ADO.NET, JWT, Argon2, etc.)
**Rules**:
- Implements technical concerns only, no business logic
- Reusable across slices
- Implements technical concerns
- No business logic
- Reusable across services
#### Domain Layer (`Domain.Entities`)
@@ -215,55 +163,21 @@ whichever slices need them
### Design Patterns
#### Vertical Slice + MediatR
**Purpose**: Organize code by feature instead of by technical layer, so
everything needed to understand or change one capability lives in one
project
**Implementation**:
- Each HTTP write operation is a `Command` (e.g. `CreateBreweryCommand` in
`Features.Breweries/Commands/CreateBrewery/`); each read operation is a
`Query` (e.g. `GetBreweryByIdQuery`)
- Controllers bind directly to the Command/Query as the request body;
there is no separate request DTO + mapping step for writes
- A single shared `ValidationBehavior<TRequest,TResponse>`
(`Shared.Application/Behaviors/`) runs FluentValidation validators in the
MediatR pipeline before any handler executes
- Query handlers map to a dedicated response `Dto`, so domain entities never
leak over the wire
**Example**:
```csharp
public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...)
: IRequest<BreweryDto>;
public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler<CreateBreweryCommand, BreweryDto>
{
public async Task<BreweryDto> Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... }
}
```
#### Repository Pattern
**Purpose**: Abstract database access behind interfaces
**Implementation**: each slice owns its own repository, scoped to that
feature only:
**Implementation**:
- `Features.Auth/Repository/IAuthRepository.cs`
- `Features.Breweries/Repository/IBreweryRepository.cs`
- `Features.UserManagement/Repository/IUserAccountRepository.cs`
- `Infrastructure.Sql/DefaultSqlConnectionFactory.cs`: the generic
connection factory every slice's repository builds on
- `IAuthRepository` - Authentication queries
- `IUserAccountRepository` - User account queries
- `DefaultSqlConnectionFactory` - Connection management
**Benefits**:
- Testable (easy to mock)
- SQL-first approach (stored procedures)
- Each slice's data access logic is self-contained
- Centralized data access logic
**Example**:
@@ -279,19 +193,17 @@ public interface IAuthRepository
**Purpose**: Loose coupling and testability
**Configuration**: `Program.cs` wires up MediatR/FluentValidation across
every `Features.*` assembly; each slice exposes its own `AddFeaturesX()`
extension method that registers its repository and slice-internal services
**Configuration**: `Program.cs` registers all services
**Lifetimes**:
- Scoped: Repositories, slice-internal services (per request)
- Singleton: `ISqlConnectionFactory`
- Scoped: Repositories, Services (per request)
- Singleton: Connection factories, JWT configuration
- Transient: Utilities, helpers
#### SQL-First Approach
**Purpose**: Push complex logic into the database
**Purpose**: Leverage database capabilities
**Strategy**:
@@ -308,13 +220,13 @@ extension method that registers its repository and slice-internal services
## Frontend Architecture
### Active Website (`web/frontend`)
### Active Website (`src/Website`)
The current website is a React Router 7 application with server-side rendering
enabled.
```text
web/frontend/
src/Website/
├── app/
│ ├── components/ Shared UI such as Navbar, FormField, SubmitButton, ToastProvider
│ ├── lib/ Auth helpers, schemas, and theme metadata
@@ -350,9 +262,10 @@ All component styling should prefer semantic tokens such as `primary`,
### Legacy Frontend
The previous Next.js frontend has been archived at `archive/next-js-web-app/`
for reference only. Active product and engineering documentation should point
to `web/frontend`.
The previous Next.js frontend has been archived at `src/Website-v1`. Active
product and engineering documentation should point to `src/Website`, while
legacy notes live in
[archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Security Architecture
@@ -477,7 +390,7 @@ scripts/
- Testing (`docker-compose.test.yaml`)
- Production (`docker-compose.prod.yaml`)
For details, see [Docker Guide](website/docker.md).
For details, see [Docker Guide](docker.md).
### Health Checks
@@ -503,10 +416,11 @@ healthcheck:
│ Integration │ ← API.Specs (Reqnroll)
│ Tests │
├──────────────┤
│ Unit Tests │ ← Features.Auth.Tests, Features.Breweries.Tests,
(per slice,Features.UserManagement.Tests, Features.Emails.Tests
│ handlers + │ (commands/queries/handlers + that slice's own
repository) │ repository, mocked with Moq/DbMocker)
│ Unit Tests │ ← Service.Auth.Tests
(Service)Repository.Tests
├──────────────┤
Unit Tests │
│ (Repository) │
└──────────────┘
```
@@ -517,4 +431,4 @@ healthcheck:
- Mock external dependencies
- Test database for integration tests
For details, see [Testing Guide](website/testing.md).
For details, see [Testing Guide](testing.md).

View File

@@ -13,7 +13,6 @@ low-resource language failures.
- [Model Bias and Language Quality](#model-bias-and-language-quality)
- [Western and Eurocentric Lens](#western-and-eurocentric-lens)
- [Wikipedia Enrichment](#wikipedia-enrichment)
- [Names-by-Country Dataset](#names-by-country-dataset)
- [The "Avoid AI Phrases" Prompt Instruction](#the-avoid-ai-phrases-prompt-instruction)
- [Known Issues](#known-issues)
- [Hallucinated Brewing Techniques](#hallucinated-brewing-techniques)
@@ -92,33 +91,6 @@ generated descriptions.
---
## Names-by-Country Dataset
`tooling/pipeline/forenames-by-country.json` and `surnames-by-country.json`
(used to sample a `Name` per ISO 3166-1 country code for user generation) are
vendored verbatim, unmodified, from
[sigpwned/popular-names-by-country-dataset](https://github.com/sigpwned/popular-names-by-country-dataset)
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
release assets), released under **CC0** (public domain). That dataset's own
forename/surname lists are pulled from Wikipedia's "Lists of most common
surnames" and "List of most popular given names" as of the week of
2023-07-08 — see that project's README for full provenance. Names are not
LLM-generated; this is curated fixture data per ROADMAP.md §2. Per-forename
gender from the source data is preserved through to the sampled `Name`
(rather than discarded during loading) so it's available for gender-aware
persona/bio generation later.
The full multinational dataset is kept as-is (106 countries for forenames,
75 for surnames) rather than trimmed to `locations.json`'s current country
list, so it doesn't need re-sourcing if more countries are added later.
`SampleName()` (a free helper in `generate_users.cc`) returns no result for
a country present in neither file; of the countries in `locations.json`,
that's currently `KE`, `SE`, `SG`, `TH`, `VN`, and `ZA``GenerateUsers`
skips cities in those countries the same way brewery generation skips
cities whose enrichment lookup fails.
---
## The "Avoid AI Phrases" Prompt Instruction
The system prompt instructs the model to avoid common AI-generated phrasing
@@ -273,7 +245,7 @@ but is semantically incoherent. This comes from limited training-data coverage
rather than prompt engineering.
Output sample:
[./french-cities.example](french-cities.example)
[./out-sample/french-cities.example](out-sample/french-cities.example)
#### Proposed Mitigations

View File

@@ -1,9 +1,8 @@
# Biergarten Pipeline
A C++20 command-line pipeline that samples city records from local JSON,
enriches each with Wikipedia context, and generates bilingual brewery names
and descriptions plus locale-grounded user profiles via a local GGUF model or
a deterministic mock.
enriches each with Wikipedia context, and generates bilingual brewery names and
descriptions via a local GGUF model or a deterministic mock.
> **This pipeline produces AI-generated data.** It is not a source of truth for
> brewing techniques, cultural representation, or local-language accuracy. See
@@ -46,7 +45,6 @@ step.
| Beer reviews and ratings | Stable brewery fixtures with enough context to anchor review pages |
| Social follow relationships | Repeatable brewery entities for feeds, follows, and saved lists |
| Geospatial brewery experiences | Latitude, longitude, and country-level metadata |
| User accounts and profiles | Locale-grounded names, bios, and an auth-ready email/date-of-birth pair for seeding real accounts |
---
@@ -96,27 +94,24 @@ Each run writes a fresh dated SQLite file such as
./biergarten-pipeline \
--model ../models/google_gemma-4-E4B-it-Q6_K.gguf \
--prompt-dir prompts \
--location-count 25 \
--temperature 1.0 --top-p 0.95 --top-k 64 --n-ctx 8192 --seed -1
```
#### CLI Flags
| Flag | Purpose |
| ------------------ | ---------------------------------------------------------------------------------------------------- |
| `--mocked` | Deterministic mock generator, no model required. |
| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. |
| `--prompt-dir` | Directory containing prompt files (e.g. `BREWERY_GENERATION.md`). Required unless `--mocked` is set. |
| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. |
| `--log-path` | Path for application logs. Default: `pipeline.log`. |
| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. |
| `--temperature` | Sampling temperature. Default: `1.0`. |
| `--top-p` | Nucleus sampling. Default: `0.95`. |
| `--top-k` | Top-k sampling. Default: `64`. |
| `--n-ctx` | Context window size. Default: `8192`. |
| `--seed` | Random seed. Default: `-1` (random at runtime). |
| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. |
| `--help, -h` | Print usage and exit. |
| Flag | Purpose |
| --------------- | ---------------------------------------------------------------------------------------------------- |
| `--mocked` | Deterministic mock generator, no model required. |
| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. |
| `--prompt-dir` | Directory containing prompt files (e.g. `BREWERY_GENERATION.md`). Required unless `--mocked` is set. |
| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. |
| `--log-path` | Path for application logs. Default: `pipeline.log`. |
| `--temperature` | Sampling temperature. Default: `1.0`. |
| `--top-p` | Nucleus sampling. Default: `0.95`. |
| `--top-k` | Top-k sampling. Default: `64`. |
| `--n-ctx` | Context window size. Default: `8192`. |
| `--seed` | Random seed. Default: `-1` (random at runtime). |
| `--help, -h` | Print usage and exit. |
`--mocked` and `--model` are mutually exclusive. Omitting both exits with an
error before the pipeline starts. Sampling flags are ignored when `--mocked` is
@@ -135,20 +130,17 @@ NVIDIA GPU.
### How it works
The container uses a two-stage build. The builder stage installs CMake/Ninja,
clones the matching llama.cpp release tag for its headers only (installed into
`/usr/local/include`), and copies prebuilt shared libraries (`libllama`,
`libggml`, and CUDA/CPU backend plugins) from `ghcr.io/ggml-org/llama.cpp:full-cuda`
into `/usr/local/lib`. With both headers and libraries present, CMake's
system-library detection (see [Build](#build) above) finds them and skips the
FetchContent source build, keeping image build times short.
The container uses a two-stage build. The first stage pulls prebuilt
`libllama`, `libggml`, and backend plugin libraries (including `libggml-cuda.so`
and the CPU variant plugins) from `ghcr.io/ggml-org/llama.cpp:full-cuda`. The
second stage copies those libraries into `/usr/local/lib` and runs `ldconfig` so
the dynamic linker and `dlopen` calls from `ggml_backend_load_all()` can resolve
the CUDA backend plugin at runtime. llama.cpp headers are cloned at the matching
tag and installed into `/usr/local/include`. CMake auto-detects both and skips
the FetchContent source build entirely, keeping image build times short.
The runtime stage copies the compiled binary, the same prebuilt shared
libraries, and config/prompt assets into a slim CUDA runtime image. It sets
`LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH` so the dynamic linker
resolves `libllama`/`libggml` at startup, and also co-locates
`libggml-cuda.so` and the CPU backend plugins next to the binary for
`ggml_backend_load_all()`'s `dlopen` scan.
`GGML_BACKEND_PATH` is set to `/usr/local/lib` so llama.cpp knows where to scan
for backend plugins.
### Build the image
@@ -173,46 +165,44 @@ docker build \
Look for `[biergarten] Found system llama.cpp — skipping FetchContent` in the
output to confirm the fast path was taken.
### Run the container
### Run in mocked mode
The container always runs the model-backed path; there is no `--mocked`
container mode (use a native build for that — see [Quick Start](#quick-start)).
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if
it is not already present at the configured path.
No model or GPU required. Useful for validating the pipeline logic and SQLite
export path.
```bash
docker run --rm \
-e BIERGARTEN_MODE=mocked \
-v "$PWD/output:/workspace/output" \
-v "$PWD/logs:/workspace/logs" \
biergarten-pipeline:latest
```
### Run in live mode
Mount your GGUF model before starting. The container validates the model path
before launching the binary.
```bash
docker run --rm \
--runtime=nvidia \
-e BIERGARTEN_MODE=live \
-e GGML_BACKEND_PATH="/usr/local/lib/libggml-cuda.so" \
-v "$PWD/models:/workspace/models" \
-v "$PWD/output:/workspace/output" \
-v "$PWD/logs:/workspace/logs" \
biergarten-pipeline:latest
```
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to
`./models/` on first run if it isn't already there. To use a pre-downloaded
model, place it at that path first — see [Model](#model) above.
#### Environment variables
| Variable | Purpose |
| ------------------------- | ----------------------------------------------------------------- |
| `BIERGARTEN_MODEL_PATH` | GGUF model path. Default: `/workspace/models/google_gemma-4-E4B-it-Q6_K.gguf`. |
| `BIERGARTEN_OUTPUT_DIR` | SQLite output directory. Default: `/workspace/output`. |
| `BIERGARTEN_LOG_PATH` | Log file path. Default: `/workspace/logs/pipeline.log`. |
| `BIERGARTEN_GL_LAYERS` | GPU layers to offload (`--n-gpu-layers`). Default: `40`. |
| `BIERGARTEN_TEMPERATURE`, `BIERGARTEN_TOP_P`, `BIERGARTEN_TOP_K`, `BIERGARTEN_N_CTX`, `BIERGARTEN_SEED` | Optional sampling overrides, unset by default (binary defaults apply). |
| `BIERGARTEN_EXTRA_ARGS` | Additional raw CLI args appended verbatim. |
`--prompt-dir` is hardcoded to `/app/prompts` inside the container and is not
configurable via environment variable.
The model must be present at `./models/google_gemma-4-E4B-it-Q6_K.gguf` on the
host. See [Model](#model) above for the download command.
### RunPod deployment
Use a GPU pod template. Mount persistent storage for `/workspace/models`,
`/workspace/output`, and `/workspace/logs`. See
`tooling/pipeline/runpod/pod-template.yaml` for a starter template — set the
environment variables listed above to match your run.
`/workspace/output`, and `/workspace/logs`. Set `BIERGARTEN_MODE=live` in the
template environment. See `tooling/pipeline/runpod/pod-template.yaml` for a
starter template.
---
@@ -220,77 +210,52 @@ environment variables listed above to match your run.
### Pipeline Stages
| Stage | Implementation |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Load | `ICuratedDataService` (`CuratedJsonDataService`) reads `locations.json`, `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json` (paths supplied via a `CuratedDataFilePaths` DTO at construction) into typed records, caching each after its first load. `--mocked` runs use `MockCuratedDataService`'s fixed in-memory dataset instead. |
| Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). |
| Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. |
| Generate Users | `GenerateUsers()` samples a persona and a forename/surname pair per enriched city (skipping countries with no name data), then `MockGenerator` or `LlamaGenerator` produces a username, bio, and activity weight around the sampled name. |
| Generate Breweries | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. |
| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `locations`, `users`, and `breweries` tables. |
| Log | `spdlog` writes results and warnings to the console. |
| Stage | Implementation |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| Load | `JsonLoader::LoadLocations()` reads `locations.json` into typed `Location` records. |
| Sample | `BiergartenDataGenerator::QueryCitiesWithCountries()` samples up to 50 locations per run. |
| Enrich | `WikipediaService` fetches city and beer context. Keeps going when a lookup fails. |
| Generate | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. |
| Store | `SqliteExportService` writes each successful brewery into a fresh dated `.sqlite` database with normalized location and brewery tables. |
| Log | `spdlog` writes results and warnings to the console. |
If name sampling, enrichment, or generation fails for a city, that city is
skipped and the pipeline continues. `GenerateUsers()` runs before
`GenerateBreweries()` in `BiergartenPipelineOrchestrator::Run()`.
If enrichment or generation fails for a city, that city is skipped and the
pipeline continues.
### Key Components
- `src/main.cc` — argument parsing and Boost.DI composition root.
- `CuratedJsonDataService` — implements `ICuratedDataService`; takes a
`CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in
its constructor, then parses and validates curated location, persona, and
forename/surname JSON, memoizing each result after its first load on a
given instance. `MockCuratedDataService` is the in-memory substitute (4
fixed locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in
`--mocked` runs.
- `WikipediaEnrichmentService` — queries Wikipedia extracts, caches results,
returns empty context on failure. `MockEnrichmentService` is the no-op
substitute used in `--mocked` runs.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for
both `GenerateBrewery` and `GenerateUser`, retries malformed responses up
to three times with corrective feedback in the retry prompt. The token
budget is fixed across attempts; it is not raised automatically on
truncation.
- `MockGenerator` — stable hash-based output so the same city/persona/name
input always produces the same brewery or user.
- `SqliteExportService` — creates a dated SQLite file per run and persists
each successful user and brewery into normalized tables.
- `JsonLoader` — validates curated location input.
- `WikipediaService` — queries Wikipedia extracts, caches results, returns empty
context on failure.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output, retries
malformed responses up to three times. If output looks truncated, the retry
raises the token budget before trying again.
- `MockGenerator` — stable hash-based output so the same city input always
produces the same brewery.
- `SqliteExportService` — creates a dated SQLite file per run and persists each
successful brewery into normalized tables.
- Brewery payloads include English and local-language name and description
fields. User payloads carry a sampled first/last name and gender, an
LLM-generated username/bio/activity weight, and a programmatically
generated (not LLM-authored) unique email and date of birth.
fields.
### Runtime Behaviour
`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a
generic "brewing" extract and a "beer in `{country}`" extract. It does not
currently query a city- or region-specific page. Each query string is cached
after its first successful (or empty) lookup.
`WikipediaService` queries city, country, and beer-related Wikipedia extracts
using its configured lookup, then caches the first successful response per query
string. The fetched extract text is included in the prompt as context for
generation.
`GetLocationContext()` returns an empty string when the web client is
unavailable or when lookup/parsing fails.
`LlamaGenerator` validates model output as structured JSON. On validation
failure it retries up to three times, replaying the previous error message in
the next prompt so the model can self-correct. All runs to date have produced
valid output on the first pass; the retry path is kept for resilience.
`LlamaGenerator` validates model output as structured JSON. The retry path
exists as a safety hatch for cases where the reasoning block consumes available
token budget and compresses the JSON output space. All runs to date have
produced valid output on the first pass; the path is kept for resilience.
`MockGenerator` uses stable hashes for repeatable output in demos and Storybook
runs.
`CuratedJsonDataService` memoizes each of `LoadLocations()`, `LoadPersonas()`,
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the
first time each is called, since `BiergartenPipelineOrchestrator` owns a
single `ICuratedDataService` instance for the whole run — later calls return
the cached result instead of re-parsing.
`GenerateUsers()` samples a forename/surname pair per city via `SampleName()`,
keyed by the city's ISO 3166-1 code. Countries present in `locations.json`
but absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`,
`ZA`) are skipped, the same way a failed enrichment or generation call skips
a city — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
### Process Flow - Activity Diagram
![An activity diagram](./diagrams/current/output/activity.svg)
@@ -303,10 +268,9 @@ a city — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
## Generated Output
Each successful run stores a `BreweryRecord` pair with the source location
and a `BreweryResult` payload, and a `UserRecord` pair with the source
location and a `UserResult` payload. The same generated records are also
written to a fresh SQLite export file named with the current UTC timestamp.
Each successful run stores a `GeneratedBrewery` pair with the source location
and a `BreweryResult` payload. The same generated records are also written to a
fresh SQLite export file named with the current UTC timestamp.
| Field | Meaning |
| ------------------- | ------------------------------------------ |
@@ -315,17 +279,6 @@ written to a fresh SQLite export file named with the current UTC timestamp.
| `name_local` | Brewery name in the local language. |
| `description_local` | Brewery description in the local language. |
| Field | Meaning |
| ----------------- | ----------------------------------------------------------------- |
| `first_name` | Sampled forename, copied from the curated name data (not LLM-invented). |
| `last_name` | Sampled surname, copied from the curated name data (not LLM-invented). |
| `gender` | Gender associated with the sampled forename in the source dataset. |
| `username` | LLM-generated handle. |
| `bio` | LLM-generated short biography. |
| `activity_weight` | Relative check-in/activity weight, reserved for a future J-curve activity profile. |
| `email` | Unique `@thebiergarten.app` address, generated programmatically from the sampled name. |
| `date_of_birth` | Randomized date of birth (age 19-48), generated programmatically. |
The log dump also includes city, country, state or province, ISO subdivision
code, latitude, and longitude for each entry.
@@ -339,6 +292,7 @@ code, latitude, and longitude for each entry.
| `local_languages` | Locale-aware copy selection |
| `name_en`, `description_en` | Default English display content |
| `name_local`, `description_local` | Local-language display content |
| `region_context` | Richer copy for cards and detail pages |
---
@@ -404,64 +358,47 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
## Fixture Strategy
- `--mocked` for stable fixtures, repeatable screenshots, and Storybook runs.
`MockCuratedDataService` swaps in for `CuratedJsonDataService`, so no
fixture files need to be present on disk.
- `--model` when geographically grounded content matters for demos.
- Keep `locations.json` structured enough to support discovery and future
filtering.
- `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` are curated/vendored fixture data, not
LLM-generated — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset
section for provenance.
- Treat SQLite output as seed material for the app's brewery and user
domains, not production data.
- Treat SQLite output as seed material for the app's brewery domain, not
production data.
---
## Repo Layout
| Path | Purpose |
| -------------------------------------- | -------------------------------------------------- |
| `tooling/pipeline/includes/` | Public headers and shared models. |
| `tooling/pipeline/src/` | Implementation files. |
| `tooling/pipeline/locations.json` | Curated city input copied into the build tree. |
| `tooling/pipeline/personas.json` | Curated user persona archetypes copied into the build tree. |
| `tooling/pipeline/forenames-by-country.json` | Vendored (CC0) forename data by ISO 3166-1 country code. |
| `tooling/pipeline/surnames-by-country.json` | Vendored (CC0) surname data by ISO 3166-1 country code. |
| `tooling/pipeline/prompts/` | System prompts used by the model-backed path. |
| `tooling/pipeline/runpod/` | Dockerfile, launcher, and RunPod pod template. |
| `docs/pipeline/diagrams/` | Architecture and pipeline diagrams. |
| `docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md` | Ethics, bias, hallucination analysis, mitigations. |
| Path | Purpose |
| ---------------------------- | -------------------------------------------------- |
| `includes/` | Public headers and shared models. |
| `src/` | Implementation files. |
| `locations.json` | Curated city input copied into the build tree. |
| `prompts/` | System prompts used by the model-backed path. |
| `diagrams/` | Architecture and pipeline diagrams. |
| `tooling/pipeline/runpod/` | Dockerfile, launcher, and RunPod pod template. |
| `ETHICS-AND-KNOWN-ISSUES.md` | Ethics, bias, hallucination analysis, mitigations. |
---
## Code Tour
Paths below are relative to `tooling/pipeline/`.
- `src/main.cc` — argument parsing and DI composition root.
- `src/biergarten_pipeline_orchestrator/` — orchestration, sampling, logging,
and export.
- `src/services/curated_data/``CuratedJsonDataService`, the file-backed
`ICuratedDataService`, and `MockCuratedDataService`, the in-memory
`ICuratedDataService` used in `--mocked` runs.
- `src/services/enrichment/wikipedia/` — enrichment service and cache.
- `src/biergarten_data_generator/` — orchestration, sampling, logging, and
export.
- `src/services/wikipedia/` — enrichment service and cache.
- `src/services/sqlite/` — SQLite export implementation.
- `src/data_generation/llama/` — local inference, prompt loading, output
validation.
- `src/data_generation/mock/` — deterministic fallback.
- `runpod/` — container build and runtime launcher.
- `tooling/pipeline/runpod/` — container build and runtime launcher.
---
## Next Steps
The pipeline currently produces city-aware brewery and user records and
dated SQLite exports. The next passes add additional fixture types so the
app can exercise the full brewery and social domains without live data. For
the detailed engineering breakdown of what's needed to reach the
architecture in [`diagrams/planned/`](./diagrams/planned/), see
[ROADMAP.md](./ROADMAP.md).
The pipeline currently produces city-aware brewery records and dated SQLite
exports. The next passes add additional fixture types so the app can exercise
the full brewery domain without live data.
### Testing — Very High Priority
@@ -483,6 +420,12 @@ Generate catalog entries with style, ABV, IBU, color, aroma notes, and food
pairing hints. Link beers back to breweries and cities. Keep style coverage wide
enough to exercise search, sort, and category filters.
### User Generation
Generate user profiles with stable names, bios, locale hints, and preference
signals. Include stable IDs for downstream fixture joins. Keep output
deterministic for screenshots while allowing larger randomized batches.
### Check-In System
Produce timestamped check-in events between users and breweries. Use a J-curve

View File

@@ -1,252 +0,0 @@
# Pipeline Roadmap — Reaching the Planned Architecture
This is the engineering breakdown for closing the gap between the **current**
pipeline and the **planned** architecture documented in
[`diagrams/planned/class.puml`](./diagrams/planned/class.puml) and
[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing
in `diagrams/planned/` is implemented yet — this file tracks what it would
take to get there. For the current implementation, see
[README.md](./README.md).
Items are grouped by layer and roughly ordered by dependency: later groups
build on types and services introduced earlier.
The only concurrency in the current pipeline is the log dispatcher thread
(`main.cc` spawns it to drain a `BoundedChannel<LogEntry>` into spdlog for the
whole run). Sampling, enrichment, generation, and export all happen
synchronously on the main thread inside `BiergartenPipelineOrchestrator::Run()`
— §6 below is what introduces the additional worker threads the planned
architecture needs.
---
## 1. Domain Models
`tooling/pipeline/includes/data_model/generated_models.h` and `models.h`.
- [ ] Add `Completeness` enum (`Full`, `Partial`, `Absent`) and a
`LocationContext` struct (`text`, `completeness`, `char_count`).
Replace `EnrichedCity::region_context` (currently a plain
`std::string`) with `context : LocationContext`.
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`,
`min_ibu`, `max_ibu`).
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads.
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
`context_provided`, `generated_with`).
- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to
`UserResult` (currently just `username`, `bio`). `first_name`/
`last_name`/`gender` are copied from the sampled `Name` (see below),
not LLM-invented.
- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result
handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`,
`gender`). Landed as a flatter shape than originally planned here: no
`NamesByCountry` wrapper class. `curated_data_service.h` instead
declares `ForenameList = std::vector<ForenameEntry>` and
`SurnameList = std::vector<std::string>`, and
`ICuratedDataService` exposes two maps keyed by ISO 3166-1 code —
`ForenamesByCountryMap = unordered_map<string, ForenameList>` and
`SurnamesByCountryMap = unordered_map<string, SurnameList>`
directly (see §2). Sampling is
a free function, `SampleName(forenames_by_country,
surnames_by_country, iso3166_1, rng)`, in
`generate_users.cc`'s anonymous namespace, not a method on a class.
Pairing a forename with a surname happens at sample time so gender
(present per-forename in the source data) is never discarded the way a
pre-flattened `{first_name, last_name}` list would lose it; see §2.
- [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`,
`metadata` (currently just `{location, brewery}`).
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
`GeneratedFollow` aggregate structs.
- [x] Add `UserRecord` (`location`, `user : UserResult`, `email`,
`date_of_birth`) as the current stand-in for the planned
`GeneratedUser``email` and `date_of_birth` are programmatically
generated by the orchestrator, never LLM-authored, so a downstream
auth-account seeding consumer can register real accounts from the
pipeline's SQLite export. No `password`, `user_id`, or `metadata`
field yet, matching `BreweryRecord`'s equally pre-`metadata` shape.
Already wired into `IExportService`/`SqliteExportService`: a `users`
table exists and `GenerateUsers()` exports each successful record via
`ProcessRecord(const UserRecord&)` (see §5) in addition to logging and
holding `generated_users_` in memory. Still missing vs. the planned
`GeneratedUser`: `user_id`, `password`, and `metadata`.
- [x] Add `UserPersona` (`name`, `description`, `style_affinities`).
## 2. Data Preloading
`tooling/pipeline/includes/services/curated_data/`, fixture files.
- [x] Extract an interface; have `CuratedJsonDataService` implement it.
Landed as `ICuratedDataService`
(`services/curated_data/curated_data_service.h`), not `DataPreloader`
`LoadLocations()`, `LoadPersonas()`, `LoadForenamesByCountry()`, and
`LoadSurnamesByCountry()` are all virtual methods returning `const&`
and take no arguments. `CuratedJsonDataService`
(`services/curated_data/curated_json_data_service.h`, originally
landed as `JsonLoader` under `json_handling/` before being renamed and
moved) takes a `CuratedDataFilePaths` DTO (`locations_path`,
`personas_path`, `forenames_path`, `surnames_path`) in its
constructor rather than a path per `Load*()` call, and memoizes each
result in a private `cache` struct (`locations`, `personas`,
`forenames_by_country`, `surnames_by_country`), so a second call on
the same instance returns the cached result instead of re-parsing —
safe because `CuratedJsonDataService` outlives every call site (owned
by `BiergartenPipelineOrchestrator` via
`unique_ptr<ICuratedDataService>` for the whole run). `main.cc`'s DI
injector also gained a `MockCuratedDataService` (fixed in-memory
data: 4 locations across `US`/`DE`/`FR`/`BE`, 3 personas, and
matching forename/surname sets for those four countries), bound in
place of `CuratedJsonDataService` under `--mocked`, mirroring the
existing `MockEnrichmentService`/`MockGenerator` pattern.
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo
and is already copied into the Docker image
(`runpod/Dockerfile`), but no loader reads it yet, and the native
CMake build doesn't copy it into `build/` at all — `CMakeLists.txt`'s
"Runtime Assets" step only copies `locations.json` and `prompts/`.
- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
- [x] Add name-by-country loading, parsing
`tooling/pipeline/forenames-by-country.json` and
`surnames-by-country.json`. Landed as two separate methods —
`LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each
returning a flat `ForenamesByCountryMap` / `SurnamesByCountryMap`
(aliases for `unordered_map<string, ForenameList>` /
`unordered_map<string, SurnameList>`) keyed by ISO 3166-1 code, rather
than one combined `LoadNamesByCountry() : NamesByCountry` call. Both
files are vendored verbatim (unmodified,
full multinational coverage) from
`sigpwned/popular-names-by-country-dataset` (CC0) — see
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for
provenance. Deliberately not pre-paired or filtered down to just the
countries in `locations.json`: keeping the source shape (including
per-forename gender) intact means the loader can support more
countries later, or gender-aware persona/bio generation, without
re-sourcing data.
## 3. Policy / Strategy Layer
Entirely new — no `includes/policy/` (or equivalent) directory exists today.
- [ ] `ContextStrategy` interface + `BreweryContextStrategy` /
`BeerContextStrategy`. Today `WikipediaEnrichmentService::GetLocationContext`
hardcodes a generic `"brewing"` query and a `"beer in {country}"`
query directly — no per-phase strategy selection.
- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing
the inline `std::ranges::sample(...)` call in
`BiergartenPipelineOrchestrator::QueryCitiesWithCountries()`.
- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to
pick styles per brewery from the `BeerStyle` palette (depends on §1
and §2).
- [ ] `CheckinDistributionStrategy` interface + `JCurveCheckinStrategy` /
`RandomCheckinStrategy` — this is the "Check-In System" item already
called out in README.md's Next Steps, made concrete.
- [ ] `FollowGenerationStrategy` interface + `RandomFollowStrategy` /
`ActivityWeightedFollowStrategy`.
## 4. Data Generation
`tooling/pipeline/includes/data_generation/`, `src/data_generation/`.
- [ ] Extend the `DataGenerator` interface with `GenerateBeer`,
`GenerateCheckin`, `GenerateRating` (today: only `GenerateBrewery` and
`GenerateUser`).
- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop
mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3
attempts with corrective feedback) and a new
`prompts/USER_GENERATION.md`. Changed the `DataGenerator::GenerateUser`
signature to
`(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`,
matching what the activity diagram actually passes. `MockGenerator::GenerateUser`
updated to match the new signature too.
- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` /
`GenerateRating`.
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
`LlamaGenerator::ValidateModelArchitecture()` so loading a GGUF that
doesn't match the configured chat template fails fast instead of
silently producing degraded output.
- [ ] Add prompt template files for beer/checkin/rating generation next to
the existing `prompts/BREWERY_GENERATION.md`.
## 5. Export Service
`tooling/pipeline/includes/services/database/`, `src/services/sqlite/`.
- [x] `IExportService::ProcessRecord(const UserRecord&)` and
`SqliteExportService`'s `users` table (see
`kCreateUsersTableSql` / `kInsertUserSql` in
`includes/services/database/sqlite_statement_helpers.h`) are
implemented — `GenerateUsers()` exports every successful user
alongside its resolved `location_id`.
- [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`,
`ProcessRating`, `ProcessFollow` (today `ProcessRecord(const
BreweryRecord&)` and `ProcessRecord(const UserRecord&)` exist).
- [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to
`SqliteExportService::InitializeSchema()` (`locations`, `breweries`,
and `users` already exist) — see `kCreateLocationsTableSql` /
`kCreateBreweriesTableSql` / `kCreateUsersTableSql` in
`includes/services/database/sqlite_statement_helpers.h` for the
existing pattern to follow.
- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and
move from one open transaction for the whole run to per-phase batched
commits (`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as
shown in the planned activity diagram.
## 6. Concurrency & Orchestration
`tooling/pipeline/includes/concurrency/`,
`src/biergarten_pipeline_orchestrator/`.
- [ ] `BoundedChannel<T>` already exists and is production-tested — but
it's only wired to the single log channel today. Stand up the
per-phase producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the
planned activity diagram describes, each with a dedicated LLM worker
thread and SQLite worker thread.
- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous
pass over `generated_breweries_` into phased methods —
`RunUserPhase``RunBreweryAndBeerPhase` (with a `RunBeerPhase`
sub-step once `brewery_pool_` is populated) → `RunCheckinPhase` /
`RunFollowPhase` (these two can run in parallel) → `RunRatingPhase`
backed by `user_pool_`, `brewery_pool_`, `beer_pool_`,
`checkin_pool_`, `follow_pool_`.
- [ ] Honor the structural-concurrency requirement already called out in a
comment on `BiergartenPipelineOrchestrator::Run()` in
`biergarten_pipeline_orchestrator.h`: once real worker threads exist,
they must be structurally joined (e.g. via `std::jthread`) before
`Run()` returns, so no worker logs to a closed channel during
teardown.
## 7. Enrichment
- [ ] Decide whether to restore the city/region-specific Wikipedia query
that's currently commented out in
`src/services/enrichment/wikipedia/get_summary.cc`
(`GetLocationContext`). The `ContextStrategy` work in §3 is a natural
place to reintroduce it via `BreweryContextStrategy::QueriesFor()`.
- [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`,
`PreWarmLocationCache` in the planned activity diagram) instead of
fetching lazily per record, so the streaming phases never block on a
cold cache mid-run.
## 8. Fixtures / Build
- [x] Author `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` (see §2).
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime
Assets" `configure_file` step in `CMakeLists.txt` so native builds and
the Docker image agree on what's available at runtime.
---
## Suggested build order
The planned activity diagram's own phase notes ("brewery*pool* is now fully
populated. Phase 1b may begin.", etc.) imply a dependency order. Roughly:
1. Domain models (§1) and data preloading (§2) — nothing else compiles
without these.
2. Export schema (§5) — so every later generation phase has somewhere to
land its output.
3. Policy/strategy layer (§3) and generator interface (§4).
4. Concurrency/orchestration rewrite (§6), which is the only piece that
actually wires §1§5 into the phased, parallel pipeline shown in the
planned diagrams.
5. Enrichment cache pre-warming and the city/region query decision (§7) —
useful at any point, but most valuable once phases run concurrently.

View File

@@ -18,40 +18,23 @@ skinparam ActivityBarColor #628A5B
skinparam SwimlaneBorderColor #547461
skinparam SwimlaneBorderThickness 0.3
title The Biergarten Data Pipeline (Current — Synchronous Data Path)
title The Biergarten Data Pipeline (Streaming Architecture)
|#F2F6F0|main.cc|
start
:Create BoundedChannel<LogEntry> log_channel;
:Spawn log dispatcher thread\n(LogDispatcher::Run());
note right
The only concurrency in the current pipeline:
a dedicated thread drains log_channel and
forwards entries to spdlog for the entire run.
Joined during shutdown after log_channel closes.
end note
:ParseArguments(argc, argv);
if (Are arguments valid?) then (no)
:Log error / usage info;
:spdlog::error usage info;
stop
else (yes)
endif
:Init OpenSSL global state & LlamaBackendState;
:di::make_injector(...);
note right
Binds ILogger, IEnrichmentService, DataGenerator,
IExportService, IPromptFormatter, WebClient based
on --mocked vs. model-backed mode.
end note
:injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>();
:BiergartenPipelineOrchestrator::Run();
note right
Run() itself is synchronous — sampling, enrichment,
generation, and export all happen on this thread.
end note
:injector.create<std::unique_ptr<BiergartenDataGenerator>>();
:BiergartenDataGenerator::Run();
|#EAF0E8|BiergartenPipelineOrchestrator|
|#EAF0E8|BiergartenDataGenerator|
:Initialize SQLite export;
|#E0EAE0|SqliteExportService|
@@ -65,106 +48,23 @@ note right
Begins Transaction
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
|#EAF0E8|BiergartenDataGenerator|
:QueryCitiesWithCountries();
|#E2EBDC|JsonLoader|
:JsonLoader::LoadLocations("locations.json");
:std::ranges::sample(all_locations, --location-count);
note right
--location-count defaults to 10.
end note
:std::ranges::sample(all_locations, 50);
|#EAF0E8|BiergartenPipelineOrchestrator|
|#EAF0E8|BiergartenDataGenerator|
while (For each sampled Location?) is (Remaining cities)
|#DCE8D8|IEnrichmentService|
|#DCE8D8|WikipediaService|
:GetLocationContext(loc);
note right
WikipediaEnrichmentService fetches a generic
"brewing" extract and a "beer in {country}" extract
(not a city/region-specific page). MockEnrichmentService
(--mocked) returns an empty string instead.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
if (Lookup failed?) then (yes)
:Log warning, skip city;
else (no)
:Store EnrichedCity{Location, region_context};
endif
:FetchExtracts(City, Country, Beer);
|#EAF0E8|BiergartenDataGenerator|
:Store EnrichedCity{Location, region_context};
endwhile (Done)
|#EAF0E8|BiergartenPipelineOrchestrator|
:GenerateUsers(enriched_cities);
|#E2EBDC|JsonLoader|
:LoadPersonas("personas.json");
:LoadForenamesByCountry("forenames-by-country.json");
:LoadSurnamesByCountry("surnames-by-country.json");
note right
Each call is cached after its first parse.
MockCuratedDataService (--mocked) returns a
fixed in-memory dataset instead and skips
these three JSON files entirely.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
while (For each EnrichedCity?) is (Remaining cities)
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1);
if (Names available for country?) then (no)
:Log warning, skip city;
else (yes)
:Sample UserPersona (uniform random);
|#E5EDE1|DataGenerator|
if (Generator Mode) then (MockGenerator)
:DeterministicHash & Format;
else (LlamaGenerator)
:prompt_directory_->Load("USER_GENERATION");
note right
Resolves to USER_GENERATION.md inside --prompt-dir.
end note
repeat
:Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
:ValidateUserJson(raw, user);
if (Is JSON Valid?) then (yes)
break
else (no)
:Attempt++;
:Append validation error to retry prompt;
endif
repeat while (Attempt < 3?) is (yes)
endif
|#EAF0E8|BiergartenPipelineOrchestrator|
if (Generation successful?) then (yes)
:BuildEmail(name, used_email_local_parts);
:GenerateDateOfBirth(rng);
note right
email and date_of_birth are generated
programmatically, never LLM-authored.
end note
|#E0EAE0|SqliteExportService|
:ProcessRecord(UserRecord);
if (Location in cache?) then (yes)
:Reuse location_id;
else (no)
:Insert Location & Cache ID;
endif
:Insert User (FK: location_id);
if (Exception caught during insert?) then (yes)
|#EAF0E8|BiergartenPipelineOrchestrator|
:Log warning "SQLite export failed";
else (no)
endif
else (no)
:Log warning "Generation failed, skipping...";
endif
endif
endwhile (Done)
|#EAF0E8|BiergartenPipelineOrchestrator|
|#EAF0E8|BiergartenDataGenerator|
:GenerateBreweries(enriched_cities);
|#E5EDE1|DataGenerator|
@@ -173,10 +73,7 @@ while (For each EnrichedCity?) is (Remaining cities)
:DeterministicHash & Format;
else (LlamaGenerator)
:PrepareRegionContext;
:prompt_directory_->Load("BREWERY_GENERATION");
note right
Resolves to BREWERY_GENERATION.md inside --prompt-dir.
end note
:LoadBrewerySystemPrompt("prompts/system.md");
repeat
:Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar);
:ValidateBreweryJson(raw, brewery);
@@ -184,16 +81,11 @@ while (For each EnrichedCity?) is (Remaining cities)
break
else (no)
:Attempt++;
:Append validation error to retry prompt;
endif
repeat while (Attempt < 3?) is (yes)
note right
max_tokens is fixed across retries —
it is not raised on truncation.
end note
endif
|#EAF0E8|BiergartenPipelineOrchestrator|
|#EAF0E8|BiergartenDataGenerator|
if (Generation successful?) then (yes)
|#E0EAE0|SqliteExportService|
:ProcessRecord(GeneratedBrewery);
@@ -205,8 +97,8 @@ while (For each EnrichedCity?) is (Remaining cities)
:Insert Brewery (FK: location_id);
if (Exception caught during insert?) then (yes)
|#EAF0E8|BiergartenPipelineOrchestrator|
:Log warning "SQLite export failed";
|#EAF0E8|BiergartenDataGenerator|
:spdlog::warn "Failed to stream record to SQLite export";
note right
Data loss is prevented per-record.
The pipeline continues running.
@@ -214,7 +106,7 @@ while (For each EnrichedCity?) is (Remaining cities)
else (no)
endif
else (no)
:Log warning "Generation failed, skipping...";
:spdlog::warn "Generation failed, skipping...";
endif
|#E5EDE1|DataGenerator|
endwhile (Done)
@@ -227,8 +119,6 @@ note right
end note
|#F2F6F0|main.cc|
:Close log_channel;
:Join log dispatcher thread;
:Return 0;
stop

View File

@@ -25,79 +25,22 @@ skinparam note {
title The Biergarten Data Pipeline - Class Diagram
class BiergartenPipelineOrchestrator {
- logger_ : std::shared_ptr<ILogger>
class BiergartenDataGenerator {
- context_service_ : std::unique_ptr<IEnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- exporter_ : std::unique_ptr<IExportService>
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
- application_options_ : ApplicationOptions
- generated_breweries_ : std::vector<BreweryRecord>
- generated_users_ : std::vector<UserRecord>
- generated_breweries_ : std::vector<GeneratedBrewery>
+ Run() : bool
- QueryCitiesWithCountries() : std::vector<Location>
- GenerateBreweries(cities : std::span<const EnrichedCity>) : void
- GenerateUsers(cities : std::span<const EnrichedCity>) : void
- LogResults() : void
}
class LogLevel <<enumeration>> {
Debug
Info
Warn
Error
}
class PipelinePhase <<enumeration>> {
Startup
UserGeneration
BreweryAndBeerGeneration
CheckinGeneration
RatingGeneration
FollowGeneration
Teardown
}
struct LogDTO {
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
}
struct LogEntry {
+ timestamp : std::chrono::system_clock::time_point
+ origin : std::source_location
+ thread_id : std::thread::id
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
}
interface ILogger <<interface>> {
+ Log(payload : LogDTO) : void
- {abstract} DoLog(entry : LogEntry) : void
}
class LogProducer {
- channel_ : BoundedChannel<LogEntry>&
- DoLog(entry : LogEntry) : void
}
class LogDispatcher {
- channel_ : BoundedChannel<LogEntry>&
+ Run() : void
- ToSpdlogLevel(level) : spdlog::level::level_enum
}
interface IEnrichmentService <<interface>> {
+ GetLocationContext(loc : const Location&) : std::string
}
class MockEnrichmentService {
+ GetLocationContext(loc : const Location&) : std::string
}
class WikipediaEnrichmentService {
class WikipediaService {
- client_ : std::unique_ptr<WebClient>
- extract_cache_ : std::unordered_map<std::string, std::string>
+ GetLocationContext(loc : const Location&) : std::string
@@ -116,27 +59,26 @@ class HttpWebClient {
interface DataGenerator <<interface>> {
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
+ GenerateUser(locale : const std::string&) : UserResult
}
class MockGenerator {
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
+ GenerateBrewery(...) : BreweryResult
+ GenerateUser(...) : UserResult
- DeterministicHash(location : const Location&) : size_t
- DeterministicHash(location : const Location&, persona : const UserPersona&, name : const Name&) : size_t
}
class LlamaGenerator {
- model_ : ModelHandle
- context_ : ContextHandle
- prompt_formatter_ : std::unique_ptr<IPromptFormatter>
- prompt_directory_ : std::unique_ptr<IPromptDirectory>
- rng_ : std::mt19937
+ GenerateBrewery(...) : BreweryResult
+ GenerateUser(...) : UserResult
- Load(model_path : const std::string&) : void
- Infer(...) : std::string
- InferFormatted(...) : std::string
- LoadBrewerySystemPrompt(...) : std::string
}
interface IPromptFormatter <<interface>> {
@@ -147,59 +89,13 @@ class Gemma4JinjaPromptFormatter {
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
}
interface IPromptDirectory <<interface>> {
+ Load(key : std::string_view) : std::string
}
class PromptDirectory {
- prompt_dir_ : std::filesystem::path
- cache_ : std::unordered_map<std::string, std::string>
+ Load(key : std::string_view) : std::string
}
interface ICuratedDataService <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
class JsonLoader {
- cache_ : cache
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
+ {static} LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
}
note right of JsonLoader
Each Load* method memoizes its result in
cache_ on first call; later calls on the
same instance return the cached value
without re-parsing.
end note
class MockCuratedDataService {
- locations_ : std::vector<Location>
- personas_ : std::vector<UserPersona>
- forenames_by_country_ : std::unordered_map<std::string, forename_list>
- surnames_by_country_ : std::unordered_map<std::string, surname_list>
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
note right of MockCuratedDataService
Fixed 4-location / 3-persona dataset
(US, DE, FR, BE) used in --mocked runs;
filepath arguments are ignored.
end note
interface IExportService <<interface>> {
+ Initialize() : void
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
+ ProcessRecord(user : const UserRecord&) : uint64_t
+ ProcessRecord(brewery : const GeneratedBrewery&) : void
+ Finalize() : void
}
@@ -207,18 +103,15 @@ class SqliteExportService {
- date_time_provider_ : std::unique_ptr<IDateTimeProvider>
- run_timestamp_utc_ : std::string
- database_path_ : std::filesystem::path
- db_handle_ : SqliteDatabaseHandle
- insert_location_stmt_ : SqliteStatementHandle
- insert_brewery_stmt_ : SqliteStatementHandle
- insert_user_stmt_ : SqliteStatementHandle
- db_handle_ : sqlite3*
- insert_location_stmt_ : sqlite3_stmt*
- insert_brewery_stmt_ : sqlite3_stmt*
- transaction_open_ : bool
- location_cache_ : std::unordered_map<std::string, sqlite3_int64>
+ Initialize() : void
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
+ ProcessRecord(user : const UserRecord&) : uint64_t
+ ProcessRecord(brewery : const GeneratedBrewery&) : void
+ Finalize() : void
- InitializeSchema() : void
- ResolveLocationId(location : const Location&) : sqlite3_int64
}
interface IDateTimeProvider <<interface>> {
@@ -230,23 +123,12 @@ class SystemDateTimeProvider {
}
' Structural Relationships / Dependency Injection
BiergartenPipelineOrchestrator *-- ILogger : owns
BiergartenPipelineOrchestrator *-- IEnrichmentService : owns
BiergartenPipelineOrchestrator *-- DataGenerator : owns
BiergartenPipelineOrchestrator *-- IExportService : owns
BiergartenPipelineOrchestrator *-- ICuratedDataService : owns
BiergartenDataGenerator *-- IEnrichmentService : owns
BiergartenDataGenerator *-- DataGenerator : owns
BiergartenDataGenerator *-- IExportService : owns
LogEntry *-- LogLevel
LogEntry *-- PipelinePhase
LogDTO *-- LogLevel
LogDTO *-- PipelinePhase
ILogger <|.. LogProducer : implements
LogProducer ..> LogEntry : emits
LogDispatcher ..> LogEntry : consumes
IEnrichmentService <|.. WikipediaEnrichmentService : implements
IEnrichmentService <|.. MockEnrichmentService : implements
WikipediaEnrichmentService *-- WebClient : owns
IEnrichmentService <|.. WikipediaService : implements
WikipediaService *-- WebClient : owns
WebClient <|.. HttpWebClient : implements
@@ -254,13 +136,10 @@ DataGenerator <|.. MockGenerator : implements
DataGenerator <|.. LlamaGenerator : implements
LlamaGenerator *-- IPromptFormatter : uses
LlamaGenerator *-- IPromptDirectory : uses
IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements
IPromptDirectory <|.. PromptDirectory : implements
ICuratedDataService <|.. JsonLoader : implements
ICuratedDataService <|.. MockCuratedDataService : implements
BiergartenDataGenerator ..> JsonLoader : uses
IExportService <|.. SqliteExportService : implements
SqliteExportService *-- IDateTimeProvider : owns

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 44 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 44 KiB

View File

@@ -43,18 +43,10 @@ fork again
:EnrichmentService::PreWarmLocationCache(sampled_locations);
end fork
fork
:JsonLoader::LoadForenamesByCountry(\n "forenames-by-country.json");
fork again
:JsonLoader::LoadSurnamesByCountry(\n "surnames-by-country.json");
:JsonLoader::LoadNamesByCountry("names-by-country.json");
fork again
:JsonLoader::LoadPersonas("personas.json");
end fork
note right
Each call is memoized after its first parse
(implemented today). MockCuratedDataService
returns a fixed in-memory dataset instead
under --mocked, skipping these JSON files.
end note
' ═══════════════════════════════════════════
' PHASE 0 — USER GENERATION
@@ -90,15 +82,11 @@ fork again
ibu_preference, checkin_weight.
end note
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1, rng);
:NamesByCountry::SampleName(\n location.iso3166_1);
note right
Deterministic lookup -- no LLM involved.
Free function (implemented today in
generate_users.cc), not a class method.
Name selected from pre-keyed maps
Name selected from pre-keyed table
and passed into the generation prompt.
Skips the city if either map has no
entry for iso3166_1.
end note
:GenerateUser(enriched_city, persona, sampled_name)\nvia DataGenerator;

View File

@@ -67,50 +67,11 @@ package "Domain: Models" {
}
class UserResult {
+ first_name : std::string
+ last_name : std::string
+ gender : std::string
+ username : std::string
+ bio : std::string
+ activity_weight : float
}
class Name {
+ first_name : std::string
+ last_name : std::string
+ gender : std::string
}
class ForenameEntry {
+ name : std::string
+ gender : std::string
}
note right of ForenameEntry
forename_list = std::unordered_set<ForenameEntry>
surname_list = std::unordered_set<std::string>
(aliases declared in curated_data_service.h).
ICuratedDataService::LoadForenamesByCountry() /
LoadSurnamesByCountry() each return a flat
std::unordered_map<std::string, forename_list>
or <std::string, surname_list> keyed by ISO
3166-1 country code -- no NamesByCountry
wrapper class. SampleName(forenames_by_country,
surnames_by_country, iso3166_1, rng) is a free
function, not a method, so it can be called by
any per-city worker without owning the maps.
Sourced from popular-names-by-country-dataset
(CC0) -- see ETHICS-AND-KNOWN-ISSUES.md. Gender
is preserved per forename so it can be threaded
through to persona/bio generation later; surnames
carry no gender in the source data. Pairing
happens at sample time, not at fixture-authoring
time, so no information from the source dataset
is discarded up front.
end note
ForenameEntry ..> Name : SampleName() produces
class CheckinResult {
+ checked_in_at : std::string
+ note : std::string
@@ -149,20 +110,9 @@ package "Domain: Models" {
+ user_id : uint64_t
+ location : Location
+ user : UserResult
+ email : std::string
+ date_of_birth : std::string
+ password : std::string
+ metadata : GenerationMetadata
}
note right of GeneratedUser
email, date_of_birth, and password are
programmatically generated by the orchestrator
(not LLM-authored) so a downstream auth-account
seeding consumer can register real accounts from
this export. See ROADMAP.md §1.
end note
class GeneratedCheckin {
+ checkin_id : uint64_t
+ user_id : uint64_t
@@ -325,29 +275,32 @@ package "Infrastructure: Logging" {
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
+ city : std::optional<std::string>
+ entity_id : std::optional<std::string>
+ worker : std::optional<std::string>
}
interface ILogger <<interface>> {
+ Log(entry : const LogEntry&) : void
interface Logger <<interface>> {
+ Log(level, phase, message,\n city, entity_id, worker) : void
}
class LogProducer {
- channel_ : BoundedChannel<LogEntry>&
+ Log(entry : const LogEntry&) : void
class PipelineLogger {
- log_ch_ : BoundedChannel<LogEntry>&
+ Log(level, phase, message,\n city, entity_id, worker) : void
}
class LogDispatcher {
- channel_ : BoundedChannel<LogEntry>&
class LogWorker {
- log_ch_ : BoundedChannel<LogEntry>&
+ Run() : void
- FormatTimestamp(tp) : std::string
- ToSpdlogLevel(level) : spdlog::level::level_enum
- ToString(phase) : std::string
}
LogEntry *-- LogLevel
LogEntry *-- PipelinePhase
ILogger <|.. LogProducer
LogProducer ..> LogEntry : emits
LogDispatcher ..> LogEntry : consumes
PipelineLogger ..> LogEntry : emits
LogWorker ..> LogEntry : consumes
}
package "Infrastructure: Pipeline Channel" {
@@ -368,43 +321,20 @@ package "Infrastructure: Pipeline Channel" {
package "Infrastructure: Data Preloading" {
interface ICuratedDataService <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
interface DataPreloader <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
}
class JsonLoader {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
}
note right of JsonLoader
Each Load* call is memoized after its first
parse -- implemented today, ahead of the rest
of this package (LoadBeerStyles is still planned).
end note
class MockCuratedDataService {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
note right of MockCuratedDataService
Fixed in-memory dataset used in --mocked mode,
implemented today for Locations/Personas/
Forenames/Surnames; would need a LoadBeerStyles
fixture too once that method is implemented.
end note
}
package "Infrastructure: Enrichment" {
@@ -453,7 +383,7 @@ package "Infrastructure: Data Generation" {
interface DataGenerator <<interface>> {
+ GenerateBrewery(location : const Location&,\n context : const LocationContext&) : BreweryResult
+ GenerateBeer(brewery_id : uint64_t,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult
+ GenerateUser(city : const EnrichedCity&,\n persona : const UserPersona&,\n name : const Name&) : UserResult
+ GenerateUser(location : const Location&) : UserResult
+ GenerateCheckin(user : const GeneratedUser&,\n brewery : const GeneratedBrewery&,\n timestamp : const std::string&) : CheckinResult
+ GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : uint64_t) : RatingResult
}
@@ -546,7 +476,7 @@ package "Infrastructure: Data Export" {
}
class BiergartenPipelineOrchestrator {
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
- preloader_ : std::unique_ptr<DataPreloader>
- enrichment_service_ : std::unique_ptr<EnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- logger_ : std::unique_ptr<Logger>
@@ -574,7 +504,7 @@ class BiergartenPipelineOrchestrator {
}
' --- Orchestration Aggregations (Services & Strategies) ---
BiergartenPipelineOrchestrator *-- ICuratedDataService
BiergartenPipelineOrchestrator *-- DataPreloader
BiergartenPipelineOrchestrator *-- EnrichmentService
BiergartenPipelineOrchestrator *-- DataGenerator
BiergartenPipelineOrchestrator *-- ExportService
@@ -593,8 +523,7 @@ BiergartenPipelineOrchestrator *-- "0..*" GeneratedCheckin : checkin_pool_
BiergartenPipelineOrchestrator *-- "0..*" GeneratedFollow : follow_pool_
' --- Interfaces & Implementations ---
ICuratedDataService <|.. JsonLoader
ICuratedDataService <|.. MockCuratedDataService
DataPreloader <|.. JsonLoader
Logger <|.. PipelineLogger
ContextStrategy <|.. BreweryContextStrategy
ContextStrategy <|.. BeerContextStrategy

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 142 KiB

After

Width:  |  Height:  |  Size: 106 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 240 KiB

After

Width:  |  Height:  |  Size: 184 KiB

File diff suppressed because one or more lines are too long

View File

@@ -4,67 +4,41 @@ skinparam backgroundColor #FFFFFF
skinparam defaultFontName Arial
skinparam packageStyle rectangle
title The Biergarten App - Vertical Slice Architecture
title The Biergarten App - Layered Architecture
package "API.Core (thin host)" #E3F2FD {
[API.Core\nASP.NET Core Web API host] as API
package "API Layer" #E3F2FD {
[API.Core\nASP.NET Core Web API] as API
note right of API
- Program.cs wiring only
- MediatR + AddApplicationPart
per Features.* assembly
- Swagger/OpenAPI, health checks
- JWT auth middleware
- Global exception filter
- Controllers (Auth, User)
- Swagger/OpenAPI
- Middleware
- Health Checks
end note
}
package "Feature Slices" #F3E5F5 {
[Features.Auth] as AuthSlice
[Features.Breweries] as BrewerySlice
[Features.UserManagement] as UserSlice
[Features.Emails] as EmailsSlice
note right of AuthSlice
Each slice owns its own:
- Controller (HTTP endpoints)
- Commands/Queries + Handlers
- Validators
- Repository
end note
note right of EmailsSlice
No controller - invoked only
via MediatR commands sent
from other slices
end note
}
package "Shared" #FCE4EC {
[Shared.Contracts] as SharedContracts
[Shared.Application] as SharedApp
note right of SharedApp
- ValidationBehavior
(MediatR pipeline)
- Cross-slice email commands
(SendRegistrationEmailCommand,
SendResendConfirmationEmailCommand)
package "Service Layer" #F3E5F5 {
[Service.Auth] as AuthSvc
[Service.UserManagement] as UserSvc
note right of AuthSvc
- Business Logic
- Validation
- Orchestration
end note
}
package "Infrastructure Layer" #FFF3E0 {
[Infrastructure.Sql] as Sql
[Infrastructure.Repository] as Repo
[Infrastructure.Jwt] as JWT
[Infrastructure.PasswordHashing] as PwdHash
[Infrastructure.Email] as Email
[Infrastructure.Email.Templates] as EmailTemplates
}
package "Domain Layer" #E8F5E9 {
[Domain.Entities] as Domain
[Domain.Exceptions] as DomainExceptions
note right of Domain
- UserAccount
- UserCredential
- UserVerification
- BreweryPost
end note
}
@@ -74,53 +48,28 @@ database "SQL Server" {
}
' Relationships
API ..> AuthSlice : AddApplicationPart\n+ MediatR scan
API ..> BrewerySlice : AddApplicationPart\n+ MediatR scan
API ..> UserSlice : AddApplicationPart\n+ MediatR scan
API ..> EmailsSlice : MediatR scan only\n(no controller)
API --> AuthSvc
API --> UserSvc
AuthSlice --> Sql
AuthSlice --> JWT
AuthSlice --> PwdHash
AuthSlice --> SharedContracts
AuthSlice --> SharedApp
AuthSvc --> Repo
AuthSvc --> JWT
AuthSvc --> PwdHash
AuthSvc --> Email
BrewerySlice --> Sql
BrewerySlice --> SharedContracts
BrewerySlice --> SharedApp
UserSvc --> Repo
UserSlice --> Sql
UserSlice --> SharedContracts
UserSlice --> SharedApp
EmailsSlice --> Email
EmailsSlice --> EmailTemplates
EmailsSlice --> SharedApp
' The one cross-slice interaction: a MediatR command contract, not a project reference
AuthSlice ..> SharedApp : sends\nSendRegistrationEmailCommand
EmailsSlice ..> SharedApp : handles\nSendRegistrationEmailCommand
Sql --> SP
Sql --> Domain
Repo --> SP
Repo --> Domain
SP --> Tables
AuthSlice --> Domain
BrewerySlice --> Domain
UserSlice --> Domain
AuthSlice --> DomainExceptions
AuthSvc --> Domain
UserSvc --> Domain
' Notes
note left of Sql
note left of Repo
SQL-first approach
All queries via
stored procedures
end note
note bottom of AuthSlice
No Features.* project
ever references another
Features.* project directly
end note
@enduml

View File

@@ -112,32 +112,38 @@ package "Test Environment\n(docker-compose.test.yaml)" #FFF3E0 {
end note
}
node "unit.tests\n(Features.*.Tests, one container)" as UnitTests {
component "xUnit + Moq + DbMocker" as UnitComp
node "Infrastructure.Repository.Tests\n(Unit Tests)" as RepoTests {
component "xUnit + DbMocker" as RepoComp
note right
Builds Dockerfile.tests, which
restores/builds against Core.slnx
then runs `dotnet test` for every
Features/*.Tests project in turn:
- Features.Auth.Tests
- Features.Breweries.Tests
- Features.UserManagement.Tests
- Features.Emails.Tests
Tests:
- AuthRepository
- UserAccountRepository
- SQL command building
Uses: Moq for handlers,
DbMocker for repositories
Uses: Mock connections
No real database needed
end note
}
node "Service.Auth.Tests\n(Unit Tests)" as SvcTests {
component "xUnit + Moq" as SvcComp
note right
Tests:
- RegisterService
- LoginService
- Token generation
Uses: Mocked dependencies
No database or infrastructure
end note
}
}
folder "test-results/\n(mounted volume)" as Results {
file "api-specs/\n results.trx" as Result1
file "Features.Auth.Tests.trx" as Result2
file "Features.Breweries.Tests.trx" as Result3
file "Features.UserManagement.Tests.trx" as Result4
file "Features.Emails.Tests.trx" as Result5
file "repository-tests/\n results.trx" as Result2
file "service-auth-tests/\n results.trx" as Result3
note bottom
TRX format
@@ -164,16 +170,17 @@ DevAPI .up.> DevSeed : depends_on
TestMig --> TestDB : 1. Migrate
TestSeed --> TestDB : 2. Seed
Specs --> TestDB : 3. Integration test
UnitTests ..> TestDB : depends_on for startup\nordering only (Moq/DbMocker,\nno real connection)
RepoTests ..> TestDB : Mock (no connection)
SvcTests ..> TestDB : Mock (no connection)
TestMig .up.> TestDB : depends_on
TestSeed .up.> TestMig : depends_on
Specs .up.> TestSeed : depends_on
UnitTests .up.> TestSeed : depends_on
' Test results export
Specs --> Results : Export TRX
UnitTests --> Results : Export TRX (one .trx per slice)
RepoTests --> Results : Export TRX
SvcTests --> Results : Export TRX
' Network notes
note bottom of DevDB

View File

@@ -13,7 +13,7 @@ The project uses Docker Compose to orchestrate multiple services:
- .NET API
- Test runners
See the [deployment diagram](diagrams-out/deployment.svg) for visual
See the [deployment diagram](diagrams/pdf/deployment.pdf) for visual
representation.
## Docker Compose Environments
@@ -25,10 +25,9 @@ representation.
**Features**:
- Persistent SQL Server volume
- NuGet package cache volume (speeds up rebuilds)
- Hot reload support
- Swagger UI enabled
- Seed data included
- Local Mailpit SMTP server for dev email testing
- `CLEAR_DATABASE=true` (drops and recreates schema)
**Services**:
@@ -38,30 +37,28 @@ sqlserver # SQL Server 2022 (port 1433)
database.migrations # DbUp migrations
database.seed # Seed initial data
api.core # Web API (ports 8080, 8081)
mailpit # Local dev SMTP server + UI (port 8025)
```
**Start Development Environment**:
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d
docker compose -f docker-compose.dev.yaml up -d
```
**Access**:
- API Swagger: http://localhost:8080/swagger
- Health Check: http://localhost:8080/health
- Mailpit UI: http://localhost:8025
- SQL Server: localhost:1433 (sa credentials from .env.dev)
**Stop Environment**:
```bash
# Stop services (keep volumes)
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down
# Stop and remove volumes (fresh start)
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml down -v
```
### 2. Testing (`docker-compose.test.yaml`)
@@ -83,22 +80,24 @@ sqlserver # Test database
database.migrations # Fresh schema
database.seed # Test data
api.specs # Reqnroll BDD tests
unit.tests # All Features.*.Tests unit test projects
repository.tests # Repository unit tests
service.auth.tests # Service unit tests
```
**Run Tests**:
```bash
# Run all tests
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
# View results
ls -la test-results/
cat test-results/api-specs/results.trx
cat test-results/Features.Auth.Tests.trx
cat test-results/repository-tests/results.trx
cat test-results/service-auth-tests/results.trx
# Clean up
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
docker compose -f docker-compose.test.yaml down -v
```
### 3. Production (`docker-compose.prod.yaml`)
@@ -125,7 +124,7 @@ api.core # Production API
**Deploy Production**:
```bash
docker compose --env-file web/.env.prod -f web/docker-compose.prod.yaml up -d
docker compose -f docker-compose.prod.yaml up -d
```
## Service Dependencies
@@ -195,6 +194,13 @@ volumes:
Test results are written to host filesystem for CI/CD integration.
**Code Volumes** (development only):
```yaml
volumes:
- ./src:/app/src # Hot reload for development
```
## Networks
Each environment uses isolated bridge networks:
@@ -217,9 +223,7 @@ environment:
DB_NAME: "${DB_NAME}"
DB_USER: "${DB_USER}"
DB_PASSWORD: "${DB_PASSWORD}"
ACCESS_TOKEN_SECRET: "${ACCESS_TOKEN_SECRET}"
REFRESH_TOKEN_SECRET: "${REFRESH_TOKEN_SECRET}"
CONFIRMATION_TOKEN_SECRET: "${CONFIRMATION_TOKEN_SECRET}"
JWT_SECRET: "${JWT_SECRET}"
```
For complete list, see [Environment Variables](environment-variables.md).
@@ -230,7 +234,7 @@ For complete list, see [Environment Variables](environment-variables.md).
```bash
# Running services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml ps
docker compose -f docker-compose.dev.yaml ps
# All containers (including stopped)
docker ps -a
@@ -240,13 +244,13 @@ docker ps -a
```bash
# All services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f
docker compose -f docker-compose.dev.yaml logs -f
# Specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f api.core
docker compose -f docker-compose.dev.yaml logs -f api.core
# Last 100 lines
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs --tail=100 api.core
docker compose -f docker-compose.dev.yaml logs --tail=100 api.core
```
### Execute Commands in Container
@@ -263,39 +267,39 @@ docker exec dev-env-sqlserver /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -
```bash
# Restart all services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart
docker compose -f docker-compose.dev.yaml restart
# Restart specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart api.core
docker compose -f docker-compose.dev.yaml restart api.core
# Rebuild and restart
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d --build api.core
docker compose -f docker-compose.dev.yaml up -d --build api.core
```
### Build Images
```bash
# Build all images
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build
docker compose -f docker-compose.dev.yaml build
# Build specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build api.core
docker compose -f docker-compose.dev.yaml build api.core
# Build without cache
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build --no-cache
docker compose -f docker-compose.dev.yaml build --no-cache
```
### Clean Up
```bash
# Stop and remove containers
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down
# Remove containers and volumes
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml down -v
# Remove containers, volumes, and images
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v --rmi all
docker compose -f docker-compose.dev.yaml down -v --rmi all
# System-wide cleanup
docker system prune -af --volumes

View File

@@ -17,7 +17,7 @@ The application uses environment variables for:
Direct environment variable access via `Environment.GetEnvironmentVariable()`.
### Frontend (`web/frontend`)
### Frontend (`src/Website`)
The active website reads runtime values from the server environment for its auth
and API integration.
@@ -130,7 +130,7 @@ ASPNETCORE_URLS=http://0.0.0.0:8080 # Binding address and port
DOTNET_RUNNING_IN_CONTAINER=true # Flag for container execution
```
## Frontend Variables (`web/frontend`)
## Frontend Variables (`src/Website`)
The active website does not use the old Next.js/Prisma environment model. Its
core runtime variables are:
@@ -147,7 +147,7 @@ NODE_ENV=development # Standard Node runtime mode
- **Required**: Yes for local development
- **Default in code**: `http://localhost:8080`
- **Used by**: `web/frontend/app/lib/auth.server.ts`
- **Used by**: `src/Website/app/lib/auth.server.ts`
- **Purpose**: Routes website auth actions to the .NET API
#### `SESSION_SECRET`
@@ -163,27 +163,15 @@ NODE_ENV=development # Standard Node runtime mode
- **Typical values**: `development`, `production`, `test`
- **Purpose**: Controls secure cookie behavior and runtime mode
### SMTP Configuration (Backend)
Read by `Infrastructure.Email/SmtpEmailProvider.cs` for sending confirmation
and account emails.
### Admin Account (Seeding)
```bash
SMTP_HOST=mailpit # Required, no default
SMTP_PORT=1025 # Optional, defaults to 587
SMTP_USERNAME=<username> # Optional, no default
SMTP_PASSWORD=<password> # Optional, no default
SMTP_USE_SSL=false # Optional, defaults to true
SMTP_FROM_EMAIL=noreply@thebiergarten.app # Required, no default
SMTP_FROM_NAME=The Biergarten App # Optional, defaults to "The Biergarten"
ADMIN_PASSWORD=SecureAdminPassword123! # Initial admin password for seeding
```
- **Implementation**: `Infrastructure.Email/SmtpEmailProvider.cs` throws on
startup if `SMTP_HOST` or `SMTP_FROM_EMAIL` is missing
- **Local dev**: point at the `mailpit` Docker service (SMTP on port 1025, web
UI on http://localhost:8025)
- **Production**: point at a real provider (SendGrid, Mailgun, Amazon SES,
etc.)
- **Required**: No (only needed for seeding)
- **Purpose**: Sets admin account password during database seeding
- **Security**: Use strong password, change immediately in production
## Docker-Specific Variables
@@ -203,33 +191,34 @@ MSSQL_PID=Express # SQL Server edition (Express, Developer, Ente
## Environment File Structure
### Backend/Docker (`web/` Directory)
### Backend/Docker (Root Directory)
```
web/.env.example # Template (tracked in Git)
web/.env.dev # Development config (gitignored)
web/.env.test # Testing config (gitignored)
web/.env.prod # Production config (gitignored)
.env.example # Template (tracked in Git)
.env.dev # Development config (gitignored)
.env.test # Testing config (gitignored)
.env.prod # Production config (gitignored)
```
**Setup**:
```bash
cp web/.env.example web/.env.dev
# Edit web/.env.dev with your values
cp .env.example .env.dev
# Edit .env.dev with your values
```
## Legacy Frontend Variables
Variables for the archived Next.js frontend (`archive/next-js-web-app/`) have
been removed from this active reference, since that app is retained for
reference only and is not run as part of the active stack.
Variables for the archived Next.js frontend (`src/Website-v1`) have been removed
from this active reference. See
[archive/legacy-website-v1.md](archive/legacy-website-v1.md) if you need the
legacy Prisma, Cloudinary, Mapbox, or SparkPost notes.
**Docker Compose Mapping**:
- `web/docker-compose.dev.yaml``web/.env.dev`
- `web/docker-compose.test.yaml``web/.env.test`
- `web/docker-compose.prod.yaml``web/.env.prod`
- `docker-compose.dev.yaml``.env.dev`
- `docker-compose.test.yaml``.env.test`
- `docker-compose.prod.yaml``.env.prod`
## Variable Reference Table
@@ -245,13 +234,6 @@ reference only and is not run as part of the active stack.
| `REFRESH_TOKEN_SECRET` | ✓ | | ✓ | Yes | Refresh token signing |
| `CONFIRMATION_TOKEN_SECRET` | ✓ | | ✓ | Yes | Confirmation token signing |
| `WEBSITE_BASE_URL` | ✓ | | | Yes | Website URL for emails |
| `SMTP_HOST` | ✓ | | ✓ | Yes | SMTP server host |
| `SMTP_PORT` | ✓ | | ✓ | No | Defaults to `587` |
| `SMTP_USERNAME` | ✓ | | ✓ | No | SMTP auth username |
| `SMTP_PASSWORD` | ✓ | | ✓ | No | SMTP auth password |
| `SMTP_USE_SSL` | ✓ | | ✓ | No | Defaults to `true` |
| `SMTP_FROM_EMAIL` | ✓ | | ✓ | Yes | Email sender address |
| `SMTP_FROM_NAME` | ✓ | | ✓ | No | Defaults to `The Biergarten` |
| `API_BASE_URL` | | ✓ | | Yes | Website-to-API base URL |
| `SESSION_SECRET` | | ✓ | | Yes | Website session signing |
| `NODE_ENV` | | ✓ | | No | Runtime mode |
@@ -272,12 +254,9 @@ reference only and is not run as part of the active stack.
Variables are validated at startup:
- Missing required variables (`ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`,
`CONFIRMATION_TOKEN_SECRET`, `SMTP_HOST`, `SMTP_FROM_EMAIL`, DB connection
values) cause the application to fail with an `InvalidOperationException`
- No minimum length is enforced on the token secrets in code; the
"minimum 32 characters" guidance above is a recommendation, not an
enforced check
- Missing required variables cause application to fail
- JWT_SECRET length is enforced (min 32 chars)
- Connection string format is validated
### Frontend Validation
@@ -305,13 +284,6 @@ REFRESH_TOKEN_SECRET=<generated-with-openssl>
CONFIRMATION_TOKEN_SECRET=<generated-with-openssl>
WEBSITE_BASE_URL=http://localhost:3000
# SMTP (Mailpit in dev)
SMTP_HOST=mailpit
SMTP_PORT=1025
SMTP_USE_SSL=false
SMTP_FROM_EMAIL=noreply@thebiergarten.app
SMTP_FROM_NAME=The Biergarten App
# Migration
CLEAR_DATABASE=true

View File

@@ -1,7 +1,7 @@
# Getting Started
This guide covers local setup for the current Biergarten stack: the .NET backend
in `web/backend` and the active React Router frontend in `web/frontend`.
in `src/Core` and the active React Router frontend in `src/Website`.
## Prerequisites
@@ -22,7 +22,7 @@ cd the-biergarten-app
### 2. Configure Backend Environment Variables
```bash
cp web/.env.example web/.env.dev
cp .env.example .env.dev
```
At minimum, ensure `.env.dev` includes valid database and token values:
@@ -43,21 +43,20 @@ See [Environment Variables](environment-variables.md) for the full list.
### 3. Start the Backend Stack
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d
docker compose -f docker-compose.dev.yaml up -d
```
This starts SQL Server, migrations, seeding, the API, and Mailpit (local dev SMTP).
This starts SQL Server, migrations, seeding, and the API.
Available endpoints:
- API Swagger: http://localhost:8080/swagger
- Health Check: http://localhost:8080/health
- Mailpit UI: http://localhost:8025
### 4. Start the Active Frontend
```bash
cd web/frontend
cd src/Website
npm install
API_BASE_URL=http://localhost:8080 SESSION_SECRET=dev-secret-change-me npm run dev
```
@@ -72,7 +71,7 @@ Required frontend runtime variables for local work:
### 5. Optional: Run Storybook
```bash
cd web/frontend
cd src/Website
npm run storybook
```
@@ -83,15 +82,15 @@ Storybook runs at http://localhost:6006 by default.
### Backend
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml logs -f
docker compose -f docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down -v
```
### Frontend
```bash
cd web/frontend
cd src/Website
npm run lint
npm run typecheck
npm run format:check
@@ -116,7 +115,7 @@ export WEBSITE_BASE_URL="http://localhost:3000"
### 2. Run Migrations and Seed
```bash
cd web/backend
cd src/Core
dotnet run --project Database/Database.Migrations/Database.Migrations.csproj
dotnet run --project Database/Database.Seed/Database.Seed.csproj
```
@@ -129,11 +128,12 @@ dotnet run --project API/API.Core/API.Core.csproj
## Legacy Frontend Note
The previous Next.js frontend lives in `archive/next-js-web-app/` for reference
only and is not the active website.
The previous Next.js frontend now lives in `src/Website-v1` and is not the
active website. Legacy setup details have been moved to
[docs/archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Next Steps
- Review [Architecture](../architecture.md)
- Review [Architecture](architecture.md)
- Run backend and frontend checks from [Testing](testing.md)
- Use [Docker Guide](docker.md) for container troubleshooting

View File

@@ -7,13 +7,9 @@ Biergarten App.
The project uses a multi-layered testing approach across backend and frontend:
- **API.Specs** - BDD integration tests using Reqnroll (Gherkin), run against
a live, seeded database
- **Features.\*.Tests** - One unit test project per backend feature slice
(`Features.Auth.Tests`, `Features.Breweries.Tests`,
`Features.UserManagement.Tests`, `Features.Emails.Tests`), covering that
slice's command/query handlers and its own repository (mocked with Moq /
DbMocker, no real database required)
- **API.Specs** - BDD integration tests using Reqnroll (Gherkin)
- **Infrastructure.Repository.Tests** - Unit tests for data access layer
- **Service.Auth.Tests** - Unit tests for authentication business logic
- **Storybook Vitest project** - Browser-based interaction tests for shared
website stories
- **Storybook Playwright suite** - Browser checks against Storybook-rendered
@@ -25,7 +21,7 @@ The easiest way to run all tests is using Docker Compose, which sets up an
isolated test environment:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
```
This command:
@@ -45,17 +41,15 @@ ls -la test-results/
# View specific test results
cat test-results/api-specs/results.trx
cat test-results/Features.Auth.Tests.trx
cat test-results/Features.Breweries.Tests.trx
cat test-results/Features.UserManagement.Tests.trx
cat test-results/Features.Emails.Tests.trx
cat test-results/repository-tests/results.trx
cat test-results/service-auth-tests/results.trx
```
### Clean Up
```bash
# Remove test containers and volumes
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
docker compose -f docker-compose.test.yaml down -v
```
## Running Tests Locally
@@ -65,7 +59,7 @@ You can run individual test projects locally without Docker:
### Integration Tests (API.Specs)
```bash
cd web/backend
cd src/Core
dotnet test API/API.Specs/API.Specs.csproj
```
@@ -75,31 +69,32 @@ dotnet test API/API.Specs/API.Specs.csproj
- Database migrated and seeded
- Environment variables set (DB connection, JWT secret)
### Feature Slice Unit Tests
Each feature slice has its own test project, covering its command/query
handlers and repository:
### Repository Tests
```bash
cd web/backend
dotnet test Features/Features.Auth.Tests/Features.Auth.Tests.csproj
dotnet test Features/Features.Breweries.Tests/Features.Breweries.Tests.csproj
dotnet test Features/Features.UserManagement.Tests/Features.UserManagement.Tests.csproj
dotnet test Features/Features.Emails.Tests/Features.Emails.Tests.csproj
# Or run all of them at once via the solution:
for proj in Features/*.Tests; do dotnet test "$proj"; done
cd src/Core
dotnet test Infrastructure/Infrastructure.Repository.Tests/Infrastructure.Repository.Tests.csproj
```
**Requirements**:
- No database required (handlers use Moq; repository tests use DbMocker to
simulate SQL Server responses)
- SQL Server instance running (uses mock data)
### Service Tests
```bash
cd src/Core
dotnet test Service/Service.Auth.Tests/Service.Auth.Tests.csproj
```
**Requirements**:
- No database required (uses Moq for mocking)
### Frontend Storybook Tests
```bash
cd web/frontend
cd src/Website
npm install
npm run test:storybook
```
@@ -113,7 +108,7 @@ npm run test:storybook
### Frontend Playwright Storybook Tests
```bash
cd web/frontend
cd src/Website
npm install
npm run test:storybook:playwright
```
@@ -129,28 +124,27 @@ npm run test:storybook:playwright
### Current Coverage
**Features.Auth.Tests**:
**Authentication & User Management**:
- User registration with validation
- User login with JWT token generation
- Password hashing and verification (Argon2id)
- Email confirmation and confirmation-email resend
- Refresh token exchange
- JWT token generation, validation, and claims handling
- `IAuthRepository` data access (DbMocker-backed)
- Invalid credentials and 404 error responses (via API.Specs)
- JWT token generation and claims
- Invalid credentials handling
- 404 error responses
**Features.Breweries.Tests**:
**Repository Layer**:
- Brewery create/update/delete commands and get-by-id/get-all queries
- User account creation
- User credential management
- GetUserByUsername queries
- Stored procedure execution
**Features.UserManagement.Tests**:
**Service Layer**:
- User get-by-id/get-all queries and the (currently unrouted) update command
**Features.Emails.Tests**:
- Registration and resend-confirmation email dispatch handlers
- Login service with password verification
- Register service with validation
- Business logic for authentication flow
**Frontend UI Coverage**:
@@ -162,7 +156,10 @@ npm run test:storybook:playwright
### Planned Coverage
- [ ] Email verification workflow
- [ ] Password reset functionality
- [ ] Token refresh mechanism
- [ ] Brewery data management
- [ ] Beer post operations
- [ ] User follow/unfollow
- [ ] Image upload service
@@ -207,28 +204,21 @@ npm run test:storybook:playwright
```
API.Specs/
├── Features/
│ ├── Registration.feature # Registration scenarios
── Login.feature # Login scenarios
│ ├── Confirmation.feature # Email confirmation scenarios
│ ├── ResendConfirmation.feature # Resend-confirmation scenarios
│ ├── TokenRefresh.feature # Refresh token scenarios
│ ├── AccessTokenValidation.feature # Protected endpoint access scenarios
│ └── NotFound.feature # 404 handling
│ ├── Authentication.feature # Login/register scenarios
── UserManagement.feature # User CRUD scenarios
├── Steps/
│ ├── AuthSteps.cs # Step definitions for the Auth features
│ └── ApiGeneralSteps.cs # Shared/general step definitions
── Mocks/
── MockEmailDispatcher.cs # Substitutes Features.Emails' IEmailDispatcher
│ └── MockEmailProvider.cs # Substitutes Infrastructure.Email's IEmailProvider
└── TestApiFactory.cs # Test server setup (swaps in the mocks above)
│ ├── AuthenticationSteps.cs # Step definitions
│ └── UserManagementSteps.cs
── Mocks/
── TestApiFactory.cs # Test server setup
```
**Example Feature**:
```gherkin
Feature: User Registration
Feature: User Authentication
As a user
I want to register
I want to register and login
So that I can access the platform
Scenario: Successful user registration
@@ -238,51 +228,45 @@ Scenario: Successful user registration
And my account should be created
```
### Features.Auth.Tests
### Infrastructure.Repository.Tests
```
Features.Auth.Tests/
├── Commands/
│ ├── RegisterUserHandlerTests.cs
│ ├── ConfirmUserHandlerTests.cs
── ResendConfirmationEmailHandlerTests.cs
│ └── RefreshTokenHandlerTests.cs
├── Queries/
│ └── LoginHandlerTests.cs
├── Repository/
│ └── AuthRepositoryTests.cs # DbMocker-backed repository tests
└── Services/
├── TokenServiceRefreshTests.cs
└── TokenServiceValidationTests.cs
Infrastructure.Repository.Tests/
├── AuthRepositoryTests.cs # Auth repository tests
├── UserAccountRepositoryTests.cs # User account tests
└── TestFixtures/
── DatabaseFixture.cs # Shared test setup
```
Each of the other three slices (`Features.Breweries.Tests`,
`Features.UserManagement.Tests`, `Features.Emails.Tests`) follows the same
shape: a `Commands/`/`Queries/` folder with one test file per handler.
### Service.Auth.Tests
```
Service.Auth.Tests/
├── LoginService.test.cs # Login business logic tests
└── RegisterService.test.cs # Registration business logic tests
```
## Writing Tests
### Unit Test Example (xUnit)
```csharp
public class LoginHandlerTests
public class LoginServiceTests
{
[Fact]
public async Task Handle_WithValidData_ReturnsPayloadWithMatchingUsername()
public async Task LoginAsync_ValidCredentials_ReturnsToken()
{
// Arrange
var authRepoMock = new Mock<IAuthRepository>();
var passwordInfraMock = new Mock<IPasswordInfrastructure>();
var tokenServiceMock = new Mock<ITokenService>();
var handler = new LoginHandler(authRepoMock.Object, passwordInfraMock.Object, tokenServiceMock.Object);
// ...set up mocks for a known user/credential...
var mockRepo = new Mock<IAuthRepository>();
var mockJwt = new Mock<IJwtService>();
var service = new AuthService(mockRepo.Object, mockJwt.Object);
// Act
var result = await handler.Handle(new LoginQuery("testuser", "password123"), CancellationToken.None);
var result = await service.LoginAsync("testuser", "password123");
// Assert
result.Should().NotBeNull();
result.AccessToken.Should().NotBeNullOrEmpty();
result.Token.Should().NotBeNullOrEmpty();
}
}
```
@@ -304,9 +288,9 @@ configuration:
```bash
# CI/CD command
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml build
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
docker compose -f docker-compose.test.yaml build
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml down -v
```
Exit codes:
@@ -318,7 +302,7 @@ Frontend UI checks should also be included in CI for the active website
workspace:
```bash
cd web/frontend
cd src/Website
npm ci
npm run test:storybook
npm run test:storybook:playwright
@@ -331,7 +315,7 @@ npm run test:storybook:playwright
Ensure SQL Server is running and environment variables are set:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml ps
docker compose -f docker-compose.test.yaml ps
```
### Port Conflicts
@@ -344,13 +328,13 @@ If port 1433 is in use, stop other SQL Server instances or modify the port in
Clean up test database:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
docker compose -f docker-compose.test.yaml down -v
```
### View Container Logs
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml logs <service-name>
docker compose -f docker-compose.test.yaml logs <service-name>
```
## Best Practices

View File

@@ -2,7 +2,8 @@
## Overview
The Core project validates JWTs across three token types:
The Core project implements comprehensive JWT token validation across three
token types:
- **Access Tokens**: Short-lived (1 hour) tokens for API authentication
- **Refresh Tokens**: Long-lived (21 days) tokens for obtaining new access
@@ -14,7 +15,7 @@ The Core project validates JWTs across three token types:
### Infrastructure Layer
#### [ITokenInfrastructure](../../web/backend/Infrastructure/Infrastructure.Jwt/ITokenInfrastructure.cs)
#### [ITokenInfrastructure](Infrastructure.Jwt/ITokenInfrastructure.cs)
Low-level JWT operations.
@@ -24,92 +25,77 @@ Low-level JWT operations.
- `ValidateJwtAsync()` - Validates token signature, expiration, and format
**Implementation:**
[JwtInfrastructure.cs](../../web/backend/Infrastructure/Infrastructure.Jwt/JwtInfrastructure.cs)
[JwtInfrastructure.cs](Infrastructure.Jwt/JwtInfrastructure.cs)
- Uses Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler
- Algorithm: HS256 (HMAC-SHA256)
- Validates token lifetime, signature, and well-formedness
### Features.Auth Slice
### Service Layer
#### [ITokenService](../../web/backend/Features/Features.Auth/Services/ITokenService.cs)
#### [ITokenValidationService](Service.Auth/ITokenValidationService.cs)
Both token generation and validation live on the same slice-internal
service (there is no separate validation service/class).
High-level token validation with context (token type, user extraction).
**Generation methods:**
**Methods:**
- `ValidateAccessTokenAsync(string token)` - Validates access tokens
- `ValidateRefreshTokenAsync(string token)` - Validates refresh tokens
- `ValidateConfirmationTokenAsync(string token)` - Validates confirmation tokens
**Returns:** `ValidatedToken` record containing:
- `UserId` (Guid)
- `Username` (string)
- `Principal` (ClaimsPrincipal) - Full JWT claims
**Implementation:**
[TokenValidationService.cs](Service.Auth/TokenValidationService.cs)
- Reads token secrets from environment variables
- Extracts and validates claims (Sub, UniqueName)
- Throws `UnauthorizedException` on validation failure
#### [ITokenService](Service.Auth/ITokenService.cs)
Token generation (existing service extended).
**Methods:**
- `GenerateAccessToken(UserAccount)` - Creates 1-hour access token
- `GenerateRefreshToken(UserAccount)` - Creates 21-day refresh token
- `GenerateConfirmationToken(UserAccount)` - Creates 30-minute confirmation
token
**Validation methods:**
- `ValidateAccessTokenAsync(string token)` - Validates access tokens
- `ValidateRefreshTokenAsync(string token)` - Validates refresh tokens
- `ValidateConfirmationTokenAsync(string token)` - Validates confirmation tokens
**Returns (validation):** `ValidatedToken` record containing:
- `UserId` (Guid)
- `Username` (string)
- `Principal` (ClaimsPrincipal) - Full JWT claims
**Implementation:** [TokenService.cs](../../web/backend/Features/Features.Auth/Services/TokenService.cs)
- Reads token secrets from environment variables
- Extracts and validates claims (Sub, UniqueName)
- Throws `UnauthorizedException` on validation failure
`TokenService` is registered by `Features.Auth`'s own
`AddFeaturesAuth()` extension method, except for the lower-level
`ITokenInfrastructure` (JWT signing/verification) it depends on, which is
registered by the host (`API.Core/Program.cs`) since
`JwtAuthenticationHandler` (host-level auth middleware) also depends on
it directly.
### Integration Points
#### [ConfirmUserHandler](../../web/backend/Features/Features.Auth/Commands/ConfirmUser/ConfirmUserHandler.cs)
#### [ConfirmationService](Service.Auth/IConfirmationService.cs)
**Flow:**
1. Receives confirmation token from user via `ConfirmUserCommand`
2. Calls `ITokenService.ValidateConfirmationTokenAsync()`
1. Receives confirmation token from user
2. Calls `TokenValidationService.ValidateConfirmationTokenAsync()`
3. Extracts user ID from validated token
4. Calls `IAuthRepository.ConfirmUserAccountAsync()` to update database
4. Calls `AuthRepository.ConfirmUserAccountAsync()` to update database
5. Returns confirmation result
#### [ResendConfirmationEmailHandler](../../web/backend/Features/Features.Auth/Commands/ResendConfirmationEmail/ResendConfirmationEmailHandler.cs)
#### [RefreshTokenService](Service.Auth/RefreshTokenService.cs)
**Flow:**
1. Receives a user ID via `ResendConfirmationEmailCommand`
2. Looks up the user and checks they aren't already verified
3. Generates a fresh confirmation token via `ITokenService`
4. Sends `SendResendConfirmationEmailCommand` over MediatR, handled by the
`Features.Emails` slice, with no direct project reference between the two
slices
1. Receives refresh token from user
2. Calls `TokenValidationService.ValidateRefreshTokenAsync()`
3. Retrieves user account via `AuthRepository.GetUserByIdAsync()`
4. Issues new access and refresh tokens via `TokenService`
5. Returns new token pair
#### [RefreshTokenHandler](../../web/backend/Features/Features.Auth/Commands/RefreshToken/RefreshTokenHandler.cs)
**Flow:**
1. Receives a refresh token via `RefreshTokenCommand`
2. Delegates to `ITokenService.RefreshTokenAsync()`, which validates the
token, retrieves the user account via `IAuthRepository.GetUserByIdAsync()`,
and issues a new access/refresh token pair
3. Maps the result onto `LoginPayload`
#### [AuthController](../../web/backend/Features/Features.Auth/Controllers/AuthController.cs)
#### [AuthController](API.Core/Controllers/AuthController.cs)
**Endpoints:**
- `POST /api/auth/register` - Register new user
- `POST /api/auth/login` - Authenticate user
- `POST /api/auth/confirm?token=...` - Confirm email
- `POST /api/auth/confirm/resend?userId=...` - Resend the confirmation email
- `POST /api/auth/refresh` - Refresh access token
## Validation Security
@@ -172,44 +158,32 @@ Validation failures return HTTP 401 Unauthorized:
1. **Generation**: During user registration (30-minute validity)
2. **Delivery**: Emailed to user in confirmation link
3. **Usage**: User clicks link, token posted to `/api/auth/confirm`
4. **Validation**: Validated by `ConfirmUserHandler`
4. **Validation**: Validated by ConfirmationService
5. **Completion**: User account marked as confirmed
6. **Expiration**: Token becomes invalid after 30 minutes
## Testing
All of the following live in `Features.Auth.Tests`.
### Unit Tests
**Services/TokenServiceValidationTests.cs**
**TokenValidationService.test.cs**
- Happy path: Valid token extraction
- Error cases: Invalid, expired, malformed tokens
- Missing/invalid claims scenarios
**Services/TokenServiceRefreshTests.cs**
**RefreshTokenService.test.cs**
- Successful refresh with valid token
- Invalid/expired refresh token rejection
- Non-existent user handling
**Commands/RefreshTokenHandlerTests.cs**
- Verifies the handler maps `ITokenService.RefreshTokenAsync()`'s result onto
`LoginPayload`
**Commands/ConfirmUserHandlerTests.cs**
**ConfirmationService.test.cs**
- Successful confirmation with valid token
- Token validation failures
- User not found scenarios
**Commands/ResendConfirmationEmailHandlerTests.cs**
- Sends a fresh confirmation email when the user exists and is unverified
- No-ops when the user doesn't exist or is already verified
### BDD Tests (Reqnroll)
**TokenRefresh.feature**

View File

@@ -1,24 +0,0 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

View File

@@ -1,74 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Pipeline Results Viewer</title>
</head>
<body>
<div class="dialog" id="window">
<div class="dialog-header">
<p>Pipeline Results Viewer</p>
</div>
<div class="tab-strip" role="tablist">
<button
type="button"
class="tab is-active"
id="tab-tables"
role="tab"
aria-selected="true"
aria-controls="panel-tables"
>
Tables
</button>
<button
type="button"
class="tab"
id="tab-query"
role="tab"
aria-selected="false"
aria-controls="panel-query"
>
Query
</button>
</div>
<div class="dialog-body">
<div
class="tab-panel"
id="panel-tables"
role="tabpanel"
aria-labelledby="tab-tables"
>
<div class="field-group">
<label for="table-select">Table</label>
<select id="table-select"></select>
</div>
<p id="status"></p>
<div id="table-container"></div>
</div>
<div
class="tab-panel"
id="panel-query"
role="tabpanel"
aria-labelledby="tab-query"
hidden
>
<div class="field-group field-group--stacked">
<label for="query-input">SQL Query</label>
<textarea
id="query-input"
rows="4"
spellcheck="false"
placeholder="SELECT * FROM ..."
></textarea>
</div>
<button type="button" id="query-run" class="btn">Run Query</button>
<p id="query-status"></p>
<div id="query-container"></div>
</div>
</div>
</div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>

File diff suppressed because it is too large Load Diff

View File

@@ -1,20 +0,0 @@
{
"name": "pipeline-results-viewer",
"version": "0.0.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview"
},
"devDependencies": {
"@types/sql.js": "^1.4.11",
"sass": "^1.101.0",
"typescript": "^6.0.3",
"vite": "^8.0.12"
},
"dependencies": {
"sql.js": "^1.14.1"
}
}

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 9.3 KiB

View File

@@ -1,147 +0,0 @@
import initSqlJs, { type Database, type SqlValue } from "sql.js";
import sqlWasmUrl from "sql.js/dist/sql-wasm.wasm?url";
import "./style.scss";
const DB_URL = "/biergarten.sqlite";
const tabs = document.querySelectorAll<HTMLButtonElement>("[role='tab']");
const panels = document.querySelectorAll<HTMLDivElement>("[role='tabpanel']");
const tableSelect = document.querySelector<HTMLSelectElement>("#table-select")!;
const tableContainer =
document.querySelector<HTMLDivElement>("#table-container")!;
const status = document.querySelector<HTMLParagraphElement>("#status")!;
const queryInput = document.querySelector<HTMLTextAreaElement>("#query-input")!;
const queryRun = document.querySelector<HTMLButtonElement>("#query-run")!;
const queryStatus = document.querySelector<HTMLParagraphElement>("#query-status")!;
const queryContainer =
document.querySelector<HTMLDivElement>("#query-container")!;
for (const tab of tabs) {
tab.addEventListener("click", () => {
for (const otherTab of tabs) {
const isActive = otherTab === tab;
otherTab.classList.toggle("is-active", isActive);
otherTab.setAttribute("aria-selected", String(isActive));
}
for (const panel of panels) {
panel.hidden = panel.id !== tab.getAttribute("aria-controls");
}
});
}
const renderResultTable = (
container: HTMLDivElement,
columns: string[],
values: SqlValue[][],
) => {
const table = document.createElement("table");
const head = table.createTHead().insertRow();
for (const column of columns) {
const th = document.createElement("th");
th.textContent = column;
head.append(th);
}
const body = table.createTBody();
for (const row of values) {
const tr = body.insertRow();
for (const cell of row) {
const td = document.createElement("td");
td.textContent = String(cell ?? "");
tr.append(td);
}
}
container.append(table);
};
const renderTable = (db: Database, name: string) => {
const result = db.exec(`SELECT * FROM "${name}"`);
tableContainer.replaceChildren();
if (result.length === 0) {
tableContainer.append(
Object.assign(document.createElement("p"), { textContent: "No rows." }),
);
return;
}
const { columns, values } = result[0];
renderResultTable(tableContainer, columns, values);
};
const runQuery = (db: Database) => {
const sql = queryInput.value.trim();
queryContainer.replaceChildren();
if (!sql) {
queryStatus.textContent = "Enter a query to run.";
return;
}
try {
const results = db.exec(sql);
if (results.length === 0) {
queryStatus.textContent = "Query executed. No rows returned.";
return;
}
const { columns, values } = results[results.length - 1];
renderResultTable(queryContainer, columns, values);
queryStatus.textContent = `${values.length} row${values.length === 1 ? "" : "s"} returned.`;
} catch (error) {
queryStatus.textContent = `Error: ${error instanceof Error ? error.message : String(error)}`;
}
};
async function main(): Promise<void> {
status.textContent = "Loading database...";
const sqlJs = await initSqlJs({ locateFile: () => sqlWasmUrl });
const response = await fetch(DB_URL);
const buffer = await response.arrayBuffer();
const db = new sqlJs.Database(new Uint8Array(buffer));
const tables =
db
.exec(
`SELECT name
FROM sqlite_master
WHERE
type = 'table'
AND
name NOT LIKE 'sqlite_%'
ORDER BY name`,
)[0]
?.values.map((row) => String(row[0])) ?? [];
for (const name of tables) {
tableSelect.append(new Option(name, name));
}
tableSelect.addEventListener("change", () =>
renderTable(db, tableSelect.value),
);
if (tables.length > 0) {
renderTable(db, tables[0]);
}
status.textContent = "";
queryRun.addEventListener("click", () => runQuery(db));
queryInput.addEventListener("keydown", (event) => {
if ((event.metaKey || event.ctrlKey) && event.key === "Enter") {
event.preventDefault();
runQuery(db);
}
});
}
main().catch((error: unknown) => {
status.textContent = `Failed to load database: ${String(error)}`;
});

View File

@@ -1,267 +0,0 @@
$gray-100: #dedede;
$gray-300: #c3c3c3;
$gray-500: #808080;
$white: #ffffff;
$black: #000000;
$navy: #000080;
$blue: #1084d0;
$teal: #008080;
$color-surface: $gray-300;
$color-surface-alt: $gray-100;
$color-surface-sunken: $white;
$color-border-highlight: $gray-100;
$color-border-lowlight: $gray-500;
$color-border-light: $white;
$color-border-dark: $black;
$color-text: $black;
$color-text-on-accent: $white;
$color-text-status: $navy;
$color-accent-start: $navy;
$color-accent-end: $blue;
$color-desktop-bg: $teal;
$color-row-stripe: $gray-100;
$color-shadow: rgba($black, 0.4);
$color-focus-ring: $black;
$bevel-size: 2px;
$font-stack: Tahoma, "MS Sans Serif", Verdana, sans-serif;
@mixin bevel-raised($size: $bevel-size) {
border-style: solid;
border-width: $size;
border-top-color: $color-border-highlight;
border-left-color: $color-border-highlight;
border-right-color: $color-border-dark;
border-bottom-color: $color-border-dark;
box-shadow:
inset $size $size 0 0 $color-border-light,
inset (-$size) (-$size) 0 0 $color-border-lowlight;
}
@mixin bevel-sunken($size: $bevel-size) {
border-style: solid;
border-width: $size;
border-top-color: $color-border-lowlight;
border-left-color: $color-border-lowlight;
border-right-color: $color-border-light;
border-bottom-color: $color-border-light;
box-shadow:
inset $size $size 0 0 $color-border-dark,
inset (-$size) (-$size) 0 0 $color-surface-alt;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
padding: 24px;
background: $color-desktop-bg;
font-family: $font-stack;
color: $color-text;
min-height: 100vh;
}
#window {
width: 100%;
max-width: 1600px;
margin: 0 auto;
background: $color-surface;
padding: 6px;
display: flex;
flex-direction: column;
gap: 6px;
@include bevel-raised(3px);
box-shadow: 8px 8px 0 0 $color-shadow;
}
.dialog-header {
background: linear-gradient(90deg, $color-accent-start, $color-accent-end);
color: $color-text-on-accent;
padding: 6px 8px;
display: flex;
align-items: center;
p {
margin: 0;
font-size: 14px;
font-weight: bold;
letter-spacing: 0.5px;
text-transform: uppercase;
}
}
.tab-strip {
display: flex;
gap: 4px;
padding: 8px 8px 0;
}
.tab {
position: relative;
top: 3px;
margin-bottom: -2px;
font-family: $font-stack;
font-size: 12px;
font-weight: bold;
text-transform: uppercase;
padding: 6px 14px;
background: $color-surface;
color: $color-text;
border-style: solid;
border-width: 2px;
border-top-color: $color-border-light;
border-left-color: $color-border-light;
border-right-color: $color-border-dark;
border-bottom-color: $color-border-dark;
border-radius: 0;
cursor: pointer;
&.is-active {
top: 0;
z-index: 1;
border-bottom-color: $color-surface;
}
&:focus {
outline: 1px dotted $color-focus-ring;
outline-offset: -4px;
}
}
.dialog-body {
border-top: 2px solid $color-border-dark;
padding: 12px;
}
.tab-panel {
display: flex;
flex-direction: column;
gap: 12px;
&[hidden] {
display: none;
}
}
.field-group {
display: flex;
align-items: center;
gap: 8px;
label {
font-size: 12px;
font-weight: bold;
text-transform: uppercase;
}
&.field-group--stacked {
flex-direction: column;
align-items: stretch;
}
}
select#table-select {
font-family: $font-stack;
font-size: 13px;
padding: 3px 6px;
background: $color-surface-sunken;
color: $color-text;
border-radius: 0;
@include bevel-sunken();
&:focus {
outline: 1px dotted $color-focus-ring;
outline-offset: 2px;
}
}
textarea#query-input {
font-family: $font-stack;
font-size: 13px;
padding: 6px;
background: $color-surface-sunken;
color: $color-text;
border-radius: 0;
resize: vertical;
@include bevel-sunken();
&:focus {
outline: 1px dotted $color-focus-ring;
outline-offset: 2px;
}
}
.btn {
align-self: flex-start;
font-family: $font-stack;
font-size: 12px;
font-weight: bold;
text-transform: uppercase;
padding: 6px 16px;
background: $color-surface;
color: $color-text;
cursor: pointer;
@include bevel-raised();
&:active {
@include bevel-sunken();
}
&:focus {
outline: 1px dotted $color-focus-ring;
outline-offset: -4px;
}
}
#status,
#query-status {
margin: 0;
font-size: 12px;
font-style: italic;
color: $color-text-status;
}
#table-container,
#query-container {
background: $color-surface-sunken;
padding: 4px;
overflow-x: auto;
@include bevel-sunken();
}
table {
border-collapse: collapse;
margin: 0 auto;
font-family: $font-stack;
}
th,
td {
border: 1px solid $color-border-lowlight;
padding: 4px 8px;
text-align: left;
font-size: 13px;
}
th {
background: $color-surface;
white-space: nowrap;
font-weight: bold;
text-transform: uppercase;
@include bevel-raised(1px);
}
td {
min-width: 120px;
max-width: 800px;
white-space: normal;
word-break: break-word;
background: $color-surface-sunken;
}
tbody tr:nth-child(even) td {
background: $color-row-stripe;
}

View File

@@ -1,24 +0,0 @@
{
"compilerOptions": {
"target": "es2023",
"module": "esnext",
"lib": ["ES2023", "DOM"],
"types": ["vite/client"],
"skipLibCheck": true,
/* Bundler mode */
"moduleResolution": "bundler",
"esModuleInterop": true,
"allowImportingTsExtensions": true,
"verbatimModuleSyntax": true,
"moduleDetection": "force",
"noEmit": true,
/* Linting */
"noUnusedLocals": true,
"noUnusedParameters": true,
"erasableSyntaxOnly": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"]
}

View File

@@ -138,9 +138,7 @@ FetchContent_MakeAvailable(cpp-httplib)
# 5. Executable & Sources
add_executable(${PROJECT_NAME}
includes/services/enrichment/mock_enrichment.h
includes/services/curated_data/mock_curated_data_service.h
includes/json_handling/pretty_print.h)
includes/services/enrichment/mock_enrichment.h)
# --- Entry point ---
target_sources(${PROJECT_NAME} PRIVATE
@@ -149,12 +147,7 @@ target_sources(${PROJECT_NAME} PRIVATE
# --- json_handling ---
target_sources(${PROJECT_NAME} PRIVATE
src/services/curated_data/curated_json_data_service.cc
)
# --- services: curated_data ---
target_sources(${PROJECT_NAME} PRIVATE
src/services/curated_data/mock_curated_data_service.cc
src/json_handling/json_loader.cc
)
# --- application_options ---
@@ -167,7 +160,6 @@ target_sources(${PROJECT_NAME} PRIVATE
src/biergarten_pipeline_orchestrator/log_results.cc
src/biergarten_pipeline_orchestrator/biergarten_pipeline_orchestrator.cc
src/biergarten_pipeline_orchestrator/generate_breweries.cc
src/biergarten_pipeline_orchestrator/generate_users.cc
src/biergarten_pipeline_orchestrator/run.cc
src/biergarten_pipeline_orchestrator/query_cities_with_countries.cc
)
@@ -211,7 +203,6 @@ target_sources(${PROJECT_NAME} PRIVATE
# --- services: sqlite ---
target_sources(${PROJECT_NAME} PRIVATE
src/services/sqlite/process_record.cc
src/services/sqlite/process_user_record.cc
src/services/sqlite/sqlite_export_service.cc
src/services/sqlite/finalize.cc
src/services/sqlite/initialize.cc
@@ -257,35 +248,14 @@ target_compile_definitions(${PROJECT_NAME} PRIVATE
$<$<CONFIG:Debug>:DEBUG>
)
target_compile_options(biergarten-pipeline PRIVATE
-fmacro-prefix-map=${CMAKE_SOURCE_DIR}/tooling/pipeline/src/=
)
# 7. Runtime Assets
configure_file(
${CMAKE_SOURCE_DIR}/locations.json
${CMAKE_BINARY_DIR}/locations.json
COPYONLY
)
configure_file(
${CMAKE_SOURCE_DIR}/personas.json
${CMAKE_BINARY_DIR}/personas.json
COPYONLY
)
configure_file(
${CMAKE_SOURCE_DIR}/forenames-by-country.json
${CMAKE_BINARY_DIR}/forenames-by-country.json
COPYONLY
)
configure_file(
${CMAKE_SOURCE_DIR}/surnames-by-country.json
${CMAKE_BINARY_DIR}/surnames-by-country.json
COPYONLY
)
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CMAKE_SOURCE_DIR}/prompts
${CMAKE_BINARY_DIR}/prompts
)

File diff suppressed because it is too large Load Diff

View File

@@ -1,72 +0,0 @@
#!/bin/bash
#
# Walks every directory under src/ and includes/ (including nested
# subdirectories), runs clang-format on the C/C++ files in that directory
# only (non-recursive), and commits the result with its own commit.
#
# Usage: ./format-per-directory.sh [-y]
# -y skip the confirmation prompt
set -euo pipefail
SKIP_CONFIRM=false
if [[ "${1:-}" == "-y" ]]; then
SKIP_CONFIRM=true
fi
cd "$(dirname "$0")"
if [ ! -f .clang-format ]; then
echo "ERROR: .clang-format file not found."
exit 1
fi
if ! command -v clang-format &>/dev/null; then
echo "ERROR: clang-format not found."
exit 1
fi
if ! command -v git &>/dev/null; then
echo "ERROR: git not found."
exit 1
fi
echo "WARNING: This script will format .cpp, .h, .cxx, .cc, .c, .hpp files"
echo "directory-by-directory under src/ and includes/, committing after each"
echo "directory is formatted."
if [[ "$SKIP_CONFIRM" == false ]]; then
read -p "Do you want to continue? (y/n) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Aborted."
exit 1
fi
fi
if [[ -n "$(git status --porcelain)" ]]; then
echo "ERROR: working tree is not clean. Commit or stash your changes first."
exit 1
fi
dirs=$(find src includes -type d | sort)
for dir in $dirs; do
files=$(find "$dir" -maxdepth 1 -type f \( -name "*.cpp" -o -name "*.hpp" -o -name "*.h" -o -name "*.c" -o -name "*.cc" -o -name "*.cxx" \))
if [[ -z "$files" ]]; then
continue
fi
echo "Formatting $dir..."
echo "$files" | xargs clang-format -i
if [[ -n "$(git status --porcelain -- "$dir")" ]]; then
git add -- "$dir"
git commit -m "Formatted $dir"
else
echo "No changes in $dir, skipping commit."
fi
done
echo "Done."

View File

@@ -1,74 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_H_
/**
* @file biergarten_pipeline.h
* @brief Master umbrella header — includes every public header in the
* Biergarten pipeline.
*/
// --- orchestrator ---
#include "biergarten_pipeline_orchestrator.h"
// --- concurrency ---
#include "concurrency/bounded_channel.h"
// --- data_generation ---
#include "data_generation/data_generator.h"
#include "data_generation/json_grammars.h"
#ifndef BIERGARTEN_MOCK_ONLY
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
#endif
#include "data_generation/mock_generator.h"
#include "data_generation/prompt_formatting/gemma4_jinja_prompt_formatter.h"
#include "data_generation/prompt_formatting/prompt_formatter.h"
// --- data_model ---
#include "data_model/generated_models.h"
#include "data_model/models.h"
// --- json_handling ---
#include "json_handling/pretty_print.h"
#include "services/curated_data/curated_json_data_service.h"
// --- llama backend ---
#ifndef BIERGARTEN_MOCK_ONLY
#include "llama_backend_state.h"
#endif
// --- services: curated_data ---
#include "services/curated_data/curated_data_service.h"
#include "services/curated_data/mock_curated_data_service.h"
// --- services: database ---
#include "services/database/export_service.h"
#include "services/database/sqlite_connection_helpers.h"
#include "services/database/sqlite_export_service.h"
#include "services/database/sqlite_export_service_helpers.h"
#include "services/database/sqlite_handle_types.h"
#include "services/database/sqlite_statement_helpers.h"
// --- services: datetime ---
#include "services/datetime/date_time_provider.h"
#include "services/datetime/timer.h"
// --- services: enrichment ---
#include "services/enrichment/enrichment_service.h"
#include "services/enrichment/mock_enrichment.h"
#include "services/enrichment/wikipedia_service.h"
// --- services: logging ---
#include "services/logging/log_dispatcher.h"
#include "services/logging/log_entry.h"
#include "services/logging/log_producer.h"
#include "services/logging/logger.h"
// --- services: prompting ---
#include "services/prompting/prompt_directory.h"
// --- web_client ---
#include "web_client/http_web_client.h"
#include "web_client/web_client.h"
#endif // BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_H_

View File

@@ -1,9 +1,12 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_ORCHESTRATOR_H_
#define BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_ORCHESTRATOR_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_
#define BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_
/**
* @file biergarten_pipeline_orchestrator.h
* @brief Orchestration for end-to-end brewery and user data generation.
* @file biergarten_data_generator.h
* @brief Orchestration for end-to-end brewery data generation pipeline.
*
* Intent: Coordinates location loading, enrichment, and generation phases
* to produce a complete dataset. Coordinates dependencies via composition root.
*/
#include <memory>
@@ -12,36 +15,32 @@
#include "data_generation/data_generator.h"
#include "data_model/generated_models.h"
#include "services/curated_data/curated_data_service.h"
#include "services/database/export_service.h"
#include "services/enrichment/enrichment_service.h"
#include "services/logging/logger.h"
/**
* @brief Main orchestrator for the Biergarten data generation pipeline.
* @brief Main data generator class for the Biergarten pipeline.
*
* Handles location loading, city enrichment, and brewery/user generation.
* This class encapsulates the core logic for generating brewery data.
* It handles location loading, city enrichment, and brewery generation.
*/
class BiergartenPipelineOrchestrator {
public:
/**
* @brief Constructs the orchestrator with injected pipeline dependencies.
*
* @param logger Sink for pipeline diagnostics.
* @param context_service Provides regional context for locations.
* @param generator Implementation (Llama or Mock) for brewery/user
* generation.
* @param exporter Database backend for persisting generated records.
* @param curated_data_service Loads curated location, persona, and name
* data used to seed generation.
* @param application_options CLI configuration and paths.
*/
/**
* @brief Constructs the orchestrator with injected pipeline dependencies.
*
* @param context_service Provides regional context for locations.
* @param generator Implementation (Llama or Mock) for brewery/user generation.
* @param exporter Database backend for persisting generated records.
* @param application_options CLI configuration and paths.
*/
BiergartenPipelineOrchestrator(
std::shared_ptr<ILogger> logger,
std::unique_ptr<IEnrichmentService> context_service,
std::unique_ptr<DataGenerator> generator,
std::unique_ptr<IExportService> exporter,
std::unique_ptr<ICuratedDataService> curated_data_service,
const ApplicationOptions& application_options);
/**
@@ -50,42 +49,32 @@ class BiergartenPipelineOrchestrator {
* Performs the following steps:
* 1. Load curated locations from JSON
* 2. Resolve context for each city using the injected context service
* 3. Generate brewery and user data for sampled cities
*
* @note STRUCTURAL CONCURRENCY REQUIREMENT:
* When transitioned to a multithreaded design, this method MUST
* structurally enforce that all deployed worker threads are joined before
* returning (e.g. by using std::jthread or a structured concurrency
* primitive). This ensures workers do not attempt to log to a closed
* channel during application teardown.
* 3. Generate brewery data for sampled cities
*
* @return true if successful, false if not
*/
bool Run();
private:
/// @brief Logger instance for emitting pipeline messages.
std::shared_ptr<ILogger> logger_;
/// @brief Owning context provider dependency.
std::unique_ptr<IEnrichmentService> context_service_;
/**
* @brief Generator implementation selected at the composition root (Llama
* or Mock).
*/
/// @brief Generator dependency selected in the composition root.
std::unique_ptr<DataGenerator> generator_;
/**
* @brief Storage backend for generated brewery and user records.
*/
std::unique_ptr<IExportService> exporter_;
std::unique_ptr<ICuratedDataService> curated_data_service_;
/// @brief Storage backend for generated brewery records.
std::unique_ptr<IExportService> exporter_;
ApplicationOptions application_options_;
/// @brief CLI configuration: paths, model settings, generation parameters.
ApplicationOptions application_options_;
/**
* @brief Load locations from JSON and sample cities.
*
* @return Vector of locations randomly sampled per
* PipelineOptions::location_count.
* @return Vector of sampled locations capped at 50 entries.
*/
std::vector<Location> QueryCitiesWithCountries();
@@ -97,20 +86,11 @@ class BiergartenPipelineOrchestrator {
void GenerateBreweries(std::span<const EnrichedCity> cities);
/**
* @brief Generate users grounded in sampled names and personas for
* enriched cities.
*
* @param cities Span of enriched city data.
*/
void GenerateUsers(std::span<const EnrichedCity> cities);
/**
* @brief Log the generated brewery and user results.
* @brief Log the generated brewery results.
*/
void LogResults() const;
std::vector<BreweryRecord> generated_breweries_;
std::vector<UserRecord> generated_users_;
/// @brief Stores generated brewery data.
std::vector<GeneratedBrewery> generated_breweries_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_PIPELINE_ORCHESTRATOR_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_

View File

@@ -9,12 +9,10 @@
/**
* @file bounded_channel.h
* @brief Thread-safe, bounded multi-producer/multi-consumer synchronous
* channel.
* @brief Thread-safe, bounded multi-producer/multi-consumer synchronous channel.
*
* Intent: Enables asynchronous inter-thread communication with backpressure.
* Models a synchronous channel where producers/consumers block on capacity
* limits.
* Models a synchronous channel where producers/consumers block on capacity limits.
*/
/**

View File

@@ -1,4 +1,4 @@
#include "concurrency/bounded_channel.h"
#include "bounded_channel.h"
template <typename T>
void BoundedChannel<T>::Send(T item) {

View File

@@ -28,16 +28,12 @@ class DataGenerator {
const std::string& region_context) = 0;
/**
* @brief Generates a user profile grounded in a sampled name and persona.
* @brief Generates a user profile for a locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype grounding the generated bio.
* @param name Sampled first/last name (and gender)
* @param locale Locale hint used by generator.
* @return User generation result.
*/
virtual UserResult GenerateUser(const EnrichedCity& city,
const UserPersona& persona,
const Name& name) = 0;
virtual UserResult GenerateUser(const std::string& locale) = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_DATA_GENERATOR_H_

View File

@@ -1,52 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_JSON_GRAMMARS_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_JSON_GRAMMARS_H_
/**
* @file data_generation/json_grammars.h
* @brief GBNF grammars constraining structured JSON output from
* LlamaGenerator inference calls.
*/
#include <string_view>
// GBNF grammar for structured user JSON output.
// thought-block permits the model to emit free-form reasoning before the
// JSON object (the prompts explicitly invite this); only the "{...}" tail is
// constrained to the expected shape.
inline constexpr std::string_view kUserJsonGrammar = R"json_user(
root ::= thought-block (
"{" ws
"\"username\"" ws ":" ws string ws "," ws
"\"bio\"" ws ":" ws string ws "," ws
"\"activity_weight\"" ws ":" ws number ws
"}" ws
)
thought-block ::= [^{]*
ws ::= [ \t\n\r]*
string ::= "\"" char+ "\""
char ::= [^"\\\x7F\x00-\x1F] | [\\] escape
escape ::= ["\\/bfnrt] | "u" hex hex hex hex
hex ::= [0-9a-fA-F]
number ::= "-"? ("0" | [1-9] [0-9]*) ("." [0-9]+)?
)json_user";
// GBNF grammar for structured brewery JSON output (see thought-block note
// above).
inline constexpr std::string_view kBreweryJsonGrammar = R"json_brewery(
root ::= thought-block (
"{" ws
"\"name_en\"" ws ":" ws string ws "," ws
"\"description_en\"" ws ":" ws string ws "," ws
"\"name_local\"" ws ":" ws string ws "," ws
"\"description_local\"" ws ":" ws string ws
"}" ws
)
thought-block ::= [^{]*
ws ::= [ \t\n\r]*
string ::= "\"" char+ "\""
char ::= [^"\\\x7F\x00-\x1F] | [\\] escape
escape ::= ["\\/bfnrt] | "u" hex hex hex hex
hex ::= [0-9a-fA-F]
)json_brewery";
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_JSON_GRAMMARS_H_

View File

@@ -1,23 +1,24 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_H_
#include <filesystem>
/**
* @file data_generation/llama_generator.h
* @brief llama.cpp-backed implementation of DataGenerator.
*/
#include <cstdint>
#include <filesystem>
#include <memory>
#include <random>
#include <string>
#include <string_view>
#include "../services/prompting/prompt_directory.h"
#include "data_generation/data_generator.h"
#include "data_generation/prompt_formatting/prompt_formatter.h"
#include "data_model/models.h"
#include "services/logging/logger.h"
#include "services/prompting/prompt_directory.h"
struct llama_model;
struct llama_context;
@@ -37,15 +38,21 @@ class LlamaGenerator final : public DataGenerator {
* @param prompt_directory Directory service for loading named prompt files.
*/
LlamaGenerator(const ApplicationOptions& options,
const std::string& model_path, std::shared_ptr<ILogger> logger,
const std::string& model_path,
std::shared_ptr<ILogger> logger,
std::unique_ptr<IPromptFormatter> prompt_formatter,
std::unique_ptr<IPromptDirectory> prompt_directory);
~LlamaGenerator() override;
// disable copy constructor
LlamaGenerator(const LlamaGenerator&) = delete;
// disable copy assignment operator
LlamaGenerator& operator=(const LlamaGenerator&) = delete;
// disable move constructor
LlamaGenerator(LlamaGenerator&&) = delete;
// disable move assignment operator
LlamaGenerator& operator=(LlamaGenerator&&) = delete;
/**
@@ -59,15 +66,12 @@ class LlamaGenerator final : public DataGenerator {
const std::string& region_context) override;
/**
* @brief Generates a user profile grounded in a sampled name and persona.
* @brief Generates a user profile for the provided locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype grounding the generated bio.
* @param name Sampled first/last name -- not LLM-invented.
* @param locale Locale hint.
* @return Generated user profile.
*/
UserResult GenerateUser(const EnrichedCity& city, const UserPersona& persona,
const Name& name) override;
UserResult GenerateUser(const std::string& locale) override;
private:
static constexpr int32_t kDefaultMaxTokens = 10000;

View File

@@ -47,18 +47,4 @@ void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
std::optional<std::string> ValidateBreweryJson(const std::string& raw,
BreweryResult& brewery_out);
/**
* @brief Validates and parses user JSON output.
*
* Only populates `username`, `bio`, and `activity_weight` -- `first_name`
* and `last_name` are not LLM-authored and are set separately from the
* sampled Name.
*
* @param raw Raw model output.
* @param user_out Parsed user payload.
* @return Validation error message if invalid, or std::nullopt on success.
*/
std::optional<std::string> ValidateUserJson(const std::string& raw,
UserResult& user_out);
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_HELPERS_H_

View File

@@ -28,16 +28,12 @@ class MockGenerator final : public DataGenerator {
const std::string& region_context) override;
/**
* @brief Generates deterministic user data grounded in a sampled name and
* persona.
* @brief Generates deterministic user data for a locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype.
* @param name Sampled first/last name, copied directly into the result.
* @param locale Locale hint.
* @return Generated user result.
*/
UserResult GenerateUser(const EnrichedCity& city, const UserPersona& persona,
const Name& name) override;
UserResult GenerateUser(const std::string& locale) override;
private:
/**
@@ -48,25 +44,12 @@ class MockGenerator final : public DataGenerator {
*/
static size_t DeterministicHash(const Location& location);
/**
* @brief Combines city, persona, and name into a stable hash value.
*
* @param location City and country names.
* @param persona Persona archetype.
* @param name Sampled first/last name.
* @return Deterministic hash value.
*/
static size_t DeterministicHash(const Location& location,
const UserPersona& persona, const Name& name);
// Hash stride constants for deterministic distribution across fixed-size
// arrays. These coprime strides spread hash values uniformly without
// clustering, ensuring diverse output across different hash inputs.
static constexpr size_t kNounHashStride = 7;
static constexpr size_t kDescriptionHashStride = 13;
static constexpr size_t kBioHashStride = 11;
static constexpr size_t kActivityWeightHashStride = 17;
static constexpr size_t kActivityWeightHashRange = 1000;
static constexpr std::array<std::string_view, 18> kBreweryAdjectives = {
"Craft", "Heritage", "Local", "Artisan", "Pioneer", "Golden",
@@ -141,8 +124,7 @@ class MockGenerator final : public DataGenerator {
"Visits breweries for the stories, stays for the flagship pours.",
"Craft beer fan mapping tasting notes and favorite brew routes.",
"Always ready to trade recommendations for underrated local breweries.",
"Keeping a running list of must-try collab releases and tap "
"takeovers."};
"Keeping a running list of must-try collab releases and tap takeovers."};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_MOCK_GENERATOR_H_

View File

@@ -3,8 +3,8 @@
/**
* @file data_model/generated_models.h
* @brief Generated output models from the pipeline: brewery/user results,
* enriched data, and complete generation results.
* @brief Generated output models from the pipeline: brewery/user results, enriched data,
* and complete generation results.
*/
#include <string>
@@ -19,24 +19,16 @@
* @brief Generated brewery payload.
*/
struct BreweryResult {
/**
* @brief Brewery display name in English.
*/
/// @brief Brewery display name in English.
std::string name_en;
/**
* @brief Brewery description text in English.
*/
/// @brief Brewery description text in English.
std::string description_en;
/**
* @brief Brewery display name in the local language.
*/
/// @brief Brewery display name in the local language.
std::string name_local;
/**
* @brief Brewery description text in the local language.
*/
/// @brief Brewery description text in the local language.
std::string description_local;
};
@@ -44,39 +36,11 @@ struct BreweryResult {
* @brief Generated user profile payload.
*/
struct UserResult {
/**
* @brief First (given) name, copied from the sampled Name -- not
* LLM-invented.
*/
std::string first_name{};
/**
* @brief Last (family) name, copied from the sampled Name -- not
* LLM-invented.
*/
std::string last_name{};
/**
* @brief Gender associated with the sampled first name, copied from the
* sampled Name -- not LLM-invented.
*/
std::string gender{};
/**
* @brief Username handle.
*/
/// @brief Username handle.
std::string username{};
/**
* @brief Short user biography.
*/
/// @brief Short user biography.
std::string bio{};
/**
* @brief Relative check-in/activity weight for this user, used to drive
* a J-curve activity profile in later pipeline phases.
*/
float activity_weight{};
};
// ============================================================================
@@ -94,23 +58,9 @@ struct EnrichedCity {
/**
* @brief Helper struct to store generated brewery data.
*/
struct BreweryRecord {
struct GeneratedBrewery {
Location location;
BreweryResult brewery;
};
/**
* @brief Helper struct to store generated user data.
*
* `email` and `date_of_birth` are programmatically generated by the
* orchestrator (never LLM-authored) so a downstream auth-account seeding
* consumer can register real accounts from the pipeline's SQLite export.
*/
struct UserRecord {
Location location;
UserResult user;
std::string email{};
std::string date_of_birth{};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATED_MODELS_H_

View File

@@ -10,7 +10,6 @@
#include <boost/program_options.hpp>
#include <cstdint>
#include <filesystem>
#include <functional>
#include <memory>
#include <optional>
#include <string>
@@ -29,106 +28,40 @@ namespace prog_opts = boost::program_options;
* @brief Canonical location record for city-level generation.
*/
struct Location {
/// @brief City name.
std::string city{};
/// @brief State or province name.
std::string state_province{};
/**
* @brief ISO 3166-2 subdivision code.
*/
/// @brief ISO 3166-2 subdivision code.
std::string iso3166_2{};
/// @brief Country name.
std::string country{};
/**
* @brief ISO 3166-1 country code.
*/
/// @brief ISO 3166-1 country code.
std::string iso3166_1{};
/**
* @brief Local language codes in priority order.
*/
/// @brief Local language codes in priority order.
std::vector<std::string> local_languages{};
/**
* @brief Latitude in decimal degrees.
*/
/// @brief Latitude in decimal degrees.
double latitude{};
/**
* @brief Longitude in decimal degrees.
*/
/// @brief Longitude in decimal degrees.
double longitude{};
};
// ============================================================================
// Name / Persona Models
// ============================================================================
/**
* @brief A sampled first/last name pair, with the source forename's gender.
*
* Produced by the SampleName() helper in generate_users.cc.
* @brief Non-owning brewery location input.
*/
struct Name {
std::string first_name{};
std::string last_name{};
struct BreweryLocation {
/// @brief City name.
std::string_view city_name;
/**
* @brief Gender associated with the sampled forename (e.g. "M", "F"), as
* reported by the source dataset.
*/
std::string gender{};
};
/**
* @brief A single forename entry from the names-by-country fixture data.
*/
struct ForenameEntry {
/**
* @brief Romanized forename.
*/
std::string name{};
/**
* @brief Gender associated with this forename, as reported by the source
* dataset (e.g. "M", "F").
*/
std::string gender{};
bool operator==(const ForenameEntry& other) const {
return name == other.name && gender == other.gender;
}
};
namespace std {
template <>
struct hash<ForenameEntry> {
size_t operator()(const ForenameEntry& entry) const noexcept {
const size_t name_hash = std::hash<std::string>{}(entry.name);
const size_t gender_hash = std::hash<std::string>{}(entry.gender);
return name_hash ^ (gender_hash << 1);
}
};
} // namespace std
/**
* @brief A persona archetype used to ground LLM-generated user bios.
*/
struct UserPersona {
/**
* @brief Persona display name (e.g. "Hophead Explorer").
*/
std::string name{};
/**
* @brief Short description of the persona's interests and voice.
*/
std::string description{};
/**
* @brief Beer styles this persona gravitates toward.
*/
std::vector<std::string> style_affinities{};
/// @brief Country name.
std::string_view country_name;
};
// ============================================================================
@@ -139,34 +72,22 @@ struct UserPersona {
* @brief LLM sampling parameters.
*/
struct SamplingOptions {
/**
* @brief LLM sampling temperature (higher = more random).
*/
/// @brief LLM sampling temperature (0.0 to 1.0, higher = more random).
float temperature = 1.0F;
/**
* @brief LLM nucleus sampling top-p parameter.
*/
/// @brief LLM nucleus sampling top-p parameter.
float top_p = 0.95F;
/**
* @brief LLM top-k sampling parameter.
*/
/// @brief LLM top-k sampling parameter.
uint32_t top_k = 64;
/**
* @brief Context window size (tokens).
*/
/// @brief Context window size (tokens).
uint32_t n_ctx = 8192;
/**
* @brief Random seed (-1 for random, otherwise non-negative).
*/
/// @brief Random seed (-1 for random, otherwise non-negative).
int seed = -1;
/**
* @brief Number of layers to offload to GPU.
*/
/// @brief Number of layers to offload to GPU.
int n_gpu_layers = 0;
};
@@ -174,20 +95,16 @@ struct SamplingOptions {
* @brief Configuration for the LLM generator component.
*/
struct GeneratorOptions {
/**
* @brief Path to the LLM model file (gguf format).
*/
/// @brief Path to the LLM model file (gguf format).
std::filesystem::path model_path;
/**
* @brief Use mocked generator instead of actual LLM inference.
*/
/// @brief Use mocked generator instead of actual LLM inference.
bool use_mocked = false;
/**
* @brief Specific sampling parameters for this generator.
* If nullopt, the application should use global defaults.
*/
/// @brief Specific sampling parameters for this generator.
/// If nullopt, the application should use global defaults.
std::optional<SamplingOptions> sampling;
};
@@ -195,23 +112,18 @@ struct GeneratorOptions {
* @brief Configuration for the pipeline execution and output.
*/
struct PipelineOptions {
/**
* @brief Directory for generated artifacts.
*/
/// @brief Directory for generated artifacts.
std::filesystem::path output_path;
/**
* @brief Directory that contains named prompt files (e.g.
* BREWERY_GENERATION.md).
*/
/// @brief Directory that contains named prompt files (e.g.
/// BREWERY_GENERATION.md).
std::filesystem::path prompt_dir;
/// @brief Path for application logs.
std::filesystem::path log_path;
/**
* @brief Number of locations to sample from the dataset
* More locations -> more users/more breweries
*/
/// @brief Number of locations to sample from the dataset
/// More locations -> more users/more breweries
uint32_t location_count;
};
@@ -227,7 +139,7 @@ struct ApplicationOptions {
// Function Declarations
// ============================================================================
std::optional<ApplicationOptions> ParseArguments(
const int argc, char** argv, std::shared_ptr<ILogger> logger = nullptr);
std::optional<ApplicationOptions> ParseArguments(const int argc, char** argv,
std::shared_ptr<ILogger> logger = nullptr);
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_MODELS_H_

View File

@@ -0,0 +1,25 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
/**
* @file json_handling/json_loader.h
* @brief Loader API for curated location data.
*/
#include <filesystem>
#include <memory>
#include <vector>
#include "data_model/models.h"
#include "services/logging/logger.h"
/// @brief Loads curated world locations from a JSON file into memory.
class JsonLoader {
public:
/// @brief Parses a JSON array file and returns all location records.
static std::vector<Location> LoadLocations(
const std::filesystem::path& filepath,
std::shared_ptr<ILogger> logger = nullptr);
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_

View File

@@ -1,109 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_PRETTY_PRINT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_PRETTY_PRINT_H_
/**
* @file json_handling/pretty_print.h
* @brief Pretty-printing utilities for JSON values.
*
* Provides formatting capability for boost::json::value with indentation and
* readable output. Adapted from Boost JSON library examples.
*/
#include <boost/json.hpp>
#include <ostream>
#include <string>
/**
* @brief Pretty-prints a JSON value to an output stream with indentation.
*
* Recursively formats JSON objects and arrays with consistent 4-space
* indentation. Adapted from:
* https://raw.githubusercontent.com/boostorg/json/refs/heads/develop/example/pretty.cpp
*
* @param outstream Output stream to write formatted JSON.
* @param json_val JSON value to format.
* @param indent Optional indentation string (managed internally on first call).
*/
inline void PrettyPrint(std::ostream& outstream,
boost::json::value const& json_val,
std::string* indent = nullptr) {
std::string str;
if (indent == nullptr) {
indent = &str;
}
switch (json_val.kind()) {
case boost::json::kind::object: {
outstream << "{\n";
indent->append(4, ' ');
auto const& obj = json_val.get_object();
if (!obj.empty()) {
const auto* iter = obj.begin();
for (;;) {
outstream << *indent << boost::json::serialize(iter->key()) << " : ";
PrettyPrint(outstream, iter->value(), indent);
iter = std::next(iter);
if (iter == obj.end()) {
break;
}
outstream << ",\n";
}
}
outstream << "\n";
indent->resize(indent->size() - 4);
outstream << *indent << "}";
break;
}
case boost::json::kind::array: {
outstream << "[\n";
indent->append(4, ' ');
auto const& arr = json_val.get_array();
if (!arr.empty()) {
const auto* iter = arr.begin();
for (;;) {
outstream << *indent;
PrettyPrint(outstream, *iter, indent);
iter = std::next(iter);
if (iter == arr.end()) {
break;
}
outstream << ",\n";
}
}
outstream << "\n";
indent->resize(indent->size() - 4);
outstream << *indent << "]";
break;
}
case boost::json::kind::string: {
outstream << serialize(json_val.get_string());
break;
}
case boost::json::kind::uint64:
case boost::json::kind::int64:
case boost::json::kind::double_:
outstream << json_val;
break;
case boost::json::kind::bool_:
if (json_val.get_bool()) {
outstream << "true";
} else {
outstream << "false";
}
break;
case boost::json::kind::null:
outstream << "null";
break;
}
if (indent->empty()) {
outstream << "\n";
}
}
#endif // BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_PRETTY_PRINT_H_

View File

@@ -16,10 +16,16 @@
*/
class LlamaBackendState {
public:
/// @brief Initializes global llama backend state.
LlamaBackendState() { llama_backend_init(); }
/// @brief Cleans up global llama backend state.
~LlamaBackendState() { llama_backend_free(); }
/// @brief Non-copyable type.
LlamaBackendState(const LlamaBackendState&) = delete;
/// @brief Non-copyable type.
LlamaBackendState& operator=(const LlamaBackendState&) = delete;
};

View File

@@ -1,50 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_CURATED_DATA_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_CURATED_DATA_SERVICE_H_
/**
* @file services/curated_data/curated_data_service.h
* @brief Abstraction for loading curated location, persona, and name data.
*/
#include <filesystem>
#include <unordered_map>
#include <unordered_set>
#include <vector>
#include "data_model/models.h"
using ForenameList = std::vector<ForenameEntry>;
using SurnameList = std::vector<std::string>;
using LocationsList = std::vector<Location>;
using PersonasList = std::vector<UserPersona>;
using ForenamesByCountryMap = std::unordered_map<std::string, ForenameList>;
using SurnamesByCountryMap = std::unordered_map<std::string, SurnameList>;
/**
* @brief Interface for services that load curated data used to seed
* brewery/user generation.
*/
class ICuratedDataService {
public:
ICuratedDataService() = default;
virtual ~ICuratedDataService() = default;
ICuratedDataService(const ICuratedDataService&) = delete;
ICuratedDataService& operator=(const ICuratedDataService&) = delete;
ICuratedDataService(ICuratedDataService&&) = delete;
ICuratedDataService& operator=(ICuratedDataService&&) = delete;
/**
* @brief Loads all curated location records.
*/
virtual const LocationsList& LoadLocations() = 0;
/**
* @brief Loads all curated persona records.
*/
virtual const PersonasList& LoadPersonas() = 0;
virtual const ForenamesByCountryMap& LoadForenamesByCountry() = 0;
virtual const SurnamesByCountryMap& LoadSurnamesByCountry() = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_CURATED_DATA_SERVICE_H_

View File

@@ -1,63 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
/**
* @file json_handling/json_loader.h
* @brief JSON-backed implementation of ICuratedDataService.
*/
#include <filesystem>
#include "data_model/models.h"
#include "services/curated_data/curated_data_service.h"
/**
* @brief File locations for the curated JSON fixtures consumed by
* CuratedJsonDataService.
*/
struct CuratedDataFilePaths {
std::filesystem::path locations_path;
std::filesystem::path personas_path;
std::filesystem::path forenames_path;
std::filesystem::path surnames_path;
};
/**
* @brief Loads curated location, persona, and name data from JSON files.
*/
class CuratedJsonDataService final : public ICuratedDataService {
struct cache {
LocationsList locations;
PersonasList personas;
ForenamesByCountryMap forenames_by_country;
SurnamesByCountryMap surnames_by_country;
cache() = default;
~cache() = default;
};
CuratedDataFilePaths filepaths_;
cache cache_;
public:
explicit CuratedJsonDataService(CuratedDataFilePaths filepaths);
/**
* @brief Parses a JSON array file and returns all location records.
*/
const LocationsList& LoadLocations() override;
/**
* @brief Parses a JSON array file and returns all persona records.
*/
const PersonasList& LoadPersonas() override;
const ForenamesByCountryMap& LoadForenamesByCountry() override;
/**
* @brief Parses a JSON file and returns all the forenames per country.
*/
const SurnamesByCountryMap& LoadSurnamesByCountry() override;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_

View File

@@ -1,38 +0,0 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_MOCK_CURATED_DATA_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_MOCK_CURATED_DATA_SERVICE_H_
/**
* @file services/curated_data/mock_curated_data_service.h
* @brief In-memory ICuratedDataService backed by a small fixed dataset, used
* when file-backed curated data is disabled (mock mode).
*/
#include <filesystem>
#include "data_model/models.h"
#include "services/curated_data/curated_data_service.h"
/**
* @brief Curated data service returning a small fixed in-memory dataset in
* place of the JSON fixture files used by JsonLoader.
*/
class MockCuratedDataService final : public ICuratedDataService {
public:
MockCuratedDataService();
const LocationsList& LoadLocations() override;
const PersonasList& LoadPersonas() override;
const ForenamesByCountryMap& LoadForenamesByCountry() override;
const SurnamesByCountryMap& LoadSurnamesByCountry() override;
private:
LocationsList locations_;
PersonasList personas_;
ForenamesByCountryMap forenames_by_country_;
SurnamesByCountryMap surnames_by_country_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_CURATED_DATA_MOCK_CURATED_DATA_SERVICE_H_

View File

@@ -16,6 +16,8 @@
class IExportService {
public:
IExportService() = default;
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IExportService() = default;
IExportService(const IExportService&) = delete;
@@ -23,9 +25,7 @@ class IExportService {
IExportService(IExportService&&) = delete;
IExportService& operator=(IExportService&&) = delete;
/**
* @brief Prepares the export destination for a new run.
*/
/// @brief Prepares the export destination for a new run.
virtual void Initialize() = 0;
/**
@@ -33,18 +33,9 @@ class IExportService {
*
* @param brewery Generated brewery payload to store.
*/
virtual uint64_t ProcessRecord(const BreweryRecord& brewery) = 0;
virtual uint64_t ProcessRecord(const GeneratedBrewery& brewery) = 0;
/**
* @brief Persists one generated user record.
*
* @param user Generated user payload to store.
*/
virtual uint64_t ProcessRecord(const UserRecord& user) = 0;
/**
* @brief Finalizes the export destination.
*/
/// @brief Finalizes the export destination.
virtual void Finalize() = 0;
};

View File

@@ -12,7 +12,7 @@
#include <string>
#include <string_view>
#include "services/database/sqlite_handle_types.h"
#include "sqlite_handle_types.h"
namespace sqlite_export_service_internal {

View File

@@ -12,9 +12,9 @@
#include <unordered_map>
#include "data_model/models.h"
#include "services/database/export_service.h"
#include "services/database/sqlite_export_service_helpers.h"
#include "services/datetime/date_time_provider.h"
#include "../datetime/date_time_provider.h"
#include "export_service.h"
#include "sqlite_export_service_helpers.h"
/**
* @brief Persists generated brewery records into a fresh SQLite database.
@@ -30,8 +30,7 @@ class SqliteExportService final : public IExportService {
SqliteExportService& operator=(SqliteExportService&&) = delete;
void Initialize() override;
uint64_t ProcessRecord(const BreweryRecord& brewery) override;
uint64_t ProcessRecord(const UserRecord& user) override;
uint64_t ProcessRecord(const GeneratedBrewery& brewery) override;
void Finalize() override;
private:
@@ -47,15 +46,6 @@ class SqliteExportService final : public IExportService {
[[nodiscard]] std::filesystem::path BuildDatabasePath() const;
[[nodiscard]] static std::string BuildLocationKey(const Location& location);
/**
* @brief Returns the row id for @p location, inserting it first if it has
* not already been seen during this run.
*
* Shared by both ProcessRecord() overloads so breweries and users
* referencing the same location resolve to the same row.
*/
[[nodiscard]] sqlite3_int64 ResolveLocationId(const Location& location);
std::unique_ptr<IDateTimeProvider> date_time_provider_;
std::filesystem::path output_path_;
std::string run_timestamp_utc_;
@@ -63,7 +53,6 @@ class SqliteExportService final : public IExportService {
SqliteDatabaseHandle db_handle_;
SqliteStatementHandle insert_location_stmt_;
SqliteStatementHandle insert_brewery_stmt_;
SqliteStatementHandle insert_user_stmt_;
bool transaction_open_ = false;
std::unordered_map<std::string, sqlite3_int64> location_cache_;
};

View File

@@ -3,8 +3,8 @@
/* Umbrella header for backward compatibility. */
#include "services/database/sqlite_connection_helpers.h"
#include "services/database/sqlite_handle_types.h"
#include "services/database/sqlite_statement_helpers.h"
#include "sqlite_connection_helpers.h"
#include "sqlite_handle_types.h"
#include "sqlite_statement_helpers.h"
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_SQLITE_EXPORT_SERVICE_HELPERS_H_

View File

@@ -24,10 +24,8 @@ using SqliteDatabaseHandle = std::unique_ptr<sqlite3, SqliteDatabaseDeleter>;
using SqliteStatementHandle =
std::unique_ptr<sqlite3_stmt, SqliteStatementDeleter>;
// Represents a parameter that is bound to a prepared SQLite statement.
// N.B. indices are 1 based.
template <typename T>
struct BoundParam {
struct BindParam {
int index;
T value;
std::string_view action;

View File

@@ -13,7 +13,7 @@
#include <string_view>
#include <vector>
#include "services/database/sqlite_handle_types.h"
#include "sqlite_handle_types.h"
namespace sqlite_export_service_internal {
@@ -50,26 +50,6 @@ CREATE INDEX IF NOT EXISTS idx_breweries_location_id ON breweries(location_id);
)sql";
inline constexpr std::string_view kCreateUsersTableSql = R"sql(
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
location_id INTEGER NOT NULL,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL,
gender TEXT NOT NULL,
username TEXT NOT NULL,
bio TEXT NOT NULL,
activity_weight REAL NOT NULL,
email TEXT NOT NULL UNIQUE,
date_of_birth TEXT NOT NULL,
FOREIGN KEY(location_id) REFERENCES locations(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_users_location_id ON users(location_id);
)sql";
inline constexpr std::string_view kInsertLocationSql = R"sql(
INSERT INTO locations (
city,
@@ -93,52 +73,20 @@ INSERT INTO breweries (
) VALUES (?, ?, ?, ?, ?);
)sql";
inline constexpr std::string_view kInsertUserSql = R"sql(
INSERT INTO users (
location_id,
first_name,
last_name,
gender,
username,
bio,
activity_weight,
email,
date_of_birth
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?);
)sql";
inline constexpr int kLocationCityBindIndex = 1;
inline constexpr int kLocationStateProvinceBindIndex = 2;
inline constexpr int kLocationIso31662BindIndex = 3;
inline constexpr int kLocationCountryBindIndex = 4;
inline constexpr int kLocationIso31661BindIndex = 5;
inline constexpr int kLocationLanguagesBindIndex = 6;
inline constexpr int kLocationLatitudeBindIndex = 7;
inline constexpr int kLocationLongitudeBindIndex = 8;
// sqlite3_bind_*() parameter indices are 1-based, matching the "?"
// placeholder order in the SQL above.
enum LocationBindIndex {
kLocationCityBindIndex = 1,
kLocationStateProvinceBindIndex,
kLocationIso31662BindIndex,
kLocationCountryBindIndex,
kLocationIso31661BindIndex,
kLocationLanguagesBindIndex,
kLocationLatitudeBindIndex,
kLocationLongitudeBindIndex,
};
enum BreweryBindIndex {
kBreweryLocationIdBindIndex = 1,
kBreweryEnglishNameBindIndex,
kBreweryEnglishDescriptionBindIndex,
kBreweryLocalNameBindIndex,
kBreweryLocalDescriptionBindIndex,
};
enum UserBindIndex {
kUserLocationIdBindIndex = 1,
kUserFirstNameBindIndex,
kUserLastNameBindIndex,
kUserGenderBindIndex,
kUserUsernameBindIndex,
kUserBioBindIndex,
kUserActivityWeightBindIndex,
kUserEmailBindIndex,
kUserDateOfBirthBindIndex,
};
inline constexpr int kBreweryLocationIdBindIndex = 1;
inline constexpr int kBreweryEnglishNameBindIndex = 2;
inline constexpr int kBreweryEnglishDescriptionBindIndex = 3;
inline constexpr int kBreweryLocalNameBindIndex = 4;
inline constexpr int kBreweryLocalDescriptionBindIndex = 5;
SqliteStatementHandle PrepareStatement(const SqliteDatabaseHandle& db_handle,
std::string_view sql,
@@ -147,13 +95,13 @@ SqliteStatementHandle PrepareStatement(const SqliteDatabaseHandle& db_handle,
void ResetStatement(SqliteStatementHandle& statement);
void Bind(const SqliteStatementHandle& statement,
const BoundParam<std::string_view>& param);
const BindParam<std::string_view>& param);
void Bind(const SqliteStatementHandle& statement,
const BoundParam<double>& param);
const BindParam<double>& param);
void Bind(const SqliteStatementHandle& statement,
const BoundParam<sqlite3_int64>& param);
const BindParam<sqlite3_int64>& param);
void StepStatement(const SqliteDatabaseHandle& db_handle,
const SqliteStatementHandle& statement,

View File

@@ -18,6 +18,7 @@
*/
class IDateTimeProvider {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IDateTimeProvider() = default;
IDateTimeProvider() = default;

View File

@@ -15,6 +15,7 @@
*/
class IEnrichmentService {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IEnrichmentService() = default;
/**

View File

@@ -1,18 +1,13 @@
//
// Created by aaronpo on 13/05/2026.
//
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_MOCK_ENRICHMENT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_MOCK_ENRICHMENT_H_
/**
* @file services/enrichment/mock_enrichment.h
* @brief No-op IEnrichmentService used when network enrichment is disabled.
*/
#include <string>
#include "services/enrichment/enrichment_service.h"
#include "enrichment_service.h"
/**
* @brief Enrichment service that returns no context for any location.
*/
class MockEnrichmentService final : public IEnrichmentService {
public:
std::string GetLocationContext(const Location& /*loc*/) override {

View File

@@ -11,37 +11,25 @@
#include <string_view>
#include <unordered_map>
#include "services/enrichment/enrichment_service.h"
#include "enrichment_service.h"
#include "services/logging/logger.h"
#include "web_client/web_client.h"
/**
* @brief Provides Wikipedia summary lookups backed by cached raw extracts.
*/
/// @brief Provides Wikipedia summary lookups backed by cached raw extracts.
class WikipediaEnrichmentService final : public IEnrichmentService {
public:
/**
* @brief Creates a new Wikipedia service with the provided web client.
*/
/// @brief Creates a new Wikipedia service with the provided web client.
explicit WikipediaEnrichmentService(std::unique_ptr<WebClient> client,
std::shared_ptr<ILogger> logger);
/**
* @brief Returns the Wikipedia-derived context for a location.
*/
/// @brief Returns the Wikipedia-derived context for a location.
[[nodiscard]] std::string GetLocationContext(const Location& loc) override;
private:
std::string FetchExtract(std::string_view query);
std::unique_ptr<WebClient> client_;
std::shared_ptr<ILogger> logger_;
/**
* @brief Cache for raw Wikipedia query extracts, keyed by query string.
*
* GetLocationContext() always queries "brewing" and reuses "beer in
* {country}" for every city in that country, so caching avoids refetching
* the same extract across locations in a run.
*/
/// @brief Canonical cache for raw Wikipedia query extracts.
std::unordered_map<std::string, std::string> extract_cache_;
};

View File

@@ -6,8 +6,8 @@
* them to spdlog on a dedicated thread.
*/
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_DISPATCHER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_DISPATCHER_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_CONSUMER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_CONSUMER_H_
#include <spdlog/spdlog.h>
@@ -23,31 +23,31 @@
*/
class LogDispatcher {
public:
/**
* @brief Construct a log dispatcher.
*
* @param channel Reference to the bounded channel used for log retrieval.
*/
explicit LogDispatcher(BoundedChannel<LogEntry>& channel);
/**
* @brief Construct a log dispatcher.
*
* @param channel Reference to the bounded channel used for log retrieval.
*/
explicit LogDispatcher(BoundedChannel<LogEntry>& channel);
LogDispatcher(const LogDispatcher&) = delete;
LogDispatcher& operator=(const LogDispatcher&) = delete;
LogDispatcher(LogDispatcher&&) = delete;
LogDispatcher& operator=(LogDispatcher&&) = delete;
~LogDispatcher() = default;
LogDispatcher(const LogDispatcher&) = delete;
LogDispatcher& operator=(const LogDispatcher&) = delete;
LogDispatcher(LogDispatcher&&) = delete;
LogDispatcher& operator=(LogDispatcher&&) = delete;
~LogDispatcher() = default;
/**
* @brief Drain the channel and forward entries to spdlog.
*
* Intended to be called once on a dedicated thread. The loop returns after
* the channel has been closed and all queued entries have been processed.
*/
void Run();
/**
* @brief Drain the channel and forward entries to spdlog.
*
* Intended to be called once on a dedicated thread. The loop returns after
* the channel has been closed and all queued entries have been processed.
*/
void Run();
private:
BoundedChannel<LogEntry>& channel_;
BoundedChannel<LogEntry>& channel_;
static spdlog::level::level_enum ToSpdlogLevel(LogLevel level);
static spdlog::level::level_enum ToSpdlogLevel(LogLevel level);
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_DISPATCHER_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_CONSUMER_H_

View File

@@ -10,32 +10,18 @@
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_ENTRY_H_
#include <chrono>
#include <source_location>
#include <string>
#include <thread>
#include <vector>
#include <string>
/**
* @enum LogLevel
* @brief Severity levels supported by the logging infra.
*/
enum class LogLevel {
/**
* @brief Development/debugging information.
*/
Debug,
/**
* @brief General informational messages.
*/
Info,
/**
* @brief Warning conditions.
*/
Warn,
/**
* @brief Error conditions.
*/
Error,
Debug, ///< Development/debugging information.
Info, ///< General informational messages.
Warn, ///< Warning conditions.
Error, ///< Error conditions.
};
/**
@@ -46,49 +32,13 @@ enum class LogLevel {
* pipeline that emitted it.
*/
enum class PipelinePhase {
/**
* @brief Initialization and validation.
*/
Startup,
/**
* @brief Location/context enrichment (e.g. Wikipedia lookups).
*/
Enrichment,
/**
* @brief User profile generation.
*/
UserGeneration,
/**
* @brief Brewery and beer data generation.
*/
BreweryAndBeerGeneration,
/**
* @brief Checkin (visit) record generation.
*/
CheckinGeneration,
/**
* @brief Rating and review generation.
*/
RatingGeneration,
/**
* @brief Follow relationship generation.
*/
FollowGeneration,
/**
* @brief Finalization and cleanup.
*/
Teardown,
};
/**
* @struct LogDTO
* @brief User-provided subset of log fields. Used to capture call-site info
* transparently.
*/
struct LogDTO {
LogLevel level;
PipelinePhase phase;
std::string message;
Startup, ///< Initialization and validation.
UserGeneration, ///< User profile generation.
BreweryAndBeerGeneration, ///< Brewery and beer data generation.
CheckinGeneration, ///< Checkin (visit) record generation.
RatingGeneration, ///< Rating and review generation.
FollowGeneration, ///< Follow relationship generation.
Teardown, ///< Finalization and cleanup.
};
/**
@@ -97,40 +47,22 @@ struct LogDTO {
*
* All fields are value types, which keeps transfer across the bounded channel
* simple and avoids shared ownership.
*
* NOTE: timestamp, thread_id, and origin must be populated by ILogger::Log()
* before the entry is dispatched.
*/
struct LogEntry {
/**
* @brief Timestamp when the entry was created.
*/
std::chrono::system_clock::time_point timestamp{};
/// @brief Timestamp when the entry was created.
std::chrono::system_clock::time_point timestamp =
std::chrono::system_clock::now();
/**
* @brief Source location where the log call was made.
*/
std::source_location origin{};
/**
* @brief Thread responsible for emitting the log.
*/
std::thread::id thread_id{};
/**
* @brief Severity level of this entry.
*/
/// @brief Severity level of this entry.
LogLevel level;
/**
* @brief Pipeline phase associated with the entry.
*/
/// @brief Pipeline phase associated with the entry.
PipelinePhase phase;
/**
* @brief Log message text.
*/
/// @brief Log message text.
std::string message;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_ENTRY_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_ENTRY_H_

View File

@@ -6,8 +6,8 @@
* a bounded channel for later processing by the dispatcher.
*/
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_PRODUCER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_PRODUCER_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_CHANNEL_LOGGER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_CHANNEL_LOGGER_H_
#include <string_view>
@@ -24,30 +24,30 @@
*/
class LogProducer final : public ILogger {
public:
/**
* @brief Construct a channel-backed producer.
*
* @param channel Reference to the bounded channel used for log transfer.
*/
explicit LogProducer(BoundedChannel<LogEntry>& channel);
/**
* @brief Construct a channel-backed producer.
*
* @param channel Reference to the bounded channel used for log transfer.
*/
explicit LogProducer(BoundedChannel<LogEntry>& channel);
LogProducer(const LogProducer&) = delete;
LogProducer& operator=(const LogProducer&) = delete;
LogProducer(LogProducer&&) = delete;
LogProducer& operator=(LogProducer&&) = delete;
LogProducer(const LogProducer&) = delete;
LogProducer& operator=(const LogProducer&) = delete;
LogProducer(LogProducer&&) = delete;
LogProducer& operator=(LogProducer&&) = delete;
~LogProducer() override = default;
~LogProducer() override = default;
/**
* @brief Queue a log message for asynchronous processing.
*
* Blocks while the channel applies backpressure. This blocking behavior
* under heavy load is an accepted trade-off for simplicity.
*/
void DoLog(LogEntry log_entry) override;
/**
* @brief Queue a log message for asynchronous processing.
*
* Blocks while the channel applies backpressure.
*/
void Log(LogLevel level, PipelinePhase phase,
std::string_view message) override;
private:
BoundedChannel<LogEntry>& channel_;
BoundedChannel<LogEntry>& channel_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOG_PRODUCER_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_CHANNEL_LOGGER_H_

View File

@@ -9,9 +9,9 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOGGER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOGGER_H_
#include <source_location>
#include <optional>
#include <string>
#include <utility>
#include <string_view>
#include "services/logging/log_entry.h"
@@ -24,42 +24,22 @@
*/
class ILogger {
public:
ILogger() = default;
ILogger(const ILogger&) = delete;
ILogger& operator=(const ILogger&) = delete;
ILogger(ILogger&&) = delete;
ILogger& operator=(ILogger&&) = delete;
virtual ~ILogger() = default;
ILogger() = default;
ILogger(const ILogger&) = delete;
ILogger& operator=(const ILogger&) = delete;
ILogger(ILogger&&) = delete;
ILogger& operator=(ILogger&&) = delete;
virtual ~ILogger() = default;
/**
* @brief Submit a log message to the logging subsystem.
*
* @param payload User-provided log data (level, phase, message).
* @param origin Auto-captured source location of the call site.
*/
void Log(LogDTO payload,
std::source_location origin = std::source_location::current(),
std::chrono::system_clock::time_point timestamp =
std::chrono::system_clock::now(),
std::thread::id thread_id = std::this_thread::get_id()) {
LogEntry entry;
entry.timestamp = timestamp;
entry.thread_id = thread_id;
entry.level = payload.level;
entry.phase = payload.phase;
entry.message = std::move(payload.message);
entry.origin = origin;
DoLog(std::move(entry));
}
protected:
/**
* @brief Underlying implementation to transport the log entry.
*
* Implementations must be thread-safe as DoLog can be called concurrently
* from multiple worker threads.
*/
virtual void DoLog(LogEntry log_entry) = 0;
/**
* @brief Submit a log message to the logging subsystem.
*
* @param level Severity of the message.
* @param phase Pipeline execution phase associated with the message.
* @param message Log message text.
*/
virtual void Log(LogLevel level, PipelinePhase phase,
std::string_view message) = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_LOGGING_LOGGER_H_

View File

@@ -1,18 +1,19 @@
/**
* @file web_client/http_web_client.h
* @brief cpp-httplib implementation of the WebClient interface.
*/
* @file web_client/http_web_client.h
* @brief cpp-httplib implementation of the WebClient interface.
*/
#ifndef BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_HTTP_WEB_CLIENT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_HTTP_WEB_CLIENT_H_
#include "web_client/web_client.h"
#include "services/logging/logger.h"
#include <memory>
#include <string>
#include <utility>
#include "services/logging/logger.h"
#include "web_client/web_client.h"
/**
* @brief WebClient implementation backed by cpp-httplib.
*
@@ -25,7 +26,7 @@
* bound to a single origin at construction time.
*/
class HttpWebClient final : public WebClient {
public:
public:
explicit HttpWebClient(std::shared_ptr<ILogger> logger)
: logger_(std::move(logger)) {}
~HttpWebClient() override = default;
@@ -33,8 +34,7 @@ class HttpWebClient final : public WebClient {
/**
* @brief Executes a blocking HTTP/HTTPS GET request against a full URL.
*
* @param url Fully-qualified URL, e.g.
* "https://en.wikipedia.org/api/rest_v1/page/summary/Berlin"
* @param url Fully-qualified URL, e.g. "https://en.wikipedia.org/api/rest_v1/page/summary/Berlin"
* @return Response body on HTTP 2xx; throws std::runtime_error otherwise.
*/
std::string Get(const std::string& url) override;
@@ -52,4 +52,5 @@ class HttpWebClient final : public WebClient {
std::shared_ptr<ILogger> logger_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_HTTP_WEB_CLIENT_H_
#endif

View File

@@ -13,6 +13,7 @@
*/
class WebClient {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~WebClient() = default;
/**

View File

@@ -1,42 +0,0 @@
[
{
"name": "Hophead Explorer",
"description": "Chases the newest hop varieties and double-digit IBUs, tracking hazy releases and limited tap takeovers across town.",
"style_affinities": ["American IPA", "Double IPA", "New England IPA", "Black IPA"]
},
{
"name": "Malt & Tradition Loyalist",
"description": "Prefers clean, balanced lagers brewed the way their region has brewed them for generations, and is quick to defend a well-made Helles.",
"style_affinities": ["Munich Helles", "Vienna Lager", "Märzen", "Festbier"]
},
{
"name": "Dark & Roasty Devotee",
"description": "Orders the darkest beer on the board year-round, drawn to roasted malt, coffee, and chocolate notes over carbonated lightness.",
"style_affinities": ["Imperial Stout", "Robust Porter", "Baltic Porter", "Oatmeal Stout"]
},
{
"name": "Sour & Wild Forager",
"description": "Seeks out barrel-aged, spontaneously fermented, and funky beers, and will travel for a fresh bottling of something wild.",
"style_affinities": ["Lambic", "Gueuze", "Flanders Red Ale", "Wild Ale"]
},
{
"name": "Session Sipper",
"description": "Values an easy-drinking pint that holds up over a long afternoon at the taproom more than raw intensity or novelty.",
"style_affinities": ["Session IPA", "Ordinary Bitter", "Cream Ale", "Blonde Ale"]
},
{
"name": "Belgian Abbey Pilgrim",
"description": "Collects abbey and trappist releases, drawn to the dried-fruit esters and warming strength of classic Belgian ales.",
"style_affinities": ["Belgian Tripel", "Belgian Dubbel", "Belgian Quadrupel", "Trappist Single"]
},
{
"name": "Wheat & Citrus Casual",
"description": "Reaches for something light, cloudy, and refreshing, usually with a citrus or coriander note, especially on a patio in summer.",
"style_affinities": ["Hefeweizen", "Witbier", "American Wheat Beer", "Berliner Weisse"]
},
{
"name": "Big & Boozy Connoisseur",
"description": "Sips slowly from a snifter, seeking out high-ABV releases meant for aging and savoring rather than quick refreshment.",
"style_affinities": ["English Barleywine", "Wee Heavy", "Doppelbock", "Old Ale"]
}
]

View File

@@ -1,122 +0,0 @@
# FULL SYSTEM PROMPT
You are a craft beer community profile writer. You write short, first-person
biographies for fictional users of a brewery-discovery app, grounded in the
exact name, locale, and persona provided. You do not invent the user's name --
it is given to you and must not be altered, translated, or abbreviated beyond
forming a username.
You will receive the inputs like this:
## NAME:
[First name] [Last name]
## GENDER:
[Gender associated with the given first name]
## CITY:
[City Name]
## COUNTRY:
[Country Name]
## PERSONA:
[Persona archetype name]
## PERSONA DESCRIPTION:
[What this persona cares about and how they talk about beer]
## STYLE AFFINITIES:
[Comma-separated beer styles this persona favors]
## CRITICAL OUTPUT FORMAT (READ CAREFULLY):
ABSOLUTELY NO MARKDOWN FORMATTING. Do NOT wrap your response in json or ```
blocks.
Do not add markdown, code fences, or postscript around the final JSON object.
Do not say "Here is the JSON" or "Enjoy!".
The JSON must contain exactly three keys ("username", "bio",
"activity_weight") in that order. Do not rename or add any other keys.
ESCAPE ALL QUOTES inside the bio field using \", or use single quotes (' ')
instead.
DO NOT use actual line breaks (\n) inside any string. Keep the bio as one
continuous string.
The bio must be between 25 and 60 words, written in first person, and must
read as something this specific person would write about themselves -- not a
generic profile blurb.
`activity_weight` is a number between 0 and 1 (inclusive) representing how
often this persona checks in at breweries relative to other users. Persona
archetypes implying frequent, habitual brewery visits should skew higher;
casual or occasional-visitor personas should skew lower. Do not default to
0.5 -- vary it meaningfully based on the persona description.
Expected JSON format:
```json
{
"username": "a handle derived from the given name",
"bio": "The first-person bio goes here.",
"activity_weight": 0.62
}
```
## CONTENT RULES AND CONSTRAINTS:
### THE USERNAME:
Derive the username from the given first and/or last name (e.g. lowercase,
abbreviated, or combined with a short word or number tied to the persona's
style affinities). Do not invent an unrelated handle. Do not include spaces.
### THE BIO:
Ground the bio in the supplied persona description and style affinities --
mention at least one specific beer style from the style affinities list, in a
way that sounds like a personal preference rather than a list. Reference the
city or country naturally if it fits, but do not force it.
### VOICE & PERSPECTIVE:
Write in the first person ("I"/"my"). The tone should match the persona:
an enthusiastic explorer sounds different from a relaxed, easy-drinking
regular. Do not write in second or third person.
### THE BLOCKLIST (FORBIDDEN CONCEPTS):
You absolutely cannot use the following words and phrases. Make sure your
final output doesn't have any of these:
- "hidden gem"
- "passion"
- "authentic"
- "craft beer enthusiast"
- "beer connoisseur"
- "journey"
#### FORBIDDEN WRITING PATTERNS
The following patterns are common AI writing pitfalls and must not appear in
the bio:
- Negative parallelism constructions: "It's not X, it's Y"
- Inflated significance phrases: "stands as a testament," "plays a vital
role," "deeply rooted"
- Superficial trailing analyses: sentences ending in -ing words that add
opinion without content
- Promotional travel-copy tone: "breathtaking," "must-visit," "vibrant"
- Overused conjunctive transitions used as sentence openers: "Moreover,"
"Furthermore," "In addition"
- Rule of three: do not consistently organise ideas or examples in triplets

View File

@@ -1,8 +0,0 @@
./biergarten-pipeline \
--model ../models/google_gemma-4-E4B-it-Q6_K.gguf \
--temperature 1.0 \
--top-p 0.95 \
--top-k 64 \
--n-ctx 8192 \
--location-count 15 \
--prompt-dir ../prompts

View File

@@ -10,10 +10,11 @@ dockerStartCmd: []
isPublic: false
isServerless: false
env:
BIERGARTEN_MODE: live
BIERGARTEN_MODEL_PATH: /workspace/models/google_gemma-4-E4B-it-Q6_K.gguf
BIERGARTEN_PROMPT_DIR: /workspace/app/build/prompts
BIERGARTEN_OUTPUT_DIR: /workspace/output
BIERGARTEN_LOG_PATH: /workspace/logs/pipeline.log
BIERGARTEN_GL_LAYERS: "40"
BIERGARTEN_TEMPERATURE: "1.0"
BIERGARTEN_TOP_P: "0.95"
BIERGARTEN_TOP_K: "64"

View File

@@ -1,20 +1,21 @@
#include <chrono>
#include <format>
#include "services/logging/logger.h"
#include <iostream>
#include <optional>
#include <sstream>
#include <string>
#include "data_model/models.h"
#include "services/logging/logger.h"
std::optional<ApplicationOptions> ParseArguments(
const int argc, char** argv, std::shared_ptr<ILogger> logger) {
std::optional<ApplicationOptions> ParseArguments(const int argc, char** argv,
std::shared_ptr<ILogger> logger) {
prog_opts::options_description desc("Pipeline Options");
auto opt = desc.add_options();
opt("help,h", "Produce help message");
// Defaults sourced from SamplingOptions{} so the CLI and LlamaGenerator
// share a single source of truth — changing the struct updates both.
auto add_sampling_options = [&]() -> void {
const SamplingOptions sampling_defaults{};
opt("temperature",
@@ -51,8 +52,7 @@ std::optional<ApplicationOptions> ParseArguments(
prog_opts::value<std::string>()->default_value("pipeline.log"),
"Path for application logs");
opt("prompt-dir", prog_opts::value<std::string>()->default_value(""),
"Directory containing named prompt files (e.g. "
"BREWERY_GENERATION.md)."
"Directory containing named prompt files (e.g. BREWERY_GENERATION.md)."
" Required when not using --mocked.");
opt("location-count", prog_opts::value<uint32_t>()->default_value(10));
};
@@ -70,12 +70,10 @@ std::optional<ApplicationOptions> ParseArguments(
return usage_stream.str();
})();
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = title});
logger->Log(LogDTO{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = usage});
logger->Log(LogLevel::Info, PipelinePhase::Startup, title);
logger->Log(LogLevel::Info, PipelinePhase::Startup, usage);
} else {
std::cout << title << std::endl << usage << std::endl;
}
return std::nullopt;
}
@@ -89,9 +87,9 @@ std::optional<ApplicationOptions> ParseArguments(
std::stringstream help_stream;
help_stream << "\n" << desc;
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = help_stream.str()});
logger->Log(LogLevel::Info, PipelinePhase::Startup, help_stream.str());
} else {
std::cout << help_stream.str() << std::endl;
}
return std::nullopt;
}
@@ -101,22 +99,19 @@ std::optional<ApplicationOptions> ParseArguments(
options.pipeline.output_path = var_map["output"].as<std::string>();
options.pipeline.log_path = var_map["log-path"].as<std::string>();
options.pipeline.prompt_dir = var_map["prompt-dir"].as<std::string>();
options.pipeline.location_count = var_map["location-count"].as<uint32_t>();
options.pipeline.location_count =
var_map["location-count"].as<uint32_t>();
const bool use_mocked = var_map["mocked"].as<bool>();
const std::string model_path = var_map["model"].as<std::string>();
const int n_gpu_layers = var_map["n-gpu-layers"].as<int>();
// Enforce mutual exclusivity before any further configuration is
// applied.
// Enforce mutual exclusivity before any further configuration is applied.
if (use_mocked && !model_path.empty()) {
const std::string msg =
"Invalid arguments: --mocked and --model are mutually "
"exclusive";
"Invalid arguments: --mocked and --model are mutually exclusive";
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Error, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
@@ -125,12 +120,9 @@ std::optional<ApplicationOptions> ParseArguments(
if (!use_mocked && model_path.empty()) {
const std::string msg =
"Invalid arguments: either --mocked or --model must be "
"specified";
"Invalid arguments: either --mocked or --model must be specified";
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Error, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
@@ -141,12 +133,9 @@ std::optional<ApplicationOptions> ParseArguments(
// generator has no use for it and should not require it to be present.
if (!use_mocked && options.pipeline.prompt_dir.empty()) {
const std::string msg =
"Invalid arguments: --prompt-dir is required when not using "
"--mocked";
"Invalid arguments: --prompt-dir is required when not using --mocked";
if (logger) {
logger->Log({.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Error, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
@@ -155,6 +144,7 @@ std::optional<ApplicationOptions> ParseArguments(
options.generator.use_mocked = use_mocked;
options.generator.model_path = model_path;
// options.generator.n_gpu_layers = n_gpu_layers;
// Only populate sampling config when the user explicitly overrides at
// least one value. Leaving it as std::nullopt lets LlamaGenerator fall
@@ -168,13 +158,11 @@ std::optional<ApplicationOptions> ParseArguments(
if (user_provided_sampling) {
// Warn but do not fail — the run is still valid, the flags are just
// silently irrelevant when no model is loaded.
if (use_mocked) {
if (use_mocked) {
const std::string msg =
"Sampling parameters are ignored when using --mocked";
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Warn,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Warn, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
@@ -198,18 +186,17 @@ std::optional<ApplicationOptions> ParseArguments(
std::string("Failed to parse command-line arguments: ") +
exception.what();
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Error, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
return std::nullopt;
} catch (...) {
const std::string msg =
"Failed to parse command-line arguments: unknown error";
const std::string msg = "Failed to parse command-line arguments: unknown error";
if (logger) {
logger->Log(LogDTO{.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = msg});
logger->Log(LogLevel::Error, PipelinePhase::Startup, msg);
} else {
std::cerr << msg << std::endl;
}
return std::nullopt;
}

View File

@@ -1,6 +1,6 @@
/**
* @file biergarten_pipeline_orchestrator/biergarten_pipeline_orchestrator.cc
* @brief BiergartenPipelineOrchestrator constructor implementation.
* @brief BiergartenDataGenerator constructor implementation.
*/
#include "biergarten_pipeline_orchestrator.h"
@@ -12,11 +12,9 @@ BiergartenPipelineOrchestrator::BiergartenPipelineOrchestrator(
std::unique_ptr<IEnrichmentService> context_service,
std::unique_ptr<DataGenerator> generator,
std::unique_ptr<IExportService> exporter,
std::unique_ptr<ICuratedDataService> curated_data_service,
const ApplicationOptions& application_options)
const ApplicationOptions &app_options)
: logger_(std::move(logger)),
context_service_(std::move(context_service)),
generator_(std::move(generator)),
exporter_(std::move(exporter)),
curated_data_service_(std::move(curated_data_service)),
application_options_(application_options) {}
application_options_(app_options) {}

View File

@@ -1,88 +1,61 @@
/**
* @file biergarten_pipeline_orchestrator/generate_breweries.cc
* @brief BiergartenPipelineOrchestrator::GenerateBreweries() implementation.
* @brief BiergartenDataGenerator::GenerateBreweries() implementation.
*/
#include <chrono>
#include <format>
#include <optional>
#include "biergarten_pipeline_orchestrator.h"
#include "services/logging/logger.h"
#include "biergarten_pipeline_orchestrator.h"
void BiergartenPipelineOrchestrator::GenerateBreweries(
std::span<const EnrichedCity> cities) {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = "=== SAMPLE BREWERY GENERATION ==="});
logger_->Log(LogLevel::Info, PipelinePhase::BreweryAndBeerGeneration,
"=== SAMPLE BREWERY GENERATION ===");
generated_breweries_.clear();
size_t skipped_count = 0;
size_t export_failed_count = 0;
const auto generate_record =
[this, &skipped_count](
const Location& location,
const std::string& region_context) -> std::optional<BreweryRecord> {
for (const auto& [location, region_context] : cities) {
try {
const BreweryResult brewery =
generator_->GenerateBrewery(location, region_context);
return BreweryRecord{.location = location, .brewery = brewery};
const GeneratedBrewery gen{.location = location, .brewery = brewery};
generated_breweries_.push_back(gen);
try {
exporter_->ProcessRecord(gen);
} catch (const std::exception& export_exception) {
++export_failed_count;
logger_->Log(LogLevel::Warn, PipelinePhase::BreweryAndBeerGeneration,
std::string("[Pipeline] Generated brewery for '") +
location.city + "' (" + location.country +
") but SQLite export failed: " +
export_exception.what());
}
} catch (const std::exception& e) {
++skipped_count;
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("[Pipeline] Skipping city '{}' ({}): brewery "
"generation failed: {}",
location.city, location.country, e.what())});
return std::nullopt;
logger_->Log(LogLevel::Warn, PipelinePhase::BreweryAndBeerGeneration,
std::string("[Pipeline] Skipping city '") + location.city +
" (" + location.country + "): brewery generation failed: " +
e.what());
}
};
const auto export_record = [this, &export_failed_count](
const BreweryRecord& record) {
try {
exporter_->ProcessRecord(record);
} catch (const std::exception& export_exception) {
++export_failed_count;
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("[Pipeline] Generated brewery for '{}' ({}) "
"but SQLite export failed: {}",
record.location.city, record.location.country,
export_exception.what())});
}
};
for (const auto& [location, region_context] : cities) {
const std::optional<BreweryRecord> record =
generate_record(location, region_context);
if (!record.has_value()) {
continue;
}
generated_breweries_.push_back(*record);
export_record(*record);
}
if (skipped_count > 0) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"[Pipeline] Skipped {} city/cities due to generation errors",
skipped_count)});
logger_->Log(LogLevel::Warn, PipelinePhase::BreweryAndBeerGeneration,
std::string("[Pipeline] Skipped ") +
std::to_string(skipped_count) +
" city/cities due to generation errors");
}
if (export_failed_count > 0) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::Teardown,
.message = std::format("[Pipeline] Failed to export {} generated "
"brewery/breweries to SQLite",
export_failed_count)});
logger_->Log(LogLevel::Warn, PipelinePhase::Teardown,
std::string("[Pipeline] Failed to export ") +
std::to_string(export_failed_count) +
" generated brewery/breweries to SQLite");
}
}

View File

@@ -1,219 +0,0 @@
/**
* @file biergarten_pipeline_orchestrator/generate_users.cc
* @brief BiergartenPipelineOrchestrator::GenerateUsers() implementation.
*/
#include <cctype>
#include <chrono>
#include <format>
#include <iterator>
#include <optional>
#include <random>
#include <stdexcept>
#include <string>
#include <string_view>
#include <unordered_set>
#include "biergarten_pipeline_orchestrator.h"
#include "services/curated_data/curated_json_data_service.h"
#include "services/logging/logger.h"
namespace {
std::string Sanitize(std::string_view value) {
std::string out;
out.reserve(value.size());
for (const char character : value) {
if (std::isalnum(static_cast<unsigned char>(character)) != 0) {
out.push_back(static_cast<char>(
std::tolower(static_cast<unsigned char>(character))));
}
}
return out;
}
std::string BuildEmail(const Name& name,
std::unordered_set<std::string>& used_local_parts) {
const std::string base =
std::format("{}.{}", Sanitize(name.first_name), Sanitize(name.last_name));
std::string local_part = base;
uint32_t suffix = 1;
while (used_local_parts.contains(local_part)) {
local_part = std::format("{}{}", base, suffix);
++suffix;
}
used_local_parts.insert(local_part);
return std::format("{}@thebiergarten.app", local_part);
}
std::string GenerateDateOfBirth(std::mt19937& rng) {
using namespace std::chrono;
constexpr int kMinAge = 19;
constexpr int kMaxAge = 48;
constexpr int kMaxDayOffset = 364;
std::uniform_int_distribution<int> age_dist(kMinAge, kMaxAge);
std::uniform_int_distribution<int> day_offset_dist(0, kMaxDayOffset);
const year_month_day today{floor<days>(system_clock::now())};
const year_month_day birth_year_anchor{today.year() - years{age_dist(rng)},
today.month(), today.day()};
const sys_days birth_date =
sys_days{birth_year_anchor} - days{day_offset_dist(rng)};
const year_month_day birth_ymd{birth_date};
return std::format("{:04}-{:02}-{:02}", static_cast<int>(birth_ymd.year()),
static_cast<unsigned>(birth_ymd.month()),
static_cast<unsigned>(birth_ymd.day()));
}
std::optional<Name> SampleName(
const ForenamesByCountryMap& forenames_by_country,
const SurnamesByCountryMap& surnames_by_country,
const std::string& iso3166_1, std::mt19937& rng) {
const auto forenames_it = forenames_by_country.find(iso3166_1);
const auto surnames_it = surnames_by_country.find(iso3166_1);
if (forenames_it == forenames_by_country.end() ||
surnames_it == surnames_by_country.end() ||
forenames_it->second.empty() || surnames_it->second.empty()) {
return std::nullopt;
}
const ForenameList& forenames = forenames_it->second;
const SurnameList& surnames = surnames_it->second;
std::uniform_int_distribution<size_t> forename_dist(0, forenames.size() - 1);
std::uniform_int_distribution<size_t> surname_dist(0, surnames.size() - 1);
auto forename_it = forenames.begin();
std::advance(forename_it, forename_dist(rng));
auto surname_it = surnames.begin();
std::advance(surname_it, surname_dist(rng));
return Name{.first_name = forename_it->name,
.last_name = *surname_it,
.gender = forename_it->gender};
}
} // namespace
void BiergartenPipelineOrchestrator::GenerateUsers(
std::span<const EnrichedCity> cities) {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::UserGeneration,
.message = "=== SAMPLE USER GENERATION ==="});
const PersonasList& personas = curated_data_service_->LoadPersonas();
if (personas.empty()) {
throw std::runtime_error(
"No personas available in personas.json for user generation");
}
const ForenamesByCountryMap& forenames_by_country =
curated_data_service_->LoadForenamesByCountry();
const SurnamesByCountryMap& surnames_by_country =
curated_data_service_->LoadSurnamesByCountry();
std::mt19937 rng(std::random_device{}());
std::uniform_int_distribution<size_t> persona_dist(0, personas.size() - 1);
generated_users_.clear();
size_t skipped_count = 0;
size_t export_failed_count = 0;
std::unordered_set<std::string> used_email_local_parts;
const auto generate_record =
[this, &rng, &skipped_count, &used_email_local_parts](
const EnrichedCity& city, const UserPersona& persona,
const Name& sampled_name) -> std::optional<UserRecord> {
try {
const UserResult user =
generator_->GenerateUser(city, persona, sampled_name);
return UserRecord{
.location = city.location,
.user = user,
.email = BuildEmail(sampled_name, used_email_local_parts),
.date_of_birth = GenerateDateOfBirth(rng),
};
} catch (const std::exception& e) {
++skipped_count;
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"[Pipeline] Skipping city '{}' ({}): "
"user generation failed: {}",
city.location.city, city.location.country, e.what())});
return std::nullopt;
}
};
const auto export_record = [this,
&export_failed_count](const UserRecord& record) {
try {
exporter_->ProcessRecord(record);
} catch (const std::exception& export_exception) {
++export_failed_count;
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format("[Pipeline] Generated user for '{}' ({}) but "
"SQLite export failed: {}",
record.location.city, record.location.country,
export_exception.what())});
}
};
for (const auto& city : cities) {
const std::optional<Name> sampled_name =
SampleName(forenames_by_country, surnames_by_country,
city.location.iso3166_1, rng);
if (!sampled_name.has_value()) {
++skipped_count;
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"[Pipeline] Skipping city '{}' ({}): no names "
"available for country '{}'",
city.location.city, city.location.country,
city.location.iso3166_1)});
continue;
}
const UserPersona& persona = personas[persona_dist(rng)];
const std::optional<UserRecord> record =
generate_record(city, persona, *sampled_name);
if (!record.has_value()) {
continue;
}
generated_users_.push_back(*record);
export_record(*record);
}
if (skipped_count > 0) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"[Pipeline] Skipped {} city/cities during user generation",
skipped_count)});
}
if (export_failed_count > 0) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Teardown,
.message = std::format("[Pipeline] Failed to export {} "
"generated user/users to SQLite",
export_failed_count)});
}
}

View File

@@ -1,59 +1,31 @@
/**
* @file biergarten_pipeline_orchestrator/log_results.cc
* @brief BiergartenPipelineOrchestrator::LogResults() implementation.
* @brief BiergartenDataGenerator::LogResults() implementation.
*/
#include <boost/json/array.hpp>
#include <chrono>
#include <format>
#include "biergarten_pipeline_orchestrator.h"
#include "json_handling/pretty_print.h"
#include "services/logging/logger.h"
#include "biergarten_pipeline_orchestrator.h"
#include <sstream>
void BiergartenPipelineOrchestrator::LogResults() const {
boost::json::array brewery_output;
std::ostringstream msg;
msg << "GENERATED DATA DUMP\n";
size_t index = 1;
for (const auto& [location, brewery] : generated_breweries_) {
brewery_output.push_back(boost::json::object{
{"name_en", brewery.name_en},
{"description_en", brewery.description_en},
{"name_local", brewery.name_local},
{"description_local", brewery.description_local},
{"location", boost::json::object{
{"city", location.city},
{"country", location.country},
{"state_province", location.state_province},
{"iso3166_2", location.iso3166_2},
{"latitude", location.latitude},
{"longitude", location.longitude},
}}});
msg << index << ". city=\"" << location.city << "\" country=\""
<< location.country << "\" state=\"" << location.state_province
<< "\" iso3166_2=" << location.iso3166_2 << " lat="
<< location.latitude << " lon=" << location.longitude << "\n";
msg << " brewery_name_en=\"" << brewery.name_en << "\"\n";
msg << " brewery_description_en=\"" << brewery.description_en
<< "\"\n";
msg << " brewery_name_local=\"" << brewery.name_local << "\"\n";
msg << " brewery_description_local=\"" << brewery.description_local
<< "\"\n";
++index;
}
std::ostringstream brewery_oss;
PrettyPrint(brewery_oss, brewery_output);
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Teardown,
.message = brewery_oss.str()});
boost::json::array user_output;
for (const auto& generated_user : generated_users_) {
user_output.push_back(boost::json::object{
{"first_name", generated_user.user.first_name},
{"last_name", generated_user.user.last_name},
{"gender", generated_user.user.gender},
{"username", generated_user.user.username},
{"bio", generated_user.user.bio},
{"activity_weight", generated_user.user.activity_weight},
{"email", generated_user.email},
{"date_of_birth", generated_user.date_of_birth},
});
}
std::ostringstream user_oss;
PrettyPrint(user_oss, user_output);
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Teardown,
.message = user_oss.str()});
logger_->Log(LogLevel::Debug, PipelinePhase::Teardown, msg.str());
}

View File

@@ -1,27 +1,27 @@
/**
* @file biergarten_pipeline_orchestrator/query_cities_with_countries.cc
* @brief BiergartenPipelineOrchestrator::QueryCitiesWithCountries()
* implementation.
* @brief BiergartenDataGenerator::QueryCitiesWithCountries() implementation.
*/
#include "services/logging/logger.h"
#include <algorithm>
#include <chrono>
#include <filesystem>
#include <format>
#include <iterator>
#include <random>
#include "biergarten_pipeline_orchestrator.h"
#include "services/curated_data/curated_json_data_service.h"
#include "services/logging/logger.h"
#include "json_handling/json_loader.h"
std::vector<Location>
BiergartenPipelineOrchestrator::QueryCitiesWithCountries() {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "=== GEOGRAPHIC DATA OVERVIEW ==="});
std::vector<Location> BiergartenPipelineOrchestrator::QueryCitiesWithCountries() {
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
"=== GEOGRAPHIC DATA OVERVIEW ===");
const std::filesystem::path locations_path = "locations.json";
auto all_locations = JsonLoader::LoadLocations(locations_path, logger_);
const std::vector<Location>& all_locations =
curated_data_service_->LoadLocations();
const size_t sample_count = std::min(
static_cast<size_t>(application_options_.pipeline.location_count),
@@ -38,13 +38,9 @@ BiergartenPipelineOrchestrator::QueryCitiesWithCountries() {
std::ranges::sample(all_locations, std::back_inserter(sampled_locations),
sample_count_signed, random_generator);
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format(" Locations available: {}",
all_locations.size())});
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format(" Sampled locations: {}",
sampled_locations.size())});
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
std::format(" Locations available: {}", all_locations.size()));
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
std::format(" Sampled locations: {}", sampled_locations.size()));
return sampled_locations;
}

View File

@@ -1,14 +1,13 @@
/**
* @file biergarten_pipeline_orchestrator/run.cc
* @brief BiergartenPipelineOrchestrator::Run() implementation.
* @brief BiergartenDataGenerator::Run() implementation.
*/
#include <chrono>
#include <format>
#include "services/logging/logger.h"
#include <utility>
#include "biergarten_pipeline_orchestrator.h"
#include "services/logging/logger.h"
bool BiergartenPipelineOrchestrator::Run() {
try {
@@ -22,40 +21,37 @@ bool BiergartenPipelineOrchestrator::Run() {
for (auto& city : cities) {
try {
std::string region_context = context_service_->GetLocationContext(city);
// logger_->Log(LogLevel::Debug, PipelinePhase::UserGeneration,
// "[Pipeline] Context for '" + city.city + "' (" +
// city.iso3166_2 + ") gathered:\n" + region_context);
enriched.push_back(
EnrichedCity{.location = std::move(city),
.region_context = std::move(region_context)});
} catch (const std::exception& exception) {
++skipped_count;
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"[Pipeline] Skipping city '{}' ({}): context "
"lookup failed: {}",
city.city, city.country, exception.what())});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("[Pipeline] Skipping city '") + city.city +
" (" + city.country + "): context lookup failed: " +
exception.what());
}
}
if (skipped_count > 0) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format("[Pipeline] Skipped {} city/cities due "
"to context lookup errors",
skipped_count)});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("[Pipeline] Skipped ") +
std::to_string(skipped_count) +
" city/cities due to context lookup errors");
}
this->GenerateUsers(enriched);
this->GenerateBreweries(enriched);
exporter_->Finalize();
this->LogResults();
return true;
} catch (const std::exception& e) {
logger_->Log({.level = LogLevel::Error,
.phase = PipelinePhase::Teardown,
.message = std::format(
"Pipeline execution failed with error: {}", e.what())});
logger_->Log(LogLevel::Error, PipelinePhase::Teardown,
std::string("Pipeline execution failed with error: ") +
e.what());
return false;
}
}

View File

@@ -4,7 +4,6 @@
* inference, and validates structured JSON output for brewery records.
*/
#include <chrono>
#include <format>
#include <optional>
#include <stdexcept>
@@ -12,7 +11,6 @@
#include <string_view>
#include <vector>
#include "data_generation/json_grammars.h"
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
@@ -33,6 +31,19 @@ static std::string FormatLocalLanguageCodes(
return formatted;
}
// GBNF grammar for structured brewery JSON output.
// @TODO move to a separate gbnf file if it grows in complexity or is shared
// across modules.
static constexpr std::string_view kBreweryJsonGrammar = R"json_brewery(
root ::= thought-block "{" ws "\"name_en\"" ws ":" ws string ws "," ws "\"description_en\"" ws ":" ws string ws "," ws "\"name_local\"" ws ":" ws string ws "," ws "\"description_local\"" ws ":" ws string ws "}" ws
thought-block ::= [^{]*
ws ::= [ \t\n\r]*
string ::= "\"" char+ "\""
char ::= [^"\\\x7F\x00-\x1F] | [\\] escape
escape ::= ["\\/bfnrt] | "u" hex hex hex hex
hex ::= [0-9a-fA-F]
)json_brewery";
static constexpr int kBreweryInitialMaxTokens = 2800;
BreweryResult LlamaGenerator::GenerateBrewery(
@@ -71,14 +82,15 @@ BreweryResult LlamaGenerator::GenerateBrewery(
/**
* RETRY LOOP with validation and error correction
* Attempts to generate valid brewery data up to 3 times, with
* feedback-based refinement
* Attempts to generate valid brewery data up to 3 times, with feedback-based
* refinement
*/
constexpr int max_attempts = 3;
std::string raw;
std::string last_error;
// Token budget: too small risks truncating valid JSON mid-string.
// Start conservatively but allow adaptive increases on truncation.
int max_tokens = kBreweryInitialMaxTokens;
// Limit output length to keep it concise and focused
@@ -87,11 +99,9 @@ BreweryResult LlamaGenerator::GenerateBrewery(
raw = this->Infer(system_prompt, user_prompt, max_tokens,
kBreweryJsonGrammar);
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: raw output (attempt {}): {}",
attempt + 1, raw)});
logger_->Log(LogLevel::Debug, PipelinePhase::BreweryAndBeerGeneration,
std::string("LlamaGenerator: raw output (attempt ") +
std::to_string(attempt + 1) + "): " + raw);
}
// Validate output: parse JSON and check required fields
@@ -104,12 +114,10 @@ BreweryResult LlamaGenerator::GenerateBrewery(
// Success: return parsed brewery data
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: successfully generated "
"brewery data on attempt {}",
attempt + 1)});
logger_->Log(LogLevel::Info, PipelinePhase::BreweryAndBeerGeneration,
std::string(
"LlamaGenerator: successfully generated brewery data on attempt ") +
std::to_string(attempt + 1));
}
return brewery;
@@ -119,41 +127,31 @@ BreweryResult LlamaGenerator::GenerateBrewery(
last_error = *validation_error;
if (logger_) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: malformed brewery JSON (attempt {}): {}",
attempt + 1, *validation_error)});
logger_->Log(LogLevel::Warn, PipelinePhase::BreweryAndBeerGeneration,
std::string("LlamaGenerator: malformed brewery JSON (attempt ") +
std::to_string(attempt + 1) + "): " + *validation_error);
}
// Update prompt with error details to guide LLM toward correct output.
user_prompt = std::format(
"Your previous response was invalid. Error: {}\nReturn the thought "
"process before the JSON if needed, then return ONLY valid JSON "
"with "
"exactly these keys, in this exact order: {{\"name_en\": "
"\"<English "
"process before the JSON if needed, then return ONLY valid JSON with "
"exactly these keys, in this exact order: {{\"name_en\": \"<English "
"brewery name>\", \"description_en\": \"<English single-paragraph "
"description>\", \"name_local\": \"<local-language brewery "
"name>\", "
"description>\", \"name_local\": \"<local-language brewery name>\", "
"\"description_local\": \"<local-language single-paragraph "
"description>\"}}.\nDo not include markdown, comments, extra keys, "
"or "
"literal placeholder values.\n\nKeep the JSON strings concise "
"enough "
"description>\"}}.\nDo not include markdown, comments, extra keys, or "
"literal placeholder values.\n\nKeep the JSON strings concise enough "
"to fit within the token budget.\n\n{}",
*validation_error, retry_location);
}
// All retry attempts exhausted: log failure and throw exception
if (logger_) {
logger_->Log({.level = LogLevel::Error,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: malformed brewery "
"response after {} attempts: {}",
max_attempts, last_error.empty() ? raw : last_error)});
logger_->Log(LogLevel::Error, PipelinePhase::BreweryAndBeerGeneration,
std::string("LlamaGenerator: malformed brewery response after ") +
std::to_string(max_attempts) + " attempts: " +
(last_error.empty() ? raw : last_error));
}
throw std::runtime_error("LlamaGenerator: malformed brewery response");
}

View File

@@ -1,115 +1,23 @@
/**
* @file data_generation/llama/generate_user.cc
* @brief Builds persona/name-grounded user prompts, performs retry-based
* inference, and validates structured JSON output for user records.
* @brief Generates locale-aware user profiles with strict two-line formatting,
* retry handling, and output sanitization for downstream parsing.
*/
#include <format>
#include <optional>
#include <stdexcept>
#include <string>
#include <string_view>
#include "data_generation/json_grammars.h"
#include <string>
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
static constexpr int kUserInitialMaxTokens = 1200;
UserResult LlamaGenerator::GenerateUser(const EnrichedCity& city,
const UserPersona& persona,
const Name& name) {
std::string style_affinities;
for (const std::string& style : persona.style_affinities) {
if (!style_affinities.empty()) {
style_affinities += ", ";
}
style_affinities += style;
}
const std::string system_prompt = prompt_directory_->Load("USER_GENERATION");
std::string user_prompt = std::format(
"## NAME:\n\n{} {}\n\n"
"## GENDER:\n\n{}\n\n"
"## CITY:\n\n{}\n\n"
"## COUNTRY:\n\n{}\n\n"
"## PERSONA:\n\n{}\n\n"
"## PERSONA DESCRIPTION:\n\n{}\n\n"
"## STYLE AFFINITIES:\n\n{}",
name.first_name, name.last_name, name.gender, city.location.city,
city.location.country, persona.name, persona.description,
style_affinities);
const std::string retry_context = std::format(
"Name: {} {}\nCity: {}, {}\nPersona: {}", name.first_name, name.last_name,
city.location.city, city.location.country, persona.name);
constexpr int max_attempts = 3;
std::string raw;
std::string last_error;
int max_tokens = kUserInitialMaxTokens;
for (int attempt = 0; attempt < max_attempts; ++attempt) {
raw = this->Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
.phase = PipelinePhase::UserGeneration,
.message = std::format("LlamaGenerator: raw output (attempt {}): {}",
attempt + 1, raw)});
}
UserResult user;
const std::optional<std::string> validation_error =
ValidateUserJson(raw, user);
if (!validation_error.has_value()) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::UserGeneration,
.message = std::format("LlamaGenerator: successfully "
"generated user data on attempt {}",
attempt + 1)});
}
user.first_name = name.first_name;
user.last_name = name.last_name;
user.gender = name.gender;
return user;
}
last_error = *validation_error;
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"LlamaGenerator: malformed user JSON (attempt {}): {}",
attempt + 1, *validation_error)});
}
user_prompt = std::format(
"Your previous response was invalid. Error: {}\nReturn the thought "
"process before the JSON if needed, then return ONLY valid JSON "
"with "
"exactly these keys, in this exact order: {{\"username\": "
"\"<handle "
"derived from the given name>\", \"bio\": \"<first-person bio "
"grounded in the persona>\", \"activity_weight\": <number between "
"0 "
"and 1>}}.\nDo not include markdown, comments, extra keys, or "
"literal placeholder values.\n\n{}",
*validation_error, retry_context);
}
if (logger_) {
logger_->Log({.level = LogLevel::Error,
.phase = PipelinePhase::UserGeneration,
.message = std::format(
"LlamaGenerator: malformed user response "
"after {} attempts: {}",
max_attempts, last_error.empty() ? raw : last_error)});
}
throw std::runtime_error("LlamaGenerator: malformed user response");
// TODO: Implement locale-aware user profile generation.
// Current implementation returns a hardcoded test value and ignores the
// locale parameter. Future implementation should:
// 1. Load a USER_GENERATION.md prompt template with locale context
// 2. Perform LLM inference with locale-specific username/bio generation
// 3. Parse and validate JSON output with retry handling (similar to brewery)
// 4. Return locale-aware username and biography
UserResult LlamaGenerator::GenerateUser(const std::string& locale) {
return {.username = "test_user",
.bio = "This is a test user profile from " + locale + "."};
}

View File

@@ -9,21 +9,18 @@
#include <boost/json.hpp>
#include <cctype>
#include <optional>
#include <span>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator_helpers.h"
#include "llama.h"
namespace {
/**
* String trimming: removes leading and trailing whitespace
*/
std::string Trim(std::string_view value) {
static std::string Trim(std::string_view value) {
constexpr std::string_view whitespace = " \t\n\r\f\v";
const size_t first_index = value.find_first_not_of(whitespace);
if (first_index == std::string_view::npos) {
@@ -38,7 +35,7 @@ std::string Trim(std::string_view value) {
* Normalize whitespace: collapses multiple spaces/tabs/newlines into single
* spaces
*/
std::string CondenseWhitespace(std::string_view text) {
static std::string CondenseWhitespace(std::string_view text) {
std::string out;
out.reserve(text.size());
@@ -64,37 +61,7 @@ std::string CondenseWhitespace(std::string_view text) {
// Guard against truncating in the first half of the string.
// This preserves the critical opening content and avoids cutting critical
// context words early in the region description.
constexpr size_t kTruncationGuardDivisor = 2;
bool ReadRequiredTrimmedStringField(const boost::json::object& obj,
std::string_view key, std::string& out,
std::string* error_out) {
const boost::json::value* field = obj.if_contains(key);
if (field == nullptr || !field->is_string()) {
return false;
}
const auto& string_value = field->as_string();
out = Trim(std::string_view(string_value.data(), string_value.size()));
return !out.empty();
}
bool HasSchemaPlaceholder(std::span<std::string* const> values) {
for (const std::string* value : values) {
std::string lowered = *value;
std::ranges::transform(lowered, lowered.begin(),
[](const unsigned char character) {
return static_cast<char>(std::tolower(character));
});
if (lowered == "string") {
return true;
}
}
return false;
}
} // namespace
static constexpr size_t kTruncationGuardDivisor = 2;
/**
* Truncate region context to fit within max length while preserving word
@@ -154,6 +121,47 @@ void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
"LlamaGenerator: failed to decode sampled token piece");
}
static bool ReadRequiredTrimmedStringField(const boost::json::object& obj,
std::string_view key,
std::string& out,
std::string* error_out) {
const boost::json::value* field = obj.if_contains(key);
if (field == nullptr || !field->is_string()) {
if (error_out != nullptr) {
*error_out =
"JSON field '" + std::string(key) + "' is missing or not a string";
}
return false;
}
const auto& string_value = field->as_string();
out = Trim(std::string_view(string_value.data(), string_value.size()));
if (out.empty()) {
if (error_out != nullptr) {
*error_out = "JSON field '" + std::string(key) + "' must not be empty";
}
return false;
}
return true;
}
static bool HasSchemaPlaceholder(const std::array<std::string*, 4>& values) {
for (const std::string* value : values) {
std::string lowered = *value;
std::ranges::transform(lowered, lowered.begin(),
[](unsigned char character) {
return static_cast<char>(std::tolower(character));
});
if (lowered == "string") {
return true;
}
}
return false;
}
std::optional<std::string> ValidateBreweryJson(const std::string& raw,
BreweryResult& brewery_out) {
boost::system::error_code error_code;
@@ -201,7 +209,7 @@ std::optional<std::string> ValidateBreweryJson(const std::string& raw,
return validation_error;
}
const std::array schema_placeholders = {
const std::array<std::string*, 4> schema_placeholders = {
&brewery_out.name_en, &brewery_out.description_en,
&brewery_out.name_local, &brewery_out.description_local};
if (HasSchemaPlaceholder(schema_placeholders)) {
@@ -210,58 +218,3 @@ std::optional<std::string> ValidateBreweryJson(const std::string& raw,
return std::nullopt;
}
std::optional<std::string> ValidateUserJson(const std::string& raw,
UserResult& user_out) {
boost::system::error_code error_code;
const std::string_view raw_view(raw);
const size_t opening_brace = raw_view.find('{');
if (opening_brace == std::string_view::npos) {
return "JSON parse error: missing opening brace '{'";
}
const std::string_view json_payload = raw_view.substr(opening_brace);
boost::json::value json_value = boost::json::parse(json_payload, error_code);
if (error_code) {
return "JSON parse error: " + error_code.message();
}
if (!json_value.is_object()) {
return "JSON root must be an object";
}
const auto& obj = json_value.get_object();
if (obj.size() != 3) {
return "JSON object must contain exactly three keys";
}
std::string validation_error;
if (!ReadRequiredTrimmedStringField(obj, "username", user_out.username,
&validation_error)) {
return validation_error;
}
if (!ReadRequiredTrimmedStringField(obj, "bio", user_out.bio,
&validation_error)) {
return validation_error;
}
const boost::json::value* activity_weight_field =
obj.if_contains("activity_weight");
if (activity_weight_field == nullptr || !activity_weight_field->is_number()) {
return "Missing or invalid numeric field: activity_weight";
}
const double activity_weight = activity_weight_field->to_number<double>();
if (activity_weight < 0.0 || activity_weight > 1.0) {
return "activity_weight must be between 0 and 1";
}
user_out.activity_weight = static_cast<float>(activity_weight);
const std::array schema_placeholders = {&user_out.username, &user_out.bio};
if (HasSchemaPlaceholder(schema_placeholders)) {
return "JSON appears to be a schema placeholder, not content";
}
return std::nullopt;
}

View File

@@ -6,18 +6,15 @@
*/
#include <algorithm>
#include <chrono>
#include <format>
#include <memory>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
#include "llama.h"
static constexpr size_t kPromptTokenSlack = 8;
// Minimum tokens to keep when using top-p sampling. Ensures at least one
@@ -108,7 +105,7 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
.top_p = sampling_top_p_,
.seed = static_cast<uint32_t>(rng_()),
};
const auto sampler = MakeSamplerChain(vocab, sampler_config, grammar);
auto sampler = MakeSamplerChain(vocab, sampler_config, grammar);
/**
* Clear KV cache to ensure clean inference state (no residual context)
@@ -173,12 +170,12 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
prompt_tokens.resize(static_cast<size_t>(token_count));
if (token_count > prompt_budget) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: prompt too long ({} tokens), "
"truncating to {} tokens to fit n_batch/n_ctx limits",
token_count, prompt_budget)});
logger_->Log(
LogLevel::Warn, PipelinePhase::BreweryAndBeerGeneration,
std::string("LlamaGenerator: prompt too long (") +
std::to_string(token_count) + ") tokens, truncating to " +
std::to_string(prompt_budget) +
" tokens to fit n_batch/n_ctx limits");
}
prompt_tokens.resize(static_cast<size_t>(prompt_budget));
token_count = prompt_budget;

View File

@@ -11,9 +11,8 @@
#include <stdexcept>
#include <string>
#include <llama.h>
#include "data_model/models.h"
#include "llama.h"
static constexpr uint32_t kMaxContextSize = 32768U;

View File

@@ -5,15 +5,13 @@
*/
#include <algorithm>
#include <chrono>
#include <stdexcept>
#include <string>
#include <utility>
#include <ggml-backend.h>
#include <llama.h>
#include "data_generation/llama_generator.h"
#include "ggml-backend.h"
#include "llama.h"
// Maximum batch size for decode operations. Capping the batch prevents
// excessive memory allocation while maintaining inference performance.
@@ -29,10 +27,8 @@ void LlamaGenerator::Load(const std::string& model_path) {
llama_model_params model_params = llama_model_default_params();
model_params.n_gpu_layers = n_gpu_layers_;
ModelHandle loaded_model(
LlamaGenerator::ModelHandle loaded_model(
llama_model_load_from_file(model_path.c_str(), model_params));
if (!loaded_model) {
throw std::runtime_error(
"LlamaGenerator: failed to load model from path: " + model_path);
@@ -42,9 +38,8 @@ void LlamaGenerator::Load(const std::string& model_path) {
context_params.n_ctx = n_ctx_;
context_params.n_batch = std::min(n_ctx_, kMaxBatchSize);
ContextHandle loaded_context(
LlamaGenerator::ContextHandle loaded_context(
llama_init_from_model(loaded_model.get(), context_params));
if (!loaded_context) {
throw std::runtime_error("LlamaGenerator: failed to create context");
}
@@ -53,9 +48,7 @@ void LlamaGenerator::Load(const std::string& model_path) {
context_ = std::move(loaded_context);
if (logger_) {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format("[LlamaGenerator] Loaded model: {} ",
model_path)});
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
std::string("[LlamaGenerator] Loaded model: ") + model_path);
}
}

View File

@@ -14,13 +14,3 @@ size_t MockGenerator::DeterministicHash(const Location& location) {
boost::hash_combine(seed, location.country);
return seed;
}
size_t MockGenerator::DeterministicHash(const Location& location,
const UserPersona& persona,
const Name& name) {
size_t seed = DeterministicHash(location);
boost::hash_combine(seed, persona.name);
boost::hash_combine(seed, name.first_name);
boost::hash_combine(seed, name.last_name);
return seed;
}

View File

@@ -1,31 +1,22 @@
/**
* @file data_generation/mock/generate_user.cc
* @brief Generates deterministic mock user profiles by hashing city, persona,
* and sampled name into predefined username and bio collections.
* @brief Generates deterministic mock user profiles by hashing locale values
* into predefined username and bio collections.
*/
#include <functional>
#include <string>
#include <string_view>
#include "data_generation/mock_generator.h"
UserResult MockGenerator::GenerateUser(const EnrichedCity& city,
const UserPersona& persona,
const Name& name) {
const size_t hash = DeterministicHash(city.location, persona, name);
UserResult MockGenerator::GenerateUser(const std::string& locale) {
const size_t hash = std::hash<std::string>{}(locale);
UserResult result;
const std::string_view username = kUsernames[hash % kUsernames.size()];
const std::string_view bio = kBios[hash / kBioHashStride % kBios.size()];
const float activity_weight =
static_cast<float>(hash / kActivityWeightHashStride %
kActivityWeightHashRange) /
static_cast<float>(kActivityWeightHashRange);
return {
.first_name = name.first_name,
.last_name = name.last_name,
.gender = name.gender,
.username = std::string(username),
.bio = std::string(bio),
.activity_weight = activity_weight,
};
result.username = username;
result.bio = bio;
return result;
}

View File

@@ -0,0 +1,109 @@
/**
* @file json_handling/json_loader.cc
* @brief Parses curated location JSON input into strongly typed Location
* records with strict field validation and descriptive error reporting.
*/
#include "json_handling/json_loader.h"
#include "services/logging/logger.h"
#include <iostream>
#include <boost/json.hpp>
#include <fstream>
#include <sstream>
#include <stdexcept>
#include <string>
#include <string_view>
static std::string ReadRequiredString(const boost::json::object& object,
const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_string()) {
throw std::runtime_error(std::string("Missing or invalid string field: ") +
key);
}
const std::string_view text = value->as_string();
return std::string(text);
}
static double ReadRequiredNumber(const boost::json::object& object,
const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_number()) {
throw std::runtime_error(std::string("Missing or invalid numeric field: ") +
key);
}
return value->to_number<double>();
}
static std::vector<std::string> ReadRequiredStringArray(
const boost::json::object& object, const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_array()) {
throw std::runtime_error(
std::string("Missing or invalid string array field: ") + key);
}
const auto& array = value->as_array();
std::vector<std::string> items;
items.reserve(array.size());
for (const auto& item : array) {
if (!item.is_string()) {
throw std::runtime_error(
std::string("Missing or invalid string array field: ") + key);
}
items.emplace_back(item.as_string());
}
return items;
}
std::vector<Location> JsonLoader::LoadLocations(
const std::filesystem::path& filepath, std::shared_ptr<ILogger> logger) {
std::ifstream input(filepath);
if (!input.is_open()) {
throw std::runtime_error("Failed to open locations file: " +
filepath.string());
}
std::stringstream buffer;
buffer << input.rdbuf();
const std::string content = buffer.str();
boost::system::error_code error;
boost::json::value root = boost::json::parse(content, error);
if (error) {
throw std::runtime_error("Failed to parse locations JSON: " +
error.message());
}
if (!root.is_array()) {
throw std::runtime_error(
"Invalid locations JSON: root element must be an array");
}
std::vector<Location> locations;
const auto& items = root.as_array();
locations.reserve(items.size());
for (const auto& item : items) {
if (!item.is_object()) {
throw std::runtime_error(
"Invalid locations JSON: each entry must be an object");
}
const auto& object = item.as_object();
locations.push_back(Location{
.city = ReadRequiredString(object, "city"),
.state_province = ReadRequiredString(object, "state_province"),
.iso3166_2 = ReadRequiredString(object, "iso3166_2"),
.country = ReadRequiredString(object, "country"),
.iso3166_1 = ReadRequiredString(object, "iso3166_1"),
.local_languages = ReadRequiredStringArray(object, "local_languages"),
.latitude = ReadRequiredNumber(object, "latitude"),
.longitude = ReadRequiredNumber(object, "longitude"),
});
}
return locations;
}

View File

@@ -4,21 +4,35 @@
* initializes shared infrastructure, and executes the pipeline entry flow.
*/
#include <spdlog/fmt/fmt.h>
#include <spdlog/spdlog.h>
#include <boost/di.hpp>
#include <boost/program_options.hpp>
#include <chrono>
#include <exception>
#include <format>
#include <iostream>
#include <memory>
#include <optional>
#include <string>
#include <thread>
#include "biergarten_pipeline.h"
#include "biergarten_pipeline_orchestrator.h"
#include "concurrency/bounded_channel.h"
#include "data_generation/llama_generator.h"
#include "data_generation/mock_generator.h"
#include "data_generation/prompt_formatting/gemma4_jinja_prompt_formatter.h"
#include "data_model/models.h"
#include "llama_backend_state.h"
#include "services/database/export_service.h"
#include "services/database/sqlite_export_service.h"
#include "services/datetime/timer.h"
#include "services/enrichment/enrichment_service.h"
#include "services/enrichment/mock_enrichment.h"
#include "services/enrichment/wikipedia_service.h"
#include "services/logging/log_dispatcher.h"
#include "services/logging/log_entry.h"
#include "services/logging/log_producer.h"
#include "services/logging/logger.h"
#include "services/prompting/prompt_directory.h"
#include "web_client/http_web_client.h"
namespace di = boost::di;
@@ -28,20 +42,11 @@ int main(const int argc, char** argv) {
spdlog::set_level(spdlog::level::debug);
spdlog::set_pattern("│ %Y-%m-%d %H:%M:%S.%e │ %^%-7l%$ │ %v");
BoundedChannel<LogEntry> log_channel(kLogMaxCount);
auto log_dispatcher = //
std::make_unique<LogDispatcher>(log_channel);
std::shared_ptr<ILogger> log_producer =
std::make_shared<LogProducer>(log_channel);
auto log_dispatcher = std::make_unique<LogDispatcher>(log_channel);
std::thread log_thread([&log_dispatcher] { log_dispatcher->Run(); });
auto shutdown = [&](const int exit_code) {
log_channel.Close();
log_thread.join();
return exit_code;
};
std::shared_ptr<ILogger> log_producer =
std::make_shared<LogProducer>(log_channel);
try {
Timer timer;
@@ -49,15 +54,15 @@ int main(const int argc, char** argv) {
const LlamaBackendState llama_backend_state;
#endif
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "STARTING PIPELINE"});
log_producer->Log(LogLevel::Info, PipelinePhase::Startup,
"STARTING PIPELINE");
const std::optional<ApplicationOptions> parsed_options =
ParseArguments(argc, argv, log_producer);
if (!parsed_options.has_value()) {
return shutdown(EXIT_FAILURE);
log_channel.Close();
log_thread.join();
return 0;
}
const auto options = *parsed_options;
@@ -65,100 +70,41 @@ int main(const int argc, char** argv) {
const auto sampling =
options.generator.sampling.value_or(SamplingOptions{});
// -----------------------------------------------------------------------
// Prompt directory
// Conditionally constructed before the injector; moved into LlamaGenerator.
// -----------------------------------------------------------------------
std::unique_ptr<IPromptDirectory> prompt_directory;
if (!options.generator.use_mocked) {
try {
prompt_directory = std::make_unique<PromptDirectory>(
options.pipeline.prompt_dir, log_producer);
} catch (const std::exception& dir_error) {
log_producer->Log({.level = LogLevel::Error,
.phase = PipelinePhase::Startup,
.message = std::format("Invalid --prompt-dir: {}",
dir_error.what())});
return shutdown(EXIT_FAILURE);
log_producer->Log(
LogLevel::Error, PipelinePhase::Startup,
std::string("Invalid --prompt-dir: ") + dir_error.what());
log_channel.Close();
log_thread.join();
return 1;
}
}
// -----------------------------------------------------------------------
// Dependency injection
// -----------------------------------------------------------------------
const auto injector = di::make_injector(
di::bind<ILogger>().to(log_producer),
di::bind<ApplicationOptions>().to(options),
di::bind<std::string>().to(model_path),
di::bind<WebClient>().to<HttpWebClient>(),
di::bind<IExportService>().to<SqliteExportService>(),
di::bind<ICuratedDataService>().to(
[options, &log_producer]() -> std::unique_ptr<ICuratedDataService> {
if (options.generator.use_mocked) {
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Curated data: mock"});
return std::make_unique<MockCuratedDataService>();
}
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Curated data: JsonLoader"});
return std::make_unique<CuratedJsonDataService>(
CuratedDataFilePaths{
.locations_path = "locations.json",
.personas_path = "personas.json",
.forenames_path = "forenames-by-country.json",
.surnames_path = "surnames-by-country.json",
});
}),
di::bind<IPromptFormatter>().to([options, log_producer] {
if (options.generator.use_mocked) {
{
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Prompt formatter: none (mock mode)"});
}
return std::unique_ptr<IPromptFormatter>(nullptr);
}
{
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Prompt formatter: Gemma4JinjaPromptFormatter"});
}
return std::unique_ptr<IPromptFormatter>(
std::make_unique<Gemma4JinjaPromptFormatter>());
}),
di::bind<WebClient>().to([options, log_producer] {
if (options.generator.use_mocked) {
{
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Web client: none (mock mode)"});
}
return std::unique_ptr<WebClient>(nullptr);
}
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Web client: HttpWebClient"});
return std::unique_ptr<WebClient>(
std::make_unique<HttpWebClient>(log_producer));
}),
di::bind<IPromptFormatter>().to<Gemma4JinjaPromptFormatter>(),
di::bind<IEnrichmentService>().to(
[options, &log_producer](
const auto& inj) -> std::unique_ptr<IEnrichmentService> {
if (options.generator.use_mocked) {
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Enrichment: mock"});
return std::make_unique<MockEnrichmentService>();
}
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Enrichment: Wikipedia"});
// if (options.generator.use_mocked) {
// return std::make_unique<MockEnrichmentService>();
// }
return std::make_unique<WikipediaEnrichmentService>(
inj.template create<std::unique_ptr<WebClient>>(),
log_producer);
@@ -167,63 +113,57 @@ int main(const int argc, char** argv) {
[&options, &model_path, &sampling, &prompt_directory,
&log_producer](const auto& inj) -> std::unique_ptr<DataGenerator> {
if (options.generator.use_mocked) {
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = "Generator: mock"});
log_producer->Log(
LogLevel::Info, PipelinePhase::Startup,
"Using MockGenerator (no model path provided)");
return std::make_unique<MockGenerator>();
}
#ifdef BIERGARTEN_MOCK_ONLY
// Unreachable: ParseArguments requires --mocked when
// BIERGARTEN_MOCK_ONLY is compiled in, so LlamaGenerator is
// never constructed and is not linked into this binary.
throw std::runtime_error(
"LlamaGenerator is unavailable in a BIERGARTEN_MOCK_ONLY "
"build");
#else
log_producer->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format(
"Generator: LlamaGenerator | model={} | "
"temp={:.2f} top_p={:.2f} top_k={} n_ctx={} seed={}",
model_path, sampling.temperature, sampling.top_p,
sampling.top_k, sampling.n_ctx, sampling.seed)});
LogLevel::Info, PipelinePhase::Startup,
"Using LlamaGenerator: " + model_path +
" (temperature=" + std::to_string(sampling.temperature) +
", top-p=" + std::to_string(sampling.top_p) +
", top-k=" + std::to_string(sampling.top_k) +
", n_ctx=" + std::to_string(sampling.n_ctx) +
", seed=" + std::to_string(sampling.seed) + ")");
return std::make_unique<LlamaGenerator>(
options, model_path, log_producer,
inj.template create<std::unique_ptr<IPromptFormatter>>(),
std::move(prompt_directory));
#endif
}));
// -----------------------------------------------------------------------
// Pipeline execution
// -----------------------------------------------------------------------
const auto orchestrator =
injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>();
if (!orchestrator->Run()) {
log_producer->Log({.level = LogLevel::Error,
.phase = PipelinePhase::Teardown,
.message = "Pipeline execution failed"});
return shutdown(EXIT_FAILURE);
log_producer->Log(LogLevel::Error, PipelinePhase::Teardown,
"Pipeline execution failed");
log_channel.Close();
log_thread.join();
return 1;
}
log_producer->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Teardown,
.message = std::format("Pipeline complete in {} ms",
timer.Elapsed())});
log_producer->Log(LogLevel::Info, PipelinePhase::Teardown,
"Pipeline executed successfully in " +
std::to_string(timer.Elapsed()) + " ms");
log_channel.Close();
log_thread.join();
return 0;
return shutdown(EXIT_SUCCESS);
} catch (const std::exception& exception) {
const LogDTO log_entry{.level = LogLevel::Error,
.phase = PipelinePhase::Teardown,
.message = exception.what()};
// Attempt to use the logging infrastructure; if channel/dispatcher are
// compromised this is a best-effort fallback.
if (log_producer) {
log_producer->Log(log_entry);
} else {
std::cerr << log_entry.message << std::endl;
log_producer->Log(
LogLevel::Error, PipelinePhase::Teardown,
std::string("Unhandled fatal error in main: ") + exception.what());
}
return shutdown(EXIT_FAILURE);
log_channel.Close();
log_thread.join();
return 1;
}
}

View File

@@ -1,264 +0,0 @@
/**
* @file json_handling/json_loader.cc
* @brief Parses curated location JSON input into strongly typed Location
* records with strict field validation and descriptive error reporting.
*/
#include "services/curated_data/curated_json_data_service.h"
#include <boost/json.hpp>
#include <format>
#include <fstream>
#include <iostream>
#include <sstream>
#include <stdexcept>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
#include "services/logging/logger.h"
static std::string ReadRequiredString(const boost::json::object& object,
const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_string()) {
throw std::runtime_error(
std::format("Missing or invalid string field: {}", key));
}
const std::string_view text = value->as_string();
return std::string(text);
}
static double ReadRequiredNumber(const boost::json::object& object,
const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_number()) {
throw std::runtime_error(
std::format("Missing or invalid numeric field: {}", key));
}
return value->to_number<double>();
}
static std::vector<std::string> ReadRequiredStringArray(
const boost::json::object& object, const char* key) {
const boost::json::value* value = object.if_contains(key);
if (value == nullptr || !value->is_array()) {
throw std::runtime_error(
std::format("Missing or invalid string array field: {}", key));
}
const auto& array = value->as_array();
std::vector<std::string> items;
items.reserve(array.size());
for (const auto& item : array) {
if (!item.is_string()) {
throw std::runtime_error(
std::format("Missing or invalid string array field: {}", key));
}
items.emplace_back(item.as_string());
}
return items;
}
namespace {
boost::json::value ParseJsonFile(const std::filesystem::path& filepath,
const char* what) {
std::ifstream input(filepath);
if (!input.is_open()) {
throw std::runtime_error(
std::format("Failed to open {} file: {}", what, filepath.string()));
}
std::stringstream buffer;
buffer << input.rdbuf();
boost::system::error_code error;
boost::json::value root = boost::json::parse(buffer.str(), error);
if (error) {
throw std::runtime_error(
std::format("Failed to parse {} JSON: {}", what, error.message()));
}
return root;
}
/**
* @brief Returns the first element of a string array field, falling back to
* `fallback_key` if `key` is missing/empty (some source entries only have a
* "localized" form and no "romanized" form).
*/
std::string ReadFirstOfStringArray(const boost::json::object& object,
const char* key, const char* fallback_key) {
for (const char* candidate_key : {key, fallback_key}) {
const boost::json::value* value = object.if_contains(candidate_key);
if (value == nullptr || !value->is_array()) {
continue;
}
const auto& array = value->as_array();
if (!array.empty() && array.front().is_string()) {
return std::string(array.front().as_string());
}
}
throw std::runtime_error(
std::format("Missing or invalid string array field: {}", key));
}
} // namespace
CuratedJsonDataService::CuratedJsonDataService(CuratedDataFilePaths filepaths)
: filepaths_(std::move(filepaths)) {}
const LocationsList& CuratedJsonDataService::LoadLocations() {
if (!cache_.locations.empty()) {
return cache_.locations;
}
const boost::json::value root =
ParseJsonFile(filepaths_.locations_path, "locations");
if (!root.is_array()) {
throw std::runtime_error(
"Invalid locations JSON: root element must be an array");
}
LocationsList locations;
const auto& items = root.as_array();
locations.reserve(items.size());
for (const auto& item : items) {
if (!item.is_object()) {
throw std::runtime_error(
"Invalid locations JSON: each entry must be an object");
}
const auto& object = item.as_object();
locations.push_back(Location{
.city = ReadRequiredString(object, "city"),
.state_province = ReadRequiredString(object, "state_province"),
.iso3166_2 = ReadRequiredString(object, "iso3166_2"),
.country = ReadRequiredString(object, "country"),
.iso3166_1 = ReadRequiredString(object, "iso3166_1"),
.local_languages = ReadRequiredStringArray(object, "local_languages"),
.latitude = ReadRequiredNumber(object, "latitude"),
.longitude = ReadRequiredNumber(object, "longitude"),
});
}
cache_.locations = std::move(locations);
return cache_.locations;
}
const PersonasList& CuratedJsonDataService::LoadPersonas() {
if (!cache_.personas.empty()) {
return cache_.personas;
}
const boost::json::value root =
ParseJsonFile(filepaths_.personas_path, "personas");
if (!root.is_array()) {
throw std::runtime_error(
"Invalid personas JSON: root element must be an array");
}
PersonasList personas;
const auto& items = root.as_array();
personas.reserve(items.size());
for (const auto& item : items) {
if (!item.is_object()) {
throw std::runtime_error(
"Invalid personas JSON: each entry must be an object");
}
const auto& object = item.as_object();
personas.push_back(UserPersona{
.name = ReadRequiredString(object, "name"),
.description = ReadRequiredString(object, "description"),
.style_affinities = ReadRequiredStringArray(object, "style_affinities"),
});
}
cache_.personas = std::move(personas);
return cache_.personas;
}
const ForenamesByCountryMap& CuratedJsonDataService::LoadForenamesByCountry() {
if (!cache_.forenames_by_country.empty()) {
return cache_.forenames_by_country;
}
const boost::json::value root =
ParseJsonFile(filepaths_.forenames_path, "forenames-by-country");
if (!root.is_object()) {
throw std::runtime_error(
"Invalid forenames-by-country JSON: root element must be an object "
"keyed by ISO 3166-1 country code");
}
ForenamesByCountryMap forenames_by_country;
for (const auto& [country_code, regions] : root.as_object()) {
if (!regions.is_array()) {
continue;
}
ForenameList entries;
for (const auto& region : regions.as_array()) {
if (!region.is_object()) {
continue;
}
const boost::json::value* names = region.as_object().if_contains("names");
if (names == nullptr || !names->is_array()) {
continue;
}
for (const auto& name_value : names->as_array()) {
if (!name_value.is_object()) {
continue;
}
const auto& name_object = name_value.as_object();
entries.emplace_back(ForenameEntry{
.name =
ReadFirstOfStringArray(name_object, "romanized", "localized"),
.gender = ReadRequiredString(name_object, "gender"),
});
}
}
forenames_by_country.emplace(country_code, std::move(entries));
}
cache_.forenames_by_country = std::move(forenames_by_country);
return cache_.forenames_by_country;
}
const SurnamesByCountryMap& CuratedJsonDataService::LoadSurnamesByCountry() {
if (!cache_.surnames_by_country.empty()) {
return cache_.surnames_by_country;
}
const boost::json::value root =
ParseJsonFile(filepaths_.surnames_path, "surnames-by-country");
if (!root.is_object()) {
throw std::runtime_error(
"Invalid surnames-by-country JSON: root element must be an object "
"keyed by ISO 3166-1 country code");
}
SurnamesByCountryMap surnames_by_country;
for (const auto& [country_code, name_entries] : root.as_object()) {
if (!name_entries.is_array()) continue;
SurnameList surnames;
for (const auto& name_value : name_entries.as_array()) {
if (!name_value.is_object()) {
continue;
}
surnames.emplace_back(ReadFirstOfStringArray(name_value.as_object(),
"romanized", "localized"));
}
surnames_by_country.emplace(country_code, std::move(surnames));
}
cache_.surnames_by_country = std::move(surnames_by_country);
return cache_.surnames_by_country;
}

View File

@@ -1,98 +0,0 @@
/**
* @file services/curated_data/mock_curated_data_service.cc
* @brief Fixed in-memory location, persona, and name dataset for mock mode.
*/
#include "services/curated_data/mock_curated_data_service.h"
MockCuratedDataService::MockCuratedDataService()
: locations_{
Location{.city = "Portland",
.state_province = "Oregon",
.iso3166_2 = "US-OR",
.country = "United States",
.iso3166_1 = "US",
.local_languages = {"en"},
.latitude = 45.5152,
.longitude = -122.6784},
Location{.city = "Munich",
.state_province = "Bavaria",
.iso3166_2 = "DE-BY",
.country = "Germany",
.iso3166_1 = "DE",
.local_languages = {"de"},
.latitude = 48.1351,
.longitude = 11.5820},
Location{.city = "Lyon",
.state_province = "Auvergne-Rhone-Alpes",
.iso3166_2 = "FR-ARA",
.country = "France",
.iso3166_1 = "FR",
.local_languages = {"fr"},
.latitude = 45.7640,
.longitude = 4.8357},
Location{.city = "Brussels",
.state_province = "Brussels-Capital",
.iso3166_2 = "BE-BRU",
.country = "Belgium",
.iso3166_1 = "BE",
.local_languages = {"nl", "fr"},
.latitude = 50.8503,
.longitude = 4.3517},
},
personas_{
UserPersona{.name = "Hophead Explorer",
.description = "Chases hop-forward IPAs and seeks out "
"taprooms with rotating drafts.",
.style_affinities = {"IPA", "Pale Ale"}},
UserPersona{.name = "Lager Traditionalist",
.description = "Prefers clean, balanced lagers and "
"classic pub styles.",
.style_affinities = {"Lager", "Pilsner"}},
UserPersona{.name = "Sour Curious",
.description = "Seeks out wild ales, sours, and "
"barrel-aged experiments.",
.style_affinities = {"Sour", "Wild Ale"}},
},
forenames_by_country_{
{"US",
ForenameList{
ForenameEntry{.name = "James", .gender = "M"},
ForenameEntry{.name = "Mary", .gender = "F"},
}},
{"DE",
ForenameList{
ForenameEntry{.name = "Lukas", .gender = "M"},
ForenameEntry{.name = "Anna", .gender = "F"},
}},
{"FR",
ForenameList{
ForenameEntry{.name = "Lucas", .gender = "M"},
ForenameEntry{.name = "Camille", .gender = "F"},
}},
{"BE",
ForenameList{
ForenameEntry{.name = "Noah", .gender = "M"},
ForenameEntry{.name = "Emma", .gender = "F"},
}},
},
surnames_by_country_{
{"US", SurnameList{"Smith", "Johnson"}},
{"DE", SurnameList{"Muller", "Schmidt"}},
{"FR", SurnameList{"Martin", "Bernard"}},
{"BE", SurnameList{"Peeters", "Janssens"}},
} {}
const LocationsList& MockCuratedDataService::LoadLocations() {
return locations_;
}
const PersonasList& MockCuratedDataService::LoadPersonas() { return personas_; }
const ForenamesByCountryMap& MockCuratedDataService::LoadForenamesByCountry() {
return forenames_by_country_;
}
const SurnamesByCountryMap& MockCuratedDataService::LoadSurnamesByCountry() {
return surnames_by_country_;
}

View File

@@ -1,6 +1,5 @@
/**
* @file wikipedia/fetch_extract.cc
* @brief WikipediaEnrichmentService::FetchExtract() implementation.
*/
#include <boost/json.hpp>
@@ -15,16 +14,15 @@
using namespace boost;
std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
const std::string cache_key(query);
// 1. Cache Lookup
if (const auto cache_it = this->extract_cache_.find(cache_key);
cache_it != this->extract_cache_.end()) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
.phase = PipelinePhase::Enrichment,
.message = std::format("Wikipedia: Cache hit for {}!", cache_key)});
logger_->Log(LogLevel::Debug, PipelinePhase::UserGeneration,
std::string("Wikipedia: Cache hit for ") + cache_key + "!");
}
return cache_it->second;
}
@@ -32,10 +30,10 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
const std::string encoded = this->client_->EncodeURL(cache_key);
const std::string url = std::format(
"https://en.wikipedia.org/w/"
"api.php?action=query&titles={}&prop=extracts&explaintext=1&format="
"json",
"api.php?action=query&titles={}&prop=extracts&explaintext=1&format=json",
encoded);
const std::string body = this->client_->Get(url);
{
using namespace std::literals::chrono_literals;
@@ -48,11 +46,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if (ec) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: JSON parse error for '{}': {}",
std::string(query), ec.message())});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: JSON parse error for '") +
std::string(query) + "': " + ec.message());
}
return {};
}
@@ -61,11 +57,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
const json::object* obj = doc.if_object();
if (obj == nullptr) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: Expected root object for '{}'",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: Expected root object for '") +
std::string(query) + "'");
}
return {};
}
@@ -78,11 +72,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if ((pages_ptr == nullptr) || !pages_ptr->is_object()) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: Missing query.pages for '{}'",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: Missing query.pages for '") +
std::string(query) + "'");
}
return {};
}
@@ -91,11 +83,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if (pages.empty()) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: No pages returned for '{}'",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: No pages returned for '") +
std::string(query) + "'");
}
this->extract_cache_.emplace(cache_key, "");
return {};
@@ -107,11 +97,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if (!page_val.is_object()) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: Unexpected page format for '{}'",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: Unexpected page format for '") +
std::string(query) + "'");
}
return {};
}
@@ -121,11 +109,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
// Handle 404/Missing status
if (page.contains("missing")) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format("WikipediaService: Page '{}' does not exist",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: Page '") + std::string(query) +
"' does not exist");
}
this->extract_cache_.emplace(cache_key, "");
return {};
@@ -135,11 +121,9 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
if ((extract_ptr == nullptr) || !extract_ptr->is_string()) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"WikipediaService: No extract string found for '{}'",
std::string(query))});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
std::string("WikipediaService: No extract string found for '") +
std::string(query) + "'");
}
this->extract_cache_.emplace(cache_key, "");
return {};
@@ -148,14 +132,13 @@ std::string WikipediaEnrichmentService::FetchExtract(std::string_view query) {
// 4. Success
std::string extract(extract_ptr->as_string());
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Enrichment,
.message = std::format("WikipediaService: Fetched {} chars for '{}'",
extract.size(), std::string(query))});
logger_->Log(LogLevel::Info, PipelinePhase::UserGeneration,
std::string("WikipediaService: Fetched ") +
std::to_string(extract.size()) + " chars for '" +
std::string(query) + "'");
}
this->extract_cache_.insert_or_assign(cache_key, extract);
return extract;
}
}

View File

@@ -1,6 +1,6 @@
/**
* @file wikipedia/get_summary.cc
* @brief WikipediaEnrichmentService::GetLocationContext() implementation.
* @brief WikipediaService::GetLocationContext() implementation.
*/
#include <chrono>
@@ -10,20 +10,25 @@
#include "services/enrichment/wikipedia_service.h"
std::string WikipediaEnrichmentService::GetLocationContext(
const Location& loc) {
std::string WikipediaEnrichmentService::GetLocationContext(const Location& loc) {
using namespace std::literals::chrono_literals;
if (!this->client_) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::Enrichment,
.message = "Wikipedia client is nullptr."});
logger_->Log(LogLevel::Warn, PipelinePhase::UserGeneration,
"Wikipedia client is nullptr.");
}
return {};
}
std::string result;
// std::string region_query(loc.city);
// if (!loc.country.empty()) {
// region_query += loc.state_province,
// region_query += ", ";
// region_query += loc.country;
// }
constexpr std::string_view brewing_query = "brewing";
const std::string location_query =
std::format("{}, {}", loc.city, loc.iso3166_2);
@@ -43,21 +48,17 @@ std::string WikipediaEnrichmentService::GetLocationContext(
append_extract(FetchExtract(brewing_query));
append_extract(FetchExtract(beer_query));
if (logger_) {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Enrichment,
.message = std::format(
"Done fetching for {}. Sleeping for 10 seconds.",
location_query)});
logger_->Log(LogLevel::Info, PipelinePhase::UserGeneration,
std::string("Done fetching for ") + location_query +
". Sleeping for 10 seconds.");
}
std::this_thread::sleep_for(10s);
} catch (const std::runtime_error& e) {
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
.phase = PipelinePhase::Enrichment,
.message = std::format("WikipediaService lookup failed for '{}': {}",
location_query, e.what())});
logger_->Log(LogLevel::Debug, PipelinePhase::UserGeneration,
std::string("WikipediaService lookup failed for '") +
location_query + "': " + e.what());
}
}
return result;

View File

@@ -1,6 +1,6 @@
/**
* @file services/wikipedia/wikipedia_service.cc
* @brief WikipediaEnrichmentService constructor implementation.
* @brief WikipediaService constructor implementation.
*/
#include "services/enrichment/wikipedia_service.h"

View File

@@ -13,31 +13,6 @@
#include "concurrency/bounded_channel.h"
#include "services/logging/log_entry.h"
namespace {
[[nodiscard]] constexpr std::string_view PipelinePhaseToString(
PipelinePhase phase) {
switch (phase) {
case PipelinePhase::Startup:
return "Startup";
case PipelinePhase::Enrichment:
return "Enrichment";
case PipelinePhase::UserGeneration:
return "User Generation";
case PipelinePhase::BreweryAndBeerGeneration:
return "Brewery & Beer Gen";
case PipelinePhase::CheckinGeneration:
return "Checkin Gen";
case PipelinePhase::RatingGeneration:
return "Rating Gen";
case PipelinePhase::FollowGeneration:
return "Follow Gen";
case PipelinePhase::Teardown:
return "Teardown";
}
return "Unknown";
}
} // namespace
LogDispatcher::LogDispatcher(BoundedChannel<LogEntry>& channel)
: channel_(channel) {}
@@ -53,11 +28,7 @@ void LogDispatcher::Run() {
const auto& log = entry.value();
logger->log(ToSpdlogLevel(log.level),
"{:<20} │ thread: {:016x} │ [{}:{}] │ {}",
PipelinePhaseToString(log.phase),
std::hash<std::thread::id>{}(log.thread_id),
log.origin.file_name(), log.origin.line(), log.message);
logger->log(ToSpdlogLevel(log.level), log.message);
}
}

View File

@@ -16,4 +16,10 @@
LogProducer::LogProducer(BoundedChannel<LogEntry>& channel)
: channel_(channel) {}
void LogProducer::DoLog(LogEntry entry) { channel_.Send(std::move(entry)); }
void LogProducer::Log(LogLevel level, PipelinePhase phase,
const std::string_view message) {
channel_.Send(LogEntry{.timestamp = std::chrono::system_clock::now(),
.level = level,
.phase = phase,
.message = std::string(message)});
}

View File

@@ -6,9 +6,7 @@
#include "services/prompting/prompt_directory.h"
#include <chrono>
#include <filesystem>
#include <format>
#include <fstream>
#include <stdexcept>
#include <string>
@@ -44,18 +42,15 @@ PromptDirectory::PromptDirectory(const std::filesystem::path& prompt_dir,
// Scenario 4: directory must be readable (probe with directory_iterator).
std::filesystem::directory_iterator probe(prompt_dir_, ec);
if (ec) {
throw std::runtime_error(std::format(
"PromptDirectory: prompt directory is not readable: {} ({})",
prompt_dir_.string(), ec.message()));
throw std::runtime_error(
"PromptDirectory: prompt directory is not readable: " +
prompt_dir_.string() + " (" + ec.message() + ")");
}
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message =
std::string("[PromptDirectory] Resolved prompt directory: ") +
prompt_dir_.string()});
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
std::string("[PromptDirectory] Resolved prompt directory: ") +
prompt_dir_.string());
}
}
@@ -70,13 +65,13 @@ std::string PromptDirectory::Load(std::string_view key) {
// Scenario 3: resolve <prompt_dir>/<key>.md and require it to exist.
const std::filesystem::path file_path =
prompt_dir_ / std::filesystem::path(std::format("{}.md", key_str));
prompt_dir_ / std::filesystem::path(key_str + ".md");
std::ifstream file(file_path);
if (!file.is_open()) {
throw std::runtime_error(
std::format("PromptDirectory: prompt file not found for key '{}': {}",
key_str, file_path.string()));
"PromptDirectory: prompt file not found for key '" + key_str +
"': " + file_path.string());
}
std::string content((std::istreambuf_iterator<char>(file)),
@@ -84,18 +79,15 @@ std::string PromptDirectory::Load(std::string_view key) {
file.close();
if (content.empty()) {
throw std::runtime_error(
std::format("PromptDirectory: prompt file for key '{}' is empty: {}",
key_str, file_path.string()));
throw std::runtime_error("PromptDirectory: prompt file for key '" +
key_str + "' is empty: " + file_path.string());
}
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format(
"[PromptDirectory] Loaded prompt '{}' from '{}' ({} chars)",
key_str, file_path.string(), content.size())});
logger_->Log(LogLevel::Info, PipelinePhase::Startup,
std::string("[PromptDirectory] Loaded prompt '") + key_str +
"' from '" + file_path.string() + "' (" +
std::to_string(content.size()) + " chars)");
}
cache_.emplace(key_str, content);

View File

@@ -14,7 +14,6 @@ void SqliteExportService::Finalize() {
}
try {
insert_user_stmt_.reset();
insert_brewery_stmt_.reset();
insert_location_stmt_.reset();
if (transaction_open_) {

View File

@@ -1,6 +1,5 @@
#include "services/database/sqlite_connection_helpers.h"
#include <format>
#include <stdexcept>
namespace sqlite_export_service_internal {
@@ -21,7 +20,7 @@ void SqliteStatementDeleter::operator()(
void ThrowSqliteError(sqlite3* db_handle, std::string_view action) {
const std::string message =
db_handle != nullptr ? sqlite3_errmsg(db_handle) : "unknown SQLite error";
throw std::runtime_error(std::format("{}: {}", action, message));
throw std::runtime_error(std::string(action) + ": " + message);
}
SqliteDatabaseHandle OpenDatabase(const std::filesystem::path& path) {
@@ -51,7 +50,7 @@ void ExecSql(const SqliteDatabaseHandle& db_handle, std::string_view sql,
? error_message
: sqlite3_errmsg(db_handle.get());
sqlite3_free(error_message);
throw std::runtime_error(std::format("{}: {}", action, message));
throw std::runtime_error(std::string(action) + ": " + message);
}
}

View File

@@ -33,7 +33,7 @@ void ResetStatement(SqliteStatementHandle& statement) {
}
void Bind(const SqliteStatementHandle& statement,
const BoundParam<std::string_view>& param) {
const BindParam<std::string_view>& param) {
const auto byte_count = param.value.size();
if (byte_count > static_cast<std::size_t>(std::numeric_limits<int>::max())) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), param.action);
@@ -60,7 +60,7 @@ void Bind(const SqliteStatementHandle& statement,
}
void Bind(const SqliteStatementHandle& statement,
const BoundParam<double>& param) {
const BindParam<double>& param) {
if (sqlite3_bind_double(statement.get(), param.index, param.value) !=
SQLITE_OK) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), param.action);
@@ -68,7 +68,7 @@ void Bind(const SqliteStatementHandle& statement,
}
void Bind(const SqliteStatementHandle& statement,
const BoundParam<sqlite3_int64>& param) {
const BindParam<sqlite3_int64>& param) {
if (sqlite3_bind_int64(statement.get(), param.index, param.value) !=
SQLITE_OK) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), param.action);

View File

@@ -4,7 +4,6 @@
*/
#include <filesystem>
#include <format>
#include <memory>
#include <stdexcept>
#include <string>
@@ -18,9 +17,9 @@ std::filesystem::path SqliteExportService::BuildDatabasePath() const {
std::filesystem::path candidate = output_path_ / base_filename;
for (int suffix = 1; std::filesystem::exists(candidate); ++suffix) {
candidate = output_path_ / std::filesystem::path(
std::format("biergarten_seed_{}-{}.sqlite",
run_timestamp_utc_, suffix));
candidate = output_path_ /
std::filesystem::path("biergarten_seed_" + run_timestamp_utc_ +
"-" + std::to_string(suffix) + ".sqlite");
}
return candidate;
@@ -33,9 +32,6 @@ void SqliteExportService::InitializeSchema() const {
sqlite_export_service_internal::ExecSql(
db_handle_, sqlite_export_service_internal::kCreateBreweriesTableSql,
"Failed to create SQLite breweries table");
sqlite_export_service_internal::ExecSql(
db_handle_, sqlite_export_service_internal::kCreateUsersTableSql,
"Failed to create SQLite users table");
}
void SqliteExportService::PrepareStatements() {
@@ -45,9 +41,6 @@ void SqliteExportService::PrepareStatements() {
insert_brewery_stmt_ = sqlite_export_service_internal::PrepareStatement(
db_handle_, sqlite_export_service_internal::kInsertBrewerySql,
"Failed to prepare SQLite brewery insert statement");
insert_user_stmt_ = sqlite_export_service_internal::PrepareStatement(
db_handle_, sqlite_export_service_internal::kInsertUserSql,
"Failed to prepare SQLite user insert statement");
}
void SqliteExportService::RollbackAndCloseNoThrow() noexcept {
@@ -60,7 +53,6 @@ void SqliteExportService::RollbackAndCloseNoThrow() noexcept {
transaction_open_ = false;
}
insert_user_stmt_.reset();
insert_brewery_stmt_.reset();
insert_location_stmt_.reset();
db_handle_.reset();

View File

@@ -1,7 +1,6 @@
/**
* @file services/sqlite/process_record.cc
* @brief SqliteExportService::ProcessRecord() implementation
* and the shared location-resolution helper.
* @brief SqliteExportService::ProcessRecord() implementation.
*/
#include <iomanip>
@@ -30,117 +29,111 @@ std::string SqliteExportService::BuildLocationKey(const Location& location) {
return key_stream.str();
}
sqlite3_int64 SqliteExportService::ResolveLocationId(const Location& location) {
const std::string location_key = BuildLocationKey(location);
const auto cached_location = location_cache_.find(location_key);
if (cached_location != location_cache_.end()) {
return cached_location->second;
}
const std::string local_languages_json =
sqlite_export_service_internal::SerializeVector(location.local_languages);
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationCityBindIndex,
.value = location.city,
.action = "Failed to bind SQLite location city"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index =
sqlite_export_service_internal::kLocationStateProvinceBindIndex,
.value = location.state_province,
.action = "Failed to bind SQLite location state/province"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationIso31662BindIndex,
.value = location.iso3166_2,
.action = "Failed to bind SQLite location ISO 3166-2 code"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationCountryBindIndex,
.value = location.country,
.action = "Failed to bind SQLite location country"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationIso31661BindIndex,
.value = location.iso3166_1,
.action = "Failed to bind SQLite location ISO 3166-1 code"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationLanguagesBindIndex,
.value = local_languages_json,
.action = "Failed to bind SQLite location languages"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam{
.index = sqlite_export_service_internal::kLocationLatitudeBindIndex,
.value = location.latitude,
.action = "Failed to bind SQLite location latitude"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BoundParam{
.index = sqlite_export_service_internal::kLocationLongitudeBindIndex,
.value = location.longitude,
.action = "Failed to bind SQLite location longitude"});
sqlite_export_service_internal::StepStatement(
db_handle_, insert_location_stmt_,
"Failed to insert SQLite location row");
const sqlite3_int64 location_id =
sqlite_export_service_internal::LastInsertRowId(db_handle_);
location_cache_.emplace(location_key, location_id);
sqlite_export_service_internal::ResetStatement(insert_location_stmt_);
return location_id;
}
uint64_t SqliteExportService::ProcessRecord(const BreweryRecord& brewery) {
uint64_t SqliteExportService::ProcessRecord(const GeneratedBrewery& brewery) {
if (db_handle_ == nullptr || !transaction_open_) {
throw std::runtime_error("SQLite export service is not initialized");
}
const sqlite3_int64 location_id = ResolveLocationId(brewery.location);
const std::string location_key = BuildLocationKey(brewery.location);
const auto cached_location = location_cache_.find(location_key);
sqlite3_int64 location_id = 0;
if (cached_location != location_cache_.end()) {
location_id = cached_location->second;
} else {
const std::string local_languages_json =
sqlite_export_service_internal::SerializeVector(
brewery.location.local_languages);
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationCityBindIndex,
.value = brewery.location.city,
.action = "Failed to bind SQLite location city"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index =
sqlite_export_service_internal::kLocationStateProvinceBindIndex,
.value = brewery.location.state_province,
.action = "Failed to bind SQLite location state/province"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationIso31662BindIndex,
.value = brewery.location.iso3166_2,
.action = "Failed to bind SQLite location ISO 3166-2 code"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationCountryBindIndex,
.value = brewery.location.country,
.action = "Failed to bind SQLite location country"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kLocationIso31661BindIndex,
.value = brewery.location.iso3166_1,
.action = "Failed to bind SQLite location ISO 3166-1 code"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam<std::string_view>{
.index =
sqlite_export_service_internal::kLocationLanguagesBindIndex,
.value = local_languages_json,
.action = "Failed to bind SQLite location languages"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam{
.index = sqlite_export_service_internal::kLocationLatitudeBindIndex,
.value = brewery.location.latitude,
.action = "Failed to bind SQLite location latitude"});
sqlite_export_service_internal::Bind(
insert_location_stmt_,
sqlite_export_service_internal::BindParam{
.index =
sqlite_export_service_internal::kLocationLongitudeBindIndex,
.value = brewery.location.longitude,
.action = "Failed to bind SQLite location longitude"});
sqlite_export_service_internal::StepStatement(
db_handle_, insert_location_stmt_,
"Failed to insert SQLite location row");
location_id = sqlite_export_service_internal::LastInsertRowId(db_handle_);
location_cache_.emplace(location_key, location_id);
sqlite_export_service_internal::ResetStatement(insert_location_stmt_);
}
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<sqlite3_int64>{
sqlite_export_service_internal::BindParam<sqlite3_int64>{
.index = sqlite_export_service_internal::kBreweryLocationIdBindIndex,
.value = location_id,
.action = "Failed to bind SQLite brewery location id"});
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kBreweryEnglishNameBindIndex,
.value = brewery.brewery.name_en,
.action = "Failed to bind SQLite brewery English name"});
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::
kBreweryEnglishDescriptionBindIndex,
.value = brewery.brewery.description_en,
.action = "Failed to bind SQLite brewery English description"});
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
sqlite_export_service_internal::BindParam<std::string_view>{
.index = sqlite_export_service_internal::kBreweryLocalNameBindIndex,
.value = brewery.brewery.name_local,
.action = "Failed to bind SQLite brewery local name"});
sqlite_export_service_internal::Bind(
insert_brewery_stmt_,
sqlite_export_service_internal::BoundParam<std::string_view>{
sqlite_export_service_internal::BindParam<std::string_view>{
.index =
sqlite_export_service_internal::kBreweryLocalDescriptionBindIndex,
.value = brewery.brewery.description_local,

Some files were not shown because too many files have changed in this diff Show More