AaronPo/the-biergarten-app
1
0
Fork 0
mirror of https://github.com/aaronpo97/the-biergarten-app.git synced 2026-07-16 17:47:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: AaronPo:pipeline/fix/mock-builds
Branches Tags
AaronPo:main-2.0 AaronPo:seeding-updates AaronPo:pipeline/fix/mock-builds AaronPo:main-1.0
..
compare: AaronPo:d40ce3436304b206e6053b80af274c554b3b3224
Branches Tags
AaronPo:seeding-updates AaronPo:main-2.0 AaronPo:pipeline/fix/mock-builds AaronPo:main-1.0

9 Commits
pipeline/f ... d40ce34363

Author SHA1 Message Date
Aaron Po
d40ce34363 Add ethics document, edit diagrams 2026-04-22 13:51:53 -04:00
Aaron Po
a8e0ced8ba Merge branch 'main-2.0' into planning/pipeline-architecture 2026-04-21 00:55:29 -04:00
Aaron Po
bbe8970bf6 updates 2026-04-21 00:11:48 -04:00
Aaron Po
6657015ee3 fjdsa 2026-04-20 22:50:29 -04:00
Aaron Po
915301fccb add svgs 2026-04-20 16:53:35 -04:00
Aaron Po
d9412df86c Add new design: concurrent design 2026-04-20 16:51:35 -04:00
Aaron Po
660e34483e Add new design: concurrent design 2026-04-20 16:48:23 -04:00
Aaron Po
aa46cf2b4b updates 2026-04-20 03:31:42 -04:00
Aaron Po
fd6ba35f68 add future plans 2026-04-20 00:07:50 -04:00
537 changed files with 185190 additions and 79757 deletions
Expand all files Collapse all files

13
.config/dotnet-tools.json Normal file
View File

@@ -0,0 +1,13 @@
{
"version": 1,
"isRoot": true,
"tools": {
"csharpier": {
"version": "1.2.1",
"commands": [
"csharpier"
],
"rollForward": false
}
}
}

4
web/backend/.csharpierrc.json → .csharpierrc.json
View File

@@ -1,5 +1,7 @@
{
"printWidth": 100,
"$schema": "https://json.schemastore.org/csharpier.json",
"printWidth": 80,
"useTabs": false,
"indentSize": 4,
"endOfLine": "lf",

6
web/.env.example → .env.example
View File

@@ -49,10 +49,8 @@ CONFIRMATION_TOKEN_SECRET=your-secure-jwt-confirmation-secret-key
# provider (e.g., SendGrid, Mailgun, Amazon SES).
SMTP_HOST=mailpit
SMTP_PORT=1025
SMTP_USERNAME=ayerble
SMTP_PASSWORD=checkers
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_USE_SSL=false
SMTP_FROM_EMAIL=noreply@thebiergarten.app
SMTP_FROM_NAME=The Biergarten App
WEBSITE_BASE_URL=http://localhost:3000

1
.gitattributes vendored
View File

@@ -1 +0,0 @@
archive/** linguist-vendored

20
.github/ISSUE_TEMPLATE/feature_request.md vendored
View File

@@ -7,7 +7,6 @@ assignees: []
---
## User Story
**As a** (who wants to accomplish something)
**I want to** (what they want to accomplish)
**So that** (why they want to accomplish that thing)
@@ -16,18 +15,29 @@ assignees: []
### Scenario 1
Given ... When ... Then ...
Given ...
When ...
Then ...
### Scenario 2
Given ... When ... Then ...
Given ...
When ...
Then ...
### Scenario 3
Given ... When ... Then ...
Given ...
When ...
Then ...
## Subtasks
- [ ] Task 1
- [ ] Task 2
- [ ] Task 3

962
LICENSE.md
View File

File diff suppressed because it is too large Load Diff

162
README.md
View File

@@ -1,56 +1,40 @@
# The Biergarten App
The Biergarten App is a full-stack directory and discovery platform for
breweries. It features a robust user authentication system, a searchable
database of brewery locations, and a custom offline data-generation pipeline
that uses LLMs (Llama.cpp) and Wikipedia to synthesize realistic seed data.
The Biergarten App is a multi-project monorepo with a .NET backend and an active React
Router frontend in `src/Website`. The current website focuses on account flows, theme
switching, shared UI components, Storybook coverage, and integration with the API.
It features:
## Documentation
- A .NET backend (Web API + database migrations/seed) under `web/backend/`
- A server-rendered React website (React Router + Vite) under `web/frontend/`
- A C++20 “pipeline” CLI for generating seed data under `tooling/pipeline/`
Specialized documentation (setup, architecture, docker, testing, diagrams, and
pipeline notes) lives under `docs/`.
## Documentation (Start Here)
Website + backend (active stack):
- [Getting Started](docs/website/getting-started.md)
- [Architecture](docs/architecture.md)
- [Docker Guide](docs/website/docker.md)
- [Testing](docs/website/testing.md)
- [Environment Variables](docs/website/environment-variables.md)
- [Token Validation](docs/website/token-validation.md)
Data generation pipeline (C++):
- [Pipeline README](docs/pipeline/README.md)
- [Ethics & Known Issues](docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md)
- [Getting Started](docs/getting-started.md) - Local setup for backend and active website
- [Architecture](docs/architecture.md) - Current backend and frontend architecture
- [Docker Guide](docs/docker.md) - Container-based backend development and testing
- [Testing](docs/testing.md) - Backend and frontend test commands
- [Environment Variables](docs/environment-variables.md) - Active configuration reference
- [Token Validation](docs/token-validation.md) - JWT validation architecture
- [Legacy Website Archive](docs/archive/legacy-website-v1.md) - Archived notes for the old Next.js frontend
## Diagrams
- [Architecture](docs/website/diagrams-out/architecture.svg)
- [Deployment](docs/website/diagrams-out/deployment.svg)
- [Authentication Flow](docs/website/diagrams-out/authentication-flow.svg)
- [Database Schema](docs/website/diagrams-out/database-schema.svg)
- [Architecture](docs/diagrams-out/architecture.svg) - Layered architecture
- [Deployment](docs/diagrams-out/deployment.svg) - Docker topology
- [Authentication Flow](docs/diagrams-out/authentication-flow.svg) - Auth sequence
- [Database Schema](docs/diagrams-out/database-schema.svg) - Entity relationships
## Current Status
Active areas in the repository:
- .NET 10 backend (vertical-slice architecture with MediatR) + SQL Server
- React 19 website (React Router 7 + Vite)
- Shared Biergarten theme system + Storybook coverage
- Auth flows and account/email integration (local Mailpit in dev compose)
- Data generation pipeline with C++ and Llama.cpp
- .NET 10 backend with layered architecture and SQL Server
- React Router 7 website in `src/Website`
- Shared Biergarten theme system with a theme guide route
- Storybook stories and browser-based checks for shared UI
- Auth demo flows for home, login, register, dashboard, logout, and confirmation
- Toast-based feedback for auth outcomes
Archived/reference areas:
Legacy area retained for reference:
- `archive/next-js-web-app/` contains an older Next.js frontend retained for
reference
- `src/Website-v1` contains the archived Next.js frontend and is no longer the active website
## Tech Stack
@@ -59,43 +43,36 @@ Archived/reference areas:
- **UI Documentation**: Storybook 10, Vitest browser mode, Playwright
- **Testing**: xUnit, Reqnroll (BDD), FluentAssertions, Moq
- **Infrastructure**: Docker, Docker Compose
- **Security**: Argon2id password hashing, JWT access/refresh/confirmation
tokens
- **Data Pipeline**: C++20, CMake, Boost, libcurl, SQLite, llama.cpp
- **Security**: Argon2id password hashing, JWT access/refresh/confirmation tokens
## Quick Start
For full setup details, use [Getting Started](docs/website/getting-started.md).
This section is the shortest path to a working dev environment.
### Backend (Docker)
### Backend
```bash
git clone https://github.com/aaronpo97/the-biergarten-app
cd the-biergarten-app
cp web/.env.example web/.env.dev
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up --build -d
cp .env.example .env.dev
docker compose -f docker-compose.dev.yaml up -d
```
Backend access:
- API Swagger: http://localhost:8080/swagger
- Health Check: http://localhost:8080/health
- Mailpit UI (dev SMTP): http://localhost:8025
### Frontend (Node)
### Frontend
```bash
cd web/frontend
cd src/Website
npm install
API_BASE_URL=http://localhost:8080 SESSION_SECRET=dev-secret-change-me npm run dev
API_BASE_URL=http://localhost:8080 SESSION_SECRET=dev-secret npm run dev
```
Optional frontend tools:
```bash
cd web/frontend
cd src/Website
npm run storybook
npm run test:storybook
npm run test:storybook:playwright
@@ -104,43 +81,62 @@ npm run test:storybook:playwright
## Repository Structure
```text
web/
backend/ .NET API: API.Core (host), Features.* (vertical slices),
Shared.*, Domain.*, Infrastructure.*, Database.* projects
frontend/ React Router website + Storybook + Playwright/Vitest
tooling/
pipeline/ C++20 seed-data generation CLI (CMake)
docs/
architecture.md High-level architecture overview
website/ Backend/frontend setup, docker, testing, diagrams
pipeline/ Pipeline docs, ethics notes, PlantUML diagrams
archive/
next-js-web-app/ Older Next.js frontend (reference only)
src/Core/ Backend projects (.NET)
src/Website/ Active React Router frontend
src/Website-v1/ Archived legacy Next.js frontend
docs/ Active project documentation
docs/archive/ Archived legacy documentation
```
## Key Features
Implemented today:
- User registration and login against the API
- JWT-based auth with access, refresh, and confirmation flows
- SQL Server migrations and seed projects
- Shared form components and auth screens
- Theme switching with Lager, Stout, Cassis, and Weizen variants
- Storybook documentation and automated story interaction tests
- Toast feedback for auth-related outcomes
Planned next:
- Brewery discovery and management
- Beer reviews and ratings
- Social follow relationships
- Geospatial brewery experiences
- Additional frontend routes beyond the auth demo
## Testing
Run the backend test stack with Docker:
Backend suites:
- `API.Specs` - integration tests
- `Infrastructure.Repository.Tests` - repository unit tests
- `Service.Auth.Tests` - service unit tests
Frontend suites:
- Storybook interaction tests via Vitest
- Storybook browser regression checks via Playwright
Run all backend tests with Docker:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
```
See [Testing](docs/website/testing.md) for the full command list.
See [Testing](docs/testing.md) for the full command list.
## Configuration
Common active variables:
- Backend/Docker: `DB_SERVER`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`,
`ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`, `CONFIRMATION_TOKEN_SECRET`,
`WEBSITE_BASE_URL`
- Frontend runtime: `API_BASE_URL`, `SESSION_SECRET`, `NODE_ENV`
- Backend: `DB_SERVER`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`, `ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`, `CONFIRMATION_TOKEN_SECRET`
- Frontend: `API_BASE_URL`, `SESSION_SECRET`, `NODE_ENV`
See [Environment Variables](docs/website/environment-variables.md) for details.
See [Environment Variables](docs/environment-variables.md) for details.
## Contributing
@@ -149,3 +145,15 @@ See [Environment Variables](docs/website/environment-variables.md) for details.
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request
### Development Workflow
1. Start development environment: `docker compose -f docker-compose.dev.yaml up -d`
2. Make changes to code
3. Run tests: `docker compose -f docker-compose.test.yaml up --abort-on-container-exit`
4. Rebuild if needed: `docker compose -f docker-compose.dev.yaml up -d --build api.core`
## Support
- **Documentation**: [docs/](docs/)
- **Architecture**: See [Architecture Guide](docs/architecture.md)

10
web/docker-compose.db.yaml → docker-compose.db.yaml
View File

@@ -13,11 +13,7 @@ services:
volumes:
- sqlserverdata-dev:/var/opt/mssql
healthcheck:
test:
[
"CMD-SHELL",
"/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1",
]
test: ["CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1"]
interval: 10s
timeout: 5s
retries: 12
@@ -32,7 +28,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend/Database
context: ./src/Core/Database
dockerfile: Database.Migrations/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -56,7 +52,7 @@ services:
database.migrations:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: Database/Database.Seed/Dockerfile
args:
BUILD_CONFIGURATION: Release

12
web/docker-compose.dev.yaml → docker-compose.dev.yaml
View File

@@ -13,11 +13,7 @@ services:
volumes:
- sqlserverdata-dev:/var/opt/mssql
healthcheck:
test:
[
"CMD-SHELL",
"/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1",
]
test: [ "CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1" ]
interval: 10s
timeout: 5s
retries: 12
@@ -32,7 +28,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend/Database
context: ./src/Core/Database
dockerfile: Database.Migrations/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -56,7 +52,7 @@ services:
database.migrations:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: Database/Database.Seed/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -79,7 +75,7 @@ services:
database.seed:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: API/API.Core/Dockerfile
args:
BUILD_CONFIGURATION: Release

10
web/docker-compose.min.yaml → docker-compose.min.yaml
View File

@@ -13,11 +13,7 @@ services:
volumes:
- sqlserverdata-dev:/var/opt/mssql
healthcheck:
test:
[
"CMD-SHELL",
"/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1",
]
test: [ "CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1" ]
interval: 10s
timeout: 5s
retries: 12
@@ -32,7 +28,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend/Database
context: ./src/Core/Database
dockerfile: Database.Migrations/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -70,7 +66,7 @@ services:
database.migrations:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: Database/Database.Seed/Dockerfile
args:
BUILD_CONFIGURATION: Release

10
web/docker-compose.prod.yaml → docker-compose.prod.yaml
View File

@@ -11,11 +11,7 @@ services:
volumes:
- sqlserverdata-prod:/var/opt/mssql
healthcheck:
test:
[
"CMD-SHELL",
"/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1",
]
test: ["CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1"]
interval: 10s
timeout: 5s
retries: 12
@@ -31,7 +27,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend/Database
context: ./src/Core/Database
dockerfile: Database.Migrations/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -54,7 +50,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend
context: ./src/Core
dockerfile: API/API.Core/Dockerfile
args:
BUILD_CONFIGURATION: Release

42
web/docker-compose.test.yaml → docker-compose.test.yaml
View File

@@ -12,11 +12,7 @@ services:
volumes:
- sqlserverdata-test:/var/opt/mssql
healthcheck:
test:
[
"CMD-SHELL",
"/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1",
]
test: [ "CMD-SHELL", "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1' || exit 1" ]
interval: 10s
timeout: 5s
retries: 12
@@ -32,7 +28,7 @@ services:
sqlserver:
condition: service_healthy
build:
context: ./backend/Database
context: ./src/Core/Database
dockerfile: Database.Migrations/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -56,7 +52,7 @@ services:
database.migrations:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: Database/Database.Seed/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -79,7 +75,7 @@ services:
database.seed:
condition: service_completed_successfully
build:
context: ./backend
context: ./src/Core
dockerfile: API/API.Specs/Dockerfile
args:
BUILD_CONFIGURATION: Release
@@ -99,16 +95,36 @@ services:
networks:
- testnet
unit.tests:
repository.tests:
env_file: ".env.test"
image: unit.tests
container_name: test-env-unit-tests
image: repository.tests
container_name: test-env-repository-tests
depends_on:
database.seed:
condition: service_completed_successfully
build:
context: ./backend
dockerfile: Dockerfile.tests
context: ./src/Core
dockerfile: Infrastructure/Infrastructure.Repository.Tests/Dockerfile
args:
BUILD_CONFIGURATION: Release
environment:
DOTNET_RUNNING_IN_CONTAINER: "true"
volumes:
- ./test-results:/app/test-results
restart: "no"
networks:
- testnet
service.auth.tests:
env_file: ".env.test"
image: service.auth.tests
container_name: test-env-service-auth-tests
depends_on:
database.seed:
condition: service_completed_successfully
build:
context: ./src/Core
dockerfile: Service/Service.Auth.Tests/Dockerfile
args:
BUILD_CONFIGURATION: Release
environment:

293
docs/architecture.md
View File

@@ -4,68 +4,60 @@ This document describes the active architecture of The Biergarten App.
## High-Level Overview
The Biergarten App is a monorepo with a clear split between the backend and the
active website:
The Biergarten App is a monorepo with a clear split between the backend and the active
website:
- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature
slices with MediatR
- **Frontend**: React 19 + React Router 7 website in `web/frontend`
- **Architecture Style**: Vertical-slice backend plus server-rendered React
frontend
- **Backend**: .NET 10 Web API with SQL Server and a layered architecture
- **Frontend**: React 19 + React Router 7 website in `src/Website`
- **Architecture Style**: Layered backend plus server-rendered React frontend
The legacy Next.js frontend has been retained in `archive/next-js-web-app/`
for reference only.
The legacy Next.js frontend has been retained in `src/Website-v1` for reference only and is
documented in [archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Diagrams
For visual representations, see:
- [architecture.svg](website/diagrams-out/architecture.svg) - Vertical-slice
architecture diagram
- [deployment.svg](website/diagrams-out/deployment.svg) - Docker deployment
diagram
- [authentication-flow.svg](website/diagrams-out/authentication-flow.svg) -
Authentication workflow
- [database-schema.svg](website/diagrams-out/database-schema.svg) - Database
relationships
- [architecture.svg](diagrams-out/architecture.svg) - Layered architecture diagram
- [deployment.svg](diagrams-out/deployment.svg) - Docker deployment diagram
- [authentication-flow.svg](diagrams-out/authentication-flow.svg) - Authentication workflow
- [database-schema.svg](diagrams-out/database-schema.svg) - Database relationships
## Backend Architecture
### Vertical Slice Architecture Pattern
### Layered Architecture Pattern
The backend organizes business capabilities as feature slices instead of
technical layers. Each feature (`Features.Auth`, `Features.Breweries`,
`Features.UserManagement`, `Features.Emails`) is a single project that owns
its own controller, MediatR commands/queries/handlers, validators, and
repository, end to end:
The backend follows a strict layered architecture:
```
┌───────────────────────────────────────────────────────────────────────┐
│ API.Core (thin host) │
│ - Program.cs wiring: MediatR + AddApplicationPart per slice │
│ - Swagger/OpenAPI, JWT auth middleware, global exception filter │
└───────────────────────────────────────────────────────────────────────┘
↓ discovers controllers via AddApplicationPart
┌───────────────┬───────────────┬───────────────────┬───────────────────┐
│ Features.Auth │Features. │ Features. │ Features.Emails │
│ │Breweries │ UserManagement │ (no controller, │
│ Controller │ Controller │ Controller │ internal only) │
│ Commands/ │ Commands/ │ Commands/ │ Commands/ │
│ Queries + │ Queries + │ Queries + │ Handlers │
│ Handlers │ Handlers │ Handlers │ │
│ Repository │ Repository │ Repository │ EmailDispatcher │
└───────────────┴───────────────┴───────────────────┴───────────────────┘
↓ each slice depends only on shared/domain/infra, never on another slice
┌─────────────────────────┬─────────────────────────┬────────────────────┐
│ Shared.Contracts │ Shared.Application │ Domain.Entities / │
│ (ResponseBody envelope) │ (ValidationBehavior, │ Domain.Exceptions │
│ │ cross-slice email cmds) │ │
└─────────────────────────┴─────────────────────────┴────────────────────┘
┌─────────────────────────────────────┐
│ API Layer (Controllers) │
│ - HTTP Endpoints │
│ - Request/Response mapping │
│ - Swagger/OpenAPI │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────────────┐
│ Infrastructure.Sql, Infrastructure.Jwt, Infrastructure.PasswordHashing, │
│ Infrastructure.Email, Infrastructure.Email.Templates │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────┐
│ Service Layer (Business Logic) │
│ - Authentication logic │
│ - User management │
│ - Validation & orchestration │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Infrastructure Layer (Tools) │
│ - JWT token generation │
│ - Password hashing (Argon2id) │
│ - Email services │
│ - Repository implementations │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Domain Layer (Entities) │
│ - UserAccount, UserCredential │
│ - Pure POCO classes │
│ - No external dependencies │
└─────────────────────────────────────┘
↓
┌─────────────────────────────────────┐
│ Database (SQL Server) │
@@ -74,123 +66,75 @@ repository, end to end:
└─────────────────────────────────────┘
```
Slices never reference each other's project. The one cross-slice
interaction (`Features.Auth` triggering a confirmation email handled by
`Features.Emails`) goes through a MediatR command (`SendRegistrationEmailCommand`)
whose contract lives in `Shared.Application`, so neither slice takes a
project reference on the other.
### Layer Responsibilities
#### API Layer (`API.Core`)
**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of
its own
**Purpose**: HTTP interface and request handling
**Components**:
- `Program.cs`: registers MediatR (scanning every `Features.*` assembly),
FluentValidation, and uses `AddApplicationPart` so each slice's controllers
are discovered by MVC
- `GlobalException.cs`: global exception filter (host-level cross-cutting
concern)
- `Authentication/JwtAuthenticationHandler.cs`: JWT auth scheme middleware
- Swagger/OpenAPI documentation, health check endpoints
- Controllers (`AuthController`, `UserController`)
- Middleware for error handling
- Swagger/OpenAPI documentation
- Health check endpoints
**Dependencies**:
- Every `Features.*` project (for controller/MediatR discovery)
- `Shared.Contracts`, `Shared.Application`
- `Infrastructure.Jwt` (for the auth middleware)
- Service layer
- ASP.NET Core framework
**Rules**:
- No controllers, no business logic, no feature-specific contracts
- Exists purely to host and wire up the feature slices
- No business logic
- Only request/response transformation
- Delegates all work to Service layer
#### Feature Slices (`Features.Auth`, `Features.Breweries`, `Features.UserManagement`, `Features.Emails`)
#### Service Layer (`Service.Auth`, `Service.UserManagement`)
**Purpose**: Each slice is the complete vertical for one business capability
**Components** (per slice):
- `Controllers/`: HTTP endpoints, binding directly to Command/Query types
as the request contract
- `Commands/<Operation>/` and `Queries/<Operation>/`: one folder per
operation, each containing the Command/Query record, its
`IRequestHandler`, and (for commands) a FluentValidation validator
- `Repository/`: the slice's own stored-procedure repository
implementation
- `Dtos/`: response shapes returned by query handlers (never the raw
domain entity)
- `DependencyInjection/`: an `AddFeaturesX()` extension method registering
the slice's repository/services
`Features.Emails` has no `Controllers/` folder. It's invoked only via
MediatR commands sent from other slices, never over HTTP.
**Dependencies**:
- `Domain.Entities`, `Domain.Exceptions`
- `Infrastructure.Sql` (generic ADO.NET plumbing) plus whichever
infrastructure project the slice needs (`Infrastructure.Jwt`/
`Infrastructure.PasswordHashing` for Auth, `Infrastructure.Email`/
`Infrastructure.Email.Templates` for Emails)
- `Shared.Contracts`, `Shared.Application`
- **Never** another `Features.*` project
**Rules**:
- All business logic for that feature lives in its command/query handlers
- No direct controller-to-repository calls; everything flows through
MediatR
- Read endpoints return a dedicated `Dto`, never the domain entity directly
#### Shared Projects (`Shared.Contracts`, `Shared.Application`)
**Purpose**: The minimum cross-slice surface area required because every
slice needs it, or because duplicating it four times would be worse than
sharing it
**Purpose**: Business logic and orchestration
**Components**:
- `Shared.Contracts`: `ResponseBody<T>`/`ResponseBody`, the API response
envelope every controller returns
- `Shared.Application`: `ValidationBehavior<TRequest,TResponse>` (the
MediatR pipeline behavior that runs FluentValidation before a handler
executes) and the cross-slice email commands
(`SendRegistrationEmailCommand`, `SendResendConfirmationEmailCommand`)
- Authentication services (login, registration)
- User management services
- Business rule validation
- Transaction coordination
**Dependencies**:
- Infrastructure layer (repositories, JWT, password hashing)
- Domain entities
**Rules**:
- Kept deliberately small: this is the exception to "no slice depends on
another slice," not a general-purpose dumping ground
- Contains all business logic
- Coordinates multiple infrastructure components
- No direct database access (uses repositories)
- Returns domain models, not DTOs
#### Infrastructure Layer
**Purpose**: Technical capabilities and external integrations, shared by
whichever slices need them
**Purpose**: Technical capabilities and external integrations
**Components**:
- **Infrastructure.Sql**: generic ADO.NET connection/command plumbing
(`ISqlConnectionFactory`, the abstract `Repository<T>` base class), not
domain-specific; each slice's own `Repository/` folder builds on this
- **Infrastructure.Repository**: Data access via stored procedures
- **Infrastructure.Jwt**: JWT token generation and validation
- **Infrastructure.PasswordHashing**: Argon2id password hashing
- **Infrastructure.Email**: Email sending capabilities (SMTP/MailKit)
- **Infrastructure.Email.Templates**: Email template rendering (Razor
components)
- **Infrastructure.Email**: Email sending capabilities
- **Infrastructure.Email.Templates**: Email template rendering
**Dependencies**:
- Domain entities
- External libraries (ADO.NET, JWT, Argon2, MailKit, etc.)
- External libraries (ADO.NET, JWT, Argon2, etc.)
**Rules**:
- Implements technical concerns only, no business logic
- Reusable across slices
- Implements technical concerns
- No business logic
- Reusable across services
#### Domain Layer (`Domain.Entities`)
@@ -215,55 +159,21 @@ whichever slices need them
### Design Patterns
#### Vertical Slice + MediatR
**Purpose**: Organize code by feature instead of by technical layer, so
everything needed to understand or change one capability lives in one
project
**Implementation**:
- Each HTTP write operation is a `Command` (e.g. `CreateBreweryCommand` in
`Features.Breweries/Commands/CreateBrewery/`); each read operation is a
`Query` (e.g. `GetBreweryByIdQuery`)
- Controllers bind directly to the Command/Query as the request body;
there is no separate request DTO + mapping step for writes
- A single shared `ValidationBehavior<TRequest,TResponse>`
(`Shared.Application/Behaviors/`) runs FluentValidation validators in the
MediatR pipeline before any handler executes
- Query handlers map to a dedicated response `Dto`, so domain entities never
leak over the wire
**Example**:
```csharp
public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...)
: IRequest<BreweryDto>;
public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler<CreateBreweryCommand, BreweryDto>
{
public async Task<BreweryDto> Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... }
}
```
#### Repository Pattern
**Purpose**: Abstract database access behind interfaces
**Implementation**: each slice owns its own repository, scoped to that
feature only:
**Implementation**:
- `Features.Auth/Repository/IAuthRepository.cs`
- `Features.Breweries/Repository/IBreweryRepository.cs`
- `Features.UserManagement/Repository/IUserAccountRepository.cs`
- `Infrastructure.Sql/DefaultSqlConnectionFactory.cs`: the generic
connection factory every slice's repository builds on
- `IAuthRepository` - Authentication queries
- `IUserAccountRepository` - User account queries
- `DefaultSqlConnectionFactory` - Connection management
**Benefits**:
- Testable (easy to mock)
- SQL-first approach (stored procedures)
- Each slice's data access logic is self-contained
- Centralized data access logic
**Example**:
@@ -279,19 +189,17 @@ public interface IAuthRepository
**Purpose**: Loose coupling and testability
**Configuration**: `Program.cs` wires up MediatR/FluentValidation across
every `Features.*` assembly; each slice exposes its own `AddFeaturesX()`
extension method that registers its repository and slice-internal services
**Configuration**: `Program.cs` registers all services
**Lifetimes**:
- Scoped: Repositories, slice-internal services (per request)
- Singleton: `ISqlConnectionFactory`
- Scoped: Repositories, Services (per request)
- Singleton: Connection factories, JWT configuration
- Transient: Utilities, helpers
#### SQL-First Approach
**Purpose**: Push complex logic into the database
**Purpose**: Leverage database capabilities
**Strategy**:
@@ -308,13 +216,12 @@ extension method that registers its repository and slice-internal services
## Frontend Architecture
### Active Website (`web/frontend`)
### Active Website (`src/Website`)
The current website is a React Router 7 application with server-side rendering
enabled.
The current website is a React Router 7 application with server-side rendering enabled.
```text
web/frontend/
src/Website/
├── app/
│ ├── components/ Shared UI such as Navbar, FormField, SubmitButton, ToastProvider
│ ├── lib/ Auth helpers, schemas, and theme metadata
@@ -337,22 +244,21 @@ web/frontend/
### Theme System
The active website uses semantic DaisyUI theme tokens backed by four Biergarten
themes:
The active website uses semantic DaisyUI theme tokens backed by four Biergarten themes:
- Biergarten Lager
- Biergarten Stout
- Biergarten Cassis
- Biergarten Weizen
All component styling should prefer semantic tokens such as `primary`,
`success`, `surface`, and `highlight` instead of hard-coded color values.
All component styling should prefer semantic tokens such as `primary`, `success`,
`surface`, and `highlight` instead of hard-coded color values.
### Legacy Frontend
The previous Next.js frontend has been archived at `archive/next-js-web-app/`
for reference only. Active product and engineering documentation should point
to `web/frontend`.
The previous Next.js frontend has been archived at `src/Website-v1`. Active product and
engineering documentation should point to `src/Website`, while legacy notes live in
[archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Security Architecture
@@ -477,12 +383,12 @@ scripts/
- Testing (`docker-compose.test.yaml`)
- Production (`docker-compose.prod.yaml`)
For details, see [Docker Guide](website/docker.md).
For details, see [Docker Guide](docker.md).
### Health Checks
**SQL Server**: Validates database connectivity **API**: Checks service health
and dependencies
**SQL Server**: Validates database connectivity **API**: Checks service health and
dependencies
**Configuration**:
@@ -503,10 +409,11 @@ healthcheck:
│ Integration │ ← API.Specs (Reqnroll)
│ Tests │
├──────────────┤
│ Unit Tests │ ← Features.Auth.Tests, Features.Breweries.Tests,
│ (per slice, │ Features.UserManagement.Tests, Features.Emails.Tests
│ handlers + │ (commands/queries/handlers + that slice's own
│ repository) │ repository, mocked with Moq/DbMocker)
│ Unit Tests │ ← Service.Auth.Tests
│ (Service) │ Repository.Tests
├──────────────┤
│ Unit Tests │
│ (Repository) │
└──────────────┘
```
@@ -517,4 +424,4 @@ healthcheck:
- Mock external dependencies
- Test database for integration tests
For details, see [Testing Guide](website/testing.md).
For details, see [Testing Guide](testing.md).

56
docs/archive/legacy-website-v1.md Normal file
View File

@@ -0,0 +1,56 @@
# Legacy Website Archive (`src/Website-v1`)
This archive captures high-level notes about the previous Biergarten frontend so active
project documentation can focus on the current website in `src/Website`.
## Status
- `src/Website-v1` is retained for historical reference only
- It is not the active frontend used by current setup, docs, or testing guidance
- New product and engineering work should target `src/Website`
## Legacy Stack Summary
The archived frontend used a different application model from the current website:
- Next.js 14
- React 18
- Prisma
- Postgres / Neon-hosted database workflows
- Next.js API routes and server-side controllers
- Additional third-party integrations such as Cloudinary, Mapbox, and SparkPost
## Why It Was Archived
The active website moved to a React Router-based frontend that talks directly to the .NET
API. As part of that shift, the main docs were updated to describe:
- `src/Website` as the active frontend
- React Router route modules and server rendering
- Storybook-based component documentation and tests
- Current frontend runtime variables: `API_BASE_URL`, `SESSION_SECRET`, and `NODE_ENV`
## Legacy Documentation Topics Moved Out of Active Docs
The following categories were removed from active documentation and intentionally archived:
- Next.js application structure guidance
- Prisma and Postgres frontend setup
- Legacy frontend environment variables
- External service setup that only applied to `src/Website-v1`
- Old frontend local setup instructions
## When To Use This Archive
Use this file only if you need to:
- inspect the historical frontend implementation
- compare old flows against the current website
- migrate or recover legacy logic from `src/Website-v1`
For all active work, use:
- [Getting Started](../getting-started.md)
- [Architecture](../architecture.md)
- [Environment Variables](../environment-variables.md)
- [Testing](../testing.md)

1
docs/diagrams-out/architecture.svg Normal file
View File

File diff suppressed because one or more lines are too long

0
docs/website/diagrams-out/authentication-flow.svg → docs/diagrams-out/authentication-flow.svg
View File

0
docs/website/diagrams-out/database-schema.svg → docs/diagrams-out/database-schema.svg
View File

0
docs/website/diagrams-out/deployment.svg → docs/diagrams-out/deployment.svg
View File

75
docs/diagrams-src/architecture.puml Normal file
View File

@@ -0,0 +1,75 @@
@startuml architecture
!theme plain
skinparam backgroundColor #FFFFFF
skinparam defaultFontName Arial
skinparam packageStyle rectangle
title The Biergarten App - Layered Architecture
package "API Layer" #E3F2FD {
[API.Core\nASP.NET Core Web API] as API
note right of API
- Controllers (Auth, User)
- Swagger/OpenAPI
- Middleware
- Health Checks
end note
}
package "Service Layer" #F3E5F5 {
[Service.Auth] as AuthSvc
[Service.UserManagement] as UserSvc
note right of AuthSvc
- Business Logic
- Validation
- Orchestration
end note
}
package "Infrastructure Layer" #FFF3E0 {
[Infrastructure.Repository] as Repo
[Infrastructure.Jwt] as JWT
[Infrastructure.PasswordHashing] as PwdHash
[Infrastructure.Email] as Email
}
package "Domain Layer" #E8F5E9 {
[Domain.Entities] as Domain
note right of Domain
- UserAccount
- UserCredential
- UserVerification
end note
}
database "SQL Server" {
[Stored Procedures] as SP
[Tables] as Tables
}
' Relationships
API --> AuthSvc
API --> UserSvc
AuthSvc --> Repo
AuthSvc --> JWT
AuthSvc --> PwdHash
AuthSvc --> Email
UserSvc --> Repo
Repo --> SP
Repo --> Domain
SP --> Tables
AuthSvc --> Domain
UserSvc --> Domain
' Notes
note left of Repo
SQL-first approach
All queries via
stored procedures
end note
@enduml

0
docs/website/diagrams-src/authentication-flow.puml → docs/diagrams-src/authentication-flow.puml
View File

0
docs/website/diagrams-src/database-schema.puml → docs/diagrams-src/database-schema.puml
View File

45
docs/website/diagrams-src/deployment.puml → docs/diagrams-src/deployment.puml
View File

@@ -112,32 +112,38 @@ package "Test Environment\n(docker-compose.test.yaml)" #FFF3E0 {
end note
}
node "unit.tests\n(Features.*.Tests, one container)" as UnitTests {
component "xUnit + Moq + DbMocker" as UnitComp
node "Infrastructure.Repository.Tests\n(Unit Tests)" as RepoTests {
component "xUnit + DbMocker" as RepoComp
note right
Builds Dockerfile.tests, which
restores/builds against Core.slnx
then runs `dotnet test` for every
Features/*.Tests project in turn:
- Features.Auth.Tests
- Features.Breweries.Tests
- Features.UserManagement.Tests
- Features.Emails.Tests
Tests:
- AuthRepository
- UserAccountRepository
- SQL command building
Uses: Moq for handlers,
DbMocker for repositories
Uses: Mock connections
No real database needed
end note
}
node "Service.Auth.Tests\n(Unit Tests)" as SvcTests {
component "xUnit + Moq" as SvcComp
note right
Tests:
- RegisterService
- LoginService
- Token generation
Uses: Mocked dependencies
No database or infrastructure
end note
}
}
folder "test-results/\n(mounted volume)" as Results {
file "api-specs/\n results.trx" as Result1
file "Features.Auth.Tests.trx" as Result2
file "Features.Breweries.Tests.trx" as Result3
file "Features.UserManagement.Tests.trx" as Result4
file "Features.Emails.Tests.trx" as Result5
file "repository-tests/\n results.trx" as Result2
file "service-auth-tests/\n results.trx" as Result3
note bottom
TRX format
@@ -164,16 +170,17 @@ DevAPI .up.> DevSeed : depends_on
TestMig --> TestDB : 1. Migrate
TestSeed --> TestDB : 2. Seed
Specs --> TestDB : 3. Integration test
UnitTests ..> TestDB : depends_on for startup\nordering only (Moq/DbMocker,\nno real connection)
RepoTests ..> TestDB : Mock (no connection)
SvcTests ..> TestDB : Mock (no connection)
TestMig .up.> TestDB : depends_on
TestSeed .up.> TestMig : depends_on
Specs .up.> TestSeed : depends_on
UnitTests .up.> TestSeed : depends_on
' Test results export
Specs --> Results : Export TRX
UnitTests --> Results : Export TRX (one .trx per slice)
RepoTests --> Results : Export TRX
SvcTests --> Results : Export TRX
' Network notes
note bottom of DevDB

87
docs/website/docker.md → docs/docker.md
View File

@@ -1,7 +1,7 @@
# Docker Guide
This document covers Docker deployment, configuration, and troubleshooting for
The Biergarten App.
This document covers Docker deployment, configuration, and troubleshooting for The
Biergarten App.
## Overview
@@ -13,8 +13,7 @@ The project uses Docker Compose to orchestrate multiple services:
- .NET API
- Test runners
See the [deployment diagram](diagrams-out/deployment.svg) for visual
representation.
See the [deployment diagram](diagrams/pdf/deployment.pdf) for visual representation.
## Docker Compose Environments
@@ -25,10 +24,9 @@ representation.
**Features**:
- Persistent SQL Server volume
- NuGet package cache volume (speeds up rebuilds)
- Hot reload support
- Swagger UI enabled
- Seed data included
- Local Mailpit SMTP server for dev email testing
- `CLEAR_DATABASE=true` (drops and recreates schema)
**Services**:
@@ -38,30 +36,28 @@ sqlserver # SQL Server 2022 (port 1433)
database.migrations # DbUp migrations
database.seed # Seed initial data
api.core # Web API (ports 8080, 8081)
mailpit # Local dev SMTP server + UI (port 8025)
```
**Start Development Environment**:
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d
docker compose -f docker-compose.dev.yaml up -d
```
**Access**:
- API Swagger: http://localhost:8080/swagger
- Health Check: http://localhost:8080/health
- Mailpit UI: http://localhost:8025
- SQL Server: localhost:1433 (sa credentials from .env.dev)
**Stop Environment**:
```bash
# Stop services (keep volumes)
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down
# Stop and remove volumes (fresh start)
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml down -v
```
### 2. Testing (`docker-compose.test.yaml`)
@@ -83,22 +79,24 @@ sqlserver # Test database
database.migrations # Fresh schema
database.seed # Test data
api.specs # Reqnroll BDD tests
unit.tests # All Features.*.Tests unit test projects
repository.tests # Repository unit tests
service.auth.tests # Service unit tests
```
**Run Tests**:
```bash
# Run all tests
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
# View results
ls -la test-results/
cat test-results/api-specs/results.trx
cat test-results/Features.Auth.Tests.trx
cat test-results/repository-tests/results.trx
cat test-results/service-auth-tests/results.trx
# Clean up
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
docker compose -f docker-compose.test.yaml down -v
```
### 3. Production (`docker-compose.prod.yaml`)
@@ -125,7 +123,7 @@ api.core # Production API
**Deploy Production**:
```bash
docker compose --env-file web/.env.prod -f web/docker-compose.prod.yaml up -d
docker compose -f docker-compose.prod.yaml up -d
```
## Service Dependencies
@@ -146,11 +144,7 @@ api.core / tests (start when ready)
```yaml
healthcheck:
test:
[
"CMD-SHELL",
"sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1'",
]
test: ['CMD-SHELL', "sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1'"]
interval: 10s
timeout: 5s
retries: 12
@@ -195,6 +189,13 @@ volumes:
Test results are written to host filesystem for CI/CD integration.
**Code Volumes** (development only):
```yaml
volumes:
- ./src:/app/src # Hot reload for development
```
## Networks
Each environment uses isolated bridge networks:
@@ -208,18 +209,16 @@ Each environment uses isolated bridge networks:
All containers are configured via environment variables from `.env` files:
```yaml
env_file: ".env.dev" # or .env.test, .env.prod
env_file: '.env.dev' # or .env.test, .env.prod
environment:
ASPNETCORE_ENVIRONMENT: "Development"
DOTNET_RUNNING_IN_CONTAINER: "true"
DB_SERVER: "${DB_SERVER}"
DB_NAME: "${DB_NAME}"
DB_USER: "${DB_USER}"
DB_PASSWORD: "${DB_PASSWORD}"
ACCESS_TOKEN_SECRET: "${ACCESS_TOKEN_SECRET}"
REFRESH_TOKEN_SECRET: "${REFRESH_TOKEN_SECRET}"
CONFIRMATION_TOKEN_SECRET: "${CONFIRMATION_TOKEN_SECRET}"
ASPNETCORE_ENVIRONMENT: 'Development'
DOTNET_RUNNING_IN_CONTAINER: 'true'
DB_SERVER: '${DB_SERVER}'
DB_NAME: '${DB_NAME}'
DB_USER: '${DB_USER}'
DB_PASSWORD: '${DB_PASSWORD}'
JWT_SECRET: '${JWT_SECRET}'
```
For complete list, see [Environment Variables](environment-variables.md).
@@ -230,7 +229,7 @@ For complete list, see [Environment Variables](environment-variables.md).
```bash
# Running services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml ps
docker compose -f docker-compose.dev.yaml ps
# All containers (including stopped)
docker ps -a
@@ -240,13 +239,13 @@ docker ps -a
```bash
# All services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f
docker compose -f docker-compose.dev.yaml logs -f
# Specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f api.core
docker compose -f docker-compose.dev.yaml logs -f api.core
# Last 100 lines
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs --tail=100 api.core
docker compose -f docker-compose.dev.yaml logs --tail=100 api.core
```
### Execute Commands in Container
@@ -263,39 +262,39 @@ docker exec dev-env-sqlserver /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -
```bash
# Restart all services
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart
docker compose -f docker-compose.dev.yaml restart
# Restart specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart api.core
docker compose -f docker-compose.dev.yaml restart api.core
# Rebuild and restart
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d --build api.core
docker compose -f docker-compose.dev.yaml up -d --build api.core
```
### Build Images
```bash
# Build all images
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build
docker compose -f docker-compose.dev.yaml build
# Build specific service
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build api.core
docker compose -f docker-compose.dev.yaml build api.core
# Build without cache
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build --no-cache
docker compose -f docker-compose.dev.yaml build --no-cache
```
### Clean Up
```bash
# Stop and remove containers
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down
# Remove containers and volumes
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml down -v
# Remove containers, volumes, and images
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v --rmi all
docker compose -f docker-compose.dev.yaml down -v --rmi all
# System-wide cleanup
docker system prune -af --volumes

104
docs/website/environment-variables.md → docs/environment-variables.md
View File

@@ -1,7 +1,7 @@
# Environment Variables
This document covers the active environment variables used by the current
Biergarten stack.
This document covers the active environment variables used by the current Biergarten
stack.
## Overview
@@ -17,10 +17,10 @@ The application uses environment variables for:
Direct environment variable access via `Environment.GetEnvironmentVariable()`.
### Frontend (`web/frontend`)
### Frontend (`src/Website`)
The active website reads runtime values from the server environment for its auth
and API integration.
The active website reads runtime values from the server environment for its auth and API
integration.
### Docker
@@ -54,15 +54,14 @@ Provide complete connection string:
DB_CONNECTION_STRING="Server=localhost,1433;Database=Biergarten;User Id=sa;Password=YourStrong!Passw0rd;TrustServerCertificate=True;"
```
**Priority**: `DB_CONNECTION_STRING` is checked first. If not found, connection
string is built from components.
**Priority**: `DB_CONNECTION_STRING` is checked first. If not found, connection string is
built from components.
**Implementation**: See `DefaultSqlConnectionFactory.cs`
### JWT Authentication Secrets (Backend)
The backend uses separate secrets for different token types to enable
independent key rotation and validation isolation.
The backend uses separate secrets for different token types to enable independent key rotation and validation isolation.
```bash
# Access token secret (1-hour tokens)
@@ -130,10 +129,10 @@ ASPNETCORE_URLS=http://0.0.0.0:8080 # Binding address and port
DOTNET_RUNNING_IN_CONTAINER=true # Flag for container execution
```
## Frontend Variables (`web/frontend`)
## Frontend Variables (`src/Website`)
The active website does not use the old Next.js/Prisma environment model. Its
core runtime variables are:
The active website does not use the old Next.js/Prisma environment model. Its core runtime
variables are:
```bash
API_BASE_URL=http://localhost:8080 # Base URL for the .NET API
@@ -147,7 +146,7 @@ NODE_ENV=development # Standard Node runtime mode
- **Required**: Yes for local development
- **Default in code**: `http://localhost:8080`
- **Used by**: `web/frontend/app/lib/auth.server.ts`
- **Used by**: `src/Website/app/lib/auth.server.ts`
- **Purpose**: Routes website auth actions to the .NET API
#### `SESSION_SECRET`
@@ -163,27 +162,15 @@ NODE_ENV=development # Standard Node runtime mode
- **Typical values**: `development`, `production`, `test`
- **Purpose**: Controls secure cookie behavior and runtime mode
### SMTP Configuration (Backend)
Read by `Infrastructure.Email/SmtpEmailProvider.cs` for sending confirmation
and account emails.
### Admin Account (Seeding)
```bash
SMTP_HOST=mailpit # Required, no default
SMTP_PORT=1025 # Optional, defaults to 587
SMTP_USERNAME=<username> # Optional, no default
SMTP_PASSWORD=<password> # Optional, no default
SMTP_USE_SSL=false # Optional, defaults to true
SMTP_FROM_EMAIL=noreply@thebiergarten.app # Required, no default
SMTP_FROM_NAME=The Biergarten App # Optional, defaults to "The Biergarten"
ADMIN_PASSWORD=SecureAdminPassword123! # Initial admin password for seeding
```
- **Implementation**: `Infrastructure.Email/SmtpEmailProvider.cs` throws on
startup if `SMTP_HOST` or `SMTP_FROM_EMAIL` is missing
- **Local dev**: point at the `mailpit` Docker service (SMTP on port 1025, web
UI on http://localhost:8025)
- **Production**: point at a real provider (SendGrid, Mailgun, Amazon SES,
etc.)
- **Required**: No (only needed for seeding)
- **Purpose**: Sets admin account password during database seeding
- **Security**: Use strong password, change immediately in production
## Docker-Specific Variables
@@ -203,33 +190,33 @@ MSSQL_PID=Express # SQL Server edition (Express, Developer, Ente
## Environment File Structure
### Backend/Docker (`web/` Directory)
### Backend/Docker (Root Directory)
```
web/.env.example # Template (tracked in Git)
web/.env.dev # Development config (gitignored)
web/.env.test # Testing config (gitignored)
web/.env.prod # Production config (gitignored)
.env.example # Template (tracked in Git)
.env.dev # Development config (gitignored)
.env.test # Testing config (gitignored)
.env.prod # Production config (gitignored)
```
**Setup**:
```bash
cp web/.env.example web/.env.dev
# Edit web/.env.dev with your values
cp .env.example .env.dev
# Edit .env.dev with your values
```
## Legacy Frontend Variables
Variables for the archived Next.js frontend (`archive/next-js-web-app/`) have
been removed from this active reference, since that app is retained for
reference only and is not run as part of the active stack.
Variables for the archived Next.js frontend (`src/Website-v1`) have been removed from this
active reference. See [archive/legacy-website-v1.md](archive/legacy-website-v1.md) if you
need the legacy Prisma, Cloudinary, Mapbox, or SparkPost notes.
**Docker Compose Mapping**:
- `web/docker-compose.dev.yaml` → `web/.env.dev`
- `web/docker-compose.test.yaml` → `web/.env.test`
- `web/docker-compose.prod.yaml` → `web/.env.prod`
- `docker-compose.dev.yaml` → `.env.dev`
- `docker-compose.test.yaml` → `.env.test`
- `docker-compose.prod.yaml` → `.env.prod`
## Variable Reference Table
@@ -245,13 +232,6 @@ reference only and is not run as part of the active stack.
| `REFRESH_TOKEN_SECRET` | ✓ | | ✓ | Yes | Refresh token signing |
| `CONFIRMATION_TOKEN_SECRET` | ✓ | | ✓ | Yes | Confirmation token signing |
| `WEBSITE_BASE_URL` | ✓ | | | Yes | Website URL for emails |
| `SMTP_HOST` | ✓ | | ✓ | Yes | SMTP server host |
| `SMTP_PORT` | ✓ | | ✓ | No | Defaults to `587` |
| `SMTP_USERNAME` | ✓ | | ✓ | No | SMTP auth username |
| `SMTP_PASSWORD` | ✓ | | ✓ | No | SMTP auth password |
| `SMTP_USE_SSL` | ✓ | | ✓ | No | Defaults to `true` |
| `SMTP_FROM_EMAIL` | ✓ | | ✓ | Yes | Email sender address |
| `SMTP_FROM_NAME` | ✓ | | ✓ | No | Defaults to `The Biergarten` |
| `API_BASE_URL` | | ✓ | | Yes | Website-to-API base URL |
| `SESSION_SECRET` | | ✓ | | Yes | Website session signing |
| `NODE_ENV` | | ✓ | | No | Runtime mode |
@@ -263,8 +243,8 @@ reference only and is not run as part of the active stack.
| `MSSQL_PID` | | | ✓ | No | SQL Server edition |
| `DOTNET_RUNNING_IN_CONTAINER` | ✓ | | ✓ | No | Container flag |
\* Either `DB_CONNECTION_STRING` OR the component variables (`DB_SERVER`,
`DB_NAME`, `DB_USER`, `DB_PASSWORD`) must be provided.
\* Either `DB_CONNECTION_STRING` OR the component variables (`DB_SERVER`, `DB_NAME`,
`DB_USER`, `DB_PASSWORD`) must be provided.
## Validation
@@ -272,17 +252,14 @@ reference only and is not run as part of the active stack.
Variables are validated at startup:
- Missing required variables (`ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`,
`CONFIRMATION_TOKEN_SECRET`, `SMTP_HOST`, `SMTP_FROM_EMAIL`, DB connection
values) cause the application to fail with an `InvalidOperationException`
- No minimum length is enforced on the token secrets in code; the
"minimum 32 characters" guidance above is a recommendation, not an
enforced check
- Missing required variables cause application to fail
- JWT_SECRET length is enforced (min 32 chars)
- Connection string format is validated
### Frontend Validation
The active website relies on runtime defaults for local development and the
surrounding server environment in deployed environments.
The active website relies on runtime defaults for local development and the surrounding
server environment in deployed environments.
- `API_BASE_URL` defaults to `http://localhost:8080`
- `SESSION_SECRET` falls back to a development-only local secret
@@ -305,13 +282,6 @@ REFRESH_TOKEN_SECRET=<generated-with-openssl>
CONFIRMATION_TOKEN_SECRET=<generated-with-openssl>
WEBSITE_BASE_URL=http://localhost:3000
# SMTP (Mailpit in dev)
SMTP_HOST=mailpit
SMTP_PORT=1025
SMTP_USE_SSL=false
SMTP_FROM_EMAIL=noreply@thebiergarten.app
SMTP_FROM_NAME=The Biergarten App
# Migration
CLEAR_DATABASE=true

31
docs/website/getting-started.md → docs/getting-started.md
View File

@@ -1,7 +1,7 @@
# Getting Started
This guide covers local setup for the current Biergarten stack: the .NET backend
in `web/backend` and the active React Router frontend in `web/frontend`.
This guide covers local setup for the current Biergarten stack: the .NET backend in
`src/Core` and the active React Router frontend in `src/Website`.
## Prerequisites
@@ -22,7 +22,7 @@ cd the-biergarten-app
### 2. Configure Backend Environment Variables
```bash
cp web/.env.example web/.env.dev
cp .env.example .env.dev
```
At minimum, ensure `.env.dev` includes valid database and token values:
@@ -43,21 +43,20 @@ See [Environment Variables](environment-variables.md) for the full list.
### 3. Start the Backend Stack
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d
docker compose -f docker-compose.dev.yaml up -d
```
This starts SQL Server, migrations, seeding, the API, and Mailpit (local dev SMTP).
This starts SQL Server, migrations, seeding, and the API.
Available endpoints:
- API Swagger: http://localhost:8080/swagger
- Health Check: http://localhost:8080/health
- Mailpit UI: http://localhost:8025
### 4. Start the Active Frontend
```bash
cd web/frontend
cd src/Website
npm install
API_BASE_URL=http://localhost:8080 SESSION_SECRET=dev-secret-change-me npm run dev
```
@@ -72,7 +71,7 @@ Required frontend runtime variables for local work:
### 5. Optional: Run Storybook
```bash
cd web/frontend
cd src/Website
npm run storybook
```
@@ -83,15 +82,15 @@ Storybook runs at http://localhost:6006 by default.
### Backend
```bash
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
docker compose -f docker-compose.dev.yaml logs -f
docker compose -f docker-compose.dev.yaml down
docker compose -f docker-compose.dev.yaml down -v
```
### Frontend
```bash
cd web/frontend
cd src/Website
npm run lint
npm run typecheck
npm run format:check
@@ -116,7 +115,7 @@ export WEBSITE_BASE_URL="http://localhost:3000"
### 2. Run Migrations and Seed
```bash
cd web/backend
cd src/Core
dotnet run --project Database/Database.Migrations/Database.Migrations.csproj
dotnet run --project Database/Database.Seed/Database.Seed.csproj
```
@@ -129,11 +128,11 @@ dotnet run --project API/API.Core/API.Core.csproj
## Legacy Frontend Note
The previous Next.js frontend lives in `archive/next-js-web-app/` for reference
only and is not the active website.
The previous Next.js frontend now lives in `src/Website-v1` and is not the active website.
Legacy setup details have been moved to [docs/archive/legacy-website-v1.md](archive/legacy-website-v1.md).
## Next Steps
- Review [Architecture](../architecture.md)
- Review [Architecture](architecture.md)
- Run backend and frontend checks from [Testing](testing.md)
- Use [Docker Guide](docker.md) for container troubleshooting

496
docs/pipeline/README.md
View File

@@ -1,496 +0,0 @@
# Biergarten Pipeline
A C++20 command-line pipeline that samples city records from local JSON,
enriches each with Wikipedia context, and generates bilingual brewery names
and descriptions plus locale-grounded user profiles via a local GGUF model or
a deterministic mock.
> **This pipeline produces AI-generated data.** It is not a source of truth for
> brewing techniques, cultural representation, or local-language accuracy. See
> [ETHICS-AND-KNOWN-ISSUES.md](./ETHICS-AND-KNOWN-ISSUES.md) for a full
> documentation of limitations, hallucination patterns, and bias.
---
## Table of Contents
- [How It Fits The Main App](#how-it-fits-the-main-app)
- [Quick Start](#quick-start)
- [Build](#build)
- [Model](#model)
- [Run](#run)
- [Docker / RunPod](#docker--runpod)
- [Architecture](#architecture)
- [Pipeline Stages](#pipeline-stages)
- [Key Components](#key-components)
- [Runtime Behaviour](#runtime-behaviour)
- [Generated Output](#generated-output)
- [Tech Stack](#tech-stack)
- [Tested Hardware](#tested-hardware)
- [Fixture Strategy](#fixture-strategy)
- [Repo Layout](#repo-layout)
- [Code Tour](#code-tour)
- [Next Steps](#next-steps)
---
## How It Fits The Main App
The pipeline is a data ingestion layer. It sits outside the web app runtime and
produces seed records the app imports at startup or during a dedicated seed
step.
| Planned app area | Pipeline contribution |
| -------------------------------- | ------------------------------------------------------------------ |
| Brewery discovery and management | Sampled city records, localized names, long-form descriptions |
| Beer reviews and ratings | Stable brewery fixtures with enough context to anchor review pages |
| Social follow relationships | Repeatable brewery entities for feeds, follows, and saved lists |
| Geospatial brewery experiences | Latitude, longitude, and country-level metadata |
| User accounts and profiles | Locale-grounded names, bios, and an auth-ready email/date-of-birth pair for seeding real accounts |
---
## Quick Start
### Build
Requirements: C++20 compiler, CMake 3.31+, OpenSSL, Boost (JSON and
ProgramOptions). SQLite is fetched from the upstream amalgamation, so no system
SQLite package is required.
```bash
cmake -S . -B build
cmake --build build
```
CMake automatically detects whether a compatible llama.cpp installation is
present on the system (`libllama`, `libggml`, `libggml-base`, and `llama.h`
visible on the default search paths). If found, it links against those
libraries and skips the FetchContent build. If not found, it fetches and builds
llama.cpp from source at tag `b9012`. No additional flags are required in
either case.
Metal is enabled automatically on Apple Silicon. CUDA or HIP/ROCm is detected
automatically on Linux when the relevant toolkit is present.
### Model
> Skip this step if you only need `--mocked`.
```bash
mkdir -p models
curl -L \
-o models/google_gemma-4-E4B-it-Q6_K.gguf \
https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF/resolve/main/google_gemma-4-E4B-it-Q6_K.gguf?download=true
```
### Run
Run from `build/` so the copied `locations.json` and `prompts/` are available.
Each run writes a fresh dated SQLite file such as
`biergarten_seed_2026-04-19T15-30-45.123456Z.sqlite` into the working directory.
```bash
./biergarten-pipeline --mocked
./biergarten-pipeline \
--model ../models/google_gemma-4-E4B-it-Q6_K.gguf \
--prompt-dir prompts \
--location-count 25 \
--temperature 1.0 --top-p 0.95 --top-k 64 --n-ctx 8192 --seed -1
```
#### CLI Flags
| Flag | Purpose |
| ------------------ | ---------------------------------------------------------------------------------------------------- |
| `--mocked` | Deterministic mock generator, no model required. |
| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. |
| `--prompt-dir` | Directory containing prompt files (e.g. `BREWERY_GENERATION.md`). Required unless `--mocked` is set. |
| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. |
| `--log-path` | Path for application logs. Default: `pipeline.log`. |
| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. |
| `--temperature` | Sampling temperature. Default: `1.0`. |
| `--top-p` | Nucleus sampling. Default: `0.95`. |
| `--top-k` | Top-k sampling. Default: `64`. |
| `--n-ctx` | Context window size. Default: `8192`. |
| `--seed` | Random seed. Default: `-1` (random at runtime). |
| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. |
| `--help, -h` | Print usage and exit. |
`--mocked` and `--model` are mutually exclusive. Omitting both exits with an
error before the pipeline starts. Sampling flags are ignored when `--mocked` is
set.
The post-build step copies `prompts/` into `build/prompts/`. Rebuild after
editing any prompt file.
---
## Docker / RunPod
The `tooling/pipeline/runpod/` directory contains a GPU-ready container
configuration for running the pipeline on RunPod or any Docker host with an
NVIDIA GPU.
### How it works
The container uses a two-stage build. The builder stage installs CMake/Ninja,
clones the matching llama.cpp release tag for its headers only (installed into
`/usr/local/include`), and copies prebuilt shared libraries (`libllama`,
`libggml`, and CUDA/CPU backend plugins) from `ghcr.io/ggml-org/llama.cpp:full-cuda`
into `/usr/local/lib`. With both headers and libraries present, CMake's
system-library detection (see [Build](#build) above) finds them and skips the
FetchContent source build, keeping image build times short.
The runtime stage copies the compiled binary, the same prebuilt shared
libraries, and config/prompt assets into a slim CUDA runtime image. It sets
`LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH` so the dynamic linker
resolves `libllama`/`libggml` at startup, and also co-locates
`libggml-cuda.so` and the CPU backend plugins next to the binary for
`ggml_backend_load_all()`'s `dlopen` scan.
### Build the image
Run from the `tooling/pipeline/` directory (the CMake project root), not from
inside `runpod/`, so the `COPY . .` step picks up the full project context.
```bash
docker build -t biergarten-pipeline:latest -f runpod/Dockerfile .
```
To monitor the full build output and confirm CMake selects the system llama.cpp:
```bash
docker build \
--progress=plain \
--no-cache \
-t biergarten-pipeline:latest \
-f runpod/Dockerfile \
. 2>&1 | tee build.log
```
Look for `[biergarten] Found system llama.cpp — skipping FetchContent` in the
output to confirm the fast path was taken.
### Run the container
The container always runs the model-backed path; there is no `--mocked`
container mode (use a native build for that — see [Quick Start](#quick-start)).
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if
it is not already present at the configured path.
```bash
docker run --rm \
--runtime=nvidia \
-v "$PWD/models:/workspace/models" \
-v "$PWD/output:/workspace/output" \
-v "$PWD/logs:/workspace/logs" \
biergarten-pipeline:latest
```
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to
`./models/` on first run if it isn't already there. To use a pre-downloaded
model, place it at that path first — see [Model](#model) above.
#### Environment variables
| Variable | Purpose |
| ------------------------- | ----------------------------------------------------------------- |
| `BIERGARTEN_MODEL_PATH` | GGUF model path. Default: `/workspace/models/google_gemma-4-E4B-it-Q6_K.gguf`. |
| `BIERGARTEN_OUTPUT_DIR` | SQLite output directory. Default: `/workspace/output`. |
| `BIERGARTEN_LOG_PATH` | Log file path. Default: `/workspace/logs/pipeline.log`. |
| `BIERGARTEN_GL_LAYERS` | GPU layers to offload (`--n-gpu-layers`). Default: `40`. |
| `BIERGARTEN_TEMPERATURE`, `BIERGARTEN_TOP_P`, `BIERGARTEN_TOP_K`, `BIERGARTEN_N_CTX`, `BIERGARTEN_SEED` | Optional sampling overrides, unset by default (binary defaults apply). |
| `BIERGARTEN_EXTRA_ARGS` | Additional raw CLI args appended verbatim. |
`--prompt-dir` is hardcoded to `/app/prompts` inside the container and is not
configurable via environment variable.
### RunPod deployment
Use a GPU pod template. Mount persistent storage for `/workspace/models`,
`/workspace/output`, and `/workspace/logs`. See
`tooling/pipeline/runpod/pod-template.yaml` for a starter template — set the
environment variables listed above to match your run.
---
## Architecture
### Pipeline Stages
| Stage | Implementation |
| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Load | `ICuratedDataService` (`CuratedJsonDataService`) reads `locations.json`, `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json` (paths supplied via a `CuratedDataFilePaths` DTO at construction) into typed records, caching each after its first load. `--mocked` runs use `MockCuratedDataService`'s fixed in-memory dataset instead. |
| Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). |
| Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. |
| Generate Users | `GenerateUsers()` samples a persona and a forename/surname pair per enriched city (skipping countries with no name data), then `MockGenerator` or `LlamaGenerator` produces a username, bio, and activity weight around the sampled name. |
| Generate Breweries | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. |
| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `locations`, `users`, and `breweries` tables. |
| Log | `spdlog` writes results and warnings to the console. |
If name sampling, enrichment, or generation fails for a city, that city is
skipped and the pipeline continues. `GenerateUsers()` runs before
`GenerateBreweries()` in `BiergartenPipelineOrchestrator::Run()`.
### Key Components
- `src/main.cc` — argument parsing and Boost.DI composition root.
- `CuratedJsonDataService` — implements `ICuratedDataService`; takes a
`CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in
its constructor, then parses and validates curated location, persona, and
forename/surname JSON, memoizing each result after its first load on a
given instance. `MockCuratedDataService` is the in-memory substitute (4
fixed locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in
`--mocked` runs.
- `WikipediaEnrichmentService` — queries Wikipedia extracts, caches results,
returns empty context on failure. `MockEnrichmentService` is the no-op
substitute used in `--mocked` runs.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for
both `GenerateBrewery` and `GenerateUser`, retries malformed responses up
to three times with corrective feedback in the retry prompt. The token
budget is fixed across attempts; it is not raised automatically on
truncation.
- `MockGenerator` — stable hash-based output so the same city/persona/name
input always produces the same brewery or user.
- `SqliteExportService` — creates a dated SQLite file per run and persists
each successful user and brewery into normalized tables.
- Brewery payloads include English and local-language name and description
fields. User payloads carry a sampled first/last name and gender, an
LLM-generated username/bio/activity weight, and a programmatically
generated (not LLM-authored) unique email and date of birth.
### Runtime Behaviour
`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a
generic "brewing" extract and a "beer in `{country}`" extract. It does not
currently query a city- or region-specific page. Each query string is cached
after its first successful (or empty) lookup.
`GetLocationContext()` returns an empty string when the web client is
unavailable or when lookup/parsing fails.
`LlamaGenerator` validates model output as structured JSON. On validation
failure it retries up to three times, replaying the previous error message in
the next prompt so the model can self-correct. All runs to date have produced
valid output on the first pass; the retry path is kept for resilience.
`MockGenerator` uses stable hashes for repeatable output in demos and Storybook
runs.
`CuratedJsonDataService` memoizes each of `LoadLocations()`, `LoadPersonas()`,
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the
first time each is called, since `BiergartenPipelineOrchestrator` owns a
single `ICuratedDataService` instance for the whole run — later calls return
the cached result instead of re-parsing.
`GenerateUsers()` samples a forename/surname pair per city via `SampleName()`,
keyed by the city's ISO 3166-1 code. Countries present in `locations.json`
but absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`,
`ZA`) are skipped, the same way a failed enrichment or generation call skips
a city — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
### Process Flow - Activity Diagram
![An activity diagram](./diagrams/current/output/activity.svg)
### Architectural Overview - Class Diagram
![A class diagram](./diagrams/current/output/class.svg)
---
## Generated Output
Each successful run stores a `BreweryRecord` pair with the source location
and a `BreweryResult` payload, and a `UserRecord` pair with the source
location and a `UserResult` payload. The same generated records are also
written to a fresh SQLite export file named with the current UTC timestamp.
| Field | Meaning |
| ------------------- | ------------------------------------------ |
| `name_en` | Brewery name in English. |
| `description_en` | Brewery description in English. |
| `name_local` | Brewery name in the local language. |
| `description_local` | Brewery description in the local language. |
| Field | Meaning |
| ----------------- | ----------------------------------------------------------------- |
| `first_name` | Sampled forename, copied from the curated name data (not LLM-invented). |
| `last_name` | Sampled surname, copied from the curated name data (not LLM-invented). |
| `gender` | Gender associated with the sampled forename in the source dataset. |
| `username` | LLM-generated handle. |
| `bio` | LLM-generated short biography. |
| `activity_weight` | Relative check-in/activity weight, reserved for a future J-curve activity profile. |
| `email` | Unique `@thebiergarten.app` address, generated programmatically from the sampled name. |
| `date_of_birth` | Randomized date of birth (age 19-48), generated programmatically. |
The log dump also includes city, country, state or province, ISO subdivision
code, latitude, and longitude for each entry.
### Consumer Data Shape
| Field | Why it matters |
| ----------------------------------- | ------------------------------------------------ |
| `city`, `state_province`, `country` | Human-readable location labels and page headings |
| `iso3166_1`, `iso3166_2` | Filtering, regional grouping, locale matching |
| `latitude`, `longitude` | Map pins and nearby brewery views |
| `local_languages` | Locale-aware copy selection |
| `name_en`, `description_en` | Default English display content |
| `name_local`, `description_local` | Local-language display content |
---
## Tech Stack
- C++20
- CMake 3.31+
- Boost.JSON, Boost.ProgramOptions, Boost.DI
- spdlog
- cpp-httplib (with OpenSSL)
- SQLite amalgamation fetched and compiled via CMake FetchContent
- llama.cpp (auto-detected from system install or fetched via FetchContent)
- Docker with NVIDIA CUDA 12.6 base image for GPU container builds
- RunPod for cloud GPU inference
The build fetches Boost.DI, spdlog, and SQLite via CMake. llama.cpp is fetched
only when a system installation is not detected. Metal is enabled on Apple
Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
> **Code Style:** Modern C++20 throughout — RAII for ownership,
> `std::unique_ptr` for injected dependencies, `std::optional` for parse
> outcomes, `std::span` for read-only views over generated city data, structured
> bindings in pipeline loops. Formatting follows the Google C++ Style Guide via
> `.clang-format` with a narrow column limit and two-space indentation.
---
## Tested Hardware
### ARM macOS — M1 Pro
| | |
| --------- | --------------------------------- |
| Host | MacBook Pro 14" (2021) |
| CPU | Apple M1 Pro (8-core) |
| GPU | Apple M1 Pro (14-core integrated) |
| Memory | 16 GB |
| Model | Gemma 4 E4B |
| Inference | llama.cpp with Metal |
### x86_64 Linux — NVIDIA RTX 2000
| | |
| --------- | ------------------------------ |
| Host | ThinkPad P1 Gen 7 (Fedora 43) |
| CPU | Intel Core Ultra 7 155H |
| GPU | NVIDIA RTX 2000 Ada Generation |
| Memory | 32 GB |
| Model | Gemma 4 E4B |
| Inference | llama.cpp with CUDA 12.x |
### x86_64 Linux — Docker / RunPod (NVIDIA CUDA)
| | |
| --------- | ------------------------------------------- |
| Host | RunPod GPU pod |
| Base | nvidia/cuda:12.6.3-devel-ubuntu24.04 |
| Model | Gemma 4 E4B Q6_K |
| Inference | llama.cpp prebuilt CUDA backends via dlopen |
---
## Fixture Strategy
- `--mocked` for stable fixtures, repeatable screenshots, and Storybook runs.
`MockCuratedDataService` swaps in for `CuratedJsonDataService`, so no
fixture files need to be present on disk.
- `--model` when geographically grounded content matters for demos.
- Keep `locations.json` structured enough to support discovery and future
filtering.
- `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` are curated/vendored fixture data, not
LLM-generated — see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset
section for provenance.
- Treat SQLite output as seed material for the app's brewery and user
domains, not production data.
---
## Repo Layout
| Path | Purpose |
| -------------------------------------- | -------------------------------------------------- |
| `tooling/pipeline/includes/` | Public headers and shared models. |
| `tooling/pipeline/src/` | Implementation files. |
| `tooling/pipeline/locations.json` | Curated city input copied into the build tree. |
| `tooling/pipeline/personas.json` | Curated user persona archetypes copied into the build tree. |
| `tooling/pipeline/forenames-by-country.json` | Vendored (CC0) forename data by ISO 3166-1 country code. |
| `tooling/pipeline/surnames-by-country.json` | Vendored (CC0) surname data by ISO 3166-1 country code. |
| `tooling/pipeline/prompts/` | System prompts used by the model-backed path. |
| `tooling/pipeline/runpod/` | Dockerfile, launcher, and RunPod pod template. |
| `docs/pipeline/diagrams/` | Architecture and pipeline diagrams. |
| `docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md` | Ethics, bias, hallucination analysis, mitigations. |
---
## Code Tour
Paths below are relative to `tooling/pipeline/`.
- `src/main.cc` — argument parsing and DI composition root.
- `src/biergarten_pipeline_orchestrator/` — orchestration, sampling, logging,
and export.
- `src/services/curated_data/` — `CuratedJsonDataService`, the file-backed
`ICuratedDataService`, and `MockCuratedDataService`, the in-memory
`ICuratedDataService` used in `--mocked` runs.
- `src/services/enrichment/wikipedia/` — enrichment service and cache.
- `src/services/sqlite/` — SQLite export implementation.
- `src/data_generation/llama/` — local inference, prompt loading, output
validation.
- `src/data_generation/mock/` — deterministic fallback.
- `runpod/` — container build and runtime launcher.
---
## Next Steps
The pipeline currently produces city-aware brewery and user records and
dated SQLite exports. The next passes add additional fixture types so the
app can exercise the full brewery and social domains without live data. For
the detailed engineering breakdown of what's needed to reach the
architecture in [`diagrams/planned/`](./diagrams/planned/), see
[ROADMAP.md](./ROADMAP.md).
### Testing — Very High Priority
- Unit test JSON validation and retry logic against malformed, truncated, and
empty model outputs.
- Integration test the enrichment pipeline with missing context, short context,
and fake context inputs.
- Adversarial context tests: feed plausible but geographically incorrect
Wikipedia extracts and verify the model does not silently blend them with
training data.
- Verify bilingual enrichment behaviour when only an English extract is
available versus when both extracts are present.
- Confirm the retry path is reachable when the reasoning block consumes
available token budget.
### Beer Generation
Generate catalog entries with style, ABV, IBU, color, aroma notes, and food
pairing hints. Link beers back to breweries and cities. Keep style coverage wide
enough to exercise search, sort, and category filters.
### Check-In System
Produce timestamped check-in events between users and breweries. Use a J-curve
activity profile — a small set of users accounts for most check-ins, the rest
appear occasionally. Add bursty behaviour around weekends and travel periods.
### Beer Ratings
Generate rating events with a strong positive skew and a long tail of lower
scores. Avoid uniform distributions. Attach timestamps and user IDs so the app
can compute averages, trends, and per-style comparisons.

252
docs/pipeline/ROADMAP.md
View File

@@ -1,252 +0,0 @@
# Pipeline Roadmap — Reaching the Planned Architecture
This is the engineering breakdown for closing the gap between the **current**
pipeline and the **planned** architecture documented in
[`diagrams/planned/class.puml`](./diagrams/planned/class.puml) and
[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing
in `diagrams/planned/` is implemented yet — this file tracks what it would
take to get there. For the current implementation, see
[README.md](./README.md).
Items are grouped by layer and roughly ordered by dependency: later groups
build on types and services introduced earlier.
The only concurrency in the current pipeline is the log dispatcher thread
(`main.cc` spawns it to drain a `BoundedChannel<LogEntry>` into spdlog for the
whole run). Sampling, enrichment, generation, and export all happen
synchronously on the main thread inside `BiergartenPipelineOrchestrator::Run()`
— §6 below is what introduces the additional worker threads the planned
architecture needs.
---
## 1. Domain Models
`tooling/pipeline/includes/data_model/generated_models.h` and `models.h`.
- [ ] Add `Completeness` enum (`Full`, `Partial`, `Absent`) and a
`LocationContext` struct (`text`, `completeness`, `char_count`).
Replace `EnrichedCity::region_context` (currently a plain
`std::string`) with `context : LocationContext`.
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`,
`min_ibu`, `max_ibu`).
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads.
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
`context_provided`, `generated_with`).
- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to
`UserResult` (currently just `username`, `bio`). `first_name`/
`last_name`/`gender` are copied from the sampled `Name` (see below),
not LLM-invented.
- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result
handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`,
`gender`). Landed as a flatter shape than originally planned here: no
`NamesByCountry` wrapper class. `curated_data_service.h` instead
declares `ForenameList = std::vector<ForenameEntry>` and
`SurnameList = std::vector<std::string>`, and
`ICuratedDataService` exposes two maps keyed by ISO 3166-1 code —
`ForenamesByCountryMap = unordered_map<string, ForenameList>` and
`SurnamesByCountryMap = unordered_map<string, SurnameList>` —
directly (see §2). Sampling is
a free function, `SampleName(forenames_by_country,
surnames_by_country, iso3166_1, rng)`, in
`generate_users.cc`'s anonymous namespace, not a method on a class.
Pairing a forename with a surname happens at sample time so gender
(present per-forename in the source data) is never discarded the way a
pre-flattened `{first_name, last_name}` list would lose it; see §2.
- [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`,
`metadata` (currently just `{location, brewery}`).
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
`GeneratedFollow` aggregate structs.
- [x] Add `UserRecord` (`location`, `user : UserResult`, `email`,
`date_of_birth`) as the current stand-in for the planned
`GeneratedUser` — `email` and `date_of_birth` are programmatically
generated by the orchestrator, never LLM-authored, so a downstream
auth-account seeding consumer can register real accounts from the
pipeline's SQLite export. No `password`, `user_id`, or `metadata`
field yet, matching `BreweryRecord`'s equally pre-`metadata` shape.
Already wired into `IExportService`/`SqliteExportService`: a `users`
table exists and `GenerateUsers()` exports each successful record via
`ProcessRecord(const UserRecord&)` (see §5) in addition to logging and
holding `generated_users_` in memory. Still missing vs. the planned
`GeneratedUser`: `user_id`, `password`, and `metadata`.
- [x] Add `UserPersona` (`name`, `description`, `style_affinities`).
## 2. Data Preloading
`tooling/pipeline/includes/services/curated_data/`, fixture files.
- [x] Extract an interface; have `CuratedJsonDataService` implement it.
Landed as `ICuratedDataService`
(`services/curated_data/curated_data_service.h`), not `DataPreloader`
— `LoadLocations()`, `LoadPersonas()`, `LoadForenamesByCountry()`, and
`LoadSurnamesByCountry()` are all virtual methods returning `const&`
and take no arguments. `CuratedJsonDataService`
(`services/curated_data/curated_json_data_service.h`, originally
landed as `JsonLoader` under `json_handling/` before being renamed and
moved) takes a `CuratedDataFilePaths` DTO (`locations_path`,
`personas_path`, `forenames_path`, `surnames_path`) in its
constructor rather than a path per `Load*()` call, and memoizes each
result in a private `cache` struct (`locations`, `personas`,
`forenames_by_country`, `surnames_by_country`), so a second call on
the same instance returns the cached result instead of re-parsing —
safe because `CuratedJsonDataService` outlives every call site (owned
by `BiergartenPipelineOrchestrator` via
`unique_ptr<ICuratedDataService>` for the whole run). `main.cc`'s DI
injector also gained a `MockCuratedDataService` (fixed in-memory
data: 4 locations across `US`/`DE`/`FR`/`BE`, 3 personas, and
matching forename/surname sets for those four countries), bound in
place of `CuratedJsonDataService` under `--mocked`, mirroring the
existing `MockEnrichmentService`/`MockGenerator` pattern.
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo
and is already copied into the Docker image
(`runpod/Dockerfile`), but no loader reads it yet, and the native
CMake build doesn't copy it into `build/` at all — `CMakeLists.txt`'s
"Runtime Assets" step only copies `locations.json` and `prompts/`.
- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
- [x] Add name-by-country loading, parsing
`tooling/pipeline/forenames-by-country.json` and
`surnames-by-country.json`. Landed as two separate methods —
`LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each
returning a flat `ForenamesByCountryMap` / `SurnamesByCountryMap`
(aliases for `unordered_map<string, ForenameList>` /
`unordered_map<string, SurnameList>`) keyed by ISO 3166-1 code, rather
than one combined `LoadNamesByCountry() : NamesByCountry` call. Both
files are vendored verbatim (unmodified,
full multinational coverage) from
`sigpwned/popular-names-by-country-dataset` (CC0) — see
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for
provenance. Deliberately not pre-paired or filtered down to just the
countries in `locations.json`: keeping the source shape (including
per-forename gender) intact means the loader can support more
countries later, or gender-aware persona/bio generation, without
re-sourcing data.
## 3. Policy / Strategy Layer
Entirely new — no `includes/policy/` (or equivalent) directory exists today.
- [ ] `ContextStrategy` interface + `BreweryContextStrategy` /
`BeerContextStrategy`. Today `WikipediaEnrichmentService::GetLocationContext`
hardcodes a generic `"brewing"` query and a `"beer in {country}"`
query directly — no per-phase strategy selection.
- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing
the inline `std::ranges::sample(...)` call in
`BiergartenPipelineOrchestrator::QueryCitiesWithCountries()`.
- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to
pick styles per brewery from the `BeerStyle` palette (depends on §1
and §2).
- [ ] `CheckinDistributionStrategy` interface + `JCurveCheckinStrategy` /
`RandomCheckinStrategy` — this is the "Check-In System" item already
called out in README.md's Next Steps, made concrete.
- [ ] `FollowGenerationStrategy` interface + `RandomFollowStrategy` /
`ActivityWeightedFollowStrategy`.
## 4. Data Generation
`tooling/pipeline/includes/data_generation/`, `src/data_generation/`.
- [ ] Extend the `DataGenerator` interface with `GenerateBeer`,
`GenerateCheckin`, `GenerateRating` (today: only `GenerateBrewery` and
`GenerateUser`).
- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop
mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3
attempts with corrective feedback) and a new
`prompts/USER_GENERATION.md`. Changed the `DataGenerator::GenerateUser`
signature to
`(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`,
matching what the activity diagram actually passes. `MockGenerator::GenerateUser`
updated to match the new signature too.
- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` /
`GenerateRating`.
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
`LlamaGenerator::ValidateModelArchitecture()` so loading a GGUF that
doesn't match the configured chat template fails fast instead of
silently producing degraded output.
- [ ] Add prompt template files for beer/checkin/rating generation next to
the existing `prompts/BREWERY_GENERATION.md`.
## 5. Export Service
`tooling/pipeline/includes/services/database/`, `src/services/sqlite/`.
- [x] `IExportService::ProcessRecord(const UserRecord&)` and
`SqliteExportService`'s `users` table (see
`kCreateUsersTableSql` / `kInsertUserSql` in
`includes/services/database/sqlite_statement_helpers.h`) are
implemented — `GenerateUsers()` exports every successful user
alongside its resolved `location_id`.
- [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`,
`ProcessRating`, `ProcessFollow` (today `ProcessRecord(const
BreweryRecord&)` and `ProcessRecord(const UserRecord&)` exist).
- [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to
`SqliteExportService::InitializeSchema()` (`locations`, `breweries`,
and `users` already exist) — see `kCreateLocationsTableSql` /
`kCreateBreweriesTableSql` / `kCreateUsersTableSql` in
`includes/services/database/sqlite_statement_helpers.h` for the
existing pattern to follow.
- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and
move from one open transaction for the whole run to per-phase batched
commits (`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as
shown in the planned activity diagram.
## 6. Concurrency & Orchestration
`tooling/pipeline/includes/concurrency/`,
`src/biergarten_pipeline_orchestrator/`.
- [ ] `BoundedChannel<T>` already exists and is production-tested — but
it's only wired to the single log channel today. Stand up the
per-phase producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the
planned activity diagram describes, each with a dedicated LLM worker
thread and SQLite worker thread.
- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous
pass over `generated_breweries_` into phased methods —
`RunUserPhase` → `RunBreweryAndBeerPhase` (with a `RunBeerPhase`
sub-step once `brewery_pool_` is populated) → `RunCheckinPhase` /
`RunFollowPhase` (these two can run in parallel) → `RunRatingPhase` —
backed by `user_pool_`, `brewery_pool_`, `beer_pool_`,
`checkin_pool_`, `follow_pool_`.
- [ ] Honor the structural-concurrency requirement already called out in a
comment on `BiergartenPipelineOrchestrator::Run()` in
`biergarten_pipeline_orchestrator.h`: once real worker threads exist,
they must be structurally joined (e.g. via `std::jthread`) before
`Run()` returns, so no worker logs to a closed channel during
teardown.
## 7. Enrichment
- [ ] Decide whether to restore the city/region-specific Wikipedia query
that's currently commented out in
`src/services/enrichment/wikipedia/get_summary.cc`
(`GetLocationContext`). The `ContextStrategy` work in §3 is a natural
place to reintroduce it via `BreweryContextStrategy::QueriesFor()`.
- [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`,
`PreWarmLocationCache` in the planned activity diagram) instead of
fetching lazily per record, so the streaming phases never block on a
cold cache mid-run.
## 8. Fixtures / Build
- [x] Author `personas.json`, `forenames-by-country.json`, and
`surnames-by-country.json` (see §2).
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime
Assets" `configure_file` step in `CMakeLists.txt` so native builds and
the Docker image agree on what's available at runtime.
---
## Suggested build order
The planned activity diagram's own phase notes ("brewery*pool* is now fully
populated. Phase 1b may begin.", etc.) imply a dependency order. Roughly:
1. Domain models (§1) and data preloading (§2) — nothing else compiles
without these.
2. Export schema (§5) — so every later generation phase has somewhere to
land its output.
3. Policy/strategy layer (§3) and generator interface (§4).
4. Concurrency/orchestration rewrite (§6), which is the only piece that
actually wires §1–§5 into the phased, parallel pipeline shown in the
planned diagrams.
5. Enrichment cache pre-warming and the city/region query decision (§7) —
useful at any point, but most valuable once phases run concurrently.

34
docs/pipeline/diagrams/biergarten-weizen-theme.puml
View File

@@ -1,34 +0,0 @@
skinparam shadowing false
skinparam backgroundColor #FCFCF7
skinparam defaultFontName "DM Sans"
skinparam defaultFontColor #14180C
skinparam titleFontName "Volkhov"
skinparam titleFontColor #14180C
skinparam ArrowColor #656F33
skinparam NoteBackgroundColor #DBEEDD
skinparam NoteFontColor #14180C
skinparam NoteBorderColor #4A5837
skinparam SwimlaneBorderColor #4A5837
skinparam SwimlaneBorderThickness 1
skinparam activityStartColor #EBECE3
skinparam activityEndColor #4A5837
skinparam activityStopColor #4A5837
skinparam ActivityBackgroundColor #EBECE3
skinparam ActivityBorderColor #4A5837
skinparam ActivityDiamondBackgroundColor #CBD2B5
skinparam ActivityDiamondBorderColor #4A5837
skinparam packageStyle rectangle
skinparam packageBackgroundColor #F1F3EA
skinparam packageBorderColor #4A5837
skinparam packageFontColor #14180C
skinparam classBackgroundColor #EBECE3
skinparam classBorderColor #4A5837
skinparam classFontColor #14180C
skinparam classAttributeFontColor #3F4724
skinparam classStereotypeFontColor #4A5837
skinparam interfaceBackgroundColor #DBEEDD
skinparam interfaceBorderColor #4A5837
skinparam interfaceFontColor #14180C
skinparam enumBackgroundColor #E4E6D8
skinparam enumBorderColor #4A5837
skinparam enumFontColor #14180C

235
docs/pipeline/diagrams/current/activity.puml
View File

@@ -1,235 +0,0 @@
@startuml
skinparam style strictuml
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 14
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
skinparam backgroundColor #FAFCF9
skinparam defaultFontColor #28342A
skinparam titleFontColor #28342A
skinparam ArrowColor #628A5B
skinparam NoteBackgroundColor #EAF0E8
skinparam NoteBorderColor #547461
skinparam ActivityBackgroundColor #FAFCF9
skinparam ActivityBorderColor #547461
skinparam ActivityDiamondBackgroundColor #FAFCF9
skinparam ActivityDiamondBorderColor #628A5B
skinparam ActivityBarColor #628A5B
skinparam SwimlaneBorderColor #547461
skinparam SwimlaneBorderThickness 0.3
title The Biergarten Data Pipeline (Current — Synchronous Data Path)
|#F2F6F0|main.cc|
start
:Create BoundedChannel<LogEntry> log_channel;
:Spawn log dispatcher thread\n(LogDispatcher::Run());
note right
The only concurrency in the current pipeline:
a dedicated thread drains log_channel and
forwards entries to spdlog for the entire run.
Joined during shutdown after log_channel closes.
end note
:ParseArguments(argc, argv);
if (Are arguments valid?) then (no)
:Log error / usage info;
stop
else (yes)
endif
:Init OpenSSL global state & LlamaBackendState;
:di::make_injector(...);
note right
Binds ILogger, IEnrichmentService, DataGenerator,
IExportService, IPromptFormatter, WebClient based
on --mocked vs. model-backed mode.
end note
:injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>();
:BiergartenPipelineOrchestrator::Run();
note right
Run() itself is synchronous — sampling, enrichment,
generation, and export all happen on this thread.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
:Initialize SQLite export;
|#E0EAE0|SqliteExportService|
:GetUtcTimestamp() from SystemDateTimeProvider;
:Initialize();
note right
Builds a fresh biergarten_seed_<UTC datetime>.sqlite filename
Appends a numeric suffix if the timestamp already exists
Opens DB Connection
Executes Schema DDL
Begins Transaction
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
:QueryCitiesWithCountries();
|#E2EBDC|JsonLoader|
:JsonLoader::LoadLocations("locations.json");
:std::ranges::sample(all_locations, --location-count);
note right
--location-count defaults to 10.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
while (For each sampled Location?) is (Remaining cities)
|#DCE8D8|IEnrichmentService|
:GetLocationContext(loc);
note right
WikipediaEnrichmentService fetches a generic
"brewing" extract and a "beer in {country}" extract
(not a city/region-specific page). MockEnrichmentService
(--mocked) returns an empty string instead.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
if (Lookup failed?) then (yes)
:Log warning, skip city;
else (no)
:Store EnrichedCity{Location, region_context};
endif
endwhile (Done)
|#EAF0E8|BiergartenPipelineOrchestrator|
:GenerateUsers(enriched_cities);
|#E2EBDC|JsonLoader|
:LoadPersonas("personas.json");
:LoadForenamesByCountry("forenames-by-country.json");
:LoadSurnamesByCountry("surnames-by-country.json");
note right
Each call is cached after its first parse.
MockCuratedDataService (--mocked) returns a
fixed in-memory dataset instead and skips
these three JSON files entirely.
end note
|#EAF0E8|BiergartenPipelineOrchestrator|
while (For each EnrichedCity?) is (Remaining cities)
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1);
if (Names available for country?) then (no)
:Log warning, skip city;
else (yes)
:Sample UserPersona (uniform random);
|#E5EDE1|DataGenerator|
if (Generator Mode) then (MockGenerator)
:DeterministicHash & Format;
else (LlamaGenerator)
:prompt_directory_->Load("USER_GENERATION");
note right
Resolves to USER_GENERATION.md inside --prompt-dir.
end note
repeat
:Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
:ValidateUserJson(raw, user);
if (Is JSON Valid?) then (yes)
break
else (no)
:Attempt++;
:Append validation error to retry prompt;
endif
repeat while (Attempt < 3?) is (yes)
endif
|#EAF0E8|BiergartenPipelineOrchestrator|
if (Generation successful?) then (yes)
:BuildEmail(name, used_email_local_parts);
:GenerateDateOfBirth(rng);
note right
email and date_of_birth are generated
programmatically, never LLM-authored.
end note
|#E0EAE0|SqliteExportService|
:ProcessRecord(UserRecord);
if (Location in cache?) then (yes)
:Reuse location_id;
else (no)
:Insert Location & Cache ID;
endif
:Insert User (FK: location_id);
if (Exception caught during insert?) then (yes)
|#EAF0E8|BiergartenPipelineOrchestrator|
:Log warning "SQLite export failed";
else (no)
endif
else (no)
:Log warning "Generation failed, skipping...";
endif
endif
endwhile (Done)
|#EAF0E8|BiergartenPipelineOrchestrator|
:GenerateBreweries(enriched_cities);
|#E5EDE1|DataGenerator|
while (For each EnrichedCity?) is (Remaining cities)
if (Generator Mode) then (MockGenerator)
:DeterministicHash & Format;
else (LlamaGenerator)
:PrepareRegionContext;
:prompt_directory_->Load("BREWERY_GENERATION");
note right
Resolves to BREWERY_GENERATION.md inside --prompt-dir.
end note
repeat
:Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar);
:ValidateBreweryJson(raw, brewery);
if (Is JSON Valid?) then (yes)
break
else (no)
:Attempt++;
:Append validation error to retry prompt;
endif
repeat while (Attempt < 3?) is (yes)
note right
max_tokens is fixed across retries —
it is not raised on truncation.
end note
endif
|#EAF0E8|BiergartenPipelineOrchestrator|
if (Generation successful?) then (yes)
|#E0EAE0|SqliteExportService|
:ProcessRecord(GeneratedBrewery);
if (Location in cache?) then (yes)
:Reuse location_id;
else (no)
:Insert Location & Cache ID;
endif
:Insert Brewery (FK: location_id);
if (Exception caught during insert?) then (yes)
|#EAF0E8|BiergartenPipelineOrchestrator|
:Log warning "SQLite export failed";
note right
Data loss is prevented per-record.
The pipeline continues running.
end note
else (no)
endif
else (no)
:Log warning "Generation failed, skipping...";
endif
|#E5EDE1|DataGenerator|
endwhile (Done)
|#E0EAE0|SqliteExportService|
:Finalize();
note right
Commits Transaction
Closes Database Connection
end note
|#F2F6F0|main.cc|
:Close log_channel;
:Join log dispatcher thread;
:Return 0;
stop
@enduml

269
docs/pipeline/diagrams/current/class.puml
View File

@@ -1,269 +0,0 @@
@startuml
skinparam style strictuml
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 14
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
skinparam backgroundColor #FAFCF9
skinparam defaultFontColor #28342A
skinparam titleFontColor #28342A
skinparam ArrowColor #628A5B
skinparam class {
BackgroundColor #FAFCF9
HeaderBackgroundColor #EAF0E8
BorderColor #547461
ArrowColor #628A5B
FontColor #28342A
}
skinparam note {
BackgroundColor #EAF0E8
BorderColor #547461
FontColor #28342A
}
title The Biergarten Data Pipeline - Class Diagram
class BiergartenPipelineOrchestrator {
- logger_ : std::shared_ptr<ILogger>
- context_service_ : std::unique_ptr<IEnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- exporter_ : std::unique_ptr<IExportService>
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
- application_options_ : ApplicationOptions
- generated_breweries_ : std::vector<BreweryRecord>
- generated_users_ : std::vector<UserRecord>
+ Run() : bool
- QueryCitiesWithCountries() : std::vector<Location>
- GenerateBreweries(cities : std::span<const EnrichedCity>) : void
- GenerateUsers(cities : std::span<const EnrichedCity>) : void
- LogResults() : void
}
class LogLevel <<enumeration>> {
Debug
Info
Warn
Error
}
class PipelinePhase <<enumeration>> {
Startup
UserGeneration
BreweryAndBeerGeneration
CheckinGeneration
RatingGeneration
FollowGeneration
Teardown
}
struct LogDTO {
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
}
struct LogEntry {
+ timestamp : std::chrono::system_clock::time_point
+ origin : std::source_location
+ thread_id : std::thread::id
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
}
interface ILogger <<interface>> {
+ Log(payload : LogDTO) : void
- {abstract} DoLog(entry : LogEntry) : void
}
class LogProducer {
- channel_ : BoundedChannel<LogEntry>&
- DoLog(entry : LogEntry) : void
}
class LogDispatcher {
- channel_ : BoundedChannel<LogEntry>&
+ Run() : void
- ToSpdlogLevel(level) : spdlog::level::level_enum
}
interface IEnrichmentService <<interface>> {
+ GetLocationContext(loc : const Location&) : std::string
}
class MockEnrichmentService {
+ GetLocationContext(loc : const Location&) : std::string
}
class WikipediaEnrichmentService {
- client_ : std::unique_ptr<WebClient>
- extract_cache_ : std::unordered_map<std::string, std::string>
+ GetLocationContext(loc : const Location&) : std::string
- FetchExtract(query : std::string_view) : std::string
}
interface WebClient <<interface>> {
+ Get(url : const std::string&) : std::string
+ UrlEncode(value : const std::string&) : std::string
}
class HttpWebClient {
+ Get(url : const std::string&) : std::string
+ UrlEncode(value : const std::string&) : std::string
}
interface DataGenerator <<interface>> {
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
}
class MockGenerator {
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
- DeterministicHash(location : const Location&) : size_t
- DeterministicHash(location : const Location&, persona : const UserPersona&, name : const Name&) : size_t
}
class LlamaGenerator {
- model_ : ModelHandle
- context_ : ContextHandle
- prompt_formatter_ : std::unique_ptr<IPromptFormatter>
- prompt_directory_ : std::unique_ptr<IPromptDirectory>
- rng_ : std::mt19937
+ GenerateBrewery(...) : BreweryResult
+ GenerateUser(...) : UserResult
- Load(model_path : const std::string&) : void
- Infer(...) : std::string
- InferFormatted(...) : std::string
}
interface IPromptFormatter <<interface>> {
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
}
class Gemma4JinjaPromptFormatter {
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
}
interface IPromptDirectory <<interface>> {
+ Load(key : std::string_view) : std::string
}
class PromptDirectory {
- prompt_dir_ : std::filesystem::path
- cache_ : std::unordered_map<std::string, std::string>
+ Load(key : std::string_view) : std::string
}
interface ICuratedDataService <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
class JsonLoader {
- cache_ : cache
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
note right of JsonLoader
Each Load* method memoizes its result in
cache_ on first call; later calls on the
same instance return the cached value
without re-parsing.
end note
class MockCuratedDataService {
- locations_ : std::vector<Location>
- personas_ : std::vector<UserPersona>
- forenames_by_country_ : std::unordered_map<std::string, forename_list>
- surnames_by_country_ : std::unordered_map<std::string, surname_list>
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
note right of MockCuratedDataService
Fixed 4-location / 3-persona dataset
(US, DE, FR, BE) used in --mocked runs;
filepath arguments are ignored.
end note
interface IExportService <<interface>> {
+ Initialize() : void
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
+ ProcessRecord(user : const UserRecord&) : uint64_t
+ Finalize() : void
}
class SqliteExportService {
- date_time_provider_ : std::unique_ptr<IDateTimeProvider>
- run_timestamp_utc_ : std::string
- database_path_ : std::filesystem::path
- db_handle_ : SqliteDatabaseHandle
- insert_location_stmt_ : SqliteStatementHandle
- insert_brewery_stmt_ : SqliteStatementHandle
- insert_user_stmt_ : SqliteStatementHandle
- transaction_open_ : bool
- location_cache_ : std::unordered_map<std::string, sqlite3_int64>
+ Initialize() : void
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
+ ProcessRecord(user : const UserRecord&) : uint64_t
+ Finalize() : void
- InitializeSchema() : void
- ResolveLocationId(location : const Location&) : sqlite3_int64
}
interface IDateTimeProvider <<interface>> {
+ GetUtcTimestamp() : std::string
}
class SystemDateTimeProvider {
+ GetUtcTimestamp() : std::string
}
' Structural Relationships / Dependency Injection
BiergartenPipelineOrchestrator *-- ILogger : owns
BiergartenPipelineOrchestrator *-- IEnrichmentService : owns
BiergartenPipelineOrchestrator *-- DataGenerator : owns
BiergartenPipelineOrchestrator *-- IExportService : owns
BiergartenPipelineOrchestrator *-- ICuratedDataService : owns
LogEntry *-- LogLevel
LogEntry *-- PipelinePhase
LogDTO *-- LogLevel
LogDTO *-- PipelinePhase
ILogger <|.. LogProducer : implements
LogProducer ..> LogEntry : emits
LogDispatcher ..> LogEntry : consumes
IEnrichmentService <|.. WikipediaEnrichmentService : implements
IEnrichmentService <|.. MockEnrichmentService : implements
WikipediaEnrichmentService *-- WebClient : owns
WebClient <|.. HttpWebClient : implements
DataGenerator <|.. MockGenerator : implements
DataGenerator <|.. LlamaGenerator : implements
LlamaGenerator *-- IPromptFormatter : uses
LlamaGenerator *-- IPromptDirectory : uses
IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements
IPromptDirectory <|.. PromptDirectory : implements
ICuratedDataService <|.. JsonLoader : implements
ICuratedDataService <|.. MockCuratedDataService : implements
IExportService <|.. SqliteExportService : implements
SqliteExportService *-- IDateTimeProvider : owns
IDateTimeProvider <|.. SystemDateTimeProvider : implements
@enduml

1
docs/pipeline/diagrams/current/output/activity.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 86 KiB

1
docs/pipeline/diagrams/current/output/class.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 87 KiB

372
docs/pipeline/diagrams/planned/activity.puml
View File

@@ -1,372 +0,0 @@
@startuml biergarten_activity
!include ../biergarten-weizen-theme.puml
skinparam defaultFontSize 13
skinparam titleFontSize 20
title The Biergarten Data Pipeline — Activity Diagram
|Main|
start
:ParseArguments(argc, argv);
if (Invalid args?) then (yes)
:spdlog::error;
stop
else (no)
endif
:Init OpenSSL global state & LlamaBackendState;
:Build DI injector;
:Initialize SqliteExportService;
note right
Opens SQLite connection.
(Transactions are now managed
per-phase via batching).
end note
:Create BoundedChannel<LogEntry> log_ch;
:Spawn Log Worker thread;
note right
Log worker drains log_ch for the
entire pipeline lifetime.
All workers emit LogEntry structs
via PipelineLogger -- never spdlog directly.
end note
:BiergartenPipelineOrchestrator::Run();
|BiergartenPipelineOrchestrator::Run()|
fork
:JsonLoader::LoadBeerStyles("beer-styles.json");
:EnrichmentService::PreWarmBeerStyleCache(beer_styles);
fork again
:JsonLoader::LoadLocations("locations.json");
:EnrichmentService::PreWarmLocationCache(sampled_locations);
end fork
fork
:JsonLoader::LoadForenamesByCountry(\n "forenames-by-country.json");
fork again
:JsonLoader::LoadSurnamesByCountry(\n "surnames-by-country.json");
fork again
:JsonLoader::LoadPersonas("personas.json");
end fork
note right
Each call is memoized after its first parse
(implemented today). MockCuratedDataService
returns a fixed in-memory dataset instead
under --mocked, skipping these JSON files.
end note
' ═══════════════════════════════════════════
' PHASE 0 — USER GENERATION
' ═══════════════════════════════════════════
|Orchestrator|
:RunUserPhase(sampled_locations);
:Create BoundedChannels\n(loc_ch, exp_ch);
fork
|Orchestrator|
:Loop: Send Locations -> loc_ch;
:Close loc_ch;
note right
Producer closes loc_ch.
LLM Worker while loop
terminates on empty + closed.
end note
fork again
|LLM Worker|
while (loc_ch has items?) is (yes)
:Receive Location;
:GetLocationContextFromCache(location);
note right
Guaranteed cache hit from startup.
end note
:IPersonaSelectionStrategy::SelectPersona(\n personas_palette_);
note right
Guaranteed cache hit from startup.
Returns a Persona struct carrying
style_affinities, abv_range,
ibu_preference, checkin_weight.
end note
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1, rng);
note right
Deterministic lookup -- no LLM involved.
Free function (implemented today in
generate_users.cc), not a class method.
Name selected from pre-keyed maps
and passed into the generation prompt.
Skips the city if either map has no
entry for iso3166_1.
end note
:GenerateUser(enriched_city, persona, sampled_name)\nvia DataGenerator;
note right
LLM receives: EnrichedCity context + persona
description + sampled name. Generates
bio and preference signals grounded
in locale and persona.
end note
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "llm");
:Send GeneratedUser -> exp_ch;
endwhile (no)
:Close exp_ch;
note right
Producer closes exp_ch.
SQLite Worker while loop
terminates on empty + closed.
end note
fork again
|SQLite Worker|
:BEGIN TRANSACTION;
while (exp_ch has items?) is (yes)
:Receive GeneratedUser;
:ProcessUser(user);
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "sqlite");
:Append -> user_pool_;
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
endwhile (no)
:COMMIT (Final);
end fork
|Orchestrator|
:Join LLM Worker, SQLite Worker;
' ═══════════════════════════════════════════
' PHASE 1a — BREWERY GENERATION
' ═══════════════════════════════════════════
:RunBreweryPhase(sampled_locations);
:Create BoundedChannels\n(loc_ch, exp_ch);
fork
|Orchestrator|
:Loop: Sample User from user_pool_
and pair with Location;
:Send BreweryTask(Location, User) -> loc_ch;
:Close loc_ch;
fork again
|LLM Worker|
while (loc_ch has items?) is (yes)
:Receive BreweryTask(Location, User);
:GetLocationContextFromCache(task.location);
note right
Guaranteed cache hit from startup.
end note
:GenerateBrewery(enriched_city, context, task.user)\nvia DataGenerator;
note right
KV cache stays warm.
Brewery is linked to the sampled owner_user_id.
end note
:PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "llm");
:Send GeneratedBrewery -> exp_ch;
endwhile (no)
:Close exp_ch;
fork again
|SQLite Worker|
:BEGIN TRANSACTION;
while (exp_ch has items?) is (yes)
:Receive GeneratedBrewery;
:ProcessBrewery(brewery);
:PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "sqlite");
:Append -> brewery_pool_;
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
endwhile (no)
:COMMIT (Final);
end fork
|Orchestrator|
:Join LLM Worker, SQLite Worker;
note right
brewery_pool_ is now fully populated.
Phase 1b may begin.
end note
' ═══════════════════════════════════════════
' PHASE 1b — BEER GENERATION
' ═══════════════════════════════════════════
:RunBeerPhase();
:Create BoundedChannels\n(brew_ch, exp_ch);
fork
|Orchestrator|
:Loop: Send Breweries -> brew_ch;
:Close brew_ch;
fork again
|LLM Worker|
while (brew_ch has items?) is (yes)
:Receive GeneratedBrewery;
:IBeerSelectionStrategy::SelectStyles(\n brewery, beer_style_palette_);
while (For each selected BeerStyle?) is (remaining)
:GetStyleContextFromCache(style);
note right
Guaranteed cache hit from startup.
KV cache stays warm across all
beer generations -- system prompt
does not change within this phase.
end note
:GenerateBeer(brewery, style_context)\nvia DataGenerator;
:Attach GeneratedBeer to bundle;
endwhile (done)
:PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "llm");
:Send BeersBundle -> exp_ch;
endwhile (no)
:Close exp_ch;
fork again
|SQLite Worker|
:BEGIN TRANSACTION;
while (exp_ch has items?) is (yes)
:Receive BeersBundle;
while (For each beer in bundle?) is (remaining)
:Set beer.brewery_id from bundle;
:ProcessBeer(beer);
:Append -> beer_pool_;
endwhile (done)
:PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "sqlite");
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
endwhile (no)
:COMMIT (Final);
end fork
|Orchestrator|
:Join LLM Worker, SQLite Worker;
note right
Both brewery_pool_ and beer_pool_
are now completely populated.
Checkin and Follow phases may
now run in parallel.
end note
' ═══════════════════════════════════════════
' PHASE 2 — CHECKIN + FOLLOW GENERATION
' (parallel — both depend only on user_pool_
' and brewery_pool_ being fully populated)
' ═══════════════════════════════════════════
fork
|Orchestrator|
:RunCheckinPhase();
:ICheckinDistributionStrategy::\nAssignActivityWeights(user_pool_);
note right
Weights seeded from each user's
persona.checkin_weight. J-curve profile
emerges from persona distribution.
end note
:BEGIN TRANSACTION;
while (For each GeneratedUser in user_pool_?) is (remaining)
:CheckinsForUser(user, brewery_pool_.size());
while (For each checkin index?) is (remaining)
:TimestampFor(user, index);
:Select brewery from brewery_pool_;
:GenerateCheckin(user, brewery, timestamp)\nvia DataGenerator;
:ProcessCheckin(checkin);
:PipelineLogger::Log(Info, CheckinGeneration,\n nullopt, checkin_id, "sqlite");
:Append -> checkin_pool_;
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
endwhile (done)
endwhile (done)
:COMMIT (Final);
fork again
|Orchestrator|
:RunFollowPhase();
:IFollowGenerationStrategy::\nAssignFollowWeights(user_pool_);
note right
For RandomFollowStrategy, weights
are uniform. For ActivityWeightedFollowStrategy,
weights derived from user.activity_weight
so high-activity users attract more followers.
end note
:BEGIN TRANSACTION;
:IFollowGenerationStrategy::\nGenerateFollows(user_pool_);
note right
Self-follow constraint (follower_id != followed_id)
enforced here and at the DB schema level.
end note
while (For each GeneratedFollow?) is (remaining)
:ProcessFollow(follow);
:PipelineLogger::Log(Info, FollowGeneration,\n nullopt, follower_id, "sqlite");
:Append -> follow_pool_;
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
endwhile (done)
:COMMIT (Final);
end fork
|Orchestrator|
:Join CheckinPhase, FollowPhase;
note right
checkin_pool_ and follow_pool_
are now fully populated.
Rating phase may begin.
end note
' ═══════════════════════════════════════════
' PHASE 3 — RATING GENERATION
' ═══════════════════════════════════════════
:RunRatingPhase();
note right
Beer selection biased by
user.persona.style_affinities and abv_range.
Rating skew modulated per persona.
end note
:BEGIN TRANSACTION;
while (For each GeneratedCheckin in checkin_pool_?) is (remaining)
:Match brewery_id, select beer from beer_pool_\n(same brewery_id, biased by persona affinities);
if (Beer exists for brewery?) then (yes)
:GenerateRating(user, beer, checkin_id)\nvia DataGenerator;
:ProcessRating(rating);
:PipelineLogger::Log(Info, RatingGeneration,\n nullopt, rating_id, "sqlite");
if (Batch size reached?) then (yes)
:COMMIT & BEGIN;
else (no)
endif
else (no)
:PipelineLogger::Log(Warn, RatingGeneration,\n nullopt, brewery_id, "sqlite");
:Skip -- brewery has no beers;
endif
endwhile (done)
:COMMIT (Final);
' ═══════════════════════════════════════════
' TEARDOWN
' ═══════════════════════════════════════════
|Orchestrator|
:Finalize SqliteExportService;
note right
Safely closes the DB connection.
end note
:Close log_ch;
|Main|
:spdlog::info "Pipeline complete in X ms";
:Join Log Worker;
note right
Drain guarantees no LogEntry is
dropped at shutdown.
end note
stop
@enduml

1
docs/pipeline/diagrams/planned/output/biergarten_activity.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 142 KiB

1
docs/pipeline/diagrams/planned/output/class.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 240 KiB

340
docs/testing.md Normal file
View File

@@ -0,0 +1,340 @@
# Testing
This document describes the testing strategy and how to run tests for The Biergarten App.
## Overview
The project uses a multi-layered testing approach across backend and frontend:
- **API.Specs** - BDD integration tests using Reqnroll (Gherkin)
- **Infrastructure.Repository.Tests** - Unit tests for data access layer
- **Service.Auth.Tests** - Unit tests for authentication business logic
- **Storybook Vitest project** - Browser-based interaction tests for shared website stories
- **Storybook Playwright suite** - Browser checks against Storybook-rendered components
## Running Tests with Docker (Recommended)
The easiest way to run all tests is using Docker Compose, which sets up an isolated test
environment:
```bash
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
```
This command:
1. Starts a fresh SQL Server instance
2. Runs database migrations
3. Seeds test data
4. Executes all test suites in parallel
5. Exports results to `./test-results/`
6. Exits when tests complete
### View Test Results
```bash
# List test result files
ls -la test-results/
# View specific test results
cat test-results/api-specs/results.trx
cat test-results/repository-tests/results.trx
cat test-results/service-auth-tests/results.trx
```
### Clean Up
```bash
# Remove test containers and volumes
docker compose -f docker-compose.test.yaml down -v
```
## Running Tests Locally
You can run individual test projects locally without Docker:
### Integration Tests (API.Specs)
```bash
cd src/Core
dotnet test API/API.Specs/API.Specs.csproj
```
**Requirements**:
- SQL Server instance running
- Database migrated and seeded
- Environment variables set (DB connection, JWT secret)
### Repository Tests
```bash
cd src/Core
dotnet test Infrastructure/Infrastructure.Repository.Tests/Infrastructure.Repository.Tests.csproj
```
**Requirements**:
- SQL Server instance running (uses mock data)
### Service Tests
```bash
cd src/Core
dotnet test Service/Service.Auth.Tests/Service.Auth.Tests.csproj
```
**Requirements**:
- No database required (uses Moq for mocking)
### Frontend Storybook Tests
```bash
cd src/Website
npm install
npm run test:storybook
```
**Purpose**:
- Verifies shared stories such as form fields, submit buttons, navbar states, toasts, and the theme gallery
- Runs in browser mode via Vitest and Storybook integration
### Frontend Playwright Storybook Tests
```bash
cd src/Website
npm install
npm run test:storybook:playwright
```
**Requirements**:
- Storybook dependencies installed
- Playwright browser dependencies installed
- The command will start or reuse the Storybook server defined in `playwright.storybook.config.ts`
## Test Coverage
### Current Coverage
**Authentication & User Management**:
- User registration with validation
- User login with JWT token generation
- Password hashing and verification (Argon2id)
- JWT token generation and claims
- Invalid credentials handling
- 404 error responses
**Repository Layer**:
- User account creation
- User credential management
- GetUserByUsername queries
- Stored procedure execution
**Service Layer**:
- Login service with password verification
- Register service with validation
- Business logic for authentication flow
**Frontend UI Coverage**:
- Shared submit button states
- Form field happy path and error presentation
- Navbar guest, authenticated, and mobile behavior
- Theme gallery rendering across Biergarten themes
- Toast interactions and themed notification display
### Planned Coverage
- [ ] Email verification workflow
- [ ] Password reset functionality
- [ ] Token refresh mechanism
- [ ] Brewery data management
- [ ] Beer post operations
- [ ] User follow/unfollow
- [ ] Image upload service
- [ ] Frontend route integration coverage beyond Storybook stories
## Testing Frameworks & Tools
### xUnit
- Primary unit testing framework
- Used for Repository and Service layer tests
- Supports parallel test execution
### Reqnroll (Gherkin/BDD)
- Behavior-driven development framework
- Used for API integration tests
- Human-readable test scenarios in `.feature` files
### FluentAssertions
- Expressive assertion library
- Makes test assertions more readable
- Used across all test projects
### Moq
- Mocking framework for .NET
- Used in Service layer tests
- Enables isolated unit testing
### DbMocker
- Database mocking for repository tests
- Simulates SQL Server responses
- No real database required for unit tests
## Test Structure
### API.Specs (Integration Tests)
```
API.Specs/
├── Features/
│ ├── Authentication.feature # Login/register scenarios
│ └── UserManagement.feature # User CRUD scenarios
├── Steps/
│ ├── AuthenticationSteps.cs # Step definitions
│ └── UserManagementSteps.cs
└── Mocks/
└── TestApiFactory.cs # Test server setup
```
**Example Feature**:
```gherkin
Feature: User Authentication
As a user
I want to register and login
So that I can access the platform
Scenario: Successful user registration
Given I have valid registration details
When I register a new account
Then I should receive a JWT token
And my account should be created
```
### Infrastructure.Repository.Tests
```
Infrastructure.Repository.Tests/
├── AuthRepositoryTests.cs # Auth repository tests
├── UserAccountRepositoryTests.cs # User account tests
└── TestFixtures/
└── DatabaseFixture.cs # Shared test setup
```
### Service.Auth.Tests
```
Service.Auth.Tests/
├── LoginService.test.cs # Login business logic tests
└── RegisterService.test.cs # Registration business logic tests
```
## Writing Tests
### Unit Test Example (xUnit)
```csharp
public class LoginServiceTests
{
[Fact]
public async Task LoginAsync_ValidCredentials_ReturnsToken()
{
// Arrange
var mockRepo = new Mock<IAuthRepository>();
var mockJwt = new Mock<IJwtService>();
var service = new AuthService(mockRepo.Object, mockJwt.Object);
// Act
var result = await service.LoginAsync("testuser", "password123");
// Assert
result.Should().NotBeNull();
result.Token.Should().NotBeNullOrEmpty();
}
}
```
### Integration Test Example (Reqnroll)
```gherkin
Scenario: User login with valid credentials
Given a registered user with username "testuser"
When I POST to "/api/auth/login" with valid credentials
Then the response status should be 200
And the response should contain a JWT token
```
## Continuous Integration
Tests run automatically in CI/CD pipelines using the test Docker Compose configuration:
```bash
# CI/CD command
docker compose -f docker-compose.test.yaml build
docker compose -f docker-compose.test.yaml up --abort-on-container-exit
docker compose -f docker-compose.test.yaml down -v
```
Exit codes:
- `0` - All tests passed
- Non-zero - Test failures occurred
Frontend UI checks should also be included in CI for the active website workspace:
```bash
cd src/Website
npm ci
npm run test:storybook
npm run test:storybook:playwright
```
## Troubleshooting
### Tests Failing Due to Database Connection
Ensure SQL Server is running and environment variables are set:
```bash
docker compose -f docker-compose.test.yaml ps
```
### Port Conflicts
If port 1433 is in use, stop other SQL Server instances or modify the port in
`docker-compose.test.yaml`.
### Stale Test Data
Clean up test database:
```bash
docker compose -f docker-compose.test.yaml down -v
```
### View Container Logs
```bash
docker compose -f docker-compose.test.yaml logs <service-name>
```
## Best Practices
1. **Isolation**: Each test should be independent and not rely on other tests
2. **Cleanup**: Use fixtures and dispose patterns for resource cleanup
3. **Mocking**: Mock external dependencies in unit tests
4. **Descriptive Names**: Use clear, descriptive test method names
5. **Arrange-Act-Assert**: Follow AAA pattern in unit tests
6. **Given-When-Then**: Follow GWT pattern in BDD scenarios

118
docs/website/token-validation.md → docs/token-validation.md
View File

@@ -2,114 +2,85 @@
## Overview
The Core project validates JWTs across three token types:
The Core project implements comprehensive JWT token validation across three token types:
- **Access Tokens**: Short-lived (1 hour) tokens for API authentication
- **Refresh Tokens**: Long-lived (21 days) tokens for obtaining new access
tokens
- **Confirmation Tokens**: Short-lived (30 minutes) tokens for email
confirmation
- **Refresh Tokens**: Long-lived (21 days) tokens for obtaining new access tokens
- **Confirmation Tokens**: Short-lived (30 minutes) tokens for email confirmation
## Components
### Infrastructure Layer
#### [ITokenInfrastructure](../../web/backend/Infrastructure/Infrastructure.Jwt/ITokenInfrastructure.cs)
#### [ITokenInfrastructure](Infrastructure.Jwt/ITokenInfrastructure.cs)
Low-level JWT operations.
**Methods:**
- `GenerateJwt()` - Creates signed JWT tokens
- `ValidateJwtAsync()` - Validates token signature, expiration, and format
**Implementation:**
[JwtInfrastructure.cs](../../web/backend/Infrastructure/Infrastructure.Jwt/JwtInfrastructure.cs)
**Implementation:** [JwtInfrastructure.cs](Infrastructure.Jwt/JwtInfrastructure.cs)
- Uses Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler
- Algorithm: HS256 (HMAC-SHA256)
- Validates token lifetime, signature, and well-formedness
### Features.Auth Slice
### Service Layer
#### [ITokenService](../../web/backend/Features/Features.Auth/Services/ITokenService.cs)
#### [ITokenValidationService](Service.Auth/ITokenValidationService.cs)
Both token generation and validation live on the same slice-internal
service (there is no separate validation service/class).
**Generation methods:**
- `GenerateAccessToken(UserAccount)` - Creates 1-hour access token
- `GenerateRefreshToken(UserAccount)` - Creates 21-day refresh token
- `GenerateConfirmationToken(UserAccount)` - Creates 30-minute confirmation
token
**Validation methods:**
High-level token validation with context (token type, user extraction).
**Methods:**
- `ValidateAccessTokenAsync(string token)` - Validates access tokens
- `ValidateRefreshTokenAsync(string token)` - Validates refresh tokens
- `ValidateConfirmationTokenAsync(string token)` - Validates confirmation tokens
**Returns (validation):** `ValidatedToken` record containing:
**Returns:** `ValidatedToken` record containing:
- `UserId` (Guid)
- `Username` (string)
- `Principal` (ClaimsPrincipal) - Full JWT claims
**Implementation:** [TokenService.cs](../../web/backend/Features/Features.Auth/Services/TokenService.cs)
**Implementation:** [TokenValidationService.cs](Service.Auth/TokenValidationService.cs)
- Reads token secrets from environment variables
- Extracts and validates claims (Sub, UniqueName)
- Throws `UnauthorizedException` on validation failure
`TokenService` is registered by `Features.Auth`'s own
`AddFeaturesAuth()` extension method, except for the lower-level
`ITokenInfrastructure` (JWT signing/verification) it depends on, which is
registered by the host (`API.Core/Program.cs`) since
`JwtAuthenticationHandler` (host-level auth middleware) also depends on
it directly.
#### [ITokenService](Service.Auth/ITokenService.cs)
Token generation (existing service extended).
**Methods:**
- `GenerateAccessToken(UserAccount)` - Creates 1-hour access token
- `GenerateRefreshToken(UserAccount)` - Creates 21-day refresh token
- `GenerateConfirmationToken(UserAccount)` - Creates 30-minute confirmation token
### Integration Points
#### [ConfirmUserHandler](../../web/backend/Features/Features.Auth/Commands/ConfirmUser/ConfirmUserHandler.cs)
#### [ConfirmationService](Service.Auth/IConfirmationService.cs)
**Flow:**
1. Receives confirmation token from user via `ConfirmUserCommand`
2. Calls `ITokenService.ValidateConfirmationTokenAsync()`
1. Receives confirmation token from user
2. Calls `TokenValidationService.ValidateConfirmationTokenAsync()`
3. Extracts user ID from validated token
4. Calls `IAuthRepository.ConfirmUserAccountAsync()` to update database
4. Calls `AuthRepository.ConfirmUserAccountAsync()` to update database
5. Returns confirmation result
#### [ResendConfirmationEmailHandler](../../web/backend/Features/Features.Auth/Commands/ResendConfirmationEmail/ResendConfirmationEmailHandler.cs)
#### [RefreshTokenService](Service.Auth/RefreshTokenService.cs)
**Flow:**
1. Receives refresh token from user
2. Calls `TokenValidationService.ValidateRefreshTokenAsync()`
3. Retrieves user account via `AuthRepository.GetUserByIdAsync()`
4. Issues new access and refresh tokens via `TokenService`
5. Returns new token pair
1. Receives a user ID via `ResendConfirmationEmailCommand`
2. Looks up the user and checks they aren't already verified
3. Generates a fresh confirmation token via `ITokenService`
4. Sends `SendResendConfirmationEmailCommand` over MediatR, handled by the
`Features.Emails` slice, with no direct project reference between the two
slices
#### [RefreshTokenHandler](../../web/backend/Features/Features.Auth/Commands/RefreshToken/RefreshTokenHandler.cs)
**Flow:**
1. Receives a refresh token via `RefreshTokenCommand`
2. Delegates to `ITokenService.RefreshTokenAsync()`, which validates the
token, retrieves the user account via `IAuthRepository.GetUserByIdAsync()`,
and issues a new access/refresh token pair
3. Maps the result onto `LoginPayload`
#### [AuthController](../../web/backend/Features/Features.Auth/Controllers/AuthController.cs)
#### [AuthController](API.Core/Controllers/AuthController.cs)
**Endpoints:**
- `POST /api/auth/register` - Register new user
- `POST /api/auth/login` - Authenticate user
- `POST /api/auth/confirm?token=...` - Confirm email
- `POST /api/auth/confirm/resend?userId=...` - Resend the confirmation email
- `POST /api/auth/refresh` - Refresh access token
## Validation Security
@@ -117,13 +88,11 @@ it directly.
### Token Secrets
Three independent secrets enable:
- **Key rotation** - Rotate each secret type independently
- **Isolation** - Compromise of one secret doesn't affect others
- **Different expiration** - Different token types can expire at different rates
**Environment Variables:**
```bash
ACCESS_TOKEN_SECRET=... # Signs 1-hour access tokens
REFRESH_TOKEN_SECRET=... # Signs 21-day refresh tokens
@@ -142,7 +111,6 @@ Each token is validated for:
### Error Handling
Validation failures return HTTP 401 Unauthorized:
- Invalid signature → "Invalid token"
- Expired token → "Invalid token" (message doesn't reveal reason for security)
- Missing claims → "Invalid token"
@@ -172,60 +140,42 @@ Validation failures return HTTP 401 Unauthorized:
1. **Generation**: During user registration (30-minute validity)
2. **Delivery**: Emailed to user in confirmation link
3. **Usage**: User clicks link, token posted to `/api/auth/confirm`
4. **Validation**: Validated by `ConfirmUserHandler`
4. **Validation**: Validated by ConfirmationService
5. **Completion**: User account marked as confirmed
6. **Expiration**: Token becomes invalid after 30 minutes
## Testing
All of the following live in `Features.Auth.Tests`.
### Unit Tests
**Services/TokenServiceValidationTests.cs**
**TokenValidationService.test.cs**
- Happy path: Valid token extraction
- Error cases: Invalid, expired, malformed tokens
- Missing/invalid claims scenarios
**Services/TokenServiceRefreshTests.cs**
**RefreshTokenService.test.cs**
- Successful refresh with valid token
- Invalid/expired refresh token rejection
- Non-existent user handling
**Commands/RefreshTokenHandlerTests.cs**
- Verifies the handler maps `ITokenService.RefreshTokenAsync()`'s result onto
`LoginPayload`
**Commands/ConfirmUserHandlerTests.cs**
**ConfirmationService.test.cs**
- Successful confirmation with valid token
- Token validation failures
- User not found scenarios
**Commands/ResendConfirmationEmailHandlerTests.cs**
- Sends a fresh confirmation email when the user exists and is unverified
- No-ops when the user doesn't exist or is already verified
### BDD Tests (Reqnroll)
**TokenRefresh.feature**
- Successful token refresh
- Invalid/expired token rejection
- Missing token validation
**Confirmation.feature**
- Successful email confirmation
- Expired/tampered token rejection
- Missing token validation
**AccessTokenValidation.feature**
- Protected endpoint access token validation
- Invalid/expired access token rejection
- Token type mismatch (refresh used as access token)

1
docs/website/diagrams-out/architecture.svg
View File

File diff suppressed because one or more lines are too long

126
docs/website/diagrams-src/architecture.puml
View File

@@ -1,126 +0,0 @@
@startuml architecture
!theme plain
skinparam backgroundColor #FFFFFF
skinparam defaultFontName Arial
skinparam packageStyle rectangle
title The Biergarten App - Vertical Slice Architecture
package "API.Core (thin host)" #E3F2FD {
[API.Core\nASP.NET Core Web API host] as API
note right of API
- Program.cs wiring only
- MediatR + AddApplicationPart
per Features.* assembly
- Swagger/OpenAPI, health checks
- JWT auth middleware
- Global exception filter
end note
}
package "Feature Slices" #F3E5F5 {
[Features.Auth] as AuthSlice
[Features.Breweries] as BrewerySlice
[Features.UserManagement] as UserSlice
[Features.Emails] as EmailsSlice
note right of AuthSlice
Each slice owns its own:
- Controller (HTTP endpoints)
- Commands/Queries + Handlers
- Validators
- Repository
end note
note right of EmailsSlice
No controller - invoked only
via MediatR commands sent
from other slices
end note
}
package "Shared" #FCE4EC {
[Shared.Contracts] as SharedContracts
[Shared.Application] as SharedApp
note right of SharedApp
- ValidationBehavior
(MediatR pipeline)
- Cross-slice email commands
(SendRegistrationEmailCommand,
SendResendConfirmationEmailCommand)
end note
}
package "Infrastructure Layer" #FFF3E0 {
[Infrastructure.Sql] as Sql
[Infrastructure.Jwt] as JWT
[Infrastructure.PasswordHashing] as PwdHash
[Infrastructure.Email] as Email
[Infrastructure.Email.Templates] as EmailTemplates
}
package "Domain Layer" #E8F5E9 {
[Domain.Entities] as Domain
[Domain.Exceptions] as DomainExceptions
note right of Domain
- UserAccount
- UserCredential
- UserVerification
- BreweryPost
end note
}
database "SQL Server" {
[Stored Procedures] as SP
[Tables] as Tables
}
' Relationships
API ..> AuthSlice : AddApplicationPart\n+ MediatR scan
API ..> BrewerySlice : AddApplicationPart\n+ MediatR scan
API ..> UserSlice : AddApplicationPart\n+ MediatR scan
API ..> EmailsSlice : MediatR scan only\n(no controller)
AuthSlice --> Sql
AuthSlice --> JWT
AuthSlice --> PwdHash
AuthSlice --> SharedContracts
AuthSlice --> SharedApp
BrewerySlice --> Sql
BrewerySlice --> SharedContracts
BrewerySlice --> SharedApp
UserSlice --> Sql
UserSlice --> SharedContracts
UserSlice --> SharedApp
EmailsSlice --> Email
EmailsSlice --> EmailTemplates
EmailsSlice --> SharedApp
' The one cross-slice interaction: a MediatR command contract, not a project reference
AuthSlice ..> SharedApp : sends\nSendRegistrationEmailCommand
EmailsSlice ..> SharedApp : handles\nSendRegistrationEmailCommand
Sql --> SP
Sql --> Domain
SP --> Tables
AuthSlice --> Domain
BrewerySlice --> Domain
UserSlice --> Domain
AuthSlice --> DomainExceptions
' Notes
note left of Sql
SQL-first approach
All queries via
stored procedures
end note
note bottom of AuthSlice
No Features.* project
ever references another
Features.* project directly
end note
@enduml

363
docs/website/testing.md
View File

@@ -1,363 +0,0 @@
# Testing
This document describes the testing strategy and how to run tests for The
Biergarten App.
## Overview
The project uses a multi-layered testing approach across backend and frontend:
- **API.Specs** - BDD integration tests using Reqnroll (Gherkin), run against
a live, seeded database
- **Features.\*.Tests** - One unit test project per backend feature slice
(`Features.Auth.Tests`, `Features.Breweries.Tests`,
`Features.UserManagement.Tests`, `Features.Emails.Tests`), covering that
slice's command/query handlers and its own repository (mocked with Moq /
DbMocker, no real database required)
- **Storybook Vitest project** - Browser-based interaction tests for shared
website stories
- **Storybook Playwright suite** - Browser checks against Storybook-rendered
components
## Running Tests with Docker (Recommended)
The easiest way to run all tests is using Docker Compose, which sets up an
isolated test environment:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
```
This command:
1. Starts a fresh SQL Server instance
2. Runs database migrations
3. Seeds test data
4. Executes all test suites in parallel
5. Exports results to `./test-results/`
6. Exits when tests complete
### View Test Results
```bash
# List test result files
ls -la test-results/
# View specific test results
cat test-results/api-specs/results.trx
cat test-results/Features.Auth.Tests.trx
cat test-results/Features.Breweries.Tests.trx
cat test-results/Features.UserManagement.Tests.trx
cat test-results/Features.Emails.Tests.trx
```
### Clean Up
```bash
# Remove test containers and volumes
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
```
## Running Tests Locally
You can run individual test projects locally without Docker:
### Integration Tests (API.Specs)
```bash
cd web/backend
dotnet test API/API.Specs/API.Specs.csproj
```
**Requirements**:
- SQL Server instance running
- Database migrated and seeded
- Environment variables set (DB connection, JWT secret)
### Feature Slice Unit Tests
Each feature slice has its own test project, covering its command/query
handlers and repository:
```bash
cd web/backend
dotnet test Features/Features.Auth.Tests/Features.Auth.Tests.csproj
dotnet test Features/Features.Breweries.Tests/Features.Breweries.Tests.csproj
dotnet test Features/Features.UserManagement.Tests/Features.UserManagement.Tests.csproj
dotnet test Features/Features.Emails.Tests/Features.Emails.Tests.csproj
# Or run all of them at once via the solution:
for proj in Features/*.Tests; do dotnet test "$proj"; done
```
**Requirements**:
- No database required (handlers use Moq; repository tests use DbMocker to
simulate SQL Server responses)
### Frontend Storybook Tests
```bash
cd web/frontend
npm install
npm run test:storybook
```
**Purpose**:
- Verifies shared stories such as form fields, submit buttons, navbar states,
toasts, and the theme gallery
- Runs in browser mode via Vitest and Storybook integration
### Frontend Playwright Storybook Tests
```bash
cd web/frontend
npm install
npm run test:storybook:playwright
```
**Requirements**:
- Storybook dependencies installed
- Playwright browser dependencies installed
- The command will start or reuse the Storybook server defined in
`playwright.storybook.config.ts`
## Test Coverage
### Current Coverage
**Features.Auth.Tests**:
- User registration with validation
- User login with JWT token generation
- Password hashing and verification (Argon2id)
- Email confirmation and confirmation-email resend
- Refresh token exchange
- JWT token generation, validation, and claims handling
- `IAuthRepository` data access (DbMocker-backed)
- Invalid credentials and 404 error responses (via API.Specs)
**Features.Breweries.Tests**:
- Brewery create/update/delete commands and get-by-id/get-all queries
**Features.UserManagement.Tests**:
- User get-by-id/get-all queries and the (currently unrouted) update command
**Features.Emails.Tests**:
- Registration and resend-confirmation email dispatch handlers
**Frontend UI Coverage**:
- Shared submit button states
- Form field happy path and error presentation
- Navbar guest, authenticated, and mobile behavior
- Theme gallery rendering across Biergarten themes
- Toast interactions and themed notification display
### Planned Coverage
- [ ] Password reset functionality
- [ ] Beer post operations
- [ ] User follow/unfollow
- [ ] Image upload service
- [ ] Frontend route integration coverage beyond Storybook stories
## Testing Frameworks & Tools
### xUnit
- Primary unit testing framework
- Used for Repository and Service layer tests
- Supports parallel test execution
### Reqnroll (Gherkin/BDD)
- Behavior-driven development framework
- Used for API integration tests
- Human-readable test scenarios in `.feature` files
### FluentAssertions
- Expressive assertion library
- Makes test assertions more readable
- Used across all test projects
### Moq
- Mocking framework for .NET
- Used in Service layer tests
- Enables isolated unit testing
### DbMocker
- Database mocking for repository tests
- Simulates SQL Server responses
- No real database required for unit tests
## Test Structure
### API.Specs (Integration Tests)
```
API.Specs/
├── Features/
│ ├── Registration.feature # Registration scenarios
│ ├── Login.feature # Login scenarios
│ ├── Confirmation.feature # Email confirmation scenarios
│ ├── ResendConfirmation.feature # Resend-confirmation scenarios
│ ├── TokenRefresh.feature # Refresh token scenarios
│ ├── AccessTokenValidation.feature # Protected endpoint access scenarios
│ └── NotFound.feature # 404 handling
├── Steps/
│ ├── AuthSteps.cs # Step definitions for the Auth features
│ └── ApiGeneralSteps.cs # Shared/general step definitions
├── Mocks/
│ ├── MockEmailDispatcher.cs # Substitutes Features.Emails' IEmailDispatcher
│ └── MockEmailProvider.cs # Substitutes Infrastructure.Email's IEmailProvider
└── TestApiFactory.cs # Test server setup (swaps in the mocks above)
```
**Example Feature**:
```gherkin
Feature: User Registration
As a user
I want to register
So that I can access the platform
Scenario: Successful user registration
Given I have valid registration details
When I register a new account
Then I should receive a JWT token
And my account should be created
```
### Features.Auth.Tests
```
Features.Auth.Tests/
├── Commands/
│ ├── RegisterUserHandlerTests.cs
│ ├── ConfirmUserHandlerTests.cs
│ ├── ResendConfirmationEmailHandlerTests.cs
│ └── RefreshTokenHandlerTests.cs
├── Queries/
│ └── LoginHandlerTests.cs
├── Repository/
│ └── AuthRepositoryTests.cs # DbMocker-backed repository tests
└── Services/
├── TokenServiceRefreshTests.cs
└── TokenServiceValidationTests.cs
```
Each of the other three slices (`Features.Breweries.Tests`,
`Features.UserManagement.Tests`, `Features.Emails.Tests`) follows the same
shape: a `Commands/`/`Queries/` folder with one test file per handler.
## Writing Tests
### Unit Test Example (xUnit)
```csharp
public class LoginHandlerTests
{
[Fact]
public async Task Handle_WithValidData_ReturnsPayloadWithMatchingUsername()
{
// Arrange
var authRepoMock = new Mock<IAuthRepository>();
var passwordInfraMock = new Mock<IPasswordInfrastructure>();
var tokenServiceMock = new Mock<ITokenService>();
var handler = new LoginHandler(authRepoMock.Object, passwordInfraMock.Object, tokenServiceMock.Object);
// ...set up mocks for a known user/credential...
// Act
var result = await handler.Handle(new LoginQuery("testuser", "password123"), CancellationToken.None);
// Assert
result.Should().NotBeNull();
result.AccessToken.Should().NotBeNullOrEmpty();
}
}
```
### Integration Test Example (Reqnroll)
```gherkin
Scenario: User login with valid credentials
Given a registered user with username "testuser"
When I POST to "/api/auth/login" with valid credentials
Then the response status should be 200
And the response should contain a JWT token
```
## Continuous Integration
Tests run automatically in CI/CD pipelines using the test Docker Compose
configuration:
```bash
# CI/CD command
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml build
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
```
Exit codes:
- `0` - All tests passed
- Non-zero - Test failures occurred
Frontend UI checks should also be included in CI for the active website
workspace:
```bash
cd web/frontend
npm ci
npm run test:storybook
npm run test:storybook:playwright
```
## Troubleshooting
### Tests Failing Due to Database Connection
Ensure SQL Server is running and environment variables are set:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml ps
```
### Port Conflicts
If port 1433 is in use, stop other SQL Server instances or modify the port in
`docker-compose.test.yaml`.
### Stale Test Data
Clean up test database:
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
```
### View Container Logs
```bash
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml logs <service-name>
```
## Best Practices
1. **Isolation**: Each test should be independent and not rely on other tests
2. **Cleanup**: Use fixtures and dispose patterns for resource cleanup
3. **Mocking**: Mock external dependencies in unit tests
4. **Descriptive Names**: Use clear, descriptive test method names
5. **Arrange-Act-Assert**: Follow AAA pattern in unit tests
6. **Given-When-Then**: Follow GWT pattern in BDD scenarios

2411
misc/raw-data/beers.csv Normal file
View File

File diff suppressed because it is too large Load Diff

559
misc/raw-data/breweries.csv Normal file
View File

@@ -0,0 +1,559 @@
,name,city,state
0,NorthGate Brewing ,Minneapolis, MN
1,Against the Grain Brewery,Louisville, KY
2,Jack's Abby Craft Lagers,Framingham, MA
3,Mike Hess Brewing Company,San Diego, CA
4,Fort Point Beer Company,San Francisco, CA
5,COAST Brewing Company,Charleston, SC
6,Great Divide Brewing Company,Denver, CO
7,Tapistry Brewing,Bridgman, MI
8,Big Lake Brewing,Holland, MI
9,The Mitten Brewing Company,Grand Rapids, MI
10,Brewery Vivant,Grand Rapids, MI
11,Petoskey Brewing,Petoskey, MI
12,Blackrocks Brewery,Marquette, MI
13,Perrin Brewing Company,Comstock Park, MI
14,Witch's Hat Brewing Company,South Lyon, MI
15,Founders Brewing Company,Grand Rapids, MI
16,Flat 12 Bierwerks,Indianapolis, IN
17,Tin Man Brewing Company,Evansville, IN
18,Black Acre Brewing Co.,Indianapolis, IN
19,Brew Link Brewing,Plainfield, IN
20,Bare Hands Brewery,Granger, IN
21,Three Pints Brewing,Martinsville, IN
22,Four Fathers Brewing ,Valparaiso, IN
23,Indiana City Brewing,Indianapolis, IN
24,Burn 'Em Brewing,Michigan City, IN
25,Sun King Brewing Company,Indianapolis, IN
26,Evil Czech Brewery,Mishawaka, IN
27,450 North Brewing Company,Columbus, IN
28,Taxman Brewing Company,Bargersville, IN
29,Cedar Creek Brewery,Seven Points, TX
30,SanTan Brewing Company,Chandler, AZ
31,Boulevard Brewing Company,Kansas City, MO
32,James Page Brewing Company,Stevens Point, WI
33,The Dudes' Brewing Company,Torrance, CA
34,Ballast Point Brewing Company,San Diego, CA
35,Anchor Brewing Company,San Francisco, CA
36,Figueroa Mountain Brewing Company,Buellton, CA
37,Avery Brewing Company,Boulder, CO
38,Twisted X Brewing Company,Dripping Springs, TX
39,Gonzo's BiggDogg Brewing,Kalamazoo, MI
40,Big Muddy Brewing,Murphysboro, IL
41,Lost Nation Brewing,East Fairfield, VT
42,Rising Tide Brewing Company,Portland, ME
43,Rivertowne Brewing Company,Export, PA
44,Revolution Brewing Company,Chicago, IL
45,Tallgrass Brewing Company,Manhattan, KS
46,Sixpoint Craft Ales,Brooklyn, NY
47,White Birch Brewing,Hooksett, NH
48,Firestone Walker Brewing Company,Paso Robles, CA
49,SweetWater Brewing Company,Atlanta, GA
50,Flying Mouse Brewery,Troutville, VA
51,Upslope Brewing Company,Boulder, CO
52,Pipeworks Brewing Company,Chicago, IL
53,Bent Brewstillery,Roseville, MN
54,Flesk Brewing Company,Lombard, IL
55,Pollyanna Brewing Company,Lemont, IL
56,BuckleDown Brewing,Lyons, IL
57,Destihl Brewery,Bloomington, IL
58,Summit Brewing Company,St. Paul, MN
59,Latitude 42 Brewing Company,Portage, MI
60,4 Hands Brewing Company,Saint Louis, MO
61,Surly Brewing Company,Brooklyn Center, MN
62,Against The Grain Brewery,Louisville, KY
63,Crazy Mountain Brewing Company,Edwards, CO
64,SlapShot Brewing Company,Chicago, IL
65,Mikerphone Brewing,Chicago, IL
66,Freetail Brewing Company,San Antonio, TX
67,3 Daughters Brewing,St Petersburg, FL
68,Red Shedman Farm Brewery and Hop...,Mt. Airy, MD
69,Appalachian Mountain Brewery,Boone, NC
70,Birdsong Brewing Company,Charlotte, NC
71,Union Craft Brewing,Baltimore, MD
72,Atwater Brewery,Detroit, MI
73,Ale Asylum,Madison, WI
74,Two Brothers Brewing Company,Warrenville, IL
75,Bent Paddle Brewing Company,Duluth, MN
76,Bell's Brewery,Kalamazoo, MI
77,Blue Owl Brewing,Austin, TX
78,Speakasy Ales & Lagers,San Francisco, CA
79,Black Tooth Brewing Company,Sheridan, WY
80,Hopworks Urban Brewery,Portland, OR
81,Epic Brewing,Denver, CO
82,New Belgium Brewing Company,Fort Collins, CO
83,Sierra Nevada Brewing Company,Chico, CA
84,Keweenaw Brewing Company,Houghton, MI
85,Brewery Terra Firma,Traverse City, MI
86,Grey Sail Brewing Company,Westerly, RI
87,Kirkwood Station Brewing Company,Kirkwood, MO
88,Goose Island Brewing Company,Chicago, IL
89,Broad Brook Brewing LLC,East Windsor, CT
90,The Lion Brewery,Wilkes-Barre, PA
91,Madtree Brewing Company,Cincinnati, OH
92,Jackie O's Pub & Brewery,Athens, OH
93,Rhinegeist Brewery,Cincinnati, OH
94,Warped Wing Brewing Company,Dayton, OH
95,Blackrocks Brewery,Marquette, MA
96,Catawba Valley Brewing Company,Morganton, NC
97,Tröegs Brewing Company,Hershey, PA
98,Mission Brewery,San Diego, CA
99,Christian Moerlein Brewing Company,Cincinnati, OH
100,West Sixth Brewing,Lexington, KY
101,Coastal Extreme Brewing Company,Newport, RI
102,King Street Brewing Company,Anchorage, AK
103,Beer Works Brewery,Lowell, MA
104,Lone Tree Brewing Company,Lone Tree, CO
105,Four String Brewing Company,Columbus, OH
106,Glabrous Brewing Company,Pineland, ME
107,Bonfire Brewing Company,Eagle, CO
108,Thomas Hooker Brewing Company,Bloomfield, CT
109,"Woodstock Inn, Station & Brewery",North Woodstock, NH
110,Renegade Brewing Company,Denver, CO
111,Mother Earth Brew Company,Vista, CA
112,Black Market Brewing Company,Temecula, CA
113,Vault Brewing Company,Yardley, PA
114,Jailbreak Brewing Company,Laurel, MD
115,Smartmouth Brewing Company,Norfolk, VA
116,Base Camp Brewing Co.,Portland, OR
117,Alameda Brewing,Portland, OR
118,Southern Star Brewing Company,Conroe, TX
119,Steamworks Brewing Company,Durango, CO
120,Horny Goat Brew Pub,Milwaukee, WI
121,Cheboygan Brewing Company,Cheboygan, MI
122,Center of the Universe Brewing C...,Ashland, VA
123,Ipswich Ale Brewery,Ipswich, MA
124,Griffin Claw Brewing Company,Birmingham, MI
125,Karbach Brewing Company,Houston, TX
126,Uncle Billy's Brewery and Smokeh...,Austin, TX
127,Deep Ellum Brewing Company,Dallas, TX
128,Real Ale Brewing Company,Blanco, TX
129,Straub Brewery,St Mary's, PA
130,Shebeen Brewing Company,Wolcott, CT
131,Stevens Point Brewery,Stevens Point, WI
132,Weston Brewing Company,Weston, MO
133,Southern Prohibition Brewing Com...,Hattiesburg, MS
134,Minhas Craft Brewery,Monroe, WI
135,Pug Ryan's Brewery,Dillon, CO
136,Hops & Grains Brewing Company,Austin, TX
137,Sietsema Orchards and Cider Mill,Ada, MI
138,Summit Brewing Company,St Paul, MN
139,Core Brewing & Distilling Company,Springdale, AR
140,Independence Brewing Company,Austin, TX
141,Cigar City Brewing Company,Tampa, FL
142,Third Street Brewhouse,Cold Spring, MN
143,Narragansett Brewing Company,Providence, RI
144,Grimm Brothers Brewhouse,Loveland, CO
145,Cisco Brewers,Nantucket, MA
146,Angry Minnow,Hayward, WI
147,Platform Beer Company,Cleveland, OH
148,Odyssey Beerwerks,Arvada, CO
149,Lonerider Brewing Company,Raleigh, NC
150,Oakshire Brewing,Eugene, OR
151,Fort Pitt Brewing Company,Latrobe, PA
152,Tin Roof Brewing Company,Baton Rouge, LA
153,Three Creeks Brewing,Sisters, OR
154,2 Towns Ciderhouse,Corvallis, OR
155,Caldera Brewing Company,Ashland, OR
156,Greenbrier Valley Brewing Company,Lewisburg, WV
157,Phoenix Ale Brewery,Phoenix, AZ
158,Lumberyard Brewing Company,Flagstaff, AZ
159,Uinta Brewing Company,Salt Lake City, UT
160,Four Peaks Brewing Company,Tempe, AZ
161,Martin House Brewing Company,Fort Worth, TX
162,Right Brain Brewery,Traverse City, MI
163,Sly Fox Brewing Company,Phoenixville, PA
164,Round Guys Brewing,Lansdale, PA
165,Great Crescent Brewery,Aurora, IN
166,Oskar Blues Brewery,Longmont, CO
167,Boxcar Brewing Company,West Chester, PA
168,High Hops Brewery,Windsor, CO
169,Crooked Fence Brewing Company,Garden City, ID
170,Everybody's Brewing,White Salmon, WA
171,Anderson Valley Brewing Company,Boonville, CA
172,Fiddlehead Brewing Company,Shelburne, VT
173,Evil Twin Brewing,Brooklyn, NY
174,New Orleans Lager & Ale Brewing ...,New Orleans, LA
175,Spiteful Brewing Company,Chicago, IL
176,Rahr & Sons Brewing Company,Fort Worth, TX
177,18th Street Brewery,Gary, IN
178,Cambridge Brewing Company,Cambridge, MA
179,Carolina Brewery,Pittsboro, NC
180,Frog Level Brewing Company,Waynesville, NC
181,Wild Wolf Brewing Company,Nellysford, VA
182,COOP Ale Works,Oklahoma City, OK
183,Seventh Son Brewing Company,Columbus, OH
184,Oasis Texas Brewing Company,Austin, TX
185,Vander Mill Ciders,Spring Lake, MI
186,St. Julian Winery,Paw Paw, MI
187,Pedernales Brewing Company,Fredericksburg, TX
188,Mother's Brewing,Springfield, MO
189,Modern Monks Brewery,Lincoln, NE
190,Two Beers Brewing Company,Seattle, WA
191,Snake River Brewing Company,Jackson, WY
192,Capital Brewery,Middleton, WI
193,Anthem Brewing Company,Oklahoma City, OK
194,Goodlife Brewing Co.,Bend, OR
195,Breakside Brewery,Portland, OR
196,Goose Island Brewery Company,Chicago, IL
197,Burnside Brewing Co.,Portland, OR
198,Hop Valley Brewing Company,Springfield, OR
199,Worthy Brewing Company,Bend, OR
200,Occidental Brewing Company,Portland, OR
201,Fearless Brewing Company,Estacada, OR
202,Upland Brewing Company,Bloomington, IN
203,Mehana Brewing Co.,Hilo, HI
204,Hawai'i Nui Brewing Co.,Hilo, HI
205,People's Brewing Company,Lafayette, IN
206,Fort George Brewery,Astoria, OR
207,Branchline Brewing Company,San Antonio, TX
208,Kalona Brewing Company,Kalona, IA
209,Modern Times Beer,San Diego, CA
210,Temperance Beer Company,Evanston, IL
211,Wisconsin Brewing Company,Verona, WI
212,Crow Peak Brewing Company,Spearfish, SD
213,Grapevine Craft Brewery,Farmers Branch, TX
214,Buffalo Bayou Brewing Company,Houston, TX
215,Texian Brewing Co.,Richmond, TX
216,Orpheus Brewing,Atlanta, GA
217,Forgotten Boardwalk,Cherry Hill, NJ
218,Laughing Dog Brewing Company,Ponderay, ID
219,Bozeman Brewing Company,Bozeman, MT
220,Big Choice Brewing,Broomfield, CO
221,Big Storm Brewing Company,Odessa, FL
222,Carton Brewing Company,Atlantic Highlands, NJ
223,Midnight Sun Brewing Company,Anchorage, AK
224,Fat Head's Brewery,Middleburg Heights, OH
225,Refuge Brewery,Temecula, CA
226,Chatham Brewing,Chatham, NY
227,DC Brau Brewing Company,Washington, DC
228,Geneva Lake Brewing Company,Lake Geneva, WI
229,Rochester Mills Brewing Company,Rochester, MI
230,Cape Ann Brewing Company,Gloucester, MA
231,Borderlands Brewing Company,Tucson, AZ
232,College Street Brewhouse and Pub,Lake Havasu City, AZ
233,Joseph James Brewing Company,Henderson, NV
234,Harpoon Brewery,Boston, MA
235,Back East Brewing Company,Bloomfield, CT
236,Champion Brewing Company,Charlottesville, VA
237,Devil's Backbone Brewing Company,Lexington, VA
238,Newburgh Brewing Company,Newburgh, NY
239,Wiseacre Brewing Company,Memphis, TN
240,Golden Road Brewing,Los Angeles, CA
241,New Republic Brewing Company,College Station, TX
242,Infamous Brewing Company,Austin, TX
243,Two Henrys Brewing Company,Plant City, FL
244,Lift Bridge Brewing Company,Stillwater, MN
245,Lucky Town Brewing Company,Jackson, MS
246,Quest Brewing Company,Greenville, SC
247,Creature Comforts,Athens, GA
248,Half Full Brewery,Stamford, CT
249,Southampton Publick House,Southampton, NY
250,Chapman's Brewing,Angola, IN
251,Barrio Brewing Company,Tucson, AZ
252,Santa Cruz Mountain Brewing,Santa Cruz, CA
253,Frankenmuth Brewery,Frankenmuth, MI
254,Meckley's Cidery,Somerset Center, MI
255,Stillwater Artisanal Ales,Baltimore, MD
256,Finch's Beer Company,Chicago, IL
257,South Austin Brewery,South Austin, TX
258,Bauhaus Brew Labs,Minneapolis, MN
259,Ozark Beer Company,Rogers, AR
260,Mountain Town Brewing Company ,Mount Pleasant, MI
261,Otter Creek Brewing,Waterbury, VT
262,The Brewer's Art,Baltimore, MD
263,Denver Beer Company,Denver, CO
264,Ska Brewing Company,Durango, CO
265,Tractor Brewing Company,Albuquerque, NM
266,Peak Organic Brewing Company,Portland, ME
267,Cape Cod Beer,Hyannis, MA
268,Long Trail Brewing Company,Bridgewater Corners, VT
269,Great Raft Brewing Company,Shreveport, LA
270,Alaskan Brewing Company,Juneau, AK
271,Notch Brewing Company,Ipswich, MA
272,The Alchemist,Waterbury, VT
273,Three Notch'd Brewing Company,Charlottesville, VA
274,Portside Brewery,Cleveland, OH
275,Otter Creek Brewing,Middlebury, VT
276,Montauk Brewing Company,Montauk, NY
277,Indeed Brewing Company,Minneapolis, MN
278,Berkshire Brewing Company,South Deerfield, MA
279,Foolproof Brewing Company,Pawtucket, RI
280,Headlands Brewing Company,Mill Valley, CA
281,Bolero Snort Brewery,Ridgefield Park, NJ
282,Thunderhead Brewing Company,Kearney, NE
283,Defiance Brewing Company,Hays, KS
284,Milwaukee Brewing Company,Milwaukee, WI
285,Catawba Island Brewing,Port Clinton, OH
286,Back Forty Beer Company,Gadsden, AL
287,Four Corners Brewing Company,Dallas, TX
288,Saint Archer Brewery,San Diego, CA
289,Rogue Ales,Newport, OR
290,Hale's Ales,Seattle, WA
291,Tommyknocker Brewery,Idaho Springs, CO
292,Baxter Brewing Company,Lewiston, ME
293,Northampton Brewery,Northamtpon, MA
294,Black Shirt Brewing Company,Denver, CO
295,Wachusett Brewing Company,Westminster, MA
296,Widmer Brothers Brewing Company,Portland, OR
297,Hop Farm Brewing Company,Pittsburgh, PA
298,Liquid Hero Brewery,York, PA
299,Matt Brewing Company,Utica, NY
300,Boston Beer Company,Boston, MA
301,Old Forge Brewing Company,Danville, PA
302,Utah Brewers Cooperative,Salt Lake City, UT
303,Magic Hat Brewing Company,South Burlington, VT
304,Blue Hills Brewery,Canton, MA
305,Night Shift Brewing,Everett, MA
306,Beach Brewing Company,Virginia Beach, VA
307,Payette Brewing Company,Garden City, ID
308,Brew Bus Brewing,Tampa, FL
309,Sockeye Brewing Company,Boise, ID
310,Pine Street Brewery,San Francisco, CA
311,Dirty Bucket Brewing Company,Woodinville, WA
312,Jackalope Brewing Company,Nashville, TN
313,Slanted Rock Brewing Company,Meridian, ID
314,Piney River Brewing Company,Bucryus, MO
315,Cutters Brewing Company,Avon, IN
316,Iron Hill Brewery & Restaurant,Wilmington, DE
317,Marshall Wharf Brewing Company,Belfast, ME
318,Banner Beer Company,Williamsburg, MA
319,Dick's Brewing Company,Centralia, WA
320,Claremont Craft Ales,Claremont, CA
321,Rivertown Brewing Company,Lockland, OH
322,Voodoo Brewery,Meadville, PA
323,D.L. Geary Brewing Company,Portland, ME
324,Pisgah Brewing Company,Black Mountain, NC
325,Neshaminy Creek Brewing Company,Croydon, PA
326,Morgan Street Brewery,Saint Louis, MO
327,Half Acre Beer Company,Chicago, IL
328,The Just Beer Project,Burlington, VT
329,The Bronx Brewery,Bronx, NY
330,Dead Armadillo Craft Brewing,Tulsa, OK
331,Catawba Brewing Company,Morganton, NC
332,La Cumbre Brewing Company,Albuquerque, NM
333,David's Ale Works,Diamond Springs, CA
334,The Traveler Beer Company,Burlington, VT
335,Fargo Brewing Company,Fargo, ND
336,Big Sky Brewing Company,Missoula, MT
337,Nebraska Brewing Company,Papillion, NE
338,Uncle John's Fruit House Winery,St. John's, MI
339,Wormtown Brewery,Worcester, MA
340,Due South Brewing Company,Boynton Beach, FL
341,Palisade Brewing Company,Palisade, CO
342,KelSo Beer Company,Brooklyn, NY
343,Hardywood Park Craft Brewery,Richmond, VA
344,Wolf Hills Brewing Company,Abingdon, VA
345,Lavery Brewing Company,Erie, PA
346,Manzanita Brewing Company,Santee, CA
347,Fullsteam Brewery,Durham, NC
348,Four Horsemen Brewing Company,South Bend, IN
349,Hinterland Brewery,Green Bay, WI
350,Central Coast Brewing Company,San Luis Obispo, CA
351,Westfield River Brewing Company,Westfield, MA
352,Elevator Brewing Company,Columbus, OH
353,Aslan Brewing Company,Bellingham, WA
354,Kulshan Brewery,Bellingham, WA
355,Pikes Peak Brewing Company,Monument, CO
356,Manayunk Brewing Company,Philadelphia, PA
357,Buckeye Brewing,Cleveland, OH
358,Daredevil Brewing Company,Shelbyville, IN
359,NoDa Brewing Company,Charlotte, NC
360,Aviator Brewing Company,Fuquay-Varina, NC
361,Wild Onion Brewing Company,Lake Barrington, IL
362,Hilliard's Beer,Seattle, WA
363,Mikkeller,Pottstown, PA
364,Bohemian Brewery,Midvale, UT
365,Great River Brewery,Davenport, IA
366,Mustang Brewing Company,Mustang, OK
367,Airways Brewing Company,Kent, WA
368,21st Amendment Brewery,San Francisco, CA
369,Eddyline Brewery & Restaurant,Buena Vista, CO
370,Pizza Port Brewing Company,Carlsbad, CA
371,Sly Fox Brewing Company,Pottstown, PA
372,Spring House Brewing Company,Conestoga, PA
373,7venth Sun,Dunedin, FL
374,Astoria Brewing Company,Astoria, OR
375,Maui Brewing Company,Lahaina, HI
376,RoughTail Brewing Company,Midwest City, OK
377,Lucette Brewing Company,Menominee, WI
378,Bold City Brewery,Jacksonville, FL
379,Grey Sail Brewing of Rhode Island,Westerly, RI
380,Blue Blood Brewing Company,Lincoln, NE
381,Swashbuckler Brewing Company,Manheim, PA
382,Blue Mountain Brewery,Afton, VA
383,Starr Hill Brewery,Crozet, VA
384,Westbrook Brewing Company,Mt. Pleasant, SC
385,Shipyard Brewing Company,Portland, ME
386,Revolution Brewing,Paonia, CO
387,Natian Brewery,Portland, OR
388,Alltech's Lexington Brewing Company,Lexington, KY
389,Oskar Blues Brewery (North Carol...,Brevard, NC
390,Orlison Brewing Company,Airway Heights, WA
391,Breckenridge Brewery,Denver, CO
392,Santa Fe Brewing Company,Santa Fe, NM
393,Miami Brewing Company,Miami, FL
394,Schilling & Company,Seattle, WA
395,Hops & Grain Brewery,Austin, TX
396,White Flame Brewing Company,Hudsonville, MI
397,Ruhstaller Beer Company,Sacramento, CA
398,Saugatuck Brewing Company,Douglas, MI
399,Moab Brewery,Moab, UT
400,Macon Beer Company,Macon, GA
401,Amnesia Brewing Company,Washougal, WA
402,Wolverine State Brewing Company,Ann Arbor, MI
403,Red Tank Cider Company,Bend, OR
404,Cascadia Ciderworks United,Portland, OR
405,Fate Brewing Company,Boulder, CO
406,Lazy Monk Brewing,Eau Claire, WI
407,Bitter Root Brewing,Hamilton, MT
408,10 Barrel Brewing Company,Bend, OR
409,Tamarack Brewing Company,Lakeside, MT
410,New England Brewing Company,Woodbridge, CT
411,Seattle Cider Company,Seattle, WA
412,Straight to Ale,Huntsville, AL
413,Austin Beerworks,Austin, TX
414,Blue Mountain Brewery,Arrington, VA
415,Coastal Empire Beer Company,Savannah, GA
416,Jack's Hard Cider (Hauser Estate...,Biglerville, PA
417,Boulder Beer Company,Boulder, CO
418,Coalition Brewing Company,Portland, OR
419,Sanitas Brewing Company,Boulder, CO
420,Gore Range Brewery,Edwards, CO
421,Redstone Meadery,Boulder, CO
422,Blue Dog Mead,Eugene, OR
423,Hess Brewing Company,San Diego, CA
424,Wynkoop Brewing Company,Denver, CO
425,Ciderboys,Stevens Point, WI
426,Armadillo Ale Works,Denton, TX
427,Roanoke Railhouse Brewery,Roanoke, VA
428,Schlafly Brewing Company,Saint Louis, MO
429,Asher Brewing Company,Boulder, CO
430,Lost Rhino Brewing Company,Ashburn, VA
431,North Country Brewing Company,Slippery Rock, PA
432,Seabright Brewery,Santa Cruz, CA
433,French Broad Brewery,Asheville, NC
434,Angry Orchard Cider Company,Cincinnati, OH
435,Two Roads Brewing Company,Stratford, CT
436,Southern Oregon Brewing Company,Medford, OR
437,Brooklyn Brewery,Brooklyn, NY
438,The Right Brain Brewery,Traverse City, MI
439,Kona Brewing Company,Kona, HI
440,MillKing It Productions,Royal Oak, MI
441,Pateros Creek Brewing Company,Fort Collins, CO
442,O'Fallon Brewery,O'Fallon, MO
443,Marble Brewery,Albuquerque, NM
444,Big Wood Brewery,Vadnais Heights, MN
445,Howard Brewing Company,Lenoir, NC
446,Downeast Cider House,Leominster, MA
447,Swamp Head Brewery,Gainesville, FL
448,Mavericks Beer Company,Half Moon Bay, CA
449,TailGate Beer,San Diego, CA
450,Northwest Brewing Company,Pacific, WA
451,Dad & Dude's Breweria,Aurora, CO
452,Centennial Beer Company,Edwards, CO
453,Denali Brewing Company,Talkeetna, AK
454,Deschutes Brewery,Bend, OR
455,Sunken City Brewing Company,Hardy, VA
456,Lucette Brewing Company,Menominie, WI
457,The Black Tooth Brewing Company,Sheridan, WY
458,Kenai River Brewing Company,Soldotna, AK
459,River North Brewery,Denver, CO
460,Fremont Brewing Company,Seattle, WA
461,Armstrong Brewing Company,South San Francisco, CA
462,AC Golden Brewing Company,Golden, CO
463,Big Bend Brewing Company,Alpine, TX
464,Good Life Brewing Company,Bend, OR
465,Engine 15 Brewing,Jacksonville Beach, FL
466,Green Room Brewing,Jacksonville, FL
467,Brindle Dog Brewing Company,Tampa Bay, FL
468,Peace Tree Brewing Company,Knoxville, IA
469,Terrapin Brewing Company,Athens, GA
470,Pete's Brewing Company,San Antonio, TX
471,Okoboji Brewing Company,Spirit Lake, IA
472,Crystal Springs Brewing Company,Boulder, CO
473,Engine House 9,Tacoma, WA
474,Tonka Beer Company,Minnetonka, MN
475,Red Hare Brewing Company,Marietta, GA
476,Hangar 24 Craft Brewery,Redlands, CA
477,Big Elm Brewing,Sheffield, MA
478,Good People Brewing Company,Birmingham, AL
479,Heavy Seas Beer,Halethorpe, MD
480,Telluride Brewing Company,Telluride, CO
481,7 Seas Brewing Company,Gig Harbor, WA
482,Confluence Brewing Company,Des Moines, IA
483,Bale Breaker Brewing Company,Yakima, WA
484,The Manhattan Brewing Company,New York, NY
485,MacTarnahans Brewing Company,Portland, OR
486,Stillmank Beer Company,Green Bay, WI
487,Redhook Brewery,Woodinville, WA
488,Dock Street Brewery,Philadelphia, PA
489,Blue Point Brewing Company,Patchogue, NY
490,Tampa Bay Brewing Company,Tampa, FL
491,Devil's Canyon Brewery,Belmont, CA
492,Stone Coast Brewing Company,Portland, ME
493,Broken Tooth Brewing Company,Anchorage, AK
494,Seven Brides Brewery,Silverton, OR
495,Newburyport Brewing Company,Newburyport, MA
496,Dry Dock Brewing Company,Aurora, CO
497,Cans Bar and Canteen,Charlotte, NC
498,Sprecher Brewing Company,Glendale, WI
499,Wildwood Brewing Company,Stevensville, MT
500,High Noon Saloon And Brewery,Leavenworth, KS
501,Woodchuck Hard Cider,Middlebury, VT
502,Sea Dog Brewing Company,Portland, ME
503,Oskar Blues Brewery,Lyons, CO
504,Carolina Beer & Beverage,Mooresville, NC
505,Krebs Brewing Company (Pete's Pl...,Krebs, OK
506,Warbird Brewing Company,Fort Wayne, IN
507,Mudshark Brewing Company,Lake Havasu City, AZ
508,Spilker Ales,Cortland, NE
509,Wingman Brewers,Tacoma, WA
510,Kettle House Brewing Company,Missoula, MT
511,Sherwood Forest Brewers,Marlborough, MA
512,Cottrell Brewing,Pawcatuck, CT
513,Arctic Craft Brewery,Colorado Springs, CO
514,Monkey Paw Pub & Brewery,San Diego, CA
515,Crabtree Brewing Company,Greeley, CO
516,Emerald City Beer Company,Seattle, WA
517,Butcher's Brewing,Carlsbad, CA
518,New South Brewing Company,Myrtle Beach, SC
519,Big River Brewing Company,Chattanooga, TN
520,Twisted Pine Brewing Company,Boulder, CO
521,Flying Dog Brewery,Frederick, MD
522,Uncommon Brewers,Santa Cruz, CA
523,Aspen Brewing Company,Aspen, CO
524,Triangle Brewing Company,Durham, NC
525,Bomb Beer Company,New York, NY
526,Churchkey Can Company,Seattle, WA
527,Intuition Ale Works,Jacksonville, FL
528,Asheville Brewing Company,Asheville, NC
529,Northwoods Brewpub,Eau Claire, WI
530,Buckbean Brewing Company,Reno, NV
531,Dolores River Brewery,Dolores, CO
532,Flat Rock Brewing Company,Smithton, PA
533,Abita Brewing Company,Abita Springs, LA
534,Mammoth Brewing Company,Mammoth Lakes, CA
535,Harvest Moon Brewing Company,Belt, MT
536,Grand Canyon Brewing Company,Williams, AZ
537,Lewis and Clark Brewing Company,Helena, MT
538,Dundee Brewing Company,Rochester, NY
539,Twin Lakes Brewing Company,Greenville, DE
540,Mother Earth Brewing Company,Kinston, NC
541,Arcadia Brewing Company,Battle Creek, MI
542,Angry Minnow Brewing Company,Hayward, WI
543,Great Northern Brewing Company,Whitefish, MT
544,Pyramid Breweries,Seattle, WA
545,Lancaster Brewing Company,Lancaster, PA
546,Upstate Brewing Company,Elmira, NY
547,Moat Mountain Smoke House & Brew...,North Conway, NH
548,Prescott Brewing Company,Prescott, AZ
549,Mogollon Brewing Company,Flagstaff, AZ
550,Wind River Brewing Company,Pinedale, WY
551,Silverton Brewery,Silverton, CO
552,Mickey Finn's Brewery,Libertyville, IL
553,Covington Brewhouse,Covington, LA
554,Dave's Brewfarm,Wilson, WI
555,Ukiah Brewing Company,Ukiah, CA
556,Butternuts Beer and Ale,Garrattsville, NY
557,Sleeping Lady Brewing Company,Anchorage, AK
1 name city state
2 0 NorthGate Brewing Minneapolis MN
3 1 Against the Grain Brewery Louisville KY
4 2 Jack's Abby Craft Lagers Framingham MA
5 3 Mike Hess Brewing Company San Diego CA
6 4 Fort Point Beer Company San Francisco CA
7 5 COAST Brewing Company Charleston SC
8 6 Great Divide Brewing Company Denver CO
9 7 Tapistry Brewing Bridgman MI
10 8 Big Lake Brewing Holland MI
11 9 The Mitten Brewing Company Grand Rapids MI
12 10 Brewery Vivant Grand Rapids MI
13 11 Petoskey Brewing Petoskey MI
14 12 Blackrocks Brewery Marquette MI
15 13 Perrin Brewing Company Comstock Park MI
16 14 Witch's Hat Brewing Company South Lyon MI
17 15 Founders Brewing Company Grand Rapids MI
18 16 Flat 12 Bierwerks Indianapolis IN
19 17 Tin Man Brewing Company Evansville IN
20 18 Black Acre Brewing Co. Indianapolis IN
21 19 Brew Link Brewing Plainfield IN
22 20 Bare Hands Brewery Granger IN
23 21 Three Pints Brewing Martinsville IN
24 22 Four Fathers Brewing Valparaiso IN
25 23 Indiana City Brewing Indianapolis IN
26 24 Burn 'Em Brewing Michigan City IN
27 25 Sun King Brewing Company Indianapolis IN
28 26 Evil Czech Brewery Mishawaka IN
29 27 450 North Brewing Company Columbus IN
30 28 Taxman Brewing Company Bargersville IN
31 29 Cedar Creek Brewery Seven Points TX
32 30 SanTan Brewing Company Chandler AZ
33 31 Boulevard Brewing Company Kansas City MO
34 32 James Page Brewing Company Stevens Point WI
35 33 The Dudes' Brewing Company Torrance CA
36 34 Ballast Point Brewing Company San Diego CA
37 35 Anchor Brewing Company San Francisco CA
38 36 Figueroa Mountain Brewing Company Buellton CA
39 37 Avery Brewing Company Boulder CO
40 38 Twisted X Brewing Company Dripping Springs TX
41 39 Gonzo's BiggDogg Brewing Kalamazoo MI
42 40 Big Muddy Brewing Murphysboro IL
43 41 Lost Nation Brewing East Fairfield VT
44 42 Rising Tide Brewing Company Portland ME
45 43 Rivertowne Brewing Company Export PA
46 44 Revolution Brewing Company Chicago IL
47 45 Tallgrass Brewing Company Manhattan KS
48 46 Sixpoint Craft Ales Brooklyn NY
49 47 White Birch Brewing Hooksett NH
50 48 Firestone Walker Brewing Company Paso Robles CA
51 49 SweetWater Brewing Company Atlanta GA
52 50 Flying Mouse Brewery Troutville VA
53 51 Upslope Brewing Company Boulder CO
54 52 Pipeworks Brewing Company Chicago IL
55 53 Bent Brewstillery Roseville MN
56 54 Flesk Brewing Company Lombard IL
57 55 Pollyanna Brewing Company Lemont IL
58 56 BuckleDown Brewing Lyons IL
59 57 Destihl Brewery Bloomington IL
60 58 Summit Brewing Company St. Paul MN
61 59 Latitude 42 Brewing Company Portage MI
62 60 4 Hands Brewing Company Saint Louis MO
63 61 Surly Brewing Company Brooklyn Center MN
64 62 Against The Grain Brewery Louisville KY
65 63 Crazy Mountain Brewing Company Edwards CO
66 64 SlapShot Brewing Company Chicago IL
67 65 Mikerphone Brewing Chicago IL
68 66 Freetail Brewing Company San Antonio TX
69 67 3 Daughters Brewing St Petersburg FL
70 68 Red Shedman Farm Brewery and Hop... Mt. Airy MD
71 69 Appalachian Mountain Brewery Boone NC
72 70 Birdsong Brewing Company Charlotte NC
73 71 Union Craft Brewing Baltimore MD
74 72 Atwater Brewery Detroit MI
75 73 Ale Asylum Madison WI
76 74 Two Brothers Brewing Company Warrenville IL
77 75 Bent Paddle Brewing Company Duluth MN
78 76 Bell's Brewery Kalamazoo MI
79 77 Blue Owl Brewing Austin TX
80 78 Speakasy Ales & Lagers San Francisco CA
81 79 Black Tooth Brewing Company Sheridan WY
82 80 Hopworks Urban Brewery Portland OR
83 81 Epic Brewing Denver CO
84 82 New Belgium Brewing Company Fort Collins CO
85 83 Sierra Nevada Brewing Company Chico CA
86 84 Keweenaw Brewing Company Houghton MI
87 85 Brewery Terra Firma Traverse City MI
88 86 Grey Sail Brewing Company Westerly RI
89 87 Kirkwood Station Brewing Company Kirkwood MO
90 88 Goose Island Brewing Company Chicago IL
91 89 Broad Brook Brewing LLC East Windsor CT
92 90 The Lion Brewery Wilkes-Barre PA
93 91 Madtree Brewing Company Cincinnati OH
94 92 Jackie O's Pub & Brewery Athens OH
95 93 Rhinegeist Brewery Cincinnati OH
96 94 Warped Wing Brewing Company Dayton OH
97 95 Blackrocks Brewery Marquette MA
98 96 Catawba Valley Brewing Company Morganton NC
99 97 Tröegs Brewing Company Hershey PA
100 98 Mission Brewery San Diego CA
101 99 Christian Moerlein Brewing Company Cincinnati OH
102 100 West Sixth Brewing Lexington KY
103 101 Coastal Extreme Brewing Company Newport RI
104 102 King Street Brewing Company Anchorage AK
105 103 Beer Works Brewery Lowell MA
106 104 Lone Tree Brewing Company Lone Tree CO
107 105 Four String Brewing Company Columbus OH
108 106 Glabrous Brewing Company Pineland ME
109 107 Bonfire Brewing Company Eagle CO
110 108 Thomas Hooker Brewing Company Bloomfield CT
111 109 Woodstock Inn, Station & Brewery North Woodstock NH
112 110 Renegade Brewing Company Denver CO
113 111 Mother Earth Brew Company Vista CA
114 112 Black Market Brewing Company Temecula CA
115 113 Vault Brewing Company Yardley PA
116 114 Jailbreak Brewing Company Laurel MD
117 115 Smartmouth Brewing Company Norfolk VA
118 116 Base Camp Brewing Co. Portland OR
119 117 Alameda Brewing Portland OR
120 118 Southern Star Brewing Company Conroe TX
121 119 Steamworks Brewing Company Durango CO
122 120 Horny Goat Brew Pub Milwaukee WI
123 121 Cheboygan Brewing Company Cheboygan MI
124 122 Center of the Universe Brewing C... Ashland VA
125 123 Ipswich Ale Brewery Ipswich MA
126 124 Griffin Claw Brewing Company Birmingham MI
127 125 Karbach Brewing Company Houston TX
128 126 Uncle Billy's Brewery and Smokeh... Austin TX
129 127 Deep Ellum Brewing Company Dallas TX
130 128 Real Ale Brewing Company Blanco TX
131 129 Straub Brewery St Mary's PA
132 130 Shebeen Brewing Company Wolcott CT
133 131 Stevens Point Brewery Stevens Point WI
134 132 Weston Brewing Company Weston MO
135 133 Southern Prohibition Brewing Com... Hattiesburg MS
136 134 Minhas Craft Brewery Monroe WI
137 135 Pug Ryan's Brewery Dillon CO
138 136 Hops & Grains Brewing Company Austin TX
139 137 Sietsema Orchards and Cider Mill Ada MI
140 138 Summit Brewing Company St Paul MN
141 139 Core Brewing & Distilling Company Springdale AR
142 140 Independence Brewing Company Austin TX
143 141 Cigar City Brewing Company Tampa FL
144 142 Third Street Brewhouse Cold Spring MN
145 143 Narragansett Brewing Company Providence RI
146 144 Grimm Brothers Brewhouse Loveland CO
147 145 Cisco Brewers Nantucket MA
148 146 Angry Minnow Hayward WI
149 147 Platform Beer Company Cleveland OH
150 148 Odyssey Beerwerks Arvada CO
151 149 Lonerider Brewing Company Raleigh NC
152 150 Oakshire Brewing Eugene OR
153 151 Fort Pitt Brewing Company Latrobe PA
154 152 Tin Roof Brewing Company Baton Rouge LA
155 153 Three Creeks Brewing Sisters OR
156 154 2 Towns Ciderhouse Corvallis OR
157 155 Caldera Brewing Company Ashland OR
158 156 Greenbrier Valley Brewing Company Lewisburg WV
159 157 Phoenix Ale Brewery Phoenix AZ
160 158 Lumberyard Brewing Company Flagstaff AZ
161 159 Uinta Brewing Company Salt Lake City UT
162 160 Four Peaks Brewing Company Tempe AZ
163 161 Martin House Brewing Company Fort Worth TX
164 162 Right Brain Brewery Traverse City MI
165 163 Sly Fox Brewing Company Phoenixville PA
166 164 Round Guys Brewing Lansdale PA
167 165 Great Crescent Brewery Aurora IN
168 166 Oskar Blues Brewery Longmont CO
169 167 Boxcar Brewing Company West Chester PA
170 168 High Hops Brewery Windsor CO
171 169 Crooked Fence Brewing Company Garden City ID
172 170 Everybody's Brewing White Salmon WA
173 171 Anderson Valley Brewing Company Boonville CA
174 172 Fiddlehead Brewing Company Shelburne VT
175 173 Evil Twin Brewing Brooklyn NY
176 174 New Orleans Lager & Ale Brewing ... New Orleans LA
177 175 Spiteful Brewing Company Chicago IL
178 176 Rahr & Sons Brewing Company Fort Worth TX
179 177 18th Street Brewery Gary IN
180 178 Cambridge Brewing Company Cambridge MA
181 179 Carolina Brewery Pittsboro NC
182 180 Frog Level Brewing Company Waynesville NC
183 181 Wild Wolf Brewing Company Nellysford VA
184 182 COOP Ale Works Oklahoma City OK
185 183 Seventh Son Brewing Company Columbus OH
186 184 Oasis Texas Brewing Company Austin TX
187 185 Vander Mill Ciders Spring Lake MI
188 186 St. Julian Winery Paw Paw MI
189 187 Pedernales Brewing Company Fredericksburg TX
190 188 Mother's Brewing Springfield MO
191 189 Modern Monks Brewery Lincoln NE
192 190 Two Beers Brewing Company Seattle WA
193 191 Snake River Brewing Company Jackson WY
194 192 Capital Brewery Middleton WI
195 193 Anthem Brewing Company Oklahoma City OK
196 194 Goodlife Brewing Co. Bend OR
197 195 Breakside Brewery Portland OR
198 196 Goose Island Brewery Company Chicago IL
199 197 Burnside Brewing Co. Portland OR
200 198 Hop Valley Brewing Company Springfield OR
201 199 Worthy Brewing Company Bend OR
202 200 Occidental Brewing Company Portland OR
203 201 Fearless Brewing Company Estacada OR
204 202 Upland Brewing Company Bloomington IN
205 203 Mehana Brewing Co. Hilo HI
206 204 Hawai'i Nui Brewing Co. Hilo HI
207 205 People's Brewing Company Lafayette IN
208 206 Fort George Brewery Astoria OR
209 207 Branchline Brewing Company San Antonio TX
210 208 Kalona Brewing Company Kalona IA
211 209 Modern Times Beer San Diego CA
212 210 Temperance Beer Company Evanston IL
213 211 Wisconsin Brewing Company Verona WI
214 212 Crow Peak Brewing Company Spearfish SD
215 213 Grapevine Craft Brewery Farmers Branch TX
216 214 Buffalo Bayou Brewing Company Houston TX
217 215 Texian Brewing Co. Richmond TX
218 216 Orpheus Brewing Atlanta GA
219 217 Forgotten Boardwalk Cherry Hill NJ
220 218 Laughing Dog Brewing Company Ponderay ID
221 219 Bozeman Brewing Company Bozeman MT
222 220 Big Choice Brewing Broomfield CO
223 221 Big Storm Brewing Company Odessa FL
224 222 Carton Brewing Company Atlantic Highlands NJ
225 223 Midnight Sun Brewing Company Anchorage AK
226 224 Fat Head's Brewery Middleburg Heights OH
227 225 Refuge Brewery Temecula CA
228 226 Chatham Brewing Chatham NY
229 227 DC Brau Brewing Company Washington DC
230 228 Geneva Lake Brewing Company Lake Geneva WI
231 229 Rochester Mills Brewing Company Rochester MI
232 230 Cape Ann Brewing Company Gloucester MA
233 231 Borderlands Brewing Company Tucson AZ
234 232 College Street Brewhouse and Pub Lake Havasu City AZ
235 233 Joseph James Brewing Company Henderson NV
236 234 Harpoon Brewery Boston MA
237 235 Back East Brewing Company Bloomfield CT
238 236 Champion Brewing Company Charlottesville VA
239 237 Devil's Backbone Brewing Company Lexington VA
240 238 Newburgh Brewing Company Newburgh NY
241 239 Wiseacre Brewing Company Memphis TN
242 240 Golden Road Brewing Los Angeles CA
243 241 New Republic Brewing Company College Station TX
244 242 Infamous Brewing Company Austin TX
245 243 Two Henrys Brewing Company Plant City FL
246 244 Lift Bridge Brewing Company Stillwater MN
247 245 Lucky Town Brewing Company Jackson MS
248 246 Quest Brewing Company Greenville SC
249 247 Creature Comforts Athens GA
250 248 Half Full Brewery Stamford CT
251 249 Southampton Publick House Southampton NY
252 250 Chapman's Brewing Angola IN
253 251 Barrio Brewing Company Tucson AZ
254 252 Santa Cruz Mountain Brewing Santa Cruz CA
255 253 Frankenmuth Brewery Frankenmuth MI
256 254 Meckley's Cidery Somerset Center MI
257 255 Stillwater Artisanal Ales Baltimore MD
258 256 Finch's Beer Company Chicago IL
259 257 South Austin Brewery South Austin TX
260 258 Bauhaus Brew Labs Minneapolis MN
261 259 Ozark Beer Company Rogers AR
262 260 Mountain Town Brewing Company Mount Pleasant MI
263 261 Otter Creek Brewing Waterbury VT
264 262 The Brewer's Art Baltimore MD
265 263 Denver Beer Company Denver CO
266 264 Ska Brewing Company Durango CO
267 265 Tractor Brewing Company Albuquerque NM
268 266 Peak Organic Brewing Company Portland ME
269 267 Cape Cod Beer Hyannis MA
270 268 Long Trail Brewing Company Bridgewater Corners VT
271 269 Great Raft Brewing Company Shreveport LA
272 270 Alaskan Brewing Company Juneau AK
273 271 Notch Brewing Company Ipswich MA
274 272 The Alchemist Waterbury VT
275 273 Three Notch'd Brewing Company Charlottesville VA
276 274 Portside Brewery Cleveland OH
277 275 Otter Creek Brewing Middlebury VT
278 276 Montauk Brewing Company Montauk NY
279 277 Indeed Brewing Company Minneapolis MN
280 278 Berkshire Brewing Company South Deerfield MA
281 279 Foolproof Brewing Company Pawtucket RI
282 280 Headlands Brewing Company Mill Valley CA
283 281 Bolero Snort Brewery Ridgefield Park NJ
284 282 Thunderhead Brewing Company Kearney NE
285 283 Defiance Brewing Company Hays KS
286 284 Milwaukee Brewing Company Milwaukee WI
287 285 Catawba Island Brewing Port Clinton OH
288 286 Back Forty Beer Company Gadsden AL
289 287 Four Corners Brewing Company Dallas TX
290 288 Saint Archer Brewery San Diego CA
291 289 Rogue Ales Newport OR
292 290 Hale's Ales Seattle WA
293 291 Tommyknocker Brewery Idaho Springs CO
294 292 Baxter Brewing Company Lewiston ME
295 293 Northampton Brewery Northamtpon MA
296 294 Black Shirt Brewing Company Denver CO
297 295 Wachusett Brewing Company Westminster MA
298 296 Widmer Brothers Brewing Company Portland OR
299 297 Hop Farm Brewing Company Pittsburgh PA
300 298 Liquid Hero Brewery York PA
301 299 Matt Brewing Company Utica NY
302 300 Boston Beer Company Boston MA
303 301 Old Forge Brewing Company Danville PA
304 302 Utah Brewers Cooperative Salt Lake City UT
305 303 Magic Hat Brewing Company South Burlington VT
306 304 Blue Hills Brewery Canton MA
307 305 Night Shift Brewing Everett MA
308 306 Beach Brewing Company Virginia Beach VA
309 307 Payette Brewing Company Garden City ID
310 308 Brew Bus Brewing Tampa FL
311 309 Sockeye Brewing Company Boise ID
312 310 Pine Street Brewery San Francisco CA
313 311 Dirty Bucket Brewing Company Woodinville WA
314 312 Jackalope Brewing Company Nashville TN
315 313 Slanted Rock Brewing Company Meridian ID
316 314 Piney River Brewing Company Bucryus MO
317 315 Cutters Brewing Company Avon IN
318 316 Iron Hill Brewery & Restaurant Wilmington DE
319 317 Marshall Wharf Brewing Company Belfast ME
320 318 Banner Beer Company Williamsburg MA
321 319 Dick's Brewing Company Centralia WA
322 320 Claremont Craft Ales Claremont CA
323 321 Rivertown Brewing Company Lockland OH
324 322 Voodoo Brewery Meadville PA
325 323 D.L. Geary Brewing Company Portland ME
326 324 Pisgah Brewing Company Black Mountain NC
327 325 Neshaminy Creek Brewing Company Croydon PA
328 326 Morgan Street Brewery Saint Louis MO
329 327 Half Acre Beer Company Chicago IL
330 328 The Just Beer Project Burlington VT
331 329 The Bronx Brewery Bronx NY
332 330 Dead Armadillo Craft Brewing Tulsa OK
333 331 Catawba Brewing Company Morganton NC
334 332 La Cumbre Brewing Company Albuquerque NM
335 333 David's Ale Works Diamond Springs CA
336 334 The Traveler Beer Company Burlington VT
337 335 Fargo Brewing Company Fargo ND
338 336 Big Sky Brewing Company Missoula MT
339 337 Nebraska Brewing Company Papillion NE
340 338 Uncle John's Fruit House Winery St. John's MI
341 339 Wormtown Brewery Worcester MA
342 340 Due South Brewing Company Boynton Beach FL
343 341 Palisade Brewing Company Palisade CO
344 342 KelSo Beer Company Brooklyn NY
345 343 Hardywood Park Craft Brewery Richmond VA
346 344 Wolf Hills Brewing Company Abingdon VA
347 345 Lavery Brewing Company Erie PA
348 346 Manzanita Brewing Company Santee CA
349 347 Fullsteam Brewery Durham NC
350 348 Four Horsemen Brewing Company South Bend IN
351 349 Hinterland Brewery Green Bay WI
352 350 Central Coast Brewing Company San Luis Obispo CA
353 351 Westfield River Brewing Company Westfield MA
354 352 Elevator Brewing Company Columbus OH
355 353 Aslan Brewing Company Bellingham WA
356 354 Kulshan Brewery Bellingham WA
357 355 Pikes Peak Brewing Company Monument CO
358 356 Manayunk Brewing Company Philadelphia PA
359 357 Buckeye Brewing Cleveland OH
360 358 Daredevil Brewing Company Shelbyville IN
361 359 NoDa Brewing Company Charlotte NC
362 360 Aviator Brewing Company Fuquay-Varina NC
363 361 Wild Onion Brewing Company Lake Barrington IL
364 362 Hilliard's Beer Seattle WA
365 363 Mikkeller Pottstown PA
366 364 Bohemian Brewery Midvale UT
367 365 Great River Brewery Davenport IA
368 366 Mustang Brewing Company Mustang OK
369 367 Airways Brewing Company Kent WA
370 368 21st Amendment Brewery San Francisco CA
371 369 Eddyline Brewery & Restaurant Buena Vista CO
372 370 Pizza Port Brewing Company Carlsbad CA
373 371 Sly Fox Brewing Company Pottstown PA
374 372 Spring House Brewing Company Conestoga PA
375 373 7venth Sun Dunedin FL
376 374 Astoria Brewing Company Astoria OR
377 375 Maui Brewing Company Lahaina HI
378 376 RoughTail Brewing Company Midwest City OK
379 377 Lucette Brewing Company Menominee WI
380 378 Bold City Brewery Jacksonville FL
381 379 Grey Sail Brewing of Rhode Island Westerly RI
382 380 Blue Blood Brewing Company Lincoln NE
383 381 Swashbuckler Brewing Company Manheim PA
384 382 Blue Mountain Brewery Afton VA
385 383 Starr Hill Brewery Crozet VA
386 384 Westbrook Brewing Company Mt. Pleasant SC
387 385 Shipyard Brewing Company Portland ME
388 386 Revolution Brewing Paonia CO
389 387 Natian Brewery Portland OR
390 388 Alltech's Lexington Brewing Company Lexington KY
391 389 Oskar Blues Brewery (North Carol... Brevard NC
392 390 Orlison Brewing Company Airway Heights WA
393 391 Breckenridge Brewery Denver CO
394 392 Santa Fe Brewing Company Santa Fe NM
395 393 Miami Brewing Company Miami FL
396 394 Schilling & Company Seattle WA
397 395 Hops & Grain Brewery Austin TX
398 396 White Flame Brewing Company Hudsonville MI
399 397 Ruhstaller Beer Company Sacramento CA
400 398 Saugatuck Brewing Company Douglas MI
401 399 Moab Brewery Moab UT
402 400 Macon Beer Company Macon GA
403 401 Amnesia Brewing Company Washougal WA
404 402 Wolverine State Brewing Company Ann Arbor MI
405 403 Red Tank Cider Company Bend OR
406 404 Cascadia Ciderworks United Portland OR
407 405 Fate Brewing Company Boulder CO
408 406 Lazy Monk Brewing Eau Claire WI
409 407 Bitter Root Brewing Hamilton MT
410 408 10 Barrel Brewing Company Bend OR
411 409 Tamarack Brewing Company Lakeside MT
412 410 New England Brewing Company Woodbridge CT
413 411 Seattle Cider Company Seattle WA
414 412 Straight to Ale Huntsville AL
415 413 Austin Beerworks Austin TX
416 414 Blue Mountain Brewery Arrington VA
417 415 Coastal Empire Beer Company Savannah GA
418 416 Jack's Hard Cider (Hauser Estate... Biglerville PA
419 417 Boulder Beer Company Boulder CO
420 418 Coalition Brewing Company Portland OR
421 419 Sanitas Brewing Company Boulder CO
422 420 Gore Range Brewery Edwards CO
423 421 Redstone Meadery Boulder CO
424 422 Blue Dog Mead Eugene OR
425 423 Hess Brewing Company San Diego CA
426 424 Wynkoop Brewing Company Denver CO
427 425 Ciderboys Stevens Point WI
428 426 Armadillo Ale Works Denton TX
429 427 Roanoke Railhouse Brewery Roanoke VA
430 428 Schlafly Brewing Company Saint Louis MO
431 429 Asher Brewing Company Boulder CO
432 430 Lost Rhino Brewing Company Ashburn VA
433 431 North Country Brewing Company Slippery Rock PA
434 432 Seabright Brewery Santa Cruz CA
435 433 French Broad Brewery Asheville NC
436 434 Angry Orchard Cider Company Cincinnati OH
437 435 Two Roads Brewing Company Stratford CT
438 436 Southern Oregon Brewing Company Medford OR
439 437 Brooklyn Brewery Brooklyn NY
440 438 The Right Brain Brewery Traverse City MI
441 439 Kona Brewing Company Kona HI
442 440 MillKing It Productions Royal Oak MI
443 441 Pateros Creek Brewing Company Fort Collins CO
444 442 O'Fallon Brewery O'Fallon MO
445 443 Marble Brewery Albuquerque NM
446 444 Big Wood Brewery Vadnais Heights MN
447 445 Howard Brewing Company Lenoir NC
448 446 Downeast Cider House Leominster MA
449 447 Swamp Head Brewery Gainesville FL
450 448 Mavericks Beer Company Half Moon Bay CA
451 449 TailGate Beer San Diego CA
452 450 Northwest Brewing Company Pacific WA
453 451 Dad & Dude's Breweria Aurora CO
454 452 Centennial Beer Company Edwards CO
455 453 Denali Brewing Company Talkeetna AK
456 454 Deschutes Brewery Bend OR
457 455 Sunken City Brewing Company Hardy VA
458 456 Lucette Brewing Company Menominie WI
459 457 The Black Tooth Brewing Company Sheridan WY
460 458 Kenai River Brewing Company Soldotna AK
461 459 River North Brewery Denver CO
462 460 Fremont Brewing Company Seattle WA
463 461 Armstrong Brewing Company South San Francisco CA
464 462 AC Golden Brewing Company Golden CO
465 463 Big Bend Brewing Company Alpine TX
466 464 Good Life Brewing Company Bend OR
467 465 Engine 15 Brewing Jacksonville Beach FL
468 466 Green Room Brewing Jacksonville FL
469 467 Brindle Dog Brewing Company Tampa Bay FL
470 468 Peace Tree Brewing Company Knoxville IA
471 469 Terrapin Brewing Company Athens GA
472 470 Pete's Brewing Company San Antonio TX
473 471 Okoboji Brewing Company Spirit Lake IA
474 472 Crystal Springs Brewing Company Boulder CO
475 473 Engine House 9 Tacoma WA
476 474 Tonka Beer Company Minnetonka MN
477 475 Red Hare Brewing Company Marietta GA
478 476 Hangar 24 Craft Brewery Redlands CA
479 477 Big Elm Brewing Sheffield MA
480 478 Good People Brewing Company Birmingham AL
481 479 Heavy Seas Beer Halethorpe MD
482 480 Telluride Brewing Company Telluride CO
483 481 7 Seas Brewing Company Gig Harbor WA
484 482 Confluence Brewing Company Des Moines IA
485 483 Bale Breaker Brewing Company Yakima WA
486 484 The Manhattan Brewing Company New York NY
487 485 MacTarnahans Brewing Company Portland OR
488 486 Stillmank Beer Company Green Bay WI
489 487 Redhook Brewery Woodinville WA
490 488 Dock Street Brewery Philadelphia PA
491 489 Blue Point Brewing Company Patchogue NY
492 490 Tampa Bay Brewing Company Tampa FL
493 491 Devil's Canyon Brewery Belmont CA
494 492 Stone Coast Brewing Company Portland ME
495 493 Broken Tooth Brewing Company Anchorage AK
496 494 Seven Brides Brewery Silverton OR
497 495 Newburyport Brewing Company Newburyport MA
498 496 Dry Dock Brewing Company Aurora CO
499 497 Cans Bar and Canteen Charlotte NC
500 498 Sprecher Brewing Company Glendale WI
501 499 Wildwood Brewing Company Stevensville MT
502 500 High Noon Saloon And Brewery Leavenworth KS
503 501 Woodchuck Hard Cider Middlebury VT
504 502 Sea Dog Brewing Company Portland ME
505 503 Oskar Blues Brewery Lyons CO
506 504 Carolina Beer & Beverage Mooresville NC
507 505 Krebs Brewing Company (Pete's Pl... Krebs OK
508 506 Warbird Brewing Company Fort Wayne IN
509 507 Mudshark Brewing Company Lake Havasu City AZ
510 508 Spilker Ales Cortland NE
511 509 Wingman Brewers Tacoma WA
512 510 Kettle House Brewing Company Missoula MT
513 511 Sherwood Forest Brewers Marlborough MA
514 512 Cottrell Brewing Pawcatuck CT
515 513 Arctic Craft Brewery Colorado Springs CO
516 514 Monkey Paw Pub & Brewery San Diego CA
517 515 Crabtree Brewing Company Greeley CO
518 516 Emerald City Beer Company Seattle WA
519 517 Butcher's Brewing Carlsbad CA
520 518 New South Brewing Company Myrtle Beach SC
521 519 Big River Brewing Company Chattanooga TN
522 520 Twisted Pine Brewing Company Boulder CO
523 521 Flying Dog Brewery Frederick MD
524 522 Uncommon Brewers Santa Cruz CA
525 523 Aspen Brewing Company Aspen CO
526 524 Triangle Brewing Company Durham NC
527 525 Bomb Beer Company New York NY
528 526 Churchkey Can Company Seattle WA
529 527 Intuition Ale Works Jacksonville FL
530 528 Asheville Brewing Company Asheville NC
531 529 Northwoods Brewpub Eau Claire WI
532 530 Buckbean Brewing Company Reno NV
533 531 Dolores River Brewery Dolores CO
534 532 Flat Rock Brewing Company Smithton PA
535 533 Abita Brewing Company Abita Springs LA
536 534 Mammoth Brewing Company Mammoth Lakes CA
537 535 Harvest Moon Brewing Company Belt MT
538 536 Grand Canyon Brewing Company Williams AZ
539 537 Lewis and Clark Brewing Company Helena MT
540 538 Dundee Brewing Company Rochester NY
541 539 Twin Lakes Brewing Company Greenville DE
542 540 Mother Earth Brewing Company Kinston NC
543 541 Arcadia Brewing Company Battle Creek MI
544 542 Angry Minnow Brewing Company Hayward WI
545 543 Great Northern Brewing Company Whitefish MT
546 544 Pyramid Breweries Seattle WA
547 545 Lancaster Brewing Company Lancaster PA
548 546 Upstate Brewing Company Elmira NY
549 547 Moat Mountain Smoke House & Brew... North Conway NH
550 548 Prescott Brewing Company Prescott AZ
551 549 Mogollon Brewing Company Flagstaff AZ
552 550 Wind River Brewing Company Pinedale WY
553 551 Silverton Brewery Silverton CO
554 552 Mickey Finn's Brewery Libertyville IL
555 553 Covington Brewhouse Covington LA
556 554 Dave's Brewfarm Wilson WI
557 555 Ukiah Brewing Company Ukiah CA
558 556 Butternuts Beer and Ale Garrattsville NY
559 557 Sleeping Lady Brewing Company Anchorage AK

162686
misc/raw-data/breweries.json Normal file
View File

File diff suppressed because it is too large Load Diff

578
misc/raw-data/ontariobreweries.json Normal file
View File

@@ -0,0 +1,578 @@
[
{
"text": "100 Acre Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/100-acre-brewing-company/"
},
{
"text": "All My Friends Beer Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/all-my-friends-beer-co/"
},
{
"text": "All or Nothing Brewhouse",
"href": "https://ontariocraftbrewers.com/brewery-profile/all-or-nothing-brewhouse/"
},
{
"text": "Anderson Craft Ales",
"href": "https://ontariocraftbrewers.com/brewery-profile/anderson-craft-ales/"
},
{
"text": "Badlands Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/badlands-brewing-company/"
},
{
"text": "Bancroft Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/bancroft-brewing-co/"
},
{
"text": "Banded Goose Brewing Comany",
"href": "https://ontariocraftbrewers.com/brewery-profile/banded-goose-brewing-comany/"
},
{
"text": "Beau’s Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/beaus-brewery/"
},
{
"text": "BeerLab! London",
"href": "https://ontariocraftbrewers.com/brewery-profile/beerlab-london/"
},
{
"text": "Bellwoods Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/bellwoods-brewery/"
},
{
"text": "Bench Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/bench-brewing/"
},
{
"text": "Beyond The Pale Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/beyond-the-pale-brewing-co/"
},
{
"text": "Bicycle Craft Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/bicycle-craft-brewery/"
},
{
"text": "Big Rig Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/big-rig-brewery/"
},
{
"text": "Big Rock Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/big-rock-brewery/"
},
{
"text": "Black Gold Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/black-gold-brewery/"
},
{
"text": "Black Oak Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/black-oak-brewing-co/"
},
{
"text": "Block 3 Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/block-3-brewing-co/"
},
{
"text": "Blood Brothers Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/blood-brothers-brewing/"
},
{
"text": "Bobcaygeon Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/bobcaygeon-brewing-company/"
},
{
"text": "Boshkung Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/boshkung-brewing-co/"
},
{
"text": "Brauwerk Hoffman – Rockland",
"href": "https://ontariocraftbrewers.com/brewery-profile/brauwerk-hoffman-rockland/"
},
{
"text": "Bridge Masters Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/bridge-masters-brewing-co/"
},
{
"text": "Broadhead Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/broadhead-brewery/"
},
{
"text": "Broken Rail Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/broken-rail-brewing/"
},
{
"text": "Burdock Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/burdock-brewery/"
},
{
"text": "C’est What Durham Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/cest-what-durham-brewing-company/"
},
{
"text": "Calabogie Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/calabogie-brewing/"
},
{
"text": "Cameron’s Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/camerons-brewing-company/"
},
{
"text": "Canvas Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/canvas-brewing-co/"
},
{
"text": "Caps Off Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/caps-off-brewing/"
},
{
"text": "Century Barn Brewing & Beverage Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/century-barn-brewing-and-beverage-company/"
},
{
"text": "Chronicle Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/chronicle-brewing-company/"
},
{
"text": "Clifford Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/clifford-brewing-co/"
},
{
"text": "Cold Bear Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/cold-bear-brewing-co/"
},
{
"text": "Collective Arts Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/collective-arts-brewing-ltd/"
},
{
"text": "Common Good Beer Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/common-good-beer-co/"
},
{
"text": "Couchiching Craft Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/couchiching-craft-brewing-company/"
},
{
"text": "Cowbell Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/cowbell-brewing-co/"
},
{
"text": "Cured Craft Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/cured-craft-brewing-co/"
},
{
"text": "Daft Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/daft-brewing-company/"
},
{
"text": "Dog House Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/dog-house-brewing-company/"
},
{
"text": "Dominion City Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/dominion-city-brewing-co/"
},
{
"text": "Eastbound Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/eastbound-brewing-co/"
},
{
"text": "Equals Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/equals-brewing-company/"
},
{
"text": "Fairweather Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/fairweather-brewing-company/"
},
{
"text": "Farm League Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/farm-league-brewing/"
},
{
"text": "Fixed Gear Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/fixed-gear-brewing-co/"
},
{
"text": "Flying Monkeys Craft Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/flying-monkeys-craft-brewery/"
},
{
"text": "Focal Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/focal-brewing-co/"
},
{
"text": "Foundry Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/foundry-brewing/"
},
{
"text": "Four Fathers Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/four-fathers-brewing-co/"
},
{
"text": "Frank Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/frank-brewing-co/"
},
{
"text": "Freddy’s",
"href": "https://ontariocraftbrewers.com/brewery-profile/freddys/"
},
{
"text": "Full Beard Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/1068-2/"
},
{
"text": "Furnace Room Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/furnace-room-brewery/"
},
{
"text": "Gateway City Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/gateway-city-brewery/"
},
{
"text": "Glasstown Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/glasstown-brewing-company/"
},
{
"text": "Godspeed Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/godspeed-brewery/"
},
{
"text": "Goldenfield Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/goldenfield-brewing/"
},
{
"text": "Grand River Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/grand-river-brewery/"
},
{
"text": "Granite Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/granite-brewery/"
},
{
"text": "Great Lakes Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/great-lakes-brewery/"
},
{
"text": "Haliburton Highlands Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/haliburton-highlands-brewery/"
},
{
"text": "Imperial City Brew House",
"href": "https://ontariocraftbrewers.com/brewery-profile/imperial-city-brew-house/"
},
{
"text": "Indie Ale House",
"href": "https://ontariocraftbrewers.com/brewery-profile/indie-ale-house/"
},
{
"text": "Jobsite Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/jobsite-brewing-co/"
},
{
"text": "Kichesippi Beer Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/kichesippi-beer-co/"
},
{
"text": "Kick and Push Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/kick-and-push-brewing-company/"
},
{
"text": "Lake of Bays Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/lake-of-bays-brewing-co/"
},
{
"text": "Lake Of The Woods Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/lake-of-the-woods-brewing-company/"
},
{
"text": "Left Field Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/left-field-brewery/"
},
{
"text": "Lightcaster Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/lightcaster-brewery/"
},
{
"text": "MacKinnon Brothers Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/mackinnon-brothers-brewing-co/"
},
{
"text": "Maclean’s Ales Inc.",
"href": "https://ontariocraftbrewers.com/brewery-profile/macleans-ales-inc/"
},
{
"text": "Magnotta Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/magnotta-brewery/"
},
{
"text": "Market Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/market-brewing/"
},
{
"text": "Mascot Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/mascot-brewery/"
},
{
"text": "Matron Fine Beer",
"href": "https://ontariocraftbrewers.com/brewery-profile/matron-fine-beer/"
},
{
"text": "Meyers Creek Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/meyers-creek-brewing-company/"
},
{
"text": "Midtown Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/midtown-brewing-co/"
},
{
"text": "Miski Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/miski-brewing/"
},
{
"text": "Muddy York Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/muddy-york-brewing-co/"
},
{
"text": "Muskoka Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/muskoka-brewery/"
},
{
"text": "Natterjack Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/natterjack-brewing-company/"
},
{
"text": "Newark Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/newark-brewing-co/"
},
{
"text": "Niagara Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/niagara-brewing-company/"
},
{
"text": "Niagara College Teaching Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/niagara-college-teaching-brewery/"
},
{
"text": "Niagara Oast House Brewers",
"href": "https://ontariocraftbrewers.com/brewery-profile/niagara-oast-house-brewers/"
},
{
"text": "Nickel Brook Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/nickel-brook-brewing-co/"
},
{
"text": "Northern Superior Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/northern-superior-brewing-co/"
},
{
"text": "Old Credit Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/old-credit-brewing-co/"
},
{
"text": "Old Flame Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/1239-2/"
},
{
"text": "Orléans Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/orleans-brewing-co/"
},
{
"text": "Overflow Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/overflow-brewing-co/"
},
{
"text": "Parsons Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/parsons-brewing-company/"
},
{
"text": "Perth Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/perth-brewery/"
},
{
"text": "Prince Eddy’s Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/prince-eddys-brewing-company/"
},
{
"text": "Quayle’s Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/quayles-brewery/"
},
{
"text": "Quetico Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/quetico-brewing-company/"
},
{
"text": "Railway City Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/railway-city-brewing-co/"
},
{
"text": "Ramblin’ Road Brewery Farm",
"href": "https://ontariocraftbrewers.com/brewery-profile/ramblin-road-brewery-farm/"
},
{
"text": "Red Barn Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/red-barn-brewing-company/"
},
{
"text": "Refined Fool Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/refined-fool-brewing-co/"
},
{
"text": "Rouge River Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/rouge-river-brewing/"
},
{
"text": "Royal City Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/royal-city-brewing-co/"
},
{
"text": "Sassy Britches Brewing Co Ltd",
"href": "https://ontariocraftbrewers.com/brewery-profile/sassy-britches-brewing-co-ltd/"
},
{
"text": "Sawdust City Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/sawdust-city-brewing/"
},
{
"text": "Shawn & Ed Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/shawn-ed-brewing-co/"
},
{
"text": "Silversmith Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/silversmith-brewing-company/"
},
{
"text": "Slake Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/slake-brewing/"
},
{
"text": "Sleeping Giant Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/sleeping-giant-brewing-co/"
},
{
"text": "Something in the Water Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/something-in-the-water-brewing-co/"
},
{
"text": "Sonnen Hill Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/sonnen-hill-brewing/"
},
{
"text": "Sons of Kent Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/sons-of-kent-brewing-company/"
},
{
"text": "Spark Beer",
"href": "https://ontariocraftbrewers.com/brewery-profile/spark-beer/"
},
{
"text": "Split Rail Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/split-rail-brewing-company/"
},
{
"text": "Stack Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/stack-brewing/"
},
{
"text": "Steam Whistle Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/steam-whistle-brewing/"
},
{
"text": "Steel Wheel Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/steel-wheel-brewery/"
},
{
"text": "Stonehooker Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/stonehooker-brewing-company/"
},
{
"text": "Stonepicker Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/stonepicker-brewery/"
},
{
"text": "Stray Dog Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/stray-dog-brewing-company/"
},
{
"text": "The Exchange Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/the-exchange-brewery/"
},
{
"text": "The Grove Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/the-grove-brewing-company/"
},
{
"text": "The Second Wedge Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/the-second-wedge-brewing-co/"
},
{
"text": "Thornbury Village Craft Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/thornbury-village-craft-brewery/"
},
{
"text": "Three Sheets Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/three-sheets-brewing/"
},
{
"text": "Tooth and Nail Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/tooth-and-nail-brewing-company/"
},
{
"text": "Torched Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/torched-brewing/"
},
{
"text": "Town Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/town-brewery/"
},
{
"text": "Trestle Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/trestle-brewing-company/"
},
{
"text": "True History Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/true-history-brewing/"
},
{
"text": "Upper Thames Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/upper-thames-brewing-co/"
},
{
"text": "Vimy Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/vimy-brewing-company/"
},
{
"text": "Walkerville Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/walkerville-brewery/"
},
{
"text": "Wave Maker Craft Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/wave-maker-craft-brewery/"
},
{
"text": "Wellington Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/wellington-brewery/"
},
{
"text": "Whiprsnapr Brewing Co.",
"href": "https://ontariocraftbrewers.com/brewery-profile/whiprsnapr-brewing-co/"
},
{
"text": "Whiskeyjack Beer Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/whiskeyjack-beer-company/"
},
{
"text": "Whitewater Brewing Company",
"href": "https://ontariocraftbrewers.com/brewery-profile/whitewater-brewing-company/"
},
{
"text": "Willibald Farm Distillery & Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/willibald-farm-distillery-brewery/"
},
{
"text": "Windmill Brewery",
"href": "https://ontariocraftbrewers.com/brewery-profile/windmill-brewery/"
},
{
"text": "Wishbone Brewing",
"href": "https://ontariocraftbrewers.com/brewery-profile/wishbone-brewing/"
}
]

0
tooling/pipeline/.clang-format → pipeline/.clang-format
View File

0
tooling/pipeline/.clang-tidy → pipeline/.clang-tidy
View File

1
tooling/pipeline/.gitignore → pipeline/.gitignore vendored
View File

@@ -6,4 +6,3 @@ data
models
*.gguf
BiergartenPipeline.png
output

181
pipeline/CMakeLists.txt Normal file
View File

@@ -0,0 +1,181 @@
cmake_minimum_required(VERSION 3.24)
project(biergarten-pipeline)
set(CMAKE_POLICY_VERSION_MINIMUM 3.5 CACHE STRING "" FORCE)
# =============================================================================
# 1. Platform & GPU Detection
# =============================================================================
if(WIN32)
message(FATAL_ERROR "[biergarten] Windows is currently not supported. Please use Linux (Fedora 43) or macOS (M1 Pro).")
endif()
if(APPLE)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "arm64")
message(STATUS "[biergarten] Apple Silicon detected — enabling Metal acceleration.")
set(GGML_METAL ON CACHE BOOL "Enable Metal for Apple Silicon" FORCE)
else()
message(STATUS "[biergarten] Intel Mac detected — using CPU / Accelerate framework.")
set(GGML_METAL OFF CACHE BOOL "Disable Metal for Intel Macs" FORCE)
endif()
elseif(UNIX AND NOT APPLE)
find_package(CUDAToolkit QUIET)
find_package(HIP QUIET)
if(CUDAToolkit_FOUND)
message(STATUS "[biergarten] NVIDIA GPU detected — enabling CUDA acceleration.")
set(GGML_CUDA ON CACHE BOOL "Enable CUDA for NVIDIA GPUs" FORCE)
set(CMAKE_CUDA_ARCHITECTURES native)
elseif(HIP_FOUND OR EXISTS "/opt/rocm")
message(STATUS "[biergarten] AMD GPU detected — enabling HIP/ROCm acceleration.")
set(GGML_HIPBLAS ON CACHE BOOL "Enable HIP for AMD GPUs" FORCE)
else()
message(STATUS "[biergarten] No NVIDIA or AMD GPU found — falling back to CPU.")
endif()
endif()
# =============================================================================
# 2. Project-wide Settings (Standard & Optimization)
# =============================================================================
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
# Release Build Optimization: Aggressive (-O3), Arch-specific, and LTO
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3 -march=native -flto")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -Og -g")
# =============================================================================
# 3. Dependencies
# =============================================================================
include(FetchContent)
find_package(CURL QUIET)
if(NOT CURL_FOUND)
message(FATAL_ERROR "[biergarten] libcurl not found. Install it (e.g. 'sudo dnf install libcurl-devel').")
endif()
# Require system Boost for JSON and Program Options to speed up build times
find_package(Boost REQUIRED COMPONENTS json program_options)
FetchContent_Declare(
sqlite_amalgamation
URL https://www.sqlite.org/2026/sqlite-amalgamation-3530000.zip
URL_HASH SHA3_256=c2325c53b3b41761469f91cfb078e96882ac5d85bac10c11b0bd8f253b031e5b
)
FetchContent_GetProperties(sqlite_amalgamation)
if(NOT sqlite_amalgamation_POPULATED)
FetchContent_Populate(sqlite_amalgamation)
endif()
if(NOT TARGET sqlite3)
add_library(sqlite3 STATIC
${sqlite_amalgamation_SOURCE_DIR}/sqlite3.c
)
target_include_directories(sqlite3 PUBLIC
${sqlite_amalgamation_SOURCE_DIR}
)
target_compile_definitions(sqlite3 PUBLIC
SQLITE_THREADSAFE=1
)
endif()
FetchContent_Declare(
llama-cpp
GIT_REPOSITORY https://github.com/ggml-org/llama.cpp.git
GIT_TAG b8742
)
FetchContent_MakeAvailable(llama-cpp)
FetchContent_Declare(
boost-di
GIT_REPOSITORY https://github.com/boost-ext/di.git
GIT_TAG v1.3.0
)
FetchContent_MakeAvailable(boost-di)
if(TARGET Boost.DI AND NOT TARGET boost::di)
add_library(boost::di ALIAS Boost.DI)
endif()
FetchContent_Declare(
spdlog
GIT_REPOSITORY https://github.com/gabime/spdlog.git
GIT_TAG v1.15.3
)
FetchContent_MakeAvailable(spdlog)
# =============================================================================
# 4. Sources
# =============================================================================
set(SOURCES
src/main.cc
src/biergarten_data_generator/biergarten_data_generator.cc
src/biergarten_data_generator/run.cc
src/biergarten_data_generator/query_cities_with_countries.cc
src/biergarten_data_generator/generate_breweries.cc
src/biergarten_data_generator/log_results.cc
src/services/wikipedia/wikipedia_service.cc
src/services/wikipedia/get_summary.cc
src/services/wikipedia/fetch_extract.cc
src/services/sqlite/sqlite_export_service.cc
src/services/sqlite/build_database_path.cc
src/services/sqlite/build_location_key.cc
src/services/sqlite/initialize_schema.cc
src/services/sqlite/prepare_statements.cc
src/services/sqlite/initialize.cc
src/services/sqlite/process_record.cc
src/services/sqlite/finalize_statements.cc
src/services/sqlite/rollback_and_close_no_throw.cc
src/services/sqlite/finalize.cc
src/web_client/curl_global_state.cc
src/web_client/curl_web_client_get.cc
src/web_client/curl_web_client_url_encode.cc
src/data_generation/llama/llama_generator.cc
src/data_generation/llama/generate_brewery.cc
src/data_generation/llama/generate_user.cc
src/data_generation/llama/helpers.cc
src/data_generation/llama/infer.cc
src/data_generation/llama/load.cc
src/data_generation/llama/load_brewery_prompt.cc
src/data_generation/prompt_formatting/gemma4_jinja_prompt_formatter.cc
src/data_generation/mock/deterministic_hash.cc
src/data_generation/mock/generate_brewery.cc
src/data_generation/mock/generate_user.cc
src/json_handling/json_loader.cc
)
# =============================================================================
# 5. Target
# =============================================================================
add_executable(${PROJECT_NAME} ${SOURCES})
target_include_directories(${PROJECT_NAME} PRIVATE
includes
${llama-cpp_SOURCE_DIR}/include
${llama-cpp_SOURCE_DIR}/common
)
target_link_libraries(${PROJECT_NAME} PRIVATE
llama
boost::di
Boost::json
Boost::program_options
spdlog::spdlog
sqlite3
CURL::libcurl
)
# =============================================================================
# 6. Runtime Assets
# =============================================================================
configure_file(
${CMAKE_SOURCE_DIR}/locations.json
${CMAKE_BINARY_DIR}/locations.json
COPYONLY
)
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CMAKE_SOURCE_DIR}/prompts
${CMAKE_BINARY_DIR}/prompts
)

82
docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md → pipeline/ETHICS-AND-KNOWN-ISSUES.md
View File

@@ -13,7 +13,6 @@ low-resource language failures.
- [Model Bias and Language Quality](#model-bias-and-language-quality)
- [Western and Eurocentric Lens](#western-and-eurocentric-lens)
- [Wikipedia Enrichment](#wikipedia-enrichment)
- [Names-by-Country Dataset](#names-by-country-dataset)
- [The "Avoid AI Phrases" Prompt Instruction](#the-avoid-ai-phrases-prompt-instruction)
- [Known Issues](#known-issues)
- [Hallucinated Brewing Techniques](#hallucinated-brewing-techniques)
@@ -45,20 +44,20 @@ distributions, rating skews, activity profiles) will be handled the same way.
as a source of truth for brewing techniques or cultural representation.**
**Natural language processing, although a powerful tool for data analysis and
generation is to be taken with scrutiny. Human language is not simply just data
points to be analyzed, but it also carries deep cultural and human meaning that
generation is to be taken with scrutiny. Human language is not simply data
points to be analyzed, but carries deep cultural and human meaning that
artificial intelligence is incapable of.**
---
## Model Bias and Language Quality
The underlying model's training biases surface within this pipeline. Output
quality tracks with how well a language is represented in the training corpus:
standard French (`fr-FR`) produces coherent text; regional variants like `fr-CD`
and `fr-CI` are noticeably weaker; low-resource languages like Welsh, Māori, and
Sicilian produce output that is syntactically plausible but often semantically
broken.
The underlying model's training biases surface measurably in this pipeline.
Output quality tracks with how well a language is represented in the training
corpus: standard French (`fr-FR`) produces coherent text; regional variants like
`fr-CD` and `fr-CI` are noticeably weaker; low-resource languages like Welsh,
Māori, and Sicilian produce output that is syntactically plausible but often
semantically broken.
This is a property of the training distribution, not something that can be
mitigated through prompt design. This is a well-documented characteristic of
@@ -92,33 +91,6 @@ generated descriptions.
---
## Names-by-Country Dataset
`tooling/pipeline/forenames-by-country.json` and `surnames-by-country.json`
(used to sample a `Name` per ISO 3166-1 country code for user generation) are
vendored verbatim, unmodified, from
[sigpwned/popular-names-by-country-dataset](https://github.com/sigpwned/popular-names-by-country-dataset)
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
release assets), released under **CC0** (public domain). That dataset's own
forename/surname lists are pulled from Wikipedia's "Lists of most common
surnames" and "List of most popular given names" as of the week of
2023-07-08 — see that project's README for full provenance. Names are not
LLM-generated; this is curated fixture data per ROADMAP.md §2. Per-forename
gender from the source data is preserved through to the sampled `Name`
(rather than discarded during loading) so it's available for gender-aware
persona/bio generation later.
The full multinational dataset is kept as-is (106 countries for forenames,
75 for surnames) rather than trimmed to `locations.json`'s current country
list, so it doesn't need re-sourcing if more countries are added later.
`SampleName()` (a free helper in `generate_users.cc`) returns no result for
a country present in neither file; of the countries in `locations.json`,
that's currently `KE`, `SE`, `SG`, `TH`, `VN`, and `ZA` — `GenerateUsers`
skips cities in those countries the same way brewery generation skips
cities whose enrichment lookup fails.
---
## The "Avoid AI Phrases" Prompt Instruction
The system prompt instructs the model to avoid common AI-generated phrasing
@@ -208,6 +180,13 @@ translated `description_local`. Output quality is reliable for high-resource
languages such as French, though it may struggle with regional variants and
idiomatic phrasing.
For languages such as Welsh (Wales), Māori (Aotearoa/New Zealand), or Sicilian
(Sicily, Italy), the model can generate text that looks syntactically plausible
but is semantically incoherent. This comes from limited training-data coverage
rather than prompt engineering. The resource-tier gradient is illustrated by
this set of French-speaking cities, where `fr-FR` (Paris) produces coherent
output while `fr-CD` (Kinshasa) and `fr-CI` (Abidjan) do not:
```json
[
{
@@ -263,17 +242,8 @@ idiomatic phrasing.
]
```
This dataset, when fed into the pipeline will often times reason that a local
variant of French is needed, but will often times just default to a standardized
dialect of French, devoid of any cultural or linguistic nuance.
For languages such as Welsh (Wales), Māori (Aotearoa/New Zealand), or Sicilian
(Sicily, Italy), the model can generate text that looks syntactically plausible
but is semantically incoherent. This comes from limited training-data coverage
rather than prompt engineering.
Output sample:
[./french-cities.example](french-cities.example)
[./out-sample/french-cities.example](out-sample/french-cities.example)
#### Proposed Mitigations
@@ -291,12 +261,11 @@ Output sample:
## Footnotes
[^llm-choke]:
CHOKE (Certain Hallucinations Overriding Known Evidence) is a hallucination
failure mode defined by Simhi et al. (2025), in which a model that can
consistently answer a question correctly produces a confident, wrong
response when the prompt is trivially perturbed. Source: Trust Me, I'm
Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer — Adi
Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, Yonatan Belinkov.
See: "CHOKE: Certain Hallucinations Overriding Known Evidence," a term
coined by AI researcher Gwern Branwen to describe a failure mode where a
model produces confident, coherent, and factually wrong output in response
to a specific-sounding prompt. Source:
[Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer](https://arxiv.org/abs/2502.12964)
[^llm-bias]:
e.g., Blasi et al. (2022), "Systematic Inequalities in Language Technology
@@ -315,12 +284,9 @@ Output sample:
[creativecommons.org/licenses/by-sa/4.0](https://creativecommons.org/licenses/by-sa/4.0/deed.en).
[^diacetyl-source]:
White Labs confirms that diacetyl is a yeast-derived fermentation byproduct:
specifically, a compound produced during amino acid metabolism that leaks
out of the yeast cell and oxidises into its characteristic buttery
off-flavour. It is generally considered undesirable at any perceived level
in most styles, though low levels are tolerated in some English ales and
European lagers. Source:
White Labs: "Diacetyl is a natural byproduct of fermentation… produced by
brewer's yeast… generally considered undesirable at any perceived level."
Source:
[whitelabs.com — Compound Spotlight: Diacetyl](https://www.whitelabs.com/news-update-detail?id=54).
[^diacetyl-rest]:

324
pipeline/README.md Normal file
View File

@@ -0,0 +1,324 @@
# Biergarten Pipeline
A C++20 command-line pipeline that samples city records from local JSON,
enriches each with Wikipedia context, and generates bilingual brewery names and
descriptions via a local GGUF model or a deterministic mock.
> **This pipeline produces AI-generated data.** It is not a source of truth for
> brewing techniques, cultural representation, or local-language accuracy. See
> [ETHICS-AND-KNOWN-ISSUES.md](ETHICS-AND-KNOWN-ISSUES.md) for full
> documentation of limitations, hallucination patterns, and bias.
---
## Table of Contents
- [How It Fits The Main App](#how-it-fits-the-main-app)
- [Quick Start](#quick-start)
- [Build](#build)
- [Model](#model)
- [Run](#run)
- [Architecture](#architecture)
- [Pipeline Stages](#pipeline-stages)
- [Key Components](#key-components)
- [Runtime Behaviour](#runtime-behaviour)
- [Generated Output](#generated-output)
- [Tech Stack](#tech-stack)
- [Tested Hardware](#tested-hardware)
- [Fixture Strategy](#fixture-strategy)
- [Repo Layout](#repo-layout)
- [Code Tour](#code-tour)
- [Next Steps](#next-steps)
---
## How It Fits The Main App
The pipeline is a data ingestion layer. It sits outside the web app runtime and
produces seed records the app imports at startup or during a dedicated seed
step.
| Planned app area | Pipeline contribution |
| -------------------------------- | ------------------------------------------------------------------ |
| Brewery discovery and management | Sampled city records, localized names, long-form descriptions |
| Beer reviews and ratings | Stable brewery fixtures with enough context to anchor review pages |
| Social follow relationships | Repeatable brewery entities for feeds, follows, and saved lists |
| Geospatial brewery experiences | Latitude, longitude, and country-level metadata |
---
## Quick Start
### Build
Requirements: C++20 compiler, CMake 3.24+, libcurl, Boost (JSON and
ProgramOptions). SQLite is fetched from the upstream amalgamation, so no system
SQLite package is required.
```bash
cmake -S . -B build
cmake --build build
```
### Model
> Skip this step if you only need `--mocked`.
```bash
mkdir -p models
curl -L \
-o models/google_gemma-4-E4B-it-Q6_K.gguf \
https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF/resolve/main/google_gemma-4-E4B-it-Q6_K.gguf?download=true
```
### Run
Run from `build/` so the copied `locations.json` and `prompts/` are available.
Each run also writes a fresh dated SQLite file such as
`biergarten_seed_2026-04-19T15-30-45.123456Z.sqlite` into the working directory.
```bash
./biergarten-pipeline --mocked
./biergarten-pipeline --model models/google_gemma-4-E4B-it-Q6_K.gguf --temperature 1.0 --top-p 0.95 --top-k 64 --n-ctx 8192 --seed -1
```
#### CLI Flags
| Flag | Purpose |
| --------------- | ------------------------------------------------------- |
| `--mocked` | Deterministic mock generator, no model required. |
| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. |
| `--temperature` | Sampling temperature. Default: `1.0`. |
| `--top-p` | Nucleus sampling. Default: `0.95`. |
| `--top-k` | Top-k sampling. Default: `64`. |
| `--n-ctx` | Context window size. Default: `8192`. |
| `--seed` | Random seed. Default: `-1` (random at runtime). |
| `--help, -h` | Print usage and exit. |
`--mocked` and `--model` are mutually exclusive. Omitting both exits with an
error before the pipeline starts. Sampling flags are ignored when `--mocked` is
set.
The post-build step copies `prompts/` into `build/prompts/`. Rebuild after
editing `prompts/system.md`.
---
## Architecture
### Pipeline Stages
| Stage | Implementation |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| Load | `JsonLoader::LoadLocations()` reads `locations.json` into typed `Location` records. |
| Sample | `BiergartenDataGenerator::QueryCitiesWithCountries()` samples up to 50 locations per run. |
| Enrich | `WikipediaService` fetches city and beer context. Keeps going when a lookup fails. |
| Generate | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. |
| Store | `SqliteExportService` writes each successful brewery into a fresh dated `.sqlite` database with normalized location and brewery tables. |
| Log | `spdlog` writes results and warnings to the console. |
If enrichment or generation fails for a city, that city is skipped and the
pipeline continues.
### Key Components
- `src/main.cc` — argument parsing and Boost.DI composition root.
- `JsonLoader` — validates curated location input.
- `WikipediaService` — queries Wikipedia extracts, caches results, returns empty
context on failure.
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output, retries
malformed responses up to three times. If output looks truncated, the retry
raises the token budget before trying again.
- `MockGenerator` — stable hash-based output so the same city input always
produces the same brewery.
- `SqliteExportService` — creates a dated SQLite file per run and persists each
successful brewery into normalized tables.
- Brewery payloads include English and local-language name and description
fields.
### Runtime Behaviour
`WikipediaService` queries city, country, and beer-related Wikipedia extracts
using its configured lookup, then caches the first successful response per query
string. The fetched extract text is included in the prompt as context for
generation.
`GetLocationContext()` returns an empty string when the web client is
unavailable or when lookup/parsing fails.
`LlamaGenerator` validates model output as structured JSON. The retry path
exists as a safety hatch for cases where the reasoning block consumes available
token budget and compresses the JSON output space. All runs to date have
produced valid output on the first pass; the path is kept for resilience.
`MockGenerator` uses stable hashes for repeatable output in demos and Storybook
runs.
### Process Flow - Activity Diagram
![An activity diagram](./diagrams/current/output/activity.svg)
### Architectural Overview - Class Diagram
![A class diagram](./diagrams/current/output/class.svg)
---
## Generated Output
Each successful run stores a `GeneratedBrewery` pair with the source location
and a `BreweryResult` payload. The same generated records are also written to a
fresh SQLite export file named with the current UTC timestamp.
| Field | Meaning |
| ------------------- | ------------------------------------------ |
| `name_en` | Brewery name in English. |
| `description_en` | Brewery description in English. |
| `name_local` | Brewery name in the local language. |
| `description_local` | Brewery description in the local language. |
The log dump also includes city, country, state or province, ISO subdivision
code, latitude, and longitude for each entry.
### Consumer Data Shape
| Field | Why it matters |
| ----------------------------------- | ------------------------------------------------ |
| `city`, `state_province`, `country` | Human-readable location labels and page headings |
| `iso3166_1`, `iso3166_2` | Filtering, regional grouping, locale matching |
| `latitude`, `longitude` | Map pins and nearby brewery views |
| `local_languages` | Locale-aware copy selection |
| `name_en`, `description_en` | Default English display content |
| `name_local`, `description_local` | Local-language display content |
| `region_context` | Richer copy for cards and detail pages |
---
## Tech Stack
- C++20
- CMake 3.24+
- Boost.JSON, Boost.ProgramOptions, Boost.DI
- spdlog
- libcurl
- SQLite amalgamation fetched and compiled via CMake FetchContent
- llama.cpp
The build fetches Boost.DI, spdlog, llama.cpp, and SQLite via CMake. Metal is
enabled on Apple Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit
is present.
> **Code Style:** Modern C++20 throughout — RAII for ownership,
> `std::unique_ptr` for injected dependencies, `std::optional` for parse
> outcomes, `std::span` for read-only views over generated city data, structured
> bindings in pipeline loops. Formatting follows the Google C++ Style Guide via
> `.clang-format` with a narrow column limit and two-space indentation.
---
## Tested Hardware
### ARM macOS - M1 Pro
| | |
| --------- | --------------------------------- |
| Host | MacBook Pro 14" (2021) |
| CPU | Apple M1 Pro (8-core) |
| GPU | Apple M1 Pro (14-core integrated) |
| Memory | 16 GB |
| Model | Gemma 4 E4B |
| Inference | llama.cpp with Metal |
### x86_64 Linux - NVIDIA RTX 2000
| | |
| --------- | ------------------------------ |
| Host | ThinkPad P1 Gen 7 (Fedora 43) |
| CPU | Intel Core Ultra 7 155H |
| GPU | NVIDIA RTX 2000 Ada Generation |
| Memory | 32 GB |
| Model | Gemma 4 E4B |
| Inference | llama.cpp with CUDA 12.x |
---
## Fixture Strategy
- `--mocked` for stable fixtures, repeatable screenshots, and Storybook runs.
- `--model` when geographically grounded content matters for demos.
- Keep `locations.json` structured enough to support discovery and future
filtering.
- Treat SQLite output as seed material for the app's brewery domain, not
production data.
---
## Repo Layout
| Path | Purpose |
| ---------------------------- | -------------------------------------------------- |
| `includes/` | Public headers and shared models. |
| `src/` | Implementation files. |
| `locations.json` | Curated city input copied into the build tree. |
| `prompts/` | System prompt used by the model-backed path. |
| `diagrams/` | Architecture and pipeline diagrams. |
| `ETHICS-AND-KNOWN-ISSUES.md` | Ethics, bias, hallucination analysis, mitigations. |
---
## Code Tour
- `src/main.cc` — argument parsing and DI composition root.
- `src/biergarten_data_generator/` — orchestration, sampling, logging, and
export.
- `src/services/wikipedia/` — enrichment service and cache.
- `src/services/sqlite/` — SQLite export implementation.
- `src/data_generation/llama/` — local inference, prompt loading, output
validation.
- `src/data_generation/mock/` — deterministic fallback.
---
## Next Steps
The pipeline currently produces city-aware brewery records and dated SQLite
exports. The next passes add additional fixture types so the app can exercise
the full brewery domain without live data.
### Testing — Very High Priority
- Unit test JSON validation and retry logic against malformed, truncated, and
empty model outputs.
- Integration test the enrichment pipeline with missing context, short context,
and fake context inputs.
- Adversarial context tests: feed plausible but geographically incorrect
Wikipedia extracts and verify the model does not silently blend them with
training data.
- Verify bilingual enrichment behaviour when only an English extract is
available versus when both extracts are present.
- Confirm the retry path is reachable when the reasoning block consumes
available token budget.
### Beer Generation
Generate catalog entries with style, ABV, IBU, color, aroma notes, and food
pairing hints. Link beers back to breweries and cities. Keep style coverage wide
enough to exercise search, sort, and category filters.
### User Generation
Generate user profiles with stable names, bios, locale hints, and preference
signals. Include stable IDs for downstream fixture joins. Keep output
deterministic for screenshots while allowing larger randomized batches.
### Check-In System
Produce timestamped check-in events between users and breweries. Use a J-curve
activity profile — a small set of users accounts for most check-ins, the rest
appear occasionally. Add bursty behaviour around weekends and travel periods.
### Beer Ratings
Generate rating events with a strong positive skew and a long tail of lower
scores. Avoid uniform distributions. Attach timestamps and user IDs so the app
can compute averages, trends, and per-style comparisons.

0
tooling/pipeline/beer-styles.json → pipeline/beer-styles.json
View File

125
pipeline/diagrams/current/activity.puml Normal file
View File

@@ -0,0 +1,125 @@
@startuml
skinparam style strictuml
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 14
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
skinparam backgroundColor #FAFCF9
skinparam defaultFontColor #28342A
skinparam titleFontColor #28342A
skinparam ArrowColor #628A5B
skinparam NoteBackgroundColor #EAF0E8
skinparam NoteBorderColor #547461
skinparam ActivityBackgroundColor #FAFCF9
skinparam ActivityBorderColor #547461
skinparam ActivityDiamondBackgroundColor #FAFCF9
skinparam ActivityDiamondBorderColor #628A5B
skinparam ActivityBarColor #628A5B
skinparam SwimlaneBorderColor #547461
skinparam SwimlaneBorderThickness 0.3
title The Biergarten Data Pipeline (Streaming Architecture)
|#F2F6F0|main.cc|
start
:ParseArguments(argc, argv);
if (Are arguments valid?) then (no)
:spdlog::error usage info;
stop
else (yes)
endif
:Init CurlGlobalState & LlamaBackendState;
:di::make_injector(...);
:injector.create<std::unique_ptr<BiergartenDataGenerator>>();
:BiergartenDataGenerator::Run();
|#EAF0E8|BiergartenDataGenerator|
:Initialize SQLite export;
|#E0EAE0|SqliteExportService|
:GetUtcTimestamp() from SystemDateTimeProvider;
:Initialize();
note right
Builds a fresh biergarten_seed_<UTC datetime>.sqlite filename
Appends a numeric suffix if the timestamp already exists
Opens DB Connection
Executes Schema DDL
Begins Transaction
end note
|#EAF0E8|BiergartenDataGenerator|
:QueryCitiesWithCountries();
|#E2EBDC|JsonLoader|
:JsonLoader::LoadLocations("locations.json");
:std::ranges::sample(all_locations, 50);
|#EAF0E8|BiergartenDataGenerator|
while (For each sampled Location?) is (Remaining cities)
|#DCE8D8|WikipediaService|
:GetLocationContext(loc);
:FetchExtracts(City, Country, Beer);
|#EAF0E8|BiergartenDataGenerator|
:Store EnrichedCity{Location, region_context};
endwhile (Done)
|#EAF0E8|BiergartenDataGenerator|
:GenerateBreweries(enriched_cities);
|#E5EDE1|DataGenerator|
while (For each EnrichedCity?) is (Remaining cities)
if (Generator Mode) then (MockGenerator)
:DeterministicHash & Format;
else (LlamaGenerator)
:PrepareRegionContext;
:LoadBrewerySystemPrompt("prompts/system.md");
repeat
:Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar);
:ValidateBreweryJson(raw, brewery);
if (Is JSON Valid?) then (yes)
break
else (no)
:Attempt++;
endif
repeat while (Attempt < 3?) is (yes)
endif
|#EAF0E8|BiergartenDataGenerator|
if (Generation successful?) then (yes)
|#E0EAE0|SqliteExportService|
:ProcessRecord(GeneratedBrewery);
if (Location in cache?) then (yes)
:Reuse location_id;
else (no)
:Insert Location & Cache ID;
endif
:Insert Brewery (FK: location_id);
if (Exception caught during insert?) then (yes)
|#EAF0E8|BiergartenDataGenerator|
:spdlog::warn "Failed to stream record to SQLite export";
note right
Data loss is prevented per-record.
The pipeline continues running.
end note
else (no)
endif
else (no)
:spdlog::warn "Generation failed, skipping...";
endif
|#E5EDE1|DataGenerator|
endwhile (Done)
|#E0EAE0|SqliteExportService|
:Finalize();
note right
Commits Transaction
Closes Database Connection
end note
|#F2F6F0|main.cc|
:Return 0;
stop
@enduml

148
pipeline/diagrams/current/class.puml Normal file
View File

@@ -0,0 +1,148 @@
@startuml
skinparam style strictuml
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 14
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
skinparam backgroundColor #FAFCF9
skinparam defaultFontColor #28342A
skinparam titleFontColor #28342A
skinparam ArrowColor #628A5B
skinparam class {
BackgroundColor #FAFCF9
HeaderBackgroundColor #EAF0E8
BorderColor #547461
ArrowColor #628A5B
FontColor #28342A
}
skinparam note {
BackgroundColor #EAF0E8
BorderColor #547461
FontColor #28342A
}
title The Biergarten Data Pipeline - Class Diagram
class BiergartenDataGenerator {
- context_service_ : std::unique_ptr<IEnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- exporter_ : std::unique_ptr<IExportService>
- generated_breweries_ : std::vector<GeneratedBrewery>
+ Run() : bool
- QueryCitiesWithCountries() : std::vector<Location>
- GenerateBreweries(cities : std::span<const EnrichedCity>) : void
- LogResults() : void
}
interface IEnrichmentService <<interface>> {
+ GetLocationContext(loc : const Location&) : std::string
}
class WikipediaService {
- client_ : std::unique_ptr<WebClient>
- extract_cache_ : std::unordered_map<std::string, std::string>
+ GetLocationContext(loc : const Location&) : std::string
- FetchExtract(query : std::string_view) : std::string
}
interface WebClient <<interface>> {
+ Get(url : const std::string&) : std::string
+ UrlEncode(value : const std::string&) : std::string
}
class CURLWebClient {
+ Get(url : const std::string&) : std::string
+ UrlEncode(value : const std::string&) : std::string
}
interface DataGenerator <<interface>> {
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
+ GenerateUser(locale : const std::string&) : UserResult
}
class MockGenerator {
+ GenerateBrewery(...) : BreweryResult
+ GenerateUser(...) : UserResult
- DeterministicHash(location : const Location&) : size_t
}
class LlamaGenerator {
- model_ : ModelHandle
- context_ : ContextHandle
- prompt_formatter_ : std::unique_ptr<IPromptFormatter>
- rng_ : std::mt19937
+ GenerateBrewery(...) : BreweryResult
+ GenerateUser(...) : UserResult
- Load(model_path : const std::string&) : void
- Infer(...) : std::string
- InferFormatted(...) : std::string
- LoadBrewerySystemPrompt(...) : std::string
}
interface IPromptFormatter <<interface>> {
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
}
class Gemma4JinjaPromptFormatter {
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
}
class JsonLoader {
+ {static} LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
}
interface IExportService <<interface>> {
+ Initialize() : void
+ ProcessRecord(brewery : const GeneratedBrewery&) : void
+ Finalize() : void
}
class SqliteExportService {
- date_time_provider_ : std::unique_ptr<IDateTimeProvider>
- run_timestamp_utc_ : std::string
- database_path_ : std::filesystem::path
- db_handle_ : sqlite3*
- insert_location_stmt_ : sqlite3_stmt*
- insert_brewery_stmt_ : sqlite3_stmt*
- transaction_open_ : bool
- location_cache_ : std::unordered_map<std::string, sqlite3_int64>
+ Initialize() : void
+ ProcessRecord(brewery : const GeneratedBrewery&) : void
+ Finalize() : void
- InitializeSchema() : void
}
interface IDateTimeProvider <<interface>> {
+ GetUtcTimestamp() : std::string
}
class SystemDateTimeProvider {
+ GetUtcTimestamp() : std::string
}
' Structural Relationships / Dependency Injection
BiergartenDataGenerator *-- IEnrichmentService : owns
BiergartenDataGenerator *-- DataGenerator : owns
BiergartenDataGenerator *-- IExportService : owns
IEnrichmentService <|.. WikipediaService : implements
WikipediaService *-- WebClient : owns
WebClient <|.. CURLWebClient : implements
DataGenerator <|.. MockGenerator : implements
DataGenerator <|.. LlamaGenerator : implements
LlamaGenerator *-- IPromptFormatter : uses
IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements
BiergartenDataGenerator ..> JsonLoader : uses
IExportService <|.. SqliteExportService : implements
SqliteExportService *-- IDateTimeProvider : owns
IDateTimeProvider <|.. SystemDateTimeProvider : implements
@enduml

1
pipeline/diagrams/current/output/activity.svg Normal file
View File

File diff suppressed because one or more lines are too long

1
pipeline/diagrams/current/output/class.svg Normal file
View File

File diff suppressed because one or more lines are too long

262
pipeline/diagrams/planned/activity.puml Normal file
View File

@@ -0,0 +1,262 @@
@startuml biergarten_activity
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 13
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
skinparam backgroundColor #FCFCF7
skinparam defaultFontColor #14180C
skinparam titleFontColor #14180C
skinparam ArrowColor #656F33
skinparam activityStartColor #EBECE3
skinparam activityEndColor #4A5837
skinparam activityStopColor #4A5837
skinparam ActivityBackgroundColor #EBECE3
skinparam ActivityBorderColor #4A5837
skinparam ActivityDiamondBackgroundColor #CBD2B5
skinparam ActivityDiamondBorderColor #4A5837
skinparam NoteBackgroundColor #DBEEDD
skinparam NoteFontColor #14180C
skinparam NoteBorderColor #4A5837
skinparam SwimlaneBorderColor #4A5837
skinparam SwimlaneBorderThickness 1
skinparam monochrome reverse
title The Biergarten Data Pipeline — Activity Diagram
|Main|
start
:ParseArguments(argc, argv);
if (Invalid args?) then (yes)
:spdlog::error;
stop
else (no)
endif
:Init CurlGlobalState & LlamaBackendState;
:Build DI injector;
:Initialize SqliteExportService;
note right
Opens SQLite connection.
Begins a single transaction
covering all five fixture types.
end note
:Create BoundedChannel<LogEntry> log_ch;
:Spawn Log Worker thread;
note right
Log worker drains log_ch for the
entire pipeline lifetime.
All workers emit LogEntry structs
via PipelineLogger — never spdlog directly.
end note
:BiergartenPipelineOrchestrator::Run();
|BiergartenPipelineOrchestrator::Run()|
:JsonLoader::LoadLocations("locations.json");
:JsonLoader::LoadBeerStyles("beer-styles.json");
:JsonLoader::LoadPersonas("personas.json");
:JsonLoader::LoadNamesByCountry("names-by-country.json");
:EnrichmentService::PreWarmBeerStyleCache(beer_styles);
note right
Beer styles do not need location context.
Wikipedia summaries for the entire palette are
fetched and cached globally at startup.
end note
:EnrichmentService::PreWarmPersonaCache(personas);
note right
Persona descriptions do not need location context.
All persona lookups are resolved and cached
globally at startup.
end note
' ═══════════════════════════════════════════
' PHASE 0 — USER GENERATION
' ═══════════════════════════════════════════
|Orchestrator|
:RunUserPhase(sampled_locations);
:Create BoundedChannels\n(loc_ch, llm_ch, exp_ch);
fork
|Orchestrator|
:Loop: Send Locations → loc_ch;
:Close loc_ch;
fork again
|LLM Worker|
while (loc_ch has items?) is (yes)
:Receive Location;
:IPersonaSelectionStrategy::SelectPersona(\n personas_palette_);
note right
Guaranteed cache hit from startup.
Returns a Persona struct carrying
style_affinities, abv_range,
ibu_preference, checkin_weight.
end note
:NamesByCountry::SampleName(\n location.iso3166_1);
note right
Deterministic lookup — no LLM involved.
Name selected from pre-keyed table
and passed into the generation prompt.
end note
:GenerateUser(location, persona, sampled_name)\nvia DataGenerator;
note right
LLM receives: Location fields + persona
description + sampled name. Generates
bio and preference signals grounded
in locale and persona.
end note
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "llm");
:Send GeneratedUser → llm_ch;
endwhile (no)
:Close llm_ch;
fork again
|SQLite Worker|
while (llm_ch has items?) is (yes)
:Receive GeneratedUser;
:ProcessUser(user) → sqlite3_int64;
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "sqlite");
:Append → user_pool_;
endwhile (no)
end fork
|Orchestrator|
:Join LLM Worker, SQLite Worker;
' ═══════════════════════════════════════════
' PHASE 1 — BREWERY & BEER GENERATION
' ═══════════════════════════════════════════
:RunBreweryAndBeerPhase(sampled_locations);
:Create BoundedChannels\n(loc_ch, llm_ch, exp_ch);
fork
|Orchestrator|
:Loop: Send Locations → loc_ch;
:Close loc_ch;
fork again
|Enrichment Workers (xN)|
while (loc_ch has items?) is (yes)
:Receive Location;
:GetLocationContext(location,\nBreweryContextStrategy);
:PipelineLogger::Log(Info,\n BreweryAndBeerGeneration,\n city, nullopt, "enrichment");
:Send EnrichedCity → llm_ch;
endwhile (no)
|Orchestrator|
:Join Enrichment Workers;
:Close llm_ch;
fork again
|LLM Worker|
while (llm_ch has items?) is (yes)
:Receive EnrichedCity;
:GenerateBrewery(location, context)\nvia DataGenerator;
:IBeerSelectionStrategy::SelectStyles(\n brewery, beer_style_palette_);
while (For each selected BeerStyle?) is (remaining)
:GetStyleContextFromCache(style);
note right
Guaranteed cache hit from startup.
end note
:GenerateBeer(brewery, style_context)\nvia DataGenerator;
:Attach GeneratedBeer to Brewery bundle;
endwhile (done)
:PipelineLogger::Log(Info,\n BreweryAndBeerGeneration,\n city, brewery_id, "llm");
:Send BreweryWithBeers Bundle → exp_ch;
endwhile (no)
:Close exp_ch;
fork again
|SQLite Worker|
while (exp_ch has items?) is (yes)
:Receive BreweryWithBeers Bundle;
:ProcessBrewery(brewery) → brewery_id;
:Append → brewery_pool_;
while (For each beer in bundle?) is (remaining)
:Set beer.brewery_id = brewery_id;
:ProcessBeer(beer) → sqlite3_int64;
:Append → beer_pool_;
endwhile (done)
:PipelineLogger::Log(Info,\n BreweryAndBeerGeneration,\n city, brewery_id, "sqlite");
endwhile (no)
end fork
|Orchestrator|
:Join LLM Worker, SQLite Worker;
note right
Both brewery_pool_ and beer_pool_
are now completely populated.
end note
' ═══════════════════════════════════════════
' PHASE 2 — CHECKIN GENERATION
' ═══════════════════════════════════════════
:RunCheckinPhase();
:ICheckinDistributionStrategy::\nAssignActivityWeights(user_pool_);
note right
Weights seeded from each user's
persona.checkin_weight. J-curve profile
emerges from persona distribution.
end note
while (For each GeneratedUser in user_pool_?) is (remaining)
:CheckinsForUser(user, brewery_pool_.size());
while (For each checkin index?) is (remaining)
:TimestampFor(user, index);
:Select brewery from brewery_pool_;
:GenerateCheckin(user, brewery, timestamp)\nvia DataGenerator;
:ProcessCheckin(checkin) → sqlite3_int64;
:PipelineLogger::Log(Info, CheckinGeneration,\n nullopt, checkin_id, "sqlite");
:Append → checkin_pool_;
endwhile (done)
endwhile (done)
' ═══════════════════════════════════════════
' PHASE 3 — RATING GENERATION
' ═══════════════════════════════════════════
:RunRatingPhase();
note right
Beer selection biased by
user.persona.style_affinities and abv_range.
Rating skew modulated per persona.
end note
while (For each GeneratedCheckin in checkin_pool_?) is (remaining)
:Match brewery_id → select beer from beer_pool_\n(same brewery_id, biased by persona affinities);
if (Beer exists for brewery?) then (yes)
:GenerateRating(user, beer, checkin_id)\nvia DataGenerator;
:ProcessRating(rating);
:PipelineLogger::Log(Info, RatingGeneration,\n nullopt, rating_id, "sqlite");
else (no)
:PipelineLogger::Log(Warn, RatingGeneration,\n nullopt, brewery_id, "sqlite");
:Skip — brewery has no beers;
endif
endwhile (done)
' ═══════════════════════════════════════════
' TEARDOWN
' ═══════════════════════════════════════════
|Main|
:Finalize SqliteExportService;
note right
COMMIT covers all five fixture types.
end note
:Close log_ch;
:Join Log Worker;
note right
Drain guarantees no LogEntry is
dropped at shutdown.
end note
:spdlog::info "Pipeline complete in X ms";
stop
@enduml

546
docs/pipeline/diagrams/planned/class.puml → pipeline/diagrams/planned/class.puml
View File

@@ -1,16 +1,51 @@
@startuml class_diagram
@startuml future_possible_architecture
' ==========================================
' CONFIGURATION & STYLING
' ==========================================
!include ../biergarten-weizen-theme.puml
skinparam classAttributeFontSize 9
skinparam defaultFontSize 25
skinparam titleFontSize 30
left to right direction
skinparam linetype ortho
title Biergarten Data Pipeline — Class Diagram
' --- Typography ---
skinparam defaultFontName "DM Sans"
skinparam defaultFontSize 14
skinparam titleFontName "Volkhov"
skinparam titleFontSize 20
package "Domain: Models" {
' --- Global Colors ---
skinparam backgroundColor #FCFCF7
skinparam defaultFontColor #14180C
skinparam titleFontColor #14180C
skinparam ArrowColor #656F33
skinparam class {
BackgroundColor #EBECE3
HeaderBackgroundColor #CBD2B5
BorderColor #4A5837
ArrowColor #656F33
FontColor #14180C
}
skinparam package {
BackgroundColor #DBEEDD
BorderColor #4A5837
FontColor #14180C
}
skinparam note {
BackgroundColor #DBEEDD
BorderColor #4A5837
FontColor #14180C
}
skinparam monochrome reverse
title The Biergarten Data Pipeline — Planned Architecture
' ==========================================
' DOMAIN MODELS
' ==========================================
package "Domain Models" {
class Location {
+ city : std::string
@@ -27,9 +62,8 @@ package "Domain: Models" {
+ text : std::string
+ completeness : Completeness
+ char_count : size_t
}
enum Completeness {
--
<<enum>> Completeness
Full
Partial
Absent
@@ -67,50 +101,11 @@ package "Domain: Models" {
}
class UserResult {
+ first_name : std::string
+ last_name : std::string
+ gender : std::string
+ username : std::string
+ bio : std::string
+ activity_weight : float
}
class Name {
+ first_name : std::string
+ last_name : std::string
+ gender : std::string
}
class ForenameEntry {
+ name : std::string
+ gender : std::string
}
note right of ForenameEntry
forename_list = std::unordered_set<ForenameEntry>
surname_list = std::unordered_set<std::string>
(aliases declared in curated_data_service.h).
ICuratedDataService::LoadForenamesByCountry() /
LoadSurnamesByCountry() each return a flat
std::unordered_map<std::string, forename_list>
or <std::string, surname_list> keyed by ISO
3166-1 country code -- no NamesByCountry
wrapper class. SampleName(forenames_by_country,
surnames_by_country, iso3166_1, rng) is a free
function, not a method, so it can be called by
any per-city worker without owning the maps.
Sourced from popular-names-by-country-dataset
(CC0) -- see ETHICS-AND-KNOWN-ISSUES.md. Gender
is preserved per forename so it can be threaded
through to persona/bio generation later; surnames
carry no gender in the source data. Pairing
happens at sample time, not at fixture-authoring
time, so no information from the source dataset
is discarded up front.
end note
ForenameEntry ..> Name : SampleName() produces
class CheckinResult {
+ checked_in_at : std::string
+ note : std::string
@@ -121,110 +116,138 @@ package "Domain: Models" {
+ note : std::string
}
class GenerationMetadata {
+ generation_id : uint64_t
+ generated_time : std::string
+ context_provided : bool
+ generated_with : std::string
}
class GeneratedBrewery {
+ brewery_id : uint64_t
+ brewery_id : sqlite3_int64
+ location : Location
+ brewery : BreweryResult
+ context_completeness : LocationContext::Completeness
+ metadata : GenerationMetadata
+ generated_at : std::string
}
class GeneratedBeer {
+ beer_id : uint64_t
+ brewery_id : uint64_t
+ beer_id : sqlite3_int64
+ brewery_id : sqlite3_int64
+ location : Location
+ style : BeerStyle
+ beer : BeerResult
+ metadata : GenerationMetadata
+ generated_at : std::string
}
class GeneratedUser {
+ user_id : uint64_t
+ user_id : sqlite3_int64
+ location : Location
+ user : UserResult
+ email : std::string
+ date_of_birth : std::string
+ password : std::string
+ metadata : GenerationMetadata
+ generated_at : std::string
}
note right of GeneratedUser
email, date_of_birth, and password are
programmatically generated by the orchestrator
(not LLM-authored) so a downstream auth-account
seeding consumer can register real accounts from
this export. See ROADMAP.md §1.
end note
class GeneratedCheckin {
+ checkin_id : uint64_t
+ user_id : uint64_t
+ brewery_id : uint64_t
+ checkin_id : sqlite3_int64
+ user_id : sqlite3_int64
+ brewery_id : sqlite3_int64
+ checkin : CheckinResult
+ metadata : GenerationMetadata
+ generated_at : std::string
}
class GeneratedRating {
+ user_id : uint64_t
+ beer_id : uint64_t
+ checkin_id : uint64_t
+ user_id : sqlite3_int64
+ beer_id : sqlite3_int64
+ checkin_id : sqlite3_int64
+ rating : RatingResult
+ metadata : GenerationMetadata
+ generated_at : std::string
}
class GeneratedFollow {
+ follower_id : uint64_t
+ followed_id : uint64_t
+ metadata : GenerationMetadata
}
class UserPersona {
+ name: std::string
+ description: std::string
+ style_affinities: std::vector<std::string>
}
LocationContext *-- Completeness
}
package "Domain: Application Configuration" {
class SamplingOptions {
+ temperature: float = 1.0F
+ top_p: float = 0.95F
+ top_k: uint32_t = 64
+ n_ctx: uint32_t = 8192
+ seed: int = -1
+ temperature : float = 1.0F
+ top_p : float = 0.95F
+ top_k : uint32_t = 64
+ n_ctx : uint32_t = 8192
+ seed : int = -1
}
class GeneratorOptions {
+ model_path: std::filesystem::path
+ use_mocked: bool = false
+ sampling: std::optional<SamplingOptions>
+ model_path : std::filesystem::path
+ use_mocked : bool = false
+ sampling : SamplingOptions
}
class PipelineOptions {
+ output_path: std::filesystem::path
+ log_path: std::filesystem::path
+ output_path : std::filesystem::path
+ log_path : std::filesystem::path
}
class ApplicationOptions {
+ generator: GeneratorOptions
+ pipeline: PipelineOptions
+ generator : GeneratorOptions
+ pipeline : PipelineOptions
}
' --- Domain Model Relationships ---
ApplicationOptions *-- GeneratorOptions
ApplicationOptions *-- PipelineOptions
GeneratorOptions o-- SamplingOptions
GeneratorOptions *-- SamplingOptions
LocationContext *-- Completeness
}
package "Domain: Policy" {
' ==========================================
' LOGGING
' ==========================================
package "Logging" {
enum LogLevel {
Debug
Info
Warn
Error
}
enum PipelinePhase {
Startup
UserGeneration
BreweryAndBeerGeneration
CheckinGeneration
RatingGeneration
Teardown
}
class LogEntry {
+ timestamp : std::chrono::system_clock::time_point
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
+ city : std::optional<std::string>
+ entity_id : std::optional<std::string>
+ worker : std::optional<std::string>
}
interface Logger <<interface>> {
+ Log(level, phase, message,\n city, entity_id, worker) : void
}
class PipelineLogger {
- log_ch_ : BoundedChannel<LogEntry>&
+ Log(level, phase, message,\n city, entity_id, worker) : void
}
class LogWorker {
- log_ch_ : BoundedChannel<LogEntry>&
+ Run() : void
- FormatTimestamp(tp) : std::string
- ToSpdlogLevel(level) : spdlog::level::level_enum
- ToString(phase) : std::string
}
' --- Logging Relationships ---
LogEntry *-- LogLevel
LogEntry *-- PipelinePhase
PipelineLogger ..> LogEntry : emits
LogWorker ..> LogEntry : consumes
}
' ==========================================
' DOMAIN POLICY
' ==========================================
package "Domain Policy" {
interface ContextStrategy <<interface>> {
+ QueriesFor(loc : const Location&) : std::vector<std::string>
@@ -274,139 +297,66 @@ package "Domain: Policy" {
+ TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string
}
class RandomCheckinStrategy {
- rng_ : std::mt19937
- min_checkins_ : size_t
- max_checkins_ : size_t
+ AssignActivityWeights(users : std::vector<GeneratedUser>&) : void
+ CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t
+ TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string
}
' ==========================================
' ORCHESTRATION
' ==========================================
package "Orchestration" {
interface DataPreloader <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
}
interface FollowGenerationStrategy <<interface>> {
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
}
class RandomFollowStrategy {
- rng_ : std::mt19937
- min_follows_ : size_t
- max_follows_ : size_t
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
}
class ActivityWeightedFollowStrategy {
- rng_ : std::mt19937
- min_follows_ : size_t
- max_follows_ : size_t
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
class BiergartenPipelineOrchestrator {
- preloader_ : std::unique_ptr<DataPreloader>
- enrichment_service_ : std::unique_ptr<EnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- logger_ : std::unique_ptr<Logger>
- exporter_ : std::unique_ptr<ExportService>
- brewery_context_strategy_ : std::unique_ptr<ContextStrategy>
- sampling_strategy_ : std::unique_ptr<SamplingStrategy>
- beer_selection_strategy_ : std::unique_ptr<BeerSelectionStrategy>
- checkin_strategy_ : std::unique_ptr<CheckinDistributionStrategy>
- beer_style_palette_ : std::vector<BeerStyle>
- options_ : ApplicationOptions
--
- user_pool_ : std::vector<GeneratedUser>
- brewery_pool_ : std::vector<GeneratedBrewery>
- beer_pool_ : std::vector<GeneratedBeer>
- checkin_pool_ : std::vector<GeneratedCheckin>
--
+ Run() : bool
- RunUserPhase(locations : const std::vector<Location>&) : void
- RunBreweryAndBeerPhase(locations : const std::vector<Location>&) : void
- RunCheckinPhase() : void
- RunRatingPhase() : void
}
}
package "Infrastructure: Logging" {
enum LogLevel {
Debug
Info
Warn
Error
}
enum PipelinePhase {
Startup
UserGeneration
BreweryAndBeerGeneration
CheckinGeneration
RatingGeneration
FollowGeneration
Teardown
}
class LogEntry {
+ timestamp : std::chrono::system_clock::time_point
+ level : LogLevel
+ phase : PipelinePhase
+ message : std::string
+ worker : std::optional<std::string>
}
interface ILogger <<interface>> {
+ Log(entry : const LogEntry&) : void
}
class LogProducer {
- channel_ : BoundedChannel<LogEntry>&
+ Log(entry : const LogEntry&) : void
}
class LogDispatcher {
- channel_ : BoundedChannel<LogEntry>&
+ Run() : void
- ToSpdlogLevel(level) : spdlog::level::level_enum
}
LogEntry *-- LogLevel
LogEntry *-- PipelinePhase
ILogger <|.. LogProducer
LogProducer ..> LogEntry : emits
LogDispatcher ..> LogEntry : consumes
}
package "Infrastructure: Pipeline Channel" {
class "BoundedChannel<T>" as BoundedChannel {
- queue_ : std::queue<T>
- mutex_ : std::mutex
- not_full_ : std::condition_variable
- not_empty_ : std::condition_variable
- capacity_ : size_t
- closed_ : bool
+ Send(item : T) : void
+ Receive() : std::optional<T>
+ Close() : void
}
}
package "Infrastructure: Data Preloading" {
interface ICuratedDataService <<interface>> {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
' ==========================================
' INFRASTRUCTURE: PRELOADING
' ==========================================
package "Infrastructure: Preloading" {
class JsonLoader {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
+ LoadLocations(filepath : const std::filesystem::path&) : std::vector<Location>
+ LoadBeerStyles(filepath : const std::filesystem::path&) : std::vector<BeerStyle>
+ LoadPersonas(filepath : const std::filesystem::path&) : std::vector<Persona>
+ LoadNamesByCountry(filepath : const std::filesystem::path&) : NamesByCountry
}
note right of JsonLoader
Each Load* call is memoized after its first
parse -- implemented today, ahead of the rest
of this package (LoadBeerStyles is still planned).
end note
class MockCuratedDataService {
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
}
note right of MockCuratedDataService
Fixed in-memory dataset used in --mocked mode,
implemented today for Locations/Personas/
Forenames/Surnames; would need a LoadBeerStyles
fixture too once that method is implemented.
end note
}
' ==========================================
' INFRASTRUCTURE: ENRICHMENT
' ==========================================
package "Infrastructure: Enrichment" {
interface EnrichmentService <<interface>> {
@@ -425,37 +375,25 @@ package "Infrastructure: Enrichment" {
+ UrlEncode(value : const std::string&) : std::string
}
class HttpWebClient {
class CURLWebClient {
+ Get(url : const std::string&) : std::string
+ UrlEncode(value : const std::string&) : std::string
}
}
package "Infrastructure: Prompting" {
interface IPromptDirectory <<interface>> {
+ Load(key : std::string_view) : std::string
}
class PromptDirectory {
- prompt_dir_ : std::filesystem::path
- cache_ : std::unordered_map<std::string, std::string>
+ PromptDirectory(prompt_dir : const std::filesystem::path&)
+ Load(key : std::string_view) : std::string
}
IPromptDirectory <|.. PromptDirectory
}
package "Infrastructure: Data Generation" {
' ==========================================
' INFRASTRUCTURE: GENERATION
' ==========================================
package "Infrastructure: Generation" {
interface DataGenerator <<interface>> {
+ GenerateBrewery(location : const Location&,\n context : const LocationContext&) : BreweryResult
+ GenerateBeer(brewery_id : uint64_t,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult
+ GenerateUser(city : const EnrichedCity&,\n persona : const UserPersona&,\n name : const Name&) : UserResult
+ GenerateBeer(brewery_id : sqlite3_int64,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult
+ GenerateUser(location : const Location&) : UserResult
+ GenerateCheckin(user : const GeneratedUser&,\n brewery : const GeneratedBrewery&,\n timestamp : const std::string&) : CheckinResult
+ GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : uint64_t) : RatingResult
+ GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : sqlite3_int64) : RatingResult
}
class MockGenerator {
@@ -471,7 +409,6 @@ package "Infrastructure: Data Generation" {
- model_ : ModelHandle
- context_ : ContextHandle
- prompt_formatter_ : std::unique_ptr<PromptFormatter>
- prompt_directory_ : std::unique_ptr<IPromptDirectory>
- rng_ : std::mt19937
+ GenerateBrewery(...) : BreweryResult
+ GenerateBeer(...) : BeerResult
@@ -495,16 +432,39 @@ package "Infrastructure: Data Generation" {
}
package "Infrastructure: Data Export" {
' ==========================================
' INFRASTRUCTURE: PIPELINE CHANNEL
' ==========================================
package "Infrastructure: Pipeline Channel" {
class "BoundedChannel<T>" as BoundedChannel {
- queue_ : std::queue<T>
- mutex_ : std::mutex
- not_full_ : std::condition_variable
- not_empty_ : std::condition_variable
- capacity_ : size_t
- closed_ : bool
+ Send(item : T) : void
+ Receive() : std::optional<T>
+ Close() : void
}
}
' ==========================================
' INFRASTRUCTURE: EXPORT
' ==========================================
package "Infrastructure: Export" {
interface ExportService <<interface>> {
+ Initialize() : void
+ ProcessBrewery(brewery : const GeneratedBrewery&) : uint64_t
+ ProcessBeer(beer : const GeneratedBeer&) : uint64_t
+ ProcessUser(user : const GeneratedUser&) : uint64_t
+ ProcessCheckin(checkin : const GeneratedCheckin&) : uint64_t
+ ProcessBrewery(brewery : const GeneratedBrewery&) : sqlite3_int64
+ ProcessBeer(beer : const GeneratedBeer&) : sqlite3_int64
+ ProcessUser(user : const GeneratedUser&) : sqlite3_int64
+ ProcessCheckin(checkin : const GeneratedCheckin&) : sqlite3_int64
+ ProcessRating(rating : const GeneratedRating&) : void
+ ProcessFollow(follow : const GeneratedFollow&) : void
+ Finalize() : void
}
@@ -517,17 +477,15 @@ package "Infrastructure: Data Export" {
- insert_user_stmt_ : SqliteStatementHandle
- insert_checkin_stmt_ : SqliteStatementHandle
- insert_rating_stmt_ : SqliteStatementHandle
- insert_follow_stmt_ : SqliteStatementHandle
- transaction_open_ : bool
- location_cache_ : std::unordered_map<std::string, uint64_t>
- brewery_cache_ : std::unordered_map<std::string, uint64_t>
- location_cache_ : std::unordered_map<std::string, sqlite3_int64>
- brewery_cache_ : std::unordered_map<std::string, sqlite3_int64>
+ Initialize() : void
+ ProcessRecord(brewery : const GeneratedBrewery&) : uint64_t
+ ProcessRecord(beer : const GeneratedBeer&) : uint64_t
+ ProcessRecord(user : const GeneratedUser&) : uint64_t
+ ProcessRecord(checkin : const GeneratedCheckin&) : uint64_t
+ ProcessRecord(rating : const GeneratedRating&) : void
+ ProcessRecord(follow : const GeneratedFollow&) : void
+ ProcessBrewery(brewery : const GeneratedBrewery&) : sqlite3_int64
+ ProcessBeer(beer : const GeneratedBeer&) : sqlite3_int64
+ ProcessUser(user : const GeneratedUser&) : sqlite3_int64
+ ProcessCheckin(checkin : const GeneratedCheckin&) : sqlite3_int64
+ ProcessRating(rating : const GeneratedRating&) : void
+ Finalize() : void
- InitializeSchema() : void
- PrepareStatements() : void
@@ -545,41 +503,17 @@ package "Infrastructure: Data Export" {
}
class BiergartenPipelineOrchestrator {
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
- enrichment_service_ : std::unique_ptr<EnrichmentService>
- generator_ : std::unique_ptr<DataGenerator>
- logger_ : std::unique_ptr<Logger>
- exporter_ : std::unique_ptr<ExportService>
- brewery_context_strategy_ : std::unique_ptr<ContextStrategy>
- sampling_strategy_ : std::unique_ptr<SamplingStrategy>
- beer_selection_strategy_ : std::unique_ptr<BeerSelectionStrategy>
- checkin_strategy_ : std::unique_ptr<CheckinDistributionStrategy>
- follow_strategy_ : std::unique_ptr<FollowGenerationStrategy>
- beer_style_palette_ : std::vector<BeerStyle>
- options_ : ApplicationOptions
--
- user_pool_ : std::vector<GeneratedUser>
- brewery_pool_ : std::vector<GeneratedBrewery>
- beer_pool_ : std::vector<GeneratedBeer>
- checkin_pool_ : std::vector<GeneratedCheckin>
- follow_pool_ : std::vector<GeneratedFollow>
--
+ Run() : bool
- RunUserPhase(locations : const std::vector<Location>&) : void
- RunBreweryAndBeerPhase(locations : const std::vector<Location>&) : void
- RunCheckinPhase() : void
- RunRatingPhase() : void
- RunFollowPhase() : void
}
' ==========================================
' GLOBAL RELATIONSHIPS
' ==========================================
' --- Orchestration Aggregations (Services & Strategies) ---
BiergartenPipelineOrchestrator *-- ICuratedDataService
BiergartenPipelineOrchestrator *-- DataPreloader
BiergartenPipelineOrchestrator *-- EnrichmentService
BiergartenPipelineOrchestrator *-- DataGenerator
BiergartenPipelineOrchestrator *-- ExportService
BiergartenPipelineOrchestrator *-- CheckinDistributionStrategy
BiergartenPipelineOrchestrator *-- FollowGenerationStrategy
BiergartenPipelineOrchestrator *-- SamplingStrategy
BiergartenPipelineOrchestrator *-- BeerSelectionStrategy
BiergartenPipelineOrchestrator *-- ApplicationOptions
@@ -590,22 +524,17 @@ BiergartenPipelineOrchestrator *-- "0..*" GeneratedUser : user_pool_
BiergartenPipelineOrchestrator *-- "0..*" GeneratedBrewery : brewery_pool_
BiergartenPipelineOrchestrator *-- "0..*" GeneratedBeer : beer_pool_
BiergartenPipelineOrchestrator *-- "0..*" GeneratedCheckin : checkin_pool_
BiergartenPipelineOrchestrator *-- "0..*" GeneratedFollow : follow_pool_
' --- Interfaces & Implementations ---
ICuratedDataService <|.. JsonLoader
ICuratedDataService <|.. MockCuratedDataService
DataPreloader <|.. JsonLoader
Logger <|.. PipelineLogger
ContextStrategy <|.. BreweryContextStrategy
ContextStrategy <|.. BeerContextStrategy
SamplingStrategy <|.. UniformSamplingStrategy
BeerSelectionStrategy <|.. RandomBeerSelectionStrategy
CheckinDistributionStrategy <|.. JCurveCheckinStrategy
CheckinDistributionStrategy <|.. RandomCheckinStrategy
FollowGenerationStrategy <|.. RandomFollowStrategy
FollowGenerationStrategy <|.. ActivityWeightedFollowStrategy
EnrichmentService <|.. WikipediaService
WebClient <|.. HttpWebClient
WebClient <|.. CURLWebClient
DataGenerator <|.. MockGenerator
DataGenerator <|.. LlamaGenerator
PromptFormatter <|.. Gemma4JinjaPromptFormatter
@@ -616,7 +545,6 @@ DateTimeProvider <|.. SystemDateTimeProvider
WikipediaService *-- WebClient
WikipediaService ..> ContextStrategy
LlamaGenerator *-- PromptFormatter
LlamaGenerator *-- IPromptDirectory
LlamaGenerator ..> GeneratorOptions
SqliteExportService *-- DateTimeProvider
@@ -629,18 +557,12 @@ EnrichedCity *-- Location
EnrichedCity *-- LocationContext
GeneratedBrewery *-- Location
GeneratedBrewery *-- BreweryResult
GeneratedBrewery *-- GenerationMetadata
GeneratedBeer *-- Location
GeneratedBeer *-- BeerStyle
GeneratedBeer *-- BeerResult
GeneratedBeer *-- GenerationMetadata
GeneratedUser *-- Location
GeneratedUser *-- UserResult
GeneratedUser *-- GenerationMetadata
GeneratedCheckin *-- CheckinResult
GeneratedCheckin *-- GenerationMetadata
GeneratedRating *-- RatingResult
GeneratedRating *-- GenerationMetadata
GeneratedFollow *-- GenerationMetadata
@enduml

1
pipeline/diagrams/planned/output/biergarten_activity.svg Normal file
View File

File diff suppressed because one or more lines are too long

1
pipeline/diagrams/planned/output/future_possible_architecture.svg Normal file
View File

File diff suppressed because one or more lines are too long

0
tooling/pipeline/format.sh → pipeline/format.sh
View File

83
pipeline/includes/biergarten_data_generator.h Normal file
View File

@@ -0,0 +1,83 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_
#define BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_
/**
* @file biergarten_data_generator.h
* @brief Core orchestration class for pipeline data generation.
*/
#include <memory>
#include <span>
#include <vector>
#include "data_generation/data_generator.h"
#include "data_model/enriched_city.h"
#include "data_model/generated_brewery.h"
#include "data_model/location.h"
#include "services/enrichment_service.h"
#include "services/export_service.h"
/**
* @brief Main data generator class for the Biergarten pipeline.
*
* This class encapsulates the core logic for generating brewery data.
* It handles location loading, city enrichment, and brewery generation.
*/
class BiergartenDataGenerator {
public:
/**
* @brief Construct a BiergartenDataGenerator with injected dependencies.
*
* @param context_service Context provider for sampled locations.
* @param generator Brewery and user data generator.
* @param exporter Storage backend for generated brewery data.
*/
BiergartenDataGenerator(std::unique_ptr<IEnrichmentService> context_service,
std::unique_ptr<DataGenerator> generator,
std::unique_ptr<IExportService> exporter);
/**
* @brief Run the data generation pipeline.
*
* Performs the following steps:
* 1. Load curated locations from JSON
* 2. Resolve context for each city using the injected context service
* 3. Generate brewery data for sampled cities
*
* @return true if successful, false if not
*/
bool Run();
private:
/// @brief Owning context provider dependency.
std::unique_ptr<IEnrichmentService> context_service_;
/// @brief Generator dependency selected in the composition root.
std::unique_ptr<DataGenerator> generator_;
/// @brief Storage backend for generated brewery records.
std::unique_ptr<IExportService> exporter_;
/**
* @brief Load locations from JSON and sample cities.
*
* @return Vector of sampled locations capped at 50 entries.
*/
static std::vector<Location> QueryCitiesWithCountries();
/**
* @brief Generate breweries for enriched cities.
*
* @param cities Span of enriched city data.
*/
void GenerateBreweries(std::span<const EnrichedCity> cities);
/**
* @brief Log the generated brewery results.
*/
void LogResults() const;
/// @brief Stores generated brewery data.
std::vector<GeneratedBrewery> generated_breweries_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_BIERGARTEN_DATA_GENERATOR_H_

14
tooling/pipeline/includes/data_generation/data_generator.h → pipeline/includes/data_generation/data_generator.h
View File

@@ -8,7 +8,9 @@
#include <string>
#include "data_model/generated_models.h"
#include "data_model/brewery_result.h"
#include "data_model/location.h"
#include "data_model/user_result.h"
/**
* @brief Interface for data generator implementations.
@@ -28,16 +30,12 @@ class DataGenerator {
const std::string& region_context) = 0;
/**
* @brief Generates a user profile grounded in a sampled name and persona.
* @brief Generates a user profile for a locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype grounding the generated bio.
* @param name Sampled first/last name (and gender)
* @param locale Locale hint used by generator.
* @return User generation result.
*/
virtual UserResult GenerateUser(const EnrichedCity& city,
const UserPersona& persona,
const Name& name) = 0;
virtual UserResult GenerateUser(const std::string& locale) = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_DATA_GENERATOR_H_

40
tooling/pipeline/includes/data_generation/llama_generator.h → pipeline/includes/data_generation/llama_generator.h
View File

@@ -1,13 +1,14 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_H_
#include <filesystem>
/**
* @file data_generation/llama_generator.h
* @brief llama.cpp-backed implementation of DataGenerator.
*/
#include <cstdint>
#include <filesystem>
#include <memory>
#include <random>
#include <string>
@@ -15,9 +16,7 @@
#include "data_generation/data_generator.h"
#include "data_generation/prompt_formatting/prompt_formatter.h"
#include "data_model/models.h"
#include "services/logging/logger.h"
#include "services/prompting/prompt_directory.h"
#include "data_model/application_options.h"
struct llama_model;
struct llama_context;
@@ -34,18 +33,21 @@ class LlamaGenerator final : public DataGenerator {
* @param options Parsed application options.
* @param model_path Filesystem path to GGUF model assets.
* @param prompt_formatter Formatter that produces model-specific prompts.
* @param prompt_directory Directory service for loading named prompt files.
*/
LlamaGenerator(const ApplicationOptions& options,
const std::string& model_path, std::shared_ptr<ILogger> logger,
std::unique_ptr<IPromptFormatter> prompt_formatter,
std::unique_ptr<IPromptDirectory> prompt_directory);
const std::string& model_path,
std::unique_ptr<IPromptFormatter> prompt_formatter);
~LlamaGenerator() override;
// disable copy constructor
LlamaGenerator(const LlamaGenerator&) = delete;
// disable copy assignment operator
LlamaGenerator& operator=(const LlamaGenerator&) = delete;
// disable move constructor
LlamaGenerator(LlamaGenerator&&) = delete;
// disable move assignment operator
LlamaGenerator& operator=(LlamaGenerator&&) = delete;
/**
@@ -59,15 +61,12 @@ class LlamaGenerator final : public DataGenerator {
const std::string& region_context) override;
/**
* @brief Generates a user profile grounded in a sampled name and persona.
* @brief Generates a user profile for the provided locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype grounding the generated bio.
* @param name Sampled first/last name -- not LLM-invented.
* @param locale Locale hint.
* @return Generated user profile.
*/
UserResult GenerateUser(const EnrichedCity& city, const UserPersona& persona,
const Name& name) override;
UserResult GenerateUser(const std::string& locale) override;
private:
static constexpr int32_t kDefaultMaxTokens = 10000;
@@ -120,6 +119,15 @@ class LlamaGenerator final : public DataGenerator {
int max_tokens = kDefaultMaxTokens,
std::string_view grammar = {});
/**
* @brief Loads the brewery system prompt from disk.
*
* @param prompt_file_path Prompt file path to try first.
* @return Loaded prompt text.
*/
std::string LoadBrewerySystemPrompt(
const std::filesystem::path& prompt_file_path);
ModelHandle model_;
ContextHandle context_;
float sampling_temperature_ = 1.0F;
@@ -127,10 +135,8 @@ class LlamaGenerator final : public DataGenerator {
uint32_t sampling_top_k_ = kDefaultSamplingTopK;
std::mt19937 rng_;
uint32_t n_ctx_ = kDefaultContextSize;
int n_gpu_layers_ = 0;
std::shared_ptr<ILogger> logger_;
std::string brewery_system_prompt_;
std::unique_ptr<IPromptFormatter> prompt_formatter_;
std::unique_ptr<IPromptDirectory> prompt_directory_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_H_

16
tooling/pipeline/includes/data_generation/llama_generator_helpers.h → pipeline/includes/data_generation/llama_generator_helpers.h
View File

@@ -12,7 +12,7 @@
#include <string>
#include <string_view>
#include "data_model/generated_models.h"
#include "data_model/brewery_result.h"
struct llama_vocab;
using llama_token = int32_t;
@@ -47,18 +47,4 @@ void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
std::optional<std::string> ValidateBreweryJson(const std::string& raw,
BreweryResult& brewery_out);
/**
* @brief Validates and parses user JSON output.
*
* Only populates `username`, `bio`, and `activity_weight` -- `first_name`
* and `last_name` are not LLM-authored and are set separately from the
* sampled Name.
*
* @param raw Raw model output.
* @param user_out Parsed user payload.
* @return Validation error message if invalid, or std::nullopt on success.
*/
std::optional<std::string> ValidateUserJson(const std::string& raw,
UserResult& user_out);
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_LLAMA_GENERATOR_HELPERS_H_

33
tooling/pipeline/includes/data_generation/mock_generator.h → pipeline/includes/data_generation/mock_generator.h
View File

@@ -28,16 +28,12 @@ class MockGenerator final : public DataGenerator {
const std::string& region_context) override;
/**
* @brief Generates deterministic user data grounded in a sampled name and
* persona.
* @brief Generates deterministic user data for a locale.
*
* @param city Enriched city the user is associated with.
* @param persona Persona archetype.
* @param name Sampled first/last name, copied directly into the result.
* @param locale Locale hint.
* @return Generated user result.
*/
UserResult GenerateUser(const EnrichedCity& city, const UserPersona& persona,
const Name& name) override;
UserResult GenerateUser(const std::string& locale) override;
private:
/**
@@ -48,26 +44,6 @@ class MockGenerator final : public DataGenerator {
*/
static size_t DeterministicHash(const Location& location);
/**
* @brief Combines city, persona, and name into a stable hash value.
*
* @param location City and country names.
* @param persona Persona archetype.
* @param name Sampled first/last name.
* @return Deterministic hash value.
*/
static size_t DeterministicHash(const Location& location,
const UserPersona& persona, const Name& name);
// Hash stride constants for deterministic distribution across fixed-size
// arrays. These coprime strides spread hash values uniformly without
// clustering, ensuring diverse output across different hash inputs.
static constexpr size_t kNounHashStride = 7;
static constexpr size_t kDescriptionHashStride = 13;
static constexpr size_t kBioHashStride = 11;
static constexpr size_t kActivityWeightHashStride = 17;
static constexpr size_t kActivityWeightHashRange = 1000;
static constexpr std::array<std::string_view, 18> kBreweryAdjectives = {
"Craft", "Heritage", "Local", "Artisan", "Pioneer", "Golden",
"Modern", "Classic", "Summit", "Northern", "Riverstone", "Barrel",
@@ -141,8 +117,7 @@ class MockGenerator final : public DataGenerator {
"Visits breweries for the stories, stays for the flagship pours.",
"Craft beer fan mapping tasting notes and favorite brew routes.",
"Always ready to trade recommendations for underrated local breweries.",
"Keeping a running list of must-try collab releases and tap "
"takeovers."};
"Keeping a running list of must-try collab releases and tap takeovers."};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_MOCK_GENERATOR_H_

5
tooling/pipeline/includes/data_generation/prompt_formatting/gemma4_jinja_prompt_formatter.h → pipeline/includes/data_generation/prompt_formatting/gemma4_jinja_prompt_formatter.h
View File

@@ -1,5 +1,4 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_GEMMA4_JINJA_PROMPT_FORMATTER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_GEMMA4_JINJA_PROMPT_FORMATTER_H_
#pragma once
#include <string>
#include <string_view>
@@ -14,5 +13,3 @@ class Gemma4JinjaPromptFormatter final : public IPromptFormatter {
[[nodiscard]] std::string Format(std::string_view system_prompt,
std::string_view user_prompt) const override;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_GEMMA4_JINJA_PROMPT_FORMATTER_H_

5
tooling/pipeline/includes/data_generation/prompt_formatting/prompt_formatter.h → pipeline/includes/data_generation/prompt_formatting/prompt_formatter.h
View File

@@ -1,5 +1,4 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_PROMPT_FORMATTER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_PROMPT_FORMATTER_H_
#pragma once
#include <string>
#include <string_view>
@@ -16,5 +15,3 @@ class IPromptFormatter {
[[nodiscard]] virtual std::string Format(
std::string_view system_prompt, std::string_view user_prompt) const = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_GENERATION_PROMPT_FORMATTING_PROMPT_FORMATTER_H_

42
pipeline/includes/data_model/application_options.h Normal file
View File

@@ -0,0 +1,42 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_APPLICATION_OPTIONS_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_APPLICATION_OPTIONS_H_
/**
* @file data_model/application_options.h
* @brief Program options for the Biergarten pipeline application.
*/
#include <cstdint>
#include <string>
/**
* @brief Program options for the Biergarten pipeline application.
*/
struct ApplicationOptions {
/// @brief Path to the LLM model file (gguf format); mutually exclusive with
/// use_mocked.
std::string model_path;
/// @brief Use mocked generator instead of LLM; mutually exclusive with
/// model_path.
bool use_mocked = false;
/// @brief LLM sampling temperature (0.0 to 1.0, higher = more random).
float temperature = 1.0F;
/// @brief LLM nucleus sampling top-p parameter (0.0 to 1.0, higher = more
/// random).
float top_p = 0.95F;
/// @brief LLM top-k sampling parameter.
uint32_t top_k = 64;
/// @brief Context window size (tokens) for LLM inference. Higher values
/// support longer prompts but use more memory.
uint32_t n_ctx = 8192;
/// @brief Random seed for sampling (-1 for random, otherwise non-negative).
int seed = -1;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_APPLICATION_OPTIONS_H_

22
pipeline/includes/data_model/brewery_location.h Normal file
View File

@@ -0,0 +1,22 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_LOCATION_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_LOCATION_H_
/**
* @file data_model/brewery_location.h
* @brief Non-owning brewery location input.
*/
#include <string_view>
/**
* @brief Non-owning brewery location input.
*/
struct BreweryLocation {
/// @brief City name.
std::string_view city_name;
/// @brief Country name.
std::string_view country_name;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_LOCATION_H_

28
pipeline/includes/data_model/brewery_result.h Normal file
View File

@@ -0,0 +1,28 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_RESULT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_RESULT_H_
/**
* @file data_model/brewery_result.h
* @brief Generated brewery payload.
*/
#include <string>
/**
* @brief Generated brewery payload.
*/
struct BreweryResult {
/// @brief Brewery display name in English.
std::string name_en;
/// @brief Brewery description text in English.
std::string description_en;
/// @brief Brewery display name in the local language.
std::string name_local;
/// @brief Brewery description text in the local language.
std::string description_local;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_BREWERY_RESULT_H_

21
pipeline/includes/data_model/enriched_city.h Normal file
View File

@@ -0,0 +1,21 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_ENRICHED_CITY_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_ENRICHED_CITY_H_
/**
* @file data_model/enriched_city.h
* @brief Enriched city data with Wikipedia context.
*/
#include <string>
#include "data_model/location.h"
/**
* @brief Enriched city data with Wikipedia context.
*/
struct EnrichedCity {
Location location;
std::string region_context{};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_ENRICHED_CITY_H_

20
pipeline/includes/data_model/generated_brewery.h Normal file
View File

@@ -0,0 +1,20 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATED_BREWERY_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATED_BREWERY_H_
/**
* @file data_model/generated_brewery.h
* @brief Helper struct to store generated brewery data.
*/
#include "data_model/brewery_result.h"
#include "data_model/location.h"
/**
* @brief Helper struct to store generated brewery data.
*/
struct GeneratedBrewery {
Location location;
BreweryResult brewery;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATED_BREWERY_H_

13
pipeline/includes/data_model/generation_models.h Normal file
View File

@@ -0,0 +1,13 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATION_MODELS_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATION_MODELS_H_
/**
* @file data_model/generation_models.h
* @brief Convenience include for shared generation payload models.
*/
#include "data_model/brewery_location.h"
#include "data_model/brewery_result.h"
#include "data_model/user_result.h"
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_GENERATION_MODELS_H_

41
pipeline/includes/data_model/location.h Normal file
View File

@@ -0,0 +1,41 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_LOCATION_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_LOCATION_H_
/**
* @file data_model/location.h
* @brief Location data model used throughout generation pipeline.
*/
#include <string>
#include <vector>
/**
* @brief Canonical location record for city-level generation.
*/
struct Location {
/// @brief City name.
std::string city{};
/// @brief State or province name.
std::string state_province{};
/// @brief ISO 3166-2 subdivision code.
std::string iso3166_2{};
/// @brief Country name.
std::string country{};
/// @brief ISO 3166-1 country code.
std::string iso3166_1{};
/// @brief Local language codes in priority order.
std::vector<std::string> local_languages{};
/// @brief Latitude in decimal degrees.
double latitude{};
/// @brief Longitude in decimal degrees.
double longitude{};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_LOCATION_H_

12
pipeline/includes/data_model/pipeline_models.h Normal file
View File

@@ -0,0 +1,12 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_PIPELINE_MODELS_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_PIPELINE_MODELS_H_
/**
* @file data_model/pipeline_models.h
* @brief Convenience include for pipeline-specific data models.
*/
#include "data_model/enriched_city.h"
#include "data_model/generated_brewery.h"
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_PIPELINE_MODELS_H_

22
pipeline/includes/data_model/user_result.h Normal file
View File

@@ -0,0 +1,22 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_USER_RESULT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_USER_RESULT_H_
/**
* @file data_model/user_result.h
* @brief Generated user profile payload.
*/
#include <string>
/**
* @brief Generated user profile payload.
*/
struct UserResult {
/// @brief Username handle.
std::string username{};
/// @brief Short user biography.
std::string bio{};
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_DATA_MODEL_USER_RESULT_H_

22
pipeline/includes/json_handling/json_loader.h Normal file
View File

@@ -0,0 +1,22 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_
/**
* @file json_handling/json_loader.h
* @brief Loader API for curated location data.
*/
#include <filesystem>
#include <vector>
#include "data_model/location.h"
/// @brief Loads curated world locations from a JSON file into memory.
class JsonLoader {
public:
/// @brief Parses a JSON array file and returns all location records.
static std::vector<Location> LoadLocations(
const std::filesystem::path& filepath);
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_JSON_HANDLING_JSON_LOADER_H_

6
tooling/pipeline/includes/llama_backend_state.h → pipeline/includes/llama_backend_state.h
View File

@@ -16,10 +16,16 @@
*/
class LlamaBackendState {
public:
/// @brief Initializes global llama backend state.
LlamaBackendState() { llama_backend_init(); }
/// @brief Cleans up global llama backend state.
~LlamaBackendState() { llama_backend_free(); }
/// @brief Non-copyable type.
LlamaBackendState(const LlamaBackendState&) = delete;
/// @brief Non-copyable type.
LlamaBackendState& operator=(const LlamaBackendState&) = delete;
};

7
tooling/pipeline/includes/services/datetime/date_time_provider.h → pipeline/includes/services/date_time_provider.h
View File

@@ -1,5 +1,5 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATETIME_DATE_TIME_PROVIDER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATETIME_DATE_TIME_PROVIDER_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATE_TIME_PROVIDER_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATE_TIME_PROVIDER_H_
/**
* @file services/date_time_provider.h
@@ -18,6 +18,7 @@
*/
class IDateTimeProvider {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IDateTimeProvider() = default;
IDateTimeProvider() = default;
@@ -62,4 +63,4 @@ class SystemDateTimeProvider final : public IDateTimeProvider {
}
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATETIME_DATE_TIME_PROVIDER_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATE_TIME_PROVIDER_H_

9
tooling/pipeline/includes/services/enrichment/enrichment_service.h → pipeline/includes/services/enrichment_service.h
View File

@@ -1,5 +1,5 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_ENRICHMENT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_ENRICHMENT_SERVICE_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_SERVICE_H_
/**
* @file services/enrichment_service.h
@@ -8,13 +8,14 @@
#include <string>
#include "data_model/models.h"
#include "data_model/location.h"
/**
* @brief Interface for services that can enrich a location with context.
*/
class IEnrichmentService {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IEnrichmentService() = default;
/**
@@ -26,4 +27,4 @@ class IEnrichmentService {
virtual std::string GetLocationContext(const Location& loc) = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_ENRICHMENT_SERVICE_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_ENRICHMENT_SERVICE_H_

29
tooling/pipeline/includes/services/database/export_service.h → pipeline/includes/services/export_service.h
View File

@@ -1,14 +1,12 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_EXPORT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_EXPORT_SERVICE_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_EXPORT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_EXPORT_SERVICE_H_
/**
* @file services/export_service.h
* @brief Abstraction for persisting generated brewery data.
*/
#include <cstdint>
#include "data_model/generated_models.h"
#include "data_model/generated_brewery.h"
/**
* @brief Interface for services that persist generated brewery records.
@@ -16,6 +14,8 @@
class IExportService {
public:
IExportService() = default;
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~IExportService() = default;
IExportService(const IExportService&) = delete;
@@ -23,9 +23,7 @@ class IExportService {
IExportService(IExportService&&) = delete;
IExportService& operator=(IExportService&&) = delete;
/**
* @brief Prepares the export destination for a new run.
*/
/// @brief Prepares the export destination for a new run.
virtual void Initialize() = 0;
/**
@@ -33,19 +31,10 @@ class IExportService {
*
* @param brewery Generated brewery payload to store.
*/
virtual uint64_t ProcessRecord(const BreweryRecord& brewery) = 0;
virtual void ProcessRecord(const GeneratedBrewery& brewery) = 0;
/**
* @brief Persists one generated user record.
*
* @param user Generated user payload to store.
*/
virtual uint64_t ProcessRecord(const UserRecord& user) = 0;
/**
* @brief Finalizes the export destination.
*/
/// @brief Finalizes the export destination.
virtual void Finalize() = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_EXPORT_SERVICE_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_EXPORT_SERVICE_H_

32
tooling/pipeline/includes/services/database/sqlite_export_service.h → pipeline/includes/services/sqlite_export_service.h
View File

@@ -1,5 +1,5 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_SQLITE_EXPORT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_SQLITE_EXPORT_SERVICE_H_
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_H_
/**
* @file services/sqlite_export_service.h
@@ -11,17 +11,16 @@
#include <string>
#include <unordered_map>
#include "data_model/models.h"
#include "services/database/export_service.h"
#include "services/database/sqlite_export_service_helpers.h"
#include "services/datetime/date_time_provider.h"
#include "services/date_time_provider.h"
#include "services/export_service.h"
#include "services/sqlite_export_service_helpers.h"
/**
* @brief Persists generated brewery records into a fresh SQLite database.
*/
class SqliteExportService final : public IExportService {
public:
explicit SqliteExportService(const ApplicationOptions& options);
SqliteExportService();
~SqliteExportService() override;
SqliteExportService(const SqliteExportService&) = delete;
@@ -30,8 +29,7 @@ class SqliteExportService final : public IExportService {
SqliteExportService& operator=(SqliteExportService&&) = delete;
void Initialize() override;
uint64_t ProcessRecord(const BreweryRecord& brewery) override;
uint64_t ProcessRecord(const UserRecord& user) override;
void ProcessRecord(const GeneratedBrewery& brewery) override;
void Finalize() override;
private:
@@ -40,32 +38,22 @@ class SqliteExportService final : public IExportService {
using SqliteStatementHandle =
sqlite_export_service_internal::SqliteStatementHandle;
void InitializeSchema() const;
void InitializeSchema();
void PrepareStatements();
void RollbackAndCloseNoThrow() noexcept;
void FinalizeStatements() noexcept;
[[nodiscard]] std::filesystem::path BuildDatabasePath() const;
[[nodiscard]] static std::string BuildLocationKey(const Location& location);
/**
* @brief Returns the row id for @p location, inserting it first if it has
* not already been seen during this run.
*
* Shared by both ProcessRecord() overloads so breweries and users
* referencing the same location resolve to the same row.
*/
[[nodiscard]] sqlite3_int64 ResolveLocationId(const Location& location);
std::unique_ptr<IDateTimeProvider> date_time_provider_;
std::filesystem::path output_path_;
std::string run_timestamp_utc_;
std::filesystem::path database_path_;
SqliteDatabaseHandle db_handle_;
SqliteStatementHandle insert_location_stmt_;
SqliteStatementHandle insert_brewery_stmt_;
SqliteStatementHandle insert_user_stmt_;
bool transaction_open_ = false;
std::unordered_map<std::string, sqlite3_int64> location_cache_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_DATABASE_SQLITE_EXPORT_SERVICE_H_
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_H_

250
pipeline/includes/services/sqlite_export_service_helpers.h Normal file
View File

@@ -0,0 +1,250 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_HELPERS_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_HELPERS_H_
/**
* @file services/sqlite_export_service_helpers.h
* @brief Internal SQLite export helpers shared across per-method translation
* units.
*/
#include <sqlite3.h>
#include <boost/json.hpp>
#include <cstddef>
#include <cstring>
#include <filesystem>
#include <limits>
#include <memory>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
namespace sqlite_export_service_internal {
struct SqliteDatabaseDeleter {
void operator()(sqlite3* handle) const noexcept {
if (handle != nullptr) {
sqlite3_close(handle);
}
}
};
struct SqliteStatementDeleter {
void operator()(sqlite3_stmt* statement) const noexcept {
if (statement != nullptr) {
sqlite3_finalize(statement);
}
}
};
using SqliteDatabaseHandle = std::unique_ptr<sqlite3, SqliteDatabaseDeleter>;
using SqliteStatementHandle =
std::unique_ptr<sqlite3_stmt, SqliteStatementDeleter>;
inline constexpr std::string_view kCreateLocationsTableSql = R"sql(
CREATE TABLE IF NOT EXISTS locations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
city TEXT NOT NULL,
state_province TEXT NOT NULL,
iso3166_2 TEXT NOT NULL,
country TEXT NOT NULL,
iso3166_1 TEXT NOT NULL,
local_languages_json TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
UNIQUE(city, state_province, iso3166_2, country, latitude, longitude)
);
)sql";
inline constexpr std::string_view kCreateBreweriesTableSql = R"sql(
CREATE TABLE IF NOT EXISTS breweries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
location_id INTEGER NOT NULL,
name_en TEXT NOT NULL,
description_en TEXT NOT NULL,
name_local TEXT NOT NULL,
description_local TEXT NOT NULL,
FOREIGN KEY(location_id) REFERENCES locations(id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_breweries_location_id ON breweries(location_id);
)sql";
inline constexpr std::string_view kInsertLocationSql = R"sql(
INSERT INTO locations (
city,
state_province,
iso3166_2,
country,
iso3166_1,
local_languages_json,
latitude,
longitude
) VALUES (?, ?, ?, ?, ?, ?, ?, ?);
)sql";
inline constexpr std::string_view kInsertBrewerySql = R"sql(
INSERT INTO breweries (
location_id,
name_en,
description_en,
name_local,
description_local
) VALUES (?, ?, ?, ?, ?);
)sql";
inline constexpr int kLocationCityBindIndex = 1;
inline constexpr int kLocationStateProvinceBindIndex = 2;
inline constexpr int kLocationIso31662BindIndex = 3;
inline constexpr int kLocationCountryBindIndex = 4;
inline constexpr int kLocationIso31661BindIndex = 5;
inline constexpr int kLocationLanguagesBindIndex = 6;
inline constexpr int kLocationLatitudeBindIndex = 7;
inline constexpr int kLocationLongitudeBindIndex = 8;
inline constexpr int kBreweryLocationIdBindIndex = 1;
inline constexpr int kBreweryEnglishNameBindIndex = 2;
inline constexpr int kBreweryEnglishDescriptionBindIndex = 3;
inline constexpr int kBreweryLocalNameBindIndex = 4;
inline constexpr int kBreweryLocalDescriptionBindIndex = 5;
inline void ThrowSqliteError(sqlite3* db_handle, std::string_view action) {
const std::string message =
db_handle != nullptr ? sqlite3_errmsg(db_handle) : "unknown SQLite error";
throw std::runtime_error(std::string(action) + ": " + message);
}
inline SqliteDatabaseHandle OpenDatabase(const std::filesystem::path& path) {
sqlite3* raw_handle = nullptr;
const std::string path_string = path.string();
const int result = sqlite3_open(path_string.c_str(), &raw_handle);
SqliteDatabaseHandle handle(raw_handle);
if (result != SQLITE_OK) {
const std::string message = raw_handle != nullptr
? sqlite3_errmsg(raw_handle)
: "unknown SQLite error";
throw std::runtime_error("Failed to open SQLite export database: " +
message);
}
return handle;
}
inline void ExecSql(const SqliteDatabaseHandle& db_handle, std::string_view sql,
const char* action) {
char* error_message = nullptr;
const std::string sql_text(sql);
const int result = sqlite3_exec(db_handle.get(), sql_text.c_str(), nullptr,
nullptr, &error_message);
if (result != SQLITE_OK) {
const std::string message = error_message != nullptr
? error_message
: sqlite3_errmsg(db_handle.get());
sqlite3_free(error_message);
throw std::runtime_error(std::string(action) + ": " + message);
}
}
inline SqliteStatementHandle PrepareStatement(
const SqliteDatabaseHandle& db_handle, std::string_view sql,
const char* action) {
sqlite3_stmt* raw_statement = nullptr;
const std::string sql_text(sql);
const int result = sqlite3_prepare_v2(db_handle.get(), sql_text.c_str(), -1,
&raw_statement, nullptr);
SqliteStatementHandle statement(raw_statement);
if (result != SQLITE_OK) {
ThrowSqliteError(db_handle.get(), action);
}
return statement;
}
inline void ResetStatement(SqliteStatementHandle& statement) {
if (statement != nullptr) {
sqlite3_reset(statement.get());
sqlite3_clear_bindings(statement.get());
}
}
inline void DeleteCharArray(void* data) noexcept {
delete[] static_cast<char*>(data);
}
inline void BindText(const SqliteStatementHandle& statement, int index,
std::string_view value, const char* action) {
const auto byte_count = value.size();
if (byte_count > static_cast<std::size_t>(std::numeric_limits<int>::max())) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), action);
}
auto buffer = std::make_unique<char[]>(byte_count + 1);
std::memcpy(buffer.get(), value.data(), byte_count);
buffer[byte_count] = '\0';
char* raw_buffer = buffer.release();
const int result =
sqlite3_bind_text(statement.get(), index, raw_buffer,
static_cast<int>(byte_count), DeleteCharArray);
if (result != SQLITE_OK) {
DeleteCharArray(raw_buffer);
ThrowSqliteError(sqlite3_db_handle(statement.get()), action);
}
}
inline void BindDouble(const SqliteStatementHandle& statement, int index,
double value, std::string_view action) {
const int result = sqlite3_bind_double(statement.get(), index, value);
if (result != SQLITE_OK) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), action);
}
}
inline void BindInt64(const SqliteStatementHandle& statement, int index,
sqlite3_int64 value, std::string_view action) {
const int result = sqlite3_bind_int64(statement.get(), index, value);
if (result != SQLITE_OK) {
ThrowSqliteError(sqlite3_db_handle(statement.get()), action);
}
}
inline void StepStatement(const SqliteDatabaseHandle& db_handle,
const SqliteStatementHandle& statement,
std::string_view action) {
const int result = sqlite3_step(statement.get());
if (result != SQLITE_DONE) {
ThrowSqliteError(db_handle.get(), action);
}
}
inline sqlite3_int64 LastInsertRowId(const SqliteDatabaseHandle& db_handle) {
return sqlite3_last_insert_rowid(db_handle.get());
}
inline void RollbackTransactionNoThrow(
const SqliteDatabaseHandle& db_handle) noexcept {
if (!db_handle) {
return;
}
sqlite3_exec(db_handle.get(), "ROLLBACK;", nullptr, nullptr, nullptr);
}
inline std::string SerializeLocalLanguages(
const std::vector<std::string>& local_languages) {
boost::json::array array;
array.reserve(local_languages.size());
for (const auto& language : local_languages) {
array.emplace_back(language);
}
return boost::json::serialize(array);
}
} // namespace sqlite_export_service_internal
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_SQLITE_EXPORT_SERVICE_HELPERS_H_

33
pipeline/includes/services/wikipedia_service.h Normal file
View File

@@ -0,0 +1,33 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_SERVICES_WIKIPEDIA_SERVICE_H_
#define BIERGARTEN_PIPELINE_INCLUDES_SERVICES_WIKIPEDIA_SERVICE_H_
/**
* @file services/wikipedia_service.h
* @brief Wikipedia summary retrieval service with in-memory caching.
*/
#include <memory>
#include <string>
#include <string_view>
#include <unordered_map>
#include "services/enrichment_service.h"
#include "web_client/web_client.h"
/// @brief Provides Wikipedia summary lookups backed by cached raw extracts.
class WikipediaService final : public IEnrichmentService {
public:
/// @brief Creates a new Wikipedia service with the provided web client.
explicit WikipediaService(std::unique_ptr<WebClient> client);
/// @brief Returns the Wikipedia-derived context for a location.
[[nodiscard]] std::string GetLocationContext(const Location& loc) override;
private:
std::string FetchExtract(std::string_view query);
std::unique_ptr<WebClient> client_;
/// @brief Canonical cache for raw Wikipedia query extracts.
std::unordered_map<std::string, std::string> extract_cache_;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_SERVICES_WIKIPEDIA_SERVICE_H_

54
pipeline/includes/web_client/curl_web_client.h Normal file
View File

@@ -0,0 +1,54 @@
#ifndef BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_CURL_WEB_CLIENT_H_
#define BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_CURL_WEB_CLIENT_H_
/**
* @file web_client/curl_web_client.h
* @brief libcurl-based WebClient implementation.
*/
#include "web_client/web_client.h"
/**
* @brief RAII wrapper for curl_global_init and curl_global_cleanup.
*
* Create one instance in application startup before using libcurl and keep it
* alive for application lifetime.
*/
class CurlGlobalState {
public:
/// @brief Initializes global libcurl state.
CurlGlobalState();
/// @brief Cleans up global libcurl state.
~CurlGlobalState();
/// @brief Non-copyable type.
CurlGlobalState(const CurlGlobalState&) = delete;
/// @brief Non-copyable type.
CurlGlobalState& operator=(const CurlGlobalState&) = delete;
};
/**
* @brief WebClient implementation backed by libcurl.
*/
class CURLWebClient : public WebClient {
public:
/**
* @brief Executes an HTTP GET request.
*
* @param url Request URL.
* @return Response body.
*/
std::string Get(const std::string& url) override;
/**
* @brief URL-encodes a string value.
*
* @param value Raw value.
* @return URL-encoded string.
*/
std::string UrlEncode(const std::string& value) override;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_CURL_WEB_CLIENT_H_

3
tooling/pipeline/includes/web_client/web_client.h → pipeline/includes/web_client/web_client.h
View File

@@ -13,6 +13,7 @@
*/
class WebClient {
public:
/// @brief Virtual destructor for polymorphic cleanup.
virtual ~WebClient() = default;
/**
@@ -29,7 +30,7 @@ class WebClient {
* @param value Raw string value.
* @return Encoded value safe for URL usage.
*/
virtual std::string EncodeURL(const std::string& value) = 0;
virtual std::string UrlEncode(const std::string& value) = 0;
};
#endif // BIERGARTEN_PIPELINE_INCLUDES_WEB_CLIENT_WEB_CLIENT_H_

0
tooling/pipeline/locations.json → pipeline/locations.json
View File

0
docs/pipeline/french-cities.example → pipeline/out-sample/french-cities.example
View File

113
pipeline/prompts/system.md Normal file
View File

@@ -0,0 +1,113 @@
# FULL SYSTEM PROMPT
You are an expert brewery copywriter, an architectural observer, and a master of zymurgy.
Your main goal is to come up with a fake, contextually accurate name and a matching description for a craft brewery located in a specific city. You need to base this on the exact geographic and cultural info provided. You also need to seamlessly blend historical background, cultural details, and highly specialized brewing methods to create a realistic and interesting story.
You will receive the inputs like this:
## CITY:
[City Name]
## COUNTRY:
[Country Name]
## LOCAL LANGUAGE CODES:
[Local language codes in priority order]
## CONTEXT:
[Information about local beer culture, history, geography, or language context]
## CRITICAL OUTPUT FORMAT (READ CAREFULLY):
ABSOLUTELY NO MARKDOWN FORMATTING. Do NOT wrap your response in json or ``` blocks.
Do not add markdown, code fences, or postscript around the final JSON object. Do not say "Here is the JSON" or "Enjoy!".
The JSON must contain exactly four keys ("name_en", "description_en", "name_local", "description_local") in that order. Do not rename or add any other keys.
ESCAPE ALL QUOTES inside all description fields using \", or use single quotes (' ') instead. This applies equally to description_en and description_local. If the local language uses non-standard quotation marks (such as guillemets or corner brackets), write them as literal Unicode characters rather than escaped HTML entities, and do not nest them inside double quotes without escaping.
DO NOT use actual line breaks (\n) inside any string. Keep all descriptions as one continuous string each.
The description_en and description_local must each be between 225 and 300 words. Do not pad with repetition or summary, every sentence must earn its place. Be concise and specific.
Expected JSON format:
```json
{
"name_en": "Fictional Local Brewery Name in English",
"description_en": "The English description goes here.",
"name_local": "Translated brewery name in the local language",
"description_local": "The localised description goes here."
}
```
## CONTENT RULES AND CONSTRAINTS:
### THE HOOK:
The first sentence must be a sensory environmental hook written as a personal observation, something the owner notices or has always noticed. It should establish the local weather, smell, or soundscape of the city. Do not open with the brewery's name or a generic welcome.
### GEOGRAPHIC & CULTURAL ANCHOR:
The story must be deeply tied to the provided geographic and cultural info. Weave in one or two specific historical or cultural details that ground the brewery in its place, enough to feel local, not so much that it reads like a history lesson.
### TECHNICAL BREWING DETAIL (VARY THIS!):
You must include one highly specialized technical brewing detail. To avoid sounding repetitive, make sure this varies a lot. Some examples: using local wild yeast (like spontaneous Brettanomyces), adjusting the water profile (like Burtonization), specific mashing techniques, or using local barrels for aging. Don't use basic concepts like generic mash temperatures.
### ARCHITECTURAL DETAIL (VARY THIS!):
You must include one specific architectural or environmental detail, highlighting the building's physical wear, structure, or history. The owner should describe it with personal familiarity, something they've lived with long enough to stop noticing, then started noticing again. Avoid overused industry clichés like repurposed dairy equipment or glycol chillers.
### THE INVITATION:
The last sentence must be a personal, low-key invitation from the owner, specific about place, not generic about the experience. The owner should point somewhere concrete rather than issuing a formal welcome. Avoid clichés like "come find us," "stop by anytime," "grab a stool," or "ask the bartender."
### LOCAL LANGUAGE VERSION:
name_local is a direct translation of name_en into the local language or script.
Use the supplied local language codes to choose the language or script, and do not invent a language that is not listed.
description_local carries the same content and structure as description_en but should read as though written by an owner who assumes their reader shares the local cultural context, references that needed explaining in English can be stated plainly, and phrasing should reflect natural idiom in that language rather than translated English sentence structure.
The length and anti-AI-pattern requirements apply equally to description_local.
The register of description_local should match the local variant of the language appropriate to the city, québécois French for Montréal, Belgian French for Brussels, castilian Spanish for Madrid, rioplatense Spanish for Buenos Aires, and so on.
### THE BLOCKLIST (FORBIDDEN CONCEPTS):
You absolutely cannot use the following words and phrases. Make sure your final output doesn't have any of these:
- "hidden gem"
- "passion"
- "authentic"
- "repurposed dairy tank"
- "repurposed industrial vat"
- "concrete eggs"
- "glycol chiller"
- "mash temperature"
- "grab a stool"
- "ask the bartender"
- "come find us"
- "stop by anytime"
#### FORBIDDEN WRITING PATTERNS
The following patterns are common AI writing pitfalls and must not appear in either description:
- Negative parallelism constructions: "It's not X, it's Y" or "We're not about X, we're about Y"
- Inflated significance phrases: "stands as a testament," "plays a vital role," "leaves a lasting impact," "watershed moment," "deeply rooted," "rich cultural heritage," "rich cultural tapestry," "enduring legacy"
- Superficial trailing analyses: sentences ending in -ing words that add opinion without content ("ensuring consistency," "reflecting the city's spirit," "highlighting our commitment")
- Promotional travel-copy tone: "breathtaking," "must-visit," "stunning," "vibrant"
- Overused conjunctive transitions used as sentence openers: "Moreover," "Furthermore," "In addition," "In contrast"
- Rule of three: do not consistently organise ideas or examples in triplets
### VOICE & PERSPECTIVE:
The description must be written in the first person, from the perspective of the brewery's owner. Favour "we" and "our" over "I" and "my." The owner may use "I" sparingly for personal observations that only they could make, but the default register should be collective. The tone should feel lived-in and a little weathered. Do not use third-person or second-person pronouns.

16
pipeline/src/biergarten_data_generator/biergarten_data_generator.cc Normal file
View File

@@ -0,0 +1,16 @@
/**
* @file biergarten_data_generator/biergarten_data_generator.cc
* @brief BiergartenDataGenerator constructor implementation.
*/
#include "biergarten_data_generator.h"
#include <utility>
BiergartenDataGenerator::BiergartenDataGenerator(
std::unique_ptr<IEnrichmentService> context_service,
std::unique_ptr<DataGenerator> generator,
std::unique_ptr<IExportService> exporter)
: context_service_(std::move(context_service)),
generator_(std::move(generator)),
exporter_(std::move(exporter)) {}

58
pipeline/src/biergarten_data_generator/generate_breweries.cc Normal file
View File

@@ -0,0 +1,58 @@
/**
* @file biergarten_data_generator/generate_breweries.cc
* @brief BiergartenDataGenerator::GenerateBreweries() implementation.
*/
#include <spdlog/spdlog.h>
#include "biergarten_data_generator.h"
void BiergartenDataGenerator::GenerateBreweries(
std::span<const EnrichedCity> cities) {
spdlog::info("\n=== SAMPLE BREWERY GENERATION ===");
generated_breweries_.clear();
size_t skipped_count = 0;
size_t export_failed_count = 0;
for (const auto& [location, region_context] : cities) {
try {
const BreweryResult brewery =
generator_->GenerateBrewery(location, region_context);
const GeneratedBrewery gen{.location = location, .brewery = brewery};
generated_breweries_.push_back(gen);
try {
exporter_->ProcessRecord(gen);
} catch (const std::exception& export_exception) {
++export_failed_count;
spdlog::warn(
"[Pipeline] Generated brewery for '{}' ({}) but SQLite export "
"failed: {}",
location.city, location.country, export_exception.what());
}
} catch (const std::exception& e) {
++skipped_count;
spdlog::warn(
"[Pipeline] Skipping city '{}' ({}): brewery generation failed: "
"{}",
location.city, location.country, e.what());
}
}
if (skipped_count > 0) {
spdlog::warn("[Pipeline] Skipped {} city/cities due to generation errors",
skipped_count);
}
if (export_failed_count > 0) {
spdlog::warn(
"[Pipeline] Failed to export {} generated brewery/breweries to "
"SQLite",
export_failed_count);
}
}

26
pipeline/src/biergarten_data_generator/log_results.cc Normal file
View File

@@ -0,0 +1,26 @@
/**
* @file biergarten_data_generator/log_results.cc
* @brief BiergartenDataGenerator::LogResults() implementation.
*/
#include <spdlog/spdlog.h>
#include "biergarten_data_generator.h"
void BiergartenDataGenerator::LogResults() const {
spdlog::info("\n=== GENERATED DATA DUMP ===");
size_t index = 1;
for (const auto& [location, brewery] : generated_breweries_) {
spdlog::info(
"{}. city=\"{}\" country=\"{}\" state=\"{}\" "
"iso3166_2={} lat={} lon={}",
index, location.city, location.country, location.state_province,
location.iso3166_2, location.latitude, location.longitude);
spdlog::info(" brewery_name_en=\"{}\"", brewery.name_en);
spdlog::info(" brewery_description_en=\"{}\"", brewery.description_en);
spdlog::info(" brewery_name_local=\"{}\"", brewery.name_local);
spdlog::info(" brewery_description_local=\"{}\"",
brewery.description_local);
++index;
}
}

41
pipeline/src/biergarten_data_generator/query_cities_with_countries.cc Normal file
View File

@@ -0,0 +1,41 @@
/**
* @file biergarten_data_generator/query_cities_with_countries.cc
* @brief BiergartenDataGenerator::QueryCitiesWithCountries() implementation.
*/
#include <spdlog/spdlog.h>
#include <algorithm>
#include <filesystem>
#include <iterator>
#include <random>
#include "biergarten_data_generator.h"
#include "json_handling/json_loader.h"
static constexpr size_t kBreweryAmount = 50;
std::vector<Location> BiergartenDataGenerator::QueryCitiesWithCountries() {
spdlog::info("\n=== GEOGRAPHIC DATA OVERVIEW ===");
const std::filesystem::path locations_path = "locations.json";
auto all_locations = JsonLoader::LoadLocations(locations_path);
spdlog::info(" Locations available: {}", all_locations.size());
const size_t sample_count = std::min(kBreweryAmount, all_locations.size());
const auto sample_count_signed =
static_cast<std::iter_difference_t<decltype(all_locations.cbegin())>>(
sample_count);
std::vector<Location> sampled_locations;
sampled_locations.reserve(sample_count);
std::random_device random_generator;
std::ranges::sample(all_locations, std::back_inserter(sampled_locations),
sample_count_signed, random_generator);
spdlog::info(" Sampled locations: {}", sampled_locations.size());
return sampled_locations;
}

52
pipeline/src/biergarten_data_generator/run.cc Normal file
View File

@@ -0,0 +1,52 @@
/**
* @file biergarten_data_generator/run.cc
* @brief BiergartenDataGenerator::Run() implementation.
*/
#include <spdlog/spdlog.h>
#include <utility>
#include "biergarten_data_generator.h"
bool BiergartenDataGenerator::Run() {
try {
exporter_->Initialize();
std::vector<Location> cities = QueryCitiesWithCountries();
std::vector<EnrichedCity> enriched;
enriched.reserve(cities.size());
size_t skipped_count = 0;
for (auto& city : cities) {
try {
std::string region_context = context_service_->GetLocationContext(city);
spdlog::debug("[Pipeline] Context for '{}' ({}) gathered:\n{}",
city.city, city.country, region_context);
enriched.push_back(
EnrichedCity{.location = std::move(city),
.region_context = std::move(region_context)});
} catch (const std::exception& exception) {
++skipped_count;
spdlog::warn(
"[Pipeline] Skipping city '{}' ({}): context lookup failed: {}",
city.city, city.country, exception.what());
}
}
if (skipped_count > 0) {
spdlog::warn(
"[Pipeline] Skipped {} city/cities due to context lookup errors",
skipped_count);
}
this->GenerateBreweries(enriched);
exporter_->Finalize();
this->LogResults();
return true;
} catch (const std::exception& e) {
spdlog::error("Pipeline execution failed with error: {}", e.what());
return false;
}
}

83
tooling/pipeline/src/data_generation/llama/generate_brewery.cc → pipeline/src/data_generation/llama/generate_brewery.cc
View File

@@ -4,7 +4,8 @@
* inference, and validates structured JSON output for brewery records.
*/
#include <chrono>
#include <spdlog/spdlog.h>
#include <format>
#include <optional>
#include <stdexcept>
@@ -12,7 +13,6 @@
#include <string_view>
#include <vector>
#include "data_generation/json_grammars.h"
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
@@ -33,6 +33,16 @@ static std::string FormatLocalLanguageCodes(
return formatted;
}
static constexpr std::string_view kBreweryJsonGrammar = R"json_brewery(
root ::= thought-block "{" ws "\"name_en\"" ws ":" ws string ws "," ws "\"description_en\"" ws ":" ws string ws "," ws "\"name_local\"" ws ":" ws string ws "," ws "\"description_local\"" ws ":" ws string ws "}" ws
thought-block ::= [^{]*
ws ::= [ \t\n\r]*
string ::= "\"" char+ "\""
char ::= [^"\\\x7F\x00-\x1F] | [\\] escape
escape ::= ["\\/bfnrt] | "u" hex hex hex hex
hex ::= [0-9a-fA-F]
)json_brewery";
static constexpr int kBreweryInitialMaxTokens = 2800;
BreweryResult LlamaGenerator::GenerateBrewery(
@@ -49,12 +59,11 @@ BreweryResult LlamaGenerator::GenerateBrewery(
location.country.empty() ? std::string{}
: std::format(", {}", location.country);
/**
* Load brewery system prompt via the injected prompt directory.
* The key "BREWERY_GENERATION" resolves to BREWERY_GENERATION.md inside
* the configured --prompt-dir. Throws on missing or empty file.
* Load brewery system prompt from file
* Falls back to minimal inline prompt if file not found
*/
const std::string system_prompt =
prompt_directory_->Load("BREWERY_GENERATION");
LoadBrewerySystemPrompt("prompts/system.md");
std::string user_prompt = std::format(
"## CITY:\n{}\n\n## COUNTRY:\n{}\n\n## LOCAL LANGUAGE CODES:\n{}\n\n## "
@@ -71,14 +80,15 @@ BreweryResult LlamaGenerator::GenerateBrewery(
/**
* RETRY LOOP with validation and error correction
* Attempts to generate valid brewery data up to 3 times, with
* feedback-based refinement
* Attempts to generate valid brewery data up to 3 times, with feedback-based
* refinement
*/
constexpr int max_attempts = 3;
std::string raw;
std::string last_error;
// Token budget: too small risks truncating valid JSON mid-string.
// Start conservatively but allow adaptive increases on truncation.
int max_tokens = kBreweryInitialMaxTokens;
// Limit output length to keep it concise and focused
@@ -86,13 +96,8 @@ BreweryResult LlamaGenerator::GenerateBrewery(
// Generate brewery data from LLM
raw = this->Infer(system_prompt, user_prompt, max_tokens,
kBreweryJsonGrammar);
if (logger_) {
logger_->Log(
{.level = LogLevel::Debug,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: raw output (attempt {}): {}",
attempt + 1, raw)});
}
spdlog::debug("LlamaGenerator: raw output (attempt {}): {}", attempt + 1,
raw);
// Validate output: parse JSON and check required fields
@@ -103,14 +108,9 @@ BreweryResult LlamaGenerator::GenerateBrewery(
if (!validation_error.has_value()) {
// Success: return parsed brewery data
if (logger_) {
logger_->Log(
{.level = LogLevel::Info,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format("LlamaGenerator: successfully generated "
"brewery data on attempt {}",
attempt + 1)});
}
spdlog::info(
"LlamaGenerator: successfully generated brewery data on attempt {}",
attempt + 1);
return brewery;
}
@@ -118,42 +118,27 @@ BreweryResult LlamaGenerator::GenerateBrewery(
// Validation failed: log error and prepare corrective feedback
last_error = *validation_error;
if (logger_) {
logger_->Log(
{.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: malformed brewery JSON (attempt {}): {}",
attempt + 1, *validation_error)});
}
spdlog::warn("LlamaGenerator: malformed brewery JSON (attempt {}): {}",
attempt + 1, *validation_error);
// Update prompt with error details to guide LLM toward correct output.
user_prompt = std::format(
"Your previous response was invalid. Error: {}\nReturn the thought "
"process before the JSON if needed, then return ONLY valid JSON "
"with "
"exactly these keys, in this exact order: {{\"name_en\": "
"\"<English "
"process before the JSON if needed, then return ONLY valid JSON with "
"exactly these keys, in this exact order: {{\"name_en\": \"<English "
"brewery name>\", \"description_en\": \"<English single-paragraph "
"description>\", \"name_local\": \"<local-language brewery "
"name>\", "
"description>\", \"name_local\": \"<local-language brewery name>\", "
"\"description_local\": \"<local-language single-paragraph "
"description>\"}}.\nDo not include markdown, comments, extra keys, "
"or "
"literal placeholder values.\n\nKeep the JSON strings concise "
"enough "
"description>\"}}.\nDo not include markdown, comments, extra keys, or "
"literal placeholder values.\n\nKeep the JSON strings concise enough "
"to fit within the token budget.\n\n{}",
*validation_error, retry_location);
}
// All retry attempts exhausted: log failure and throw exception
if (logger_) {
logger_->Log({.level = LogLevel::Error,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: malformed brewery "
"response after {} attempts: {}",
max_attempts, last_error.empty() ? raw : last_error)});
}
spdlog::error(
"LlamaGenerator: malformed brewery response after {} attempts: "
"{}",
max_attempts, last_error.empty() ? raw : last_error);
throw std::runtime_error("LlamaGenerator: malformed brewery response");
}

18
pipeline/src/data_generation/llama/generate_user.cc Normal file
View File

@@ -0,0 +1,18 @@
/**
* @file data_generation/llama/generate_user.cc
* @brief Generates locale-aware user profiles with strict two-line formatting,
* retry handling, and output sanitization for downstream parsing.
*/
#include <spdlog/spdlog.h>
#include <stdexcept>
#include <string>
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
UserResult LlamaGenerator::GenerateUser(const std::string& locale) {
return {.username = "test_user",
.bio = "This is a test user profile from " + locale + "."};
}

145
tooling/pipeline/src/data_generation/llama/helpers.cc → pipeline/src/data_generation/llama/helpers.cc
View File

@@ -9,21 +9,18 @@
#include <boost/json.hpp>
#include <cctype>
#include <optional>
#include <span>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator_helpers.h"
#include "llama.h"
namespace {
/**
* String trimming: removes leading and trailing whitespace
*/
std::string Trim(std::string_view value) {
static std::string Trim(std::string_view value) {
constexpr std::string_view whitespace = " \t\n\r\f\v";
const size_t first_index = value.find_first_not_of(whitespace);
if (first_index == std::string_view::npos) {
@@ -38,7 +35,7 @@ std::string Trim(std::string_view value) {
* Normalize whitespace: collapses multiple spaces/tabs/newlines into single
* spaces
*/
std::string CondenseWhitespace(std::string_view text) {
static std::string CondenseWhitespace(std::string_view text) {
std::string out;
out.reserve(text.size());
@@ -61,41 +58,6 @@ std::string CondenseWhitespace(std::string_view text) {
return out;
}
// Guard against truncating in the first half of the string.
// This preserves the critical opening content and avoids cutting critical
// context words early in the region description.
constexpr size_t kTruncationGuardDivisor = 2;
bool ReadRequiredTrimmedStringField(const boost::json::object& obj,
std::string_view key, std::string& out,
std::string* error_out) {
const boost::json::value* field = obj.if_contains(key);
if (field == nullptr || !field->is_string()) {
return false;
}
const auto& string_value = field->as_string();
out = Trim(std::string_view(string_value.data(), string_value.size()));
return !out.empty();
}
bool HasSchemaPlaceholder(std::span<std::string* const> values) {
for (const std::string* value : values) {
std::string lowered = *value;
std::ranges::transform(lowered, lowered.begin(),
[](const unsigned char character) {
return static_cast<char>(std::tolower(character));
});
if (lowered == "string") {
return true;
}
}
return false;
}
} // namespace
/**
* Truncate region context to fit within max length while preserving word
* boundaries
@@ -109,8 +71,7 @@ std::string PrepareRegionContext(std::string_view region_context,
normalized.resize(max_chars);
const size_t last_space = normalized.find_last_of(' ');
if (last_space != std::string::npos &&
last_space > max_chars / kTruncationGuardDivisor) {
if (last_space != std::string::npos && last_space > max_chars / 2) {
normalized.resize(last_space);
}
@@ -154,6 +115,47 @@ void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
"LlamaGenerator: failed to decode sampled token piece");
}
static bool ReadRequiredTrimmedStringField(const boost::json::object& obj,
std::string_view key,
std::string& out,
std::string* error_out) {
const boost::json::value* field = obj.if_contains(key);
if (field == nullptr || !field->is_string()) {
if (error_out != nullptr) {
*error_out =
"JSON field '" + std::string(key) + "' is missing or not a string";
}
return false;
}
const auto& string_value = field->as_string();
out = Trim(std::string_view(string_value.data(), string_value.size()));
if (out.empty()) {
if (error_out != nullptr) {
*error_out = "JSON field '" + std::string(key) + "' must not be empty";
}
return false;
}
return true;
}
static bool HasSchemaPlaceholder(const std::array<std::string*, 4>& values) {
for (const std::string* value : values) {
std::string lowered = *value;
std::ranges::transform(lowered, lowered.begin(),
[](unsigned char character) {
return static_cast<char>(std::tolower(character));
});
if (lowered == "string") {
return true;
}
}
return false;
}
std::optional<std::string> ValidateBreweryJson(const std::string& raw,
BreweryResult& brewery_out) {
boost::system::error_code error_code;
@@ -201,7 +203,7 @@ std::optional<std::string> ValidateBreweryJson(const std::string& raw,
return validation_error;
}
const std::array schema_placeholders = {
const std::array<std::string*, 4> schema_placeholders = {
&brewery_out.name_en, &brewery_out.description_en,
&brewery_out.name_local, &brewery_out.description_local};
if (HasSchemaPlaceholder(schema_placeholders)) {
@@ -210,58 +212,3 @@ std::optional<std::string> ValidateBreweryJson(const std::string& raw,
return std::nullopt;
}
std::optional<std::string> ValidateUserJson(const std::string& raw,
UserResult& user_out) {
boost::system::error_code error_code;
const std::string_view raw_view(raw);
const size_t opening_brace = raw_view.find('{');
if (opening_brace == std::string_view::npos) {
return "JSON parse error: missing opening brace '{'";
}
const std::string_view json_payload = raw_view.substr(opening_brace);
boost::json::value json_value = boost::json::parse(json_payload, error_code);
if (error_code) {
return "JSON parse error: " + error_code.message();
}
if (!json_value.is_object()) {
return "JSON root must be an object";
}
const auto& obj = json_value.get_object();
if (obj.size() != 3) {
return "JSON object must contain exactly three keys";
}
std::string validation_error;
if (!ReadRequiredTrimmedStringField(obj, "username", user_out.username,
&validation_error)) {
return validation_error;
}
if (!ReadRequiredTrimmedStringField(obj, "bio", user_out.bio,
&validation_error)) {
return validation_error;
}
const boost::json::value* activity_weight_field =
obj.if_contains("activity_weight");
if (activity_weight_field == nullptr || !activity_weight_field->is_number()) {
return "Missing or invalid numeric field: activity_weight";
}
const double activity_weight = activity_weight_field->to_number<double>();
if (activity_weight < 0.0 || activity_weight > 1.0) {
return "activity_weight must be between 0 and 1";
}
user_out.activity_weight = static_cast<float>(activity_weight);
const std::array schema_placeholders = {&user_out.username, &user_out.bio};
if (HasSchemaPlaceholder(schema_placeholders)) {
return "JSON appears to be a schema placeholder, not content";
}
return std::nullopt;
}

26
tooling/pipeline/src/data_generation/llama/infer.cc → pipeline/src/data_generation/llama/infer.cc
View File

@@ -5,24 +5,20 @@
* output tokens back to text for system+user chat prompts.
*/
#include <spdlog/spdlog.h>
#include <algorithm>
#include <chrono>
#include <format>
#include <memory>
#include <stdexcept>
#include <string>
#include <string_view>
#include <vector>
#include <llama.h>
#include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h"
#include "llama.h"
static constexpr size_t kPromptTokenSlack = 8;
// Minimum tokens to keep when using top-p sampling. Ensures at least one
// candidate token remains available even with very restrictive top-p values.
static constexpr size_t kTopPMinKeep = 1;
namespace {
@@ -66,7 +62,7 @@ SamplerHandle MakeSamplerChain(const llama_vocab* vocab,
"LlamaGenerator: failed to initialize temperature sampler");
add_sampler(llama_sampler_init_top_k(static_cast<int32_t>(config.top_k)),
"LlamaGenerator: failed to initialize top-k sampler");
add_sampler(llama_sampler_init_top_p(config.top_p, kTopPMinKeep),
add_sampler(llama_sampler_init_top_p(config.top_p, 1),
"LlamaGenerator: failed to initialize top-p sampler");
add_sampler(llama_sampler_init_dist(config.seed),
"LlamaGenerator: failed to initialize distribution sampler");
@@ -108,7 +104,7 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
.top_p = sampling_top_p_,
.seed = static_cast<uint32_t>(rng_()),
};
const auto sampler = MakeSamplerChain(vocab, sampler_config, grammar);
auto sampler = MakeSamplerChain(vocab, sampler_config, grammar);
/**
* Clear KV cache to ensure clean inference state (no residual context)
@@ -172,14 +168,10 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
*/
prompt_tokens.resize(static_cast<size_t>(token_count));
if (token_count > prompt_budget) {
if (logger_) {
logger_->Log({.level = LogLevel::Warn,
.phase = PipelinePhase::BreweryAndBeerGeneration,
.message = std::format(
"LlamaGenerator: prompt too long ({} tokens), "
"truncating to {} tokens to fit n_batch/n_ctx limits",
token_count, prompt_budget)});
}
spdlog::warn(
"LlamaGenerator: prompt too long ({} tokens), truncating to {} "
"tokens to fit n_batch/n_ctx limits",
token_count, prompt_budget);
prompt_tokens.resize(static_cast<size_t>(prompt_budget));
token_count = prompt_budget;
}

44
tooling/pipeline/src/data_generation/llama/llama_generator.cc → pipeline/src/data_generation/llama/llama_generator.cc
View File

@@ -11,9 +11,8 @@
#include <stdexcept>
#include <string>
#include <llama.h>
#include "data_model/models.h"
#include "data_model/application_options.h"
#include "llama.h"
static constexpr uint32_t kMaxContextSize = 32768U;
@@ -33,13 +32,9 @@ void LlamaGenerator::ContextDeleter::operator()(
LlamaGenerator::LlamaGenerator(
const ApplicationOptions& options, const std::string& model_path,
std::shared_ptr<ILogger> logger,
std::unique_ptr<IPromptFormatter> prompt_formatter,
std::unique_ptr<IPromptDirectory> prompt_directory)
std::unique_ptr<IPromptFormatter> prompt_formatter)
: rng_(std::random_device{}()),
logger_(std::move(logger)),
prompt_formatter_(std::move(prompt_formatter)),
prompt_directory_(std::move(prompt_directory)) {
prompt_formatter_(std::move(prompt_formatter)) {
if (model_path.empty()) {
throw std::runtime_error("LlamaGenerator: model path must not be empty");
}
@@ -49,50 +44,41 @@ LlamaGenerator::LlamaGenerator(
"LlamaGenerator: prompt formatter dependency must not be null");
}
if (!prompt_directory_) {
throw std::runtime_error(
"LlamaGenerator: prompt directory dependency must not be null");
}
const auto sampling = options.generator.sampling.value_or(SamplingOptions{});
if (sampling.temperature < 0.0F) {
if (options.temperature < 0.0F) {
throw std::runtime_error(
"LlamaGenerator: sampling temperature must be >= 0");
}
if (sampling.top_p <= 0.0F || sampling.top_p > 1.0F) {
if (options.top_p <= 0.0F || options.top_p > 1.0F) {
throw std::runtime_error(
"LlamaGenerator: sampling top-p must be in (0, 1]");
}
if (sampling.top_k == 0U) {
if (options.top_k == 0U) {
throw std::runtime_error("LlamaGenerator: sampling top-k must be > 0");
}
if (sampling.seed < -1) {
if (options.seed < -1) {
throw std::runtime_error(
"LlamaGenerator: seed must be >= 0, or -1 for random");
}
if (sampling.n_ctx == 0 || sampling.n_ctx > kMaxContextSize) {
if (options.n_ctx == 0 || options.n_ctx > kMaxContextSize) {
throw std::runtime_error(
"LlamaGenerator: context size must be in range [1, 32768]");
}
sampling_temperature_ = sampling.temperature;
sampling_top_p_ = sampling.top_p;
sampling_top_k_ = sampling.top_k;
sampling_temperature_ = options.temperature;
sampling_top_p_ = options.top_p;
sampling_top_k_ = options.top_k;
if (sampling.seed == -1) {
if (options.seed == -1) {
std::random_device random_device;
rng_.seed(random_device());
} else {
rng_.seed(static_cast<uint32_t>(sampling.seed));
rng_.seed(static_cast<uint32_t>(options.seed));
}
n_ctx_ = sampling.n_ctx;
n_gpu_layers_ = sampling.n_gpu_layers;
n_ctx_ = options.n_ctx;
this->Load(model_path);
}

34
tooling/pipeline/src/data_generation/llama/load.cc → pipeline/src/data_generation/llama/load.cc
View File

@@ -4,35 +4,23 @@
* context, and resets prior resources during model initialization.
*/
#include <spdlog/spdlog.h>
#include <algorithm>
#include <chrono>
#include <stdexcept>
#include <string>
#include <utility>
#include <ggml-backend.h>
#include <llama.h>
#include "data_generation/llama_generator.h"
// Maximum batch size for decode operations. Capping the batch prevents
// excessive memory allocation while maintaining inference performance.
static constexpr uint32_t kMaxBatchSize = 5000U;
#include "llama.h"
void LlamaGenerator::Load(const std::string& model_path) {
context_.reset();
model_.reset();
// Specifically load dynamic ggml backends (like CUDA) that are provided
// externally before attempting to load a model.
ggml_backend_load_all();
llama_model_params model_params = llama_model_default_params();
model_params.n_gpu_layers = n_gpu_layers_;
ModelHandle loaded_model(
const llama_model_params model_params = llama_model_default_params();
LlamaGenerator::ModelHandle loaded_model(
llama_model_load_from_file(model_path.c_str(), model_params));
if (!loaded_model) {
throw std::runtime_error(
"LlamaGenerator: failed to load model from path: " + model_path);
@@ -40,11 +28,10 @@ void LlamaGenerator::Load(const std::string& model_path) {
llama_context_params context_params = llama_context_default_params();
context_params.n_ctx = n_ctx_;
context_params.n_batch = std::min(n_ctx_, kMaxBatchSize);
context_params.n_batch = std::min(n_ctx_, static_cast<uint32_t>(5000));
ContextHandle loaded_context(
LlamaGenerator::ContextHandle loaded_context(
llama_init_from_model(loaded_model.get(), context_params));
if (!loaded_context) {
throw std::runtime_error("LlamaGenerator: failed to create context");
}
@@ -52,10 +39,5 @@ void LlamaGenerator::Load(const std::string& model_path) {
model_ = std::move(loaded_model);
context_ = std::move(loaded_context);
if (logger_) {
logger_->Log({.level = LogLevel::Info,
.phase = PipelinePhase::Startup,
.message = std::format("[LlamaGenerator] Loaded model: {} ",
model_path)});
}
spdlog::info("[LlamaGenerator] Loaded model: {}", model_path);
}

Some files were not shown because too many files have changed in this diff Show More