diff --git a/.claude/skills/doc-drift-pipeline/SKILL.md b/.claude/skills/doc-drift-pipeline/SKILL.md new file mode 100644 index 00000000..a53142bc --- /dev/null +++ b/.claude/skills/doc-drift-pipeline/SKILL.md @@ -0,0 +1,52 @@ +--- +name: doc-drift-pipeline +description: Checks whether recent changes under tooling/pipeline/ have made docs/pipeline/ stale. Use when the user asks to check for doc drift on the pipeline, before committing changes to tooling/pipeline, or when asked "does this need a doc update" for the C++ seed-data pipeline. +--- + +# Doc Drift Check — Pipeline + +Given a diff scoped to `tooling/pipeline/` (uncommitted changes, a range of +commits, or a specific PR), determine whether it invalidates any factual +claim in `docs/pipeline/`. + +## Steps + +1. Run `git diff -- tooling/pipeline/` (or + `git diff ... -- tooling/pipeline/` if a range is given) to + see what changed. If the diff also touches `web/**`, note that it's out of + scope here — use the `doc-drift-web` skill for that half. +2. Based on which files changed, identify the relevant doc(s): + - `tooling/pipeline/src/**`, `includes/**` (implemented behavior) → + `docs/pipeline/README.md`, `docs/pipeline/diagrams/current/*.mmd` + (activity, class) + - Work that's scaffolded/flagged but not wired into the default run path + (roadmap-only) → `docs/pipeline/ROADMAP.md`, + `docs/pipeline/diagrams/planned/*.mmd` — do not touch `current/` for + planned work that hasn't landed + - Changes to pipeline bias/hallucination handling, model prompts, sampling + logic, or known-limitation workarounds → + `docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md` + - `tooling/pipeline/runpod/Dockerfile`, build/run scripts → + `docs/pipeline/README.md` (setup/usage sections) + - `docs/pipeline/french-cities.example` — check it still matches the input + schema the pipeline actually reads + - A new file added under `docs/pipeline/**` → also check `docs/index.md` + and `mkdocs.yml` nav are updated to reference it +3. Read the relevant doc section(s) and check each specific factual claim + (tables, "currently X", "not yet implemented", roadmap checkboxes) against + the diff. For diagrams, this means reading the `.mmd` source (not just the + wrapping `.md`) and checking nodes/edges/labels against the diff — a + pipeline stage or control-flow change can invalidate a diagram without + touching any prose. +4. When a diff adds real behavior that was previously only in + `diagrams/planned/`, check whether that planned diagram/roadmap entry + should now move to `current/`/README rather than just adding a duplicate + claim in both places. +5. Report only what's actually stale — do not rewrite unrelated prose, do not + invent claims not grounded in the diff. +6. If nothing is affected, say so plainly and make no edits. +7. For confirmed drift, propose the exact edit (old text → new text, or the + changed `.mmd` node/edge) rather than committing directly, unless + explicitly told to apply it. Flag `.mmd` changes separately since they may + need a re-render (`docs/rendered-diagrams/pipeline/**`) via + `docs/makefile`. diff --git a/.claude/skills/doc-drift-web/SKILL.md b/.claude/skills/doc-drift-web/SKILL.md new file mode 100644 index 00000000..08651900 --- /dev/null +++ b/.claude/skills/doc-drift-web/SKILL.md @@ -0,0 +1,55 @@ +--- +name: doc-drift-web +description: Checks whether recent changes under web/ have made docs/web/ stale. Use when the user asks to check for doc drift on the web app, before committing changes to web/backend or web/frontend, or when asked "does this need a doc update" for web/API/frontend work. +--- + +# Doc Drift Check — Web + +Given a diff scoped to `web/` (uncommitted changes, a range of commits, or a +specific PR), determine whether it invalidates any factual claim in +`docs/web/`. + +## Steps + +1. Run `git diff -- web/` (or `git diff ... -- web/` if a range is + given) to see what changed. If the diff also touches `tooling/pipeline/**`, + note that it's out of scope here — use the `doc-drift-pipeline` skill for + that half. +2. Based on which files changed, identify the relevant doc(s): + - `web/backend/API/**`, `Features.*/**` (general) → `docs/web/architecture.md` + - `web/backend/Features/Features.Users/**/Authentication/**`, + identity/token code (`ITokenService`, `RefreshToken*`, `ConfirmEmail*`, + login/registration handlers) → `docs/web/token-validation.md`, + `docs/web/diagrams/auth/*.mmd` (pick the matching flow: login, + registration, refresh-token, confirm-user, delete-account, + update-email/password/profile/username, resend-confirmation, + error-handling) + - `web/backend/Database/**` (entities, `Database.Migrations/scripts/**`) → + `docs/web/database.md`, `docs/web/diagrams/database-schema/*.mmd` (pick + the matching schema: beer, brewery, identity, location, media) + - Host wiring / DI registration (`API/**Program.cs`, + `Database.Seed/**Program.cs`) → `docs/web/diagrams/vertical-slice/host-wiring.mmd` + - New project references between `Features.*`/`Shared.*`/`Domain.*` + projects (new `.csproj` ``) → + `docs/web/diagrams/vertical-slice/project-references.mmd` + - `web/backend/Dockerfile*`, `web/frontend/Dockerfile*`, compose/deploy + config → `docs/web/docker.md`, `docs/web/diagrams/deployment/*.mmd` + (dev, test, minimal, db-only, prod), `docs/web/diagrams/deployment.md` + - `web/.env.example`, `web/.env.test`, appsettings → `docs/web/environment-variables.md` + - `web/backend/API/API.Specs/**`, test infra/fixtures → `docs/web/testing.md` + - New backend project setup, dev tooling → `docs/web/getting-started.md` + - `web/frontend/**` → `docs/web/architecture.md` (Frontend architecture section) + - A new file added under `docs/web/**` → also check `docs/index.md` and + `mkdocs.yml` nav are updated to reference it +3. Read the relevant doc section(s) and check each specific factual claim + (tables, "currently X", "not yet implemented", roadmap checkboxes) against + the diff. For diagrams, this means reading the `.mmd` source (not just the + wrapping `.md`) and checking nodes/edges/labels against the diff — a + schema or flow change can invalidate a diagram without touching any prose. +4. Report only what's actually stale — do not rewrite unrelated prose, do not + invent claims not grounded in the diff. +5. If nothing is affected, say so plainly and make no edits. +6. For confirmed drift, propose the exact edit (old text → new text, or the + changed `.mmd` node/edge) rather than committing directly, unless + explicitly told to apply it. Flag `.mmd` changes separately since they may + need a re-render (`docs/rendered-diagrams/web/**`) via `docs/makefile`. diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml new file mode 100644 index 00000000..c8565418 --- /dev/null +++ b/.github/workflows/deploy-docs.yml @@ -0,0 +1,51 @@ +name: Deploy Docs to GitHub Pages + +on: + push: + branches: + - main + - docs + paths: + - "docs/**" + - "mkdocs.yml" + - ".github/workflows/deploy-docs.yml" + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + deploy: + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install dependencies + run: pip install mkdocs mkdocs-material mkdocs-glightbox mkdocs-panzoom-plugin mkdocs-minify-plugin + + - name: Build site + run: mkdocs build + + - name: Upload artifact + uses: actions/upload-pages-artifact@v3 + with: + path: "site" + + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 \ No newline at end of file diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e6630881..aaba03c0 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -21,7 +21,8 @@ jobs: run: ./generate-env.sh .env.test - name: Build test images - run: docker compose --env-file .env.test -f docker-compose.test.yaml build + run: + docker compose --env-file .env.test -f docker-compose.test.yaml build - name: Run tests run: | @@ -43,7 +44,8 @@ jobs: - name: Show service logs on failure if: failure() - run: docker compose --env-file .env.test -f docker-compose.test.yaml logs + run: + docker compose --env-file .env.test -f docker-compose.test.yaml logs - name: Upload test results if: always() @@ -54,4 +56,6 @@ jobs: - name: Tear down if: always() - run: docker compose --env-file .env.test -f docker-compose.test.yaml down -v + run: + docker compose --env-file .env.test -f docker-compose.test.yaml down + -v diff --git a/.gitignore b/.gitignore index 3e4c26ab..0bc52002 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,7 @@ scratchpad.txt /web/frontend/playwright-report/ /web/frontend/test-results/ /test-results/ +/docs/rendered-diagrams/ # misc .DS_Store @@ -503,3 +504,6 @@ FodyWeavers.xsd storybook-static web/test-results/frontend/vitest-storybook.xml web/test-results/frontend/playwright-storybook.xml + +# mkdocs build output +/site diff --git a/README.md b/README.md index 17222621..9cb41d9a 100644 --- a/README.md +++ b/README.md @@ -22,32 +22,25 @@ pipeline notes) lives under `docs/`. Website + backend (active stack): -- [Getting Started](docs/website/getting-started.md) -- [Architecture](docs/architecture.md) -- [Database](docs/website/database.md) -- [Docker Guide](docs/website/docker.md) -- [Testing](docs/website/testing.md) -- [Environment Variables](docs/website/environment-variables.md) -- [Token Validation](docs/website/token-validation.md) +- [Getting Started](docs/web/getting-started.md) +- [Architecture](docs/web/architecture.md) +- [Database](docs/web/database.md) +- [Docker Guide](docs/web/docker.md) +- [Testing](docs/web/testing.md) +- [Environment Variables](docs/web/environment-variables.md) +- [Token Validation](docs/web/token-validation.md) Data generation pipeline (C++): - [Pipeline README](docs/pipeline/README.md) - [Ethics & Known Issues](docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md) -## Diagrams - -- [Architecture](docs/website/diagrams-out/architecture.svg) -- [Deployment](docs/website/diagrams-out/deployment.svg) -- [Authentication Flow](docs/website/diagrams-out/authentication-flow.svg) -- [Database Schema](docs/website/diagrams-out/database-schema.svg) - ## Current status Active areas in the repository: - .NET 10 backend (vertical-slice architecture with MediatR) + SQL Server -- React 19 website (React Router 7 + Vite) +- React 19 website (React Router 8 + Vite) - Shared Biergarten theme system + Storybook coverage - Auth flows and account/email integration (local Mailpit in dev compose) - Photo upload to S3-compatible object storage (local SeaweedFS in dev compose) @@ -61,7 +54,7 @@ Archived/reference areas: ## Tech stack - **Backend**: .NET 10, ASP.NET Core, SQL Server 2022, DbUp -- **Frontend**: React 19, React Router 7, Vite 7, Tailwind CSS 4, DaisyUI 5, +- **Frontend**: React 19, React Router 8, Vite 7, Tailwind CSS 4, DaisyUI 5, Leaflet/React Leaflet (brewery maps) - **UI Documentation**: Storybook 10, Vitest browser mode, Playwright - **Testing**: xUnit, Reqnroll (BDD), FluentAssertions, Moq @@ -73,15 +66,15 @@ Archived/reference areas: ## Requirements - .NET SDK 10 or later -- Node.js 18 or later +- Node.js 22 or later - Docker Desktop, or an equivalent Docker Engine setup -See [Getting Started](docs/website/getting-started.md) for the full +See [Getting Started](docs/web/getting-started.md) for the full prerequisite list, including the pipeline's C++ toolchain. ## Quick start -For full setup details, use [Getting Started](docs/website/getting-started.md). +For full setup details, use [Getting Started](docs/web/getting-started.md). This section is the shortest path to a working dev environment. ### Full stack (Docker) @@ -90,10 +83,15 @@ This section is the shortest path to a working dev environment. git clone https://github.com/aaronpo97/the-biergarten-app cd the-biergarten-app -cp web/.env.example web/.env.dev +cd web && ./generate-env.sh .env.dev && cd .. docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up --build -d ``` +`generate-env.sh` copies `.env.example` and replaces the secret-bearing values +with freshly randomized ones; a plain `cp` leaves placeholder secrets in +place. See [Environment Variables](docs/web/environment-variables.md#generating-an-env-file) +for details. + Access: - Website: http://localhost:3000 @@ -131,9 +129,8 @@ tooling/ pipeline/ C++20 seed-data generation CLI (CMake) docs/ - architecture.md High-level architecture overview - website/ Backend/frontend setup, docker, testing, diagrams - pipeline/ Pipeline docs, ethics notes, PlantUML diagrams + web/ Architecture, setup, docker, testing, diagrams + pipeline/ Pipeline docs, ethics notes, diagrams archive/ next-js-web-app/ Older Next.js frontend (reference only) @@ -145,13 +142,13 @@ Create a test environment file, then run the full test stack (backend and frontend) with Docker: ```bash -cp web/.env.example web/.env.test +cd web && ./generate-env.sh .env.test && cd .. docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit ``` The same command runs in CI via [`.github/workflows/tests.yml`](.github/workflows/tests.yml) on every push -and pull request to `main`. See [Testing](docs/website/testing.md) for the +and pull request to `main`. See [Testing](docs/web/testing.md) for the full command list. ## Configuration @@ -163,14 +160,14 @@ Common active variables: `WEBSITE_BASE_URL` - Frontend runtime: `API_BASE_URL`, `SESSION_SECRET`, `NODE_ENV` -See [Environment Variables](docs/website/environment-variables.md) for details. +See [Environment Variables](docs/web/environment-variables.md) for details. ## Contributing 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/amazing-feature`) 3. Commit your changes (`git commit -m 'Add amazing feature'`) -4. Run the checks in [Testing](docs/website/testing.md) (backend tests, +4. Run the checks in [Testing](docs/web/testing.md) (backend tests, `npm run lint`, `npm run typecheck`) 5. Push to the branch (`git push origin feature/amazing-feature`) 6. Open a Pull Request diff --git a/docs/architecture.md b/docs/architecture.md deleted file mode 100644 index 6a905811..00000000 --- a/docs/architecture.md +++ /dev/null @@ -1,612 +0,0 @@ -# Architecture - -This document describes the active architecture of The Biergarten App. - -## High-level overview - -The Biergarten App is a monorepo with a clear split between the backend and the -active website: - -- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature - slices with MediatR -- **Frontend**: React 19 + React Router 7 website in `web/frontend` -- **Architecture Style**: Vertical-slice backend plus server-rendered React - frontend - -The legacy Next.js frontend has been retained in `archive/next-js-web-app/` for -reference only. - -## Diagrams - -For visual representations, see: - -- [architecture.svg](website/diagrams-out/architecture.svg) - Vertical-slice - architecture diagram -- [deployment.svg](website/diagrams-out/deployment.svg) - Docker deployment - diagram -- [authentication-flow.svg](website/diagrams-out/authentication-flow.svg) - - Authentication workflow -- [database-schema.svg](website/diagrams-out/database-schema.svg) - Database - relationships - -## Backend architecture - -### Vertical-slice architecture pattern - -The backend organizes business capabilities as feature slices instead of -technical layers. Each feature (`Features.Users`, `Features.Breweries`, -`Features.Emails`, `Features.Locations`, `Features.PhotoUpload`) is a single -project that owns its own MediatR commands/queries/handlers, validators, and -repository end to end. `Features.Emails`, `Features.Locations`, and -`Features.PhotoUpload` have no `Controllers/` folder — all three are invoked -internally by other slices via MediatR/DI, never over HTTP: - -``` -┌───────────────────────────────────────────────────────────────────────┐ -│ API.Core (thin host) │ -│ - Program.cs wiring: MediatR + AddApplicationPart per slice │ -│ - Swagger/OpenAPI, JWT auth middleware, global exception filter │ -└───────────────────────────────────────────────────────────────────────┘ - ↓ discovers controllers via AddApplicationPart -┌────────────────┬───────────────┬──────────────────┬──────────────────┬──────────────────┐ -│ Features.Users │Features. │ Features.Emails │ Features. │ Features. │ -│ │Breweries │ (no controller, │ Locations │ PhotoUpload │ -│ Controllers │ Controller │ internal only) │ (no controller, │ (no controller, │ -│ Commands/ │ Commands/ │ Commands/ │ internal only) │ internal only) │ -│ Queries + │ Queries + │ Handlers │ Repository │ Commands/ │ -│ Handlers │ Handlers │ │ │ Handlers │ -│ Repository │ Repository │ EmailDispatcher │ │ Repository │ -└────────────────┴───────────────┴──────────────────┴──────────────────┴──────────────────┘ - ↓ each slice depends only on shared/domain/infra, never on another slice -┌─────────────────────────┬─────────────────────────┬────────────────────┐ -│ Shared.Contracts │ Shared.Application │ Domain.Entities / │ -│ (ResponseBody envelope) │ (ValidationBehavior, │ Domain.Exceptions │ -│ │ cross-slice email cmds) │ │ -└─────────────────────────┴─────────────────────────┴────────────────────┘ - ↓ -┌─────────────────────────────────────────────────────────────────────────┐ -│ Database.Connection, Infrastructure.Jwt, Infrastructure.PasswordHashing,│ -│ Infrastructure.Email, Infrastructure.Email.Templates │ -└─────────────────────────────────────────────────────────────────────────┘ - ↓ -┌─────────────────────────────────────┐ -│ Database (SQL Server) │ -│ - Tables & constraints │ -└─────────────────────────────────────┘ -``` - -Slices never reference each other's project. The one cross-slice interaction -(`Features.Users` triggering a confirmation email handled by `Features.Emails`) -goes through a MediatR command (`SendRegistrationEmailCommand`) whose contract -lives in `Shared.Application`, so neither slice takes a project reference on the -other. - -### Layer responsibilities - -#### API layer (`API.Core`) - -**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of its -own - -**Components**: - -- `Program.cs`: registers MediatR (scanning every `Features.*` assembly), - FluentValidation, and uses `AddApplicationPart` so each slice's controllers - are discovered by MVC -- `GlobalException.cs`: global exception filter (host-level cross-cutting - concern) -- `Authentication/JwtAuthenticationHandler.cs`: JWT auth scheme middleware -- Swagger/OpenAPI documentation, health check endpoints - -**Dependencies**: - -- Every `Features.*` project (for controller/MediatR discovery) -- `Shared.Contracts`, `Shared.Application` -- `Infrastructure.Jwt` (for the auth middleware) - -**Rules**: - -- No controllers, no business logic, no feature-specific contracts -- Exists purely to host and wire up the feature slices - -#### Feature slices (`Features.Users`, `Features.Breweries`, `Features.Emails`, `Features.Locations`, `Features.PhotoUpload`) - -**Purpose**: Each slice is the complete vertical for one business capability - -**Components** (per slice): - -- `Controllers/`: HTTP endpoints, binding directly to Command/Query types as the - request contract -- `Commands//` and `Queries//`: one folder per operation, - each containing the Command/Query record, its `IRequestHandler`, and (for - commands) a FluentValidation validator -- `Repository/`: the slice's own Dapper repository implementation -- `Dtos/`: response shapes returned by query handlers (never the raw domain - entity) -- `DependencyInjection/`: an `AddFeaturesX()` extension method registering the - slice's repository/services - -`Features.Emails` has no `Controllers/` folder. It's invoked only via MediatR -commands sent from other slices, never over HTTP. `Features.Locations` also has -no `Controllers/` folder and no MediatR handlers — it exposes -`ILocationRepository` directly as a plain service, currently consumed only by -`Database.Seed` to look up cities while seeding brewery locations. Other slices -(for example, `Features.Breweries`) do not yet depend on it; they run their own -`CityID` existence checks inline. `Features.PhotoUpload` also has no -`Controllers/` folder — its `UploadPhotoCommand` is sent by other features' -handlers (for example, a future brewery photo upload command) rather than -bound to an HTTP route directly. The one exception today is the `/upload-test` -minimal API endpoint wired directly in `API.Core/Program.cs`, a temporary route -for exercising the upload path end to end. - -**Dependencies**: - -- `Domain.Entities`, `Domain.Exceptions` -- `Infrastructure.Sql` (generic ADO.NET plumbing) plus whichever infrastructure - project the slice needs (`Infrastructure.Jwt`/ - `Infrastructure.PasswordHashing` for Users, `Infrastructure.Email`/ - `Infrastructure.Email.Templates` for Emails, `Infrastructure.FileUpload` for - PhotoUpload) -- `Shared.Contracts`, `Shared.Application` -- **Never** another `Features.*` project - -**Rules**: - -- All business logic for that feature lives in its command/query handlers -- No direct controller-to-repository calls; everything flows through MediatR -- Read endpoints return a dedicated `Dto`, never the domain entity directly - -#### Shared projects (`Shared.Contracts`, `Shared.Application`) - -**Purpose**: The minimum cross-slice surface area required because every slice -needs it, or because duplicating it four times would be worse than sharing it - -**Components**: - -- `Shared.Contracts`: `ResponseBody`/`ResponseBody`, the API response - envelope every controller returns -- `Shared.Application`: `ValidationBehavior` (the MediatR - pipeline behavior that runs FluentValidation before a handler executes) and - the cross-slice email commands (`SendRegistrationEmailCommand`, - `SendResendConfirmationEmailCommand`) - -**Rules**: - -- Kept deliberately small: this is the exception to "no slice depends on another - slice," not a general-purpose dumping ground - -#### Infrastructure layer - -**Purpose**: Technical capabilities and external integrations, shared by -whichever slices need them - -**Components**: - -- **Database.Connection**: generic ADO.NET connection/command plumbing - (`DefaultSqlConnectionFactory`, the abstract `DapperRepository` base class), not - domain-specific; each slice's own `Repository/` folder builds on this -- **Infrastructure.Jwt**: JWT token generation and validation -- **Infrastructure.PasswordHashing**: Argon2id password hashing -- **Infrastructure.Email**: Email sending capabilities (SMTP/MailKit) -- **Infrastructure.Email.Templates**: Email template rendering (Razor - components) -- **Infrastructure.FileUpload**: `IFileStorageProvider` abstraction over - S3-compatible object storage, implemented by `S3FileStorageProvider` - (targets the SeaweedFS container in dev; any S3-compatible endpoint in - other environments) using the AWS SDK's `AmazonS3Client` with - `ForcePathStyle = true` - -**Dependencies**: - -- Domain entities -- External libraries (ADO.NET, JWT, Argon2, MailKit, AWSSDK.S3, and so on) - -**Rules**: - -- Implements technical concerns only, no business logic -- Reusable across slices - -#### Domain layer (`Domain.Entities`) - -**Purpose**: Core business entities and models - -**Components**: - -- `UserAccount` - User profile data -- `UserCredential` - Authentication credentials -- `UserVerification` - Account verification state -- `BreweryPost` - A user-submitted brewery listing -- `BreweryPostLocation` - A brewery's address and coordinates -- `City` - A city referenced by a brewery location -- `StateProvince` - A state/province referenced by a city -- `Country` - A country referenced by a state/province -- `BeerStyle` - A beer style/category (schema table exists; no repository yet) -- `BeerPost` - A beer listing tied to a brewery and style (schema table - exists; no repository yet) -- `Photo` - A photo uploaded by a user account, written via - `Features.PhotoUpload`'s `IPhotoUploadRepository` - -**Dependencies**: - -- None (pure domain) - -**Rules**: - -- Plain Old CLR Objects (POCOs) -- No framework dependencies -- No infrastructure references -- Represents business concepts - -### Design patterns - -#### Vertical slice + MediatR - -**Purpose**: Organize code by feature instead of by technical layer, so -everything needed to understand or change one capability lives in one project - -**Implementation**: - -- Each HTTP write operation is a `Command` (for example, `CreateBreweryCommand` - in `Features.Breweries/Commands/CreateBrewery/`); each read operation is a - `Query` (for example, `GetBreweryByIdQuery`) -- Controllers bind directly to the Command/Query as the request body; there is - no separate request DTO + mapping step for writes -- A single shared `ValidationBehavior` - (`Shared.Application/Behaviors/`) runs FluentValidation validators in the - MediatR pipeline before any handler executes -- Query handlers map to a dedicated response `Dto`, so domain entities never - leak over the wire - -**Example**: - -```csharp -public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...) - : IRequest; - -public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler -{ - public async Task Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... } -} -``` - -#### Repository pattern - -**Purpose**: Abstract database access behind interfaces - -**Implementation**: each slice owns its own repository, scoped to that feature -only: - -- `Features.Users/Repository/IUserListRepository.cs`, - `Features.Users/Repository/IUserProfileRepository.cs` -- `Features.Breweries/Repository/IBreweryRepository.cs` -- `Features.PhotoUpload/Repository/IPhotoUploadRepository.cs` -- `Database.Connection/DefaultSqlConnectionFactory.cs`: the generic connection - factory every slice's repository builds on -- `Database.Connection/DapperRepository.cs`: shared abstract base class every - repository extends, providing `CreateConnection()` and a - `RollbackQuietlyAsync()` helper that rolls back a transaction while - swallowing any exception the rollback itself raises, so the exception that - triggered the rollback is what propagates - -**Benefits**: - -- Testable (easy to mock) -- Each slice's data access logic is self-contained - -**Example**: - -```csharp -public interface IUserListRepository -{ - Task GetByIdAsync(Guid id); - Task> GetAllAsync(int? limit, int? offset); -} -``` - -#### Dependency injection - -**Purpose**: Loose coupling and testability - -**Configuration**: `Program.cs` wires up MediatR/FluentValidation across every -`Features.*` assembly; each slice exposes its own `AddFeaturesX()` extension -method that registers its repository and slice-internal services - -**Lifetimes**: - -- Scoped: Repositories, slice-internal services (per request) -- Singleton: `DefaultSqlConnectionFactory` -- Transient: Utilities, helpers - -#### Direct SQL via Dapper/ADO.NET - -**Purpose**: Keep data access explicit and colocated with the slice that owns -it, without an ORM's change-tracking or mapping magic - -**Strategy**: - -- No ORM (Entity Framework not used); each repository issues inline SQL - through Dapper, with no manual `IDbCommand` parameter binding. The one - wrinkle is the `GEOGRAPHY` column on `BreweryPostLocation`, which Dapper - can't deserialize as a UDT: reads select `CONVERT(varbinary(max), - Coordinates)` so the value comes back as a plain `byte[]` Dapper can bind. - Proximity search (`GET /api/brewery/locations/nearby`) filters and orders - rows server-side with `Coordinates.STDistance(@Origin) <= @RangeInMetres`, - avoiding a client-side distance calculation over every row. No spatial - index backs the query yet; `schema.sql` sketches one, commented out, for - when the table grows large enough to need it -- Referential checks that a stored procedure used to perform (e.g. "does this - `CityId` exist?") are now explicit `SELECT 1 ...` existence checks in the - repository method, run inside the same transaction as the write -- Optimistic concurrency uses each table's `RowVersion` (`ROWVERSION`) column, - checked in the `UPDATE ... WHERE ... AND RowVersion = @RowVersion` clause -- Repositories throw `Domain.Exceptions` types (`NotFoundException`, - `ConflictException`, ...) directly when a check fails, rather than relying on - a database-side `THROW`; `API.Core`'s `GlobalExceptionFilter` maps those - exception types to HTTP status codes -- Application focuses on orchestration; the database enforces integrity via - keys, `CHECK` constraints, and cascades - -See [Database](website/database.md) for the schema and the app's current SQL -error-handling approach in more detail. - -## Frontend architecture - -### Active website (`web/frontend`) - -The current website is a React Router 7 application with server-side rendering -enabled. - -```text -web/frontend/ -├── app/ -│ ├── components/ Shared UI: Navbar, FormField, SubmitButton, ToastProvider, -│ │ RouteErrorState, ClientOnly (defers client-only children, -│ │ for example Leaflet maps, past hydration) -│ ├── features/ One folder per feature slice, each owning its own -│ │ routes/, components/, hooks/, schemas, and server-side -│ │ loader helpers: -│ │ - account profile/dashboard management -│ │ - auth login, register, confirm, logout -│ │ - breweries index/directory/detail routes, Leaflet -│ │ map components, nearby-search resource -│ │ route, distance/address utils -│ │ - catalog beers, beer styles -│ │ - home landing page -│ │ - theme theme switcher/guide -│ ├── hooks/ Shared hooks not tied to one feature, for example useMediaQuery -│ ├── routes.ts Route table (React Router 7 config-based routing) -│ ├── root.tsx App shell and global providers -│ └── app.css Theme tokens and global styling -├── .storybook/ Storybook config and preview setup -├── stories/ Storybook stories for shared UI and themes -├── tests/playwright/ Storybook Playwright coverage -└── package.json Frontend scripts and dependencies -``` - -### Frontend responsibilities - -- Render brewery discovery routes (`/breweries` index, `/breweries/directory`, - `/breweries/:id`) with Leaflet-powered maps and geolocation-based "nearby" - search -- Render the auth demo, account dashboard, and theme guide routes -- Manage cookie-backed website session state -- Call the .NET API for login, registration, token refresh, confirmation, and - brewery/location data -- Provide shared UI building blocks for forms, navigation, themes, and toasts -- Supply Storybook documentation and browser-based component verification - -### Theme system - -The active website uses semantic DaisyUI theme tokens backed by four Biergarten -themes: - -- Biergarten Lager -- Biergarten Stout -- Biergarten Cassis -- Biergarten Weizen - -All component styling should prefer semantic tokens such as `primary`, -`success`, `surface`, and `highlight` instead of hard-coded color values. - -### Legacy frontend - -The previous Next.js frontend has been archived at `archive/next-js-web-app/` -for reference only. Active product and engineering documentation should point to -`web/frontend`. - -## Security architecture - -### Authentication flow - -1. **Registration**: - - User submits credentials - - Password hashed with Argon2id - - User account created (unverified) and a confirmation email is dispatched - via `Features.Emails` - -2. **Email confirmation**: - - User follows the confirmation link/token from the email - - `Features.Users`' `ConfirmUser` command marks the account verified - -3. **Login**: - - User submits credentials - - Password verified against hash - - Access and refresh JWTs are issued; the refresh token is stored server-side - against the user's credential row - -4. **Token refresh**: - - Client exchanges a valid, unexpired refresh token for a new access/refresh - pair via `Features.Users`' `RefreshToken` command - - Refresh tokens are stateless JWTs, not tracked server-side: the previous - refresh token is not invalidated and remains usable until its own - expiration - -5. **API Requests**: - - Client sends the access JWT in the `Authorization` header - - `JwtAuthenticationHandler` validates the token - - Request proceeds if valid - -### Password security - -**Algorithm**: Argon2id - -- Memory: 64MB -- Iterations: 4 -- Parallelism: CPU core count -- Salt: 128-bit (16 bytes) -- Hash: 256-bit (32 bytes) - -### JWT tokens - -**Algorithm**: HS256 (HMAC-SHA256) - -**Claims**: - -- `sub` - User ID -- `unique_name` - Username -- `jti` - Unique token ID -- `iat` - Issued at timestamp -- `exp` - Expiration timestamp - -**Configuration** (appsettings.json): - -```json -{ - "Jwt": { - "ExpirationMinutes": 60, - "Issuer": "biergarten-api", - "Audience": "biergarten-users" - } -} -``` - -## Database architecture - -For the table-by-table schema and the custom SQL error code scheme, see -[Database](website/database.md). - -### SQL-first philosophy - -**Principles**: - -1. Database is source of truth -2. Queries live in the owning repository, not scattered across the app -3. Database handles referential integrity -4. Application orchestrates, database executes - -**Benefits**: - -- Performance optimization via execution plans -- Query logic scoped to the repository that owns it, not spread across layers -- Version-controlled schema (migrations) -- Easier query profiling and tuning - -### Migration strategy - -**Tool**: DbUp - -**Process**: - -1. Write a SQL script under `Database.Migrations/scripts/` -2. Embed it in the `Database.Migrations` project -3. `Database.Migrations` runs each script against the target database on - startup, tracked so a script never re-runs once applied - -**Migration Files**: - -``` -scripts/ -└── 01-schema/ - ├── 01-UserAccount.sql - ├── 02-Photo.sql - ├── ... - ├── 16-UserProfile.sql - └── 17-UserAvatar.sql # one script per CREATE TABLE -``` - -Each table has its own script, numbered so a table's foreign keys always point -at an earlier-numbered script. As the schema evolves, new numbered scripts get -added rather than editing an existing one in place, so DbUp's already-applied -tracking stays valid. - -### Data seeding - -**Purpose**: Populate development/test databases with realistic data - -**Implementation**: `Database.Seed` project, using data produced by the C++ -pipeline under `tooling/pipeline/` (see [Pipeline README](pipeline/README.md)) - -**Seed Data**: - -- Countries, state/provinces, and cities (via `Features.Locations`' - `ILocationRepository`, which creates `Country`/`StateProvince` rows on - demand while resolving a city) -- User accounts (via `Features.Users`' repository) -- Brewery posts with locations (via `Features.Breweries`' repository) - -`Photo` rows are written via `Features.PhotoUpload`'s `IPhotoUploadRepository`, -but `Database.Seed` doesn't call it, so photos aren't part of the seed data. -Tables that exist in the schema but have no repository yet (`UserAvatar`, -`UserFollow`, `BeerStyle`, `BeerPost`, `BeerPostPhoto`, `BeerPostComment`, -`BreweryPostPhoto`) are not seeded either. - -## Deployment architecture - -### Docker containerization - -**Container Structure**: - -- `sqlserver` - SQL Server 2022 -- `database.migrations` - Schema migration runner -- `database.seed` - Data seeder -- `api.core` - ASP.NET Core Web API -- `mailpit` - Local dev/test SMTP server + web UI (not used in production) -- `seaweedfs` - S3-compatible object storage for uploaded photos (dev only) - -**Environments**: - -- Development (`docker-compose.dev.yaml`) -- Testing (`docker-compose.test.yaml`) -- Production (`docker-compose.prod.yaml`) - -For details, see [Docker Guide](website/docker.md). - -### Health checks - -- **SQL Server**: Validates database connectivity -- **API**: Checks service health and dependencies - -**Configuration**: - -```yaml -healthcheck: - test: ["CMD-SHELL", "sqlcmd health check"] - interval: 10s - retries: 12 - start_period: 30s -``` - -## Testing architecture - -### Test pyramid - -``` - ┌──────────────┐ - │ Integration │ ← API.Specs (Reqnroll) - │ Tests │ - ├──────────────┤ - │ Unit Tests │ ← Features.Users.Tests, Features.Breweries.Tests, - │ (per slice, │ Features.Emails.Tests, Features.PhotoUpload.Tests - │ handler │ (commands/queries/handlers, with repository/service - │ tests) │ dependencies mocked via Moq) - └──────────────┘ -``` - -**Strategy**: - -- Many unit tests (fast, isolated) -- Fewer integration tests (slower, e2e) -- Mock external dependencies -- Test database for integration tests - -For details, see [Testing Guide](website/testing.md). diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..2ae98957 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,33 @@ +--- +title: The Biergarten App +last-updated: 2026-08-31 +tags: + - overview + - index +--- + +Welcome to the documentation for The Biergarten App. + +## Documentation Sections + +- **[Architecture](web/architecture.md)** — System design and components +- **[Website](web/getting-started.md)** — Backend and frontend documentation +- **[Pipeline](pipeline/README.md)** — Data generation pipeline + +## AI Disclosure + +Parts of this documentation are drafted and maintained with AI assistance +(Claude, Sonnet or Opus, via Claude Code), including prose (architecture +notes, guides) and diagrams (the Mermaid sources under `diagrams/`). +AI-authored changes are reviewed by a maintainer before merging, same as any +other contribution. + +Staleness checks are partly automated too, to guard against doc-rot: the +repo's `doc-drift-web` and `doc-drift-pipeline` Claude Code skills +(`.claude/skills/`) diff recent code changes against these docs and flag +claims that may need updating. + +This is separate from the pipeline's own use of AI to *generate seed data* +(brewery names, descriptions, user profiles) — see +[ETHICS-AND-KNOWN-ISSUES.md](pipeline/ETHICS-AND-KNOWN-ISSUES.md) for that +disclosure. diff --git a/docs/makefile b/docs/makefile new file mode 100644 index 00000000..f95fc772 --- /dev/null +++ b/docs/makefile @@ -0,0 +1,43 @@ +# Makefile for rendering all Mermaid (.mmd) diagrams to SVG, +# preserving the source directory structure under OUT_ROOT. +# +# Usage: +# make # render all diagrams +# make clean # remove all rendered output +# make FORMAT=png # render as PNG instead of SVG +# make FORMAT=pdf # render as PDF instead of SVG +# +# Requires: @mermaid-js/mermaid-cli (mmdc) on PATH +# npm install -g @mermaid-js/mermaid-cli + +SRC_ROOT := . +OUT_ROOT := rendered-diagrams +FORMAT := svg + +# Optional: point to a shared Mermaid config/theme file if you have one. +# CONFIG := ./mermaidconfig.json +# CONFIG_FLAG := -c $(CONFIG) +CONFIG_FLAG := + +# Find every .mmd file under SRC_ROOT, excluding the output directory itself. +SRC_FILES := $(shell find $(SRC_ROOT) -type f -name "*.mmd" -not -path "./$(OUT_ROOT)/*") + +# Map each source file to its corresponding output file, preserving subdirectories. +OUT_FILES := $(patsubst $(SRC_ROOT)/%.mmd,$(OUT_ROOT)/%.$(FORMAT),$(SRC_FILES)) + +.PHONY: all clean list + +all: $(OUT_FILES) + +# Pattern rule: build any output file from its matching .mmd source file, +# creating the destination subdirectory as needed. +$(OUT_ROOT)/%.$(FORMAT): $(SRC_ROOT)/%.mmd + @mkdir -p $(dir $@) + mmdc $(CONFIG_FLAG) -i $< -o $@ + +clean: + rm -rf $(OUT_ROOT) + +# Debug helper: print which source files were discovered. +list: + @echo "$(SRC_FILES)" | tr ' ' '\n' diff --git a/docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md b/docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md index 08d9ca9b..ebb649e9 100644 --- a/docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md +++ b/docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md @@ -1,4 +1,13 @@ -# Ethics, bias, and known issues +--- +title: Pipeline ethics, bias, and known issues +last-updated: 2026-08-31 +tags: + - pipeline + - ethics + - bias + - known-issues + - ai-generated +--- This document covers the ethical context of the Biergarten Pipeline's output, the model's biases, and known issues including hallucinated brewing science and @@ -47,7 +56,7 @@ as a source of truth for brewing techniques or cultural representation.** **Natural language processing is a powerful tool for data analysis and generation, but its output should be treated with scrutiny. Human language is -not simply data points to be analyzed — it carries cultural and human meaning +not simply data points to be analyzed: it carries cultural and human meaning that artificial intelligence cannot capture.** --- @@ -102,7 +111,7 @@ vendored verbatim, unmodified, from (the `common-forenames-by-country.json` / `common-surnames-by-country.json` release assets), released under **CC0** (public domain). That dataset's own forename/surname lists are pulled from Wikipedia's "Lists of most common -surnames" and "List of most popular given names" as of the week of 2023-07-08 — +surnames" and "List of most popular given names" as of the week of 2023-07-08; see that project's README for full provenance. Names are not LLM-generated; this is curated fixture data per ROADMAP.md §2. Per-forename gender from the source data is preserved through to the sampled `Name` (rather than discarded during @@ -113,7 +122,7 @@ for surnames) rather than trimmed to `locations.json`'s current country list, so it doesn't need re-sourcing if more countries are added later. `SampleName()` (a free helper in `generate_users.cc`) returns no result for a country present in neither file; of the countries in `locations.json`, that's currently `KE`, `SE`, -`SG`, `TH`, `VN`, and `ZA` — `GenerateUsers` skips cities in those countries the +`SG`, `TH`, `VN`, and `ZA`; `GenerateUsers` skips cities in those countries the same way brewery generation skips cities whose enrichment lookup fails. --- @@ -290,11 +299,20 @@ Each brewery and user address carries a longitude/latitude pair, sampled uniformly at random within a 5 km disc centred on the city's curated `latitude`/`longitude` from `locations.json`. -The resulting point is **plausible by construction** — it always falls within -the city's vicinity — but it is not looked up from any real address registry. -It does not correspond to a real address, business, or resident, and should -not be treated as accurate geocoding data. This is consistent with the rest of -the dataset: fixture data for a proof-of-concept, not a source of truth. +The coordinate pair itself is **plausible by construction**, always falling +within the city's vicinity, but does not correspond to a real business or +resident, and should not be treated as accurate geocoding data. This is +consistent with the rest of the dataset: fixture data for a proof-of-concept, +not a source of truth. + +For brewery addresses specifically (not user addresses, which carry no street +address at all), `address_line1`/`postal_code` _are_ looked up from a real +address registry — `NominatimAddressService` reverse-geocodes the synthetic +coordinate pair against the live OpenStreetMap/Nominatim database, so the street +address is a genuine nearby address, just not the address of an actual brewery. +This lookup only happens for non-`--mocked` runs; `--mocked` runs use +`MockAddressService`'s fixed placeholder address instead, which is not looked up +from any real registry. --- diff --git a/docs/pipeline/README.md b/docs/pipeline/README.md index 7916007e..56aad149 100644 --- a/docs/pipeline/README.md +++ b/docs/pipeline/README.md @@ -1,4 +1,13 @@ -# Biergarten Pipeline +--- +title: Biergarten Pipeline — C++ CLI for generating seed data +last-updated: 2026-09-02 +tags: + - pipeline + - cpp + - llm + - seed-data + - docker +--- A C++20 command-line pipeline that samples city records from local JSON, enriches each with Wikipedia context, and generates bilingual brewery names and @@ -7,12 +16,12 @@ deterministic mock. > **This pipeline produces AI-generated data.** It is not a source of truth for > brewing techniques, cultural representation, or local-language accuracy. See -> [ETHICS-AND-KNOWN-ISSUES.md](./ETHICS-AND-KNOWN-ISSUES.md) for a full +> [ETHICS-AND-KNOWN-ISSUES.md](./ETHICS-AND-KNOWN-ISSUES.md) for full > documentation of limitations, hallucination patterns, and bias. --- -## Table of Contents +## Table of contents - [How it fits the main app](#how-it-fits-the-main-app) - [Quick start](#quick-start) @@ -101,25 +110,27 @@ Each run writes a fresh dated SQLite file such as #### CLI flags -| Flag | Purpose | -| ------------------ | ------------------------------------------------------------------------------------------------------------ | -| `--mocked` | Deterministic mock generator, no model required. | -| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. | -| `--prompt-dir` | Directory containing prompt files (for example, `BREWERY_GENERATION.md`). Required unless `--mocked` is set. | -| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. | -| `--log-path` | Path for application logs. Default: `pipeline.log`. | -| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. | -| `--temperature` | Sampling temperature. Default: `1.0`. | -| `--top-p` | Nucleus sampling. Default: `0.95`. | -| `--top-k` | Top-k sampling. Default: `64`. | -| `--n-ctx` | Context window size. Default: `8192`. | -| `--seed` | Random seed. Default: `-1` (random at runtime). | -| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. | -| `--help, -h` | Print usage and exit. | +| Flag | Purpose | +| ------------------ | ---------------------------------------------------------------------------------------------------------------- | +| `--mocked` | Deterministic mock generator, no model required. | +| `--model, -m` | Path to a GGUF file. Selects the local Llama.cpp generator. | +| `--openai` | Use the OpenAI API (Chat Completions) for brewery/user data. Requires the `OPENAI_API_KEY` environment variable. | +| `--openai-model` | OpenAI model ID used when `--openai` is set. Default: `gpt-4o-mini`. | +| `--prompt-dir` | Directory containing prompt files (for example, `BREWERY_GENERATION.md`). Required unless `--mocked` is set. | +| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. | +| `--log-path` | Path for application logs. Default: `pipeline.log`. | +| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. | +| `--temperature` | Sampling temperature. Default: `1.0`. | +| `--top-p` | Nucleus sampling. Default: `0.95`. | +| `--top-k` | Top-k sampling. Default: `64`. | +| `--n-ctx` | Context window size. Default: `8192`. | +| `--seed` | Random seed. Default: `-1` (random at runtime). | +| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. | +| `--help, -h` | Print usage and exit. | -`--mocked` and `--model` are mutually exclusive. Omitting both exits with an -error before the pipeline starts. Sampling flags are ignored when `--mocked` is -set. +`--mocked`, `--model`, and `--openai` are mutually exclusive. Omitting all three +exits with an error before the pipeline starts. Sampling flags are ignored +unless `--model` is set (Llama). The post-build step copies `prompts/` into `build/prompts/`. Rebuild after editing any prompt file. @@ -176,7 +187,7 @@ output to confirm the fast path was taken. ### Run the container The container always runs the model-backed path; there is no `--mocked` -container mode (use a native build for that — see [Quick start](#quick-start)). +container mode (use a native build for that; see [Quick start](#quick-start)). The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if it is not already present at the configured path. @@ -191,7 +202,7 @@ docker run --rm \ By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to `./models/` on first run if it isn't already there. To use a pre-downloaded model, place it at -that path first — see [Model](#model) above. +that path first; see [Model](#model). #### Environment variables @@ -211,8 +222,9 @@ configurable via environment variable. Use a GPU pod template. Mount persistent storage for `/workspace/models`, `/workspace/output`, and `/workspace/logs`. See -`tooling/pipeline/runpod/pod-template.yaml` for a starter template — set the -environment variables listed above to match your run. +`tooling/pipeline/runpod/pod-template.yaml` for a starter template; set the +environment variables listed in [Environment variables](#environment-variables) +to match your run. --- @@ -226,8 +238,8 @@ environment variables listed above to match your run. | Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). | | Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. | | Generate Users | `GenerateUsers()` samples a persona and a forename/surname pair per enriched city (skipping countries with no name data), then `MockGenerator` or `LlamaGenerator` produces a username, bio, and activity weight around the sampled name. | -| Generate Breweries | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. | -| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `locations`, `users`, and `breweries` tables. | +| Generate Breweries | `MockGenerator`, `LlamaGenerator`, or `OpenAIGenerator` produces brewery names and descriptions in English and the local language, then `IAddressService::ReverseGeocode()` resolves a street address for the brewery's jittered coordinates. | +| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `cities`, `breweries`, `brewery_addresses`, `users`, and `user_addresses` tables. | | Log | `spdlog` writes results and warnings to the console. | If name sampling, enrichment, or generation fails for a city, that city is @@ -236,67 +248,36 @@ skipped and the pipeline continues. `GenerateUsers()` runs before ### Key components -- `src/main.cc` — argument parsing and Boost.DI composition root. -- `CuratedJsonDataService` — implements `ICuratedDataService`; takes a - `CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in - its constructor, then parses and validates curated location, persona, and - forename/surname JSON, memoizing each result after its first load on a given - instance. `MockCuratedDataService` is the in-memory substitute (4 fixed - locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in - `--mocked` runs. -- `WikipediaEnrichmentService` — queries Wikipedia extracts, caches results, - returns empty context on failure. `MockEnrichmentService` is the no-op - substitute used in `--mocked` runs. -- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for both - `GenerateBrewery` and `GenerateUser`, retries malformed responses up to three - times with corrective feedback in the retry prompt. The token budget is fixed - across attempts; it is not raised automatically on truncation. -- `MockGenerator` — stable hash-based output so the same city/persona/name input - always produces the same brewery or user. -- `SqliteExportService` — creates a dated SQLite file per run and persists each - successful user and brewery into normalized tables. -- Brewery payloads include English and local-language name and description - fields. User payloads carry a sampled first/last name and gender, an - LLM-generated username/bio/activity weight, and a programmatically generated - (not LLM-authored) unique email and date of birth. +| Component | Description | +| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/main.cc` | Argument parsing and the Boost.DI composition root. | +| `CuratedJsonDataService` | Implements `ICuratedDataService`; takes a `CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in its constructor, parses and validates curated location, persona, and forename/surname JSON, and memoizes each result after its first load on a given instance. `MockCuratedDataService` is the in-memory substitute (4 fixed locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in `--mocked` runs. | +| `WikipediaEnrichmentService` | Queries Wikipedia extracts, caches results, and returns empty context on failure. `MockEnrichmentService` is the no-op substitute used in `--mocked` runs. | +| `LlamaGenerator` | Formats prompts for Gemma 4, validates JSON output for both `GenerateBrewery` and `GenerateUser`, and retries malformed responses up to three times with corrective feedback in the retry prompt. The token budget is fixed across attempts; it is not raised automatically on truncation. | +| `MockGenerator` | Uses stable hash-based output so the same city/persona/name input always produces the same brewery or user. | +| `NominatimAddressService` | Reverse-geocodes each brewery's jittered coordinates into a street address (`address_line1`, `postal_code`) via the public [Nominatim API](https://nominatim.openstreetmap.org), respecting its one-request-per-second usage policy. Falls back to a placeholder `address_line1` ("Address unavailable") and empty `postal_code` when the lookup fails or returns no usable street address. `MockAddressService` is the no-network substitute (`"123 Mock Street"`, `"00000"`) used in `--mocked` runs. Only breweries get a resolved street address; generated users carry a coordinate pair with no address fields. | +| `SqliteExportService` | Creates a dated SQLite file per run and persists each successful user and brewery into normalized tables. | +| Brewery/user payloads | Brewery payloads include English and local-language name and description fields. User payloads carry a sampled first/last name and gender, an LLM-generated username/bio/activity weight, and a programmatically generated (not LLM-authored) unique email and date of birth. | ### Runtime behaviour -`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a generic -"brewing" extract and a "beer in `{country}`" extract. It does not currently -query a city- or region-specific page. Each query string is cached after its -first successful (or empty) lookup. - -`GetLocationContext()` returns an empty string when the web client is -unavailable or when lookup/parsing fails. - -`LlamaGenerator` validates model output as structured JSON. On validation -failure it retries up to three times, replaying the previous error message in -the next prompt so the model can self-correct. All runs to date have produced -valid output on the first pass; the retry path is kept for resilience. - -`MockGenerator` uses stable hashes for repeatable output in demos and Storybook -runs. - -`CuratedJsonDataService` memoizes each of `LoadLocations()`, `LoadPersonas()`, -`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the -first time each is called, since `BiergartenPipelineOrchestrator` owns a single -`ICuratedDataService` instance for the whole run — later calls return the cached -result instead of re-parsing. - -`GenerateUsers()` samples a forename/surname pair per city via `SampleName()`, -keyed by the city's ISO 3166-1 code. Countries present in `locations.json` but -absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`, `ZA`) -are skipped, the same way a failed enrichment or generation call skips a city — -see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section. +| Component / behavior | Details | +| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `WikipediaEnrichmentService` | Fetches two Wikipedia extracts per city: a generic "brewing" extract and a "beer in `{country}`" extract. It does not currently query a city- or region-specific page. Each query string is cached after its first successful (or empty) lookup. | +| `GetLocationContext()` | Returns an empty string when the web client is unavailable or when lookup/parsing fails. | +| `LlamaGenerator` | Validates model output as structured JSON. On validation failure it retries up to three times, replaying the previous error message in the next prompt so the model can self-correct. All runs to date have produced valid output on the first pass; the retry path is kept for resilience. | +| `MockGenerator` | Uses stable hashes for repeatable output in demos and Storybook runs. | +| `NominatimAddressService.ReverseGeocode()` | Sleeps one second after every request to respect Nominatim's usage policy, and returns `std::nullopt` (logged as a warning, not a skipped city) if the request fails, the response has no `address` object, or neither `house_number` nor `road` is present — the brewery is still generated and exported, just with a placeholder address. | +| `CuratedJsonDataService` | Memoizes each of `LoadLocations()`, `LoadPersonas()`, `LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the first time each is called, since `BiergartenPipelineOrchestrator` owns a single `ICuratedDataService` instance for the whole run; later calls return the cached result instead of re-parsing. | +| `GenerateUsers()` | Samples a forename/surname pair per city via `SampleName()`, keyed by the city's ISO 3166-1 code. Countries present in `locations.json` but absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`, `ZA`) are skipped, the same way a failed enrichment or generation call skips a city; see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section. | ### Process flow - activity diagram -![An activity diagram](./diagrams/current/output/activity.svg) +See [Activity diagram](./diagrams/current/activity.md). ### Architectural overview - class diagram -![A class diagram](./diagrams/current/output/class.svg) +See [Class diagram](./diagrams/current/class.md). --- @@ -307,12 +288,14 @@ Each successful run stores a `BreweryRecord` pair with the source location and a `UserResult` payload. The same generated records are also written to a fresh SQLite export file named with the current UTC timestamp. -| Field | Meaning | -| ------------------- | ------------------------------------------ | -| `name_en` | Brewery name in English. | -| `description_en` | Brewery description in English. | -| `name_local` | Brewery name in the local language. | -| `description_local` | Brewery description in the local language. | +| Field | Meaning | +| ------------------- | ------------------------------------------------------------------------------------- | +| `name_en` | Brewery name in English. | +| `description_en` | Brewery description in English. | +| `name_local` | Brewery name in the local language. | +| `description_local` | Brewery description in the local language. | +| `address_line1` | Street address resolved via reverse geocoding, or a placeholder if resolution failed. | +| `postal_code` | Postal code resolved via reverse geocoding, or empty if resolution failed. | | Field | Meaning | | ----------------- | -------------------------------------------------------------------------------------- | @@ -357,15 +340,14 @@ The build fetches Boost.DI, spdlog, and SQLite via CMake. llama.cpp is fetched only when a system installation is not detected. Metal is enabled on Apple Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present. -> **Code Style:** Code style generally follows the Google Style Guide and -> features modern C++20 throughout. Formatting follows the Google C++ Style -> Guide, with the exception of indentation width (3 spaces vs. Google's 2). +> **Code style:** The code follows the Google C++ Style Guide, except that +> indentation is 3 spaces instead of Google's 2, and uses C++20 throughout. --- ## Tested hardware -### ARM macOS — M1 Pro +### ARM macOS (M1 Pro) | | | | --------- | --------------------------------- | @@ -376,7 +358,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present. | Model | Gemma 4 E4B | | Inference | llama.cpp with Metal | -### x86_64 Linux — NVIDIA RTX 2000 +### x86_64 Linux (NVIDIA RTX 2000) | | | | --------- | ------------------------------ | @@ -387,7 +369,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present. | Model | Gemma 4 E4B | | Inference | llama.cpp with CUDA 12.x | -### x86_64 Linux — Docker / RunPod (NVIDIA CUDA) +### x86_64 Linux (Docker / RunPod, NVIDIA CUDA) | | | | --------- | ------------------------------------------- | @@ -407,7 +389,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present. - Keep `locations.json` structured enough to support discovery and future filtering. - `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json` - are curated/vendored fixture data, not LLM-generated — see + are curated/vendored fixture data, not LLM-generated; see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for provenance. - Treat SQLite output as seed material for the app's brewery and user domains, not production data. @@ -435,30 +417,32 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present. Paths below are relative to `tooling/pipeline/`. -- `src/main.cc` — argument parsing and DI composition root. -- `src/biergarten_pipeline_orchestrator/` — orchestration, sampling, logging, - and export. -- `src/services/curated_data/` — `CuratedJsonDataService`, the file-backed - `ICuratedDataService`, and `MockCuratedDataService`, the in-memory - `ICuratedDataService` used in `--mocked` runs. -- `src/services/enrichment/wikipedia/` — enrichment service and cache. -- `src/services/sqlite/` — SQLite export implementation. -- `src/data_generation/llama/` — local inference, prompt loading, output - validation. -- `src/data_generation/mock/` — deterministic fallback. -- `runpod/` — container build and runtime launcher. +| Path | Purpose | +| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/main.cc` | Argument parsing and DI composition root. | +| `src/biergarten_pipeline_orchestrator/` | Orchestration, sampling, logging, and export. | +| `src/services/curated_data/` | `CuratedJsonDataService`, the file-backed `ICuratedDataService`, and `MockCuratedDataService`, the in-memory `ICuratedDataService` used in `--mocked` runs. | +| `src/services/enrichment/wikipedia/` | Enrichment service and cache. | +| `src/services/sqlite/` | SQLite export implementation. | +| `src/data_generation/llama/` | Local inference, prompt loading, and output validation. | +| `src/data_generation/mock/` | Deterministic fallback. | +| `runpod/` | Container build and runtime launcher. | --- ## Next steps The pipeline currently produces city-aware brewery and user records and dated -SQLite exports. The next passes add additional fixture types so the app can -exercise the full brewery and social domains without live data. For the detailed -engineering breakdown of what's needed to reach the architecture in -[`diagrams/planned/`](./diagrams/planned/), see [ROADMAP.md](./ROADMAP.md). +SQLite exports. -### Testing — very high priority +The next passes add additional fixture types so the app can exercise the full +brewery and social domains without live data. + +For the detailed engineering breakdown of what's needed to reach the +architecture in [`diagrams/planned/`](./diagrams/planned/), see +[ROADMAP.md](./ROADMAP.md). + +### Testing - Unit test JSON validation and retry logic against malformed, truncated, and empty model outputs. @@ -481,7 +465,7 @@ enough to exercise search, sort, and category filters. ### Check-in system Produce timestamped check-in events between users and breweries. Use a J-curve -activity profile — a small set of users accounts for most check-ins, the rest +activity profile: a small set of users accounts for most check-ins, the rest appear occasionally. Add bursty behaviour around weekends and travel periods. ### Beer ratings diff --git a/docs/pipeline/ROADMAP.md b/docs/pipeline/ROADMAP.md index 71e01f06..33d0229b 100644 --- a/docs/pipeline/ROADMAP.md +++ b/docs/pipeline/ROADMAP.md @@ -1,21 +1,24 @@ -# Pipeline Roadmap — Reaching the Planned Architecture +--- +title: Pipeline roadmap — reaching the planned architecture +last-updated: 2026-09-01 +tags: + - pipeline + - roadmap + - cpp + - concurrency +--- -This is the engineering breakdown for closing the gap between the **current** -pipeline and the **planned** architecture documented in -[`diagrams/planned/class.puml`](./diagrams/planned/class.puml) and -[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing in -`diagrams/planned/` is implemented yet — this file tracks what it would take to -get there. For the current implementation, see [README.md](./README.md). +Engineering breakdown for closing the gap between the **current** pipeline and +the **planned** architecture in +[`diagrams/planned/class.md`](./diagrams/planned/class.md) / +[`diagrams/planned/activity.md`](./diagrams/planned/activity.md). Items are +grouped by layer, roughly by dependency order. For the current implementation, +see [README.md](./README.md). -Items are grouped by layer and roughly ordered by dependency: later groups build -on types and services introduced earlier. - -The only concurrency in the current pipeline is the log dispatcher thread -(`main.cc` spawns it to drain a `BoundedChannel` into spdlog for the -whole run). Sampling, enrichment, generation, and export all happen -synchronously on the main thread inside `BiergartenPipelineOrchestrator::Run()` -— §6 below is what introduces the additional worker threads the planned -architecture needs. +The only concurrency today is a log dispatcher thread draining a +`BoundedChannel` into spdlog; sampling, enrichment, generation, and +export run synchronously on the main thread — §6 introduces the planned worker +threads. --- @@ -23,222 +26,183 @@ architecture needs. `tooling/pipeline/includes/data_model/generated_models.h` and `models.h`. -- [ ] Add `Completeness` enum (`Full`, `Partial`, `Absent`) and a - `LocationContext` struct (`text`, `completeness`, `char_count`). Replace - `EnrichedCity::region_context` (currently a plain `std::string`) with - `context : LocationContext`. +- [ ] Add `Completeness` enum (`Full`/`Partial`/`Absent`) and a + `LocationContext` struct (`text`, `completeness`, `char_count`); replace + `EnrichedCity::region_context` (a plain `string` today) with it. - [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`, `min_ibu`, `max_ibu`). -- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads. +- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` payloads. - [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`, `context_provided`, `generated_with`). -- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to - `UserResult` (currently just `username`, `bio`). `first_name`/ - `last_name`/`gender` are copied from the sampled `Name` (see below), not +- [x] `UserResult` gained `first_name`/`last_name`/`gender`/`activity_weight` + (was just `username`/`bio`), copied from the sampled `Name`, not LLM-invented. -- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result - handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`, - `gender`). Landed as a flatter shape than originally planned here: no - `NamesByCountry` wrapper class. `curated_data_service.h` instead declares - `ForenameList = std::vector` and - `SurnameList = std::vector`, and `ICuratedDataService` - exposes two maps keyed by ISO 3166-1 code — - `ForenamesByCountryMap = unordered_map` and - `SurnamesByCountryMap = unordered_map` — directly - (see §2). Sampling is a free function, - `SampleName(forenames_by_country, surnames_by_country, iso3166_1, rng)`, - in `generate_users.cc`'s anonymous namespace, not a method on a class. - Pairing a forename with a surname happens at sample time so gender - (present per-forename in the source data) is never discarded the way a - pre-flattened `{first_name, last_name}` list would lose it; see §2. +- [x] Added `Name` (`first_name`, `last_name`, `gender`) and `ForenameEntry` + (`name`, `gender`). Flatter than planned: no `NamesByCountry` wrapper — + `ICuratedDataService` exposes `ForenamesByCountryMap`/ + `SurnamesByCountryMap` (keyed by ISO 3166-1) directly, and sampling is a + free function, `SampleName(forenames, surnames, iso3166_1, rng)`, not a + class method. - [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`, `metadata` (currently just `{location, brewery}`). - [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`, `GeneratedFollow` aggregate structs. -- [x] Add `UserRecord` (`location`, `user : UserResult`, `email`, - `date_of_birth`) as the current stand-in for the planned `GeneratedUser` — - `email` and `date_of_birth` are programmatically generated by the - orchestrator, never LLM-authored, so a downstream auth-account seeding - consumer can register real accounts from the pipeline's SQLite export. No - `password`, `user_id`, or `metadata` field yet, matching `BreweryRecord`'s - equally pre-`metadata` shape. Already wired into - `IExportService`/`SqliteExportService`: a `users` table exists and - `GenerateUsers()` exports each successful record via - `ProcessRecord(const UserRecord&)` (see §5) in addition to logging and - holding `generated_users_` in memory. Still missing vs. the planned - `GeneratedUser`: `user_id`, `password`, and `metadata`. -- [x] Add `UserPersona` (`name`, `description`, `style_affinities`). +- [x] Added `UserRecord` (`location`, `user`, `email`, `date_of_birth`), the + current stand-in for the planned `GeneratedUser`. `email`/ `date_of_birth` + are programmatic (never LLM-authored), for downstream auth-account + seeding. Missing `user_id`/`password`/`metadata`; already exported via + `ProcessRecord(const UserRecord&)` (§5). +- [x] Added `UserPersona` (`name`, `description`, `style_affinities`). ## 2. Data Preloading `tooling/pipeline/includes/services/curated_data/`, fixture files. -- [x] Extract an interface; have `CuratedJsonDataService` implement it. Landed - as `ICuratedDataService` (`services/curated_data/curated_data_service.h`), - not `DataPreloader` — `LoadLocations()`, `LoadPersonas()`, - `LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` are all virtual - methods returning `const&` and take no arguments. `CuratedJsonDataService` - (`services/curated_data/curated_json_data_service.h`, originally landed as - `JsonLoader` under `json_handling/` before being renamed and moved) takes - a `CuratedDataFilePaths` DTO (`locations_path`, `personas_path`, - `forenames_path`, `surnames_path`) in its constructor rather than a path - per `Load*()` call, and memoizes each result in a private `cache` struct - (`locations`, `personas`, `forenames_by_country`, `surnames_by_country`), - so a second call on the same instance returns the cached result instead of - re-parsing — safe because `CuratedJsonDataService` outlives every call - site (owned by `BiergartenPipelineOrchestrator` via - `unique_ptr` for the whole run). `main.cc`'s DI - injector also gained a `MockCuratedDataService` (fixed in-memory data: 4 - locations across `US`/`DE`/`FR`/`BE`, 3 personas, and matching - forename/surname sets for those four countries), bound in place of - `CuratedJsonDataService` under `--mocked`, mirroring the existing - `MockEnrichmentService`/`MockGenerator` pattern. -- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo and - is already copied into the Docker image (`runpod/Dockerfile`), but no - loader reads it yet, and the native CMake build doesn't copy it into - `build/` at all — `CMakeLists.txt`'s "Runtime Assets" step only copies - `locations.json` and `prompts/`. -- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet). -- [x] Add name-by-country loading, parsing - `tooling/pipeline/forenames-by-country.json` and - `surnames-by-country.json`. Landed as two separate methods — - `LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each returning - a flat `ForenamesByCountryMap` / `SurnamesByCountryMap` (aliases for - `unordered_map` / - `unordered_map`) keyed by ISO 3166-1 code, rather - than one combined `LoadNamesByCountry() : NamesByCountry` call. Both files - are vendored verbatim (unmodified, full multinational coverage) from - `sigpwned/popular-names-by-country-dataset` (CC0) — see - ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for - provenance. Deliberately not pre-paired or filtered down to just the - countries in `locations.json`: keeping the source shape (including - per-forename gender) intact means the loader can support more countries - later, or gender-aware persona/bio generation, without re-sourcing data. +- [x] Extracted `ICuratedDataService` (not `DataPreloader`); `Load*()` methods + take no arguments, return `const&`. `CuratedJsonDataService` (renamed from + `JsonLoader`) takes a `CuratedDataFilePaths` DTO at construction and + memoizes each `Load*()` result. `MockCuratedDataService` (fixed data for + `US`/`DE`/`FR`/`BE`) binds in its place under `--mocked`. +- [ ] Add `LoadBeerStyles()`. `beer-styles.json` exists and is copied into the + Docker image, but no loader reads it and the native CMake build doesn't + copy it into `build/` at all. +- [x] Added `LoadPersonas()` and authored `personas.json`. +- [x] Added `LoadForenamesByCountry()` / `LoadSurnamesByCountry()` (two methods, + not one combined `LoadNamesByCountry()`), parsing vendored CC0 data from + `sigpwned/popular-names-by-country-dataset` — see + ETHICS-AND-KNOWN-ISSUES.md for provenance. Deliberately not pre-paired or + filtered to `locations.json`'s countries, to keep per-forename gender and + support more countries later without re-sourcing. ## 3. Policy / Strategy Layer -Entirely new — no `includes/policy/` (or equivalent) directory exists today. +Entirely new — no `includes/policy/` (or equivalent) exists today. -- [ ] `ContextStrategy` interface + `BreweryContextStrategy` / - `BeerContextStrategy`. Today - `WikipediaEnrichmentService::GetLocationContext` hardcodes a generic - `"brewing"` query and a `"beer in {country}"` query directly — no - per-phase strategy selection. -- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing the - inline `std::ranges::sample(...)` call in - `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()`. -- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to pick - styles per brewery from the `BeerStyle` palette (depends on §1 and §2). -- [ ] `CheckinDistributionStrategy` interface + `JCurveCheckinStrategy` / - `RandomCheckinStrategy` — this is the "Check-In System" item already - called out in README.md's Next Steps, made concrete. -- [ ] `FollowGenerationStrategy` interface + `RandomFollowStrategy` / +- [ ] `ContextStrategy` + `BreweryContextStrategy`/`BeerContextStrategy`. Today + `WikipediaEnrichmentService::GetLocationContext` hardcodes a `"brewing"` + query and a `"beer in {country}"` query directly. +- [ ] `SamplingStrategy` + `UniformSamplingStrategy`, replacing the inline + `std::ranges::sample(...)` in `QueryCitiesWithCountries()`. +- [ ] `BeerSelectionStrategy` + `RandomBeerSelectionStrategy` (depends on §1, + §2). +- [ ] `CheckinDistributionStrategy` + `JCurveCheckinStrategy` / + `RandomCheckinStrategy` — the "Check-In System" README next step, made + concrete. +- [ ] `FollowGenerationStrategy` + `RandomFollowStrategy` / `ActivityWeightedFollowStrategy`. ## 4. Data Generation `tooling/pipeline/includes/data_generation/`, `src/data_generation/`. -- [ ] Extend the `DataGenerator` interface with `GenerateBeer`, - `GenerateCheckin`, `GenerateRating` (today: only `GenerateBrewery` and - `GenerateUser`). -- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop - mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3 - attempts with corrective feedback) and a new `prompts/USER_GENERATION.md`. - Changed the `DataGenerator::GenerateUser` signature to - `(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`, - matching what the activity diagram actually passes. - `MockGenerator::GenerateUser` updated to match the new signature too. -- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` / - `GenerateRating`. +- [ ] Extend `DataGenerator` with `GenerateBeer`, `GenerateCheckin`, + `GenerateRating` (today: only `GenerateBrewery`/`GenerateUser`). +- [x] Added `OpenAIGenerator`, a third backend calling the OpenAI Chat + Completions API with Structured Outputs (`json_schema`, `strict`) — + schema-valid JSON with no `IPromptFormatter`/GBNF grammar needed. Selected + via `--openai`/`GeneratorMode::kOpenAI`; landed ahead of §3/§4. Now shown + in `diagrams/planned/class.md` alongside `MockGenerator`/`LlamaGenerator`, + noting only `GenerateBrewery`/`GenerateUser` are implemented so far. +- [x] Implemented `LlamaGenerator::GenerateUser` for real (GBNF grammar, + `ValidateUserJson`, 3 retries with corrective feedback, + `prompts/USER_GENERATION.md`). Signature is + `(city, persona, name) : UserResult`, matching the activity diagram. +- [ ] Implement + `MockGenerator::GenerateBeer`/`GenerateCheckin`/`GenerateRating`. - [ ] Add `IPromptFormatter::ExpectedArchitecture()` and - `LlamaGenerator::ValidateModelArchitecture()` so loading a GGUF that - doesn't match the configured chat template fails fast instead of silently - producing degraded output. -- [ ] Add prompt template files for beer/checkin/rating generation next to the - existing `prompts/BREWERY_GENERATION.md`. + `LlamaGenerator::ValidateModelArchitecture()` so a mismatched GGUF chat + template fails fast instead of degrading silently. +- [ ] Add prompt templates for beer/checkin/rating, next to + `prompts/BREWERY_GENERATION.md`. ## 5. Export Service `tooling/pipeline/includes/services/database/`, `src/services/sqlite/`. -- [x] `IExportService::ProcessRecord(const UserRecord&)` and - `SqliteExportService`'s `users` table (see `kCreateUsersTableSql` / - `kInsertUserSql` in - `includes/services/database/sqlite_statement_helpers.h`) are implemented — - `GenerateUsers()` exports every successful user alongside its resolved +- [x] `IExportService::ProcessRecord(const UserRecord&)` and the `users` table + are implemented — every successful user is exported with its resolved `location_id`. - [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`, - `ProcessRating`, `ProcessFollow` (today - `ProcessRecord(const BreweryRecord&)` and - `ProcessRecord(const UserRecord&)` exist). + `ProcessRating`, `ProcessFollow`. - [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to - `SqliteExportService::InitializeSchema()` (`locations`, `breweries`, and - `users` already exist) — see `kCreateLocationsTableSql` / - `kCreateBreweriesTableSql` / `kCreateUsersTableSql` in - `includes/services/database/sqlite_statement_helpers.h` for the existing - pattern to follow. -- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and move - from one open transaction for the whole run to per-phase batched commits - (`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as shown in the - planned activity diagram. + `InitializeSchema()` (`cities`/`breweries`/`brewery_addresses`/`users`/ + `user_addresses` already exist — see `sqlite_statement_helpers.h` for the + pattern). +- [ ] Add a `brewery_cache_` alongside `city_cache_`, and move from one open + transaction for the whole run to per-phase batched commits (`BEGIN` / + `COMMIT & BEGIN` on a batch threshold), per the planned activity diagram. ## 6. Concurrency & Orchestration `tooling/pipeline/includes/concurrency/`, `src/biergarten_pipeline_orchestrator/`. -- [ ] `BoundedChannel` already exists and is production-tested — but it's - only wired to the single log channel today. Stand up the per-phase - producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the planned activity - diagram describes, each with a dedicated LLM worker thread and SQLite - worker thread. -- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous pass - over `generated_breweries_` into phased methods — `RunUserPhase` → - `RunBreweryAndBeerPhase` (with a `RunBeerPhase` sub-step once - `brewery_pool_` is populated) → `RunCheckinPhase` / `RunFollowPhase` - (these two can run in parallel) → `RunRatingPhase` — backed by - `user_pool_`, `brewery_pool_`, `beer_pool_`, `checkin_pool_`, - `follow_pool_`. -- [ ] Honor the structural-concurrency requirement already called out in a - comment on `BiergartenPipelineOrchestrator::Run()` in - `biergarten_pipeline_orchestrator.h`: once real worker threads exist, they - must be structurally joined (e.g. via `std::jthread`) before `Run()` - returns, so no worker logs to a closed channel during teardown. +- [ ] `BoundedChannel` exists and is production-tested but only wired to the + log channel. Stand up the per-phase producer/consumer channels (`loc_ch`, + `exp_ch`, etc.) with a dedicated LLM worker thread and SQLite worker + thread each; the brewery phase needs a third, a Geocode Worker between + them on its own `geo_ch` (§9). +- [ ] Rewrite `Run()` from one synchronous pass into phased methods — + `RunUserPhase` → `RunBreweryAndBeerPhase` (`RunBeerPhase` sub-step once + `brewery_pool_` fills) → `RunCheckinPhase`/`RunFollowPhase` (parallel) → + `RunRatingPhase` — backed by `user_pool_`, `brewery_pool_`, `beer_pool_`, + `checkin_pool_`, `follow_pool_`. +- [ ] Structurally join worker threads (e.g. `std::jthread`) before `Run()` + returns, per the existing comment in `biergarten_pipeline_orchestrator.h`, + so nothing logs to a closed channel during teardown. ## 7. Enrichment -- [ ] Decide whether to restore the city/region-specific Wikipedia query that's - currently commented out in - `src/services/enrichment/wikipedia/get_summary.cc` (`GetLocationContext`). - The `ContextStrategy` work in §3 is a natural place to reintroduce it via - `BreweryContextStrategy::QueriesFor()`. +- [ ] Decide whether to restore the city/region-specific Wikipedia query + currently commented out in `get_summary.cc` — the §3 `ContextStrategy` + work is a natural place to reintroduce it. - [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`, - `PreWarmLocationCache` in the planned activity diagram) instead of - fetching lazily per record, so the streaming phases never block on a cold - cache mid-run. + `PreWarmLocationCache`) instead of fetching lazily per record. ## 8. Fixtures / Build -- [x] Author `personas.json`, `forenames-by-country.json`, and - `surnames-by-country.json` (see §2). +- [x] Authored `personas.json`, `forenames-by-country.json`, + `surnames-by-country.json` (§2). - [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime Assets" - `configure_file` step in `CMakeLists.txt` so native builds and the Docker - image agree on what's available at runtime. + `configure_file` step in `CMakeLists.txt`. + +## 9. Geocoding + +`tooling/pipeline/includes/services/address/`, `src/services/address/` — added +entirely after `diagrams/planned/` was authored; wasn't tracked here until now. + +- [x] Added `IAddressService` + (`ReverseGeocode(lon, lat) : optional`), + `NominatimAddressService` (public Nominatim API, 1s sleep per request, + `nullopt` on failure or no usable address), and `MockAddressService` + (fixed placeholder for `--mocked`). Called once per brewery, after + `GenerateBrewery` and before export, to resolve + `BreweryAddress::address_line1`/`postal_code` — users aren't geocoded. + `diagrams/planned/class.md` now models this as an + `InfrastructureGeocoding` namespace owned by the orchestrator. +- [ ] Give the brewery phase its own **Geocode Worker** thread, between the LLM + worker and the SQLite worker (`loc_ch` → LLM Worker → `geo_ch` → Geocode + Worker → `exp_ch` → SQLite Worker), rather than calling `ReverseGeocode` + inline — Nominatim's rate limit is unrelated to inference speed, so + coupling them stalls whichever side is slower and needlessly cools the + LLM's KV cache. It never skips a record on lookup failure, only attaches a + placeholder; shown as its own subgraph (`B_GEO`) in + `diagrams/planned/activity.md`. One worker per phase is safe today since + `RunUserPhase` joins before `RunBreweryPhase` starts; concurrent phases + plus user-geocoding would need a shared rate-limited client instead, since + the throttle is global, not per-thread. --- ## Suggested build order -The planned activity diagram's own phase notes ("brewery*pool* is now fully -populated. Phase 1b may begin.", etc.) imply a dependency order. Roughly: +The planned activity diagram's phase notes imply a dependency order: 1. Domain models (§1) and data preloading (§2) — nothing else compiles without these. -2. Export schema (§5) — so every later generation phase has somewhere to land - its output. +2. Export schema (§5) — so every generation phase has somewhere to land output. 3. Policy/strategy layer (§3) and generator interface (§4). -4. Concurrency/orchestration rewrite (§6), which is the only piece that actually - wires §1–§5 into the phased, parallel pipeline shown in the planned diagrams. -5. Enrichment cache pre-warming and the city/region query decision (§7) — useful - at any point, but most valuable once phases run concurrently. +4. Concurrency/orchestration rewrite (§6) — wires §1–§5 into the phased, + parallel pipeline the planned diagrams show. +5. Enrichment cache pre-warming and the city/region query decision (§7) — most + valuable once phases run concurrently. diff --git a/docs/pipeline/diagrams/biergarten-weizen-theme.puml b/docs/pipeline/diagrams/biergarten-weizen-theme.puml deleted file mode 100644 index b31305d6..00000000 --- a/docs/pipeline/diagrams/biergarten-weizen-theme.puml +++ /dev/null @@ -1,34 +0,0 @@ -skinparam shadowing false -skinparam backgroundColor #FCFCF7 -skinparam defaultFontName "DM Sans" -skinparam defaultFontColor #14180C -skinparam titleFontName "Volkhov" -skinparam titleFontColor #14180C -skinparam ArrowColor #656F33 -skinparam NoteBackgroundColor #DBEEDD -skinparam NoteFontColor #14180C -skinparam NoteBorderColor #4A5837 -skinparam SwimlaneBorderColor #4A5837 -skinparam SwimlaneBorderThickness 1 -skinparam activityStartColor #EBECE3 -skinparam activityEndColor #4A5837 -skinparam activityStopColor #4A5837 -skinparam ActivityBackgroundColor #EBECE3 -skinparam ActivityBorderColor #4A5837 -skinparam ActivityDiamondBackgroundColor #CBD2B5 -skinparam ActivityDiamondBorderColor #4A5837 -skinparam packageStyle rectangle -skinparam packageBackgroundColor #F1F3EA -skinparam packageBorderColor #4A5837 -skinparam packageFontColor #14180C -skinparam classBackgroundColor #EBECE3 -skinparam classBorderColor #4A5837 -skinparam classFontColor #14180C -skinparam classAttributeFontColor #3F4724 -skinparam classStereotypeFontColor #4A5837 -skinparam interfaceBackgroundColor #DBEEDD -skinparam interfaceBorderColor #4A5837 -skinparam interfaceFontColor #14180C -skinparam enumBackgroundColor #E4E6D8 -skinparam enumBorderColor #4A5837 -skinparam enumFontColor #14180C diff --git a/docs/pipeline/diagrams/current/activity.md b/docs/pipeline/diagrams/current/activity.md new file mode 100644 index 00000000..49eb862b --- /dev/null +++ b/docs/pipeline/diagrams/current/activity.md @@ -0,0 +1,11 @@ +# The Biergarten Data Pipeline (Current — Synchronous Data Path) + +The only concurrency in the current pipeline is a dedicated log dispatcher +thread that drains `log_channel` and forwards entries to spdlog for the entire +run (joined during shutdown after `log_channel` closes). +`BiergartenPipelineOrchestrator::Run()` itself is synchronous — sampling, +enrichment, generation, and export all happen on the main thread. + +```mermaid +--8<-- "pipeline/diagrams/current/activity.mmd" +``` diff --git a/docs/pipeline/diagrams/current/activity.mmd b/docs/pipeline/diagrams/current/activity.mmd new file mode 100644 index 00000000..d98403bc --- /dev/null +++ b/docs/pipeline/diagrams/current/activity.mmd @@ -0,0 +1,86 @@ +flowchart TD + Start([start]) --> ParseArgs["main.cc: ParseArguments(argc, argv)"] + ParseArgs --> ArgsValid{Arguments valid?} + ArgsValid -- no --> LogErr[Log error / usage info] --> Stop1([stop]) + ArgsValid -- yes --> Init["main.cc: Init OpenSSL global state and LlamaBackendState"] + Init --> DI["main.cc: di::make_injector(...)
binds ILogger, IEnrichmentService, DataGenerator, IExportService, IPromptFormatter, WebClient based on --mocked vs. model-backed mode"] + DI --> LogThread["main.cc: create BoundedChannel log_channel, spawn log dispatcher thread (LogDispatcher::Run)"] + LogThread --> CreateOrch["main.cc: injector.create BiergartenPipelineOrchestrator"] + CreateOrch --> Run["BiergartenPipelineOrchestrator::Run()"] + + Run --> SqliteInit["SqliteExportService.Initialize()
builds biergarten_seed_<UTC datetime>.sqlite (numeric suffix if it exists), opens DB connection, executes schema DDL, begins transaction"] + SqliteInit --> QueryLoc[Orchestrator: QueryLocations] + QueryLoc --> LoadLoc["CuratedJsonDataService.LoadLocations()"] + LoadLoc --> SampleLoc["std::ranges::sample(all_locations, --location-count) — defaults to 10"] + + SampleLoc --> EnrichLoop{For each sampled Location} + EnrichLoop --> GetContext["IEnrichmentService.GetLocationContext(loc)
WikipediaEnrichmentService fetches a generic 'brewing' extract and a 'beer in {country}' extract (not city/region-specific); MockEnrichmentService (--mocked) returns an empty string"] + GetContext --> LookupFailed{Lookup failed?} + LookupFailed -- yes --> SkipCity1[Log warning, skip city] --> EnrichLoop + LookupFailed -- no --> StoreEnriched["Store EnrichedCity{Location, region_context}"] --> EnrichLoop + EnrichLoop -- done --> GenUsers[Orchestrator: GenerateUsers enriched_cities] + SkipCity1 ~~~ GenUsers + StoreEnriched ~~~ GenUsers + + GenUsers --> LoadUserData["CuratedJsonDataService: LoadPersonas / LoadForenamesByCountry / LoadSurnamesByCountry
each memoized after first parse; MockCuratedDataService (--mocked) returns a fixed in-memory dataset instead, skipping these JSON files"] + + LoadUserData --> UserLoop{For each EnrichedCity} + UserLoop --> SampleName["SampleName(forenames_by_country, surnames_by_country, location.iso3166_1)"] + SampleName --> NamesAvail{Names available for country?} + NamesAvail -- no --> SkipCity2[Log warning, skip city] --> UserLoop + NamesAvail -- yes --> SamplePersona["Sample UserPersona (uniform random)"] + SamplePersona --> UserGenMode{Generator mode} + + UserGenMode -- MockGenerator --> UserMockGen[DeterministicHash and Format] + UserGenMode -- LlamaGenerator --> UserLlamaLoad["prompt_directory_.Load(USER_GENERATION) → USER_GENERATION.md in --prompt-dir"] + UserLlamaLoad --> UserLlamaRetry["Infer(...) with kUserJsonGrammar → ValidateUserJson
retry up to 3 attempts, appending validation error to the retry prompt on failure"] + UserGenMode -- OpenAIGenerator --> UserOAILoad["prompt_directory_.Load(USER_GENERATION) — same prompt file as LlamaGenerator, content is backend-agnostic"] + UserOAILoad --> UserOAIRetry["CallChatCompletionsApi(...) with kUserJsonSchema → ValidateUserJson
retry up to 2 attempts — Structured Outputs (json_schema, strict) already guarantees schema shape, so retries only cover transient failures / placeholder text"] + + UserMockGen --> UserGenDone{Generation successful?} + UserLlamaRetry --> UserGenDone + UserOAIRetry --> UserGenDone + + UserGenDone -- yes --> UserPostProc["GetRandomCoordsWithinRange (5km jitter), BuildEmail, GenerateDateOfBirth
email and date_of_birth are generated programmatically, never LLM-authored; generation + coordinate jitter run inside one try/catch per record"] + UserGenDone -- no --> UserGenFail["Log warning: generation failed, skipping"] --> UserLoop + + UserPostProc --> UserExport["SqliteExportService.ProcessRecord(UserRecord)
reuse cached city_id or insert City and cache its id; insert User + UserAddress (FK city_id)"] + UserExport --> UserExportFail{Exception during insert?} + UserExportFail -- yes --> UserExportWarn["Log warning: SQLite export failed"] --> UserLoop + UserExportFail -- no --> UserLoop + UserLoop -- done --> GenBrew[Orchestrator: GenerateBreweries enriched_cities] + SkipCity2 ~~~ GenBrew + UserGenFail ~~~ GenBrew + UserExportWarn ~~~ GenBrew + UserExportFail ~~~ GenBrew + + GenBrew --> BrewLoop{For each EnrichedCity} + BrewLoop --> BrewGenMode{Generator mode} + + BrewGenMode -- MockGenerator --> BrewMockGen[DeterministicHash and Format] + BrewGenMode -- LlamaGenerator --> BrewLlamaLoad["PrepareRegionContext, prompt_directory_.Load(BREWERY_GENERATION) → BREWERY_GENERATION.md"] + BrewLlamaLoad --> BrewLlamaRetry["Infer(...) with kBreweryJsonGrammar → ValidateBreweryJson
retry up to 3 attempts; max_tokens is fixed across retries, not raised on truncation"] + BrewGenMode -- OpenAIGenerator --> BrewOAILoad["PrepareRegionContext, prompt_directory_.Load(BREWERY_GENERATION) — same prompt file as LlamaGenerator"] + BrewOAILoad --> BrewOAIRetry["CallChatCompletionsApi(...) with kBreweryJsonSchema → ValidateBreweryJson
retry up to 2 attempts — Structured Outputs already guarantees schema shape"] + + BrewMockGen --> BrewGenDone{Generation successful?} + BrewLlamaRetry --> BrewGenDone + BrewOAIRetry --> BrewGenDone + + BrewGenDone -- yes --> BrewPostProc["GetRandomCoordsWithinRange (5km jitter)
generation + jitter run inside one try/catch per record"] + BrewGenDone -- no --> BrewGenFail["Log warning: generation failed, skipping"] --> BrewLoop + + BrewPostProc --> BrewGeocode["IAddressService.ReverseGeocode(longitude, latitude)
NominatimAddressService hits the live Nominatim API (1 req/sec) and returns nullopt on failure; MockAddressService (--mocked) returns a fixed placeholder address instead"] + BrewGeocode --> BrewGeocodeFailed{Geocode resolved?} + BrewGeocodeFailed -- no --> BrewPlaceholder["Log warning: could not reverse geocode; use placeholder address_line1, empty postal_code"] --> BrewExport + BrewGeocodeFailed -- yes --> BrewExport["SqliteExportService.ProcessRecord(BreweryRecord)
reuse cached city_id or insert City and cache its id; insert Brewery + BreweryAddress (FK city_id)"] + BrewExport --> BrewExportFail{Exception during insert?} + BrewExportFail -- yes --> BrewExportWarn["Log warning: SQLite export failed (data loss prevented per-record; pipeline continues)"] --> BrewLoop + BrewExportFail -- no --> BrewLoop + BrewLoop -- done --> Finalize["SqliteExportService.Finalize()
commits transaction, closes database connection"] + BrewGenFail ~~~ Finalize + BrewExportWarn ~~~ Finalize + BrewExportFail ~~~ Finalize + + Finalize --> CloseLog["main.cc: close log_channel, join log dispatcher thread"] + CloseLog --> Stop2(["return 0 / stop"]) diff --git a/docs/pipeline/diagrams/current/activity.puml b/docs/pipeline/diagrams/current/activity.puml deleted file mode 100644 index 998a742b..00000000 --- a/docs/pipeline/diagrams/current/activity.puml +++ /dev/null @@ -1,293 +0,0 @@ -@startuml -skinparam style strictuml -skinparam defaultFontName "DM Sans" -skinparam defaultFontSize 14 -skinparam titleFontName "Volkhov" -skinparam titleFontSize 20 -skinparam backgroundColor #FAFCF9 -skinparam defaultFontColor #28342A -skinparam titleFontColor #28342A -skinparam ArrowColor #628A5B -skinparam NoteBackgroundColor #EAF0E8 -skinparam NoteBorderColor #547461 -skinparam ActivityBackgroundColor #FAFCF9 -skinparam ActivityBorderColor #547461 -skinparam ActivityDiamondBackgroundColor #FAFCF9 -skinparam ActivityDiamondBorderColor #628A5B -skinparam ActivityBarColor #628A5B -skinparam SwimlaneBorderColor #547461 -skinparam SwimlaneBorderThickness 0.3 - -title The Biergarten Data Pipeline (Current — Synchronous Data Path) - -|#F2F6F0|main.cc| -start -:Create BoundedChannel log_channel; -:Spawn log dispatcher thread\n(LogDispatcher::Run()); -note right - The only concurrency in the current pipeline: - a dedicated thread drains log_channel and - forwards entries to spdlog for the entire run. - Joined during shutdown after log_channel closes. -end note -:ParseArguments(argc, argv); -if (Are arguments valid?) then (no) - :Log error / usage info; - stop -else (yes) -endif - -:Init OpenSSL global state & LlamaBackendState; -:di::make_injector(...); -note right - Binds ILogger, IEnrichmentService, DataGenerator, - IExportService, IPromptFormatter, WebClient based - on --mocked vs. model-backed mode. -end note -:injector.create>(); -:BiergartenPipelineOrchestrator::Run(); -note right - Run() itself is synchronous — sampling, enrichment, - generation, and export all happen on this thread. -end note - -|#EAF0E8|BiergartenPipelineOrchestrator| -:Initialize SQLite export; - -|#E0EAE0|SqliteExportService| -:GetUtcTimestamp() from SystemDateTimeProvider; -:Initialize(); -note right - Builds a fresh biergarten_seed_.sqlite filename - Appends a numeric suffix if the timestamp already exists - Opens DB Connection - Executes Schema DDL - Begins Transaction -end note - -|#EAF0E8|BiergartenPipelineOrchestrator| -:QueryLocations(); - -|#E2EBDC|JsonLoader| -:JsonLoader::LoadLocations("locations.json"); -:std::ranges::sample(all_locations, --location-count); -note right - --location-count defaults to 10. -end note - -|#EAF0E8|BiergartenPipelineOrchestrator| -while (For each sampled Location?) is (Remaining cities) - |#DCE8D8|IEnrichmentService| - :GetLocationContext(loc); - note right - WikipediaEnrichmentService fetches a generic - "brewing" extract and a "beer in {country}" extract - (not a city/region-specific page). MockEnrichmentService - (--mocked) returns an empty string instead. - end note - |#EAF0E8|BiergartenPipelineOrchestrator| - if (Lookup failed?) then (yes) - :Log warning, skip city; - else (no) - :Store EnrichedCity{Location, region_context}; - endif -endwhile (Done) - -|#EAF0E8|BiergartenPipelineOrchestrator| -:GenerateUsers(enriched_cities); - -|#E2EBDC|JsonLoader| -:LoadPersonas("personas.json"); -:LoadForenamesByCountry("forenames-by-country.json"); -:LoadSurnamesByCountry("surnames-by-country.json"); -note right - Each call is cached after its first parse. - MockCuratedDataService (--mocked) returns a - fixed in-memory dataset instead and skips - these three JSON files entirely. -end note - -|#EAF0E8|BiergartenPipelineOrchestrator| -while (For each EnrichedCity?) is (Remaining cities) - :SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1); - if (Names available for country?) then (no) - :Log warning, skip city; - else (yes) - :Sample UserPersona (uniform random); - - |#E5EDE1|DataGenerator| - if (Generator Mode) then (MockGenerator) - :DeterministicHash & Format; - elseif (Generator Mode) then (LlamaGenerator) - :prompt_directory_->Load("USER_GENERATION"); - note right - Resolves to USER_GENERATION.md inside --prompt-dir. - end note - repeat - :Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar); - :ValidateUserJson(raw, user); - if (Is JSON Valid?) then (yes) - break - else (no) - :Attempt++; - :Append validation error to retry prompt; - endif - repeat while (Attempt < 3?) is (yes) - else (OpenAIGenerator) - :prompt_directory_->Load("USER_GENERATION"); - note right - Same prompt file as LlamaGenerator - content - is backend-agnostic. - end note - repeat - :CallChatCompletionsApi(system_prompt, user_prompt,\nkUserJsonSchema, max_tokens); - :ValidateUserJson(raw, user); - if (Is JSON Valid?) then (yes) - break - else (no) - :Attempt++; - :Append validation error to retry prompt; - endif - repeat while (Attempt < 2?) is (yes) - note right - Structured Outputs (response_format: - json_schema, strict: true) already - guarantees schema shape, so only 2 - attempts are needed - retries here cover - transient failures / placeholder text, - not malformed JSON. - end note - endif - - |#EAF0E8|BiergartenPipelineOrchestrator| - if (Generation successful?) then (yes) - :GetRandomCoordsWithinRange(city_centre, 5km, rng); - :BuildEmail(name, used_email_local_parts); - :GenerateDateOfBirth(rng); - note right - email and date_of_birth are generated - programmatically, never LLM-authored. - Generation + coordinate jitter run inside - one try/catch per record. UserRecord.address - now carries {city, longitude, latitude}. - end note - - |#E0EAE0|SqliteExportService| - :ProcessRecord(UserRecord); - if (Location in cache?) then (yes) - :Reuse location_id; - else (no) - :Insert Location & Cache ID; - endif - :Insert User (FK: location_id); - - if (Exception caught during insert?) then (yes) - |#EAF0E8|BiergartenPipelineOrchestrator| - :Log warning "SQLite export failed"; - else (no) - endif - else (no) - :Log warning "Generation failed, skipping..."; - endif - endif -endwhile (Done) - -|#EAF0E8|BiergartenPipelineOrchestrator| -:GenerateBreweries(enriched_cities); - -|#E5EDE1|DataGenerator| -while (For each EnrichedCity?) is (Remaining cities) - if (Generator Mode) then (MockGenerator) - :DeterministicHash & Format; - elseif (Generator Mode) then (LlamaGenerator) - :PrepareRegionContext; - :prompt_directory_->Load("BREWERY_GENERATION"); - note right - Resolves to BREWERY_GENERATION.md inside --prompt-dir. - end note - repeat - :Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar); - :ValidateBreweryJson(raw, brewery); - if (Is JSON Valid?) then (yes) - break - else (no) - :Attempt++; - :Append validation error to retry prompt; - endif - repeat while (Attempt < 3?) is (yes) - note right - max_tokens is fixed across retries — - it is not raised on truncation. - end note - else (OpenAIGenerator) - :PrepareRegionContext; - :prompt_directory_->Load("BREWERY_GENERATION"); - note right - Same prompt file as LlamaGenerator - content - is backend-agnostic. - end note - repeat - :CallChatCompletionsApi(system_prompt, user_prompt,\nkBreweryJsonSchema, max_tokens); - :ValidateBreweryJson(raw, brewery); - if (Is JSON Valid?) then (yes) - break - else (no) - :Attempt++; - :Append validation error to retry prompt; - endif - repeat while (Attempt < 2?) is (yes) - note right - Structured Outputs already guarantees schema - shape, so only 2 attempts are needed - retries - here cover transient failures / placeholder - text, not malformed JSON. - end note - endif - - |#EAF0E8|BiergartenPipelineOrchestrator| - if (Generation successful?) then (yes) - :GetRandomCoordsWithinRange(city_centre, 5km, rng); - note right - Generation + coordinate jitter run inside one - try/catch per record. BreweryRecord.address - now carries {city, longitude, latitude}. - end note - - |#E0EAE0|SqliteExportService| - :ProcessRecord(GeneratedBrewery); - if (Location in cache?) then (yes) - :Reuse location_id; - else (no) - :Insert Location & Cache ID; - endif - :Insert Brewery (FK: location_id); - - if (Exception caught during insert?) then (yes) - |#EAF0E8|BiergartenPipelineOrchestrator| - :Log warning "SQLite export failed"; - note right - Data loss is prevented per-record. - The pipeline continues running. - end note - else (no) - endif - else (no) - :Log warning "Generation failed, skipping..."; - endif - |#E5EDE1|DataGenerator| -endwhile (Done) - -|#E0EAE0|SqliteExportService| -:Finalize(); -note right - Commits Transaction - Closes Database Connection -end note - -|#F2F6F0|main.cc| -:Close log_channel; -:Join log dispatcher thread; -:Return 0; -stop - -@enduml diff --git a/docs/pipeline/diagrams/current/class.md b/docs/pipeline/diagrams/current/class.md new file mode 100644 index 00000000..aa3251f2 --- /dev/null +++ b/docs/pipeline/diagrams/current/class.md @@ -0,0 +1,5 @@ +# The Biergarten Data Pipeline — Class Diagram (Current) + +```mermaid +--8<-- "pipeline/diagrams/current/class.mmd" +``` diff --git a/docs/pipeline/diagrams/current/class.mmd b/docs/pipeline/diagrams/current/class.mmd new file mode 100644 index 00000000..cee4ed4d --- /dev/null +++ b/docs/pipeline/diagrams/current/class.mmd @@ -0,0 +1,285 @@ +classDiagram + class BiergartenPipelineOrchestrator { + -shared_ptr~ILogger~ logger_ + -unique_ptr~IEnrichmentService~ context_service_ + -unique_ptr~DataGenerator~ generator_ + -unique_ptr~IExportService~ exporter_ + -unique_ptr~ICuratedDataService~ curated_data_service_ + -unique_ptr~IAddressService~ address_service_ + -ApplicationOptions application_options_ + -vector~BreweryRecord~ generated_breweries_ + -vector~UserRecord~ generated_users_ + +Run() bool + -QueryLocations() vector~City~ + -GenerateBreweries(cities) void + -GenerateUsers(cities) void + -LogResults() void + } + note for BiergartenPipelineOrchestrator "GenerateBreweries/GenerateUsers jitter each address via GetRandomCoordsWithinRange(), uniformly within 5km of the city centre. GenerateBreweries additionally reverse-geocodes the jittered coordinates via address_service_." + + class LogLevel { + <> + Debug + Info + Warn + Error + } + + class PipelinePhase { + <> + Startup + Enrichment + UserGeneration + BreweryAndBeerGeneration + CheckinGeneration + RatingGeneration + FollowGeneration + Teardown + } + + class LogDTO { + <> + +LogLevel level + +PipelinePhase phase + +string message + } + + class LogEntry { + <> + +time_point timestamp + +source_location origin + +thread::id thread_id + +LogLevel level + +PipelinePhase phase + +string message + } + + class ILogger { + <> + +Log(payload) void + -DoLog(entry)* void + } + + class LogProducer { + -BoundedChannel~LogEntry~ channel_ + -DoLog(entry) void + } + + class LogDispatcher { + -BoundedChannel~LogEntry~ channel_ + +Run() void + -ToSpdlogLevel(level) level_enum + } + + class IEnrichmentService { + <> + +GetLocationContext(loc) string + } + + class MockEnrichmentService { + +GetLocationContext(loc) string + } + + class WikipediaEnrichmentService { + -unique_ptr~WebClient~ client_ + -unordered_map~string, string~ extract_cache_ + +GetLocationContext(loc) string + -FetchExtract(query) string + } + + class WebClient { + <> + +Get(url) string + +UrlEncode(value) string + } + + class HttpWebClient { + +Get(url) string + +UrlEncode(value) string + } + + class DataGenerator { + <> + +GenerateBrewery(city) BreweryResult + +GenerateUser(city, persona, name) UserResult + } + + class MockGenerator { + +GenerateBrewery(city) BreweryResult + +GenerateUser(city, persona, name) UserResult + -DeterministicHash(location) size_t + -DeterministicHash(location, persona, name) size_t + } + + class LlamaGenerator { + -ModelHandle model_ + -ContextHandle context_ + -unique_ptr~IPromptFormatter~ prompt_formatter_ + -unique_ptr~IPromptDirectory~ prompt_directory_ + -mt19937 rng_ + +GenerateBrewery(...) BreweryResult + +GenerateUser(...) UserResult + -Load(model_path) void + -Infer(...) string + -InferFormatted(...) string + } + + class OpenAIGenerator { + -string api_key_ + -string model_ + -shared_ptr~ILogger~ logger_ + -unique_ptr~IPromptDirectory~ prompt_directory_ + -unique_ptr~WebClient~ web_client_ + +GenerateBrewery(city) BreweryResult + +GenerateUser(city, persona, name) UserResult + -CallChatCompletionsApi(...) string + } + note for OpenAIGenerator "Chat Completions with Structured Outputs (json_schema, strict) guarantees schema-valid JSON, replacing llama.cpp's GBNF grammar — no IPromptFormatter needed" + + class IPromptFormatter { + <> + +Format(system_prompt, user_prompt) string + } + + class Gemma4JinjaPromptFormatter { + +Format(system_prompt, user_prompt) string + } + + class IPromptDirectory { + <> + +Load(key) string + } + + class PromptDirectory { + -path prompt_dir_ + -unordered_map~string, string~ cache_ + +Load(key) string + } + + class ICuratedDataService { + <> + +LoadLocations() vector~Location~ + +LoadPersonas() vector~UserPersona~ + +LoadForenamesByCountry() unordered_map~string, forename_list~ + +LoadSurnamesByCountry() unordered_map~string, surname_list~ + } + + class CuratedJsonDataService { + -CuratedDataFilePaths filepaths_ + -cache cache_ + +LoadLocations() vector~Location~ + +LoadPersonas() vector~UserPersona~ + +LoadForenamesByCountry() unordered_map~string, forename_list~ + +LoadSurnamesByCountry() unordered_map~string, surname_list~ + } + note for CuratedJsonDataService "Takes a CuratedDataFilePaths DTO (locations/personas/forenames/surnames paths) once at construction, not per Load* call. Each Load* method memoizes its result in cache_ after its first call." + + class MockCuratedDataService { + -vector~Location~ locations_ + -vector~UserPersona~ personas_ + -unordered_map~string, forename_list~ forenames_by_country_ + -unordered_map~string, surname_list~ surnames_by_country_ + +LoadLocations() vector~Location~ + +LoadPersonas() vector~UserPersona~ + +LoadForenamesByCountry() unordered_map~string, forename_list~ + +LoadSurnamesByCountry() unordered_map~string, surname_list~ + } + note for MockCuratedDataService "Fixed 4-location/3-persona dataset (US, DE, FR, BE) for --mocked runs" + + class IAddressService { + <> + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + + class NominatimAddressService { + -unique_ptr~WebClient~ client_ + -shared_ptr~ILogger~ logger_ + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + note for NominatimAddressService "Queries the public Nominatim API; sleeps 1s per request per its usage policy; returns nullopt on failure or when no usable street address is found" + + class MockAddressService { + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + note for MockAddressService "Returns a fixed placeholder address ('123 Mock Street', '00000') for --mocked runs, no network call" + + class IExportService { + <> + +Initialize() void + +ProcessRecord(brewery) uint64_t + +ProcessRecord(user) uint64_t + +Finalize() void + } + + class SqliteExportService { + -unique_ptr~IDateTimeProvider~ date_time_provider_ + -string run_timestamp_utc_ + -path database_path_ + -SqliteDatabaseHandle db_handle_ + -SqliteStatementHandle insert_city_stmt_ + -SqliteStatementHandle insert_brewery_stmt_ + -SqliteStatementHandle insert_brewery_address_stmt_ + -SqliteStatementHandle insert_user_stmt_ + -SqliteStatementHandle insert_user_address_stmt_ + -bool transaction_open_ + -unordered_map~string, sqlite3_int64~ city_cache_ + +Initialize() void + +ProcessRecord(brewery) uint64_t + +ProcessRecord(user) uint64_t + +Finalize() void + -InitializeSchema() void + -ResolveCityId(city) sqlite3_int64 + } + + class IDateTimeProvider { + <> + +GetUtcTimestamp() string + } + + class SystemDateTimeProvider { + +GetUtcTimestamp() string + } + + BiergartenPipelineOrchestrator *-- ILogger : owns + BiergartenPipelineOrchestrator *-- IEnrichmentService : owns + BiergartenPipelineOrchestrator *-- DataGenerator : owns + BiergartenPipelineOrchestrator *-- IExportService : owns + BiergartenPipelineOrchestrator *-- ICuratedDataService : owns + BiergartenPipelineOrchestrator *-- IAddressService : owns + + LogEntry *-- LogLevel + LogEntry *-- PipelinePhase + LogDTO *-- LogLevel + LogDTO *-- PipelinePhase + ILogger <|.. LogProducer : implements + LogProducer ..> LogEntry : emits + LogDispatcher ..> LogEntry : consumes + + IEnrichmentService <|.. WikipediaEnrichmentService : implements + IEnrichmentService <|.. MockEnrichmentService : implements + WikipediaEnrichmentService *-- WebClient : owns + + WebClient <|.. HttpWebClient : implements + + DataGenerator <|.. MockGenerator : implements + DataGenerator <|.. LlamaGenerator : implements + DataGenerator <|.. OpenAIGenerator : implements + + LlamaGenerator *-- IPromptFormatter : uses + LlamaGenerator *-- IPromptDirectory : uses + + OpenAIGenerator *-- IPromptDirectory : uses + OpenAIGenerator *-- WebClient : uses + + IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements + IPromptDirectory <|.. PromptDirectory : implements + + ICuratedDataService <|.. CuratedJsonDataService : implements + ICuratedDataService <|.. MockCuratedDataService : implements + + IAddressService <|.. NominatimAddressService : implements + IAddressService <|.. MockAddressService : implements + + IExportService <|.. SqliteExportService : implements + SqliteExportService *-- IDateTimeProvider : owns + IDateTimeProvider <|.. SystemDateTimeProvider : implements diff --git a/docs/pipeline/diagrams/current/class.puml b/docs/pipeline/diagrams/current/class.puml deleted file mode 100644 index 935274a6..00000000 --- a/docs/pipeline/diagrams/current/class.puml +++ /dev/null @@ -1,301 +0,0 @@ -@startuml -skinparam style strictuml -skinparam defaultFontName "DM Sans" -skinparam defaultFontSize 14 -skinparam titleFontName "Volkhov" -skinparam titleFontSize 20 -skinparam backgroundColor #FAFCF9 -skinparam defaultFontColor #28342A -skinparam titleFontColor #28342A -skinparam ArrowColor #628A5B - -skinparam class { - BackgroundColor #FAFCF9 - HeaderBackgroundColor #EAF0E8 - BorderColor #547461 - ArrowColor #628A5B - FontColor #28342A -} - -skinparam note { - BackgroundColor #EAF0E8 - BorderColor #547461 - FontColor #28342A -} - -title The Biergarten Data Pipeline - Class Diagram - -class BiergartenPipelineOrchestrator { - - logger_ : std::shared_ptr - - context_service_ : std::unique_ptr - - generator_ : std::unique_ptr - - exporter_ : std::unique_ptr - - curated_data_service_ : std::unique_ptr - - application_options_ : ApplicationOptions - - generated_breweries_ : std::vector - - generated_users_ : std::vector - + Run() : bool - - QueryLocations() : std::vector - - GenerateBreweries(cities : std::span) : void - - GenerateUsers(cities : std::span) : void - - LogResults() const : void -} - -note right of BiergartenPipelineOrchestrator - GenerateBreweries and GenerateUsers both call the - free function GetRandomCoordsWithinRange() - (biergarten_pipeline_orchestrator/coords.h) to jitter - a coordinate pair uniformly within 5km of the city - centre for each generated record's address. -end note - -class LogLevel <> { - Debug - Info - Warn - Error -} - -class PipelinePhase <> { - Startup - Enrichment - UserGeneration - BreweryAndBeerGeneration - CheckinGeneration - RatingGeneration - FollowGeneration - Teardown -} - -struct LogDTO { - + level : LogLevel - + phase : PipelinePhase - + message : std::string -} - -struct LogEntry { - + timestamp : std::chrono::system_clock::time_point - + origin : std::source_location - + thread_id : std::thread::id - + level : LogLevel - + phase : PipelinePhase - + message : std::string -} - -interface ILogger <> { - + Log(payload : LogDTO) : void - - {abstract} DoLog(entry : LogEntry) : void -} - -class LogProducer { - - channel_ : BoundedChannel& - - DoLog(entry : LogEntry) : void -} - -class LogDispatcher { - - channel_ : BoundedChannel& - + Run() : void - - ToSpdlogLevel(level) : spdlog::level::level_enum -} - -interface IEnrichmentService <> { - + GetLocationContext(loc : const Location&) : std::string -} - -class MockEnrichmentService { - + GetLocationContext(loc : const Location&) : std::string -} - -class WikipediaEnrichmentService { - - client_ : std::unique_ptr - - extract_cache_ : std::unordered_map - + GetLocationContext(loc : const Location&) : std::string - - FetchExtract(query : std::string_view) : std::string -} - -interface WebClient <> { - + Get(url : const std::string&) : std::string - + UrlEncode(value : const std::string&) : std::string -} - -class HttpWebClient { - + Get(url : const std::string&) : std::string - + UrlEncode(value : const std::string&) : std::string -} - -interface DataGenerator <> { - + GenerateBrewery(city : const EnrichedCity&) : BreweryResult - + GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult -} - -class MockGenerator { - + GenerateBrewery(city : const EnrichedCity&) : BreweryResult - + GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult - - DeterministicHash(location : const Location&) : size_t - - DeterministicHash(location : const Location&, persona : const UserPersona&, name : const Name&) : size_t -} - -class LlamaGenerator { - - model_ : ModelHandle - - context_ : ContextHandle - - prompt_formatter_ : std::unique_ptr - - prompt_directory_ : std::unique_ptr - - rng_ : std::mt19937 - + GenerateBrewery(...) : BreweryResult - + GenerateUser(...) : UserResult - - Load(model_path : const std::string&) : void - - Infer(...) : std::string - - InferFormatted(...) : std::string -} - -class OpenAIGenerator { - - api_key_ : std::string - - model_ : std::string - - logger_ : std::shared_ptr - - prompt_directory_ : std::unique_ptr - - web_client_ : std::unique_ptr - + GenerateBrewery(city : const EnrichedCity&) : BreweryResult - + GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult - - CallChatCompletionsApi(...) : std::string -} - -note right of OpenAIGenerator - Chat Completions with Structured Outputs - (response_format: json_schema, strict: true) - guarantees schema-valid JSON, replacing - llama.cpp's GBNF grammar constraint - - no IPromptFormatter needed. -end note - -interface IPromptFormatter <> { - + Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string -} - -class Gemma4JinjaPromptFormatter { - + Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string -} - -interface IPromptDirectory <> { - + Load(key : std::string_view) : std::string -} - -class PromptDirectory { - - prompt_dir_ : std::filesystem::path - - cache_ : std::unordered_map - + Load(key : std::string_view) : std::string -} - -interface ICuratedDataService <> { - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& -} - -class JsonLoader { - - cache_ : cache - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& -} - -note right of JsonLoader - Each Load* method memoizes its result in - cache_ on first call; later calls on the - same instance return the cached value - without re-parsing. -end note - -class MockCuratedDataService { - - locations_ : std::vector - - personas_ : std::vector - - forenames_by_country_ : std::unordered_map - - surnames_by_country_ : std::unordered_map - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map& -} - -note right of MockCuratedDataService - Fixed 4-location / 3-persona dataset - (US, DE, FR, BE) used in --mocked runs; - filepath arguments are ignored. -end note - -interface IExportService <> { - + Initialize() : void - + ProcessRecord(brewery : const BreweryRecord&) : uint64_t - + ProcessRecord(user : const UserRecord&) : uint64_t - + Finalize() : void -} - -class SqliteExportService { - - date_time_provider_ : std::unique_ptr - - run_timestamp_utc_ : std::string - - database_path_ : std::filesystem::path - - db_handle_ : SqliteDatabaseHandle - - insert_location_stmt_ : SqliteStatementHandle - - insert_brewery_stmt_ : SqliteStatementHandle - - insert_user_stmt_ : SqliteStatementHandle - - transaction_open_ : bool - - location_cache_ : std::unordered_map - + Initialize() : void - + ProcessRecord(brewery : const BreweryRecord&) : uint64_t - + ProcessRecord(user : const UserRecord&) : uint64_t - + Finalize() : void - - InitializeSchema() : void - - ResolveLocationId(location : const Location&) : sqlite3_int64 -} - -interface IDateTimeProvider <> { - + GetUtcTimestamp() : std::string -} - -class SystemDateTimeProvider { - + GetUtcTimestamp() : std::string -} - -' Structural Relationships / Dependency Injection -BiergartenPipelineOrchestrator *-- ILogger : owns -BiergartenPipelineOrchestrator *-- IEnrichmentService : owns -BiergartenPipelineOrchestrator *-- DataGenerator : owns -BiergartenPipelineOrchestrator *-- IExportService : owns -BiergartenPipelineOrchestrator *-- ICuratedDataService : owns - -LogEntry *-- LogLevel -LogEntry *-- PipelinePhase -LogDTO *-- LogLevel -LogDTO *-- PipelinePhase -ILogger <|.. LogProducer : implements -LogProducer ..> LogEntry : emits -LogDispatcher ..> LogEntry : consumes - -IEnrichmentService <|.. WikipediaEnrichmentService : implements -IEnrichmentService <|.. MockEnrichmentService : implements -WikipediaEnrichmentService *-- WebClient : owns - -WebClient <|.. HttpWebClient : implements - -DataGenerator <|.. MockGenerator : implements -DataGenerator <|.. LlamaGenerator : implements -DataGenerator <|.. OpenAIGenerator : implements - -LlamaGenerator *-- IPromptFormatter : uses -LlamaGenerator *-- IPromptDirectory : uses - -OpenAIGenerator *-- IPromptDirectory : uses -OpenAIGenerator *-- WebClient : uses - -IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements -IPromptDirectory <|.. PromptDirectory : implements - -ICuratedDataService <|.. JsonLoader : implements -ICuratedDataService <|.. MockCuratedDataService : implements - -IExportService <|.. SqliteExportService : implements -SqliteExportService *-- IDateTimeProvider : owns -IDateTimeProvider <|.. SystemDateTimeProvider : implements - -@enduml diff --git a/docs/pipeline/diagrams/current/output/activity.svg b/docs/pipeline/diagrams/current/output/activity.svg deleted file mode 100644 index 858ca813..00000000 --- a/docs/pipeline/diagrams/current/output/activity.svg +++ /dev/null @@ -1 +0,0 @@ -The Biergarten Data Pipeline (Current — Synchronous Data Path)The Biergarten Data Pipeline (Current — Synchronous Data Path)Create BoundedChannel<LogEntry> log_channelThe only concurrency in the current pipeline:a dedicated thread drains log_channel andforwards entries to spdlog for the entire run.Joined during shutdown after log_channel closes.Spawn log dispatcher thread(LogDispatcher::Run())ParseArguments(argc, argv)Log error / usage infonoAre arguments valid?yesInit OpenSSL global state & LlamaBackendStateBinds ILogger, IEnrichmentService, DataGenerator,IExportService, IPromptFormatter, WebClient basedon --mocked vs. model-backed mode.di::make_injector(...)injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>()Run() itself is synchronous — sampling, enrichment,generation, and export all happen on this thread.BiergartenPipelineOrchestrator::Run()Close log_channelJoin log dispatcher threadReturn 0Initialize SQLite exportQueryLocations()Lookup failed?yesnoLog warning, skip cityStore EnrichedCity{Location, region_context}Remaining citiesFor each sampled Location?DoneGenerateUsers(enriched_cities)SampleName(forenames_by_country,surnames_by_country, location.iso3166_1)Names available for country?noyesLog warning, skip citySample UserPersona (uniform random)Generation successful?yesnoGetRandomCoordsWithinRange(city_centre, 5km, rng)BuildEmail(name, used_email_local_parts)email and date_of_birth are generatedprogrammatically, never LLM-authored.Generation + coordinate jitter run insideone try/catch per record. UserRecord.addressnow carries {city, longitude, latitude}.GenerateDateOfBirth(rng)Log warning "SQLite export failed"Log warning "Generation failed, skipping..."Remaining citiesFor each EnrichedCity?DoneGenerateBreweries(enriched_cities)Generation successful?yesnoGeneration + coordinate jitter run inside onetry/catch per record. BreweryRecord.addressnow carries {city, longitude, latitude}.GetRandomCoordsWithinRange(city_centre, 5km, rng)Data loss is prevented per-record.The pipeline continues running.Log warning "SQLite export failed"Log warning "Generation failed, skipping..."GetUtcTimestamp() from SystemDateTimeProviderBuilds a fresh biergarten_seed_<UTC datetime>.sqlite filenameAppends a numeric suffix if the timestamp already existsOpens DB ConnectionExecutes Schema DDLBegins TransactionInitialize()ProcessRecord(UserRecord)Location in cache?yesnoReuse location_idInsert Location & Cache IDInsert User (FK: location_id)yesException caught during insert?noProcessRecord(GeneratedBrewery)Location in cache?yesnoReuse location_idInsert Location & Cache IDInsert Brewery (FK: location_id)yesException caught during insert?noCommits TransactionCloses Database ConnectionFinalize()JsonLoader::LoadLocations("locations.json")--location-count defaults to 10.std::ranges::sample(all_locations, --location-count)LoadPersonas("personas.json")LoadForenamesByCountry("forenames-by-country.json")Each call is cached after its first parse.MockCuratedDataService (--mocked) returns afixed in-memory dataset instead and skipsthese three JSON files entirely.LoadSurnamesByCountry("surnames-by-country.json")WikipediaEnrichmentService fetches a generic"brewing" extract and a "beer in {country}" extract(not a city/region-specific page). MockEnrichmentService(--mocked) returns an empty string instead.GetLocationContext(loc)MockGeneratorGenerator ModeDeterministicHash & FormatLlamaGeneratorGenerator ModeOpenAIGeneratorResolves to USER_GENERATION.md inside --prompt-dir.prompt_directory_->Load("USER_GENERATION")Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar)ValidateUserJson(raw, user)Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 3?yesSame prompt file as LlamaGenerator - contentis backend-agnostic.prompt_directory_->Load("USER_GENERATION")CallChatCompletionsApi(system_prompt, user_prompt,kUserJsonSchema, max_tokens)ValidateUserJson(raw, user)Structured Outputs (response_format:json_schema, strict: true) alreadyguarantees schema shape, so only 2attempts are needed - retries here covertransient failures / placeholder text,not malformed JSON.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 2?yesMockGeneratorGenerator ModeDeterministicHash & FormatLlamaGeneratorGenerator ModeOpenAIGeneratorPrepareRegionContextResolves to BREWERY_GENERATION.md inside --prompt-dir.prompt_directory_->Load("BREWERY_GENERATION")Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar)ValidateBreweryJson(raw, brewery)max_tokens is fixed across retries —it is not raised on truncation.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 3?yesPrepareRegionContextSame prompt file as LlamaGenerator - contentis backend-agnostic.prompt_directory_->Load("BREWERY_GENERATION")CallChatCompletionsApi(system_prompt, user_prompt,kBreweryJsonSchema, max_tokens)ValidateBreweryJson(raw, brewery)Structured Outputs already guarantees schemashape, so only 2 attempts are needed - retrieshere cover transient failures / placeholdertext, not malformed JSON.Is JSON Valid?yesnoAttempt++Append validation error to retry promptAttempt < 2?yesRemaining citiesFor each EnrichedCity?Donemain.ccBiergartenPipelineOrchestratorSqliteExportServiceJsonLoaderIEnrichmentServiceDataGenerator \ No newline at end of file diff --git a/docs/pipeline/diagrams/current/output/class.svg b/docs/pipeline/diagrams/current/output/class.svg deleted file mode 100644 index b4751c3a..00000000 --- a/docs/pipeline/diagrams/current/output/class.svg +++ /dev/null @@ -1 +0,0 @@ -The Biergarten Data Pipeline - Class DiagramThe Biergarten Data Pipeline - Class DiagramBiergartenPipelineOrchestratorlogger_ : std::shared_ptr<ILogger>context_service_ : std::unique_ptr<IEnrichmentService>generator_ : std::unique_ptr<DataGenerator>exporter_ : std::unique_ptr<IExportService>curated_data_service_ : std::unique_ptr<ICuratedDataService>application_options_ : ApplicationOptionsgenerated_breweries_ : std::vector<BreweryRecord>generated_users_ : std::vector<UserRecord>Run() : boolQueryLocations() : std::vector<City>GenerateBreweries(cities : std::span<const EnrichedCity>) : voidGenerateUsers(cities : std::span<const EnrichedCity>) : voidLogResults() const : voidGenerateBreweries and GenerateUsers both call thefree function GetRandomCoordsWithinRange()(biergarten_pipeline_orchestrator/coords.h) to jittera coordinate pair uniformly within 5km of the citycentre for each generated record's address.«enumeration»LogLevelDebugInfoWarnError«enumeration»PipelinePhaseStartupEnrichmentUserGenerationBreweryAndBeerGenerationCheckinGenerationRatingGenerationFollowGenerationTeardownLogDTOlevel : LogLevelphase : PipelinePhasemessage : std::stringLogEntrytimestamp : std::chrono::system_clock::time_pointorigin : std::source_locationthread_id : std::thread::idlevel : LogLevelphase : PipelinePhasemessage : std::string«interface»ILoggerLog(payload : LogDTO) : voidDoLog(entry : LogEntry) : voidLogProducerchannel_ : BoundedChannel<LogEntry>&DoLog(entry : LogEntry) : voidLogDispatcherchannel_ : BoundedChannel<LogEntry>&Run() : voidToSpdlogLevel(level) : spdlog::level::level_enum«interface»IEnrichmentServiceGetLocationContext(loc : const Location&) : std::stringMockEnrichmentServiceGetLocationContext(loc : const Location&) : std::stringWikipediaEnrichmentServiceclient_ : std::unique_ptr<WebClient>extract_cache_ : std::unordered_map<std::string, std::string>GetLocationContext(loc : const Location&) : std::stringFetchExtract(query : std::string_view) : std::string«interface»WebClientGet(url : const std::string&) : std::stringUrlEncode(value : const std::string&) : std::stringHttpWebClientGet(url : const std::string&) : std::stringUrlEncode(value : const std::string&) : std::string«interface»DataGeneratorGenerateBrewery(city : const EnrichedCity&) : BreweryResultGenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResultMockGeneratorGenerateBrewery(city : const EnrichedCity&) : BreweryResultGenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResultDeterministicHash(location : const Location&) : size_tDeterministicHash(location : const Location&, persona : const UserPersona&, name : const Name&) : size_tLlamaGeneratormodel_ : ModelHandlecontext_ : ContextHandleprompt_formatter_ : std::unique_ptr<IPromptFormatter>prompt_directory_ : std::unique_ptr<IPromptDirectory>rng_ : std::mt19937GenerateBrewery(...) : BreweryResultGenerateUser(...) : UserResultLoad(model_path : const std::string&) : voidInfer(...) : std::stringInferFormatted(...) : std::stringOpenAIGeneratorapi_key_ : std::stringmodel_ : std::stringlogger_ : std::shared_ptr<ILogger>prompt_directory_ : std::unique_ptr<IPromptDirectory>web_client_ : std::unique_ptr<WebClient>GenerateBrewery(city : const EnrichedCity&) : BreweryResultGenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResultCallChatCompletionsApi(...) : std::stringChat Completions with Structured Outputs(response_format: json_schema, strict: true)guarantees schema-valid JSON, replacingllama.cpp's GBNF grammar constraint -no IPromptFormatter needed.«interface»IPromptFormatterFormat(system_prompt : std::string_view, user_prompt : std::string_view) : std::stringGemma4JinjaPromptFormatterFormat(system_prompt : std::string_view, user_prompt : std::string_view) : std::string«interface»IPromptDirectoryLoad(key : std::string_view) : std::stringPromptDirectoryprompt_dir_ : std::filesystem::pathcache_ : std::unordered_map<std::string, std::string>Load(key : std::string_view) : std::string«interface»ICuratedDataServiceLoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&JsonLoadercache_ : cacheLoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&Each Load* method memoizes its result incache_ on first call; later calls on thesame instance return the cached valuewithout re-parsing.MockCuratedDataServicelocations_ : std::vector<Location>personas_ : std::vector<UserPersona>forenames_by_country_ : std::unordered_map<std::string, forename_list>surnames_by_country_ : std::unordered_map<std::string, surname_list>LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&Fixed 4-location / 3-persona dataset(US, DE, FR, BE) used in --mocked runs;filepath arguments are ignored.«interface»IExportServiceInitialize() : voidProcessRecord(brewery : const BreweryRecord&) : uint64_tProcessRecord(user : const UserRecord&) : uint64_tFinalize() : voidSqliteExportServicedate_time_provider_ : std::unique_ptr<IDateTimeProvider>run_timestamp_utc_ : std::stringdatabase_path_ : std::filesystem::pathdb_handle_ : SqliteDatabaseHandleinsert_location_stmt_ : SqliteStatementHandleinsert_brewery_stmt_ : SqliteStatementHandleinsert_user_stmt_ : SqliteStatementHandletransaction_open_ : boollocation_cache_ : std::unordered_map<std::string, sqlite3_int64>Initialize() : voidProcessRecord(brewery : const BreweryRecord&) : uint64_tProcessRecord(user : const UserRecord&) : uint64_tFinalize() : voidInitializeSchema() : voidResolveLocationId(location : const Location&) : sqlite3_int64«interface»IDateTimeProviderGetUtcTimestamp() : std::stringSystemDateTimeProviderGetUtcTimestamp() : std::stringownsownsownsownsownsimplementsemitsconsumesimplementsimplementsownsimplementsimplementsimplementsimplementsusesusesusesusesimplementsimplementsimplementsimplementsimplementsownsimplements \ No newline at end of file diff --git a/docs/pipeline/diagrams/planned/activity.md b/docs/pipeline/diagrams/planned/activity.md new file mode 100644 index 00000000..2485be4a --- /dev/null +++ b/docs/pipeline/diagrams/planned/activity.md @@ -0,0 +1,16 @@ +# The Biergarten Data Pipeline — Activity Diagram (Planned) + +Each generation phase runs a producer/consumer chain concurrently: an +**Orchestrator** producer feeds a bounded channel, an **LLM Worker** consumes it +and generates records (closing its output channel when the input is drained and +closed), and a **SQLite Worker** consumes that output channel and commits in +batches. The brewery phase inserts a fourth stage, a **Geocode Worker**, between +the LLM Worker and the SQLite Worker: reverse-geocoding is network-bound and +rate-limited independently of LLM inference (Nominatim's 1-request-per-second +policy), so keeping it off the LLM Worker's thread means neither stage stalls +waiting on the other's bottleneck. The Checkin and Follow phases run in parallel +with each other once the user and brewery pools are populated. + +```mermaid +--8<-- "pipeline/diagrams/planned/activity.mmd" +``` diff --git a/docs/pipeline/diagrams/planned/activity.mmd b/docs/pipeline/diagrams/planned/activity.mmd new file mode 100644 index 00000000..e19b76eb --- /dev/null +++ b/docs/pipeline/diagrams/planned/activity.mmd @@ -0,0 +1,208 @@ +flowchart TD + Start([start]) --> ParseArgs["Main: ParseArguments(argc, argv)"] + ParseArgs --> InvalidArgs{Invalid args?} + InvalidArgs -- yes --> LogErr["spdlog::error"] --> Stop1([stop]) + InvalidArgs -- no --> InitOpenSSL["Main: Init OpenSSL global state and LlamaBackendState"] + InitOpenSSL --> BuildDI["Main: Build DI injector"] + BuildDI --> InitSqlite["Main: Initialize SqliteExportService
opens SQLite connection; transactions now managed per-phase via batching"] + InitSqlite --> LogSetup["Main: create BoundedChannel log_ch, spawn Log Worker thread
drains log_ch for the pipeline lifetime; all workers emit LogEntry via PipelineLogger, never spdlog directly"] + LogSetup --> OrchRun["BiergartenPipelineOrchestrator::Run()"] + + OrchRun --> PL1["JsonLoader.LoadBeerStyles → EnrichmentService.PreWarmBeerStyleCache"] + OrchRun --> PL2["JsonLoader.LoadLocations → EnrichmentService.PreWarmLocationCache"] + OrchRun --> PL3["JsonLoader.LoadForenamesByCountry"] + OrchRun --> PL4["JsonLoader.LoadSurnamesByCountry"] + OrchRun --> PL5["JsonLoader.LoadPersonas"] + PL1 & PL2 & PL3 & PL4 & PL5 --> UserPhase["Orchestrator: RunUserPhase(sampled_locations)
each Load* call memoized after first parse; MockCuratedDataService (--mocked) returns a fixed in-memory dataset instead, skipping these JSON files"] + + UserPhase --> UCreate["Create BoundedChannels (loc_ch, exp_ch)"] + + subgraph USERGEN["Phase 0 — User Generation (concurrent)"] + direction LR + subgraph U_PROD["Orchestrator (producer)"] + UP1["Loop: send Location → loc_ch"] + UP1 --> UP2["Close loc_ch"] + end + subgraph U_LLM["LLM Worker"] + UL1{"loc_ch has items?"} + UL1 -- yes --> UL2["Receive Location"] + UL2 --> UL3["GetLocationContextFromCache(location) — guaranteed cache hit from startup"] + UL3 --> UL4["IPersonaSelectionStrategy.SelectPersona(personas_palette_) — guaranteed cache hit; returns Persona{style_affinities, abv_range, ibu_preference, checkin_weight}"] + UL4 --> UL5["SampleName(forenames_by_country, surnames_by_country, iso3166_1, rng)
deterministic, no LLM; free function; skips the city if either map has no entry for iso3166_1"] + UL5 --> UL6["GenerateUser(enriched_city, persona, sampled_name) via DataGenerator
LLM receives EnrichedCity context + persona description + sampled name"] + UL6 --> UL7["PipelineLogger.Log(Info, UserGeneration, city, user_id, llm)"] + UL7 --> UL8["Send GeneratedUser → exp_ch"] + UL8 --> UL1 + UL1 -- no --> UL9["Close exp_ch"] + end + subgraph U_SQL["SQLite Worker"] + US1["BEGIN TRANSACTION"] --> US2{"exp_ch has items?"} + US2 -- yes --> US3["Receive GeneratedUser"] + US3 --> US4["ProcessUser(user)"] + US4 --> US5["PipelineLogger.Log(Info, UserGeneration, city, user_id, sqlite)"] + US5 --> US6["Append → user_pool_"] + US6 --> US7{"Batch size reached?"} + US7 -- yes --> US8["COMMIT and BEGIN"] --> US2 + US7 -- no --> US2 + US2 -- no --> US9["COMMIT (Final)"] + end + end + + UCreate --> UP1 + UP2 -.->|loc_ch| UL1 + UL9 -.->|exp_ch| US2 + UL9 & US9 --> UserJoin["Orchestrator: Join LLM Worker, SQLite Worker"] + + UserJoin --> BreweryPhase["Orchestrator: RunBreweryPhase(sampled_locations)"] + BreweryPhase --> BCreate["Create BoundedChannels (loc_ch, geo_ch, exp_ch)"] + + subgraph BREWGEN["Phase 1a — Brewery Generation (concurrent)"] + direction LR + subgraph B_PROD["Orchestrator (producer)"] + BP1["Loop: sample User from user_pool_, pair with Location"] + BP1 --> BP2["Send BreweryTask(Location, User) → loc_ch"] + BP2 --> BP3["Close loc_ch"] + end + subgraph B_LLM["LLM Worker"] + BL1{"loc_ch has items?"} + BL1 -- yes --> BL2["Receive BreweryTask(Location, User)"] + BL2 --> BL3["GetLocationContextFromCache(task.location) — guaranteed cache hit"] + BL3 --> BL4["GenerateBrewery(enriched_city, context, task.user) via DataGenerator
KV cache stays warm; brewery linked to sampled owner_user_id"] + BL4 --> BL5["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, llm)"] + BL5 --> BL6["Send GeneratedBrewery → geo_ch"] + BL6 --> BL1 + BL1 -- no --> BL7["Close geo_ch"] + end + subgraph B_GEO["Geocode Worker"] + BG1{"geo_ch has items?"} + BG1 -- yes --> BG2["Receive GeneratedBrewery"] + BG2 --> BG3["AddressService.ReverseGeocode(brewery.longitude, brewery.latitude)
rate-limited independently of LLM throughput — NominatimAddressService sleeps 1s per request; MockAddressService (--mocked) returns a fixed placeholder, no network call"] + BG3 --> BG4{"Address resolved?"} + BG4 -- yes --> BG5["Attach resolved AddressLookupResult to brewery"] + BG4 -- no --> BG6["Log warning: could not reverse geocode; attach placeholder address_line1, empty postal_code — brewery is never dropped"] + BG5 --> BG7["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, geocode)"] + BG6 --> BG7 + BG7 --> BG8["Send GeneratedBrewery → exp_ch"] + BG8 --> BG1 + BG1 -- no --> BG9["Close exp_ch"] + end + subgraph B_SQL["SQLite Worker"] + BS1["BEGIN TRANSACTION"] --> BS2{"exp_ch has items?"} + BS2 -- yes --> BS3["Receive GeneratedBrewery"] + BS3 --> BS4["ProcessBrewery(brewery)"] + BS4 --> BS5["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, sqlite)"] + BS5 --> BS6["Append → brewery_pool_"] + BS6 --> BS7{"Batch size reached?"} + BS7 -- yes --> BS8["COMMIT and BEGIN"] --> BS2 + BS7 -- no --> BS2 + BS2 -- no --> BS9["COMMIT (Final)"] + end + end + + BCreate --> BP1 + BP3 -.->|loc_ch| BL1 + BL7 -.->|geo_ch| BG1 + BG9 -.->|exp_ch| BS2 + BL7 & BG9 & BS9 --> BrewJoin["Orchestrator: Join LLM Worker, Geocode Worker, SQLite Worker
brewery_pool_ is now fully populated"] + + BrewJoin --> BeerPhase["Orchestrator: RunBeerPhase()"] + BeerPhase --> BeCreate["Create BoundedChannels (brew_ch, exp_ch)"] + + subgraph BEERGEN["Phase 1b — Beer Generation (concurrent)"] + direction LR + subgraph BE_PROD["Orchestrator (producer)"] + BeP1["Loop: send Breweries → brew_ch"] + BeP1 --> BeP2["Close brew_ch"] + end + subgraph BE_LLM["LLM Worker"] + BeL1{"brew_ch has items?"} + BeL1 -- yes --> BeL2["Receive GeneratedBrewery"] + BeL2 --> BeL3["IBeerSelectionStrategy.SelectStyles(brewery, beer_style_palette_)"] + BeL3 --> BeL4{"For each selected BeerStyle"} + BeL4 -- remaining --> BeL5["GetStyleContextFromCache(style) — guaranteed cache hit; KV cache stays warm across all beer generations in this phase"] + BeL5 --> BeL6["GenerateBeer(brewery, style_context) via DataGenerator"] + BeL6 --> BeL7["Attach GeneratedBeer to bundle"] --> BeL4 + BeL4 -- done --> BeL8["PipelineLogger.Log(Info, BeerGeneration, city, brewery_id, llm)"] + BeL8 --> BeL9["Send BeersBundle → exp_ch"] + BeL9 --> BeL1 + BeL1 -- no --> BeL10["Close exp_ch"] + end + subgraph BE_SQL["SQLite Worker"] + BeS1["BEGIN TRANSACTION"] --> BeS2{"exp_ch has items?"} + BeS2 -- yes --> BeS3["Receive BeersBundle"] + BeS3 --> BeS4{"For each beer in bundle"} + BeS4 -- remaining --> BeS5["Set beer.brewery_id from bundle"] + BeS5 --> BeS6["ProcessBeer(beer)"] + BeS6 --> BeS7["Append → beer_pool_"] --> BeS4 + BeS4 -- done --> BeS8["PipelineLogger.Log(Info, BeerGeneration, city, brewery_id, sqlite)"] + BeS8 --> BeS9{"Batch size reached?"} + BeS9 -- yes --> BeS10["COMMIT and BEGIN"] --> BeS2 + BeS9 -- no --> BeS2 + BeS2 -- no --> BeS11["COMMIT (Final)"] + end + end + + BeCreate --> BeP1 + BeP2 -.->|brew_ch| BeL1 + BeL10 -.->|exp_ch| BeS2 + BeL10 & BeS11 --> BeerJoin["Orchestrator: Join LLM Worker, SQLite Worker
brewery_pool_ and beer_pool_ are now completely populated — Checkin and Follow phases may run in parallel"] + + BeerJoin --> CheckinPhase["Orchestrator: RunCheckinPhase()"] + BeerJoin --> FollowPhase["Orchestrator: RunFollowPhase()"] + + subgraph CHECKIN["Checkin Generation"] + CK1["ICheckinDistributionStrategy.AssignActivityWeights(user_pool_)
weights seeded from persona.checkin_weight; J-curve profile emerges from persona distribution"] + CK1 --> CK2["BEGIN TRANSACTION"] + CK2 --> CK3{"For each GeneratedUser in user_pool_"} + CK3 -- remaining --> CK4["CheckinsForUser(user, brewery_pool_.size())"] + CK4 --> CK5{"For each checkin index"} + CK5 -- remaining --> CK6["TimestampFor(user, index)"] + CK6 --> CK7["Select brewery from brewery_pool_"] + CK7 --> CK8["GenerateCheckin(user, brewery, timestamp) via DataGenerator"] + CK8 --> CK9["ProcessCheckin(checkin)"] + CK9 --> CK10["PipelineLogger.Log(Info, CheckinGeneration, nullopt, checkin_id, sqlite)"] + CK10 --> CK11["Append → checkin_pool_"] + CK11 --> CK12{"Batch size reached?"} + CK12 -- yes --> CK13["COMMIT and BEGIN"] --> CK5 + CK12 -- no --> CK5 + CK5 -- done --> CK3 + CK3 -- done --> CK14["COMMIT (Final)"] + end + + subgraph FOLLOW["Follow Generation"] + FL1["IFollowGenerationStrategy.AssignFollowWeights(user_pool_)
RandomFollowStrategy: uniform weights; ActivityWeightedFollowStrategy: derived from user.activity_weight so high-activity users attract more followers"] + FL1 --> FL2["BEGIN TRANSACTION"] + FL2 --> FL3["IFollowGenerationStrategy.GenerateFollows(user_pool_)
self-follow constraint (follower_id != followed_id) enforced here and at the DB schema level"] + FL3 --> FL4{"For each GeneratedFollow"} + FL4 -- remaining --> FL5["ProcessFollow(follow)"] + FL5 --> FL6["PipelineLogger.Log(Info, FollowGeneration, nullopt, follower_id, sqlite)"] + FL6 --> FL7["Append → follow_pool_"] + FL7 --> FL8{"Batch size reached?"} + FL8 -- yes --> FL9["COMMIT and BEGIN"] --> FL4 + FL8 -- no --> FL4 + FL4 -- done --> FL10["COMMIT (Final)"] + end + + CheckinPhase --> CK1 + FollowPhase --> FL1 + CK14 & FL10 --> PhaseJoin["Orchestrator: Join CheckinPhase, FollowPhase
checkin_pool_ and follow_pool_ are now fully populated"] + + PhaseJoin --> RatingPhase["Orchestrator: RunRatingPhase()
beer selection biased by user.persona.style_affinities and abv_range; rating skew modulated per persona"] + RatingPhase --> RT1["BEGIN TRANSACTION"] + RT1 --> RT2{"For each GeneratedCheckin in checkin_pool_"} + RT2 -- remaining --> RT3["Match brewery_id, select beer from beer_pool_ (same brewery_id, biased by persona affinities)"] + RT3 --> RT4{"Beer exists for brewery?"} + RT4 -- yes --> RT5["GenerateRating(user, beer, checkin_id) via DataGenerator"] + RT5 --> RT6["ProcessRating(rating)"] + RT6 --> RT7["PipelineLogger.Log(Info, RatingGeneration, nullopt, rating_id, sqlite)"] + RT7 --> RT8{"Batch size reached?"} + RT8 -- yes --> RT9["COMMIT and BEGIN"] --> RT2 + RT8 -- no --> RT2 + RT4 -- no --> RT10["Log warning; skip — brewery has no beers"] --> RT2 + RT2 -- done --> RT11["COMMIT (Final)"] + + RT11 --> Teardown["Orchestrator: Finalize SqliteExportService — safely closes the DB connection"] + Teardown --> CloseLogCh["Orchestrator: close log_ch"] + CloseLogCh --> MainDone["Main: spdlog::info 'Pipeline complete in X ms'"] + MainDone --> JoinLog["Main: Join Log Worker — drain guarantees no LogEntry is dropped at shutdown"] + JoinLog --> Stop2([stop]) diff --git a/docs/pipeline/diagrams/planned/activity.puml b/docs/pipeline/diagrams/planned/activity.puml deleted file mode 100644 index 67cabae3..00000000 --- a/docs/pipeline/diagrams/planned/activity.puml +++ /dev/null @@ -1,372 +0,0 @@ -@startuml biergarten_activity -!include ../biergarten-weizen-theme.puml -skinparam defaultFontSize 13 -skinparam titleFontSize 20 - -title The Biergarten Data Pipeline — Activity Diagram - -|Main| -start -:ParseArguments(argc, argv); -if (Invalid args?) then (yes) - :spdlog::error; - stop -else (no) -endif -:Init OpenSSL global state & LlamaBackendState; -:Build DI injector; - -:Initialize SqliteExportService; -note right - Opens SQLite connection. - (Transactions are now managed - per-phase via batching). -end note - -:Create BoundedChannel log_ch; -:Spawn Log Worker thread; -note right - Log worker drains log_ch for the - entire pipeline lifetime. - All workers emit LogEntry structs - via PipelineLogger -- never spdlog directly. -end note - -:BiergartenPipelineOrchestrator::Run(); -|BiergartenPipelineOrchestrator::Run()| - -fork - :JsonLoader::LoadBeerStyles("beer-styles.json"); - :EnrichmentService::PreWarmBeerStyleCache(beer_styles); -fork again - :JsonLoader::LoadLocations("locations.json"); - :EnrichmentService::PreWarmLocationCache(sampled_locations); -end fork -fork - :JsonLoader::LoadForenamesByCountry(\n "forenames-by-country.json"); -fork again - :JsonLoader::LoadSurnamesByCountry(\n "surnames-by-country.json"); -fork again - :JsonLoader::LoadPersonas("personas.json"); -end fork -note right - Each call is memoized after its first parse - (implemented today). MockCuratedDataService - returns a fixed in-memory dataset instead - under --mocked, skipping these JSON files. -end note - -' ═══════════════════════════════════════════ -' PHASE 0 — USER GENERATION -' ═══════════════════════════════════════════ -|Orchestrator| -:RunUserPhase(sampled_locations); -:Create BoundedChannels\n(loc_ch, exp_ch); - -fork - |Orchestrator| - :Loop: Send Locations -> loc_ch; - :Close loc_ch; - note right - Producer closes loc_ch. - LLM Worker while loop - terminates on empty + closed. - end note -fork again - |LLM Worker| - while (loc_ch has items?) is (yes) - :Receive Location; - - :GetLocationContextFromCache(location); - note right - Guaranteed cache hit from startup. - end note - - :IPersonaSelectionStrategy::SelectPersona(\n personas_palette_); - note right - Guaranteed cache hit from startup. - Returns a Persona struct carrying - style_affinities, abv_range, - ibu_preference, checkin_weight. - end note - - :SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1, rng); - note right - Deterministic lookup -- no LLM involved. - Free function (implemented today in - generate_users.cc), not a class method. - Name selected from pre-keyed maps - and passed into the generation prompt. - Skips the city if either map has no - entry for iso3166_1. - end note - - :GenerateUser(enriched_city, persona, sampled_name)\nvia DataGenerator; - note right - LLM receives: EnrichedCity context + persona - description + sampled name. Generates - bio and preference signals grounded - in locale and persona. - end note - - :PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "llm"); - :Send GeneratedUser -> exp_ch; - endwhile (no) - :Close exp_ch; - note right - Producer closes exp_ch. - SQLite Worker while loop - terminates on empty + closed. - end note -fork again - |SQLite Worker| - :BEGIN TRANSACTION; - while (exp_ch has items?) is (yes) - :Receive GeneratedUser; - :ProcessUser(user); - :PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "sqlite"); - :Append -> user_pool_; - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - endwhile (no) - :COMMIT (Final); -end fork - -|Orchestrator| -:Join LLM Worker, SQLite Worker; - -' ═══════════════════════════════════════════ -' PHASE 1a — BREWERY GENERATION -' ═══════════════════════════════════════════ -:RunBreweryPhase(sampled_locations); -:Create BoundedChannels\n(loc_ch, exp_ch); - -fork - |Orchestrator| - :Loop: Sample User from user_pool_ - and pair with Location; - :Send BreweryTask(Location, User) -> loc_ch; - :Close loc_ch; -fork again - |LLM Worker| - while (loc_ch has items?) is (yes) - :Receive BreweryTask(Location, User); - - :GetLocationContextFromCache(task.location); - note right - Guaranteed cache hit from startup. - end note - - :GenerateBrewery(enriched_city, context, task.user)\nvia DataGenerator; - note right - KV cache stays warm. - Brewery is linked to the sampled owner_user_id. - end note - :PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "llm"); - :Send GeneratedBrewery -> exp_ch; - endwhile (no) - :Close exp_ch; -fork again - |SQLite Worker| - :BEGIN TRANSACTION; - while (exp_ch has items?) is (yes) - :Receive GeneratedBrewery; - :ProcessBrewery(brewery); - :PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "sqlite"); - :Append -> brewery_pool_; - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - endwhile (no) - :COMMIT (Final); -end fork - -|Orchestrator| -:Join LLM Worker, SQLite Worker; -note right - brewery_pool_ is now fully populated. - Phase 1b may begin. -end note - -' ═══════════════════════════════════════════ -' PHASE 1b — BEER GENERATION -' ═══════════════════════════════════════════ -:RunBeerPhase(); -:Create BoundedChannels\n(brew_ch, exp_ch); - -fork - |Orchestrator| - :Loop: Send Breweries -> brew_ch; - :Close brew_ch; -fork again - |LLM Worker| - while (brew_ch has items?) is (yes) - :Receive GeneratedBrewery; - :IBeerSelectionStrategy::SelectStyles(\n brewery, beer_style_palette_); - - while (For each selected BeerStyle?) is (remaining) - :GetStyleContextFromCache(style); - note right - Guaranteed cache hit from startup. - KV cache stays warm across all - beer generations -- system prompt - does not change within this phase. - end note - :GenerateBeer(brewery, style_context)\nvia DataGenerator; - :Attach GeneratedBeer to bundle; - endwhile (done) - - :PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "llm"); - :Send BeersBundle -> exp_ch; - endwhile (no) - :Close exp_ch; -fork again - |SQLite Worker| - :BEGIN TRANSACTION; - while (exp_ch has items?) is (yes) - :Receive BeersBundle; - while (For each beer in bundle?) is (remaining) - :Set beer.brewery_id from bundle; - :ProcessBeer(beer); - :Append -> beer_pool_; - endwhile (done) - :PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "sqlite"); - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - endwhile (no) - :COMMIT (Final); -end fork - -|Orchestrator| -:Join LLM Worker, SQLite Worker; -note right - Both brewery_pool_ and beer_pool_ - are now completely populated. - Checkin and Follow phases may - now run in parallel. -end note - -' ═══════════════════════════════════════════ -' PHASE 2 — CHECKIN + FOLLOW GENERATION -' (parallel — both depend only on user_pool_ -' and brewery_pool_ being fully populated) -' ═══════════════════════════════════════════ -fork - |Orchestrator| - :RunCheckinPhase(); - :ICheckinDistributionStrategy::\nAssignActivityWeights(user_pool_); - note right - Weights seeded from each user's - persona.checkin_weight. J-curve profile - emerges from persona distribution. - end note - - :BEGIN TRANSACTION; - while (For each GeneratedUser in user_pool_?) is (remaining) - :CheckinsForUser(user, brewery_pool_.size()); - while (For each checkin index?) is (remaining) - :TimestampFor(user, index); - :Select brewery from brewery_pool_; - :GenerateCheckin(user, brewery, timestamp)\nvia DataGenerator; - :ProcessCheckin(checkin); - :PipelineLogger::Log(Info, CheckinGeneration,\n nullopt, checkin_id, "sqlite"); - :Append -> checkin_pool_; - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - endwhile (done) - endwhile (done) - :COMMIT (Final); - -fork again - |Orchestrator| - :RunFollowPhase(); - :IFollowGenerationStrategy::\nAssignFollowWeights(user_pool_); - note right - For RandomFollowStrategy, weights - are uniform. For ActivityWeightedFollowStrategy, - weights derived from user.activity_weight - so high-activity users attract more followers. - end note - - :BEGIN TRANSACTION; - :IFollowGenerationStrategy::\nGenerateFollows(user_pool_); - note right - Self-follow constraint (follower_id != followed_id) - enforced here and at the DB schema level. - end note - while (For each GeneratedFollow?) is (remaining) - :ProcessFollow(follow); - :PipelineLogger::Log(Info, FollowGeneration,\n nullopt, follower_id, "sqlite"); - :Append -> follow_pool_; - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - endwhile (done) - :COMMIT (Final); - -end fork - -|Orchestrator| -:Join CheckinPhase, FollowPhase; -note right - checkin_pool_ and follow_pool_ - are now fully populated. - Rating phase may begin. -end note - -' ═══════════════════════════════════════════ -' PHASE 3 — RATING GENERATION -' ═══════════════════════════════════════════ -:RunRatingPhase(); -note right - Beer selection biased by - user.persona.style_affinities and abv_range. - Rating skew modulated per persona. -end note - -:BEGIN TRANSACTION; -while (For each GeneratedCheckin in checkin_pool_?) is (remaining) - :Match brewery_id, select beer from beer_pool_\n(same brewery_id, biased by persona affinities); - if (Beer exists for brewery?) then (yes) - :GenerateRating(user, beer, checkin_id)\nvia DataGenerator; - :ProcessRating(rating); - :PipelineLogger::Log(Info, RatingGeneration,\n nullopt, rating_id, "sqlite"); - if (Batch size reached?) then (yes) - :COMMIT & BEGIN; - else (no) - endif - else (no) - :PipelineLogger::Log(Warn, RatingGeneration,\n nullopt, brewery_id, "sqlite"); - :Skip -- brewery has no beers; - endif -endwhile (done) -:COMMIT (Final); - -' ═══════════════════════════════════════════ -' TEARDOWN -' ═══════════════════════════════════════════ -|Orchestrator| -:Finalize SqliteExportService; -note right - Safely closes the DB connection. -end note -:Close log_ch; - -|Main| -:spdlog::info "Pipeline complete in X ms"; -:Join Log Worker; -note right - Drain guarantees no LogEntry is - dropped at shutdown. -end note -stop - -@enduml diff --git a/docs/pipeline/diagrams/planned/class.md b/docs/pipeline/diagrams/planned/class.md new file mode 100644 index 00000000..64b63917 --- /dev/null +++ b/docs/pipeline/diagrams/planned/class.md @@ -0,0 +1,17 @@ +# Biergarten Data Pipeline — Class Diagram (Planned) + +```mermaid +--8<-- "pipeline/diagrams/planned/class.mmd" +``` + +## Notes + +- `Logger`, `PipelineLogger`, and `LogWorker` are referenced only in + relationships in the source PlantUML (not given their own class blocks there + either) — they appear here as implicit nodes for the same reason. +- Package grouping from the PlantUML source (`Domain: Models`, + `Domain: Application Configuration`, `Domain: Policy`, + `Infrastructure: Logging`, `Infrastructure: Pipeline Channel`, + `Infrastructure: Data Preloading`, `Infrastructure: Enrichment`, + `Infrastructure: Prompting`, `Infrastructure: Data Generation`, + `Infrastructure: Data Export`) is preserved as Mermaid `namespace` blocks. diff --git a/docs/pipeline/diagrams/planned/class.mmd b/docs/pipeline/diagrams/planned/class.mmd new file mode 100644 index 00000000..3fb645ef --- /dev/null +++ b/docs/pipeline/diagrams/planned/class.mmd @@ -0,0 +1,643 @@ +classDiagram + namespace DomainModels { + class Location { + +string city + +string state_province + +string iso3166_2 + +string country + +string iso3166_1 + +vector~string~ local_languages + +double latitude + +double longitude + } + + class LocationContext { + +string text + +Completeness completeness + +size_t char_count + } + + class Completeness { + <> + Full + Partial + Absent + } + + class EnrichedCity { + +Location location + +LocationContext context + } + + class BeerStyle { + +string name + +string description + +float min_abv + +float max_abv + +int min_ibu + +int max_ibu + } + + class BreweryResult { + +string name_en + +string description_en + +string name_local + +string description_local + } + + class BeerResult { + +string name_en + +string description_en + +string name_local + +string description_local + +string style + +float abv + +int ibu + } + + class UserResult { + +string first_name + +string last_name + +string gender + +string username + +string bio + +float activity_weight + } + + class Name { + +string first_name + +string last_name + +string gender + } + + class ForenameEntry { + +string name + +string gender + } + + class CheckinResult { + +string checked_in_at + +string note + } + + class RatingResult { + +float score + +string note + } + + class GenerationMetadata { + +uint64_t generation_id + +string generated_time + +bool context_provided + +string generated_with + } + + class GeneratedBrewery { + +uint64_t brewery_id + +Location location + +BreweryResult brewery + +Completeness context_completeness + +GenerationMetadata metadata + } + + class GeneratedBeer { + +uint64_t beer_id + +uint64_t brewery_id + +Location location + +BeerStyle style + +BeerResult beer + +GenerationMetadata metadata + } + + class GeneratedUser { + +uint64_t user_id + +Location location + +UserResult user + +string email + +string date_of_birth + +string password + +GenerationMetadata metadata + } + + class GeneratedCheckin { + +uint64_t checkin_id + +uint64_t user_id + +uint64_t brewery_id + +CheckinResult checkin + +GenerationMetadata metadata + } + + class GeneratedRating { + +uint64_t user_id + +uint64_t beer_id + +uint64_t checkin_id + +RatingResult rating + +GenerationMetadata metadata + } + + class GeneratedFollow { + +uint64_t follower_id + +uint64_t followed_id + +GenerationMetadata metadata + } + + class UserPersona { + +string name + +string description + +vector~string~ style_affinities + } + } + note for ForenameEntry "forename_list/surname_list are unordered_set aliases, keyed by ISO 3166-1 country code via ICuratedDataService. SampleName() is a free function, not a method, sourced from popular-names-by-country-dataset (CC0)." + note for GeneratedUser "email, date_of_birth, and password are generated programmatically by the orchestrator (not LLM-authored) so a downstream auth-account seeding consumer can register real accounts from this export." + ForenameEntry ..> Name : SampleName() produces + LocationContext *-- Completeness + + namespace DomainApplicationConfiguration { + class SamplingOptions { + +float temperature + +float top_p + +uint32_t top_k + +uint32_t n_ctx + +int seed + } + + class GeneratorMode { + <> + kMock + kLlama + kOpenAI + } + + class GeneratorOptions { + +path model_path + +GeneratorMode mode + +optional~SamplingOptions~ sampling + +string openai_api_key + +string openai_model + } + + class PipelineOptions { + +path output_path + +path log_path + } + + class ApplicationOptions { + +GeneratorOptions generator + +PipelineOptions pipeline + } + } + note for GeneratorOptions "Already implemented (models.h), not planned: the three-way GeneratorMode split (kMock/kLlama/kOpenAI) and the openai_api_key/openai_model fields ship today, ahead of the rest of this diagram." + ApplicationOptions *-- GeneratorOptions + ApplicationOptions *-- PipelineOptions + GeneratorOptions *-- GeneratorMode + GeneratorOptions o-- SamplingOptions + + namespace DomainPolicy { + class ContextStrategy { + <> + +QueriesFor(loc) vector~string~ + +MaxContextChars() size_t + } + + class BreweryContextStrategy { + +QueriesFor(loc) vector~string~ + +MaxContextChars() size_t + } + + class BeerContextStrategy { + +QueriesFor(loc) vector~string~ + +MaxContextChars() size_t + } + + class SamplingStrategy { + <> + +Sample(locations) vector~Location~ + } + + class UniformSamplingStrategy { + -size_t sample_size_ + +Sample(locations) vector~Location~ + } + + class BeerSelectionStrategy { + <> + +SelectStyles(brewery, palette) vector~BeerStyle~ + } + + class RandomBeerSelectionStrategy { + -mt19937 rng_ + -size_t min_beers_ + -size_t max_beers_ + +SelectStyles(brewery, palette) vector~BeerStyle~ + } + + class CheckinDistributionStrategy { + <> + +AssignActivityWeights(users) void + +CheckinsForUser(user, brewery_count) size_t + +TimestampFor(user, index) string + } + + class JCurveCheckinStrategy { + -mt19937 rng_ + +AssignActivityWeights(users) void + +CheckinsForUser(user, brewery_count) size_t + +TimestampFor(user, index) string + } + + class RandomCheckinStrategy { + -mt19937 rng_ + -size_t min_checkins_ + -size_t max_checkins_ + +AssignActivityWeights(users) void + +CheckinsForUser(user, brewery_count) size_t + +TimestampFor(user, index) string + } + + class FollowGenerationStrategy { + <> + +GenerateFollows(users) vector~GeneratedFollow~ + } + + class RandomFollowStrategy { + -mt19937 rng_ + -size_t min_follows_ + -size_t max_follows_ + +GenerateFollows(users) vector~GeneratedFollow~ + } + + class ActivityWeightedFollowStrategy { + -mt19937 rng_ + -size_t min_follows_ + -size_t max_follows_ + +GenerateFollows(users) vector~GeneratedFollow~ + } + } + + namespace InfrastructureLogging { + class LogLevel { + <> + Debug + Info + Warn + Error + } + + class PipelinePhase { + <> + Startup + UserGeneration + BreweryAndBeerGeneration + CheckinGeneration + RatingGeneration + FollowGeneration + Teardown + } + + class LogEntry { + +time_point timestamp + +LogLevel level + +PipelinePhase phase + +string message + +optional~string~ worker + } + + class ILogger { + <> + +Log(entry) void + } + + class LogProducer { + -BoundedChannel~LogEntry~ channel_ + +Log(entry) void + } + + class LogDispatcher { + -BoundedChannel~LogEntry~ channel_ + +Run() void + -ToSpdlogLevel(level) level_enum + } + } + LogEntry *-- LogLevel + LogEntry *-- PipelinePhase + ILogger <|.. LogProducer + LogProducer ..> LogEntry : emits + LogDispatcher ..> LogEntry : consumes + + namespace InfrastructurePipelineChannel { + class BoundedChannel~T~ { + -queue~T~ queue_ + -mutex mutex_ + -condition_variable not_full_ + -condition_variable not_empty_ + -size_t capacity_ + -bool closed_ + +Send(item) void + +Receive() optional~T~ + +Close() void + } + } + + namespace InfrastructureDataPreloading { + class ICuratedDataService { + <> + +LoadLocations(filepath) vector~Location~ + +LoadBeerStyles(filepath) vector~BeerStyle~ + +LoadPersonas(filepath) vector~UserPersona~ + +LoadForenamesByCountry(filepath) unordered_map~string, forename_list~ + +LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~ + } + + class JsonLoader { + +LoadLocations(filepath) vector~Location~ + +LoadBeerStyles(filepath) vector~BeerStyle~ + +LoadPersonas(filepath) vector~UserPersona~ + +LoadForenamesByCountry(filepath) unordered_map~string, forename_list~ + +LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~ + } + + class MockCuratedDataService { + +LoadLocations(filepath) vector~Location~ + +LoadBeerStyles(filepath) vector~BeerStyle~ + +LoadPersonas(filepath) vector~UserPersona~ + +LoadForenamesByCountry(filepath) unordered_map~string, forename_list~ + +LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~ + } + } + note for JsonLoader "Each Load* call is memoized after its first parse — implemented today, ahead of LoadBeerStyles which is still planned." + note for MockCuratedDataService "Fixed in-memory dataset for --mocked mode, implemented today for Locations/Personas/Forenames/Surnames; would need a LoadBeerStyles fixture too once that method is implemented." + + namespace InfrastructureEnrichment { + class EnrichmentService { + <> + +GetLocationContext(loc, strategy) LocationContext + } + + class WikipediaService { + -unique_ptr~WebClient~ client_ + -unordered_map~string, string~ extract_cache_ + +GetLocationContext(loc, strategy) LocationContext + -FetchExtract(query) string + } + + class WebClient { + <> + +Get(url) string + +UrlEncode(value) string + } + + class HttpWebClient { + +Get(url) string + +UrlEncode(value) string + } + } + + namespace InfrastructureGeocoding { + class AddressLookupResult { + +string address_line1 + +string postal_code + } + + class AddressService { + <> + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + + class NominatimAddressService { + -unique_ptr~WebClient~ client_ + -Logger logger_ + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + + class MockAddressService { + +ReverseGeocode(longitude, latitude) optional~AddressLookupResult~ + } + } + note for NominatimAddressService "Already implemented (services/address/), not planned: queries the public Nominatim API, sleeping 1s per request per its usage policy; returns nullopt on failure or when no usable street address is found. Ships today, ahead of the rest of this diagram." + note for MockAddressService "Already implemented, not planned: returns a fixed placeholder address ('123 Mock Street', '00000') for --mocked runs, no network call." + AddressService <|.. NominatimAddressService + AddressService <|.. MockAddressService + AddressService ..> AddressLookupResult : returns + NominatimAddressService *-- WebClient + + namespace InfrastructurePrompting { + class IPromptDirectory { + <> + +Load(key) string + } + + class PromptDirectory { + -path prompt_dir_ + -unordered_map~string, string~ cache_ + +PromptDirectory(prompt_dir) + +Load(key) string + } + } + IPromptDirectory <|.. PromptDirectory + + namespace InfrastructureDataGeneration { + class DataGenerator { + <> + +GenerateBrewery(location, context) BreweryResult + +GenerateBeer(brewery_id, location, context, style) BeerResult + +GenerateUser(city, persona, name) UserResult + +GenerateCheckin(user, brewery, timestamp) CheckinResult + +GenerateRating(user, beer, checkin_id) RatingResult + } + + class MockGenerator { + +GenerateBrewery(...) BreweryResult + +GenerateBeer(...) BeerResult + +GenerateUser(...) UserResult + +GenerateCheckin(...) CheckinResult + +GenerateRating(...) RatingResult + -DeterministicHash(location) size_t + } + + class LlamaGenerator { + -ModelHandle model_ + -ContextHandle context_ + -unique_ptr~PromptFormatter~ prompt_formatter_ + -unique_ptr~IPromptDirectory~ prompt_directory_ + -mt19937 rng_ + +GenerateBrewery(...) BreweryResult + +GenerateBeer(...) BeerResult + +GenerateUser(...) UserResult + +GenerateCheckin(...) CheckinResult + +GenerateRating(...) RatingResult + -Load(opts) void + -Infer(system_prompt, user_prompt, max_tokens, grammar) string + -ValidateModelArchitecture() void + } + + class PromptFormatter { + <> + +Format(system_prompt, user_prompt) string + +ExpectedArchitecture() string_view + } + + class Gemma4JinjaPromptFormatter { + +Format(system_prompt, user_prompt) string + +ExpectedArchitecture() string_view + } + + class OpenAIGenerator { + -string api_key_ + -string model_ + -Logger logger_ + -unique_ptr~IPromptDirectory~ prompt_directory_ + -unique_ptr~WebClient~ web_client_ + +GenerateBrewery(location, context) BreweryResult + +GenerateBeer(brewery_id, location, context, style) BeerResult + +GenerateUser(city, persona, name) UserResult + +GenerateCheckin(user, brewery, timestamp) CheckinResult + +GenerateRating(user, beer, checkin_id) RatingResult + -CallChatCompletionsApi(...) string + } + } + note for OpenAIGenerator "Already implemented (data_generation/openai/), not planned: calls the OpenAI Chat Completions API with Structured Outputs (json_schema, strict) to guarantee schema-valid JSON, needing no IPromptFormatter or GBNF grammar. Ships today for GenerateBrewery/GenerateUser only, ahead of the rest of this diagram — GenerateBeer/GenerateCheckin/GenerateRating are shown here for interface completeness but not yet implemented on any backend." + + namespace InfrastructureDataExport { + class ExportService { + <> + +Initialize() void + +ProcessBrewery(brewery) uint64_t + +ProcessBeer(beer) uint64_t + +ProcessUser(user) uint64_t + +ProcessCheckin(checkin) uint64_t + +ProcessRating(rating) void + +ProcessFollow(follow) void + +Finalize() void + } + + class SqliteExportService { + -unique_ptr~DateTimeProvider~ date_time_provider_ + -SqliteDatabaseHandle db_handle_ + -SqliteStatementHandle insert_location_stmt_ + -SqliteStatementHandle insert_brewery_stmt_ + -SqliteStatementHandle insert_beer_stmt_ + -SqliteStatementHandle insert_user_stmt_ + -SqliteStatementHandle insert_checkin_stmt_ + -SqliteStatementHandle insert_rating_stmt_ + -SqliteStatementHandle insert_follow_stmt_ + -bool transaction_open_ + -unordered_map~string, uint64_t~ location_cache_ + -unordered_map~string, uint64_t~ brewery_cache_ + +Initialize() void + +ProcessRecord(brewery) uint64_t + +ProcessRecord(beer) uint64_t + +ProcessRecord(user) uint64_t + +ProcessRecord(checkin) uint64_t + +ProcessRecord(rating) void + +ProcessRecord(follow) void + +Finalize() void + -InitializeSchema() void + -PrepareStatements() void + -RollbackAndCloseNoThrow() void + -FinalizeStatements() void + } + + class DateTimeProvider { + <> + +GetUtcTimestamp() string + } + + class SystemDateTimeProvider { + +GetUtcTimestamp() string + } + } + + class BiergartenPipelineOrchestrator { + -unique_ptr~ICuratedDataService~ curated_data_service_ + -unique_ptr~EnrichmentService~ enrichment_service_ + -unique_ptr~DataGenerator~ generator_ + -unique_ptr~Logger~ logger_ + -unique_ptr~ExportService~ exporter_ + -unique_ptr~AddressService~ address_service_ + -unique_ptr~ContextStrategy~ brewery_context_strategy_ + -unique_ptr~SamplingStrategy~ sampling_strategy_ + -unique_ptr~BeerSelectionStrategy~ beer_selection_strategy_ + -unique_ptr~CheckinDistributionStrategy~ checkin_strategy_ + -unique_ptr~FollowGenerationStrategy~ follow_strategy_ + -vector~BeerStyle~ beer_style_palette_ + -ApplicationOptions options_ + -vector~GeneratedUser~ user_pool_ + -vector~GeneratedBrewery~ brewery_pool_ + -vector~GeneratedBeer~ beer_pool_ + -vector~GeneratedCheckin~ checkin_pool_ + -vector~GeneratedFollow~ follow_pool_ + +Run() bool + -RunUserPhase(locations) void + -RunBreweryAndBeerPhase(locations) void + -RunCheckinPhase() void + -RunRatingPhase() void + -RunFollowPhase() void + } + + BiergartenPipelineOrchestrator *-- ICuratedDataService + BiergartenPipelineOrchestrator *-- EnrichmentService + BiergartenPipelineOrchestrator *-- DataGenerator + BiergartenPipelineOrchestrator *-- ExportService + BiergartenPipelineOrchestrator *-- AddressService + BiergartenPipelineOrchestrator *-- CheckinDistributionStrategy + BiergartenPipelineOrchestrator *-- FollowGenerationStrategy + BiergartenPipelineOrchestrator *-- SamplingStrategy + BiergartenPipelineOrchestrator *-- BeerSelectionStrategy + BiergartenPipelineOrchestrator *-- ApplicationOptions + BiergartenPipelineOrchestrator *-- Logger + + BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedUser : user_pool_ + BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedBrewery : brewery_pool_ + BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedBeer : beer_pool_ + BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedCheckin : checkin_pool_ + BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedFollow : follow_pool_ + + ICuratedDataService <|.. JsonLoader + ICuratedDataService <|.. MockCuratedDataService + Logger <|.. PipelineLogger + ContextStrategy <|.. BreweryContextStrategy + ContextStrategy <|.. BeerContextStrategy + SamplingStrategy <|.. UniformSamplingStrategy + BeerSelectionStrategy <|.. RandomBeerSelectionStrategy + CheckinDistributionStrategy <|.. JCurveCheckinStrategy + CheckinDistributionStrategy <|.. RandomCheckinStrategy + FollowGenerationStrategy <|.. RandomFollowStrategy + FollowGenerationStrategy <|.. ActivityWeightedFollowStrategy + EnrichmentService <|.. WikipediaService + WebClient <|.. HttpWebClient + DataGenerator <|.. MockGenerator + DataGenerator <|.. LlamaGenerator + DataGenerator <|.. OpenAIGenerator + PromptFormatter <|.. Gemma4JinjaPromptFormatter + ExportService <|.. SqliteExportService + DateTimeProvider <|.. SystemDateTimeProvider + + WikipediaService *-- WebClient + WikipediaService ..> ContextStrategy + LlamaGenerator *-- PromptFormatter + LlamaGenerator *-- IPromptDirectory + LlamaGenerator ..> GeneratorOptions + OpenAIGenerator *-- IPromptDirectory + OpenAIGenerator *-- WebClient + SqliteExportService *-- DateTimeProvider + + PipelineLogger o-- BoundedChannel : logs to + LogWorker o-- BoundedChannel : drains from + + EnrichedCity *-- Location + EnrichedCity *-- LocationContext + GeneratedBrewery *-- Location + GeneratedBrewery *-- BreweryResult + GeneratedBrewery *-- GenerationMetadata + GeneratedBeer *-- Location + GeneratedBeer *-- BeerStyle + GeneratedBeer *-- BeerResult + GeneratedBeer *-- GenerationMetadata + GeneratedUser *-- Location + GeneratedUser *-- UserResult + GeneratedUser *-- GenerationMetadata + GeneratedCheckin *-- CheckinResult + GeneratedCheckin *-- GenerationMetadata + GeneratedRating *-- RatingResult + GeneratedRating *-- GenerationMetadata + GeneratedFollow *-- GenerationMetadata diff --git a/docs/pipeline/diagrams/planned/class.puml b/docs/pipeline/diagrams/planned/class.puml deleted file mode 100644 index 68876efa..00000000 --- a/docs/pipeline/diagrams/planned/class.puml +++ /dev/null @@ -1,646 +0,0 @@ -@startuml class_diagram - -' ========================================== -' CONFIGURATION & STYLING -' ========================================== -!include ../biergarten-weizen-theme.puml -skinparam classAttributeFontSize 9 -skinparam defaultFontSize 25 -skinparam titleFontSize 30 - -title Biergarten Data Pipeline — Class Diagram - -package "Domain: Models" { - - class Location { - + city : std::string - + state_province : std::string - + iso3166_2 : std::string - + country : std::string - + iso3166_1 : std::string - + local_languages : std::vector - + latitude : double - + longitude : double - } - - class LocationContext { - + text : std::string - + completeness : Completeness - + char_count : size_t - } - - enum Completeness { - Full - Partial - Absent - } - - class EnrichedCity { - + location : Location - + context : LocationContext - } - - class BeerStyle { - + name : std::string - + description : std::string - + min_abv : float - + max_abv : float - + min_ibu : int - + max_ibu : int - } - - class BreweryResult { - + name_en : std::string - + description_en : std::string - + name_local : std::string - + description_local : std::string - } - - class BeerResult { - + name_en : std::string - + description_en : std::string - + name_local : std::string - + description_local : std::string - + style : std::string - + abv : float - + ibu : int - } - - class UserResult { - + first_name : std::string - + last_name : std::string - + gender : std::string - + username : std::string - + bio : std::string - + activity_weight : float - } - - class Name { - + first_name : std::string - + last_name : std::string - + gender : std::string - } - - class ForenameEntry { - + name : std::string - + gender : std::string - } - - note right of ForenameEntry - forename_list = std::unordered_set - surname_list = std::unordered_set - (aliases declared in curated_data_service.h). - ICuratedDataService::LoadForenamesByCountry() / - LoadSurnamesByCountry() each return a flat - std::unordered_map - or keyed by ISO - 3166-1 country code -- no NamesByCountry - wrapper class. SampleName(forenames_by_country, - surnames_by_country, iso3166_1, rng) is a free - function, not a method, so it can be called by - any per-city worker without owning the maps. - Sourced from popular-names-by-country-dataset - (CC0) -- see ETHICS-AND-KNOWN-ISSUES.md. Gender - is preserved per forename so it can be threaded - through to persona/bio generation later; surnames - carry no gender in the source data. Pairing - happens at sample time, not at fixture-authoring - time, so no information from the source dataset - is discarded up front. - end note - - ForenameEntry ..> Name : SampleName() produces - - class CheckinResult { - + checked_in_at : std::string - + note : std::string - } - - class RatingResult { - + score : float - + note : std::string - } - - class GenerationMetadata { - + generation_id : uint64_t - + generated_time : std::string - + context_provided : bool - + generated_with : std::string - } - - class GeneratedBrewery { - + brewery_id : uint64_t - + location : Location - + brewery : BreweryResult - + context_completeness : LocationContext::Completeness - + metadata : GenerationMetadata - } - - class GeneratedBeer { - + beer_id : uint64_t - + brewery_id : uint64_t - + location : Location - + style : BeerStyle - + beer : BeerResult - + metadata : GenerationMetadata - } - - class GeneratedUser { - + user_id : uint64_t - + location : Location - + user : UserResult - + email : std::string - + date_of_birth : std::string - + password : std::string - + metadata : GenerationMetadata - } - - note right of GeneratedUser - email, date_of_birth, and password are - programmatically generated by the orchestrator - (not LLM-authored) so a downstream auth-account - seeding consumer can register real accounts from - this export. See ROADMAP.md §1. - end note - - class GeneratedCheckin { - + checkin_id : uint64_t - + user_id : uint64_t - + brewery_id : uint64_t - + checkin : CheckinResult - + metadata : GenerationMetadata - } - - class GeneratedRating { - + user_id : uint64_t - + beer_id : uint64_t - + checkin_id : uint64_t - + rating : RatingResult - + metadata : GenerationMetadata - } - - class GeneratedFollow { - + follower_id : uint64_t - + followed_id : uint64_t - + metadata : GenerationMetadata - } - - class UserPersona { - + name: std::string - + description: std::string - + style_affinities: std::vector - } - - LocationContext *-- Completeness -} - -package "Domain: Application Configuration" { - class SamplingOptions { - + temperature: float = 1.0F - + top_p: float = 0.95F - + top_k: uint32_t = 64 - + n_ctx: uint32_t = 8192 - + seed: int = -1 - } - - class GeneratorOptions { - + model_path: std::filesystem::path - + use_mocked: bool = false - + sampling: std::optional - } - - class PipelineOptions { - + output_path: std::filesystem::path - + log_path: std::filesystem::path - } - - class ApplicationOptions { - + generator: GeneratorOptions - + pipeline: PipelineOptions - } - - ApplicationOptions *-- GeneratorOptions - ApplicationOptions *-- PipelineOptions - GeneratorOptions o-- SamplingOptions -} - -package "Domain: Policy" { - - interface ContextStrategy <> { - + QueriesFor(loc : const Location&) : std::vector - + MaxContextChars() : size_t - } - - class BreweryContextStrategy { - + QueriesFor(loc : const Location&) : std::vector - + MaxContextChars() : size_t - } - - class BeerContextStrategy { - + QueriesFor(loc : const Location&) : std::vector - + MaxContextChars() : size_t - } - - interface SamplingStrategy <> { - + Sample(locations : const std::vector&) : std::vector - } - - class UniformSamplingStrategy { - - sample_size_ : size_t - + Sample(locations : const std::vector&) : std::vector - } - - interface BeerSelectionStrategy <> { - + SelectStyles(brewery : const GeneratedBrewery&,\n palette : std::span) : std::vector - } - - class RandomBeerSelectionStrategy { - - rng_ : std::mt19937 - - min_beers_ : size_t - - max_beers_ : size_t - + SelectStyles(brewery : const GeneratedBrewery&,\n palette : std::span) : std::vector - } - - interface CheckinDistributionStrategy <> { - + AssignActivityWeights(users : std::vector&) : void - + CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t - + TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string - } - - class JCurveCheckinStrategy { - - rng_ : std::mt19937 - + AssignActivityWeights(users : std::vector&) : void - + CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t - + TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string - } - - class RandomCheckinStrategy { - - rng_ : std::mt19937 - - min_checkins_ : size_t - - max_checkins_ : size_t - + AssignActivityWeights(users : std::vector&) : void - + CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t - + TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string - } - - interface FollowGenerationStrategy <> { - + GenerateFollows(users : const std::vector&) : std::vector - } - - class RandomFollowStrategy { - - rng_ : std::mt19937 - - min_follows_ : size_t - - max_follows_ : size_t - + GenerateFollows(users : const std::vector&) : std::vector - } - - class ActivityWeightedFollowStrategy { - - rng_ : std::mt19937 - - min_follows_ : size_t - - max_follows_ : size_t - + GenerateFollows(users : const std::vector&) : std::vector - } -} - -package "Infrastructure: Logging" { - enum LogLevel { - Debug - Info - Warn - Error - } - - enum PipelinePhase { - Startup - UserGeneration - BreweryAndBeerGeneration - CheckinGeneration - RatingGeneration - FollowGeneration - Teardown - } - - class LogEntry { - + timestamp : std::chrono::system_clock::time_point - + level : LogLevel - + phase : PipelinePhase - + message : std::string - + worker : std::optional - } - - interface ILogger <> { - + Log(entry : const LogEntry&) : void - } - - class LogProducer { - - channel_ : BoundedChannel& - + Log(entry : const LogEntry&) : void - } - - class LogDispatcher { - - channel_ : BoundedChannel& - + Run() : void - - ToSpdlogLevel(level) : spdlog::level::level_enum - } - - LogEntry *-- LogLevel - LogEntry *-- PipelinePhase - ILogger <|.. LogProducer - LogProducer ..> LogEntry : emits - LogDispatcher ..> LogEntry : consumes -} - -package "Infrastructure: Pipeline Channel" { - - class "BoundedChannel" as BoundedChannel { - - queue_ : std::queue - - mutex_ : std::mutex - - not_full_ : std::condition_variable - - not_empty_ : std::condition_variable - - capacity_ : size_t - - closed_ : bool - + Send(item : T) : void - + Receive() : std::optional - + Close() : void - } - -} - -package "Infrastructure: Data Preloading" { - - interface ICuratedDataService <> { - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map& - } - - class JsonLoader { - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map& - } - - note right of JsonLoader - Each Load* call is memoized after its first - parse -- implemented today, ahead of the rest - of this package (LoadBeerStyles is still planned). - end note - - class MockCuratedDataService { - + LoadLocations(filepath : const std::filesystem::path&) : const std::vector& - + LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector& - + LoadPersonas(filepath : const std::filesystem::path&) : const std::vector& - + LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map& - + LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map& - } - - note right of MockCuratedDataService - Fixed in-memory dataset used in --mocked mode, - implemented today for Locations/Personas/ - Forenames/Surnames; would need a LoadBeerStyles - fixture too once that method is implemented. - end note - -} - -package "Infrastructure: Enrichment" { - - interface EnrichmentService <> { - + GetLocationContext(loc : const Location&,\n strategy : const ContextStrategy&) : LocationContext - } - - class WikipediaService { - - client_ : std::unique_ptr - - extract_cache_ : std::unordered_map - + GetLocationContext(loc : const Location&,\n strategy : const ContextStrategy&) : LocationContext - - FetchExtract(query : std::string_view) : std::string - } - - interface WebClient <> { - + Get(url : const std::string&) : std::string - + UrlEncode(value : const std::string&) : std::string - } - - class HttpWebClient { - + Get(url : const std::string&) : std::string - + UrlEncode(value : const std::string&) : std::string - } - -} - -package "Infrastructure: Prompting" { - - interface IPromptDirectory <> { - + Load(key : std::string_view) : std::string - } - - class PromptDirectory { - - prompt_dir_ : std::filesystem::path - - cache_ : std::unordered_map - + PromptDirectory(prompt_dir : const std::filesystem::path&) - + Load(key : std::string_view) : std::string - } - - IPromptDirectory <|.. PromptDirectory -} - -package "Infrastructure: Data Generation" { - - interface DataGenerator <> { - + GenerateBrewery(location : const Location&,\n context : const LocationContext&) : BreweryResult - + GenerateBeer(brewery_id : uint64_t,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult - + GenerateUser(city : const EnrichedCity&,\n persona : const UserPersona&,\n name : const Name&) : UserResult - + GenerateCheckin(user : const GeneratedUser&,\n brewery : const GeneratedBrewery&,\n timestamp : const std::string&) : CheckinResult - + GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : uint64_t) : RatingResult - } - - class MockGenerator { - + GenerateBrewery(...) : BreweryResult - + GenerateBeer(...) : BeerResult - + GenerateUser(...) : UserResult - + GenerateCheckin(...) : CheckinResult - + GenerateRating(...) : RatingResult - - DeterministicHash(location : const Location&) : size_t - } - - class LlamaGenerator { - - model_ : ModelHandle - - context_ : ContextHandle - - prompt_formatter_ : std::unique_ptr - - prompt_directory_ : std::unique_ptr - - rng_ : std::mt19937 - + GenerateBrewery(...) : BreweryResult - + GenerateBeer(...) : BeerResult - + GenerateUser(...) : UserResult - + GenerateCheckin(...) : CheckinResult - + GenerateRating(...) : RatingResult - - Load(opts : const GeneratorOptions&) : void - - Infer(system_prompt, user_prompt,\n max_tokens, grammar) : std::string - - ValidateModelArchitecture() : void - } - - interface PromptFormatter <> { - + Format(system_prompt : std::string_view,\n user_prompt : std::string_view) : std::string - + ExpectedArchitecture() : std::string_view - } - - class Gemma4JinjaPromptFormatter { - + Format(...) : std::string - + ExpectedArchitecture() : std::string_view - } - -} - -package "Infrastructure: Data Export" { - - interface ExportService <> { - + Initialize() : void - + ProcessBrewery(brewery : const GeneratedBrewery&) : uint64_t - + ProcessBeer(beer : const GeneratedBeer&) : uint64_t - + ProcessUser(user : const GeneratedUser&) : uint64_t - + ProcessCheckin(checkin : const GeneratedCheckin&) : uint64_t - + ProcessRating(rating : const GeneratedRating&) : void - + ProcessFollow(follow : const GeneratedFollow&) : void - + Finalize() : void - } - - class SqliteExportService { - - date_time_provider_ : std::unique_ptr - - db_handle_ : SqliteDatabaseHandle - - insert_location_stmt_ : SqliteStatementHandle - - insert_brewery_stmt_ : SqliteStatementHandle - - insert_beer_stmt_ : SqliteStatementHandle - - insert_user_stmt_ : SqliteStatementHandle - - insert_checkin_stmt_ : SqliteStatementHandle - - insert_rating_stmt_ : SqliteStatementHandle - - insert_follow_stmt_ : SqliteStatementHandle - - transaction_open_ : bool - - location_cache_ : std::unordered_map - - brewery_cache_ : std::unordered_map - + Initialize() : void - + ProcessRecord(brewery : const GeneratedBrewery&) : uint64_t - + ProcessRecord(beer : const GeneratedBeer&) : uint64_t - + ProcessRecord(user : const GeneratedUser&) : uint64_t - + ProcessRecord(checkin : const GeneratedCheckin&) : uint64_t - + ProcessRecord(rating : const GeneratedRating&) : void - + ProcessRecord(follow : const GeneratedFollow&) : void - + Finalize() : void - - InitializeSchema() : void - - PrepareStatements() : void - - RollbackAndCloseNoThrow() : void - - FinalizeStatements() : void - } - - interface DateTimeProvider <> { - + GetUtcTimestamp() : std::string - } - - class SystemDateTimeProvider { - + GetUtcTimestamp() : std::string - } - -} - -class BiergartenPipelineOrchestrator { - - curated_data_service_ : std::unique_ptr - - enrichment_service_ : std::unique_ptr - - generator_ : std::unique_ptr - - logger_ : std::unique_ptr - - exporter_ : std::unique_ptr - - brewery_context_strategy_ : std::unique_ptr - - sampling_strategy_ : std::unique_ptr - - beer_selection_strategy_ : std::unique_ptr - - checkin_strategy_ : std::unique_ptr - - follow_strategy_ : std::unique_ptr - - beer_style_palette_ : std::vector - - options_ : ApplicationOptions - -- - - user_pool_ : std::vector - - brewery_pool_ : std::vector - - beer_pool_ : std::vector - - checkin_pool_ : std::vector - - follow_pool_ : std::vector - -- - + Run() : bool - - RunUserPhase(locations : const std::vector&) : void - - RunBreweryAndBeerPhase(locations : const std::vector&) : void - - RunCheckinPhase() : void - - RunRatingPhase() : void - - RunFollowPhase() : void -} - -' --- Orchestration Aggregations (Services & Strategies) --- -BiergartenPipelineOrchestrator *-- ICuratedDataService -BiergartenPipelineOrchestrator *-- EnrichmentService -BiergartenPipelineOrchestrator *-- DataGenerator -BiergartenPipelineOrchestrator *-- ExportService -BiergartenPipelineOrchestrator *-- CheckinDistributionStrategy -BiergartenPipelineOrchestrator *-- FollowGenerationStrategy -BiergartenPipelineOrchestrator *-- SamplingStrategy -BiergartenPipelineOrchestrator *-- BeerSelectionStrategy -BiergartenPipelineOrchestrator *-- ApplicationOptions -BiergartenPipelineOrchestrator *-- Logger - -' --- Orchestration Aggregations (Data Pools) --- -BiergartenPipelineOrchestrator *-- "0..*" GeneratedUser : user_pool_ -BiergartenPipelineOrchestrator *-- "0..*" GeneratedBrewery : brewery_pool_ -BiergartenPipelineOrchestrator *-- "0..*" GeneratedBeer : beer_pool_ -BiergartenPipelineOrchestrator *-- "0..*" GeneratedCheckin : checkin_pool_ -BiergartenPipelineOrchestrator *-- "0..*" GeneratedFollow : follow_pool_ - -' --- Interfaces & Implementations --- -ICuratedDataService <|.. JsonLoader -ICuratedDataService <|.. MockCuratedDataService -Logger <|.. PipelineLogger -ContextStrategy <|.. BreweryContextStrategy -ContextStrategy <|.. BeerContextStrategy -SamplingStrategy <|.. UniformSamplingStrategy -BeerSelectionStrategy <|.. RandomBeerSelectionStrategy -CheckinDistributionStrategy <|.. JCurveCheckinStrategy -CheckinDistributionStrategy <|.. RandomCheckinStrategy -FollowGenerationStrategy <|.. RandomFollowStrategy -FollowGenerationStrategy <|.. ActivityWeightedFollowStrategy -EnrichmentService <|.. WikipediaService -WebClient <|.. HttpWebClient -DataGenerator <|.. MockGenerator -DataGenerator <|.. LlamaGenerator -PromptFormatter <|.. Gemma4JinjaPromptFormatter -ExportService <|.. SqliteExportService -DateTimeProvider <|.. SystemDateTimeProvider - -' --- Service Compositions & Dependencies --- -WikipediaService *-- WebClient -WikipediaService ..> ContextStrategy -LlamaGenerator *-- PromptFormatter -LlamaGenerator *-- IPromptDirectory -LlamaGenerator ..> GeneratorOptions -SqliteExportService *-- DateTimeProvider - -' --- Cross-Component Aggregations (Held References) --- -PipelineLogger o-- BoundedChannel : logs to -LogWorker o-- BoundedChannel : drains from - -' --- Domain Containment --- -EnrichedCity *-- Location -EnrichedCity *-- LocationContext -GeneratedBrewery *-- Location -GeneratedBrewery *-- BreweryResult -GeneratedBrewery *-- GenerationMetadata -GeneratedBeer *-- Location -GeneratedBeer *-- BeerStyle -GeneratedBeer *-- BeerResult -GeneratedBeer *-- GenerationMetadata -GeneratedUser *-- Location -GeneratedUser *-- UserResult -GeneratedUser *-- GenerationMetadata -GeneratedCheckin *-- CheckinResult -GeneratedCheckin *-- GenerationMetadata -GeneratedRating *-- RatingResult -GeneratedRating *-- GenerationMetadata -GeneratedFollow *-- GenerationMetadata - -@enduml diff --git a/docs/pipeline/diagrams/planned/output/biergarten_activity.svg b/docs/pipeline/diagrams/planned/output/biergarten_activity.svg deleted file mode 100644 index e565241d..00000000 --- a/docs/pipeline/diagrams/planned/output/biergarten_activity.svg +++ /dev/null @@ -1 +0,0 @@ -The Biergarten Data Pipeline — Activity DiagramParseArguments(argc, argv)spdlog::erroryesInvalid args?noInit OpenSSL global state & LlamaBackendStateBuild DI injectorOpens SQLite connection.(Transactions are now managedper-phase via batching).Initialize SqliteExportServiceCreate BoundedChannel<LogEntry> log_chLog worker drains log_ch for theentire pipeline lifetime.All workers emit LogEntry structsvia PipelineLogger -- never spdlog directly.Spawn Log Worker threadBiergartenPipelineOrchestrator::Run()spdlog::info "Pipeline complete in X ms"Drain guarantees no LogEntry isdropped at shutdown.Join Log WorkerJsonLoader::LoadBeerStyles("beer-styles.json")EnrichmentService::PreWarmBeerStyleCache(beer_styles)JsonLoader::LoadLocations("locations.json")EnrichmentService::PreWarmLocationCache(sampled_locations)Each call is memoized after its first parse(implemented today). MockCuratedDataServicereturns a fixed in-memory dataset insteadunder --mocked, skipping these JSON files.JsonLoader::LoadForenamesByCountry("forenames-by-country.json")JsonLoader::LoadSurnamesByCountry("surnames-by-country.json")JsonLoader::LoadPersonas("personas.json")RunUserPhase(sampled_locations)Create BoundedChannels(loc_ch, exp_ch)Loop: Send Locations -> loc_chProducer closes loc_ch.LLM Worker while loopterminates on empty + closed.Close loc_chJoin LLM Worker, SQLite WorkerRunBreweryPhase(sampled_locations)Create BoundedChannels(loc_ch, exp_ch)Loop: Sample User from user_pool_and pair with LocationSend BreweryTask(Location, User) -> loc_chClose loc_chbrewery_pool_ is now fully populated.Phase 1b may begin.Join LLM Worker, SQLite WorkerRunBeerPhase()Create BoundedChannels(brew_ch, exp_ch)Loop: Send Breweries -> brew_chClose brew_chBoth brewery_pool_ and beer_pool_are now completely populated.Checkin and Follow phases maynow run in parallel.Join LLM Worker, SQLite WorkerRunCheckinPhase()Weights seeded from each user'spersona.checkin_weight. J-curve profileemerges from persona distribution.ICheckinDistributionStrategy::AssignActivityWeights(user_pool_)BEGIN TRANSACTIONCheckinsForUser(user, brewery_pool_.size())TimestampFor(user, index)Select brewery from brewery_pool_GenerateCheckin(user, brewery, timestamp)via DataGeneratorProcessCheckin(checkin)PipelineLogger::Log(Info, CheckinGeneration,nullopt, checkin_id, "sqlite")Append -> checkin_pool_COMMIT & BEGINyesBatch size reached?noremainingFor each checkin index?doneremainingFor each GeneratedUser in user_pool_?doneCOMMIT (Final)RunFollowPhase()For RandomFollowStrategy, weightsare uniform. For ActivityWeightedFollowStrategy,weights derived from user.activity_weightso high-activity users attract more followers.IFollowGenerationStrategy::AssignFollowWeights(user_pool_)BEGIN TRANSACTIONSelf-follow constraint (follower_id != followed_id)enforced here and at the DB schema level.IFollowGenerationStrategy::GenerateFollows(user_pool_)ProcessFollow(follow)PipelineLogger::Log(Info, FollowGeneration,nullopt, follower_id, "sqlite")Append -> follow_pool_COMMIT & BEGINyesBatch size reached?noremainingFor each GeneratedFollow?doneCOMMIT (Final)checkin_pool_ and follow_pool_are now fully populated.Rating phase may begin.Join CheckinPhase, FollowPhaseBeer selection biased byuser.persona.style_affinities and abv_range.Rating skew modulated per persona.RunRatingPhase()BEGIN TRANSACTIONMatch brewery_id, select beer from beer_pool_(same brewery_id, biased by persona affinities)Beer exists for brewery?yesnoGenerateRating(user, beer, checkin_id)via DataGeneratorProcessRating(rating)PipelineLogger::Log(Info, RatingGeneration,nullopt, rating_id, "sqlite")COMMIT & BEGINyesBatch size reached?noPipelineLogger::Log(Warn, RatingGeneration,nullopt, brewery_id, "sqlite")Skip -- brewery has no beersremainingFor each GeneratedCheckin in checkin_pool_?doneCOMMIT (Final)Safely closes the DB connection.Finalize SqliteExportServiceClose log_chReceive LocationGuaranteed cache hit from startup.GetLocationContextFromCache(location)Guaranteed cache hit from startup.Returns a Persona struct carryingstyle_affinities, abv_range,ibu_preference, checkin_weight.IPersonaSelectionStrategy::SelectPersona(personas_palette_)Deterministic lookup -- no LLM involved.Free function (implemented today ingenerate_users.cc), not a class method.Name selected from pre-keyed mapsand passed into the generation prompt.Skips the city if either map has noentry for iso3166_1.SampleName(forenames_by_country,surnames_by_country, location.iso3166_1, rng)LLM receives: EnrichedCity context + personadescription + sampled name. Generatesbio and preference signals groundedin locale and persona.GenerateUser(enriched_city, persona, sampled_name)via DataGeneratorPipelineLogger::Log(Info, UserGeneration,city, user_id, "llm")Send GeneratedUser -> exp_chyesloc_ch has items?noProducer closes exp_ch.SQLite Worker while loopterminates on empty + closed.Close exp_chReceive BreweryTask(Location, User)Guaranteed cache hit from startup.GetLocationContextFromCache(task.location)KV cache stays warm.Brewery is linked to the sampled owner_user_id.GenerateBrewery(enriched_city, context, task.user)via DataGeneratorPipelineLogger::Log(Info,BreweryGeneration,city, brewery_id, "llm")Send GeneratedBrewery -> exp_chyesloc_ch has items?noClose exp_chReceive GeneratedBreweryIBeerSelectionStrategy::SelectStyles(brewery, beer_style_palette_)Guaranteed cache hit from startup.KV cache stays warm across allbeer generations -- system promptdoes not change within this phase.GetStyleContextFromCache(style)GenerateBeer(brewery, style_context)via DataGeneratorAttach GeneratedBeer to bundleremainingFor each selected BeerStyle?donePipelineLogger::Log(Info,BeerGeneration,city, brewery_id, "llm")Send BeersBundle -> exp_chyesbrew_ch has items?noClose exp_chBEGIN TRANSACTIONReceive GeneratedUserProcessUser(user)PipelineLogger::Log(Info, UserGeneration,city, user_id, "sqlite")Append -> user_pool_COMMIT & BEGINyesBatch size reached?noyesexp_ch has items?noCOMMIT (Final)BEGIN TRANSACTIONReceive GeneratedBreweryProcessBrewery(brewery)PipelineLogger::Log(Info,BreweryGeneration,city, brewery_id, "sqlite")Append -> brewery_pool_COMMIT & BEGINyesBatch size reached?noyesexp_ch has items?noCOMMIT (Final)BEGIN TRANSACTIONReceive BeersBundleSet beer.brewery_id from bundleProcessBeer(beer)Append -> beer_pool_remainingFor each beer in bundle?donePipelineLogger::Log(Info,BeerGeneration,city, brewery_id, "sqlite")COMMIT & BEGINyesBatch size reached?noyesexp_ch has items?noCOMMIT (Final)MainBiergartenPipelineOrchestrator::Run()OrchestratorLLM WorkerSQLite Worker \ No newline at end of file diff --git a/docs/pipeline/diagrams/planned/output/class.svg b/docs/pipeline/diagrams/planned/output/class.svg deleted file mode 100644 index f796ee6e..00000000 --- a/docs/pipeline/diagrams/planned/output/class.svg +++ /dev/null @@ -1 +0,0 @@ -Biergarten Data Pipeline — Class DiagramDomain: ModelsDomain: Application ConfigurationDomain: PolicyInfrastructure: LoggingInfrastructure: Pipeline ChannelInfrastructure: Data PreloadingInfrastructure: EnrichmentInfrastructure: PromptingInfrastructure: Data GenerationInfrastructure: Data ExportLocationcity : std::stringstate_province : std::stringiso3166_2 : std::stringcountry : std::stringiso3166_1 : std::stringlocal_languages : std::vector<std::string>latitude : doublelongitude : doubleLocationContexttext : std::stringcompleteness : Completenesschar_count : size_tCompletenessFullPartialAbsentEnrichedCitylocation : Locationcontext : LocationContextBeerStylename : std::stringdescription : std::stringmin_abv : floatmax_abv : floatmin_ibu : intmax_ibu : intBreweryResultname_en : std::stringdescription_en : std::stringname_local : std::stringdescription_local : std::stringBeerResultname_en : std::stringdescription_en : std::stringname_local : std::stringdescription_local : std::stringstyle : std::stringabv : floatibu : intUserResultfirst_name : std::stringlast_name : std::stringgender : std::stringusername : std::stringbio : std::stringactivity_weight : floatNamefirst_name : std::stringlast_name : std::stringgender : std::stringForenameEntryname : std::stringgender : std::stringforename_list = std::unordered_set<ForenameEntry>surname_list = std::unordered_set<std::string>(aliases declared in curated_data_service.h).ICuratedDataService::LoadForenamesByCountry() /LoadSurnamesByCountry() each return a flatstd::unordered_map<std::string, forename_list>or <std::string, surname_list> keyed by ISO3166-1 country code -- no NamesByCountrywrapper class. SampleName(forenames_by_country,surnames_by_country, iso3166_1, rng) is a freefunction, not a method, so it can be called byany per-city worker without owning the maps.Sourced from popular-names-by-country-dataset(CC0) -- see ETHICS-AND-KNOWN-ISSUES.md. Genderis preserved per forename so it can be threadedthrough to persona/bio generation later; surnamescarry no gender in the source data. Pairinghappens at sample time, not at fixture-authoringtime, so no information from the source datasetis discarded up front.CheckinResultchecked_in_at : std::stringnote : std::stringRatingResultscore : floatnote : std::stringGenerationMetadatageneration_id : uint64_tgenerated_time : std::stringcontext_provided : boolgenerated_with : std::stringGeneratedBrewerybrewery_id : uint64_tlocation : Locationbrewery : BreweryResultcontext_completeness : LocationContext::Completenessmetadata : GenerationMetadataGeneratedBeerbeer_id : uint64_tbrewery_id : uint64_tlocation : Locationstyle : BeerStylebeer : BeerResultmetadata : GenerationMetadataGeneratedUseruser_id : uint64_tlocation : Locationuser : UserResultemail : std::stringdate_of_birth : std::stringpassword : std::stringmetadata : GenerationMetadataemail, date_of_birth, and password areprogrammatically generated by the orchestrator(not LLM-authored) so a downstream auth-accountseeding consumer can register real accounts fromthis export. See ROADMAP.md §1.GeneratedCheckincheckin_id : uint64_tuser_id : uint64_tbrewery_id : uint64_tcheckin : CheckinResultmetadata : GenerationMetadataGeneratedRatinguser_id : uint64_tbeer_id : uint64_tcheckin_id : uint64_trating : RatingResultmetadata : GenerationMetadataGeneratedFollowfollower_id : uint64_tfollowed_id : uint64_tmetadata : GenerationMetadataUserPersonaname: std::stringdescription: std::stringstyle_affinities: std::vector<std::string>SamplingOptionstemperature: float = 1.0Ftop_p: float = 0.95Ftop_k: uint32_t = 64n_ctx: uint32_t = 8192seed: int = -1GeneratorOptionsmodel_path: std::filesystem::pathuse_mocked: bool = falsesampling: std::optional<SamplingOptions>PipelineOptionsoutput_path: std::filesystem::pathlog_path: std::filesystem::pathApplicationOptionsgenerator: GeneratorOptionspipeline: PipelineOptions«interface»ContextStrategyQueriesFor(loc : const Location&) : std::vector<std::string>MaxContextChars() : size_tBreweryContextStrategyQueriesFor(loc : const Location&) : std::vector<std::string>MaxContextChars() : size_tBeerContextStrategyQueriesFor(loc : const Location&) : std::vector<std::string>MaxContextChars() : size_t«interface»SamplingStrategySample(locations : const std::vector<Location>&) : std::vector<Location>UniformSamplingStrategysample_size_ : size_tSample(locations : const std::vector<Location>&) : std::vector<Location>«interface»BeerSelectionStrategySelectStyles(brewery : const GeneratedBrewery&,palette : std::span<const BeerStyle>) : std::vector<BeerStyle>RandomBeerSelectionStrategyrng_ : std::mt19937min_beers_ : size_tmax_beers_ : size_tSelectStyles(brewery : const GeneratedBrewery&,palette : std::span<const BeerStyle>) : std::vector<BeerStyle>«interface»CheckinDistributionStrategyAssignActivityWeights(users : std::vector<GeneratedUser>&) : voidCheckinsForUser(user : const GeneratedUser&,brewery_count : size_t) : size_tTimestampFor(user : const GeneratedUser&,index : size_t) : std::stringJCurveCheckinStrategyrng_ : std::mt19937AssignActivityWeights(users : std::vector<GeneratedUser>&) : voidCheckinsForUser(user : const GeneratedUser&,brewery_count : size_t) : size_tTimestampFor(user : const GeneratedUser&,index : size_t) : std::stringRandomCheckinStrategyrng_ : std::mt19937min_checkins_ : size_tmax_checkins_ : size_tAssignActivityWeights(users : std::vector<GeneratedUser>&) : voidCheckinsForUser(user : const GeneratedUser&,brewery_count : size_t) : size_tTimestampFor(user : const GeneratedUser&,index : size_t) : std::string«interface»FollowGenerationStrategyGenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>RandomFollowStrategyrng_ : std::mt19937min_follows_ : size_tmax_follows_ : size_tGenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>ActivityWeightedFollowStrategyrng_ : std::mt19937min_follows_ : size_tmax_follows_ : size_tGenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>LogLevelDebugInfoWarnErrorPipelinePhaseStartupUserGenerationBreweryAndBeerGenerationCheckinGenerationRatingGenerationFollowGenerationTeardownLogEntrytimestamp : std::chrono::system_clock::time_pointlevel : LogLevelphase : PipelinePhasemessage : std::stringworker : std::optional<std::string>«interface»ILoggerLog(entry : const LogEntry&) : voidLogProducerchannel_ : BoundedChannel<LogEntry>&Log(entry : const LogEntry&) : voidLogDispatcherchannel_ : BoundedChannel<LogEntry>&Run() : voidToSpdlogLevel(level) : spdlog::level::level_enumBoundedChannelTqueue_ : std::queue<T>mutex_ : std::mutexnot_full_ : std::condition_variablenot_empty_ : std::condition_variablecapacity_ : size_tclosed_ : boolSend(item : T) : voidReceive() : std::optional<T>Close() : void«interface»ICuratedDataServiceLoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&JsonLoaderLoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&Each Load* call is memoized after its firstparse -- implemented today, ahead of the restof this package (LoadBeerStyles is still planned).MockCuratedDataServiceLoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&Fixed in-memory dataset used in --mocked mode,implemented today for Locations/Personas/Forenames/Surnames; would need a LoadBeerStylesfixture too once that method is implemented.«interface»EnrichmentServiceGetLocationContext(loc : const Location&,strategy : const ContextStrategy&) : LocationContextWikipediaServiceclient_ : std::unique_ptr<WebClient>extract_cache_ : std::unordered_map<std::string, std::string>GetLocationContext(loc : const Location&,strategy : const ContextStrategy&) : LocationContextFetchExtract(query : std::string_view) : std::string«interface»WebClientGet(url : const std::string&) : std::stringUrlEncode(value : const std::string&) : std::stringHttpWebClientGet(url : const std::string&) : std::stringUrlEncode(value : const std::string&) : std::string«interface»IPromptDirectoryLoad(key : std::string_view) : std::stringPromptDirectoryprompt_dir_ : std::filesystem::pathcache_ : std::unordered_map<std::string, std::string>PromptDirectory(prompt_dir : const std::filesystem::path&)Load(key : std::string_view) : std::string«interface»DataGeneratorGenerateBrewery(location : const Location&,context : const LocationContext&) : BreweryResultGenerateBeer(brewery_id : uint64_t,location : const Location&,context : const LocationContext&,style : const BeerStyle&) : BeerResultGenerateUser(city : const EnrichedCity&,persona : const UserPersona&,name : const Name&) : UserResultGenerateCheckin(user : const GeneratedUser&,brewery : const GeneratedBrewery&,timestamp : const std::string&) : CheckinResultGenerateRating(user : const GeneratedUser&,beer : const GeneratedBeer&,checkin_id : uint64_t) : RatingResultMockGeneratorGenerateBrewery(...) : BreweryResultGenerateBeer(...) : BeerResultGenerateUser(...) : UserResultGenerateCheckin(...) : CheckinResultGenerateRating(...) : RatingResultDeterministicHash(location : const Location&) : size_tLlamaGeneratormodel_ : ModelHandlecontext_ : ContextHandleprompt_formatter_ : std::unique_ptr<PromptFormatter>prompt_directory_ : std::unique_ptr<IPromptDirectory>rng_ : std::mt19937GenerateBrewery(...) : BreweryResultGenerateBeer(...) : BeerResultGenerateUser(...) : UserResultGenerateCheckin(...) : CheckinResultGenerateRating(...) : RatingResultLoad(opts : const GeneratorOptions&) : voidInfer(system_prompt, user_prompt,max_tokens, grammar) : std::stringValidateModelArchitecture() : void«interface»PromptFormatterFormat(system_prompt : std::string_view,user_prompt : std::string_view) : std::stringExpectedArchitecture() : std::string_viewGemma4JinjaPromptFormatterFormat(...) : std::stringExpectedArchitecture() : std::string_view«interface»ExportServiceInitialize() : voidProcessBrewery(brewery : const GeneratedBrewery&) : uint64_tProcessBeer(beer : const GeneratedBeer&) : uint64_tProcessUser(user : const GeneratedUser&) : uint64_tProcessCheckin(checkin : const GeneratedCheckin&) : uint64_tProcessRating(rating : const GeneratedRating&) : voidProcessFollow(follow : const GeneratedFollow&) : voidFinalize() : voidSqliteExportServicedate_time_provider_ : std::unique_ptr<DateTimeProvider>db_handle_ : SqliteDatabaseHandleinsert_location_stmt_ : SqliteStatementHandleinsert_brewery_stmt_ : SqliteStatementHandleinsert_beer_stmt_ : SqliteStatementHandleinsert_user_stmt_ : SqliteStatementHandleinsert_checkin_stmt_ : SqliteStatementHandleinsert_rating_stmt_ : SqliteStatementHandleinsert_follow_stmt_ : SqliteStatementHandletransaction_open_ : boollocation_cache_ : std::unordered_map<std::string, uint64_t>brewery_cache_ : std::unordered_map<std::string, uint64_t>Initialize() : voidProcessRecord(brewery : const GeneratedBrewery&) : uint64_tProcessRecord(beer : const GeneratedBeer&) : uint64_tProcessRecord(user : const GeneratedUser&) : uint64_tProcessRecord(checkin : const GeneratedCheckin&) : uint64_tProcessRecord(rating : const GeneratedRating&) : voidProcessRecord(follow : const GeneratedFollow&) : voidFinalize() : voidInitializeSchema() : voidPrepareStatements() : voidRollbackAndCloseNoThrow() : voidFinalizeStatements() : void«interface»DateTimeProviderGetUtcTimestamp() : std::stringSystemDateTimeProviderGetUtcTimestamp() : std::stringBiergartenPipelineOrchestratorcurated_data_service_ : std::unique_ptr<ICuratedDataService>enrichment_service_ : std::unique_ptr<EnrichmentService>generator_ : std::unique_ptr<DataGenerator>logger_ : std::unique_ptr<Logger>exporter_ : std::unique_ptr<ExportService>brewery_context_strategy_ : std::unique_ptr<ContextStrategy>sampling_strategy_ : std::unique_ptr<SamplingStrategy>beer_selection_strategy_ : std::unique_ptr<BeerSelectionStrategy>checkin_strategy_ : std::unique_ptr<CheckinDistributionStrategy>follow_strategy_ : std::unique_ptr<FollowGenerationStrategy>beer_style_palette_ : std::vector<BeerStyle>options_ : ApplicationOptionsuser_pool_ : std::vector<GeneratedUser>brewery_pool_ : std::vector<GeneratedBrewery>beer_pool_ : std::vector<GeneratedBeer>checkin_pool_ : std::vector<GeneratedCheckin>follow_pool_ : std::vector<GeneratedFollow>Run() : boolRunUserPhase(locations : const std::vector<Location>&) : voidRunBreweryAndBeerPhase(locations : const std::vector<Location>&) : voidRunCheckinPhase() : voidRunRatingPhase() : voidRunFollowPhase() : voidLoggerPipelineLoggerLogWorkerSampleName() producesemitsconsumesuser_pool_0..*brewery_pool_0..*beer_pool_0..*checkin_pool_0..*follow_pool_0..*logs todrains from \ No newline at end of file diff --git a/docs/web/architecture.md b/docs/web/architecture.md new file mode 100644 index 00000000..de5614e4 --- /dev/null +++ b/docs/web/architecture.md @@ -0,0 +1,609 @@ +# Architecture + +This document describes the architecture of The Biergarten App. + +## High-level overview + +The Biergarten App is a monorepo split between the backend and the website: + +- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature + slices with MediatR +- **Frontend**: React 19 + React Router 8 website in `web/frontend` +- **Architecture Style**: Vertical-slice backend plus server-rendered React + frontend + +`archive/next-js-web-app/` contains an archived Next.js frontend, kept for +reference. + +## Diagrams + +For visual representations, see: + +- [Vertical Slice](diagrams/vertical-slice.md) - host wiring and project + reference diagrams +- [Deployment](diagrams/deployment.md) - Docker deployment diagram +- [Database Schema](diagrams/database-schema.md) - database relationships + +Authentication workflow diagrams, one per flow, under +[`diagrams/auth/`](diagrams/auth/): + +- [Registration](diagrams/auth/registration.md) +- [Login](diagrams/auth/login.md) +- [Refresh Token](diagrams/auth/refresh-token.md) +- [Confirm User](diagrams/auth/confirm-user.md) +- [Resend Confirmation](diagrams/auth/resend-confirmation.md) +- [Update Username](diagrams/auth/update-username.md) +- [Update Email](diagrams/auth/update-email.md) +- [Update Password](diagrams/auth/update-password.md) +- [Update Profile](diagrams/auth/update-profile.md) +- [Delete Account](diagrams/auth/delete-account.md) +- [Error Handling](diagrams/auth/error-handling.md) - shared + `GlobalExceptionFilter` mapping used by all flows above + +## Backend architecture + +### Vertical-slice architecture pattern + +The backend organizes business capabilities as feature slices instead of +technical layers. + +Each feature (`Features.Users`, `Features.Breweries`, `Features.Emails`, +`Features.Locations`, `Features.PhotoUpload`) is a single project that owns its +own MediatR commands/queries/handlers, validators, and repository end to end. + +`Features.Emails`, `Features.Locations`, and `Features.PhotoUpload` have no +`Controllers/` folder; all three are invoked internally by other slices via +MediatR/DI, never over HTTP: + +```mermaid +flowchart TB + subgraph HOST["API.Core (thin host)"] + direction LR + H1["Program.cs wiring:
MediatR + AddApplicationPart per slice"] + H2["Swagger / OpenAPI"] + H3["JWT auth middleware"] + H4["GlobalExceptionFilter"] + end + + subgraph SLICES["Feature Slices"] + direction LR + S1["Features.Users
Controllers, Commands/Queries,
Handlers, Repository"] + S2["Features.Breweries
Controller, Commands/Queries,
Handlers, Repository"] + S3["Features.Emails
(internal only)
Commands, Handlers,
EmailDispatcher"] + S4["Features.Locations
(internal only)
Repository"] + S5["Features.PhotoUpload
(internal only)
Commands, Handlers,
Repository"] + end + + subgraph SHARED["Shared Projects"] + direction LR + SH1["Shared.Contracts
ResponseBody envelope"] + SH2["Shared.Application
ValidationBehavior,
cross-slice email commands"] + end + + subgraph DOMAIN["Domain Layer"] + direction LR + D1["Domain.Entities"] + D2["Domain.Exceptions"] + end + + subgraph INFRA["Infrastructure Layer"] + direction LR + I1["Database.Connection"] + I2["Infrastructure.Jwt"] + I3["Infrastructure.PasswordHashing"] + I4["Infrastructure.Email /
Email.Templates"] + I5["Infrastructure.FileUpload"] + end + + DB[("Database (SQL Server)")] + + HOST -- "discovers controllers via
AddApplicationPart" --> SLICES + S1 -- "SendRegistrationEmailCommand
(via Shared.Application contract)" --> S3 + S1 -- "ProjectReference
(avatar upload)" --> S5 + + SLICES --> SHARED + SLICES --> DOMAIN + SLICES --> INFRA + + INFRA --> DB + + style S3 stroke-dasharray: 4 3 + style S4 stroke-dasharray: 4 3 + style S5 stroke-dasharray: 4 3 +``` + +### Layer responsibilities + +#### API layer (`API.Core`) + +**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of its +own + +**Components**: + +| Component | Description | +| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `Program.cs` | Registers MediatR (scanning every `Features.*` assembly), FluentValidation, and uses `AddApplicationPart` so each slice's controllers are discovered by MVC. | +| `GlobalException.cs` | Global exception filter (host-level cross-cutting concern). | +| `Authentication/JwtAuthenticationHandler.cs` | JWT auth scheme middleware. | +| Swagger/OpenAPI documentation, health check endpoints | Host-level documentation and health endpoints. | + +**Dependencies**: + +- Every `Features.*` project (for controller/MediatR discovery) +- `Shared.Contracts`, `Shared.Application` +- `Infrastructure.Jwt` (for the auth middleware) + +**Rules**: + +- No controllers, no business logic, no feature-specific contracts +- Exists purely to host and wire up the feature slices + +#### Feature slices (`Features.Users`, `Features.Breweries`, `Features.Emails`, `Features.Locations`, `Features.PhotoUpload`) + +**Purpose**: Each slice is the complete vertical for one business capability + +**Components** (per slice): + +| Component | Description | +| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `Controllers/` | HTTP endpoints, binding directly to Command/Query types as the request contract | +| `Commands//` and `Queries//` | One folder per operation, each containing the Command/Query record, its `IRequestHandler`, and (for commands) a FluentValidation validator | +| `Repository/` | The slice's own Dapper repository implementation | +| `Dtos/` | Response shapes returned by query handlers (never the raw domain entity) | +| `DependencyInjection/` | An `AddFeaturesX()` extension method registering the slice's repository/services | + +##### Note + +`Features.Emails` has no `Controllers/` folder. It's invoked only via MediatR +commands sent from other slices, never over HTTP. + +`Features.PhotoUpload` also has no `Controllers/` folder; its +`UploadPhotoCommand` is sent by other features' handlers (for example, a future +brewery photo upload command) rather than bound to an HTTP route directly. + +**Dependencies**: + +- `Domain.Entities`, `Domain.Exceptions` +- `Database.Connection` (generic ADO.NET connection/command plumbing) plus + whichever infrastructure project the slice needs (`Infrastructure.Jwt`/ + `Infrastructure.PasswordHashing` for Users, `Infrastructure.Email`/ + `Infrastructure.Email.Templates` for Emails, `Infrastructure.FileUpload` for + PhotoUpload) +- `Shared.Contracts`, `Shared.Application` +- No other `Features.*` project, with one exception: `Features.Users` references + `Features.PhotoUpload` directly for avatar uploads + +**Rules**: + +- All business logic for that feature lives in its command/query handlers +- No direct controller-to-repository calls; everything flows through MediatR +- Read endpoints return a dedicated `Dto`, never the domain entity directly + +#### Shared projects (`Shared.Contracts`, `Shared.Application`) + +**Purpose**: The minimum cross-slice surface area required because every slice +needs it, or because duplicating it four times would be worse than sharing it + +**Components**: + +| Project | Purpose | +| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Shared.Contracts` | `ResponseBody`/`ResponseBody`, the API response envelope every controller returns | +| `Shared.Application` | `ValidationBehavior` (the MediatR pipeline behavior that runs FluentValidation before a handler executes) and the cross-slice email commands (`SendRegistrationEmailCommand`, `SendResendConfirmationEmailCommand`) | + +**Rules**: + +- Kept deliberately small: this is the exception to "no slice depends on another + slice". + +#### Infrastructure layer + +**Purpose**: Technical capabilities and external integrations, shared by +whichever slices need them + +**Components**: + +| Project | Description | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Database.Connection` | Generic ADO.NET connection/command plumbing (`DefaultSqlConnectionFactory`, the abstract `DapperRepository` base class), not domain-specific; each slice's own `Repository/` folder builds on this. | +| `Infrastructure.Jwt` | JWT token generation and validation. | +| `Infrastructure.PasswordHashing` | Argon2id password hashing. | +| `Infrastructure.Email` | Email sending capabilities (SMTP/MailKit). | +| `Infrastructure.Email.Templates` | Email template rendering (Razor components). | +| `Infrastructure.FileUpload` | `IFileStorageProvider` abstraction over S3-compatible object storage, implemented by `S3FileStorageProvider` (targets the SeaweedFS container in dev; any S3-compatible endpoint in other environments) using the AWS SDK's `AmazonS3Client` with `ForcePathStyle = true`. | + +**Dependencies**: + +- Domain entities +- External libraries (ADO.NET, JWT, Argon2, MailKit, AWSSDK.S3, and so on) + +**Rules**: + +- Implements technical concerns only, no business logic +- Reusable across slices + +#### Domain layer (`Domain.Entities`) + +**Purpose**: Core business entities and models + +**Components**: + +| Entity | Description | +| --------------------- | -------------------------------------------------------------------------------------------------- | +| `UserAccount` | User profile data. | +| `UserCredential` | Authentication credentials. | +| `UserVerification` | Account verification state. | +| `BreweryPost` | A user-submitted brewery listing. | +| `BreweryPostLocation` | A brewery's address and coordinates. | +| `City` | A city referenced by a brewery location. | +| `StateProvince` | A state/province referenced by a city. | +| `Country` | A country referenced by a state/province. | +| `BeerStyle` | A beer style/category (schema table exists; no repository yet). | +| `BeerPost` | A beer listing tied to a brewery and style (schema table exists; no repository yet). | +| `Photo` | A photo uploaded by a user account, written via `Features.PhotoUpload`'s `IPhotoUploadRepository`. | + +**Dependencies**: + +- None (pure domain) + +**Rules**: + +- Plain Old CLR Objects (POCOs) +- No framework dependencies +- No infrastructure references +- Represents business concepts + +### Design patterns + +#### Vertical slice + MediatR + +**Purpose**: Organize code by feature instead of by technical layer, so +everything needed to understand or change one capability lives in one project + +**Implementation**: + +- Each HTTP write operation is a `Command` (for example, `CreateBreweryCommand` + in `Features.Breweries/Commands/CreateBrewery/`); each read operation is a + `Query` (for example, `GetBreweryByIdQuery`) +- Controllers generally bind directly to the Command/Query as the request body; + there is no separate request DTO + mapping step for writes + - The exception to this is for authenticated routes, when the current user id + needs to be extracted from the authentication token to be used in a command. +- A single shared `ValidationBehavior` + (`Shared.Application/Behaviors/`) runs FluentValidation validators in the + MediatR pipeline before any handler executes +- Query handlers map to a dedicated response `Dto`, so domain entities never + leak + +**Example**: + +```csharp +public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...) + : IRequest; + +public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler +{ + public async Task Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... } +} +``` + +#### Repository pattern + +**Purpose**: Abstract database access behind interfaces + +**Implementation**: each slice owns its own repository, scoped to that feature +only: + +- `Features.Users/Repository/IUserListRepository.cs`, + `Features.Users/Repository/IUserProfileRepository.cs` +- `Features.Breweries/Repository/IBreweryRepository.cs` +- `Features.PhotoUpload/Repository/IPhotoUploadRepository.cs` +- `Database.Connection/DefaultSqlConnectionFactory.cs`: the generic connection + factory every slice's repository builds on +- `Database.Connection/DapperRepository.cs`: shared abstract base class every + repository extends, providing `CreateConnection()` and a + `RollbackQuietlyAsync()` helper that rolls back a transaction while swallowing + any exception the rollback itself raises, so the exception that triggered the + rollback is what propagates + +**Benefits**: + +- Testable (easy to mock) +- Each slice's data access logic is self-contained + +**Example**: + +```csharp +public interface IUserListRepository +{ + Task GetByIdAsync(Guid id); + Task> GetAllAsync(int? limit, int? offset); +} +``` + +#### Dependency injection + +**Purpose**: Loose coupling and testability + +**Configuration**: `Program.cs` wires up MediatR/FluentValidation across every +`Features.*` assembly; each slice exposes its own `AddFeaturesX()` extension +method that registers its repository and slice-internal services + +**Lifetimes**: + +- Scoped: Repositories, slice-internal services (per request) +- Singleton: `DefaultSqlConnectionFactory` +- Transient: Utilities, helpers + +#### Direct SQL via Dapper/ADO.NET + +**Purpose**: Keep data access explicit and colocated with the slice that owns it + +**Strategy**: + +- Each repository issues inline SQL through Dapper, which handles parameter + binding. +- Proximity search (`GET /api/brewery/locations/nearby`) filters and orders rows + server-side with `Coordinates.STDistance(@Origin) <= @RangeInMetres`, avoiding + a client-side distance calculation over every row. +- Referential checks (for example, "does this `CityId` exist?") are explicit + `SELECT 1 ...` existence checks in the repository method, run inside the same + transaction as the write +- Optimistic concurrency uses each table's `RowVersion` (`ROWVERSION`) column, + checked in the `UPDATE ... WHERE ... AND RowVersion = @RowVersion` clause +- Repositories throw `Domain.Exceptions` types (`NotFoundException`, + `ConflictException`, ...) directly when a check fails; `API.Core`'s + `GlobalExceptionFilter` maps those exception types to HTTP status codes +- Application focuses on orchestration; the database enforces integrity via + keys, `CHECK` constraints, and cascades + +See [Database](database.md) for the schema and the app's SQL error-handling +approach in more detail. + +## Frontend architecture + +### Website (`web/frontend`) + +The website is a React Router 8 application with server-side rendering enabled. + +```text +web/frontend/ +├── app/ +│ ├── components/ Shared UI: Navbar, FormField, SubmitButton, ToastProvider, +│ │ RouteErrorState, ClientOnly (defers client-only children, +│ │ for example Leaflet maps, past hydration) +│ ├── features/ One folder per feature slice, each owning its own +│ │ routes/, components/, hooks/, schemas, and server-side +│ │ loader helpers: +│ │ - account profile/dashboard management +│ │ - auth login, register, confirm, logout +│ │ - breweries index/directory/detail routes, Leaflet +│ │ map components, nearby-search resource +│ │ route, distance/address utils +│ │ - catalog beers, beer styles +│ │ - home landing page +│ │ - theme theme switcher/guide +│ ├── hooks/ Shared hooks not tied to one feature, for example useMediaQuery +│ ├── routes.ts Route table (React Router 8 config-based routing) +│ ├── root.tsx App shell and global providers +│ └── app.css Theme tokens and global styling +├── .storybook/ Storybook config and preview setup +├── stories/ Storybook stories for shared UI and themes +├── tests/playwright/ Storybook Playwright coverage +└── package.json Frontend scripts and dependencies +``` + +### Frontend responsibilities + +- Render brewery discovery routes (`/breweries` index, `/breweries/directory`, + `/breweries/:id`) with Leaflet-powered maps and geolocation-based "nearby" + search +- Render the auth demo, account dashboard, and theme guide routes +- Manage cookie-backed website session state +- Call the .NET API for login, registration, token refresh, confirmation, and + brewery/location data +- Provide shared UI building blocks for forms, navigation, themes, and toasts +- Supply Storybook documentation and browser-based component verification + +### Theme system + +The website uses semantic DaisyUI theme tokens backed by four Biergarten themes: + +- Biergarten Lager +- Biergarten Stout +- Biergarten Cassis +- Biergarten Weizen + +All component styling should prefer semantic tokens such as `primary`, +`success`, `surface`, and `highlight` instead of hard-coded color values. + +### Archived frontend + +`archive/next-js-web-app/` contains an archived Next.js frontend, kept for +reference. Product and engineering documentation points to `web/frontend`. + +## Security architecture + +### Authentication flow + +| Flow | Details | +| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Registration** | User submits credentials; password is hashed with Argon2id; user account is created as unverified and a confirmation email is dispatched via `Features.Emails`. | +| **Email confirmation** | User follows the confirmation link/token from the email; `Features.Users`' `ConfirmUser` command marks the account as verified. | +| **Login** | User submits credentials; password is verified against the hash; access and refresh JWTs are issued and returned to the client; neither is stored server-side. | +| **Token refresh** | Client exchanges a valid, unexpired refresh token for a new access/refresh pair via `Features.Users`' `RefreshToken` command; refresh tokens are stateless JWTs, not tracked server-side, and the previous refresh token remains usable until its own expiration. | +| **API requests** | Client sends the access JWT in the `Authorization` header; `JwtAuthenticationHandler` validates the token; request proceeds if valid. | + +### Password security + +**Algorithm**: Argon2id + +| Setting | Value | +| ----------- | ---------------------------------------------- | +| Memory | 64MB | +| Iterations | 4 | +| Parallelism | 2 (hardcoded, to avoid thread-pool exhaustion) | +| Salt | 128-bit (16 bytes) | +| Hash | 256-bit (32 bytes) | + +### JWT tokens + +**Algorithm**: HS256 (HMAC-SHA256) + +**Claims**: + +| Claim | Meaning | +| ------------- | -------------------- | +| `sub` | User ID | +| `unique_name` | Username | +| `jti` | Unique token ID | +| `iat` | Issued at timestamp | +| `exp` | Expiration timestamp | + +**Configuration** (appsettings.json): + +```json +{ + "Jwt": { + "ExpirationMinutes": 60, + "Issuer": "biergarten-api", + "Audience": "biergarten-users" + } +} +``` + +This block exists in `appsettings.json`, but nothing in the code currently reads +it: actual token lifetimes come from the hardcoded `TokenServiceExpirationHours` +constants, and `JwtInfrastructure` hardcodes `ValidateIssuer`/`ValidateAudience` +to `false`, so `Issuer`/`Audience` are not enforced. + +## Database architecture + +For the table-by-table schema and the custom SQL error code scheme, see +[Database](database.md). + +### SQL-first philosophy + +**Principles**: + +1. Database is source of truth +2. Queries live in the owning repository, not scattered across the app +3. Database handles referential integrity +4. Application orchestrates, database executes + +**Benefits**: + +- Performance optimization via execution plans +- Query logic scoped to the repository that owns it, not spread across layers +- Version-controlled schema (migrations) +- Easier query profiling and tuning + +### Migration strategy + +**Tool**: DbUp + +**Process**: + +1. Write a SQL script under `Database.Migrations/scripts/` +2. Embed it in the `Database.Migrations` project +3. `Database.Migrations` runs each script against the target database on + startup, tracked so a script never re-runs once applied + +**Migration Files**: + +``` +scripts/ +└── 01-schema/ + ├── 01-UserAccount.sql + ├── 02-Photo.sql + ├── ... + ├── 16-UserProfile.sql + └── 17-UserAvatar.sql # one script per CREATE TABLE +``` + +Each table has its own script, numbered so a table's foreign keys always point +at an earlier-numbered script. As the schema evolves, new numbered scripts get +added rather than editing an existing one in place, so DbUp's already-applied +tracking stays valid. + +### Data seeding + +**Purpose**: Populate development/test databases with realistic data + +**Implementation**: `Database.Seed` project, using data produced by the C++ +pipeline under `tooling/pipeline/` (see +[Pipeline README](../pipeline/README.md)) + +**Seed Data**: + +- Countries, state/provinces, and cities (via `Features.Locations`' + `ILocationRepository`, which creates `Country`/`StateProvince` rows on demand + while resolving a city) +- User accounts (via `Features.Users`' repository) +- Brewery posts with locations (via `Features.Breweries`' repository) + +Tables that exist in the schema but have no repository yet ( `UserFollow`, +`BeerStyle`, `BeerPost`, `BeerPostPhoto`, `BeerPostComment`, `BreweryPostPhoto`) +are not seeded either. + +## Deployment architecture + +### Docker containerization + +**Container Structure**: + +| Container | Purpose | +| --------------------- | ------------------------------------------------------------ | +| `sqlserver` | SQL Server 2022 | +| `database.migrations` | Schema migration runner | +| `database.seed` | Data seeder | +| `api.core` | ASP.NET Core Web API | +| `mailpit` | Local dev/test SMTP server + web UI (not used in production) | +| `seaweedfs` | S3-compatible object storage for uploaded photos (dev only) | + +**Environments**: + +| Environment | Compose file | +| ----------- | -------------------------- | +| Development | `docker-compose.dev.yaml` | +| Testing | `docker-compose.test.yaml` | +| Production | `docker-compose.prod.yaml` | + +For details, see [Docker Guide](docker.md). + +### Health checks + +- **SQL Server**: Validates database connectivity +- **API**: Checks service health and dependencies + +**Configuration**: + +```yaml +healthcheck: + test: ["CMD-SHELL", "sqlcmd health check"] + interval: 10s + retries: 12 + start_period: 30s +``` + +## Testing architecture + +### Test pyramid + +```mermaid +flowchart TB + IT["Integration Tests
API.Specs (Reqnroll)"] + UT["Unit Tests (per slice, handler tests)
Features.Users.Tests, Features.Breweries.Tests,
Features.Emails.Tests, Features.PhotoUpload.Tests
(commands/queries/handlers, with repository/service
dependencies mocked via Moq)"] + + IT --- UT +``` + +**Strategy**: + +- Many unit tests (fast, isolated) +- Fewer integration tests (slower, e2e) +- Mock external dependencies +- Test database for integration tests + +For details, see [Testing Guide](testing.md). diff --git a/docs/web/database.md b/docs/web/database.md new file mode 100644 index 00000000..34f5c0e8 --- /dev/null +++ b/docs/web/database.md @@ -0,0 +1,125 @@ +--- +title: SQL Server database schema, migrations, and error handling +last-updated: 2026-08-31 +tags: + - sql + - repository + - database + - sql-server + - schema +--- + +SQL Server 2022 schema, migration approach, and the app's SQL error-handling +convention. For the vertical-slice code that talks to this schema, see +[Architecture](architecture.md). + +## Schema + +The full schema lives under +`web/backend/Database/Database.Migrations/scripts/01-schema/` with one numbered +script per table. + +``` +01-schema + ├── 01-UserAccount.sql + ├── 02-Photo.sql + ├── 03-UserVerification.sql + ├── 04-UserCredential.sql + ├── 05-UserFollow.sql + ├── 06-Country.sql + ├── 07-StateProvince.sql + ├── 08-City.sql + ├── 09-BreweryPost.sql + ├── 10-BreweryPostLocation.sql + ├── 11-BreweryPostPhoto.sql + ├── 12-BeerStyle.sql + ├── 13-BeerPost.sql + ├── 14-BeerPostPhoto.sql + ├── 15-BeerPostComment.sql + ├── 16-UserProfile.sql + └── 17-UserAvatar.sql +``` + +Each script is run in order by `Database.Migrations` (DbUp) on startup. Each +table is queried directly by the owning feature slice's repository (see +[Direct SQL via Dapper/ADO.NET](architecture.md#direct-sql-via-dapperadonet)). + +A generated DBML copy of the schema (for import into a diagramming tool such as +dbdiagram.io) lives at `web/backend/Database/Database.Migrations/schema.dbml`. + +All tables use a `UNIQUEIDENTIFIER` primary key defaulting to `NEWID()`, and +most carry a `RowVersion ROWVERSION` column used for optimistic concurrency on +updates (`UPDATE ... WHERE ... AND RowVersion = @RowVersion`). + +### Tables backed by the API today + +| Table | Backed by | Notes | +| --------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `UserAccount` | `Domain.Entities.UserAccount`, `Features.Users` | `Username` and `Email` are unique | +| `UserCredential` | `Domain.Entities.UserCredential`, `Features.Users` | Argon2id password hash; `IsRevoked`/`RevokedAt` track credential rotation, not JWTs | +| `UserVerification` | `Domain.Entities.UserVerification`, `Features.Users` | Written when a user confirms their email | +| `UserProfile` | `Domain.Entities.UserProfile`, `Features.Users` | 1:1 with `UserAccount`; holds the user's biography; written via `IUserProfileRepository` | +| `UserAvatar` | `Domain.Entities.UserAvatar`, `Features.Users` | Links a `UserProfile` to a `Photo` as its avatar; written via `IUserProfileRepository.SaveAvatarAsync` | +| `BreweryPost` | `Domain.Entities.BreweryPost`, `Features.Breweries` | A user-submitted brewery listing | +| `BreweryPostLocation` | `Domain.Entities.BreweryPostLocation`, `Features.Breweries` | 1:1 with `BreweryPost`; `Coordinates` is a `GEOGRAPHY` column, read via a plain Dapper query that selects `CONVERT(varbinary(max), Coordinates)`, since Dapper can't deserialize SQL Server UDTs directly | +| `City` | `Domain.Entities.City`, `Features.Locations` | Referenced by `BreweryPostLocation.CityID`; written via `ILocationRepository`, read via `ILocationRepository` and `IBreweryRepository`'s location joins, currently written only by `Database.Seed` | +| `StateProvince` | `Domain.Entities.StateProvince`, `Features.Locations` | Referenced by `City.StateProvinceID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins | +| `Country` | `Domain.Entities.Country`, `Features.Locations` | Referenced by `StateProvince.CountryID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins | +| `Photo` | `Domain.Entities.Photo`, `Features.PhotoUpload` | Written via `IPhotoUploadRepository.CreateAsync`, sent as an `UploadPhotoCommand` from other features' handlers; not yet seeded by `Database.Seed` | + +### Tables defined in the schema but not yet wired to a feature slice + +These exist so the schema can support planned functionality, but have no +repository or seed data yet: + +| Table | Purpose (from schema) | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `UserFollow` | One user following another (`CannotFollowOwnAccount` check constraint) | +| `BeerStyle` | A beer style/category; has a `Domain.Entities.BeerStyle` type, but no repository yet | +| `BeerPost` | A beer listing, linked to a `BreweryPost` and `BeerStyle`; `ABV`/`IBU` are constrained (`0-67`, `0-120`); has a `Domain.Entities.BeerPost` type, but no repository yet | +| `BeerPostPhoto` | Links a `BeerPost` to a `Photo` | +| `BeerPostComment` | A comment + 1-5 `Rating` on a `BeerPost` | +| `BreweryPostPhoto` | Links a `BreweryPost` to a `Photo` | + +## Managing Migrations (DbUp) + +Migrations are handled by the `Database.Migrations` container using DbUp. +Scripts are placed in the `scripts/` directory, applied in order, and tracked so +that each one runs only once. + +**Note:** The schema is currently considered volatile. At this stage of +development, fresh database instances are provisioned for both development and +testing, so no production data exists yet. + +### How to Make Schema Changes + +- **Now (no production data):** Update the schema by editing the original DDL + statements directly. +- **Later (with production data):** Add a new, numbered script for any schema + changes. Never edit an existing script in place — DbUp tracks completion by + script name, so modifying an already-applied script means the changes will be + ignored on existing databases. + +## Data seeding + +`Database.Seed` populates a target database from data produced by the C++ +pipeline (see [Pipeline README](../pipeline/README.md)): countries, +state/provinces, and cities (created on demand while resolving each brewery's +location), user accounts, and brewery posts with locations. It doesn't call +every feature slice that has a write path: `Photo`, for example, has a +repository (`Features.PhotoUpload`) but no seed data. + +## Database error handling + +- Optimistic-concurrency conflicts are detected by an `UPDATE` affecting zero + rows against the `RowVersion` column. +- On failure, the repository throws a `Domain.Exceptions` type directly + (`NotFoundException`, `ConflictException`, `ArgumentException`, ...). + `API.Core`'s `GlobalExceptionFilter` maps these to HTTP status codes (404, + 409, 400, ...); an unmapped `SqlException` (for example, a genuine + connectivity or constraint-violation failure) maps to 503. + + - One exception: `Features.Users`' `DapperUserStore` does catch `SqlException` + directly, to detect a duplicate-key race (SQL Server error 2601 or 2627) + when two requests try to verify the same account concurrently; see + `DapperUserStore.IsDuplicateKeyViolation`. diff --git a/docs/web/diagrams/auth/confirm-user.md b/docs/web/diagrams/auth/confirm-user.md new file mode 100644 index 00000000..ee5402c0 --- /dev/null +++ b/docs/web/diagrams/auth/confirm-user.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Confirm User + +```mermaid +--8<-- "web/diagrams/auth/confirm-user.mmd" +``` diff --git a/docs/web/diagrams/auth/confirm-user.mmd b/docs/web/diagrams/auth/confirm-user.mmd new file mode 100644 index 00000000..dc680e76 --- /dev/null +++ b/docs/web/diagrams/auth/confirm-user.mmd @@ -0,0 +1,79 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant ConfirmHandler as ConfirmUserHandler + participant UserMgr as UserManager(ApplicationUser) + participant TokenSvc as TokenService + end + box Infrastructure Layer + participant JWT as JWT Infrastructure + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: POST /api/auth/confirm?token={token} (requires Bearer access token) + activate API + API->>Mediator: Send(ConfirmUserCommand(token)) + activate Mediator + Mediator->>ConfirmHandler: Handle(command) + activate ConfirmHandler + + ConfirmHandler->>TokenSvc: ValidateConfirmationTokenAsync(token) + activate TokenSvc + TokenSvc->>JWT: Validate against confirmation token secret, extract userId + activate JWT + JWT-->>TokenSvc: ValidatedToken or throws + deactivate JWT + TokenSvc-->>ConfirmHandler: ValidatedToken + deactivate TokenSvc + + alt Token invalid or expired + ConfirmHandler->>Mediator: throw UnauthorizedException + Mediator->>API: propagate + API->>User: 401 Unauthorized + else Token valid + ConfirmHandler->>UserMgr: FindByIdAsync(validatedToken.UserId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userId) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id + activate DB + DB-->>UserStore: row or null + deactivate DB + UserStore-->>UserMgr: ApplicationUser or null + deactivate UserStore + UserMgr-->>ConfirmHandler: ApplicationUser or null + deactivate UserMgr + + alt User not found + ConfirmHandler->>Mediator: throw UnauthorizedException "User account not found" + Mediator->>API: propagate + API->>User: 401 Unauthorized + else User found + ConfirmHandler->>UserStore: SetEmailConfirmedAsync(user, true) + Note right of UserStore: IUserEmailStore(ApplicationUser) is injected directly into the handler (not via UserManager), setting the confirmed flag in-memory here + + ConfirmHandler->>UserMgr: UpdateAsync(user) + activate UserMgr + UserMgr->>UserStore: UpdateAsync(user) + activate UserStore + UserStore->>DB: UPDATE UserAccount SET ... + UserStore->>DB: INSERT INTO UserVerification (UserAccountId, VerificationDateTime) + Note right of UserStore: Idempotent — skipped (duplicate-key exception swallowed) if a verification row already exists + UserStore-->>UserMgr: IdentityResult.Success + deactivate UserStore + UserMgr-->>ConfirmHandler: IdentityResult + deactivate UserMgr + + ConfirmHandler-->>Mediator: ConfirmationPayload(user.Id, DateTime.UtcNow) + deactivate ConfirmHandler + Mediator-->>API: ConfirmationPayload + deactivate Mediator + API->>User: 200 OK {message: "User with ID {id} is confirmed.", payload: {...}} + end + end + deactivate API diff --git a/docs/web/diagrams/auth/delete-account.md b/docs/web/diagrams/auth/delete-account.md new file mode 100644 index 00000000..aa4b91e3 --- /dev/null +++ b/docs/web/diagrams/auth/delete-account.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Delete Account + +```mermaid +--8<-- "web/diagrams/auth/delete-account.mmd" +``` diff --git a/docs/web/diagrams/auth/delete-account.mmd b/docs/web/diagrams/auth/delete-account.mmd new file mode 100644 index 00000000..c1788cb2 --- /dev/null +++ b/docs/web/diagrams/auth/delete-account.mmd @@ -0,0 +1,65 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant DeleteAccountHandler + participant UserMgr as UserManager(ApplicationUser) + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: DELETE /api/auth/account (requires Bearer access token) + activate API + API->>Mediator: Send(DeleteAccountCommand(GetAuthenticatedUserId())) + activate Mediator + Mediator->>DeleteAccountHandler: Handle(command) + activate DeleteAccountHandler + + DeleteAccountHandler->>UserMgr: FindByIdAsync(userAccountId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userAccountId) + UserStore-->>UserMgr: ApplicationUser or null + UserMgr-->>DeleteAccountHandler: ApplicationUser or null + deactivate UserMgr + + alt Account not found + DeleteAccountHandler->>Mediator: throw NotFoundException + Mediator->>API: propagate + API->>User: 404 Not Found + else Account found + DeleteAccountHandler->>UserMgr: DeleteAsync(user) + activate UserMgr + UserMgr->>UserStore: DeleteAsync(user) + activate UserStore + UserStore->>DB: DELETE FROM UserAccount WHERE UserAccountID = @Id + Note right of UserStore: UserCredential, UserVerification, and UserProfile cascade on delete
(UserAvatar cascades transitively through UserProfile).
Posts, comments, photos, and follows use ON DELETE NO ACTION
if any reference this account, the DELETE throws a foreign key violation (SQL error 547) + + alt Foreign key violation (dependent records exist) + UserStore-->>UserMgr: IdentityResult.Failed (AccountHasDependentRecords) + else Delete succeeds + UserStore-->>UserMgr: IdentityResult.Success + end + deactivate UserStore + UserMgr-->>DeleteAccountHandler: IdentityResult + deactivate UserMgr + + alt Account has dependent records + DeleteAccountHandler->>Mediator: throw ConflictException
"Account cannot be deleted while it still has associated posts, comments, photos, or follows." + Mediator->>API: propagate + API->>User: 409 Conflict + else !result.Succeeded (other failure) + DeleteAccountHandler->>Mediator: throw ConflictException + Mediator->>API: propagate + API->>User: 409 Conflict + else result.Succeeded + DeleteAccountHandler-->>Mediator: (void) + deactivate DeleteAccountHandler + Mediator-->>API: (void) + deactivate Mediator + API->>User: 200 OK {message: "Account deleted successfully."} + end + end + deactivate API diff --git a/docs/web/diagrams/auth/error-handling.md b/docs/web/diagrams/auth/error-handling.md new file mode 100644 index 00000000..0057ba00 --- /dev/null +++ b/docs/web/diagrams/auth/error-handling.md @@ -0,0 +1,11 @@ +# User Authentication Flow — Error Handling + +Global exception mapping performed by `API.Core`'s `GlobalExceptionFilter`. +Applies uniformly across every authentication endpoint (registration, login, +refresh, confirm, resend, update-username/email/password/profile, +delete-account) — see the per-flow diagrams for where each exception type is +thrown. + +```mermaid +--8<-- "web/diagrams/auth/error-handling.mmd" +``` diff --git a/docs/web/diagrams/auth/error-handling.mmd b/docs/web/diagrams/auth/error-handling.mmd new file mode 100644 index 00000000..e4ef9a36 --- /dev/null +++ b/docs/web/diagrams/auth/error-handling.mmd @@ -0,0 +1,12 @@ +sequenceDiagram + participant API as API Controller + + Note over API: GlobalExceptionFilter catches: + Note over API: FluentValidation.ValidationException → 400 Bad Request {message, errors} (grouped by property) + Note over API: ConflictException → 409 Conflict + Note over API: NotFoundException → 404 Not Found + Note over API: UnauthorizedException → 401 Unauthorized + Note over API: ForbiddenException → 403 Forbidden + Note over API: Domain.Exceptions.ValidationException → 400 Bad Request + Note over API: Microsoft.Data.SqlClient.SqlException → 503 Service Unavailable + Note over API: All others → 500 Internal Server Error diff --git a/docs/web/diagrams/auth/login.md b/docs/web/diagrams/auth/login.md new file mode 100644 index 00000000..07ac61c3 --- /dev/null +++ b/docs/web/diagrams/auth/login.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Login + +```mermaid +--8<-- "web/diagrams/auth/login.mmd" +``` diff --git a/docs/web/diagrams/auth/login.mmd b/docs/web/diagrams/auth/login.mmd new file mode 100644 index 00000000..55d55356 --- /dev/null +++ b/docs/web/diagrams/auth/login.mmd @@ -0,0 +1,93 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant LoginHandler + participant UserMgr as UserManager(ApplicationUser) + participant TokenSvc as TokenService + end + box Infrastructure Layer + participant Argon2 as Argon2PasswordHasher + participant JWT as JWT Infrastructure + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: POST /api/auth/login {username, password} + activate API + API->>Mediator: Send(LoginCommand) + activate Mediator + Note right of Mediator: ValidationBehavior runs LoginValidator — username and password must both be non-empty + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request {message, errors} + else Validation succeeds + Mediator->>LoginHandler: Handle(command) + activate LoginHandler + + LoginHandler->>UserMgr: FindByNameAsync(username) + activate UserMgr + UserMgr->>UserStore: FindByNameAsync(username) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE Username = @Username + activate DB + DB-->>UserStore: UserAccount row or null + deactivate DB + UserStore->>UserStore: Hydrate (active UserCredential.Hash, UserVerification → EmailConfirmed) + UserStore-->>UserMgr: ApplicationUser or null + deactivate UserStore + UserMgr-->>LoginHandler: ApplicationUser or null + deactivate UserMgr + + alt User not found + LoginHandler->>Mediator: throw UnauthorizedException "Invalid username or password." + Mediator->>API: propagate + API->>User: 401 Unauthorized + else User found + LoginHandler->>UserMgr: CheckPasswordAsync(user, password) + activate UserMgr + Note right of UserMgr: HasPasswordAsync short-circuits to false (no active credential hydrated) without calling the hasher — otherwise delegates to Argon2PasswordHasher + UserMgr->>Argon2: VerifyHashedPassword(user, user.PasswordHash, password) + activate Argon2 + Note right of Argon2: Parse stored PHC string ($argon2id$v=19$m=...,t=...,p=...$salt$hash), rehash password with the embedded salt/parameters, constant-time compare (CryptographicOperations.FixedTimeEquals) + Argon2-->>UserMgr: Success/Failed + deactivate Argon2 + UserMgr-->>LoginHandler: true/false + deactivate UserMgr + + alt Password invalid + LoginHandler->>Mediator: throw UnauthorizedException + Mediator->>API: propagate + API->>User: 401 Unauthorized + else Password valid + LoginHandler->>TokenSvc: GenerateAccessToken(user) + activate TokenSvc + TokenSvc->>JWT: GenerateJwt(...) + activate JWT + JWT-->>TokenSvc: Access Token + deactivate JWT + TokenSvc-->>LoginHandler: Access Token + deactivate TokenSvc + + LoginHandler->>TokenSvc: GenerateRefreshToken(user) + activate TokenSvc + TokenSvc->>JWT: GenerateJwt(...) + activate JWT + JWT-->>TokenSvc: Refresh Token + deactivate JWT + TokenSvc-->>LoginHandler: Refresh Token + deactivate TokenSvc + + LoginHandler-->>Mediator: LoginPayload(userAccountId, username, refreshToken, accessToken) + deactivate LoginHandler + Mediator-->>API: LoginPayload + deactivate Mediator + API->>User: 200 OK {message: "Logged in successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/auth/refresh-token.md b/docs/web/diagrams/auth/refresh-token.md new file mode 100644 index 00000000..5a06743d --- /dev/null +++ b/docs/web/diagrams/auth/refresh-token.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Refresh Token + +```mermaid +--8<-- "web/diagrams/auth/refresh-token.mmd" +``` diff --git a/docs/web/diagrams/auth/refresh-token.mmd b/docs/web/diagrams/auth/refresh-token.mmd new file mode 100644 index 00000000..4e23bb4f --- /dev/null +++ b/docs/web/diagrams/auth/refresh-token.mmd @@ -0,0 +1,77 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant RefreshHandler as RefreshTokenHandler + participant UserMgr as UserManager(ApplicationUser) + participant TokenSvc as TokenService + end + box Infrastructure Layer + participant JWT as JWT Infrastructure + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: POST /api/auth/refresh
Header: X-Refresh-Token: {token} + activate API + API->>Mediator: Send(RefreshTokenCommand(refreshToken)) + activate Mediator + Note right of Mediator: ValidationBehavior runs RefreshTokenValidator — refreshToken must be non-empty. Anonymous endpoint, no Bearer token required. + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request {message, errors} + else Validation succeeds + Mediator->>RefreshHandler: Handle(command) + activate RefreshHandler + + RefreshHandler->>TokenSvc: RefreshTokenAsync(refreshToken) + activate TokenSvc + TokenSvc->>JWT: ValidateRefreshTokenAsync(refreshToken) + activate JWT + Note right of JWT: Validates signature/expiry against the refresh token secret, extracts userId/username claims + JWT-->>TokenSvc: ValidatedToken or throws + deactivate JWT + + alt Token invalid or expired + TokenSvc->>RefreshHandler: throw UnauthorizedException + RefreshHandler->>Mediator: propagate + Mediator->>API: propagate + API->>User: 401 Unauthorized + else Token valid + TokenSvc->>UserMgr: FindByIdAsync(validated.UserId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userId) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id + activate DB + DB-->>UserStore: row or null + deactivate DB + UserStore-->>UserMgr: ApplicationUser or null + deactivate UserStore + UserMgr-->>TokenSvc: ApplicationUser or null + deactivate UserMgr + + alt User not found + TokenSvc->>RefreshHandler: throw UnauthorizedException "User account not found" + RefreshHandler->>Mediator: propagate + Mediator->>API: propagate + API->>User: 401 Unauthorized + else User found + TokenSvc->>JWT: GenerateAccessToken(user) + GenerateRefreshToken(user) + JWT-->>TokenSvc: new access + refresh tokens + TokenSvc-->>RefreshHandler: RefreshTokenResult + deactivate TokenSvc + + RefreshHandler-->>Mediator: LoginPayload(userId, username, refreshToken, accessToken) + deactivate RefreshHandler + Mediator-->>API: LoginPayload + deactivate Mediator + API->>User: 200 OK {message: "Token refreshed successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/auth/registration.md b/docs/web/diagrams/auth/registration.md new file mode 100644 index 00000000..cc2c1ac9 --- /dev/null +++ b/docs/web/diagrams/auth/registration.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Registration + +```mermaid +--8<-- "web/diagrams/auth/registration.mmd" +``` diff --git a/docs/web/diagrams/auth/registration.mmd b/docs/web/diagrams/auth/registration.mmd new file mode 100644 index 00000000..9403f83c --- /dev/null +++ b/docs/web/diagrams/auth/registration.mmd @@ -0,0 +1,165 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant RegHandler as RegisterUserHandler + participant UserMgr as UserManager(ApplicationUser) + participant TokenSvc as TokenService + end + box Infrastructure Layer + participant Argon2 as Argon2PasswordHasher + participant JWT as JWT Infrastructure + participant SMTP as Email Provider + participant Template as Template Provider + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + participant ProfileHandler as CreateUserProfileHandler
(Profile slice) + participant EmailDispatcher as EmailDispatcher
(Features.Emails) + end + participant DB as SQL Server + + User->>API: POST /api/auth/register {username, firstName, lastName, email, dateOfBirth, password} + activate API + API->>Mediator: Send(RegisterUserCommand) + activate Mediator + Note right of Mediator: ValidationBehavior runs RegisterUserValidator — username 3-64 chars alphanumeric+[._-], valid email ≤128 chars, password min 8 chars with upper/lower/number/special, DateOfBirth 19+ years old + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request {message, errors} + else Validation succeeds + Mediator->>RegHandler: Handle(command) + activate RegHandler + RegHandler->>RegHandler: user = new UserAccount{...}.ToApplicationUser() + + RegHandler->>UserMgr: CreateAsync(user, password) + activate UserMgr + UserMgr->>Argon2: HashPassword(user, password) + activate Argon2 + Note right of Argon2: Argon2id — salt 16 bytes, memory 64MB, iterations 4, parallelism 2 (fixed, to avoid thread-pool exhaustion), hash output 32 bytes + Argon2-->>UserMgr: PHC string in Argon2id format (algorithm, version, memory/time/parallelism params, salt, hash) + deactivate Argon2 + + UserMgr->>UserStore: SetPasswordHashAsync(user, hash) + Note right of UserStore: user.Id is still empty here — only sets the in-memory hash — CreateAsync writes it to UserCredential itself + + UserMgr->>UserStore: FindByNameAsync(username) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE Username = @Username + activate DB + DB-->>UserStore: null (user doesn't exist) + deactivate DB + deactivate UserStore + + UserMgr->>UserStore: FindByEmailAsync(email) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE Email = @Email + activate DB + DB-->>UserStore: null (email doesn't exist) + deactivate DB + deactivate UserStore + Note right of UserMgr: Default UserManager validators check uniqueness before CreateAsync — DapperUserStore.CreateAsync also guards a genuine race via a caught duplicate-key SqlException (2601/2627) + + UserMgr->>UserStore: CreateAsync(user) + activate UserStore + UserStore->>DB: BEGIN TRANSACTION + activate DB + UserStore->>DB: INSERT INTO UserAccount (...) OUTPUT INSERTED.UserAccountID + DB-->>UserStore: UserAccountId (GUID) + UserStore->>DB: INSERT INTO UserCredential (UserAccountId, Hash) + UserStore->>DB: COMMIT TRANSACTION + deactivate DB + Note right of UserStore: Both statements run via Dapper against a single DbTransaction + + UserStore-->>UserMgr: IdentityResult.Success (user.Id set) + deactivate UserStore + UserMgr-->>RegHandler: IdentityResult + deactivate UserMgr + + alt !result.Succeeded + RegHandler->>Mediator: throw ConflictException (DuplicateUserName/DuplicateEmail → "Username or email already exists", otherwise joined error descriptions) + Mediator->>API: propagate + API->>User: 409 Conflict "Username or email already exists" + else result.Succeeded + RegHandler->>Mediator: Send(CreateUserProfileCommand(user.Id, "")) + activate Mediator + Mediator->>ProfileHandler: Handle(command) + activate ProfileHandler + ProfileHandler->>DB: INSERT INTO UserProfile (UserAccountId, Biography) + activate DB + DB-->>ProfileHandler: UserProfileId + deactivate DB + ProfileHandler-->>Mediator: UserProfileId + deactivate ProfileHandler + Mediator-->>RegHandler: UserProfileId + deactivate Mediator + + RegHandler->>TokenSvc: GenerateAccessToken(user) + activate TokenSvc + TokenSvc->>JWT: GenerateJwt(userId, username, expiry) + activate JWT + Note right of JWT: HS256, expires 1 hour, claims sub/unique_name/iat/jti + JWT->>JWT: Create JWT with claims, sign with secret key + JWT-->>TokenSvc: Access Token + deactivate JWT + TokenSvc-->>RegHandler: Access Token + deactivate TokenSvc + + RegHandler->>TokenSvc: GenerateRefreshToken(user) + activate TokenSvc + TokenSvc->>JWT: GenerateJwt(userId, username, expiry) + activate JWT + Note right of JWT: Expires 21 days (504h), same structure as access token, signed with a separate secret + JWT-->>TokenSvc: Refresh Token + deactivate JWT + TokenSvc-->>RegHandler: Refresh Token + deactivate TokenSvc + + RegHandler->>TokenSvc: GenerateConfirmationToken(user) + activate TokenSvc + TokenSvc->>JWT: GenerateJwt(userId, username, expiry) + activate JWT + Note right of JWT: Expires 30 minutes, signed with a third, separate secret + JWT-->>TokenSvc: Confirmation Token + deactivate JWT + TokenSvc-->>RegHandler: Confirmation Token + deactivate TokenSvc + + RegHandler->>Mediator: Send(SendRegistrationEmailCommand) (Shared.Application cross-slice contract) + activate Mediator + Note right of Mediator: Routed to Features.Emails' SendRegistrationEmailHandler — the second of two cross-slice dispatches in this flow (profile creation is the first) + Mediator->>EmailDispatcher: SendRegistrationEmailAsync(firstName, email, confirmationToken) + activate EmailDispatcher + + EmailDispatcher->>Template: RenderUserRegisteredEmailAsync(firstName, confirmationLink) + activate Template + Note right of Template: Razor component — header/branding, welcome message, confirmation button, footer + Template->>Template: Render Razor component to HTML + Template-->>EmailDispatcher: HTML email content + deactivate Template + + EmailDispatcher->>SMTP: SendAsync(email, subject, body) + activate SMTP + Note right of SMTP: MailKit — host/port from SMTP_HOST/SMTP_PORT, StartTLS when SMTP_USE_SSL=true, optional username/password auth + SMTP->>SMTP: Build MIME message, connect, authenticate, send, disconnect + SMTP-->>EmailDispatcher: Success / throws + deactivate SMTP + deactivate EmailDispatcher + deactivate Mediator + + alt Email sent successfully + RegHandler->>RegHandler: emailSent = true + else mediator.Send throws + RegHandler->>RegHandler: catch(Exception), log to stderr, emailSent = false + end + + RegHandler-->>Mediator: RegistrationPayload(userAccountId, username, refreshToken, accessToken, confirmationEmailSent) + deactivate RegHandler + Mediator-->>API: RegistrationPayload + deactivate Mediator + API->>User: 201 Created {message: "User registered successfully.", payload: {...}} + end + end + deactivate API diff --git a/docs/web/diagrams/auth/resend-confirmation.md b/docs/web/diagrams/auth/resend-confirmation.md new file mode 100644 index 00000000..ccdcfd4f --- /dev/null +++ b/docs/web/diagrams/auth/resend-confirmation.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Resend Confirmation Email + +```mermaid +--8<-- "web/diagrams/auth/resend-confirmation.mmd" +``` diff --git a/docs/web/diagrams/auth/resend-confirmation.mmd b/docs/web/diagrams/auth/resend-confirmation.mmd new file mode 100644 index 00000000..cd985da3 --- /dev/null +++ b/docs/web/diagrams/auth/resend-confirmation.mmd @@ -0,0 +1,71 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant ResendHandler as ResendConfirmationEmailHandler + participant UserMgr as UserManager(ApplicationUser) + participant TokenSvc as TokenService + end + box Infrastructure Layer + participant SMTP as Email Provider + participant Template as Template Provider + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + participant EmailDispatcher as EmailDispatcher
(Features.Emails) + end + participant DB as SQL Server + + User->>API: POST /api/auth/confirm/resend?userId={id} (requires Bearer access token) + activate API + API->>Mediator: Send(ResendConfirmationEmailCommand(userId)) + activate Mediator + Mediator->>ResendHandler: Handle(command) + activate ResendHandler + + ResendHandler->>UserMgr: FindByIdAsync(userId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userId) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id + activate DB + DB-->>UserStore: row or null + deactivate DB + UserStore-->>UserMgr: ApplicationUser or null + deactivate UserStore + UserMgr-->>ResendHandler: ApplicationUser or null + deactivate UserMgr + + alt User not found + ResendHandler->>ResendHandler: return silently (prevents user enumeration) + else User found + ResendHandler->>UserMgr: IsEmailConfirmedAsync(user) + UserMgr-->>ResendHandler: bool + + alt Already confirmed + ResendHandler->>ResendHandler: return silently (no-op) + else Not yet confirmed + ResendHandler->>TokenSvc: GenerateConfirmationToken(user) + activate TokenSvc + TokenSvc-->>ResendHandler: confirmation token + deactivate TokenSvc + + ResendHandler->>Mediator: Send(SendResendConfirmationEmailCommand) + activate Mediator + Mediator->>EmailDispatcher: SendResendConfirmationEmailAsync(firstName, email, confirmationToken) + activate EmailDispatcher + EmailDispatcher->>Template: Render email (same Razor pipeline as registration) + EmailDispatcher->>SMTP: SendAsync(email, subject, body) + deactivate EmailDispatcher + deactivate Mediator + end + end + + ResendHandler-->>Mediator: (void) + deactivate ResendHandler + Mediator-->>API: (void) + deactivate Mediator + API->>User: 200 OK {message: "confirmation email has been resent"} + Note right of API: Always 200 OK regardless of outcome above — same user-enumeration precaution as a not-found user or an already-confirmed account + deactivate API diff --git a/docs/web/diagrams/auth/update-email.md b/docs/web/diagrams/auth/update-email.md new file mode 100644 index 00000000..67b1a79e --- /dev/null +++ b/docs/web/diagrams/auth/update-email.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Update Email + +```mermaid +--8<-- "web/diagrams/auth/update-email.mmd" +``` diff --git a/docs/web/diagrams/auth/update-email.mmd b/docs/web/diagrams/auth/update-email.mmd new file mode 100644 index 00000000..14f029e4 --- /dev/null +++ b/docs/web/diagrams/auth/update-email.mmd @@ -0,0 +1,64 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant UpdateEmailHandler + participant UserMgr as UserManager(ApplicationUser) + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: PATCH /api/auth/email {newEmail} (requires Bearer access token) + activate API + API->>Mediator: Send(UpdateEmailCommand(GetAuthenticatedUserId(), newEmail)) + activate Mediator + Note right of Mediator: ValidationBehavior runs UpdateEmailValidator (same format rules as registration) + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request + else Validation succeeds + Mediator->>UpdateEmailHandler: Handle(command) + activate UpdateEmailHandler + + UpdateEmailHandler->>UserMgr: FindByIdAsync(userAccountId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userAccountId) + UserStore->>DB: SELECT ... + UserStore-->>UserMgr: ApplicationUser or null + UserMgr-->>UpdateEmailHandler: ApplicationUser or null + deactivate UserMgr + + alt Account not found + UpdateEmailHandler->>Mediator: throw NotFoundException + Mediator->>API: propagate + API->>User: 404 Not Found + else Account found + UpdateEmailHandler->>UserMgr: SetEmailAsync(user, newEmail) + activate UserMgr + UserMgr->>UserStore: FindByEmailAsync(newEmail) + Note right of UserMgr: Uniqueness check. Also resets user.EmailConfirmed = false — caller must re-confirm via POST /api/auth/confirm/resend + UserMgr->>UserStore: UpdateAsync(user) + UserStore->>DB: UPDATE UserAccount SET Email = @Email, ... + UserStore->>DB: DELETE FROM UserVerification WHERE UserAccountID = @Id + UserStore-->>UserMgr: IdentityResult + UserMgr-->>UpdateEmailHandler: IdentityResult + deactivate UserMgr + + alt !result.Succeeded + UpdateEmailHandler->>Mediator: throw ConflictException (DuplicateEmail → "Email address already exists") + Mediator->>API: propagate + API->>User: 409 Conflict + else result.Succeeded + UpdateEmailHandler-->>Mediator: UpdateEmailPayload(user.Id, user.Email, user.EmailConfirmed = false) + deactivate UpdateEmailHandler + Mediator-->>API: UpdateEmailPayload + deactivate Mediator + API->>User: 200 OK {message: "Email updated successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/auth/update-password.md b/docs/web/diagrams/auth/update-password.md new file mode 100644 index 00000000..9d538077 --- /dev/null +++ b/docs/web/diagrams/auth/update-password.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Update Password + +```mermaid +--8<-- "web/diagrams/auth/update-password.mmd" +``` diff --git a/docs/web/diagrams/auth/update-password.mmd b/docs/web/diagrams/auth/update-password.mmd new file mode 100644 index 00000000..ba09c5d0 --- /dev/null +++ b/docs/web/diagrams/auth/update-password.mmd @@ -0,0 +1,84 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant UpdatePasswordHandler + participant UserMgr as UserManager(ApplicationUser) + end + box Infrastructure Layer + participant Argon2 as Argon2PasswordHasher + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: PATCH /api/auth/password {currentPassword, newPassword} (requires Bearer access token) + activate API + API->>Mediator: Send(UpdatePasswordCommand(GetAuthenticatedUserId(), currentPassword, newPassword)) + activate Mediator + Note right of Mediator: ValidationBehavior runs UpdatePasswordValidator (same password strength rules as registration) + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request + else Validation succeeds + Mediator->>UpdatePasswordHandler: Handle(command) + activate UpdatePasswordHandler + + UpdatePasswordHandler->>UserMgr: FindByIdAsync(userAccountId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userAccountId) + UserStore-->>UserMgr: ApplicationUser or null + UserMgr-->>UpdatePasswordHandler: ApplicationUser or null + deactivate UserMgr + + alt Account not found + UpdatePasswordHandler->>Mediator: throw NotFoundException + Mediator->>API: propagate + API->>User: 404 Not Found + else Account found + UpdatePasswordHandler->>UserMgr: ChangePasswordAsync(user, currentPassword, newPassword) + activate UserMgr + UserMgr->>Argon2: VerifyHashedPassword(user, user.PasswordHash, currentPassword) + activate Argon2 + Argon2-->>UserMgr: Success/Failed + deactivate Argon2 + + alt Current password mismatch + UserMgr-->>UpdatePasswordHandler: IdentityResult.Failed (PasswordMismatch) + else Current password matches + UserMgr->>Argon2: HashPassword(user, newPassword) + activate Argon2 + Argon2-->>UserMgr: new hash + deactivate Argon2 + UserMgr->>UserStore: SetPasswordHashAsync(user, newHash) + activate UserStore + UserStore->>DB: UPDATE UserCredential SET IsRevoked = 1, RevokedAt = GETDATE() WHERE UserAccountId = @Id AND IsRevoked = 0 + UserStore->>DB: INSERT INTO UserCredential (UserAccountId, Hash) + Note right of UserStore: Revoke-then-insert runs inside one DbTransaction (credential rotation) + UserStore-->>UserMgr: (void) + deactivate UserStore + UserMgr-->>UpdatePasswordHandler: IdentityResult.Success + end + deactivate UserMgr + + alt Current password is incorrect + UpdatePasswordHandler->>Mediator: throw UnauthorizedException "Current password is incorrect" + Mediator->>API: propagate + API->>User: 401 Unauthorized + else !result.Succeeded (other failure) + UpdatePasswordHandler->>Mediator: throw ConflictException + Mediator->>API: propagate + API->>User: 409 Conflict + else result.Succeeded + UpdatePasswordHandler-->>Mediator: UpdatePasswordPayload(user.Id, DateTime.UtcNow) + deactivate UpdatePasswordHandler + Mediator-->>API: UpdatePasswordPayload + deactivate Mediator + API->>User: 200 OK {message: "Password updated successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/auth/update-profile.md b/docs/web/diagrams/auth/update-profile.md new file mode 100644 index 00000000..1e2bdd4b --- /dev/null +++ b/docs/web/diagrams/auth/update-profile.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Update Profile + +```mermaid +--8<-- "web/diagrams/auth/update-profile.mmd" +``` diff --git a/docs/web/diagrams/auth/update-profile.mmd b/docs/web/diagrams/auth/update-profile.mmd new file mode 100644 index 00000000..4180f9c4 --- /dev/null +++ b/docs/web/diagrams/auth/update-profile.mmd @@ -0,0 +1,63 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant UpdateProfileHandler + participant UserMgr as UserManager(ApplicationUser) + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: PATCH /api/auth/profile {firstName, lastName, dateOfBirth} (requires Bearer access token) + activate API + API->>Mediator: Send(UpdateProfileCommand(GetAuthenticatedUserId(), firstName, lastName, dateOfBirth)) + activate Mediator + Note right of Mediator: ValidationBehavior runs UpdateProfileValidator (same name/DateOfBirth rules as registration) + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request + else Validation succeeds + Mediator->>UpdateProfileHandler: Handle(command) + activate UpdateProfileHandler + + UpdateProfileHandler->>UserMgr: FindByIdAsync(userAccountId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userAccountId) + UserStore-->>UserMgr: ApplicationUser or null + UserMgr-->>UpdateProfileHandler: ApplicationUser or null + deactivate UserMgr + + alt Account not found + UpdateProfileHandler->>Mediator: throw NotFoundException + Mediator->>API: propagate + API->>User: 404 Not Found + else Account found + UpdateProfileHandler->>UpdateProfileHandler: user.FirstName = ...
user.LastName = ...
user.DateOfBirth = ... + Note right of UpdateProfileHandler: First/last name and date of birth aren't tracked by any Identity store interface — mutated directly on the ApplicationUser, then saved via UserManager.UpdateAsync + + UpdateProfileHandler->>UserMgr: UpdateAsync(user) + activate UserMgr + UserMgr->>UserStore: UpdateAsync(user) + UserStore->>DB: UPDATE UserAccount SET FirstName = @FirstName, LastName = @LastName, DateOfBirth = @DateOfBirth, ... + UserStore-->>UserMgr: IdentityResult + UserMgr-->>UpdateProfileHandler: IdentityResult + deactivate UserMgr + + alt !result.Succeeded + UpdateProfileHandler->>Mediator: throw ConflictException + Mediator->>API: propagate + API->>User: 409 Conflict + else result.Succeeded + UpdateProfileHandler-->>Mediator: UpdateProfilePayload(user.Id, firstName, lastName, dateOfBirth) + deactivate UpdateProfileHandler + Mediator-->>API: UpdateProfilePayload + deactivate Mediator + API->>User: 200 OK {message: "Profile updated successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/auth/update-username.md b/docs/web/diagrams/auth/update-username.md new file mode 100644 index 00000000..6c841334 --- /dev/null +++ b/docs/web/diagrams/auth/update-username.md @@ -0,0 +1,5 @@ +# User Authentication Flow — Update Username + +```mermaid +--8<-- "web/diagrams/auth/update-username.mmd" +``` diff --git a/docs/web/diagrams/auth/update-username.mmd b/docs/web/diagrams/auth/update-username.mmd new file mode 100644 index 00000000..d25c075d --- /dev/null +++ b/docs/web/diagrams/auth/update-username.mmd @@ -0,0 +1,68 @@ +sequenceDiagram + actor User + participant API as API Controller + participant Mediator as MediatR pipeline
(ValidationBehavior) + box Handler Layer + participant UpdateUsernameHandler + participant UserMgr as UserManager(ApplicationUser) + end + box Repository / Cross-slice + participant UserStore as DapperUserStore
(IUserStore/IUserPasswordStore/IUserEmailStore) + end + participant DB as SQL Server + + User->>API: PATCH /api/auth/username {newUsername} (requires Bearer access token) + activate API + API->>Mediator: Send(UpdateUsernameCommand(GetAuthenticatedUserId(), newUsername)) + activate Mediator + Note right of Mediator: ValidationBehavior runs UpdateUsernameValidator (same format rules as registration) + + alt Validation fails + Mediator->>API: throws FluentValidation.ValidationException + API->>User: 400 Bad Request + else Validation succeeds + Mediator->>UpdateUsernameHandler: Handle(command) + activate UpdateUsernameHandler + + UpdateUsernameHandler->>UserMgr: FindByIdAsync(userAccountId) + activate UserMgr + UserMgr->>UserStore: FindByIdAsync(userAccountId) + activate UserStore + UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id + activate DB + DB-->>UserStore: row or null + deactivate DB + UserStore-->>UserMgr: ApplicationUser or null + deactivate UserStore + UserMgr-->>UpdateUsernameHandler: ApplicationUser or null + deactivate UserMgr + + alt Account not found + UpdateUsernameHandler->>Mediator: throw NotFoundException + Mediator->>API: propagate + API->>User: 404 Not Found + else Account found + UpdateUsernameHandler->>UserMgr: SetUserNameAsync(user, newUsername) + activate UserMgr + UserMgr->>UserStore: FindByNameAsync(newUsername) + Note right of UserMgr: Uniqueness check before the UPDATE + UserMgr->>UserStore: UpdateAsync(user) + UserStore->>DB: UPDATE UserAccount SET Username = @UserName, ... + UserStore-->>UserMgr: IdentityResult + UserMgr-->>UpdateUsernameHandler: IdentityResult + deactivate UserMgr + + alt !result.Succeeded + UpdateUsernameHandler->>Mediator: throw ConflictException (DuplicateUserName → "Username already exists") + Mediator->>API: propagate + API->>User: 409 Conflict + else result.Succeeded + UpdateUsernameHandler-->>Mediator: UpdateUsernamePayload(user.Id, user.UserName) + deactivate UpdateUsernameHandler + Mediator-->>API: UpdateUsernamePayload + deactivate Mediator + API->>User: 200 OK {message: "Username updated successfully.", payload: {...}} + end + end + end + deactivate API diff --git a/docs/web/diagrams/database-schema.md b/docs/web/diagrams/database-schema.md new file mode 100644 index 00000000..fb4e219a --- /dev/null +++ b/docs/web/diagrams/database-schema.md @@ -0,0 +1,40 @@ +# Database Schema + +Five diagrams, one per schema category — mirrors the `identity` / `media` / +`location` / `brewery` / `beer` schema split. Every table also has a +`RowVersion ROWVERSION` column (optimistic concurrency), omitted below for +brevity. Tables marked `(*)` have no repository/feature slice yet. + +Boxes with no attribute list (e.g. `Photo` in the identity diagram) are owned by +a different schema and shown only to keep the relationship visible; see the +diagram named after that schema for its full definition. + +## Identity (`identity`) + +```mermaid +--8<-- "web/diagrams/database-schema/identity.mmd" +``` + +## Media (`media`) + +```mermaid +--8<-- "web/diagrams/database-schema/media.mmd" +``` + +## Location (`location`) + +```mermaid +--8<-- "web/diagrams/database-schema/location.mmd" +``` + +## Brewery (`brewery`) + +```mermaid +--8<-- "web/diagrams/database-schema/brewery.mmd" +``` + +## Beer (`beer`) + +```mermaid +--8<-- "web/diagrams/database-schema/beer.mmd" +``` diff --git a/docs/web/diagrams/database-schema/beer.mmd b/docs/web/diagrams/database-schema/beer.mmd new file mode 100644 index 00000000..f0e7b387 --- /dev/null +++ b/docs/web/diagrams/database-schema/beer.mmd @@ -0,0 +1,38 @@ +erDiagram + BeerStyle ||--o{ Beer : "categorizes" + Beer ||--o{ BeerPhoto : "has photos" + Beer ||--o{ BeerComment : "has comments" + + %% Cross-schema: defined in full in identity.mmd / brewery.mmd / media.mmd + User ||--o{ Beer : "posts" + Brewery ||--o{ Beer : "brews" + User ||--o{ BeerComment : "comments" + + BeerStyle { + uniqueidentifier BeerStyleID PK + nvarchar StyleName UK + nvarchar Description "nullable" + } + + Beer { + uniqueidentifier BeerPostID PK + nvarchar Name + nvarchar Description + decimal ABV + int IBU + uniqueidentifier PostedByID FK + uniqueidentifier BeerStyleID FK + uniqueidentifier BrewedByID FK + datetime CreatedAt + datetime UpdatedAt "nullable" + } + + BeerComment { + uniqueidentifier BeerPostCommentID PK + nvarchar Comment + uniqueidentifier BeerPostID FK + uniqueidentifier CommentedByID FK + int Rating + datetime CreatedAt + datetime UpdatedAt "nullable" + } diff --git a/docs/web/diagrams/database-schema/brewery.mmd b/docs/web/diagrams/database-schema/brewery.mmd new file mode 100644 index 00000000..1b831d0b --- /dev/null +++ b/docs/web/diagrams/database-schema/brewery.mmd @@ -0,0 +1,16 @@ +erDiagram + Brewery ||--o| BreweryLoc : "has location" + Brewery ||--o{ BreweryPhoto : "has photos" + Brewery ||--o{ Beer : "brews" + + %% Cross-schema: defined in full in identity.mmd / location.mmd / media.mmd / beer.mmd + User ||--o{ Brewery : "posts" + + Brewery { + uniqueidentifier BreweryPostID PK + nvarchar BreweryName + uniqueidentifier PostedByID FK + nvarchar Description + datetime CreatedAt + datetime UpdatedAt "nullable" + } diff --git a/docs/web/diagrams/database-schema/identity.mmd b/docs/web/diagrams/database-schema/identity.mmd new file mode 100644 index 00000000..38395a92 --- /dev/null +++ b/docs/web/diagrams/database-schema/identity.mmd @@ -0,0 +1,53 @@ +erDiagram + User ||--o{ Cred : "has credentials" + User ||--o| Verify : "has verification" + User ||--o| Profile : "has profile" + User ||--o{ Follow : "follows" + User ||--o{ Follow : "followed by" + + %% Cross-schema: defined in full in media.mmd / brewery.mmd / beer.mmd + Profile ||--o{ Avatar : "has avatar history" + User ||--o{ Photo : "uploads" + User ||--o{ Brewery : "posts" + User ||--o{ Beer : "posts" + User ||--o{ BeerComment : "comments" + + User { + uniqueidentifier UserAccountID PK + varchar Username UK + nvarchar FirstName + nvarchar LastName + varchar Email UK + datetime CreatedAt + datetime UpdatedAt "nullable" + date DateOfBirth + } + + Cred { + uniqueidentifier UserCredentialID PK + uniqueidentifier UserAccountID FK + datetime CreatedAt + datetime Expiry + nvarchar Hash + bit IsRevoked + datetime RevokedAt "nullable" + } + + Verify { + uniqueidentifier UserVerificationID PK + uniqueidentifier UserAccountID "FK, UK" + datetime VerificationDateTime + } + + Profile { + uniqueidentifier UserProfileID PK + uniqueidentifier UserAccountID "FK, UK" + nvarchar Biography + } + + Follow { + uniqueidentifier UserFollowID PK + uniqueidentifier UserAccountID FK + uniqueidentifier FollowingID FK + datetime CreatedAt + } diff --git a/docs/web/diagrams/database-schema/location.mmd b/docs/web/diagrams/database-schema/location.mmd new file mode 100644 index 00000000..4840a233 --- /dev/null +++ b/docs/web/diagrams/database-schema/location.mmd @@ -0,0 +1,36 @@ +erDiagram + Country ||--o{ StateProvince : "has" + StateProvince ||--o{ City : "has" + City ||--o{ BreweryLoc : "located in" + + %% Cross-schema: defined in full in brewery.mmd + Brewery ||--o| BreweryLoc : "has location" + + Country { + uniqueidentifier CountryID PK + nvarchar CountryName + char ISO3166_1 UK + } + + StateProvince { + uniqueidentifier StateProvinceID PK + nvarchar StateProvinceName + char ISO3166_2 UK + uniqueidentifier CountryID FK + } + + City { + uniqueidentifier CityID PK + nvarchar CityName + uniqueidentifier StateProvinceID FK + } + + BreweryLoc { + uniqueidentifier BreweryPostLocationID PK + uniqueidentifier BreweryPostID "FK, UK" + nvarchar AddressLine1 + nvarchar AddressLine2 "nullable" + nvarchar PostalCode + uniqueidentifier CityID FK + geography Coordinates "nullable" + } diff --git a/docs/web/diagrams/database-schema/media.mmd b/docs/web/diagrams/database-schema/media.mmd new file mode 100644 index 00000000..4b006716 --- /dev/null +++ b/docs/web/diagrams/database-schema/media.mmd @@ -0,0 +1,39 @@ +erDiagram + Avatar }o--|| Photo : "uses photo" + Photo ||--o{ BreweryPhoto : "used in" + Photo ||--o{ BeerPhoto : "used in" + + %% Cross-schema: defined in full in identity.mmd / brewery.mmd / beer.mmd + Profile ||--o{ Avatar : "has avatar history" + User ||--o{ Photo : "uploads" + Brewery ||--o{ BreweryPhoto : "has photos" + Beer ||--o{ BeerPhoto : "has photos" + + Photo { + uniqueidentifier PhotoID PK + nvarchar Hyperlink "nullable" + uniqueidentifier UploadedByID FK + datetime UploadedAt + } + + Avatar { + uniqueidentifier UserAvatarID PK + uniqueidentifier UserProfileID FK + uniqueidentifier PhotoID FK + datetime2 ValidFrom + datetime2 ValidTo "nullable; at most one row per UserProfileID has ValidTo IS NULL (active avatar)" + } + + BreweryPhoto { + uniqueidentifier BreweryPostPhotoID PK + uniqueidentifier BreweryPostID FK + uniqueidentifier PhotoID FK + datetime LinkedAt + } + + BeerPhoto { + uniqueidentifier BeerPostPhotoID PK + uniqueidentifier BeerPostID FK + uniqueidentifier PhotoID FK + datetime LinkedAt + } diff --git a/docs/web/diagrams/deployment.md b/docs/web/diagrams/deployment.md new file mode 100644 index 00000000..ec52c4c9 --- /dev/null +++ b/docs/web/diagrams/deployment.md @@ -0,0 +1,48 @@ +# Docker Deployment Architecture + +Five diagrams, one per Compose file — mixing all five networks in one graph made +it impossible to tell which containers actually talk to each other. Solid arrows +are real container traffic; dashed arrows are `depends_on` startup gating with +no data flowing over them. + +## Development (`docker-compose.dev.yaml`) + +```mermaid +--8<-- "web/diagrams/deployment/dev.mmd" +``` + +## Test (`docker-compose.test.yaml`) + +```mermaid +--8<-- "web/diagrams/deployment/test.mmd" +``` + +## Production (`docker-compose.prod.yaml`) + +```mermaid +--8<-- "web/diagrams/deployment/prod.mmd" +``` + +## Database-only (`docker-compose.db.yaml`) + +```mermaid +--8<-- "web/diagrams/deployment/db-only.mmd" +``` + +## Minimal (`docker-compose.min.yaml`) + +```mermaid +--8<-- "web/diagrams/deployment/minimal.mmd" +``` + +## Notes + +| Environment | Details | +| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Dev network** (`devnet`, bridge) | Internal DNS resolves service names such as `sqlserver` and `api.core`. Volumes: `sqlserverdata-dev`, `nuget-cache-dev`, `seaweedfsdata-dev`. Startup order: SQL Server (health check) → migrations (run-once) → seed (run-once, depends on `seaweedfs`) → API (long-running, depends on seed). | +| **Test network** (`testnet`, bridge) | All test components are isolated; fresh DB instance each run (`CLEAR_DATABASE=true`, ephemeral `sqlserverdata-test` / `seaweedfsdata-test` volumes). `api.specs`, `unit.tests`, and `frontend.tests` all run as separate containers, in parallel once seeding completes; results are aggregated. | +| **Prod network** (`prodnet`, bridge) | Isolated from dev/test networks. Volumes: `sqlserverdata-prod`, `nuget-cache-prod`. No seed step in production. | +| **Database-only environment** | Same devnet volumes/images as `docker-compose.dev.yaml`, minus `api.core`, `frontend`, and `mailpit` — for working against the database (and object storage, since `seaweedfs` is still included) without running the API or website. | +| **Minimal environment** | Database + migrations + seed + `seaweedfs` + mailpit, no API or frontend container — for frontend development against an `API.Core` process run outside Docker. | +| `unit.tests` | Builds `Dockerfile.tests`, which restores/builds against `Biergarten.API.slnx` and then runs `dotnet test` for every `Features/*.Tests` project in turn (`Features.Users.Tests`, `Features.Breweries.Tests`, `Features.Emails.Tests`, `Features.PhotoUpload.Tests`), mocking each handler's repository/service dependencies with Moq — no real database needed. | +| `.trx` results | Readable by Visual Studio, Azure DevOps, and GitHub Actions. | diff --git a/docs/web/diagrams/deployment/db-only.mmd b/docs/web/diagrams/deployment/db-only.mmd new file mode 100644 index 00000000..dff55fc1 --- /dev/null +++ b/docs/web/diagrams/deployment/db-only.mmd @@ -0,0 +1,13 @@ +sequenceDiagram + box Database-only Environment (docker-compose.db.yaml) + participant DbOnlyDB as SQL Server
:1434 + participant DbOnlySeaweedFS as seaweedfs
S3-compatible object storage + participant DbOnlyMig as Migrations (run-once) + participant DbOnlySeed as Seed (run-once) + end + + DbOnlyMig-->>DbOnlyDB: depends_on + DbOnlyMig->>DbOnlyDB: Run migrations + DbOnlySeed-->>DbOnlyMig: depends_on + DbOnlySeed-->>DbOnlySeaweedFS: depends_on + DbOnlySeed->>DbOnlyDB: Seed data diff --git a/docs/web/diagrams/deployment/dev.mmd b/docs/web/diagrams/deployment/dev.mmd new file mode 100644 index 00000000..6d37734c --- /dev/null +++ b/docs/web/diagrams/deployment/dev.mmd @@ -0,0 +1,20 @@ +sequenceDiagram + actor Developer + participant Host as Docker Host + box Development Environment (docker-compose.dev.yaml) + participant DevDB as SQL Server 2022
Biergarten Database
:1434 + participant DevMig as Migrations (run-once)
Database.Migrations + participant DevSeed as Seed (run-once)
Database.Seed + participant DevAPI as API Container
ASP.NET Core 10
:8080 HTTP / :8081 HTTPS + participant DevMailpit as Mailpit
:8025 Web UI / :1025 SMTP + end + + Developer->>Host: docker compose up + DevMig-->>DevDB: depends_on (healthy) + DevMig->>DevDB: Run migrations + DevSeed-->>DevMig: depends_on + DevSeed->>DevDB: Seed data + DevAPI-->>DevSeed: depends_on + DevAPI->>DevDB: Connect and serve + DevAPI-->>DevMailpit: SMTP_HOST/PORT + Host->>DevAPI: http://localhost:8080 diff --git a/docs/web/diagrams/deployment/minimal.mmd b/docs/web/diagrams/deployment/minimal.mmd new file mode 100644 index 00000000..2d508544 --- /dev/null +++ b/docs/web/diagrams/deployment/minimal.mmd @@ -0,0 +1,15 @@ +sequenceDiagram + box Minimal Environment (docker-compose.min.yaml) + participant MinDB as SQL Server
:1433 + participant MinSeaweedFS as seaweedfs
S3-compatible object storage + participant MinMig as Migrations (run-once) + participant MinSeed as Seed (run-once) + participant MinMailpit as Mailpit
:8025 / :1025 + end + + MinMig-->>MinDB: depends_on + MinMig->>MinDB: Run migrations + MinSeed-->>MinMig: depends_on + MinSeed-->>MinSeaweedFS: depends_on + MinSeed->>MinDB: Seed data + Note over MinDB,MinMailpit: consumed by an externally run API.Core process (not shown) diff --git a/docs/web/diagrams/deployment/prod.mmd b/docs/web/diagrams/deployment/prod.mmd new file mode 100644 index 00000000..6c267b92 --- /dev/null +++ b/docs/web/diagrams/deployment/prod.mmd @@ -0,0 +1,11 @@ +sequenceDiagram + box Production Environment (docker-compose.prod.yaml) + participant ProdDB as SQL Server 2022
Biergarten Database
no host port published + participant ProdMig as Migrations (run-once) + participant ProdAPI as API Container
ASP.NET Core 10
:8080 HTTP / :8081 HTTPS + end + + ProdMig-->>ProdDB: depends_on + ProdMig->>ProdDB: Run migrations + ProdAPI-->>ProdDB: depends_on: sql-server healthy (not migrations/seed) + ProdAPI->>ProdDB: Connect and serve diff --git a/docs/web/diagrams/deployment/test.mmd b/docs/web/diagrams/deployment/test.mmd new file mode 100644 index 00000000..2e6089d2 --- /dev/null +++ b/docs/web/diagrams/deployment/test.mmd @@ -0,0 +1,29 @@ +sequenceDiagram + box Test Environment (docker-compose.test.yaml) + participant TestDB as SQL Server, isolated instance
Biergarten Test Database
no host port published + participant TestSeaweedFS as seaweedfs
S3-compatible object storage + participant TestMig as Migrations (test) + participant TestSeed as Seed (test)
minimal: test.user only + participant Specs as API.Specs
Reqnroll + xUnit
(integration tests) + participant UnitTests as unit.tests
xUnit + Moq
Features.*.Tests, one container + participant FrontendTests as frontend.tests
Playwright/Vitest + end + participant Results as test-results/ (mounted volume) + + TestMig-->>TestDB: depends_on + TestMig->>TestDB: Migrate + TestSeed-->>TestMig: depends_on + TestSeed-->>TestSeaweedFS: depends_on + TestSeed->>TestDB: Seed + + par Specs + Specs-->>TestSeed: depends_on + Specs->>TestDB: Integration test + Specs->>Results: Export TRX + and unit.tests + UnitTests-->>TestSeed: depends_on + UnitTests-->>TestDB: depends_on (startup ordering only, no real connection) + UnitTests->>Results: Export TRX, one per slice + and frontend.tests + FrontendTests->>Results: Export results + end diff --git a/docs/web/diagrams/vertical-slice.md b/docs/web/diagrams/vertical-slice.md new file mode 100644 index 00000000..69d4dacb --- /dev/null +++ b/docs/web/diagrams/vertical-slice.md @@ -0,0 +1,103 @@ +# The Biergarten App — Vertical Slice Architecture + +Two diagrams, split by concern: which host wires up which slice at startup, and +which projects each slice actually compiles against. Mixing DI wiring and +compile-time references in one graph is what made the old diagram unreadable. + +## Host wiring + +Each host (`API.Core`, `Database.Seed`) decides for itself which `Features.*` +assemblies to register — there's no shared bootstrap. `Database.Migrations` +doesn't wire up any slice; it applies schema directly. + +```mermaid +--8<-- "web/diagrams/vertical-slice/host-wiring.mmd" +``` + +## Project reference dependencies + +Solid arrows are compile-time project references. Dashed arrows are the two +places slices talk to each other without a direct reference: the email MediatR +contract routed through `Shared.Application`, and the one place that pattern is +broken outright. Every DB-backed slice depends on the same three foundation +projects (`Shared.Contracts`, `Domain.Entities`, `Database.Connection`) — drawn +as one box instead of fifteen separate arrows; see the notes for what else each +of those foundation projects pulls in. + +```mermaid +--8<-- "web/diagrams/vertical-slice/project-references.mmd" +``` + +## Notes + +- **API.Core**: Program.cs wiring only + - MediatR + `AddApplicationPart` per HTTP-exposed `Features.*` assembly + - Swagger/OpenAPI, health checks, JWT auth middleware, global exception filter + - Only `Features.Users` and `Features.Breweries` get `AddApplicationPart` + (controllers) + - `Features.PhotoUpload` and `Features.Emails` scanned for MediatR handlers + only + - `Features.Locations` and `Features.Beers` not referenced by `API.Core` at + all + +- **Feature slices**: each owns its own controller (where present), + commands/queries + handlers, validators, and Dapper repository + - `Features.Users` covers auth (`Commands/Authentication`), account management + (`Commands/Account`), and profile/avatar (`Commands/Profile`) + - internally namespaced `Features.Auth`, despite project name + `Features.Users` + - `Features.Emails` and `Features.PhotoUpload` have no controller — invoked + only via MediatR commands sent from other slices + - `Features.Locations` has no controller; one MediatR handler + (`GetOrCreateCityHandler`) registered by `Database.Seed` only — `API.Core` + never scans it + - `Features.Beers` is scaffolded (project + references only, no source files + yet) + +- **The one exception to slice isolation**: `Features.Users` takes a direct + project reference to `Features.PhotoUpload` + - sends its `UploadPhotoCommand` via MediatR from `UploadAvatarHandler` + - unlike the email flow, the command type lives in the target slice itself, + not in `Shared.Application` + +- **Shared.Application**: + - `ValidationBehavior` (MediatR pipeline) + - Cross-slice email commands (`SendRegistrationEmailCommand`, + `SendResendConfirmationEmailCommand`) + - MediatR command contract (not project reference) + - Every slice (including `Features.Emails`) references it for the pipeline + behavior alone + +- **Domain.Entities**: + - `UserAccount`, `UserCredential`, `UserVerification`, `UserProfile`, + `UserAvatar` + - `BreweryPost`, `BreweryPostLocation` + - `City`, `StateProvince`, `Country` + - `BeerPost`, `BeerStyle`, `Photo` (schema tables exist for + `BeerPost`/`BeerStyle`; no repository yet since `Features.Beers` has no + code) + +- **Database.Connection**: the shared connection-factory project + - each slice's own repository issues inline SQL via Dapper against it + - no ORM, no stored procedures + - Replaces the old `Infrastructure.Sql` project + +- **Database.Migrations**: separate console app + - `schema.dbml` + SQL scripts + - applies DDL directly + - distinct from `Database.Seed`, which loads data through the feature slices' + own DI extensions + +- **Database.Seed**: + - calls each slice's `AddFeatures*()` DI extension directly + - builds its own MediatR registration + - no API host, no shared pipeline with `API.Core` + +- **Omitted from the diagram**: + - `Domain.Exceptions` (referenced by nearly every project directly, including + all of `Shared.Contracts`, `Infrastructure.Jwt`, and each feature slice) + - `Infrastructure.Configuration` (the shared options-binding project + referenced by `Database.Connection`, `Infrastructure.Email`, + `Infrastructure.FileUpload`, and `Features.Emails` directly) + - both are plumbing-level dependencies of nearly everything above, and drawing + them would just add a second copy of the same fan-in diff --git a/docs/web/diagrams/vertical-slice/host-wiring.mmd b/docs/web/diagrams/vertical-slice/host-wiring.mmd new file mode 100644 index 00000000..009d65c7 --- /dev/null +++ b/docs/web/diagrams/vertical-slice/host-wiring.mmd @@ -0,0 +1,31 @@ +--- +config: +flowchart: +curve: stepAfter +layout: elk +--- + +flowchart TB +API["API.Core
(ASP.NET Core Web API)"] +Seed["Database.Seed
(console app)"] +Migrations["Database.Migrations
(console app)"] + +subgraph SLICES["Feature Slices"] +direction LR +UsersSlice["Features.Users
(controller)"] +BrewerySlice["Features.Breweries
(controller)"] +PhotoSlice["Features.PhotoUpload
(MediatR only)"] +EmailsSlice["Features.Emails
(MediatR only)"] +LocationsSlice["Features.Locations
(plain DI service)"] +BeersSlice["Features.Beers
(scaffolded, no code yet)"] +end + +API -->|AddApplicationPart + MediatR scan| UsersSlice +API -->|AddApplicationPart + MediatR scan| BrewerySlice +API -->|MediatR scan only| PhotoSlice +API -->|MediatR scan only| EmailsSlice + +Seed -->|AddFeaturesUsers| UsersSlice +Seed -->|AddFeaturesBreweries| BrewerySlice +Seed -->|AddFeaturesPhotoUpload| PhotoSlice +Seed -->|AddFeaturesLocations| LocationsSlice diff --git a/docs/web/diagrams/vertical-slice/project-references.mmd b/docs/web/diagrams/vertical-slice/project-references.mmd new file mode 100644 index 00000000..d4dc5508 --- /dev/null +++ b/docs/web/diagrams/vertical-slice/project-references.mmd @@ -0,0 +1,43 @@ +--- +config: +flowchart: {} +layout: fixed +--- + +flowchart TB +subgraph SLICES["Feature Slices"] +direction LR +BrewerySlice["Features.Breweries"] +LocationsSlice["Features.Locations"] +BeersSlice["Features.Beers
(scaffolded)"] +PhotoSlice["Features.PhotoUpload"] +UsersSlice["Features.Users"] +EmailsSlice["Features.Emails"] +end + +subgraph FOUNDATION["Foundation (every DB-backed slice depends on all three)"] +direction LR +SharedContracts["Shared.Contracts"] +Domain["Domain.Entities"] +DBConn["Database.Connection"] +end + +subgraph INFRA["Slice-specific Infrastructure"] +direction LR +JWT["Infrastructure.Jwt"] +PwdHash["Infrastructure.PasswordHashing"] +FileUpload["Infrastructure.FileUpload"] +Email["Infrastructure.Email"] +EmailTemplates["Infrastructure.Email.Templates"] +end + +UsersSlice == project ref, sends UploadPhotoCommand ==> PhotoSlice +UsersSlice -. sends email commands .-> SharedApp["Shared.Application
(email command contract)"] +EmailsSlice -. handles email commands .-> SharedApp +BrewerySlice --> FOUNDATION +LocationsSlice --> FOUNDATION +BeersSlice --> FOUNDATION +PhotoSlice --> FOUNDATION & FileUpload +UsersSlice --> FOUNDATION & JWT & PwdHash +EmailsSlice --> Email & EmailTemplates +DBConn -- Dapper SQL, no ORM --> Tables[("SQL Server
Tables")] diff --git a/docs/website/docker.md b/docs/web/docker.md similarity index 90% rename from docs/website/docker.md rename to docs/web/docker.md index 20b882cc..a097d10f 100644 --- a/docs/website/docker.md +++ b/docs/web/docker.md @@ -1,4 +1,11 @@ -# Docker guide +--- +title: Docker deployment, configuration, and troubleshooting +last-updated: 2026-08-31 +tags: + - docker + - docker-compose + - deployment +--- This document covers Docker deployment, configuration, and troubleshooting for The Biergarten App. @@ -14,8 +21,7 @@ The project uses Docker Compose to orchestrate multiple services: - React Router website (SSR frontend) - Test runners -See the [deployment diagram](diagrams-out/deployment.svg) for visual -representation. +See the [deployment diagram](diagrams/deployment.md) for visual representation. ## Docker Compose environments @@ -100,7 +106,8 @@ frontend.tests # Storybook Vitest + Playwright suites (own Playwright image) ```bash # Run all tests -docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d +docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests # View results ls -la test-results/ @@ -111,6 +118,13 @@ cat test-results/Features.Users.Tests.trx docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v ``` +`wait` blocks until `api.specs`, `unit.tests`, and `frontend.tests` have all +stopped, regardless of which one finishes first; `up --abort-on-container-exit` +isn't used here because the one-shot `database.migrations`/`database.seed` jobs +and `frontend.tests` (no database dependency) can exit before the other test +containers, which would end the run prematurely. See +[Testing](testing.md#running-tests-with-docker-recommended) for details. + ### 3. Production (`docker-compose.prod.yaml`) **Purpose**: Production-ready deployment @@ -123,8 +137,8 @@ docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v - Health checks enabled - Restart policies (unless-stopped) - Security hardening -- Only `frontend` publishes a port to the host; `sqlserver` and `api.core` - are reachable only from other containers on `prodnet` +- Only `frontend` publishes a port to the host; `sqlserver` and `api.core` are + reachable only from other containers on `prodnet` **Services**: @@ -198,8 +212,8 @@ graph TD E[seaweedfs: container started] --> D ``` -In development, `api.core` also depends on `seaweedfs` (`condition: -service_started`, not a health check) alongside `database.seed`. +In development, `api.core` also depends on `seaweedfs` +(`condition: service_started`, not a health check) alongside `database.seed`. **Health Check Example** (SQL Server): @@ -233,8 +247,7 @@ api.core: - `sqlserverdata-dev` - Database files persist between restarts - `nuget-cache-dev` - NuGet package cache (speeds up builds) -- `seaweedfsdata-dev` - SeaweedFS object storage data persists between - restarts +- `seaweedfsdata-dev` - SeaweedFS object storage data persists between restarts **Testing**: diff --git a/docs/website/environment-variables.md b/docs/web/environment-variables.md similarity index 87% rename from docs/website/environment-variables.md rename to docs/web/environment-variables.md index 8b1534b1..beae6701 100644 --- a/docs/website/environment-variables.md +++ b/docs/web/environment-variables.md @@ -1,7 +1,14 @@ -# Environment variables +--- +title: Environment variables for the backend, frontend, and Docker +last-updated: 2026-08-31 +tags: + - environment-variables + - configuration + - secrets + - docker +--- -This document covers the active environment variables used by the current -Biergarten stack. +This document covers the environment variables used by the Biergarten stack. ## Overview @@ -19,10 +26,10 @@ Direct environment variable access via `Environment.GetEnvironmentVariable()`. ### Frontend (`web/frontend`) -The active website reads runtime values from the server environment for its auth -and API integration. In Docker, the `frontend` service is wired into the same +The website reads runtime values from the server environment for its auth and +API integration. In Docker, the `frontend` service is wired into the same `web/.env.dev` / `.env.test` / `.env.prod` files as the backend; running it -locally with `npm run dev` still reads from the shell environment. +locally with `npm run dev` reads from the shell environment. ### Docker @@ -38,9 +45,9 @@ Environment-specific `.env` files loaded via `env_file:` in docker-compose.yaml: secret-bearing values (`DB_PASSWORD`, `ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`, `CONFIRMATION_TOKEN_SECRET`, `SESSION_SECRET`, `SMTP_USERNAME`, `SMTP_PASSWORD`, `SEAWEEDFS_ACCESS_KEY_ID`, -`SEAWEEDFS_SECRET_ACCESS_KEY`) -with freshly randomized ones via `openssl`. Non-secret values are left -untouched. The output file is written with `600` permissions. +`SEAWEEDFS_SECRET_ACCESS_KEY`) with freshly randomized ones via `openssl`. +Non-secret values are left untouched. The output file is written with `600` +permissions. ```bash cd web @@ -104,7 +111,8 @@ WEBSITE_BASE_URL=https://thebiergarten.app # Base URL for the website **Security Requirements**: - Use at least 32 characters per secret -- Use 127 or more characters in production +- Use secrets generated from 127 or more random bytes in production (base64 + encoding turns 127 bytes into a longer string, well over 127 characters) - Generate secrets with a cryptographically secure random function - Never reuse secrets across token types or environments - Rotate secrets periodically in production @@ -170,10 +178,17 @@ ASPNETCORE_URLS=http://0.0.0.0:8080 # Binding address and port DOTNET_RUNNING_IN_CONTAINER=true # Flag for container execution ``` +`ASPNETCORE_URLS` is hardcoded directly in `docker-compose.dev.yaml` and +`docker-compose.prod.yaml` (not sourced from the `.env` file), so it's required +only for Docker. Outside Docker, `dotnet run` falls back to the `http` profile +in `API.Core/Properties/launchSettings.json` (`http://localhost:5069`), so +manual backend setup (see +[Getting Started](getting-started.md#manual-backend-setup)) doesn't need to set +it. + ## Frontend variables (`web/frontend`) -The active website does not use the old Next.js/Prisma environment model. Its -core runtime variables are: +The website's core runtime variables are: ```bash API_BASE_URL=http://api.core:8080 # Base URL for the .NET API (container DNS name in Docker) @@ -184,8 +199,8 @@ PORT=3000 # Port the built server listens on The `frontend` Docker service hardcodes `API_BASE_URL` to `http://api.core:8080` (the backend's in-network service name) and `PORT` to `3000`; only -`SESSION_SECRET` is sourced from the env file, and `NODE_ENV` is set per -compose file (`development` in dev, `production` in prod). +`SESSION_SECRET` is sourced from the env file, and `NODE_ENV` is set per compose +file (`development` in dev, `production` in prod). ### Frontend variable details @@ -287,15 +302,15 @@ web/.env.prod # Production config (gitignored) **Setup**: ```bash -cp web/.env.example web/.env.dev -# Edit web/.env.dev with your values +cd web +./generate-env.sh .env.dev ``` -## Legacy frontend variables - -Variables for the archived Next.js frontend (`archive/next-js-web-app/`) have -been removed from this active reference, since that app is retained for -reference only and is not run as part of the active stack. +See [Generating an env file](#generating-an-env-file) above — `generate-env.sh` +randomizes the secret-bearing values, so it's the preferred way to create any +`.env.dev`/`.env.test`/`.env.prod` file. A plain +`cp web/.env.example web/.env.dev` works too, but leaves the placeholder secrets +from the template in place; edit them manually if you go that route. **Docker Compose Mapping**: @@ -334,7 +349,7 @@ reference only and is not run as part of the active stack. | `PORT` | | ✓ | ✓ | No | Frontend server port | | `CLEAR_DATABASE` | ✓ | | ✓ | No | Dev/test reset flag | | `ASPNETCORE_ENVIRONMENT` | ✓ | | ✓ | Yes | ASP.NET environment | -| `ASPNETCORE_URLS` | ✓ | | ✓ | Yes | API binding address | +| `ASPNETCORE_URLS` | ✓ | | ✓ | Yes\*\* | API binding address | | `SA_PASSWORD` | | | ✓ | Yes | SQL Server container | | `ACCEPT_EULA` | | | ✓ | Yes | SQL Server EULA | | `MSSQL_PID` | | | ✓ | No | SQL Server edition | @@ -343,6 +358,10 @@ reference only and is not run as part of the active stack. \* Either `DB_CONNECTION_STRING` OR the component variables (`DB_SERVER`, `DB_NAME`, `DB_USER`, `DB_PASSWORD`) must be provided. +\*\* Required for Docker, where it's hardcoded in the compose files. Outside +Docker, `dotnet run` falls back to the `applicationUrl` in +`launchSettings.json`. + ## Validation ### Backend validation @@ -352,15 +371,15 @@ Variables are validated at startup: - Missing required variables (`ACCESS_TOKEN_SECRET`, `REFRESH_TOKEN_SECRET`, `CONFIRMATION_TOKEN_SECRET`, `SMTP_HOST`, `SMTP_FROM_EMAIL`, `SEAWEEDFS_SERVICE_URL`, `SEAWEEDFS_ACCESS_KEY_ID`, - `SEAWEEDFS_SECRET_ACCESS_KEY`, `SEAWEEDFS_BUCKET`, DB connection values) - cause the application to fail with an `InvalidOperationException` + `SEAWEEDFS_SECRET_ACCESS_KEY`, `SEAWEEDFS_BUCKET`, DB connection values) cause + the application to fail with an `InvalidOperationException` - No minimum length is enforced on the token secrets in code; the "minimum 32 characters" guidance above is a recommendation, not an enforced check ### Frontend validation -The active website relies on runtime defaults for local development and the -surrounding server environment in deployed environments. +The website relies on runtime defaults for local development and on the server +environment when deployed. - `API_BASE_URL` defaults to `http://localhost:8080` - `SESSION_SECRET` falls back to a development-only local secret diff --git a/docs/website/getting-started.md b/docs/web/getting-started.md similarity index 78% rename from docs/website/getting-started.md rename to docs/web/getting-started.md index e0492922..dcfa6517 100644 --- a/docs/website/getting-started.md +++ b/docs/web/getting-started.md @@ -1,12 +1,19 @@ -# Getting started +--- +title: Getting started — local setup for backend and frontend +last-updated: 2026-08-31 +tags: + - getting-started + - setup + - onboarding +--- -This guide covers local setup for the current Biergarten stack: the .NET backend -in `web/backend` and the active React Router frontend in `web/frontend`. +This guide covers local setup for the Biergarten stack: the .NET backend in +`web/backend` and the React Router frontend in `web/frontend`. ## Prerequisites - **.NET SDK 10+** -- **Node.js 18+** +- **Node.js 22+** - **Docker Desktop** or equivalent Docker Engine setup - **Java 8+** if you want to regenerate PlantUML diagrams @@ -58,18 +65,19 @@ section for details on `generate-env.sh`. docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d ``` -This starts SQL Server, migrations, seeding, the API, Mailpit (local dev -SMTP), and SeaweedFS (local dev S3-compatible object storage for photo -uploads). +This starts SQL Server, migrations, seeding, the API, Mailpit (local dev SMTP), +and SeaweedFS (local dev S3-compatible object storage for photo uploads). Available endpoints: -- API Swagger: http://localhost:8080/swagger -- Health Check: http://localhost:8080/health -- Mailpit UI: http://localhost:8025 -- SeaweedFS S3 API: http://localhost:8333 +| Service | URL | +| ---------------- | ----------------------------- | +| API Swagger | http://localhost:8080/swagger | +| Health Check | http://localhost:8080/health | +| Mailpit UI | http://localhost:8025 | +| SeaweedFS S3 API | http://localhost:8333 | -### 4. Start the active frontend +### 4. Start the frontend ```bash cd web/frontend @@ -146,13 +154,13 @@ dotnet run --project Database/Database.Seed/Database.Seed.csproj dotnet run --project API/API.Core/API.Core.csproj ``` -## Legacy frontend note +## Archived frontend -The previous Next.js frontend lives in `archive/next-js-web-app/` for reference -only and is not the active website. +`archive/next-js-web-app/` contains an archived Next.js frontend, kept for +reference; it is not part of the running stack. ## Next steps -- Review [Architecture](../architecture.md) +- Review [Architecture](architecture.md) - Run backend and frontend checks from [Testing](testing.md) - Use [Docker Guide](docker.md) for container troubleshooting diff --git a/docs/website/testing.md b/docs/web/testing.md similarity index 73% rename from docs/website/testing.md rename to docs/web/testing.md index 9c9167c1..0eb1c74d 100644 --- a/docs/website/testing.md +++ b/docs/web/testing.md @@ -1,4 +1,13 @@ -# Testing +--- +title: Testing strategy — unit, integration, and Storybook suites +last-updated: 2026-08-31 +tags: + - testing + - ci + - xunit + - reqnroll + - storybook +--- This document describes the testing strategy and how to run tests for The Biergarten App. @@ -7,22 +16,17 @@ Biergarten App. The project uses a multi-layered testing approach across backend and frontend: -- **API.Specs** - BDD integration tests using Reqnroll (Gherkin), run against a - live, seeded database -- **Features.\*.Tests** - One unit test project per backend feature slice - (`Features.Users.Tests`, `Features.Breweries.Tests`, - `Features.Emails.Tests`, `Features.PhotoUpload.Tests`), covering that - slice's command/query handlers, with dependencies mocked via Moq (no real - database required) -- **Storybook Vitest project** - Browser-based interaction tests for shared - website stories -- **Storybook Playwright suite** - Browser checks against Storybook-rendered - components +| Suite | Purpose | +| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **API.Specs** | BDD integration tests using Reqnroll (Gherkin), run against a live, seeded database | +| **Features.\*.Tests** | One unit test project per backend feature slice (`Features.Users.Tests`, `Features.Breweries.Tests`, `Features.Emails.Tests`, `Features.PhotoUpload.Tests`), covering that slice's command/query handlers, with dependencies mocked via Moq (no real database required) | +| **Storybook Vitest project** | Browser-based interaction tests for shared website stories | +| **Storybook Playwright suite** | Browser checks against Storybook-rendered components | ## Running tests with Docker (recommended) -The easiest way to run all tests is using Docker Compose, which sets up an -isolated test environment: +To run all tests, use Docker Compose, which sets up an isolated test +environment: ```bash docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d @@ -42,8 +46,8 @@ This command: `wait` blocks until `api.specs`, `unit.tests`, and `frontend.tests` have all stopped, regardless of which one finishes first. Long-running services (`sqlserver`, `seaweedfs`) and the one-shot `database.migrations` / -`database.seed` jobs keep running in the background alongside them and are -torn down separately (see below) — `wait` doesn't touch them. +`database.seed` jobs keep running in the background alongside them and are torn +down separately (see below); `wait` doesn't touch them. ### View test results @@ -72,19 +76,6 @@ docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v You can run individual test projects locally without Docker: -### Integration tests (API.Specs) - -```bash -cd web/backend -dotnet test API/API.Specs/API.Specs.csproj -``` - -**Requirements**: - -- SQL Server instance running -- Database migrated and seeded -- Environment variables set (DB connection, JWT secret) - ### Feature slice unit tests Each feature slice has its own test project, covering its command/query @@ -114,6 +105,19 @@ npm install npm run test:storybook ``` +### Integration tests (API.Specs) + +```bash +cd web/backend +dotnet test API/API.Specs/API.Specs.csproj +``` + +**Requirements**: + +- SQL Server instance running +- Database migrated and seeded +- Environment variables set (DB connection, JWT secret) + **Purpose**: - Verifies shared stories such as form fields, submit buttons, navbar states, @@ -175,8 +179,8 @@ The default (`headless: true`) is set in `web/frontend/vite.config.ts`. **Features.Breweries.Tests**: - Brewery create/update/delete commands and get-by-id/get-all queries -- Brewery location queries: all locations, and locations within a given range - of a coordinate +- Brewery location queries: all locations, and locations within a given range of + a coordinate **Features.Emails.Tests**: @@ -186,9 +190,8 @@ The default (`headless: true`) is set in `web/frontend/vite.config.ts`. - Upload command handler: storage key construction, content type passthrough, and photo persistence via the mocked storage provider and repository -- Upload validator: min/max file size bounds, and PNG/JPEG/WebP signature - checks (including rejection of files with a mismatched or missing - signature) +- Upload validator: min/max file size bounds, and PNG/JPEG/WebP signature checks + (including rejection of files with a mismatched or missing signature) **Frontend UI Coverage**: @@ -295,8 +298,8 @@ Features.Users.Tests/ Each of the other slices (`Features.Breweries.Tests`, `Features.Emails.Tests`, `Features.PhotoUpload.Tests`) follows the same shape: a `Commands/`/`Queries/` -folder with one test file per handler (`Features.PhotoUpload.Tests` also adds -a validator test file alongside its handler test). +folder with one test file per handler (`Features.PhotoUpload.Tests` also adds a +validator test file alongside its handler test). ## Writing tests @@ -337,10 +340,10 @@ Scenario: User login with valid credentials ## Continuous integration -The `.github/workflows/tests.yml` GitHub Actions workflow runs on every push -and pull request to `main`. It runs the same test Docker Compose -configuration used locally, so backend and frontend tests execute in the same -containerized environment in CI as they do on a developer machine: +The `.github/workflows/tests.yml` GitHub Actions workflow runs on every push and +pull request to `main`. It runs the same test Docker Compose configuration used +locally, so backend and frontend tests execute in the same containerized +environment in CI as they do on a developer machine: ```bash # CI/CD command @@ -351,19 +354,15 @@ docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v ``` The workflow inspects the exit code of each test container -(`test-env-api-specs`, `test-env-unit-tests`, `test-env-frontend-tests`) -after `wait` returns and fails the job if any of them is non-zero. -`./test-results/` is uploaded as a build artifact regardless of outcome. +(`test-env-api-specs`, `test-env-unit-tests`, `test-env-frontend-tests`) after +`wait` returns and fails the job if any of them is non-zero. `./test-results/` +is uploaded as a build artifact regardless of outcome. -Earlier versions of this workflow used -`up --abort-on-container-exit`, which stops the entire stack the instant -*any* watched container exits. That's wrong here: the one-shot -`database.migrations`/`database.seed` jobs are expected to exit 0 partway -through, and even scoping the flag to just the three test services doesn't -help, since `frontend.tests` (no database dependency) reliably finishes -before `api.specs`/`unit.tests` even start, aborting them prematurely. -`up -d` + `wait` avoids both races by not tying container teardown to any -single container's exit. +`up -d` + `wait` keeps container teardown independent of any single container's +exit. The one-shot `database.migrations`/`database.seed` jobs exit 0 partway +through the run, and `frontend.tests` (which has no database dependency) +reliably finishes before `api.specs`/`unit.tests` start; because `wait` blocks +on all three test containers, neither early exit ends the run prematurely. Exit codes: @@ -371,9 +370,9 @@ Exit codes: - Non-zero - Test failures occurred The frontend's Storybook Vitest and Playwright suites run inside the -`frontend.tests` container (see [Docker Guide](docker.md)) as part of that -same compose run, so no separate frontend CI step is needed. To run them -locally without Docker: +`frontend.tests` container (see [Docker Guide](docker.md)) as part of that same +compose run, so no separate frontend CI step is needed. To run them locally +without Docker: ```bash cd web/frontend @@ -384,9 +383,9 @@ npm run test:storybook:playwright ### Running the workflow locally with `act` -[`act`](https://github.com/nektos/act) replays `.github/workflows/*.yml` on -your own machine using Docker, so you can reproduce a CI run (or a fix for -one) without pushing a branch. +[`act`](https://github.com/nektos/act) replays `.github/workflows/*.yml` on your +own machine using Docker, so you can reproduce a CI run (or a fix for one) +without pushing a branch. **Install** (macOS): @@ -404,34 +403,31 @@ act push \ --container-architecture linux/amd64 ``` -- `-j containerized-tests` runs only that job (there's currently just the - one). -- `-P ubuntu-latest=catthehacker/ubuntu:act-latest` pins the runner image - act uses to impersonate `ubuntu-latest`. Without it, act's first run - prompts interactively to choose a default image size, which hangs/fails - under a non-interactive shell. +- `-j containerized-tests` runs only that job (there's currently just the one). +- `-P ubuntu-latest=catthehacker/ubuntu:act-latest` pins the runner image act + uses to impersonate `ubuntu-latest`. Without it, act's first run prompts + interactively to choose a default image size, which hangs/fails under a + non-interactive shell. - `--container-architecture linux/amd64` is required on Apple Silicon: the - `sqlserver` service's image is amd64-only, and act needs to know to - emulate that platform for the whole job container, not just that one - service. -- No secrets or `.actrc` are needed — `generate-env.sh` creates its own + `sqlserver` service's image is amd64-only, and act needs to know to emulate + that platform for the whole job container, not just that one service. +- No secrets or `.actrc` are needed: `generate-env.sh` creates its own `.env.test` with freshly randomized values, same as in real CI. **Known act-only limitation**: the "Upload test results" step -(`actions/upload-artifact@v4`) fails locally with `Unable to get the -ACTIONS_RUNTIME_TOKEN env variable`. act doesn't provide a real Actions -artifact backend, so this step — and only this step — is expected to fail -under act even when everything else passes. It works normally on +(`actions/upload-artifact@v4`) fails locally with +`Unable to get the ACTIONS_RUNTIME_TOKEN env variable`. act doesn't provide a +real Actions artifact backend, so this step, and only this step, is expected to +fail under act even when everything else passes. It works normally on GitHub-hosted runners. -**Stale local state**: `docker-compose.test.yaml` uses fixed container -names and named volumes (`sqlserverdata-test`, `seaweedfsdata-test`) scoped -to the `web` compose project. If a previous local run (via `act` or a -manual `docker compose` invocation) didn't get torn down, its SQL Server -volume can persist with an old `SA_PASSWORD` baked in, while a fresh -`.env.test` generates a new one each run — causing `Login failed for user -'sa'` errors that look like a test bug but are really a leftover volume. -Clear it before re-running: +**Stale local state**: `docker-compose.test.yaml` uses fixed container names and +named volumes (`sqlserverdata-test`, `seaweedfsdata-test`) scoped to the `web` +compose project. If a previous local run (via `act` or a manual `docker compose` +invocation) didn't get torn down, its SQL Server volume can persist with an old +`SA_PASSWORD` baked in, while a fresh `.env.test` generates a new one each run, +causing `Login failed for user 'sa'` errors that look like a test bug but are +really a leftover volume. Clear it before re-running: ```bash docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v diff --git a/docs/website/token-validation.md b/docs/web/token-validation.md similarity index 57% rename from docs/website/token-validation.md rename to docs/web/token-validation.md index 7cd3e769..bfb0ad65 100644 --- a/docs/website/token-validation.md +++ b/docs/web/token-validation.md @@ -1,14 +1,22 @@ -# Token validation architecture +--- +title: JWT token validation architecture +last-updated: 2026-08-31 +tags: + - jwt + - authentication + - security + - tokens +--- ## Overview -The Core project validates JSON Web Tokens (JWTs) across three token types: +The Biergarten API validates JSON Web Tokens (JWTs) across three token types: -- **Access Tokens**: Short-lived (1 hour) tokens for API authentication -- **Refresh Tokens**: Long-lived (21 days) tokens for obtaining new access - tokens -- **Confirmation Tokens**: Short-lived (30 minutes) tokens for email - confirmation +| Token Type | Lifetime | Purpose | +| ----------------------- | ---------- | --------------------------- | +| **Access Tokens** | 1 hour | API authentication | +| **Refresh Tokens** | 21 days | Obtaining new access tokens | +| **Confirmation Tokens** | 30 minutes | Email confirmation | ## Components @@ -34,22 +42,27 @@ Low-level JWT operations. #### [ITokenService](../../web/backend/Features/Features.Users/Services/ITokenService.cs) -Both token generation and validation live on the same slice-internal service -(there is no separate validation service/class). +Token generation and validation live on the same slice-internal service. **Generation methods:** -- `GenerateAccessToken(UserAccount)` - Creates 1-hour access token -- `GenerateRefreshToken(UserAccount)` - Creates 21-day refresh token -- `GenerateConfirmationToken(UserAccount)` - Creates 30-minute confirmation +- `GenerateAccessToken(Guid userId, string username)` - Creates 1-hour access token +- `GenerateRefreshToken(Guid userId, string username)` - Creates 21-day refresh + token +- `GenerateConfirmationToken(Guid userId, string username)` - Creates 30-minute + confirmation token **Validation methods:** -- `ValidateAccessTokenAsync(string token)` - Validates access tokens - `ValidateRefreshTokenAsync(string token)` - Validates refresh tokens - `ValidateConfirmationTokenAsync(string token)` - Validates confirmation tokens +There is no `ValidateAccessTokenAsync` on `ITokenService`. Access-token +validation doesn't go through `ITokenService` at all: `JwtAuthenticationHandler` +(the host-level auth middleware) calls `ITokenInfrastructure.ValidateJwtAsync()` +directly, bypassing the `Features.Users` slice entirely. + **Returns (validation):** `ValidatedToken` record containing: - `UserId` (Guid) @@ -65,10 +78,9 @@ Both token generation and validation live on the same slice-internal service `TokenService` is registered by `Features.Users`' own `AddFeaturesUsers()` extension method, except for the lower-level `ITokenInfrastructure` (JWT -signing/verification) it -depends on, which is registered by the host (`API.Core/Program.cs`) since -`JwtAuthenticationHandler` (host-level auth middleware) also depends on it -directly. +signing/verification) it depends on, which is registered by the host +(`API.Core/Program.cs`) since `JwtAuthenticationHandler` (host-level auth +middleware) also depends on it directly. ### Integration points @@ -78,10 +90,20 @@ directly. 1. Receives confirmation token from user via `ConfirmUserCommand` 2. Calls `ITokenService.ValidateConfirmationTokenAsync()` -3. Extracts user ID from validated token and looks it up via `UserManager` -4. Marks the account confirmed via `IUserEmailStore.SetEmailConfirmedAsync()` +3. Extracts user ID from validated token and looks it up via + `UserManager` +4. Marks the account confirmed via + `IUserEmailStore.SetEmailConfirmedAsync()`, then persists it + via `UserManager.UpdateAsync()` 5. Returns confirmation result +`UserManager` and `IUserEmailStore` are +ASP.NET Core Identity's abstractions, not EF Core's default Identity store: the +app backs them with its own `DapperUserStore` (see +[Database](database.md#database-error-handling)), which persists against the +existing `UserAccount`/`UserCredential`/`UserVerification` tables via Dapper, +consistent with the rest of the app's repository pattern. + #### [ResendConfirmationEmailHandler](../../web/backend/Features/Features.Users/Commands/Authentication/ResendConfirmationEmail/ResendConfirmationEmailHandler.cs) **Flow:** @@ -156,40 +178,45 @@ Each token is validated for: ### Error handling -Validation failures return HTTP 401 Unauthorized: - -- Invalid signature → "Invalid token" -- Expired token → "Invalid token" (message doesn't reveal reason for security) -- Missing claims → "Invalid token" -- Malformed claims → "Invalid token" +Validation failures return HTTP 401 Unauthorized. Regardless of the specific +cause (invalid signature, expired token, missing header, malformed claims), +`JwtAuthenticationHandler.HandleChallengeAsync` always returns the same generic +message: `"Unauthorized: Invalid or missing authentication token"` — it doesn't +distinguish failure reasons in the response. ## Token lifecycle ### Access token lifecycle -1. **Generation**: During login (1-hour validity) -2. **Usage**: Included in Authorization header on API requests -3. **Validation**: Validated on protected endpoints -4. **Expiration**: Token becomes invalid after 1 hour -5. **Refresh**: Use refresh token to obtain new access token +| Step | Description | +| ---------- | ------------------------------------------------ | +| Generation | During login (1-hour validity) | +| Usage | Included in Authorization header on API requests | +| Validation | Validated on protected endpoints | +| Expiration | Token becomes invalid after 1 hour | +| Refresh | Use refresh token to obtain new access token | ### Refresh token lifecycle -1. **Generation**: During login (21-day validity) -2. **Storage**: Client-side (secure storage) -3. **Usage**: Posted to `/api/auth/refresh` endpoint -4. **Validation**: Validated by `ITokenService.RefreshTokenAsync()` -5. **Rotation**: New refresh token issued on successful refresh -6. **Expiration**: Token becomes invalid after 21 days +| Step | Description | +| ---------- | ------------------------------------------------ | +| Generation | During login (21-day validity) | +| Storage | Client-side (secure storage) | +| Usage | Posted to `/api/auth/refresh` endpoint | +| Validation | Validated by `ITokenService.RefreshTokenAsync()` | +| Rotation | New refresh token issued on successful refresh | +| Expiration | Token becomes invalid after 21 days | ### Confirmation token lifecycle -1. **Generation**: During user registration (30-minute validity) -2. **Delivery**: Emailed to user in confirmation link -3. **Usage**: User clicks link, token posted to `/api/auth/confirm` -4. **Validation**: Validated by `ConfirmUserHandler` -5. **Completion**: User account marked as confirmed -6. **Expiration**: Token becomes invalid after 30 minutes +| Step | Description | +| ---------- | ----------------------------------------------------- | +| Generation | During user registration (30-minute validity) | +| Delivery | Emailed to user in confirmation link | +| Usage | User clicks link, token posted to `/api/auth/confirm` | +| Validation | Validated by `ConfirmUserHandler` | +| Completion | User account marked as confirmed | +| Expiration | Token becomes invalid after 30 minutes | ## Testing @@ -249,22 +276,18 @@ All of the following live in `Features.Users.Tests`. ### Stretch goals -1. **Middleware for Access Token Validation** - - Automatically validate access tokens on protected routes - - Populate HttpContext.User from token claims - - Return 401 for invalid/missing tokens +1. **Middleware for Access Token Validation** - Automatically validate access + tokens on protected routes - Populate HttpContext.User from token claims - + Return 401 for invalid/missing tokens -2. **Token Denylisting** - - Implement token revocation (for example, on logout) - - Store denylisted tokens in cache/database - - Check the denylist during validation +2. **Token Denylisting** - Implement token revocation (for example, on logout) - + Store denylisted tokens in cache/database - Check the denylist during + validation -3. **Refresh Token Rotation Strategy** - - Detect token reuse (replay attacks) - - Automatically invalidate entire token chain on reuse - - Log suspicious activity +3. **Refresh Token Rotation Strategy** - Detect token reuse (replay attacks) - + Automatically invalidate entire token chain on reuse - Log suspicious + activity -4. **Structured Logging** - - Log token validation attempts - - Track failed validation reasons - - Alert on repeated validation failures (brute force detection) +4. **Structured Logging** - Log token validation attempts - Track failed + validation reasons - Alert on repeated validation failures (brute force + detection) diff --git a/docs/website/database.md b/docs/website/database.md deleted file mode 100644 index e98dcf10..00000000 --- a/docs/website/database.md +++ /dev/null @@ -1,87 +0,0 @@ -# Database - -SQL Server 2022 schema, migration approach, and the app's SQL error-handling -convention. For the vertical-slice code that talks to this schema, see -[Architecture](../architecture.md). - -## Schema - -The full schema lives under -`web/backend/Database/Database.Migrations/scripts/01-schema/`, one numbered -script per table (`01-UserAccount.sql`, `02-Photo.sql`, ...), run in order by -`Database.Migrations` (DbUp) on startup. Each table is queried directly by the -owning feature slice's repository (see -[Direct SQL via Dapper/ADO.NET](../architecture.md#direct-sql-via-dapperadonet)). - -A generated DBML copy of the schema (for import into a diagramming tool such -as dbdiagram.io) lives at -`web/backend/Database/Database.Migrations/schema.dbml`. - -All tables use a `UNIQUEIDENTIFIER` primary key defaulting to `NEWID()`, and -most carry a `RowVersion ROWVERSION` column used for optimistic concurrency on -updates (`UPDATE ... WHERE ... AND RowVersion = @RowVersion`). - -### Tables backed by the API today - -| Table | Backed by | Notes | -| --------------------- | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `UserAccount` | `Domain.Entities.UserAccount`, `Features.Users` | `Username` and `Email` are unique | -| `UserCredential` | `Domain.Entities.UserCredential`, `Features.Users` | Argon2id password hash; `IsRevoked`/`RevokedAt` track credential rotation, not JWTs | -| `UserVerification` | `Domain.Entities.UserVerification`, `Features.Users` | Written when a user confirms their email | -| `UserProfile` | `Domain.Entities.UserProfile`, `Features.Users` | 1:1 with `UserAccount`; holds the user's biography; written via `IUserProfileRepository` | -| `UserAvatar` | `Domain.Entities.UserAvatar`, `Features.Users` | Links a `UserProfile` to a `Photo` as its avatar; written via `IUserProfileRepository.SaveAvatarAsync` | -| `BreweryPost` | `Domain.Entities.BreweryPost`, `Features.Breweries` | A user-submitted brewery listing | -| `BreweryPostLocation` | `Domain.Entities.BreweryPostLocation`, `Features.Breweries` | 1:1 with `BreweryPost`; `Coordinates` is a `GEOGRAPHY` column, read via a plain Dapper query that selects `CONVERT(varbinary(max), Coordinates)`, since Dapper can't deserialize SQL Server UDTs directly | -| `City` | `Domain.Entities.City`, `Features.Locations` | Referenced by `BreweryPostLocation.CityID`; written via `ILocationRepository`, read via `ILocationRepository` and `IBreweryRepository`'s location joins, currently written only by `Database.Seed` | -| `StateProvince` | `Domain.Entities.StateProvince`, `Features.Locations` | Referenced by `City.StateProvinceID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins | -| `Country` | `Domain.Entities.Country`, `Features.Locations` | Referenced by `StateProvince.CountryID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins | -| `Photo` | `Domain.Entities.Photo`, `Features.PhotoUpload` | Written via `IPhotoUploadRepository.CreateAsync`, sent as an `UploadPhotoCommand` from other features' handlers; not yet seeded by `Database.Seed` | - -### Tables defined in the schema but not yet wired to a feature slice - -These exist so the schema can support planned functionality, but have no -repository or seed data yet: - -| Table | Purpose (from schema) | -| ------------------ | -------------------------------------------------------------------------------------------------------- | -| `UserFollow` | One user following another (`CannotFollowOwnAccount` check constraint) | -| `BeerStyle` | A beer style/category; has a `Domain.Entities.BeerStyle` type, but no repository yet | -| `BeerPost` | A beer listing, linked to a `BreweryPost` and `BeerStyle`; `ABV`/`IBU` are constrained (`0-67`, `0-120`); has a `Domain.Entities.BeerPost` type, but no repository yet | -| `BeerPostPhoto` | Links a `BeerPost` to a `Photo` | -| `BeerPostComment` | A comment + 1-5 `Rating` on a `BeerPost` | -| `BreweryPostPhoto` | Links a `BreweryPost` to a `Photo` | - -## Migrations - -**Tool**: DbUp, run by the `Database.Migrations` project/container. - -Scripts live under `scripts/`, applied in order and tracked so a script never -re-runs once it has succeeded. Today that's `01-schema/`, with one script per -table, numbered so a table's foreign keys always point at an earlier-numbered -script. When the schema needs to change, add a new numbered script rather than -editing an existing one in place — DbUp's tracking is per-script, so editing -an already-applied script means it won't re-run against existing databases. - -## Data seeding - -`Database.Seed` populates a target database from data produced by the C++ -pipeline (see [Pipeline README](../pipeline/README.md)): countries, -state/provinces, and cities (created on demand while resolving each brewery's -location), user accounts, and brewery posts with locations. It doesn't call -every feature slice that has a write path — `Photo`, for example, has a -repository (`Features.PhotoUpload`) but no seed data. - -## Database error handling - -- Optimistic-concurrency conflicts are detected by an `UPDATE` affecting zero - rows against the `RowVersion` column. -- On failure, the repository throws a `Domain.Exceptions` type directly - (`NotFoundException`, `ConflictException`, `ArgumentException`, ...). - `API.Core`'s `GlobalExceptionFilter` maps these to HTTP status codes (404, - 409, 400, ...); an unmapped `SqlException` (e.g. a genuine connectivity or - constraint-violation failure) maps to 503. - - - One exception: `Features.Users`' `DapperUserStore` does catch - `SqlException` directly, to detect a duplicate-key race (SQL Server error - 2601 or 2627) when two requests try to verify the same account - concurrently — see `DapperUserStore.IsDuplicateKeyViolation`. diff --git a/docs/website/diagrams-out/architecture.svg b/docs/website/diagrams-out/architecture.svg deleted file mode 100644 index b07f8b34..00000000 --- a/docs/website/diagrams-out/architecture.svg +++ /dev/null @@ -1 +0,0 @@ -The Biergarten App - Vertical Slice ArchitectureAPI.Core (thin host)Feature SlicesSharedInfrastructure LayerDomain LayerSQL ServerAPI.CoreASP.NET Core Web API host- Program.cs wiring only- MediatR + AddApplicationPartper Features.* assembly- Swagger/OpenAPI, health checks- JWT auth middleware- Global exception filterFeatures.UsersFeatures.BreweriesFeatures.EmailsFeatures.LocationsEach slice owns its own:- Controller (HTTP endpoints)- Commands/Queries + Handlers- Validators- Repository (Dapper) Features.Users covers auth(Commands/Authentication),account management(Commands/Account), andprofile/avatar (Commands/Profile)No controller - invoked onlyvia MediatR commands sentfrom other slicesNo controller and no MediatRhandlers - exposes ILocationRepositorydirectly as a plain DI service.Not registered by API.Core at all;only Database.Seed depends on it todayShared.ContractsShared.Application- ValidationBehavior(MediatR pipeline)- Cross-slice email commands(SendRegistrationEmailCommand,SendResendConfirmationEmailCommand)Infrastructure.SqlInfrastructure.JwtInfrastructure.PasswordHashingInfrastructure.EmailInfrastructure.Email.TemplatesDomain.EntitiesDomain.Exceptions- UserAccount, UserCredential, UserVerification- UserProfile, UserAvatar- BreweryPost, BreweryPostLocation- City, StateProvince, Country- BeerPost, BeerStyle(schema tables exist; no repository yet)TablesDatabase.Seed(separate console app)SQL-first approachEach slice's own repository issuesinline SQL via Dapper - no ORM,no stored proceduresNo Features.* projectever references anotherFeatures.* project directlyAddApplicationPart+ MediatR scanAddApplicationPart+ MediatR scanMediatR scan only(no controller)AddFeaturesUsers()(DI only, no API/MediatR)AddFeaturesBreweries()AddFeaturesLocations()sendsSendRegistrationEmailCommandhandlesSendRegistrationEmailCommandDapper SQLDapper SQLDapper SQL \ No newline at end of file diff --git a/docs/website/diagrams-out/authentication-flow.svg b/docs/website/diagrams-out/authentication-flow.svg deleted file mode 100644 index e90a7d3e..00000000 --- a/docs/website/diagrams-out/authentication-flow.svg +++ /dev/null @@ -1 +0,0 @@ -User Authentication Flow - ExpandedUser Authentication Flow - ExpandedHandler LayerInfrastructure LayerRepository / Cross-sliceAPIAPIAPIAPIAPIAPIAPIAPIAPIAPIMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineRegisterUserHandlerLoginHandlerRefreshTokenHandlerConfirmUserHandlerResendConfirmationEmailHandlerUpdateUsernameHandlerUpdateEmailHandlerUpdatePasswordHandlerUpdateProfileHandlerDeleteAccountHandlerUserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.TokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceArgon2PasswordHasherArgon2PasswordHasherArgon2PasswordHasherArgon2PasswordHasherJWTJWTJWTJWTJWTJWTJWTEmailTemplateDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreEmailDispatcherEmailDispatcherSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerUserAPIMediatR pipelineRegisterUserHandlerLoginHandlerRefreshTokenHandlerConfirmUserHandlerResendConfirmationEmailHandlerUpdateUsernameHandlerUpdateEmailHandlerUpdatePasswordHandlerUpdateProfileHandlerDeleteAccountHandlerUserManager.ApplicationUser.TokenServiceArgon2PasswordHasherJWTEmailTemplateDapperUserStoreEmailDispatcherSQL ServerUserUserAPIControllerAPIControllerMediatR pipeline(ValidationBehavior)MediatR pipeline(ValidationBehavior)RegisterUserHandlerRegisterUserHandlerLoginHandlerLoginHandlerRefreshTokenHandlerRefreshTokenHandlerConfirmUserHandlerConfirmUserHandlerResendConfirmationEmailHandlerResendConfirmationEmailHandlerUpdateUsernameHandlerUpdateUsernameHandlerUpdateEmailHandlerUpdateEmailHandlerUpdatePasswordHandlerUpdatePasswordHandlerUpdateProfileHandlerUpdateProfileHandlerDeleteAccountHandlerDeleteAccountHandlerUserManager<ApplicationUser>UserManager<ApplicationUser>TokenServiceTokenServiceArgon2PasswordHasherArgon2PasswordHasherJWTInfrastructureJWTInfrastructureEmailProviderEmailProviderTemplateProviderTemplateProviderDapperUserStore(IUserStore/IUserPasswordStore/IUserEmailStore)DapperUserStore(IUserStore/IUserPasswordStore/IUserEmailStore)EmailDispatcher(Features.Emails)EmailDispatcher(Features.Emails)SQL ServerSQL ServerAPIAPIAPIAPIAPIAPIAPIAPIAPIAPIMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineMediatR pipelineRegisterUserHandlerLoginHandlerRefreshTokenHandlerConfirmUserHandlerResendConfirmationEmailHandlerUpdateUsernameHandlerUpdateEmailHandlerUpdatePasswordHandlerUpdateProfileHandlerDeleteAccountHandlerUserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.UserManager.ApplicationUser.TokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceTokenServiceArgon2PasswordHasherArgon2PasswordHasherArgon2PasswordHasherArgon2PasswordHasherJWTJWTJWTJWTJWTJWTJWTEmailTemplateDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreDapperUserStoreEmailDispatcherEmailDispatcherSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerSQL ServerASP.NET Core Identity's UserManager<TUser>replaces the deleted AuthRepository: handlersno longer issue SQL or call Argon2 directly,they delegate to UserManager, which in turncalls DapperUserStore (persistence) andArgon2PasswordHasher (IPasswordHasher<TUser>).Registration FlowPOST /api/auth/register{username, firstName,lastName,email, dateOfBirth, password}Send(RegisterUserCommand)ValidationBehavior runs RegisterUserValidator(FluentValidation) before the handler:- Username: 3-64 chars, alphanumeric + [._-]- Email: valid format, max 128 chars- Password: min 8 chars, uppercase,\n lowercase, number, special char- DateOfBirth: must be 19+ years oldalt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request{message: "Validation failed",errors: {...}}[Validation succeeds]Handle(command)user = new UserAccount{...}.ToApplicationUser()CreateAsync(user, password)HashPassword(user, password)Delegates to IPasswordInfrastructure.Hash:Argon2id parameters:- Salt: 16 bytes (128-bit)- Memory: 64MB- Iterations: 4- Parallelism: CPU count- Hash output: 32 bytes"base64(salt):base64(hash)"SetPasswordHashAsync(user,hash)user.Id is still empty at this point -only sets the in-memory hash. CreateAsyncwrites it to dbo.UserCredential itself.FindByNameAsync(username)SELECT UserAccountID,Username, ...FROM dbo.UserAccountWHERE Username =@Usernamenull (user doesn't exist)FindByEmailAsync(email)SELECT UserAccountID,Username, ...FROM dbo.UserAccountWHERE Email = @Emailnull (email doesn't exist)Default UserManager validators checkuniqueness before CreateAsync is called;DapperUserStore.CreateAsync also guardsagainst a genuine race via a caughtduplicate-key SqlException (2601/2627).CreateAsync(user)BEGIN TRANSACTIONINSERT INTO dbo.UserAccount(Username, FirstName,LastName,DateOfBirth, Email)OUTPUTINSERTED.UserAccountIDUserAccountId (GUID)INSERT INTOdbo.UserCredential(UserAccountId, Hash)COMMIT TRANSACTIONBoth statements run via Dapper'sExecuteScalarAsync/ExecuteAsync againsta single DbTransaction, not a storedprocedure - no manual parameter bindingIdentityResult.Success(user.Id set)IdentityResultalt[!result.Succeeded]throw ConflictException(DuplicateUserName/DuplicateEmailcodes ->"Username or email alreadyexists";otherwise joined errordescriptions)propagate409 Conflict"Username or email alreadyexists"[result.Succeeded]GenerateAccessToken(user)GenerateJwt(userId, username,expiry)JWT Configuration:- Algorithm: HS256- Expires: 1 hour- Claims:* sub: userId* unique_name: username* iat / jtiCreate JWT with claimsSign with secret keyAccess TokenAccess TokenGenerateRefreshToken(user)GenerateJwt(userId, username,expiry)Refresh Token:- Expires: 21 days (504 hours)- Same structure as access token,\n signed with a separate secretRefresh TokenRefresh TokenGenerateConfirmationToken(user)GenerateJwt(userId, username,expiry)Confirmation Token:- Expires: 30 minutes (0.5 hours)- Signed with a third, separate secretConfirmation TokenConfirmation TokenSend(SendRegistrationEmailCommand)(Shared.Application cross-slicecontract)SendRegistrationEmailAsync(  firstName, email,confirmationToken)Routed to Features.Emails'SendRegistrationEmailHandler,the only cross-slice call inthe backendRenderUserRegisteredEmailAsync(  firstName, confirmationLink)Razor Component:- Header with branding- Welcome message- Confirmation button- FooterRender Razor componentto HTMLHTML email contentSendAsync(email, subject, body)SMTP Configuration (MailKit):- Host/port: from env (SMTP_HOST/SMTP_PORT)- TLS: StartTLS when SMTP_USE_SSL=true- Auth: username/password, if configuredBuild MIME messageConnect, authenticate,send, disconnectSuccess / throwsalt[Email sent successfully]emailSent = true[mediator.Send throws]catch(Exception),log to stderr,emailSent = falseRegistrationPayload(  userAccountId, username,  refreshToken, accessToken,  confirmationEmailSent)RegistrationPayload201 Created{  message: "User registeredsuccessfully.",  payload: {    userAccountId, username,    refreshToken, accessToken,    confirmationEmailSent  }}Login FlowPOST /api/auth/login{username, password}Send(LoginCommand)ValidationBehavior runs LoginValidator:username and password must both be non-emptyalt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request{message: "Validation failed",errors: {...}}[Validation succeeds]Handle(command)FindByNameAsync(username)FindByNameAsync(username)SELECT UserAccountID,Username, ...FROM dbo.UserAccountWHERE Username =@UsernameUserAccount row or nullHydrateAsync(load activeUserCredential.Hash,UserVerification ->EmailConfirmed)ApplicationUser or nullApplicationUser or nullalt[User not found]throw UnauthorizedException"Invalid username or password."propagate401 Unauthorized[User found]CheckPasswordAsync(user,password)HasPasswordAsync short-circuits to false(no active UserCredential was hydrated)without calling the hasher; otherwisedelegates to Argon2PasswordHasher.VerifyHashedPassword(  user, user.PasswordHash,password)Delegates to IPasswordInfrastructure.Verify:1. Split stored hash: "salt:hash"2. Hash provided password with the same salt3. Constant-time comparison\n (CryptographicOperations.FixedTimeEquals)Success/Failedtrue/falsealt[Password invalid]throw UnauthorizedExceptionpropagate401 Unauthorized[Password valid]GenerateAccessToken(user)GenerateJwt(...)Access TokenAccess TokenGenerateRefreshToken(user)GenerateJwt(...)Refresh TokenRefresh TokenLoginPayload(  userAccountId, username,  refreshToken, accessToken)LoginPayload200 OK{  message: "Logged insuccessfully.",  payload: {    userAccountId, username,    refreshToken, accessToken  }}Refresh Token FlowPOST /api/auth/refreshHeader: X-Refresh-Token:<token>Send(RefreshTokenCommand(refreshToken))ValidationBehavior runs RefreshTokenValidator:refreshToken must be non-empty. Anonymousendpoint - no Bearer token required.alt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request{message: "Validation failed",errors: {...}}[Validation succeeds]Handle(command)RefreshTokenAsync(refreshToken)ValidateRefreshTokenAsync(refreshToken)Validates signature/expiry against therefresh token secret, extracts userId/username claims.ValidatedToken or throwsalt[Token invalid or expired]throw UnauthorizedExceptionpropagatepropagate401 Unauthorized[Token valid]FindByIdAsync(validated.UserId)FindByIdAsync(userId)SELECT UserAccountID,Username, ...FROM dbo.UserAccountWHERE UserAccountID = @Idrow or nullApplicationUser or nullApplicationUser or nullalt[User not found]throw UnauthorizedException"User account not found"propagatepropagate401 Unauthorized[User found]GenerateAccessToken(user)GenerateRefreshToken(user)new access + refresh tokensRefreshTokenResultLoginPayload(  userId, username,  refreshToken, accessToken)LoginPayload200 OK{message: "Token refreshedsuccessfully.",payload: {...}}Confirm User FlowPOST/api/auth/confirm?token=<token>(requires Bearer access token)Send(ConfirmUserCommand(token))Handle(command)ValidateConfirmationTokenAsync(token)Validate against confirmationtoken secret, extract userIdValidatedToken or throwsValidatedTokenalt[Token invalid or expired]throw UnauthorizedExceptionpropagate401 Unauthorized[Token valid]FindByIdAsync(validatedToken.UserId)FindByIdAsync(userId)SELECT ...FROM dbo.UserAccountWHERE UserAccountID = @Idrow or nullApplicationUser or nullApplicationUser or nullalt[User not found]throw UnauthorizedException"User account not found"propagate401 Unauthorized[User found]SetEmailConfirmedAsync(user,true)IUserEmailStore<ApplicationUser> is injecteddirectly into the handler (not via UserManager),setting the confirmed flag in-memory here.UpdateAsync(user)UpdateAsync(user)UPDATE dbo.UserAccount SET ...INSERT INTOdbo.UserVerification(UserAccountId,VerificationDateTime)Idempotent: skipped (duplicate-keyexception swallowed) if a verificationrow already exists.IdentityResult.SuccessIdentityResultConfirmationPayload(  user.Id, DateTime.UtcNow)ConfirmationPayload200 OK{message: "User with ID {id} isconfirmed.",payload: {...}}Resend Confirmation Email FlowPOST/api/auth/confirm/resend?userId=<id>(requires Bearer access token)Send(ResendConfirmationEmailCommand(userId))Handle(command)FindByIdAsync(userId)FindByIdAsync(userId)SELECT ...FROM dbo.UserAccountWHERE UserAccountID = @Idrow or nullApplicationUser or nullApplicationUser or nullalt[User not found]return silently(prevents user enumeration)[User found]IsEmailConfirmedAsync(user)boolalt[Already confirmed]return silently (no-op)[Not yet confirmed]GenerateConfirmationToken(user)confirmation tokenSend(SendResendConfirmationEmailCommand)SendResendConfirmationEmailAsync(  firstName, email,confirmationToken)Render email(same Razor pipeline asregistration)SendAsync(email, subject, body)(void)(void)200 OK{message: "confirmation emailhas been resent"}Always 200 OK regardless of outcome above -same user-enumeration precaution as a not-founduser or an already-confirmed account.Update Username FlowPATCH /api/auth/username{newUsername}(requires Bearer access token)Send(UpdateUsernameCommand(  GetAuthenticatedUserId(),newUsername))ValidationBehavior runs UpdateUsernameValidator(same format rules as registration)alt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request[Validation succeeds]Handle(command)FindByIdAsync(userAccountId)FindByIdAsync(userAccountId)SELECT ...FROM dbo.UserAccountWHERE UserAccountID = @Idrow or nullApplicationUser or nullApplicationUser or nullalt[Account not found]throw NotFoundExceptionpropagate404 Not Found[Account found]SetUserNameAsync(user,newUsername)FindByNameAsync(newUsername)Uniqueness check before the UPDATE.UpdateAsync(user)UPDATE dbo.UserAccountSET Username = @UserName, ...IdentityResultIdentityResultalt[!result.Succeeded]throw ConflictException(DuplicateUserName code ->"Username already exists")propagate409 Conflict[result.Succeeded]UpdateUsernamePayload(  user.Id, user.UserName)UpdateUsernamePayload200 OK{message: "Username updatedsuccessfully.",payload: {...}}Update Email FlowPATCH /api/auth/email{newEmail}(requires Bearer access token)Send(UpdateEmailCommand(  GetAuthenticatedUserId(),newEmail))ValidationBehavior runs UpdateEmailValidator(same format rules as registration)alt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request[Validation succeeds]Handle(command)FindByIdAsync(userAccountId)FindByIdAsync(userAccountId)SELECT ...ApplicationUser or nullApplicationUser or nullalt[Account not found]throw NotFoundExceptionpropagate404 Not Found[Account found]SetEmailAsync(user, newEmail)FindByEmailAsync(newEmail)Uniqueness check. Also resetsuser.EmailConfirmed = false -the caller must re-confirm viaPOST /api/auth/confirm/resend.UpdateAsync(user)UPDATE dbo.UserAccountSET Email = @Email, ...DELETE FROMdbo.UserVerificationWHERE UserAccountID = @IdIdentityResultIdentityResultalt[!result.Succeeded]throw ConflictException(DuplicateEmail code ->"Email address already exists")propagate409 Conflict[result.Succeeded]UpdateEmailPayload(  user.Id, user.Email,  user.EmailConfirmed = false)UpdateEmailPayload200 OK{message: "Email updatedsuccessfully.",payload: {...}}Update Password FlowPATCH /api/auth/password{currentPassword,newPassword}(requires Bearer access token)Send(UpdatePasswordCommand(  GetAuthenticatedUserId(),  currentPassword,newPassword))ValidationBehavior runs UpdatePasswordValidator(same password strength rules as registration)alt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request[Validation succeeds]Handle(command)FindByIdAsync(userAccountId)FindByIdAsync(userAccountId)ApplicationUser or nullApplicationUser or nullalt[Account not found]throw NotFoundExceptionpropagate404 Not Found[Account found]ChangePasswordAsync(  user, currentPassword,newPassword)VerifyHashedPassword(  user, user.PasswordHash,currentPassword)Success/Failedalt[Current password mismatch]IdentityResult.Failed(PasswordMismatch)[Current password matches]HashPassword(user,newPassword)new hashSetPasswordHashAsync(user,newHash)UPDATE dbo.UserCredentialSET IsRevoked = 1, RevokedAt =GETDATE()WHERE UserAccountId = @IdAND IsRevoked = 0INSERT INTOdbo.UserCredential(UserAccountId, Hash)Revoke-then-insert runs inside oneDbTransaction (credential rotation).(void)IdentityResult.Successalt[Current password is incorrect]throw UnauthorizedException"Current password is incorrect"propagate401 Unauthorized[!result.Succeeded (other failure)]throw ConflictExceptionpropagate409 Conflict[result.Succeeded]UpdatePasswordPayload(  user.Id, DateTime.UtcNow)UpdatePasswordPayload200 OK{message: "Password updatedsuccessfully.",payload: {...}}Update Profile FlowPATCH /api/auth/profile{firstName, lastName,dateOfBirth}(requires Bearer access token)Send(UpdateProfileCommand(  GetAuthenticatedUserId(),  firstName, lastName,dateOfBirth))ValidationBehavior runs UpdateProfileValidator(same name/DateOfBirth rules as registration)alt[Validation fails]throwsFluentValidation.ValidationException400 Bad Request[Validation succeeds]Handle(command)FindByIdAsync(userAccountId)FindByIdAsync(userAccountId)ApplicationUser or nullApplicationUser or nullalt[Account not found]throw NotFoundExceptionpropagate404 Not Found[Account found]user.FirstName = ...user.LastName = ...user.DateOfBirth = ...First/last name and date of birth aren'ttracked by any Identity store interface -mutated directly on the ApplicationUser,then saved via UserManager.UpdateAsync.UpdateAsync(user)UpdateAsync(user)UPDATE dbo.UserAccountSET FirstName = @FirstName,    LastName = @LastName,    DateOfBirth = @DateOfBirth,...IdentityResultIdentityResultalt[!result.Succeeded]throw ConflictExceptionpropagate409 Conflict[result.Succeeded]UpdateProfilePayload(  user.Id, firstName,  lastName, dateOfBirth)UpdateProfilePayload200 OK{message: "Profile updatedsuccessfully.",payload: {...}}Delete Account FlowDELETE /api/auth/account(requires Bearer access token)Send(DeleteAccountCommand(  GetAuthenticatedUserId()))Handle(command)FindByIdAsync(userAccountId)FindByIdAsync(userAccountId)ApplicationUser or nullApplicationUser or nullalt[Account not found]throw NotFoundExceptionpropagate404 Not Found[Account found]DeleteAsync(user)DeleteAsync(user)DELETE FROM dbo.UserAccountWHERE UserAccountID = @IdUserCredential/UserVerification/UserAvatarcascade on delete. Posts, comments, photos,and follows use ON DELETE NO ACTION - if anyreference this account, the DELETE throws aforeign key violation (SQL error 547).alt[Foreign key violation (dependent records exist)]IdentityResult.Failed(AccountHasDependentRecords)[Delete succeeds]IdentityResult.SuccessIdentityResultalt[Account has dependent records]throw ConflictException"Account cannot be deletedwhile it stillhas associated posts,comments, photos,or follows."propagate409 Conflict[!result.Succeeded (other failure)]throw ConflictExceptionpropagate409 Conflict[result.Succeeded](void)(void)200 OK{message: "Account deletedsuccessfully."}Error Handling (Global Exception Filter)API.Core's GlobalExceptionFilter catches:- FluentValidation.ValidationException -> 400 Bad Request\n {message, errors} (grouped by property)- ConflictException -> 409 Conflict- NotFoundException -> 404 Not Found- UnauthorizedException -> 401 Unauthorized- ForbiddenException -> 403 Forbidden- Domain.Exceptions.ValidationException -> 400 Bad Request- Microsoft.Data.SqlClient.SqlException -> 503 Service Unavailable- All others -> 500 Internal Server Error \ No newline at end of file diff --git a/docs/website/diagrams-out/database-schema.svg b/docs/website/diagrams-out/database-schema.svg deleted file mode 100644 index 2c9e70c5..00000000 --- a/docs/website/diagrams-out/database-schema.svg +++ /dev/null @@ -1 +0,0 @@ -Database SchemaUser & AuthLocationBreweryBeer (*)UserAccountUserAccountID: UNIQUEIDENTIFIER «PK»Username: VARCHAR(64) «UNIQUE»FirstName: NVARCHAR(128)LastName: NVARCHAR(128)Email: VARCHAR(128) «UNIQUE»CreatedAt: DATETIMEUpdatedAt: DATETIMEDateOfBirth: DATEUserCredentialUserCredentialID: UNIQUEIDENTIFIER «PK»UserAccountID: UNIQUEIDENTIFIER «FK»CreatedAt: DATETIMEExpiry: DATETIMEHash: NVARCHAR(256)IsRevoked: BITRevokedAt: DATETIMEUserVerificationUserVerificationID: UNIQUEIDENTIFIER «PK»UserAccountID: UNIQUEIDENTIFIER «FK, UNIQUE»VerificationDateTime: DATETIMEUserProfileUserProfileID: UNIQUEIDENTIFIER «PK»UserAccountID: UNIQUEIDENTIFIER «FK, UNIQUE»Biography: NVARCHAR(MAX)UserAvatarUserAvatarID: UNIQUEIDENTIFIER «PK»UserProfileID: UNIQUEIDENTIFIER «FK»PhotoID: UNIQUEIDENTIFIER «FK»ValidFrom: DATETIME2(3)ValidTo: DATETIME2(3)At most one row per UserProfileIDhas ValidTo IS NULL (active avatar)UserFollow (*)UserFollowID: UNIQUEIDENTIFIER «PK»UserAccountID: UNIQUEIDENTIFIER «FK»FollowingID: UNIQUEIDENTIFIER «FK»CreatedAt: DATETIMEPhoto (*)PhotoID: UNIQUEIDENTIFIER «PK»Hyperlink: NVARCHAR(256)UploadedByID: UNIQUEIDENTIFIER «FK»UploadedAt: DATETIMECountryCountryID: UNIQUEIDENTIFIER «PK»CountryName: NVARCHAR(100)ISO3166_1: CHAR(2) «UNIQUE»StateProvinceStateProvinceID: UNIQUEIDENTIFIER «PK»StateProvinceName: NVARCHAR(100)ISO3166_2: CHAR(6) «UNIQUE»CountryID: UNIQUEIDENTIFIER «FK»CityCityID: UNIQUEIDENTIFIER «PK»CityName: NVARCHAR(100)StateProvinceID: UNIQUEIDENTIFIER «FK»BreweryPostBreweryPostID: UNIQUEIDENTIFIER «PK»BreweryName: NVARCHAR(256)PostedByID: UNIQUEIDENTIFIER «FK»Description: NVARCHAR(512)CreatedAt: DATETIMEUpdatedAt: DATETIMEBreweryPostLocationBreweryPostLocationID: UNIQUEIDENTIFIER «PK»BreweryPostID: UNIQUEIDENTIFIER «FK, UNIQUE»AddressLine1: NVARCHAR(256)AddressLine2: NVARCHAR(256)PostalCode: NVARCHAR(20)CityID: UNIQUEIDENTIFIER «FK»Coordinates: GEOGRAPHYBreweryPostPhoto (*)BreweryPostPhotoID: UNIQUEIDENTIFIER «PK»BreweryPostID: UNIQUEIDENTIFIER «FK»PhotoID: UNIQUEIDENTIFIER «FK»LinkedAt: DATETIMEBeerStyle (*)BeerStyleID: UNIQUEIDENTIFIER «PK»StyleName: NVARCHAR(100) «UNIQUE»Description: NVARCHAR(MAX)BeerPost (*)BeerPostID: UNIQUEIDENTIFIER «PK»Name: NVARCHAR(100)Description: NVARCHAR(MAX)ABV: DECIMAL(4,2)IBU: INTPostedByID: UNIQUEIDENTIFIER «FK»BeerStyleID: UNIQUEIDENTIFIER «FK»BrewedByID: UNIQUEIDENTIFIER «FK»CreatedAt: DATETIMEUpdatedAt: DATETIMEBeerPostPhoto (*)BeerPostPhotoID: UNIQUEIDENTIFIER «PK»BeerPostID: UNIQUEIDENTIFIER «FK»PhotoID: UNIQUEIDENTIFIER «FK»LinkedAt: DATETIMEBeerPostComment (*)BeerPostCommentID: UNIQUEIDENTIFIER «PK»Comment: NVARCHAR(250)BeerPostID: UNIQUEIDENTIFIER «FK»CommentedByID: UNIQUEIDENTIFIER «FK»Rating: INTCreatedAt: DATETIMEUpdatedAt: DATETIMEAuth reads/writes (registration, login, credentialrotation, verification) go through ASP.NET CoreIdentity's UserManager<ApplicationUser>, backed byFeatures.Users' DapperUserStore - inline SQL, nostored procedures. Profile/avatar data goes throughFeatures.Users' UserProfileRepository.Backed by Features.Breweries' BreweryRepository(Dapper); create/update/delete run inline SQLinside a transaction, with optimistic concurrencyvia RowVersionhashashashas (history)refers tofollows (UserAccountID)followed by (FollowingID)uploadedhashaspostedhaslocated inhaslinked topostedcategorizesbrewedhaslinked tohascommentedEvery table also has a RowVersion ROWVERSION column (optimistic concurrency), omitted below for brevity.Tables marked (*) have no repository/feature slice yet. \ No newline at end of file diff --git a/docs/website/diagrams-out/deployment.svg b/docs/website/diagrams-out/deployment.svg deleted file mode 100644 index a0ca5751..00000000 --- a/docs/website/diagrams-out/deployment.svg +++ /dev/null @@ -1 +0,0 @@ -Docker Deployment ArchitectureDevelopment Environment(docker-compose.dev.yaml)SQL Server(mcr.microsoft.com/mssql/server:2022-latest)BiergartenDatabaseAPI Container(API.Core)Migrations(run-once)Seed(run-once)Mailpit(axllent/mailpit)Test Environment(docker-compose.test.yaml)SQL Server(isolated instance)BiergartenTest DatabaseMigrations(test)Seed(test)API.Specs(Integration Tests)unit.tests(Features.*.Tests, one container)Production Environment(docker-compose.prod.yaml)SQL Server(mcr.microsoft.com/mssql/server:2022-latest)Migrations(run-once)API Container(API.Core)Database-only Environment(docker-compose.db.yaml)SQL ServerMinimal Environment(docker-compose.min.yaml)SQL ServerMailpittest-results/(mounted volume)Environment:- ACCEPT_EULA=Y- SA_PASSWORD=***- MSSQL_PID=Developer Volumes:- biergarten-dev-data1433ASP.NET Core 108080:8080 (HTTP)8081:8081 (HTTPS)Environment:- ASPNETCORE_ENVIRONMENT=Development- DB_SERVER=sql-server- DB_NAME=Biergarten- DB_USER/PASSWORD- JWT_SECRET- SMTP_* (10+ variables) Health Check:/health endpointDatabase.MigrationsRuns: DbUp migrationsEnvironment:- CLEAR_DATABASE=falseDepends on: sql-serverDatabase.SeedCreates:- 100 test users- Location data (US/CA/MX)- test.user accountDepends on: migrations8025 (Web UI)1025 (SMTP)Local SMTP catcher for dev email testing.Environment:- MP_SMTP_AUTH_ACCEPT_ANY=1- MP_SMTP_AUTH_ALLOW_INSECURE=1Fresh instance each runCLEAR_DATABASE=true Volumes:- biergarten-test-data(ephemeral)1434Database.MigrationsDatabase.SeedMinimal seed:- test.user only- Essential dataReqnroll + xUnitTests:- Registration flow- Login flow- Validation rules- 404 handling Uses: TestApiFactoryMocks: Email servicesxUnit + MoqBuilds Dockerfile.tests, whichrestores/builds against Core.slnxthen runs `dotnet test` for everyFeatures/*.Tests project in turn:- Features.Users.Tests- Features.Breweries.Tests- Features.Emails.Tests Uses: Moq to mock each handler'srepository/service dependenciesNo real database neededProd Network (bridge: prodnet)Isolated from dev/test networks.Volumes: sqlserverdata-prod,nuget-cache-prodBiergartenDatabaseEnvironment:- ACCEPT_EULA=Y- SA_PASSWORD=***- MSSQL_PID=Express Volumes:- sqlserverdata-prodNo host port published.Database.MigrationsDepends on: sql-server (healthy)No seed step in production.ASP.NET Core 108080:8080 (HTTP)8081:8081 (HTTPS)Environment:- ASPNETCORE_ENVIRONMENT=Production- DB_SERVER/NAME/USER/PASSWORD- ACCESS/REFRESH/CONFIRMATION_TOKEN_SECRET- WEBSITE_BASE_URL Depends on: sql-server (healthy) -does not wait on migrations/seed.Same devnet volumes/images asdocker-compose.dev.yaml, minusapi.core and mailpit - for workingagainst the database withoutrunning the API.BiergartenDatabaseMigrations(run-once)Seed(run-once)Database + mailpit only, no APIcontainer - for frontend developmentagainst an API.Core process runoutside Docker.BiergartenDatabaseMigrations(run-once)8025 / 1025Seed(run-once)api-specs/results.trxFeatures.Users.Tests.trxFeatures.Breweries.Tests.trxFeatures.Emails.Tests.trxTRX formatReadable by:- Visual Studio- Azure DevOps- GitHub ActionsDeveloperDocker HostDev Network (bridge: biergarten-dev)Internal DNS:- sql-server (resolves to SQL container)- api (resolves to API container)Test Network (bridge: biergarten-test)All test components isolatedStartup Order:1. SQL Server (health check)2. Migrations (run-once)3. Seed (run-once)4. API (long-running)Test Execution:All tests run in parallelResults aggregated1. Run migrationsdepends_on2. Connect & servedepends_on1. Run migrationsdepends_on2. Seed datadepends_on1. Run migrationsdepends_on2. Seed datadepends_ondocker compose uphttp://localhost:80801. Run migrationsdepends_on2. Seed data3. Connect & serve4. SMTP_HOST/PORTdepends_ondepends_on1. Migratedepends_on2. Seed3. Integration testdepends_on for startupordering only (Moq mocks,no real connection)depends_ondepends_ondepends_onExport TRXExport TRX (one .trx per slice) \ No newline at end of file diff --git a/docs/website/diagrams-src/architecture.puml b/docs/website/diagrams-src/architecture.puml deleted file mode 100644 index e634f6c2..00000000 --- a/docs/website/diagrams-src/architecture.puml +++ /dev/null @@ -1,146 +0,0 @@ -@startuml architecture -!theme plain -skinparam backgroundColor #FFFFFF -skinparam defaultFontName Arial -skinparam packageStyle rectangle - -title The Biergarten App - Vertical Slice Architecture - -package "API.Core (thin host)" #E3F2FD { - [API.Core\nASP.NET Core Web API host] as API - note right of API - - Program.cs wiring only - - MediatR + AddApplicationPart - per Features.* assembly - - Swagger/OpenAPI, health checks - - JWT auth middleware - - Global exception filter - end note -} - -package "Feature Slices" #F3E5F5 { - [Features.Users] as UsersSlice - [Features.Breweries] as BrewerySlice - [Features.Emails] as EmailsSlice - [Features.Locations] as LocationsSlice - note right of UsersSlice - Each slice owns its own: - - Controller (HTTP endpoints) - - Commands/Queries + Handlers - - Validators - - Repository (Dapper) - - Features.Users covers auth - (Commands/Authentication), - account management - (Commands/Account), and - profile/avatar (Commands/Profile) - end note - note right of EmailsSlice - No controller - invoked only - via MediatR commands sent - from other slices - end note - note right of LocationsSlice - No controller and no MediatR - handlers - exposes ILocationRepository - directly as a plain DI service. - Not registered by API.Core at all; - only Database.Seed depends on it today - end note -} - -package "Shared" #FCE4EC { - [Shared.Contracts] as SharedContracts - [Shared.Application] as SharedApp - note right of SharedApp - - ValidationBehavior - (MediatR pipeline) - - Cross-slice email commands - (SendRegistrationEmailCommand, - SendResendConfirmationEmailCommand) - end note -} - -package "Infrastructure Layer" #FFF3E0 { - [Infrastructure.Sql] as Sql - [Infrastructure.Jwt] as JWT - [Infrastructure.PasswordHashing] as PwdHash - [Infrastructure.Email] as Email - [Infrastructure.Email.Templates] as EmailTemplates -} - -package "Domain Layer" #E8F5E9 { - [Domain.Entities] as Domain - [Domain.Exceptions] as DomainExceptions - note right of Domain - - UserAccount, UserCredential, UserVerification - - UserProfile, UserAvatar - - BreweryPost, BreweryPostLocation - - City, StateProvince, Country - - BeerPost, BeerStyle - (schema tables exist; no repository yet) - end note -} - -database "SQL Server" { - [Tables] as Tables -} - -node "Database.Seed\n(separate console app)" as Seed - -' Relationships -API ..> UsersSlice : AddApplicationPart\n+ MediatR scan -API ..> BrewerySlice : AddApplicationPart\n+ MediatR scan -API ..> EmailsSlice : MediatR scan only\n(no controller) - -Seed ..> UsersSlice : AddFeaturesUsers()\n(DI only, no API/MediatR) -Seed ..> BrewerySlice : AddFeaturesBreweries() -Seed ..> LocationsSlice : AddFeaturesLocations() - -UsersSlice --> Sql -UsersSlice --> JWT -UsersSlice --> PwdHash -UsersSlice --> SharedContracts -UsersSlice --> SharedApp - -BrewerySlice --> Sql -BrewerySlice --> SharedContracts -BrewerySlice --> SharedApp - -LocationsSlice --> Sql - -EmailsSlice --> Email -EmailsSlice --> EmailTemplates -EmailsSlice --> SharedApp - -' The one cross-slice interaction: a MediatR command contract, not a project reference -UsersSlice ..> SharedApp : sends\nSendRegistrationEmailCommand -EmailsSlice ..> SharedApp : handles\nSendRegistrationEmailCommand - -Sql --> Domain - -UsersSlice --> Tables : Dapper SQL -BrewerySlice --> Tables : Dapper SQL -LocationsSlice --> Tables : Dapper SQL - -UsersSlice --> Domain -BrewerySlice --> Domain -LocationsSlice --> Domain -UsersSlice --> DomainExceptions - -' Notes -note left of Sql - SQL-first approach - Each slice's own repository issues - inline SQL via Dapper - no ORM, - no stored procedures -end note - -note bottom of UsersSlice - No Features.* project - ever references another - Features.* project directly -end note - -@enduml diff --git a/docs/website/diagrams-src/authentication-flow.puml b/docs/website/diagrams-src/authentication-flow.puml deleted file mode 100644 index b5c97f47..00000000 --- a/docs/website/diagrams-src/authentication-flow.puml +++ /dev/null @@ -1,927 +0,0 @@ -@startuml authentication-flow -!theme plain -skinparam backgroundColor #FFFFFF -skinparam defaultFontName Arial -skinparam sequenceMessageAlign center -skinparam maxMessageSize 200 - -title User Authentication Flow - Expanded - -actor User -participant "API\nController" as API -participant "MediatR pipeline\n(ValidationBehavior)" as Mediator -box "Handler Layer" #LightBlue -participant "RegisterUserHandler" as RegHandler -participant "LoginHandler" as LoginHandler -participant "RefreshTokenHandler" as RefreshHandler -participant "ConfirmUserHandler" as ConfirmHandler -participant "ResendConfirmationEmailHandler" as ResendHandler -participant "UpdateUsernameHandler" as UpdateUsernameHandler -participant "UpdateEmailHandler" as UpdateEmailHandler -participant "UpdatePasswordHandler" as UpdatePasswordHandler -participant "UpdateProfileHandler" as UpdateProfileHandler -participant "DeleteAccountHandler" as DeleteAccountHandler -participant "UserManager" as UserMgr -participant "TokenService" as TokenSvc -end box -box "Infrastructure Layer" #LightGreen -participant "Argon2PasswordHasher" as Argon2 -participant "JWT\nInfrastructure" as JWT -participant "Email\nProvider" as SMTP -participant "Template\nProvider" as Template -end box -box "Repository / Cross-slice" #LightYellow -participant "DapperUserStore\n(IUserStore/IUserPasswordStore/\nIUserEmailStore)" as UserStore -participant "EmailDispatcher\n(Features.Emails)" as EmailDispatcher -end box -database "SQL Server" as DB - -note over UserMgr - ASP.NET Core Identity's UserManager - replaces the deleted AuthRepository: handlers - no longer issue SQL or call Argon2 directly, - they delegate to UserManager, which in turn - calls DapperUserStore (persistence) and - Argon2PasswordHasher (IPasswordHasher). -end note - -== Registration Flow == - -User -> API: POST /api/auth/register\n{username, firstName, lastName,\nemail, dateOfBirth, password} -activate API - -API -> Mediator: Send(RegisterUserCommand) -activate Mediator - -note right of Mediator - ValidationBehavior runs RegisterUserValidator - (FluentValidation) before the handler: - - Username: 3-64 chars, alphanumeric + [._-] - - Email: valid format, max 128 chars - - Password: min 8 chars, uppercase,\n lowercase, number, special char - - DateOfBirth: must be 19+ years old -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request\n{message: "Validation failed", errors: {...}} -else Validation succeeds - Mediator -> RegHandler: Handle(command) - activate RegHandler - - RegHandler -> RegHandler: user = new UserAccount{...}\n.ToApplicationUser() - - RegHandler -> UserMgr: CreateAsync(user, password) - activate UserMgr - - UserMgr -> Argon2: HashPassword(user, password) - activate Argon2 - note right of Argon2 - Delegates to IPasswordInfrastructure.Hash: - Argon2id parameters: - - Salt: 16 bytes (128-bit) - - Memory: 64MB - - Iterations: 4 - - Parallelism: CPU count - - Hash output: 32 bytes - end note - Argon2 --> UserMgr: "base64(salt):base64(hash)" - deactivate Argon2 - - UserMgr -> UserStore: SetPasswordHashAsync(user, hash) - note right of UserStore - user.Id is still empty at this point - - only sets the in-memory hash. CreateAsync - writes it to dbo.UserCredential itself. - end note - - UserMgr -> UserStore: FindByNameAsync(username) - activate UserStore - UserStore -> DB: SELECT UserAccountID, Username, ...\nFROM dbo.UserAccount\nWHERE Username = @Username - activate DB - DB --> UserStore: null (user doesn't exist) - deactivate DB - deactivate UserStore - - UserMgr -> UserStore: FindByEmailAsync(email) - activate UserStore - UserStore -> DB: SELECT UserAccountID, Username, ...\nFROM dbo.UserAccount\nWHERE Email = @Email - activate DB - DB --> UserStore: null (email doesn't exist) - deactivate DB - deactivate UserStore - note right of UserMgr - Default UserManager validators check - uniqueness before CreateAsync is called; - DapperUserStore.CreateAsync also guards - against a genuine race via a caught - duplicate-key SqlException (2601/2627). - end note - - UserMgr -> UserStore: CreateAsync(user) - activate UserStore - - UserStore -> DB: BEGIN TRANSACTION - activate DB - UserStore -> DB: INSERT INTO dbo.UserAccount\n(Username, FirstName, LastName,\nDateOfBirth, Email)\nOUTPUT INSERTED.UserAccountID - DB --> UserStore: UserAccountId (GUID) - UserStore -> DB: INSERT INTO dbo.UserCredential\n(UserAccountId, Hash) - UserStore -> DB: COMMIT TRANSACTION - deactivate DB - note right of UserStore - Both statements run via Dapper's - ExecuteScalarAsync/ExecuteAsync against - a single DbTransaction, not a stored - procedure - no manual parameter binding - end note - - UserStore --> UserMgr: IdentityResult.Success\n(user.Id set) - deactivate UserStore - - UserMgr --> RegHandler: IdentityResult - deactivate UserMgr - - alt !result.Succeeded - RegHandler -> Mediator: throw ConflictException\n(DuplicateUserName/DuplicateEmail codes ->\n"Username or email already exists";\notherwise joined error descriptions) - Mediator -> API: propagate - API -> User: 409 Conflict\n"Username or email already exists" - else result.Succeeded - - RegHandler -> TokenSvc: GenerateAccessToken(user) - activate TokenSvc - TokenSvc -> JWT: GenerateJwt(userId, username, expiry) - activate JWT - note right of JWT - JWT Configuration: - - Algorithm: HS256 - - Expires: 1 hour - - Claims: - * sub: userId - * unique_name: username - * iat / jti - end note - JWT -> JWT: Create JWT with claims - JWT -> JWT: Sign with secret key - JWT --> TokenSvc: Access Token - deactivate JWT - TokenSvc --> RegHandler: Access Token - deactivate TokenSvc - - RegHandler -> TokenSvc: GenerateRefreshToken(user) - activate TokenSvc - TokenSvc -> JWT: GenerateJwt(userId, username, expiry) - activate JWT - note right of JWT - Refresh Token: - - Expires: 21 days (504 hours) - - Same structure as access token,\n signed with a separate secret - end note - JWT --> TokenSvc: Refresh Token - deactivate JWT - TokenSvc --> RegHandler: Refresh Token - deactivate TokenSvc - - RegHandler -> TokenSvc: GenerateConfirmationToken(user) - activate TokenSvc - TokenSvc -> JWT: GenerateJwt(userId, username, expiry) - activate JWT - note right of JWT - Confirmation Token: - - Expires: 30 minutes (0.5 hours) - - Signed with a third, separate secret - end note - JWT --> TokenSvc: Confirmation Token - deactivate JWT - TokenSvc --> RegHandler: Confirmation Token - deactivate TokenSvc - - RegHandler -> Mediator: Send(SendRegistrationEmailCommand)\n(Shared.Application cross-slice contract) - activate Mediator - Mediator -> EmailDispatcher: SendRegistrationEmailAsync(\n firstName, email, confirmationToken) - note right of Mediator - Routed to Features.Emails' - SendRegistrationEmailHandler, - the only cross-slice call in - the backend - end note - activate EmailDispatcher - - EmailDispatcher -> Template: RenderUserRegisteredEmailAsync(\n firstName, confirmationLink) - activate Template - note right of Template - Razor Component: - - Header with branding - - Welcome message - - Confirmation button - - Footer - end note - Template -> Template: Render Razor component\nto HTML - Template --> EmailDispatcher: HTML email content - deactivate Template - - EmailDispatcher -> SMTP: SendAsync(email, subject, body) - activate SMTP - note right of SMTP - SMTP Configuration (MailKit): - - Host/port: from env (SMTP_HOST/SMTP_PORT) - - TLS: StartTLS when SMTP_USE_SSL=true - - Auth: username/password, if configured - end note - SMTP -> SMTP: Build MIME message - SMTP -> SMTP: Connect, authenticate,\nsend, disconnect - SMTP --> EmailDispatcher: Success / throws - deactivate SMTP - - deactivate EmailDispatcher - deactivate Mediator - - alt Email sent successfully - RegHandler -> RegHandler: emailSent = true - else mediator.Send throws - RegHandler -> RegHandler: catch(Exception),\nlog to stderr,\nemailSent = false - end - - RegHandler --> Mediator: RegistrationPayload(\n userAccountId, username,\n refreshToken, accessToken,\n confirmationEmailSent) - deactivate RegHandler - Mediator --> API: RegistrationPayload - deactivate Mediator - - API -> User: 201 Created\n{\n message: "User registered successfully.",\n payload: {\n userAccountId, username,\n refreshToken, accessToken,\n confirmationEmailSent\n }\n} - end -end -deactivate API - -== Login Flow == - -User -> API: POST /api/auth/login\n{username, password} -activate API - -API -> Mediator: Send(LoginCommand) -activate Mediator - -note right of Mediator - ValidationBehavior runs LoginValidator: - username and password must both be non-empty -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request\n{message: "Validation failed", errors: {...}} -else Validation succeeds - - Mediator -> LoginHandler: Handle(command) - activate LoginHandler - - LoginHandler -> UserMgr: FindByNameAsync(username) - activate UserMgr - UserMgr -> UserStore: FindByNameAsync(username) - activate UserStore - UserStore -> DB: SELECT UserAccountID, Username, ...\nFROM dbo.UserAccount\nWHERE Username = @Username - activate DB - DB --> UserStore: UserAccount row or null - deactivate DB - UserStore -> UserStore: HydrateAsync\n(load active UserCredential.Hash,\nUserVerification -> EmailConfirmed) - UserStore --> UserMgr: ApplicationUser or null - deactivate UserStore - UserMgr --> LoginHandler: ApplicationUser or null - deactivate UserMgr - - alt User not found - LoginHandler -> Mediator: throw UnauthorizedException\n"Invalid username or password." - Mediator -> API: propagate - API -> User: 401 Unauthorized - else User found - - LoginHandler -> UserMgr: CheckPasswordAsync(user, password) - activate UserMgr - note right of UserMgr - HasPasswordAsync short-circuits to false - (no active UserCredential was hydrated) - without calling the hasher; otherwise - delegates to Argon2PasswordHasher. - end note - UserMgr -> Argon2: VerifyHashedPassword(\n user, user.PasswordHash, password) - activate Argon2 - note right of Argon2 - Delegates to IPasswordInfrastructure.Verify: - 1. Split stored hash: "salt:hash" - 2. Hash provided password with the same salt - 3. Constant-time comparison\n (CryptographicOperations.FixedTimeEquals) - end note - Argon2 --> UserMgr: Success/Failed - deactivate Argon2 - UserMgr --> LoginHandler: true/false - deactivate UserMgr - - alt Password invalid - LoginHandler -> Mediator: throw UnauthorizedException - Mediator -> API: propagate - API -> User: 401 Unauthorized - else Password valid - - LoginHandler -> TokenSvc: GenerateAccessToken(user) - activate TokenSvc - TokenSvc -> JWT: GenerateJwt(...) - activate JWT - JWT --> TokenSvc: Access Token - deactivate JWT - TokenSvc --> LoginHandler: Access Token - deactivate TokenSvc - - LoginHandler -> TokenSvc: GenerateRefreshToken(user) - activate TokenSvc - TokenSvc -> JWT: GenerateJwt(...) - activate JWT - JWT --> TokenSvc: Refresh Token - deactivate JWT - TokenSvc --> LoginHandler: Refresh Token - deactivate TokenSvc - - LoginHandler --> Mediator: LoginPayload(\n userAccountId, username,\n refreshToken, accessToken) - deactivate LoginHandler - Mediator --> API: LoginPayload - deactivate Mediator - - API -> User: 200 OK\n{\n message: "Logged in successfully.",\n payload: {\n userAccountId, username,\n refreshToken, accessToken\n }\n} - end - end - end -deactivate API - -== Refresh Token Flow == - -User -> API: POST /api/auth/refresh\nHeader: X-Refresh-Token: -activate API - -API -> Mediator: Send(RefreshTokenCommand(refreshToken)) -activate Mediator - -note right of Mediator - ValidationBehavior runs RefreshTokenValidator: - refreshToken must be non-empty. Anonymous - endpoint - no Bearer token required. -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request\n{message: "Validation failed", errors: {...}} -else Validation succeeds - - Mediator -> RefreshHandler: Handle(command) - activate RefreshHandler - - RefreshHandler -> TokenSvc: RefreshTokenAsync(refreshToken) - activate TokenSvc - - TokenSvc -> JWT: ValidateRefreshTokenAsync(refreshToken) - activate JWT - note right of JWT - Validates signature/expiry against the - refresh token secret, extracts userId/ - username claims. - end note - JWT --> TokenSvc: ValidatedToken or throws - deactivate JWT - - alt Token invalid or expired - TokenSvc -> RefreshHandler: throw UnauthorizedException - RefreshHandler -> Mediator: propagate - Mediator -> API: propagate - API -> User: 401 Unauthorized - else Token valid - - TokenSvc -> UserMgr: FindByIdAsync(validated.UserId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userId) - activate UserStore - UserStore -> DB: SELECT UserAccountID, Username, ...\nFROM dbo.UserAccount\nWHERE UserAccountID = @Id - activate DB - DB --> UserStore: row or null - deactivate DB - UserStore --> UserMgr: ApplicationUser or null - deactivate UserStore - UserMgr --> TokenSvc: ApplicationUser or null - deactivate UserMgr - - alt User not found - TokenSvc -> RefreshHandler: throw UnauthorizedException\n"User account not found" - RefreshHandler -> Mediator: propagate - Mediator -> API: propagate - API -> User: 401 Unauthorized - else User found - TokenSvc -> JWT: GenerateAccessToken(user)\nGenerateRefreshToken(user) - JWT --> TokenSvc: new access + refresh tokens - TokenSvc --> RefreshHandler: RefreshTokenResult - deactivate TokenSvc - - RefreshHandler --> Mediator: LoginPayload(\n userId, username,\n refreshToken, accessToken) - deactivate RefreshHandler - Mediator --> API: LoginPayload - deactivate Mediator - - API -> User: 200 OK\n{message: "Token refreshed successfully.",\npayload: {...}} - end - end -end -deactivate API - -== Confirm User Flow == - -User -> API: POST /api/auth/confirm?token=\n(requires Bearer access token) -activate API - -API -> Mediator: Send(ConfirmUserCommand(token)) -activate Mediator - -Mediator -> ConfirmHandler: Handle(command) -activate ConfirmHandler - -ConfirmHandler -> TokenSvc: ValidateConfirmationTokenAsync(token) -activate TokenSvc -TokenSvc -> JWT: Validate against confirmation\ntoken secret, extract userId -activate JWT -JWT --> TokenSvc: ValidatedToken or throws -deactivate JWT -TokenSvc --> ConfirmHandler: ValidatedToken -deactivate TokenSvc - -alt Token invalid or expired - ConfirmHandler -> Mediator: throw UnauthorizedException - Mediator -> API: propagate - API -> User: 401 Unauthorized -else Token valid - - ConfirmHandler -> UserMgr: FindByIdAsync(validatedToken.UserId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userId) - activate UserStore - UserStore -> DB: SELECT ...\nFROM dbo.UserAccount\nWHERE UserAccountID = @Id - activate DB - DB --> UserStore: row or null - deactivate DB - UserStore --> UserMgr: ApplicationUser or null - deactivate UserStore - UserMgr --> ConfirmHandler: ApplicationUser or null - deactivate UserMgr - - alt User not found - ConfirmHandler -> Mediator: throw UnauthorizedException\n"User account not found" - Mediator -> API: propagate - API -> User: 401 Unauthorized - else User found - - ConfirmHandler -> UserStore: SetEmailConfirmedAsync(user, true) - note right of UserStore - IUserEmailStore is injected - directly into the handler (not via UserManager), - setting the confirmed flag in-memory here. - end note - - ConfirmHandler -> UserMgr: UpdateAsync(user) - activate UserMgr - UserMgr -> UserStore: UpdateAsync(user) - activate UserStore - UserStore -> DB: UPDATE dbo.UserAccount SET ... - UserStore -> DB: INSERT INTO dbo.UserVerification\n(UserAccountId, VerificationDateTime) - note right of UserStore - Idempotent: skipped (duplicate-key - exception swallowed) if a verification - row already exists. - end note - UserStore --> UserMgr: IdentityResult.Success - deactivate UserStore - UserMgr --> ConfirmHandler: IdentityResult - deactivate UserMgr - - ConfirmHandler --> Mediator: ConfirmationPayload(\n user.Id, DateTime.UtcNow) - deactivate ConfirmHandler - Mediator --> API: ConfirmationPayload - deactivate Mediator - - API -> User: 200 OK\n{message: "User with ID {id} is confirmed.",\npayload: {...}} - end -end -deactivate API - -== Resend Confirmation Email Flow == - -User -> API: POST /api/auth/confirm/resend?userId=\n(requires Bearer access token) -activate API - -API -> Mediator: Send(ResendConfirmationEmailCommand(userId)) -activate Mediator - -Mediator -> ResendHandler: Handle(command) -activate ResendHandler - -ResendHandler -> UserMgr: FindByIdAsync(userId) -activate UserMgr -UserMgr -> UserStore: FindByIdAsync(userId) -activate UserStore -UserStore -> DB: SELECT ...\nFROM dbo.UserAccount\nWHERE UserAccountID = @Id -activate DB -DB --> UserStore: row or null -deactivate DB -UserStore --> UserMgr: ApplicationUser or null -deactivate UserStore -UserMgr --> ResendHandler: ApplicationUser or null -deactivate UserMgr - -alt User not found - ResendHandler -> ResendHandler: return silently\n(prevents user enumeration) -else User found - - ResendHandler -> UserMgr: IsEmailConfirmedAsync(user) - UserMgr --> ResendHandler: bool - - alt Already confirmed - ResendHandler -> ResendHandler: return silently (no-op) - else Not yet confirmed - - ResendHandler -> TokenSvc: GenerateConfirmationToken(user) - activate TokenSvc - TokenSvc --> ResendHandler: confirmation token - deactivate TokenSvc - - ResendHandler -> Mediator: Send(SendResendConfirmationEmailCommand) - activate Mediator - Mediator -> EmailDispatcher: SendResendConfirmationEmailAsync(\n firstName, email, confirmationToken) - activate EmailDispatcher - EmailDispatcher -> Template: Render email\n(same Razor pipeline as registration) - EmailDispatcher -> SMTP: SendAsync(email, subject, body) - deactivate EmailDispatcher - deactivate Mediator - end -end - -ResendHandler --> Mediator: (void) -deactivate ResendHandler -Mediator --> API: (void) -deactivate Mediator - -API -> User: 200 OK\n{message: "confirmation email has been resent"} -note right of API - Always 200 OK regardless of outcome above - - same user-enumeration precaution as a not-found - user or an already-confirmed account. -end note -deactivate API - -== Update Username Flow == - -User -> API: PATCH /api/auth/username\n{newUsername}\n(requires Bearer access token) -activate API - -API -> Mediator: Send(UpdateUsernameCommand(\n GetAuthenticatedUserId(), newUsername)) -activate Mediator - -note right of Mediator - ValidationBehavior runs UpdateUsernameValidator - (same format rules as registration) -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request -else Validation succeeds - - Mediator -> UpdateUsernameHandler: Handle(command) - activate UpdateUsernameHandler - - UpdateUsernameHandler -> UserMgr: FindByIdAsync(userAccountId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userAccountId) - activate UserStore - UserStore -> DB: SELECT ...\nFROM dbo.UserAccount\nWHERE UserAccountID = @Id - activate DB - DB --> UserStore: row or null - deactivate DB - UserStore --> UserMgr: ApplicationUser or null - deactivate UserStore - UserMgr --> UpdateUsernameHandler: ApplicationUser or null - deactivate UserMgr - - alt Account not found - UpdateUsernameHandler -> Mediator: throw NotFoundException - Mediator -> API: propagate - API -> User: 404 Not Found - else Account found - - UpdateUsernameHandler -> UserMgr: SetUserNameAsync(user, newUsername) - activate UserMgr - UserMgr -> UserStore: FindByNameAsync(newUsername) - note right of UserMgr - Uniqueness check before the UPDATE. - end note - UserMgr -> UserStore: UpdateAsync(user) - UserStore -> DB: UPDATE dbo.UserAccount\nSET Username = @UserName, ... - UserStore --> UserMgr: IdentityResult - UserMgr --> UpdateUsernameHandler: IdentityResult - deactivate UserMgr - - alt !result.Succeeded - UpdateUsernameHandler -> Mediator: throw ConflictException\n(DuplicateUserName code ->\n"Username already exists") - Mediator -> API: propagate - API -> User: 409 Conflict - else result.Succeeded - UpdateUsernameHandler --> Mediator: UpdateUsernamePayload(\n user.Id, user.UserName) - deactivate UpdateUsernameHandler - Mediator --> API: UpdateUsernamePayload - deactivate Mediator - API -> User: 200 OK\n{message: "Username updated successfully.",\npayload: {...}} - end - end -end -deactivate API - -== Update Email Flow == - -User -> API: PATCH /api/auth/email\n{newEmail}\n(requires Bearer access token) -activate API - -API -> Mediator: Send(UpdateEmailCommand(\n GetAuthenticatedUserId(), newEmail)) -activate Mediator - -note right of Mediator - ValidationBehavior runs UpdateEmailValidator - (same format rules as registration) -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request -else Validation succeeds - - Mediator -> UpdateEmailHandler: Handle(command) - activate UpdateEmailHandler - - UpdateEmailHandler -> UserMgr: FindByIdAsync(userAccountId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userAccountId) - UserStore -> DB: SELECT ... - UserStore --> UserMgr: ApplicationUser or null - UserMgr --> UpdateEmailHandler: ApplicationUser or null - deactivate UserMgr - - alt Account not found - UpdateEmailHandler -> Mediator: throw NotFoundException - Mediator -> API: propagate - API -> User: 404 Not Found - else Account found - - UpdateEmailHandler -> UserMgr: SetEmailAsync(user, newEmail) - activate UserMgr - UserMgr -> UserStore: FindByEmailAsync(newEmail) - note right of UserMgr - Uniqueness check. Also resets - user.EmailConfirmed = false - - the caller must re-confirm via - POST /api/auth/confirm/resend. - end note - UserMgr -> UserStore: UpdateAsync(user) - UserStore -> DB: UPDATE dbo.UserAccount\nSET Email = @Email, ... - UserStore -> DB: DELETE FROM dbo.UserVerification\nWHERE UserAccountID = @Id - UserStore --> UserMgr: IdentityResult - UserMgr --> UpdateEmailHandler: IdentityResult - deactivate UserMgr - - alt !result.Succeeded - UpdateEmailHandler -> Mediator: throw ConflictException\n(DuplicateEmail code ->\n"Email address already exists") - Mediator -> API: propagate - API -> User: 409 Conflict - else result.Succeeded - UpdateEmailHandler --> Mediator: UpdateEmailPayload(\n user.Id, user.Email,\n user.EmailConfirmed = false) - deactivate UpdateEmailHandler - Mediator --> API: UpdateEmailPayload - deactivate Mediator - API -> User: 200 OK\n{message: "Email updated successfully.",\npayload: {...}} - end - end -end -deactivate API - -== Update Password Flow == - -User -> API: PATCH /api/auth/password\n{currentPassword, newPassword}\n(requires Bearer access token) -activate API - -API -> Mediator: Send(UpdatePasswordCommand(\n GetAuthenticatedUserId(),\n currentPassword, newPassword)) -activate Mediator - -note right of Mediator - ValidationBehavior runs UpdatePasswordValidator - (same password strength rules as registration) -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request -else Validation succeeds - - Mediator -> UpdatePasswordHandler: Handle(command) - activate UpdatePasswordHandler - - UpdatePasswordHandler -> UserMgr: FindByIdAsync(userAccountId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userAccountId) - UserStore --> UserMgr: ApplicationUser or null - UserMgr --> UpdatePasswordHandler: ApplicationUser or null - deactivate UserMgr - - alt Account not found - UpdatePasswordHandler -> Mediator: throw NotFoundException - Mediator -> API: propagate - API -> User: 404 Not Found - else Account found - - UpdatePasswordHandler -> UserMgr: ChangePasswordAsync(\n user, currentPassword, newPassword) - activate UserMgr - UserMgr -> Argon2: VerifyHashedPassword(\n user, user.PasswordHash, currentPassword) - activate Argon2 - Argon2 --> UserMgr: Success/Failed - deactivate Argon2 - - alt Current password mismatch - UserMgr --> UpdatePasswordHandler: IdentityResult.Failed\n(PasswordMismatch) - else Current password matches - UserMgr -> Argon2: HashPassword(user, newPassword) - activate Argon2 - Argon2 --> UserMgr: new hash - deactivate Argon2 - UserMgr -> UserStore: SetPasswordHashAsync(user, newHash) - activate UserStore - UserStore -> DB: UPDATE dbo.UserCredential\nSET IsRevoked = 1, RevokedAt = GETDATE()\nWHERE UserAccountId = @Id AND IsRevoked = 0 - UserStore -> DB: INSERT INTO dbo.UserCredential\n(UserAccountId, Hash) - note right of UserStore - Revoke-then-insert runs inside one - DbTransaction (credential rotation). - end note - UserStore --> UserMgr: (void) - deactivate UserStore - UserMgr --> UpdatePasswordHandler: IdentityResult.Success - end - deactivate UserMgr - - alt Current password is incorrect - UpdatePasswordHandler -> Mediator: throw UnauthorizedException\n"Current password is incorrect" - Mediator -> API: propagate - API -> User: 401 Unauthorized - else !result.Succeeded (other failure) - UpdatePasswordHandler -> Mediator: throw ConflictException - Mediator -> API: propagate - API -> User: 409 Conflict - else result.Succeeded - UpdatePasswordHandler --> Mediator: UpdatePasswordPayload(\n user.Id, DateTime.UtcNow) - deactivate UpdatePasswordHandler - Mediator --> API: UpdatePasswordPayload - deactivate Mediator - API -> User: 200 OK\n{message: "Password updated successfully.",\npayload: {...}} - end - end -end -deactivate API - -== Update Profile Flow == - -User -> API: PATCH /api/auth/profile\n{firstName, lastName, dateOfBirth}\n(requires Bearer access token) -activate API - -API -> Mediator: Send(UpdateProfileCommand(\n GetAuthenticatedUserId(),\n firstName, lastName, dateOfBirth)) -activate Mediator - -note right of Mediator - ValidationBehavior runs UpdateProfileValidator - (same name/DateOfBirth rules as registration) -end note - -alt Validation fails - Mediator -> API: throws FluentValidation.ValidationException - API -> User: 400 Bad Request -else Validation succeeds - - Mediator -> UpdateProfileHandler: Handle(command) - activate UpdateProfileHandler - - UpdateProfileHandler -> UserMgr: FindByIdAsync(userAccountId) - activate UserMgr - UserMgr -> UserStore: FindByIdAsync(userAccountId) - UserStore --> UserMgr: ApplicationUser or null - UserMgr --> UpdateProfileHandler: ApplicationUser or null - deactivate UserMgr - - alt Account not found - UpdateProfileHandler -> Mediator: throw NotFoundException - Mediator -> API: propagate - API -> User: 404 Not Found - else Account found - - UpdateProfileHandler -> UpdateProfileHandler: user.FirstName = ...\nuser.LastName = ...\nuser.DateOfBirth = ... - note right of UpdateProfileHandler - First/last name and date of birth aren't - tracked by any Identity store interface - - mutated directly on the ApplicationUser, - then saved via UserManager.UpdateAsync. - end note - - UpdateProfileHandler -> UserMgr: UpdateAsync(user) - activate UserMgr - UserMgr -> UserStore: UpdateAsync(user) - UserStore -> DB: UPDATE dbo.UserAccount\nSET FirstName = @FirstName,\n LastName = @LastName,\n DateOfBirth = @DateOfBirth, ... - UserStore --> UserMgr: IdentityResult - UserMgr --> UpdateProfileHandler: IdentityResult - deactivate UserMgr - - alt !result.Succeeded - UpdateProfileHandler -> Mediator: throw ConflictException - Mediator -> API: propagate - API -> User: 409 Conflict - else result.Succeeded - UpdateProfileHandler --> Mediator: UpdateProfilePayload(\n user.Id, firstName,\n lastName, dateOfBirth) - deactivate UpdateProfileHandler - Mediator --> API: UpdateProfilePayload - deactivate Mediator - API -> User: 200 OK\n{message: "Profile updated successfully.",\npayload: {...}} - end - end -end -deactivate API - -== Delete Account Flow == - -User -> API: DELETE /api/auth/account\n(requires Bearer access token) -activate API - -API -> Mediator: Send(DeleteAccountCommand(\n GetAuthenticatedUserId())) -activate Mediator - -Mediator -> DeleteAccountHandler: Handle(command) -activate DeleteAccountHandler - -DeleteAccountHandler -> UserMgr: FindByIdAsync(userAccountId) -activate UserMgr -UserMgr -> UserStore: FindByIdAsync(userAccountId) -UserStore --> UserMgr: ApplicationUser or null -UserMgr --> DeleteAccountHandler: ApplicationUser or null -deactivate UserMgr - -alt Account not found - DeleteAccountHandler -> Mediator: throw NotFoundException - Mediator -> API: propagate - API -> User: 404 Not Found -else Account found - - DeleteAccountHandler -> UserMgr: DeleteAsync(user) - activate UserMgr - UserMgr -> UserStore: DeleteAsync(user) - activate UserStore - UserStore -> DB: DELETE FROM dbo.UserAccount\nWHERE UserAccountID = @Id - note right of UserStore - UserCredential/UserVerification/UserAvatar - cascade on delete. Posts, comments, photos, - and follows use ON DELETE NO ACTION - if any - reference this account, the DELETE throws a - foreign key violation (SQL error 547). - end note - - alt Foreign key violation (dependent records exist) - UserStore --> UserMgr: IdentityResult.Failed\n(AccountHasDependentRecords) - else Delete succeeds - UserStore --> UserMgr: IdentityResult.Success - end - deactivate UserStore - UserMgr --> DeleteAccountHandler: IdentityResult - deactivate UserMgr - - alt Account has dependent records - DeleteAccountHandler -> Mediator: throw ConflictException\n"Account cannot be deleted while it still\nhas associated posts, comments, photos,\nor follows." - Mediator -> API: propagate - API -> User: 409 Conflict - else !result.Succeeded (other failure) - DeleteAccountHandler -> Mediator: throw ConflictException - Mediator -> API: propagate - API -> User: 409 Conflict - else result.Succeeded - DeleteAccountHandler --> Mediator: (void) - deactivate DeleteAccountHandler - Mediator --> API: (void) - deactivate Mediator - API -> User: 200 OK\n{message: "Account deleted successfully."} - end -end -deactivate API - -== Error Handling (Global Exception Filter) == - -note over API - API.Core's GlobalExceptionFilter catches: - - FluentValidation.ValidationException -> 400 Bad Request\n {message, errors} (grouped by property) - - ConflictException -> 409 Conflict - - NotFoundException -> 404 Not Found - - UnauthorizedException -> 401 Unauthorized - - ForbiddenException -> 403 Forbidden - - Domain.Exceptions.ValidationException -> 400 Bad Request - - Microsoft.Data.SqlClient.SqlException -> 503 Service Unavailable - - All others -> 500 Internal Server Error -end note - -@enduml diff --git a/docs/website/diagrams-src/database-schema.puml b/docs/website/diagrams-src/database-schema.puml deleted file mode 100644 index 37a1269a..00000000 --- a/docs/website/diagrams-src/database-schema.puml +++ /dev/null @@ -1,221 +0,0 @@ -@startuml database-schema -!theme plain -skinparam backgroundColor #FFFFFF -skinparam defaultFontName Arial -skinparam linetype ortho -skinparam packageStyle rectangle - -title Database Schema -caption Every table also has a RowVersion ROWVERSION column (optimistic concurrency), omitted below for brevity.\nTables marked (*) have no repository/feature slice yet. - -package "User & Auth" { - entity "UserAccount" as User { - * UserAccountID: UNIQUEIDENTIFIER <> - -- - * Username: VARCHAR(64) <> - * FirstName: NVARCHAR(128) - * LastName: NVARCHAR(128) - * Email: VARCHAR(128) <> - * CreatedAt: DATETIME - UpdatedAt: DATETIME - * DateOfBirth: DATE - } - - entity "UserCredential" as Cred { - * UserCredentialID: UNIQUEIDENTIFIER <> - -- - * UserAccountID: UNIQUEIDENTIFIER <> - * CreatedAt: DATETIME - * Expiry: DATETIME - * Hash: NVARCHAR(256) - * IsRevoked: BIT - RevokedAt: DATETIME - } - - entity "UserVerification" as Verify { - * UserVerificationID: UNIQUEIDENTIFIER <> - -- - * UserAccountID: UNIQUEIDENTIFIER <> - * VerificationDateTime: DATETIME - } - - entity "UserProfile" as Profile { - * UserProfileID: UNIQUEIDENTIFIER <> - -- - * UserAccountID: UNIQUEIDENTIFIER <> - * Biography: NVARCHAR(MAX) - } - - entity "UserAvatar" as Avatar { - * UserAvatarID: UNIQUEIDENTIFIER <> - -- - * UserProfileID: UNIQUEIDENTIFIER <> - * PhotoID: UNIQUEIDENTIFIER <> - * ValidFrom: DATETIME2(3) - ValidTo: DATETIME2(3) - .. - At most one row per UserProfileID - has ValidTo IS NULL (active avatar) - } - - entity "UserFollow (*)" as Follow { - * UserFollowID: UNIQUEIDENTIFIER <> - -- - * UserAccountID: UNIQUEIDENTIFIER <> - * FollowingID: UNIQUEIDENTIFIER <> - * CreatedAt: DATETIME - } - - entity "Photo (*)" as Photo { - * PhotoID: UNIQUEIDENTIFIER <> - -- - Hyperlink: NVARCHAR(256) - * UploadedByID: UNIQUEIDENTIFIER <> - * UploadedAt: DATETIME - } -} - -package "Location" { - entity "Country" as Country { - * CountryID: UNIQUEIDENTIFIER <> - -- - * CountryName: NVARCHAR(100) - * ISO3166_1: CHAR(2) <> - } - - entity "StateProvince" as StateProvince { - * StateProvinceID: UNIQUEIDENTIFIER <> - -- - * StateProvinceName: NVARCHAR(100) - * ISO3166_2: CHAR(6) <> - * CountryID: UNIQUEIDENTIFIER <> - } - - entity "City" as City { - * CityID: UNIQUEIDENTIFIER <> - -- - * CityName: NVARCHAR(100) - * StateProvinceID: UNIQUEIDENTIFIER <> - } -} - -package "Brewery" { - entity "BreweryPost" as Brewery { - * BreweryPostID: UNIQUEIDENTIFIER <> - -- - * BreweryName: NVARCHAR(256) - * PostedByID: UNIQUEIDENTIFIER <> - * Description: NVARCHAR(512) - * CreatedAt: DATETIME - UpdatedAt: DATETIME - } - - entity "BreweryPostLocation" as BreweryLoc { - * BreweryPostLocationID: UNIQUEIDENTIFIER <> - -- - * BreweryPostID: UNIQUEIDENTIFIER <> - * AddressLine1: NVARCHAR(256) - AddressLine2: NVARCHAR(256) - * PostalCode: NVARCHAR(20) - * CityID: UNIQUEIDENTIFIER <> - Coordinates: GEOGRAPHY - } - - entity "BreweryPostPhoto (*)" as BreweryPhoto { - * BreweryPostPhotoID: UNIQUEIDENTIFIER <> - -- - * BreweryPostID: UNIQUEIDENTIFIER <> - * PhotoID: UNIQUEIDENTIFIER <> - * LinkedAt: DATETIME - } -} - -package "Beer (*)" { - entity "BeerStyle (*)" as BeerStyle { - * BeerStyleID: UNIQUEIDENTIFIER <> - -- - * StyleName: NVARCHAR(100) <> - Description: NVARCHAR(MAX) - } - - entity "BeerPost (*)" as Beer { - * BeerPostID: UNIQUEIDENTIFIER <> - -- - * Name: NVARCHAR(100) - * Description: NVARCHAR(MAX) - * ABV: DECIMAL(4,2) - * IBU: INT - * PostedByID: UNIQUEIDENTIFIER <> - * BeerStyleID: UNIQUEIDENTIFIER <> - * BrewedByID: UNIQUEIDENTIFIER <> - * CreatedAt: DATETIME - UpdatedAt: DATETIME - } - - entity "BeerPostPhoto (*)" as BeerPhoto { - * BeerPostPhotoID: UNIQUEIDENTIFIER <> - -- - * BeerPostID: UNIQUEIDENTIFIER <> - * PhotoID: UNIQUEIDENTIFIER <> - * LinkedAt: DATETIME - } - - entity "BeerPostComment (*)" as BeerComment { - * BeerPostCommentID: UNIQUEIDENTIFIER <> - -- - * Comment: NVARCHAR(250) - * BeerPostID: UNIQUEIDENTIFIER <> - * CommentedByID: UNIQUEIDENTIFIER <> - * Rating: INT - * CreatedAt: DATETIME - UpdatedAt: DATETIME - } -} - -' Relationships: User & Auth -User ||--o{ Cred : "has" -User ||--o| Verify : "has" -User ||--o| Profile : "has" -Profile ||--o{ Avatar : "has (history)" -Avatar }o--|| Photo : "refers to" -User ||--o{ Follow : "follows (UserAccountID)" -User ||--o{ Follow : "followed by (FollowingID)" -User ||--o{ Photo : "uploaded" - -' Relationships: Location -Country ||--o{ StateProvince : "has" -StateProvince ||--o{ City : "has" - -' Relationships: Brewery -User ||--o{ Brewery : "posted" -Brewery ||--o| BreweryLoc : "has" -City ||--o{ BreweryLoc : "located in" -Brewery ||--o{ BreweryPhoto : "has" -Photo ||--o{ BreweryPhoto : "linked to" - -' Relationships: Beer -User ||--o{ Beer : "posted" -BeerStyle ||--o{ Beer : "categorizes" -Brewery ||--o{ Beer : "brewed" -Beer ||--o{ BeerPhoto : "has" -Photo ||--o{ BeerPhoto : "linked to" -Beer ||--o{ BeerComment : "has" -User ||--o{ BeerComment : "commented" - -note bottom of User - Auth reads/writes (registration, login, credential - rotation, verification) go through ASP.NET Core - Identity's UserManager, backed by - Features.Users' DapperUserStore - inline SQL, no - stored procedures. Profile/avatar data goes through - Features.Users' UserProfileRepository. -end note - -note bottom of Brewery - Backed by Features.Breweries' BreweryRepository - (Dapper); create/update/delete run inline SQL - inside a transaction, with optimistic concurrency - via RowVersion -end note - -@enduml diff --git a/docs/website/diagrams-src/deployment.puml b/docs/website/diagrams-src/deployment.puml deleted file mode 100644 index cdaa2e64..00000000 --- a/docs/website/diagrams-src/deployment.puml +++ /dev/null @@ -1,332 +0,0 @@ -@startuml deployment -!theme plain -skinparam backgroundColor #FFFFFF -skinparam defaultFontName Arial -skinparam linetype ortho - -title Docker Deployment Architecture - -' External systems -actor Developer -cloud "Docker Host" as Host - -package "Development Environment\n(docker-compose.dev.yaml)" #E3F2FD { - - node "SQL Server\n(mcr.microsoft.com/mssql/server:2022-latest)" as DevDB { - database "Biergarten\nDatabase" as DevDBInner { - portin "1433" - } - note right - Environment: - - ACCEPT_EULA=Y - - SA_PASSWORD=*** - - MSSQL_PID=Developer - - Volumes: - - biergarten-dev-data - end note - } - - node "API Container\n(API.Core)" as DevAPI { - component "ASP.NET Core 10" as API1 - portin "8080:8080 (HTTP)" as DevPort1 - portin "8081:8081 (HTTPS)" as DevPort2 - - note right - Environment: - - ASPNETCORE_ENVIRONMENT=Development - - DB_SERVER=sql-server - - DB_NAME=Biergarten - - DB_USER/PASSWORD - - JWT_SECRET - - SMTP_* (10+ variables) - - Health Check: - /health endpoint - end note - } - - node "Migrations\n(run-once)" as DevMig { - component "Database.Migrations" as Mig1 - note bottom - Runs: DbUp migrations - Environment: - - CLEAR_DATABASE=false - Depends on: sql-server - end note - } - - node "Seed\n(run-once)" as DevSeed { - component "Database.Seed" as Seed1 - note bottom - Creates: - - 100 test users - - Location data (US/CA/MX) - - test.user account - Depends on: migrations - end note - } - - node "Mailpit\n(axllent/mailpit)" as DevMailpit { - portin "8025 (Web UI)" as DevMailpitWeb - portin "1025 (SMTP)" as DevMailpitSmtp - note right - Local SMTP catcher for dev email testing. - Environment: - - MP_SMTP_AUTH_ACCEPT_ANY=1 - - MP_SMTP_AUTH_ALLOW_INSECURE=1 - end note - } - -} - -package "Test Environment\n(docker-compose.test.yaml)" #FFF3E0 { - - node "SQL Server\n(isolated instance)" as TestDB { - database "Biergarten\nTest Database" as TestDBInner { - portin "1434" - } - note right - Fresh instance each run - CLEAR_DATABASE=true - - Volumes: - - biergarten-test-data - (ephemeral) - end note - } - - node "Migrations\n(test)" as TestMig { - component "Database.Migrations" - } - - node "Seed\n(test)" as TestSeed { - component "Database.Seed" - note bottom - Minimal seed: - - test.user only - - Essential data - end note - } - - node "API.Specs\n(Integration Tests)" as Specs { - component "Reqnroll + xUnit" as SpecsComp - note right - Tests: - - Registration flow - - Login flow - - Validation rules - - 404 handling - - Uses: TestApiFactory - Mocks: Email services - end note - } - - node "unit.tests\n(Features.*.Tests, one container)" as UnitTests { - component "xUnit + Moq" as UnitComp - note right - Builds Dockerfile.tests, which - restores/builds against Core.slnx - then runs `dotnet test` for every - Features/*.Tests project in turn: - - Features.Users.Tests - - Features.Breweries.Tests - - Features.Emails.Tests - - Uses: Moq to mock each handler's - repository/service dependencies - No real database needed - end note - } - -} - -package "Production Environment\n(docker-compose.prod.yaml)" #FCE4EC { - - node "SQL Server\n(mcr.microsoft.com/mssql/server:2022-latest)" as ProdDB { - database "Biergarten\nDatabase" as ProdDBInner - note right - Environment: - - ACCEPT_EULA=Y - - SA_PASSWORD=*** - - MSSQL_PID=Express - - Volumes: - - sqlserverdata-prod - No host port published. - end note - } - - node "Migrations\n(run-once)" as ProdMig { - component "Database.Migrations" as ProdMig1 - note bottom - Depends on: sql-server (healthy) - No seed step in production. - end note - } - - node "API Container\n(API.Core)" as ProdAPI { - component "ASP.NET Core 10" as ProdAPI1 - portin "8080:8080 (HTTP)" as ProdPort1 - portin "8081:8081 (HTTPS)" as ProdPort2 - note right - Environment: - - ASPNETCORE_ENVIRONMENT=Production - - DB_SERVER/NAME/USER/PASSWORD - - ACCESS/REFRESH/CONFIRMATION_TOKEN_SECRET - - WEBSITE_BASE_URL - - Depends on: sql-server (healthy) - - does not wait on migrations/seed. - end note - } - - ProdMig --> ProdDB : 1. Run migrations - ProdAPI --> ProdDB : 2. Connect & serve - ProdMig .up.> ProdDB : depends_on - ProdAPI .up.> ProdDB : depends_on - - note bottom of ProdDB - Prod Network (bridge: prodnet) - Isolated from dev/test networks. - Volumes: sqlserverdata-prod, - nuget-cache-prod - end note -} - -package "Database-only Environment\n(docker-compose.db.yaml)" #EDE7F6 { - - node "SQL Server" as DbOnlyDB { - database "Biergarten\nDatabase" as DbOnlyDBInner { - portin "1434" - } - } - - node "Migrations\n(run-once)" as DbOnlyMig { - component "Database.Migrations" - } - - node "Seed\n(run-once)" as DbOnlySeed { - component "Database.Seed" - } - - note bottom of DbOnlyDB - Same devnet volumes/images as - docker-compose.dev.yaml, minus - api.core and mailpit - for working - against the database without - running the API. - end note - - DbOnlyMig --> DbOnlyDB : 1. Run migrations - DbOnlySeed --> DbOnlyDB : 2. Seed data - DbOnlyMig .up.> DbOnlyDB : depends_on - DbOnlySeed .up.> DbOnlyMig : depends_on -} - -package "Minimal Environment\n(docker-compose.min.yaml)" #FFF9C4 { - - node "SQL Server" as MinDB { - database "Biergarten\nDatabase" as MinDBInner { - portin "1433" - } - } - - node "Migrations\n(run-once)" as MinMig { - component "Database.Migrations" - } - - node "Mailpit" as MinMailpit { - portin "8025 / 1025" - } - - node "Seed\n(run-once)" as MinSeed { - component "Database.Seed" - } - - note bottom of MinDB - Database + mailpit only, no API - container - for frontend development - against an API.Core process run - outside Docker. - end note - - MinMig --> MinDB : 1. Run migrations - MinSeed --> MinDB : 2. Seed data - MinMig .up.> MinDB : depends_on - MinSeed .up.> MinMig : depends_on -} - -folder "test-results/\n(mounted volume)" as Results { - file "api-specs/\n results.trx" as Result1 - file "Features.Users.Tests.trx" as Result2 - file "Features.Breweries.Tests.trx" as Result3 - file "Features.Emails.Tests.trx" as Result5 - - note bottom - TRX format - Readable by: - - Visual Studio - - Azure DevOps - - GitHub Actions - end note -} - -' External access -Developer --> Host : docker compose up -Host --> DevAPI : http://localhost:8080 - -' Development dependencies -DevMig --> DevDB : 1. Run migrations -DevSeed --> DevDB : 2. Seed data -DevAPI --> DevDB : 3. Connect & serve -DevAPI ..> DevMailpit : 4. SMTP_HOST/PORT -DevMig .up.> DevDB : depends_on -DevSeed .up.> DevMig : depends_on -DevAPI .up.> DevSeed : depends_on - -' Test dependencies -TestMig --> TestDB : 1. Migrate -TestSeed --> TestDB : 2. Seed -Specs --> TestDB : 3. Integration test -UnitTests ..> TestDB : depends_on for startup\nordering only (Moq mocks,\nno real connection) - -TestMig .up.> TestDB : depends_on -TestSeed .up.> TestMig : depends_on -Specs .up.> TestSeed : depends_on -UnitTests .up.> TestSeed : depends_on - -' Test results export -Specs --> Results : Export TRX -UnitTests --> Results : Export TRX (one .trx per slice) - -' Network notes -note bottom of DevDB - Dev Network (bridge: biergarten-dev) - Internal DNS: - - sql-server (resolves to SQL container) - - api (resolves to API container) -end note - -note bottom of TestDB - Test Network (bridge: biergarten-test) - All test components isolated -end note - -' Startup sequence notes -note top of DevMig - Startup Order: - 1. SQL Server (health check) - 2. Migrations (run-once) - 3. Seed (run-once) - 4. API (long-running) -end note - -note top of Specs - Test Execution: - All tests run in parallel - Results aggregated -end note - -@enduml diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 00000000..1c3205c3 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,67 @@ +site_name: The Biergarten App +site_url: https://github.io +repo_url: https://github.com +repo_name: the-biergarten-app + +theme: + name: material +nav: + - Home: index.md + - Web: + - Architecture: web/architecture.md + - Getting Started: web/getting-started.md + - Database: web/database.md + - Docker: web/docker.md + - Environment Variables: web/environment-variables.md + - Testing: web/testing.md + - Token Validation: web/token-validation.md + - Diagrams: + - Vertical Slice: web/diagrams/vertical-slice.md + - Database Schema: web/diagrams/database-schema.md + - Deployment: web/diagrams/deployment.md + - Auth Flows: + - Registration: web/diagrams/auth/registration.md + - Login: web/diagrams/auth/login.md + - Refresh Token: web/diagrams/auth/refresh-token.md + - Confirm User: web/diagrams/auth/confirm-user.md + - Resend Confirmation: web/diagrams/auth/resend-confirmation.md + - Update Username: web/diagrams/auth/update-username.md + - Update Email: web/diagrams/auth/update-email.md + - Update Password: web/diagrams/auth/update-password.md + - Update Profile: web/diagrams/auth/update-profile.md + - Delete Account: web/diagrams/auth/delete-account.md + - Error Handling: web/diagrams/auth/error-handling.md + - Pipeline: + - Overview: pipeline/README.md + - Roadmap: pipeline/ROADMAP.md + - Ethics & Known Issues: pipeline/ETHICS-AND-KNOWN-ISSUES.md + - Diagrams: + - Current: + - Activity: pipeline/diagrams/current/activity.md + - Class: pipeline/diagrams/current/class.md + - Planned: + - Activity: pipeline/diagrams/planned/activity.md + - Class: pipeline/diagrams/planned/class.md + +plugins: + - search + - glightbox + - panzoom: + mermaid: true + - minify: + minify_html: true + minify_js: true + minify_css: true + htmlmin_opts: + remove_comments: true + cache_safe: true + +markdown_extensions: + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format + - pymdownx.snippets: + base_path: [docs] + check_paths: true