mirror of
https://github.com/aaronpo97/the-biergarten-app.git
synced 2026-10-15 10:04:44 +00:00
Compare commits
82 Commits
feat/auth-
...
81a76c9baf
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
81a76c9baf | ||
|
|
693f74bb5b | ||
|
|
6a53167cb8 | ||
|
|
998e5c478b | ||
|
|
cac6d7f193 | ||
|
|
e8499181cb | ||
|
|
a4da01586d | ||
|
|
3087d599ae | ||
|
|
48ee6b0fef | ||
|
|
7a0620337d | ||
|
|
64302286fb | ||
|
|
88abb8e995 | ||
|
|
2ed655e03d | ||
|
|
be0fe9d405 | ||
|
|
492d5d280e | ||
|
|
f3f704da8d | ||
|
|
1d1edd37d0 | ||
|
|
417fa35ced | ||
|
|
66d84dee9e | ||
|
|
fcca12b49d | ||
|
|
84e9e79d81 | ||
|
|
5b2eceb6b8 | ||
|
|
3154c2feed | ||
|
|
3df657dc54 | ||
|
|
a238db1d37 | ||
|
|
5d48d6757a | ||
|
|
34a036d7b3 | ||
|
|
59053fcddd | ||
|
|
d4c7260127 | ||
|
|
be90522eae | ||
|
|
d055a634c2 | ||
|
|
933b728135 | ||
|
|
ac98a2a22e | ||
|
|
d723f53cba | ||
|
|
1885d9859f | ||
|
|
03467cbb65 | ||
|
|
1f25d25dce | ||
|
|
67bc5bfc2a | ||
|
|
eb32a55831 | ||
|
|
fad1611685 | ||
|
|
74f848d015 | ||
|
|
86db1c2827 | ||
|
|
9bcf4c6879 | ||
|
|
d9b10df436 | ||
|
|
4f0f56d7d7 | ||
|
|
6c07edf845 | ||
|
|
969125e3a5 | ||
|
|
ffabd85b6d | ||
|
|
4fae5ead08 | ||
|
|
efd0d4f2aa | ||
|
|
522381357a | ||
|
|
5f3706bb93 | ||
|
|
a52b14c49a | ||
|
|
4ebd28c06b | ||
|
|
9593a07c13 | ||
|
|
f020671703 | ||
|
|
fbcf438381 | ||
| 2b8a900d12 | |||
|
|
880f73e004 | ||
|
|
3711591db1 | ||
|
|
194509f9c9 | ||
|
|
67d6350425 | ||
|
|
cee2917556 | ||
|
|
6f8e5dc722 | ||
|
|
db3ed3581d | ||
|
|
b54f70ccbe | ||
|
|
d4734ae9e7 | ||
|
|
27f28d5207 | ||
|
|
bf0d29fa54 | ||
|
|
4772f59a68 | ||
|
|
2e5d4923c0 | ||
|
|
62766b7638 | ||
|
|
744d4c7f8c | ||
|
|
2f28ac9350 | ||
|
|
c8bddae817 | ||
|
|
b298bd02d3 | ||
|
|
e2e5eb89e3 | ||
|
|
3dd44fca5a | ||
|
|
0c55d6c83f | ||
|
|
58cbc5b5ba | ||
|
|
5bef24696b | ||
|
|
79ae18b3e2 |
@@ -1,52 +0,0 @@
|
|||||||
---
|
|
||||||
name: doc-drift-pipeline
|
|
||||||
description: Checks whether recent changes under tooling/pipeline/ have made docs/pipeline/ stale. Use when the user asks to check for doc drift on the pipeline, before committing changes to tooling/pipeline, or when asked "does this need a doc update" for the C++ seed-data pipeline.
|
|
||||||
---
|
|
||||||
|
|
||||||
# Doc Drift Check — Pipeline
|
|
||||||
|
|
||||||
Given a diff scoped to `tooling/pipeline/` (uncommitted changes, a range of
|
|
||||||
commits, or a specific PR), determine whether it invalidates any factual
|
|
||||||
claim in `docs/pipeline/`.
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
1. Run `git diff -- tooling/pipeline/` (or
|
|
||||||
`git diff <base>...<head> -- tooling/pipeline/` if a range is given) to
|
|
||||||
see what changed. If the diff also touches `web/**`, note that it's out of
|
|
||||||
scope here — use the `doc-drift-web` skill for that half.
|
|
||||||
2. Based on which files changed, identify the relevant doc(s):
|
|
||||||
- `tooling/pipeline/src/**`, `includes/**` (implemented behavior) →
|
|
||||||
`docs/pipeline/README.md`, `docs/pipeline/diagrams/current/*.mmd`
|
|
||||||
(activity, class)
|
|
||||||
- Work that's scaffolded/flagged but not wired into the default run path
|
|
||||||
(roadmap-only) → `docs/pipeline/ROADMAP.md`,
|
|
||||||
`docs/pipeline/diagrams/planned/*.mmd` — do not touch `current/` for
|
|
||||||
planned work that hasn't landed
|
|
||||||
- Changes to pipeline bias/hallucination handling, model prompts, sampling
|
|
||||||
logic, or known-limitation workarounds →
|
|
||||||
`docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md`
|
|
||||||
- `tooling/pipeline/runpod/Dockerfile`, build/run scripts →
|
|
||||||
`docs/pipeline/README.md` (setup/usage sections)
|
|
||||||
- `docs/pipeline/french-cities.example` — check it still matches the input
|
|
||||||
schema the pipeline actually reads
|
|
||||||
- A new file added under `docs/pipeline/**` → also check `docs/index.md`
|
|
||||||
and `mkdocs.yml` nav are updated to reference it
|
|
||||||
3. Read the relevant doc section(s) and check each specific factual claim
|
|
||||||
(tables, "currently X", "not yet implemented", roadmap checkboxes) against
|
|
||||||
the diff. For diagrams, this means reading the `.mmd` source (not just the
|
|
||||||
wrapping `.md`) and checking nodes/edges/labels against the diff — a
|
|
||||||
pipeline stage or control-flow change can invalidate a diagram without
|
|
||||||
touching any prose.
|
|
||||||
4. When a diff adds real behavior that was previously only in
|
|
||||||
`diagrams/planned/`, check whether that planned diagram/roadmap entry
|
|
||||||
should now move to `current/`/README rather than just adding a duplicate
|
|
||||||
claim in both places.
|
|
||||||
5. Report only what's actually stale — do not rewrite unrelated prose, do not
|
|
||||||
invent claims not grounded in the diff.
|
|
||||||
6. If nothing is affected, say so plainly and make no edits.
|
|
||||||
7. For confirmed drift, propose the exact edit (old text → new text, or the
|
|
||||||
changed `.mmd` node/edge) rather than committing directly, unless
|
|
||||||
explicitly told to apply it. Flag `.mmd` changes separately since they may
|
|
||||||
need a re-render (`docs/rendered-diagrams/pipeline/**`) via
|
|
||||||
`docs/makefile`.
|
|
||||||
@@ -1,55 +0,0 @@
|
|||||||
---
|
|
||||||
name: doc-drift-web
|
|
||||||
description: Checks whether recent changes under web/ have made docs/web/ stale. Use when the user asks to check for doc drift on the web app, before committing changes to web/backend or web/frontend, or when asked "does this need a doc update" for web/API/frontend work.
|
|
||||||
---
|
|
||||||
|
|
||||||
# Doc Drift Check — Web
|
|
||||||
|
|
||||||
Given a diff scoped to `web/` (uncommitted changes, a range of commits, or a
|
|
||||||
specific PR), determine whether it invalidates any factual claim in
|
|
||||||
`docs/web/`.
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
1. Run `git diff -- web/` (or `git diff <base>...<head> -- web/` if a range is
|
|
||||||
given) to see what changed. If the diff also touches `tooling/pipeline/**`,
|
|
||||||
note that it's out of scope here — use the `doc-drift-pipeline` skill for
|
|
||||||
that half.
|
|
||||||
2. Based on which files changed, identify the relevant doc(s):
|
|
||||||
- `web/backend/API/**`, `Features.*/**` (general) → `docs/web/architecture.md`
|
|
||||||
- `web/backend/Features/Features.Users/**/Authentication/**`,
|
|
||||||
identity/token code (`ITokenService`, `RefreshToken*`, `ConfirmEmail*`,
|
|
||||||
login/registration handlers) → `docs/web/token-validation.md`,
|
|
||||||
`docs/web/diagrams/auth/*.mmd` (pick the matching flow: login,
|
|
||||||
registration, refresh-token, confirm-user, delete-account,
|
|
||||||
update-email/password/profile/username, resend-confirmation,
|
|
||||||
error-handling)
|
|
||||||
- `web/backend/Database/**` (entities, `Database.Migrations/scripts/**`) →
|
|
||||||
`docs/web/database.md`, `docs/web/diagrams/database-schema/*.mmd` (pick
|
|
||||||
the matching schema: beer, brewery, identity, location, media)
|
|
||||||
- Host wiring / DI registration (`API/**Program.cs`,
|
|
||||||
`Database.Seed/**Program.cs`) → `docs/web/diagrams/vertical-slice/host-wiring.mmd`
|
|
||||||
- New project references between `Features.*`/`Shared.*`/`Domain.*`
|
|
||||||
projects (new `.csproj` `<ProjectReference>`) →
|
|
||||||
`docs/web/diagrams/vertical-slice/project-references.mmd`
|
|
||||||
- `web/backend/Dockerfile*`, `web/frontend/Dockerfile*`, compose/deploy
|
|
||||||
config → `docs/web/docker.md`, `docs/web/diagrams/deployment/*.mmd`
|
|
||||||
(dev, test, minimal, db-only, prod), `docs/web/diagrams/deployment.md`
|
|
||||||
- `web/.env.example`, `web/.env.test`, appsettings → `docs/web/environment-variables.md`
|
|
||||||
- `web/backend/API/API.Specs/**`, test infra/fixtures → `docs/web/testing.md`
|
|
||||||
- New backend project setup, dev tooling → `docs/web/getting-started.md`
|
|
||||||
- `web/frontend/**` → `docs/web/architecture.md` (Frontend architecture section)
|
|
||||||
- A new file added under `docs/web/**` → also check `docs/index.md` and
|
|
||||||
`mkdocs.yml` nav are updated to reference it
|
|
||||||
3. Read the relevant doc section(s) and check each specific factual claim
|
|
||||||
(tables, "currently X", "not yet implemented", roadmap checkboxes) against
|
|
||||||
the diff. For diagrams, this means reading the `.mmd` source (not just the
|
|
||||||
wrapping `.md`) and checking nodes/edges/labels against the diff — a
|
|
||||||
schema or flow change can invalidate a diagram without touching any prose.
|
|
||||||
4. Report only what's actually stale — do not rewrite unrelated prose, do not
|
|
||||||
invent claims not grounded in the diff.
|
|
||||||
5. If nothing is affected, say so plainly and make no edits.
|
|
||||||
6. For confirmed drift, propose the exact edit (old text → new text, or the
|
|
||||||
changed `.mmd` node/edge) rather than committing directly, unless
|
|
||||||
explicitly told to apply it. Flag `.mmd` changes separately since they may
|
|
||||||
need a re-render (`docs/rendered-diagrams/web/**`) via `docs/makefile`.
|
|
||||||
101
.github/pull_request_template.md
vendored
101
.github/pull_request_template.md
vendored
@@ -1,101 +0,0 @@
|
|||||||
## Summary
|
|
||||||
|
|
||||||
<!--
|
|
||||||
State WHAT is changing and WHY, in two to five sentences.
|
|
||||||
A reader should understand the change without opening the diff.
|
|
||||||
Describe the problem first, then the outcome.
|
|
||||||
-->
|
|
||||||
|
|
||||||
## Type of change
|
|
||||||
|
|
||||||
<!-- Tick the one option that best describes this PR. -->
|
|
||||||
|
|
||||||
- [ ] Bug fix
|
|
||||||
- [ ] New feature
|
|
||||||
- [ ] Refactor (no change in behaviour)
|
|
||||||
- [ ] Performance improvement
|
|
||||||
- [ ] Documentation
|
|
||||||
- [ ] Tests only
|
|
||||||
- [ ] Build, CI or dependency update
|
|
||||||
- [ ] Other (please specify):
|
|
||||||
|
|
||||||
## Related issues
|
|
||||||
|
|
||||||
<!--
|
|
||||||
Link the issue(s) this PR relates to. Use one keyword per issue so that each is closed on merge:
|
|
||||||
|
|
||||||
Fixes #123
|
|
||||||
Fixes #124, fixes #125
|
|
||||||
Fixes owner/other-repo#456
|
|
||||||
|
|
||||||
Closing keywords only take effect when the PR targets the repository's default branch.
|
|
||||||
Use "Related to #123" if the PR should not close the issue.
|
|
||||||
If there is no issue, write "N/A" and explain the motivation in the Summary.
|
|
||||||
-->
|
|
||||||
|
|
||||||
## Approach and context
|
|
||||||
|
|
||||||
<!--
|
|
||||||
Explain how the change works and why this approach was chosen.
|
|
||||||
Include, where relevant:
|
|
||||||
- The decisions you made that are not obvious from the code.
|
|
||||||
- Alternatives you considered and why you rejected them.
|
|
||||||
- Known limitations or shortcomings, and any planned follow-up work.
|
|
||||||
- Background such as benchmark results or design documents.
|
|
||||||
|
|
||||||
Summarise the key points of any external link. Links may be inaccessible to future readers.
|
|
||||||
-->
|
|
||||||
|
|
||||||
## How this was tested
|
|
||||||
|
|
||||||
<!--
|
|
||||||
Describe what you ran and what you observed. Be specific, and state only what you actually did.
|
|
||||||
For example: the automated tests added or changed, manual steps and their outcome,
|
|
||||||
environments or platforms covered, and anything you were unable to test.
|
|
||||||
For a bug fix, note the test that fails without the fix.
|
|
||||||
-->
|
|
||||||
|
|
||||||
## Compatibility and risk
|
|
||||||
|
|
||||||
- [ ] This change is backwards compatible.
|
|
||||||
- [ ] This change is **breaking** (complete the details below).
|
|
||||||
- [ ] This change deprecates existing behaviour (complete the details below).
|
|
||||||
|
|
||||||
<!--
|
|
||||||
If breaking or deprecating, describe: what changes, who is affected,
|
|
||||||
and the migration path or recommended alternative.
|
|
||||||
Also note any deployment, configuration, data migration or rollback considerations.
|
|
||||||
-->
|
|
||||||
|
|
||||||
## User-facing change
|
|
||||||
|
|
||||||
<!--
|
|
||||||
Does this PR change behaviour that users, operators or API consumers will notice?
|
|
||||||
If not, write "None".
|
|
||||||
If so, write one or two sentences suitable for release notes, in plain language and from the
|
|
||||||
user's point of view. If users must take action when upgrading, state clearly that action is required.
|
|
||||||
-->
|
|
||||||
|
|
||||||
```release-note
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes for reviewers
|
|
||||||
|
|
||||||
<!--
|
|
||||||
Help reviewers focus. For example:
|
|
||||||
- Where to start, or a suggested order in which to read the files.
|
|
||||||
- Areas where you would particularly value scrutiny, or where you are uncertain.
|
|
||||||
- How any mechanical, generated or bulk changes were produced (for example a search-and-replace
|
|
||||||
or a code generator), so that reviewers can reproduce the steps.
|
|
||||||
- Screenshots or recordings for visual changes (include before and after where useful).
|
|
||||||
-->
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
- [ ] The PR has a single, focused purpose and contains no unrelated changes.
|
|
||||||
- [ ] The title is a clear, imperative summary of the change (for example, "Add retry to payment client").
|
|
||||||
- [ ] I have added or updated tests that cover this change.
|
|
||||||
- [ ] The build succeeds and the tests pass locally.
|
|
||||||
- [ ] The code follows the existing patterns and style of this repository.
|
|
||||||
- [ ] I have updated documentation, changelogs or API references where required.
|
|
||||||
- [ ] I have reviewed my own diff, and this description reflects the final state of the change.
|
|
||||||
51
.github/workflows/deploy-docs.yml
vendored
51
.github/workflows/deploy-docs.yml
vendored
@@ -1,51 +0,0 @@
|
|||||||
name: Deploy Docs to GitHub Pages
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches:
|
|
||||||
- main
|
|
||||||
- docs
|
|
||||||
paths:
|
|
||||||
- "docs/**"
|
|
||||||
- "mkdocs.yml"
|
|
||||||
- ".github/workflows/deploy-docs.yml"
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
pages: write
|
|
||||||
id-token: write
|
|
||||||
|
|
||||||
concurrency:
|
|
||||||
group: pages
|
|
||||||
cancel-in-progress: false
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
deploy:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
environment:
|
|
||||||
name: github-pages
|
|
||||||
url: ${{ steps.deployment.outputs.page_url }}
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: "3.11"
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: pip install mkdocs mkdocs-material mkdocs-glightbox mkdocs-panzoom-plugin mkdocs-minify-plugin
|
|
||||||
|
|
||||||
- name: Build site
|
|
||||||
run: mkdocs build
|
|
||||||
|
|
||||||
- name: Upload artifact
|
|
||||||
uses: actions/upload-pages-artifact@v3
|
|
||||||
with:
|
|
||||||
path: "site"
|
|
||||||
|
|
||||||
- name: Deploy to GitHub Pages
|
|
||||||
id: deployment
|
|
||||||
uses: actions/deploy-pages@v4
|
|
||||||
61
.github/workflows/tests.yml
vendored
61
.github/workflows/tests.yml
vendored
@@ -1,61 +0,0 @@
|
|||||||
name: Tests
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
pull_request:
|
|
||||||
branches: [main]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
containerized-tests:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
defaults:
|
|
||||||
run:
|
|
||||||
working-directory: web
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout
|
|
||||||
uses: actions/checkout@v5
|
|
||||||
|
|
||||||
- name: Generate test environment file
|
|
||||||
run: ./generate-env.sh .env.test
|
|
||||||
|
|
||||||
- name: Build test images
|
|
||||||
run:
|
|
||||||
docker compose --env-file .env.test -f docker-compose.test.yaml build
|
|
||||||
|
|
||||||
- name: Run tests
|
|
||||||
run: |
|
|
||||||
docker compose --env-file .env.test -f docker-compose.test.yaml up -d
|
|
||||||
docker compose --env-file .env.test -f docker-compose.test.yaml wait api.specs unit.tests frontend.tests || true
|
|
||||||
|
|
||||||
- name: Check test container exit codes
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
status=0
|
|
||||||
for name in test-env-database-migrations test-env-database-seed test-env-api-specs test-env-unit-tests test-env-frontend-tests; do
|
|
||||||
code="$(docker inspect -f '{{.State.ExitCode}}' "$name" 2>/dev/null || echo 1)"
|
|
||||||
echo "$name exited with $code"
|
|
||||||
if [ "$code" != "0" ]; then
|
|
||||||
status=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
exit $status
|
|
||||||
|
|
||||||
- name: Show service logs on failure
|
|
||||||
if: failure()
|
|
||||||
run:
|
|
||||||
docker compose --env-file .env.test -f docker-compose.test.yaml logs
|
|
||||||
|
|
||||||
- name: Upload test results
|
|
||||||
if: always()
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: test-results
|
|
||||||
path: web/test-results
|
|
||||||
|
|
||||||
- name: Tear down
|
|
||||||
if: always()
|
|
||||||
run:
|
|
||||||
docker compose --env-file .env.test -f docker-compose.test.yaml down
|
|
||||||
-v
|
|
||||||
18
.gitignore
vendored
18
.gitignore
vendored
@@ -1,5 +1,5 @@
|
|||||||
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
|
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
|
||||||
scratchpad.txt
|
|
||||||
# dependencies
|
# dependencies
|
||||||
/node_modules
|
/node_modules
|
||||||
/.pnp
|
/.pnp
|
||||||
@@ -16,13 +16,12 @@ scratchpad.txt
|
|||||||
/build
|
/build
|
||||||
|
|
||||||
# project-specific build artifacts
|
# project-specific build artifacts
|
||||||
/web/frontend/build/
|
/src/Website/build/
|
||||||
/web/frontend/storybook-static/
|
/src/Website/storybook-static/
|
||||||
/web/frontend/.react-router/
|
/src/Website/.react-router/
|
||||||
/web/frontend/playwright-report/
|
/src/Website/playwright-report/
|
||||||
/web/frontend/test-results/
|
/src/Website/test-results/
|
||||||
/test-results/
|
/test-results/
|
||||||
/docs/rendered-diagrams/
|
|
||||||
|
|
||||||
# misc
|
# misc
|
||||||
.DS_Store
|
.DS_Store
|
||||||
@@ -502,8 +501,3 @@ FodyWeavers.xsd
|
|||||||
|
|
||||||
*storybook.log
|
*storybook.log
|
||||||
storybook-static
|
storybook-static
|
||||||
web/test-results/frontend/vitest-storybook.xml
|
|
||||||
web/test-results/frontend/playwright-storybook.xml
|
|
||||||
|
|
||||||
# mkdocs build output
|
|
||||||
/site
|
|
||||||
|
|||||||
67
README.md
67
README.md
@@ -22,28 +22,34 @@ pipeline notes) lives under `docs/`.
|
|||||||
|
|
||||||
Website + backend (active stack):
|
Website + backend (active stack):
|
||||||
|
|
||||||
- [Getting Started](docs/web/getting-started.md)
|
- [Getting Started](docs/website/getting-started.md)
|
||||||
- [Architecture](docs/web/architecture.md)
|
- [Architecture](docs/architecture.md)
|
||||||
- [Database](docs/web/database.md)
|
- [Database](docs/website/database.md)
|
||||||
- [Docker Guide](docs/web/docker.md)
|
- [Docker Guide](docs/website/docker.md)
|
||||||
- [Testing](docs/web/testing.md)
|
- [Testing](docs/website/testing.md)
|
||||||
- [Environment Variables](docs/web/environment-variables.md)
|
- [Environment Variables](docs/website/environment-variables.md)
|
||||||
- [Token Validation](docs/web/token-validation.md)
|
- [Token Validation](docs/website/token-validation.md)
|
||||||
|
|
||||||
Data generation pipeline (C++):
|
Data generation pipeline (C++):
|
||||||
|
|
||||||
- [Pipeline README](docs/pipeline/README.md)
|
- [Pipeline README](docs/pipeline/README.md)
|
||||||
- [Ethics & Known Issues](docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md)
|
- [Ethics & Known Issues](docs/pipeline/ETHICS-AND-KNOWN-ISSUES.md)
|
||||||
|
|
||||||
|
## Diagrams
|
||||||
|
|
||||||
|
- [Architecture](docs/website/diagrams-out/architecture.svg)
|
||||||
|
- [Deployment](docs/website/diagrams-out/deployment.svg)
|
||||||
|
- [Authentication Flow](docs/website/diagrams-out/authentication-flow.svg)
|
||||||
|
- [Database Schema](docs/website/diagrams-out/database-schema.svg)
|
||||||
|
|
||||||
## Current status
|
## Current status
|
||||||
|
|
||||||
Active areas in the repository:
|
Active areas in the repository:
|
||||||
|
|
||||||
- .NET 10 backend (vertical-slice architecture with MediatR) + SQL Server
|
- .NET 10 backend (vertical-slice architecture with MediatR) + SQL Server
|
||||||
- React 19 website (React Router 8 + Vite)
|
- React 19 website (React Router 7 + Vite)
|
||||||
- Shared Biergarten theme system + Storybook coverage
|
- Shared Biergarten theme system + Storybook coverage
|
||||||
- Auth flows and account/email integration (local Mailpit in dev compose)
|
- Auth flows and account/email integration (local Mailpit in dev compose)
|
||||||
- Photo upload to S3-compatible object storage (local SeaweedFS in dev compose)
|
|
||||||
- Data generation pipeline with C++ and Llama.cpp
|
- Data generation pipeline with C++ and Llama.cpp
|
||||||
|
|
||||||
Archived/reference areas:
|
Archived/reference areas:
|
||||||
@@ -54,8 +60,7 @@ Archived/reference areas:
|
|||||||
## Tech stack
|
## Tech stack
|
||||||
|
|
||||||
- **Backend**: .NET 10, ASP.NET Core, SQL Server 2022, DbUp
|
- **Backend**: .NET 10, ASP.NET Core, SQL Server 2022, DbUp
|
||||||
- **Frontend**: React 19, React Router 8, Vite 7, Tailwind CSS 4, DaisyUI 5,
|
- **Frontend**: React 19, React Router 7, Vite 7, Tailwind CSS 4, DaisyUI 5
|
||||||
Leaflet/React Leaflet (brewery maps)
|
|
||||||
- **UI Documentation**: Storybook 10, Vitest browser mode, Playwright
|
- **UI Documentation**: Storybook 10, Vitest browser mode, Playwright
|
||||||
- **Testing**: xUnit, Reqnroll (BDD), FluentAssertions, Moq
|
- **Testing**: xUnit, Reqnroll (BDD), FluentAssertions, Moq
|
||||||
- **Infrastructure**: Docker, Docker Compose
|
- **Infrastructure**: Docker, Docker Compose
|
||||||
@@ -66,41 +71,34 @@ Archived/reference areas:
|
|||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
- .NET SDK 10 or later
|
- .NET SDK 10 or later
|
||||||
- Node.js 22 or later
|
- Node.js 18 or later
|
||||||
- Docker Desktop, or an equivalent Docker Engine setup
|
- Docker Desktop, or an equivalent Docker Engine setup
|
||||||
|
|
||||||
See [Getting Started](docs/web/getting-started.md) for the full
|
See [Getting Started](docs/website/getting-started.md) for the full
|
||||||
prerequisite list, including the pipeline's C++ toolchain.
|
prerequisite list, including the pipeline's C++ toolchain.
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
For full setup details, use [Getting Started](docs/web/getting-started.md).
|
For full setup details, use [Getting Started](docs/website/getting-started.md).
|
||||||
This section is the shortest path to a working dev environment.
|
This section is the shortest path to a working dev environment.
|
||||||
|
|
||||||
### Full stack (Docker)
|
### Backend (Docker)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/aaronpo97/the-biergarten-app
|
git clone https://github.com/aaronpo97/the-biergarten-app
|
||||||
cd the-biergarten-app
|
cd the-biergarten-app
|
||||||
|
|
||||||
cd web && ./generate-env.sh .env.dev && cd ..
|
cp web/.env.example web/.env.dev
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up --build -d
|
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
`generate-env.sh` copies `.env.example` and replaces the secret-bearing values
|
Backend access:
|
||||||
with freshly randomized ones; a plain `cp` leaves placeholder secrets in
|
|
||||||
place. See [Environment Variables](docs/web/environment-variables.md#generating-an-env-file)
|
|
||||||
for details.
|
|
||||||
|
|
||||||
Access:
|
|
||||||
|
|
||||||
- Website: http://localhost:3000
|
|
||||||
- API Swagger: http://localhost:8080/swagger
|
- API Swagger: http://localhost:8080/swagger
|
||||||
- Health Check: http://localhost:8080/health
|
- Health Check: http://localhost:8080/health
|
||||||
- Mailpit UI (dev SMTP): http://localhost:8025
|
- Mailpit UI (dev SMTP): http://localhost:8025
|
||||||
- SeaweedFS S3 API (dev object storage): http://localhost:8333
|
|
||||||
|
|
||||||
### Frontend (Node, outside Docker)
|
### Frontend (Node)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd web/frontend
|
cd web/frontend
|
||||||
@@ -129,8 +127,9 @@ tooling/
|
|||||||
pipeline/ C++20 seed-data generation CLI (CMake)
|
pipeline/ C++20 seed-data generation CLI (CMake)
|
||||||
|
|
||||||
docs/
|
docs/
|
||||||
web/ Architecture, setup, docker, testing, diagrams
|
architecture.md High-level architecture overview
|
||||||
pipeline/ Pipeline docs, ethics notes, diagrams
|
website/ Backend/frontend setup, docker, testing, diagrams
|
||||||
|
pipeline/ Pipeline docs, ethics notes, PlantUML diagrams
|
||||||
|
|
||||||
archive/
|
archive/
|
||||||
next-js-web-app/ Older Next.js frontend (reference only)
|
next-js-web-app/ Older Next.js frontend (reference only)
|
||||||
@@ -138,18 +137,14 @@ archive/
|
|||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
Create a test environment file, then run the full test stack (backend and
|
Create a test environment file, then run the backend test stack with Docker:
|
||||||
frontend) with Docker:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd web && ./generate-env.sh .env.test && cd ..
|
cp web/.env.example web/.env.test
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
|
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up --abort-on-container-exit
|
||||||
```
|
```
|
||||||
|
|
||||||
The same command runs in CI via
|
See [Testing](docs/website/testing.md) for the full command list.
|
||||||
[`.github/workflows/tests.yml`](.github/workflows/tests.yml) on every push
|
|
||||||
and pull request to `main`. See [Testing](docs/web/testing.md) for the
|
|
||||||
full command list.
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -160,14 +155,14 @@ Common active variables:
|
|||||||
`WEBSITE_BASE_URL`
|
`WEBSITE_BASE_URL`
|
||||||
- Frontend runtime: `API_BASE_URL`, `SESSION_SECRET`, `NODE_ENV`
|
- Frontend runtime: `API_BASE_URL`, `SESSION_SECRET`, `NODE_ENV`
|
||||||
|
|
||||||
See [Environment Variables](docs/web/environment-variables.md) for details.
|
See [Environment Variables](docs/website/environment-variables.md) for details.
|
||||||
|
|
||||||
## Contributing
|
## Contributing
|
||||||
|
|
||||||
1. Fork the repository
|
1. Fork the repository
|
||||||
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
||||||
3. Commit your changes (`git commit -m 'Add amazing feature'`)
|
3. Commit your changes (`git commit -m 'Add amazing feature'`)
|
||||||
4. Run the checks in [Testing](docs/web/testing.md) (backend tests,
|
4. Run the checks in [Testing](docs/website/testing.md) (backend tests,
|
||||||
`npm run lint`, `npm run typecheck`)
|
`npm run lint`, `npm run typecheck`)
|
||||||
5. Push to the branch (`git push origin feature/amazing-feature`)
|
5. Push to the branch (`git push origin feature/amazing-feature`)
|
||||||
6. Open a Pull Request
|
6. Open a Pull Request
|
||||||
|
|||||||
@@ -1,15 +0,0 @@
|
|||||||
CREATE OR ALTER FUNCTION dbo.UDF_GetCountryIdByCode(
|
|
||||||
@CountryCode NVARCHAR(2)
|
|
||||||
)
|
|
||||||
RETURNS UNIQUEIDENTIFIER
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
DECLARE
|
|
||||||
@CountryId UNIQUEIDENTIFIER;
|
|
||||||
|
|
||||||
SELECT @CountryId = CountryID
|
|
||||||
FROM dbo.Country
|
|
||||||
WHERE ISO3166_1 = @CountryCode;
|
|
||||||
|
|
||||||
RETURN @CountryId;
|
|
||||||
END;
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
CREATE OR ALTER FUNCTION dbo.UDF_GetStateProvinceIdByCode(
|
|
||||||
@StateProvinceCode NVARCHAR(6)
|
|
||||||
)
|
|
||||||
RETURNS UNIQUEIDENTIFIER
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
DECLARE
|
|
||||||
@StateProvinceId UNIQUEIDENTIFIER;
|
|
||||||
SELECT @StateProvinceId = StateProvinceID
|
|
||||||
FROM dbo.StateProvince
|
|
||||||
WHERE ISO3166_2 = @StateProvinceCode;
|
|
||||||
RETURN @StateProvinceId;
|
|
||||||
END;
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_CreateUserAccount(
|
|
||||||
@UserAccountId UNIQUEIDENTIFIER OUTPUT,
|
|
||||||
@Username VARCHAR(64),
|
|
||||||
@FirstName NVARCHAR(128),
|
|
||||||
@LastName NVARCHAR(128),
|
|
||||||
@DateOfBirth DATETIME,
|
|
||||||
@Email VARCHAR(128)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
DECLARE
|
|
||||||
@Inserted TABLE
|
|
||||||
(
|
|
||||||
UserAccountID UNIQUEIDENTIFIER
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO UserAccount
|
|
||||||
(Username,
|
|
||||||
FirstName,
|
|
||||||
LastName,
|
|
||||||
DateOfBirth,
|
|
||||||
Email)
|
|
||||||
OUTPUT INSERTED.UserAccountID INTO @Inserted
|
|
||||||
VALUES (@Username, @FirstName, @LastName, @DateOfBirth, @Email);
|
|
||||||
|
|
||||||
SELECT @UserAccountId = UserAccountID
|
|
||||||
FROM @Inserted;
|
|
||||||
END;
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_DeleteUserAccount(
|
|
||||||
@UserAccountId UNIQUEIDENTIFIER
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET NOCOUNT ON
|
|
||||||
|
|
||||||
IF NOT EXISTS (SELECT 1 FROM UserAccount WHERE UserAccountId = @UserAccountId)
|
|
||||||
BEGIN
|
|
||||||
RAISERROR
|
|
||||||
('UserAccount with the specified ID does not exist.', 16,
|
|
||||||
1);
|
|
||||||
ROLLBACK TRANSACTION
|
|
||||||
RETURN
|
|
||||||
END
|
|
||||||
|
|
||||||
DELETE
|
|
||||||
FROM UserAccount
|
|
||||||
WHERE UserAccountId = @UserAccountId;
|
|
||||||
END;
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_GetAllUserAccounts
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET NOCOUNT ON;
|
|
||||||
SELECT UserAccountID,
|
|
||||||
Username,
|
|
||||||
FirstName,
|
|
||||||
LastName,
|
|
||||||
Email,
|
|
||||||
CreatedAt,
|
|
||||||
UpdatedAt,
|
|
||||||
DateOfBirth,
|
|
||||||
Timer
|
|
||||||
FROM dbo.UserAccount;
|
|
||||||
END;
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_GetUserAccountByEmail(
|
|
||||||
@Email VARCHAR(128)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
SELECT UserAccountID,
|
|
||||||
Username,
|
|
||||||
FirstName,
|
|
||||||
LastName,
|
|
||||||
Email,
|
|
||||||
CreatedAt,
|
|
||||||
UpdatedAt,
|
|
||||||
DateOfBirth,
|
|
||||||
Timer
|
|
||||||
FROM dbo.UserAccount
|
|
||||||
WHERE Email = @Email;
|
|
||||||
END;
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE USP_GetUserAccountById(
|
|
||||||
@UserAccountId UNIQUEIDENTIFIER
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
SELECT UserAccountID,
|
|
||||||
Username,
|
|
||||||
FirstName,
|
|
||||||
LastName,
|
|
||||||
Email,
|
|
||||||
CreatedAt,
|
|
||||||
UpdatedAt,
|
|
||||||
DateOfBirth,
|
|
||||||
Timer
|
|
||||||
FROM dbo.UserAccount
|
|
||||||
WHERE UserAccountID = @UserAccountId;
|
|
||||||
END
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_GetUserAccountByUsername(
|
|
||||||
@Username VARCHAR(64)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
SELECT UserAccountID,
|
|
||||||
Username,
|
|
||||||
FirstName,
|
|
||||||
LastName,
|
|
||||||
Email,
|
|
||||||
CreatedAt,
|
|
||||||
UpdatedAt,
|
|
||||||
DateOfBirth,
|
|
||||||
Timer
|
|
||||||
FROM dbo.UserAccount
|
|
||||||
WHERE Username = @Username;
|
|
||||||
END;
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE usp_UpdateUserAccount(
|
|
||||||
@Username VARCHAR(64),
|
|
||||||
@FirstName NVARCHAR(128),
|
|
||||||
@LastName NVARCHAR(128),
|
|
||||||
@DateOfBirth DATETIME,
|
|
||||||
@Email VARCHAR(128),
|
|
||||||
@UserAccountId UNIQUEIDENTIFIER
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
UPDATE UserAccount
|
|
||||||
SET Username = @Username,
|
|
||||||
FirstName = @FirstName,
|
|
||||||
LastName = @LastName,
|
|
||||||
DateOfBirth = @DateOfBirth,
|
|
||||||
Email = @Email
|
|
||||||
WHERE UserAccountId = @UserAccountId;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@@ROWCOUNT = 0
|
|
||||||
BEGIN
|
|
||||||
THROW
|
|
||||||
50001, 'UserAccount with the specified ID does not exist.', 1;
|
|
||||||
END
|
|
||||||
END;
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_GetActiveUserCredentialByUserAccountId(
|
|
||||||
@UserAccountId UNIQUEIDENTIFIER
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
SELECT UserCredentialId,
|
|
||||||
UserAccountId,
|
|
||||||
Hash,
|
|
||||||
IsRevoked,
|
|
||||||
CreatedAt,
|
|
||||||
RevokedAt
|
|
||||||
FROM dbo.UserCredential
|
|
||||||
WHERE UserAccountId = @UserAccountId
|
|
||||||
AND IsRevoked = 0;
|
|
||||||
END;
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_InvalidateUserCredential(
|
|
||||||
@UserAccountId_ UNIQUEIDENTIFIER
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
EXEC dbo.USP_GetUserAccountByID @UserAccountId = @UserAccountId_;
|
|
||||||
IF
|
|
||||||
@@ROWCOUNT = 0
|
|
||||||
THROW 50001, 'User account not found', 1;
|
|
||||||
|
|
||||||
-- invalidate all other credentials by setting them to revoked
|
|
||||||
UPDATE dbo.UserCredential
|
|
||||||
SET IsRevoked = 1,
|
|
||||||
RevokedAt = GETDATE()
|
|
||||||
WHERE UserAccountId = @UserAccountId_
|
|
||||||
AND IsRevoked != 1;
|
|
||||||
|
|
||||||
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
END;
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_RegisterUser(
|
|
||||||
@Username VARCHAR(64),
|
|
||||||
@FirstName NVARCHAR(128),
|
|
||||||
@LastName NVARCHAR(128),
|
|
||||||
@DateOfBirth DATETIME,
|
|
||||||
@Email VARCHAR(128),
|
|
||||||
@Hash NVARCHAR(MAX)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
DECLARE
|
|
||||||
@UserAccountId_ UNIQUEIDENTIFIER;
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
EXEC usp_CreateUserAccount
|
|
||||||
@UserAccountId = @UserAccountId_ OUTPUT,
|
|
||||||
@Username = @Username,
|
|
||||||
@FirstName = @FirstName,
|
|
||||||
@LastName = @LastName,
|
|
||||||
@DateOfBirth = @DateOfBirth,
|
|
||||||
@Email = @Email;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@UserAccountId_ IS NULL
|
|
||||||
BEGIN
|
|
||||||
THROW
|
|
||||||
50000, 'Failed to create user account.', 1;
|
|
||||||
END
|
|
||||||
|
|
||||||
INSERT INTO dbo.UserCredential
|
|
||||||
(UserAccountId, Hash)
|
|
||||||
VALUES (@UserAccountId_, @Hash);
|
|
||||||
|
|
||||||
IF
|
|
||||||
@@ROWCOUNT = 0
|
|
||||||
BEGIN
|
|
||||||
THROW
|
|
||||||
50002, 'Failed to create user credential.', 1;
|
|
||||||
END
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
|
|
||||||
SELECT @UserAccountId_ AS UserAccountId;
|
|
||||||
END
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_RotateUserCredential(
|
|
||||||
@UserAccountId_ UNIQUEIDENTIFIER,
|
|
||||||
@Hash NVARCHAR(MAX)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
EXEC USP_GetUserAccountByID @UserAccountId = @UserAccountId_
|
|
||||||
|
|
||||||
IF @@ROWCOUNT = 0
|
|
||||||
THROW 50001, 'User account not found', 1;
|
|
||||||
|
|
||||||
|
|
||||||
-- invalidate all other credentials -- set them to revoked
|
|
||||||
UPDATE dbo.UserCredential
|
|
||||||
SET IsRevoked = 1,
|
|
||||||
RevokedAt = GETDATE()
|
|
||||||
WHERE UserAccountId = @UserAccountId_;
|
|
||||||
|
|
||||||
INSERT INTO dbo.UserCredential
|
|
||||||
(UserAccountId, Hash)
|
|
||||||
VALUES (@UserAccountId_, @Hash);
|
|
||||||
|
|
||||||
|
|
||||||
END;
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_CreateUserVerification @UserAccountID_ UNIQUEIDENTIFIER,
|
|
||||||
@VerificationDateTime DATETIME = NULL
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@VerificationDateTime IS NULL
|
|
||||||
SET @VerificationDateTime = GETDATE();
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
EXEC USP_GetUserAccountByID @UserAccountId = @UserAccountID_;
|
|
||||||
IF
|
|
||||||
@@ROWCOUNT = 0
|
|
||||||
THROW 50001, 'Could not find a user with that id', 1;
|
|
||||||
|
|
||||||
INSERT INTO dbo.UserVerification
|
|
||||||
(UserAccountId, VerificationDateTime)
|
|
||||||
VALUES (@UserAccountID_, @VerificationDateTime);
|
|
||||||
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
END
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_CreateCity(
|
|
||||||
@CityName NVARCHAR(100),
|
|
||||||
@StateProvinceCode NVARCHAR(6)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION
|
|
||||||
DECLARE
|
|
||||||
@StateProvinceId UNIQUEIDENTIFIER = dbo.UDF_GetStateProvinceIdByCode(@StateProvinceCode);
|
|
||||||
IF
|
|
||||||
@StateProvinceId IS NULL
|
|
||||||
BEGIN
|
|
||||||
THROW
|
|
||||||
50001, 'State/province does not exist', 1;
|
|
||||||
END
|
|
||||||
|
|
||||||
IF
|
|
||||||
EXISTS (SELECT 1
|
|
||||||
FROM dbo.City
|
|
||||||
WHERE CityName = @CityName
|
|
||||||
AND StateProvinceID = @StateProvinceId)
|
|
||||||
BEGIN
|
|
||||||
|
|
||||||
THROW
|
|
||||||
50002, 'City already exists.', 1;
|
|
||||||
END
|
|
||||||
|
|
||||||
INSERT INTO dbo.City
|
|
||||||
(StateProvinceID, CityName)
|
|
||||||
VALUES (@StateProvinceId, @CityName);
|
|
||||||
COMMIT TRANSACTION
|
|
||||||
END;
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_CreateCountry(
|
|
||||||
@CountryName NVARCHAR(100),
|
|
||||||
@ISO3166_1 NVARCHAR(2)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
IF
|
|
||||||
EXISTS (SELECT 1
|
|
||||||
FROM dbo.Country
|
|
||||||
WHERE ISO3166_1 = @ISO3166_1)
|
|
||||||
THROW 50001, 'Country already exists', 1;
|
|
||||||
|
|
||||||
INSERT INTO dbo.Country
|
|
||||||
(CountryName, ISO3166_1)
|
|
||||||
VALUES (@CountryName, @ISO3166_1);
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
END;
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_CreateStateProvince(
|
|
||||||
@StateProvinceName NVARCHAR(100),
|
|
||||||
@ISO3166_2 NVARCHAR(6),
|
|
||||||
@CountryCode NVARCHAR(2)
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
IF
|
|
||||||
EXISTS (SELECT 1
|
|
||||||
FROM dbo.StateProvince
|
|
||||||
WHERE ISO3166_2 = @ISO3166_2)
|
|
||||||
RETURN;
|
|
||||||
|
|
||||||
DECLARE
|
|
||||||
@CountryId UNIQUEIDENTIFIER = dbo.UDF_GetCountryIdByCode(@CountryCode);
|
|
||||||
IF
|
|
||||||
@CountryId IS NULL
|
|
||||||
BEGIN
|
|
||||||
THROW
|
|
||||||
50001, 'Country does not exist', 1;
|
|
||||||
|
|
||||||
END
|
|
||||||
|
|
||||||
INSERT INTO dbo.StateProvince
|
|
||||||
(StateProvinceName, ISO3166_2, CountryID)
|
|
||||||
VALUES (@StateProvinceName, @ISO3166_2, @CountryId);
|
|
||||||
END;
|
|
||||||
@@ -1,59 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_CreateBrewery(
|
|
||||||
@BreweryName NVARCHAR(256),
|
|
||||||
@Description NVARCHAR(512),
|
|
||||||
@PostedByID UNIQUEIDENTIFIER,
|
|
||||||
@CityID UNIQUEIDENTIFIER,
|
|
||||||
@AddressLine1 NVARCHAR(256),
|
|
||||||
@AddressLine2 NVARCHAR(256) = NULL,
|
|
||||||
@PostalCode NVARCHAR(20),
|
|
||||||
@Coordinates GEOGRAPHY = NULL
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@BreweryName IS NULL
|
|
||||||
THROW 50001, 'Brewery name cannot be null.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@Description IS NULL
|
|
||||||
THROW 50002, 'Brewery description cannot be null.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
NOT EXISTS (SELECT 1
|
|
||||||
FROM dbo.UserAccount
|
|
||||||
WHERE UserAccountID = @PostedByID)
|
|
||||||
THROW 50404, 'User not found.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
NOT EXISTS (SELECT 1
|
|
||||||
FROM dbo.City
|
|
||||||
WHERE CityID = @CityID)
|
|
||||||
THROW 50404, 'City not found.', 1;
|
|
||||||
|
|
||||||
DECLARE
|
|
||||||
@NewBreweryID UNIQUEIDENTIFIER = NEWID();
|
|
||||||
DECLARE
|
|
||||||
@NewBrewerLocationID UNIQUEIDENTIFIER = NEWID();
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
INSERT INTO dbo.BreweryPost
|
|
||||||
(BreweryPostID, BreweryName, Description, PostedByID)
|
|
||||||
VALUES (@NewBreweryID, @BreweryName, @Description, @PostedByID);
|
|
||||||
|
|
||||||
INSERT INTO dbo.BreweryPostLocation
|
|
||||||
(BreweryPostLocationID, BreweryPostID, CityID, AddressLine1, AddressLine2, PostalCode, Coordinates)
|
|
||||||
VALUES (@NewBrewerLocationID, @NewBreweryID, @CityID, @AddressLine1, @AddressLine2, @PostalCode, @Coordinates);
|
|
||||||
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
|
|
||||||
SELECT @NewBreweryID AS BreweryPostID,
|
|
||||||
@NewBrewerLocationID AS BreweryPostLocationID;
|
|
||||||
|
|
||||||
END
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_DeleteBrewery @BreweryPostID UNIQUEIDENTIFIER
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
IF
|
|
||||||
NOT EXISTS (SELECT 1
|
|
||||||
FROM dbo.BreweryPost
|
|
||||||
WHERE BreweryPostID = @BreweryPostID)
|
|
||||||
THROW 50404, 'Brewery not found.', 1;
|
|
||||||
|
|
||||||
-- BreweryPostLocation and BreweryPostPhoto cascade-delete with their parent BreweryPost.
|
|
||||||
DELETE
|
|
||||||
FROM dbo.BreweryPost
|
|
||||||
WHERE BreweryPostID = @BreweryPostID;
|
|
||||||
END
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_GetAllBreweries(
|
|
||||||
@Limit INT = NULL,
|
|
||||||
@Offset INT = NULL
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
|
|
||||||
SELECT bp.BreweryPostID,
|
|
||||||
bp.PostedByID,
|
|
||||||
bp.BreweryName,
|
|
||||||
bp.Description,
|
|
||||||
bp.CreatedAt,
|
|
||||||
bp.UpdatedAt,
|
|
||||||
bp.Timer,
|
|
||||||
bpl.BreweryPostLocationID,
|
|
||||||
bpl.CityID,
|
|
||||||
bpl.AddressLine1,
|
|
||||||
bpl.AddressLine2,
|
|
||||||
bpl.PostalCode,
|
|
||||||
bpl.Coordinates
|
|
||||||
FROM dbo.BreweryPost bp
|
|
||||||
LEFT JOIN dbo.BreweryPostLocation bpl
|
|
||||||
ON bp.BreweryPostID = bpl.BreweryPostID
|
|
||||||
ORDER BY bp.CreatedAt DESC
|
|
||||||
OFFSET ISNULL(@Offset, 0) ROWS FETCH NEXT ISNULL(@Limit, 2147483647) ROWS ONLY;
|
|
||||||
END
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_GetBreweryById @BreweryPostID UNIQUEIDENTIFIER
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SELECT *
|
|
||||||
FROM BreweryPost bp
|
|
||||||
INNER JOIN BreweryPostLocation bpl
|
|
||||||
ON bp.BreweryPostID = bpl.BreweryPostID
|
|
||||||
WHERE bp.BreweryPostID = @BreweryPostID;
|
|
||||||
END
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
CREATE OR ALTER PROCEDURE dbo.USP_UpdateBrewery(
|
|
||||||
@BreweryPostID UNIQUEIDENTIFIER,
|
|
||||||
@BreweryName NVARCHAR(256),
|
|
||||||
@Description NVARCHAR(512),
|
|
||||||
@BreweryPostLocationID UNIQUEIDENTIFIER = NULL,
|
|
||||||
@CityID UNIQUEIDENTIFIER = NULL,
|
|
||||||
@AddressLine1 NVARCHAR(256) = NULL,
|
|
||||||
@AddressLine2 NVARCHAR(256) = NULL,
|
|
||||||
@PostalCode NVARCHAR(20) = NULL,
|
|
||||||
@Coordinates GEOGRAPHY = NULL
|
|
||||||
)
|
|
||||||
AS
|
|
||||||
BEGIN
|
|
||||||
SET
|
|
||||||
NOCOUNT ON;
|
|
||||||
SET
|
|
||||||
XACT_ABORT ON;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@BreweryName IS NULL
|
|
||||||
THROW 50001, 'Brewery name cannot be null.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@Description IS NULL
|
|
||||||
THROW 50002, 'Brewery description cannot be null.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
NOT EXISTS (SELECT 1
|
|
||||||
FROM dbo.BreweryPost
|
|
||||||
WHERE BreweryPostID = @BreweryPostID)
|
|
||||||
THROW 50404, 'Brewery not found.', 1;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@CityID IS NOT NULL AND NOT EXISTS (SELECT 1
|
|
||||||
FROM dbo.City
|
|
||||||
WHERE CityID = @CityID)
|
|
||||||
THROW 50404, 'City not found.', 1;
|
|
||||||
|
|
||||||
BEGIN
|
|
||||||
TRANSACTION;
|
|
||||||
|
|
||||||
UPDATE dbo.BreweryPost
|
|
||||||
SET BreweryName = @BreweryName,
|
|
||||||
Description = @Description,
|
|
||||||
UpdatedAt = GETDATE()
|
|
||||||
WHERE BreweryPostID = @BreweryPostID;
|
|
||||||
|
|
||||||
IF
|
|
||||||
@CityID IS NULL
|
|
||||||
BEGIN
|
|
||||||
-- No location supplied: clear any existing location for this brewery.
|
|
||||||
DELETE
|
|
||||||
FROM dbo.BreweryPostLocation
|
|
||||||
WHERE BreweryPostID = @BreweryPostID;
|
|
||||||
END
|
|
||||||
ELSE
|
|
||||||
IF EXISTS (SELECT 1
|
|
||||||
FROM dbo.BreweryPostLocation
|
|
||||||
WHERE BreweryPostID = @BreweryPostID)
|
|
||||||
BEGIN
|
|
||||||
UPDATE dbo.BreweryPostLocation
|
|
||||||
SET CityID = @CityID,
|
|
||||||
AddressLine1 = @AddressLine1,
|
|
||||||
AddressLine2 = @AddressLine2,
|
|
||||||
PostalCode = @PostalCode,
|
|
||||||
Coordinates = @Coordinates
|
|
||||||
WHERE BreweryPostID = @BreweryPostID;
|
|
||||||
END
|
|
||||||
ELSE
|
|
||||||
BEGIN
|
|
||||||
INSERT INTO dbo.BreweryPostLocation
|
|
||||||
(BreweryPostLocationID, BreweryPostID, CityID, AddressLine1, AddressLine2, PostalCode, Coordinates)
|
|
||||||
VALUES (ISNULL(@BreweryPostLocationID, NEWID()), @BreweryPostID, @CityID, @AddressLine1, @AddressLine2,
|
|
||||||
@PostalCode, @Coordinates);
|
|
||||||
END
|
|
||||||
COMMIT TRANSACTION;
|
|
||||||
END
|
|
||||||
558
docs/architecture.md
Normal file
558
docs/architecture.md
Normal file
@@ -0,0 +1,558 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
This document describes the active architecture of The Biergarten App.
|
||||||
|
|
||||||
|
## High-level overview
|
||||||
|
|
||||||
|
The Biergarten App is a monorepo with a clear split between the backend and the
|
||||||
|
active website:
|
||||||
|
|
||||||
|
- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature
|
||||||
|
slices with MediatR
|
||||||
|
- **Frontend**: React 19 + React Router 7 website in `web/frontend`
|
||||||
|
- **Architecture Style**: Vertical-slice backend plus server-rendered React
|
||||||
|
frontend
|
||||||
|
|
||||||
|
The legacy Next.js frontend has been retained in `archive/next-js-web-app/` for
|
||||||
|
reference only.
|
||||||
|
|
||||||
|
## Diagrams
|
||||||
|
|
||||||
|
For visual representations, see:
|
||||||
|
|
||||||
|
- [architecture.svg](website/diagrams-out/architecture.svg) - Vertical-slice
|
||||||
|
architecture diagram
|
||||||
|
- [deployment.svg](website/diagrams-out/deployment.svg) - Docker deployment
|
||||||
|
diagram
|
||||||
|
- [authentication-flow.svg](website/diagrams-out/authentication-flow.svg) -
|
||||||
|
Authentication workflow
|
||||||
|
- [database-schema.svg](website/diagrams-out/database-schema.svg) - Database
|
||||||
|
relationships
|
||||||
|
|
||||||
|
## Backend architecture
|
||||||
|
|
||||||
|
### Vertical-slice architecture pattern
|
||||||
|
|
||||||
|
The backend organizes business capabilities as feature slices instead of
|
||||||
|
technical layers. Each feature (`Features.Auth`, `Features.Breweries`,
|
||||||
|
`Features.UserManagement`, `Features.Emails`, `Features.Locations`) is a single
|
||||||
|
project that owns its own MediatR commands/queries/handlers, validators, and
|
||||||
|
repository end to end. `Features.Emails` and `Features.Locations` have no
|
||||||
|
`Controllers/` folder — both are invoked internally by other slices via
|
||||||
|
MediatR/DI, never over HTTP:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌───────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ API.Core (thin host) │
|
||||||
|
│ - Program.cs wiring: MediatR + AddApplicationPart per slice │
|
||||||
|
│ - Swagger/OpenAPI, JWT auth middleware, global exception filter │
|
||||||
|
└───────────────────────────────────────────────────────────────────────┘
|
||||||
|
↓ discovers controllers via AddApplicationPart
|
||||||
|
┌───────────────┬───────────────┬───────────────────┬──────────────────┬──────────────────┐
|
||||||
|
│ Features.Auth │Features. │ Features. │ Features.Emails │ Features. │
|
||||||
|
│ │Breweries │ UserManagement │ (no controller, │ Locations │
|
||||||
|
│ Controller │ Controller │ Controller │ internal only) │ (no controller, │
|
||||||
|
│ Commands/ │ Commands/ │ Commands/ │ Commands/ │ internal only) │
|
||||||
|
│ Queries + │ Queries + │ Queries + │ Handlers │ Repository │
|
||||||
|
│ Handlers │ Handlers │ Handlers │ │ │
|
||||||
|
│ Repository │ Repository │ Repository │ EmailDispatcher │ │
|
||||||
|
└───────────────┴───────────────┴───────────────────┴──────────────────┴──────────────────┘
|
||||||
|
↓ each slice depends only on shared/domain/infra, never on another slice
|
||||||
|
┌─────────────────────────┬─────────────────────────┬────────────────────┐
|
||||||
|
│ Shared.Contracts │ Shared.Application │ Domain.Entities / │
|
||||||
|
│ (ResponseBody envelope) │ (ValidationBehavior, │ Domain.Exceptions │
|
||||||
|
│ │ cross-slice email cmds) │ │
|
||||||
|
└─────────────────────────┴─────────────────────────┴────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ Infrastructure.Sql, Infrastructure.Jwt, Infrastructure.PasswordHashing, │
|
||||||
|
│ Infrastructure.Email, Infrastructure.Email.Templates │
|
||||||
|
└─────────────────────────────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────┐
|
||||||
|
│ Database (SQL Server) │
|
||||||
|
│ - Stored procedures │
|
||||||
|
│ - Tables & constraints │
|
||||||
|
└─────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Slices never reference each other's project. The one cross-slice interaction
|
||||||
|
(`Features.Auth` triggering a confirmation email handled by `Features.Emails`)
|
||||||
|
goes through a MediatR command (`SendRegistrationEmailCommand`) whose contract
|
||||||
|
lives in `Shared.Application`, so neither slice takes a project reference on the
|
||||||
|
other.
|
||||||
|
|
||||||
|
### Layer responsibilities
|
||||||
|
|
||||||
|
#### API layer (`API.Core`)
|
||||||
|
|
||||||
|
**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of its
|
||||||
|
own
|
||||||
|
|
||||||
|
**Components**:
|
||||||
|
|
||||||
|
- `Program.cs`: registers MediatR (scanning every `Features.*` assembly),
|
||||||
|
FluentValidation, and uses `AddApplicationPart` so each slice's controllers
|
||||||
|
are discovered by MVC
|
||||||
|
- `GlobalException.cs`: global exception filter (host-level cross-cutting
|
||||||
|
concern)
|
||||||
|
- `Authentication/JwtAuthenticationHandler.cs`: JWT auth scheme middleware
|
||||||
|
- Swagger/OpenAPI documentation, health check endpoints
|
||||||
|
|
||||||
|
**Dependencies**:
|
||||||
|
|
||||||
|
- Every `Features.*` project (for controller/MediatR discovery)
|
||||||
|
- `Shared.Contracts`, `Shared.Application`
|
||||||
|
- `Infrastructure.Jwt` (for the auth middleware)
|
||||||
|
|
||||||
|
**Rules**:
|
||||||
|
|
||||||
|
- No controllers, no business logic, no feature-specific contracts
|
||||||
|
- Exists purely to host and wire up the feature slices
|
||||||
|
|
||||||
|
#### Feature slices (`Features.Auth`, `Features.Breweries`, `Features.UserManagement`, `Features.Emails`, `Features.Locations`)
|
||||||
|
|
||||||
|
**Purpose**: Each slice is the complete vertical for one business capability
|
||||||
|
|
||||||
|
**Components** (per slice):
|
||||||
|
|
||||||
|
- `Controllers/`: HTTP endpoints, binding directly to Command/Query types as the
|
||||||
|
request contract
|
||||||
|
- `Commands/<Operation>/` and `Queries/<Operation>/`: one folder per operation,
|
||||||
|
each containing the Command/Query record, its `IRequestHandler`, and (for
|
||||||
|
commands) a FluentValidation validator
|
||||||
|
- `Repository/`: the slice's own stored-procedure repository implementation
|
||||||
|
- `Dtos/`: response shapes returned by query handlers (never the raw domain
|
||||||
|
entity)
|
||||||
|
- `DependencyInjection/`: an `AddFeaturesX()` extension method registering the
|
||||||
|
slice's repository/services
|
||||||
|
|
||||||
|
`Features.Emails` has no `Controllers/` folder. It's invoked only via MediatR
|
||||||
|
commands sent from other slices, never over HTTP. `Features.Locations` also has
|
||||||
|
no `Controllers/` folder and no MediatR handlers — it exposes
|
||||||
|
`ILocationRepository` directly as a plain service, currently consumed only by
|
||||||
|
`Database.Seed` to look up cities while seeding brewery locations. Other slices
|
||||||
|
(for example, `Features.Breweries`) do not yet depend on it; they run their own
|
||||||
|
`CityID` existence checks inline.
|
||||||
|
|
||||||
|
**Dependencies**:
|
||||||
|
|
||||||
|
- `Domain.Entities`, `Domain.Exceptions`
|
||||||
|
- `Infrastructure.Sql` (generic ADO.NET plumbing) plus whichever infrastructure
|
||||||
|
project the slice needs (`Infrastructure.Jwt`/
|
||||||
|
`Infrastructure.PasswordHashing` for Auth, `Infrastructure.Email`/
|
||||||
|
`Infrastructure.Email.Templates` for Emails)
|
||||||
|
- `Shared.Contracts`, `Shared.Application`
|
||||||
|
- **Never** another `Features.*` project
|
||||||
|
|
||||||
|
**Rules**:
|
||||||
|
|
||||||
|
- All business logic for that feature lives in its command/query handlers
|
||||||
|
- No direct controller-to-repository calls; everything flows through MediatR
|
||||||
|
- Read endpoints return a dedicated `Dto`, never the domain entity directly
|
||||||
|
|
||||||
|
#### Shared projects (`Shared.Contracts`, `Shared.Application`)
|
||||||
|
|
||||||
|
**Purpose**: The minimum cross-slice surface area required because every slice
|
||||||
|
needs it, or because duplicating it four times would be worse than sharing it
|
||||||
|
|
||||||
|
**Components**:
|
||||||
|
|
||||||
|
- `Shared.Contracts`: `ResponseBody<T>`/`ResponseBody`, the API response
|
||||||
|
envelope every controller returns
|
||||||
|
- `Shared.Application`: `ValidationBehavior<TRequest,TResponse>` (the MediatR
|
||||||
|
pipeline behavior that runs FluentValidation before a handler executes) and
|
||||||
|
the cross-slice email commands (`SendRegistrationEmailCommand`,
|
||||||
|
`SendResendConfirmationEmailCommand`)
|
||||||
|
|
||||||
|
**Rules**:
|
||||||
|
|
||||||
|
- Kept deliberately small: this is the exception to "no slice depends on another
|
||||||
|
slice," not a general-purpose dumping ground
|
||||||
|
|
||||||
|
#### Infrastructure layer
|
||||||
|
|
||||||
|
**Purpose**: Technical capabilities and external integrations, shared by
|
||||||
|
whichever slices need them
|
||||||
|
|
||||||
|
**Components**:
|
||||||
|
|
||||||
|
- **Infrastructure.Sql**: generic ADO.NET connection/command plumbing
|
||||||
|
(`ISqlConnectionFactory`, the abstract `Repository<T>` base class), not
|
||||||
|
domain-specific; each slice's own `Repository/` folder builds on this
|
||||||
|
- **Infrastructure.Jwt**: JWT token generation and validation
|
||||||
|
- **Infrastructure.PasswordHashing**: Argon2id password hashing
|
||||||
|
- **Infrastructure.Email**: Email sending capabilities (SMTP/MailKit)
|
||||||
|
- **Infrastructure.Email.Templates**: Email template rendering (Razor
|
||||||
|
components)
|
||||||
|
|
||||||
|
**Dependencies**:
|
||||||
|
|
||||||
|
- Domain entities
|
||||||
|
- External libraries (ADO.NET, JWT, Argon2, MailKit, and so on)
|
||||||
|
|
||||||
|
**Rules**:
|
||||||
|
|
||||||
|
- Implements technical concerns only, no business logic
|
||||||
|
- Reusable across slices
|
||||||
|
|
||||||
|
#### Domain layer (`Domain.Entities`)
|
||||||
|
|
||||||
|
**Purpose**: Core business entities and models
|
||||||
|
|
||||||
|
**Components**:
|
||||||
|
|
||||||
|
- `UserAccount` - User profile data
|
||||||
|
- `UserCredential` - Authentication credentials
|
||||||
|
- `UserVerification` - Account verification state
|
||||||
|
- `BreweryPost` - A user-submitted brewery listing
|
||||||
|
- `BreweryPostLocation` - A brewery's address and coordinates
|
||||||
|
- `City` - A city referenced by a brewery location
|
||||||
|
|
||||||
|
**Dependencies**:
|
||||||
|
|
||||||
|
- None (pure domain)
|
||||||
|
|
||||||
|
**Rules**:
|
||||||
|
|
||||||
|
- Plain Old CLR Objects (POCOs)
|
||||||
|
- No framework dependencies
|
||||||
|
- No infrastructure references
|
||||||
|
- Represents business concepts
|
||||||
|
|
||||||
|
### Design patterns
|
||||||
|
|
||||||
|
#### Vertical slice + MediatR
|
||||||
|
|
||||||
|
**Purpose**: Organize code by feature instead of by technical layer, so
|
||||||
|
everything needed to understand or change one capability lives in one project
|
||||||
|
|
||||||
|
**Implementation**:
|
||||||
|
|
||||||
|
- Each HTTP write operation is a `Command` (for example, `CreateBreweryCommand`
|
||||||
|
in `Features.Breweries/Commands/CreateBrewery/`); each read operation is a
|
||||||
|
`Query` (for example, `GetBreweryByIdQuery`)
|
||||||
|
- Controllers bind directly to the Command/Query as the request body; there is
|
||||||
|
no separate request DTO + mapping step for writes
|
||||||
|
- A single shared `ValidationBehavior<TRequest,TResponse>`
|
||||||
|
(`Shared.Application/Behaviors/`) runs FluentValidation validators in the
|
||||||
|
MediatR pipeline before any handler executes
|
||||||
|
- Query handlers map to a dedicated response `Dto`, so domain entities never
|
||||||
|
leak over the wire
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...)
|
||||||
|
: IRequest<BreweryDto>;
|
||||||
|
|
||||||
|
public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler<CreateBreweryCommand, BreweryDto>
|
||||||
|
{
|
||||||
|
public async Task<BreweryDto> Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Repository pattern
|
||||||
|
|
||||||
|
**Purpose**: Abstract database access behind interfaces
|
||||||
|
|
||||||
|
**Implementation**: each slice owns its own repository, scoped to that feature
|
||||||
|
only:
|
||||||
|
|
||||||
|
- `Features.Auth/Repository/IAuthRepository.cs`
|
||||||
|
- `Features.Breweries/Repository/IBreweryRepository.cs`
|
||||||
|
- `Features.UserManagement/Repository/IUserAccountRepository.cs`
|
||||||
|
- `Infrastructure.Sql/DefaultSqlConnectionFactory.cs`: the generic connection
|
||||||
|
factory every slice's repository builds on
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
|
||||||
|
- Testable (easy to mock)
|
||||||
|
- Each slice's data access logic is self-contained
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
|
||||||
|
```csharp
|
||||||
|
public interface IAuthRepository
|
||||||
|
{
|
||||||
|
Task<UserCredential> GetUserCredentialAsync(string username);
|
||||||
|
Task<int> CreateUserAccountAsync(UserAccount user, UserCredential credential);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Dependency injection
|
||||||
|
|
||||||
|
**Purpose**: Loose coupling and testability
|
||||||
|
|
||||||
|
**Configuration**: `Program.cs` wires up MediatR/FluentValidation across every
|
||||||
|
`Features.*` assembly; each slice exposes its own `AddFeaturesX()` extension
|
||||||
|
method that registers its repository and slice-internal services
|
||||||
|
|
||||||
|
**Lifetimes**:
|
||||||
|
|
||||||
|
- Scoped: Repositories, slice-internal services (per request)
|
||||||
|
- Singleton: `ISqlConnectionFactory`
|
||||||
|
- Transient: Utilities, helpers
|
||||||
|
|
||||||
|
#### Direct SQL via Dapper/ADO.NET
|
||||||
|
|
||||||
|
**Purpose**: Keep data access explicit and colocated with the slice that owns
|
||||||
|
it, without an ORM's change-tracking or mapping magic
|
||||||
|
|
||||||
|
**Strategy**:
|
||||||
|
|
||||||
|
- No ORM (Entity Framework not used); each
|
||||||
|
repository issues inline SQL (a mix of Dapper and raw
|
||||||
|
`DbCommand`/`DbDataReader` for the few reads that need manual column mapping,
|
||||||
|
such as the `GEOGRAPHY` column on `BreweryPostLocation`)
|
||||||
|
- Referential checks that a stored procedure used to perform (e.g. "does this
|
||||||
|
`CityId` exist?") are now explicit `SELECT 1 ...` existence checks in the
|
||||||
|
repository method, run inside the same transaction as the write
|
||||||
|
- Optimistic concurrency uses each table's `Timer` (`ROWVERSION`) column,
|
||||||
|
checked in the `UPDATE ... WHERE ... AND Timer = @Timer` clause
|
||||||
|
- Repositories throw `Domain.Exceptions` types (`NotFoundException`,
|
||||||
|
`ConflictException`, ...) directly when a check fails, rather than relying on
|
||||||
|
a database-side `THROW`; `API.Core`'s `GlobalExceptionFilter` maps those
|
||||||
|
exception types to HTTP status codes
|
||||||
|
- Application focuses on orchestration; the database enforces integrity via
|
||||||
|
keys, `CHECK` constraints, and cascades
|
||||||
|
|
||||||
|
See [Database](website/database.md) for the schema and the app's current SQL
|
||||||
|
error-handling approach in more detail.
|
||||||
|
|
||||||
|
## Frontend architecture
|
||||||
|
|
||||||
|
### Active website (`web/frontend`)
|
||||||
|
|
||||||
|
The current website is a React Router 7 application with server-side rendering
|
||||||
|
enabled.
|
||||||
|
|
||||||
|
```text
|
||||||
|
web/frontend/
|
||||||
|
├── app/
|
||||||
|
│ ├── components/ Shared UI such as Navbar, FormField, SubmitButton, ToastProvider
|
||||||
|
│ ├── lib/ Auth helpers, schemas, and theme metadata
|
||||||
|
│ ├── routes/ Route modules for home, login, register, dashboard, confirm, theme
|
||||||
|
│ ├── root.tsx App shell and global providers
|
||||||
|
│ └── app.css Theme tokens and global styling
|
||||||
|
├── .storybook/ Storybook config and preview setup
|
||||||
|
├── stories/ Storybook stories for shared UI and themes
|
||||||
|
├── tests/playwright/ Storybook Playwright coverage
|
||||||
|
└── package.json Frontend scripts and dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend responsibilities
|
||||||
|
|
||||||
|
- Render the auth demo and theme guide routes
|
||||||
|
- Manage cookie-backed website session state
|
||||||
|
- Call the .NET API for login, registration, token refresh, and confirmation
|
||||||
|
- Provide shared UI building blocks for forms, navigation, themes, and toasts
|
||||||
|
- Supply Storybook documentation and browser-based component verification
|
||||||
|
|
||||||
|
### Theme system
|
||||||
|
|
||||||
|
The active website uses semantic DaisyUI theme tokens backed by four Biergarten
|
||||||
|
themes:
|
||||||
|
|
||||||
|
- Biergarten Lager
|
||||||
|
- Biergarten Stout
|
||||||
|
- Biergarten Cassis
|
||||||
|
- Biergarten Weizen
|
||||||
|
|
||||||
|
All component styling should prefer semantic tokens such as `primary`,
|
||||||
|
`success`, `surface`, and `highlight` instead of hard-coded color values.
|
||||||
|
|
||||||
|
### Legacy frontend
|
||||||
|
|
||||||
|
The previous Next.js frontend has been archived at `archive/next-js-web-app/`
|
||||||
|
for reference only. Active product and engineering documentation should point to
|
||||||
|
`web/frontend`.
|
||||||
|
|
||||||
|
## Security architecture
|
||||||
|
|
||||||
|
### Authentication flow
|
||||||
|
|
||||||
|
1. **Registration**:
|
||||||
|
- User submits credentials
|
||||||
|
- Password hashed with Argon2id
|
||||||
|
- User account created (unverified) and a confirmation email is dispatched
|
||||||
|
via `Features.Emails`
|
||||||
|
|
||||||
|
2. **Email confirmation**:
|
||||||
|
- User follows the confirmation link/token from the email
|
||||||
|
- `Features.Auth`'s `ConfirmUser` command marks the account verified
|
||||||
|
|
||||||
|
3. **Login**:
|
||||||
|
- User submits credentials
|
||||||
|
- Password verified against hash
|
||||||
|
- Access and refresh JWTs are issued; the refresh token is stored server-side
|
||||||
|
against the user's credential row
|
||||||
|
|
||||||
|
4. **Token refresh**:
|
||||||
|
- Client exchanges a valid, unexpired refresh token for a new access/refresh
|
||||||
|
pair via `Features.Auth`'s `RefreshToken` command
|
||||||
|
- Refresh tokens are stateless JWTs, not tracked server-side: the previous
|
||||||
|
refresh token is not invalidated and remains usable until its own
|
||||||
|
expiration
|
||||||
|
|
||||||
|
5. **API Requests**:
|
||||||
|
- Client sends the access JWT in the `Authorization` header
|
||||||
|
- `JwtAuthenticationHandler` validates the token
|
||||||
|
- Request proceeds if valid
|
||||||
|
|
||||||
|
### Password security
|
||||||
|
|
||||||
|
**Algorithm**: Argon2id
|
||||||
|
|
||||||
|
- Memory: 64MB
|
||||||
|
- Iterations: 4
|
||||||
|
- Parallelism: CPU core count
|
||||||
|
- Salt: 128-bit (16 bytes)
|
||||||
|
- Hash: 256-bit (32 bytes)
|
||||||
|
|
||||||
|
### JWT tokens
|
||||||
|
|
||||||
|
**Algorithm**: HS256 (HMAC-SHA256)
|
||||||
|
|
||||||
|
**Claims**:
|
||||||
|
|
||||||
|
- `sub` - User ID
|
||||||
|
- `unique_name` - Username
|
||||||
|
- `jti` - Unique token ID
|
||||||
|
- `iat` - Issued at timestamp
|
||||||
|
- `exp` - Expiration timestamp
|
||||||
|
|
||||||
|
**Configuration** (appsettings.json):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"Jwt": {
|
||||||
|
"ExpirationMinutes": 60,
|
||||||
|
"Issuer": "biergarten-api",
|
||||||
|
"Audience": "biergarten-users"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database architecture
|
||||||
|
|
||||||
|
For the table-by-table schema, the stored procedure reference (parameters,
|
||||||
|
behavior, and exactly what each one throws), and the custom SQL error code
|
||||||
|
scheme, see [Database](website/database.md).
|
||||||
|
|
||||||
|
### SQL-first philosophy
|
||||||
|
|
||||||
|
**Principles**:
|
||||||
|
|
||||||
|
1. Database is source of truth
|
||||||
|
2. Complex queries in stored procedures
|
||||||
|
3. Database handles referential integrity
|
||||||
|
4. Application orchestrates, database executes
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
|
||||||
|
- Performance optimization via execution plans
|
||||||
|
- Centralized query logic
|
||||||
|
- Version-controlled schema (migrations)
|
||||||
|
- Easier query profiling and tuning
|
||||||
|
|
||||||
|
### Migration strategy
|
||||||
|
|
||||||
|
**Tool**: DbUp
|
||||||
|
|
||||||
|
**Process**:
|
||||||
|
|
||||||
|
1. Write a SQL script under `Database.Migrations/scripts/`
|
||||||
|
2. Embed it in the `Database.Migrations` project
|
||||||
|
3. `Database.Migrations` runs each script against the target database on
|
||||||
|
startup, tracked so a script never re-runs once applied
|
||||||
|
|
||||||
|
**Migration Files**:
|
||||||
|
|
||||||
|
```
|
||||||
|
scripts/
|
||||||
|
└── 01-schema/
|
||||||
|
└── schema.sql # full table/index/constraint definitions
|
||||||
|
```
|
||||||
|
|
||||||
|
Currently the whole schema is a single versioned script. As the schema evolves,
|
||||||
|
new numbered scripts get added alongside it rather than editing `schema.sql` in
|
||||||
|
place, so DbUp's already-applied tracking stays valid.
|
||||||
|
|
||||||
|
### Data seeding
|
||||||
|
|
||||||
|
**Purpose**: Populate development/test databases with realistic data
|
||||||
|
|
||||||
|
**Implementation**: `Database.Seed` project, using data produced by the C++
|
||||||
|
pipeline under `tooling/pipeline/` (see [Pipeline README](pipeline/README.md))
|
||||||
|
|
||||||
|
**Seed Data**:
|
||||||
|
|
||||||
|
- Cities (via `Features.Locations`' `ILocationRepository`)
|
||||||
|
- User accounts (via `Features.Auth`'s repository)
|
||||||
|
- Brewery posts with locations (via `Features.Breweries`' repository)
|
||||||
|
|
||||||
|
Tables that exist in the schema but have no corresponding `Domain.Entities` type
|
||||||
|
or feature slice yet (`Photo`, `UserAvatar`, `UserFollow`, `Country`,
|
||||||
|
`StateProvince`, `BeerStyle`, `BeerPost`, `BeerPostPhoto`, `BeerPostComment`,
|
||||||
|
`BreweryPostPhoto`) are not seeded.
|
||||||
|
|
||||||
|
## Deployment architecture
|
||||||
|
|
||||||
|
### Docker containerization
|
||||||
|
|
||||||
|
**Container Structure**:
|
||||||
|
|
||||||
|
- `sqlserver` - SQL Server 2022
|
||||||
|
- `database.migrations` - Schema migration runner
|
||||||
|
- `database.seed` - Data seeder
|
||||||
|
- `api.core` - ASP.NET Core Web API
|
||||||
|
- `mailpit` - Local dev/test SMTP server + web UI (not used in production)
|
||||||
|
|
||||||
|
**Environments**:
|
||||||
|
|
||||||
|
- Development (`docker-compose.dev.yaml`)
|
||||||
|
- Testing (`docker-compose.test.yaml`)
|
||||||
|
- Production (`docker-compose.prod.yaml`)
|
||||||
|
|
||||||
|
For details, see [Docker Guide](website/docker.md).
|
||||||
|
|
||||||
|
### Health checks
|
||||||
|
|
||||||
|
- **SQL Server**: Validates database connectivity
|
||||||
|
- **API**: Checks service health and dependencies
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "sqlcmd health check"]
|
||||||
|
interval: 10s
|
||||||
|
retries: 12
|
||||||
|
start_period: 30s
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing architecture
|
||||||
|
|
||||||
|
### Test pyramid
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────┐
|
||||||
|
│ Integration │ ← API.Specs (Reqnroll)
|
||||||
|
│ Tests │
|
||||||
|
├──────────────┤
|
||||||
|
│ Unit Tests │ ← Features.Auth.Tests, Features.Breweries.Tests,
|
||||||
|
│ (per slice, │ Features.UserManagement.Tests, Features.Emails.Tests
|
||||||
|
│ handlers + │ (commands/queries/handlers + that slice's own
|
||||||
|
│ repository) │ repository, mocked with Moq/DbMocker)
|
||||||
|
└──────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Strategy**:
|
||||||
|
|
||||||
|
- Many unit tests (fast, isolated)
|
||||||
|
- Fewer integration tests (slower, e2e)
|
||||||
|
- Mock external dependencies
|
||||||
|
- Test database for integration tests
|
||||||
|
|
||||||
|
For details, see [Testing Guide](website/testing.md).
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
---
|
|
||||||
title: The Biergarten App
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- overview
|
|
||||||
- index
|
|
||||||
---
|
|
||||||
|
|
||||||
Welcome to the documentation for The Biergarten App.
|
|
||||||
|
|
||||||
## Documentation Sections
|
|
||||||
|
|
||||||
- **[Architecture](web/architecture.md)** — System design and components
|
|
||||||
- **[Website](web/getting-started.md)** — Backend and frontend documentation
|
|
||||||
- **[Pipeline](pipeline/README.md)** — Data generation pipeline
|
|
||||||
|
|
||||||
## AI Disclosure
|
|
||||||
|
|
||||||
Parts of this documentation are drafted and maintained with AI assistance
|
|
||||||
(Claude, Sonnet or Opus, via Claude Code), including prose (architecture
|
|
||||||
notes, guides) and diagrams (the Mermaid sources under `diagrams/`).
|
|
||||||
AI-authored changes are reviewed by a maintainer before merging, same as any
|
|
||||||
other contribution.
|
|
||||||
|
|
||||||
Staleness checks are partly automated too, to guard against doc-rot: the
|
|
||||||
repo's `doc-drift-web` and `doc-drift-pipeline` Claude Code skills
|
|
||||||
(`.claude/skills/`) diff recent code changes against these docs and flag
|
|
||||||
claims that may need updating.
|
|
||||||
|
|
||||||
This is separate from the pipeline's own use of AI to *generate seed data*
|
|
||||||
(brewery names, descriptions, user profiles) — see
|
|
||||||
[ETHICS-AND-KNOWN-ISSUES.md](pipeline/ETHICS-AND-KNOWN-ISSUES.md) for that
|
|
||||||
disclosure.
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
# Makefile for rendering all Mermaid (.mmd) diagrams to SVG,
|
|
||||||
# preserving the source directory structure under OUT_ROOT.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# make # render all diagrams
|
|
||||||
# make clean # remove all rendered output
|
|
||||||
# make FORMAT=png # render as PNG instead of SVG
|
|
||||||
# make FORMAT=pdf # render as PDF instead of SVG
|
|
||||||
#
|
|
||||||
# Requires: @mermaid-js/mermaid-cli (mmdc) on PATH
|
|
||||||
# npm install -g @mermaid-js/mermaid-cli
|
|
||||||
|
|
||||||
SRC_ROOT := .
|
|
||||||
OUT_ROOT := rendered-diagrams
|
|
||||||
FORMAT := svg
|
|
||||||
|
|
||||||
# Optional: point to a shared Mermaid config/theme file if you have one.
|
|
||||||
# CONFIG := ./mermaidconfig.json
|
|
||||||
# CONFIG_FLAG := -c $(CONFIG)
|
|
||||||
CONFIG_FLAG :=
|
|
||||||
|
|
||||||
# Find every .mmd file under SRC_ROOT, excluding the output directory itself.
|
|
||||||
SRC_FILES := $(shell find $(SRC_ROOT) -type f -name "*.mmd" -not -path "./$(OUT_ROOT)/*")
|
|
||||||
|
|
||||||
# Map each source file to its corresponding output file, preserving subdirectories.
|
|
||||||
OUT_FILES := $(patsubst $(SRC_ROOT)/%.mmd,$(OUT_ROOT)/%.$(FORMAT),$(SRC_FILES))
|
|
||||||
|
|
||||||
.PHONY: all clean list
|
|
||||||
|
|
||||||
all: $(OUT_FILES)
|
|
||||||
|
|
||||||
# Pattern rule: build any output file from its matching .mmd source file,
|
|
||||||
# creating the destination subdirectory as needed.
|
|
||||||
$(OUT_ROOT)/%.$(FORMAT): $(SRC_ROOT)/%.mmd
|
|
||||||
@mkdir -p $(dir $@)
|
|
||||||
mmdc $(CONFIG_FLAG) -i $< -o $@
|
|
||||||
|
|
||||||
clean:
|
|
||||||
rm -rf $(OUT_ROOT)
|
|
||||||
|
|
||||||
# Debug helper: print which source files were discovered.
|
|
||||||
list:
|
|
||||||
@echo "$(SRC_FILES)" | tr ' ' '\n'
|
|
||||||
@@ -1,13 +1,4 @@
|
|||||||
---
|
# Ethics, bias, and known issues
|
||||||
title: Pipeline ethics, bias, and known issues
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- pipeline
|
|
||||||
- ethics
|
|
||||||
- bias
|
|
||||||
- known-issues
|
|
||||||
- ai-generated
|
|
||||||
---
|
|
||||||
|
|
||||||
This document covers the ethical context of the Biergarten Pipeline's output,
|
This document covers the ethical context of the Biergarten Pipeline's output,
|
||||||
the model's biases, and known issues including hallucinated brewing science and
|
the model's biases, and known issues including hallucinated brewing science and
|
||||||
@@ -27,7 +18,7 @@ low-resource language failures.
|
|||||||
- [Known issues](#known-issues)
|
- [Known issues](#known-issues)
|
||||||
- [Hallucinated brewing techniques](#hallucinated-brewing-techniques)
|
- [Hallucinated brewing techniques](#hallucinated-brewing-techniques)
|
||||||
- [Low-resource language hallucination](#low-resource-language-hallucination)
|
- [Low-resource language hallucination](#low-resource-language-hallucination)
|
||||||
- [Synthetic coordinates](#synthetic-coordinates)
|
- [Synthetic postal codes](#synthetic-postal-codes)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -56,7 +47,7 @@ as a source of truth for brewing techniques or cultural representation.**
|
|||||||
|
|
||||||
**Natural language processing is a powerful tool for data analysis and
|
**Natural language processing is a powerful tool for data analysis and
|
||||||
generation, but its output should be treated with scrutiny. Human language is
|
generation, but its output should be treated with scrutiny. Human language is
|
||||||
not simply data points to be analyzed: it carries cultural and human meaning
|
not simply data points to be analyzed — it carries cultural and human meaning
|
||||||
that artificial intelligence cannot capture.**
|
that artificial intelligence cannot capture.**
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -111,7 +102,7 @@ vendored verbatim, unmodified, from
|
|||||||
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
|
(the `common-forenames-by-country.json` / `common-surnames-by-country.json`
|
||||||
release assets), released under **CC0** (public domain). That dataset's own
|
release assets), released under **CC0** (public domain). That dataset's own
|
||||||
forename/surname lists are pulled from Wikipedia's "Lists of most common
|
forename/surname lists are pulled from Wikipedia's "Lists of most common
|
||||||
surnames" and "List of most popular given names" as of the week of 2023-07-08;
|
surnames" and "List of most popular given names" as of the week of 2023-07-08 —
|
||||||
see that project's README for full provenance. Names are not LLM-generated; this
|
see that project's README for full provenance. Names are not LLM-generated; this
|
||||||
is curated fixture data per ROADMAP.md §2. Per-forename gender from the source
|
is curated fixture data per ROADMAP.md §2. Per-forename gender from the source
|
||||||
data is preserved through to the sampled `Name` (rather than discarded during
|
data is preserved through to the sampled `Name` (rather than discarded during
|
||||||
@@ -122,7 +113,7 @@ for surnames) rather than trimmed to `locations.json`'s current country list, so
|
|||||||
it doesn't need re-sourcing if more countries are added later. `SampleName()` (a
|
it doesn't need re-sourcing if more countries are added later. `SampleName()` (a
|
||||||
free helper in `generate_users.cc`) returns no result for a country present in
|
free helper in `generate_users.cc`) returns no result for a country present in
|
||||||
neither file; of the countries in `locations.json`, that's currently `KE`, `SE`,
|
neither file; of the countries in `locations.json`, that's currently `KE`, `SE`,
|
||||||
`SG`, `TH`, `VN`, and `ZA`; `GenerateUsers` skips cities in those countries the
|
`SG`, `TH`, `VN`, and `ZA` — `GenerateUsers` skips cities in those countries the
|
||||||
same way brewery generation skips cities whose enrichment lookup fails.
|
same way brewery generation skips cities whose enrichment lookup fails.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -293,27 +284,21 @@ Output sample: [./french-cities.example](french-cities.example)
|
|||||||
SQLite schema so downstream applications can filter or flag potentially
|
SQLite schema so downstream applications can filter or flag potentially
|
||||||
hallucinated text by language tier.
|
hallucinated text by language tier.
|
||||||
|
|
||||||
### Synthetic coordinates
|
### Synthetic postal codes
|
||||||
|
|
||||||
Each brewery and user address carries a longitude/latitude pair, sampled
|
When the pipeline is run without `--mocked`, `XegerPostalCodeService` generates
|
||||||
uniformly at random within a 5 km disc centred on the city's curated
|
a fresh postal code per brewery/user address by reverse-matching one of the
|
||||||
`latitude`/`longitude` from `locations.json`.
|
city's curated postal-code regexes from `locations.json` (falling back to the
|
||||||
|
country format regex), rather than returning one of `MockPostalCodeService`'s
|
||||||
|
curated real-world examples.
|
||||||
|
|
||||||
The coordinate pair itself is **plausible by construction**, always falling
|
The generated code is **format-conformant by construction** — it always
|
||||||
within the city's vicinity, but does not correspond to a real business or
|
satisfies the regex it was generated from — but it is not looked up from any
|
||||||
resident, and should not be treated as accurate geocoding data. This is
|
real postal registry. It does not correspond to a real address, business, or
|
||||||
consistent with the rest of the dataset: fixture data for a proof-of-concept,
|
resident, and should not be treated as accurate geocoding or delivery data. This
|
||||||
|
is consistent with the rest of the dataset: fixture data for a proof-of-concept,
|
||||||
not a source of truth.
|
not a source of truth.
|
||||||
|
|
||||||
For brewery addresses specifically (not user addresses, which carry no street
|
|
||||||
address at all), `address_line1`/`postal_code` _are_ looked up from a real
|
|
||||||
address registry — `NominatimAddressService` reverse-geocodes the synthetic
|
|
||||||
coordinate pair against the live OpenStreetMap/Nominatim database, so the street
|
|
||||||
address is a genuine nearby address, just not the address of an actual brewery.
|
|
||||||
This lookup only happens for non-`--mocked` runs; `--mocked` runs use
|
|
||||||
`MockAddressService`'s fixed placeholder address instead, which is not looked up
|
|
||||||
from any real registry.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Footnotes
|
## Footnotes
|
||||||
|
|||||||
@@ -1,13 +1,4 @@
|
|||||||
---
|
# Biergarten Pipeline
|
||||||
title: Biergarten Pipeline — C++ CLI for generating seed data
|
|
||||||
last-updated: 2026-09-02
|
|
||||||
tags:
|
|
||||||
- pipeline
|
|
||||||
- cpp
|
|
||||||
- llm
|
|
||||||
- seed-data
|
|
||||||
- docker
|
|
||||||
---
|
|
||||||
|
|
||||||
A C++20 command-line pipeline that samples city records from local JSON,
|
A C++20 command-line pipeline that samples city records from local JSON,
|
||||||
enriches each with Wikipedia context, and generates bilingual brewery names and
|
enriches each with Wikipedia context, and generates bilingual brewery names and
|
||||||
@@ -16,12 +7,12 @@ deterministic mock.
|
|||||||
|
|
||||||
> **This pipeline produces AI-generated data.** It is not a source of truth for
|
> **This pipeline produces AI-generated data.** It is not a source of truth for
|
||||||
> brewing techniques, cultural representation, or local-language accuracy. See
|
> brewing techniques, cultural representation, or local-language accuracy. See
|
||||||
> [ETHICS-AND-KNOWN-ISSUES.md](./ETHICS-AND-KNOWN-ISSUES.md) for full
|
> [ETHICS-AND-KNOWN-ISSUES.md](./ETHICS-AND-KNOWN-ISSUES.md) for a full
|
||||||
> documentation of limitations, hallucination patterns, and bias.
|
> documentation of limitations, hallucination patterns, and bias.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Table of contents
|
## Table of Contents
|
||||||
|
|
||||||
- [How it fits the main app](#how-it-fits-the-main-app)
|
- [How it fits the main app](#how-it-fits-the-main-app)
|
||||||
- [Quick start](#quick-start)
|
- [Quick start](#quick-start)
|
||||||
@@ -110,27 +101,25 @@ Each run writes a fresh dated SQLite file such as
|
|||||||
|
|
||||||
#### CLI flags
|
#### CLI flags
|
||||||
|
|
||||||
| Flag | Purpose |
|
| Flag | Purpose |
|
||||||
| ------------------ | ---------------------------------------------------------------------------------------------------------------- |
|
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
|
||||||
| `--mocked` | Deterministic mock generator, no model required. |
|
| `--mocked` | Deterministic mock generator, no model required. |
|
||||||
| `--model, -m` | Path to a GGUF file. Selects the local Llama.cpp generator. |
|
| `--model, -m` | Path to a GGUF file. Required unless `--mocked` is set. |
|
||||||
| `--openai` | Use the OpenAI API (Chat Completions) for brewery/user data. Requires the `OPENAI_API_KEY` environment variable. |
|
| `--prompt-dir` | Directory containing prompt files (for example, `BREWERY_GENERATION.md`). Required unless `--mocked` is set. |
|
||||||
| `--openai-model` | OpenAI model ID used when `--openai` is set. Default: `gpt-4o-mini`. |
|
| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. |
|
||||||
| `--prompt-dir` | Directory containing prompt files (for example, `BREWERY_GENERATION.md`). Required unless `--mocked` is set. |
|
| `--log-path` | Path for application logs. Default: `pipeline.log`. |
|
||||||
| `--output, -o` | Directory for generated SQLite artifacts. Default: `output`. |
|
| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. |
|
||||||
| `--log-path` | Path for application logs. Default: `pipeline.log`. |
|
| `--temperature` | Sampling temperature. Default: `1.0`. |
|
||||||
| `--location-count` | Number of cities to sample from `locations.json` per run. Default: `10`. |
|
| `--top-p` | Nucleus sampling. Default: `0.95`. |
|
||||||
| `--temperature` | Sampling temperature. Default: `1.0`. |
|
| `--top-k` | Top-k sampling. Default: `64`. |
|
||||||
| `--top-p` | Nucleus sampling. Default: `0.95`. |
|
| `--n-ctx` | Context window size. Default: `8192`. |
|
||||||
| `--top-k` | Top-k sampling. Default: `64`. |
|
| `--seed` | Random seed. Default: `-1` (random at runtime). |
|
||||||
| `--n-ctx` | Context window size. Default: `8192`. |
|
| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. |
|
||||||
| `--seed` | Random seed. Default: `-1` (random at runtime). |
|
| `--help, -h` | Print usage and exit. |
|
||||||
| `--n-gpu-layers` | Number of model layers to offload to GPU. Default: `0`. |
|
|
||||||
| `--help, -h` | Print usage and exit. |
|
|
||||||
|
|
||||||
`--mocked`, `--model`, and `--openai` are mutually exclusive. Omitting all three
|
`--mocked` and `--model` are mutually exclusive. Omitting both exits with an
|
||||||
exits with an error before the pipeline starts. Sampling flags are ignored
|
error before the pipeline starts. Sampling flags are ignored when `--mocked` is
|
||||||
unless `--model` is set (Llama).
|
set.
|
||||||
|
|
||||||
The post-build step copies `prompts/` into `build/prompts/`. Rebuild after
|
The post-build step copies `prompts/` into `build/prompts/`. Rebuild after
|
||||||
editing any prompt file.
|
editing any prompt file.
|
||||||
@@ -187,7 +176,7 @@ output to confirm the fast path was taken.
|
|||||||
### Run the container
|
### Run the container
|
||||||
|
|
||||||
The container always runs the model-backed path; there is no `--mocked`
|
The container always runs the model-backed path; there is no `--mocked`
|
||||||
container mode (use a native build for that; see [Quick start](#quick-start)).
|
container mode (use a native build for that — see [Quick start](#quick-start)).
|
||||||
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if it
|
The entrypoint, `runpod/start.sh`, downloads the GGUF model automatically if it
|
||||||
is not already present at the configured path.
|
is not already present at the configured path.
|
||||||
|
|
||||||
@@ -202,7 +191,7 @@ docker run --rm \
|
|||||||
|
|
||||||
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to `./models/` on
|
By default this downloads `google_gemma-4-E4B-it-Q6_K.gguf` to `./models/` on
|
||||||
first run if it isn't already there. To use a pre-downloaded model, place it at
|
first run if it isn't already there. To use a pre-downloaded model, place it at
|
||||||
that path first; see [Model](#model).
|
that path first — see [Model](#model) above.
|
||||||
|
|
||||||
#### Environment variables
|
#### Environment variables
|
||||||
|
|
||||||
@@ -222,9 +211,8 @@ configurable via environment variable.
|
|||||||
|
|
||||||
Use a GPU pod template. Mount persistent storage for `/workspace/models`,
|
Use a GPU pod template. Mount persistent storage for `/workspace/models`,
|
||||||
`/workspace/output`, and `/workspace/logs`. See
|
`/workspace/output`, and `/workspace/logs`. See
|
||||||
`tooling/pipeline/runpod/pod-template.yaml` for a starter template; set the
|
`tooling/pipeline/runpod/pod-template.yaml` for a starter template — set the
|
||||||
environment variables listed in [Environment variables](#environment-variables)
|
environment variables listed above to match your run.
|
||||||
to match your run.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -238,8 +226,8 @@ to match your run.
|
|||||||
| Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). |
|
| Sample | `BiergartenPipelineOrchestrator::QueryCitiesWithCountries()` samples `--location-count` locations per run (default `10`). |
|
||||||
| Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. |
|
| Enrich | `WikipediaEnrichmentService` fetches brewing and beer-related context. Keeps going when a lookup fails. `--mocked` runs use `MockEnrichmentService` instead and skip Wikipedia entirely. |
|
||||||
| Generate Users | `GenerateUsers()` samples a persona and a forename/surname pair per enriched city (skipping countries with no name data), then `MockGenerator` or `LlamaGenerator` produces a username, bio, and activity weight around the sampled name. |
|
| Generate Users | `GenerateUsers()` samples a persona and a forename/surname pair per enriched city (skipping countries with no name data), then `MockGenerator` or `LlamaGenerator` produces a username, bio, and activity weight around the sampled name. |
|
||||||
| Generate Breweries | `MockGenerator`, `LlamaGenerator`, or `OpenAIGenerator` produces brewery names and descriptions in English and the local language, then `IAddressService::ReverseGeocode()` resolves a street address for the brewery's jittered coordinates. |
|
| Generate Breweries | `MockGenerator` or `LlamaGenerator` produces brewery names and descriptions in English and the local language. |
|
||||||
| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `cities`, `breweries`, `brewery_addresses`, `users`, and `user_addresses` tables. |
|
| Store | `SqliteExportService` writes each successful user and brewery into a fresh dated `.sqlite` database with normalized `locations`, `users`, and `breweries` tables. |
|
||||||
| Log | `spdlog` writes results and warnings to the console. |
|
| Log | `spdlog` writes results and warnings to the console. |
|
||||||
|
|
||||||
If name sampling, enrichment, or generation fails for a city, that city is
|
If name sampling, enrichment, or generation fails for a city, that city is
|
||||||
@@ -248,36 +236,67 @@ skipped and the pipeline continues. `GenerateUsers()` runs before
|
|||||||
|
|
||||||
### Key components
|
### Key components
|
||||||
|
|
||||||
| Component | Description |
|
- `src/main.cc` — argument parsing and Boost.DI composition root.
|
||||||
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
- `CuratedJsonDataService` — implements `ICuratedDataService`; takes a
|
||||||
| `src/main.cc` | Argument parsing and the Boost.DI composition root. |
|
`CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in
|
||||||
| `CuratedJsonDataService` | Implements `ICuratedDataService`; takes a `CuratedDataFilePaths` DTO (locations/personas/forenames/surnames paths) in its constructor, parses and validates curated location, persona, and forename/surname JSON, and memoizes each result after its first load on a given instance. `MockCuratedDataService` is the in-memory substitute (4 fixed locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in `--mocked` runs. |
|
its constructor, then parses and validates curated location, persona, and
|
||||||
| `WikipediaEnrichmentService` | Queries Wikipedia extracts, caches results, and returns empty context on failure. `MockEnrichmentService` is the no-op substitute used in `--mocked` runs. |
|
forename/surname JSON, memoizing each result after its first load on a given
|
||||||
| `LlamaGenerator` | Formats prompts for Gemma 4, validates JSON output for both `GenerateBrewery` and `GenerateUser`, and retries malformed responses up to three times with corrective feedback in the retry prompt. The token budget is fixed across attempts; it is not raised automatically on truncation. |
|
instance. `MockCuratedDataService` is the in-memory substitute (4 fixed
|
||||||
| `MockGenerator` | Uses stable hash-based output so the same city/persona/name input always produces the same brewery or user. |
|
locations, 3 personas, and name data for `US`/`DE`/`FR`/`BE`) used in
|
||||||
| `NominatimAddressService` | Reverse-geocodes each brewery's jittered coordinates into a street address (`address_line1`, `postal_code`) via the public [Nominatim API](https://nominatim.openstreetmap.org), respecting its one-request-per-second usage policy. Falls back to a placeholder `address_line1` ("Address unavailable") and empty `postal_code` when the lookup fails or returns no usable street address. `MockAddressService` is the no-network substitute (`"123 Mock Street"`, `"00000"`) used in `--mocked` runs. Only breweries get a resolved street address; generated users carry a coordinate pair with no address fields. |
|
`--mocked` runs.
|
||||||
| `SqliteExportService` | Creates a dated SQLite file per run and persists each successful user and brewery into normalized tables. |
|
- `WikipediaEnrichmentService` — queries Wikipedia extracts, caches results,
|
||||||
| Brewery/user payloads | Brewery payloads include English and local-language name and description fields. User payloads carry a sampled first/last name and gender, an LLM-generated username/bio/activity weight, and a programmatically generated (not LLM-authored) unique email and date of birth. |
|
returns empty context on failure. `MockEnrichmentService` is the no-op
|
||||||
|
substitute used in `--mocked` runs.
|
||||||
|
- `LlamaGenerator` — formats prompts for Gemma 4, validates JSON output for both
|
||||||
|
`GenerateBrewery` and `GenerateUser`, retries malformed responses up to three
|
||||||
|
times with corrective feedback in the retry prompt. The token budget is fixed
|
||||||
|
across attempts; it is not raised automatically on truncation.
|
||||||
|
- `MockGenerator` — stable hash-based output so the same city/persona/name input
|
||||||
|
always produces the same brewery or user.
|
||||||
|
- `SqliteExportService` — creates a dated SQLite file per run and persists each
|
||||||
|
successful user and brewery into normalized tables.
|
||||||
|
- Brewery payloads include English and local-language name and description
|
||||||
|
fields. User payloads carry a sampled first/last name and gender, an
|
||||||
|
LLM-generated username/bio/activity weight, and a programmatically generated
|
||||||
|
(not LLM-authored) unique email and date of birth.
|
||||||
|
|
||||||
### Runtime behaviour
|
### Runtime behaviour
|
||||||
|
|
||||||
| Component / behavior | Details |
|
`WikipediaEnrichmentService` fetches two Wikipedia extracts per city: a generic
|
||||||
| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
"brewing" extract and a "beer in `{country}`" extract. It does not currently
|
||||||
| `WikipediaEnrichmentService` | Fetches two Wikipedia extracts per city: a generic "brewing" extract and a "beer in `{country}`" extract. It does not currently query a city- or region-specific page. Each query string is cached after its first successful (or empty) lookup. |
|
query a city- or region-specific page. Each query string is cached after its
|
||||||
| `GetLocationContext()` | Returns an empty string when the web client is unavailable or when lookup/parsing fails. |
|
first successful (or empty) lookup.
|
||||||
| `LlamaGenerator` | Validates model output as structured JSON. On validation failure it retries up to three times, replaying the previous error message in the next prompt so the model can self-correct. All runs to date have produced valid output on the first pass; the retry path is kept for resilience. |
|
|
||||||
| `MockGenerator` | Uses stable hashes for repeatable output in demos and Storybook runs. |
|
`GetLocationContext()` returns an empty string when the web client is
|
||||||
| `NominatimAddressService.ReverseGeocode()` | Sleeps one second after every request to respect Nominatim's usage policy, and returns `std::nullopt` (logged as a warning, not a skipped city) if the request fails, the response has no `address` object, or neither `house_number` nor `road` is present — the brewery is still generated and exported, just with a placeholder address. |
|
unavailable or when lookup/parsing fails.
|
||||||
| `CuratedJsonDataService` | Memoizes each of `LoadLocations()`, `LoadPersonas()`, `LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the first time each is called, since `BiergartenPipelineOrchestrator` owns a single `ICuratedDataService` instance for the whole run; later calls return the cached result instead of re-parsing. |
|
|
||||||
| `GenerateUsers()` | Samples a forename/surname pair per city via `SampleName()`, keyed by the city's ISO 3166-1 code. Countries present in `locations.json` but absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`, `ZA`) are skipped, the same way a failed enrichment or generation call skips a city; see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section. |
|
`LlamaGenerator` validates model output as structured JSON. On validation
|
||||||
|
failure it retries up to three times, replaying the previous error message in
|
||||||
|
the next prompt so the model can self-correct. All runs to date have produced
|
||||||
|
valid output on the first pass; the retry path is kept for resilience.
|
||||||
|
|
||||||
|
`MockGenerator` uses stable hashes for repeatable output in demos and Storybook
|
||||||
|
runs.
|
||||||
|
|
||||||
|
`CuratedJsonDataService` memoizes each of `LoadLocations()`, `LoadPersonas()`,
|
||||||
|
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` independently the
|
||||||
|
first time each is called, since `BiergartenPipelineOrchestrator` owns a single
|
||||||
|
`ICuratedDataService` instance for the whole run — later calls return the cached
|
||||||
|
result instead of re-parsing.
|
||||||
|
|
||||||
|
`GenerateUsers()` samples a forename/surname pair per city via `SampleName()`,
|
||||||
|
keyed by the city's ISO 3166-1 code. Countries present in `locations.json` but
|
||||||
|
absent from either name fixture (currently `KE`, `SE`, `SG`, `TH`, `VN`, `ZA`)
|
||||||
|
are skipped, the same way a failed enrichment or generation call skips a city —
|
||||||
|
see ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section.
|
||||||
|
|
||||||
### Process flow - activity diagram
|
### Process flow - activity diagram
|
||||||
|
|
||||||
See [Activity diagram](./diagrams/current/activity.md).
|

|
||||||
|
|
||||||
### Architectural overview - class diagram
|
### Architectural overview - class diagram
|
||||||
|
|
||||||
See [Class diagram](./diagrams/current/class.md).
|

|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -288,14 +307,12 @@ Each successful run stores a `BreweryRecord` pair with the source location and a
|
|||||||
`UserResult` payload. The same generated records are also written to a fresh
|
`UserResult` payload. The same generated records are also written to a fresh
|
||||||
SQLite export file named with the current UTC timestamp.
|
SQLite export file named with the current UTC timestamp.
|
||||||
|
|
||||||
| Field | Meaning |
|
| Field | Meaning |
|
||||||
| ------------------- | ------------------------------------------------------------------------------------- |
|
| ------------------- | ------------------------------------------ |
|
||||||
| `name_en` | Brewery name in English. |
|
| `name_en` | Brewery name in English. |
|
||||||
| `description_en` | Brewery description in English. |
|
| `description_en` | Brewery description in English. |
|
||||||
| `name_local` | Brewery name in the local language. |
|
| `name_local` | Brewery name in the local language. |
|
||||||
| `description_local` | Brewery description in the local language. |
|
| `description_local` | Brewery description in the local language. |
|
||||||
| `address_line1` | Street address resolved via reverse geocoding, or a placeholder if resolution failed. |
|
|
||||||
| `postal_code` | Postal code resolved via reverse geocoding, or empty if resolution failed. |
|
|
||||||
|
|
||||||
| Field | Meaning |
|
| Field | Meaning |
|
||||||
| ----------------- | -------------------------------------------------------------------------------------- |
|
| ----------------- | -------------------------------------------------------------------------------------- |
|
||||||
@@ -340,14 +357,15 @@ The build fetches Boost.DI, spdlog, and SQLite via CMake. llama.cpp is fetched
|
|||||||
only when a system installation is not detected. Metal is enabled on Apple
|
only when a system installation is not detected. Metal is enabled on Apple
|
||||||
Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
||||||
|
|
||||||
> **Code style:** The code follows the Google C++ Style Guide, except that
|
> **Code Style:** Code style generally follows the Google Style Guide and
|
||||||
> indentation is 3 spaces instead of Google's 2, and uses C++20 throughout.
|
> features modern C++20 throughout. Formatting follows the Google C++ Style
|
||||||
|
> Guide, with the exception of indentation width (3 spaces vs. Google's 2).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Tested hardware
|
## Tested hardware
|
||||||
|
|
||||||
### ARM macOS (M1 Pro)
|
### ARM macOS — M1 Pro
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
| --------- | --------------------------------- |
|
| --------- | --------------------------------- |
|
||||||
@@ -358,7 +376,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
|||||||
| Model | Gemma 4 E4B |
|
| Model | Gemma 4 E4B |
|
||||||
| Inference | llama.cpp with Metal |
|
| Inference | llama.cpp with Metal |
|
||||||
|
|
||||||
### x86_64 Linux (NVIDIA RTX 2000)
|
### x86_64 Linux — NVIDIA RTX 2000
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
| --------- | ------------------------------ |
|
| --------- | ------------------------------ |
|
||||||
@@ -369,7 +387,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
|||||||
| Model | Gemma 4 E4B |
|
| Model | Gemma 4 E4B |
|
||||||
| Inference | llama.cpp with CUDA 12.x |
|
| Inference | llama.cpp with CUDA 12.x |
|
||||||
|
|
||||||
### x86_64 Linux (Docker / RunPod, NVIDIA CUDA)
|
### x86_64 Linux — Docker / RunPod (NVIDIA CUDA)
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
| --------- | ------------------------------------------- |
|
| --------- | ------------------------------------------- |
|
||||||
@@ -389,7 +407,7 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
|||||||
- Keep `locations.json` structured enough to support discovery and future
|
- Keep `locations.json` structured enough to support discovery and future
|
||||||
filtering.
|
filtering.
|
||||||
- `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json`
|
- `personas.json`, `forenames-by-country.json`, and `surnames-by-country.json`
|
||||||
are curated/vendored fixture data, not LLM-generated; see
|
are curated/vendored fixture data, not LLM-generated — see
|
||||||
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for provenance.
|
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for provenance.
|
||||||
- Treat SQLite output as seed material for the app's brewery and user domains,
|
- Treat SQLite output as seed material for the app's brewery and user domains,
|
||||||
not production data.
|
not production data.
|
||||||
@@ -417,32 +435,30 @@ Silicon; CUDA or HIP/ROCm is detected on Linux when the toolkit is present.
|
|||||||
|
|
||||||
Paths below are relative to `tooling/pipeline/`.
|
Paths below are relative to `tooling/pipeline/`.
|
||||||
|
|
||||||
| Path | Purpose |
|
- `src/main.cc` — argument parsing and DI composition root.
|
||||||
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
- `src/biergarten_pipeline_orchestrator/` — orchestration, sampling, logging,
|
||||||
| `src/main.cc` | Argument parsing and DI composition root. |
|
and export.
|
||||||
| `src/biergarten_pipeline_orchestrator/` | Orchestration, sampling, logging, and export. |
|
- `src/services/curated_data/` — `CuratedJsonDataService`, the file-backed
|
||||||
| `src/services/curated_data/` | `CuratedJsonDataService`, the file-backed `ICuratedDataService`, and `MockCuratedDataService`, the in-memory `ICuratedDataService` used in `--mocked` runs. |
|
`ICuratedDataService`, and `MockCuratedDataService`, the in-memory
|
||||||
| `src/services/enrichment/wikipedia/` | Enrichment service and cache. |
|
`ICuratedDataService` used in `--mocked` runs.
|
||||||
| `src/services/sqlite/` | SQLite export implementation. |
|
- `src/services/enrichment/wikipedia/` — enrichment service and cache.
|
||||||
| `src/data_generation/llama/` | Local inference, prompt loading, and output validation. |
|
- `src/services/sqlite/` — SQLite export implementation.
|
||||||
| `src/data_generation/mock/` | Deterministic fallback. |
|
- `src/data_generation/llama/` — local inference, prompt loading, output
|
||||||
| `runpod/` | Container build and runtime launcher. |
|
validation.
|
||||||
|
- `src/data_generation/mock/` — deterministic fallback.
|
||||||
|
- `runpod/` — container build and runtime launcher.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Next steps
|
## Next steps
|
||||||
|
|
||||||
The pipeline currently produces city-aware brewery and user records and dated
|
The pipeline currently produces city-aware brewery and user records and dated
|
||||||
SQLite exports.
|
SQLite exports. The next passes add additional fixture types so the app can
|
||||||
|
exercise the full brewery and social domains without live data. For the detailed
|
||||||
|
engineering breakdown of what's needed to reach the architecture in
|
||||||
|
[`diagrams/planned/`](./diagrams/planned/), see [ROADMAP.md](./ROADMAP.md).
|
||||||
|
|
||||||
The next passes add additional fixture types so the app can exercise the full
|
### Testing — very high priority
|
||||||
brewery and social domains without live data.
|
|
||||||
|
|
||||||
For the detailed engineering breakdown of what's needed to reach the
|
|
||||||
architecture in [`diagrams/planned/`](./diagrams/planned/), see
|
|
||||||
[ROADMAP.md](./ROADMAP.md).
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
|
|
||||||
- Unit test JSON validation and retry logic against malformed, truncated, and
|
- Unit test JSON validation and retry logic against malformed, truncated, and
|
||||||
empty model outputs.
|
empty model outputs.
|
||||||
@@ -465,7 +481,7 @@ enough to exercise search, sort, and category filters.
|
|||||||
### Check-in system
|
### Check-in system
|
||||||
|
|
||||||
Produce timestamped check-in events between users and breweries. Use a J-curve
|
Produce timestamped check-in events between users and breweries. Use a J-curve
|
||||||
activity profile: a small set of users accounts for most check-ins, the rest
|
activity profile — a small set of users accounts for most check-ins, the rest
|
||||||
appear occasionally. Add bursty behaviour around weekends and travel periods.
|
appear occasionally. Add bursty behaviour around weekends and travel periods.
|
||||||
|
|
||||||
### Beer ratings
|
### Beer ratings
|
||||||
|
|||||||
@@ -1,24 +1,21 @@
|
|||||||
---
|
# Pipeline Roadmap — Reaching the Planned Architecture
|
||||||
title: Pipeline roadmap — reaching the planned architecture
|
|
||||||
last-updated: 2026-09-01
|
|
||||||
tags:
|
|
||||||
- pipeline
|
|
||||||
- roadmap
|
|
||||||
- cpp
|
|
||||||
- concurrency
|
|
||||||
---
|
|
||||||
|
|
||||||
Engineering breakdown for closing the gap between the **current** pipeline and
|
This is the engineering breakdown for closing the gap between the **current**
|
||||||
the **planned** architecture in
|
pipeline and the **planned** architecture documented in
|
||||||
[`diagrams/planned/class.md`](./diagrams/planned/class.md) /
|
[`diagrams/planned/class.puml`](./diagrams/planned/class.puml) and
|
||||||
[`diagrams/planned/activity.md`](./diagrams/planned/activity.md). Items are
|
[`diagrams/planned/activity.puml`](./diagrams/planned/activity.puml). Nothing in
|
||||||
grouped by layer, roughly by dependency order. For the current implementation,
|
`diagrams/planned/` is implemented yet — this file tracks what it would take to
|
||||||
see [README.md](./README.md).
|
get there. For the current implementation, see [README.md](./README.md).
|
||||||
|
|
||||||
The only concurrency today is a log dispatcher thread draining a
|
Items are grouped by layer and roughly ordered by dependency: later groups build
|
||||||
`BoundedChannel<LogEntry>` into spdlog; sampling, enrichment, generation, and
|
on types and services introduced earlier.
|
||||||
export run synchronously on the main thread — §6 introduces the planned worker
|
|
||||||
threads.
|
The only concurrency in the current pipeline is the log dispatcher thread
|
||||||
|
(`main.cc` spawns it to drain a `BoundedChannel<LogEntry>` into spdlog for the
|
||||||
|
whole run). Sampling, enrichment, generation, and export all happen
|
||||||
|
synchronously on the main thread inside `BiergartenPipelineOrchestrator::Run()`
|
||||||
|
— §6 below is what introduces the additional worker threads the planned
|
||||||
|
architecture needs.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -26,183 +23,222 @@ threads.
|
|||||||
|
|
||||||
`tooling/pipeline/includes/data_model/generated_models.h` and `models.h`.
|
`tooling/pipeline/includes/data_model/generated_models.h` and `models.h`.
|
||||||
|
|
||||||
- [ ] Add `Completeness` enum (`Full`/`Partial`/`Absent`) and a
|
- [ ] Add `Completeness` enum (`Full`, `Partial`, `Absent`) and a
|
||||||
`LocationContext` struct (`text`, `completeness`, `char_count`); replace
|
`LocationContext` struct (`text`, `completeness`, `char_count`). Replace
|
||||||
`EnrichedCity::region_context` (a plain `string` today) with it.
|
`EnrichedCity::region_context` (currently a plain `std::string`) with
|
||||||
|
`context : LocationContext`.
|
||||||
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`, `min_ibu`,
|
- [ ] Add `BeerStyle` (`name`, `description`, `min_abv`, `max_abv`, `min_ibu`,
|
||||||
`max_ibu`).
|
`max_ibu`).
|
||||||
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` payloads.
|
- [ ] Add `BeerResult`, `CheckinResult`, `RatingResult` result payloads.
|
||||||
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
|
- [ ] Add `GenerationMetadata` (`generation_id`, `generated_time`,
|
||||||
`context_provided`, `generated_with`).
|
`context_provided`, `generated_with`).
|
||||||
- [x] `UserResult` gained `first_name`/`last_name`/`gender`/`activity_weight`
|
- [x] Add `first_name`, `last_name`, `gender`, and `activity_weight` to
|
||||||
(was just `username`/`bio`), copied from the sampled `Name`, not
|
`UserResult` (currently just `username`, `bio`). `first_name`/
|
||||||
|
`last_name`/`gender` are copied from the sampled `Name` (see below), not
|
||||||
LLM-invented.
|
LLM-invented.
|
||||||
- [x] Added `Name` (`first_name`, `last_name`, `gender`) and `ForenameEntry`
|
- [x] Add `Name` (`first_name`, `last_name`, `gender`) — the sampled result
|
||||||
(`name`, `gender`). Flatter than planned: no `NamesByCountry` wrapper —
|
handed to `DataGenerator::GenerateUser`. Add `ForenameEntry` (`name`,
|
||||||
`ICuratedDataService` exposes `ForenamesByCountryMap`/
|
`gender`). Landed as a flatter shape than originally planned here: no
|
||||||
`SurnamesByCountryMap` (keyed by ISO 3166-1) directly, and sampling is a
|
`NamesByCountry` wrapper class. `curated_data_service.h` instead declares
|
||||||
free function, `SampleName(forenames, surnames, iso3166_1, rng)`, not a
|
`ForenameList = std::vector<ForenameEntry>` and
|
||||||
class method.
|
`SurnameList = std::vector<std::string>`, and `ICuratedDataService`
|
||||||
|
exposes two maps keyed by ISO 3166-1 code —
|
||||||
|
`ForenamesByCountryMap = unordered_map<string, ForenameList>` and
|
||||||
|
`SurnamesByCountryMap = unordered_map<string, SurnameList>` — directly
|
||||||
|
(see §2). Sampling is a free function,
|
||||||
|
`SampleName(forenames_by_country, surnames_by_country, iso3166_1, rng)`,
|
||||||
|
in `generate_users.cc`'s anonymous namespace, not a method on a class.
|
||||||
|
Pairing a forename with a surname happens at sample time so gender
|
||||||
|
(present per-forename in the source data) is never discarded the way a
|
||||||
|
pre-flattened `{first_name, last_name}` list would lose it; see §2.
|
||||||
- [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`,
|
- [ ] Extend `GeneratedBrewery` with `brewery_id`, `context_completeness`,
|
||||||
`metadata` (currently just `{location, brewery}`).
|
`metadata` (currently just `{location, brewery}`).
|
||||||
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
|
- [ ] Add `GeneratedBeer`, `GeneratedCheckin`, `GeneratedRating`,
|
||||||
`GeneratedFollow` aggregate structs.
|
`GeneratedFollow` aggregate structs.
|
||||||
- [x] Added `UserRecord` (`location`, `user`, `email`, `date_of_birth`), the
|
- [x] Add `UserRecord` (`location`, `user : UserResult`, `email`,
|
||||||
current stand-in for the planned `GeneratedUser`. `email`/ `date_of_birth`
|
`date_of_birth`) as the current stand-in for the planned `GeneratedUser` —
|
||||||
are programmatic (never LLM-authored), for downstream auth-account
|
`email` and `date_of_birth` are programmatically generated by the
|
||||||
seeding. Missing `user_id`/`password`/`metadata`; already exported via
|
orchestrator, never LLM-authored, so a downstream auth-account seeding
|
||||||
`ProcessRecord(const UserRecord&)` (§5).
|
consumer can register real accounts from the pipeline's SQLite export. No
|
||||||
- [x] Added `UserPersona` (`name`, `description`, `style_affinities`).
|
`password`, `user_id`, or `metadata` field yet, matching `BreweryRecord`'s
|
||||||
|
equally pre-`metadata` shape. Already wired into
|
||||||
|
`IExportService`/`SqliteExportService`: a `users` table exists and
|
||||||
|
`GenerateUsers()` exports each successful record via
|
||||||
|
`ProcessRecord(const UserRecord&)` (see §5) in addition to logging and
|
||||||
|
holding `generated_users_` in memory. Still missing vs. the planned
|
||||||
|
`GeneratedUser`: `user_id`, `password`, and `metadata`.
|
||||||
|
- [x] Add `UserPersona` (`name`, `description`, `style_affinities`).
|
||||||
|
|
||||||
## 2. Data Preloading
|
## 2. Data Preloading
|
||||||
|
|
||||||
`tooling/pipeline/includes/services/curated_data/`, fixture files.
|
`tooling/pipeline/includes/services/curated_data/`, fixture files.
|
||||||
|
|
||||||
- [x] Extracted `ICuratedDataService` (not `DataPreloader`); `Load*()` methods
|
- [x] Extract an interface; have `CuratedJsonDataService` implement it. Landed
|
||||||
take no arguments, return `const&`. `CuratedJsonDataService` (renamed from
|
as `ICuratedDataService` (`services/curated_data/curated_data_service.h`),
|
||||||
`JsonLoader`) takes a `CuratedDataFilePaths` DTO at construction and
|
not `DataPreloader` — `LoadLocations()`, `LoadPersonas()`,
|
||||||
memoizes each `Load*()` result. `MockCuratedDataService` (fixed data for
|
`LoadForenamesByCountry()`, and `LoadSurnamesByCountry()` are all virtual
|
||||||
`US`/`DE`/`FR`/`BE`) binds in its place under `--mocked`.
|
methods returning `const&` and take no arguments. `CuratedJsonDataService`
|
||||||
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` exists and is copied into the
|
(`services/curated_data/curated_json_data_service.h`, originally landed as
|
||||||
Docker image, but no loader reads it and the native CMake build doesn't
|
`JsonLoader` under `json_handling/` before being renamed and moved) takes
|
||||||
copy it into `build/` at all.
|
a `CuratedDataFilePaths` DTO (`locations_path`, `personas_path`,
|
||||||
- [x] Added `LoadPersonas()` and authored `personas.json`.
|
`forenames_path`, `surnames_path`) in its constructor rather than a path
|
||||||
- [x] Added `LoadForenamesByCountry()` / `LoadSurnamesByCountry()` (two methods,
|
per `Load*()` call, and memoizes each result in a private `cache` struct
|
||||||
not one combined `LoadNamesByCountry()`), parsing vendored CC0 data from
|
(`locations`, `personas`, `forenames_by_country`, `surnames_by_country`),
|
||||||
`sigpwned/popular-names-by-country-dataset` — see
|
so a second call on the same instance returns the cached result instead of
|
||||||
ETHICS-AND-KNOWN-ISSUES.md for provenance. Deliberately not pre-paired or
|
re-parsing — safe because `CuratedJsonDataService` outlives every call
|
||||||
filtered to `locations.json`'s countries, to keep per-forename gender and
|
site (owned by `BiergartenPipelineOrchestrator` via
|
||||||
support more countries later without re-sourcing.
|
`unique_ptr<ICuratedDataService>` for the whole run). `main.cc`'s DI
|
||||||
|
injector also gained a `MockCuratedDataService` (fixed in-memory data: 4
|
||||||
|
locations across `US`/`DE`/`FR`/`BE`, 3 personas, and matching
|
||||||
|
forename/surname sets for those four countries), bound in place of
|
||||||
|
`CuratedJsonDataService` under `--mocked`, mirroring the existing
|
||||||
|
`MockEnrichmentService`/`MockGenerator` pattern.
|
||||||
|
- [ ] Add `LoadBeerStyles()`. `beer-styles.json` already exists in the repo and
|
||||||
|
is already copied into the Docker image (`runpod/Dockerfile`), but no
|
||||||
|
loader reads it yet, and the native CMake build doesn't copy it into
|
||||||
|
`build/` at all — `CMakeLists.txt`'s "Runtime Assets" step only copies
|
||||||
|
`locations.json` and `prompts/`.
|
||||||
|
- [x] Add `LoadPersonas()` and author `personas.json` (doesn't exist yet).
|
||||||
|
- [x] Add name-by-country loading, parsing
|
||||||
|
`tooling/pipeline/forenames-by-country.json` and
|
||||||
|
`surnames-by-country.json`. Landed as two separate methods —
|
||||||
|
`LoadForenamesByCountry()` and `LoadSurnamesByCountry()` — each returning
|
||||||
|
a flat `ForenamesByCountryMap` / `SurnamesByCountryMap` (aliases for
|
||||||
|
`unordered_map<string, ForenameList>` /
|
||||||
|
`unordered_map<string, SurnameList>`) keyed by ISO 3166-1 code, rather
|
||||||
|
than one combined `LoadNamesByCountry() : NamesByCountry` call. Both files
|
||||||
|
are vendored verbatim (unmodified, full multinational coverage) from
|
||||||
|
`sigpwned/popular-names-by-country-dataset` (CC0) — see
|
||||||
|
ETHICS-AND-KNOWN-ISSUES.md's Names-by-Country Dataset section for
|
||||||
|
provenance. Deliberately not pre-paired or filtered down to just the
|
||||||
|
countries in `locations.json`: keeping the source shape (including
|
||||||
|
per-forename gender) intact means the loader can support more countries
|
||||||
|
later, or gender-aware persona/bio generation, without re-sourcing data.
|
||||||
|
|
||||||
## 3. Policy / Strategy Layer
|
## 3. Policy / Strategy Layer
|
||||||
|
|
||||||
Entirely new — no `includes/policy/` (or equivalent) exists today.
|
Entirely new — no `includes/policy/` (or equivalent) directory exists today.
|
||||||
|
|
||||||
- [ ] `ContextStrategy` + `BreweryContextStrategy`/`BeerContextStrategy`. Today
|
- [ ] `ContextStrategy` interface + `BreweryContextStrategy` /
|
||||||
`WikipediaEnrichmentService::GetLocationContext` hardcodes a `"brewing"`
|
`BeerContextStrategy`. Today
|
||||||
query and a `"beer in {country}"` query directly.
|
`WikipediaEnrichmentService::GetLocationContext` hardcodes a generic
|
||||||
- [ ] `SamplingStrategy` + `UniformSamplingStrategy`, replacing the inline
|
`"brewing"` query and a `"beer in {country}"` query directly — no
|
||||||
`std::ranges::sample(...)` in `QueryCitiesWithCountries()`.
|
per-phase strategy selection.
|
||||||
- [ ] `BeerSelectionStrategy` + `RandomBeerSelectionStrategy` (depends on §1,
|
- [ ] `SamplingStrategy` interface + `UniformSamplingStrategy`, replacing the
|
||||||
§2).
|
inline `std::ranges::sample(...)` call in
|
||||||
- [ ] `CheckinDistributionStrategy` + `JCurveCheckinStrategy` /
|
`BiergartenPipelineOrchestrator::QueryCitiesWithCountries()`.
|
||||||
`RandomCheckinStrategy` — the "Check-In System" README next step, made
|
- [ ] `BeerSelectionStrategy` interface + `RandomBeerSelectionStrategy`, to pick
|
||||||
concrete.
|
styles per brewery from the `BeerStyle` palette (depends on §1 and §2).
|
||||||
- [ ] `FollowGenerationStrategy` + `RandomFollowStrategy` /
|
- [ ] `CheckinDistributionStrategy` interface + `JCurveCheckinStrategy` /
|
||||||
|
`RandomCheckinStrategy` — this is the "Check-In System" item already
|
||||||
|
called out in README.md's Next Steps, made concrete.
|
||||||
|
- [ ] `FollowGenerationStrategy` interface + `RandomFollowStrategy` /
|
||||||
`ActivityWeightedFollowStrategy`.
|
`ActivityWeightedFollowStrategy`.
|
||||||
|
|
||||||
## 4. Data Generation
|
## 4. Data Generation
|
||||||
|
|
||||||
`tooling/pipeline/includes/data_generation/`, `src/data_generation/`.
|
`tooling/pipeline/includes/data_generation/`, `src/data_generation/`.
|
||||||
|
|
||||||
- [ ] Extend `DataGenerator` with `GenerateBeer`, `GenerateCheckin`,
|
- [ ] Extend the `DataGenerator` interface with `GenerateBeer`,
|
||||||
`GenerateRating` (today: only `GenerateBrewery`/`GenerateUser`).
|
`GenerateCheckin`, `GenerateRating` (today: only `GenerateBrewery` and
|
||||||
- [x] Added `OpenAIGenerator`, a third backend calling the OpenAI Chat
|
`GenerateUser`).
|
||||||
Completions API with Structured Outputs (`json_schema`, `strict`) —
|
- [x] Implement `LlamaGenerator::GenerateUser` for real, with a retry loop
|
||||||
schema-valid JSON with no `IPromptFormatter`/GBNF grammar needed. Selected
|
mirroring `GenerateBrewery` (GBNF grammar, `ValidateUserJson`, up to 3
|
||||||
via `--openai`/`GeneratorMode::kOpenAI`; landed ahead of §3/§4. Now shown
|
attempts with corrective feedback) and a new `prompts/USER_GENERATION.md`.
|
||||||
in `diagrams/planned/class.md` alongside `MockGenerator`/`LlamaGenerator`,
|
Changed the `DataGenerator::GenerateUser` signature to
|
||||||
noting only `GenerateBrewery`/`GenerateUser` are implemented so far.
|
`(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult`,
|
||||||
- [x] Implemented `LlamaGenerator::GenerateUser` for real (GBNF grammar,
|
matching what the activity diagram actually passes.
|
||||||
`ValidateUserJson`, 3 retries with corrective feedback,
|
`MockGenerator::GenerateUser` updated to match the new signature too.
|
||||||
`prompts/USER_GENERATION.md`). Signature is
|
- [ ] Implement `MockGenerator::GenerateBeer` / `GenerateCheckin` /
|
||||||
`(city, persona, name) : UserResult`, matching the activity diagram.
|
`GenerateRating`.
|
||||||
- [ ] Implement
|
|
||||||
`MockGenerator::GenerateBeer`/`GenerateCheckin`/`GenerateRating`.
|
|
||||||
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
|
- [ ] Add `IPromptFormatter::ExpectedArchitecture()` and
|
||||||
`LlamaGenerator::ValidateModelArchitecture()` so a mismatched GGUF chat
|
`LlamaGenerator::ValidateModelArchitecture()` so loading a GGUF that
|
||||||
template fails fast instead of degrading silently.
|
doesn't match the configured chat template fails fast instead of silently
|
||||||
- [ ] Add prompt templates for beer/checkin/rating, next to
|
producing degraded output.
|
||||||
`prompts/BREWERY_GENERATION.md`.
|
- [ ] Add prompt template files for beer/checkin/rating generation next to the
|
||||||
|
existing `prompts/BREWERY_GENERATION.md`.
|
||||||
|
|
||||||
## 5. Export Service
|
## 5. Export Service
|
||||||
|
|
||||||
`tooling/pipeline/includes/services/database/`, `src/services/sqlite/`.
|
`tooling/pipeline/includes/services/database/`, `src/services/sqlite/`.
|
||||||
|
|
||||||
- [x] `IExportService::ProcessRecord(const UserRecord&)` and the `users` table
|
- [x] `IExportService::ProcessRecord(const UserRecord&)` and
|
||||||
are implemented — every successful user is exported with its resolved
|
`SqliteExportService`'s `users` table (see `kCreateUsersTableSql` /
|
||||||
|
`kInsertUserSql` in
|
||||||
|
`includes/services/database/sqlite_statement_helpers.h`) are implemented —
|
||||||
|
`GenerateUsers()` exports every successful user alongside its resolved
|
||||||
`location_id`.
|
`location_id`.
|
||||||
- [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`,
|
- [ ] Extend `IExportService` with `ProcessBeer`, `ProcessCheckin`,
|
||||||
`ProcessRating`, `ProcessFollow`.
|
`ProcessRating`, `ProcessFollow` (today
|
||||||
|
`ProcessRecord(const BreweryRecord&)` and
|
||||||
|
`ProcessRecord(const UserRecord&)` exist).
|
||||||
- [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to
|
- [ ] Add `beers`, `checkins`, `ratings`, `follows` tables to
|
||||||
`InitializeSchema()` (`cities`/`breweries`/`brewery_addresses`/`users`/
|
`SqliteExportService::InitializeSchema()` (`locations`, `breweries`, and
|
||||||
`user_addresses` already exist — see `sqlite_statement_helpers.h` for the
|
`users` already exist) — see `kCreateLocationsTableSql` /
|
||||||
pattern).
|
`kCreateBreweriesTableSql` / `kCreateUsersTableSql` in
|
||||||
- [ ] Add a `brewery_cache_` alongside `city_cache_`, and move from one open
|
`includes/services/database/sqlite_statement_helpers.h` for the existing
|
||||||
transaction for the whole run to per-phase batched commits (`BEGIN` /
|
pattern to follow.
|
||||||
`COMMIT & BEGIN` on a batch threshold), per the planned activity diagram.
|
- [ ] Add a `brewery_cache_` alongside the existing `location_cache_`, and move
|
||||||
|
from one open transaction for the whole run to per-phase batched commits
|
||||||
|
(`BEGIN` / `COMMIT & BEGIN` on a batch-size threshold), as shown in the
|
||||||
|
planned activity diagram.
|
||||||
|
|
||||||
## 6. Concurrency & Orchestration
|
## 6. Concurrency & Orchestration
|
||||||
|
|
||||||
`tooling/pipeline/includes/concurrency/`,
|
`tooling/pipeline/includes/concurrency/`,
|
||||||
`src/biergarten_pipeline_orchestrator/`.
|
`src/biergarten_pipeline_orchestrator/`.
|
||||||
|
|
||||||
- [ ] `BoundedChannel<T>` exists and is production-tested but only wired to the
|
- [ ] `BoundedChannel<T>` already exists and is production-tested — but it's
|
||||||
log channel. Stand up the per-phase producer/consumer channels (`loc_ch`,
|
only wired to the single log channel today. Stand up the per-phase
|
||||||
`exp_ch`, etc.) with a dedicated LLM worker thread and SQLite worker
|
producer/consumer channels (`loc_ch`, `exp_ch`, etc.) the planned activity
|
||||||
thread each; the brewery phase needs a third, a Geocode Worker between
|
diagram describes, each with a dedicated LLM worker thread and SQLite
|
||||||
them on its own `geo_ch` (§9).
|
worker thread.
|
||||||
- [ ] Rewrite `Run()` from one synchronous pass into phased methods —
|
- [ ] Rewrite `BiergartenPipelineOrchestrator::Run()` from one synchronous pass
|
||||||
`RunUserPhase` → `RunBreweryAndBeerPhase` (`RunBeerPhase` sub-step once
|
over `generated_breweries_` into phased methods — `RunUserPhase` →
|
||||||
`brewery_pool_` fills) → `RunCheckinPhase`/`RunFollowPhase` (parallel) →
|
`RunBreweryAndBeerPhase` (with a `RunBeerPhase` sub-step once
|
||||||
`RunRatingPhase` — backed by `user_pool_`, `brewery_pool_`, `beer_pool_`,
|
`brewery_pool_` is populated) → `RunCheckinPhase` / `RunFollowPhase`
|
||||||
`checkin_pool_`, `follow_pool_`.
|
(these two can run in parallel) → `RunRatingPhase` — backed by
|
||||||
- [ ] Structurally join worker threads (e.g. `std::jthread`) before `Run()`
|
`user_pool_`, `brewery_pool_`, `beer_pool_`, `checkin_pool_`,
|
||||||
returns, per the existing comment in `biergarten_pipeline_orchestrator.h`,
|
`follow_pool_`.
|
||||||
so nothing logs to a closed channel during teardown.
|
- [ ] Honor the structural-concurrency requirement already called out in a
|
||||||
|
comment on `BiergartenPipelineOrchestrator::Run()` in
|
||||||
|
`biergarten_pipeline_orchestrator.h`: once real worker threads exist, they
|
||||||
|
must be structurally joined (e.g. via `std::jthread`) before `Run()`
|
||||||
|
returns, so no worker logs to a closed channel during teardown.
|
||||||
|
|
||||||
## 7. Enrichment
|
## 7. Enrichment
|
||||||
|
|
||||||
- [ ] Decide whether to restore the city/region-specific Wikipedia query
|
- [ ] Decide whether to restore the city/region-specific Wikipedia query that's
|
||||||
currently commented out in `get_summary.cc` — the §3 `ContextStrategy`
|
currently commented out in
|
||||||
work is a natural place to reintroduce it.
|
`src/services/enrichment/wikipedia/get_summary.cc` (`GetLocationContext`).
|
||||||
|
The `ContextStrategy` work in §3 is a natural place to reintroduce it via
|
||||||
|
`BreweryContextStrategy::QueriesFor()`.
|
||||||
- [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`,
|
- [ ] Pre-warm caches at startup (`PreWarmBeerStyleCache`,
|
||||||
`PreWarmLocationCache`) instead of fetching lazily per record.
|
`PreWarmLocationCache` in the planned activity diagram) instead of
|
||||||
|
fetching lazily per record, so the streaming phases never block on a cold
|
||||||
|
cache mid-run.
|
||||||
|
|
||||||
## 8. Fixtures / Build
|
## 8. Fixtures / Build
|
||||||
|
|
||||||
- [x] Authored `personas.json`, `forenames-by-country.json`,
|
- [x] Author `personas.json`, `forenames-by-country.json`, and
|
||||||
`surnames-by-country.json` (§2).
|
`surnames-by-country.json` (see §2).
|
||||||
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime Assets"
|
- [ ] Fix the `beer-styles.json` build-tree gap: add it to the "Runtime Assets"
|
||||||
`configure_file` step in `CMakeLists.txt`.
|
`configure_file` step in `CMakeLists.txt` so native builds and the Docker
|
||||||
|
image agree on what's available at runtime.
|
||||||
## 9. Geocoding
|
|
||||||
|
|
||||||
`tooling/pipeline/includes/services/address/`, `src/services/address/` — added
|
|
||||||
entirely after `diagrams/planned/` was authored; wasn't tracked here until now.
|
|
||||||
|
|
||||||
- [x] Added `IAddressService`
|
|
||||||
(`ReverseGeocode(lon, lat) : optional<AddressLookupResult>`),
|
|
||||||
`NominatimAddressService` (public Nominatim API, 1s sleep per request,
|
|
||||||
`nullopt` on failure or no usable address), and `MockAddressService`
|
|
||||||
(fixed placeholder for `--mocked`). Called once per brewery, after
|
|
||||||
`GenerateBrewery` and before export, to resolve
|
|
||||||
`BreweryAddress::address_line1`/`postal_code` — users aren't geocoded.
|
|
||||||
`diagrams/planned/class.md` now models this as an
|
|
||||||
`InfrastructureGeocoding` namespace owned by the orchestrator.
|
|
||||||
- [ ] Give the brewery phase its own **Geocode Worker** thread, between the LLM
|
|
||||||
worker and the SQLite worker (`loc_ch` → LLM Worker → `geo_ch` → Geocode
|
|
||||||
Worker → `exp_ch` → SQLite Worker), rather than calling `ReverseGeocode`
|
|
||||||
inline — Nominatim's rate limit is unrelated to inference speed, so
|
|
||||||
coupling them stalls whichever side is slower and needlessly cools the
|
|
||||||
LLM's KV cache. It never skips a record on lookup failure, only attaches a
|
|
||||||
placeholder; shown as its own subgraph (`B_GEO`) in
|
|
||||||
`diagrams/planned/activity.md`. One worker per phase is safe today since
|
|
||||||
`RunUserPhase` joins before `RunBreweryPhase` starts; concurrent phases
|
|
||||||
plus user-geocoding would need a shared rate-limited client instead, since
|
|
||||||
the throttle is global, not per-thread.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Suggested build order
|
## Suggested build order
|
||||||
|
|
||||||
The planned activity diagram's phase notes imply a dependency order:
|
The planned activity diagram's own phase notes ("brewery*pool* is now fully
|
||||||
|
populated. Phase 1b may begin.", etc.) imply a dependency order. Roughly:
|
||||||
|
|
||||||
1. Domain models (§1) and data preloading (§2) — nothing else compiles without
|
1. Domain models (§1) and data preloading (§2) — nothing else compiles without
|
||||||
these.
|
these.
|
||||||
2. Export schema (§5) — so every generation phase has somewhere to land output.
|
2. Export schema (§5) — so every later generation phase has somewhere to land
|
||||||
|
its output.
|
||||||
3. Policy/strategy layer (§3) and generator interface (§4).
|
3. Policy/strategy layer (§3) and generator interface (§4).
|
||||||
4. Concurrency/orchestration rewrite (§6) — wires §1–§5 into the phased,
|
4. Concurrency/orchestration rewrite (§6), which is the only piece that actually
|
||||||
parallel pipeline the planned diagrams show.
|
wires §1–§5 into the phased, parallel pipeline shown in the planned diagrams.
|
||||||
5. Enrichment cache pre-warming and the city/region query decision (§7) — most
|
5. Enrichment cache pre-warming and the city/region query decision (§7) — useful
|
||||||
valuable once phases run concurrently.
|
at any point, but most valuable once phases run concurrently.
|
||||||
|
|||||||
34
docs/pipeline/diagrams/biergarten-weizen-theme.puml
Normal file
34
docs/pipeline/diagrams/biergarten-weizen-theme.puml
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
skinparam shadowing false
|
||||||
|
skinparam backgroundColor #FCFCF7
|
||||||
|
skinparam defaultFontName "DM Sans"
|
||||||
|
skinparam defaultFontColor #14180C
|
||||||
|
skinparam titleFontName "Volkhov"
|
||||||
|
skinparam titleFontColor #14180C
|
||||||
|
skinparam ArrowColor #656F33
|
||||||
|
skinparam NoteBackgroundColor #DBEEDD
|
||||||
|
skinparam NoteFontColor #14180C
|
||||||
|
skinparam NoteBorderColor #4A5837
|
||||||
|
skinparam SwimlaneBorderColor #4A5837
|
||||||
|
skinparam SwimlaneBorderThickness 1
|
||||||
|
skinparam activityStartColor #EBECE3
|
||||||
|
skinparam activityEndColor #4A5837
|
||||||
|
skinparam activityStopColor #4A5837
|
||||||
|
skinparam ActivityBackgroundColor #EBECE3
|
||||||
|
skinparam ActivityBorderColor #4A5837
|
||||||
|
skinparam ActivityDiamondBackgroundColor #CBD2B5
|
||||||
|
skinparam ActivityDiamondBorderColor #4A5837
|
||||||
|
skinparam packageStyle rectangle
|
||||||
|
skinparam packageBackgroundColor #F1F3EA
|
||||||
|
skinparam packageBorderColor #4A5837
|
||||||
|
skinparam packageFontColor #14180C
|
||||||
|
skinparam classBackgroundColor #EBECE3
|
||||||
|
skinparam classBorderColor #4A5837
|
||||||
|
skinparam classFontColor #14180C
|
||||||
|
skinparam classAttributeFontColor #3F4724
|
||||||
|
skinparam classStereotypeFontColor #4A5837
|
||||||
|
skinparam interfaceBackgroundColor #DBEEDD
|
||||||
|
skinparam interfaceBorderColor #4A5837
|
||||||
|
skinparam interfaceFontColor #14180C
|
||||||
|
skinparam enumBackgroundColor #E4E6D8
|
||||||
|
skinparam enumBorderColor #4A5837
|
||||||
|
skinparam enumFontColor #14180C
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
# The Biergarten Data Pipeline (Current — Synchronous Data Path)
|
|
||||||
|
|
||||||
The only concurrency in the current pipeline is a dedicated log dispatcher
|
|
||||||
thread that drains `log_channel` and forwards entries to spdlog for the entire
|
|
||||||
run (joined during shutdown after `log_channel` closes).
|
|
||||||
`BiergartenPipelineOrchestrator::Run()` itself is synchronous — sampling,
|
|
||||||
enrichment, generation, and export all happen on the main thread.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "pipeline/diagrams/current/activity.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
flowchart LR
|
|
||||||
Start([start]) --> ParseArgs["main.cc: ParseArguments(argc, argv)"]
|
|
||||||
ParseArgs --> ArgsValid{Arguments valid?}
|
|
||||||
ArgsValid -- no --> LogErr[Log error / usage info] --> Stop1([stop])
|
|
||||||
ArgsValid -- yes --> Init["main.cc: Init OpenSSL global state and LlamaBackendState"]
|
|
||||||
Init --> DI["main.cc: di::make_injector(...)<br/>binds ILogger, IEnrichmentService, DataGenerator, IExportService, IPromptFormatter, WebClient based on --mocked vs. model-backed mode"]
|
|
||||||
DI --> LogThread["main.cc: create BoundedChannel log_channel, spawn log dispatcher thread (LogDispatcher::Run)"]
|
|
||||||
LogThread --> CreateOrch["main.cc: injector.create BiergartenPipelineOrchestrator"]
|
|
||||||
CreateOrch --> Run["BiergartenPipelineOrchestrator::Run()"]
|
|
||||||
|
|
||||||
Run --> SqliteInit["SqliteExportService.Initialize()<br/>builds biergarten_seed_<UTC datetime>.sqlite (numeric suffix if it exists), opens DB connection, executes schema DDL, begins transaction"]
|
|
||||||
SqliteInit --> QueryLoc[Orchestrator: QueryLocations]
|
|
||||||
QueryLoc --> LoadLoc["CuratedJsonDataService.LoadLocations()"]
|
|
||||||
LoadLoc --> SampleLoc["std::ranges::sample(all_locations, --location-count) — defaults to 10"]
|
|
||||||
|
|
||||||
SampleLoc --> EnrichLoop{For each sampled Location}
|
|
||||||
EnrichLoop --> GetContext["IEnrichmentService.GetLocationContext(loc)<br/>WikipediaEnrichmentService fetches a generic 'brewing' extract and a 'beer in {country}' extract (not city/region-specific); MockEnrichmentService (--mocked) returns an empty string"]
|
|
||||||
GetContext --> LookupFailed{Lookup failed?}
|
|
||||||
LookupFailed -- yes --> SkipCity1[Log warning, skip city] --> EnrichLoop
|
|
||||||
LookupFailed -- no --> StoreEnriched["Store EnrichedCity{Location, region_context}"] --> EnrichLoop
|
|
||||||
EnrichLoop -- done --> GenUsers[Orchestrator: GenerateUsers enriched_cities]
|
|
||||||
SkipCity1 ~~~ GenUsers
|
|
||||||
StoreEnriched ~~~ GenUsers
|
|
||||||
|
|
||||||
GenUsers --> LoadUserData["CuratedJsonDataService: LoadPersonas / LoadForenamesByCountry / LoadSurnamesByCountry<br/>each memoized after first parse; MockCuratedDataService (--mocked) returns a fixed in-memory dataset instead, skipping these JSON files"]
|
|
||||||
|
|
||||||
LoadUserData --> UserLoop{For each EnrichedCity}
|
|
||||||
UserLoop --> SampleName["SampleName(forenames_by_country, surnames_by_country, location.iso3166_1)"]
|
|
||||||
SampleName --> NamesAvail{Names available for country?}
|
|
||||||
NamesAvail -- no --> SkipCity2[Log warning, skip city] --> UserLoop
|
|
||||||
NamesAvail -- yes --> SamplePersona["Sample UserPersona (uniform random)"]
|
|
||||||
SamplePersona --> UserGenMode{Generator mode}
|
|
||||||
|
|
||||||
UserGenMode -- MockGenerator --> UserMockGen[DeterministicHash and Format]
|
|
||||||
UserGenMode -- LlamaGenerator --> UserLlamaLoad["prompt_directory_.Load(USER_GENERATION) → USER_GENERATION.md in --prompt-dir"]
|
|
||||||
UserLlamaLoad --> UserLlamaRetry["Infer(...) with kUserJsonGrammar → ValidateUserJson<br/>retry up to 3 attempts, appending validation error to the retry prompt on failure"]
|
|
||||||
UserGenMode -- OpenAIGenerator --> UserOAILoad["prompt_directory_.Load(USER_GENERATION) — same prompt file as LlamaGenerator, content is backend-agnostic"]
|
|
||||||
UserOAILoad --> UserOAIRetry["CallChatCompletionsApi(...) with kUserJsonSchema → ValidateUserJson<br/>retry up to 2 attempts — Structured Outputs (json_schema, strict) already guarantees schema shape, so retries only cover transient failures / placeholder text"]
|
|
||||||
|
|
||||||
UserMockGen --> UserGenDone{Generation successful?}
|
|
||||||
UserLlamaRetry --> UserGenDone
|
|
||||||
UserOAIRetry --> UserGenDone
|
|
||||||
|
|
||||||
UserGenDone -- yes --> UserPostProc["GetRandomCoordsWithinRange (5km jitter), BuildEmail, GenerateDateOfBirth<br/>email and date_of_birth are generated programmatically, never LLM-authored; generation + coordinate jitter run inside one try/catch per record"]
|
|
||||||
UserGenDone -- no --> UserGenFail["Log warning: generation failed, skipping"] --> UserLoop
|
|
||||||
|
|
||||||
UserPostProc --> UserExport["SqliteExportService.ProcessRecord(UserRecord)<br/>reuse cached city_id or insert City and cache its id; insert User + UserAddress (FK city_id)"]
|
|
||||||
UserExport --> UserExportFail{Exception during insert?}
|
|
||||||
UserExportFail -- yes --> UserExportWarn["Log warning: SQLite export failed"] --> UserLoop
|
|
||||||
UserExportFail -- no --> UserLoop
|
|
||||||
UserLoop -- done --> GenBrew[Orchestrator: GenerateBreweries enriched_cities]
|
|
||||||
SkipCity2 ~~~ GenBrew
|
|
||||||
UserGenFail ~~~ GenBrew
|
|
||||||
UserExportWarn ~~~ GenBrew
|
|
||||||
UserExportFail ~~~ GenBrew
|
|
||||||
|
|
||||||
GenBrew --> BrewLoop{For each EnrichedCity}
|
|
||||||
BrewLoop --> BrewGenMode{Generator mode}
|
|
||||||
|
|
||||||
BrewGenMode -- MockGenerator --> BrewMockGen[DeterministicHash and Format]
|
|
||||||
BrewGenMode -- LlamaGenerator --> BrewLlamaLoad["PrepareRegionContext, prompt_directory_.Load(BREWERY_GENERATION) → BREWERY_GENERATION.md"]
|
|
||||||
BrewLlamaLoad --> BrewLlamaRetry["Infer(...) with kBreweryJsonGrammar → ValidateBreweryJson<br/>retry up to 3 attempts; max_tokens is fixed across retries, not raised on truncation"]
|
|
||||||
BrewGenMode -- OpenAIGenerator --> BrewOAILoad["PrepareRegionContext, prompt_directory_.Load(BREWERY_GENERATION) — same prompt file as LlamaGenerator"]
|
|
||||||
BrewOAILoad --> BrewOAIRetry["CallChatCompletionsApi(...) with kBreweryJsonSchema → ValidateBreweryJson<br/>retry up to 2 attempts — Structured Outputs already guarantees schema shape"]
|
|
||||||
|
|
||||||
BrewMockGen --> BrewGenDone{Generation successful?}
|
|
||||||
BrewLlamaRetry --> BrewGenDone
|
|
||||||
BrewOAIRetry --> BrewGenDone
|
|
||||||
|
|
||||||
BrewGenDone -- yes --> BrewPostProc["GetRandomCoordsWithinRange (5km jitter)<br/>generation + jitter run inside one try/catch per record"]
|
|
||||||
BrewGenDone -- no --> BrewGenFail["Log warning: generation failed, skipping"] --> BrewLoop
|
|
||||||
|
|
||||||
BrewPostProc --> BrewGeocode["IAddressService.ReverseGeocode(longitude, latitude)<br/>NominatimAddressService hits the live Nominatim API (1 req/sec) and returns nullopt on failure; MockAddressService (--mocked) returns a fixed placeholder address instead"]
|
|
||||||
BrewGeocode --> BrewGeocodeFailed{Geocode resolved?}
|
|
||||||
BrewGeocodeFailed -- no --> BrewPlaceholder["Log warning: could not reverse geocode; use placeholder address_line1, empty postal_code"] --> BrewExport
|
|
||||||
BrewGeocodeFailed -- yes --> BrewExport["SqliteExportService.ProcessRecord(BreweryRecord)<br/>reuse cached city_id or insert City and cache its id; insert Brewery + BreweryAddress (FK city_id)"]
|
|
||||||
BrewExport --> BrewExportFail{Exception during insert?}
|
|
||||||
BrewExportFail -- yes --> BrewExportWarn["Log warning: SQLite export failed (data loss prevented per-record; pipeline continues)"] --> BrewLoop
|
|
||||||
BrewExportFail -- no --> BrewLoop
|
|
||||||
BrewLoop -- done --> Finalize["SqliteExportService.Finalize()<br/>commits transaction, closes database connection"]
|
|
||||||
BrewGenFail ~~~ Finalize
|
|
||||||
BrewExportWarn ~~~ Finalize
|
|
||||||
BrewExportFail ~~~ Finalize
|
|
||||||
|
|
||||||
Finalize --> CloseLog["main.cc: close log_channel, join log dispatcher thread"]
|
|
||||||
CloseLog --> Stop2(["return 0 / stop"])
|
|
||||||
235
docs/pipeline/diagrams/current/activity.puml
Normal file
235
docs/pipeline/diagrams/current/activity.puml
Normal file
@@ -0,0 +1,235 @@
|
|||||||
|
@startuml
|
||||||
|
skinparam style strictuml
|
||||||
|
skinparam defaultFontName "DM Sans"
|
||||||
|
skinparam defaultFontSize 14
|
||||||
|
skinparam titleFontName "Volkhov"
|
||||||
|
skinparam titleFontSize 20
|
||||||
|
skinparam backgroundColor #FAFCF9
|
||||||
|
skinparam defaultFontColor #28342A
|
||||||
|
skinparam titleFontColor #28342A
|
||||||
|
skinparam ArrowColor #628A5B
|
||||||
|
skinparam NoteBackgroundColor #EAF0E8
|
||||||
|
skinparam NoteBorderColor #547461
|
||||||
|
skinparam ActivityBackgroundColor #FAFCF9
|
||||||
|
skinparam ActivityBorderColor #547461
|
||||||
|
skinparam ActivityDiamondBackgroundColor #FAFCF9
|
||||||
|
skinparam ActivityDiamondBorderColor #628A5B
|
||||||
|
skinparam ActivityBarColor #628A5B
|
||||||
|
skinparam SwimlaneBorderColor #547461
|
||||||
|
skinparam SwimlaneBorderThickness 0.3
|
||||||
|
|
||||||
|
title The Biergarten Data Pipeline (Current — Synchronous Data Path)
|
||||||
|
|
||||||
|
|#F2F6F0|main.cc|
|
||||||
|
start
|
||||||
|
:Create BoundedChannel<LogEntry> log_channel;
|
||||||
|
:Spawn log dispatcher thread\n(LogDispatcher::Run());
|
||||||
|
note right
|
||||||
|
The only concurrency in the current pipeline:
|
||||||
|
a dedicated thread drains log_channel and
|
||||||
|
forwards entries to spdlog for the entire run.
|
||||||
|
Joined during shutdown after log_channel closes.
|
||||||
|
end note
|
||||||
|
:ParseArguments(argc, argv);
|
||||||
|
if (Are arguments valid?) then (no)
|
||||||
|
:Log error / usage info;
|
||||||
|
stop
|
||||||
|
else (yes)
|
||||||
|
endif
|
||||||
|
|
||||||
|
:Init OpenSSL global state & LlamaBackendState;
|
||||||
|
:di::make_injector(...);
|
||||||
|
note right
|
||||||
|
Binds ILogger, IEnrichmentService, DataGenerator,
|
||||||
|
IExportService, IPromptFormatter, WebClient based
|
||||||
|
on --mocked vs. model-backed mode.
|
||||||
|
end note
|
||||||
|
:injector.create<std::unique_ptr<BiergartenPipelineOrchestrator>>();
|
||||||
|
:BiergartenPipelineOrchestrator::Run();
|
||||||
|
note right
|
||||||
|
Run() itself is synchronous — sampling, enrichment,
|
||||||
|
generation, and export all happen on this thread.
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:Initialize SQLite export;
|
||||||
|
|
||||||
|
|#E0EAE0|SqliteExportService|
|
||||||
|
:GetUtcTimestamp() from SystemDateTimeProvider;
|
||||||
|
:Initialize();
|
||||||
|
note right
|
||||||
|
Builds a fresh biergarten_seed_<UTC datetime>.sqlite filename
|
||||||
|
Appends a numeric suffix if the timestamp already exists
|
||||||
|
Opens DB Connection
|
||||||
|
Executes Schema DDL
|
||||||
|
Begins Transaction
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:QueryCitiesWithCountries();
|
||||||
|
|
||||||
|
|#E2EBDC|JsonLoader|
|
||||||
|
:JsonLoader::LoadLocations("locations.json");
|
||||||
|
:std::ranges::sample(all_locations, --location-count);
|
||||||
|
note right
|
||||||
|
--location-count defaults to 10.
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
while (For each sampled Location?) is (Remaining cities)
|
||||||
|
|#DCE8D8|IEnrichmentService|
|
||||||
|
:GetLocationContext(loc);
|
||||||
|
note right
|
||||||
|
WikipediaEnrichmentService fetches a generic
|
||||||
|
"brewing" extract and a "beer in {country}" extract
|
||||||
|
(not a city/region-specific page). MockEnrichmentService
|
||||||
|
(--mocked) returns an empty string instead.
|
||||||
|
end note
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
if (Lookup failed?) then (yes)
|
||||||
|
:Log warning, skip city;
|
||||||
|
else (no)
|
||||||
|
:Store EnrichedCity{Location, region_context};
|
||||||
|
endif
|
||||||
|
endwhile (Done)
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:GenerateUsers(enriched_cities);
|
||||||
|
|
||||||
|
|#E2EBDC|JsonLoader|
|
||||||
|
:LoadPersonas("personas.json");
|
||||||
|
:LoadForenamesByCountry("forenames-by-country.json");
|
||||||
|
:LoadSurnamesByCountry("surnames-by-country.json");
|
||||||
|
note right
|
||||||
|
Each call is cached after its first parse.
|
||||||
|
MockCuratedDataService (--mocked) returns a
|
||||||
|
fixed in-memory dataset instead and skips
|
||||||
|
these three JSON files entirely.
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
while (For each EnrichedCity?) is (Remaining cities)
|
||||||
|
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1);
|
||||||
|
if (Names available for country?) then (no)
|
||||||
|
:Log warning, skip city;
|
||||||
|
else (yes)
|
||||||
|
:Sample UserPersona (uniform random);
|
||||||
|
|
||||||
|
|#E5EDE1|DataGenerator|
|
||||||
|
if (Generator Mode) then (MockGenerator)
|
||||||
|
:DeterministicHash & Format;
|
||||||
|
else (LlamaGenerator)
|
||||||
|
:prompt_directory_->Load("USER_GENERATION");
|
||||||
|
note right
|
||||||
|
Resolves to USER_GENERATION.md inside --prompt-dir.
|
||||||
|
end note
|
||||||
|
repeat
|
||||||
|
:Infer(system_prompt, user_prompt, max_tokens, kUserJsonGrammar);
|
||||||
|
:ValidateUserJson(raw, user);
|
||||||
|
if (Is JSON Valid?) then (yes)
|
||||||
|
break
|
||||||
|
else (no)
|
||||||
|
:Attempt++;
|
||||||
|
:Append validation error to retry prompt;
|
||||||
|
endif
|
||||||
|
repeat while (Attempt < 3?) is (yes)
|
||||||
|
endif
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
if (Generation successful?) then (yes)
|
||||||
|
:BuildEmail(name, used_email_local_parts);
|
||||||
|
:GenerateDateOfBirth(rng);
|
||||||
|
note right
|
||||||
|
email and date_of_birth are generated
|
||||||
|
programmatically, never LLM-authored.
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#E0EAE0|SqliteExportService|
|
||||||
|
:ProcessRecord(UserRecord);
|
||||||
|
if (Location in cache?) then (yes)
|
||||||
|
:Reuse location_id;
|
||||||
|
else (no)
|
||||||
|
:Insert Location & Cache ID;
|
||||||
|
endif
|
||||||
|
:Insert User (FK: location_id);
|
||||||
|
|
||||||
|
if (Exception caught during insert?) then (yes)
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:Log warning "SQLite export failed";
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
else (no)
|
||||||
|
:Log warning "Generation failed, skipping...";
|
||||||
|
endif
|
||||||
|
endif
|
||||||
|
endwhile (Done)
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:GenerateBreweries(enriched_cities);
|
||||||
|
|
||||||
|
|#E5EDE1|DataGenerator|
|
||||||
|
while (For each EnrichedCity?) is (Remaining cities)
|
||||||
|
if (Generator Mode) then (MockGenerator)
|
||||||
|
:DeterministicHash & Format;
|
||||||
|
else (LlamaGenerator)
|
||||||
|
:PrepareRegionContext;
|
||||||
|
:prompt_directory_->Load("BREWERY_GENERATION");
|
||||||
|
note right
|
||||||
|
Resolves to BREWERY_GENERATION.md inside --prompt-dir.
|
||||||
|
end note
|
||||||
|
repeat
|
||||||
|
:Infer(system_prompt, user_prompt, max_tokens, kBreweryJsonGrammar);
|
||||||
|
:ValidateBreweryJson(raw, brewery);
|
||||||
|
if (Is JSON Valid?) then (yes)
|
||||||
|
break
|
||||||
|
else (no)
|
||||||
|
:Attempt++;
|
||||||
|
:Append validation error to retry prompt;
|
||||||
|
endif
|
||||||
|
repeat while (Attempt < 3?) is (yes)
|
||||||
|
note right
|
||||||
|
max_tokens is fixed across retries —
|
||||||
|
it is not raised on truncation.
|
||||||
|
end note
|
||||||
|
endif
|
||||||
|
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
if (Generation successful?) then (yes)
|
||||||
|
|#E0EAE0|SqliteExportService|
|
||||||
|
:ProcessRecord(GeneratedBrewery);
|
||||||
|
if (Location in cache?) then (yes)
|
||||||
|
:Reuse location_id;
|
||||||
|
else (no)
|
||||||
|
:Insert Location & Cache ID;
|
||||||
|
endif
|
||||||
|
:Insert Brewery (FK: location_id);
|
||||||
|
|
||||||
|
if (Exception caught during insert?) then (yes)
|
||||||
|
|#EAF0E8|BiergartenPipelineOrchestrator|
|
||||||
|
:Log warning "SQLite export failed";
|
||||||
|
note right
|
||||||
|
Data loss is prevented per-record.
|
||||||
|
The pipeline continues running.
|
||||||
|
end note
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
else (no)
|
||||||
|
:Log warning "Generation failed, skipping...";
|
||||||
|
endif
|
||||||
|
|#E5EDE1|DataGenerator|
|
||||||
|
endwhile (Done)
|
||||||
|
|
||||||
|
|#E0EAE0|SqliteExportService|
|
||||||
|
:Finalize();
|
||||||
|
note right
|
||||||
|
Commits Transaction
|
||||||
|
Closes Database Connection
|
||||||
|
end note
|
||||||
|
|
||||||
|
|#F2F6F0|main.cc|
|
||||||
|
:Close log_channel;
|
||||||
|
:Join log dispatcher thread;
|
||||||
|
:Return 0;
|
||||||
|
stop
|
||||||
|
|
||||||
|
@enduml
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# The Biergarten Data Pipeline — Class Diagram (Current)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "pipeline/diagrams/current/class.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,290 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
classDiagram
|
|
||||||
class BiergartenPipelineOrchestrator {
|
|
||||||
-shared_ptr~ILogger~ logger_
|
|
||||||
-unique_ptr~IEnrichmentService~ context_service_
|
|
||||||
-unique_ptr~DataGenerator~ generator_
|
|
||||||
-unique_ptr~IExportService~ exporter_
|
|
||||||
-unique_ptr~ICuratedDataService~ curated_data_service_
|
|
||||||
-unique_ptr~IAddressService~ address_service_
|
|
||||||
-ApplicationOptions application_options_
|
|
||||||
-vector~BreweryRecord~ generated_breweries_
|
|
||||||
-vector~UserRecord~ generated_users_
|
|
||||||
+Run() bool
|
|
||||||
-QueryLocations() vector~City~
|
|
||||||
-GenerateBreweries(cities) void
|
|
||||||
-GenerateUsers(cities) void
|
|
||||||
-LogResults() void
|
|
||||||
}
|
|
||||||
note for BiergartenPipelineOrchestrator "GenerateBreweries/GenerateUsers jitter each address via GetRandomCoordsWithinRange(), uniformly within 5km of the city centre. GenerateBreweries additionally reverse-geocodes the jittered coordinates via address_service_."
|
|
||||||
|
|
||||||
class LogLevel {
|
|
||||||
<<enumeration>>
|
|
||||||
Debug
|
|
||||||
Info
|
|
||||||
Warn
|
|
||||||
Error
|
|
||||||
}
|
|
||||||
|
|
||||||
class PipelinePhase {
|
|
||||||
<<enumeration>>
|
|
||||||
Startup
|
|
||||||
Enrichment
|
|
||||||
UserGeneration
|
|
||||||
BreweryAndBeerGeneration
|
|
||||||
CheckinGeneration
|
|
||||||
RatingGeneration
|
|
||||||
FollowGeneration
|
|
||||||
Teardown
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogDTO {
|
|
||||||
<<struct>>
|
|
||||||
+LogLevel level
|
|
||||||
+PipelinePhase phase
|
|
||||||
+string message
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogEntry {
|
|
||||||
<<struct>>
|
|
||||||
+time_point timestamp
|
|
||||||
+source_location origin
|
|
||||||
+thread::id thread_id
|
|
||||||
+LogLevel level
|
|
||||||
+PipelinePhase phase
|
|
||||||
+string message
|
|
||||||
}
|
|
||||||
|
|
||||||
class ILogger {
|
|
||||||
<<interface>>
|
|
||||||
+Log(payload) void
|
|
||||||
-DoLog(entry)* void
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogProducer {
|
|
||||||
-BoundedChannel~LogEntry~ channel_
|
|
||||||
-DoLog(entry) void
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogDispatcher {
|
|
||||||
-BoundedChannel~LogEntry~ channel_
|
|
||||||
+Run() void
|
|
||||||
-ToSpdlogLevel(level) level_enum
|
|
||||||
}
|
|
||||||
|
|
||||||
class IEnrichmentService {
|
|
||||||
<<interface>>
|
|
||||||
+GetLocationContext(loc) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class MockEnrichmentService {
|
|
||||||
+GetLocationContext(loc) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class WikipediaEnrichmentService {
|
|
||||||
-unique_ptr~WebClient~ client_
|
|
||||||
-unordered_map~string, string~ extract_cache_
|
|
||||||
+GetLocationContext(loc) string
|
|
||||||
-FetchExtract(query) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class WebClient {
|
|
||||||
<<interface>>
|
|
||||||
+Get(url) string
|
|
||||||
+UrlEncode(value) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class HttpWebClient {
|
|
||||||
+Get(url) string
|
|
||||||
+UrlEncode(value) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class DataGenerator {
|
|
||||||
<<interface>>
|
|
||||||
+GenerateBrewery(city) BreweryResult
|
|
||||||
+GenerateUser(city, persona, name) UserResult
|
|
||||||
}
|
|
||||||
|
|
||||||
class MockGenerator {
|
|
||||||
+GenerateBrewery(city) BreweryResult
|
|
||||||
+GenerateUser(city, persona, name) UserResult
|
|
||||||
-DeterministicHash(location) size_t
|
|
||||||
-DeterministicHash(location, persona, name) size_t
|
|
||||||
}
|
|
||||||
|
|
||||||
class LlamaGenerator {
|
|
||||||
-ModelHandle model_
|
|
||||||
-ContextHandle context_
|
|
||||||
-unique_ptr~IPromptFormatter~ prompt_formatter_
|
|
||||||
-unique_ptr~IPromptDirectory~ prompt_directory_
|
|
||||||
-mt19937 rng_
|
|
||||||
+GenerateBrewery(...) BreweryResult
|
|
||||||
+GenerateUser(...) UserResult
|
|
||||||
-Load(model_path) void
|
|
||||||
-Infer(...) string
|
|
||||||
-InferFormatted(...) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class OpenAIGenerator {
|
|
||||||
-string api_key_
|
|
||||||
-string model_
|
|
||||||
-shared_ptr~ILogger~ logger_
|
|
||||||
-unique_ptr~IPromptDirectory~ prompt_directory_
|
|
||||||
-unique_ptr~WebClient~ web_client_
|
|
||||||
+GenerateBrewery(city) BreweryResult
|
|
||||||
+GenerateUser(city, persona, name) UserResult
|
|
||||||
-CallChatCompletionsApi(...) string
|
|
||||||
}
|
|
||||||
note for OpenAIGenerator "Chat Completions with Structured Outputs (json_schema, strict) guarantees schema-valid JSON, replacing llama.cpp's GBNF grammar — no IPromptFormatter needed"
|
|
||||||
|
|
||||||
class IPromptFormatter {
|
|
||||||
<<interface>>
|
|
||||||
+Format(system_prompt, user_prompt) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class Gemma4JinjaPromptFormatter {
|
|
||||||
+Format(system_prompt, user_prompt) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class IPromptDirectory {
|
|
||||||
<<interface>>
|
|
||||||
+Load(key) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class PromptDirectory {
|
|
||||||
-path prompt_dir_
|
|
||||||
-unordered_map~string, string~ cache_
|
|
||||||
+Load(key) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class ICuratedDataService {
|
|
||||||
<<interface>>
|
|
||||||
+LoadLocations() vector~Location~
|
|
||||||
+LoadPersonas() vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry() unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry() unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
|
|
||||||
class CuratedJsonDataService {
|
|
||||||
-CuratedDataFilePaths filepaths_
|
|
||||||
-cache cache_
|
|
||||||
+LoadLocations() vector~Location~
|
|
||||||
+LoadPersonas() vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry() unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry() unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
note for CuratedJsonDataService "Takes a CuratedDataFilePaths DTO (locations/personas/forenames/surnames paths) once at construction, not per Load* call. Each Load* method memoizes its result in cache_ after its first call."
|
|
||||||
|
|
||||||
class MockCuratedDataService {
|
|
||||||
-vector~Location~ locations_
|
|
||||||
-vector~UserPersona~ personas_
|
|
||||||
-unordered_map~string, forename_list~ forenames_by_country_
|
|
||||||
-unordered_map~string, surname_list~ surnames_by_country_
|
|
||||||
+LoadLocations() vector~Location~
|
|
||||||
+LoadPersonas() vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry() unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry() unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
note for MockCuratedDataService "Fixed 4-location/3-persona dataset (US, DE, FR, BE) for --mocked runs"
|
|
||||||
|
|
||||||
class IAddressService {
|
|
||||||
<<interface>>
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
|
|
||||||
class NominatimAddressService {
|
|
||||||
-unique_ptr~WebClient~ client_
|
|
||||||
-shared_ptr~ILogger~ logger_
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
note for NominatimAddressService "Queries the public Nominatim API; sleeps 1s per request per its usage policy; returns nullopt on failure or when no usable street address is found"
|
|
||||||
|
|
||||||
class MockAddressService {
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
note for MockAddressService "Returns a fixed placeholder address ('123 Mock Street', '00000') for --mocked runs, no network call"
|
|
||||||
|
|
||||||
class IExportService {
|
|
||||||
<<interface>>
|
|
||||||
+Initialize() void
|
|
||||||
+ProcessRecord(brewery) uint64_t
|
|
||||||
+ProcessRecord(user) uint64_t
|
|
||||||
+Finalize() void
|
|
||||||
}
|
|
||||||
|
|
||||||
class SqliteExportService {
|
|
||||||
-unique_ptr~IDateTimeProvider~ date_time_provider_
|
|
||||||
-string run_timestamp_utc_
|
|
||||||
-path database_path_
|
|
||||||
-SqliteDatabaseHandle db_handle_
|
|
||||||
-SqliteStatementHandle insert_city_stmt_
|
|
||||||
-SqliteStatementHandle insert_brewery_stmt_
|
|
||||||
-SqliteStatementHandle insert_brewery_address_stmt_
|
|
||||||
-SqliteStatementHandle insert_user_stmt_
|
|
||||||
-SqliteStatementHandle insert_user_address_stmt_
|
|
||||||
-bool transaction_open_
|
|
||||||
-unordered_map~string, sqlite3_int64~ city_cache_
|
|
||||||
+Initialize() void
|
|
||||||
+ProcessRecord(brewery) uint64_t
|
|
||||||
+ProcessRecord(user) uint64_t
|
|
||||||
+Finalize() void
|
|
||||||
-InitializeSchema() void
|
|
||||||
-ResolveCityId(city) sqlite3_int64
|
|
||||||
}
|
|
||||||
|
|
||||||
class IDateTimeProvider {
|
|
||||||
<<interface>>
|
|
||||||
+GetUtcTimestamp() string
|
|
||||||
}
|
|
||||||
|
|
||||||
class SystemDateTimeProvider {
|
|
||||||
+GetUtcTimestamp() string
|
|
||||||
}
|
|
||||||
|
|
||||||
BiergartenPipelineOrchestrator *-- ILogger : owns
|
|
||||||
BiergartenPipelineOrchestrator *-- IEnrichmentService : owns
|
|
||||||
BiergartenPipelineOrchestrator *-- DataGenerator : owns
|
|
||||||
BiergartenPipelineOrchestrator *-- IExportService : owns
|
|
||||||
BiergartenPipelineOrchestrator *-- ICuratedDataService : owns
|
|
||||||
BiergartenPipelineOrchestrator *-- IAddressService : owns
|
|
||||||
|
|
||||||
LogEntry *-- LogLevel
|
|
||||||
LogEntry *-- PipelinePhase
|
|
||||||
LogDTO *-- LogLevel
|
|
||||||
LogDTO *-- PipelinePhase
|
|
||||||
ILogger <|.. LogProducer : implements
|
|
||||||
LogProducer ..> LogEntry : emits
|
|
||||||
LogDispatcher ..> LogEntry : consumes
|
|
||||||
|
|
||||||
IEnrichmentService <|.. WikipediaEnrichmentService : implements
|
|
||||||
IEnrichmentService <|.. MockEnrichmentService : implements
|
|
||||||
WikipediaEnrichmentService *-- WebClient : owns
|
|
||||||
|
|
||||||
WebClient <|.. HttpWebClient : implements
|
|
||||||
|
|
||||||
DataGenerator <|.. MockGenerator : implements
|
|
||||||
DataGenerator <|.. LlamaGenerator : implements
|
|
||||||
DataGenerator <|.. OpenAIGenerator : implements
|
|
||||||
|
|
||||||
LlamaGenerator *-- IPromptFormatter : uses
|
|
||||||
LlamaGenerator *-- IPromptDirectory : uses
|
|
||||||
|
|
||||||
OpenAIGenerator *-- IPromptDirectory : uses
|
|
||||||
OpenAIGenerator *-- WebClient : uses
|
|
||||||
|
|
||||||
IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements
|
|
||||||
IPromptDirectory <|.. PromptDirectory : implements
|
|
||||||
|
|
||||||
ICuratedDataService <|.. CuratedJsonDataService : implements
|
|
||||||
ICuratedDataService <|.. MockCuratedDataService : implements
|
|
||||||
|
|
||||||
IAddressService <|.. NominatimAddressService : implements
|
|
||||||
IAddressService <|.. MockAddressService : implements
|
|
||||||
|
|
||||||
IExportService <|.. SqliteExportService : implements
|
|
||||||
SqliteExportService *-- IDateTimeProvider : owns
|
|
||||||
IDateTimeProvider <|.. SystemDateTimeProvider : implements
|
|
||||||
269
docs/pipeline/diagrams/current/class.puml
Normal file
269
docs/pipeline/diagrams/current/class.puml
Normal file
@@ -0,0 +1,269 @@
|
|||||||
|
@startuml
|
||||||
|
skinparam style strictuml
|
||||||
|
skinparam defaultFontName "DM Sans"
|
||||||
|
skinparam defaultFontSize 14
|
||||||
|
skinparam titleFontName "Volkhov"
|
||||||
|
skinparam titleFontSize 20
|
||||||
|
skinparam backgroundColor #FAFCF9
|
||||||
|
skinparam defaultFontColor #28342A
|
||||||
|
skinparam titleFontColor #28342A
|
||||||
|
skinparam ArrowColor #628A5B
|
||||||
|
|
||||||
|
skinparam class {
|
||||||
|
BackgroundColor #FAFCF9
|
||||||
|
HeaderBackgroundColor #EAF0E8
|
||||||
|
BorderColor #547461
|
||||||
|
ArrowColor #628A5B
|
||||||
|
FontColor #28342A
|
||||||
|
}
|
||||||
|
|
||||||
|
skinparam note {
|
||||||
|
BackgroundColor #EAF0E8
|
||||||
|
BorderColor #547461
|
||||||
|
FontColor #28342A
|
||||||
|
}
|
||||||
|
|
||||||
|
title The Biergarten Data Pipeline - Class Diagram
|
||||||
|
|
||||||
|
class BiergartenPipelineOrchestrator {
|
||||||
|
- logger_ : std::shared_ptr<ILogger>
|
||||||
|
- context_service_ : std::unique_ptr<IEnrichmentService>
|
||||||
|
- generator_ : std::unique_ptr<DataGenerator>
|
||||||
|
- exporter_ : std::unique_ptr<IExportService>
|
||||||
|
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
|
||||||
|
- application_options_ : ApplicationOptions
|
||||||
|
- generated_breweries_ : std::vector<BreweryRecord>
|
||||||
|
- generated_users_ : std::vector<UserRecord>
|
||||||
|
+ Run() : bool
|
||||||
|
- QueryCitiesWithCountries() : std::vector<Location>
|
||||||
|
- GenerateBreweries(cities : std::span<const EnrichedCity>) : void
|
||||||
|
- GenerateUsers(cities : std::span<const EnrichedCity>) : void
|
||||||
|
- LogResults() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogLevel <<enumeration>> {
|
||||||
|
Debug
|
||||||
|
Info
|
||||||
|
Warn
|
||||||
|
Error
|
||||||
|
}
|
||||||
|
|
||||||
|
class PipelinePhase <<enumeration>> {
|
||||||
|
Startup
|
||||||
|
UserGeneration
|
||||||
|
BreweryAndBeerGeneration
|
||||||
|
CheckinGeneration
|
||||||
|
RatingGeneration
|
||||||
|
FollowGeneration
|
||||||
|
Teardown
|
||||||
|
}
|
||||||
|
|
||||||
|
struct LogDTO {
|
||||||
|
+ level : LogLevel
|
||||||
|
+ phase : PipelinePhase
|
||||||
|
+ message : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
struct LogEntry {
|
||||||
|
+ timestamp : std::chrono::system_clock::time_point
|
||||||
|
+ origin : std::source_location
|
||||||
|
+ thread_id : std::thread::id
|
||||||
|
+ level : LogLevel
|
||||||
|
+ phase : PipelinePhase
|
||||||
|
+ message : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ILogger <<interface>> {
|
||||||
|
+ Log(payload : LogDTO) : void
|
||||||
|
- {abstract} DoLog(entry : LogEntry) : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogProducer {
|
||||||
|
- channel_ : BoundedChannel<LogEntry>&
|
||||||
|
- DoLog(entry : LogEntry) : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogDispatcher {
|
||||||
|
- channel_ : BoundedChannel<LogEntry>&
|
||||||
|
+ Run() : void
|
||||||
|
- ToSpdlogLevel(level) : spdlog::level::level_enum
|
||||||
|
}
|
||||||
|
|
||||||
|
interface IEnrichmentService <<interface>> {
|
||||||
|
+ GetLocationContext(loc : const Location&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class MockEnrichmentService {
|
||||||
|
+ GetLocationContext(loc : const Location&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class WikipediaEnrichmentService {
|
||||||
|
- client_ : std::unique_ptr<WebClient>
|
||||||
|
- extract_cache_ : std::unordered_map<std::string, std::string>
|
||||||
|
+ GetLocationContext(loc : const Location&) : std::string
|
||||||
|
- FetchExtract(query : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface WebClient <<interface>> {
|
||||||
|
+ Get(url : const std::string&) : std::string
|
||||||
|
+ UrlEncode(value : const std::string&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class HttpWebClient {
|
||||||
|
+ Get(url : const std::string&) : std::string
|
||||||
|
+ UrlEncode(value : const std::string&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface DataGenerator <<interface>> {
|
||||||
|
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
|
||||||
|
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
|
||||||
|
}
|
||||||
|
|
||||||
|
class MockGenerator {
|
||||||
|
+ GenerateBrewery(location : const Location&, region_context : const std::string&) : BreweryResult
|
||||||
|
+ GenerateUser(city : const EnrichedCity&, persona : const UserPersona&, name : const Name&) : UserResult
|
||||||
|
- DeterministicHash(location : const Location&) : size_t
|
||||||
|
- DeterministicHash(location : const Location&, persona : const UserPersona&, name : const Name&) : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
class LlamaGenerator {
|
||||||
|
- model_ : ModelHandle
|
||||||
|
- context_ : ContextHandle
|
||||||
|
- prompt_formatter_ : std::unique_ptr<IPromptFormatter>
|
||||||
|
- prompt_directory_ : std::unique_ptr<IPromptDirectory>
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
+ GenerateBrewery(...) : BreweryResult
|
||||||
|
+ GenerateUser(...) : UserResult
|
||||||
|
- Load(model_path : const std::string&) : void
|
||||||
|
- Infer(...) : std::string
|
||||||
|
- InferFormatted(...) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface IPromptFormatter <<interface>> {
|
||||||
|
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class Gemma4JinjaPromptFormatter {
|
||||||
|
+ Format(system_prompt : std::string_view, user_prompt : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface IPromptDirectory <<interface>> {
|
||||||
|
+ Load(key : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class PromptDirectory {
|
||||||
|
- prompt_dir_ : std::filesystem::path
|
||||||
|
- cache_ : std::unordered_map<std::string, std::string>
|
||||||
|
+ Load(key : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ICuratedDataService <<interface>> {
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
class JsonLoader {
|
||||||
|
- cache_ : cache
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of JsonLoader
|
||||||
|
Each Load* method memoizes its result in
|
||||||
|
cache_ on first call; later calls on the
|
||||||
|
same instance return the cached value
|
||||||
|
without re-parsing.
|
||||||
|
end note
|
||||||
|
|
||||||
|
class MockCuratedDataService {
|
||||||
|
- locations_ : std::vector<Location>
|
||||||
|
- personas_ : std::vector<UserPersona>
|
||||||
|
- forenames_by_country_ : std::unordered_map<std::string, forename_list>
|
||||||
|
- surnames_by_country_ : std::unordered_map<std::string, surname_list>
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of MockCuratedDataService
|
||||||
|
Fixed 4-location / 3-persona dataset
|
||||||
|
(US, DE, FR, BE) used in --mocked runs;
|
||||||
|
filepath arguments are ignored.
|
||||||
|
end note
|
||||||
|
|
||||||
|
interface IExportService <<interface>> {
|
||||||
|
+ Initialize() : void
|
||||||
|
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
|
||||||
|
+ ProcessRecord(user : const UserRecord&) : uint64_t
|
||||||
|
+ Finalize() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class SqliteExportService {
|
||||||
|
- date_time_provider_ : std::unique_ptr<IDateTimeProvider>
|
||||||
|
- run_timestamp_utc_ : std::string
|
||||||
|
- database_path_ : std::filesystem::path
|
||||||
|
- db_handle_ : SqliteDatabaseHandle
|
||||||
|
- insert_location_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_brewery_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_user_stmt_ : SqliteStatementHandle
|
||||||
|
- transaction_open_ : bool
|
||||||
|
- location_cache_ : std::unordered_map<std::string, sqlite3_int64>
|
||||||
|
+ Initialize() : void
|
||||||
|
+ ProcessRecord(brewery : const BreweryRecord&) : uint64_t
|
||||||
|
+ ProcessRecord(user : const UserRecord&) : uint64_t
|
||||||
|
+ Finalize() : void
|
||||||
|
- InitializeSchema() : void
|
||||||
|
- ResolveLocationId(location : const Location&) : sqlite3_int64
|
||||||
|
}
|
||||||
|
|
||||||
|
interface IDateTimeProvider <<interface>> {
|
||||||
|
+ GetUtcTimestamp() : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class SystemDateTimeProvider {
|
||||||
|
+ GetUtcTimestamp() : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
' Structural Relationships / Dependency Injection
|
||||||
|
BiergartenPipelineOrchestrator *-- ILogger : owns
|
||||||
|
BiergartenPipelineOrchestrator *-- IEnrichmentService : owns
|
||||||
|
BiergartenPipelineOrchestrator *-- DataGenerator : owns
|
||||||
|
BiergartenPipelineOrchestrator *-- IExportService : owns
|
||||||
|
BiergartenPipelineOrchestrator *-- ICuratedDataService : owns
|
||||||
|
|
||||||
|
LogEntry *-- LogLevel
|
||||||
|
LogEntry *-- PipelinePhase
|
||||||
|
LogDTO *-- LogLevel
|
||||||
|
LogDTO *-- PipelinePhase
|
||||||
|
ILogger <|.. LogProducer : implements
|
||||||
|
LogProducer ..> LogEntry : emits
|
||||||
|
LogDispatcher ..> LogEntry : consumes
|
||||||
|
|
||||||
|
IEnrichmentService <|.. WikipediaEnrichmentService : implements
|
||||||
|
IEnrichmentService <|.. MockEnrichmentService : implements
|
||||||
|
WikipediaEnrichmentService *-- WebClient : owns
|
||||||
|
|
||||||
|
WebClient <|.. HttpWebClient : implements
|
||||||
|
|
||||||
|
DataGenerator <|.. MockGenerator : implements
|
||||||
|
DataGenerator <|.. LlamaGenerator : implements
|
||||||
|
|
||||||
|
LlamaGenerator *-- IPromptFormatter : uses
|
||||||
|
LlamaGenerator *-- IPromptDirectory : uses
|
||||||
|
|
||||||
|
IPromptFormatter <|.. Gemma4JinjaPromptFormatter : implements
|
||||||
|
IPromptDirectory <|.. PromptDirectory : implements
|
||||||
|
|
||||||
|
ICuratedDataService <|.. JsonLoader : implements
|
||||||
|
ICuratedDataService <|.. MockCuratedDataService : implements
|
||||||
|
|
||||||
|
IExportService <|.. SqliteExportService : implements
|
||||||
|
SqliteExportService *-- IDateTimeProvider : owns
|
||||||
|
IDateTimeProvider <|.. SystemDateTimeProvider : implements
|
||||||
|
|
||||||
|
@enduml
|
||||||
1
docs/pipeline/diagrams/current/output/activity.svg
Normal file
1
docs/pipeline/diagrams/current/output/activity.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 86 KiB |
1
docs/pipeline/diagrams/current/output/class.svg
Normal file
1
docs/pipeline/diagrams/current/output/class.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 87 KiB |
@@ -1,16 +0,0 @@
|
|||||||
# The Biergarten Data Pipeline — Activity Diagram (Planned)
|
|
||||||
|
|
||||||
Each generation phase runs a producer/consumer chain concurrently: an
|
|
||||||
**Orchestrator** producer feeds a bounded channel, an **LLM Worker** consumes it
|
|
||||||
and generates records (closing its output channel when the input is drained and
|
|
||||||
closed), and a **SQLite Worker** consumes that output channel and commits in
|
|
||||||
batches. The brewery phase inserts a fourth stage, a **Geocode Worker**, between
|
|
||||||
the LLM Worker and the SQLite Worker: reverse-geocoding is network-bound and
|
|
||||||
rate-limited independently of LLM inference (Nominatim's 1-request-per-second
|
|
||||||
policy), so keeping it off the LLM Worker's thread means neither stage stalls
|
|
||||||
waiting on the other's bottleneck. The Checkin and Follow phases run in parallel
|
|
||||||
with each other once the user and brewery pools are populated.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "pipeline/diagrams/planned/activity.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,213 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
flowchart LR
|
|
||||||
Start([start]) --> ParseArgs["Main: ParseArguments(argc, argv)"]
|
|
||||||
ParseArgs --> InvalidArgs{Invalid args?}
|
|
||||||
InvalidArgs -- yes --> LogErr["spdlog::error"] --> Stop1([stop])
|
|
||||||
InvalidArgs -- no --> InitOpenSSL["Main: Init OpenSSL global state and LlamaBackendState"]
|
|
||||||
InitOpenSSL --> BuildDI["Main: Build DI injector"]
|
|
||||||
BuildDI --> InitSqlite["Main: Initialize SqliteExportService<br/>opens SQLite connection; transactions now managed per-phase via batching"]
|
|
||||||
InitSqlite --> LogSetup["Main: create BoundedChannel log_ch, spawn Log Worker thread<br/>drains log_ch for the pipeline lifetime; all workers emit LogEntry via PipelineLogger, never spdlog directly"]
|
|
||||||
LogSetup --> OrchRun["BiergartenPipelineOrchestrator::Run()"]
|
|
||||||
|
|
||||||
OrchRun --> PL1["JsonLoader.LoadBeerStyles → EnrichmentService.PreWarmBeerStyleCache"]
|
|
||||||
OrchRun --> PL2["JsonLoader.LoadLocations → EnrichmentService.PreWarmLocationCache"]
|
|
||||||
OrchRun --> PL3["JsonLoader.LoadForenamesByCountry"]
|
|
||||||
OrchRun --> PL4["JsonLoader.LoadSurnamesByCountry"]
|
|
||||||
OrchRun --> PL5["JsonLoader.LoadPersonas"]
|
|
||||||
PL1 & PL2 & PL3 & PL4 & PL5 --> UserPhase["Orchestrator: RunUserPhase(sampled_locations)<br/>each Load* call memoized after first parse; MockCuratedDataService (--mocked) returns a fixed in-memory dataset instead, skipping these JSON files"]
|
|
||||||
|
|
||||||
UserPhase --> UCreate["Create BoundedChannels (loc_ch, exp_ch)"]
|
|
||||||
|
|
||||||
subgraph USERGEN["Phase 0 — User Generation (concurrent)"]
|
|
||||||
direction LR
|
|
||||||
subgraph U_PROD["Orchestrator (producer)"]
|
|
||||||
UP1["Loop: send Location → loc_ch"]
|
|
||||||
UP1 --> UP2["Close loc_ch"]
|
|
||||||
end
|
|
||||||
subgraph U_LLM["LLM Worker"]
|
|
||||||
UL1{"loc_ch has items?"}
|
|
||||||
UL1 -- yes --> UL2["Receive Location"]
|
|
||||||
UL2 --> UL3["GetLocationContextFromCache(location) — guaranteed cache hit from startup"]
|
|
||||||
UL3 --> UL4["IPersonaSelectionStrategy.SelectPersona(personas_palette_) — guaranteed cache hit; returns Persona{style_affinities, abv_range, ibu_preference, checkin_weight}"]
|
|
||||||
UL4 --> UL5["SampleName(forenames_by_country, surnames_by_country, iso3166_1, rng)<br/>deterministic, no LLM; free function; skips the city if either map has no entry for iso3166_1"]
|
|
||||||
UL5 --> UL6["GenerateUser(enriched_city, persona, sampled_name) via DataGenerator<br/>LLM receives EnrichedCity context + persona description + sampled name"]
|
|
||||||
UL6 --> UL7["PipelineLogger.Log(Info, UserGeneration, city, user_id, llm)"]
|
|
||||||
UL7 --> UL8["Send GeneratedUser → exp_ch"]
|
|
||||||
UL8 --> UL1
|
|
||||||
UL1 -- no --> UL9["Close exp_ch"]
|
|
||||||
end
|
|
||||||
subgraph U_SQL["SQLite Worker"]
|
|
||||||
US1["BEGIN TRANSACTION"] --> US2{"exp_ch has items?"}
|
|
||||||
US2 -- yes --> US3["Receive GeneratedUser"]
|
|
||||||
US3 --> US4["ProcessUser(user)"]
|
|
||||||
US4 --> US5["PipelineLogger.Log(Info, UserGeneration, city, user_id, sqlite)"]
|
|
||||||
US5 --> US6["Append → user_pool_"]
|
|
||||||
US6 --> US7{"Batch size reached?"}
|
|
||||||
US7 -- yes --> US8["COMMIT and BEGIN"] --> US2
|
|
||||||
US7 -- no --> US2
|
|
||||||
US2 -- no --> US9["COMMIT (Final)"]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
UCreate --> UP1
|
|
||||||
UP2 -.->|loc_ch| UL1
|
|
||||||
UL9 -.->|exp_ch| US2
|
|
||||||
UL9 & US9 --> UserJoin["Orchestrator: Join LLM Worker, SQLite Worker"]
|
|
||||||
|
|
||||||
UserJoin --> BreweryPhase["Orchestrator: RunBreweryPhase(sampled_locations)"]
|
|
||||||
BreweryPhase --> BCreate["Create BoundedChannels (loc_ch, geo_ch, exp_ch)"]
|
|
||||||
|
|
||||||
subgraph BREWGEN["Phase 1a — Brewery Generation (concurrent)"]
|
|
||||||
direction LR
|
|
||||||
subgraph B_PROD["Orchestrator (producer)"]
|
|
||||||
BP1["Loop: sample User from user_pool_, pair with Location"]
|
|
||||||
BP1 --> BP2["Send BreweryTask(Location, User) → loc_ch"]
|
|
||||||
BP2 --> BP3["Close loc_ch"]
|
|
||||||
end
|
|
||||||
subgraph B_LLM["LLM Worker"]
|
|
||||||
BL1{"loc_ch has items?"}
|
|
||||||
BL1 -- yes --> BL2["Receive BreweryTask(Location, User)"]
|
|
||||||
BL2 --> BL3["GetLocationContextFromCache(task.location) — guaranteed cache hit"]
|
|
||||||
BL3 --> BL4["GenerateBrewery(enriched_city, context, task.user) via DataGenerator<br/>KV cache stays warm; brewery linked to sampled owner_user_id"]
|
|
||||||
BL4 --> BL5["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, llm)"]
|
|
||||||
BL5 --> BL6["Send GeneratedBrewery → geo_ch"]
|
|
||||||
BL6 --> BL1
|
|
||||||
BL1 -- no --> BL7["Close geo_ch"]
|
|
||||||
end
|
|
||||||
subgraph B_GEO["Geocode Worker"]
|
|
||||||
BG1{"geo_ch has items?"}
|
|
||||||
BG1 -- yes --> BG2["Receive GeneratedBrewery"]
|
|
||||||
BG2 --> BG3["AddressService.ReverseGeocode(brewery.longitude, brewery.latitude)<br/>rate-limited independently of LLM throughput — NominatimAddressService sleeps 1s per request; MockAddressService (--mocked) returns a fixed placeholder, no network call"]
|
|
||||||
BG3 --> BG4{"Address resolved?"}
|
|
||||||
BG4 -- yes --> BG5["Attach resolved AddressLookupResult to brewery"]
|
|
||||||
BG4 -- no --> BG6["Log warning: could not reverse geocode; attach placeholder address_line1, empty postal_code — brewery is never dropped"]
|
|
||||||
BG5 --> BG7["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, geocode)"]
|
|
||||||
BG6 --> BG7
|
|
||||||
BG7 --> BG8["Send GeneratedBrewery → exp_ch"]
|
|
||||||
BG8 --> BG1
|
|
||||||
BG1 -- no --> BG9["Close exp_ch"]
|
|
||||||
end
|
|
||||||
subgraph B_SQL["SQLite Worker"]
|
|
||||||
BS1["BEGIN TRANSACTION"] --> BS2{"exp_ch has items?"}
|
|
||||||
BS2 -- yes --> BS3["Receive GeneratedBrewery"]
|
|
||||||
BS3 --> BS4["ProcessBrewery(brewery)"]
|
|
||||||
BS4 --> BS5["PipelineLogger.Log(Info, BreweryGeneration, city, brewery_id, sqlite)"]
|
|
||||||
BS5 --> BS6["Append → brewery_pool_"]
|
|
||||||
BS6 --> BS7{"Batch size reached?"}
|
|
||||||
BS7 -- yes --> BS8["COMMIT and BEGIN"] --> BS2
|
|
||||||
BS7 -- no --> BS2
|
|
||||||
BS2 -- no --> BS9["COMMIT (Final)"]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
BCreate --> BP1
|
|
||||||
BP3 -.->|loc_ch| BL1
|
|
||||||
BL7 -.->|geo_ch| BG1
|
|
||||||
BG9 -.->|exp_ch| BS2
|
|
||||||
BL7 & BG9 & BS9 --> BrewJoin["Orchestrator: Join LLM Worker, Geocode Worker, SQLite Worker<br/>brewery_pool_ is now fully populated"]
|
|
||||||
|
|
||||||
BrewJoin --> BeerPhase["Orchestrator: RunBeerPhase()"]
|
|
||||||
BeerPhase --> BeCreate["Create BoundedChannels (brew_ch, exp_ch)"]
|
|
||||||
|
|
||||||
subgraph BEERGEN["Phase 1b — Beer Generation (concurrent)"]
|
|
||||||
direction LR
|
|
||||||
subgraph BE_PROD["Orchestrator (producer)"]
|
|
||||||
BeP1["Loop: send Breweries → brew_ch"]
|
|
||||||
BeP1 --> BeP2["Close brew_ch"]
|
|
||||||
end
|
|
||||||
subgraph BE_LLM["LLM Worker"]
|
|
||||||
BeL1{"brew_ch has items?"}
|
|
||||||
BeL1 -- yes --> BeL2["Receive GeneratedBrewery"]
|
|
||||||
BeL2 --> BeL3["IBeerSelectionStrategy.SelectStyles(brewery, beer_style_palette_)"]
|
|
||||||
BeL3 --> BeL4{"For each selected BeerStyle"}
|
|
||||||
BeL4 -- remaining --> BeL5["GetStyleContextFromCache(style) — guaranteed cache hit; KV cache stays warm across all beer generations in this phase"]
|
|
||||||
BeL5 --> BeL6["GenerateBeer(brewery, style_context) via DataGenerator"]
|
|
||||||
BeL6 --> BeL7["Attach GeneratedBeer to bundle"] --> BeL4
|
|
||||||
BeL4 -- done --> BeL8["PipelineLogger.Log(Info, BeerGeneration, city, brewery_id, llm)"]
|
|
||||||
BeL8 --> BeL9["Send BeersBundle → exp_ch"]
|
|
||||||
BeL9 --> BeL1
|
|
||||||
BeL1 -- no --> BeL10["Close exp_ch"]
|
|
||||||
end
|
|
||||||
subgraph BE_SQL["SQLite Worker"]
|
|
||||||
BeS1["BEGIN TRANSACTION"] --> BeS2{"exp_ch has items?"}
|
|
||||||
BeS2 -- yes --> BeS3["Receive BeersBundle"]
|
|
||||||
BeS3 --> BeS4{"For each beer in bundle"}
|
|
||||||
BeS4 -- remaining --> BeS5["Set beer.brewery_id from bundle"]
|
|
||||||
BeS5 --> BeS6["ProcessBeer(beer)"]
|
|
||||||
BeS6 --> BeS7["Append → beer_pool_"] --> BeS4
|
|
||||||
BeS4 -- done --> BeS8["PipelineLogger.Log(Info, BeerGeneration, city, brewery_id, sqlite)"]
|
|
||||||
BeS8 --> BeS9{"Batch size reached?"}
|
|
||||||
BeS9 -- yes --> BeS10["COMMIT and BEGIN"] --> BeS2
|
|
||||||
BeS9 -- no --> BeS2
|
|
||||||
BeS2 -- no --> BeS11["COMMIT (Final)"]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
BeCreate --> BeP1
|
|
||||||
BeP2 -.->|brew_ch| BeL1
|
|
||||||
BeL10 -.->|exp_ch| BeS2
|
|
||||||
BeL10 & BeS11 --> BeerJoin["Orchestrator: Join LLM Worker, SQLite Worker<br/>brewery_pool_ and beer_pool_ are now completely populated — Checkin and Follow phases may run in parallel"]
|
|
||||||
|
|
||||||
BeerJoin --> CheckinPhase["Orchestrator: RunCheckinPhase()"]
|
|
||||||
BeerJoin --> FollowPhase["Orchestrator: RunFollowPhase()"]
|
|
||||||
|
|
||||||
subgraph CHECKIN["Checkin Generation"]
|
|
||||||
CK1["ICheckinDistributionStrategy.AssignActivityWeights(user_pool_)<br/>weights seeded from persona.checkin_weight; J-curve profile emerges from persona distribution"]
|
|
||||||
CK1 --> CK2["BEGIN TRANSACTION"]
|
|
||||||
CK2 --> CK3{"For each GeneratedUser in user_pool_"}
|
|
||||||
CK3 -- remaining --> CK4["CheckinsForUser(user, brewery_pool_.size())"]
|
|
||||||
CK4 --> CK5{"For each checkin index"}
|
|
||||||
CK5 -- remaining --> CK6["TimestampFor(user, index)"]
|
|
||||||
CK6 --> CK7["Select brewery from brewery_pool_"]
|
|
||||||
CK7 --> CK8["GenerateCheckin(user, brewery, timestamp) via DataGenerator"]
|
|
||||||
CK8 --> CK9["ProcessCheckin(checkin)"]
|
|
||||||
CK9 --> CK10["PipelineLogger.Log(Info, CheckinGeneration, nullopt, checkin_id, sqlite)"]
|
|
||||||
CK10 --> CK11["Append → checkin_pool_"]
|
|
||||||
CK11 --> CK12{"Batch size reached?"}
|
|
||||||
CK12 -- yes --> CK13["COMMIT and BEGIN"] --> CK5
|
|
||||||
CK12 -- no --> CK5
|
|
||||||
CK5 -- done --> CK3
|
|
||||||
CK3 -- done --> CK14["COMMIT (Final)"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph FOLLOW["Follow Generation"]
|
|
||||||
FL1["IFollowGenerationStrategy.AssignFollowWeights(user_pool_)<br/>RandomFollowStrategy: uniform weights; ActivityWeightedFollowStrategy: derived from user.activity_weight so high-activity users attract more followers"]
|
|
||||||
FL1 --> FL2["BEGIN TRANSACTION"]
|
|
||||||
FL2 --> FL3["IFollowGenerationStrategy.GenerateFollows(user_pool_)<br/>self-follow constraint (follower_id != followed_id) enforced here and at the DB schema level"]
|
|
||||||
FL3 --> FL4{"For each GeneratedFollow"}
|
|
||||||
FL4 -- remaining --> FL5["ProcessFollow(follow)"]
|
|
||||||
FL5 --> FL6["PipelineLogger.Log(Info, FollowGeneration, nullopt, follower_id, sqlite)"]
|
|
||||||
FL6 --> FL7["Append → follow_pool_"]
|
|
||||||
FL7 --> FL8{"Batch size reached?"}
|
|
||||||
FL8 -- yes --> FL9["COMMIT and BEGIN"] --> FL4
|
|
||||||
FL8 -- no --> FL4
|
|
||||||
FL4 -- done --> FL10["COMMIT (Final)"]
|
|
||||||
end
|
|
||||||
|
|
||||||
CheckinPhase --> CK1
|
|
||||||
FollowPhase --> FL1
|
|
||||||
CK14 & FL10 --> PhaseJoin["Orchestrator: Join CheckinPhase, FollowPhase<br/>checkin_pool_ and follow_pool_ are now fully populated"]
|
|
||||||
|
|
||||||
PhaseJoin --> RatingPhase["Orchestrator: RunRatingPhase()<br/>beer selection biased by user.persona.style_affinities and abv_range; rating skew modulated per persona"]
|
|
||||||
RatingPhase --> RT1["BEGIN TRANSACTION"]
|
|
||||||
RT1 --> RT2{"For each GeneratedCheckin in checkin_pool_"}
|
|
||||||
RT2 -- remaining --> RT3["Match brewery_id, select beer from beer_pool_ (same brewery_id, biased by persona affinities)"]
|
|
||||||
RT3 --> RT4{"Beer exists for brewery?"}
|
|
||||||
RT4 -- yes --> RT5["GenerateRating(user, beer, checkin_id) via DataGenerator"]
|
|
||||||
RT5 --> RT6["ProcessRating(rating)"]
|
|
||||||
RT6 --> RT7["PipelineLogger.Log(Info, RatingGeneration, nullopt, rating_id, sqlite)"]
|
|
||||||
RT7 --> RT8{"Batch size reached?"}
|
|
||||||
RT8 -- yes --> RT9["COMMIT and BEGIN"] --> RT2
|
|
||||||
RT8 -- no --> RT2
|
|
||||||
RT4 -- no --> RT10["Log warning; skip — brewery has no beers"] --> RT2
|
|
||||||
RT2 -- done --> RT11["COMMIT (Final)"]
|
|
||||||
|
|
||||||
RT11 --> Teardown["Orchestrator: Finalize SqliteExportService — safely closes the DB connection"]
|
|
||||||
Teardown --> CloseLogCh["Orchestrator: close log_ch"]
|
|
||||||
CloseLogCh --> MainDone["Main: spdlog::info 'Pipeline complete in X ms'"]
|
|
||||||
MainDone --> JoinLog["Main: Join Log Worker — drain guarantees no LogEntry is dropped at shutdown"]
|
|
||||||
JoinLog --> Stop2([stop])
|
|
||||||
372
docs/pipeline/diagrams/planned/activity.puml
Normal file
372
docs/pipeline/diagrams/planned/activity.puml
Normal file
@@ -0,0 +1,372 @@
|
|||||||
|
@startuml biergarten_activity
|
||||||
|
!include ../biergarten-weizen-theme.puml
|
||||||
|
skinparam defaultFontSize 13
|
||||||
|
skinparam titleFontSize 20
|
||||||
|
|
||||||
|
title The Biergarten Data Pipeline — Activity Diagram
|
||||||
|
|
||||||
|
|Main|
|
||||||
|
start
|
||||||
|
:ParseArguments(argc, argv);
|
||||||
|
if (Invalid args?) then (yes)
|
||||||
|
:spdlog::error;
|
||||||
|
stop
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
:Init OpenSSL global state & LlamaBackendState;
|
||||||
|
:Build DI injector;
|
||||||
|
|
||||||
|
:Initialize SqliteExportService;
|
||||||
|
note right
|
||||||
|
Opens SQLite connection.
|
||||||
|
(Transactions are now managed
|
||||||
|
per-phase via batching).
|
||||||
|
end note
|
||||||
|
|
||||||
|
:Create BoundedChannel<LogEntry> log_ch;
|
||||||
|
:Spawn Log Worker thread;
|
||||||
|
note right
|
||||||
|
Log worker drains log_ch for the
|
||||||
|
entire pipeline lifetime.
|
||||||
|
All workers emit LogEntry structs
|
||||||
|
via PipelineLogger -- never spdlog directly.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:BiergartenPipelineOrchestrator::Run();
|
||||||
|
|BiergartenPipelineOrchestrator::Run()|
|
||||||
|
|
||||||
|
fork
|
||||||
|
:JsonLoader::LoadBeerStyles("beer-styles.json");
|
||||||
|
:EnrichmentService::PreWarmBeerStyleCache(beer_styles);
|
||||||
|
fork again
|
||||||
|
:JsonLoader::LoadLocations("locations.json");
|
||||||
|
:EnrichmentService::PreWarmLocationCache(sampled_locations);
|
||||||
|
end fork
|
||||||
|
fork
|
||||||
|
:JsonLoader::LoadForenamesByCountry(\n "forenames-by-country.json");
|
||||||
|
fork again
|
||||||
|
:JsonLoader::LoadSurnamesByCountry(\n "surnames-by-country.json");
|
||||||
|
fork again
|
||||||
|
:JsonLoader::LoadPersonas("personas.json");
|
||||||
|
end fork
|
||||||
|
note right
|
||||||
|
Each call is memoized after its first parse
|
||||||
|
(implemented today). MockCuratedDataService
|
||||||
|
returns a fixed in-memory dataset instead
|
||||||
|
under --mocked, skipping these JSON files.
|
||||||
|
end note
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' PHASE 0 — USER GENERATION
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
|Orchestrator|
|
||||||
|
:RunUserPhase(sampled_locations);
|
||||||
|
:Create BoundedChannels\n(loc_ch, exp_ch);
|
||||||
|
|
||||||
|
fork
|
||||||
|
|Orchestrator|
|
||||||
|
:Loop: Send Locations -> loc_ch;
|
||||||
|
:Close loc_ch;
|
||||||
|
note right
|
||||||
|
Producer closes loc_ch.
|
||||||
|
LLM Worker while loop
|
||||||
|
terminates on empty + closed.
|
||||||
|
end note
|
||||||
|
fork again
|
||||||
|
|LLM Worker|
|
||||||
|
while (loc_ch has items?) is (yes)
|
||||||
|
:Receive Location;
|
||||||
|
|
||||||
|
:GetLocationContextFromCache(location);
|
||||||
|
note right
|
||||||
|
Guaranteed cache hit from startup.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:IPersonaSelectionStrategy::SelectPersona(\n personas_palette_);
|
||||||
|
note right
|
||||||
|
Guaranteed cache hit from startup.
|
||||||
|
Returns a Persona struct carrying
|
||||||
|
style_affinities, abv_range,
|
||||||
|
ibu_preference, checkin_weight.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:SampleName(forenames_by_country,\n surnames_by_country, location.iso3166_1, rng);
|
||||||
|
note right
|
||||||
|
Deterministic lookup -- no LLM involved.
|
||||||
|
Free function (implemented today in
|
||||||
|
generate_users.cc), not a class method.
|
||||||
|
Name selected from pre-keyed maps
|
||||||
|
and passed into the generation prompt.
|
||||||
|
Skips the city if either map has no
|
||||||
|
entry for iso3166_1.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:GenerateUser(enriched_city, persona, sampled_name)\nvia DataGenerator;
|
||||||
|
note right
|
||||||
|
LLM receives: EnrichedCity context + persona
|
||||||
|
description + sampled name. Generates
|
||||||
|
bio and preference signals grounded
|
||||||
|
in locale and persona.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "llm");
|
||||||
|
:Send GeneratedUser -> exp_ch;
|
||||||
|
endwhile (no)
|
||||||
|
:Close exp_ch;
|
||||||
|
note right
|
||||||
|
Producer closes exp_ch.
|
||||||
|
SQLite Worker while loop
|
||||||
|
terminates on empty + closed.
|
||||||
|
end note
|
||||||
|
fork again
|
||||||
|
|SQLite Worker|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
while (exp_ch has items?) is (yes)
|
||||||
|
:Receive GeneratedUser;
|
||||||
|
:ProcessUser(user);
|
||||||
|
:PipelineLogger::Log(Info, UserGeneration,\n city, user_id, "sqlite");
|
||||||
|
:Append -> user_pool_;
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
endwhile (no)
|
||||||
|
:COMMIT (Final);
|
||||||
|
end fork
|
||||||
|
|
||||||
|
|Orchestrator|
|
||||||
|
:Join LLM Worker, SQLite Worker;
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' PHASE 1a — BREWERY GENERATION
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
:RunBreweryPhase(sampled_locations);
|
||||||
|
:Create BoundedChannels\n(loc_ch, exp_ch);
|
||||||
|
|
||||||
|
fork
|
||||||
|
|Orchestrator|
|
||||||
|
:Loop: Sample User from user_pool_
|
||||||
|
and pair with Location;
|
||||||
|
:Send BreweryTask(Location, User) -> loc_ch;
|
||||||
|
:Close loc_ch;
|
||||||
|
fork again
|
||||||
|
|LLM Worker|
|
||||||
|
while (loc_ch has items?) is (yes)
|
||||||
|
:Receive BreweryTask(Location, User);
|
||||||
|
|
||||||
|
:GetLocationContextFromCache(task.location);
|
||||||
|
note right
|
||||||
|
Guaranteed cache hit from startup.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:GenerateBrewery(enriched_city, context, task.user)\nvia DataGenerator;
|
||||||
|
note right
|
||||||
|
KV cache stays warm.
|
||||||
|
Brewery is linked to the sampled owner_user_id.
|
||||||
|
end note
|
||||||
|
:PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "llm");
|
||||||
|
:Send GeneratedBrewery -> exp_ch;
|
||||||
|
endwhile (no)
|
||||||
|
:Close exp_ch;
|
||||||
|
fork again
|
||||||
|
|SQLite Worker|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
while (exp_ch has items?) is (yes)
|
||||||
|
:Receive GeneratedBrewery;
|
||||||
|
:ProcessBrewery(brewery);
|
||||||
|
:PipelineLogger::Log(Info,\n BreweryGeneration,\n city, brewery_id, "sqlite");
|
||||||
|
:Append -> brewery_pool_;
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
endwhile (no)
|
||||||
|
:COMMIT (Final);
|
||||||
|
end fork
|
||||||
|
|
||||||
|
|Orchestrator|
|
||||||
|
:Join LLM Worker, SQLite Worker;
|
||||||
|
note right
|
||||||
|
brewery_pool_ is now fully populated.
|
||||||
|
Phase 1b may begin.
|
||||||
|
end note
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' PHASE 1b — BEER GENERATION
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
:RunBeerPhase();
|
||||||
|
:Create BoundedChannels\n(brew_ch, exp_ch);
|
||||||
|
|
||||||
|
fork
|
||||||
|
|Orchestrator|
|
||||||
|
:Loop: Send Breweries -> brew_ch;
|
||||||
|
:Close brew_ch;
|
||||||
|
fork again
|
||||||
|
|LLM Worker|
|
||||||
|
while (brew_ch has items?) is (yes)
|
||||||
|
:Receive GeneratedBrewery;
|
||||||
|
:IBeerSelectionStrategy::SelectStyles(\n brewery, beer_style_palette_);
|
||||||
|
|
||||||
|
while (For each selected BeerStyle?) is (remaining)
|
||||||
|
:GetStyleContextFromCache(style);
|
||||||
|
note right
|
||||||
|
Guaranteed cache hit from startup.
|
||||||
|
KV cache stays warm across all
|
||||||
|
beer generations -- system prompt
|
||||||
|
does not change within this phase.
|
||||||
|
end note
|
||||||
|
:GenerateBeer(brewery, style_context)\nvia DataGenerator;
|
||||||
|
:Attach GeneratedBeer to bundle;
|
||||||
|
endwhile (done)
|
||||||
|
|
||||||
|
:PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "llm");
|
||||||
|
:Send BeersBundle -> exp_ch;
|
||||||
|
endwhile (no)
|
||||||
|
:Close exp_ch;
|
||||||
|
fork again
|
||||||
|
|SQLite Worker|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
while (exp_ch has items?) is (yes)
|
||||||
|
:Receive BeersBundle;
|
||||||
|
while (For each beer in bundle?) is (remaining)
|
||||||
|
:Set beer.brewery_id from bundle;
|
||||||
|
:ProcessBeer(beer);
|
||||||
|
:Append -> beer_pool_;
|
||||||
|
endwhile (done)
|
||||||
|
:PipelineLogger::Log(Info,\n BeerGeneration,\n city, brewery_id, "sqlite");
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
endwhile (no)
|
||||||
|
:COMMIT (Final);
|
||||||
|
end fork
|
||||||
|
|
||||||
|
|Orchestrator|
|
||||||
|
:Join LLM Worker, SQLite Worker;
|
||||||
|
note right
|
||||||
|
Both brewery_pool_ and beer_pool_
|
||||||
|
are now completely populated.
|
||||||
|
Checkin and Follow phases may
|
||||||
|
now run in parallel.
|
||||||
|
end note
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' PHASE 2 — CHECKIN + FOLLOW GENERATION
|
||||||
|
' (parallel — both depend only on user_pool_
|
||||||
|
' and brewery_pool_ being fully populated)
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
fork
|
||||||
|
|Orchestrator|
|
||||||
|
:RunCheckinPhase();
|
||||||
|
:ICheckinDistributionStrategy::\nAssignActivityWeights(user_pool_);
|
||||||
|
note right
|
||||||
|
Weights seeded from each user's
|
||||||
|
persona.checkin_weight. J-curve profile
|
||||||
|
emerges from persona distribution.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
while (For each GeneratedUser in user_pool_?) is (remaining)
|
||||||
|
:CheckinsForUser(user, brewery_pool_.size());
|
||||||
|
while (For each checkin index?) is (remaining)
|
||||||
|
:TimestampFor(user, index);
|
||||||
|
:Select brewery from brewery_pool_;
|
||||||
|
:GenerateCheckin(user, brewery, timestamp)\nvia DataGenerator;
|
||||||
|
:ProcessCheckin(checkin);
|
||||||
|
:PipelineLogger::Log(Info, CheckinGeneration,\n nullopt, checkin_id, "sqlite");
|
||||||
|
:Append -> checkin_pool_;
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
endwhile (done)
|
||||||
|
endwhile (done)
|
||||||
|
:COMMIT (Final);
|
||||||
|
|
||||||
|
fork again
|
||||||
|
|Orchestrator|
|
||||||
|
:RunFollowPhase();
|
||||||
|
:IFollowGenerationStrategy::\nAssignFollowWeights(user_pool_);
|
||||||
|
note right
|
||||||
|
For RandomFollowStrategy, weights
|
||||||
|
are uniform. For ActivityWeightedFollowStrategy,
|
||||||
|
weights derived from user.activity_weight
|
||||||
|
so high-activity users attract more followers.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
:IFollowGenerationStrategy::\nGenerateFollows(user_pool_);
|
||||||
|
note right
|
||||||
|
Self-follow constraint (follower_id != followed_id)
|
||||||
|
enforced here and at the DB schema level.
|
||||||
|
end note
|
||||||
|
while (For each GeneratedFollow?) is (remaining)
|
||||||
|
:ProcessFollow(follow);
|
||||||
|
:PipelineLogger::Log(Info, FollowGeneration,\n nullopt, follower_id, "sqlite");
|
||||||
|
:Append -> follow_pool_;
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
endwhile (done)
|
||||||
|
:COMMIT (Final);
|
||||||
|
|
||||||
|
end fork
|
||||||
|
|
||||||
|
|Orchestrator|
|
||||||
|
:Join CheckinPhase, FollowPhase;
|
||||||
|
note right
|
||||||
|
checkin_pool_ and follow_pool_
|
||||||
|
are now fully populated.
|
||||||
|
Rating phase may begin.
|
||||||
|
end note
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' PHASE 3 — RATING GENERATION
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
:RunRatingPhase();
|
||||||
|
note right
|
||||||
|
Beer selection biased by
|
||||||
|
user.persona.style_affinities and abv_range.
|
||||||
|
Rating skew modulated per persona.
|
||||||
|
end note
|
||||||
|
|
||||||
|
:BEGIN TRANSACTION;
|
||||||
|
while (For each GeneratedCheckin in checkin_pool_?) is (remaining)
|
||||||
|
:Match brewery_id, select beer from beer_pool_\n(same brewery_id, biased by persona affinities);
|
||||||
|
if (Beer exists for brewery?) then (yes)
|
||||||
|
:GenerateRating(user, beer, checkin_id)\nvia DataGenerator;
|
||||||
|
:ProcessRating(rating);
|
||||||
|
:PipelineLogger::Log(Info, RatingGeneration,\n nullopt, rating_id, "sqlite");
|
||||||
|
if (Batch size reached?) then (yes)
|
||||||
|
:COMMIT & BEGIN;
|
||||||
|
else (no)
|
||||||
|
endif
|
||||||
|
else (no)
|
||||||
|
:PipelineLogger::Log(Warn, RatingGeneration,\n nullopt, brewery_id, "sqlite");
|
||||||
|
:Skip -- brewery has no beers;
|
||||||
|
endif
|
||||||
|
endwhile (done)
|
||||||
|
:COMMIT (Final);
|
||||||
|
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
' TEARDOWN
|
||||||
|
' ═══════════════════════════════════════════
|
||||||
|
|Orchestrator|
|
||||||
|
:Finalize SqliteExportService;
|
||||||
|
note right
|
||||||
|
Safely closes the DB connection.
|
||||||
|
end note
|
||||||
|
:Close log_ch;
|
||||||
|
|
||||||
|
|Main|
|
||||||
|
:spdlog::info "Pipeline complete in X ms";
|
||||||
|
:Join Log Worker;
|
||||||
|
note right
|
||||||
|
Drain guarantees no LogEntry is
|
||||||
|
dropped at shutdown.
|
||||||
|
end note
|
||||||
|
stop
|
||||||
|
|
||||||
|
@enduml
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
# Biergarten Data Pipeline — Class Diagram (Planned)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "pipeline/diagrams/planned/class.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- `Logger`, `PipelineLogger`, and `LogWorker` are referenced only in
|
|
||||||
relationships in the source PlantUML (not given their own class blocks there
|
|
||||||
either) — they appear here as implicit nodes for the same reason.
|
|
||||||
- Package grouping from the PlantUML source (`Domain: Models`,
|
|
||||||
`Domain: Application Configuration`, `Domain: Policy`,
|
|
||||||
`Infrastructure: Logging`, `Infrastructure: Pipeline Channel`,
|
|
||||||
`Infrastructure: Data Preloading`, `Infrastructure: Enrichment`,
|
|
||||||
`Infrastructure: Prompting`, `Infrastructure: Data Generation`,
|
|
||||||
`Infrastructure: Data Export`) is preserved as Mermaid `namespace` blocks.
|
|
||||||
@@ -1,647 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
classDiagram
|
|
||||||
namespace DomainModels {
|
|
||||||
class Location {
|
|
||||||
+string city
|
|
||||||
+string state_province
|
|
||||||
+string iso3166_2
|
|
||||||
+string country
|
|
||||||
+string iso3166_1
|
|
||||||
+vector~string~ local_languages
|
|
||||||
+double latitude
|
|
||||||
+double longitude
|
|
||||||
}
|
|
||||||
|
|
||||||
class LocationContext {
|
|
||||||
+string text
|
|
||||||
+Completeness completeness
|
|
||||||
+size_t char_count
|
|
||||||
}
|
|
||||||
|
|
||||||
class Completeness {
|
|
||||||
<<enumeration>>
|
|
||||||
Full
|
|
||||||
Partial
|
|
||||||
Absent
|
|
||||||
}
|
|
||||||
|
|
||||||
class EnrichedCity {
|
|
||||||
+Location location
|
|
||||||
+LocationContext context
|
|
||||||
}
|
|
||||||
|
|
||||||
class BeerStyle {
|
|
||||||
+string name
|
|
||||||
+string description
|
|
||||||
+float min_abv
|
|
||||||
+float max_abv
|
|
||||||
+int min_ibu
|
|
||||||
+int max_ibu
|
|
||||||
}
|
|
||||||
|
|
||||||
class BreweryResult {
|
|
||||||
+string name_en
|
|
||||||
+string description_en
|
|
||||||
+string name_local
|
|
||||||
+string description_local
|
|
||||||
}
|
|
||||||
|
|
||||||
class BeerResult {
|
|
||||||
+string name_en
|
|
||||||
+string description_en
|
|
||||||
+string name_local
|
|
||||||
+string description_local
|
|
||||||
+string style
|
|
||||||
+float abv
|
|
||||||
+int ibu
|
|
||||||
}
|
|
||||||
|
|
||||||
class UserResult {
|
|
||||||
+string first_name
|
|
||||||
+string last_name
|
|
||||||
+string gender
|
|
||||||
+string username
|
|
||||||
+string bio
|
|
||||||
+float activity_weight
|
|
||||||
}
|
|
||||||
|
|
||||||
class Name {
|
|
||||||
+string first_name
|
|
||||||
+string last_name
|
|
||||||
+string gender
|
|
||||||
}
|
|
||||||
|
|
||||||
class ForenameEntry {
|
|
||||||
+string name
|
|
||||||
+string gender
|
|
||||||
}
|
|
||||||
|
|
||||||
class CheckinResult {
|
|
||||||
+string checked_in_at
|
|
||||||
+string note
|
|
||||||
}
|
|
||||||
|
|
||||||
class RatingResult {
|
|
||||||
+float score
|
|
||||||
+string note
|
|
||||||
}
|
|
||||||
|
|
||||||
class GenerationMetadata {
|
|
||||||
+uint64_t generation_id
|
|
||||||
+string generated_time
|
|
||||||
+bool context_provided
|
|
||||||
+string generated_with
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedBrewery {
|
|
||||||
+uint64_t brewery_id
|
|
||||||
+Location location
|
|
||||||
+BreweryResult brewery
|
|
||||||
+Completeness context_completeness
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedBeer {
|
|
||||||
+uint64_t beer_id
|
|
||||||
+uint64_t brewery_id
|
|
||||||
+Location location
|
|
||||||
+BeerStyle style
|
|
||||||
+BeerResult beer
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedUser {
|
|
||||||
+uint64_t user_id
|
|
||||||
+Location location
|
|
||||||
+UserResult user
|
|
||||||
+string email
|
|
||||||
+string date_of_birth
|
|
||||||
+string password
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedCheckin {
|
|
||||||
+uint64_t checkin_id
|
|
||||||
+uint64_t user_id
|
|
||||||
+uint64_t brewery_id
|
|
||||||
+CheckinResult checkin
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedRating {
|
|
||||||
+uint64_t user_id
|
|
||||||
+uint64_t beer_id
|
|
||||||
+uint64_t checkin_id
|
|
||||||
+RatingResult rating
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratedFollow {
|
|
||||||
+uint64_t follower_id
|
|
||||||
+uint64_t followed_id
|
|
||||||
+GenerationMetadata metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
class UserPersona {
|
|
||||||
+string name
|
|
||||||
+string description
|
|
||||||
+vector~string~ style_affinities
|
|
||||||
}
|
|
||||||
}
|
|
||||||
note for ForenameEntry "forename_list/surname_list are unordered_set aliases, keyed by ISO 3166-1 country code via ICuratedDataService. SampleName() is a free function, not a method, sourced from popular-names-by-country-dataset (CC0)."
|
|
||||||
note for GeneratedUser "email, date_of_birth, and password are generated programmatically by the orchestrator (not LLM-authored) so a downstream auth-account seeding consumer can register real accounts from this export."
|
|
||||||
ForenameEntry ..> Name : SampleName() produces
|
|
||||||
LocationContext *-- Completeness
|
|
||||||
|
|
||||||
namespace DomainApplicationConfiguration {
|
|
||||||
class SamplingOptions {
|
|
||||||
+float temperature
|
|
||||||
+float top_p
|
|
||||||
+uint32_t top_k
|
|
||||||
+uint32_t n_ctx
|
|
||||||
+int seed
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratorMode {
|
|
||||||
<<enumeration>>
|
|
||||||
kMock
|
|
||||||
kLlama
|
|
||||||
kOpenAI
|
|
||||||
}
|
|
||||||
|
|
||||||
class GeneratorOptions {
|
|
||||||
+path model_path
|
|
||||||
+GeneratorMode mode
|
|
||||||
+optional~SamplingOptions~ sampling
|
|
||||||
+string openai_api_key
|
|
||||||
+string openai_model
|
|
||||||
}
|
|
||||||
|
|
||||||
class PipelineOptions {
|
|
||||||
+path output_path
|
|
||||||
+path log_path
|
|
||||||
}
|
|
||||||
|
|
||||||
class ApplicationOptions {
|
|
||||||
+GeneratorOptions generator
|
|
||||||
+PipelineOptions pipeline
|
|
||||||
}
|
|
||||||
}
|
|
||||||
note for GeneratorOptions "Already implemented (models.h), not planned: the three-way GeneratorMode split (kMock/kLlama/kOpenAI) and the openai_api_key/openai_model fields ship today, ahead of the rest of this diagram."
|
|
||||||
ApplicationOptions *-- GeneratorOptions
|
|
||||||
ApplicationOptions *-- PipelineOptions
|
|
||||||
GeneratorOptions *-- GeneratorMode
|
|
||||||
GeneratorOptions o-- SamplingOptions
|
|
||||||
|
|
||||||
namespace DomainPolicy {
|
|
||||||
class ContextStrategy {
|
|
||||||
<<interface>>
|
|
||||||
+QueriesFor(loc) vector~string~
|
|
||||||
+MaxContextChars() size_t
|
|
||||||
}
|
|
||||||
|
|
||||||
class BreweryContextStrategy {
|
|
||||||
+QueriesFor(loc) vector~string~
|
|
||||||
+MaxContextChars() size_t
|
|
||||||
}
|
|
||||||
|
|
||||||
class BeerContextStrategy {
|
|
||||||
+QueriesFor(loc) vector~string~
|
|
||||||
+MaxContextChars() size_t
|
|
||||||
}
|
|
||||||
|
|
||||||
class SamplingStrategy {
|
|
||||||
<<interface>>
|
|
||||||
+Sample(locations) vector~Location~
|
|
||||||
}
|
|
||||||
|
|
||||||
class UniformSamplingStrategy {
|
|
||||||
-size_t sample_size_
|
|
||||||
+Sample(locations) vector~Location~
|
|
||||||
}
|
|
||||||
|
|
||||||
class BeerSelectionStrategy {
|
|
||||||
<<interface>>
|
|
||||||
+SelectStyles(brewery, palette) vector~BeerStyle~
|
|
||||||
}
|
|
||||||
|
|
||||||
class RandomBeerSelectionStrategy {
|
|
||||||
-mt19937 rng_
|
|
||||||
-size_t min_beers_
|
|
||||||
-size_t max_beers_
|
|
||||||
+SelectStyles(brewery, palette) vector~BeerStyle~
|
|
||||||
}
|
|
||||||
|
|
||||||
class CheckinDistributionStrategy {
|
|
||||||
<<interface>>
|
|
||||||
+AssignActivityWeights(users) void
|
|
||||||
+CheckinsForUser(user, brewery_count) size_t
|
|
||||||
+TimestampFor(user, index) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class JCurveCheckinStrategy {
|
|
||||||
-mt19937 rng_
|
|
||||||
+AssignActivityWeights(users) void
|
|
||||||
+CheckinsForUser(user, brewery_count) size_t
|
|
||||||
+TimestampFor(user, index) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class RandomCheckinStrategy {
|
|
||||||
-mt19937 rng_
|
|
||||||
-size_t min_checkins_
|
|
||||||
-size_t max_checkins_
|
|
||||||
+AssignActivityWeights(users) void
|
|
||||||
+CheckinsForUser(user, brewery_count) size_t
|
|
||||||
+TimestampFor(user, index) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class FollowGenerationStrategy {
|
|
||||||
<<interface>>
|
|
||||||
+GenerateFollows(users) vector~GeneratedFollow~
|
|
||||||
}
|
|
||||||
|
|
||||||
class RandomFollowStrategy {
|
|
||||||
-mt19937 rng_
|
|
||||||
-size_t min_follows_
|
|
||||||
-size_t max_follows_
|
|
||||||
+GenerateFollows(users) vector~GeneratedFollow~
|
|
||||||
}
|
|
||||||
|
|
||||||
class ActivityWeightedFollowStrategy {
|
|
||||||
-mt19937 rng_
|
|
||||||
-size_t min_follows_
|
|
||||||
-size_t max_follows_
|
|
||||||
+GenerateFollows(users) vector~GeneratedFollow~
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
namespace InfrastructureLogging {
|
|
||||||
class LogLevel {
|
|
||||||
<<enumeration>>
|
|
||||||
Debug
|
|
||||||
Info
|
|
||||||
Warn
|
|
||||||
Error
|
|
||||||
}
|
|
||||||
|
|
||||||
class PipelinePhase {
|
|
||||||
<<enumeration>>
|
|
||||||
Startup
|
|
||||||
UserGeneration
|
|
||||||
BreweryAndBeerGeneration
|
|
||||||
CheckinGeneration
|
|
||||||
RatingGeneration
|
|
||||||
FollowGeneration
|
|
||||||
Teardown
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogEntry {
|
|
||||||
+time_point timestamp
|
|
||||||
+LogLevel level
|
|
||||||
+PipelinePhase phase
|
|
||||||
+string message
|
|
||||||
+optional~string~ worker
|
|
||||||
}
|
|
||||||
|
|
||||||
class ILogger {
|
|
||||||
<<interface>>
|
|
||||||
+Log(entry) void
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogProducer {
|
|
||||||
-BoundedChannel~LogEntry~ channel_
|
|
||||||
+Log(entry) void
|
|
||||||
}
|
|
||||||
|
|
||||||
class LogDispatcher {
|
|
||||||
-BoundedChannel~LogEntry~ channel_
|
|
||||||
+Run() void
|
|
||||||
-ToSpdlogLevel(level) level_enum
|
|
||||||
}
|
|
||||||
}
|
|
||||||
LogEntry *-- LogLevel
|
|
||||||
LogEntry *-- PipelinePhase
|
|
||||||
ILogger <|.. LogProducer
|
|
||||||
LogProducer ..> LogEntry : emits
|
|
||||||
LogDispatcher ..> LogEntry : consumes
|
|
||||||
|
|
||||||
namespace InfrastructurePipelineChannel {
|
|
||||||
class BoundedChannel~T~ {
|
|
||||||
-queue~T~ queue_
|
|
||||||
-mutex mutex_
|
|
||||||
-condition_variable not_full_
|
|
||||||
-condition_variable not_empty_
|
|
||||||
-size_t capacity_
|
|
||||||
-bool closed_
|
|
||||||
+Send(item) void
|
|
||||||
+Receive() optional~T~
|
|
||||||
+Close() void
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
namespace InfrastructureDataPreloading {
|
|
||||||
class ICuratedDataService {
|
|
||||||
<<interface>>
|
|
||||||
+LoadLocations(filepath) vector~Location~
|
|
||||||
+LoadBeerStyles(filepath) vector~BeerStyle~
|
|
||||||
+LoadPersonas(filepath) vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry(filepath) unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
|
|
||||||
class JsonLoader {
|
|
||||||
+LoadLocations(filepath) vector~Location~
|
|
||||||
+LoadBeerStyles(filepath) vector~BeerStyle~
|
|
||||||
+LoadPersonas(filepath) vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry(filepath) unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
|
|
||||||
class MockCuratedDataService {
|
|
||||||
+LoadLocations(filepath) vector~Location~
|
|
||||||
+LoadBeerStyles(filepath) vector~BeerStyle~
|
|
||||||
+LoadPersonas(filepath) vector~UserPersona~
|
|
||||||
+LoadForenamesByCountry(filepath) unordered_map~string, forename_list~
|
|
||||||
+LoadSurnamesByCountry(filepath) unordered_map~string, surname_list~
|
|
||||||
}
|
|
||||||
}
|
|
||||||
note for JsonLoader "Each Load* call is memoized after its first parse — implemented today, ahead of LoadBeerStyles which is still planned."
|
|
||||||
note for MockCuratedDataService "Fixed in-memory dataset for --mocked mode, implemented today for Locations/Personas/Forenames/Surnames; would need a LoadBeerStyles fixture too once that method is implemented."
|
|
||||||
|
|
||||||
namespace InfrastructureEnrichment {
|
|
||||||
class EnrichmentService {
|
|
||||||
<<interface>>
|
|
||||||
+GetLocationContext(loc, strategy) LocationContext
|
|
||||||
}
|
|
||||||
|
|
||||||
class WikipediaService {
|
|
||||||
-unique_ptr~WebClient~ client_
|
|
||||||
-unordered_map~string, string~ extract_cache_
|
|
||||||
+GetLocationContext(loc, strategy) LocationContext
|
|
||||||
-FetchExtract(query) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class WebClient {
|
|
||||||
<<interface>>
|
|
||||||
+Get(url) string
|
|
||||||
+UrlEncode(value) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class HttpWebClient {
|
|
||||||
+Get(url) string
|
|
||||||
+UrlEncode(value) string
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
namespace InfrastructureGeocoding {
|
|
||||||
class AddressLookupResult {
|
|
||||||
+string address_line1
|
|
||||||
+string postal_code
|
|
||||||
}
|
|
||||||
|
|
||||||
class AddressService {
|
|
||||||
<<interface>>
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
|
|
||||||
class NominatimAddressService {
|
|
||||||
-unique_ptr~WebClient~ client_
|
|
||||||
-Logger logger_
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
|
|
||||||
class MockAddressService {
|
|
||||||
+ReverseGeocode(longitude, latitude) optional~AddressLookupResult~
|
|
||||||
}
|
|
||||||
}
|
|
||||||
note for NominatimAddressService "Already implemented (services/address/), not planned: queries the public Nominatim API, sleeping 1s per request per its usage policy; returns nullopt on failure or when no usable street address is found. Ships today, ahead of the rest of this diagram."
|
|
||||||
note for MockAddressService "Already implemented, not planned: returns a fixed placeholder address ('123 Mock Street', '00000') for --mocked runs, no network call."
|
|
||||||
AddressService <|.. NominatimAddressService
|
|
||||||
AddressService <|.. MockAddressService
|
|
||||||
AddressService ..> AddressLookupResult : returns
|
|
||||||
NominatimAddressService *-- WebClient
|
|
||||||
|
|
||||||
namespace InfrastructurePrompting {
|
|
||||||
class IPromptDirectory {
|
|
||||||
<<interface>>
|
|
||||||
+Load(key) string
|
|
||||||
}
|
|
||||||
|
|
||||||
class PromptDirectory {
|
|
||||||
-path prompt_dir_
|
|
||||||
-unordered_map~string, string~ cache_
|
|
||||||
+PromptDirectory(prompt_dir)
|
|
||||||
+Load(key) string
|
|
||||||
}
|
|
||||||
}
|
|
||||||
IPromptDirectory <|.. PromptDirectory
|
|
||||||
|
|
||||||
namespace InfrastructureDataGeneration {
|
|
||||||
class DataGenerator {
|
|
||||||
<<interface>>
|
|
||||||
+GenerateBrewery(location, context) BreweryResult
|
|
||||||
+GenerateBeer(brewery_id, location, context, style) BeerResult
|
|
||||||
+GenerateUser(city, persona, name) UserResult
|
|
||||||
+GenerateCheckin(user, brewery, timestamp) CheckinResult
|
|
||||||
+GenerateRating(user, beer, checkin_id) RatingResult
|
|
||||||
}
|
|
||||||
|
|
||||||
class MockGenerator {
|
|
||||||
+GenerateBrewery(...) BreweryResult
|
|
||||||
+GenerateBeer(...) BeerResult
|
|
||||||
+GenerateUser(...) UserResult
|
|
||||||
+GenerateCheckin(...) CheckinResult
|
|
||||||
+GenerateRating(...) RatingResult
|
|
||||||
-DeterministicHash(location) size_t
|
|
||||||
}
|
|
||||||
|
|
||||||
class LlamaGenerator {
|
|
||||||
-ModelHandle model_
|
|
||||||
-ContextHandle context_
|
|
||||||
-unique_ptr~PromptFormatter~ prompt_formatter_
|
|
||||||
-unique_ptr~IPromptDirectory~ prompt_directory_
|
|
||||||
-mt19937 rng_
|
|
||||||
+GenerateBrewery(...) BreweryResult
|
|
||||||
+GenerateBeer(...) BeerResult
|
|
||||||
+GenerateUser(...) UserResult
|
|
||||||
+GenerateCheckin(...) CheckinResult
|
|
||||||
+GenerateRating(...) RatingResult
|
|
||||||
-Load(opts) void
|
|
||||||
-Infer(system_prompt, user_prompt, max_tokens, grammar) string
|
|
||||||
-ValidateModelArchitecture() void
|
|
||||||
}
|
|
||||||
|
|
||||||
class PromptFormatter {
|
|
||||||
<<interface>>
|
|
||||||
+Format(system_prompt, user_prompt) string
|
|
||||||
+ExpectedArchitecture() string_view
|
|
||||||
}
|
|
||||||
|
|
||||||
class Gemma4JinjaPromptFormatter {
|
|
||||||
+Format(system_prompt, user_prompt) string
|
|
||||||
+ExpectedArchitecture() string_view
|
|
||||||
}
|
|
||||||
|
|
||||||
class OpenAIGenerator {
|
|
||||||
-string api_key_
|
|
||||||
-string model_
|
|
||||||
-Logger logger_
|
|
||||||
-unique_ptr~IPromptDirectory~ prompt_directory_
|
|
||||||
-unique_ptr~WebClient~ web_client_
|
|
||||||
+GenerateBrewery(location, context) BreweryResult
|
|
||||||
+GenerateBeer(brewery_id, location, context, style) BeerResult
|
|
||||||
+GenerateUser(city, persona, name) UserResult
|
|
||||||
+GenerateCheckin(user, brewery, timestamp) CheckinResult
|
|
||||||
+GenerateRating(user, beer, checkin_id) RatingResult
|
|
||||||
-CallChatCompletionsApi(...) string
|
|
||||||
}
|
|
||||||
}
|
|
||||||
note for OpenAIGenerator "Already implemented (data_generation/openai/), not planned: calls the OpenAI Chat Completions API with Structured Outputs (json_schema, strict) to guarantee schema-valid JSON, needing no IPromptFormatter or GBNF grammar. Ships today for GenerateBrewery/GenerateUser only, ahead of the rest of this diagram — GenerateBeer/GenerateCheckin/GenerateRating are shown here for interface completeness but not yet implemented on any backend."
|
|
||||||
|
|
||||||
namespace InfrastructureDataExport {
|
|
||||||
class ExportService {
|
|
||||||
<<interface>>
|
|
||||||
+Initialize() void
|
|
||||||
+ProcessBrewery(brewery) uint64_t
|
|
||||||
+ProcessBeer(beer) uint64_t
|
|
||||||
+ProcessUser(user) uint64_t
|
|
||||||
+ProcessCheckin(checkin) uint64_t
|
|
||||||
+ProcessRating(rating) void
|
|
||||||
+ProcessFollow(follow) void
|
|
||||||
+Finalize() void
|
|
||||||
}
|
|
||||||
|
|
||||||
class SqliteExportService {
|
|
||||||
-unique_ptr~DateTimeProvider~ date_time_provider_
|
|
||||||
-SqliteDatabaseHandle db_handle_
|
|
||||||
-SqliteStatementHandle insert_location_stmt_
|
|
||||||
-SqliteStatementHandle insert_brewery_stmt_
|
|
||||||
-SqliteStatementHandle insert_beer_stmt_
|
|
||||||
-SqliteStatementHandle insert_user_stmt_
|
|
||||||
-SqliteStatementHandle insert_checkin_stmt_
|
|
||||||
-SqliteStatementHandle insert_rating_stmt_
|
|
||||||
-SqliteStatementHandle insert_follow_stmt_
|
|
||||||
-bool transaction_open_
|
|
||||||
-unordered_map~string, uint64_t~ location_cache_
|
|
||||||
-unordered_map~string, uint64_t~ brewery_cache_
|
|
||||||
+Initialize() void
|
|
||||||
+ProcessRecord(brewery) uint64_t
|
|
||||||
+ProcessRecord(beer) uint64_t
|
|
||||||
+ProcessRecord(user) uint64_t
|
|
||||||
+ProcessRecord(checkin) uint64_t
|
|
||||||
+ProcessRecord(rating) void
|
|
||||||
+ProcessRecord(follow) void
|
|
||||||
+Finalize() void
|
|
||||||
-InitializeSchema() void
|
|
||||||
-PrepareStatements() void
|
|
||||||
-RollbackAndCloseNoThrow() void
|
|
||||||
-FinalizeStatements() void
|
|
||||||
}
|
|
||||||
|
|
||||||
class DateTimeProvider {
|
|
||||||
<<interface>>
|
|
||||||
+GetUtcTimestamp() string
|
|
||||||
}
|
|
||||||
|
|
||||||
class SystemDateTimeProvider {
|
|
||||||
+GetUtcTimestamp() string
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
class BiergartenPipelineOrchestrator {
|
|
||||||
-unique_ptr~ICuratedDataService~ curated_data_service_
|
|
||||||
-unique_ptr~EnrichmentService~ enrichment_service_
|
|
||||||
-unique_ptr~DataGenerator~ generator_
|
|
||||||
-unique_ptr~Logger~ logger_
|
|
||||||
-unique_ptr~ExportService~ exporter_
|
|
||||||
-unique_ptr~AddressService~ address_service_
|
|
||||||
-unique_ptr~ContextStrategy~ brewery_context_strategy_
|
|
||||||
-unique_ptr~SamplingStrategy~ sampling_strategy_
|
|
||||||
-unique_ptr~BeerSelectionStrategy~ beer_selection_strategy_
|
|
||||||
-unique_ptr~CheckinDistributionStrategy~ checkin_strategy_
|
|
||||||
-unique_ptr~FollowGenerationStrategy~ follow_strategy_
|
|
||||||
-vector~BeerStyle~ beer_style_palette_
|
|
||||||
-ApplicationOptions options_
|
|
||||||
-vector~GeneratedUser~ user_pool_
|
|
||||||
-vector~GeneratedBrewery~ brewery_pool_
|
|
||||||
-vector~GeneratedBeer~ beer_pool_
|
|
||||||
-vector~GeneratedCheckin~ checkin_pool_
|
|
||||||
-vector~GeneratedFollow~ follow_pool_
|
|
||||||
+Run() bool
|
|
||||||
-RunUserPhase(locations) void
|
|
||||||
-RunBreweryAndBeerPhase(locations) void
|
|
||||||
-RunCheckinPhase() void
|
|
||||||
-RunRatingPhase() void
|
|
||||||
-RunFollowPhase() void
|
|
||||||
}
|
|
||||||
|
|
||||||
BiergartenPipelineOrchestrator *-- ICuratedDataService
|
|
||||||
BiergartenPipelineOrchestrator *-- EnrichmentService
|
|
||||||
BiergartenPipelineOrchestrator *-- DataGenerator
|
|
||||||
BiergartenPipelineOrchestrator *-- ExportService
|
|
||||||
BiergartenPipelineOrchestrator *-- AddressService
|
|
||||||
BiergartenPipelineOrchestrator *-- CheckinDistributionStrategy
|
|
||||||
BiergartenPipelineOrchestrator *-- FollowGenerationStrategy
|
|
||||||
BiergartenPipelineOrchestrator *-- SamplingStrategy
|
|
||||||
BiergartenPipelineOrchestrator *-- BeerSelectionStrategy
|
|
||||||
BiergartenPipelineOrchestrator *-- ApplicationOptions
|
|
||||||
BiergartenPipelineOrchestrator *-- Logger
|
|
||||||
|
|
||||||
BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedUser : user_pool_
|
|
||||||
BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedBrewery : brewery_pool_
|
|
||||||
BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedBeer : beer_pool_
|
|
||||||
BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedCheckin : checkin_pool_
|
|
||||||
BiergartenPipelineOrchestrator "1" *-- "0..*" GeneratedFollow : follow_pool_
|
|
||||||
|
|
||||||
ICuratedDataService <|.. JsonLoader
|
|
||||||
ICuratedDataService <|.. MockCuratedDataService
|
|
||||||
Logger <|.. PipelineLogger
|
|
||||||
ContextStrategy <|.. BreweryContextStrategy
|
|
||||||
ContextStrategy <|.. BeerContextStrategy
|
|
||||||
SamplingStrategy <|.. UniformSamplingStrategy
|
|
||||||
BeerSelectionStrategy <|.. RandomBeerSelectionStrategy
|
|
||||||
CheckinDistributionStrategy <|.. JCurveCheckinStrategy
|
|
||||||
CheckinDistributionStrategy <|.. RandomCheckinStrategy
|
|
||||||
FollowGenerationStrategy <|.. RandomFollowStrategy
|
|
||||||
FollowGenerationStrategy <|.. ActivityWeightedFollowStrategy
|
|
||||||
EnrichmentService <|.. WikipediaService
|
|
||||||
WebClient <|.. HttpWebClient
|
|
||||||
DataGenerator <|.. MockGenerator
|
|
||||||
DataGenerator <|.. LlamaGenerator
|
|
||||||
DataGenerator <|.. OpenAIGenerator
|
|
||||||
PromptFormatter <|.. Gemma4JinjaPromptFormatter
|
|
||||||
ExportService <|.. SqliteExportService
|
|
||||||
DateTimeProvider <|.. SystemDateTimeProvider
|
|
||||||
|
|
||||||
WikipediaService *-- WebClient
|
|
||||||
WikipediaService ..> ContextStrategy
|
|
||||||
LlamaGenerator *-- PromptFormatter
|
|
||||||
LlamaGenerator *-- IPromptDirectory
|
|
||||||
LlamaGenerator ..> GeneratorOptions
|
|
||||||
OpenAIGenerator *-- IPromptDirectory
|
|
||||||
OpenAIGenerator *-- WebClient
|
|
||||||
SqliteExportService *-- DateTimeProvider
|
|
||||||
|
|
||||||
PipelineLogger o-- BoundedChannel : logs to
|
|
||||||
LogWorker o-- BoundedChannel : drains from
|
|
||||||
|
|
||||||
EnrichedCity *-- Location
|
|
||||||
EnrichedCity *-- LocationContext
|
|
||||||
GeneratedBrewery *-- Location
|
|
||||||
GeneratedBrewery *-- BreweryResult
|
|
||||||
GeneratedBrewery *-- GenerationMetadata
|
|
||||||
GeneratedBeer *-- Location
|
|
||||||
GeneratedBeer *-- BeerStyle
|
|
||||||
GeneratedBeer *-- BeerResult
|
|
||||||
GeneratedBeer *-- GenerationMetadata
|
|
||||||
GeneratedUser *-- Location
|
|
||||||
GeneratedUser *-- UserResult
|
|
||||||
GeneratedUser *-- GenerationMetadata
|
|
||||||
GeneratedCheckin *-- CheckinResult
|
|
||||||
GeneratedCheckin *-- GenerationMetadata
|
|
||||||
GeneratedRating *-- RatingResult
|
|
||||||
GeneratedRating *-- GenerationMetadata
|
|
||||||
GeneratedFollow *-- GenerationMetadata
|
|
||||||
646
docs/pipeline/diagrams/planned/class.puml
Normal file
646
docs/pipeline/diagrams/planned/class.puml
Normal file
@@ -0,0 +1,646 @@
|
|||||||
|
@startuml class_diagram
|
||||||
|
|
||||||
|
' ==========================================
|
||||||
|
' CONFIGURATION & STYLING
|
||||||
|
' ==========================================
|
||||||
|
!include ../biergarten-weizen-theme.puml
|
||||||
|
skinparam classAttributeFontSize 9
|
||||||
|
skinparam defaultFontSize 25
|
||||||
|
skinparam titleFontSize 30
|
||||||
|
|
||||||
|
title Biergarten Data Pipeline — Class Diagram
|
||||||
|
|
||||||
|
package "Domain: Models" {
|
||||||
|
|
||||||
|
class Location {
|
||||||
|
+ city : std::string
|
||||||
|
+ state_province : std::string
|
||||||
|
+ iso3166_2 : std::string
|
||||||
|
+ country : std::string
|
||||||
|
+ iso3166_1 : std::string
|
||||||
|
+ local_languages : std::vector<std::string>
|
||||||
|
+ latitude : double
|
||||||
|
+ longitude : double
|
||||||
|
}
|
||||||
|
|
||||||
|
class LocationContext {
|
||||||
|
+ text : std::string
|
||||||
|
+ completeness : Completeness
|
||||||
|
+ char_count : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
enum Completeness {
|
||||||
|
Full
|
||||||
|
Partial
|
||||||
|
Absent
|
||||||
|
}
|
||||||
|
|
||||||
|
class EnrichedCity {
|
||||||
|
+ location : Location
|
||||||
|
+ context : LocationContext
|
||||||
|
}
|
||||||
|
|
||||||
|
class BeerStyle {
|
||||||
|
+ name : std::string
|
||||||
|
+ description : std::string
|
||||||
|
+ min_abv : float
|
||||||
|
+ max_abv : float
|
||||||
|
+ min_ibu : int
|
||||||
|
+ max_ibu : int
|
||||||
|
}
|
||||||
|
|
||||||
|
class BreweryResult {
|
||||||
|
+ name_en : std::string
|
||||||
|
+ description_en : std::string
|
||||||
|
+ name_local : std::string
|
||||||
|
+ description_local : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class BeerResult {
|
||||||
|
+ name_en : std::string
|
||||||
|
+ description_en : std::string
|
||||||
|
+ name_local : std::string
|
||||||
|
+ description_local : std::string
|
||||||
|
+ style : std::string
|
||||||
|
+ abv : float
|
||||||
|
+ ibu : int
|
||||||
|
}
|
||||||
|
|
||||||
|
class UserResult {
|
||||||
|
+ first_name : std::string
|
||||||
|
+ last_name : std::string
|
||||||
|
+ gender : std::string
|
||||||
|
+ username : std::string
|
||||||
|
+ bio : std::string
|
||||||
|
+ activity_weight : float
|
||||||
|
}
|
||||||
|
|
||||||
|
class Name {
|
||||||
|
+ first_name : std::string
|
||||||
|
+ last_name : std::string
|
||||||
|
+ gender : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class ForenameEntry {
|
||||||
|
+ name : std::string
|
||||||
|
+ gender : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of ForenameEntry
|
||||||
|
forename_list = std::unordered_set<ForenameEntry>
|
||||||
|
surname_list = std::unordered_set<std::string>
|
||||||
|
(aliases declared in curated_data_service.h).
|
||||||
|
ICuratedDataService::LoadForenamesByCountry() /
|
||||||
|
LoadSurnamesByCountry() each return a flat
|
||||||
|
std::unordered_map<std::string, forename_list>
|
||||||
|
or <std::string, surname_list> keyed by ISO
|
||||||
|
3166-1 country code -- no NamesByCountry
|
||||||
|
wrapper class. SampleName(forenames_by_country,
|
||||||
|
surnames_by_country, iso3166_1, rng) is a free
|
||||||
|
function, not a method, so it can be called by
|
||||||
|
any per-city worker without owning the maps.
|
||||||
|
Sourced from popular-names-by-country-dataset
|
||||||
|
(CC0) -- see ETHICS-AND-KNOWN-ISSUES.md. Gender
|
||||||
|
is preserved per forename so it can be threaded
|
||||||
|
through to persona/bio generation later; surnames
|
||||||
|
carry no gender in the source data. Pairing
|
||||||
|
happens at sample time, not at fixture-authoring
|
||||||
|
time, so no information from the source dataset
|
||||||
|
is discarded up front.
|
||||||
|
end note
|
||||||
|
|
||||||
|
ForenameEntry ..> Name : SampleName() produces
|
||||||
|
|
||||||
|
class CheckinResult {
|
||||||
|
+ checked_in_at : std::string
|
||||||
|
+ note : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class RatingResult {
|
||||||
|
+ score : float
|
||||||
|
+ note : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class GenerationMetadata {
|
||||||
|
+ generation_id : uint64_t
|
||||||
|
+ generated_time : std::string
|
||||||
|
+ context_provided : bool
|
||||||
|
+ generated_with : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratedBrewery {
|
||||||
|
+ brewery_id : uint64_t
|
||||||
|
+ location : Location
|
||||||
|
+ brewery : BreweryResult
|
||||||
|
+ context_completeness : LocationContext::Completeness
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratedBeer {
|
||||||
|
+ beer_id : uint64_t
|
||||||
|
+ brewery_id : uint64_t
|
||||||
|
+ location : Location
|
||||||
|
+ style : BeerStyle
|
||||||
|
+ beer : BeerResult
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratedUser {
|
||||||
|
+ user_id : uint64_t
|
||||||
|
+ location : Location
|
||||||
|
+ user : UserResult
|
||||||
|
+ email : std::string
|
||||||
|
+ date_of_birth : std::string
|
||||||
|
+ password : std::string
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of GeneratedUser
|
||||||
|
email, date_of_birth, and password are
|
||||||
|
programmatically generated by the orchestrator
|
||||||
|
(not LLM-authored) so a downstream auth-account
|
||||||
|
seeding consumer can register real accounts from
|
||||||
|
this export. See ROADMAP.md §1.
|
||||||
|
end note
|
||||||
|
|
||||||
|
class GeneratedCheckin {
|
||||||
|
+ checkin_id : uint64_t
|
||||||
|
+ user_id : uint64_t
|
||||||
|
+ brewery_id : uint64_t
|
||||||
|
+ checkin : CheckinResult
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratedRating {
|
||||||
|
+ user_id : uint64_t
|
||||||
|
+ beer_id : uint64_t
|
||||||
|
+ checkin_id : uint64_t
|
||||||
|
+ rating : RatingResult
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratedFollow {
|
||||||
|
+ follower_id : uint64_t
|
||||||
|
+ followed_id : uint64_t
|
||||||
|
+ metadata : GenerationMetadata
|
||||||
|
}
|
||||||
|
|
||||||
|
class UserPersona {
|
||||||
|
+ name: std::string
|
||||||
|
+ description: std::string
|
||||||
|
+ style_affinities: std::vector<std::string>
|
||||||
|
}
|
||||||
|
|
||||||
|
LocationContext *-- Completeness
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Domain: Application Configuration" {
|
||||||
|
class SamplingOptions {
|
||||||
|
+ temperature: float = 1.0F
|
||||||
|
+ top_p: float = 0.95F
|
||||||
|
+ top_k: uint32_t = 64
|
||||||
|
+ n_ctx: uint32_t = 8192
|
||||||
|
+ seed: int = -1
|
||||||
|
}
|
||||||
|
|
||||||
|
class GeneratorOptions {
|
||||||
|
+ model_path: std::filesystem::path
|
||||||
|
+ use_mocked: bool = false
|
||||||
|
+ sampling: std::optional<SamplingOptions>
|
||||||
|
}
|
||||||
|
|
||||||
|
class PipelineOptions {
|
||||||
|
+ output_path: std::filesystem::path
|
||||||
|
+ log_path: std::filesystem::path
|
||||||
|
}
|
||||||
|
|
||||||
|
class ApplicationOptions {
|
||||||
|
+ generator: GeneratorOptions
|
||||||
|
+ pipeline: PipelineOptions
|
||||||
|
}
|
||||||
|
|
||||||
|
ApplicationOptions *-- GeneratorOptions
|
||||||
|
ApplicationOptions *-- PipelineOptions
|
||||||
|
GeneratorOptions o-- SamplingOptions
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Domain: Policy" {
|
||||||
|
|
||||||
|
interface ContextStrategy <<interface>> {
|
||||||
|
+ QueriesFor(loc : const Location&) : std::vector<std::string>
|
||||||
|
+ MaxContextChars() : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
class BreweryContextStrategy {
|
||||||
|
+ QueriesFor(loc : const Location&) : std::vector<std::string>
|
||||||
|
+ MaxContextChars() : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
class BeerContextStrategy {
|
||||||
|
+ QueriesFor(loc : const Location&) : std::vector<std::string>
|
||||||
|
+ MaxContextChars() : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
interface SamplingStrategy <<interface>> {
|
||||||
|
+ Sample(locations : const std::vector<Location>&) : std::vector<Location>
|
||||||
|
}
|
||||||
|
|
||||||
|
class UniformSamplingStrategy {
|
||||||
|
- sample_size_ : size_t
|
||||||
|
+ Sample(locations : const std::vector<Location>&) : std::vector<Location>
|
||||||
|
}
|
||||||
|
|
||||||
|
interface BeerSelectionStrategy <<interface>> {
|
||||||
|
+ SelectStyles(brewery : const GeneratedBrewery&,\n palette : std::span<const BeerStyle>) : std::vector<BeerStyle>
|
||||||
|
}
|
||||||
|
|
||||||
|
class RandomBeerSelectionStrategy {
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
- min_beers_ : size_t
|
||||||
|
- max_beers_ : size_t
|
||||||
|
+ SelectStyles(brewery : const GeneratedBrewery&,\n palette : std::span<const BeerStyle>) : std::vector<BeerStyle>
|
||||||
|
}
|
||||||
|
|
||||||
|
interface CheckinDistributionStrategy <<interface>> {
|
||||||
|
+ AssignActivityWeights(users : std::vector<GeneratedUser>&) : void
|
||||||
|
+ CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t
|
||||||
|
+ TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class JCurveCheckinStrategy {
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
+ AssignActivityWeights(users : std::vector<GeneratedUser>&) : void
|
||||||
|
+ CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t
|
||||||
|
+ TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class RandomCheckinStrategy {
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
- min_checkins_ : size_t
|
||||||
|
- max_checkins_ : size_t
|
||||||
|
+ AssignActivityWeights(users : std::vector<GeneratedUser>&) : void
|
||||||
|
+ CheckinsForUser(user : const GeneratedUser&,\n brewery_count : size_t) : size_t
|
||||||
|
+ TimestampFor(user : const GeneratedUser&,\n index : size_t) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface FollowGenerationStrategy <<interface>> {
|
||||||
|
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
|
||||||
|
}
|
||||||
|
|
||||||
|
class RandomFollowStrategy {
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
- min_follows_ : size_t
|
||||||
|
- max_follows_ : size_t
|
||||||
|
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
|
||||||
|
}
|
||||||
|
|
||||||
|
class ActivityWeightedFollowStrategy {
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
- min_follows_ : size_t
|
||||||
|
- max_follows_ : size_t
|
||||||
|
+ GenerateFollows(users : const std::vector<GeneratedUser>&) : std::vector<GeneratedFollow>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Logging" {
|
||||||
|
enum LogLevel {
|
||||||
|
Debug
|
||||||
|
Info
|
||||||
|
Warn
|
||||||
|
Error
|
||||||
|
}
|
||||||
|
|
||||||
|
enum PipelinePhase {
|
||||||
|
Startup
|
||||||
|
UserGeneration
|
||||||
|
BreweryAndBeerGeneration
|
||||||
|
CheckinGeneration
|
||||||
|
RatingGeneration
|
||||||
|
FollowGeneration
|
||||||
|
Teardown
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogEntry {
|
||||||
|
+ timestamp : std::chrono::system_clock::time_point
|
||||||
|
+ level : LogLevel
|
||||||
|
+ phase : PipelinePhase
|
||||||
|
+ message : std::string
|
||||||
|
+ worker : std::optional<std::string>
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ILogger <<interface>> {
|
||||||
|
+ Log(entry : const LogEntry&) : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogProducer {
|
||||||
|
- channel_ : BoundedChannel<LogEntry>&
|
||||||
|
+ Log(entry : const LogEntry&) : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class LogDispatcher {
|
||||||
|
- channel_ : BoundedChannel<LogEntry>&
|
||||||
|
+ Run() : void
|
||||||
|
- ToSpdlogLevel(level) : spdlog::level::level_enum
|
||||||
|
}
|
||||||
|
|
||||||
|
LogEntry *-- LogLevel
|
||||||
|
LogEntry *-- PipelinePhase
|
||||||
|
ILogger <|.. LogProducer
|
||||||
|
LogProducer ..> LogEntry : emits
|
||||||
|
LogDispatcher ..> LogEntry : consumes
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Pipeline Channel" {
|
||||||
|
|
||||||
|
class "BoundedChannel<T>" as BoundedChannel {
|
||||||
|
- queue_ : std::queue<T>
|
||||||
|
- mutex_ : std::mutex
|
||||||
|
- not_full_ : std::condition_variable
|
||||||
|
- not_empty_ : std::condition_variable
|
||||||
|
- capacity_ : size_t
|
||||||
|
- closed_ : bool
|
||||||
|
+ Send(item : T) : void
|
||||||
|
+ Receive() : std::optional<T>
|
||||||
|
+ Close() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Data Preloading" {
|
||||||
|
|
||||||
|
interface ICuratedDataService <<interface>> {
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
class JsonLoader {
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of JsonLoader
|
||||||
|
Each Load* call is memoized after its first
|
||||||
|
parse -- implemented today, ahead of the rest
|
||||||
|
of this package (LoadBeerStyles is still planned).
|
||||||
|
end note
|
||||||
|
|
||||||
|
class MockCuratedDataService {
|
||||||
|
+ LoadLocations(filepath : const std::filesystem::path&) : const std::vector<Location>&
|
||||||
|
+ LoadBeerStyles(filepath : const std::filesystem::path&) : const std::vector<BeerStyle>&
|
||||||
|
+ LoadPersonas(filepath : const std::filesystem::path&) : const std::vector<UserPersona>&
|
||||||
|
+ LoadForenamesByCountry(forenames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, forename_list>&
|
||||||
|
+ LoadSurnamesByCountry(surnames_filepath : const std::filesystem::path&) : const std::unordered_map<std::string, surname_list>&
|
||||||
|
}
|
||||||
|
|
||||||
|
note right of MockCuratedDataService
|
||||||
|
Fixed in-memory dataset used in --mocked mode,
|
||||||
|
implemented today for Locations/Personas/
|
||||||
|
Forenames/Surnames; would need a LoadBeerStyles
|
||||||
|
fixture too once that method is implemented.
|
||||||
|
end note
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Enrichment" {
|
||||||
|
|
||||||
|
interface EnrichmentService <<interface>> {
|
||||||
|
+ GetLocationContext(loc : const Location&,\n strategy : const ContextStrategy&) : LocationContext
|
||||||
|
}
|
||||||
|
|
||||||
|
class WikipediaService {
|
||||||
|
- client_ : std::unique_ptr<WebClient>
|
||||||
|
- extract_cache_ : std::unordered_map<std::string, std::string>
|
||||||
|
+ GetLocationContext(loc : const Location&,\n strategy : const ContextStrategy&) : LocationContext
|
||||||
|
- FetchExtract(query : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
interface WebClient <<interface>> {
|
||||||
|
+ Get(url : const std::string&) : std::string
|
||||||
|
+ UrlEncode(value : const std::string&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class HttpWebClient {
|
||||||
|
+ Get(url : const std::string&) : std::string
|
||||||
|
+ UrlEncode(value : const std::string&) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Prompting" {
|
||||||
|
|
||||||
|
interface IPromptDirectory <<interface>> {
|
||||||
|
+ Load(key : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class PromptDirectory {
|
||||||
|
- prompt_dir_ : std::filesystem::path
|
||||||
|
- cache_ : std::unordered_map<std::string, std::string>
|
||||||
|
+ PromptDirectory(prompt_dir : const std::filesystem::path&)
|
||||||
|
+ Load(key : std::string_view) : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
IPromptDirectory <|.. PromptDirectory
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Data Generation" {
|
||||||
|
|
||||||
|
interface DataGenerator <<interface>> {
|
||||||
|
+ GenerateBrewery(location : const Location&,\n context : const LocationContext&) : BreweryResult
|
||||||
|
+ GenerateBeer(brewery_id : uint64_t,\n location : const Location&,\n context : const LocationContext&,\n style : const BeerStyle&) : BeerResult
|
||||||
|
+ GenerateUser(city : const EnrichedCity&,\n persona : const UserPersona&,\n name : const Name&) : UserResult
|
||||||
|
+ GenerateCheckin(user : const GeneratedUser&,\n brewery : const GeneratedBrewery&,\n timestamp : const std::string&) : CheckinResult
|
||||||
|
+ GenerateRating(user : const GeneratedUser&,\n beer : const GeneratedBeer&,\n checkin_id : uint64_t) : RatingResult
|
||||||
|
}
|
||||||
|
|
||||||
|
class MockGenerator {
|
||||||
|
+ GenerateBrewery(...) : BreweryResult
|
||||||
|
+ GenerateBeer(...) : BeerResult
|
||||||
|
+ GenerateUser(...) : UserResult
|
||||||
|
+ GenerateCheckin(...) : CheckinResult
|
||||||
|
+ GenerateRating(...) : RatingResult
|
||||||
|
- DeterministicHash(location : const Location&) : size_t
|
||||||
|
}
|
||||||
|
|
||||||
|
class LlamaGenerator {
|
||||||
|
- model_ : ModelHandle
|
||||||
|
- context_ : ContextHandle
|
||||||
|
- prompt_formatter_ : std::unique_ptr<PromptFormatter>
|
||||||
|
- prompt_directory_ : std::unique_ptr<IPromptDirectory>
|
||||||
|
- rng_ : std::mt19937
|
||||||
|
+ GenerateBrewery(...) : BreweryResult
|
||||||
|
+ GenerateBeer(...) : BeerResult
|
||||||
|
+ GenerateUser(...) : UserResult
|
||||||
|
+ GenerateCheckin(...) : CheckinResult
|
||||||
|
+ GenerateRating(...) : RatingResult
|
||||||
|
- Load(opts : const GeneratorOptions&) : void
|
||||||
|
- Infer(system_prompt, user_prompt,\n max_tokens, grammar) : std::string
|
||||||
|
- ValidateModelArchitecture() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PromptFormatter <<interface>> {
|
||||||
|
+ Format(system_prompt : std::string_view,\n user_prompt : std::string_view) : std::string
|
||||||
|
+ ExpectedArchitecture() : std::string_view
|
||||||
|
}
|
||||||
|
|
||||||
|
class Gemma4JinjaPromptFormatter {
|
||||||
|
+ Format(...) : std::string
|
||||||
|
+ ExpectedArchitecture() : std::string_view
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
package "Infrastructure: Data Export" {
|
||||||
|
|
||||||
|
interface ExportService <<interface>> {
|
||||||
|
+ Initialize() : void
|
||||||
|
+ ProcessBrewery(brewery : const GeneratedBrewery&) : uint64_t
|
||||||
|
+ ProcessBeer(beer : const GeneratedBeer&) : uint64_t
|
||||||
|
+ ProcessUser(user : const GeneratedUser&) : uint64_t
|
||||||
|
+ ProcessCheckin(checkin : const GeneratedCheckin&) : uint64_t
|
||||||
|
+ ProcessRating(rating : const GeneratedRating&) : void
|
||||||
|
+ ProcessFollow(follow : const GeneratedFollow&) : void
|
||||||
|
+ Finalize() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
class SqliteExportService {
|
||||||
|
- date_time_provider_ : std::unique_ptr<DateTimeProvider>
|
||||||
|
- db_handle_ : SqliteDatabaseHandle
|
||||||
|
- insert_location_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_brewery_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_beer_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_user_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_checkin_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_rating_stmt_ : SqliteStatementHandle
|
||||||
|
- insert_follow_stmt_ : SqliteStatementHandle
|
||||||
|
- transaction_open_ : bool
|
||||||
|
- location_cache_ : std::unordered_map<std::string, uint64_t>
|
||||||
|
- brewery_cache_ : std::unordered_map<std::string, uint64_t>
|
||||||
|
+ Initialize() : void
|
||||||
|
+ ProcessRecord(brewery : const GeneratedBrewery&) : uint64_t
|
||||||
|
+ ProcessRecord(beer : const GeneratedBeer&) : uint64_t
|
||||||
|
+ ProcessRecord(user : const GeneratedUser&) : uint64_t
|
||||||
|
+ ProcessRecord(checkin : const GeneratedCheckin&) : uint64_t
|
||||||
|
+ ProcessRecord(rating : const GeneratedRating&) : void
|
||||||
|
+ ProcessRecord(follow : const GeneratedFollow&) : void
|
||||||
|
+ Finalize() : void
|
||||||
|
- InitializeSchema() : void
|
||||||
|
- PrepareStatements() : void
|
||||||
|
- RollbackAndCloseNoThrow() : void
|
||||||
|
- FinalizeStatements() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
interface DateTimeProvider <<interface>> {
|
||||||
|
+ GetUtcTimestamp() : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
class SystemDateTimeProvider {
|
||||||
|
+ GetUtcTimestamp() : std::string
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
class BiergartenPipelineOrchestrator {
|
||||||
|
- curated_data_service_ : std::unique_ptr<ICuratedDataService>
|
||||||
|
- enrichment_service_ : std::unique_ptr<EnrichmentService>
|
||||||
|
- generator_ : std::unique_ptr<DataGenerator>
|
||||||
|
- logger_ : std::unique_ptr<Logger>
|
||||||
|
- exporter_ : std::unique_ptr<ExportService>
|
||||||
|
- brewery_context_strategy_ : std::unique_ptr<ContextStrategy>
|
||||||
|
- sampling_strategy_ : std::unique_ptr<SamplingStrategy>
|
||||||
|
- beer_selection_strategy_ : std::unique_ptr<BeerSelectionStrategy>
|
||||||
|
- checkin_strategy_ : std::unique_ptr<CheckinDistributionStrategy>
|
||||||
|
- follow_strategy_ : std::unique_ptr<FollowGenerationStrategy>
|
||||||
|
- beer_style_palette_ : std::vector<BeerStyle>
|
||||||
|
- options_ : ApplicationOptions
|
||||||
|
--
|
||||||
|
- user_pool_ : std::vector<GeneratedUser>
|
||||||
|
- brewery_pool_ : std::vector<GeneratedBrewery>
|
||||||
|
- beer_pool_ : std::vector<GeneratedBeer>
|
||||||
|
- checkin_pool_ : std::vector<GeneratedCheckin>
|
||||||
|
- follow_pool_ : std::vector<GeneratedFollow>
|
||||||
|
--
|
||||||
|
+ Run() : bool
|
||||||
|
- RunUserPhase(locations : const std::vector<Location>&) : void
|
||||||
|
- RunBreweryAndBeerPhase(locations : const std::vector<Location>&) : void
|
||||||
|
- RunCheckinPhase() : void
|
||||||
|
- RunRatingPhase() : void
|
||||||
|
- RunFollowPhase() : void
|
||||||
|
}
|
||||||
|
|
||||||
|
' --- Orchestration Aggregations (Services & Strategies) ---
|
||||||
|
BiergartenPipelineOrchestrator *-- ICuratedDataService
|
||||||
|
BiergartenPipelineOrchestrator *-- EnrichmentService
|
||||||
|
BiergartenPipelineOrchestrator *-- DataGenerator
|
||||||
|
BiergartenPipelineOrchestrator *-- ExportService
|
||||||
|
BiergartenPipelineOrchestrator *-- CheckinDistributionStrategy
|
||||||
|
BiergartenPipelineOrchestrator *-- FollowGenerationStrategy
|
||||||
|
BiergartenPipelineOrchestrator *-- SamplingStrategy
|
||||||
|
BiergartenPipelineOrchestrator *-- BeerSelectionStrategy
|
||||||
|
BiergartenPipelineOrchestrator *-- ApplicationOptions
|
||||||
|
BiergartenPipelineOrchestrator *-- Logger
|
||||||
|
|
||||||
|
' --- Orchestration Aggregations (Data Pools) ---
|
||||||
|
BiergartenPipelineOrchestrator *-- "0..*" GeneratedUser : user_pool_
|
||||||
|
BiergartenPipelineOrchestrator *-- "0..*" GeneratedBrewery : brewery_pool_
|
||||||
|
BiergartenPipelineOrchestrator *-- "0..*" GeneratedBeer : beer_pool_
|
||||||
|
BiergartenPipelineOrchestrator *-- "0..*" GeneratedCheckin : checkin_pool_
|
||||||
|
BiergartenPipelineOrchestrator *-- "0..*" GeneratedFollow : follow_pool_
|
||||||
|
|
||||||
|
' --- Interfaces & Implementations ---
|
||||||
|
ICuratedDataService <|.. JsonLoader
|
||||||
|
ICuratedDataService <|.. MockCuratedDataService
|
||||||
|
Logger <|.. PipelineLogger
|
||||||
|
ContextStrategy <|.. BreweryContextStrategy
|
||||||
|
ContextStrategy <|.. BeerContextStrategy
|
||||||
|
SamplingStrategy <|.. UniformSamplingStrategy
|
||||||
|
BeerSelectionStrategy <|.. RandomBeerSelectionStrategy
|
||||||
|
CheckinDistributionStrategy <|.. JCurveCheckinStrategy
|
||||||
|
CheckinDistributionStrategy <|.. RandomCheckinStrategy
|
||||||
|
FollowGenerationStrategy <|.. RandomFollowStrategy
|
||||||
|
FollowGenerationStrategy <|.. ActivityWeightedFollowStrategy
|
||||||
|
EnrichmentService <|.. WikipediaService
|
||||||
|
WebClient <|.. HttpWebClient
|
||||||
|
DataGenerator <|.. MockGenerator
|
||||||
|
DataGenerator <|.. LlamaGenerator
|
||||||
|
PromptFormatter <|.. Gemma4JinjaPromptFormatter
|
||||||
|
ExportService <|.. SqliteExportService
|
||||||
|
DateTimeProvider <|.. SystemDateTimeProvider
|
||||||
|
|
||||||
|
' --- Service Compositions & Dependencies ---
|
||||||
|
WikipediaService *-- WebClient
|
||||||
|
WikipediaService ..> ContextStrategy
|
||||||
|
LlamaGenerator *-- PromptFormatter
|
||||||
|
LlamaGenerator *-- IPromptDirectory
|
||||||
|
LlamaGenerator ..> GeneratorOptions
|
||||||
|
SqliteExportService *-- DateTimeProvider
|
||||||
|
|
||||||
|
' --- Cross-Component Aggregations (Held References) ---
|
||||||
|
PipelineLogger o-- BoundedChannel : logs to
|
||||||
|
LogWorker o-- BoundedChannel : drains from
|
||||||
|
|
||||||
|
' --- Domain Containment ---
|
||||||
|
EnrichedCity *-- Location
|
||||||
|
EnrichedCity *-- LocationContext
|
||||||
|
GeneratedBrewery *-- Location
|
||||||
|
GeneratedBrewery *-- BreweryResult
|
||||||
|
GeneratedBrewery *-- GenerationMetadata
|
||||||
|
GeneratedBeer *-- Location
|
||||||
|
GeneratedBeer *-- BeerStyle
|
||||||
|
GeneratedBeer *-- BeerResult
|
||||||
|
GeneratedBeer *-- GenerationMetadata
|
||||||
|
GeneratedUser *-- Location
|
||||||
|
GeneratedUser *-- UserResult
|
||||||
|
GeneratedUser *-- GenerationMetadata
|
||||||
|
GeneratedCheckin *-- CheckinResult
|
||||||
|
GeneratedCheckin *-- GenerationMetadata
|
||||||
|
GeneratedRating *-- RatingResult
|
||||||
|
GeneratedRating *-- GenerationMetadata
|
||||||
|
GeneratedFollow *-- GenerationMetadata
|
||||||
|
|
||||||
|
@enduml
|
||||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 142 KiB |
1
docs/pipeline/diagrams/planned/output/class.svg
Normal file
1
docs/pipeline/diagrams/planned/output/class.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 240 KiB |
@@ -1,611 +0,0 @@
|
|||||||
# Architecture
|
|
||||||
|
|
||||||
This document describes the architecture of The Biergarten App.
|
|
||||||
|
|
||||||
## High-level overview
|
|
||||||
|
|
||||||
The Biergarten App is a monorepo split between the backend and the website:
|
|
||||||
|
|
||||||
- **Backend**: .NET 10 Web API with SQL Server, organized as vertical feature
|
|
||||||
slices with MediatR
|
|
||||||
- **Frontend**: React 19 + React Router 8 website in `web/frontend`
|
|
||||||
- **Architecture Style**: Vertical-slice backend plus server-rendered React
|
|
||||||
frontend
|
|
||||||
|
|
||||||
`archive/next-js-web-app/` contains an archived Next.js frontend, kept for
|
|
||||||
reference.
|
|
||||||
|
|
||||||
## Diagrams
|
|
||||||
|
|
||||||
For visual representations, see:
|
|
||||||
|
|
||||||
- [Vertical Slice](diagrams/vertical-slice.md) - host wiring and project
|
|
||||||
reference diagrams
|
|
||||||
- [Deployment](diagrams/deployment.md) - Docker deployment diagram
|
|
||||||
- [Database Schema](diagrams/database-schema.md) - database relationships
|
|
||||||
|
|
||||||
Authentication workflow diagrams, one per flow, under
|
|
||||||
[`diagrams/auth/`](diagrams/auth/):
|
|
||||||
|
|
||||||
- [Registration](diagrams/auth/registration.md)
|
|
||||||
- [Login](diagrams/auth/login.md)
|
|
||||||
- [Refresh Token](diagrams/auth/refresh-token.md)
|
|
||||||
- [Confirm User](diagrams/auth/confirm-user.md)
|
|
||||||
- [Resend Confirmation](diagrams/auth/resend-confirmation.md)
|
|
||||||
- [Update Username](diagrams/auth/update-username.md)
|
|
||||||
- [Update Email](diagrams/auth/update-email.md)
|
|
||||||
- [Update Password](diagrams/auth/update-password.md)
|
|
||||||
- [Update Profile](diagrams/auth/update-profile.md)
|
|
||||||
- [Delete Account](diagrams/auth/delete-account.md)
|
|
||||||
- [Error Handling](diagrams/auth/error-handling.md) - shared
|
|
||||||
`GlobalExceptionFilter` mapping used by all flows above
|
|
||||||
|
|
||||||
## Backend architecture
|
|
||||||
|
|
||||||
### Vertical-slice architecture pattern
|
|
||||||
|
|
||||||
The backend organizes business capabilities as feature slices instead of
|
|
||||||
technical layers.
|
|
||||||
|
|
||||||
Each feature (`Features.Users`, `Features.Breweries`, `Features.Emails`,
|
|
||||||
`Features.Locations`, `Features.PhotoUpload`) is a single project that owns its
|
|
||||||
own MediatR commands/queries/handlers, validators, and repository end to end.
|
|
||||||
|
|
||||||
`Features.Emails`, `Features.Locations`, and `Features.PhotoUpload` have no
|
|
||||||
`Controllers/` folder; all three are invoked internally by other slices via
|
|
||||||
MediatR/DI, never over HTTP:
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TB
|
|
||||||
subgraph HOST["API.Core (thin host)"]
|
|
||||||
direction LR
|
|
||||||
H1["Program.cs wiring:<br/>MediatR + AddApplicationPart per slice"]
|
|
||||||
H2["Swagger / OpenAPI"]
|
|
||||||
H3["JWT auth middleware"]
|
|
||||||
H4["GlobalExceptionFilter"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph SLICES["Feature Slices"]
|
|
||||||
direction LR
|
|
||||||
S1["Features.Users<br/>Controllers, Commands/Queries,<br/>Handlers, Repository"]
|
|
||||||
S2["Features.Breweries<br/>Controller, Commands/Queries,<br/>Handlers, Repository"]
|
|
||||||
S3["Features.Emails<br/>(internal only)<br/>Commands, Handlers,<br/>EmailDispatcher"]
|
|
||||||
S4["Features.Locations<br/>(internal only)<br/>Repository"]
|
|
||||||
S5["Features.PhotoUpload<br/>(internal only)<br/>Commands, Handlers,<br/>Repository"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph SHARED["Shared Projects"]
|
|
||||||
direction LR
|
|
||||||
SH1["Shared.Contracts<br/>ResponseBody envelope"]
|
|
||||||
SH2["Shared.Application<br/>ValidationBehavior"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph DOMAIN["Domain Layer"]
|
|
||||||
direction LR
|
|
||||||
D1["Domain.Entities"]
|
|
||||||
D2["Domain.Exceptions"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph INFRA["Infrastructure Layer"]
|
|
||||||
direction LR
|
|
||||||
I1["Database.Connection"]
|
|
||||||
I3["Infrastructure.PasswordHashing"]
|
|
||||||
I4["Email.Templates"]
|
|
||||||
end
|
|
||||||
|
|
||||||
DB[("Database (SQL Server)")]
|
|
||||||
|
|
||||||
HOST -- "discovers controllers via<br/>AddApplicationPart" --> SLICES
|
|
||||||
S3 == "ProjectReference<br/>(handles UserRegisteredNotification & ConfirmationEmailResendRequestedNotification)" ==> S1
|
|
||||||
S1 -- "ProjectReference<br/>(avatar upload)" --> S5
|
|
||||||
|
|
||||||
SLICES --> SHARED
|
|
||||||
SLICES --> DOMAIN
|
|
||||||
SLICES --> INFRA
|
|
||||||
|
|
||||||
INFRA --> DB
|
|
||||||
|
|
||||||
style S3 stroke-dasharray: 4 3
|
|
||||||
style S4 stroke-dasharray: 4 3
|
|
||||||
style S5 stroke-dasharray: 4 3
|
|
||||||
```
|
|
||||||
|
|
||||||
### Layer responsibilities
|
|
||||||
|
|
||||||
#### API layer (`API.Core`)
|
|
||||||
|
|
||||||
**Purpose**: Thin ASP.NET Core host: no business logic, no controllers of its
|
|
||||||
own
|
|
||||||
|
|
||||||
**Components**:
|
|
||||||
|
|
||||||
| Component | Description |
|
|
||||||
| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
||||||
| `Program.cs` | Thin composition root: builds the host and calls the `ServiceCollectionExtensions` methods below. |
|
|
||||||
| `ServiceCollectionExtensions.cs` | Registers MediatR (scanning every `Features.*` assembly), FluentValidation, and uses `AddApplicationPart` so each slice's controllers are discovered by MVC. |
|
|
||||||
| `GlobalException.cs` | Global exception filter (host-level cross-cutting concern). |
|
|
||||||
| `Authentication/JwtAuthenticationHandler.cs` | JWT auth scheme middleware. |
|
|
||||||
| Swagger/OpenAPI documentation, health check endpoints | Host-level documentation and health endpoints. |
|
|
||||||
|
|
||||||
**Dependencies**:
|
|
||||||
|
|
||||||
- Every `Features.*` project (for controller/MediatR discovery)
|
|
||||||
- `Shared.Contracts`, `Shared.Application`
|
|
||||||
|
|
||||||
**Rules**:
|
|
||||||
|
|
||||||
- No controllers, no business logic, no feature-specific contracts
|
|
||||||
- Exists purely to host and wire up the feature slices
|
|
||||||
|
|
||||||
#### Feature slices (`Features.Users`, `Features.Breweries`, `Features.Emails`, `Features.Locations`, `Features.PhotoUpload`)
|
|
||||||
|
|
||||||
**Purpose**: Each slice is the complete vertical for one business capability
|
|
||||||
|
|
||||||
**Components** (per slice):
|
|
||||||
|
|
||||||
| Component | Description |
|
|
||||||
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
||||||
| `Controllers/` | HTTP endpoints, binding directly to Command/Query types as the request contract |
|
|
||||||
| `Commands/<Operation>/` and `Queries/<Operation>/` | One folder per operation, each containing the Command/Query record, its `IRequestHandler`, and (for commands) a FluentValidation validator |
|
|
||||||
| `Repository/` | The slice's own Dapper repository implementation |
|
|
||||||
| `Dtos/` | Response shapes returned by query handlers (never the raw domain entity) |
|
|
||||||
| `DependencyInjection/` | An `AddFeaturesX()` extension method registering the slice's repository/services |
|
|
||||||
|
|
||||||
##### Note
|
|
||||||
|
|
||||||
`Features.Emails` has no `Controllers/` folder. It's invoked only via MediatR
|
|
||||||
commands sent from other slices, never over HTTP.
|
|
||||||
|
|
||||||
`Features.PhotoUpload` also has no `Controllers/` folder; its
|
|
||||||
`UploadPhotoCommand` is sent by other features' handlers (for example, a future
|
|
||||||
brewery photo upload command) rather than bound to an HTTP route directly.
|
|
||||||
|
|
||||||
**Dependencies**:
|
|
||||||
|
|
||||||
- `Domain.Entities`, `Domain.Exceptions`
|
|
||||||
- `Database.Connection` (generic ADO.NET connection/command plumbing) plus
|
|
||||||
whichever infrastructure project the slice needs (`Infrastructure.PasswordHashing`
|
|
||||||
for Users, `Infrastructure.Email.Templates` for Emails)
|
|
||||||
- `Shared.Contracts`, `Shared.Application`
|
|
||||||
- No other `Features.*` project, with two exceptions: `Features.Users`
|
|
||||||
references `Features.PhotoUpload` directly for avatar uploads, and
|
|
||||||
`Features.Emails` references `Features.Users` directly to handle
|
|
||||||
`UserRegisteredNotification` and `ConfirmationEmailResendRequestedNotification`
|
|
||||||
|
|
||||||
**Rules**:
|
|
||||||
|
|
||||||
- All business logic for that feature lives in its command/query handlers
|
|
||||||
- No direct controller-to-repository calls; everything flows through MediatR
|
|
||||||
- Read endpoints return a dedicated `Dto` rather than the domain entity
|
|
||||||
directly. The one exception is `UserController.GetAuthenticated`, which
|
|
||||||
returns the caller's own `UserAccount` entity
|
|
||||||
|
|
||||||
#### Shared projects (`Shared.Contracts`, `Shared.Application`)
|
|
||||||
|
|
||||||
**Purpose**: The minimum cross-slice surface area required because every slice
|
|
||||||
needs it, or because duplicating it four times would be worse than sharing it
|
|
||||||
|
|
||||||
**Components**:
|
|
||||||
|
|
||||||
| Project | Purpose |
|
|
||||||
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `Shared.Contracts` | `ResponseBody<T>`/`ResponseBody`, the API response envelope `AuthController`, `BreweryController`, and `CityController` return; `UserController`'s read endpoints return their DTOs unwrapped |
|
|
||||||
| `Shared.Application` | `ValidationBehavior<TRequest,TResponse>`, the MediatR pipeline behavior that runs FluentValidation before a handler executes |
|
|
||||||
|
|
||||||
**Rules**:
|
|
||||||
|
|
||||||
- Kept deliberately small: this is the exception to "no slice depends on another
|
|
||||||
slice".
|
|
||||||
|
|
||||||
#### Infrastructure layer
|
|
||||||
|
|
||||||
**Purpose**: Technical capabilities and external integrations, shared by
|
|
||||||
whichever slices need them
|
|
||||||
|
|
||||||
**Components**:
|
|
||||||
|
|
||||||
| Project | Description |
|
|
||||||
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `Database.Connection` | Generic ADO.NET connection/command plumbing (`DefaultSqlConnectionFactory`, the abstract `DapperRepository` base class), not domain-specific; each slice's own `Repository/` folder builds on this. |
|
|
||||||
| `Infrastructure.PasswordHashing` | Argon2id password hashing. |
|
|
||||||
| `Infrastructure.Email.Templates` | Email template rendering (Razor components). |
|
|
||||||
|
|
||||||
**Dependencies**:
|
|
||||||
|
|
||||||
- Domain entities
|
|
||||||
- External libraries (ADO.NET, JWT, Argon2, MailKit, AWSSDK.S3, and so on)
|
|
||||||
|
|
||||||
**Rules**:
|
|
||||||
|
|
||||||
- Implements technical concerns only, no business logic
|
|
||||||
- Reusable across slices
|
|
||||||
|
|
||||||
#### Domain layer (`Domain.Entities`)
|
|
||||||
|
|
||||||
**Purpose**: Core business entities and models
|
|
||||||
|
|
||||||
**Components**:
|
|
||||||
|
|
||||||
| Entity | Description |
|
|
||||||
| --------------------- | -------------------------------------------------------------------------------------------------- |
|
|
||||||
| `UserAccount` | User profile data. |
|
|
||||||
| `UserCredential` | Authentication credentials. |
|
|
||||||
| `UserVerification` | Account verification state. |
|
|
||||||
| `BreweryPost` | A user-submitted brewery listing. |
|
|
||||||
| `BreweryPostLocation` | A brewery's address and coordinates. |
|
|
||||||
| `City` | A city referenced by a brewery location. |
|
|
||||||
| `StateProvince` | A state/province referenced by a city. |
|
|
||||||
| `Country` | A country referenced by a state/province. |
|
|
||||||
| `BeerStyle` | A beer style/category (schema table exists; no repository yet). |
|
|
||||||
| `BeerPost` | A beer listing tied to a brewery and style (schema table exists; no repository yet). |
|
|
||||||
| `Photo` | A photo uploaded by a user account, written via `Features.PhotoUpload`'s `IPhotoUploadRepository`. |
|
|
||||||
|
|
||||||
**Dependencies**:
|
|
||||||
|
|
||||||
- None (pure domain)
|
|
||||||
|
|
||||||
**Rules**:
|
|
||||||
|
|
||||||
- Plain Old CLR Objects (POCOs)
|
|
||||||
- No framework dependencies
|
|
||||||
- No infrastructure references
|
|
||||||
- Represents business concepts
|
|
||||||
|
|
||||||
### Design patterns
|
|
||||||
|
|
||||||
#### Vertical slice + MediatR
|
|
||||||
|
|
||||||
**Purpose**: Organize code by feature instead of by technical layer, so
|
|
||||||
everything needed to understand or change one capability lives in one project
|
|
||||||
|
|
||||||
**Implementation**:
|
|
||||||
|
|
||||||
- Each HTTP write operation is a `Command` (for example, `CreateBreweryCommand`
|
|
||||||
in `Features.Breweries/Commands/CreateBrewery/`); each read operation is a
|
|
||||||
`Query` (for example, `GetBreweryByIdQuery`)
|
|
||||||
- Controllers generally bind directly to the Command/Query as the request body;
|
|
||||||
there is no separate request DTO + mapping step for writes
|
|
||||||
- The exception to this is for authenticated routes, when the current user id
|
|
||||||
needs to be extracted from the authentication token to be used in a command
|
|
||||||
or query.
|
|
||||||
- A single shared `ValidationBehavior<TRequest,TResponse>`
|
|
||||||
(`Shared.Application/Behaviors/`) runs FluentValidation validators in the
|
|
||||||
MediatR pipeline before any handler executes
|
|
||||||
- Query handlers map to a dedicated response `Dto`, so domain entities never
|
|
||||||
leak
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```csharp
|
|
||||||
public record CreateBreweryCommand(Guid PostedById, string BreweryName, string Description, ...)
|
|
||||||
: IRequest<BreweryDto>;
|
|
||||||
|
|
||||||
public class CreateBreweryHandler(IBreweryRepository repository) : IRequestHandler<CreateBreweryCommand, BreweryDto>
|
|
||||||
{
|
|
||||||
public async Task<BreweryDto> Handle(CreateBreweryCommand request, CancellationToken cancellationToken) { ... }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Repository pattern
|
|
||||||
|
|
||||||
**Purpose**: Abstract database access behind interfaces
|
|
||||||
|
|
||||||
**Implementation**: each slice owns its own repository, scoped to that feature
|
|
||||||
only:
|
|
||||||
|
|
||||||
- `Features.Users/Repository/IUserListRepository.cs`,
|
|
||||||
`Features.Users/Repository/IUserProfileRepository.cs`
|
|
||||||
- `Features.Breweries/Repository/IBreweryRepository.cs`
|
|
||||||
- `Features.PhotoUpload/Repository/IPhotoUploadRepository.cs`
|
|
||||||
- `Database.Connection/DefaultSqlConnectionFactory.cs`: the generic connection
|
|
||||||
factory every slice's repository builds on
|
|
||||||
- `Database.Connection/DapperRepository.cs`: shared abstract base class every
|
|
||||||
repository extends, providing `CreateConnection()` and a
|
|
||||||
`RollbackQuietlyAsync()` helper that rolls back a transaction while swallowing
|
|
||||||
any exception the rollback itself raises, so the exception that triggered the
|
|
||||||
rollback is what propagates
|
|
||||||
|
|
||||||
**Benefits**:
|
|
||||||
|
|
||||||
- Testable (easy to mock)
|
|
||||||
- Each slice's data access logic is self-contained
|
|
||||||
|
|
||||||
**Example**:
|
|
||||||
|
|
||||||
```csharp
|
|
||||||
public interface IUserListRepository
|
|
||||||
{
|
|
||||||
Task<UserAccount?> GetByIdAsync(Guid id);
|
|
||||||
Task<IEnumerable<UserAccount>> GetAllAsync(int? limit, int? offset);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Dependency injection
|
|
||||||
|
|
||||||
**Purpose**: Loose coupling and testability
|
|
||||||
|
|
||||||
**Configuration**: `Program.cs` wires up MediatR/FluentValidation across every
|
|
||||||
`Features.*` assembly; each slice exposes its own `AddFeaturesX()` extension
|
|
||||||
method that registers its repository and slice-internal services
|
|
||||||
|
|
||||||
**Lifetimes**:
|
|
||||||
|
|
||||||
- Scoped: Repositories, slice-internal services (per request)
|
|
||||||
- Singleton: `DefaultSqlConnectionFactory`
|
|
||||||
- Transient: Utilities, helpers
|
|
||||||
|
|
||||||
#### Direct SQL via Dapper/ADO.NET
|
|
||||||
|
|
||||||
**Purpose**: Keep data access explicit and colocated with the slice that owns it
|
|
||||||
|
|
||||||
**Strategy**:
|
|
||||||
|
|
||||||
- Each repository issues inline SQL through Dapper, which handles parameter
|
|
||||||
binding.
|
|
||||||
- Proximity search (`GET /api/brewery/locations/nearby`) filters and orders rows
|
|
||||||
server-side with `Coordinates.STDistance(@Origin) <= @RangeInMetres`, avoiding
|
|
||||||
a client-side distance calculation over every row.
|
|
||||||
- Referential checks (for example, "does this `CityId` exist?") are explicit
|
|
||||||
`SELECT 1 ...` existence checks in the repository method, run inside the same
|
|
||||||
transaction as the write
|
|
||||||
- Optimistic concurrency uses each table's `RowVersion` (`ROWVERSION`) column,
|
|
||||||
checked in the `UPDATE ... WHERE ... AND RowVersion = @RowVersion` clause
|
|
||||||
- Repositories throw `Domain.Exceptions` types (`NotFoundException`,
|
|
||||||
`ConflictException`, ...) directly when a check fails; `API.Core`'s
|
|
||||||
`GlobalExceptionFilter` maps those exception types to HTTP status codes
|
|
||||||
- Application focuses on orchestration; the database enforces integrity via
|
|
||||||
keys, `CHECK` constraints, and cascades
|
|
||||||
|
|
||||||
See [Database](database.md) for the schema and the app's SQL error-handling
|
|
||||||
approach in more detail.
|
|
||||||
|
|
||||||
## Frontend architecture
|
|
||||||
|
|
||||||
### Website (`web/frontend`)
|
|
||||||
|
|
||||||
The website is a React Router 8 application with server-side rendering enabled.
|
|
||||||
|
|
||||||
```text
|
|
||||||
web/frontend/
|
|
||||||
├── app/
|
|
||||||
│ ├── components/ Shared UI: Navbar, FormField, SubmitButton, ToastProvider,
|
|
||||||
│ │ RouteErrorState, ClientOnly (defers client-only children,
|
|
||||||
│ │ for example Leaflet maps, past hydration)
|
|
||||||
│ ├── features/ One folder per feature slice, each owning its own
|
|
||||||
│ │ routes/, components/, hooks/, schemas, and server-side
|
|
||||||
│ │ loader helpers:
|
|
||||||
│ │ - account profile/dashboard management
|
|
||||||
│ │ - auth login, register, confirm, logout
|
|
||||||
│ │ - breweries index/directory/detail routes, Leaflet
|
|
||||||
│ │ map components, nearby-search resource
|
|
||||||
│ │ route, distance/address utils
|
|
||||||
│ │ - catalog beers, beer styles
|
|
||||||
│ │ - home landing page
|
|
||||||
│ │ - theme theme switcher/guide
|
|
||||||
│ │ - users public user profile page (/users/:id)
|
|
||||||
│ ├── hooks/ Shared hooks not tied to one feature, for example useMediaQuery
|
|
||||||
│ ├── routes.ts Route table (React Router 8 config-based routing)
|
|
||||||
│ ├── root.tsx App shell and global providers
|
|
||||||
│ └── app.css Theme tokens and global styling
|
|
||||||
├── .storybook/ Storybook config and preview setup
|
|
||||||
├── stories/ Storybook stories for shared UI and themes
|
|
||||||
├── tests/playwright/ Storybook Playwright coverage
|
|
||||||
└── package.json Frontend scripts and dependencies
|
|
||||||
```
|
|
||||||
|
|
||||||
### Frontend responsibilities
|
|
||||||
|
|
||||||
- Render brewery discovery routes (`/breweries` index, `/breweries/directory`,
|
|
||||||
`/breweries/:id`) with Leaflet-powered maps and geolocation-based "nearby"
|
|
||||||
search
|
|
||||||
- Render the auth demo, account dashboard, and theme guide routes
|
|
||||||
- Render the public user profile route (`/users/:id`), including a user's
|
|
||||||
following list and (via brewery comments) links back to commenter profiles
|
|
||||||
- Manage cookie-backed website session state
|
|
||||||
- Call the .NET API for login, registration, token refresh, confirmation, and
|
|
||||||
brewery/location data
|
|
||||||
- Provide shared UI building blocks for forms, navigation, themes, and toasts
|
|
||||||
- Supply Storybook documentation and browser-based component verification
|
|
||||||
|
|
||||||
### Theme system
|
|
||||||
|
|
||||||
The website uses semantic DaisyUI theme tokens backed by four Biergarten themes:
|
|
||||||
|
|
||||||
- Biergarten Lager
|
|
||||||
- Biergarten Stout
|
|
||||||
- Biergarten Cassis
|
|
||||||
- Biergarten Weizen
|
|
||||||
|
|
||||||
All component styling should prefer semantic tokens such as `primary`,
|
|
||||||
`success`, `surface`, and `highlight` instead of hard-coded color values.
|
|
||||||
|
|
||||||
### Archived frontend
|
|
||||||
|
|
||||||
`archive/next-js-web-app/` contains an archived Next.js frontend, kept for
|
|
||||||
reference. Product and engineering documentation points to `web/frontend`.
|
|
||||||
|
|
||||||
## Security architecture
|
|
||||||
|
|
||||||
### Authentication flow
|
|
||||||
|
|
||||||
| Flow | Details |
|
|
||||||
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| **Registration** | User submits credentials; password is hashed with Argon2id; user account is created as unverified and a confirmation email is dispatched via `Features.Emails`. |
|
|
||||||
| **Email confirmation** | User follows the confirmation link/token from the email; `Features.Users`' `ConfirmUser` command marks the account as verified. |
|
|
||||||
| **Login** | User submits credentials; password is verified against the hash; access and refresh JWTs are issued and returned to the client; neither is stored server-side. |
|
|
||||||
| **Token refresh** | Client exchanges a valid, unexpired refresh token for a new access/refresh pair via `Features.Users`' `RefreshToken` command; refresh tokens are stateless JWTs, not tracked server-side, and the previous refresh token remains usable until its own expiration. |
|
|
||||||
| **API requests** | Client sends the access JWT in the `Authorization` header; `JwtAuthenticationHandler` validates the token; request proceeds if valid. |
|
|
||||||
|
|
||||||
### Password security
|
|
||||||
|
|
||||||
**Algorithm**: Argon2id
|
|
||||||
|
|
||||||
| Setting | Value |
|
|
||||||
| ----------- | ---------------------------------------------- |
|
|
||||||
| Memory | 64MB |
|
|
||||||
| Iterations | 4 |
|
|
||||||
| Parallelism | 2 (hardcoded, to avoid thread-pool exhaustion) |
|
|
||||||
| Salt | 128-bit (16 bytes) |
|
|
||||||
| Hash | 256-bit (32 bytes) |
|
|
||||||
|
|
||||||
### JWT tokens
|
|
||||||
|
|
||||||
**Algorithm**: HS256 (HMAC-SHA256)
|
|
||||||
|
|
||||||
**Claims**:
|
|
||||||
|
|
||||||
| Claim | Meaning |
|
|
||||||
| ------------- | -------------------- |
|
|
||||||
| `sub` | User ID |
|
|
||||||
| `unique_name` | Username |
|
|
||||||
| `jti` | Unique token ID |
|
|
||||||
| `iat` | Issued at timestamp |
|
|
||||||
| `exp` | Expiration timestamp |
|
|
||||||
|
|
||||||
**Configuration** (appsettings.json):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"Jwt": {
|
|
||||||
"ExpirationMinutes": 60,
|
|
||||||
"Issuer": "biergarten-api",
|
|
||||||
"Audience": "biergarten-users"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This block exists in `appsettings.json`, but nothing in the code currently reads
|
|
||||||
it: actual token lifetimes come from the hardcoded `TokenServiceExpirationHours`
|
|
||||||
constants, and `JwtInfrastructure` hardcodes `ValidateIssuer`/`ValidateAudience`
|
|
||||||
to `false`, so `Issuer`/`Audience` are not enforced.
|
|
||||||
|
|
||||||
## Database architecture
|
|
||||||
|
|
||||||
For the table-by-table schema and the custom SQL error code scheme, see
|
|
||||||
[Database](database.md).
|
|
||||||
|
|
||||||
### SQL-first philosophy
|
|
||||||
|
|
||||||
**Principles**:
|
|
||||||
|
|
||||||
1. Database is source of truth
|
|
||||||
2. Queries live in the owning repository, not scattered across the app
|
|
||||||
3. Database handles referential integrity
|
|
||||||
4. Application orchestrates, database executes
|
|
||||||
|
|
||||||
**Benefits**:
|
|
||||||
|
|
||||||
- Performance optimization via execution plans
|
|
||||||
- Query logic scoped to the repository that owns it, not spread across layers
|
|
||||||
- Version-controlled schema (migrations)
|
|
||||||
- Easier query profiling and tuning
|
|
||||||
|
|
||||||
### Migration strategy
|
|
||||||
|
|
||||||
**Tool**: DbUp
|
|
||||||
|
|
||||||
**Process**:
|
|
||||||
|
|
||||||
1. Write a SQL script under `Database.Migrations/scripts/`
|
|
||||||
2. Embed it in the `Database.Migrations` project
|
|
||||||
3. `Database.Migrations` runs each script against the target database on
|
|
||||||
startup, tracked so a script never re-runs once applied
|
|
||||||
|
|
||||||
**Migration Files**:
|
|
||||||
|
|
||||||
```
|
|
||||||
scripts/
|
|
||||||
└── 01-schema/
|
|
||||||
├── 01-UserAccount.sql
|
|
||||||
├── 02-Photo.sql
|
|
||||||
├── ...
|
|
||||||
├── 16-UserProfile.sql
|
|
||||||
└── 17-UserAvatar.sql # one script per CREATE TABLE
|
|
||||||
```
|
|
||||||
|
|
||||||
Each table has its own script, numbered so a table's foreign keys always point
|
|
||||||
at an earlier-numbered script. As the schema evolves, new numbered scripts get
|
|
||||||
added rather than editing an existing one in place, so DbUp's already-applied
|
|
||||||
tracking stays valid.
|
|
||||||
|
|
||||||
### Data seeding
|
|
||||||
|
|
||||||
**Purpose**: Populate development/test databases with realistic data
|
|
||||||
|
|
||||||
**Implementation**: `Database.Seed` project, using data produced by the C++
|
|
||||||
pipeline under `tooling/pipeline/` (see
|
|
||||||
[Pipeline README](../pipeline/README.md))
|
|
||||||
|
|
||||||
**Seed Data**:
|
|
||||||
|
|
||||||
- Countries, state/provinces, and cities (via `Features.Locations`'
|
|
||||||
`ILocationRepository`, which creates `Country`/`StateProvince` rows on demand
|
|
||||||
while resolving a city)
|
|
||||||
- User accounts (via `Features.Users`' repository)
|
|
||||||
- Brewery posts with locations (via `Features.Breweries`' repository)
|
|
||||||
|
|
||||||
Tables that exist in the schema but have no repository yet ( `UserFollow`,
|
|
||||||
`BeerStyle`, `BeerPost`, `BeerPostPhoto`, `BeerPostComment`, `BreweryPostPhoto`)
|
|
||||||
are not seeded either.
|
|
||||||
|
|
||||||
## Deployment architecture
|
|
||||||
|
|
||||||
### Docker containerization
|
|
||||||
|
|
||||||
**Container Structure**:
|
|
||||||
|
|
||||||
| Container | Purpose |
|
|
||||||
| --------------------- | ------------------------------------------------------------ |
|
|
||||||
| `sqlserver` | SQL Server 2022 |
|
|
||||||
| `database.migrations` | Schema migration runner |
|
|
||||||
| `database.seed` | Data seeder |
|
|
||||||
| `api.core` | ASP.NET Core Web API |
|
|
||||||
| `mailpit` | Local dev/test SMTP server + web UI (not used in production) |
|
|
||||||
| `seaweedfs` | S3-compatible object storage for uploaded photos (dev only) |
|
|
||||||
|
|
||||||
**Environments**:
|
|
||||||
|
|
||||||
| Environment | Compose file |
|
|
||||||
| --------------- | ------------------------------- |
|
|
||||||
| Development | `docker-compose.dev.yaml` |
|
|
||||||
| Testing | `docker-compose.test.yaml` |
|
|
||||||
| Production | `docker-compose.prod.yaml` |
|
|
||||||
| Production seed | `docker-compose.prod.seed.yaml` |
|
|
||||||
|
|
||||||
For details, see [Docker Guide](docker.md).
|
|
||||||
|
|
||||||
### Health checks
|
|
||||||
|
|
||||||
- **SQL Server**: Validates database connectivity
|
|
||||||
- **API**: Checks service health and dependencies
|
|
||||||
|
|
||||||
**Configuration**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD-SHELL", "sqlcmd health check"]
|
|
||||||
interval: 10s
|
|
||||||
retries: 12
|
|
||||||
start_period: 30s
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing architecture
|
|
||||||
|
|
||||||
### Test pyramid
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TB
|
|
||||||
IT["Integration Tests<br/>API.Specs (Reqnroll)"]
|
|
||||||
UT["Unit Tests (per slice, handler tests)<br/>Features.Users.Tests, Features.Breweries.Tests,<br/>Features.Emails.Tests, Features.PhotoUpload.Tests<br/>(commands/queries/handlers, with repository/service<br/>dependencies mocked via Moq)"]
|
|
||||||
|
|
||||||
IT --- UT
|
|
||||||
```
|
|
||||||
|
|
||||||
**Strategy**:
|
|
||||||
|
|
||||||
- Many unit tests (fast, isolated)
|
|
||||||
- Fewer integration tests (slower, e2e)
|
|
||||||
- Mock external dependencies
|
|
||||||
- Test database for integration tests
|
|
||||||
|
|
||||||
For details, see [Testing Guide](testing.md).
|
|
||||||
@@ -1,143 +0,0 @@
|
|||||||
---
|
|
||||||
title: SQL Server database schema, migrations, and error handling
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- sql
|
|
||||||
- repository
|
|
||||||
- database
|
|
||||||
- sql-server
|
|
||||||
- schema
|
|
||||||
---
|
|
||||||
|
|
||||||
SQL Server 2022 schema, migration approach, and the app's SQL error-handling
|
|
||||||
convention. For the vertical-slice code that talks to this schema, see
|
|
||||||
[Architecture](architecture.md).
|
|
||||||
|
|
||||||
## Schema
|
|
||||||
|
|
||||||
The schema is split into six SQL Server schemas, one per feature area, each
|
|
||||||
living in its own numbered folder under
|
|
||||||
`web/backend/Database/Database.Migrations/scripts/`. The folder numbers
|
|
||||||
(`01`-`06`) fix the load order so foreign-key dependencies resolve — DbUp runs
|
|
||||||
embedded scripts in alphabetical-by-resource-name order, so e.g. `04-Brewery`
|
|
||||||
(which references `Auth`, `Geolocation`, and `Media`) runs after all three.
|
|
||||||
Each folder's `00-Schema.sql` creates its SQL schema before its tables run:
|
|
||||||
|
|
||||||
```
|
|
||||||
scripts
|
|
||||||
├── 01-Auth (UserAccount, UserVerification, UserCredential)
|
|
||||||
│ ├── 00-Schema.sql
|
|
||||||
│ ├── 01-UserAccount.sql
|
|
||||||
│ ├── 02-UserVerification.sql
|
|
||||||
│ └── 03-UserCredential.sql
|
|
||||||
├── 02-Geolocation (Country, StateProvince, City)
|
|
||||||
│ ├── 00-Schema.sql
|
|
||||||
│ ├── 01-Country.sql
|
|
||||||
│ ├── 02-StateProvince.sql
|
|
||||||
│ └── 03-City.sql
|
|
||||||
├── 03-Media (Photo)
|
|
||||||
│ ├── 00-Schema.sql
|
|
||||||
│ └── 01-Photo.sql
|
|
||||||
├── 04-Brewery (BreweryPost, BreweryPostLocation, BreweryPostPhoto)
|
|
||||||
│ ├── 00-Schema.sql
|
|
||||||
│ ├── 01-BreweryPost.sql
|
|
||||||
│ ├── 02-BreweryPostLocation.sql
|
|
||||||
│ └── 03-BreweryPostPhoto.sql
|
|
||||||
├── 05-Beer (BeerStyle, BeerPost, BeerPostPhoto, BeerPostComment)
|
|
||||||
│ ├── 00-Schema.sql
|
|
||||||
│ ├── 01-BeerStyle.sql
|
|
||||||
│ ├── 02-BeerPost.sql
|
|
||||||
│ ├── 03-BeerPostPhoto.sql
|
|
||||||
│ └── 04-BeerPostComment.sql
|
|
||||||
└── 06-Social (UserFollow, UserProfile, UserAvatar)
|
|
||||||
├── 00-Schema.sql
|
|
||||||
├── 01-UserFollow.sql
|
|
||||||
├── 02-UserProfile.sql
|
|
||||||
└── 03-UserAvatar.sql
|
|
||||||
```
|
|
||||||
|
|
||||||
Each script is run in order by `Database.Migrations` (DbUp) on startup. Tables
|
|
||||||
are referenced schema-qualified in code (e.g. `Auth.UserAccount`), not under
|
|
||||||
the default `dbo` schema. Each table is queried directly by the owning
|
|
||||||
feature slice's repository (see
|
|
||||||
[Direct SQL via Dapper/ADO.NET](architecture.md#direct-sql-via-dapperadonet)).
|
|
||||||
|
|
||||||
A generated DBML copy of the schema (for import into a diagramming tool such as
|
|
||||||
dbdiagram.io) lives at `web/backend/Database/Database.Migrations/schema.dbml`.
|
|
||||||
|
|
||||||
All tables use a `UNIQUEIDENTIFIER` primary key defaulting to `NEWID()`, and
|
|
||||||
most carry a `RowVersion ROWVERSION` column used for optimistic concurrency on
|
|
||||||
updates (`UPDATE ... WHERE ... AND RowVersion = @RowVersion`).
|
|
||||||
|
|
||||||
### Tables backed by the API today
|
|
||||||
|
|
||||||
| Table | Backed by | Notes |
|
|
||||||
| --------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `UserAccount` | `Domain.Entities.UserAccount`, `Features.Users` | `Username` and `Email` are unique |
|
|
||||||
| `UserCredential` | `Domain.Entities.UserCredential`, `Features.Users` | Argon2id password hash; `IsRevoked`/`RevokedAt` track credential rotation, not JWTs |
|
|
||||||
| `UserVerification` | `Domain.Entities.UserVerification`, `Features.Users` | Written when a user confirms their email |
|
|
||||||
| `UserProfile` | `Domain.Entities.UserProfile`, `Features.Users` | 1:1 with `UserAccount`; holds the user's biography; written via `IUserProfileRepository`, read via `IUserListRepository`'s left joins in both `GetByIdAsync` and `GetAllAsync` |
|
|
||||||
| `UserAvatar` | `Domain.Entities.UserAvatar`, `Features.Users` | Links a `UserProfile` to a `Photo` as its avatar; written via `IUserProfileRepository.SaveAvatarAsync`, read via `IUserListRepository`'s `ValidTo IS NULL` join for the active avatar |
|
|
||||||
| `BreweryPost` | `Domain.Entities.BreweryPost`, `Features.Breweries` | A user-submitted brewery listing |
|
|
||||||
| `BreweryPostLocation` | `Domain.Entities.BreweryPostLocation`, `Features.Breweries` | 1:1 with `BreweryPost`; `Coordinates` is a `GEOGRAPHY` column, read via a plain Dapper query that selects `CONVERT(varbinary(max), Coordinates)`, since Dapper can't deserialize SQL Server UDTs directly |
|
|
||||||
| `City` | `Domain.Entities.City`, `Features.Locations` | Referenced by `BreweryPostLocation.CityID`; `CityName` and `StateProvinceID` together are unique; written via `ILocationRepository`, read via `ILocationRepository` and `IBreweryRepository`'s location joins, currently written only by `Database.Seed` |
|
|
||||||
| `StateProvince` | `Domain.Entities.StateProvince`, `Features.Locations` | Referenced by `City.StateProvinceID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins |
|
|
||||||
| `Country` | `Domain.Entities.Country`, `Features.Locations` | Referenced by `StateProvince.CountryID`; created on demand by `ILocationRepository.GetOrCreateCityIdAsync`, read via `IBreweryRepository`'s location joins |
|
|
||||||
| `Photo` | `Domain.Entities.Photo`, `Features.PhotoUpload` | Written via `IPhotoUploadRepository.CreateAsync`, sent as an `UploadPhotoCommand` from other features' handlers; `Hyperlink` holds the object key within the bucket, which readers sign into a time-limited URL via `IFileStorageProvider.GetPresignedUrl`; the active avatar's row is also read via `IUserListRepository`'s `Media.Photo` join; not yet seeded by `Database.Seed` |
|
|
||||||
|
|
||||||
### Tables defined in the schema but not yet wired to a feature slice
|
|
||||||
|
|
||||||
These exist so the schema can support planned functionality, but have no
|
|
||||||
repository or seed data yet:
|
|
||||||
|
|
||||||
| Table | Purpose (from schema) |
|
|
||||||
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `UserFollow` | One user following another (`CannotFollowOwnAccount` check constraint) |
|
|
||||||
| `BeerStyle` | A beer style/category; has a `Domain.Entities.BeerStyle` type, but no repository yet |
|
|
||||||
| `BeerPost` | A beer listing, linked to a `BreweryPost` and `BeerStyle`; `ABV`/`IBU` are constrained (`0-67`, `0-120`); has a `Domain.Entities.BeerPost` type, but no repository yet |
|
|
||||||
| `BeerPostPhoto` | Links a `BeerPost` to a `Photo` |
|
|
||||||
| `BeerPostComment` | A comment + 1-5 `Rating` on a `BeerPost` |
|
|
||||||
| `BreweryPostPhoto` | Links a `BreweryPost` to a `Photo` |
|
|
||||||
|
|
||||||
## Managing Migrations (DbUp)
|
|
||||||
|
|
||||||
Migrations are handled by the `Database.Migrations` container using DbUp.
|
|
||||||
Scripts are placed in the `scripts/` directory, applied in order, and tracked so
|
|
||||||
that each one runs only once.
|
|
||||||
|
|
||||||
**Note:** The schema is currently considered volatile. At this stage of
|
|
||||||
development, fresh database instances are provisioned for both development and
|
|
||||||
testing, so no production data exists yet.
|
|
||||||
|
|
||||||
### How to Make Schema Changes
|
|
||||||
|
|
||||||
- **Now (no production data):** Update the schema by editing the original DDL
|
|
||||||
statements directly.
|
|
||||||
- **Later (with production data):** Add a new, numbered script for any schema
|
|
||||||
changes. Never edit an existing script in place — DbUp tracks completion by
|
|
||||||
script name, so modifying an already-applied script means the changes will be
|
|
||||||
ignored on existing databases.
|
|
||||||
|
|
||||||
## Data seeding
|
|
||||||
|
|
||||||
`Database.Seed` populates a target database from data produced by the C++
|
|
||||||
pipeline (see [Pipeline README](../pipeline/README.md)): countries,
|
|
||||||
state/provinces, and cities (created on demand while resolving each brewery's
|
|
||||||
location), user accounts, and brewery posts with locations. It doesn't call
|
|
||||||
every feature slice that has a write path: `Photo`, for example, has a
|
|
||||||
repository (`Features.PhotoUpload`) but no seed data.
|
|
||||||
|
|
||||||
## Database error handling
|
|
||||||
|
|
||||||
- Optimistic-concurrency conflicts are detected by an `UPDATE` affecting zero
|
|
||||||
rows against the `RowVersion` column.
|
|
||||||
- On failure, the repository throws a `Domain.Exceptions` type directly
|
|
||||||
(`NotFoundException`, `ConflictException`, `ArgumentException`, ...).
|
|
||||||
`API.Core`'s `GlobalExceptionFilter` maps these to HTTP status codes (404,
|
|
||||||
409, 400, ...); an unmapped `SqlException` (for example, a genuine
|
|
||||||
connectivity or constraint-violation failure) maps to 503.
|
|
||||||
|
|
||||||
- One exception: `Features.Users`' `DapperUserStore` does catch `SqlException`
|
|
||||||
directly, to detect a duplicate-key race (SQL Server error 2601 or 2627)
|
|
||||||
when two requests try to verify the same account concurrently; see
|
|
||||||
`DapperUserStore.IsDuplicateKeyViolation`.
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Confirm User
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/confirm-user.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant ConfirmHandler as ConfirmUserHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
participant TokenSvc as TokenService
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant JWT as JWT Infrastructure
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: POST /api/auth/confirm?token={token} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(ConfirmUserCommand(token))
|
|
||||||
activate Mediator
|
|
||||||
Mediator->>ConfirmHandler: Handle(command)
|
|
||||||
activate ConfirmHandler
|
|
||||||
|
|
||||||
ConfirmHandler->>TokenSvc: ValidateConfirmationTokenAsync(token)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: Validate against confirmation token secret, extract userId
|
|
||||||
activate JWT
|
|
||||||
JWT-->>TokenSvc: ValidatedToken or throws
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>ConfirmHandler: ValidatedToken
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
alt Token invalid or expired
|
|
||||||
ConfirmHandler->>Mediator: throw UnauthorizedException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else Token valid
|
|
||||||
ConfirmHandler->>UserMgr: FindByIdAsync(validatedToken.UserId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userId)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: row or null
|
|
||||||
deactivate DB
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>ConfirmHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt User not found
|
|
||||||
ConfirmHandler->>Mediator: throw UnauthorizedException "User account not found"
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else User found
|
|
||||||
ConfirmHandler->>UserStore: SetEmailConfirmedAsync(user, true)
|
|
||||||
Note right of UserStore: IUserEmailStore(ApplicationUser) is injected directly into the handler (not via UserManager), setting the confirmed flag in-memory here
|
|
||||||
|
|
||||||
ConfirmHandler->>UserMgr: UpdateAsync(user)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: UpdateAsync(user)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: UPDATE UserAccount SET ...
|
|
||||||
UserStore->>DB: INSERT INTO UserVerification (UserAccountId, VerificationDateTime)
|
|
||||||
Note right of UserStore: Idempotent — skipped (duplicate-key exception swallowed) if a verification row already exists
|
|
||||||
UserStore-->>UserMgr: IdentityResult.Success
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>ConfirmHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
ConfirmHandler-->>Mediator: ConfirmationPayload(user.Id, DateTime.UtcNow)
|
|
||||||
deactivate ConfirmHandler
|
|
||||||
Mediator-->>API: ConfirmationPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "User with ID {id} is confirmed.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Delete Account
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/delete-account.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant DeleteAccountHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: DELETE /api/auth/account (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(DeleteAccountCommand(GetAuthenticatedUserId()))
|
|
||||||
activate Mediator
|
|
||||||
Mediator->>DeleteAccountHandler: Handle(command)
|
|
||||||
activate DeleteAccountHandler
|
|
||||||
|
|
||||||
DeleteAccountHandler->>UserMgr: FindByIdAsync(userAccountId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userAccountId)
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
UserMgr-->>DeleteAccountHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account not found
|
|
||||||
DeleteAccountHandler->>Mediator: throw NotFoundException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 404 Not Found
|
|
||||||
else Account found
|
|
||||||
DeleteAccountHandler->>UserMgr: DeleteAsync(user)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: DeleteAsync(user)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: DELETE FROM UserAccount WHERE UserAccountID = @Id
|
|
||||||
Note right of UserStore: UserCredential, UserVerification, and UserProfile cascade on delete<br/>(UserAvatar cascades transitively through UserProfile).<br/>Posts, comments, photos, and follows use ON DELETE NO ACTION<br/>if any reference this account, the DELETE throws a foreign key violation (SQL error 547)
|
|
||||||
|
|
||||||
alt Foreign key violation (dependent records exist)
|
|
||||||
UserStore-->>UserMgr: IdentityResult.Failed (AccountHasDependentRecords)
|
|
||||||
else Delete succeeds
|
|
||||||
UserStore-->>UserMgr: IdentityResult.Success
|
|
||||||
end
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>DeleteAccountHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account has dependent records
|
|
||||||
DeleteAccountHandler->>Mediator: throw ConflictException<br>"Account cannot be deleted while it still has associated posts, comments, photos, or follows."
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else !result.Succeeded (other failure)
|
|
||||||
DeleteAccountHandler->>Mediator: throw ConflictException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else result.Succeeded
|
|
||||||
DeleteAccountHandler-->>Mediator: (void)
|
|
||||||
deactivate DeleteAccountHandler
|
|
||||||
Mediator-->>API: (void)
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Account deleted successfully."}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
# User Authentication Flow — Error Handling
|
|
||||||
|
|
||||||
Global exception mapping performed by `API.Core`'s `GlobalExceptionFilter`.
|
|
||||||
Applies uniformly across every authentication endpoint (registration, login,
|
|
||||||
refresh, confirm, resend, update-username/email/password/profile,
|
|
||||||
delete-account) — see the per-flow diagrams for where each exception type is
|
|
||||||
thrown.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/error-handling.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
participant API as API Controller
|
|
||||||
|
|
||||||
Note over API: GlobalExceptionFilter catches:
|
|
||||||
Note over API: FluentValidation.ValidationException → 400 Bad Request {message, errors} (grouped by property)
|
|
||||||
Note over API: ConflictException → 409 Conflict
|
|
||||||
Note over API: NotFoundException → 404 Not Found
|
|
||||||
Note over API: UnauthorizedException → 401 Unauthorized
|
|
||||||
Note over API: ForbiddenException → 403 Forbidden
|
|
||||||
Note over API: Domain.Exceptions.ValidationException → 400 Bad Request
|
|
||||||
Note over API: Microsoft.Data.SqlClient.SqlException → 503 Service Unavailable
|
|
||||||
Note over API: All others → 500 Internal Server Error
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Login
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/login.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,93 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant LoginHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
participant TokenSvc as TokenService
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant Argon2 as Argon2PasswordHasher
|
|
||||||
participant JWT as JWT Infrastructure
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: POST /api/auth/login {username, password}
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(LoginCommand)
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs LoginValidator — username and password must both be non-empty
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request {message, errors}
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>LoginHandler: Handle(command)
|
|
||||||
activate LoginHandler
|
|
||||||
|
|
||||||
LoginHandler->>UserMgr: FindByNameAsync(username)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByNameAsync(username)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE Username = @Username
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: UserAccount row or null
|
|
||||||
deactivate DB
|
|
||||||
UserStore->>UserStore: Hydrate (active UserCredential.Hash, UserVerification → EmailConfirmed)
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>LoginHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt User not found
|
|
||||||
LoginHandler->>Mediator: throw UnauthorizedException "Invalid username or password."
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else User found
|
|
||||||
LoginHandler->>UserMgr: CheckPasswordAsync(user, password)
|
|
||||||
activate UserMgr
|
|
||||||
Note right of UserMgr: HasPasswordAsync short-circuits to false (no active credential hydrated) without calling the hasher — otherwise delegates to Argon2PasswordHasher
|
|
||||||
UserMgr->>Argon2: VerifyHashedPassword(user, user.PasswordHash, password)
|
|
||||||
activate Argon2
|
|
||||||
Note right of Argon2: Parse stored PHC string ($argon2id$v=19$m=...,t=...,p=...$salt$hash), rehash password with the embedded salt/parameters, constant-time compare (CryptographicOperations.FixedTimeEquals)
|
|
||||||
Argon2-->>UserMgr: Success/Failed
|
|
||||||
deactivate Argon2
|
|
||||||
UserMgr-->>LoginHandler: true/false
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Password invalid
|
|
||||||
LoginHandler->>Mediator: throw UnauthorizedException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else Password valid
|
|
||||||
LoginHandler->>TokenSvc: GenerateAccessToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: GenerateJwt(...)
|
|
||||||
activate JWT
|
|
||||||
JWT-->>TokenSvc: Access Token
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>LoginHandler: Access Token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
LoginHandler->>TokenSvc: GenerateRefreshToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: GenerateJwt(...)
|
|
||||||
activate JWT
|
|
||||||
JWT-->>TokenSvc: Refresh Token
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>LoginHandler: Refresh Token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
LoginHandler-->>Mediator: LoginPayload(userAccountId, username, refreshToken, accessToken)
|
|
||||||
deactivate LoginHandler
|
|
||||||
Mediator-->>API: LoginPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Logged in successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Refresh Token
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/refresh-token.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant RefreshHandler as RefreshTokenHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
participant TokenSvc as TokenService
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant JWT as JWT Infrastructure
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: POST /api/auth/refresh<br/>Header: X-Refresh-Token: {token}
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(RefreshTokenCommand(refreshToken))
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs RefreshTokenValidator — refreshToken must be non-empty. Anonymous endpoint, no Bearer token required.
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request {message, errors}
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>RefreshHandler: Handle(command)
|
|
||||||
activate RefreshHandler
|
|
||||||
|
|
||||||
RefreshHandler->>TokenSvc: RefreshTokenAsync(refreshToken)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: ValidateRefreshTokenAsync(refreshToken)
|
|
||||||
activate JWT
|
|
||||||
Note right of JWT: Validates signature/expiry against the refresh token secret, extracts userId/username claims
|
|
||||||
JWT-->>TokenSvc: ValidatedToken or throws
|
|
||||||
deactivate JWT
|
|
||||||
|
|
||||||
alt Token invalid or expired
|
|
||||||
TokenSvc->>RefreshHandler: throw UnauthorizedException
|
|
||||||
RefreshHandler->>Mediator: propagate
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else Token valid
|
|
||||||
TokenSvc->>UserMgr: FindByIdAsync(validated.UserId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userId)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: row or null
|
|
||||||
deactivate DB
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>TokenSvc: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt User not found
|
|
||||||
TokenSvc->>RefreshHandler: throw UnauthorizedException "User account not found"
|
|
||||||
RefreshHandler->>Mediator: propagate
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else User found
|
|
||||||
TokenSvc->>JWT: GenerateAccessToken(user) + GenerateRefreshToken(user)
|
|
||||||
JWT-->>TokenSvc: new access + refresh tokens
|
|
||||||
TokenSvc-->>RefreshHandler: RefreshTokenResult
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
RefreshHandler-->>Mediator: LoginPayload(userId, username, refreshToken, accessToken)
|
|
||||||
deactivate RefreshHandler
|
|
||||||
Mediator-->>API: LoginPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Token refreshed successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Registration
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/registration.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,165 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant RegHandler as RegisterUserHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
participant TokenSvc as TokenService
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant Argon2 as Argon2PasswordHasher
|
|
||||||
participant JWT as JWT Infrastructure
|
|
||||||
participant SMTP as Email Provider
|
|
||||||
participant Template as Template Provider
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
participant ProfileHandler as CreateUserProfileHandler<br/>(Profile slice)
|
|
||||||
participant EmailDispatcher as EmailDispatcher<br/>(Features.Emails)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: POST /api/auth/register {username, firstName, lastName, email, dateOfBirth, password}
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(RegisterUserCommand)
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs RegisterUserValidator — username 3-64 chars alphanumeric+[._-], valid email ≤128 chars, password min 8 chars with upper/lower/number/special, DateOfBirth 19+ years old
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request {message, errors}
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>RegHandler: Handle(command)
|
|
||||||
activate RegHandler
|
|
||||||
RegHandler->>RegHandler: user = new UserAccount{...}.ToApplicationUser()
|
|
||||||
|
|
||||||
RegHandler->>UserMgr: CreateAsync(user, password)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>Argon2: HashPassword(user, password)
|
|
||||||
activate Argon2
|
|
||||||
Note right of Argon2: Argon2id — salt 16 bytes, memory 64MB, iterations 4, parallelism 2 (fixed, to avoid thread-pool exhaustion), hash output 32 bytes
|
|
||||||
Argon2-->>UserMgr: PHC string in Argon2id format (algorithm, version, memory/time/parallelism params, salt, hash)
|
|
||||||
deactivate Argon2
|
|
||||||
|
|
||||||
UserMgr->>UserStore: SetPasswordHashAsync(user, hash)
|
|
||||||
Note right of UserStore: user.Id is still empty here — only sets the in-memory hash — CreateAsync writes it to UserCredential itself
|
|
||||||
|
|
||||||
UserMgr->>UserStore: FindByNameAsync(username)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE Username = @Username
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: null (user doesn't exist)
|
|
||||||
deactivate DB
|
|
||||||
deactivate UserStore
|
|
||||||
|
|
||||||
UserMgr->>UserStore: FindByEmailAsync(email)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE Email = @Email
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: null (email doesn't exist)
|
|
||||||
deactivate DB
|
|
||||||
deactivate UserStore
|
|
||||||
Note right of UserMgr: Default UserManager validators check uniqueness before CreateAsync — DapperUserStore.CreateAsync also guards a genuine race via a caught duplicate-key SqlException (2601/2627)
|
|
||||||
|
|
||||||
UserMgr->>UserStore: CreateAsync(user)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: BEGIN TRANSACTION
|
|
||||||
activate DB
|
|
||||||
UserStore->>DB: INSERT INTO UserAccount (...) OUTPUT INSERTED.UserAccountID
|
|
||||||
DB-->>UserStore: UserAccountId (GUID)
|
|
||||||
UserStore->>DB: INSERT INTO UserCredential (UserAccountId, Hash)
|
|
||||||
UserStore->>DB: COMMIT TRANSACTION
|
|
||||||
deactivate DB
|
|
||||||
Note right of UserStore: Both statements run via Dapper against a single DbTransaction
|
|
||||||
|
|
||||||
UserStore-->>UserMgr: IdentityResult.Success (user.Id set)
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>RegHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt !result.Succeeded
|
|
||||||
RegHandler->>Mediator: throw ConflictException (DuplicateUserName/DuplicateEmail → "Username or email already exists", otherwise joined error descriptions)
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict "Username or email already exists"
|
|
||||||
else result.Succeeded
|
|
||||||
RegHandler->>Mediator: Send(CreateUserProfileCommand(user.Id, ""))
|
|
||||||
activate Mediator
|
|
||||||
Mediator->>ProfileHandler: Handle(command)
|
|
||||||
activate ProfileHandler
|
|
||||||
ProfileHandler->>DB: INSERT INTO UserProfile (UserAccountId, Biography)
|
|
||||||
activate DB
|
|
||||||
DB-->>ProfileHandler: UserProfileId
|
|
||||||
deactivate DB
|
|
||||||
ProfileHandler-->>Mediator: UserProfileId
|
|
||||||
deactivate ProfileHandler
|
|
||||||
Mediator-->>RegHandler: UserProfileId
|
|
||||||
deactivate Mediator
|
|
||||||
|
|
||||||
RegHandler->>TokenSvc: GenerateAccessToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: GenerateJwt(userId, username, expiry)
|
|
||||||
activate JWT
|
|
||||||
Note right of JWT: HS256, expires 1 hour, claims sub/unique_name/iat/jti
|
|
||||||
JWT->>JWT: Create JWT with claims, sign with secret key
|
|
||||||
JWT-->>TokenSvc: Access Token
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>RegHandler: Access Token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
RegHandler->>TokenSvc: GenerateRefreshToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: GenerateJwt(userId, username, expiry)
|
|
||||||
activate JWT
|
|
||||||
Note right of JWT: Expires 21 days (504h), same structure as access token, signed with a separate secret
|
|
||||||
JWT-->>TokenSvc: Refresh Token
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>RegHandler: Refresh Token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
RegHandler->>TokenSvc: GenerateConfirmationToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc->>JWT: GenerateJwt(userId, username, expiry)
|
|
||||||
activate JWT
|
|
||||||
Note right of JWT: Expires 30 minutes, signed with a third, separate secret
|
|
||||||
JWT-->>TokenSvc: Confirmation Token
|
|
||||||
deactivate JWT
|
|
||||||
TokenSvc-->>RegHandler: Confirmation Token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
RegHandler->>Mediator: Publish(UserRegisteredNotification)
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: Features.Emails has a direct ProjectReference on Features.Users and handles the notification via UserRegisteredNotificationHandler — the second of two cross-slice dispatches in this flow (profile creation is the first)
|
|
||||||
Mediator->>EmailDispatcher: SendRegistrationEmailAsync(firstName, email, confirmationToken)
|
|
||||||
activate EmailDispatcher
|
|
||||||
|
|
||||||
EmailDispatcher->>Template: RenderUserRegisteredEmailAsync(firstName, confirmationLink)
|
|
||||||
activate Template
|
|
||||||
Note right of Template: Razor component — header/branding, welcome message, confirmation button, footer
|
|
||||||
Template->>Template: Render Razor component to HTML
|
|
||||||
Template-->>EmailDispatcher: HTML email content
|
|
||||||
deactivate Template
|
|
||||||
|
|
||||||
EmailDispatcher->>SMTP: SendAsync(email, subject, body)
|
|
||||||
activate SMTP
|
|
||||||
Note right of SMTP: MailKit — host/port from SMTP_HOST/SMTP_PORT, StartTLS when SMTP_USE_SSL=true, optional username/password auth
|
|
||||||
SMTP->>SMTP: Build MIME message, connect, authenticate, send, disconnect
|
|
||||||
SMTP-->>EmailDispatcher: Success / throws
|
|
||||||
deactivate SMTP
|
|
||||||
deactivate EmailDispatcher
|
|
||||||
|
|
||||||
alt Email sent successfully
|
|
||||||
Note right of Mediator: UserRegisteredNotificationHandler completes normally
|
|
||||||
else EmailDispatcher throws
|
|
||||||
Note right of Mediator: UserRegisteredNotificationHandler catches and logs the exception — registration is unaffected
|
|
||||||
end
|
|
||||||
deactivate Mediator
|
|
||||||
|
|
||||||
RegHandler-->>Mediator: RegistrationPayload(userAccountId, username, refreshToken, accessToken)
|
|
||||||
deactivate RegHandler
|
|
||||||
Mediator-->>API: RegistrationPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 201 Created {message: "User registered successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Resend Confirmation Email
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/resend-confirmation.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant ResendHandler as ResendConfirmationEmailHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
participant TokenSvc as TokenService
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant SMTP as Email Provider
|
|
||||||
participant Template as Template Provider
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
participant EmailDispatcher as EmailDispatcher<br/>(Features.Emails)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: POST /api/auth/confirm/resend?userId={id} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(ResendConfirmationEmailCommand(userId))
|
|
||||||
activate Mediator
|
|
||||||
Mediator->>ResendHandler: Handle(command)
|
|
||||||
activate ResendHandler
|
|
||||||
|
|
||||||
ResendHandler->>UserMgr: FindByIdAsync(userId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userId)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: row or null
|
|
||||||
deactivate DB
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>ResendHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt User not found
|
|
||||||
ResendHandler->>ResendHandler: return silently (prevents user enumeration)
|
|
||||||
else User found
|
|
||||||
ResendHandler->>UserMgr: IsEmailConfirmedAsync(user)
|
|
||||||
UserMgr-->>ResendHandler: bool
|
|
||||||
|
|
||||||
alt Already confirmed
|
|
||||||
ResendHandler->>ResendHandler: return silently (no-op)
|
|
||||||
else Not yet confirmed
|
|
||||||
ResendHandler->>TokenSvc: GenerateConfirmationToken(user)
|
|
||||||
activate TokenSvc
|
|
||||||
TokenSvc-->>ResendHandler: confirmation token
|
|
||||||
deactivate TokenSvc
|
|
||||||
|
|
||||||
ResendHandler->>Mediator: Publish(ConfirmationEmailResendRequestedNotification)
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: Features.Emails has a direct ProjectReference on Features.Users and handles the notification via ConfirmationEmailResendRequestedNotificationHandler
|
|
||||||
Mediator->>EmailDispatcher: SendResendConfirmationEmailAsync(firstName, email, confirmationToken)
|
|
||||||
activate EmailDispatcher
|
|
||||||
EmailDispatcher->>Template: Render email (same Razor pipeline as registration)
|
|
||||||
EmailDispatcher->>SMTP: SendAsync(email, subject, body)
|
|
||||||
deactivate EmailDispatcher
|
|
||||||
|
|
||||||
alt Email sent successfully
|
|
||||||
Note right of Mediator: ConfirmationEmailResendRequestedNotificationHandler completes normally
|
|
||||||
else EmailDispatcher throws
|
|
||||||
Note right of Mediator: ConfirmationEmailResendRequestedNotificationHandler catches and logs the exception — the resend request is unaffected
|
|
||||||
end
|
|
||||||
deactivate Mediator
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
ResendHandler-->>Mediator: (void)
|
|
||||||
deactivate ResendHandler
|
|
||||||
Mediator-->>API: (void)
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "confirmation email has been resent"}
|
|
||||||
Note right of API: Always 200 OK regardless of outcome above — same user-enumeration precaution as a not-found user or an already-confirmed account
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Update Email
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/update-email.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant UpdateEmailHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: PATCH /api/auth/email {newEmail} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(UpdateEmailCommand(GetAuthenticatedUserId(), newEmail))
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs UpdateEmailValidator (same format rules as registration)
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>UpdateEmailHandler: Handle(command)
|
|
||||||
activate UpdateEmailHandler
|
|
||||||
|
|
||||||
UpdateEmailHandler->>UserMgr: FindByIdAsync(userAccountId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userAccountId)
|
|
||||||
UserStore->>DB: SELECT ...
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
UserMgr-->>UpdateEmailHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account not found
|
|
||||||
UpdateEmailHandler->>Mediator: throw NotFoundException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 404 Not Found
|
|
||||||
else Account found
|
|
||||||
UpdateEmailHandler->>UserMgr: SetEmailAsync(user, newEmail)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByEmailAsync(newEmail)
|
|
||||||
Note right of UserMgr: Uniqueness check. Also resets user.EmailConfirmed = false — caller must re-confirm via POST /api/auth/confirm/resend
|
|
||||||
UserMgr->>UserStore: UpdateAsync(user)
|
|
||||||
UserStore->>DB: UPDATE UserAccount SET Email = @Email, ...
|
|
||||||
UserStore->>DB: DELETE FROM UserVerification WHERE UserAccountID = @Id
|
|
||||||
UserStore-->>UserMgr: IdentityResult
|
|
||||||
UserMgr-->>UpdateEmailHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt !result.Succeeded
|
|
||||||
UpdateEmailHandler->>Mediator: throw ConflictException (DuplicateEmail → "Email address already exists")
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else result.Succeeded
|
|
||||||
UpdateEmailHandler-->>Mediator: UpdateEmailPayload(user.Id, user.Email, user.EmailConfirmed = false)
|
|
||||||
deactivate UpdateEmailHandler
|
|
||||||
Mediator-->>API: UpdateEmailPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Email updated successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Update Password
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/update-password.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant UpdatePasswordHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
end
|
|
||||||
box Infrastructure Layer
|
|
||||||
participant Argon2 as Argon2PasswordHasher
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: PATCH /api/auth/password {currentPassword, newPassword} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(UpdatePasswordCommand(GetAuthenticatedUserId(), currentPassword, newPassword))
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs UpdatePasswordValidator (same password strength rules as registration)
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>UpdatePasswordHandler: Handle(command)
|
|
||||||
activate UpdatePasswordHandler
|
|
||||||
|
|
||||||
UpdatePasswordHandler->>UserMgr: FindByIdAsync(userAccountId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userAccountId)
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
UserMgr-->>UpdatePasswordHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account not found
|
|
||||||
UpdatePasswordHandler->>Mediator: throw NotFoundException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 404 Not Found
|
|
||||||
else Account found
|
|
||||||
UpdatePasswordHandler->>UserMgr: ChangePasswordAsync(user, currentPassword, newPassword)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>Argon2: VerifyHashedPassword(user, user.PasswordHash, currentPassword)
|
|
||||||
activate Argon2
|
|
||||||
Argon2-->>UserMgr: Success/Failed
|
|
||||||
deactivate Argon2
|
|
||||||
|
|
||||||
alt Current password mismatch
|
|
||||||
UserMgr-->>UpdatePasswordHandler: IdentityResult.Failed (PasswordMismatch)
|
|
||||||
else Current password matches
|
|
||||||
UserMgr->>Argon2: HashPassword(user, newPassword)
|
|
||||||
activate Argon2
|
|
||||||
Argon2-->>UserMgr: new hash
|
|
||||||
deactivate Argon2
|
|
||||||
UserMgr->>UserStore: SetPasswordHashAsync(user, newHash)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: UPDATE UserCredential SET IsRevoked = 1, RevokedAt = GETDATE() WHERE UserAccountId = @Id AND IsRevoked = 0
|
|
||||||
UserStore->>DB: INSERT INTO UserCredential (UserAccountId, Hash)
|
|
||||||
Note right of UserStore: Revoke-then-insert runs inside one DbTransaction (credential rotation)
|
|
||||||
UserStore-->>UserMgr: (void)
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>UpdatePasswordHandler: IdentityResult.Success
|
|
||||||
end
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Current password is incorrect
|
|
||||||
UpdatePasswordHandler->>Mediator: throw UnauthorizedException "Current password is incorrect"
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 401 Unauthorized
|
|
||||||
else !result.Succeeded (other failure)
|
|
||||||
UpdatePasswordHandler->>Mediator: throw ConflictException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else result.Succeeded
|
|
||||||
UpdatePasswordHandler-->>Mediator: UpdatePasswordPayload(user.Id, DateTime.UtcNow)
|
|
||||||
deactivate UpdatePasswordHandler
|
|
||||||
Mediator-->>API: UpdatePasswordPayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Password updated successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Update Profile
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/update-profile.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,63 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant UpdateProfileHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: PATCH /api/auth/profile {firstName, lastName, dateOfBirth} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(UpdateProfileCommand(GetAuthenticatedUserId(), firstName, lastName, dateOfBirth))
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs UpdateProfileValidator (same name/DateOfBirth rules as registration)
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>UpdateProfileHandler: Handle(command)
|
|
||||||
activate UpdateProfileHandler
|
|
||||||
|
|
||||||
UpdateProfileHandler->>UserMgr: FindByIdAsync(userAccountId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userAccountId)
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
UserMgr-->>UpdateProfileHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account not found
|
|
||||||
UpdateProfileHandler->>Mediator: throw NotFoundException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 404 Not Found
|
|
||||||
else Account found
|
|
||||||
UpdateProfileHandler->>UpdateProfileHandler: user.FirstName = ...<br/>user.LastName = ...<br/>user.DateOfBirth = ...
|
|
||||||
Note right of UpdateProfileHandler: First/last name and date of birth aren't tracked by any Identity store interface — mutated directly on the ApplicationUser, then saved via UserManager.UpdateAsync
|
|
||||||
|
|
||||||
UpdateProfileHandler->>UserMgr: UpdateAsync(user)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: UpdateAsync(user)
|
|
||||||
UserStore->>DB: UPDATE UserAccount SET FirstName = @FirstName, LastName = @LastName, DateOfBirth = @DateOfBirth, ...
|
|
||||||
UserStore-->>UserMgr: IdentityResult
|
|
||||||
UserMgr-->>UpdateProfileHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt !result.Succeeded
|
|
||||||
UpdateProfileHandler->>Mediator: throw ConflictException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else result.Succeeded
|
|
||||||
UpdateProfileHandler-->>Mediator: UpdateProfilePayload(user.Id, firstName, lastName, dateOfBirth)
|
|
||||||
deactivate UpdateProfileHandler
|
|
||||||
Mediator-->>API: UpdateProfilePayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Profile updated successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
# User Authentication Flow — Update Username
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/auth/update-username.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor User
|
|
||||||
participant API as API Controller
|
|
||||||
participant Mediator as MediatR pipeline<br/>(ValidationBehavior)
|
|
||||||
box Handler Layer
|
|
||||||
participant UpdateUsernameHandler
|
|
||||||
participant UserMgr as UserManager(ApplicationUser)
|
|
||||||
end
|
|
||||||
box Repository / Cross-slice
|
|
||||||
participant UserStore as DapperUserStore<br/>(IUserStore/IUserPasswordStore/IUserEmailStore)
|
|
||||||
end
|
|
||||||
participant DB as SQL Server
|
|
||||||
|
|
||||||
User->>API: PATCH /api/auth/username {newUsername} (requires Bearer access token)
|
|
||||||
activate API
|
|
||||||
API->>Mediator: Send(UpdateUsernameCommand(GetAuthenticatedUserId(), newUsername))
|
|
||||||
activate Mediator
|
|
||||||
Note right of Mediator: ValidationBehavior runs UpdateUsernameValidator (same format rules as registration)
|
|
||||||
|
|
||||||
alt Validation fails
|
|
||||||
Mediator->>API: throws FluentValidation.ValidationException
|
|
||||||
API->>User: 400 Bad Request
|
|
||||||
else Validation succeeds
|
|
||||||
Mediator->>UpdateUsernameHandler: Handle(command)
|
|
||||||
activate UpdateUsernameHandler
|
|
||||||
|
|
||||||
UpdateUsernameHandler->>UserMgr: FindByIdAsync(userAccountId)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByIdAsync(userAccountId)
|
|
||||||
activate UserStore
|
|
||||||
UserStore->>DB: SELECT ... FROM UserAccount WHERE UserAccountID = @Id
|
|
||||||
activate DB
|
|
||||||
DB-->>UserStore: row or null
|
|
||||||
deactivate DB
|
|
||||||
UserStore-->>UserMgr: ApplicationUser or null
|
|
||||||
deactivate UserStore
|
|
||||||
UserMgr-->>UpdateUsernameHandler: ApplicationUser or null
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt Account not found
|
|
||||||
UpdateUsernameHandler->>Mediator: throw NotFoundException
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 404 Not Found
|
|
||||||
else Account found
|
|
||||||
UpdateUsernameHandler->>UserMgr: SetUserNameAsync(user, newUsername)
|
|
||||||
activate UserMgr
|
|
||||||
UserMgr->>UserStore: FindByNameAsync(newUsername)
|
|
||||||
Note right of UserMgr: Uniqueness check before the UPDATE
|
|
||||||
UserMgr->>UserStore: UpdateAsync(user)
|
|
||||||
UserStore->>DB: UPDATE UserAccount SET Username = @UserName, ...
|
|
||||||
UserStore-->>UserMgr: IdentityResult
|
|
||||||
UserMgr-->>UpdateUsernameHandler: IdentityResult
|
|
||||||
deactivate UserMgr
|
|
||||||
|
|
||||||
alt !result.Succeeded
|
|
||||||
UpdateUsernameHandler->>Mediator: throw ConflictException (DuplicateUserName → "Username already exists")
|
|
||||||
Mediator->>API: propagate
|
|
||||||
API->>User: 409 Conflict
|
|
||||||
else result.Succeeded
|
|
||||||
UpdateUsernameHandler-->>Mediator: UpdateUsernamePayload(user.Id, user.UserName)
|
|
||||||
deactivate UpdateUsernameHandler
|
|
||||||
Mediator-->>API: UpdateUsernamePayload
|
|
||||||
deactivate Mediator
|
|
||||||
API->>User: 200 OK {message: "Username updated successfully.", payload: {...}}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
deactivate API
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
# Database Schema
|
|
||||||
|
|
||||||
Six diagrams, one per SQL Server schema — mirrors the `Auth` / `Geolocation` /
|
|
||||||
`Media` / `Brewery` / `Beer` / `Social` schema split under
|
|
||||||
`web/backend/Database/Database.Migrations/scripts/`. Every table also has a
|
|
||||||
`RowVersion ROWVERSION` column (optimistic concurrency), omitted below for
|
|
||||||
brevity. Tables marked `(*)` have no repository/feature slice yet.
|
|
||||||
|
|
||||||
Boxes with no attribute list (e.g. `Photo` in the auth diagram) are owned by a
|
|
||||||
different schema and shown only to keep the relationship visible; see the
|
|
||||||
diagram named after that schema for its full definition.
|
|
||||||
|
|
||||||
## Auth (`Auth`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/auth.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Social (`Social`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/social.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Media (`Media`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/media.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Geolocation (`Geolocation`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/geolocation.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Brewery (`Brewery`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/brewery.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Beer (`Beer`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/database-schema/beer.mmd"
|
|
||||||
```
|
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
erDiagram
|
|
||||||
User ||--o{ Cred : "has credentials"
|
|
||||||
User ||--o| Verify : "has verification"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in social.mmd / media.mmd / brewery.mmd / beer.mmd
|
|
||||||
User ||--o| Profile : "has profile"
|
|
||||||
User ||--o{ Follow : "follows"
|
|
||||||
User ||--o{ Follow : "followed by"
|
|
||||||
User ||--o{ Photo : "uploads"
|
|
||||||
User ||--o{ Brewery : "posts"
|
|
||||||
User ||--o{ Beer : "posts"
|
|
||||||
User ||--o{ BeerComment : "comments"
|
|
||||||
|
|
||||||
User {
|
|
||||||
uniqueidentifier UserAccountID PK
|
|
||||||
varchar Username UK
|
|
||||||
nvarchar FirstName
|
|
||||||
nvarchar LastName
|
|
||||||
varchar Email UK
|
|
||||||
datetime CreatedAt
|
|
||||||
datetime UpdatedAt "nullable"
|
|
||||||
date DateOfBirth
|
|
||||||
}
|
|
||||||
|
|
||||||
Cred {
|
|
||||||
uniqueidentifier UserCredentialID PK
|
|
||||||
uniqueidentifier UserAccountID FK
|
|
||||||
datetime CreatedAt
|
|
||||||
datetime Expiry
|
|
||||||
nvarchar Hash
|
|
||||||
bit IsRevoked
|
|
||||||
datetime RevokedAt "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
Verify {
|
|
||||||
uniqueidentifier UserVerificationID PK
|
|
||||||
uniqueidentifier UserAccountID "FK, UK"
|
|
||||||
datetime VerificationDateTime
|
|
||||||
}
|
|
||||||
@@ -1,51 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
erDiagram
|
|
||||||
BeerStyle ||--o{ Beer : "categorizes"
|
|
||||||
Beer ||--o{ BeerPhoto : "has photos"
|
|
||||||
Beer ||--o{ BeerComment : "has comments"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in auth.mmd / brewery.mmd / media.mmd
|
|
||||||
User ||--o{ Beer : "posts"
|
|
||||||
Brewery ||--o{ Beer : "brews"
|
|
||||||
User ||--o{ BeerComment : "comments"
|
|
||||||
BeerPhoto }o--|| Photo : "uses photo"
|
|
||||||
|
|
||||||
BeerStyle {
|
|
||||||
uniqueidentifier BeerStyleID PK
|
|
||||||
nvarchar StyleName UK
|
|
||||||
nvarchar Description "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
Beer {
|
|
||||||
uniqueidentifier BeerPostID PK
|
|
||||||
nvarchar Name
|
|
||||||
nvarchar Description
|
|
||||||
decimal ABV
|
|
||||||
int IBU
|
|
||||||
uniqueidentifier PostedByID FK
|
|
||||||
uniqueidentifier BeerStyleID FK
|
|
||||||
uniqueidentifier BrewedByID FK
|
|
||||||
datetime CreatedAt
|
|
||||||
datetime UpdatedAt "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
BeerComment {
|
|
||||||
uniqueidentifier BeerPostCommentID PK
|
|
||||||
nvarchar Comment
|
|
||||||
uniqueidentifier BeerPostID FK
|
|
||||||
uniqueidentifier CommentedByID FK
|
|
||||||
int Rating
|
|
||||||
datetime CreatedAt
|
|
||||||
datetime UpdatedAt "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
BeerPhoto {
|
|
||||||
uniqueidentifier BeerPostPhotoID PK
|
|
||||||
uniqueidentifier BeerPostID FK
|
|
||||||
uniqueidentifier PhotoID FK
|
|
||||||
datetime LinkedAt
|
|
||||||
}
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
erDiagram
|
|
||||||
Brewery ||--o| BreweryLocation : "has location"
|
|
||||||
Brewery ||--o{ BreweryPhoto : "has photos"
|
|
||||||
Brewery ||--o{ Beer : "brews"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in auth.mmd / geolocation.mmd / media.mmd / beer.mmd
|
|
||||||
User ||--o{ Brewery : "posts"
|
|
||||||
BreweryLocation }o--|| City : "located in"
|
|
||||||
BreweryPhoto }o--|| Photo : "uses photo"
|
|
||||||
|
|
||||||
Brewery {
|
|
||||||
uniqueidentifier BreweryPostID PK
|
|
||||||
nvarchar BreweryName
|
|
||||||
uniqueidentifier PostedByID FK
|
|
||||||
nvarchar Description
|
|
||||||
datetime CreatedAt
|
|
||||||
datetime UpdatedAt "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
BreweryLocation {
|
|
||||||
uniqueidentifier BreweryPostLocationID PK
|
|
||||||
uniqueidentifier BreweryPostID "FK, UK"
|
|
||||||
nvarchar AddressLine1
|
|
||||||
nvarchar AddressLine2 "nullable"
|
|
||||||
nvarchar PostalCode
|
|
||||||
uniqueidentifier CityID FK
|
|
||||||
geography Coordinates "nullable"
|
|
||||||
}
|
|
||||||
|
|
||||||
BreweryPhoto {
|
|
||||||
uniqueidentifier BreweryPostPhotoID PK
|
|
||||||
uniqueidentifier BreweryPostID FK
|
|
||||||
uniqueidentifier PhotoID FK
|
|
||||||
datetime LinkedAt
|
|
||||||
}
|
|
||||||
@@ -1,26 +0,0 @@
|
|||||||
erDiagram
|
|
||||||
Country ||--o{ StateProvince : "has"
|
|
||||||
StateProvince ||--o{ City : "has"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in brewery.mmd
|
|
||||||
City ||--o{ BreweryLocation : "located in"
|
|
||||||
|
|
||||||
Country {
|
|
||||||
uniqueidentifier CountryID PK
|
|
||||||
nvarchar CountryName
|
|
||||||
char ISO3166_1 UK
|
|
||||||
}
|
|
||||||
|
|
||||||
StateProvince {
|
|
||||||
uniqueidentifier StateProvinceID PK
|
|
||||||
nvarchar StateProvinceName
|
|
||||||
char ISO3166_2 UK
|
|
||||||
uniqueidentifier CountryID FK
|
|
||||||
}
|
|
||||||
|
|
||||||
%% CityName + StateProvinceID form a composite unique constraint (AK_CityStateProvince)
|
|
||||||
City {
|
|
||||||
uniqueidentifier CityID PK
|
|
||||||
nvarchar CityName UK
|
|
||||||
uniqueidentifier StateProvinceID FK,UK
|
|
||||||
}
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
erDiagram
|
|
||||||
User ||--o{ Photo : "uploads"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in auth.mmd / social.mmd / brewery.mmd / beer.mmd
|
|
||||||
Avatar }o--|| Photo : "uses photo"
|
|
||||||
Photo ||--o{ BreweryPhoto : "used in"
|
|
||||||
Photo ||--o{ BeerPhoto : "used in"
|
|
||||||
|
|
||||||
Photo {
|
|
||||||
uniqueidentifier PhotoID PK
|
|
||||||
nvarchar Hyperlink "nullable"
|
|
||||||
uniqueidentifier UploadedByID FK
|
|
||||||
datetime UploadedAt
|
|
||||||
}
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
erDiagram
|
|
||||||
User ||--o| Profile : "has profile"
|
|
||||||
User ||--o{ Follow : "follows"
|
|
||||||
User ||--o{ Follow : "followed by"
|
|
||||||
Profile ||--o{ Avatar : "has avatar history"
|
|
||||||
|
|
||||||
%% Cross-schema: defined in full in auth.mmd / media.mmd
|
|
||||||
Avatar }o--|| Photo : "uses photo"
|
|
||||||
|
|
||||||
Profile {
|
|
||||||
uniqueidentifier UserProfileID PK
|
|
||||||
uniqueidentifier UserAccountID "FK, UK"
|
|
||||||
nvarchar Biography
|
|
||||||
}
|
|
||||||
|
|
||||||
Follow {
|
|
||||||
uniqueidentifier UserFollowID PK
|
|
||||||
uniqueidentifier UserAccountID FK
|
|
||||||
uniqueidentifier FollowingID FK
|
|
||||||
datetime CreatedAt
|
|
||||||
}
|
|
||||||
|
|
||||||
Avatar {
|
|
||||||
uniqueidentifier UserAvatarID PK
|
|
||||||
uniqueidentifier UserProfileID FK
|
|
||||||
uniqueidentifier PhotoID FK
|
|
||||||
datetime2 ValidFrom
|
|
||||||
datetime2 ValidTo "nullable; at most one row per UserProfileID has ValidTo IS NULL (active avatar)"
|
|
||||||
}
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
# Docker Deployment Architecture
|
|
||||||
|
|
||||||
Five diagrams, one per Compose file — mixing all five networks in one graph made
|
|
||||||
it impossible to tell which containers actually talk to each other. Solid arrows
|
|
||||||
are real container traffic; dashed arrows are `depends_on` startup gating with
|
|
||||||
no data flowing over them.
|
|
||||||
|
|
||||||
## Development (`docker-compose.dev.yaml`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/deployment/dev.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Test (`docker-compose.test.yaml`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/deployment/test.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Production (`docker-compose.prod.yaml`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/deployment/prod.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Database-only (`docker-compose.db.yaml`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/deployment/db-only.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Minimal (`docker-compose.min.yaml`)
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/deployment/minimal.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
| Environment | Details |
|
|
||||||
| ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| **Dev network** (`devnet`, bridge) | Internal DNS resolves service names such as `sqlserver` and `api.core`. Volumes: `sqlserverdata-dev`, `nuget-cache-dev`, `seaweedfsdata-dev`. Startup order: SQL Server (health check) → migrations (run-once) → seed (run-once, depends on `seaweedfs`) → API (long-running, depends on seed). |
|
|
||||||
| **Test network** (`testnet`, bridge) | All test components are isolated; fresh DB instance each run (`CLEAR_DATABASE=true`, ephemeral `sqlserverdata-test` / `seaweedfsdata-test` volumes). `api.specs`, `unit.tests`, and `frontend.tests` all run as separate containers, in parallel once seeding completes; results are aggregated. |
|
|
||||||
| **Prod network** (`prodnet`, bridge, named `biergarten-prodnet`) | Isolated from dev/test networks. Volumes: `biergartendata-prod` and `seaweedfsdata-prod`, both external, so `down -v` keeps them. Startup order: SeaweedFS (health check) → API (health check) → frontend, with `frontend` the only service publishing a host port, bound to `127.0.0.1`. Migrating and seeding run on demand as their own run-once jobs joining `biergarten-prodnet`, so an `up` or a reboot replays neither. |
|
|
||||||
| **Database-only environment** | Same devnet volumes/images as `docker-compose.dev.yaml`, minus `api.core`, `frontend`, and `mailpit` — for working against the database (and object storage, since `seaweedfs` is still included) without running the API or website. |
|
|
||||||
| **Minimal environment** | Database + migrations + seed + `seaweedfs` + mailpit, no API or frontend container — for frontend development against an `API.Core` process run outside Docker. |
|
|
||||||
| `unit.tests` | Builds `Dockerfile.tests`, which restores/builds against `Biergarten.API.slnx` and then runs `dotnet test` for every `Features/*.Tests` project in turn (`Features.Users.Tests`, `Features.Breweries.Tests`, `Features.Emails.Tests`, `Features.PhotoUpload.Tests`), mocking each handler's repository/service dependencies with Moq — no real database needed. |
|
|
||||||
| `.trx` results | Readable by Visual Studio, Azure DevOps, and GitHub Actions. |
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
box Database-only Environment (docker-compose.db.yaml)
|
|
||||||
participant DbOnlyDB as SQL Server<br/>:1434
|
|
||||||
participant DbOnlySeaweedFS as seaweedfs<br/>S3-compatible object storage
|
|
||||||
participant DbOnlyMig as Migrations (run-once)
|
|
||||||
participant DbOnlySeed as Seed (run-once)
|
|
||||||
end
|
|
||||||
|
|
||||||
DbOnlyMig-->>DbOnlyDB: depends_on
|
|
||||||
DbOnlyMig->>DbOnlyDB: Run migrations
|
|
||||||
DbOnlySeed-->>DbOnlyMig: depends_on
|
|
||||||
DbOnlySeed-->>DbOnlySeaweedFS: depends_on
|
|
||||||
DbOnlySeed->>DbOnlyDB: Seed data
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
actor Developer
|
|
||||||
participant Host as Docker Host
|
|
||||||
box Development Environment (docker-compose.dev.yaml)
|
|
||||||
participant DevDB as SQL Server 2022<br/>Biergarten Database<br/>:1434
|
|
||||||
participant DevMig as Migrations (run-once)<br/>Database.Migrations
|
|
||||||
participant DevSeed as Seed (run-once)<br/>Database.Seed
|
|
||||||
participant DevAPI as API Container<br/>ASP.NET Core 10<br/>:8080 HTTP / :8081 HTTPS
|
|
||||||
participant DevMailpit as Mailpit<br/>:8025 Web UI / :1025 SMTP
|
|
||||||
end
|
|
||||||
|
|
||||||
Developer->>Host: docker compose up
|
|
||||||
DevMig-->>DevDB: depends_on (healthy)
|
|
||||||
DevMig->>DevDB: Run migrations
|
|
||||||
DevSeed-->>DevMig: depends_on
|
|
||||||
DevSeed->>DevDB: Seed data
|
|
||||||
DevAPI-->>DevSeed: depends_on
|
|
||||||
DevAPI->>DevDB: Connect and serve
|
|
||||||
DevAPI-->>DevMailpit: SMTP_HOST/PORT
|
|
||||||
Host->>DevAPI: http://localhost:8080
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
box Minimal Environment (docker-compose.min.yaml)
|
|
||||||
participant MinDB as SQL Server<br/>:1433
|
|
||||||
participant MinSeaweedFS as seaweedfs<br/>S3-compatible object storage
|
|
||||||
participant MinMig as Migrations (run-once)
|
|
||||||
participant MinSeed as Seed (run-once)
|
|
||||||
participant MinMailpit as Mailpit<br/>:8025 / :1025
|
|
||||||
end
|
|
||||||
|
|
||||||
MinMig-->>MinDB: depends_on
|
|
||||||
MinMig->>MinDB: Run migrations
|
|
||||||
MinSeed-->>MinMig: depends_on
|
|
||||||
MinSeed-->>MinSeaweedFS: depends_on
|
|
||||||
MinSeed->>MinDB: Seed data
|
|
||||||
Note over MinDB,MinMailpit: consumed by an externally run API.Core process (not shown)
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
box Production Environment (docker-compose.prod.yaml)
|
|
||||||
participant ProdDB as SQL Server 2022<br/>Biergarten Database<br/>no host port published
|
|
||||||
participant ProdS3 as SeaweedFS<br/>S3 gateway :8333<br/>no host port published
|
|
||||||
participant ProdAPI as API Container<br/>ASP.NET Core 10<br/>:8080 HTTP / :8081 HTTPS
|
|
||||||
participant ProdWeb as Frontend Container<br/>Express + React Router<br/>127.0.0.1:5656
|
|
||||||
end
|
|
||||||
|
|
||||||
ProdAPI-->>ProdS3: depends_on: seaweedfs healthy
|
|
||||||
ProdWeb-->>ProdAPI: depends_on: api.core healthy
|
|
||||||
ProdAPI->>ProdDB: Connect and serve
|
|
||||||
ProdAPI->>ProdS3: Store and read photos
|
|
||||||
Note over ProdDB: Migrating and seeding are separate run-once jobs,<br/>run at deploy time rather than by the stack
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
sequenceDiagram
|
|
||||||
box Test Environment (docker-compose.test.yaml)
|
|
||||||
participant TestDB as SQL Server, isolated instance<br/>Biergarten Test Database<br/>no host port published
|
|
||||||
participant TestSeaweedFS as seaweedfs<br/>S3-compatible object storage
|
|
||||||
participant TestMig as Migrations (test)
|
|
||||||
participant TestSeed as Seed (test)<br/>minimal: test.user only
|
|
||||||
participant Specs as API.Specs<br/>Reqnroll + xUnit<br/>(integration tests)
|
|
||||||
participant UnitTests as unit.tests<br/>xUnit + Moq<br/>Features.*.Tests, one container
|
|
||||||
participant FrontendTests as frontend.tests<br/>Playwright/Vitest
|
|
||||||
end
|
|
||||||
participant Results as test-results/ (mounted volume)
|
|
||||||
|
|
||||||
TestMig-->>TestDB: depends_on
|
|
||||||
TestMig->>TestDB: Migrate
|
|
||||||
TestSeed-->>TestMig: depends_on
|
|
||||||
TestSeed-->>TestSeaweedFS: depends_on
|
|
||||||
TestSeed->>TestDB: Seed
|
|
||||||
|
|
||||||
par Specs
|
|
||||||
Specs-->>TestSeed: depends_on
|
|
||||||
Specs->>TestDB: Integration test
|
|
||||||
Specs->>Results: Export TRX
|
|
||||||
and unit.tests
|
|
||||||
UnitTests-->>TestSeed: depends_on
|
|
||||||
UnitTests-->>TestDB: depends_on (startup ordering only, no real connection)
|
|
||||||
UnitTests->>Results: Export TRX, one per slice
|
|
||||||
and frontend.tests
|
|
||||||
FrontendTests->>Results: Export results
|
|
||||||
end
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
# The Biergarten App — Vertical Slice Architecture
|
|
||||||
|
|
||||||
Two diagrams, split by concern: which host wires up which slice at startup, and
|
|
||||||
which projects each slice actually compiles against. Mixing DI wiring and
|
|
||||||
compile-time references in one graph is what made the old diagram unreadable.
|
|
||||||
|
|
||||||
## Host wiring
|
|
||||||
|
|
||||||
Each host (`API.Core`, `Database.Seed`) decides for itself which `Features.*`
|
|
||||||
assemblies to register — there's no shared bootstrap. `Database.Migrations`
|
|
||||||
doesn't wire up any slice; it applies schema directly.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/vertical-slice/host-wiring.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Project reference dependencies
|
|
||||||
|
|
||||||
Solid arrows are compile-time project references — every slice-to-slice
|
|
||||||
dependency is one now that the email MediatR contract has moved off
|
|
||||||
`Shared.Application`: `Features.Users` on `Features.PhotoUpload` for avatar
|
|
||||||
uploads, and `Features.Emails` on `Features.Users` for the registration and
|
|
||||||
resend-confirmation notifications. Every DB-backed slice depends on the same
|
|
||||||
three foundation projects (`Shared.Contracts`, `Domain.Entities`,
|
|
||||||
`Database.Connection`) — drawn as one box instead of fifteen separate arrows;
|
|
||||||
see the notes for what else each of those foundation projects pulls in.
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
--8<-- "web/diagrams/vertical-slice/project-references.mmd"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- **API.Core**: Program.cs wiring only
|
|
||||||
- MediatR + `AddApplicationPart` per HTTP-exposed `Features.*` assembly
|
|
||||||
- Swagger/OpenAPI, health checks, JWT auth middleware, global exception filter
|
|
||||||
- Only `Features.Users` and `Features.Breweries` get `AddApplicationPart`
|
|
||||||
(controllers)
|
|
||||||
- `Features.PhotoUpload` and `Features.Emails` scanned for MediatR handlers
|
|
||||||
only
|
|
||||||
- `Features.Locations` and `Features.Beers` not referenced by `API.Core` at
|
|
||||||
all
|
|
||||||
|
|
||||||
- **Feature slices**: each owns its own controller (where present),
|
|
||||||
commands/queries + handlers, validators, and Dapper repository
|
|
||||||
- `Features.Users` covers auth (`Commands/Authentication`), account management
|
|
||||||
(`Commands/Account`), and profile/avatar (`Commands/Profile`)
|
|
||||||
- `Features.Emails` and `Features.PhotoUpload` have no controller — invoked
|
|
||||||
only via MediatR commands sent from other slices
|
|
||||||
- `Features.Locations` has no controller; one MediatR handler
|
|
||||||
(`GetOrCreateCityHandler`) registered by `Database.Seed` only — `API.Core`
|
|
||||||
never scans it
|
|
||||||
- `Features.Beers` is scaffolded (project + references only, no source files
|
|
||||||
yet)
|
|
||||||
|
|
||||||
- **The exceptions to slice isolation**:
|
|
||||||
- `Features.Users` takes a direct project reference to `Features.PhotoUpload`
|
|
||||||
- sends its `UploadPhotoCommand` via MediatR from `UploadAvatarHandler`
|
|
||||||
- the command type lives in the target slice itself, not in
|
|
||||||
`Shared.Application`
|
|
||||||
- `Features.Emails` takes a direct project reference to `Features.Users`
|
|
||||||
- handles `UserRegisteredNotification` (published from
|
|
||||||
`RegisterUserHandler`) and `ConfirmationEmailResendRequestedNotification`
|
|
||||||
(published from `ResendConfirmationEmailHandler`)
|
|
||||||
- both notification types live in `Features.Users`
|
|
||||||
(`Features.Users.Notifications`), not in `Shared.Application`
|
|
||||||
|
|
||||||
- **Shared.Application**:
|
|
||||||
- `ValidationBehavior` (MediatR pipeline)
|
|
||||||
- Every slice (including `Features.Emails`) references it for the pipeline
|
|
||||||
behavior alone
|
|
||||||
|
|
||||||
- **Domain.Entities**:
|
|
||||||
- `UserAccount`, `UserCredential`, `UserVerification`, `UserProfile`,
|
|
||||||
`UserAvatar`
|
|
||||||
- `BreweryPost`, `BreweryPostLocation`
|
|
||||||
- `City`, `StateProvince`, `Country`
|
|
||||||
- `BeerPost`, `BeerStyle`, `Photo` (schema tables exist for
|
|
||||||
`BeerPost`/`BeerStyle`; no repository yet since `Features.Beers` has no
|
|
||||||
code)
|
|
||||||
|
|
||||||
- **Database.Connection**: the shared connection-factory project
|
|
||||||
- each slice's own repository issues inline SQL via Dapper against it
|
|
||||||
- no ORM, no stored procedures
|
|
||||||
- Replaces the old `Infrastructure.Sql` project
|
|
||||||
|
|
||||||
- **Database.Migrations**: separate console app
|
|
||||||
- `schema.dbml` + SQL scripts
|
|
||||||
- applies DDL directly
|
|
||||||
- distinct from `Database.Seed`, which loads data through the feature slices'
|
|
||||||
own DI extensions
|
|
||||||
|
|
||||||
- **Database.Seed**:
|
|
||||||
- calls each slice's `AddFeatures*()` DI extension directly
|
|
||||||
- builds its own MediatR registration
|
|
||||||
- no API host, no shared pipeline with `API.Core`
|
|
||||||
|
|
||||||
- **Omitted from the diagram**:
|
|
||||||
- `Domain.Exceptions` (referenced by nearly every project directly, including
|
|
||||||
all of `Shared.Contracts` and each feature slice)
|
|
||||||
- `Configuration` (the shared options-binding project
|
|
||||||
referenced by `Database.Connection`, `Features.PhotoUpload`,
|
|
||||||
and `Features.Emails` directly)
|
|
||||||
- both are plumbing-level dependencies of nearly everything above, and drawing
|
|
||||||
them would just add a second copy of the same fan-in
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
flowchart TB
|
|
||||||
API["API.Core<br/>(ASP.NET Core Web API)"]
|
|
||||||
Seed["Database.Seed<br/>(console app)"]
|
|
||||||
Migrations["Database.Migrations<br/>(console app)"]
|
|
||||||
|
|
||||||
subgraph SLICES["Feature Slices"]
|
|
||||||
direction LR
|
|
||||||
UsersSlice["Features.Users<br/>(controller)"]
|
|
||||||
BrewerySlice["Features.Breweries<br/>(controller)"]
|
|
||||||
PhotoSlice["Features.PhotoUpload<br/>(MediatR only)"]
|
|
||||||
EmailsSlice["Features.Emails<br/>(MediatR only)"]
|
|
||||||
LocationsSlice["Features.Locations<br/>(plain DI service)"]
|
|
||||||
BeersSlice["Features.Beers<br/>(scaffolded, no code yet)"]
|
|
||||||
end
|
|
||||||
|
|
||||||
API -->|AddApplicationPart + MediatR scan| UsersSlice
|
|
||||||
API -->|AddApplicationPart + MediatR scan| BrewerySlice
|
|
||||||
API -->|MediatR scan only| PhotoSlice
|
|
||||||
API -->|MediatR scan only| EmailsSlice
|
|
||||||
|
|
||||||
Seed -->|AddFeaturesUsers| UsersSlice
|
|
||||||
Seed -->|AddFeaturesBreweries| BrewerySlice
|
|
||||||
Seed -->|AddFeaturesPhotoUpload| PhotoSlice
|
|
||||||
Seed -->|AddFeaturesLocations| LocationsSlice
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
---
|
|
||||||
config:
|
|
||||||
layout: elk
|
|
||||||
---
|
|
||||||
|
|
||||||
flowchart TB
|
|
||||||
subgraph SLICES["Feature Slices"]
|
|
||||||
direction LR
|
|
||||||
BrewerySlice["Features.Breweries"]
|
|
||||||
LocationsSlice["Features.Locations"]
|
|
||||||
BeersSlice["Features.Beers<br>(scaffolded)"]
|
|
||||||
PhotoSlice["Features.PhotoUpload"]
|
|
||||||
UsersSlice["Features.Users"]
|
|
||||||
EmailsSlice["Features.Emails"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph FOUNDATION["Foundation (every DB-backed slice depends on all three)"]
|
|
||||||
direction LR
|
|
||||||
SharedContracts["Shared.Contracts"]
|
|
||||||
Domain["Domain.Entities"]
|
|
||||||
DBConn["Database.Connection"]
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph INFRA["Slice-specific Infrastructure"]
|
|
||||||
direction LR
|
|
||||||
PwdHash["Infrastructure.PasswordHashing"]
|
|
||||||
EmailTemplates["Infrastructure.Email.Templates"]
|
|
||||||
end
|
|
||||||
|
|
||||||
UsersSlice == project ref, sends UploadPhotoCommand ==> PhotoSlice
|
|
||||||
EmailsSlice == project ref, handles UserRegisteredNotification & ConfirmationEmailResendRequestedNotification ==> UsersSlice
|
|
||||||
BrewerySlice --> FOUNDATION
|
|
||||||
LocationsSlice --> FOUNDATION
|
|
||||||
BeersSlice --> FOUNDATION
|
|
||||||
PhotoSlice --> FOUNDATION
|
|
||||||
UsersSlice --> FOUNDATION & PwdHash
|
|
||||||
EmailsSlice --> EmailTemplates
|
|
||||||
DBConn -- Dapper SQL, no ORM --> Tables[("SQL Server<br>Tables")]
|
|
||||||
@@ -1,558 +0,0 @@
|
|||||||
---
|
|
||||||
title: Docker deployment, configuration, and troubleshooting
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- docker
|
|
||||||
- docker-compose
|
|
||||||
- deployment
|
|
||||||
---
|
|
||||||
|
|
||||||
This document covers Docker deployment, configuration, and troubleshooting for
|
|
||||||
The Biergarten App.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The project uses Docker Compose to orchestrate multiple services:
|
|
||||||
|
|
||||||
- SQL Server 2022 database
|
|
||||||
- Database migrations runner (DbUp)
|
|
||||||
- Database seeder
|
|
||||||
- .NET API
|
|
||||||
- React Router website (SSR frontend)
|
|
||||||
- Test runners
|
|
||||||
|
|
||||||
See the [deployment diagram](diagrams/deployment.md) for visual representation.
|
|
||||||
|
|
||||||
## Docker Compose environments
|
|
||||||
|
|
||||||
### 1. Development (`docker-compose.dev.yaml`)
|
|
||||||
|
|
||||||
**Purpose**: Local development with persistent data
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- Persistent SQL Server volume
|
|
||||||
- NuGet package cache volume (speeds up rebuilds)
|
|
||||||
- Swagger UI enabled
|
|
||||||
- Seed data included
|
|
||||||
- Local Mailpit SMTP server for dev email testing
|
|
||||||
- Local SeaweedFS S3-compatible object storage for photo uploads
|
|
||||||
- `CLEAR_DATABASE=true` (drops and recreates schema)
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sqlserver # SQL Server 2022 (port 1434)
|
|
||||||
database.migrations # DbUp migrations
|
|
||||||
database.seed # Seed initial data
|
|
||||||
api.core # Web API (ports 8080, 8081)
|
|
||||||
frontend # React Router website (port 3000)
|
|
||||||
mailpit # Local dev SMTP server + UI (port 8025)
|
|
||||||
seaweedfs # S3-compatible object storage (ports 8333, 8888, 9333, 23646)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Start Development Environment**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
**Access**:
|
|
||||||
|
|
||||||
- Website: http://localhost:3000
|
|
||||||
- API Swagger: http://localhost:8080/swagger
|
|
||||||
- Health Check: http://localhost:8080/health
|
|
||||||
- Mailpit UI: http://localhost:8025
|
|
||||||
- SQL Server: localhost:1434 (sa credentials from .env.dev)
|
|
||||||
- SeaweedFS S3 API: http://localhost:8333 (credentials from
|
|
||||||
`SEAWEEDFS_ACCESS_KEY_ID`/`SEAWEEDFS_SECRET_ACCESS_KEY` in .env.dev)
|
|
||||||
- SeaweedFS Filer UI: http://localhost:8888
|
|
||||||
- SeaweedFS Master UI: http://localhost:9333
|
|
||||||
|
|
||||||
**Stop Environment**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Stop services (keep volumes)
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
|
|
||||||
|
|
||||||
# Stop and remove volumes (fresh start)
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Testing (`docker-compose.test.yaml`)
|
|
||||||
|
|
||||||
**Purpose**: Automated CI/CD testing in an isolated environment
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- Fresh database each run
|
|
||||||
- All test suites execute in parallel
|
|
||||||
- Test results exported to `./test-results/`
|
|
||||||
- Containers auto-exit after completion
|
|
||||||
- Fully isolated testnet network
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sqlserver # Test database
|
|
||||||
database.migrations # Fresh schema
|
|
||||||
database.seed # Test data
|
|
||||||
api.specs # Reqnroll BDD tests
|
|
||||||
unit.tests # All Features.*.Tests and Shared.*.Tests unit test projects
|
|
||||||
frontend.tests # Storybook Vitest + Playwright suites (own Playwright image)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Run Tests**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run all tests
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests
|
|
||||||
|
|
||||||
# View results
|
|
||||||
ls -la test-results/
|
|
||||||
cat test-results/api-specs/results.trx
|
|
||||||
cat test-results/Features.Users.Tests.trx
|
|
||||||
|
|
||||||
# Clean up
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
`wait` blocks until `api.specs`, `unit.tests`, and `frontend.tests` have all
|
|
||||||
stopped, regardless of which one finishes first; `up --abort-on-container-exit`
|
|
||||||
isn't used here because the one-shot `database.migrations`/`database.seed` jobs
|
|
||||||
and `frontend.tests` (no database dependency) can exit before the other test
|
|
||||||
containers, which would end the run prematurely. See
|
|
||||||
[Testing](testing.md#running-tests-with-docker-recommended) for details.
|
|
||||||
|
|
||||||
### 3. Production (`docker-compose.prod.yaml`)
|
|
||||||
|
|
||||||
**Purpose**: Production-ready deployment
|
|
||||||
|
|
||||||
**Features**:
|
|
||||||
|
|
||||||
- Production logging levels
|
|
||||||
- No database clearing
|
|
||||||
- Optimized build configurations
|
|
||||||
- Health checks on every long-running service
|
|
||||||
- Restart policies that survive a reboot (see
|
|
||||||
[Restart tolerance](#restart-tolerance))
|
|
||||||
- Security hardening
|
|
||||||
- Only `frontend` publishes a port to the host, bound to `127.0.0.1` so a reverse
|
|
||||||
proxy can sit in front of it; `sqlserver`, `api.core` and `seaweedfs` are
|
|
||||||
reachable only from other containers on `prodnet` (named `biergarten-prodnet`
|
|
||||||
so the seed job can join it from its own Compose project)
|
|
||||||
- Seeding is a separate, explicitly run compose file
|
|
||||||
(`docker-compose.prod.seed.yaml`), so an `up` or a restart leaves the existing
|
|
||||||
catalogue as it is. See [Production data](#production-data) below.
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sqlserver # Production SQL Server (internal only)
|
|
||||||
api.core # Production API (internal only, no published ports)
|
|
||||||
seaweedfs # S3-compatible object storage (internal only)
|
|
||||||
frontend # Production website (127.0.0.1:5656, the only host binding)
|
|
||||||
```
|
|
||||||
|
|
||||||
Migrating and seeding are run-once jobs kept outside this file, so an `up` or a
|
|
||||||
restart touches neither the schema nor the catalogue. Run them explicitly at
|
|
||||||
deploy time: see [Migrating production](#migrating-production) and
|
|
||||||
[Seeding production](#seeding-production-docker-composeprodseedyaml).
|
|
||||||
|
|
||||||
**Deploy Production**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# One time, before the first `up`: the data volumes are external, so Compose
|
|
||||||
# expects them to exist already.
|
|
||||||
docker volume create biergartendata-prod
|
|
||||||
docker volume create seaweedfsdata-prod
|
|
||||||
|
|
||||||
docker compose --env-file web/.env.prod -f web/docker-compose.prod.yaml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Docker seeds each volume with the ownership the image expects and labels it for
|
|
||||||
SELinux, so the host needs no `chown` and no `:z`/`:Z` mount options on
|
|
||||||
enforcing distributions such as RHEL, AlmaLinux and Fedora.
|
|
||||||
|
|
||||||
#### Migrating production
|
|
||||||
|
|
||||||
`database.migrations` is a run-once job rather than a service in the prod stack,
|
|
||||||
so the schema is migrated when an operator asks for it instead of on every boot.
|
|
||||||
Build the image and run it against the network the prod stack creates:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -t database.migrations \
|
|
||||||
-f web/backend/Database/Database.Migrations/Dockerfile \
|
|
||||||
--build-arg BUILD_CONFIGURATION=Release --build-arg APP_UID=1000 \
|
|
||||||
web/backend
|
|
||||||
|
|
||||||
docker run --rm --env-file web/.env.prod --network biergarten-prodnet \
|
|
||||||
-e DOTNET_RUNNING_IN_CONTAINER=true database.migrations
|
|
||||||
```
|
|
||||||
|
|
||||||
DbUp records every script it has applied, so a re-run applies only what is new.
|
|
||||||
Run this with `sqlserver` already up and healthy, and before the seed job.
|
|
||||||
|
|
||||||
#### Restart tolerance
|
|
||||||
|
|
||||||
`sqlserver`, `api.core`, `seaweedfs` and `frontend` run under
|
|
||||||
`restart: unless-stopped`, so the stack returns on its own after a container
|
|
||||||
crash, a Docker daemon restart or a host reboot, provided Docker itself starts at
|
|
||||||
boot (`systemctl enable docker`). The migration and seed jobs stay out of the
|
|
||||||
stack, so a reboot replays neither.
|
|
||||||
|
|
||||||
The daemon replays restart policies without replaying the `depends_on` graph, so
|
|
||||||
after a reboot the containers come back in arbitrary order and each one tolerates
|
|
||||||
a dependency that arrives late. `api.core` opens its database and S3 connections
|
|
||||||
per request rather than at startup, and `frontend` answers with its own error
|
|
||||||
states while `api.core` is still coming up.
|
|
||||||
|
|
||||||
Health checks report when the stack has converged:
|
|
||||||
|
|
||||||
| Service | Probe |
|
|
||||||
| --- | --- |
|
|
||||||
| `sqlserver` | `sqlcmd -Q 'SELECT 1'`, with a 30s start period and 12 retries |
|
|
||||||
| `api.core` | `GET /health` spoken over bash's `/dev/tcp` — the .NET runtime image carries bash and no HTTP client |
|
|
||||||
| `seaweedfs` | `curl http://localhost:8333/status`, the S3 gateway, which serves once the master, volume and filer servers are up |
|
|
||||||
| `frontend` | `wget --spider http://127.0.0.1:5656/healthz`, answered by Express alone and so independent of the API |
|
|
||||||
|
|
||||||
`sqlserver` also carries `stop_grace_period: 60s`, giving a checkpoint time to
|
|
||||||
finish on shutdown instead of being recovered on the next start.
|
|
||||||
|
|
||||||
Deploy and wait for the stack to report healthy, or check convergence after a
|
|
||||||
reboot:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.prod -f web/docker-compose.prod.yaml up -d --wait
|
|
||||||
docker compose --env-file web/.env.prod -f web/docker-compose.prod.yaml ps
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Seeding production (`docker-compose.prod.seed.yaml`)
|
|
||||||
|
|
||||||
`database.seed` lives in its own compose file, run on demand once the prod stack
|
|
||||||
is up, `sqlserver` is healthy, and the schema is migrated:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.prod -f web/docker-compose.prod.seed.yaml \
|
|
||||||
run --rm --build database.seed
|
|
||||||
```
|
|
||||||
|
|
||||||
The job runs as the `biergarten-prod-seed` Compose project and attaches to the
|
|
||||||
external `biergarten-prodnet` network, which the main prod stack creates, so it
|
|
||||||
reaches `sqlserver` and `seaweedfs` under the same hostnames the API uses. It
|
|
||||||
exits when seeding finishes; `--rm` removes the container. Bring the prod stack
|
|
||||||
up first, because the seed project has no visibility into the other project's
|
|
||||||
health checks, so a run started before SQL Server is accepting connections fails.
|
|
||||||
|
|
||||||
#### Production data
|
|
||||||
|
|
||||||
A production deployment seeded this way serves the AI-generated fixture data
|
|
||||||
produced by the pipeline. Per
|
|
||||||
[Pipeline ethics, bias, and known issues](../pipeline/ETHICS-AND-KNOWN-ISSUES.md),
|
|
||||||
anyone interacting with an application seeded from it has to be told upfront that
|
|
||||||
the content is AI-generated.
|
|
||||||
|
|
||||||
The website meets that obligation with `AiDisclosureDialog`
|
|
||||||
(`web/frontend/app/features/home/components/AiDisclosureDialog.tsx`), which
|
|
||||||
the root route renders on every full page load, whichever route the visitor lands
|
|
||||||
on. It is part of the server-rendered HTML rather than mounted after hydration, so
|
|
||||||
the disclosure is on screen with the first paint. Acknowledging it hides the
|
|
||||||
dialog for the remainder of that page load; a new load shows it again.
|
|
||||||
|
|
||||||
Keep the seed job and the dialog together: any deployment that runs
|
|
||||||
`docker-compose.prod.seed.yaml` needs the disclosure in place to satisfy the
|
|
||||||
ethics documentation.
|
|
||||||
|
|
||||||
### 4. Database only (`docker-compose.db.yaml`)
|
|
||||||
|
|
||||||
**Purpose**: Run just the database for local backend development outside Docker
|
|
||||||
(for example, `dotnet run` against a containerized SQL Server)
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sqlserver # SQL Server 2022 (port 1434)
|
|
||||||
database.migrations # DbUp migrations
|
|
||||||
database.seed # Seed initial data
|
|
||||||
```
|
|
||||||
|
|
||||||
No `api.core` or `mailpit` container. Uses `.env.dev`, same as development.
|
|
||||||
|
|
||||||
**Start**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.db.yaml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Minimal (`docker-compose.min.yaml`)
|
|
||||||
|
|
||||||
**Purpose**: Local dev where the API runs outside Docker but the database and
|
|
||||||
mail capture still run in containers
|
|
||||||
|
|
||||||
**Services**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sqlserver # SQL Server 2022 (port 1433)
|
|
||||||
database.migrations # DbUp migrations
|
|
||||||
database.seed # Seed initial data
|
|
||||||
mailpit # Local dev SMTP server + UI (port 8025)
|
|
||||||
```
|
|
||||||
|
|
||||||
No `api.core` container. Uses `.env.local` instead of `.env.dev`, and exposes
|
|
||||||
SQL Server on the standard port 1433 (`docker-compose.db.yaml` and
|
|
||||||
`docker-compose.dev.yaml` both use 1434).
|
|
||||||
|
|
||||||
**Start**:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.local -f web/docker-compose.min.yaml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## Service dependencies
|
|
||||||
|
|
||||||
Docker Compose manages startup order using health checks:
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
graph TD
|
|
||||||
A[sqlserver: health check passes] --> B[database.migrations: completes successfully]
|
|
||||||
B --> C[database.seed: completes successfully]
|
|
||||||
C --> D[api.core / tests: start when ready]
|
|
||||||
E[seaweedfs: container started] --> D
|
|
||||||
```
|
|
||||||
|
|
||||||
In development, `api.core` also depends on `seaweedfs`
|
|
||||||
(`condition: service_started`, not a health check) alongside `database.seed`.
|
|
||||||
|
|
||||||
Production has a shorter chain, because the migration and seed jobs run outside
|
|
||||||
the stack: `api.core` waits for `seaweedfs` to pass its health check, and
|
|
||||||
`frontend` waits for `api.core` to report healthy. This ordering applies to
|
|
||||||
`docker compose up`; see [Restart tolerance](#restart-tolerance) for what happens
|
|
||||||
on a host reboot.
|
|
||||||
|
|
||||||
**Health Check Example** (SQL Server):
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
healthcheck:
|
|
||||||
test:
|
|
||||||
[
|
|
||||||
"CMD-SHELL",
|
|
||||||
"sqlcmd -S localhost -U sa -P '${DB_PASSWORD}' -C -Q 'SELECT 1'",
|
|
||||||
]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 12
|
|
||||||
start_period: 30s
|
|
||||||
```
|
|
||||||
|
|
||||||
**Dependency Configuration**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
api.core:
|
|
||||||
depends_on:
|
|
||||||
database.seed:
|
|
||||||
condition: service_completed_successfully
|
|
||||||
```
|
|
||||||
|
|
||||||
## Volumes
|
|
||||||
|
|
||||||
### Persistent volumes
|
|
||||||
|
|
||||||
**Development**:
|
|
||||||
|
|
||||||
- `sqlserverdata-dev` - Database files persist between restarts
|
|
||||||
- `nuget-cache-dev` - NuGet package cache (speeds up builds)
|
|
||||||
- `seaweedfsdata-dev` - SeaweedFS object storage data persists between restarts
|
|
||||||
|
|
||||||
**Testing**:
|
|
||||||
|
|
||||||
- `sqlserverdata-test` - Temporary, typically removed after tests
|
|
||||||
|
|
||||||
**Production**:
|
|
||||||
|
|
||||||
- `biergartendata-prod` - Database files, external; survives `down -v`
|
|
||||||
- `seaweedfsdata-prod` - SeaweedFS object storage, external; survives `down -v`
|
|
||||||
|
|
||||||
See [Production data volumes](#production-data-volumes).
|
|
||||||
|
|
||||||
### Mounted volumes
|
|
||||||
|
|
||||||
**Test Results**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
volumes:
|
|
||||||
- ./test-results:/app/test-results
|
|
||||||
```
|
|
||||||
|
|
||||||
Test results are written to host filesystem for CI/CD integration.
|
|
||||||
|
|
||||||
### Production data volumes
|
|
||||||
|
|
||||||
`biergartendata-prod` and `seaweedfsdata-prod` are declared `external: true`.
|
|
||||||
Compose removes only the volumes it owns, so `docker compose down -v` tears the
|
|
||||||
stack down and leaves the database and the uploaded photos in place. Deleting
|
|
||||||
them takes an explicit `docker volume rm`.
|
|
||||||
|
|
||||||
The trade-off is that Compose creates neither one, and `up` fails with a clear
|
|
||||||
error until both exist, hence the `docker volume create` step in
|
|
||||||
[Deploy Production](#3-production-docker-composeprodyaml). External volumes also
|
|
||||||
carry their literal name rather than a `<project>_` prefix, so `docker volume ls`
|
|
||||||
shows `biergartendata-prod` and `seaweedfsdata-prod` unprefixed, unlike the
|
|
||||||
`web_`-prefixed volumes of the dev and test stacks.
|
|
||||||
|
|
||||||
Both live under `/var/lib/docker/volumes`. To put them on a specific disk, point
|
|
||||||
the daemon's `data-root` at it in `/etc/docker/daemon.json`, or mount the disk at
|
|
||||||
`/var/lib/docker`.
|
|
||||||
|
|
||||||
Back up the database through SQL Server rather than by copying its files, since a
|
|
||||||
file copy taken while the engine is running can be torn:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec prod-env-sqlserver /opt/mssql-tools18/bin/sqlcmd \
|
|
||||||
-S localhost -U sa -P '<db-password>' -C \
|
|
||||||
-Q "BACKUP DATABASE [<db-name>] TO DISK = '/var/opt/mssql/data/backup.bak'"
|
|
||||||
docker cp prod-env-sqlserver:/var/opt/mssql/data/backup.bak .
|
|
||||||
```
|
|
||||||
|
|
||||||
Object storage can be archived directly, with the stack stopped:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker run --rm -v seaweedfsdata-prod:/data -v "$PWD:/backup" \
|
|
||||||
alpine tar czf /backup/seaweedfs.tar.gz -C /data .
|
|
||||||
```
|
|
||||||
|
|
||||||
## Networks
|
|
||||||
|
|
||||||
Each environment uses isolated bridge networks:
|
|
||||||
|
|
||||||
- `devnet` - Development network
|
|
||||||
- `testnet` - Testing network (fully isolated)
|
|
||||||
- `prodnet` - Production network
|
|
||||||
|
|
||||||
## Environment variables
|
|
||||||
|
|
||||||
All containers are configured via environment variables from `.env` files:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
env_file: ".env.dev" # or .env.test, .env.prod
|
|
||||||
|
|
||||||
environment:
|
|
||||||
ASPNETCORE_ENVIRONMENT: "Development"
|
|
||||||
DOTNET_RUNNING_IN_CONTAINER: "true"
|
|
||||||
DB_SERVER: "${DB_SERVER}"
|
|
||||||
DB_NAME: "${DB_NAME}"
|
|
||||||
DB_USER: "${DB_USER}"
|
|
||||||
DB_PASSWORD: "${DB_PASSWORD}"
|
|
||||||
ACCESS_TOKEN_SECRET: "${ACCESS_TOKEN_SECRET}"
|
|
||||||
REFRESH_TOKEN_SECRET: "${REFRESH_TOKEN_SECRET}"
|
|
||||||
CONFIRMATION_TOKEN_SECRET: "${CONFIRMATION_TOKEN_SECRET}"
|
|
||||||
SEAWEEDFS_SERVICE_URL: "${SEAWEEDFS_SERVICE_URL}"
|
|
||||||
SEAWEEDFS_ACCESS_KEY_ID: "${SEAWEEDFS_ACCESS_KEY_ID}"
|
|
||||||
SEAWEEDFS_SECRET_ACCESS_KEY: "${SEAWEEDFS_SECRET_ACCESS_KEY}"
|
|
||||||
SEAWEEDFS_BUCKET: "${SEAWEEDFS_BUCKET}"
|
|
||||||
```
|
|
||||||
|
|
||||||
For complete list, see [Environment Variables](environment-variables.md).
|
|
||||||
|
|
||||||
## Common commands
|
|
||||||
|
|
||||||
### View services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Running services
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml ps
|
|
||||||
|
|
||||||
# All containers (including stopped)
|
|
||||||
docker ps -a
|
|
||||||
```
|
|
||||||
|
|
||||||
### View logs
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# All services
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f
|
|
||||||
|
|
||||||
# Specific service
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs -f api.core
|
|
||||||
|
|
||||||
# Last 100 lines
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml logs --tail=100 api.core
|
|
||||||
```
|
|
||||||
|
|
||||||
### Execute commands in container
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Interactive shell
|
|
||||||
docker exec -it dev-env-api-core bash
|
|
||||||
|
|
||||||
# Run command (replace <db-password> with the value of DB_PASSWORD from .env.dev)
|
|
||||||
docker exec dev-env-sqlserver /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '<db-password>' -C
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restart services
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Restart all services
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart
|
|
||||||
|
|
||||||
# Restart specific service
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml restart api.core
|
|
||||||
|
|
||||||
# Rebuild and restart
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml up -d --build api.core
|
|
||||||
```
|
|
||||||
|
|
||||||
### Build images
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Build all images
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build
|
|
||||||
|
|
||||||
# Build specific service
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build api.core
|
|
||||||
|
|
||||||
# Build without cache
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml build --no-cache
|
|
||||||
```
|
|
||||||
|
|
||||||
### Clean up
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Stop and remove containers
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down
|
|
||||||
|
|
||||||
# Remove containers and volumes
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v
|
|
||||||
|
|
||||||
# Remove containers, volumes, and images
|
|
||||||
docker compose --env-file web/.env.dev -f web/docker-compose.dev.yaml down -v --rmi all
|
|
||||||
|
|
||||||
# System-wide cleanup
|
|
||||||
docker system prune -af --volumes
|
|
||||||
```
|
|
||||||
|
|
||||||
## Dockerfile structure
|
|
||||||
|
|
||||||
### Multi-stage build
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
# Stage 1: Build
|
|
||||||
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
|
|
||||||
WORKDIR /src
|
|
||||||
COPY ["Project/Project.csproj", "Project/"]
|
|
||||||
RUN dotnet restore
|
|
||||||
COPY . .
|
|
||||||
RUN dotnet build -c Release
|
|
||||||
|
|
||||||
# Stage 2: Runtime
|
|
||||||
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS final
|
|
||||||
WORKDIR /app
|
|
||||||
COPY --from=build /app/build .
|
|
||||||
ENTRYPOINT ["dotnet", "Project.dll"]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Additional resources
|
|
||||||
|
|
||||||
- [Docker Compose Documentation](https://docs.docker.com/compose/)
|
|
||||||
- [.NET Docker Images](https://hub.docker.com/_/microsoft-dotnet)
|
|
||||||
- [SQL Server Docker Images](https://hub.docker.com/_/microsoft-mssql-server)
|
|
||||||
@@ -1,488 +0,0 @@
|
|||||||
---
|
|
||||||
title: Testing strategy — unit, integration, and Storybook suites
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- testing
|
|
||||||
- ci
|
|
||||||
- xunit
|
|
||||||
- reqnroll
|
|
||||||
- storybook
|
|
||||||
---
|
|
||||||
|
|
||||||
This document describes the testing strategy and how to run tests for The
|
|
||||||
Biergarten App.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The project uses a multi-layered testing approach across backend and frontend:
|
|
||||||
|
|
||||||
| Suite | Purpose |
|
|
||||||
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| **API.Specs** | BDD integration tests using Reqnroll (Gherkin), run against a live, seeded database |
|
|
||||||
| **Features.\*.Tests** | One unit test project per backend feature slice (`Features.Users.Tests`, `Features.Breweries.Tests`, `Features.Emails.Tests`, `Features.PhotoUpload.Tests`), covering that slice's command/query handlers, with dependencies mocked via Moq (no real database required) |
|
|
||||||
| **Storybook Vitest project** | Browser-based interaction tests for shared website stories |
|
|
||||||
| **Storybook Playwright suite** | Browser checks against Storybook-rendered components |
|
|
||||||
|
|
||||||
## Running tests with Docker (recommended)
|
|
||||||
|
|
||||||
To run all tests, use Docker Compose, which sets up an isolated test
|
|
||||||
environment:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests
|
|
||||||
```
|
|
||||||
|
|
||||||
This command:
|
|
||||||
|
|
||||||
1. Starts a fresh SQL Server instance
|
|
||||||
2. Runs database migrations
|
|
||||||
3. Seeds test data
|
|
||||||
4. Executes all test suites in parallel, including the frontend's Storybook
|
|
||||||
Vitest and Playwright suites (`frontend.tests`, which needs no database)
|
|
||||||
5. Exports results to `./test-results/`
|
|
||||||
6. Returns once all three test containers have exited
|
|
||||||
|
|
||||||
`wait` blocks until `api.specs`, `unit.tests`, and `frontend.tests` have all
|
|
||||||
stopped, regardless of which one finishes first. Long-running services
|
|
||||||
(`sqlserver`, `seaweedfs`) and the one-shot `database.migrations` /
|
|
||||||
`database.seed` jobs keep running in the background alongside them and are torn
|
|
||||||
down separately (see below); `wait` doesn't touch them.
|
|
||||||
|
|
||||||
### View test results
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List test result files
|
|
||||||
ls -la test-results/
|
|
||||||
|
|
||||||
# View specific test results
|
|
||||||
cat test-results/api-specs/results.trx
|
|
||||||
cat test-results/Features.Users.Tests.trx
|
|
||||||
cat test-results/Features.Breweries.Tests.trx
|
|
||||||
cat test-results/Features.Emails.Tests.trx
|
|
||||||
cat test-results/Features.PhotoUpload.Tests.trx
|
|
||||||
cat test-results/frontend/vitest-storybook.xml
|
|
||||||
cat test-results/frontend/playwright-storybook.xml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Clean up
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Remove test containers and volumes
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
## Running tests locally
|
|
||||||
|
|
||||||
You can run individual test projects locally without Docker:
|
|
||||||
|
|
||||||
### Feature slice and shared unit tests
|
|
||||||
|
|
||||||
Each feature slice has its own test project, covering its command/query
|
|
||||||
handlers. Shared cross-cutting code (e.g. MediatR pipeline behaviors) has
|
|
||||||
its own test project under `Shared/`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd web/backend
|
|
||||||
dotnet test Features/Features.Users.Tests/Features.Users.Tests.csproj
|
|
||||||
dotnet test Features/Features.Breweries.Tests/Features.Breweries.Tests.csproj
|
|
||||||
dotnet test Features/Features.Emails.Tests/Features.Emails.Tests.csproj
|
|
||||||
dotnet test Features/Features.PhotoUpload.Tests/Features.PhotoUpload.Tests.csproj
|
|
||||||
dotnet test Shared/Shared.Application.Tests/Shared.Application.Tests.csproj
|
|
||||||
|
|
||||||
# Or run all of them at once via the solution:
|
|
||||||
for proj in Features/*.Tests Shared/*.Tests; do dotnet test "$proj"; done
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements**:
|
|
||||||
|
|
||||||
- No database required (handlers are tested with their repository/service
|
|
||||||
dependencies mocked via Moq)
|
|
||||||
|
|
||||||
### Frontend Storybook tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd web/frontend
|
|
||||||
npm install
|
|
||||||
npm run test:storybook
|
|
||||||
```
|
|
||||||
|
|
||||||
### Integration tests (API.Specs)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd web/backend
|
|
||||||
dotnet test API/API.Specs/API.Specs.csproj
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements**:
|
|
||||||
|
|
||||||
- SQL Server instance running
|
|
||||||
- Database migrated and seeded
|
|
||||||
- Environment variables set (DB connection, JWT secret)
|
|
||||||
|
|
||||||
**Purpose**:
|
|
||||||
|
|
||||||
- Verifies shared stories such as form fields, submit buttons, navbar states,
|
|
||||||
toasts, and the theme gallery
|
|
||||||
- Runs in browser mode via Vitest and Storybook integration
|
|
||||||
|
|
||||||
### Frontend Playwright Storybook tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd web/frontend
|
|
||||||
npm install
|
|
||||||
npm run test:storybook:playwright
|
|
||||||
```
|
|
||||||
|
|
||||||
**Requirements**:
|
|
||||||
|
|
||||||
- Storybook dependencies installed
|
|
||||||
- Playwright browser dependencies installed
|
|
||||||
- The command will start or reuse the Storybook server defined in
|
|
||||||
`playwright.storybook.config.ts`
|
|
||||||
|
|
||||||
### Running Storybook tests with a visible browser
|
|
||||||
|
|
||||||
Both of the above run headless by default. To watch the browser while debugging
|
|
||||||
a failure:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Playwright Storybook suite
|
|
||||||
npx playwright test -c playwright.storybook.config.ts --headed
|
|
||||||
# or the interactive step-through runner
|
|
||||||
npx playwright test -c playwright.storybook.config.ts --ui
|
|
||||||
|
|
||||||
# Vitest/story play-function suite
|
|
||||||
npx vitest run --project storybook --browser.headless=false
|
|
||||||
# or Vitest's own UI (drop `run` for watch mode)
|
|
||||||
npx vitest --project storybook --browser.headless=false --ui
|
|
||||||
```
|
|
||||||
|
|
||||||
The default (`headless: true`) is set in `web/frontend/vite.config.ts`.
|
|
||||||
|
|
||||||
## Test coverage
|
|
||||||
|
|
||||||
### Current coverage
|
|
||||||
|
|
||||||
**Features.Users.Tests**:
|
|
||||||
|
|
||||||
- User registration with validation
|
|
||||||
- User login with JWT token generation
|
|
||||||
- Password hashing and verification (Argon2id)
|
|
||||||
- Email confirmation and confirmation-email resend
|
|
||||||
- Refresh token exchange
|
|
||||||
- JWT token generation, validation, and claims handling
|
|
||||||
- Invalid credentials and 404 error responses (via API.Specs)
|
|
||||||
- Account management: username, email, password, and profile updates, and
|
|
||||||
account deletion (blocked while dependent posts, comments, photos, or follows
|
|
||||||
exist)
|
|
||||||
- User get-by-id/get-all queries
|
|
||||||
|
|
||||||
**Features.Breweries.Tests**:
|
|
||||||
|
|
||||||
- Brewery create/update/delete commands and get-by-id/get-all queries
|
|
||||||
- Brewery location queries: all locations, and locations within a given range of
|
|
||||||
a coordinate
|
|
||||||
|
|
||||||
**Features.Emails.Tests**:
|
|
||||||
|
|
||||||
- Registration and resend-confirmation email dispatch handlers
|
|
||||||
|
|
||||||
**Features.PhotoUpload.Tests**:
|
|
||||||
|
|
||||||
- Upload command handler: storage key construction, content type passthrough,
|
|
||||||
and photo persistence via the mocked storage provider and repository
|
|
||||||
- Upload validator: min/max file size bounds, and PNG/JPEG/WebP signature checks
|
|
||||||
(including rejection of files with a mismatched or missing signature)
|
|
||||||
|
|
||||||
**Frontend UI Coverage**:
|
|
||||||
|
|
||||||
- Shared submit button states
|
|
||||||
- Form field happy path and error presentation
|
|
||||||
- Navbar guest, authenticated, and mobile behavior
|
|
||||||
- Theme gallery rendering across Biergarten themes
|
|
||||||
- Toast interactions and themed notification display
|
|
||||||
|
|
||||||
### Planned coverage
|
|
||||||
|
|
||||||
- [ ] Password reset functionality
|
|
||||||
- [ ] Beer post operations
|
|
||||||
- [ ] User follow/unfollow
|
|
||||||
- [ ] Frontend route integration coverage beyond Storybook stories
|
|
||||||
|
|
||||||
## Testing frameworks and tools
|
|
||||||
|
|
||||||
### xUnit
|
|
||||||
|
|
||||||
- Primary unit testing framework
|
|
||||||
- Used for handler and service layer tests
|
|
||||||
- Supports parallel test execution
|
|
||||||
|
|
||||||
### Reqnroll (Gherkin/BDD)
|
|
||||||
|
|
||||||
- Behavior-driven development framework
|
|
||||||
- Used for API integration tests
|
|
||||||
- Human-readable test scenarios in `.feature` files
|
|
||||||
|
|
||||||
### FluentAssertions
|
|
||||||
|
|
||||||
- Expressive assertion library
|
|
||||||
- Makes test assertions more readable
|
|
||||||
- Used across all test projects
|
|
||||||
|
|
||||||
### Moq
|
|
||||||
|
|
||||||
- Mocking framework for .NET
|
|
||||||
- Used in Service layer tests
|
|
||||||
- Enables isolated unit testing
|
|
||||||
|
|
||||||
## Test structure
|
|
||||||
|
|
||||||
### API.Specs (integration tests)
|
|
||||||
|
|
||||||
```
|
|
||||||
API.Specs/
|
|
||||||
├── Features/
|
|
||||||
│ ├── Registration.feature # Registration scenarios
|
|
||||||
│ ├── Login.feature # Login scenarios
|
|
||||||
│ ├── Confirmation.feature # Email confirmation scenarios
|
|
||||||
│ ├── ResendConfirmation.feature # Resend-confirmation scenarios
|
|
||||||
│ ├── TokenRefresh.feature # Refresh token scenarios
|
|
||||||
│ ├── AccessTokenValidation.feature # Protected endpoint access scenarios
|
|
||||||
│ ├── NotFound.feature # 404 handling
|
|
||||||
│ └── PublicUserProfile.feature # Public profile & authenticated account scenarios
|
|
||||||
├── Steps/
|
|
||||||
│ ├── AuthSteps.cs # Step definitions for the Auth features
|
|
||||||
│ ├── UserSteps.cs # Step definitions for the public profile & user-account features
|
|
||||||
│ └── ApiGeneralSteps.cs # Shared/general step definitions
|
|
||||||
├── Mocks/
|
|
||||||
│ ├── MockEmailDispatcher.cs # Substitutes Features.Emails' IEmailDispatcher
|
|
||||||
│ ├── MockEmailProvider.cs # Substitutes Features.Emails' IEmailProvider
|
|
||||||
│ └── MockFileStorageProvider.cs # Substitutes Features.ImageUploads' IFileStorageProvider
|
|
||||||
└── TestApiFactory.cs # Test server setup (swaps in the mocks above)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Example Feature**:
|
|
||||||
|
|
||||||
```gherkin
|
|
||||||
Feature: User Registration
|
|
||||||
As a user
|
|
||||||
I want to register
|
|
||||||
So that I can access the platform
|
|
||||||
|
|
||||||
Scenario: Successful user registration
|
|
||||||
Given I have valid registration details
|
|
||||||
When I register a new account
|
|
||||||
Then I should receive a JWT token
|
|
||||||
And my account should be created
|
|
||||||
```
|
|
||||||
|
|
||||||
### Features.Users.Tests
|
|
||||||
|
|
||||||
```
|
|
||||||
Features.Users.Tests/
|
|
||||||
├── Commands/
|
|
||||||
│ ├── RegisterUserHandlerTests.cs
|
|
||||||
│ ├── RegisterUserValidatorTests.cs
|
|
||||||
│ ├── ConfirmUserHandlerTests.cs
|
|
||||||
│ ├── ResendConfirmationEmailHandlerTests.cs
|
|
||||||
│ ├── RefreshTokenHandlerTests.cs
|
|
||||||
│ ├── RefreshTokenValidatorTests.cs
|
|
||||||
│ ├── LoginValidatorTests.cs # tests LoginCommand's validator
|
|
||||||
│ ├── UpdateUsernameHandlerTests.cs
|
|
||||||
│ ├── UpdateUsernameValidatorTests.cs
|
|
||||||
│ ├── UpdateEmailHandlerTests.cs
|
|
||||||
│ ├── UpdateEmailValidatorTests.cs
|
|
||||||
│ ├── UpdatePasswordHandlerTests.cs
|
|
||||||
│ ├── UpdatePasswordValidatorTests.cs
|
|
||||||
│ ├── UpdateProfileHandlerTests.cs
|
|
||||||
│ ├── UpdateProfileValidatorTests.cs
|
|
||||||
│ └── DeleteAccountHandlerTests.cs
|
|
||||||
├── Queries/
|
|
||||||
│ ├── LoginHandlerTests.cs # tests LoginCommand, despite the folder name
|
|
||||||
│ ├── GetAllUsersHandlerTests.cs
|
|
||||||
│ ├── GetPublicUserProfileByIdHandlerTests.cs
|
|
||||||
│ └── GetUserByIdHandlerTests.cs
|
|
||||||
├── Services/
|
|
||||||
│ ├── TokenServiceRefreshTests.cs
|
|
||||||
│ └── TokenServiceValidationTests.cs
|
|
||||||
└── TestSupport/ # shared test fixtures/helpers
|
|
||||||
```
|
|
||||||
|
|
||||||
`Features.Breweries.Tests` and `Features.PhotoUpload.Tests` follow the same
|
|
||||||
shape: a `Commands/`/`Queries/` folder with one test file per handler, plus a
|
|
||||||
`*ValidatorTests.cs` file beside each handler whose command or query has a
|
|
||||||
FluentValidation validator (`Features.Breweries.Tests` has one of these in
|
|
||||||
`Queries/` as well, for the nearby-search query). `Features.Emails.Tests`
|
|
||||||
differs: its handlers respond to MediatR notifications, so its test files live
|
|
||||||
under `Notifications/`.
|
|
||||||
|
|
||||||
## Writing tests
|
|
||||||
|
|
||||||
### Unit test example (xUnit)
|
|
||||||
|
|
||||||
```csharp
|
|
||||||
public class LoginHandlerTests
|
|
||||||
{
|
|
||||||
[Fact]
|
|
||||||
public async Task Handle_WithValidData_ReturnsPayloadWithMatchingUsername()
|
|
||||||
{
|
|
||||||
// Arrange
|
|
||||||
var authRepoMock = new Mock<IAuthRepository>();
|
|
||||||
var passwordInfraMock = new Mock<IPasswordInfrastructure>();
|
|
||||||
var tokenServiceMock = new Mock<ITokenService>();
|
|
||||||
var handler = new LoginHandler(authRepoMock.Object, passwordInfraMock.Object, tokenServiceMock.Object);
|
|
||||||
// ...set up mocks for a known user/credential...
|
|
||||||
|
|
||||||
// Act
|
|
||||||
var result = await handler.Handle(new LoginCommand("testuser", "password123"), CancellationToken.None);
|
|
||||||
|
|
||||||
// Assert
|
|
||||||
result.Should().NotBeNull();
|
|
||||||
result.AccessToken.Should().NotBeNullOrEmpty();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Integration test example (Reqnroll)
|
|
||||||
|
|
||||||
```gherkin
|
|
||||||
Scenario: User login with valid credentials
|
|
||||||
Given a registered user with username "testuser"
|
|
||||||
When I POST to "/api/auth/login" with valid credentials
|
|
||||||
Then the response status should be 200
|
|
||||||
And the response should contain a JWT token
|
|
||||||
```
|
|
||||||
|
|
||||||
## Continuous integration
|
|
||||||
|
|
||||||
The `.github/workflows/tests.yml` GitHub Actions workflow runs on every push and
|
|
||||||
pull request to `main`. It runs the same test Docker Compose configuration used
|
|
||||||
locally, so backend and frontend tests execute in the same containerized
|
|
||||||
environment in CI as they do on a developer machine:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# CI/CD command
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml build
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml up -d
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml wait api.specs unit.tests frontend.tests
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
The workflow inspects the exit code of each test container
|
|
||||||
(`test-env-api-specs`, `test-env-unit-tests`, `test-env-frontend-tests`) after
|
|
||||||
`wait` returns and fails the job if any of them is non-zero. `./test-results/`
|
|
||||||
is uploaded as a build artifact regardless of outcome.
|
|
||||||
|
|
||||||
`up -d` + `wait` keeps container teardown independent of any single container's
|
|
||||||
exit. The one-shot `database.migrations`/`database.seed` jobs exit 0 partway
|
|
||||||
through the run, and `frontend.tests` (which has no database dependency)
|
|
||||||
reliably finishes before `api.specs`/`unit.tests` start; because `wait` blocks
|
|
||||||
on all three test containers, neither early exit ends the run prematurely.
|
|
||||||
|
|
||||||
Exit codes:
|
|
||||||
|
|
||||||
- `0` - All tests passed
|
|
||||||
- Non-zero - Test failures occurred
|
|
||||||
|
|
||||||
The frontend's Storybook Vitest and Playwright suites run inside the
|
|
||||||
`frontend.tests` container (see [Docker Guide](docker.md)) as part of that same
|
|
||||||
compose run, so no separate frontend CI step is needed. To run them locally
|
|
||||||
without Docker:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd web/frontend
|
|
||||||
npm ci
|
|
||||||
npm run test:storybook
|
|
||||||
npm run test:storybook:playwright
|
|
||||||
```
|
|
||||||
|
|
||||||
### Running the workflow locally with `act`
|
|
||||||
|
|
||||||
[`act`](https://github.com/nektos/act) replays `.github/workflows/*.yml` on your
|
|
||||||
own machine using Docker, so you can reproduce a CI run (or a fix for one)
|
|
||||||
without pushing a branch.
|
|
||||||
|
|
||||||
**Install** (macOS):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
brew install act
|
|
||||||
```
|
|
||||||
|
|
||||||
**Run the workflow** from the repo root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
act push \
|
|
||||||
-j containerized-tests \
|
|
||||||
-W .github/workflows/tests.yml \
|
|
||||||
-P ubuntu-latest=catthehacker/ubuntu:act-latest \
|
|
||||||
--container-architecture linux/amd64
|
|
||||||
```
|
|
||||||
|
|
||||||
- `-j containerized-tests` runs only that job (there's currently just the one).
|
|
||||||
- `-P ubuntu-latest=catthehacker/ubuntu:act-latest` pins the runner image act
|
|
||||||
uses to impersonate `ubuntu-latest`. Without it, act's first run prompts
|
|
||||||
interactively to choose a default image size, which hangs/fails under a
|
|
||||||
non-interactive shell.
|
|
||||||
- `--container-architecture linux/amd64` is required on Apple Silicon: the
|
|
||||||
`sqlserver` service's image is amd64-only, and act needs to know to emulate
|
|
||||||
that platform for the whole job container, not just that one service.
|
|
||||||
- No secrets or `.actrc` are needed: `generate-env.sh` creates its own
|
|
||||||
`.env.test` with freshly randomized values, same as in real CI.
|
|
||||||
|
|
||||||
**Known act-only limitation**: the "Upload test results" step
|
|
||||||
(`actions/upload-artifact@v4`) fails locally with
|
|
||||||
`Unable to get the ACTIONS_RUNTIME_TOKEN env variable`. act doesn't provide a
|
|
||||||
real Actions artifact backend, so this step, and only this step, is expected to
|
|
||||||
fail under act even when everything else passes. It works normally on
|
|
||||||
GitHub-hosted runners.
|
|
||||||
|
|
||||||
**Stale local state**: `docker-compose.test.yaml` uses fixed container names and
|
|
||||||
named volumes (`sqlserverdata-test`, `seaweedfsdata-test`) scoped to the `web`
|
|
||||||
compose project. If a previous local run (via `act` or a manual `docker compose`
|
|
||||||
invocation) didn't get torn down, its SQL Server volume can persist with an old
|
|
||||||
`SA_PASSWORD` baked in, while a fresh `.env.test` generates a new one each run,
|
|
||||||
causing `Login failed for user 'sa'` errors that look like a test bug but are
|
|
||||||
really a leftover volume. Clear it before re-running:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Tests failing due to database connection
|
|
||||||
|
|
||||||
Ensure SQL Server is running and environment variables are set:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml ps
|
|
||||||
```
|
|
||||||
|
|
||||||
### Port conflicts
|
|
||||||
|
|
||||||
If port 1433 is in use, stop other SQL Server instances or modify the port in
|
|
||||||
`docker-compose.test.yaml`.
|
|
||||||
|
|
||||||
### Stale test data
|
|
||||||
|
|
||||||
Clean up test database:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml down -v
|
|
||||||
```
|
|
||||||
|
|
||||||
### View container logs
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file web/.env.test -f web/docker-compose.test.yaml logs <service-name>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Best practices
|
|
||||||
|
|
||||||
1. **Isolation**: Each test should be independent and not rely on other tests
|
|
||||||
2. **Cleanup**: Use fixtures and dispose patterns for resource cleanup
|
|
||||||
3. **Mocking**: Mock external dependencies in unit tests
|
|
||||||
4. **Descriptive Names**: Use clear, descriptive test method names
|
|
||||||
5. **Arrange-Act-Assert**: Follow AAA pattern in unit tests
|
|
||||||
6. **Given-When-Then**: Follow GWT pattern in BDD scenarios
|
|
||||||
@@ -1,293 +0,0 @@
|
|||||||
---
|
|
||||||
title: JWT token validation architecture
|
|
||||||
last-updated: 2026-08-31
|
|
||||||
tags:
|
|
||||||
- jwt
|
|
||||||
- authentication
|
|
||||||
- security
|
|
||||||
- tokens
|
|
||||||
---
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Biergarten API validates JSON Web Tokens (JWTs) across three token types:
|
|
||||||
|
|
||||||
| Token Type | Lifetime | Purpose |
|
|
||||||
| ----------------------- | ---------- | --------------------------- |
|
|
||||||
| **Access Tokens** | 1 hour | API authentication |
|
|
||||||
| **Refresh Tokens** | 21 days | Obtaining new access tokens |
|
|
||||||
| **Confirmation Tokens** | 30 minutes | Email confirmation |
|
|
||||||
|
|
||||||
## Components
|
|
||||||
|
|
||||||
### Infrastructure layer
|
|
||||||
|
|
||||||
#### [ITokenInfrastructure](../../web/backend/Features/Features.Users/Services/ITokenInfrastructure.cs)
|
|
||||||
|
|
||||||
Low-level JWT operations.
|
|
||||||
|
|
||||||
**Methods:**
|
|
||||||
|
|
||||||
- `GenerateJwt()` - Creates signed JWT tokens
|
|
||||||
- `ValidateJwtAsync()` - Validates token signature, expiration, and format
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
[JwtInfrastructure.cs](../../web/backend/Features/Features.Users/Services/JwtInfrastructure.cs)
|
|
||||||
|
|
||||||
- Uses Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler
|
|
||||||
- Algorithm: HS256 (HMAC-SHA256)
|
|
||||||
- Validates token lifetime, signature, and well-formedness
|
|
||||||
|
|
||||||
### Features.Users slice
|
|
||||||
|
|
||||||
#### [ITokenService](../../web/backend/Features/Features.Users/Services/ITokenService.cs)
|
|
||||||
|
|
||||||
Token generation and validation live on the same slice-internal service.
|
|
||||||
|
|
||||||
**Generation methods:**
|
|
||||||
|
|
||||||
- `GenerateAccessToken(Guid userId, string username)` - Creates 1-hour access
|
|
||||||
token
|
|
||||||
- `GenerateRefreshToken(Guid userId, string username)` - Creates 21-day refresh
|
|
||||||
token
|
|
||||||
- `GenerateConfirmationToken(Guid userId, string username)` - Creates 30-minute
|
|
||||||
confirmation token
|
|
||||||
|
|
||||||
**Validation methods:**
|
|
||||||
|
|
||||||
- `ValidateRefreshTokenAsync(string token)` - Validates refresh tokens
|
|
||||||
- `ValidateConfirmationTokenAsync(string token)` - Validates confirmation tokens
|
|
||||||
|
|
||||||
There is no `ValidateAccessTokenAsync` on `ITokenService`. Access-token
|
|
||||||
validation doesn't go through `ITokenService` at all: `JwtAuthenticationHandler`
|
|
||||||
(the host-level auth middleware) calls `ITokenInfrastructure.ValidateJwtAsync()`
|
|
||||||
directly, bypassing the `Features.Users` slice entirely.
|
|
||||||
|
|
||||||
**Returns (validation):** `ValidatedToken` record containing:
|
|
||||||
|
|
||||||
- `UserId` (Guid)
|
|
||||||
- `Username` (string)
|
|
||||||
- `Principal` (ClaimsPrincipal) - Full JWT claims
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
[TokenService.cs](../../web/backend/Features/Features.Users/Services/TokenService.cs)
|
|
||||||
|
|
||||||
- Reads token secrets from environment variables
|
|
||||||
- Extracts and validates claims (Sub, UniqueName)
|
|
||||||
- Throws `UnauthorizedException` on validation failure
|
|
||||||
|
|
||||||
`TokenService` and the lower-level `ITokenInfrastructure` (JWT
|
|
||||||
signing/verification) it depends on are both registered by `Features.Users`'
|
|
||||||
own `AddFeaturesUsers()` extension method. `JwtAuthenticationHandler`
|
|
||||||
(host-level auth middleware) also depends on `ITokenInfrastructure` directly,
|
|
||||||
resolving it from the same DI container.
|
|
||||||
|
|
||||||
### Integration points
|
|
||||||
|
|
||||||
#### [ConfirmUserHandler](../../web/backend/Features/Features.Users/Commands/Authentication/ConfirmUser/ConfirmUserHandler.cs)
|
|
||||||
|
|
||||||
**Flow:**
|
|
||||||
|
|
||||||
1. Receives confirmation token from user via `ConfirmUserCommand`
|
|
||||||
2. Calls `ITokenService.ValidateConfirmationTokenAsync()`
|
|
||||||
3. Extracts user ID from validated token and looks it up via
|
|
||||||
`UserManager<ApplicationUser>`
|
|
||||||
4. Marks the account confirmed via
|
|
||||||
`IUserEmailStore<ApplicationUser>.SetEmailConfirmedAsync()`, then persists it
|
|
||||||
via `UserManager<ApplicationUser>.UpdateAsync()`
|
|
||||||
5. Returns confirmation result
|
|
||||||
|
|
||||||
`UserManager<ApplicationUser>` and `IUserEmailStore<ApplicationUser>` are
|
|
||||||
ASP.NET Core Identity's abstractions, not EF Core's default Identity store: the
|
|
||||||
app backs them with its own `DapperUserStore` (see
|
|
||||||
[Database](database.md#database-error-handling)), which persists against the
|
|
||||||
existing `UserAccount`/`UserCredential`/`UserVerification` tables via Dapper,
|
|
||||||
consistent with the rest of the app's repository pattern.
|
|
||||||
|
|
||||||
#### [ResendConfirmationEmailHandler](../../web/backend/Features/Features.Users/Commands/Authentication/ResendConfirmationEmail/ResendConfirmationEmailHandler.cs)
|
|
||||||
|
|
||||||
**Flow:**
|
|
||||||
|
|
||||||
1. Receives a user ID via `ResendConfirmationEmailCommand`
|
|
||||||
2. Looks up the user and checks they aren't already verified
|
|
||||||
3. Generates a fresh confirmation token via `ITokenService`
|
|
||||||
4. Sends `SendResendConfirmationEmailCommand` over MediatR, handled by the
|
|
||||||
`Features.Emails` slice, with no direct project reference between the two
|
|
||||||
slices
|
|
||||||
|
|
||||||
#### [RefreshTokenHandler](../../web/backend/Features/Features.Users/Commands/Authentication/RefreshToken/RefreshTokenHandler.cs)
|
|
||||||
|
|
||||||
**Flow:**
|
|
||||||
|
|
||||||
1. Receives a refresh token via `RefreshTokenCommand`
|
|
||||||
2. Delegates to `ITokenService.RefreshTokenAsync()`, which validates the token,
|
|
||||||
retrieves the user account, and issues a new access/refresh token pair
|
|
||||||
3. Maps the result onto `LoginPayload`
|
|
||||||
|
|
||||||
#### [AuthController](../../web/backend/Features/Features.Users/Controllers/AuthController.cs)
|
|
||||||
|
|
||||||
**Endpoints:**
|
|
||||||
|
|
||||||
- `POST /api/auth/register` - Register new user
|
|
||||||
- `POST /api/auth/login` - Authenticate user
|
|
||||||
- `POST /api/auth/confirm?token=...` - Confirm email
|
|
||||||
- `POST /api/auth/confirm/resend?userId=...` - Resend the confirmation email
|
|
||||||
- `POST /api/auth/refresh` - Refresh access token
|
|
||||||
|
|
||||||
Account management (require a valid access token; user ID comes from the token,
|
|
||||||
never the request body):
|
|
||||||
|
|
||||||
- `PATCH /api/auth/username` - Change the authenticated user's username
|
|
||||||
- `PATCH /api/auth/email` - Change the authenticated user's email address;
|
|
||||||
resets `EmailConfirmed` to false, so the user must re-confirm via
|
|
||||||
`POST /api/auth/confirm/resend`
|
|
||||||
- `PATCH /api/auth/password` - Change the authenticated user's password;
|
|
||||||
requires the current password
|
|
||||||
- `PATCH /api/auth/profile` - Update the authenticated user's profile (first
|
|
||||||
name, last name, date of birth)
|
|
||||||
- `DELETE /api/auth/account` - Permanently delete the authenticated user's
|
|
||||||
account; blocked while the account still has associated posts, comments,
|
|
||||||
photos, or follows
|
|
||||||
|
|
||||||
## Validation security
|
|
||||||
|
|
||||||
### Token secrets
|
|
||||||
|
|
||||||
Three independent secrets enable:
|
|
||||||
|
|
||||||
- **Key rotation** - Rotate each secret type independently
|
|
||||||
- **Isolation** - Compromise of one secret doesn't affect others
|
|
||||||
- **Different expiration** - Different token types can expire at different rates
|
|
||||||
|
|
||||||
**Environment Variables:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ACCESS_TOKEN_SECRET=... # Signs 1-hour access tokens
|
|
||||||
REFRESH_TOKEN_SECRET=... # Signs 21-day refresh tokens
|
|
||||||
CONFIRMATION_TOKEN_SECRET=... # Signs 30-minute confirmation tokens
|
|
||||||
```
|
|
||||||
|
|
||||||
### Validation checks
|
|
||||||
|
|
||||||
Each token is validated for:
|
|
||||||
|
|
||||||
1. **Signature Verification** - Token must be signed with correct secret
|
|
||||||
2. **Expiration** - Token must not be expired (checked against current time)
|
|
||||||
3. **Claims Presence** - Required claims (Sub, UniqueName) must be present
|
|
||||||
4. **Claims Format** - UserId claim must be a valid GUID
|
|
||||||
|
|
||||||
### Error handling
|
|
||||||
|
|
||||||
Validation failures return HTTP 401 Unauthorized. Regardless of the specific
|
|
||||||
cause (invalid signature, expired token, missing header, malformed claims),
|
|
||||||
`JwtAuthenticationHandler.HandleChallengeAsync` always returns the same generic
|
|
||||||
message: `"Unauthorized: Invalid or missing authentication token"` — it doesn't
|
|
||||||
distinguish failure reasons in the response.
|
|
||||||
|
|
||||||
## Token lifecycle
|
|
||||||
|
|
||||||
### Access token lifecycle
|
|
||||||
|
|
||||||
| Step | Description |
|
|
||||||
| ---------- | ------------------------------------------------ |
|
|
||||||
| Generation | During login (1-hour validity) |
|
|
||||||
| Usage | Included in Authorization header on API requests |
|
|
||||||
| Validation | Validated on protected endpoints |
|
|
||||||
| Expiration | Token becomes invalid after 1 hour |
|
|
||||||
| Refresh | Use refresh token to obtain new access token |
|
|
||||||
|
|
||||||
### Refresh token lifecycle
|
|
||||||
|
|
||||||
| Step | Description |
|
|
||||||
| ---------- | ------------------------------------------------ |
|
|
||||||
| Generation | During login (21-day validity) |
|
|
||||||
| Storage | Client-side (secure storage) |
|
|
||||||
| Usage | Posted to `/api/auth/refresh` endpoint |
|
|
||||||
| Validation | Validated by `ITokenService.RefreshTokenAsync()` |
|
|
||||||
| Rotation | New refresh token issued on successful refresh |
|
|
||||||
| Expiration | Token becomes invalid after 21 days |
|
|
||||||
|
|
||||||
### Confirmation token lifecycle
|
|
||||||
|
|
||||||
| Step | Description |
|
|
||||||
| ---------- | ----------------------------------------------------- |
|
|
||||||
| Generation | During user registration (30-minute validity) |
|
|
||||||
| Delivery | Emailed to user in confirmation link |
|
|
||||||
| Usage | User clicks link, token posted to `/api/auth/confirm` |
|
|
||||||
| Validation | Validated by `ConfirmUserHandler` |
|
|
||||||
| Completion | User account marked as confirmed |
|
|
||||||
| Expiration | Token becomes invalid after 30 minutes |
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
All of the following live in `Features.Users.Tests`.
|
|
||||||
|
|
||||||
### Unit tests
|
|
||||||
|
|
||||||
**Services/TokenServiceValidationTests.cs**
|
|
||||||
|
|
||||||
- Happy path: Valid token extraction
|
|
||||||
- Error cases: Invalid, expired, malformed tokens
|
|
||||||
- Missing/invalid claims scenarios
|
|
||||||
|
|
||||||
**Services/TokenServiceRefreshTests.cs**
|
|
||||||
|
|
||||||
- Successful refresh with valid token
|
|
||||||
- Invalid/expired refresh token rejection
|
|
||||||
- Non-existent user handling
|
|
||||||
|
|
||||||
**Commands/RefreshTokenHandlerTests.cs**
|
|
||||||
|
|
||||||
- Verifies the handler maps `ITokenService.RefreshTokenAsync()`'s result onto
|
|
||||||
`LoginPayload`
|
|
||||||
|
|
||||||
**Commands/ConfirmUserHandlerTests.cs**
|
|
||||||
|
|
||||||
- Successful confirmation with valid token
|
|
||||||
- Token validation failures
|
|
||||||
- User not found scenarios
|
|
||||||
|
|
||||||
**Commands/ResendConfirmationEmailHandlerTests.cs**
|
|
||||||
|
|
||||||
- Sends a fresh confirmation email when the user exists and is unverified
|
|
||||||
- No-ops when the user doesn't exist or is already verified
|
|
||||||
|
|
||||||
### BDD tests (Reqnroll)
|
|
||||||
|
|
||||||
**TokenRefresh.feature**
|
|
||||||
|
|
||||||
- Successful token refresh
|
|
||||||
- Invalid/expired token rejection
|
|
||||||
- Missing token validation
|
|
||||||
|
|
||||||
**Confirmation.feature**
|
|
||||||
|
|
||||||
- Successful email confirmation
|
|
||||||
- Expired/tampered token rejection
|
|
||||||
- Missing token validation
|
|
||||||
|
|
||||||
**AccessTokenValidation.feature**
|
|
||||||
|
|
||||||
- Protected endpoint access token validation
|
|
||||||
- Invalid/expired access token rejection
|
|
||||||
- Token type mismatch (refresh used as access token)
|
|
||||||
|
|
||||||
## Future enhancements
|
|
||||||
|
|
||||||
### Stretch goals
|
|
||||||
|
|
||||||
1. **Middleware for Access Token Validation** - Automatically validate access
|
|
||||||
tokens on protected routes - Populate HttpContext.User from token claims -
|
|
||||||
Return 401 for invalid/missing tokens
|
|
||||||
|
|
||||||
2. **Token Denylisting** - Implement token revocation (for example, on logout) -
|
|
||||||
Store denylisted tokens in cache/database - Check the denylist during
|
|
||||||
validation
|
|
||||||
|
|
||||||
3. **Refresh Token Rotation Strategy** - Detect token reuse (replay attacks) -
|
|
||||||
Automatically invalidate entire token chain on reuse - Log suspicious
|
|
||||||
activity
|
|
||||||
|
|
||||||
4. **Structured Logging** - Log token validation attempts - Track failed
|
|
||||||
validation reasons - Alert on repeated validation failures (brute force
|
|
||||||
detection)
|
|
||||||
78
docs/website/database.md
Normal file
78
docs/website/database.md
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
# Database
|
||||||
|
|
||||||
|
SQL Server 2022 schema, migration approach, and the app's SQL error-handling
|
||||||
|
convention. For the vertical-slice code that talks to this schema, see
|
||||||
|
[Architecture](../architecture.md).
|
||||||
|
|
||||||
|
## Schema
|
||||||
|
|
||||||
|
The full schema lives in one script:
|
||||||
|
`web/backend/Database/Database.Migrations/scripts/01-schema/schema.sql`, run by
|
||||||
|
`Database.Migrations` (DbUp) on startup. Each table is queried directly by the
|
||||||
|
owning feature slice's repository (see
|
||||||
|
[Direct SQL via Dapper/ADO.NET](../architecture.md#direct-sql-via-dapperadonet)).
|
||||||
|
|
||||||
|
All tables use a `UNIQUEIDENTIFIER` primary key defaulting to `NEWID()`, and
|
||||||
|
most carry a `Timer ROWVERSION` column used for optimistic concurrency on
|
||||||
|
updates (`UPDATE ... WHERE ... AND Timer = @Timer`).
|
||||||
|
|
||||||
|
### Tables backed by the API today
|
||||||
|
|
||||||
|
| Table | Backed by | Notes |
|
||||||
|
| --------------------- | ------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| `UserAccount` | `Domain.Entities.UserAccount`, `Features.Auth`/`Features.UserManagement` | `Username` and `Email` are unique |
|
||||||
|
| `UserCredential` | `Domain.Entities.UserCredential`, `Features.Auth` | Argon2id password hash; `IsRevoked`/`RevokedAt` track credential rotation, not JWTs |
|
||||||
|
| `UserVerification` | `Domain.Entities.UserVerification`, `Features.Auth` | Written when a user confirms their email |
|
||||||
|
| `BreweryPost` | `Domain.Entities.BreweryPost`, `Features.Breweries` | A user-submitted brewery listing |
|
||||||
|
| `BreweryPostLocation` | `Domain.Entities.BreweryPostLocation`, `Features.Breweries` | 1:1 with `BreweryPost`; `Coordinates` is a `GEOGRAPHY` column, read manually via `DbDataReader.GetFieldValue<byte[]>` because Dapper can't deserialize SQL Server UDTs |
|
||||||
|
| `City` | `Domain.Entities.City`, `Features.Locations` | Referenced by `BreweryPostLocation.CityID`; looked up via `ILocationRepository`, currently consumed only by `Database.Seed` |
|
||||||
|
|
||||||
|
### Tables defined in the schema but not yet wired to a feature slice
|
||||||
|
|
||||||
|
These exist so the schema can support planned functionality, but have no
|
||||||
|
`Domain.Entities` type, repository, or seed data yet:
|
||||||
|
|
||||||
|
| Table | Purpose (from schema) |
|
||||||
|
| ------------------ | -------------------------------------------------------------------------------------------------------- |
|
||||||
|
| `Photo` | A photo uploaded by a user account |
|
||||||
|
| `UserAvatar` | Links a `UserAccount` to a `Photo` as its avatar |
|
||||||
|
| `UserFollow` | One user following another (`CannotFollowOwnAccount` check constraint) |
|
||||||
|
| `StateProvince` | Referenced by `City.StateProvinceID` |
|
||||||
|
| `Country` | Referenced by `StateProvince.CountryID` |
|
||||||
|
| `BeerStyle` | A beer style/category |
|
||||||
|
| `BeerPost` | A beer listing, linked to a `BreweryPost` and `BeerStyle`; `ABV`/`IBU` are constrained (`0-67`, `0-120`) |
|
||||||
|
| `BeerPostPhoto` | Links a `BeerPost` to a `Photo` |
|
||||||
|
| `BeerPostComment` | A comment + 1-5 `Rating` on a `BeerPost` |
|
||||||
|
| `BreweryPostPhoto` | Links a `BreweryPost` to a `Photo` |
|
||||||
|
|
||||||
|
## Migrations
|
||||||
|
|
||||||
|
**Tool**: DbUp, run by the `Database.Migrations` project/container.
|
||||||
|
|
||||||
|
Scripts live under `scripts/`, applied in order and tracked so a script never
|
||||||
|
re-runs once it has succeeded. Today that's a single `01-schema/schema.sql`.
|
||||||
|
When the schema needs to change, add a new numbered script rather than editing
|
||||||
|
`schema.sql` in place — DbUp's tracking is per-script, so editing an
|
||||||
|
already-applied script means it won't re-run against existing databases.
|
||||||
|
|
||||||
|
## Data seeding
|
||||||
|
|
||||||
|
`Database.Seed` populates a target database from data produced by the C++
|
||||||
|
pipeline (see [Pipeline README](../pipeline/README.md)): cities, user accounts,
|
||||||
|
and brewery posts with locations. It only seeds tables that have a feature slice
|
||||||
|
to write through (see the table above).
|
||||||
|
|
||||||
|
## Database error handling
|
||||||
|
|
||||||
|
- Optimistic-concurrency conflicts are detected by an `UPDATE` affecting zero
|
||||||
|
rows against the `Timer` column.
|
||||||
|
- On failure, the repository throws a `Domain.Exceptions` type directly
|
||||||
|
(`NotFoundException`, `ConflictException`, `ArgumentException`, ...).
|
||||||
|
`API.Core`'s `GlobalExceptionFilter` maps these to HTTP status codes (404,
|
||||||
|
409, 400, ...); an unmapped `SqlException` (e.g. a genuine connectivity or
|
||||||
|
constraint-violation failure) maps to 503.
|
||||||
|
|
||||||
|
- One repository-level exception: `Features.Auth`'s `AuthRepository` does
|
||||||
|
catch `SqlException` directly, to detect a duplicate-key race (SQL Server
|
||||||
|
error 2601 or 2627) when two requests try to verify the same account
|
||||||
|
concurrently — see `Auth Repository.IsDuplicateVerificationViolation`.
|
||||||
1
docs/website/diagrams-out/architecture.svg
Normal file
1
docs/website/diagrams-out/architecture.svg
Normal file
File diff suppressed because one or more lines are too long
1
docs/website/diagrams-out/authentication-flow.svg
Normal file
1
docs/website/diagrams-out/authentication-flow.svg
Normal file
File diff suppressed because one or more lines are too long
1
docs/website/diagrams-out/database-schema.svg
Normal file
1
docs/website/diagrams-out/database-schema.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 63 KiB |
1
docs/website/diagrams-out/deployment.svg
Normal file
1
docs/website/diagrams-out/deployment.svg
Normal file
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user