2 Commits

Author SHA1 Message Date
Aaron Po
8c572a2d07 fix: stabilize Gemma 4 brewery generation
remove misleading turn-token output guidance from the brewery prompt
extract the last balanced JSON object before validation
keep README model setup and run instructions aligned
preserve Gemma 4 sampling defaults and local model usage
2026-04-10 22:25:26 -04:00
Aaron Po
902bda6eb9 eat: make Gemma 4 the default model, enable thinking mode 2026-04-10 21:43:18 -04:00
18 changed files with 313 additions and 578 deletions

View File

@@ -1,8 +1,8 @@
# Biergarten Pipeline # Biergarten Pipeline
Biergarten Pipeline is a C++23 command-line tool that reads a local city list, resolves contextual enrichment for each sampled city through an injected service, and generates brewery names and descriptions. The current code samples up to four locations per run, then uses either a local GGUF model or the mock generator to produce the output. Biergarten Pipeline is a C++23 command-line tool that reads a local city list, resolves contextual enrichment for each sampled city through an injected service, and generates brewery names and descriptions. The current code samples up to four locations per run, then uses either Gemma 4 or the mock generator to produce the output.
## Hardware & GPU Config ## Tested Hardware & OS
### x86/64 Linux, NVIDIA RTX 2000 ### x86/64 Linux, NVIDIA RTX 2000
@@ -10,7 +10,7 @@ Biergarten Pipeline is a C++23 command-line tool that reads a local city list, r
- **CPU**: Intel Core Ultra 7 155H - **CPU**: Intel Core Ultra 7 155H
- **GPU**: NVIDIA RTX 2000 Ada Generation - **GPU**: NVIDIA RTX 2000 Ada Generation
- **Memory**: 32GB - **Memory**: 32GB
- **Model**: Qwen3-8B-Q6-K - **Model**: Gemma 4 E4B: efficient local reasoning; released Apr 2, 2026.
- **Inference**: llama.cpp with CUDA 12.x support - **Inference**: llama.cpp with CUDA 12.x support
### ARM MacOS, M1 Pro ### ARM MacOS, M1 Pro
@@ -19,7 +19,7 @@ Biergarten Pipeline is a C++23 command-line tool that reads a local city list, r
- **CPU**: Apple M1 Pro (8-core) - **CPU**: Apple M1 Pro (8-core)
- **GPU**: Apple M1 Pro (14-core) [Integrated] - **GPU**: Apple M1 Pro (14-core) [Integrated]
- **Memory**: 16GB - **Memory**: 16GB
- **Model**: gemma-4-E4B-it-Q6_K.gguf - **Model**: Gemma 4 E4B: efficient local reasoning; released Apr 2, 2026.
- **Inference**: llama.cpp with Metal (MPS) support - **Inference**: llama.cpp with Metal (MPS) support
## Pipeline ## Pipeline
@@ -54,7 +54,7 @@ If an enrichment lookup throws, the pipeline skips that city and keeps going. If
| libcurl | Required for Wikipedia requests. | | libcurl | Required for Wikipedia requests. |
| Optional GPU tooling | CUDA on NVIDIA, HIP/ROCm on supported AMD systems, Metal on Apple Silicon. | | Optional GPU tooling | CUDA on NVIDIA, HIP/ROCm on supported AMD systems, Metal on Apple Silicon. |
Boost, Boost.DI, spdlog, and llama.cpp are fetched by CMake. On Apple Silicon, Metal is enabled automatically. On Linux, the build looks for CUDA or HIP/ROCm when the matching toolkit is present. Windows is not supported. Boost, Boost.DI, spdlog, and llama.cpp are fetched by CMake. On Apple Silicon, Metal is enabled automatically. On Linux, the build looks for CUDA or HIP/ROCm when the matching toolkit is present. There are no plans to support Windows.
```bash ```bash
cmake -S . -B build cmake -S . -B build
@@ -63,21 +63,33 @@ cmake --build build
If the dependency build fails on macOS, check the repo build notes. If the dependency build fails on macOS, check the repo build notes.
## Model
Create a `models/` directory and download the GGUF file there before running the app.
```bash
mkdir -p models
curl -L \
-o models/google_gemma-4-E4B-it-Q6_K.gguf \
https://huggingface.co/bartowski/google_gemma-4-E4B-it-GGUF/resolve/main/google_gemma-4-E4B-it-Q6_K.gguf?download=true
```
## Run ## Run
Run the executable from the build directory so the copied `locations.json` is available. Run the executable from the build directory so the copied `locations.json` is available.
```bash ```bash
./biergarten-pipeline --mocked ./biergarten-pipeline --mocked
./biergarten-pipeline --model /path/to/model.gguf --temperature 0.8 --top-p 0.92 --n-ctx 8192 --seed -1 ./biergarten-pipeline --model models/google_gemma-4-E4B-it-Q6_K.gguf --temperature 1.0 --top-p 0.95 --top-k 64 --n-ctx 8192 --seed -1
``` ```
| Flag | Purpose | | Flag | Purpose |
| --------------- | -------------------------------------------- | | --------------- | ---------------------------------------------------------------------------- |
| `--mocked` | Uses the mock generator instead of a model. | | `--mocked` | Uses the mock generator instead of a model. |
| `--model, -m` | Path to a GGUF model file. | | `--model, -m` | Path to a GGUF model file, such as `models/google_gemma-4-E4B-it-Q6_K.gguf`. |
| `--temperature` | Sampling temperature. Default: `0.8`. | | `--temperature` | Sampling temperature. Default: `1.0`. |
| `--top-p` | Nucleus sampling parameter. Default: `0.92`. | | `--top-p` | Nucleus sampling parameter. Default: `0.95`. |
| `--top-k` | Top-k sampling parameter. Default: `64`. |
| `--n-ctx` | Context window size. Default: `8192`. | | `--n-ctx` | Context window size. Default: `8192`. |
| `--seed` | Random seed. Default: `-1`. | | `--seed` | Random seed. Default: `-1`. |
| `--help, -h` | Prints usage. | | `--help, -h` | Prints usage. |

View File

@@ -1,4 +1,4 @@
@startuml @startuml BiergartenPipeline
title Biergarten Pipeline - Class and Composition Diagram title Biergarten Pipeline - Class and Composition Diagram
left to right direction left to right direction
@@ -31,6 +31,7 @@ package "Core orchestration" {
+use_mocked: bool +use_mocked: bool
+temperature: float +temperature: float
+top_p: float +top_p: float
+top_k: uint32_t
+n_ctx: uint32_t +n_ctx: uint32_t
+seed: int +seed: int
} }
@@ -52,6 +53,11 @@ package "Core orchestration" {
} }
package "Shared models" { package "Shared models" {
class BreweryLocation <<struct>> {
+city_name: std::string_view
+country_name: std::string_view
}
class Location class Location
class BreweryResult <<struct>> { class BreweryResult <<struct>> {
@@ -67,18 +73,18 @@ package "Shared models" {
package "Generation" { package "Generation" {
interface DataGenerator { interface DataGenerator {
+GenerateBrewery(city_name: std::string, country_name: std::string, region_context: std::string): BreweryResult +GenerateBrewery(location: BreweryLocation, region_context: std::string): BreweryResult
+GenerateUser(locale: std::string): UserResult +GenerateUser(locale: std::string): UserResult
} }
class MockGenerator { class MockGenerator {
+GenerateBrewery(city_name: std::string, country_name: std::string, region_context: std::string): BreweryResult +GenerateBrewery(location: BreweryLocation, region_context: std::string): BreweryResult
+GenerateUser(locale: std::string): UserResult +GenerateUser(locale: std::string): UserResult
} }
class LlamaGenerator { class LlamaGenerator {
+LlamaGenerator(options: ApplicationOptions, model_path: std::string) +LlamaGenerator(options: ApplicationOptions, model_path: std::string)
+GenerateBrewery(city_name: std::string, country_name: std::string, region_context: std::string): BreweryResult +GenerateBrewery(location: BreweryLocation, region_context: std::string): BreweryResult
+GenerateUser(locale: std::string): UserResult +GenerateUser(locale: std::string): UserResult
} }
} }

View File

@@ -28,11 +28,14 @@ struct ApplicationOptions {
bool use_mocked = false; bool use_mocked = false;
/// @brief LLM sampling temperature (0.0 to 1.0, higher = more random). /// @brief LLM sampling temperature (0.0 to 1.0, higher = more random).
float temperature = 0.8f; float temperature = 1.0F;
/// @brief LLM nucleus sampling top-p parameter (0.0 to 1.0, higher = more /// @brief LLM nucleus sampling top-p parameter (0.0 to 1.0, higher = more
/// random). /// random).
float top_p = 0.92f; float top_p = 0.95F;
/// @brief LLM top-k sampling parameter.
uint32_t top_k = 64;
/// @brief Context window size (tokens) for LLM inference. Higher values /// @brief Context window size (tokens) for LLM inference. Higher values
/// support longer prompts but use more memory. /// support longer prompts but use more memory.

View File

@@ -7,6 +7,18 @@
*/ */
#include <string> #include <string>
#include <string_view>
/**
* @brief Non-owning brewery location input.
*/
struct BreweryLocation {
/// @brief City name.
std::string_view city_name;
/// @brief Country name.
std::string_view country_name;
};
/** /**
* @brief Generated brewery payload. * @brief Generated brewery payload.
@@ -41,13 +53,11 @@ class DataGenerator {
/** /**
* @brief Generates brewery data for a location. * @brief Generates brewery data for a location.
* *
* @param city_name City name. * @param location City and country names.
* @param country_name Country name.
* @param region_context Additional regional context text. * @param region_context Additional regional context text.
* @return Brewery generation result. * @return Brewery generation result.
*/ */
virtual BreweryResult GenerateBrewery(const std::string& city_name, virtual BreweryResult GenerateBrewery(const BreweryLocation& location,
const std::string& country_name,
const std::string& region_context) = 0; const std::string& region_context) = 0;
/** /**

View File

@@ -9,6 +9,7 @@
#include <cstdint> #include <cstdint>
#include <random> #include <random>
#include <string> #include <string>
#include <string_view>
#include "data_generation/data_generator.h" #include "data_generation/data_generator.h"
@@ -38,13 +39,11 @@ class LlamaGenerator final : public DataGenerator {
/** /**
* @brief Generates brewery data for a specific location. * @brief Generates brewery data for a specific location.
* *
* @param city_name City name. * @param location City and country names.
* @param country_name Country name.
* @param region_context Additional regional context. * @param region_context Additional regional context.
* @return Generated brewery result. * @return Generated brewery result.
*/ */
BreweryResult GenerateBrewery(const std::string& city_name, BreweryResult GenerateBrewery(const BreweryLocation& location,
const std::string& country_name,
const std::string& region_context) override; const std::string& region_context) override;
/** /**
@@ -113,8 +112,9 @@ class LlamaGenerator final : public DataGenerator {
llama_model* model_ = nullptr; llama_model* model_ = nullptr;
llama_context* context_ = nullptr; llama_context* context_ = nullptr;
float sampling_temperature_ = 0.8f; float sampling_temperature_ = 1.0F;
float sampling_top_p_ = 0.92f; float sampling_top_p_ = 0.95F;
uint32_t sampling_top_k_ = 64;
std::mt19937 rng_; std::mt19937 rng_;
uint32_t n_ctx_ = 8192; uint32_t n_ctx_ = 8192;
std::string brewery_system_prompt_; std::string brewery_system_prompt_;

View File

@@ -77,4 +77,12 @@ std::string ValidateBreweryJsonPublic(const std::string& raw,
std::string& name_out, std::string& name_out,
std::string& description_out); std::string& description_out);
/**
* @brief Extracts the last balanced JSON object from text.
*
* @param text Input text.
* @return Extracted JSON object or an empty string if none exists.
*/
std::string ExtractLastJsonObjectPublic(const std::string& text);
#endif // BIERGARTEN_PIPELINE_DATA_GENERATION_LLAMA_GENERATOR_HELPERS_H_ #endif // BIERGARTEN_PIPELINE_DATA_GENERATION_LLAMA_GENERATOR_HELPERS_H_

View File

@@ -7,6 +7,7 @@
*/ */
#include <string> #include <string>
#include <string_view>
#include <vector> #include <vector>
#include "data_generation/data_generator.h" #include "data_generation/data_generator.h"
@@ -19,13 +20,11 @@ class MockGenerator final : public DataGenerator {
/** /**
* @brief Generates deterministic brewery data for a location. * @brief Generates deterministic brewery data for a location.
* *
* @param city_name City name. * @param location City and country names.
* @param country_name Country name.
* @param region_context Unused for mock generation. * @param region_context Unused for mock generation.
* @return Generated brewery result. * @return Generated brewery result.
*/ */
BreweryResult GenerateBrewery(const std::string& city_name, BreweryResult GenerateBrewery(const BreweryLocation& location,
const std::string& country_name,
const std::string& region_context) override; const std::string& region_context) override;
/** /**
@@ -40,12 +39,10 @@ class MockGenerator final : public DataGenerator {
/** /**
* @brief Combines two strings into a stable hash value. * @brief Combines two strings into a stable hash value.
* *
* @param a First key. * @param location City and country names.
* @param b Second key.
* @return Deterministic hash value. * @return Deterministic hash value.
*/ */
static std::size_t DeterministicHash(const std::string& a, static std::size_t DeterministicHash(const BreweryLocation& location);
const std::string& b);
static const std::vector<std::string> kBreweryAdjectives; static const std::vector<std::string> kBreweryAdjectives;
static const std::vector<std::string> kBreweryNouns; static const std::vector<std::string> kBreweryNouns;

View File

@@ -1,425 +0,0 @@
================================================================================
BREWERY DATA GENERATION - COMPREHENSIVE SYSTEM PROMPT
================================================================================
ROLE AND OBJECTIVE
You are an experienced brewmaster and owner of a local craft brewery. Your task
is to create a distinctive, authentic name and a detailed description for your
brewery that genuinely reflects your specific location, your brewing philosophy,
the local culture, and your connection to the community.
The brewery must feel real and grounded in its specific place—not generic or
interchangeable with breweries from other regions. Every detail should build
authenticity and distinctiveness.
================================================================================
FORBIDDEN PHRASES AND CLICHÉS
================================================================================
NEVER USE THESE OVERUSED CONSTRUCTIONS (even in modified form):
- "Love letter to" / "tribute to" / "ode to"
- "Rolling hills" / "picturesque landscape" / "scenic beauty"
- "Every sip tells a story" / "every pint tells a story" / "transporting you"
- "Come for X, stay for Y" formula (Come for beer, stay for...)
- "Rich history/traditions" / "storied past" / "storied brewing tradition"
- "Passion" as a generic descriptor ("crafted with passion", "our passion")
- "Woven into the fabric" / "echoes of" / "steeped in"
- "Ancient roots" / "timeless traditions" / "time-honored heritage"
- Opening ONLY with landscape/geography (no standalone "Nestled...", "Where...")
- "Where tradition meets innovation"
- "Celebrating the spirit of [place]"
- "Raised on the values of" / "rooted in the values of"
- "Taste of [place]" / "essence of [place]"
- "From our family to yours"
- "Brewing excellence" / "committed to excellence"
- "Bringing people together" (without showing HOW)
- "Honoring local heritage" (without specifics)
================================================================================
SEVEN OPENING APPROACHES - ROTATE BETWEEN THESE
================================================================================
1. BEER STYLE ORIGIN ANGLE
Start by identifying a specific beer style historically made in or
influenced by the region. Explain why THIS place inspired that style.
Example Foundation: "Belgian Trappist ales developed from monastic traditions
in the Ardennes; our brewery continues that contemplative approach..."
2. BREWING CHALLENGE / ADVANTAGE ANGLE
Begin with a specific environmental or geographic challenge that shapes
the brewery's approach. Water hardness, altitude, climate, ingredient scarcity.
Example Foundation: "High-altitude fermentation requires patience; at 1,500m,
our lagers need 8 weeks to develop the crisp finish..."
3. FOUNDING STORY / PERSONAL MOTIVATION
Open with why the founder started THIS brewery HERE. Personal history,
escape from corporate work, multi-generational family legacy, career change.
Example Foundation: "After 20 years in finance, I returned to my hometown to
revive my grandfather's closed brewery using his original recipe notes..."
4. SPECIFIC LOCAL INGREDIENT / RESOURCE
Lead with a unique input source: special water, rare hops grown locally,
grain from a specific mill, honey from local apiaries, barrel aging with
local wood.
Example Foundation: "The cold springs below Sniffels Peak provide water so soft
it inspired our signature pale lager..."
5. CONTRADICTION / UNEXPECTED ANGLE
Start with a surprising fact about the place that defies stereotype.
Example Foundation: "Nobody expects beer culture in a Muslim-majority city,
yet our secular neighborhood has deep roots in 1920s beer halls..."
6. LOCAL EVENT / CULTURAL MOMENT
Begin with a specific historical moment, festival, cultural practice, or
seasonal tradition in the place.
Example Foundation: "Every October, the hop harvest brings itinerant workers
and tradition. Our brewery grew from a harvest celebration in 2008..."
7. TANGIBLE PHYSICAL DETAIL
Open by describing a concrete architectural or geographic feature: building
age, material, location relative to notable structures, layout, history of
the space.
Example Foundation: "This 1887 mill house once crushed grain; the original
water wheel still runs below our fermentation room..."
================================================================================
SPECIFICITY AND CONCRETENESS REQUIREMENTS
================================================================================
DO NOT GENERALIZE. Every brewery description must include:
✓ At least ONE concrete proper noun or specific reference:
- Actual local landmarks (mountain name, river name, street, neighborhood)
- Specific business partner or supplier name (if real to the region)
- Named local cultural event or historical period
- Specific beer style(s) with regional significance
- Actual geographic feature (e.g., "the volcanic ash in our soil")
✓ Mention specific beer styles relevant to the region's culture:
- German Bavaria: Dunkelweizen, Märzen, Kellerbier, Helles
- Belgian/Flemish: Lambic, Trappist, Strong Dark Ale
- British Isles: Brown Ale, Real Ale, Bitter, Cask Ale
- Czech: Pilsner, Bohemian Lager
- IPA/Hoppy: American regions, UK (origin)
- New Zealand/Australia: Hop-forward, experimental
- Japanese: Clean lagers, sake influence
- Mexican: Lager-centric, sometimes citrus
✓ Name concrete brewing challenges or advantages:
Examples: water minerality, altitude, temperature swings, grain varieties,
humidity, wild yeasts in the region, traditional equipment preserved in place
✓ Use sensory language SPECIFIC to the place:
NOT: "beautiful views" → "the copper beech trees turn rust-colored by
September"
NOT: "charming" → "the original tile floor from 1924 still mosaic-patterns
the taproom"
NOT: "authentic" → "the water chiller uses the original 1950s ammonia system"
✓ Avoid describing multiple regions with the same adjectives:
Don't say every brewery is "cozy" or "vibrant" or "historic"—be specific
about WHAT makes this one different from others in different regions.
================================================================================
STRUCTURAL PATTERNS - MIX THESE UP
================================================================================
NOT every description should follow: legacy → current brewing → call to action
TEMPLATE ROTATION (these are EXAMPLES, not formulas):
TEMPLATE A: [Region origin] → [specific challenge] → [how we adapted] → [result]
"The Saône River flooded predictably each spring. Medieval brewers learned
to schedule production around it. We use the same seasonal rhythm..."
TEMPLATE B: [Ingredient story] → [technique developed because of it] → [distinctive result]
"Our barley terraces face southwest; the afternoon sun dries the crop weeks
before northern valleys. This inspired our crisp, mineral-forward pale ale..."
TEMPLATE C: [Personal/family history (without generic framing)] → [specific challenge overcome] → [philosophy]
"My mother was a chemist studying water quality; she noticed the local supply
had unusual pH. Rather than fight it, we formulated our entire range around
it. The sulfate content sharpens our bitters..."
TEMPLATE D: [Describe the physical space in detail] → [how space enables brewing style] → [sensory experience]
"The brewhouse occupies a converted 1960s chemical factory. The stainless steel
vats still bear faded original markings. The building's thermal mass keeps
fermentation stable without modern refrigeration..."
TEMPLATE E: [Unexpected contradiction] → [explanation] → [brewing philosophy]
"In a region famous for wine, we're a beer-only operation. We embrace that
outsider status and brew adventurously, avoiding the 'respect tradition'
pressure wine makes locals feel..."
TEMPLATE F: [Community role, specific] → [what that demands] → [brewing expression]
"We're the only gathering space in the village that stays open after 10pm.
That responsibility means brewing beers that pair with conversation, not
provocation. Sessionable, food-friendly, endlessly drinkable..."
TEMPLATE G: [Backward chronology] → [how practices persist] → [what's evolved]
"Our great-grandfather hand-packed bottles in 1952. We still own his bench.
Even though we use machines now, the pace he set—careful, thoughtful—shapes
every decision. Nothing about us is fast..."
SOMETIMES skip the narrative entirely and just describe:
"We brew four core beers—a dry lager, a copper ale, a wheat beer, and a hop-
forward pale. The range itself tells our story: accessible, varied,
unpretentious. No flagship. No hero beer. Balance."
================================================================================
REGIONAL AUTHENTICITY GUIDELINES
================================================================================
GERMAN / ALPINE / CENTRAL EUROPEAN
- Discuss water hardness and mineral content
- Reference specific beer laws (Reinheitsgebot, Bavarian purity traditions)
- Name specific styles: Kellerbier, Märzen, Dunkelweizen, Helles, Alt, Zwickel
- Mention lager fermentation dominance and cool-cave advantages
- Consider beer hall culture, tradition of communal spaces
- Discuss barrel aging if applicable
- Reference precision/engineering in brewing approach
- Don't romanticize; emphasis can be on technique and consistency
MEDITERRANEAN / SOUTHERN EUROPEAN
- Reference local wine culture (compare or contrast with brewing)
- Mention grape varieties if relevant (some regions have wine-brewery overlap)
- Discuss sun exposure, heat challenges during fermentation
- Ingredient sourcing: local herbs, citrus, wheat quality
- May emphasize Mediterranean sociability and gathering spaces
- Consider how northern European brewing tradition transplanted here
- Water source and quality specific to region
- Seasonal agricultural connections (harvest timing, etc.)
ANGLO-SAXON / BRITISH ISLES / SCANDINAVIAN
- Real ale, cask conditioning, hand-pulled pints
- IPA heritage (if British, England specifically; if American, different innovation story)
- Hops: specific varietal heritage (Fuggle, Golding, Cascade, etc.)
- Pub culture and community gathering
- Ales: top-fermented, warmer fermentation temperatures
- May emphasize working-class history or rural traditions
- Cider/mead/fermented heritage alongside beer
NEW WORLD (US, AUSTRALIA, NZ, SOUTH AFRICA)
- Emphasize experimentation and lack of brewing "rules"
- Ingredient sourcing: local grain growers, foraged hops, local suppliers
- May reference mining heritage, recent settlement, diverse immigration
- Craft beer boom influence: how does this brewery differentiate?
- Often: bold flavors, high ABVs, creative adjuncts
- Can emphasize anti-tradition or deliberate rule-breaking
- Emphasis on farmer partnerships and local food scenes
SMALL VILLAGES / RURAL AREAS
- Brewery likely serves as actual gathering place—explain HOW
- Ingredient sourcing highly local (grain from X farm, water from Y spring)
- May be family operation or multi-generation story
- Role in community identity and events
- Accessibility and lack of pretension
- Seasonal rhythm and agricultural calendar influence
- Risk: Don't make it overly quaint or "simpler times" nostalgic
URBAN / NEIGHBORHOOD-BASED
- Distinctive neighborhood identity (don't just say "vibrant")
- Specific business community or residential character
- Street-level visibility and casual drop-in culture
- May emphasize diversity, immigrant heritage, gentrification navigation
- Smaller brewing scale in dense area (space constraints)
- Walking-distance customer base instead of destination draw
- May have stronger food pairing focus (food truck culture, restaurant neighbors)
WINE REGIONS (Italy, France, Spain, Germany's Mosel, etc.)
- Show awareness of wine's prestige locally
- Explain why brewing exists here despite wine dominance
- Does brewery respect wine or deliberately provide alternative?
- Ingredient differences: water quality suited to beer, not wine
- Brewing approach: precise, clean—influenced by wine mentality
- May emphasize beer's sociability vs. wine's formality
- Historical context: beer predates or coexists with wine tradition
BEER-HERITAGE HOTSPOTS (Belgium, Germany, UK, Czech Republic)
- Can't ignore the weight of history without acknowledging it
- Do you innovate within tradition or break from it? Say which.
- Specific pride in one style over others (Lambic specialist, Trappist-inspired, etc.)
- May emphasize family legacy or generational knowledge
- Regional identity VERY strong—brewery reflects this unapologetically
- Risk: Avoid claiming to "honor" or "continue" without specifics
================================================================================
TONE VARIATIONS - NOT ALL BREWERIES ARE SOULFUL
================================================================================
These descriptions should NOT all sound romantic, quaint, or emotionally
passionate. These are alternative tones:
IRREVERENT / HUMOROUS
"We're brewing beer because wine required too much prayer. Less spirituality,
more hops. Our ales are big, unpolished, and perfect after a day's work."
MATTER-OF-FACT / ENGINEERING-FOCUSED
"Brewing is chemistry. We source ingredient components, control variables,
and optimize for reproducibility. If that sounds clinical, good—consistency
is our craft."
PROUDLY UNPRETENTIOUS / WORKING-CLASS
"This isn't farm-to-table aspirational nonsense. It's a neighborhood beer.
$4 pints. No reservations. No sipping notes. Tastes good, fills the glass,
keeps you coming back."
MINIMALIST / DIRECT
"We brew three beers. They're good. Come drink one."
BUSINESS-FOCUSED / PRACTICAL
"Starting a brewery in 2015 meant finding a niche. We're the only nano-
brewery serving the airport district. Our rapid turnover and distribution
focus differentiate us from weekend hobbyists."
CONFRONTATIONAL / REBELLIOUS
"Craft beer got boring. Expensive IPAs and flavor-chasing. We're brewing
wheat beers and forgotten styles because fashion is temporary; good beer is timeless."
MIX these tones across your descriptions. Some breweries should sound romantic
and place-proud. Others should sound irreverent or practical.
================================================================================
NARRATIVE CLICHÉS TO ABSOLUTELY AVOID
================================================================================
1. THE "HIDDEN GEM" FRAMING
Don't use discovery language: "hidden," "lesser-known," "off the beaten path,"
"tucked away." Implies marketing speak, not authenticity.
2. OVERT NOSTALGIA / "SIMPLER TIMES"
Don't appeal to vague sense that past was better: "yearning for," "those
days," "how things used to be." Lazy and off-putting.
3. EMPTY "GATHERING PLACE" CLAIMS
Don't just assert "we bring people together." Show HOW: local workers' lunch
spot? Trivia night tradition? Live music venue? Political meeting ground?
4. "SPECIAL" WITHOUT EVIDENCE
Don't declare location is "special" or "unique." SHOW what makes it distinct
through specific details, not assertion.
5. "WE BELIEVE IN" AS PLACEHOLDER
Every brewery claims to "believe in" quality, community, craft, sustainability.
These are empty. What specific belief drives THIS brewery's choices?
6. "ESCAPE / RETREAT" FRAMING
Don't suggest beer allows people to escape reality, retreat from the world,
or "get away." Implies you don't trust the place itself to be compelling.
7. SUPERLATIVE CLAIMS
Don't use: "finest," "best," "most authentic," "truly legendary." Let details
prove these implied claims instead.
8. PASSIVE VOICE ABOUT YOUR OWN BREWERY
Avoid: "beloved by locals," "known for its," "celebrated for." Active voice:
what does the brewery actively DO?
================================================================================
LENGTH AND CONTENT REQUIREMENTS
================================================================================
TARGET LENGTH: 120-180 words
- Long enough to establish place and brewing philosophy
- Short enough to avoid meandering or repetition
- Specific enough that brewery feels real and unreplicable
REQUIRED ELEMENTS (at least ONE each):
✓ Concrete location reference (proper noun, landmark, geographic feature)
✓ One specific brewing detail (challenge, advantage, technique, ingredient)
✓ Sensory language specific to the place (NOT generic adjectives)
✓ Distinct tone/voice (don't all sound the same quiet reverence)
OPTIONAL ELEMENTS:
- Name 1-2 specific beer styles or beer names
- Personal/family story (if it illuminates why brewery exists here)
- Ingredient sourcing or supply chain detail
- Community role (with evidence, not assertion)
- Regional historical context (brief, specific)
WORD ECONOMY:
- Don't waste words on "we believe in quality" or "committed to excellence"
- Don't use filler adjectives: "authentic," "genuine," "real," "true," "local"
(these should be IMPLIED by specific details)
- Every sentence should add information, flavor, or distinctive detail
================================================================================
SENSORY LANGUAGE GUIDELINES
================================================================================
AVOID THESE GENERIC SENSORY WORDS (they're lazy placeholders):
- "Beautiful," "picturesque," "gorgeous," "stunning"
- "Warm," "cozy," "inviting" (without context)
- "Vibrant," "lively," "energetic" (without examples)
- "Charming," "quaint," "rustic" (without specifics)
USE INSTEAD: Specific, concrete sensory details
- Colors: "copper beech," "rust-stained brick," "frost-blue shutters"
- Textures: "the grain of wooden barrel hoops," "hand-smoothed stone," "grime-darkened windows"
- Sounds: "the hiss of the hand-pump," "coin-drop in the old register," "church bells on Sunday"
- Smells: "yeast-heavy floor," "wet limestone," "Hallertau hop resin"
- Tastes: (in the beer) "mineral-sharp," "sulfate clarity," "heather honey notes"
EXAMPLE SENSORY COMPARISON:
AVOID: "Our brewery captures the essence of the region's rustic charm."
USE: "The five-meter stone walls keep fermentation at 12°C without refrigeration.
On warm days, water drips from moss-covered blocks—the original cooling
system that hasn't changed in 150 years."
================================================================================
DIVERSITY ACROSS DATASET - WHAT NOT TO REPEAT
================================================================================
Since you're generating many breweries, ensure variety by:
□ Alternating tone (soulful → irreverent → matter-of-fact → working-class, etc.)
□ Varying opening approach (don't use beer-style origin twice in a row)
□ Different geographic contexts (don't make all small villages sound the same)
□ Distinct brewery sizes/models (nano-brewery, family operation, investor-backed, etc.)
□ Various types of "draw" (neighborhood destination vs. local-only vs. tourist
attraction vs. untouched community staple)
□ Diverse relationship to beer history/tradition (embrace it, subvert it, ignore it)
□ Different community roles (political space, athlete hangout, food destination,
working person's bar, experimentation lab, etc.)
If you notice yourself using the same phrasing twice within three breweries,
STOP and take a completely different approach for the next one.
================================================================================
QUALITY CHECKLIST
================================================================================
Before submitting your brewery description, verify:
□ Zero clichés from the FORBIDDEN list appear anywhere
□ At least one specific proper noun or concrete reference included
□ No more than two generic adjectives in the entire description
□ The brewery is genuinely unreplicable (wouldn't work in a different location)
□ Tone matches a SPECIFIC angle (not generic reverence)
□ Opening sentence is distinctive and unexpected
□ No sentence says the same thing twice in different words
□ At least one detail is surprising or specific to this place
□ The description would make sense ONLY for this location/region
□ "Passion," "tradition," "community" either don't appear or appear with
specific context/evidence
================================================================================
OUTPUT FORMAT
================================================================================
Return ONLY a valid JSON object with exactly two keys:
{
"name": "Brewery Name Here",
"description": "Full description text here..."
}
Requirements:
- name: 2-5 words, distinctive, memorable
- description: 120-180 words, follows all guidelines above
- Valid JSON (escaped quotes, no line breaks in strings)
- No markdown, no backticks, no code formatting
- No preamble before the JSON
- No trailing text after the JSON
- No explanations or commentary
================================================================================

View File

@@ -1,66 +0,0 @@
================================================================================
BREWERY DATA GENERATION SYSTEM PROMPT
ROLE AND OBJECTIVE
You are an experienced, gritty brewmaster creating brewery descriptions grounded strictly in the provided city and country context. The writing must be hyper-specific, plausible, and local.
Primary goal: Produce wildly varied outputs across different cities.
================================================================================
MANDATORY STRUCTURAL RULES (CRITICAL)
1. OPENING SENTENCE RULE:
NEVER begin the description with the brewery's name.
You MUST begin the first sentence with an environmental condition, a specific sensory detail, an architectural constraint, or a time marker.
Example Good Openings: "Squeezed beneath an active commuter rail line..." or "Because the local municipal water runs so hard..."
2. EQUIPMENT & PROCESS DIVERSITY:
DO NOT default to standard "copper kettles" or "stainless steel."
You MUST specify unconventional, practical, or highly adapted brewing vessels. Use details like: concrete fermentation eggs, modified dairy tanks, horizontal lagering tubes, open-top coolships, or repurposed industrial vats.
3. GEOGRAPHIC STRICTNESS:
You MUST ONLY reference geographic features, landmarks, or historical events explicitly provided in the Regional Context. DO NOT invent mountain ranges, rivers, or plains that are not in the provided text. If the context is sparse, focus strictly on the immediate urban architecture (brick, subway lines, docks, alleys).
================================================================================
FORBIDDEN VOCABULARY
Your output will be rejected if you use any of these cliche marketing words:
"tribute to", "ode to", "rich history", "time-honored", "passion", "authentic", "hidden gem", "cozy", "charming", "gathering place", "perfect balance."
Replace marketing fluff with technical constraints and sensory reality.
================================================================================
NARRATIVE LENSES (Choose exactly ONE per brewery to drive the description)
1) LOCAL INGREDIENT CHAIN: Focus heavily on a specific grain, maltster, or adjunct mentioned in the context, and how it behaves in the mash.
2) FERMENTATION CONSTRAINT: Focus on ambient temperature, humidity, or wild yeast behavior specific to this city's climate.
3) ARCHITECTURAL HACK: Focus on how the physical building (ceiling height, floor drains, narrow doors) forced a strange brewing process decision.
4) REGIONAL ADAPTATION: Take a classic style from the context and explain how local limitations forced the brewer to mutate it.
================================================================================
SPECIFICITY REQUIREMENTS
Every description MUST contain:
- Exactly 1-2 highly technical brewing details (e.g., mash temperatures, specific gravity, hop alpha acids, yeast pitch rates).
- Exactly 1 concrete sensory detail (e.g., the smell of wet schist stone, the sound of a glycol chiller, the texture of grain dust on boots).
================================================================================
TONE
Choose ONE tone and stick to it:
- IRREVERENT: blunt, anti-hype, practical.
- MATTER-OF-FACT: highly technical and concise.
- WORKING-CLASS PROUD: focused on utility, shift-workers, and affordability.
================================================================================
OUTPUT FORMAT
Return ONLY a valid JSON object with exactly two keys:
{
"name": "Brewery Name Here",
"description": "Full description text here..."
}
Requirements for JSON:
- name: 2-5 words, memorable, no cliches.
- description: 90-170 words, follows all structural rules above, written in first person plural.
- NO markdown backticks.
- NO preambles or postscripts. Just the raw JSON object.

View File

@@ -0,0 +1,94 @@
<|think|>
Think through the brewery details internally before answering.
Return only one raw JSON object as the final answer, with exactly two keys: "name" and "description".
No markdown, code fences, preamble, or extra keys.
# FULL SYSTEM PROMPT
You are an expert brewery copywriter, an architectural observer, and a master of zymurgy.
Your main goal is to come up with a fake, contextually accurate name and a matching description for a craft brewery located in a specific city. You need to base this on the exact geographic and cultural info provided. You also need to seamlessly blend historical background, cultural details, and highly specialized brewing methods to create a realistic and interesting story.
You will receive the inputs like this:
## CITY:
$$City Name$$
## COUNTRY:
$$Country Name$$
## CONTEXT:
$$Information about local beer culture, history, or geography$$
## CRITICAL OUTPUT FORMAT (READ CAREFULLY):
You have to return a reasoning block first, then ONLY raw, perfectly valid JSON as the final answer. Any mistake with the JSON means the data pipeline breaks.
ABSOLUTELY NO MARKDOWN FORMATTING. Do NOT wrap your response in json or ``` blocks.
NO PREAMBLE OR POSTSCRIPT outside the reasoning block. Do not say "Here is the JSON" or "Enjoy!".
The JSON must contain exactly two keys ("name" and "description"); do not rename or add any other keys.
ESCAPE ALL QUOTES inside the description using ", or use single quotes (' ') instead. Escaping quotes perfectly is super important to avoid errors later.
DO NOT use actual line breaks (\n) inside the string. Keep the description as one continuous string.
Expected JSON format:
{ "name": "Fictional Brewery Name", "description": "The description goes here." }
## CONTENT RULES AND CONSTRAINTS:
### THE HOOK:
The first sentence must be an immersive, sensory environmental hook. It needs to clearly establish the weather, smells, or sounds typical of that city. Do not start by using the brewery's name or standard welcoming phrases.
### GEOGRAPHIC & CULTURAL ANCHOR:
The story must be deeply tied to the provided geographic and cultural info. It should mix historical brewing facts with the gritty reality of modern craft brewing, making sure it fits the local culture perfectly.
### TECHNICAL BREWING DETAIL (VARY THIS!):
You must include one highly specialized technical brewing detail. To avoid sounding repetitive, make sure this varies a lot. Some examples: using local wild yeast (like spontaneous Brettanomyces), adjusting the water profile (like Burtonization), specific mashing techniques, or using local barrels for aging. Don't use basic concepts like generic mash temperatures.
### ARCHITECTURAL DETAIL (VARY THIS!):
You must include one specific architectural or environmental detail, highlighting the building's physical wear, structure, or history. Examples include rusty steel beams, weird acoustics from an old factory, decaying brickwork, or worn-out local infrastructure. Avoid overused industry clichés like repurposed dairy equipment or glycol chillers.
### THE INVITATION:
The last sentence must be an atmospheric invitation to hang out in the space, kept totally objective. Good examples include suggesting where to stand, like "Observation may commence near the foundational supports," or "Positioning adjacent to the exterior loading apparatus is suggested." Avoid regular sayings like telling people to grab a seat or ask the bartender.
### THE BLOCKLIST (FORBIDDEN CONCEPTS):
You absolutely cannot use the following words and phrases because they are overused and too casual. Make sure your final output doesn't have any of these:
- "hidden gem"
- "passion"
- "authentic"
- "repurposed dairy tank"
- "repurposed industrial vat"
- "concrete eggs"
- "glycol chiller"
- "mash temperature"
- "grab a stool"
- "ask the bartender"
### VOICE & PERSPECTIVE:
The description must be written strictly in the third-person objective. You need to act like a detached architectural observer looking at the space and the brewing process from the outside. Do not use first-person or second-person pronouns, keeping an atmosphere of academic distance and professionalism.
## EXAMPLE:
Input:
CITY: Sapporo
COUNTRY: Japan
CONTEXT: Sapporo is the capital of Hokkaido, Japan's northernmost main island, with a subarctic climate: winters are severe and protracted, with the city averaging over 6 metres of cumulative snowfall per season...
$$Truncated for brevity, but assumes full context provided$$
Output:
{ "name": "Tokachi Grain & Ferment", "description": "By February, the powder snow blowing off the Teine range buries the bicycle racks on Susukino's side streets to the crossbar. Sapporo has been in the business of serious lager since 1876, but Tokachi Grain & Ferment isn't interested in replicating the macro-brew legacy. Instead, they source base malt exclusively from Obihiro-area farms and run the entire grain bill through a rigorous Burtonization protocol, driving up calcium sulfate levels to pull a sharp, mineral snap into the finish. The taproom is carved from a former Meiji-era goods shed, where a single run of oxidized copper piping bisects the ceiling and weeps green verdigris onto the communal timber table below. Observation may commence beneath the deteriorating copper, where the pale ale may be procured while the surrounding acoustics are analyzed." }

View File

@@ -17,7 +17,8 @@ void BiergartenDataGenerator::GenerateBreweries(
for (const auto& enriched_city : cities) { for (const auto& enriched_city : cities) {
try { try {
auto brewery = generator_->GenerateBrewery( auto brewery = generator_->GenerateBrewery(
enriched_city.location.city, enriched_city.location.country, BreweryLocation{enriched_city.location.city,
enriched_city.location.country},
enriched_city.region_context); enriched_city.region_context);
generatedBreweries_.push_back(GeneratedBrewery{ generatedBreweries_.push_back(GeneratedBrewery{
.location = enriched_city.location, .brewery = brewery}); .location = enriched_city.location, .brewery = brewery});

View File

@@ -27,6 +27,10 @@ LlamaGenerator::LlamaGenerator(const ApplicationOptions& options,
"LlamaGenerator: sampling top-p must be in (0, 1]"); "LlamaGenerator: sampling top-p must be in (0, 1]");
} }
if (options.top_k == 0U) {
throw std::runtime_error("LlamaGenerator: sampling top-k must be > 0");
}
if (options.seed < -1) { if (options.seed < -1) {
throw std::runtime_error( throw std::runtime_error(
"LlamaGenerator: seed must be >= 0, or -1 for random"); "LlamaGenerator: seed must be >= 0, or -1 for random");
@@ -39,6 +43,7 @@ LlamaGenerator::LlamaGenerator(const ApplicationOptions& options,
sampling_temperature_ = options.temperature; sampling_temperature_ = options.temperature;
sampling_top_p_ = options.top_p; sampling_top_p_ = options.top_p;
sampling_top_k_ = options.top_k;
if (options.seed == -1) { if (options.seed == -1) {
std::random_device random_device; std::random_device random_device;
rng_.seed(random_device()); rng_.seed(random_device());

View File

@@ -6,15 +6,61 @@
#include <spdlog/spdlog.h> #include <spdlog/spdlog.h>
#include <array>
#include <stdexcept> #include <stdexcept>
#include <string> #include <string>
#include "data_generation/llama_generator.h" #include "data_generation/llama_generator.h"
#include "data_generation/llama_generator_helpers.h" #include "data_generation/llama_generator_helpers.h"
BreweryResult LlamaGenerator::GenerateBrewery( namespace {
const std::string& city_name, const std::string& country_name,
const std::string& region_context) { auto ExtractFinalJsonPayload(std::string raw_response) -> std::string {
auto trim = [](std::string_view text) -> std::string_view {
const std::size_t first = text.find_first_not_of(" \t\n\r");
if (first == std::string_view::npos) {
return {};
}
const std::size_t last = text.find_last_not_of(" \t\n\r");
return text.substr(first, last - first + 1);
};
static const std::array<std::string_view, 6> separator_tokens = {
"<|think|>", "<think|>", "<|turn|>",
"<turn|>", "<channel|>", "<|channel|>"};
std::size_t separator_pos = std::string::npos;
std::size_t separator_length = 0;
for (const std::string_view token : separator_tokens) {
const std::size_t candidate_pos = raw_response.rfind(token);
if (candidate_pos != std::string::npos &&
(separator_pos == std::string::npos ||
candidate_pos > separator_pos)) {
separator_pos = candidate_pos;
separator_length = token.size();
}
}
if (separator_pos != std::string::npos) {
raw_response.erase(0, separator_pos + separator_length);
}
const std::string_view trimmed = trim(raw_response);
std::string json_candidate =
ExtractLastJsonObjectPublic(std::string(trimmed));
if (!json_candidate.empty()) {
return ExtractLastJsonObjectPublic(std::string(trimmed));
}
return std::string(trimmed);
}
} // namespace
auto LlamaGenerator::GenerateBrewery(const BreweryLocation& location,
const std::string& region_context)
-> BreweryResult {
/** /**
* Preprocess and truncate region context to manageable size * Preprocess and truncate region context to manageable size
*/ */
@@ -24,10 +70,9 @@ BreweryResult LlamaGenerator::GenerateBrewery(
/** /**
* Load brewery system prompt from file * Load brewery system prompt from file
* Falls back to minimal inline prompt if file not found * Falls back to minimal inline prompt if file not found
* Default path: prompts/brewery_system_prompt_expanded.txt
*/ */
const std::string system_prompt = const std::string system_prompt =
LoadBrewerySystemPrompt("prompts/brewery_system_prompt_expanded.txt"); LoadBrewerySystemPrompt("prompts/system.md");
/** /**
* User prompt: provides geographic context to guide generation towards * User prompt: provides geographic context to guide generation towards
@@ -35,21 +80,28 @@ BreweryResult LlamaGenerator::GenerateBrewery(
*/ */
std::string prompt = std::string prompt =
"Write a brewery name and place-specific long description for a craft " "Write a brewery name and place-specific long description for a craft "
"brewery in " + "brewery in ";
city_name + prompt.append(location.city_name);
(country_name.empty() ? std::string("") if (!location.country_name.empty()) {
: std::string(", ") + country_name) + prompt.append(", ");
(safe_region_context.empty() prompt.append(location.country_name);
? std::string(".") }
: std::string(". Regional context: ") + safe_region_context); if (safe_region_context.empty()) {
prompt.append(".");
} else {
prompt.append(". Regional context: ");
prompt.append(safe_region_context);
}
/** /**
* Store location context for retry prompts (without repeating full context) * Store location context for retry prompts (without repeating full context)
*/ */
const std::string retry_location = std::string retry_location = "Location: ";
"Location: " + city_name + retry_location.append(location.city_name);
(country_name.empty() ? std::string("") if (!location.country_name.empty()) {
: std::string(", ") + country_name); retry_location.append(", ");
retry_location.append(location.country_name);
}
/** /**
* RETRY LOOP with validation and error correction * RETRY LOOP with validation and error correction
@@ -72,8 +124,9 @@ BreweryResult LlamaGenerator::GenerateBrewery(
std::string name; std::string name;
std::string description; std::string description;
const std::string json_only = ExtractFinalJsonPayload(raw);
const std::string validation_error = const std::string validation_error =
ValidateBreweryJsonPublic(raw, name, description); ValidateBreweryJsonPublic(json_only, name, description);
if (validation_error.empty()) { if (validation_error.empty()) {
// Success: return parsed brewery data // Success: return parsed brewery data
return {std::move(name), std::move(description)}; return {std::move(name), std::move(description)};
@@ -90,11 +143,13 @@ BreweryResult LlamaGenerator::GenerateBrewery(
// limits. // limits.
prompt = prompt =
"Your previous response was invalid. Error: " + validation_error + "Your previous response was invalid. Error: " + validation_error +
"\nReturn ONLY valid JSON with this exact schema: " "\nReturn ONLY valid JSON with exactly these keys: "
"{\"name\": \"string\", \"description\": \"string\"}." "{\"name\": \"<brewery name>\", "
"\nDo not include markdown, comments, or extra keys." "\"description\": \"<single-paragraph description>\"}."
"\n\n" + "\nDo not include markdown, comments, extra keys, or literal "
retry_location; "placeholder values.";
prompt += "\n\n";
prompt += retry_location;
} }
// All retry attempts exhausted: log failure and throw exception // All retry attempts exhausted: log failure and throw exception

View File

@@ -263,12 +263,14 @@ static void AppendTokenPiece(const llama_vocab* vocab, llama_token token,
output.append(buffer.data(), static_cast<std::size_t>(bytes)); output.append(buffer.data(), static_cast<std::size_t>(bytes));
} }
static bool ExtractFirstJsonObject(const std::string& text, static bool ExtractLastJsonObject(const std::string& text,
std::string& json_out) { std::string& json_out) {
std::size_t start = std::string::npos; std::size_t start = std::string::npos;
int depth = 0; int depth = 0;
bool in_string = false; bool in_string = false;
bool escaped = false; bool escaped = false;
bool found = false;
std::string candidate;
for (std::size_t i = 0; i < text.size(); ++i) { for (std::size_t i = 0; i < text.size(); ++i) {
const char ch = text[i]; const char ch = text[i];
@@ -303,13 +305,27 @@ static bool ExtractFirstJsonObject(const std::string& text,
} }
--depth; --depth;
if (depth == 0 && start != std::string::npos) { if (depth == 0 && start != std::string::npos) {
json_out = text.substr(start, i - start + 1); candidate = text.substr(start, i - start + 1);
return true; found = true;
} }
} }
} }
if (!found) {
return false; return false;
}
json_out = std::move(candidate);
return true;
}
std::string ExtractLastJsonObjectPublic(const std::string& text) {
std::string extracted;
if (ExtractLastJsonObject(text, extracted)) {
return extracted;
}
return {};
} }
static std::string ValidateBreweryJson(const std::string& raw, static std::string ValidateBreweryJson(const std::string& raw,
@@ -371,7 +387,7 @@ static std::string ValidateBreweryJson(const std::string& raw,
std::string validation_error; std::string validation_error;
if (ec) { if (ec) {
std::string extracted; std::string extracted;
if (!ExtractFirstJsonObject(raw, extracted)) { if (!ExtractLastJsonObject(raw, extracted)) {
return "JSON parse error: " + ec.message(); return "JSON parse error: " + ec.message();
} }

View File

@@ -119,7 +119,8 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
/** /**
* SAMPLER CONFIGURATION PHASE * SAMPLER CONFIGURATION PHASE
* Set up the probabilistic token selection pipeline (sampler chain) * Set up the probabilistic token selection pipeline (sampler chain)
* Samplers are applied in sequence: temperature -> top-p -> distribution * Samplers are applied in sequence: temperature -> top-k -> top-p ->
* distribution
*/ */
llama_sampler_chain_params sampler_params = llama_sampler_chain_params sampler_params =
llama_sampler_chain_default_params(); llama_sampler_chain_default_params();
@@ -135,6 +136,13 @@ std::string LlamaGenerator::InferFormatted(const std::string& formatted_prompt,
*/ */
llama_sampler_chain_add(sampler.get(), llama_sampler_chain_add(sampler.get(),
llama_sampler_init_temp(sampling_temperature_)); llama_sampler_init_temp(sampling_temperature_));
/**
* Top-K: limits sampling to the most likely tokens before nucleus
* sampling
*/
llama_sampler_chain_add(
sampler.get(),
llama_sampler_init_top_k(static_cast<int32_t>(sampling_top_k_)));
/** /**
* Top-P: nucleus sampling - filters to most likely tokens summing to top_p * Top-P: nucleus sampling - filters to most likely tokens summing to top_p
* probability * probability

View File

@@ -9,10 +9,10 @@
#include "data_generation/mock_generator.h" #include "data_generation/mock_generator.h"
std::size_t MockGenerator::DeterministicHash(const std::string& a, auto MockGenerator::DeterministicHash(const BreweryLocation& location)
const std::string& b) { -> std::size_t {
std::size_t seed = 0; std::size_t seed = 0;
boost::hash_combine(seed, a); boost::hash_combine(seed, location.city_name);
boost::hash_combine(seed, b); boost::hash_combine(seed, location.country_name);
return seed; return seed;
} }

View File

@@ -8,11 +8,10 @@
#include "data_generation/mock_generator.h" #include "data_generation/mock_generator.h"
auto MockGenerator::GenerateBrewery(const std::string& city_name, auto MockGenerator::GenerateBrewery(const BreweryLocation& location,
const std::string& country_name,
const std::string& /*region_context*/) const std::string& /*region_context*/)
-> BreweryResult { -> BreweryResult {
const std::size_t hash = DeterministicHash(city_name, country_name); const std::size_t hash = DeterministicHash(location);
const std::string& adjective = const std::string& adjective =
kBreweryAdjectives.at(hash % kBreweryAdjectives.size()); kBreweryAdjectives.at(hash % kBreweryAdjectives.size());
@@ -21,11 +20,20 @@ auto MockGenerator::GenerateBrewery(const std::string& city_name,
const std::string& base_description = const std::string& base_description =
kBreweryDescriptions.at((hash / 13) % kBreweryDescriptions.size()); kBreweryDescriptions.at((hash / 13) % kBreweryDescriptions.size());
const std::string name = city_name + " " + adjective + " " + noun; std::string name(location.city_name);
const std::string description = name.append(" ");
base_description + " Based in " + city_name + name.append(adjective);
(country_name.empty() ? std::string(".") name.append(" ");
: std::string(", ") + country_name + "."); name.append(noun);
std::string description = base_description;
description.append(" Based in ");
description.append(location.city_name);
if (!location.country_name.empty()) {
description.append(", ");
description.append(location.country_name);
}
description.append(".");
return {name, description}; return {name, description};
} }

View File

@@ -40,10 +40,12 @@ auto ParseArguments(const int argc, char** argv,
"Use mocked generator for brewery/user data")( "Use mocked generator for brewery/user data")(
"model,m", prog_opts::value<std::string>()->default_value(""), "model,m", prog_opts::value<std::string>()->default_value(""),
"Path to LLM model (gguf)")( "Path to LLM model (gguf)")(
"temperature", prog_opts::value<float>()->default_value(0.8f), "temperature", prog_opts::value<float>()->default_value(1.0F),
"Sampling temperature (higher = more random)")( "Sampling temperature (higher = more random)")(
"top-p", prog_opts::value<float>()->default_value(0.92f), "top-p", prog_opts::value<float>()->default_value(0.95F),
"Nucleus sampling top-p in (0,1] (higher = more random)")( "Nucleus sampling top-p in (0,1] (higher = more random)")(
"top-k", prog_opts::value<uint32_t>()->default_value(64),
"Top-k sampling parameter (higher = more candidate tokens)")(
"n-ctx", prog_opts::value<uint32_t>()->default_value(8192), "n-ctx", prog_opts::value<uint32_t>()->default_value(8192),
"Context window size in tokens (1-32768)")( "Context window size in tokens (1-32768)")(
"seed", prog_opts::value<int>()->default_value(-1), "seed", prog_opts::value<int>()->default_value(-1),
@@ -88,11 +90,12 @@ auto ParseArguments(const int argc, char** argv,
const bool has_llm_params = !variables_map["temperature"].defaulted() || const bool has_llm_params = !variables_map["temperature"].defaulted() ||
!variables_map["top-p"].defaulted() || !variables_map["top-p"].defaulted() ||
!variables_map["top-k"].defaulted() ||
!variables_map["seed"].defaulted(); !variables_map["seed"].defaulted();
if (use_mocked && has_llm_params) { if (use_mocked && has_llm_params) {
spdlog::warn( spdlog::warn(
"Sampling parameters (--temperature, --top-p, --seed) are" "Sampling parameters (--temperature, --top-p, --top-k, --seed) are"
" ignored when using --mocked"); " ignored when using --mocked");
} }
@@ -100,6 +103,7 @@ auto ParseArguments(const int argc, char** argv,
options.model_path = model_path; options.model_path = model_path;
options.temperature = variables_map["temperature"].as<float>(); options.temperature = variables_map["temperature"].as<float>();
options.top_p = variables_map["top-p"].as<float>(); options.top_p = variables_map["top-p"].as<float>();
options.top_k = variables_map["top-k"].as<uint32_t>();
options.n_ctx = variables_map["n-ctx"].as<uint32_t>(); options.n_ctx = variables_map["n-ctx"].as<uint32_t>();
options.seed = variables_map["seed"].as<int>(); options.seed = variables_map["seed"].as<int>();
@@ -140,10 +144,9 @@ auto main(const int argc, char** argv) noexcept -> int {
spdlog::info( spdlog::info(
"[Generator] Using LlamaGenerator: {} (temperature={}, " "[Generator] Using LlamaGenerator: {} (temperature={}, "
"top-p={}, " "top-p={}, top-k={}, n_ctx={}, seed={})",
"n_ctx={}, seed={})",
options.model_path, options.temperature, options.top_p, options.model_path, options.temperature, options.top_p,
options.n_ctx, options.seed); options.top_k, options.n_ctx, options.seed);
return injector.template create<std::unique_ptr<LlamaGenerator>>(); return injector.template create<std::unique_ptr<LlamaGenerator>>();
})); }));