AI Engineer
MarketBrew Oracle Agent
An eval-first agent on top of the MarketBrew knowledge base, with answers cited back to the atoms.
There is no public URL for this one. The write-up below covers the architecture and the decisions behind it, and we can walk you through it on a call.
The agent layer built on top of the MarketBrew knowledge base (see the Reference Manual project). An eval-first agentic scaffold on PostgreSQL: contracts and fixtures are defined before generation, and every recommendation is evidence-cited back to the underlying knowledge atoms. The scaffold pattern transfers across client domains.
The problem
Most LLM-backed recommendation tools generate first and validate later. When the generation is wrong, the team is left chasing why. The eval-first pattern flips that order: define the contract, build the fixtures, generate against the contract. Outputs are checkable from the moment they exist.
What we built
A PostgreSQL-backed agent layered on the MarketBrew knowledge atoms, with contracts and fixtures defined before any generation happens. Every recommendation cites the supporting atom, so a reviewer can trace the path from output back to evidence. The eval-first scaffold is the deliverable: a foundation that is checkable from the first output rather than debugged after the fact.
What was delivered
- PostgreSQL schema for inputs, fixtures, and outputs
- Contract-and-fixture-first architecture
- Evidence-cited recommendation pattern with traceable provenance
- Reusable scaffold for adjacent eval-first agentic work
Outcomes
- Outputs are checkable from the moment they exist
- Recommendations cite their supporting data
- Scaffold pattern transfers to adjacent client domains
- Demonstrates the eval-first discipline Sonoma Solutions applies to AI engagements
Services: PostgreSQL, Anthropic API, Node.js, Eval-first design