Benchmark one agent task before paying to train a model
Turn one narrow agent job into a ten-example evaluation pack, a pass/fail rubric and a versioned baseline before committing to training compute.
A small agent-evaluation pack with ten anonymised examples, a written pass/fail rubric, a dated baseline, reviewed failure notes and one specific upgrade trigger.
Start with the behaviour you need to measure, not a training membership. It does not run an external agent, train a model, test provider performance or claim a customer result. The workflow visual is a FreeStack editorial illustration, not an Overmind or provider screenshot.
About this guide & its examples
This original FreeStack workflow uses a fictional support-routing task and ongoing-free software to make one agent decision inspectable. Official sources checked 28 October 2026.
Illustrations are FreeStack concepts, not provider screenshots. Check the free limits below and test your own files before delivery.
Free boundaries before you start
What’s free, whether you need an account, and when we last checked.
Git · checked 2026-10-28
Free GPLv2 distributed version-control software for local repositories, with no provider project or history quota. Remote hosting, backup, access control and large-file storage are separate services.
No account is required for a local Git repository. Any remote host or sync service can require an account.
A small example to aim for
Route one support note
One job.
One rubric. One decision.
Safe examples · reviewed traces · named checkpoints · specific upgrade trigger
FreeStack editorial workflow illustration · not an Overmind screenshot, tested result or customer outcome.
Illustrative example · adapt it to your project.
Jump to a step
Write one observable job
Define one input and one useful output: route a fictional support note to billing, access or product and include one sentence explaining the choice. Exclude replies, refunds, account access and automatic publishing. Name who reviews the decision, what data must never enter the pack and the point where the run must stop. A narrow job makes the evaluation useful before infrastructure or model choice obscures the question.
Create ten safe examples
Write ten fictional or properly anonymised notes that cover ordinary cases, ambiguous wording and one out-of-scope request. Store each input beside the expected route and the reason it belongs there. Do not copy customer messages into an unapproved model or trace store. Ten examples are enough to learn whether the rubric works; they are not a production benchmark or representative dataset.
Write the pass/fail rubric before the run
Give each example three checks: the route matches the expected category, the explanation cites only the supplied note, and the agent refuses the out-of-scope action. Mark an example failed if any required check fails. Record how ambiguous cases are handled and who can change the expected answer. A rubric prevents a persuasive response from becoming an automatic pass.
Run a baseline and keep the evidence bounded
Use Overmind with an approved local or external model route to record the ten inputs, outputs and rubric decisions. Keep secrets and personal data out of traces. Record the model identifier, prompt version, tool permissions and date. Review every failure rather than optimising against an aggregate number alone. FreeStack has not run this example or validated a provider result.
Version the contract and the decision
Save the task contract, example set, rubric and reviewed result summary as ordinary files, then create named Git checkpoints for dataset-v1, rubric-v1 and baseline-v1. Do not commit credentials, raw private messages or model secrets. Local Git history is not an independent backup, and a remote host adds its own account, permission and retention decisions.
Pay only for a repeated, valuable gap
Keep the free workflow while a small reviewed baseline answers the question. FreeStack editorial upgrade triggers: repeated rubric failures on a valuable task, a validated dataset large enough to justify training, governed team review, managed trace retention, supported deployment or compute the current environment cannot provide. Estimate the training and operating cost against the value of fixing the named failure before buying a membership.
Your workflow, connected.
Follow the steps and carry the output into the next stage. These are suggested handoffs, not automatic integrations.
- 02Make the dataset
marimo is Apache-2.0 local software and stores notebooks as Python files. Packages, models, compute and hosted deployment remain separate.
Ten reviewable examples - 03Define success
Overmind's platform is AGPL-3.0 and self-hostable; its SDK and CLI are MIT. Model inference, storage and compute remain separate.
Rubric + review rule - 05Make change visible
Git is GPLv2 free software. Repository hosting, backup and access control are separate.
Named evaluation checkpoints
Provider information
Check current plans and documentation before committing to a workflow.
A couple of useful questions
Do I need to train a model to finish the guide?
No. The useful result is a reviewed baseline and a decision. Training is justified only when a repeatable failure, suitable data and enough value remain after prompt, tool and workflow changes.
What stays free?
Overmind publishes a self-hostable AGPL platform and MIT SDK/CLI. marimo and Git are ongoing-free software. Your model route, GPUs, storage, server, remote repository and team operations remain separate.
Is ten examples enough?
It is a deliberately small learning set for checking the task and rubric. It is not evidence of production accuracy or coverage. Expand only after the first examples reveal what needs representation.
What does Save this toolkit keep?
The existing FreeStack flow saves Overmind, marimo and Git as catalogue links in a private stack after sign-in. It does not copy data, run an evaluation, connect a model, create a repository or purchase compute.
What would you try?
Share your result or ask a specific question about this workflow.
Join the conversation