Request access
Product
Use cases
Docs
Resources
PricingRequest access

What it does

And around it

Compose. Version. Prove. Publish.

One lifecycle for every kind of AI asset.

{{ }}

Who it is for

Who uses a prompt library

Four ways people use it today.

Start here

Look things up

Documentation

Guides, the tool reference and the glossary.

Read

Know us

Read, ask, follow along

Notes from building Prompt Verse.

evalshow-to

Test a prompt with five cases and no tooling

A practical way to check whether a changed prompt is better: pick five inputs, write what a good answer contains, score both versions and keep the winner.

The Prompt Verse team, AVS Media
· 2 min read

“Is the new wording better?” is the most common question about prompts and the least often answered with evidence. You do not need a test framework to answer it. You need five cases and an honest hour.

Step 1: pick five inputs that matter#

Do not pick five typical inputs. Pick the ones that have gone wrong, or could:

  1. The angry message.
  2. The message with a missing detail (no order number).
  3. The message that is not a question at all.
  4. The longest message you have seen.
  5. The one that nearly caused an incident.

If you cannot find five, find three. The point is to cover failure, not the middle.

Step 2: write what a good answer contains#

For each input, write one line about the answer. Not the whole answer, just a check you can apply:

  • “Says what happens next and when.”
  • “Does not promise a refund.”
  • “Asks for the order number.”
  • “Stays under six sentences.”

Where you can, make the check mechanical. “Contains the words ‘order number’” is a check a machine can apply. “Sounds kind” needs a judge, either a person or a model.

Step 3: choose a scoring method#

Prompt Verse offers five: exact, contains, structural, judge and hybrid. Use the simplest that fits.

  • contains for “must mention” checks.
  • structural when the answer must be valid JSON with certain keys.
  • judge when quality is the point and you accept that a model does the grading.
  • hybrid when you want a rule and a judgement together.

Step 4: run both versions on the same cases#

This is the part that matters. Run the old wording and the new wording on the same five inputs and score each answer from 0 to 1. Average them. Set a threshold you are willing to defend, 0.8 is the default in Prompt Verse.

In the pre-beta Prompt Verse runs no model, so your own AI does this step: it reads the saved cases, runs each version and scores the answers in the conversation.

Step 5: keep the winner and say why#

Commit the version that scored higher with a message that includes the numbers: “Shorter reply. Mean 0.90, was 0.78 on five cases.” Six months from now that sentence is worth more than the diff.

What this will and will not tell you#

Five cases will not prove a prompt is good. They will catch the regressions you already know about, and they will make you articulate what “good” means, which is half the value. When a new failure appears in the wild, add it as case six. That is how a test set grows, and it is how prompts stop being fragile.

Keep reading