Original tutorialDocs reviewed

Check saved outputs with Promptfoo CSV regression fixtures

A local evaluation archive and a boundary for independent real outputs.

Use caseUse __expected exact assertions and a local JavaScript provider to check saved outputs. An intentional failure verifies reporting without an LLM judge. Includes a complete fixture archive.

Source · JevLog editorialIntermediate15 min

Start reading

Use __expected exact assertions and a local JavaScript provider to check saved outputs. An intentional failure verifies reporting without an LLM judge. Includes a complete fixture archive.

Source · JevLog editorial · Original guide

Source cover · Promptfoo

Source cover · Promptfoo

Published 2026-10-02 from current official docs with original exercises. No live service calls; source publication dates not established.

Problem

Regression checks detect changed behavior. A provider returning expected labels creates fake passes, so store expectations and predictions separately. This archive checks fictional outputs and does not call Jev.

Before you begin

Install Node.js and Promptfoo and extract promptfoo-local-regression.zip into a new directory.
Use equals assertions without an LLM judge. Installation needs network; the provider reads local files only.
Official docs
Official docs

Original JevLog diagram, not a product screenshot or measured result.

Original JevLog diagram, not a product screenshot or measured result.

Steps

1. Freeze expected labels

cases.csv has case_id, text and __expected with equals: labels. c03 deliberately has a wrong prediction to verify reporting. Do not change labels to fit predictions.

case_id,text,__expected
c01,Download crashes,equals:bug
c02,Please explain a setting,equals:question
c03,Thanks for your help,equals:other

2. Read independent predictions

Read results.json by case ID without reading __expected. Missing IDs return error, not empty classifications. Save as provider.cjs in CommonJS format.

const fs = require('node:fs');
const path = require('node:path');
module.exports = class SavedResults {
 id() { return 'saved-results-fixture'; }
 async callApi(prompt, context) {
  const results = JSON.parse(fs.readFileSync(path.join(__dirname, 'results.json'), 'utf8'));
  const id = context?.vars?.case_id;
  if (!Object.prototype.hasOwnProperty.call(results, id)) return {error: 'Missing result: ' + id};
  return {output: results[id], metadata: {mode: 'fictional-local-fixture'}};
 }
};

3. Configure and generate a report

Keep configuration beside provider.cjs and cases.csv. Run npx promptfoo eval -c promptfooconfig.yaml, then npx promptfoo view. The archive includes these files; this article is not a measured model evaluation.

description: Local saved-output regression fixture
prompts:
 - '{{text}}'
providers:
 - file://provider.cjs
tests: file://cases.csv

4. Confirm the intentional mismatch

results.json predicts question for c03, conflicting with other. If all pass, inspect loaded files and provider logic. Two correct fixtures and one mismatch are not Jev accuracy.

5. Replace fixtures with independently generated outputs

Label and scrub your inputs, then generate predictions independently from live calls. Version models, questions and hashes, and retain errors. Missing outputs are failures; measure confidence and review volume separately.

Practice sample: download promptfoo-local-regression.zip for local use; not a live run here.

Result and cautions