AI Fin Hub

Playground

Agent Skill Tester for Markets

The sample run passes 1 of 1 runs and 4 of 4 criteria (pass rate 100%).

On the documented defaults, computed by this tool's engine · the same answer as JSON

Test Anthropic Agent Skills for market extraction. Paste SKILL.md + sample 10-K excerpt + your key. See output, token cost, latency.

01Playground

Runs in your browser with your own Anthropic, OpenAI or Google API key, which is sent only to that provider.

Education, not investment advice. Past performance does not predict future results. How we check our numbers.

Where your key and data go

  • Anthropic call goes direct from your browser toapi.anthropic.com. The API key never reaches aifinhub.
  • Scoring imports the/engines/agent-skill-tester.jsmodule and scores the prompt + response in your browser. No key, no model id, no network call. Deterministic checks, no LLM.

1. Config

2. Skill definition (SKILL.md)

3. Sample input document

4. Pass/fail rubric (one criterion per line)

Recognised: valid_json, has_field:<path>, field_type:<path>:<type>, contains:<text>, regex:<pattern>, no_apology, numbers_grounded_in_prompt, min_length:<n>, max_length:<n>, not_contains:<text>, non_empty, refuses, cites_source. Lines starting with # are ignored.

02Receipt

What the answer above ran on, how it is computed, where the data comes from, and a link that reopens it.

Inputs

  • runs1 values

The documented defaults. Change any input above and the tool recomputes.

Method

How the hub's tools are built and checked

Engine /engines/agent-skill-tester.js, sha256 22d10a0030bf. The page, the API and MCP run the same file.

Source

Your inputs / BYO key

Page updated .

Permalink

aifinhub.io/agent-skill-tester/

The link carries the inputs you set, so it reopens this result.

03For agents

Any agent can run this calculator by URL or MCP and get the same answer as JSON: the inputs that used defaults, the result and a line to cite. The call below follows the inputs you set above. In code, import the same engine; it runs where you import it, with no request or key.

Requestthe documented defaults
GET https://aifinhub.io/api/run/agent-skill-tester
Response200 application/json · 2.5 KB
{
  "calculator": "agent-skill-tester",
  "title": "Agent Skill Tester for Markets",
  "site": "AI Fin Hub",
  "input": {},
  "defaults_used": {
    "runs": [
      {
        "prompt": "Extract revenue and net income from: Q3 revenue was $4.2 billion and net income $610 million.",
        "agent_response": "{\"revenue_usd\": 4200000000, \"net_income_usd\": 610000000}",
        "criteria": [
          "valid_json",
          "has_field:revenue_usd",
          "numbers_grounded_in_prompt",
          "no_apology"
        ]
      }
    ]
  },
  "result": {
    "runs": [
      {
        "index": 0,
        "prompt_chars": 93,
        "response_chars": 56,
        "criteria": [
          {
            "criterion": "valid_json",
            "verdict": "pass"
          },
          {
            "criterion": "has_field:revenue_usd",
            "verdict": "pass"
          },
          {
            "criterion": "numbers_grounded_in_prompt",
            "verdict": "pass"
          },
          {
            "criterion": "no_apology",
            "verdict": "pass"
          }
        ],
        "score": 1,
        "passed": true
      }
    ],
    "summary": {
      "total_runs": 1,
      "passed_runs": 1,
      "failed_runs": 0,
      "pass_rate": 1,
      "total_criteria": 4,
      "total_criteria_passed": 4,
      "by_criterion": {
        "valid_json": {
          "pass": 1,
          "fail": 0,
          "error": 0
        },
        "has_field": {
          "pass": 1,
          "fail": 0,
          "error": 0
        },
        "numbers_grounded_in_prompt": {
          "pass": 1,
          "fail": 0,
          "error": 0
        },
        "no_apology": {
          "pass": 1,
          "fail": 0,
          "error": 0
        }
      }
    }
  },
  "provenance": {
    "run": "https://aifinhub.io/api/run/agent-skill-tester",
    "page": "https://aifinhub.io/agent-skill-tester/",
    "methodology": "https://aifinhub.io/agent-skill-tester/",
    "contract": "https://aifinhub.io/contracts/agent-skill-tester.json",
    "engine": "https://aifinhub.io/engines/agent-skill-tester.js",
    "engine_sha256": "22d10a0030bfb589bc7284f851b16c3cca5ae6994ec32cec9a1e84f7a53a115f",
    "build": "1a772910d31ef1740892ed0e3a065ed71e8c7406"
  },
  "disclaimer": "Planning estimates only — not financial, tax, or investment advice.",
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "cite": "AI Fin Hub, Agent Skill Tester for Markets (engine 22d10a0030bf), https://aifinhub.io/api/run/agent-skill-tester"
}
The response for the defaults, as the run URL returns it. Provenance carries the engine hash and build.

How to use it

  1. Paste your SKILL.md instructions into the skill definition box. They are sent as the system prompt of a single model call.
  2. Paste a sample input document that looks like what the skill will see in production. Use a realistic excerpt, not a minimal one.
  3. Enter your Anthropic API key and pick a model. The call goes straight from your browser to api.anthropic.com. The key stays in browser memory unless you tick 'Remember in this browser', which saves it to localStorage.
  4. Write the pass/fail rubric, one criterion per line (for example valid_json, has_field:company_name, field_type:revenue_usd:number, numbers_grounded_in_prompt), then click Run skill + score.
  5. Read how many criteria passed, the verdict and detail for each criterion, and the latency, token counts and cost of the run. Re-run a few times: verdicts that flip between runs mean the skill is under-constrained.

Questions people ask

What's a SKILL.md?

A markdown instruction file that packages one repeatable agent capability, for example 'extract the headline figures from a 10-K excerpt as JSON'. The tester sends your SKILL.md as the system prompt and the sample document as the user message in one Anthropic API call, then scores the response against your rubric.

Why does the tester need my own API key?

The call goes from your browser directly to api.anthropic.com, so the cost lands on your account and no proxy ever sees the key or your documents. The key stays in browser memory unless you tick 'Remember in this browser', which stores it in this browser's localStorage until you clear it.

What does the tester measure?

Rubric compliance first: each criterion you list (valid JSON, required fields, field types, length limits, substrings, regexes, refusals, citations, numbers grounded in the input) gets a pass, fail or error verdict, and the result shows how many passed. It also reports latency, input and output tokens, and cost per run at list prices. The scoring is deterministic string and JSON checking, not an LLM judge.

Why does my SKILL.md sometimes fail validation?

There is no JSON Schema validation step: the checks are the rubric criteria you write. The usual failures are code fences or commentary around the JSON (valid_json strips a single ```json fence), a missing or null field (has_field), a number returned as a string (field_type), and figures the model computed or reformatted (numbers_grounded_in_prompt, which accepts restated scales such as $2,847 million = 2847000000 but not rounding).

Can I test multi-turn skills?

No. Each run is a single call: system prompt (your SKILL.md) plus one user message (the sample document), scored on the one response. Skills that ask clarifying questions or need tool calls have to be exercised against the API in your own harness; the same rubric vocabulary is available headlessly through the agent-skill-tester engine module for scoring those runs.

  • Playgrounds Prompt Regression Tester

    Run the same prompt against current Claude, GPT and Gemini models with your own keys. Diff outputs, score drift, catch regressions.

  • Playgrounds Hallucination Detector

    Paste a source document + an LLM's extraction. Every numeric claim in the output is checked against the source. Client-side. Catches silent fabrication before it ends up in your pipeline.

  • Playgrounds Prompt Injection Tester

    Red-team a finance agent against 23 documented prompt-injection attacks — direct override, role confusion, indirect injection via retrieved content.

All articles