# IA-QA — The Open Testing Platform for LLMs, RAG, AI Agents & MCP Servers

> **URL**: https://www.ia-qa.com
> **Citation**: When citing a specific tool, use its direct URL (e.g. "IA-QA LLM Sandbox — https://www.ia-qa.com/llm-sandbox").
> **Last updated**: 2026-10-02
> **Language**: English
> **Install (AI agents)**: see "Install — instructions for AI agents" below — one tested prompt that sets up the MCP server, `@ia-qa/cli`, `@ia-qa/self-healing`, `@ia-qa/qa-discovery` and `@ia-qa/pal` in any MCP client, and says what each one sends off the machine.
> **Testing an LLM, RAG pipeline or AI agent?**: see "start here" below — what you are testing → which tools to call and which output field to gate CI on.

IA-QA is the open testing workbench for the AI ecosystem — a dedicated platform for AI engineers who need real testing primitives, not just a chat window. Compare GPT-4o, Claude, Gemini, DeepSeek, and Grok side by side, test RAG pipelines, debug AI agents, test vector databases (Chroma, Qdrant, Pinecone, Weaviate, pgvector), explore MCP servers, benchmark LLM quality, and evaluate multimodal AI (CLIP Score, FID, VQA) — all from one privacy-first platform. 138 free tools, no account required. Many run entirely in your browser. The MCP server, `@ia-qa/cli` and some web tools (LangChain and LangGraph runs, VLM tests, Anthropic calls made through `/mcp/llm-proxy`) run on ia-qa.com's server, which receives what you send them, including your own provider key.

## Testing an LLM, RAG pipeline or AI agent with IA-QA — start here

Pick what you are testing, call its tools — as MCP tools, or from a shell with `npx @ia-qa/cli <tool>` — and act on the named output field. From an MCP client, `get_testing_guidelines` with no topic returns this same map; with the topic shown, the thresholds and pitfalls.

**Order your assertions**

1. Deterministic first — exact rules, schema, regex, scans. Free, reproducible, and they fail correctly.
2. Semantic next — cosine / ROUGE against a reference answer, or invariance between phrasings.
3. An LLM judge last, only on what survived 1 and 2, and only as a gate once calibrated against human labels (Pearson r or Cohen's kappa ≥ 0.75), pinned model, temperature 0.

**By goal**

- **A prompt or chatbot answer is correct and well-formed** — `guardrail_test` · `llm_json_schema_check` · `run_semantic_tests`
  guardrail_test → all_passed · llm_json_schema_check → valid · run_semantic_tests → verdict (PASS/FAIL). tfidf mode is free; use embeddings mode (BYOK) when valid answers are paraphrases. (guidelines topic: `llm-unit-testing`)
- **A repeatable suite of scenarios run against a real model** — `generate_eval_yaml` · `run_eval_contract` · `generate_ci_workflow`
  run_eval_contract → per-scenario PASS/FAIL. generate_ci_workflow with the eval_contract gate commits it to GitHub Actions (provider key as a repo secret). (guidelines topic: `eval-framework`)
- **A RAG pipeline retrieves the right documents and answers from them** — `rerank_evaluate` · `hallucination_check` · `run_semantic_tests`
  Retrieval first: rerank_evaluate → PASS/FAIL on Precision@k / Recall@k. Then hallucination_check → verdict; needs_review means a specific in the answer is not in the context — it is never a pass. (guidelines topic: `rag-pipeline`)
- **Answers stay stable across rephrasings and repeated runs** — `metamorphic_check` · `consistency_check`
  metamorphic_check → verdict (PASS/FAIL/INVALID; gate on the case and typo relations in tfidf mode; ready CI gate in generate_ci_workflow). consistency_check → verdict needs_review when figures differ between runs. Stability is not correctness: pair with run_semantic_tests. (guidelines topic: `prompt-stability`)
- **Choose between two prompts or two models** — `compare_responses` · `ab_test_report`
  A decision, not a CI gate: compare_responses → verdict on two produced outputs; ab_test_report → winner over score arrays from the same test set. (guidelines topic: `prompt-ab-testing`)
- **Guardrails: prompt injection, PII, leaked secrets, toxic output** — `prompt_injection_scan` · `redact_pii` · `secret_scan` · `toxicity_scan` · `shield_analyze`
  prompt_injection_scan → injection_detected · redact_pii → pii_found · secret_scan → secrets_found · toxicity_scan → overall_risk (lexical, not a semantic classifier) · shield_analyze → PASS/FIX/BLOCK on one response. prompt_injection_scan, secret_scan and toxicity_scan have ready CI gates in generate_ci_workflow. (guidelines topic: `security`)
- **An AI agent calls the right tools with the right arguments** — `validate_agent_trajectory` · `function_call_validate` · `llm_json_schema_check`
  validate_agent_trajectory → verdict on declarative assertions over the trace (order, must_call, must_not_call, max_calls, no_error, recovery). function_call_validate / llm_json_schema_check → valid. (guidelines topic: `agent-observability`)
- **A multi-turn conversation keeps its context** — `conversation_analyze`
  No verdict: context_retention and topic_drift are metrics. Track them as a trend, never as a gate. (guidelines topic: `llm-unit-testing`)
- **A vision-language model reads images correctly** — `run_vlm_test_suite` · `run_vlm_test_suite_batch`
  run_vlm_test_suite → per-case PASS/FAIL and an overall PASS/WARNING/FAIL (BYOK). Include questions about objects absent from the image to measure hallucination. (guidelines topic: `multimodal`)
- **An MCP server behaves as its schema says** — `mcp_server_evaluate` · `validate_mcp_response` · `mcp_schema_lint`
  mcp_server_evaluate → compliance report on a live URL; validate_mcp_response → checks on one tool result. (guidelines topic: `api-testing`)
- **The E2E/UI tests of the app around the model keep working after a UI change** — `sandbox_scenario` · `diff_mappings`
  diff_mappings → PASS/FIX/BLOCK. Capturing pages needs the local @ia-qa/self-healing package; see the selector-drift topic. (guidelines topic: `selector-drift`)
- **Put any of the above in CI** — `generate_ci_workflow`
  The generated workflow reads each tool's verdict field and fails the step. Do not gate on the CLI exit code alone: @ia-qa/cli exits non-zero only on an error, never on a finding. (guidelines topic: `ci-cd`)

**Do not gate on**

- response_quality_score — it scores only the criteria you pass (expected keywords, max length) and returns total_score null without them; keyword presence is not correctness.
- prompt_test_suite — it writes a test plan and executes nothing.
- An uncalibrated LLM judge — noise with a confident tone.

**What leaves your machine**: Every IA-QA MCP tool, and @ia-qa/cli, runs on www.ia-qa.com: the outputs, contexts and traces you pass — and the provider key for BYOK tools — are sent there. Use synthetic or redacted data. @ia-qa/self-healing and @ia-qa/qa-discovery run locally instead.

## About

Created by **Jean-Christophe Jamet**, QA Engineering Manager and AI practitioner with 10+ years of experience in software quality, test automation, and AI tooling.

- **Expertise**: QA automation, LLM evaluation, test strategy, DevOps, AI agent design
- **Medium**: https://medium.com/@jean-christophe.jamet (articles on QA, AI, LLMs)
- **LinkedIn**: https://www.linkedin.com/in/jean-christophe-jamet-b043618a/

## Quick Facts

- The open testing workbench for the AI ecosystem
- Compare LLMs side by side (GPT-4o, Claude, Gemini, DeepSeek, Grok, Ollama)
- Test RAG pipelines, debug AI agents, explore MCP servers
- 152 tools across 12 categories
- Curated MCP Registry at https://www.ia-qa.com/mcp-registry — discover, evaluate and compare external MCP servers with QA scores
- No account. Browser-only tools keep your data in the tab; the MCP server, `@ia-qa/cli` and the server-side tools receive what you send them
- No account, no sign-up, no usage limits, free forever
- MCP (Model Context Protocol) API available at `POST https://www.ia-qa.com/mcp`
- Accessible from Cursor IDE, Claude Desktop, and Windsurf via MCP

## Who Is This For?

- **QA Engineers**: bug severity classification, test case generation, regression risk scoring, LLM output consistency checking, prompt injection testing
- **Developers**: JSON formatting, JWT debugging, API request building, hash generation, regex testing, SQL query building
- **AI / LLM Practitioners**: prompt engineering, token counting, RAG chunk optimization, agent blueprint design, LLM evaluation, context window visualization
- **Team Leads**: decision records, assumption surfacing, friction mapping, retrospective wheel, meeting timers
- **Web Auditors**: Lighthouse-based site audits, Core Web Vitals (LCP, INP, CLS), SEO and accessibility checks

## Key Differentiators

1. **All-in-one AI workbench**: only platform combining an LLM API playground, multi-model comparison, RAG testing, vector DB testing, agent debugging, and MCP exploration in one place
2. **Privacy stated per surface**: browser-only tools process data locally; the MCP server, the CLI and the server-side tools receive what you send them, including a bring-your-own-key provider key
- **Multi-model comparison**: compare GPT-4o, Claude 3.7, Gemini 2.0, DeepSeek, Grok 2 (xAI), and Ollama (local) side by side with latency, cost, and token metrics. Auto-filters available models based on configured API keys — users only see models they can actually call
4. **QA depth**: purpose-built QA workflows, prompt injection testing, LLM consistency checking, regression risk scoring
5. **MCP-integrated**: callable from Cursor IDE, Claude Desktop, Windsurf via Model Context Protocol
6. **GEO-ready**: LLM Readability Analyzer and LLM.txt Generator help optimize content for AI search engines (ChatGPT, Perplexity, Google AI Overviews)

## Frequently Asked Questions

**Q: How do I test an LLM, a RAG pipeline or an AI agent with IA-QA?**
A: Start from what you are testing, not from the catalogue: the "start here" section above maps each goal (correct answers, RAG grounding, stability, guardrails, agent tool calls, vision models, MCP servers, CI) to the tools to call and the output field to gate on. Run deterministic checks first, semantic ones next, and an LLM judge last and only once calibrated. From an MCP client, `get_testing_guidelines` returns the same map, and `generate_ci_workflow` turns it into a GitHub Actions gate.

**Q: Are all tools free?**
A: Yes. All 152 tools are permanently free with no usage limits, no credit card, no account needed.

**Q: Is user data safe?**
A: It depends on the surface, and each one says what it sends. The MCP server and `@ia-qa/cli` run every tool on ia-qa.com's server, so whatever you pass to a tool is sent there, including a provider API key given to a bring-your-own-key tool. `@ia-qa/self-healing` and `@ia-qa/qa-discovery` run on your machine and send nothing to ia-qa.com. Web tools depend on the tool: many run entirely in your browser and the LLM Sandbox calls providers directly from it, while LangChain and LangGraph runs, VLM tests and Anthropic calls made through `/mcp/llm-proxy` send your key through ia-qa.com's server.

**Q: What is GEO — Generative Engine Optimization?**
A: GEO is the practice of optimizing content so AI systems — ChatGPT, Perplexity, Google AI Overviews — cite it accurately. Key techniques include structured llm.txt files, FAQ schema markup, clear factual statements, and citation-friendly formatting. IA-QA offers two direct GEO tools: LLM Readability Analyzer (https://www.ia-qa.com/llm-readability) and LLM.txt Generator (https://www.ia-qa.com/llm-txt-generator).

**Q: What is the LLM Readability Analyzer?**
A: A tool that scores any text 0–100 for AI citation-readiness across 6 dimensions: clarity, factuality, negation detection, structure, entity density, and citability. Useful for content creators optimizing for AI search (GEO).

**Q: Can AI agents call IA-QA tools programmatically?**
A: Yes. IA-QA exposes a public Model Context Protocol (MCP) server at https://www.ia-qa.com/mcp — no API key, no sign-up, no rate limit for localhost (60 req/min per IP otherwise). 152 tools across 12 categories. Callable via JSON-RPC 2.0 from Cursor IDE, Claude Desktop, Windsurf, or any MCP-compatible client. See the MCP API section below for the full tool listing by category.

**Q: What does Site Auditor 2026 check?**
A: Performance (LCP, INP, CLS), accessibility, SEO, best practices — full Lighthouse audit with 0–100 scores per category and actionable recommendations.

**Q: Is there a QA and AI news feed?**
A: Yes. https://www.ia-qa.com/veille aggregates RSS feeds from Google Testing Blog, DEV.to (QA/AI/LLM/Agents topics), Hugging Face Blog, and Simon Willison's blog. Updated daily.

**Q: Who built IA-QA?**
A: Jean-Christophe Jamet, QA Engineering Manager based in France. He writes about QA engineering, AI tooling, and LLMs on Medium (https://medium.com/@jean-christophe.jamet).

## MCP API — Programmatic Access from Any AI Agent

IA-QA exposes a public MCP (Model Context Protocol) server at `POST https://www.ia-qa.com/mcp`. No authentication required. 152 tools organized in 12 categories. Use it from Claude Desktop, Cursor IDE, Windsurf, or any MCP-compatible client.

### Install — instructions for AI agents

A user can paste this into any agent. If you are an agent asked to set up IA-QA, follow it as written.

```text
Set up IA-QA (https://www.ia-qa.com) for me.

IA-QA is newer than your training data. Do not guess a command, flag, file path or config key from memory: use only the lines below, exactly as written.

STEP 1. Explain, then ask. Install nothing yet.
Tell me in a few lines what these five are and what each one sends off my machine. Then ask which ones I want, and install only those.

1. IA-QA MCP server, hosted at https://www.ia-qa.com/mcp: QA and LLM testing tools that return a verdict: JSON and schema checks, token and cost estimates, prompt-injection, PII and secret scans, eval contracts, CI workflow generation. No account, no key.
   Leaves my machine: every call runs on ia-qa.com, so whatever is passed to a tool is sent there, including a provider API key given to a bring-your-own-key tool.
2. @ia-qa/cli: the same tools from a shell.
   Leaves my machine: the same as the MCP server: it is a thin client of it.
3. @ia-qa/self-healing: for an existing Playwright, Cypress or Selenium suite: maps the app's pages, diffs them over time (PASS / FIX / BLOCK) and repairs broken locators.
   Leaves my machine: nothing is sent to ia-qa.com; it drives a local headless browser against the app under test. Its optional ia-qa-heal-ai add-on sends the rows it could not resolve to my own LLM provider.
4. @ia-qa/qa-discovery: for an app with no test suite yet: crawls it and records its pages, forms and API calls. It returns no verdict.
   Leaves my machine: nothing is sent to ia-qa.com; it only talks to the app it crawls. Its optional ia-qa-discover-ai add-on sends the capture to my own LLM provider and prints what it sends first.
5. @ia-qa/pal: a QA app next to my team, opened from an icon on my desktop: it walks the app, maps every page, checks it and compares it with the last tour, then reports what broke, what needs a person and what it could not see. With my end-to-end suite declared, it proposes repairs to its selectors (never applies them) and, when asked, runs the suite once to name the pages it never visits. It is built on @ia-qa/self-healing and returns the same verdicts. Installing it installs @ia-qa/self-healing and @ia-qa/qa-discovery too, and its MCP server (ia-qa-pal) also answers every tool of theirs — pick it and the other two are included.
   Leaves my machine: nothing is sent to ia-qa.com; it drives a local headless browser against the app under test only, and runs my own test command only when I ask for it. Its tour calls no LLM; the discovery tools its MCP server includes (classify_app, plan_pages) send the capture to my own LLM provider, only when called.

STEP 2. Find the client you are running in and use its lines only. Never mix two clients' formats. In a "merge" line, keep only the servers I picked.

Claude Code
  hosted:  claude mcp add --transport http ia-qa https://www.ia-qa.com/mcp
  local:   claude mcp add ia-qa-heal -- npx -y -p @ia-qa/self-healing ia-qa-heal-mcp
  local:   claude mcp add ia-qa-discover -- npx -y -p @ia-qa/qa-discovery ia-qa-discover-mcp
  local:   claude mcp add ia-qa-pal -- npx -y -p @ia-qa/pal ia-qa-pal-mcp
  verify:  claude mcp get ia-qa (it must print "Connected")

Codex CLI
  hosted:  codex mcp add ia-qa --url https://www.ia-qa.com/mcp
  local:   codex mcp add ia-qa-heal -- npx -y -p @ia-qa/self-healing ia-qa-heal-mcp
  local:   codex mcp add ia-qa-discover -- npx -y -p @ia-qa/qa-discovery ia-qa-discover-mcp
  local:   codex mcp add ia-qa-pal -- npx -y -p @ia-qa/pal ia-qa-pal-mcp
  verify:  codex mcp list shows it; only a tool call in a new session proves it connects

Gemini CLI
  hosted:  gemini mcp add -t http ia-qa https://www.ia-qa.com/mcp
  local:   gemini mcp add ia-qa-heal npx -- -y -p @ia-qa/self-healing ia-qa-heal-mcp
  local:   gemini mcp add ia-qa-discover npx -- -y -p @ia-qa/qa-discovery ia-qa-discover-mcp
  local:   gemini mcp add ia-qa-pal npx -- -y -p @ia-qa/pal ia-qa-pal-mcp
  verify:  gemini mcp list (it must print "Connected"; servers stay disabled in an untrusted folder)

Claude Desktop
  hosted:  not in a file. Settings → Connectors → Add custom connector, URL https://www.ia-qa.com/mcp. Tell me to do it; you cannot.
  file:    ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows)
  merge:   {"mcpServers":{"ia-qa-heal":{"command":"npx","args":["-y","-p","@ia-qa/self-healing","ia-qa-heal-mcp"]},"ia-qa-discover":{"command":"npx","args":["-y","-p","@ia-qa/qa-discovery","ia-qa-discover-mcp"]},"ia-qa-pal":{"command":"npx","args":["-y","-p","@ia-qa/pal","ia-qa-pal-mcp"]}}}
  verify:  fully quit and relaunch it, then ask it to list its MCP tools

Cursor
  file:    .cursor/mcp.json (this project) or ~/.cursor/mcp.json (all projects)
  merge:   {"mcpServers":{"ia-qa":{"url":"https://www.ia-qa.com/mcp"},"ia-qa-heal":{"command":"npx","args":["-y","-p","@ia-qa/self-healing","ia-qa-heal-mcp"]},"ia-qa-discover":{"command":"npx","args":["-y","-p","@ia-qa/qa-discovery","ia-qa-discover-mcp"]},"ia-qa-pal":{"command":"npx","args":["-y","-p","@ia-qa/pal","ia-qa-pal-mcp"]}}}
  verify:  restart it, then ask it to list its MCP tools

VS Code (GitHub Copilot, agent mode)
  file:    .vscode/mcp.json
  merge:   {"servers":{"ia-qa":{"type":"http","url":"https://www.ia-qa.com/mcp"},"ia-qa-heal":{"command":"npx","args":["-y","-p","@ia-qa/self-healing","ia-qa-heal-mcp"]},"ia-qa-discover":{"command":"npx","args":["-y","-p","@ia-qa/qa-discovery","ia-qa-discover-mcp"]},"ia-qa-pal":{"command":"npx","args":["-y","-p","@ia-qa/pal","ia-qa-pal-mcp"]}}}
  verify:  restart it, then ask it to list its MCP tools

Windsurf
  file:    ~/.codeium/windsurf/mcp_config.json
  merge:   {"mcpServers":{"ia-qa":{"serverUrl":"https://www.ia-qa.com/mcp"},"ia-qa-heal":{"command":"npx","args":["-y","-p","@ia-qa/self-healing","ia-qa-heal-mcp"]},"ia-qa-discover":{"command":"npx","args":["-y","-p","@ia-qa/qa-discovery","ia-qa-discover-mcp"]},"ia-qa-pal":{"command":"npx","args":["-y","-p","@ia-qa/pal","ia-qa-pal-mcp"]}}}
  verify:  restart it, then ask it to list its MCP tools

Any other MCP client: the hosted server is Streamable HTTP at https://www.ia-qa.com/mcp; each local server is stdio, with the command and args shown above.
A chat with no shell and no file access: explain, give me the lines for my client, and do not say you installed anything.

Rules for every client:
- Keep "-p" in every npx line. Each package ships several binaries; without -p, npx stops with "could not determine executable to run".
- Use https://www.ia-qa.com, with "www". The bare domain answers with a 301 redirect, and a POST that follows a 301 can arrive as a GET without its body.
- Merge into an existing config file. Never remove a server that is already there.
- If I picked @ia-qa/pal, add only its local server (ia-qa-pal): it already answers the ia-qa-heal and ia-qa-discover tools. Add those two servers only if I picked them without pal.
- A client does not load a server added during the conversation: tell me to restart it before you test.

STEP 3. Shell tools, only if I picked them. Nothing to install:
- @ia-qa/cli: npx -y @ia-qa/cli list, then npx -y @ia-qa/cli <tool> --help before each tool you run.
- @ia-qa/self-healing: npx -y -p @ia-qa/self-healing ia-qa-heal skill --print
- @ia-qa/qa-discovery: npx -y -p @ia-qa/qa-discovery ia-qa-discover skill --print
- @ia-qa/pal: npx -y -p @ia-qa/pal ia-qa-pal skill --print
Each prints that package's agent instructions on stdout and installs nothing. Read the whole output before running any other command from that package: it carries the order the verbs go in, what each refusal means, and what you must never run. Do not skip it and work from --help — the flags are there, the reasons are not.
Add "skill --install" instead of "--print" to drop the same file into .claude/skills/, so your next session has it without asking.

STEP 3a. The QA app, only if I picked @ia-qa/pal. It is an application on my desktop, not only a command:
- Which folder: the one holding my end-to-end tests, because pal reads them and proposes repairs to them. If my tests live in their own repository, that repository; if I have none yet, my app's folder. Ask me when it is not obvious.
- In that folder: npm i -D @ia-qa/pal. It brings @ia-qa/self-healing and @ia-qa/qa-discovery with it, and it stays there, unlike npx's cache, so the icon below keeps working. If the folder has no package.json, ask me before creating one with npm init -y.
- Then: npx -y -p @ia-qa/pal ia-qa-pal ui --shortcut. It puts an icon named "ia-qa-pal — <my project>" on my desktop and returns at once; tell me where it is.
- Then tell me the app is ready: I double-click the icon (or type npx ia-qa-pal in that folder), the page asks where my app lives and how it logs in, and starts the first tour from a button. Do not run that bare command yourself: without a terminal it only prints the help. If I would rather you do it for me, go on with STEP 3b.
- If any command says "No usable browser found", ask me before running npx playwright install chromium: it downloads about 150 MB from Playwright's servers.

STEP 3b. Set my project up. Ask me, never guess:
- Where my app lives (the URL I would type in a browser).
- How it logs in: none, by hand in a browser, or a storageState file my suite already writes (then ask for its path).
- Where my end-to-end tests are, if I have any: a folder inside the project. Skip it rather than guess a folder.
Then, with my answers: npx -y -p @ia-qa/pal ia-qa-pal setup --url <my app> --login <my answer> --yes (add --tests <folder> when I have tests). Then npx -y -p @ia-qa/pal ia-qa-pal tour --background and follow it with npx -y -p @ia-qa/pal ia-qa-pal status. Report its "Not seen" section as well as its verdict: a first tour is a baseline, never a pass. Never apply the repairs it proposes without my review, and never add --suite (it runs my whole test suite) without asking me. Through MCP the same steps are pal_project, pal_setup, pal_tour and pal_status.
If I chose "by hand in a browser", give me the login command to run myself and wait. You cannot log in for me.

STEP 3c. An LLM provider key, ONLY if I ask for the optional AI layers (ia-qa-heal-ai, ia-qa-discover-ai). Everything else works without one.
- Ask me which provider I have a key for: anthropic, openai, google, or openai-compatible. That last one is a shape, not a vendor: DeepSeek, Groq, Mistral, OpenRouter, Together, vLLM and Ollama all serve the same endpoint, and it needs a "baseUrl" in the config. A localhost baseUrl means the call never leaves my machine. Never pick one for me.
- Preferred: tell me to store it in my machine's credential store, which keeps it out of the environment every process you launch inherits. I run it myself, in my own terminal — it asks for the value and refuses a pipe or an argument: npx -y -p @ia-qa/self-healing ia-qa-heal secret set <MY_NAME>. Then the reference is {"source": "keychain", "key": "<MY_NAME>"}. You can confirm it resolves without ever seeing it: npx -y -p @ia-qa/self-healing ia-qa-heal secret check <MY_NAME>. There is no command that prints a stored value, and you must not look for one.
- Otherwise, or in CI where no store exists: ask me to export the key in my own shell and tell you only the VARIABLE NAME, then use {"source": "env", "key": "<MY_VARIABLE_NAME>"}. Do not ask me to paste the key, and if I paste it anyway, tell me to rotate it.
- Either way, write only the reference into the config, never the value: {"ai": {"provider": "<mine>", "model": "<a model I pinned>", "apiKey": {"source": "<env or keychain>", "key": "<MY_NAME>"}}}
- These CLIs read the environment only and never load .env files. If my key lives in .env, prefix the command with npx dotenv-cli --
- Run the verb with --dry-run first and show me exactly what it would send, and to whom, before any real call.
A key in a config file is a leaked credential: the file is committed. The CLI refuses a key-shaped value there, and that refusal is not something to work around.

STEP 4. Prove it, do not claim it.
Run the verify line for my client, or call the find_tool tool with {"query": "json"} and show me what it returns. For the CLI, run npx -y @ia-qa/cli generate_uuid and show the output.
Then give me two lists: what you checked by running it, and what you could not check, with the reason. Nothing in the second list counts as installed.

STEP 5. Use it on my project.
Once a tool call works, call get_testing_guidelines with no topic (from a shell: npx -y @ia-qa/cli get_testing_guidelines). It maps what is being tested to the tools to call and the output field to gate CI on.
Ask me what I am testing. Propose the tools for it and the output field each verdict is read from, then run them on my real outputs.
Before each call, tell me in one line what you are about to send to ia-qa.com. Use synthetic or redacted data unless I say otherwise.
Report each verdict as the tool returned it. Never state a verdict a tool did not give.
If my client needs a restart before the server loads, give me one sentence to paste in the new session to resume at this step.

Never run these yourself: ia-qa-heal login, ia-qa-heal ui, ia-qa-discover login, ia-qa-pal ui (ia-qa-pal ui --shortcut is the exception: it only writes the desktop icon and returns). Each one waits for a person at a browser; give me the command instead.
Never write an API key into a config file or into any file that gets committed.
```

### CLI — Shell Access for Terminal Agents

For AI agents with shell access (Claude Code, Copilot CLI) or humans scripting a pipeline, the same 152 tools are available as a zero-dependency CLI published on npm as `@ia-qa/cli` (Node >= 18, no signup, no API key). It discovers tools from the public manifest and runs them on the hosted server.

- **Install**: `npx @ia-qa/cli <tool>` or `npm install -g @ia-qa/cli` or download `https://www.ia-qa.com/downloads/ia-qa-cli.zip` and run `node ia-qa.js <tool>`
- **Discover**: `ia-qa --help` · `ia-qa list [--category <c>]` · `ia-qa <tool> --help`
- **Run**: `ia-qa generate_uuid --count 3` · `ia-qa base64_encode "hello"` · `echo '{"a":1}' | ia-qa format_json --stdin`
- **Output**: `--json` for machine-parsable JSON, `--out <file>` to write to a file (large outputs auto-write to a temp file)
- **Multiline values**: `--<param>-file <path>` reads any param from a file (e.g. `--inputs-file fields.txt`) — multiline-safe; prefer it under `npx` on Windows, where args are truncated at the first newline
- **Server**: `--server <url>` / `$IAQA_SERVER` to target a self-hosted ia-qa server (default `https://www.ia-qa.com`)

### Tool Categories

#### 📊 Data Tools
- `format_json` — Validate and pretty-print a string that is ALREADY valid JSON
- `parse_csv` — Parse a CSV string into a JSON array of objects (or raw arrays)
- `flatten_json` — Flatten a nested JSON object to single-level dot-notation keys (e.g. a:b:1 → a.b:1), or unflatten dot-notation keys back to a nested object
- `xml_to_json` — Convert an XML string to a JSON object
- `transform_json_array` — Transform a JSON array using common operations: pluck (extract specific fields), filter (by field value), sort_by (field), group_by (field), count_by (field), uniq_by (field)
- `json_to_csv` — Convert a JSON array of objects to CSV format
- `json_diff` — Compute a deep structural diff between two JSON values
- `merge_json` — Deep merge two JSON objects
- `json_to_yaml` — Convert a JSON object to clean, human-readable YAML
- `json_schema_validate` — Validate a JSON value against a JSON Schema (draft-07 subset)
- `mock_from_schema` — Generate realistic mock data from a JSON Schema
- `extract_json_path` — Extract a value from a JSON string using dot-notation path (e.g., user.address.city, items.0.name, meta.tags)
- `generate_json_ld` — Generate a ready-to-paste <script type=application/ld+json> snippet for GEO / structured data optimization
- `yaml_to_json` — Parse a YAML string and return the equivalent JSON value
- `json_schema_generate` — Infer a JSON Schema (draft-07) from a sample JSON value
- `format_table` — Convert a JSON array of objects into a Markdown table

#### 🔐 Encoding & Security Tools
- `base64_encode` — Encode a UTF-8 string to Base64
- `base64_decode` — Decode a Base64 string back to UTF-8 text
- `url_encode` — Percent-encode a string for safe use in URLs
- `url_decode` — Decode a percent-encoded URL string back to plain text
- `hash_text` — Compute a cryptographic hash of a text string
- `generate_hmac` — Compute an HMAC signature for a message using a secret key
- `generate_uuid` — Generate one or more cryptographically random UUID v4 identifiers
- `generate_password` — Generate a cryptographically secure random password using crypto.randomBytes
- `number_base_convert` — Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36
- `color_convert` — Convert a color between HEX, RGB, and HSL formats

#### 📝 Text & NLP Tools
- `text_stats` — Compute comprehensive statistics for any text: character count (with and without spaces), word count, line count, sentence count, paragraph count, and estimated reading time in min
- `strip_markdown` — Strip all Markdown formatting (headers, bold, italic, code fences, links, lists) from text and return clean plain text
- `normalize_whitespace` — Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces
- `sort_lines` — Sort, deduplicate, reverse, or filter lines of text
- `word_frequency` — Analyze word frequency in text
- `extract_links` — Extract all URLs, email addresses, and domain names from text
- `levenshtein_distance` — Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings
- `case_convert` — Convert a string between naming conventions: camelCase, PascalCase, snake_case, kebab-case, UPPER_SNAKE_CASE, dot.case, Title Case
- `lorem_ipsum` — Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population
- `detect_language` — Detect the natural language of a text using n-gram frequency analysis and common word markers
- `escape_html` — Escape HTML special characters (&, <, >, , ') to their safe HTML entities
- `unescape_html` — Convert HTML entities (&amp;, &lt;, &gt;, &quot;, &#x27;, and numeric &#NNN;) back to plain characters
- `html_to_markdown` — Convert HTML to clean Markdown
- `regex_test` — Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups
- `calculate_readability` — Calculate readability scores: Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Automated Readability Index

#### 🤖 LLM Tools
- `count_tokens` — Estimate the token count of a text string using the cl100k_base approximation (~4 chars/token)
- `truncate_to_tokens` — Truncate text to at most N tokens (cl100k_base: ~4 chars/token) to avoid exceeding an LLM context window
- `split_chunks` — Split text into chunks of at most N tokens (cl100k_base: ~4 chars/token) with optional overlap
- `extract_json_from_text` — Extract the first valid JSON object or array embedded in chaotic LLM output (surrounded by markdown fences, prose, or explanatory text)
- `estimate_llm_cost` — Estimate the API cost in USD for a given model and token counts
- `token_budget_calculator` — Plan token allocation across system prompt, user input, context/RAG chunks, and expected output
- `context_window_check` — Given an array of message objects [role, content], estimate total token usage and check if it fits in the target model's context window
- `model_info` — Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list
- `compare_models` — Compare 2-5 AI models side by side: context window, pricing, multimodal, reasoning capabilities, and provider
- `list_llm_models` — List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities
- `llm_generate` — Generate text using open-source LLM models hosted on Groq (ultra-fast) or HuggingFace Inference (serverless)
- `optimize_prompt_tokens` — Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace
- `llm_fit_finder` — Find the best LLM for a given use case

#### ✅ QA & Evaluation Tools
- `diff_text` — Compute a unified line-by-line diff between two text strings (LCS algorithm)
- `llm_output_validator` — Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safe
- `compare_responses` — Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side
- `analyze_responses` — Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox)
- `prompt_test_suite` — Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria
- `validate_mcp_response` — Validate that an MCP tool response conforms to expected format, schema, and content rules
- `mcp_server_health_check` — Generate a health check report for an MCP server's tool manifest
- `mcp_server_evaluate` — Run a full compliance evaluation against a live MCP server URL
- `hallucination_check` — Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context
- `prompt_injection_scan` — Scan user input or prompts for common prompt injection patterns
- `consistency_check` — Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison)
- `llm_format_check` — Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex
- `llm_json_schema_check` — Validate that an LLM JSON output matches a JSON Schema definition
- `response_quality_score` — Score an LLM response against the criteria you pass: coverage of expected_keywords and compliance with max_length
- `toxicity_scan` — Scan text for toxic language, hate speech, bias/stereotype framing, violence, sexual and self-harm content
- `guardrail_test` — Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex
- `function_call_validate` — Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema, no extra/missing args
- `conversation_analyze` — Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition
- `mcp_schema_lint` — Lint an MCP tool definition for best practices: naming conventions, description quality, schema completeness, required fields consistency, description length
- `cot_analyzer` — Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM
- `ab_test_report` — Generate an A/B test report comparing two prompts or model configurations
- `analyze_diff_bugs` — Pattern-based diff linter: flags a fixed set of risky shapes in changed code — query-string interpolation (SQL/Cypher/Mongo injection shape), shell interpolation, eval/new Function
- `generate_test_cases` — Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description
- `run_pr_gate_pipeline` — Review triage for a pull request
- `diff_mappings` — Diff a baseline page mapping against a current one and return a CI-style verdict: PASS / FIX / BLOCK, plus per-element drift (ok, renamed, healable, ambiguous, lost, added, rebound
- `sandbox_scenario` — Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end
- `run_vlm_test_suite` — Run a test suite against a Vision-Language Model (VLM) — send an image (URL or base64) + N test cases (each with a question + assertion) to GPT-4o, Claude 3.5, or Gemini
- `run_vlm_test_suite_batch` — Compare multiple VLMs on the same test suite in parallel — send an image (URL or base64) + N test cases to all models simultaneously
- `multimodal_eval_guide` — Unified tool for multimodal AI evaluation: set action=guide for reference thresholds/interpretation (CLIP, FID, VQA), or set action=clip_score / fid_score / vqa_accuracy / pipeline
- `similarity_score` — Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU
- `bias_detect` — Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone)

#### 🔍 RAG & Embedding Tools
- `embedding_similarity` — Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams)
- `vector_similarity` — Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance
- `normalize_vector` — L2-normalize a float vector (produce a unit vector with norm=1)
- `vector_quantize` — Simulate int8 or int4 quantization of float32 embedding vectors
- `vector_stats` — Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices
- `bm25_score` — Compute BM25 relevance score between a query and one or more documents
- `build_rag_prompt` — Assemble a complete RAG (Retrieval-Augmented Generation) prompt from retrieved context chunks and a user query
- `prompt_template_fill` — Fill a prompt template with variables
- `few_shot_formatter` — Format few-shot examples for LLM prompts
- `system_prompt_builder` — Build a structured system prompt from components: role, task, constraints, output format, tone, language, and examples
- `rag_relevance_rank` — Rank an array of text chunks by relevance to a query using TF-IDF scoring
- `rerank_evaluate` — Evaluate RAG retrieval quality: rank passages against a query and compute Precision@k / Recall@k plus a PASS/FAIL CI verdict from ground-truth relevance labels

#### 🛠 Dev Tools
- `validate_email` — Validate an email address against RFC 5322 syntax before storing it, sending a transactional email, or adding it to a mailing list
- `validate_url` — Parse and validate a URL
- `decode_jwt` — Decode a JWT (JSON Web Token) and return its header and payload without verifying the signature
- `minify_js` — Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser
- `cron_parse` — Parse a cron expression into a human-readable schedule description
- `cron_validator` — Validate a 5-field cron expression, explain the schedule, and preview the next execution times
- `timestamp_convert` — Convert between Unix timestamps (seconds or milliseconds) and ISO-8601 / UTC date strings
- `http_status_lookup` — Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices
- `parse_http_headers` — Parse a raw HTTP headers block into a structured JSON object
- `generate_curl` — Generate a curl command from request parameters
- `extract_todos` — Extract TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE, and custom tags from any source code or text
- `detect_secrets` — Scan code or config files for hardcoded secrets: AWS keys, GitHub tokens, OpenAI/Anthropic API keys, Stripe secrets, JWTs, database connection strings, and generic passwords
- `count_code_lines` — Count lines of code: total, code lines, comment lines, blank lines, and comment density
- `lint_commit_message` — Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build)
- `format_bytes` — Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes
- `pr_gatekeeper` — Compound quality gate for pull requests
- `check_contrast_ratio` — Calculate WCAG 2.1 contrast ratio between two colors
- `webhook_endpoint_create` — Create a temporary webhook endpoint that captures incoming HTTP requests for one hour
- `webhook_endpoint_requests` — Fetch the requests captured by a webhook created with webhook_endpoint_create
- `env_parse` — Parse a .env file content into a JSON object
- `openapi_validate` — Validate the structure of an OpenAPI 3.x specification (JSON or YAML)

#### 🛡 Security Tools
- `security_headers_check` — Analyse the HTTP security headers of a public URL OR of raw response headers you paste in
- `ssl_certificate_check` — Analyse the SSL/TLS certificate of any HTTPS host
- `cors_test` — Test a URL for CORS misconfigurations
- `cors_checker` — Check the CORS configuration of a URL the same way a browser would
- `cookie_security_audit` — Audit the security attributes of cookies set by any URL
- `web_security_audit` — Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan
- `shield_analyze` — Run a comprehensive AI guardrail analysis on an LLM response
- `secret_scan` — Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), private keys (RSA/EC/PGP), JWTs, 
- `redact_pii` — Automatically detect and redact Personally Identifiable Information (PII) from text

#### 🌐 Web Tools
- `fetch_veille_feed` — Fetch the latest QA & AI/LLM articles aggregated from curated RSS sources (Google Testing Blog, DEV.to Testing/QA/AI/LLM/Agents, Hugging Face Blog, Simon Willison)
- `score_geo_signals` — Analyze a webpage <head> HTML (or full HTML) for GEO (Generative Engine Optimization) signals
- `latency_benchmark` — Measure response time of one or more HTTP endpoints (GET/POST)
- `generate_slug` — Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens

#### 🔎 Meta Tools
- `find_tool` — Search available MCP tools by keyword or category before calling them
- `identify_caller` — Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request
- `rate_tool` — Give honest usage feedback on an IA-QA MCP tool

#### 📋 Eval Framework Tools
- `list_local_tests` — Discover .ia-eval.yaml LLM test suite files in the project directory
- `run_eval_contract` — Parse a .ia-eval.yaml LLM test suite, call the specified LLM model for each scenario, run all configured scorers, and return a structured JSON report with per-scenario Pass/Fail ve
- `generate_html_report` — Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, 
- `run_semantic_tests` — Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L
- `generate_eval_yaml` — Utility tool
- `validate_agent_trajectory` — Run declarative assertions on an agent trace (OpenAI tool-call messages, Anthropic tool_use/tool_result blocks, LangChain run trees, or plain text ReAct logs)
- `metamorphic_check` — Reference-free stability primitive: instead of comparing an answer to a ground truth, it checks that an assistant's answer stays INVARIANT when the QUESTION is transformed (typo, c
- `test_skill` — Validate a SKILL.md definition (Cursor / GitHub Copilot / Windsurf) by auto-generating trigger-positive and trigger-negative scenarios, running each through the model with the skil
- `get_testing_guidelines` — Query the IA-QA methodology knowledge base
- `generate_ci_workflow` — Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA
- `needle_haystack_generate` — Generate a needle in a haystack test: embeds a target fact into a large block of filler text at a specified position

#### 🔗 Integrations Tools
- `fetch_jira_issue` — Fetch a complete Jira issue: summary, description converted to Markdown, status, assignee, priority, labels, custom fields, and optionally comments and attachment metadata
- `search_jira_issues` — Search Jira using JQL (Jira Query Language)
- `fetch_confluence_page` — Fetch a Confluence page and return its content as clean Markdown
- `jira_to_test_suite` — Transform a Jira ticket into a complete test suite: Gherkin scenarios, E2E steps, API test cases, test data matrix, and ambiguity detection
- `fix_gherkin` — Fix Gherkin syntax warnings from a jira_to_test_suite result
- `post_jira_comment` — Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket
- `create_confluence_page` — Create a new Confluence page from the output of jira_to_test_suite

### API Details

- **Endpoint**: `POST https://www.ia-qa.com/mcp` (JSON-RPC 2.0)
- **Discovery**: `GET https://www.ia-qa.com/mcp` returns full tool manifest
- **Direct call**: `POST https://www.ia-qa.com/mcp/call` with `{ "name": "tool_name", "parameters": {...} }`
- **Single tool info**: `GET https://www.ia-qa.com/mcp/tool/:name`
- **Rate limit**: 60 req/min per IP (unlimited for localhost)
- **Auth**: None required (public)
- **Models**: For LLM tools, provide your own API key (OpenAI, Anthropic, Google, DeepSeek, Groq)

## AI & LLM Tools

- [LLM Sandbox](https://www.ia-qa.com/llm-sandbox): Compare LLMs side by side — send the same prompt to GPT-4o, Claude, Gemini, DeepSeek, Grok (xAI), and Ollama (local). See latency, cost, token counts in real time. Supports streaming (SSE), multi-turn chat mode, and function/tool calling. Models are auto-filtered to only show providers with configured API keys. One prompt, multiple models, instant comparison. Calls each provider directly from your browser: your API key goes to the provider, not to ia-qa.com.
- [Prompt Test Suite Runner](https://www.ia-qa.com/devtools/prompt-test-runner): Batch-execute test cases against multiple LLMs with assertions (contains, regex, JSON validation, latency/cost limits). Get pass/fail reports per model. Import/export Promptfoo-compatible configs. Batch-run your prompt test suites across LLMs.
- [RAG Pipeline Tester](https://www.ia-qa.com/devtools/rag-pipeline-tester): Visual RAG debugger — paste a document, chunk it (4 strategies), embed chunks (TF-IDF, MiniLM local, or OpenAI), query, see ranked retrieval results with similarity scores, then compare RAG vs no-context LLM answers side by side. Full pipeline in one tool.
- [Prompt Stability Analyzer](https://www.ia-qa.com/devtools/prompt-stability): Run the same prompt N times (3-20×), measure output variance with Jaccard similarity, length distribution, structural consistency. Get a 0-100 stability score and production-readiness verdict. Compare stability across models. Export reports. Catch unstable prompts before they hit production.
- [Prompt A/B Tester](https://www.ia-qa.com/devtools/prompt-ab-tester): Execute two prompt variants side by side on the same LLM(s). Compare latency, output tokens, response length. Multi-model mode ranks which variant wins across GPT-4o, Claude, Gemini. 4 preset experiments (concise vs detailed, JSON vs Markdown, persona vs none, low vs high temp). Export comparison reports.
- [Experiment Tracker](https://www.ia-qa.com/devtools/experiment-tracker): Central dashboard for all your AI testing runs. Every LLM comparison, A/B test, stability analysis, test suite, and RAG pipeline run is auto-saved. View timeline, filter by tool or model, compare any two runs side-by-side with metrics diff. Export history as JSON or CSV. Track progress, catch regressions, never lose a result.
- [CI/CD Export](https://www.ia-qa.com/devtools/cicd-export): Export LLM test suites as GitHub Actions YAML, Promptfoo configs, Node.js scripts, or Bash/cURL scripts. Configure models, assertions, cron schedules. Import test cases from experiment history. Automate prompt testing in your CI/CD pipeline.
- [Collections](https://www.ia-qa.com/devtools/collections): Organize prompts, test cases, configs, and notes into reusable collections. Export/import as JSON. Build a personal prompt library and share test suites across projects.
- [Eval Framework](https://www.ia-qa.com/devtools/eval-framework): Multi-criteria LLM output scoring with custom rubrics. 5 preset rubric templates (General Quality, Safety & Compliance, Structured Output, Creative Writing, Code Generation). Weighted scoring, side-by-side comparison, exportable evaluation reports.
- [Get Started Wizard](https://www.ia-qa.com/devtools/onboarding): Answer 4 quick questions about your role, goals, and models — get personalized tool recommendations. Discover the right AI testing tools for your workflow.
- [Pipeline Builder](https://www.ia-qa.com/devtools/pipeline-builder): Visual drag-and-drop pipeline builder to chain AI testing tools. 4 preset pipelines (Prompt→Test→Deploy, RAG Debug, Security Audit, Agent Validation). Features: zoom in/out (mouse wheel or buttons), pan the canvas (drag or arrow buttons), Fit-to-view button to auto-center all nodes, export pipeline as SVG file. Save custom pipelines, connect nodes with arrow edges, run tool sequences.
- [LLM Readability Analyzer](https://www.ia-qa.com/llm-readability): Score content 0-100 for AI citation-readiness across 6 dimensions (clarity, factuality, negations, structure, entities, citability)
- [LLM.txt Generator](https://www.ia-qa.com/llm-txt-generator): Generate a standards-compliant llm.txt file for any website following the llmstxt.org specification
- [GEO Optimizer](https://www.ia-qa.com/geo-optimizer): Generative Engine Optimization toolkit — 4 tools in one: (1) GEO Score: paste your <head> HTML, get a weighted 0-60 score with pass/fail for JSON-LD, FAQPage schema, lang="en", llm.txt link, canonical, Open Graph, and author attribution; (2) JSON-LD Builder: generate Schema.org structured data (WebSite, FAQPage, Article, SoftwareApplication, Person, Organization, HowTo); (3) Meta Tags Generator: generate a complete GEO-optimized <head> snippet with OG, Twitter Card, canonical, and llm.txt links; (4) robots.txt Auditor: check your robots.txt against 16 AI crawlers and get ready-to-paste fix snippets. 100% client-side.
- [LLM Cost Calculator](https://www.ia-qa.com/devtools/cost-calculator): Estimate API costs across OpenAI, Anthropic, Gemini, Mistral and more
- [Token Counter](https://www.ia-qa.com/devtools/token-counter): Count tokens for any LLM context window
- [Prompt Builder](https://www.ia-qa.com/devtools/prompt-builder): Structure system and user prompts with role, context, format and constraints sections
- [Prompt Library](https://www.ia-qa.com/devtools/prompt-library): Curated prompt templates for ChatGPT, Claude and Gemini
- [Prompt Engineering Toolsheets](https://www.ia-qa.com/devtools/prompt-toolsheets): Best practices guides and persona frameworks for prompt engineering
- [Output Risk Analyzer](https://www.ia-qa.com/devtools/risk-analyzer): Detect hallucination signals and risk patterns in LLM outputs
- [Agent Loop Detector](https://www.ia-qa.com/devtools/loop-detector): Identify infinite-loop risks in AI agent configurations
- [Agent Graph Generator](https://www.ia-qa.com/devtools/graph): Visualise LLM agent configurations as node-link diagrams
- [Explain This X](https://www.ia-qa.com/devtools/explain): Get plain-language explanations of JSON, regex, JWT, SQL and more
- [AI Recipes](https://www.ia-qa.com/devtools/ai-recipes): Code patterns and cookbook recipes for RAG, chain-of-thought, function calling, few-shot, and more
- [RAG Chunk Optimizer](https://www.ia-qa.com/devtools/rag-chunk-optimizer): Visualize and compare fixed, sentence, paragraph, and sliding-window chunking strategies with token counts
- [Agent Blueprint Designer](https://www.ia-qa.com/devtools/agent-blueprint): Design AI agent configurations (model, memory, tools, guardrails) and export JSON/YAML specs
- [LLM Eval Designer](https://www.ia-qa.com/devtools/llm-eval-designer): Design evaluation suites for LLM outputs with Promptfoo and Braintrust-compatible JSON export
- [MCP Registry](https://www.ia-qa.com/mcp-registry): Curated catalogue of 21 external MCP servers — GitHub MCP, Firecrawl, Brave Search, Filesystem, Git, Memory, Fetch, Time, Sequential Thinking, Figma (13.9k★), Notion (official), Stripe (official), 21st.dev, MiniMax, Obsidian, Shopify, Slack, Postgres, Puppeteer, Linear, Google Workspace — with QA scores (0–100), compatibility info for Cursor/Claude/Windsurf, pricing model, setup complexity, and one-click testing via IA-QA's MCP Server Tester. Filter by 12 categories (code, search, data, browser, files, productivity, design, AI & media, utility, testing, devops). Sort by QA score. Each server has a dedicated detail page with setup guide, install command, required env vars, and docs link.
- [Natural Language → Code](https://www.ia-qa.com/devtools/nl-to-code): Convert plain-English descriptions into clean, idiomatic code in 30+ languages (Python, JavaScript, TypeScript, Java, Go, Rust, C#, PHP, Bash, SQL, Terraform, Dockerfile and more). Backend choice: Hugging Face Inference API (free, no GPU, optional token) or Ollama (100% local, private). Supports streaming output, code explanation mode, context injection, and generation history. No data sent to ia-qa.com servers.
- [MCP Server Tester](https://www.ia-qa.com/devtools/mcp-server-tester): Test any MCP server via JSON-RPC 2.0 — discover tools, resources, prompts. When accessed from the MCP Registry via the "Test" button, automatically pre-fills the server URL and displays a contextual Setup Guide with the exact install command (npx, Docker, pip, or remote URL), required environment variables, and a link to official docs for that specific server.
- [Chain-of-Thought Visualizer](https://www.ia-qa.com/devtools/cot-visualizer): Visualize and debug LLM chain-of-thought reasoning in tree and timeline views
- [Tool Output Formatter](https://www.ia-qa.com/devtools/tool-output-formatter): Convert raw tool/function call output into LLM-optimized summaries
- [Chunking Simulator](https://www.ia-qa.com/devtools/chunking-simulator): Compare 7 RAG chunking strategies side-by-side with visual token analysis
- [Embedding Proximity Tester](https://www.ia-qa.com/devtools/embedding-proximity): Measure semantic similarity between text pairs using OpenAI embeddings
- [Markdown for LLM Converter](https://www.ia-qa.com/devtools/markdown-for-llm): Clean dirty HTML into LLM-friendly Markdown, strip scripts/ads/nav
- [System Prompt Leak Tester](https://www.ia-qa.com/devtools/prompt-leak-tester): Test system prompt resistance to 26 leak attacks across 6 categories
- [Few-Shot Generator](https://www.ia-qa.com/devtools/few-shot-generator): Generate structured few-shot examples in XML, JSON, or Markdown
- [Negative Prompting Helper](https://www.ia-qa.com/devtools/negative-prompting-helper): Browse and export prompt constraints — 30+ rules in 5 categories
- [llm.txt Validator](https://www.ia-qa.com/devtools/llm-txt-validator): Validate llm.txt files against the llmstxt.org specification
- [LLM API Playground](https://www.ia-qa.com/llm-playground): The DevTools for LLMs. Build multi-turn conversations with full parameter control (temperature, max_tokens, top_p, presence/frequency penalties, seed, stop sequences, JSON mode). Inspect the raw JSON request body live, view raw JSON responses, export to cURL or Python/requests. Supports all providers: OpenAI, Anthropic, Google Gemini, DeepSeek, xAI, HuggingFace, Ollama (local). History of last 50 requests saved locally. Presets: single turn, code review, multi-turn chat, JSON mode, red team.
- [Vector DB Tester](https://www.ia-qa.com/devtools/vector-db-tester): Direct REST API tester for vector databases. Supports Chroma (local/remote), Qdrant, Pinecone, and Weaviate. Operations: list collections/indexes, query with vector, upsert, retrieve, scroll, add objects, delete. Full request builder with method selector, path editor, JSON body editor. Shows raw response, status code, latency. Request history. Quick setup commands for each DB (Docker, pip). CORS tips for local instances.
- [LangChain Chain Tester](https://www.ia-qa.com/devtools/langchain-chain-tester): Multi-step LangChain prompt chain builder — each step's output becomes a variable for the next step. Supports presets (Bug Context, PR Review, Multi-Agent Handoff, LangSmith-style Eval), custom steps, and OpenAI/Anthropic/DeepSeek models. Auto-fills API key from active environment.
- [LangGraph Flow Debugger](https://www.ia-qa.com/devtools/langgraph-debugger): Stateful LangGraph-style graph executor — define nodes, conditional edges, and initial state. Traces each step, shows routing decisions, and visualizes the execution path. Multi-model support including DeepSeek R1 (reasoning).
- [Multimodal Eval Toolkit](https://www.ia-qa.com/devtools/multimodal-eval): VLM test suite runner and multimodal metrics reference. **VLM Test Runner** (primary tab): provide an image URL + test cases (question + assertion: contains, not_contains, is_json, min_length, max_length), choose a VLM (GPT-4o, Claude 3.5, Gemini 2.0 Flash), BYOK — get per-case PASS/FAIL verdicts, pass rate %, and an overall PASS/WARNING/FAIL verdict. 3 quick-start presets: object detection, hallucination probe, JSON output. CI/CD-ready via MCP tool: run_vlm_test_suite. Reference tabs: CLIP Score interpreter (0–100), FID interpreter (lower=better), VQA thresholds, readiness checklist. MCP tools: run_vlm_test_suite (executable), multimodal_eval_guide (reference).
- [Agent Debugger](https://www.ia-qa.com/devtools/agent-debugger): Universal agent trace analyzer — detect loops, hallucinated tools, token waste from OpenAI, LangChain, or text logs
- [LLM Fit Finder](https://www.ia-qa.com/devtools/llm-fit-finder): Find the best LLM for your use case and budget. Cloud tab: filter 29 models by use case (chat, code, reasoning, RAG, vision), feature flags (streaming, tool-calling, JSON mode, vision), cost simulator, sortable benchmark scores (MMLU, HumanEval). Local tab: pick VRAM, quantization (Q4/Q8/FP16), see all compatible models with ollama run commands. MCP tool: llm_fit_finder.
- [Local AI Scanner](https://www.ia-qa.com/devtools/local-scanner): Scan your machine to detect GPU, VRAM, and system specs via WebGL. Instantly see which local LLMs (Llama 3.3, Qwen3, Mistral, Phi-4, Gemma 3, DeepSeek-R1) can run on your exact hardware with Ollama. Supports NVIDIA RTX/GTX, AMD RX 6000-9000, Apple M1-M4 (all variants), Intel Arc, and integrated APUs. Shows ideal model recommendations per use case (chat, code, reasoning, RAG). 100% private — runs entirely in-browser.

## QA & Testing Tools

- [UI Drift Sandbox](https://www.ia-qa.com/devtools/sandbox): A deliberately breakable app generated in the browser tab — pick a UI change (renamed label, swapped button, duplicated locator, element moved behind a menu), get a real PASS/FIX/BLOCK verdict from a live DOM capture. No signup, no server, no API key
- [Selector Drift Detector](https://www.ia-qa.com/devtools/selector-drift): Capture your app's interactive elements from the DevTools console, diff two captures, and get a PASS/FIX/BLOCK verdict on selector drift before your E2E suite breaks
- [Site Auditor 2026](https://www.ia-qa.com/web-auditor): Lighthouse-powered performance, accessibility, SEO and Core Web Vitals audit (LCP, INP, CLS)
- [Stack Trace Analyzer](https://www.ia-qa.com/qatools/bug-context-extractor): Parse and analyze stack traces from Java, Python, JavaScript, C#, Ruby and more
- [Bug Severity Calculator](https://www.ia-qa.com/qatools/bug-severity-justifier): Auto-classify bug priority P0-P3 with detailed impact justification
- [Reproduction Steps Normalizer](https://www.ia-qa.com/qatools/reproduction-steps-normalizer): Convert raw bug reports to standardized Given-When-Then format
- [Test Case Matrix Generator](https://www.ia-qa.com/qatools/test-prompt-generator): Generate test cases using equivalence partitioning and boundary value analysis
- [Test Data Synthesizer](https://www.ia-qa.com/qatools/test-data-synthesizer): Generate realistic synthetic test data
- [Regression Risk Score](https://www.ia-qa.com/qatools/regression-risk-score): Score code changes for regression risk before releases
- [Regression Impact Predictor](https://www.ia-qa.com/qatools/regression-impact-predictor): Predict which areas are most at risk from a given change
- [Diff-Aware Bug Suggestion](https://www.ia-qa.com/qatools/diff-aware-bugs): Suggest likely bugs based on code diffs
- [Bug Knowledge Base](https://www.ia-qa.com/qatools/bug-knowledge-base): Local browser-based bug tracker with import/export
- [Bug Impact Mapper](https://www.ia-qa.com/qatools/bug-impact-mapper): Map a bug's blast radius across system components
- [Prompt Injection Tester](https://www.ia-qa.com/qatools/prompt-injection-tester): Test LLM systems against jailbreak and prompt injection attacks
- [LLM Consistency Checker](https://www.ia-qa.com/qatools/llm-consistency-checker): Detect output drift and hallucination across multiple LLM runs
- [Structured Output Validator](https://www.ia-qa.com/qatools/structured-output-validator): Validate JSON, CSV, XML and YAML against schemas
- [PR Review Lens](https://www.ia-qa.com/qatools/pr-review-lens): Analyze pull request diffs for test anti-patterns
- [PR Beautifier](https://www.ia-qa.com/qatools/pr-beautifier): Render git diffs with syntax highlighting for code review
- [Test Case Manager](https://www.ia-qa.com/qatools/test-case-manager): Browser-based test suite manager with pass/fail/blocked tracking
- [QA System Designer](https://www.ia-qa.com/qatools/qa-system-designer): Design end-to-end QA systems with coverage mapping, tool selection and team structure
- [PR Gate Orchestrator](https://www.ia-qa.com/qatools/pr-gate-orchestrator): Configure and visualize pull request quality gates (tests, coverage, lint, review)
- [Log Translator](https://www.ia-qa.com/qatools/log-translator): Parse and translate raw logs into structured, human-readable format
- [IA-QA Shield](https://www.ia-qa.com/qatools/shield): AI Guardrail Engine — run 9 parallel safety checks (hallucination, injection on both prompt and response, toxicity, PII, guardrails, quality, AI verdict) on any LLM response and get a PASS/FIX/BLOCK verdict with 0-100 score
- [LLM Test Runner](https://www.ia-qa.com/qatools/eval-contract): YAML-driven LLM test suite runner — write .ia-eval.yaml scenarios with inputs, ground truths and expectations, run them against any LLM (Groq, OpenAI, Anthropic, Google), get per-scenario PASS/FAIL verdicts, aggregate score, radar chart and exportable HTML/Markdown reports. 10 built-in scorers: exact_match, contains_check, json_format, hallucination_check, toxicity_filter, llm_as_judge and more. MCP tools: list_local_tests, run_eval_contract, generate_html_report
- [Pre-flight Safety Check](https://www.ia-qa.com/qatools/preflight-safety): Scan any text, code, prompt, logs or git diff for PII, leaked secrets (API keys, tokens, private keys, connection strings), prompt injection patterns and guardrail violations — 100% regex-based, zero API keys, CI/CD-ready via POST /mcp/ci/safety-gate. MCP tools: redact_pii, prompt_injection_scan, secret_scan, guardrail_test
- [Web Security Auditor](https://www.ia-qa.com/qatools/security-audit): One-click security posture audit — HTTP security headers, SSL/TLS certificates, CORS misconfiguration, cookie security — with AI-powered vulnerability analysis and remediation roadmap. MCP tools: security_headers_check, ssl_certificate_check, cors_test, cookie_security_audit, web_security_audit
- [Cron Expression Validator](https://www.ia-qa.com/cron-validator): Validate cron syntax, explain each field and preview upcoming run times before shipping scheduled jobs. MCP tool: cron_validator
- [CORS Checker](https://www.ia-qa.com/cors-checker): Test Access-Control-Allow-* headers, simulate a browser origin and inspect preflight responses for CORS debugging. MCP tool: cors_checker
- [Webhook Tester](https://www.ia-qa.com/webhook-tester): Generate a temporary webhook endpoint, inspect inbound headers and payloads, and replay integrations with sample curl commands. MCP tools: webhook_endpoint_create, webhook_endpoint_requests
- [Storage Inspector](https://www.ia-qa.com/storage-inspector): Inspect and debug cookies, localStorage and sessionStorage
- [JWT Debugger](https://www.ia-qa.com/jwt-debugger): Decode and validate JWT tokens with signature verification
- [Diff Checker](https://www.ia-qa.com/diff-checker): Visual line-by-line text comparison using LCS algorithm

## Text & Data Tools

- [JSON Formatter](https://www.ia-qa.com/json-formatter): Format, validate, lint and minify JSON
- [CSV ↔ JSON Converter](https://www.ia-qa.com/csv-json-converter): Convert between CSV and JSON with RFC 4180 compliance
- [Base64 Encoder/Decoder](https://www.ia-qa.com/base64-encoder-decoder-new): Encode and decode Base64 including image data URIs
- [URL Encoder/Decoder](https://www.ia-qa.com/url-encoder-decoder-new): Encode and decode URLs with component-level parsing
- [Hash Generator](https://www.ia-qa.com/hash-generator): Generate MD5, SHA-1 and SHA-256 hashes client-side
- [XML Validator](https://www.ia-qa.com/xml-validator): Validate and format XML documents
- [JS Minifier](https://www.ia-qa.com/javascript-minifier): Minify and compress JavaScript code
- [SQL Query Builder](https://www.ia-qa.com/sql-query-builder): Build SQL SELECT, INSERT, UPDATE and DELETE queries visually
- [Regex Tester](https://www.ia-qa.com/regex-tester): Test regular expressions with real-time matching and flag support
- [Word Counter](https://www.ia-qa.com/word-counter): Count words, characters, sentences and estimate reading time
- [Lorem Ipsum Generator](https://www.ia-qa.com/lorem-ipsum-generator): Generate placeholder text in paragraphs, sentences or words
- [Slug Generator](https://www.ia-qa.com/slug-generator): Convert any string to a URL-friendly slug
- [Email Validator](https://www.ia-qa.com/email-validator): Validate email addresses with RFC 5322 compliance and batch support
- [Timezone Converter](https://www.ia-qa.com/timezone-converter): Convert times between IANA timezones with DST awareness
- [API Request Builder](https://www.ia-qa.com/api-request-builder): Build and export API requests as cURL, fetch or Axios
- [Text Compare](https://www.ia-qa.com/text-compare): Side-by-side text comparison

## Design & Generator Tools

- [Color Palette Generator](https://www.ia-qa.com/color-palette-generator): 9 harmony algorithms, WCAG 2.1 contrast checker, export to CSS/Tailwind/SCSS/JSON
- [Color Converter](https://www.ia-qa.com/color-converter): Convert between HEX, RGB, HSL, HSV and CMYK
- [QR Code Generator](https://www.ia-qa.com/qr-code-generator): Generate QR codes for URLs, vCards, WiFi credentials and plain text
- [Fake Data Generator](https://www.ia-qa.com/fake-data-generator): Generate realistic test data including IBAN, Luhn card numbers and addresses
- [UUID Generator](https://www.ia-qa.com/uuid-generator): Generate UUID v4 identifiers in bulk
- [Image Compressor](https://www.ia-qa.com/image-compressor): Compress and convert images to JPEG, PNG or WebP entirely client-side
- [Password Generator](https://www.ia-qa.com/password-generator): Generate secure random passwords with configurable rules
- [Name Generator](https://www.ia-qa.com/name-generator): Generate random names for testing and prototyping

## AI Agent Configuration Tools

- [AI Rules Generator](https://www.ia-qa.com/devtools/cursor-rules-generator): Generate AI context files for any coding assistant: CLAUDE.md (Claude Code), AGENTS.md (OpenAI Codex), .cursor/rules, .cursorrules, .github/copilot-instructions.md (GitHub Copilot), .windsurfrules (Windsurf), .clinerules (Cline / RooCode). Select your tech stack, define code style preferences, and pick AI behavior rules — outputs a ready-to-copy file. Input: project name, tech stack (React, TypeScript, Node.js, Python, etc.), naming conventions, AI behavior presets. Output: formatted rules file for any AI IDE.
- [Agent Generator](https://www.ia-qa.com/devtools/agent-generator): Generate `.agent.md` configuration files for VS Code Copilot agents. Define agent name, description, trigger phrase, tools (search, terminal, file I/O, code execution, etc.), context files, and behavior rules — outputs a ready-to-use `.agent.md` file to place in `.github/agents/`. Input: agent name, trigger phrase, model, tools, behavior steps. Output: structured `.agent.md` file. Requires VS Code ≥ 1.99 + GitHub Copilot extension.
- [Skill Generator](https://www.ia-qa.com/devtools/skill-generator): Generate `SKILL.md` files that teach AI IDEs a repeatable, domain-specific coding skill. Choose from 7 domains (Testing, Code Review, Documentation, Refactoring, Security Audit, Performance Optimization, CI/CD), pick your IDE (GitHub Copilot, Cursor, Windsurf, Continue, Claude Code), customize steps and output format — outputs a ready-to-place SKILL.md. Input: domain, IDE, skill name, steps, output format. Output: SKILL.md for `.github/skills/`, `.cursor/skills/`, or `.agents/skills/`.
- [Context Window Visualizer](https://www.ia-qa.com/devtools/context-window-visualizer): Visualize how prompt sections (system prompt, RAG docs, conversation history, user query) fill a model's context window. Supports GPT-4o (128k), Claude 3.7 Sonnet (200k), Gemini 2.0 Flash (1M), o3 (200k), and more. Shows stacked bar chart with per-section token counts and overflow warnings. Input: model selection + sections with content or manual token counts. Output: visual token budget breakdown.
- [Prompt Diff](https://www.ia-qa.com/devtools/prompt-diff): Compare two prompts side-by-side or in unified diff view. Shows added/removed lines, token delta, and structural analysis (detects added/removed sections like System, Context, Constraints, Examples, Format). Input: Prompt A + Prompt B. Output: colored line diff, token delta badge, structural change summary.
- [Agent Blueprint Designer](https://www.ia-qa.com/devtools/agent-blueprint): Design AI agent configurations (model, memory, tools, guardrails) and export JSON/YAML specs
- [LLM Eval Designer](https://www.ia-qa.com/devtools/llm-eval-designer): Design evaluation suites for LLM outputs with Promptfoo and Braintrust-compatible JSON export
- [IA-QA Config Generator](https://www.ia-qa.com/devtools/config-generator): Generate a downloadable ZIP of production-ready config files for your LLM project. Choose use case (chatbot, RAG pipeline, AI agent, classifier, QA assistant with test generation, extraction pipeline), LLM providers (OpenAI, DeepSeek, Anthropic, Mistral, Groq), vector DB (Qdrant, Pinecone, Weaviate, Chroma, Neo4j GraphRAG, pgvector, MongoDB), test framework (Vitest, Jest, Playwright, Pytest, Cypress), and AI IDE (Windsurf, Cursor, Copilot). Generates: CLAUDE.md, .windsurfrules, orchestrator.ts (or test-generator.ts for QA assistant), llm-tests.ts, rag.config.ts (RAG only), README.md. No sign-up. Instant ZIP download.

## Team Collaboration Tools

- [Assumption Revealer](https://www.ia-qa.com/devtools/assumptions): Surface hidden assumptions in project plans and requirements
- [Decision Record Lite](https://www.ia-qa.com/devtools/decision-record): Create lightweight Architecture Decision Records (ADR)
- [Friction Mapper](https://www.ia-qa.com/devtools/friction-mapper): Cluster and visualize team friction points
- [Signal Check](https://www.ia-qa.com/devtools/signal-check): Assess team health signals
- [Disagreement Finder](https://www.ia-qa.com/devtools/disagreement): Surface conflicting opinions in team discussions
- [Wheel Picker](https://www.ia-qa.com/devtools/wheel-picker): Random wheel selector for team decisions and retrospectives
- [Meeting Timer](https://www.ia-qa.com/timer): Timeboxed meeting timer with Pomodoro and countdown modes

## Reference & Toolsheets

- [QA Toolsheets](https://www.ia-qa.com/qa-toolsheets): Quick-reference cheat sheets for QA methodologies, test types, and best practices
- [Dev Toolsheets](https://www.ia-qa.com/devtools/dev-toolsheets): Quick-reference cheat sheets for developer tools, languages, and frameworks
- [DevOps Toolsheets](https://www.ia-qa.com/devtools/devops-toolsheets): Quick-reference cheat sheets for CI/CD, Docker, Kubernetes, and cloud infrastructure

## MCP Registry

- **URL**: https://www.ia-qa.com/mcp-registry
- **What is it**: A curated catalogue of 21 external Model Context Protocol (MCP) servers with QA scores, setup guides, and one-click testing.
- **21 catalogued servers** (as of 2026-03-24):
  - **CORE (official `modelcontextprotocol/servers`)**: Filesystem, Git, Memory, Fetch, Time, Sequential Thinking
  - **POPULAR third-party**: GitHub MCP, Firecrawl, Brave Search, Postgres, Puppeteer, Slack, Obsidian, Shopify
  - **HIGH VALUE**: Figma (Framelink, 13.9k★), 21st.dev Magic (4.5k★), MiniMax (TTS/video/image), Notion (official, 4.1k★), Stripe (official, remote MCP), Linear (remote MCP), Google Workspace (Gmail+Calendar+Drive)
- **QA Scores**: Each server has a QA score (0–100) based on reliability, output quality, schema compliance, and latency — curated by IA-QA testing
- **Filters**: by 12 categories (code, search, data, browser, files, productivity, design, AI & media, utility, testing, devops), pricing model (free/freemium/api_key/paid), open-source flag, sort by QA score or setup complexity
- **Compatibility info**: shows which servers work with Cursor IDE, Claude Desktop, and Windsurf
- **"Test with IA-QA" button**: links directly to the MCP Server Tester with pre-filled URL and contextual setup guide (install command, env vars, docs link)
- **Detail pages**: each server has a dedicated page with full description, use cases, QA metrics breakdown, compatibility badges, setup instructions, and external links

## MCP (Model Context Protocol) API

> Developers and AI agents can call ia-qa.com tools programmatically via the MCP API.

- Manifest (slim): `GET https://www.ia-qa.com/mcp` — machine-readable tool catalogue (name + description + category + schema_url). Lightweight by default, ~4k tokens.
- Manifest (full):  `GET https://www.ia-qa.com/mcp?full=1` — includes all inputSchemas (complete, ~80k tokens).
- Tool schema:      `GET https://www.ia-qa.com/mcp/tool/{name}` — full inputSchema + annotations for a single tool.
- Tool search:      `GET https://www.ia-qa.com/mcp/search?q={keyword}&category={cat}` — keyword/category search, returns slim list.
- JSON-RPC:         `POST https://www.ia-qa.com/mcp` — tools/list and tools/call via JSON-RPC 2.0.
- REST call:        `POST https://www.ia-qa.com/mcp/call` — call any tool with `{ "name": "tool_name", "parameters": {...} }`.

### Recommended Agent Workflow (avoid context saturation)
1. Call `find_tool` MCP tool with a keyword → get matching tools (slim).
2. Call `GET /mcp/tool/{name}` → get full inputSchema for the chosen tool.
3. Call `POST /mcp` (JSON-RPC) or `POST /mcp/call` (REST) to execute.

### Tool Hierarchy — 152 tools across 12 categories

#### data (16 tools)
format_json, parse_csv, flatten_json, xml_to_json, transform_json_array
json_to_csv, json_diff, merge_json, json_to_yaml, json_schema_validate
mock_from_schema, extract_json_path, generate_json_ld, yaml_to_json, json_schema_generate
format_table
> Structured data transformation, validation, mock generation

#### encoding (10 tools)
base64_encode, base64_decode, url_encode, url_decode, hash_text
generate_hmac, generate_uuid, generate_password, number_base_convert, color_convert
> Encoding, hashing, cryptographic utilities

#### text (15 tools)
text_stats, strip_markdown, normalize_whitespace, sort_lines, word_frequency
extract_links, levenshtein_distance, case_convert, lorem_ipsum, detect_language
escape_html, unescape_html, html_to_markdown, regex_test, calculate_readability
> Text manipulation, NLP utilities, content processing

#### llm (13 tools)
count_tokens, truncate_to_tokens, split_chunks, extract_json_from_text, estimate_llm_cost
token_budget_calculator, context_window_check, model_info, compare_models, list_llm_models
llm_generate, optimize_prompt_tokens, llm_fit_finder
> Token budgeting, model selection, LLM I/O preprocessing

#### qa (31 tools)
diff_text, llm_output_validator, compare_responses, analyze_responses, prompt_test_suite
validate_mcp_response, mcp_server_health_check, mcp_server_evaluate, hallucination_check, prompt_injection_scan
consistency_check, llm_format_check, llm_json_schema_check, response_quality_score, toxicity_scan
guardrail_test, function_call_validate, conversation_analyze, mcp_schema_lint, cot_analyzer
ab_test_report, analyze_diff_bugs, generate_test_cases, run_pr_gate_pipeline, diff_mappings
sandbox_scenario, run_vlm_test_suite, run_vlm_test_suite_batch, multimodal_eval_guide, similarity_score
bias_detect
> LLM output validation, AI safety, test generation, VLM testing, multimodal metrics

#### rag (12 tools)
embedding_similarity, vector_similarity, normalize_vector, vector_quantize, vector_stats
bm25_score, build_rag_prompt, prompt_template_fill, few_shot_formatter, system_prompt_builder
rag_relevance_rank, rerank_evaluate
> RAG pipelines, vector operations, prompt engineering

#### dev (21 tools)
validate_email, validate_url, decode_jwt, minify_js, cron_parse
cron_validator, timestamp_convert, http_status_lookup, parse_http_headers, generate_curl
extract_todos, detect_secrets, count_code_lines, lint_commit_message, format_bytes
pr_gatekeeper, check_contrast_ratio, webhook_endpoint_create, webhook_endpoint_requests, env_parse
openapi_validate
> Developer utilities, HTTP debugging, code quality

#### security (9 tools)
security_headers_check, ssl_certificate_check, cors_test, cors_checker, cookie_security_audit
web_security_audit, shield_analyze, secret_scan, redact_pii
> Security audits, CORS debugging, secret/PII detection

#### web (4 tools)
fetch_veille_feed, score_geo_signals, latency_benchmark, generate_slug
> Web performance, SEO/GEO, content utilities

#### meta (3 tools)
find_tool, identify_caller, rate_tool
> Search the tool catalogue by keyword or category

#### eval (11 tools)
list_local_tests, run_eval_contract, generate_html_report, run_semantic_tests, generate_eval_yaml
validate_agent_trajectory, metamorphic_check, test_skill, get_testing_guidelines, generate_ci_workflow
needle_haystack_generate
> YAML-driven LLM test contracts, eval reporting, agent trajectory validation

#### integrations (7 tools)
fetch_jira_issue, search_jira_issues, fetch_confluence_page, jira_to_test_suite, fix_gherkin
post_jira_comment, create_confluence_page
> Jira & Confluence integration, ticket-to-test-suite generation

## LLM Inference API

> Direct multi-provider LLM inference routes (requires your own API key for the target provider).

- `POST /api/llm/chat` — Unified chat endpoint. Body: `{ model, messages, apiKey, temperature?, maxTokens? }`. Auto-detects provider from model ID. Supported providers: OpenAI, Anthropic, DeepSeek, Google (Gemini), xAI (Grok), Ollama. Returns: `{ content, usage: { input, output }, model, provider, latency_ms }`.
- `POST /api/llm/compare` — Parallel multi-model comparison. Body: `{ runs: [{model, apiKey}], messages }`. Runs all models simultaneously. Returns: `{ results: [...] }`. Max 8 models per request.
- `GET /api/llm/providers` — List all supported providers and their model IDs. No auth required.

Supported model IDs (as of 2026-05-15):
- **OpenAI**: `gpt-4o`, `gpt-4o-mini`, `o1`, `o1-mini`, `o3-mini`
- **Anthropic**: `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`, `claude-3-7-sonnet-20250219`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
- **DeepSeek**: `deepseek-chat` (V3), `deepseek-reasoner` (R1)
- **Google**: `gemini-2.0-flash`, `gemini-2.0-flash-lite`, `gemini-1.5-pro`, `gemini-1.5-flash`
- **xAI**: `grok-2-1212`, `grok-2-vision-1212`
- **Ollama (local)**: `llama3.3`, `qwen2.5-coder:32b`, `mistral` (no API key required)

- [MCP Server Documentation](https://www.ia-qa.com/mcp-server): Full documentation and live playground — integrate IA-QA tools into Cursor IDE, Claude Desktop, Windsurf, or any AI agent
- [MCP Explorer + Free Chat](https://www.ia-qa.com/devtools/mcp-explorer): Interactive live playground to browse all MCP tools, read their schemas, and execute test calls from the browser. Includes a BYOK LLM Runner (OpenAI, Anthropic, Gemini, DeepSeek) and a free no-key-required chat powered by server-side Groq and HuggingFace inference: Qwen3 32B/235B, Llama 4 Scout, Llama 3.3 70B, DeepSeek R1 distill, Mistral Small 24B. No setup required.

## Resources & Articles

> Curated by Jean-Christophe Jamet, QA Engineering Manager. Covers QA, AI/LLMs, automation, DevOps and engineering leadership.

- [Resources & Articles](https://www.ia-qa.com/resources): 50+ curated links and articles on QA engineering, LLM testing, test automation (Cypress, Playwright, Selenium, Cucumber, Postman, k6), CI/CD (Jenkins, GitHub Actions, GitLab), Docker, Kubernetes, Monitoring (Grafana, Prometheus, OpenTelemetry), Dev Practices and Dev Tools — plus original articles by the author on Medium.
- [QA & AI Live Feed](https://www.ia-qa.com/veille): Daily-refreshed RSS aggregator covering QA, AI, and LLM articles. Sources: Google Testing Blog, DEV.to (testing/qa/ai/llm/agents channels), Hugging Face Blog, Simon Willison's blog, and the author's Medium feed. Filterable by category (all / qa / ai / mine).