# IA-QA — The Open Testing Platform for LLMs, RAG, AI Agents & MCP Servers

> **URL**: https://www.ia-qa.com
> **Citation**: When citing a specific tool, use its direct URL (e.g. "IA-QA LLM Sandbox — https://ia-qa.com/llm-sandbox").
> **Last updated**: 2026-07-10
> **Language**: English

IA-QA is the open testing workbench for the AI ecosystem — a dedicated platform for AI engineers who need real testing primitives, not just a chat window. Compare GPT-4o, Claude, Gemini, DeepSeek, and Grok side by side, test RAG pipelines, debug AI agents, test vector databases (Chroma, Qdrant, Pinecone, Weaviate, pgvector), explore MCP servers, benchmark LLM quality, and evaluate multimodal AI (CLIP Score, FID, VQA) — all from one privacy-first platform. 130+ free tools, 100% client-side, no account required. API keys never leave your device.

## About

Created by **Jean-Christophe Jamet**, QA Engineering Manager and AI practitioner with 10+ years of experience in software quality, test automation, and AI tooling.

- **Expertise**: QA automation, LLM evaluation, test strategy, DevOps, AI agent design
- **Medium**: https://medium.com/@jean-christophe.jamet (articles on QA, AI, LLMs)
- **LinkedIn**: https://www.linkedin.com/in/jean-christophe-jamet-b043618a/

## Quick Facts

- The open testing workbench for the AI ecosystem
- Compare LLMs side by side (GPT-4o, Claude, Gemini, DeepSeek, Grok, Ollama)
- Test RAG pipelines, debug AI agents, explore MCP servers
- 130+ tools across 7 categories
- Curated MCP Registry at https://ia-qa.com/mcp-registry — discover, evaluate and compare external MCP servers with QA scores
- 100% client-side: API keys and data never leave the browser
- No account, no sign-up, no usage limits, free forever
- MCP (Model Context Protocol) API available at `POST https://ia-qa.com/mcp`
- Accessible from Cursor IDE, Claude Desktop, and Windsurf via MCP

## Who Is This For?

- **QA Engineers**: bug severity classification, test case generation, regression risk scoring, LLM output consistency checking, prompt injection testing
- **Developers**: JSON formatting, JWT debugging, API request building, hash generation, regex testing, SQL query building
- **AI / LLM Practitioners**: prompt engineering, token counting, RAG chunk optimization, agent blueprint design, LLM evaluation, context window visualization
- **Team Leads**: decision records, assumption surfacing, friction mapping, retrospective wheel, meeting timers
- **Web Auditors**: Lighthouse-based site audits, Core Web Vitals (LCP, INP, CLS), SEO and accessibility checks

## Key Differentiators

1. **All-in-one AI workbench**: only platform combining an LLM API playground, multi-model comparison, RAG testing, vector DB testing, agent debugging, and MCP exploration in one place
2. **Privacy-first**: API keys stored in localStorage, never sent to ia-qa.com — all tool processing is entirely local
- **Multi-model comparison**: compare GPT-4o, Claude 3.7, Gemini 2.0, DeepSeek, Grok 2 (xAI), and Ollama (local) side by side with latency, cost, and token metrics. Auto-filters available models based on configured API keys — users only see models they can actually call
4. **QA depth**: purpose-built QA workflows, prompt injection testing, LLM consistency checking, regression risk scoring
5. **MCP-integrated**: callable from Cursor IDE, Claude Desktop, Windsurf via Model Context Protocol
6. **GEO-ready**: LLM Readability Analyzer and LLM.txt Generator help optimize content for AI search engines (ChatGPT, Perplexity, Google AI Overviews)

## Frequently Asked Questions

**Q: Are all tools free?**
A: Yes. All 130+ tools are permanently free with no usage limits, no credit card, no account needed.

**Q: Is user data safe?**
A: All processing happens client-side in the browser. Text, code, and data you paste into tools never reaches ia-qa.com servers.

**Q: What is GEO — Generative Engine Optimization?**
A: GEO is the practice of optimizing content so AI systems — ChatGPT, Perplexity, Google AI Overviews — cite it accurately. Key techniques include structured llm.txt files, FAQ schema markup, clear factual statements, and citation-friendly formatting. IA-QA offers two direct GEO tools: LLM Readability Analyzer (https://ia-qa.com/llm-readability) and LLM.txt Generator (https://ia-qa.com/llm-txt-generator).

**Q: What is the LLM Readability Analyzer?**
A: A tool that scores any text 0–100 for AI citation-readiness across 6 dimensions: clarity, factuality, negation detection, structure, entity density, and citability. Useful for content creators optimizing for AI search (GEO).

**Q: Can AI agents call IA-QA tools programmatically?**
A: Yes. IA-QA exposes a public Model Context Protocol (MCP) server at https://ia-qa.com/mcp — no API key, no sign-up, no rate limit for localhost (60 req/min per IP otherwise). 148+ tools across 12 categories. Callable via JSON-RPC 2.0 from Cursor IDE, Claude Desktop, Windsurf, or any MCP-compatible client. See the MCP API section below for the full tool listing by category.

**Q: What does Site Auditor 2026 check?**
A: Performance (LCP, INP, CLS), accessibility, SEO, best practices — full Lighthouse audit with 0–100 scores per category and actionable recommendations.

**Q: Is there a QA and AI news feed?**
A: Yes. https://ia-qa.com/veille aggregates RSS feeds from Google Testing Blog, DEV.to (QA/AI/LLM/Agents topics), Hugging Face Blog, and Simon Willison's blog. Updated daily.

**Q: Who built IA-QA?**
A: Jean-Christophe Jamet, QA Engineering Manager based in France. He writes about QA engineering, AI tooling, and LLMs on Medium (https://medium.com/@jean-christophe.jamet).

## MCP API — Programmatic Access from Any AI Agent

IA-QA exposes a public MCP (Model Context Protocol) server at `POST https://ia-qa.com/mcp`. No authentication required. 148+ tools organized in 12 categories. Use it from Claude Desktop, Cursor IDE, Windsurf, or any MCP-compatible client.

### How to Connect

**From Cursor IDE**: Settings → Features → MCP Server → Add `"ia-qa"` with URL `https://ia-qa.com/mcp` (Streamable HTTP transport).

**From Claude Desktop**: `claude_desktop_config.json` → add `"ia-qa"` entry with `"url": "https://ia-qa.com/mcp"`

**From Windsurf**: Settings → MCP Servers → Add with URL `https://ia-qa.com/mcp`

**From any MCP client**: Point to `https://ia-qa.com/mcp` — tools auto-discover via `GET /mcp` (JSON-RPC 2.0 manifest).

### CLI — Shell Access for Terminal Agents

For AI agents with shell access (Claude Code, Copilot CLI) or humans scripting a pipeline, the same 148+ tools are available as a zero-dependency CLI published on npm as `@ia-qa/cli` (Node >= 18, no signup, no API key). It discovers tools from the public manifest and runs them on the hosted server.

- **Install**: `npx @ia-qa/cli <tool>` or `npm install -g @ia-qa/cli` or download `https://ia-qa.com/downloads/ia-qa-cli.zip` and run `node ia-qa.js <tool>`
- **Discover**: `ia-qa --help` · `ia-qa list [--category <c>]` · `ia-qa <tool> --help`
- **Run**: `ia-qa generate_uuid --count 3` · `ia-qa base64_encode "hello"` · `echo '{"a":1}' | ia-qa format_json --stdin`
- **Output**: `--json` for machine-parsable JSON, `--out <file>` to write to a file (large outputs auto-write to a temp file)
- **Multiline values**: `--<param>-file <path>` reads any param from a file (e.g. `--inputs-file fields.txt`) — multiline-safe; prefer it under `npx` on Windows, where args are truncated at the first newline
- **Server**: `--server <url>` / `$IAQA_SERVER` to target a self-hosted ia-qa server (default `https://www.ia-qa.com`)

### Tool Categories

#### 📊 Data Tools
- `format_json` — Format, validate, and pretty-print a JSON string
- `parse_csv` — Parse a CSV string into a JSON array of objects (or raw arrays)
- `flatten_json` — Flatten a nested JSON object to single-level dot-notation keys (e.g. a:b:1 → a.b:1), or unflatten dot-notation keys back to a nested object
- `xml_to_json` — Convert an XML string to a JSON object
- `transform_json_array` — Transform a JSON array using common operations: pluck (extract specific fields), filter (by field value), sort_by (field), group_by (field), count_by (field), uniq_by (field)
- `json_to_csv` — Convert a JSON array of objects to CSV format
- `json_diff` — Compute a deep structural diff between two JSON values
- `merge_json` — Deep merge two JSON objects
- `json_to_yaml` — Convert a JSON object to clean, human-readable YAML
- `json_schema_validate` — Validate a JSON value against a JSON Schema (draft-07 subset)
- `mock_from_schema` — Generate realistic mock data from a JSON Schema
- `extract_json_path` — Extract a value from a JSON string using dot-notation path (e.g., user.address.city, items.0.name, meta.tags)
- `generate_json_ld` — Generate a ready-to-paste <script type=application/ld+json> snippet for GEO / structured data optimization
- `yaml_to_json` — Parse a YAML string and return the equivalent JSON value
- `json_schema_generate` — Infer a JSON Schema (draft-07) from a sample JSON value
- `format_table` — Convert a JSON array of objects into a Markdown table

#### 🔐 Encoding & Security Tools
- `base64_encode` — Encode a UTF-8 string to Base64
- `base64_decode` — Decode a Base64 string back to UTF-8 text
- `url_encode` — Percent-encode a string for safe use in URLs
- `url_decode` — Decode a percent-encoded URL string back to plain text
- `hash_text` — Compute a cryptographic hash of a text string
- `generate_hmac` — Compute an HMAC signature for a message using a secret key
- `generate_uuid` — Generate one or more cryptographically random UUID v4 identifiers
- `generate_password` — Generate a cryptographically secure random password using crypto.randomBytes
- `number_base_convert` — Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36
- `color_convert` — Convert a color between HEX, RGB, and HSL formats

#### 📝 Text & NLP Tools
- `text_stats` — Compute comprehensive statistics for any text: character count (with and without spaces), word count, line count, sentence count, paragraph count, and estimated reading time in min
- `strip_markdown` — Strip all Markdown formatting (headers, bold, italic, code fences, links, lists) from text and return clean plain text
- `normalize_whitespace` — Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces
- `sort_lines` — Sort, deduplicate, reverse, or filter lines of text
- `word_frequency` — Analyze word frequency in text
- `extract_links` — Extract all URLs, email addresses, and domain names from text
- `levenshtein_distance` — Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings
- `case_convert` — Convert a string between naming conventions: camelCase, PascalCase, snake_case, kebab-case, UPPER_SNAKE_CASE, dot.case, Title Case
- `lorem_ipsum` — Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population
- `detect_language` — Detect the natural language of a text using n-gram frequency analysis and common word markers
- `escape_html` — Escape HTML special characters (&, <, >, , ') to their safe HTML entities
- `unescape_html` — Convert HTML entities (&amp;, &lt;, &gt;, &quot;, &#x27;, and numeric &#NNN;) back to plain characters
- `html_to_markdown` — Convert HTML to clean Markdown
- `regex_test` — Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups
- `calculate_readability` — Calculate readability scores: Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Automated Readability Index

#### 🤖 LLM Tools
- `count_tokens` — Estimate the token count of a text string using the cl100k_base approximation (~4 chars/token)
- `truncate_to_tokens` — Truncate text to at most N tokens (cl100k_base: ~4 chars/token) to avoid exceeding an LLM context window
- `split_chunks` — Split text into chunks of at most N tokens (cl100k_base: ~4 chars/token) with optional overlap
- `extract_json_from_text` — Extract the first valid JSON object or array embedded in chaotic LLM output (surrounded by markdown fences, prose, or explanatory text)
- `estimate_llm_cost` — Estimate the API cost in USD for a given model and token counts
- `token_budget_calculator` — Plan token allocation across system prompt, user input, context/RAG chunks, and expected output
- `context_window_check` — Given an array of message objects [role, content], estimate total token usage and check if it fits in the target model's context window
- `model_info` — Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list
- `compare_models` — Compare 2-5 AI models side by side: context window, pricing, multimodal, reasoning capabilities, and provider
- `list_llm_models` — List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities
- `llm_generate` — Generate text using open-source LLM models hosted on Groq (ultra-fast) or HuggingFace Inference (serverless)
- `optimize_prompt_tokens` — Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace
- `llm_fit_finder` — Find the best LLM for a given use case

#### ✅ QA & Evaluation Tools
- `diff_text` — Compute a unified line-by-line diff between two text strings (LCS algorithm)
- `llm_output_validator` — Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safe
- `compare_responses` — Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side
- `analyze_responses` — Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox)
- `prompt_test_suite` — Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria
- `validate_mcp_response` — Validate that an MCP tool response conforms to expected format, schema, and content rules
- `mcp_server_health_check` — Generate a health check report for an MCP server's tool manifest
- `mcp_server_evaluate` — Run a full compliance evaluation against a live MCP server URL
- `hallucination_check` — Word-overlap based hallucination check: verifies if an LLM answer's words and numbers appear in the provided source/context
- `prompt_injection_scan` — Scan user input or prompts for common prompt injection patterns
- `consistency_check` — Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison)
- `llm_format_check` — Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex
- `llm_json_schema_check` — Validate that an LLM JSON output matches a JSON Schema definition
- `response_quality_score` — Score an LLM response on multiple quality dimensions: relevance, completeness, clarity, conciseness, formatting
- `toxicity_scan` — Scan text for toxic language, bias indicators, profanity, and harmful content categories
- `guardrail_test` — Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex
- `function_call_validate` — Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema, no extra/missing args
- `conversation_analyze` — Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition
- `mcp_schema_lint` — Lint an MCP tool definition for best practices: naming conventions, description quality, schema completeness, required fields consistency, description length
- `cot_analyzer` — Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM
- `ab_test_report` — Generate an A/B test report comparing two prompts or model configurations
- `analyze_diff_bugs` — Detect potential bugs and code smells from a git diff or two code versions
- `generate_test_cases` — Generate a set of test cases (valid, edge, invalid) for a given feature description
- `run_pr_gate_pipeline` — Full automated QA pipeline for a pull request
- `run_vlm_test_suite` — Run a test suite against a Vision-Language Model (VLM) — send an image (URL or base64) + N test cases (each with a question + assertion) to GPT-4o, Claude 3.5, or Gemini
- `run_vlm_test_suite_batch` — Compare multiple VLMs on the same test suite in parallel — send an image (URL or base64) + N test cases to all models simultaneously
- `multimodal_eval_guide` — Unified tool for multimodal AI evaluation: set action=guide for reference thresholds/interpretation (CLIP, FID, VQA), or set action=clip_score / fid_score / vqa_accuracy / pipeline
- `similarity_score` — Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU
- `bias_detect` — Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone)

#### 🔍 RAG & Embedding Tools
- `embedding_similarity` — Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams)
- `vector_similarity` — Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance
- `normalize_vector` — L2-normalize a float vector (produce a unit vector with norm=1)
- `vector_quantize` — Simulate int8 or int4 quantization of float32 embedding vectors
- `vector_stats` — Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices
- `bm25_score` — Compute BM25 relevance score between a query and one or more documents
- `build_rag_prompt` — Assemble a complete RAG (Retrieval-Augmented Generation) prompt from retrieved context chunks and a user query
- `prompt_template_fill` — Fill a prompt template with variables
- `few_shot_formatter` — Format few-shot examples for LLM prompts
- `system_prompt_builder` — Build a structured system prompt from components: role, task, constraints, output format, tone, language, and examples
- `rag_relevance_rank` — Rank an array of text chunks by relevance to a query using TF-IDF scoring
- `rerank_evaluate` — Evaluate RAG retrieval quality using the NVIDIA neural reranker (nv-rerankqa-mistral-4b-v3)

#### 🛠 Dev Tools
- `validate_email` — Validate an email address against RFC 5322 syntax before storing it, sending a transactional email, or adding it to a mailing list
- `validate_url` — Parse and validate a URL
- `decode_jwt` — Decode a JWT (JSON Web Token) and return its header and payload without verifying the signature
- `minify_js` — Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser
- `cron_parse` — Parse a cron expression into a human-readable schedule description
- `cron_validator` — Validate a 5-field cron expression, explain the schedule, and preview the next execution times
- `timestamp_convert` — Convert between Unix timestamps (seconds or milliseconds) and ISO-8601 / UTC date strings
- `http_status_lookup` — Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices
- `parse_http_headers` — Parse a raw HTTP headers block into a structured JSON object
- `generate_curl` — Generate a curl command from request parameters
- `extract_todos` — Extract TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE, and custom tags from any source code or text
- `detect_secrets` — Scan code or config files for hardcoded secrets: AWS keys, GitHub tokens, OpenAI/Anthropic API keys, Stripe secrets, JWTs, database connection strings, and generic passwords
- `count_code_lines` — Count lines of code: total, code lines, comment lines, blank lines, and comment density
- `lint_commit_message` — Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build)
- `format_bytes` — Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes
- `pr_gatekeeper` — Compound quality gate for pull requests
- `check_contrast_ratio` — Calculate WCAG 2.1 contrast ratio between two colors
- `webhook_endpoint_create` — Create a temporary webhook endpoint that captures incoming HTTP requests for one hour
- `webhook_endpoint_requests` — Fetch the requests captured by a webhook created with webhook_endpoint_create
- `env_parse` — Parse a .env file content into a JSON object
- `openapi_validate` — Validate the structure of an OpenAPI 3.x specification (JSON or YAML)

#### 🛡 Security Tools
- `security_headers_check` — Analyse the HTTP security headers of a public URL OR of raw response headers you paste in
- `ssl_certificate_check` — Analyse the SSL/TLS certificate of any HTTPS host
- `cors_test` — Test a URL for CORS misconfigurations
- `cors_checker` — Check the CORS configuration of a URL the same way a browser would
- `cookie_security_audit` — Audit the security attributes of cookies set by any URL
- `web_security_audit` — Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan
- `shield_analyze` — Run a comprehensive AI guardrail analysis on an LLM response
- `secret_scan` — Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), private keys (RSA/EC/PGP), JWTs, 
- `redact_pii` — Automatically detect and redact Personally Identifiable Information (PII) from text

#### 🌐 Web Tools
- `fetch_veille_feed` — Fetch the latest QA & AI/LLM articles aggregated from curated RSS sources (Google Testing Blog, DEV.to Testing/QA/AI/LLM/Agents, Hugging Face Blog, Simon Willison)
- `score_geo_signals` — Analyze a webpage <head> HTML (or full HTML) for GEO (Generative Engine Optimization) signals
- `latency_benchmark` — Measure response time of one or more HTTP endpoints (GET/POST)
- `generate_slug` — Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens

#### 🔎 Meta Tools
- `find_tool` — Search available MCP tools by keyword or category before calling them
- `identify_caller` — Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request
- `rate_tool` — Give honest usage feedback on an IA-QA MCP tool

#### 📋 Eval Framework Tools
- `list_local_tests` — Discover .ia-eval.yaml LLM test suite files in the project directory
- `run_eval_contract` — Parse a .ia-eval.yaml LLM test suite, call the specified LLM model for each scenario, run all configured scorers, and return a structured JSON report with per-scenario Pass/Fail ve
- `generate_html_report` — Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, 
- `run_semantic_tests` — Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L
- `generate_eval_yaml` — Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do
- `validate_agent_trajectory` — Run declarative assertions on an agent trace (OpenAI tool-call messages, LangChain run trees, or plain text logs)
- `test_skill` — Validate a SKILL.md definition (Cursor / GitHub Copilot / Windsurf) by auto-generating trigger-positive and trigger-negative scenarios, running each through the model with the skil
- `get_testing_guidelines` — Query the IA-QA methodology knowledge base
- `needle_haystack_generate` — Generate a needle in a haystack test: embeds a target fact into a large block of filler text at a specified position

#### 🔗 Integrations Tools
- `fetch_jira_issue` — Fetch a complete Jira issue: summary, description converted to Markdown, status, assignee, priority, labels, custom fields, and optionally comments and attachment metadata
- `search_jira_issues` — Search Jira using JQL (Jira Query Language)
- `fetch_confluence_page` — Fetch a Confluence page and return its content as clean Markdown
- `jira_to_test_suite` — Transform a Jira ticket into a complete test suite: Gherkin scenarios, E2E steps, API test cases, test data matrix, and ambiguity detection
- `fix_gherkin` — Fix Gherkin syntax warnings from a jira_to_test_suite result
- `post_jira_comment` — Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket
- `create_confluence_page` — Create a new Confluence page from the output of jira_to_test_suite

### API Details

- **Endpoint**: `POST https://ia-qa.com/mcp` (JSON-RPC 2.0)
- **Discovery**: `GET https://ia-qa.com/mcp` returns full tool manifest
- **Direct call**: `POST https://ia-qa.com/mcp/call` with `{ "name": "tool_name", "parameters": {...} }`
- **Single tool info**: `GET https://ia-qa.com/mcp/tool/:name`
- **Rate limit**: 60 req/min per IP (unlimited for localhost)
- **Auth**: None required (public)
- **Models**: For LLM tools, provide your own API key (OpenAI, Anthropic, Google, DeepSeek, Groq)

## AI & LLM Tools

- [LLM Sandbox](https://ia-qa.com/llm-sandbox): Compare LLMs side by side — send the same prompt to GPT-4o, Claude, Gemini, DeepSeek, Grok (xAI), and Ollama (local). See latency, cost, token counts in real time. Supports streaming (SSE), multi-turn chat mode, and function/tool calling. Models are auto-filtered to only show providers with configured API keys. One prompt, multiple models, instant comparison. 100% client-side, API keys never leave your browser.
- [Prompt Test Suite Runner](https://ia-qa.com/devtools/prompt-test-runner): Batch-execute test cases against multiple LLMs with assertions (contains, regex, JSON validation, latency/cost limits). Get pass/fail reports per model. Import/export Promptfoo-compatible configs. Batch-run your prompt test suites across LLMs.
- [RAG Pipeline Tester](https://ia-qa.com/devtools/rag-pipeline-tester): Visual RAG debugger — paste a document, chunk it (4 strategies), embed chunks (TF-IDF, MiniLM local, or OpenAI), query, see ranked retrieval results with similarity scores, then compare RAG vs no-context LLM answers side by side. Full pipeline in one tool.
- [Prompt Stability Analyzer](https://ia-qa.com/devtools/prompt-stability): Run the same prompt N times (3-20×), measure output variance with Jaccard similarity, length distribution, structural consistency. Get a 0-100 stability score and production-readiness verdict. Compare stability across models. Export reports. Catch unstable prompts before they hit production.
- [Prompt A/B Tester](https://ia-qa.com/devtools/prompt-ab-tester): Execute two prompt variants side by side on the same LLM(s). Compare latency, output tokens, response length. Multi-model mode ranks which variant wins across GPT-4o, Claude, Gemini. 4 preset experiments (concise vs detailed, JSON vs Markdown, persona vs none, low vs high temp). Export comparison reports.
- [Experiment Tracker](https://ia-qa.com/devtools/experiment-tracker): Central dashboard for all your AI testing runs. Every LLM comparison, A/B test, stability analysis, test suite, and RAG pipeline run is auto-saved. View timeline, filter by tool or model, compare any two runs side-by-side with metrics diff. Export history as JSON or CSV. Track progress, catch regressions, never lose a result.
- [CI/CD Export](https://ia-qa.com/devtools/cicd-export): Export LLM test suites as GitHub Actions YAML, Promptfoo configs, Node.js scripts, or Bash/cURL scripts. Configure models, assertions, cron schedules. Import test cases from experiment history. Automate prompt testing in your CI/CD pipeline.
- [Collections](https://ia-qa.com/devtools/collections): Organize prompts, test cases, configs, and notes into reusable collections. Export/import as JSON. Build a personal prompt library and share test suites across projects.
- [Eval Framework](https://ia-qa.com/devtools/eval-framework): Multi-criteria LLM output scoring with custom rubrics. 5 preset rubric templates (General Quality, Safety & Compliance, Structured Output, Creative Writing, Code Generation). Weighted scoring, side-by-side comparison, exportable evaluation reports.
- [Get Started Wizard](https://ia-qa.com/devtools/onboarding): Answer 4 quick questions about your role, goals, and models — get personalized tool recommendations. Discover the right AI testing tools for your workflow.
- [Pipeline Builder](https://ia-qa.com/devtools/pipeline-builder): Visual drag-and-drop pipeline builder to chain AI testing tools. 4 preset pipelines (Prompt→Test→Deploy, RAG Debug, Security Audit, Agent Validation). Features: zoom in/out (mouse wheel or buttons), pan the canvas (drag or arrow buttons), Fit-to-view button to auto-center all nodes, export pipeline as SVG file. Save custom pipelines, connect nodes with arrow edges, run tool sequences.
- [LLM Readability Analyzer](https://ia-qa.com/llm-readability): Score content 0-100 for AI citation-readiness across 6 dimensions (clarity, factuality, negations, structure, entities, citability)
- [LLM.txt Generator](https://ia-qa.com/llm-txt-generator): Generate a standards-compliant llm.txt file for any website following the llmstxt.org specification
- [GEO Optimizer](https://ia-qa.com/geo-optimizer): Generative Engine Optimization toolkit — 4 tools in one: (1) GEO Score: paste your <head> HTML, get a weighted 0-60 score with pass/fail for JSON-LD, FAQPage schema, lang="en", llm.txt link, canonical, Open Graph, and author attribution; (2) JSON-LD Builder: generate Schema.org structured data (WebSite, FAQPage, Article, SoftwareApplication, Person, Organization, HowTo); (3) Meta Tags Generator: generate a complete GEO-optimized <head> snippet with OG, Twitter Card, canonical, and llm.txt links; (4) robots.txt Auditor: check your robots.txt against 16 AI crawlers and get ready-to-paste fix snippets. 100% client-side.
- [LLM Cost Calculator](https://ia-qa.com/devtools/cost-calculator): Estimate API costs across OpenAI, Anthropic, Gemini, Mistral and more
- [Token Counter](https://ia-qa.com/devtools/token-counter): Count tokens for any LLM context window
- [Prompt Builder](https://ia-qa.com/devtools/prompt-builder): Structure system and user prompts with role, context, format and constraints sections
- [Prompt Library](https://ia-qa.com/devtools/prompt-library): Curated prompt templates for ChatGPT, Claude and Gemini
- [Prompt Engineering Toolsheets](https://ia-qa.com/devtools/prompt-toolsheets): Best practices guides and persona frameworks for prompt engineering
- [Output Risk Analyzer](https://ia-qa.com/devtools/risk-analyzer): Detect hallucination signals and risk patterns in LLM outputs
- [Agent Loop Detector](https://ia-qa.com/devtools/loop-detector): Identify infinite-loop risks in AI agent configurations
- [Agent Graph Generator](https://ia-qa.com/devtools/graph): Visualise LLM agent configurations as node-link diagrams
- [Explain This X](https://ia-qa.com/devtools/explain): Get plain-language explanations of JSON, regex, JWT, SQL and more
- [AI Recipes](https://ia-qa.com/devtools/ai-recipes): Code patterns and cookbook recipes for RAG, chain-of-thought, function calling, few-shot, and more
- [RAG Chunk Optimizer](https://ia-qa.com/devtools/rag-chunk-optimizer): Visualize and compare fixed, sentence, paragraph, and sliding-window chunking strategies with token counts
- [Agent Blueprint Designer](https://ia-qa.com/devtools/agent-blueprint): Design AI agent configurations (model, memory, tools, guardrails) and export JSON/YAML specs
- [LLM Eval Designer](https://ia-qa.com/devtools/llm-eval-designer): Design evaluation suites for LLM outputs with Promptfoo and Braintrust-compatible JSON export
- [MCP Registry](https://ia-qa.com/mcp-registry): Curated catalogue of 21 external MCP servers — GitHub MCP, Firecrawl, Brave Search, Filesystem, Git, Memory, Fetch, Time, Sequential Thinking, Figma (13.9k★), Notion (official), Stripe (official), 21st.dev, MiniMax, Obsidian, Shopify, Slack, Postgres, Puppeteer, Linear, Google Workspace — with QA scores (0–100), compatibility info for Cursor/Claude/Windsurf, pricing model, setup complexity, and one-click testing via IA-QA's MCP Server Tester. Filter by 12 categories (code, search, data, browser, files, productivity, design, AI & media, utility, testing, devops). Sort by QA score. Each server has a dedicated detail page with setup guide, install command, required env vars, and docs link.
- [Natural Language → Code](https://ia-qa.com/devtools/nl-to-code): Convert plain-English descriptions into clean, idiomatic code in 30+ languages (Python, JavaScript, TypeScript, Java, Go, Rust, C#, PHP, Bash, SQL, Terraform, Dockerfile and more). Backend choice: Hugging Face Inference API (free, no GPU, optional token) or Ollama (100% local, private). Supports streaming output, code explanation mode, context injection, and generation history. No data sent to ia-qa.com servers.
- [MCP Server Tester](https://ia-qa.com/devtools/mcp-server-tester): Test any MCP server via JSON-RPC 2.0 — discover tools, resources, prompts. When accessed from the MCP Registry via the "Test" button, automatically pre-fills the server URL and displays a contextual Setup Guide with the exact install command (npx, Docker, pip, or remote URL), required environment variables, and a link to official docs for that specific server.
- [Chain-of-Thought Visualizer](https://ia-qa.com/devtools/cot-visualizer): Visualize and debug LLM chain-of-thought reasoning in tree and timeline views
- [Tool Output Formatter](https://ia-qa.com/devtools/tool-output-formatter): Convert raw tool/function call output into LLM-optimized summaries
- [Chunking Simulator](https://ia-qa.com/devtools/chunking-simulator): Compare 7 RAG chunking strategies side-by-side with visual token analysis
- [Embedding Proximity Tester](https://ia-qa.com/devtools/embedding-proximity): Measure semantic similarity between text pairs using OpenAI embeddings
- [Markdown for LLM Converter](https://ia-qa.com/devtools/markdown-for-llm): Clean dirty HTML into LLM-friendly Markdown, strip scripts/ads/nav
- [System Prompt Leak Tester](https://ia-qa.com/devtools/prompt-leak-tester): Test system prompt resistance to 26 leak attacks across 6 categories
- [Few-Shot Generator](https://ia-qa.com/devtools/few-shot-generator): Generate structured few-shot examples in XML, JSON, or Markdown
- [Negative Prompting Helper](https://ia-qa.com/devtools/negative-prompting-helper): Browse and export prompt constraints — 30+ rules in 5 categories
- [llm.txt Validator](https://ia-qa.com/devtools/llm-txt-validator): Validate llm.txt files against the llmstxt.org specification
- [LLM API Playground](https://ia-qa.com/llm-playground): The DevTools for LLMs. Build multi-turn conversations with full parameter control (temperature, max_tokens, top_p, presence/frequency penalties, seed, stop sequences, JSON mode). Inspect the raw JSON request body live, view raw JSON responses, export to cURL or Python/requests. Supports all providers: OpenAI, Anthropic, Google Gemini, DeepSeek, xAI, HuggingFace, Ollama (local). History of last 50 requests saved locally. Presets: single turn, code review, multi-turn chat, JSON mode, red team.
- [Vector DB Tester](https://ia-qa.com/devtools/vector-db-tester): Direct REST API tester for vector databases. Supports Chroma (local/remote), Qdrant, Pinecone, and Weaviate. Operations: list collections/indexes, query with vector, upsert, retrieve, scroll, add objects, delete. Full request builder with method selector, path editor, JSON body editor. Shows raw response, status code, latency. Request history. Quick setup commands for each DB (Docker, pip). CORS tips for local instances.
- [LangChain Chain Tester](https://ia-qa.com/devtools/langchain-chain-tester): Multi-step LangChain prompt chain builder — each step's output becomes a variable for the next step. Supports presets (Bug Context, PR Review, Multi-Agent Handoff, LangSmith-style Eval), custom steps, and OpenAI/Anthropic/DeepSeek models. Auto-fills API key from active environment.
- [LangGraph Flow Debugger](https://ia-qa.com/devtools/langgraph-debugger): Stateful LangGraph-style graph executor — define nodes, conditional edges, and initial state. Traces each step, shows routing decisions, and visualizes the execution path. Multi-model support including DeepSeek R1 (reasoning).
- [Multimodal Eval Toolkit](https://ia-qa.com/devtools/multimodal-eval): VLM test suite runner and multimodal metrics reference. **VLM Test Runner** (primary tab): provide an image URL + test cases (question + assertion: contains, not_contains, is_json, min_length, max_length), choose a VLM (GPT-4o, Claude 3.5, Gemini 2.0 Flash), BYOK — get per-case PASS/FAIL verdicts, pass rate %, and an overall PASS/WARNING/FAIL verdict. 3 quick-start presets: object detection, hallucination probe, JSON output. CI/CD-ready via MCP tool: run_vlm_test_suite. Reference tabs: CLIP Score interpreter (0–100), FID interpreter (lower=better), VQA thresholds, readiness checklist. MCP tools: run_vlm_test_suite (executable), multimodal_eval_guide (reference).
- [Agent Debugger](https://ia-qa.com/devtools/agent-debugger): Universal agent trace analyzer — detect loops, hallucinated tools, token waste from OpenAI, LangChain, or text logs
- [LLM Fit Finder](https://ia-qa.com/devtools/llm-fit-finder): Find the best LLM for your use case and budget. Cloud tab: filter 29 models by use case (chat, code, reasoning, RAG, vision), feature flags (streaming, tool-calling, JSON mode, vision), cost simulator, sortable benchmark scores (MMLU, HumanEval). Local tab: pick VRAM, quantization (Q4/Q8/FP16), see all compatible models with ollama run commands. MCP tool: llm_fit_finder.
- [Local AI Scanner](https://ia-qa.com/devtools/local-scanner): Scan your machine to detect GPU, VRAM, and system specs via WebGL. Instantly see which local LLMs (Llama 3.3, Qwen3, Mistral, Phi-4, Gemma 3, DeepSeek-R1) can run on your exact hardware with Ollama. Supports NVIDIA RTX/GTX, AMD RX 6000-9000, Apple M1-M4 (all variants), Intel Arc, and integrated APUs. Shows ideal model recommendations per use case (chat, code, reasoning, RAG). 100% private — runs entirely in-browser.

## QA & Testing Tools

- [Site Auditor 2026](https://ia-qa.com/web-auditor): Lighthouse-powered performance, accessibility, SEO and Core Web Vitals audit (LCP, INP, CLS)
- [Stack Trace Analyzer](https://ia-qa.com/qatools/bug-context-extractor): Parse and analyze stack traces from Java, Python, JavaScript, C#, Ruby and more
- [Bug Severity Calculator](https://ia-qa.com/qatools/bug-severity-justifier): Auto-classify bug priority P0-P3 with detailed impact justification
- [Reproduction Steps Normalizer](https://ia-qa.com/qatools/reproduction-steps-normalizer): Convert raw bug reports to standardized Given-When-Then format
- [Test Case Matrix Generator](https://ia-qa.com/qatools/test-prompt-generator): Generate test cases using equivalence partitioning and boundary value analysis
- [Test Data Synthesizer](https://ia-qa.com/qatools/test-data-synthesizer): Generate realistic synthetic test data
- [Regression Risk Score](https://ia-qa.com/qatools/regression-risk-score): Score code changes for regression risk before releases
- [Regression Impact Predictor](https://ia-qa.com/qatools/regression-impact-predictor): Predict which areas are most at risk from a given change
- [Diff-Aware Bug Suggestion](https://ia-qa.com/qatools/diff-aware-bugs): Suggest likely bugs based on code diffs
- [Bug Knowledge Base](https://ia-qa.com/qatools/bug-knowledge-base): Local browser-based bug tracker with import/export
- [Bug Impact Mapper](https://ia-qa.com/qatools/bug-impact-mapper): Map a bug's blast radius across system components
- [Prompt Injection Tester](https://ia-qa.com/qatools/prompt-injection-tester): Test LLM systems against jailbreak and prompt injection attacks
- [LLM Consistency Checker](https://ia-qa.com/qatools/llm-consistency-checker): Detect output drift and hallucination across multiple LLM runs
- [Structured Output Validator](https://ia-qa.com/qatools/structured-output-validator): Validate JSON, CSV, XML and YAML against schemas
- [PR Review Lens](https://ia-qa.com/qatools/pr-review-lens): Analyze pull request diffs for test anti-patterns
- [PR Beautifier](https://ia-qa.com/qatools/pr-beautifier): Render git diffs with syntax highlighting for code review
- [Test Case Manager](https://ia-qa.com/qatools/test-case-manager): Browser-based test suite manager with pass/fail/blocked tracking
- [QA System Designer](https://ia-qa.com/qatools/qa-system-designer): Design end-to-end QA systems with coverage mapping, tool selection and team structure
- [PR Gate Orchestrator](https://ia-qa.com/qatools/pr-gate-orchestrator): Configure and visualize pull request quality gates (tests, coverage, lint, review)
- [Log Translator](https://ia-qa.com/qatools/log-translator): Parse and translate raw logs into structured, human-readable format
- [IA-QA Shield](https://ia-qa.com/qatools/shield): AI Guardrail Engine — run 6 parallel safety checks (hallucination, injection, toxicity, PII, guardrails, quality) on any LLM response and get a PASS/FIX/BLOCK verdict with 0-100 score
- [LLM Test Runner](https://ia-qa.com/qatools/eval-contract): YAML-driven LLM test suite runner — write .ia-eval.yaml scenarios with inputs, ground truths and expectations, run them against any LLM (Groq, OpenAI, Anthropic, Google), get per-scenario PASS/FAIL verdicts, aggregate score, radar chart and exportable HTML/Markdown reports. 10 built-in scorers: exact_match, contains_check, json_format, hallucination_check, toxicity_filter, llm_as_judge and more. MCP tools: list_local_tests, run_eval_contract, generate_html_report
- [Pre-flight Safety Check](https://ia-qa.com/qatools/preflight-safety): Scan any text, code, prompt, logs or git diff for PII, leaked secrets (API keys, tokens, private keys, connection strings), prompt injection patterns and guardrail violations — 100% regex-based, zero API keys, CI/CD-ready via POST /mcp/ci/safety-gate. MCP tools: redact_pii, prompt_injection_scan, secret_scan, guardrail_test
- [Web Security Auditor](https://ia-qa.com/qatools/security-audit): One-click security posture audit — HTTP security headers, SSL/TLS certificates, CORS misconfiguration, cookie security — with AI-powered vulnerability analysis and remediation roadmap. MCP tools: security_headers_check, ssl_certificate_check, cors_test, cookie_security_audit, web_security_audit
- [Cron Expression Validator](https://ia-qa.com/cron-validator): Validate cron syntax, explain each field and preview upcoming run times before shipping scheduled jobs. MCP tool: cron_validator
- [CORS Checker](https://ia-qa.com/cors-checker): Test Access-Control-Allow-* headers, simulate a browser origin and inspect preflight responses for CORS debugging. MCP tool: cors_checker
- [Webhook Tester](https://ia-qa.com/webhook-tester): Generate a temporary webhook endpoint, inspect inbound headers and payloads, and replay integrations with sample curl commands. MCP tools: webhook_endpoint_create, webhook_endpoint_requests
- [Storage Inspector](https://ia-qa.com/storage-inspector): Inspect and debug cookies, localStorage and sessionStorage
- [JWT Debugger](https://ia-qa.com/jwt-debugger): Decode and validate JWT tokens with signature verification
- [Diff Checker](https://ia-qa.com/diff-checker): Visual line-by-line text comparison using LCS algorithm

## Text & Data Tools

- [JSON Formatter](https://ia-qa.com/json-formatter): Format, validate, lint and minify JSON
- [CSV ↔ JSON Converter](https://ia-qa.com/csv-json-converter): Convert between CSV and JSON with RFC 4180 compliance
- [Base64 Encoder/Decoder](https://ia-qa.com/base64-encoder-decoder-new): Encode and decode Base64 including image data URIs
- [URL Encoder/Decoder](https://ia-qa.com/url-encoder-decoder-new): Encode and decode URLs with component-level parsing
- [Hash Generator](https://ia-qa.com/hash-generator): Generate MD5, SHA-1 and SHA-256 hashes client-side
- [XML Validator](https://ia-qa.com/xml-validator): Validate and format XML documents
- [JS Minifier](https://ia-qa.com/javascript-minifier): Minify and compress JavaScript code
- [SQL Query Builder](https://ia-qa.com/sql-query-builder): Build SQL SELECT, INSERT, UPDATE and DELETE queries visually
- [Regex Tester](https://ia-qa.com/regex-tester): Test regular expressions with real-time matching and flag support
- [Word Counter](https://ia-qa.com/word-counter): Count words, characters, sentences and estimate reading time
- [Lorem Ipsum Generator](https://ia-qa.com/lorem-ipsum-generator): Generate placeholder text in paragraphs, sentences or words
- [Slug Generator](https://ia-qa.com/slug-generator): Convert any string to a URL-friendly slug
- [Email Validator](https://ia-qa.com/email-validator): Validate email addresses with RFC 5322 compliance and batch support
- [Timezone Converter](https://ia-qa.com/timezone-converter): Convert times between IANA timezones with DST awareness
- [API Request Builder](https://ia-qa.com/api-request-builder): Build and export API requests as cURL, fetch or Axios
- [Text Compare](https://ia-qa.com/text-compare): Side-by-side text comparison

## Design & Generator Tools

- [Color Palette Generator](https://ia-qa.com/color-palette-generator): 9 harmony algorithms, WCAG 2.1 contrast checker, export to CSS/Tailwind/SCSS/JSON
- [Color Converter](https://ia-qa.com/color-converter): Convert between HEX, RGB, HSL, HSV and CMYK
- [QR Code Generator](https://ia-qa.com/qr-code-generator): Generate QR codes for URLs, vCards, WiFi credentials and plain text
- [Fake Data Generator](https://ia-qa.com/fake-data-generator): Generate realistic test data including IBAN, Luhn card numbers and addresses
- [UUID Generator](https://ia-qa.com/uuid-generator): Generate UUID v4 identifiers in bulk
- [Image Compressor](https://ia-qa.com/image-compressor): Compress and convert images to JPEG, PNG or WebP entirely client-side
- [Password Generator](https://ia-qa.com/password-generator): Generate secure random passwords with configurable rules
- [Name Generator](https://ia-qa.com/name-generator): Generate random names for testing and prototyping

## AI Agent Configuration Tools

- [AI Rules Generator](https://ia-qa.com/devtools/cursor-rules-generator): Generate AI context files for any coding assistant: CLAUDE.md (Claude Code), AGENTS.md (OpenAI Codex), .cursor/rules, .cursorrules, .github/copilot-instructions.md (GitHub Copilot), .windsurfrules (Windsurf), .clinerules (Cline / RooCode). Select your tech stack, define code style preferences, and pick AI behavior rules — outputs a ready-to-copy file. Input: project name, tech stack (React, TypeScript, Node.js, Python, etc.), naming conventions, AI behavior presets. Output: formatted rules file for any AI IDE.
- [Agent Generator](https://ia-qa.com/devtools/agent-generator): Generate `.agent.md` configuration files for VS Code Copilot agents. Define agent name, description, trigger phrase, tools (search, terminal, file I/O, code execution, etc.), context files, and behavior rules — outputs a ready-to-use `.agent.md` file to place in `.github/agents/`. Input: agent name, trigger phrase, model, tools, behavior steps. Output: structured `.agent.md` file. Requires VS Code ≥ 1.99 + GitHub Copilot extension.
- [Skill Generator](https://ia-qa.com/devtools/skill-generator): Generate `SKILL.md` files that teach AI IDEs a repeatable, domain-specific coding skill. Choose from 7 domains (Testing, Code Review, Documentation, Refactoring, Security Audit, Performance Optimization, CI/CD), pick your IDE (GitHub Copilot, Cursor, Windsurf, Continue, Claude Code), customize steps and output format — outputs a ready-to-place SKILL.md. Input: domain, IDE, skill name, steps, output format. Output: SKILL.md for `.github/skills/`, `.cursor/skills/`, or `.agents/skills/`.
- [Context Window Visualizer](https://ia-qa.com/devtools/context-window-visualizer): Visualize how prompt sections (system prompt, RAG docs, conversation history, user query) fill a model's context window. Supports GPT-4o (128k), Claude 3.7 Sonnet (200k), Gemini 2.0 Flash (1M), o3 (200k), and more. Shows stacked bar chart with per-section token counts and overflow warnings. Input: model selection + sections with content or manual token counts. Output: visual token budget breakdown.
- [Prompt Diff](https://ia-qa.com/devtools/prompt-diff): Compare two prompts side-by-side or in unified diff view. Shows added/removed lines, token delta, and structural analysis (detects added/removed sections like System, Context, Constraints, Examples, Format). Input: Prompt A + Prompt B. Output: colored line diff, token delta badge, structural change summary.
- [Agent Blueprint Designer](https://ia-qa.com/devtools/agent-blueprint): Design AI agent configurations (model, memory, tools, guardrails) and export JSON/YAML specs
- [LLM Eval Designer](https://ia-qa.com/devtools/llm-eval-designer): Design evaluation suites for LLM outputs with Promptfoo and Braintrust-compatible JSON export
- [IA-QA Config Generator](https://ia-qa.com/devtools/config-generator): Generate a downloadable ZIP of production-ready config files for your LLM project. Choose use case (chatbot, RAG pipeline, AI agent, classifier, QA assistant with test generation, extraction pipeline), LLM providers (OpenAI, DeepSeek, Anthropic, Mistral, Groq), vector DB (Qdrant, Pinecone, Weaviate, Chroma, Neo4j GraphRAG, pgvector, MongoDB), test framework (Vitest, Jest, Playwright, Pytest, Cypress), and AI IDE (Windsurf, Cursor, Copilot). Generates: CLAUDE.md, .windsurfrules, orchestrator.ts (or test-generator.ts for QA assistant), llm-tests.ts, rag.config.ts (RAG only), README.md. No sign-up. Instant ZIP download.

## Team Collaboration Tools

- [Assumption Revealer](https://ia-qa.com/devtools/assumptions): Surface hidden assumptions in project plans and requirements
- [Decision Record Lite](https://ia-qa.com/devtools/decision-record): Create lightweight Architecture Decision Records (ADR)
- [Friction Mapper](https://ia-qa.com/devtools/friction-mapper): Cluster and visualize team friction points
- [Signal Check](https://ia-qa.com/devtools/signal-check): Assess team health signals
- [Disagreement Finder](https://ia-qa.com/devtools/disagreement): Surface conflicting opinions in team discussions
- [Wheel Picker](https://ia-qa.com/devtools/wheel-picker): Random wheel selector for team decisions and retrospectives
- [Meeting Timer](https://ia-qa.com/timer): Timeboxed meeting timer with Pomodoro and countdown modes

## Reference & Toolsheets

- [QA Toolsheets](https://ia-qa.com/qa-toolsheets): Quick-reference cheat sheets for QA methodologies, test types, and best practices
- [Dev Toolsheets](https://ia-qa.com/devtools/dev-toolsheets): Quick-reference cheat sheets for developer tools, languages, and frameworks
- [DevOps Toolsheets](https://ia-qa.com/devtools/devops-toolsheets): Quick-reference cheat sheets for CI/CD, Docker, Kubernetes, and cloud infrastructure

## MCP Registry

- **URL**: https://ia-qa.com/mcp-registry
- **What is it**: A curated catalogue of 21 external Model Context Protocol (MCP) servers with QA scores, setup guides, and one-click testing.
- **21 catalogued servers** (as of 2026-03-24):
  - **CORE (official `modelcontextprotocol/servers`)**: Filesystem, Git, Memory, Fetch, Time, Sequential Thinking
  - **POPULAR third-party**: GitHub MCP, Firecrawl, Brave Search, Postgres, Puppeteer, Slack, Obsidian, Shopify
  - **HIGH VALUE**: Figma (Framelink, 13.9k★), 21st.dev Magic (4.5k★), MiniMax (TTS/video/image), Notion (official, 4.1k★), Stripe (official, remote MCP), Linear (remote MCP), Google Workspace (Gmail+Calendar+Drive)
- **QA Scores**: Each server has a QA score (0–100) based on reliability, output quality, schema compliance, and latency — curated by IA-QA testing
- **Filters**: by 12 categories (code, search, data, browser, files, productivity, design, AI & media, utility, testing, devops), pricing model (free/freemium/api_key/paid), open-source flag, sort by QA score or setup complexity
- **Compatibility info**: shows which servers work with Cursor IDE, Claude Desktop, and Windsurf
- **"Test with IA-QA" button**: links directly to the MCP Server Tester with pre-filled URL and contextual setup guide (install command, env vars, docs link)
- **Detail pages**: each server has a dedicated page with full description, use cases, QA metrics breakdown, compatibility badges, setup instructions, and external links

## MCP (Model Context Protocol) API

> Developers and AI agents can call ia-qa.com tools programmatically via the MCP API.

- Manifest (slim): `GET https://ia-qa.com/mcp` — machine-readable tool catalogue (name + description + category + schema_url). Lightweight by default, ~4k tokens.
- Manifest (full):  `GET https://ia-qa.com/mcp?full=1` — includes all inputSchemas (complete, ~80k tokens).
- Tool schema:      `GET https://ia-qa.com/mcp/tool/{name}` — full inputSchema + annotations for a single tool.
- Tool search:      `GET https://ia-qa.com/mcp/search?q={keyword}&category={cat}` — keyword/category search, returns slim list.
- JSON-RPC:         `POST https://ia-qa.com/mcp` — tools/list and tools/call via JSON-RPC 2.0.
- REST call:        `POST https://ia-qa.com/mcp/call` — call any tool with `{ "name": "tool_name", "parameters": {...} }`.

### Recommended Agent Workflow (avoid context saturation)
1. Call `find_tool` MCP tool with a keyword → get matching tools (slim).
2. Call `GET /mcp/tool/{name}` → get full inputSchema for the chosen tool.
3. Call `POST /mcp` (JSON-RPC) or `POST /mcp/call` (REST) to execute.

### Tool Hierarchy — 148 tools across 12 categories

#### data (16 tools)
format_json, parse_csv, flatten_json, xml_to_json, transform_json_array
json_to_csv, json_diff, merge_json, json_to_yaml, json_schema_validate
mock_from_schema, extract_json_path, generate_json_ld, yaml_to_json, json_schema_generate
format_table
> Structured data transformation, validation, mock generation

#### encoding (10 tools)
base64_encode, base64_decode, url_encode, url_decode, hash_text
generate_hmac, generate_uuid, generate_password, number_base_convert, color_convert
> Encoding, hashing, cryptographic utilities

#### text (15 tools)
text_stats, strip_markdown, normalize_whitespace, sort_lines, word_frequency
extract_links, levenshtein_distance, case_convert, lorem_ipsum, detect_language
escape_html, unescape_html, html_to_markdown, regex_test, calculate_readability
> Text manipulation, NLP utilities, content processing

#### llm (13 tools)
count_tokens, truncate_to_tokens, split_chunks, extract_json_from_text, estimate_llm_cost
token_budget_calculator, context_window_check, model_info, compare_models, list_llm_models
llm_generate, optimize_prompt_tokens, llm_fit_finder
> Token budgeting, model selection, LLM I/O preprocessing

#### qa (29 tools)
diff_text, llm_output_validator, compare_responses, analyze_responses, prompt_test_suite
validate_mcp_response, mcp_server_health_check, mcp_server_evaluate, hallucination_check, prompt_injection_scan
consistency_check, llm_format_check, llm_json_schema_check, response_quality_score, toxicity_scan
guardrail_test, function_call_validate, conversation_analyze, mcp_schema_lint, cot_analyzer
ab_test_report, analyze_diff_bugs, generate_test_cases, run_pr_gate_pipeline, run_vlm_test_suite
run_vlm_test_suite_batch, multimodal_eval_guide, similarity_score, bias_detect
> LLM output validation, AI safety, test generation, VLM testing, multimodal metrics

#### rag (12 tools)
embedding_similarity, vector_similarity, normalize_vector, vector_quantize, vector_stats
bm25_score, build_rag_prompt, prompt_template_fill, few_shot_formatter, system_prompt_builder
rag_relevance_rank, rerank_evaluate
> RAG pipelines, vector operations, prompt engineering

#### dev (21 tools)
validate_email, validate_url, decode_jwt, minify_js, cron_parse
cron_validator, timestamp_convert, http_status_lookup, parse_http_headers, generate_curl
extract_todos, detect_secrets, count_code_lines, lint_commit_message, format_bytes
pr_gatekeeper, check_contrast_ratio, webhook_endpoint_create, webhook_endpoint_requests, env_parse
openapi_validate
> Developer utilities, HTTP debugging, code quality

#### security (9 tools)
security_headers_check, ssl_certificate_check, cors_test, cors_checker, cookie_security_audit
web_security_audit, shield_analyze, secret_scan, redact_pii
> Security audits, CORS debugging, secret/PII detection

#### web (4 tools)
fetch_veille_feed, score_geo_signals, latency_benchmark, generate_slug
> Web performance, SEO/GEO, content utilities

#### meta (3 tools)
find_tool, identify_caller, rate_tool
> Search the tool catalogue by keyword or category

#### eval (9 tools)
list_local_tests, run_eval_contract, generate_html_report, run_semantic_tests, generate_eval_yaml
validate_agent_trajectory, test_skill, get_testing_guidelines, needle_haystack_generate
> YAML-driven LLM test contracts, eval reporting, agent trajectory validation

#### integrations (7 tools)
fetch_jira_issue, search_jira_issues, fetch_confluence_page, jira_to_test_suite, fix_gherkin
post_jira_comment, create_confluence_page
> Jira & Confluence integration, ticket-to-test-suite generation

## LLM Inference API

> Direct multi-provider LLM inference routes (requires your own API key for the target provider).

- `POST /api/llm/chat` — Unified chat endpoint. Body: `{ model, messages, apiKey, temperature?, maxTokens? }`. Auto-detects provider from model ID. Supported providers: OpenAI, Anthropic, DeepSeek, Google (Gemini), xAI (Grok), Ollama. Returns: `{ content, usage: { input, output }, model, provider, latency_ms }`.
- `POST /api/llm/compare` — Parallel multi-model comparison. Body: `{ runs: [{model, apiKey}], messages }`. Runs all models simultaneously. Returns: `{ results: [...] }`. Max 8 models per request.
- `GET /api/llm/providers` — List all supported providers and their model IDs. No auth required.

Supported model IDs (as of 2026-05-15):
- **OpenAI**: `gpt-4o`, `gpt-4o-mini`, `o1`, `o1-mini`, `o3-mini`
- **Anthropic**: `claude-opus-4-7`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`, `claude-3-7-sonnet-20250219`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
- **DeepSeek**: `deepseek-chat` (V3), `deepseek-reasoner` (R1)
- **Google**: `gemini-2.0-flash`, `gemini-2.0-flash-lite`, `gemini-1.5-pro`, `gemini-1.5-flash`
- **xAI**: `grok-2-1212`, `grok-2-vision-1212`
- **Ollama (local)**: `llama3.3`, `qwen2.5-coder:32b`, `mistral` (no API key required)

- [MCP Server Documentation](https://ia-qa.com/mcp-server): Full documentation and live playground — integrate IA-QA tools into Cursor IDE, Claude Desktop, Windsurf, or any AI agent
- [MCP Explorer + Free Chat](https://ia-qa.com/devtools/mcp-explorer): Interactive live playground to browse all MCP tools, read their schemas, and execute test calls from the browser. Includes a BYOK LLM Runner (OpenAI, Anthropic, Gemini, DeepSeek) and a free no-key-required chat powered by server-side Groq and HuggingFace inference: Qwen3 32B/235B, Llama 4 Scout, Llama 3.3 70B, DeepSeek R1 distill, Mistral Small 24B. No setup required.

## Resources & Articles

> Curated by Jean-Christophe Jamet, QA Engineering Manager. Covers QA, AI/LLMs, automation, DevOps and engineering leadership.

- [Resources & Articles](https://ia-qa.com/resources): 50+ curated links and articles on QA engineering, LLM testing, test automation (Cypress, Playwright, Selenium, Cucumber, Postman, k6), CI/CD (Jenkins, GitHub Actions, GitLab), Docker, Kubernetes, Monitoring (Grafana, Prometheus, OpenTelemetry), Dev Practices and Dev Tools — plus original articles by the author on Medium.
- [QA & AI Live Feed](https://ia-qa.com/veille): Daily-refreshed RSS aggregator covering QA, AI, and LLM articles. Sources: Google Testing Blog, DEV.to (testing/qa/ai/llm/agents channels), Hugging Face Blog, Simon Willison's blog, and the author's Medium feed. Filterable by category (all / qa / ai / mine).