{"manifest":{"name":"API Tester","version":"1.0.0","description":"Parse HTTP examples from skill markdown, execute requests, validate responses, test error paths. For API-type skills.","tags":["verification","api","tester","meta","toolkit"],"standard":"agentskills.io","standard_version":"1.0","content_checksum":"98a0b028802826c8f3cb8dc8a48609b8658e3083a8bb4ba13328b1a12ef1e23c","bundle_checksum":null,"metadata":{},"files":[]},"files":{"SKILL.md":"# API Tester — Verification Toolkit\n\n> **Purpose:** Teach an AI agent how to test API-based skills by parsing HTTP examples from markdown, executing requests, and validating responses.\n> Optional third step in the verification pipeline for `api_workflow` skills.\n\n---\n\n## 1. Overview\n\nThe API Tester extracts HTTP examples from a skill's markdown content, executes them against the target API, and validates that responses match expected patterns. It produces an execution trace with detailed results.\n\nThis skill is only relevant when the **Skill Classifier** identifies a skill as `api_workflow` with `api_access: true`.\n\n---\n\n## 2. Input\n\nThe API Tester receives:\n\n1. **Skill content** — markdown with HTTP examples\n2. **Base URL** — the API endpoint to test against\n3. **Auth token** — Bearer token for authenticated requests\n4. **Classification** — from the Skill Classifier (confirms this is an API skill)\n\n---\n\n## 3. HTTP Example Extraction\n\n### 3a. Parsing HTTP Blocks\n\nLook for code blocks tagged with `http`:\n\n````markdown\n```http\nGET /api/skills?status=active&limit=5\nAuthorization: Bearer \n```\n````\n\nParse these into structured requests:\n\n```json\n{\n \"method\": \"GET\",\n \"path\": \"/api/skills\",\n \"query\": { \"status\": \"active\", \"limit\": \"5\" },\n \"headers\": { \"Authorization\": \"Bearer \" },\n \"body\": null\n}\n```\n\n### 3b. Parsing Response Examples\n\nLook for JSON response blocks immediately following HTTP blocks:\n\n````markdown\n```json\n{\n \"skills\": [...],\n \"total\": 42\n}\n```\n````\n\nThese define the expected response shape for validation.\n\n### 3c. Common Patterns\n\n| Pattern | Parsing Rule |\n|---------|-------------|\n| `GET /path` | No body expected |\n| `POST /path` + JSON block | Body is the JSON block |\n| `Authorization: Bearer ` | Replace `` with actual token |\n| `Content-Type: application/json` | Body should be parsed as JSON |\n\n---\n\n## 4. Request Execution\n\n### 4a. Safety Rules\n\nBefore executing any request:\n\n1. **Never execute destructive operations** (DELETE, DROP) against production endpoints\n2. **Only execute GET requests** unless the test environment is confirmed as safe\n3. **Replace placeholder tokens** with actual auth tokens\n4. **Respect rate limits** — add delays between requests, handle 429s\n5. **Do not send real credentials** in test requests\n\n### 4b. Execution Steps\n\nFor each extracted HTTP example:\n\n1. Build the full URL: `{base_url}{path}`\n2. Set headers (replace `` placeholders)\n3. Set body if applicable\n4. Execute the request\n5. Record timing and response\n\n### 4c. Recording Results\n\nFor each request, create trace steps:\n\n```json\n[\n {\n \"type\": \"api_request\",\n \"timestamp\": \"2026-02-09T...\",\n \"method\": \"GET\",\n \"url\": \"https://example.com/api/skills\",\n \"headers\": { \"Authorization\": \"Bearer ey...\" }\n },\n {\n \"type\": \"api_response\",\n \"timestamp\": \"2026-02-09T...\",\n \"status\": 200,\n \"body_preview\": \"{\\\"skills\\\": [...], \\\"total\\\": 42}\"\n }\n]\n```\n\n---\n\n## 5. Response Validation\n\n### 5a. Status Code Validation\n\n| Expected | Acceptable | Notes |\n|----------|-----------|-------|\n| 200 | 200 | Exact match required |\n| 201 | 201 | Exact match required |\n| 4xx | 400-499 | Any client error is valid if documented |\n| Not specified | 200-299 | Assume success range |\n\n### 5b. Shape Validation\n\nCompare the response body against the expected shape:\n\n1. **Field presence** — all documented fields should exist\n2. **Field types** — types should match (string, number, array, object)\n3. **Array contents** — if expected shows an array, actual should too\n4. **Nested objects** — recursively validate structure\n\nRecord an assertion for each validation:\n\n```json\n{\n \"type\": \"assertion\",\n \"timestamp\": \"2026-02-09T...\",\n \"description\": \"Response contains 'skills' array\",\n \"passed\": true,\n \"expected\": \"array\",\n \"actual\": \"array (5 items)\"\n}\n```\n\n### 5c. Error Path Testing\n\nIf the skill documents error responses:\n\n1. Intentionally trigger the error (e.g., send invalid data)\n2. Verify the error response matches the documented format\n3. Verify the status code matches\n\n---\n\n## 6. Output Format\n\nThe API Tester produces an execution trace:\n\n```json\n{\n \"version\": \"1.0\",\n \"started_at\": \"2026-02-09T...\",\n \"completed_at\": \"2026-02-09T...\",\n \"steps\": [\n { \"type\": \"info\", \"message\": \"Extracted 5 HTTP examples from skill\" },\n { \"type\": \"api_request\", \"method\": \"GET\", \"url\": \"...\" },\n { \"type\": \"api_response\", \"status\": 200, ... },\n { \"type\": \"assertion\", \"description\": \"Status 200\", \"passed\": true },\n { \"type\": \"assertion\", \"description\": \"Response has 'skills' field\", \"passed\": true },\n ...\n ],\n \"summary\": \"5/5 API tests passed\"\n}\n```\n\n---\n\n## 7. Scoring\n\nAPI test results contribute to the quality analysis scores:\n\n| Dimension | Impact |\n|-----------|--------|\n| Executability | +0.1 if all tests pass, -0.1 per failure |\n| Completeness | +0.05 if error paths are tested |\n| Quality | +0.05 if response shapes match exactly |\n\n---\n\n## 8. Limitations\n\n- Only tests HTTP-based APIs (no GraphQL, gRPC, WebSocket)\n- Cannot test endpoints requiring complex authentication flows (OAuth dance)\n- Cannot test endpoints that require specific server state\n- Rate limit handling is best-effort — may not work with all APIs\n- Does not test idempotency or concurrency\n\n---\n\n## 9. Integration\n\nThis tester's trace feeds into:\n- The **Skill Verifier** orchestrator (as part of the execution_trace)\n- The overall executability and quality scores\n- The verification record's `execution_trace` field\n\n## Playground\n\n
API TesterExample · SkillSlap
Input: Endpoint + expected
POST /api/auth/login\nBody: {\n  \"email\": \"user@example.com\",\n  \"password\": \"correct_password\"\n}\nExpected: 200, body.token exists
Output: Test result
✅ PASS  POST /api/auth/login\n\nStatus:  200 OK  (143 ms)\nHeaders: Content-Type: application/json\n\nBody:\n{\n  \"token\": \"eyJhbGci...\"  ✅\n  \"expires_at\": \"2026-...\"  ✅\n  \"user_id\": \"abc123\"\n}\n\nAssertions:\n  ✅ status === 200\n  ✅ body.token is string (length 187)\n  ✅ body.expires_at is ISO timestamp
"}}