# API Tester

Parse HTTP examples from skill markdown, execute requests, validate responses, test error paths. For API-type skills.

## Quick Reference

# API Tester — Verification Toolkit

> **Purpose:** Teach an AI agent how to test API-based skills by parsing HTTP examples from markdown, executing requests, and validating responses.
> Optional third step in the verification pipeline for `api_workflow` skills.

---

## 1. Overview

The API Tester extracts HTTP examples from a skill's markdown content, executes them against the target API, and validates that responses match expected patterns. It produces an execution trace with detailed results.

This skill is only relevant when the **Skill Classifier** identifies a skill as `api_workflow` with `api_access: true`.

---

## 2. Input

The API Tester receives:

1. **Skill content** — markdown with HTTP examples
2. **Base URL** — the API endpoint to test against
3. **Auth token** — Bearer token for authenticated requests
4. **Classification** — from the Skill Classifier (confirms this is an API skill)

---

## 3. HTTP Example Extraction

### 3a. Parsing HTTP Blocks

Look for code blocks tagged with `http`:

````markdown
```http
GET /api/skills?status=active&limit=5
Authorization: Bearer <token>
```
````

Parse these into structured requests:

```json
{
  "method": "GET",
  "path": "/api/skills",
  "query": { "status": "active", "limit": "5" },
  "headers": { "Authorization": "Bearer <token>" },
  "body": null
}
```

### 3b. Parsing Response Examples

Look for JSON response blocks immediately following HTTP blocks:

````markdown
```json
{
  "skills": [...],
  "total": 42
}
```
````

These define the expected response shape for validation.

### 3c. Common Patterns

| Pattern | Parsing Rule |
|---------|-------------|
| `GET /path` | No body expected |
| `POST /path` + JSON block | Body is the JSON block |
| `Authorization: Bearer <token>` | Replace `<token>` with actual token |
| `Content-Type: application/json` | Body should be parsed as JSON |

---

## 4. Request Execution

### 4a. Safety Rules

Before executing any request:

1. **Never execute destructive operations** (DELETE, DROP) against production endpoints
2. **Only execute GET requests** unless the test environment is confirmed as safe
3. **Replace placeholder tokens** with actual auth tokens
4. **Respect rate limits** — add delays between requests, handle 429s
5. **Do not send real credentials** in test requests

### 4b. Execution Steps

For each extracted HTTP example:

1. Build the full URL: `{base_url}{path}`
2. Set headers (replace `<token>` placeholders)
3. Set body if applicable
4. Execute the request
5. Record timing and response

### 4c. Recording Results

For each request, create trace steps:

```json
[
  {
    "type": "api_request",
    "timestamp": "2026-02-09T...",
    "method": "GET",
    "url": "https://example.com/api/skills",
    "headers": { "Authorization": "Bearer ey..." }
  },
  {
    "type": "api_response",
    "timestamp": "2026-02-09T...",
    "status": 200,
    "body_preview": "{\"skills\": [...], \"total\": 42}"
  }
]
```

---

## 5. Response Validation

### 5a. Status Code Validation

| Expected | Acceptable | Notes |
|----------|-----------|-------|
| 200 | 200 | Exact match required |
| 201 | 201 | Exact match required |
| 4xx | 400-499 | Any client error is valid if documented |
| Not specified | 200-299 | Assume success range |

### 5b. Shape Validation

Compare the response body against the expected shape:

1. **Field presence** — all documented fields should exist
2. **Field types** — types should match (string, number, array, object)
3. **Array contents** — if expected shows an array, actual should too
4. **Nested objects** — recursively validate structure

Record an assertion for each validation:

```json
{
  "type": "assertion",
  "timestamp": "2026-02-09T...",
  "description": "Response contains 'skills' array",
  "passed": true,
  "expected": "array",
  "actual": "array (5 items)"
}
```

### 5c. Error Path Testing

If the skill documents error responses:

1. Intentionally trigger the error (e.g., send invalid data)
2. Verify the error response matches the documented format
3. Verify the status code matches

---

## 6. Output Format

The API Tester produces an execution trace:

```json
{
  "version": "1.0",
  "started_at": "2026-02-09T...",
  "completed_at": "2026-02-09T...",
  "steps": [
    { "type": "info", "message": "Extracted 5 HTTP examples from skill" },
    { "type": "api_request", "method": "GET", "url": "..." },
    { "type": "api_response", "status": 200, ... },
    { "type": "assertion", "description": "Status 200", "passed": true },
    { "type": "assertion", "description": "Response has 'skills' field", "passed": true },
    ...
  ],
  "summary": "5/5 API tests passed"
}
```

---

## 7. Scoring

API test results contribute to the quality analysis scores:

| Dimension | Impact |
|-----------|--------|
| Executability | +0.1 if all tests pass, -0.1 per failure |
| Completeness | +0.05 if error paths are tested |
| Quality | +0.05 if response shapes match exactly |

---

## 8. Limitations

- Only tests HTTP-based APIs (no GraphQL, gRPC, WebSocket)
- Cannot test endpoints requiring complex authentication flows (OAuth dance)
- Cannot test endpoints that require specific server state
- Rate limit handling is best-effort — may not work with all APIs
- Does not test idempotency or concurrency

---

## 9. Integration

This tester's trace feeds into:
- The **Skill Verifier** orchestrator (as part of the execution_trace)
- The overall executability and quality scores
- The verification record's `execution_trace` field

## Playground

<!DOCTYPE html><html><head><meta charset='utf-8'><style>*{box-sizing:border-box;margin:0;padding:0}body{background:#0d1117;color:#e6edf3;font-family:monospace;font-size:12px;height:100vh;display:flex;flex-direction:column;overflow:hidden}.header{background:#161b22;border-bottom:1px solid #30363d;padding:8px 14px;font-size:11px;color:#8b949e;display:flex;justify-content:space-between;align-items:center;flex-shrink:0}.title{color:#58a6ff;font-weight:bold;font-size:13px}.panels{display:flex;flex:1;overflow:hidden}.panel{flex:1;overflow:auto;padding:12px;border-right:1px solid #30363d}.panel:last-child{border-right:none}.label{font-size:10px;color:#8b949e;text-transform:uppercase;letter-spacing:.08em;margin-bottom:6px}pre{white-space:pre-wrap;word-break:break-word;line-height:1.5}</style></head><body><div class='header'><span class='title'>API Tester</span><span>Example · SkillSlap</span></div><div class='panels'><div class='panel'><div class='label'>Input: Endpoint + expected</div><pre><span style='color:#58a6ff'>POST</span> <span style='color:#8b949e'>/api/auth/login</span>
<span style='color:#8b949e'>Body: {</span>
<span style='color:#8b949e'>  "email": "user@example.com",</span>
<span style='color:#8b949e'>  "password": "correct_password"</span>
<span style='color:#8b949e'>}</span>
<span style='color:#8b949e'>Expected: 200, body.token exists</span></pre></div><div class='panel'><div class='label'>Output: Test result</div><pre><span style='color:#3fb950'>✅ PASS  POST /api/auth/login</span>

<span style='color:#8b949e'>Status:  200 OK  (143 ms)</span>
<span style='color:#8b949e'>Headers: Content-Type: application/json</span>

<span style='color:#8b949e'>Body:</span>
<span style='color:#8b949e'>{</span>
<span style='color:#8b949e'>  "token": "eyJhbGci..."  </span><span style='color:#3fb950'>✅</span>
<span style='color:#8b949e'>  "expires_at": "2026-..."  </span><span style='color:#3fb950'>✅</span>
<span style='color:#8b949e'>  "user_id": "abc123"</span>
<span style='color:#8b949e'>}</span>

<span style='color:#3fb950'>Assertions:</span>
<span style='color:#3fb950'>  ✅ status === 200</span>
<span style='color:#3fb950'>  ✅ body.token is string (length 187)</span>
<span style='color:#3fb950'>  ✅ body.expires_at is ISO timestamp</span></pre></div></div></body></html>