-
Notifications
You must be signed in to change notification settings - Fork 14
Description
I've been playing with building a Claude Code environment for government guidance content design.
I've been using a little fetcher that it made to grab legislation and turn it into markdown. This is so it can check guidance accuracy against the law.
I thought I'd try integrating lex instead. It worked, but seems to choke on some point at a complex query.
I should say I'm an amateur - a government content designer hoping to make something for the community - so I may be steering Claude to do something stupid. But I asked it to log the problem, and I've pasted it below.
MCP lex Server Testing Log
Date: 2026-01-16
Server: https://lex.lab.i.ai.gov.uk/mcp
Purpose: Testing MCP integration for legislation lookups in GOV.UK editing workflow
Test Results Summary
| Test | Status | Response Time |
|---|---|---|
| Health check | ✅ Pass | ~1-2s |
| Stats | ✅ Pass | ~1-2s |
| Lookup (DPA 2018) | ✅ Pass | ~1-2s |
| Search Acts (product safety) | ✅ Pass | ~1-2s |
| Get sections (uksi/2024/1332) | ✅ Pass | ~1-2s |
| Search explanatory notes | ✅ Pass | ~1-2s |
| Search amendments | ✅ Pass (empty result) | ~1-2s |
| Search sections (group registration) | ✅ Pass | ~1-2s |
| Search sections (holding company) | ✅ Pass | ~1-2s |
| Search sections (registration fee) | ✅ Pass | ~1-2s |
| Search sections (compliance scheme) | ✅ Pass | ~1-2s |
| Search sections (28 days notify) | ✅ Pass | ~1-2s |
| Search sections (complex query) | ❌ Timeout | AbortError |
| Get sections (retry) | ❌ Timeout | AbortError |
Total calls before failure: ~12-13
Published rate limit: 60 requests/min, 1000/hour per IP
Detailed Test Log
1. Health Check
Tool: mcp__lex__health_check_healthcheck_get
Result:
{
"status": "healthy",
"database": "qdrant",
"collections": 8,
"collection_details": {
"caselaw_section": {"points": 4723735, "status": "green"},
"explanatory_note": {"points": 83527, "status": "green"},
"legislation_section": {"points": 2096731, "status": "green"},
"caselaw_summary": {"points": 61107, "status": "green"},
"embedding_cache": {"points": 238600, "status": "green"},
"amendment": {"points": 892210, "status": "green"},
"legislation": {"points": 219685, "status": "green"},
"caselaw": {"points": 69970, "status": "green"}
}
}
2. Stats
Tool: mcp__lex__get_live_stats_api_stats_get
Result:
{
"acts_and_sis": "219,685",
"provisions": "2,096,731",
"explanatory_sections": "83,527",
"amendments": "892,210",
"pdf_legislation": "94,419",
"pdf_provisions": "1,086,007",
"last_updated": "12:45 UTC"
}
3. Lookup Legislation (DPA 2018)
Tool: mcp__lex__lookup_legislation
Params: legislation_type=ukpga, year=2018, number=12
Result: Successfully returned metadata for Data Protection Act 2018
- 1176 provisions
- Status: revised
- Extent: United Kingdom
4. Search Acts (Product Safety)
Tool: mcp__lex__search_for_legislation_acts
Params: query="product safety consumer protection", limit=5, include_text=false
Result: Returned 5 results including:
- General Product Safety Regulations 2005 (uksi/2005/1803)
- General Product Safety Regulations 1994 (uksi/1994/2328)
- EU Directive 2001/95/EC
- Recreational Craft Regulations 2017
- Consumer Safety Act 1978 Commencement Order
5. Get Legislation Sections
Tool: mcp__lex__get_legislation_sections
Params: legislation_id=uksi/2024/1332, limit=5
Result: Successfully returned 5 sections with full text:
- Regulation 2 (Interpretation)
- Schedule 4 (Producer reporting information)
- Regulation 30 (Determination of applications)
- Regulation 48 (Conditions of approval)
- Regulation 58 (Scheme administrator)
6. Search Explanatory Notes
Tool: mcp__lex__search_explanatory_note
Params: query="product safety market surveillance", size=3
Result: Returned 3 explanatory notes including:
- Section 18 interpretation (ukpga/2020/29)
- Schedule 11 construction products (ukpga/2022/30)
- Section 146 construction products (ukpga/2022/30)
7. Search Amendments
Tool: mcp__lex__search_amendments
Params: legislation_id=uksi/2005/1803, size=5
Result: Empty array (no amendments found for GPSR 2005)
Note: May be data gap or genuinely no amendments indexed
8-12. Multiple Section Searches (all successful)
Various searches against uksi/2024/1332 for:
- Group registration / holding company / subsidiary
- UK establishment requirements
- Registration fees
- Compliance scheme exemptions
- 28-day notification requirements
All returned relevant provisions from Schedule 9 (Corporate Groups) and Schedule 1 (Charges).
13. Failed Query
Tool: mcp__lex__search_for_legislation_sections
Params: query="compliance scheme obligation exempt registration reporting recycling not disposal", legislation_id=uksi/2024/1332, size=3
Error: MCP error -32001: AbortError: The operation was aborted.
14. Retry (also failed)
Tool: mcp__lex__get_legislation_sections
Params: legislation_id=uksi/2024/1332, limit=3
Error: MCP error -32001: AbortError: The operation was aborted.
Error Analysis
The AbortError: The operation was aborted error occurred after ~12-13 successful requests over approximately 10-15 minutes. This is well below the stated rate limits.
Possible causes:
- Server-side timeout on query processing
- Network interruption
- Server load/resource constraints
- Connection pool exhaustion
Not likely rate limiting because:
- Error message doesn't indicate rate limit (no 429, no "too many requests")
- Request count was ~13, far below 60/min limit
- Simple retry query also failed immediately
Overall Assessment
The MCP works well when it works:
- Fast responses (~1-2s)
- Good semantic search results
- Clean markdown output
- Explanatory notes very useful
- Section-level retrieval excellent for accuracy checking
Issues:
- Intermittent timeout/abort errors
- No graceful degradation or retry logic apparent
- Empty amendments result for GPSR 2005 (data gap?)
Recommendation: Suitable for production use with fallback to local files or fetch-legislation.js when MCP is unavailable.
Follow-up Test: Native HTTP Transport
After the initial failures, we tested whether the mcp-remote proxy was the issue by switching to Claude Code's native HTTP transport:
claude mcp add --transport http lex https://lex.lab.i.ai.gov.uk/mcpResults
| Test | Transport | Status |
|---|---|---|
| Health check | mcp-remote (npx) | ❌ AbortError |
| Get sections | mcp-remote (npx) | ❌ AbortError |
| Health check | Native HTTP | ❌ AbortError |
| Get sections | Native HTTP | ❌ AbortError |
Conclusion: Same AbortError: The operation was aborted with both transport methods. This rules out mcp-remote as the cause — the problem is server-side.
Configuration
.mcp.json:
{
"mcpServers": {
"lex": {
"command": "npx",
"args": [
"-y",
"mcp-remote@latest",
"https://lex.lab.i.ai.gov.uk/mcp"
]
}
}
}Log generated by Claude Code during MCP testing session