Server data from the Official MCP Registry
Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.
About
Continuous behavioral drift monitoring for LLM apps — catches silent provider model updates.
Security Report
Valid MCP server (3 strong, 4 medium validity signals). No known CVEs in dependencies. Package registry verified. Imported from the Official MCP Registry.
8 files analyzed · 1 issue found
Security scores are indicators to help you make informed decisions, not guarantees. Always review permissions before connecting any MCP server.
Permissions Required
This plugin requests these system permissions. Most are normal for its category.
What You'll Need
Set these up before or after installing:
Environment variable: MODELWATCH_API_KEY
Environment variable: MODELWATCH_API_BASE
How to Install
Add this to your MCP configuration file:
{
"mcpServers": {
"io-github-bch1212-modelwatch": {
"env": {
"MODELWATCH_API_KEY": "your-modelwatch-api-key-here",
"MODELWATCH_API_BASE": "your-modelwatch-api-base-here"
},
"args": [
"-y",
"modelwatch-mcp"
],
"command": "npx"
}
}
}Documentation
View on GitHubFrom the project's GitHub README.
ModelWatch
Continuous behavioral drift monitoring for LLM-powered applications. Define a spec (input and expected behavior), ModelWatch replays it on a schedule against your model endpoint, and alerts when the output drifts from the baseline.
Customers bring their own OpenAI or Anthropic key. Auth is a workspace key minted at POST /api/auth/signup (Authorization: Bearer mw_...).
- API:
https://api.modelwatch.app - App:
https://modelwatch.app - HTTP reference:
frontend/docs.html
pip install -r requirements.txt
pytest tests/ -q
Tool monitoring
ModelWatch also watches the tools an agent calls. A decorator in your process records latency, success, and a fingerprint of the response shape. The API compares that fingerprint with the dominant shape from the last 24 hours and alerts on schema drift. Response values are not uploaded.
pip install -e sdk
export MODELWATCH_API_KEY=mw_...
from modelwatch import monitor
@monitor(tool_name="search_web", agent_id="researcher")
async def search_web(query: str) -> dict:
...
Calls go to https://api.modelwatch.app/api/tools/ingest. Read recent calls at GET /api/tools and schema changes at GET /api/tools/schema-drift. The same mw_ key and the same email/Slack settings are used. Full SDK notes, including the TypeScript wrapper, are in sdk/README.md.
Quotas for tool calls are separate from spec runs: Free 5,000/month, Pro 100,000, Team 1,000,000, Enterprise unlimited.
This feature replaces a standalone ToolPulse deployment for the same buyer. Point clients at ModelWatch (MODELWATCH_API_KEY, https://api.modelwatch.app). The decorator does not call api.toolpulse.io.
Reviews
No reviews yet
Be the first to review this server!
More Developer Tools MCP Servers
Rea
Freeby Morluto · Developer Tools
Reverse engineer anything from your terminal or agent with one CLI and MCP server.
Paperclip
Freeby Paperclipai · Developer Tools
Trending hip-hop artist momentum scores across four cultural dimensions.
Git
Freeby Modelcontextprotocol · Developer Tools
Read, search, and manipulate Git repositories programmatically
Toleno
Freeby Toleno · Developer Tools
Toleno Network MCP Server — Manage your Toleno mining account with Claude AI using natural language.
MCP Marketplace
Freeby mcp-marketplace · Developer Tools
Search and install MCP servers from inside your AI client.
mcp-creator-python
Freeby mcp-marketplace · Developer Tools
Create, build, and publish Python MCP servers to PyPI — conversationally.