Back to Browse

RagScore MCP Server

by HZYAI
AI & MLUse Caution3.8MCP RegistryLocal
Free

Server data from the Official MCP Registry

Generate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.

About

Generate QA datasets & evaluate RAG systems with failure diagnosis. Any LLM.

Security Report

3.8
Use Caution3.8High Risk

RAGScore is an AI/ML evaluation tool with reasonable security practices for its intended purpose. The codebase properly handles external API interactions, uses environment variables for credentials, and includes appropriate logging controls. However, there are concerns about telemetry collection in MCP mode, insufficient input validation on user-supplied endpoints, and potential path traversal vulnerabilities in document processing that should be addressed. Supply chain analysis found 30 known vulnerabilities in dependencies (0 critical, 14 high severity). Package verification found 1 issue.

4 files analyzed · 38 issues found

Security scores are indicators to help you make informed decisions, not guarantees. Always review permissions before connecting any MCP server.

Permissions Required

This plugin requests these system permissions. Most are normal for its category.

File System Read

Reads files on your machine. Normal for tools that analyze or process local data.

File System Write

Writes or modifies files on your machine. Check that this is expected for the tool.

HTTP Network Access

Connects to external APIs or services over the internet.

env_vars

Check that this permission is expected for this type of plugin.

process_spawn

Check that this permission is expected for this type of plugin.

What You'll Need

Set these up before or after installing:

OpenAI API key (if using OpenAI provider)Required

Environment variable: OPENAI_API_KEY

Anthropic API key (if using Anthropic provider)Required

Environment variable: ANTHROPIC_API_KEY

How to Install

Add this to your MCP configuration file:

{
  "mcpServers": {
    "io-github-hzyai-ragscore": {
      "env": {
        "OPENAI_API_KEY": "your-openai-api-key-here",
        "ANTHROPIC_API_KEY": "your-anthropic-api-key-here"
      },
      "args": [
        "ragscore"
      ],
      "command": "uvx"
    }
  }
}

Reviews

No reviews yet

Be the first to review this server!