WeChat Article Scraper MCP Server

This is an MCP Server designed to scrape and retrieve WeChat Official Account articles for AI Agents. It allows Agents to add official accounts, trigger article scraping, and retrieve scraped articles in Markdown format.

Features

• Add Official Account: Add WeChat Official Accounts to a monitoring list.
• List Official Accounts: View all currently monitored official accounts.
• Scrape Articles: Trigger a scraping task for a specific official account to fetch its historical articles.
• Get Scraped Articles: Retrieve scraped articles based on various criteria (e.g., official account, keywords, date range).

Installation

1. Clone the repository:

   git clone <repository_url>
   cd mcp-server-wechat-article-scraper

2. Create and activate a virtual environment:

   python -m venv .venv
   # On Windows
   .venv\Scripts\activate
   # On macOS/Linux
   source .venv/bin/activate

3. Install dependencies:

   pip install -e .

4. Install a web browser: This server uses a headless browser for scraping. Ensure you have Chrome or Edge installed on your system.

Usage

To run the MCP Server, execute the server.py file:

python src/mcp_server_wechat_article_scraper/server.py

The server will then listen for commands via STDIO. AI Agents can interact with it using the defined tools.

Configuration

You can configure the server using environment variables:

• WECHAT_SCRAPER_BROWSER_PATH: (Optional) Specify the path to your browser executable (e.g., C:\Program Files\Google\Chrome\Application\chrome.exe). If not set, the system will attempt to find it automatically.
• WECHAT_SCRAPER_DATA_DIR: (Optional) Specify the directory for data storage (e.g., database files, scraping cache). Defaults to a data folder within the server's working directory.

Tools

This MCP Server exposes the following tools:

• add_official_account: Add a WeChat Official Account to monitor.
• list_official_accounts: List all monitored WeChat Official Accounts.
• scrape_official_account_articles: Trigger an article scraping task for a specific account.
• get_scraped_articles: Retrieve previously scraped articles.

Refer to the spec.md file in the .trae/specs/wechat-article-mcp-server/ directory for detailed tool specifications, input/output schemas, and error handling.

Development

Running Tests

pytest tests/

Project Structure

mcp-server-wechat-article-scraper/
├── src/
│   └── mcp_server_wechat_article_scraper/
│       ├── server.py         # MCP server main file
│       ├── tools/            # Tool definitions
│       │   ├── __init__.py
│       │   ├── add_official_account.py
│       │   ├── list_official_accounts.py
│       │   ├── scrape_official_account_articles.py
│       │   └── get_scraped_articles.py
│       └── utils/            # Utility functions and modules
│           ├── __init__.py
│           ├── api_client.py # API request helpers (for internal use, e.g., browser control)
│           ├── formatters.py # Response formatting
│           └── errors.py     # Error handling
├── pyproject.toml
├── README.md
└── tests/
    └── ...