MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
-
Updated
Mar 27, 2026 - Python
#
trafilatura
Here are 28 public repositories matching this topic...
pebkac Chrome Nonautomation - A Local LLM-Driven Web Co-Browser using Smolagents, Zendriver, Trafilatura.
chrome automation ai openai webscraping atlas claude llms claude-ai trafilatura nodriver ai-browser-automation smolagents ai-browser-control aibrowser zendriver ai-browser pebkac
-
Updated
Apr 14, 2026 - Python
Fast, accurate web content extraction in Rust. ML page-type classification, per-type extraction, confidence scoring. F1=0.966 on ScrapingHub (#1), F1=0.859 across 2,008 annotated pages (1,497 development + 511 held-out test
-
Updated
Apr 3, 2026 - Rust
web Scrapper In Python
-
Updated
Sep 6, 2023 - Python
Telegram Mini App that saves internet articles to read them later
-
Updated
Feb 19, 2026 - Python
Lean Python tool for extracting clean, LLM-optimized markdown from web pages. Handles dynamic content with Playwright + Trafilatura for maximum information extraction efficiency.
-
Updated
Jul 6, 2025 - HTML
Tools for LLMs to anonymously search and browse the web
mcp selenium tor duckduckgo selenium-webdriver tor-network geckodriver tor-hidden-services trafilatura mcp-server fastmcp ddgs tbselenium
-
Updated
Mar 17, 2026 - Python
Selective web content extraction for AI agents — URL + query returns only the chunks that matter (Python library + MCP server)
python mcp embeddings web-scraping content-extraction ai-agents rag playwright llm trafilatura bge-m3 mcp-server
-
Updated
Apr 17, 2026 - Python
ChatGPT AI Clone
python dart websockets gemini web-scraping flutter fastapi sentence-transformers flutter-web-app trafilatura tavily ai-clones
-
Updated
Jul 4, 2025 - Dart
A pipe-based news article scraping and metadata extraction library for Python
python crawler data-science scraper spider data-collection news-scraper osint-python llms article-extraction trafilatura newspaper4k
-
Updated
Mar 20, 2026 - Python
Real-time AI search and chat backend with WebSocket streaming, powered by Tavily web search and Google Gemini for Flutter apps.
-
Updated
Aug 2, 2025 - Python
Protocole de collecte et d'analyse d'archives de la Wayback Machine pour une analyse textuelle et statistique
-
Updated
Apr 12, 2026 - Python
Tools for LLMs to anonymously search and browse the web
mcp selenium tor duckduckgo selenium-webdriver tor-network geckodriver tor-hidden-services trafilatura mcp-server fastmcp ddgs tbselenium
-
Updated
Mar 17, 2026
🕷️ Clean, chunked documentation crawler optimized for RAG & AnythingLLM. Dockerized.
-
Updated
Jan 13, 2026 - Python
🕵️♂️ Enable anonymous web searches for your LLM with the first-ever Model Context Protocol server utilizing Tor for secure and private information retrieval.
api wrapper server mcp selenium tor duckduckgo geckodriver nyaa thepiratebay ygg trafilatura mcp-server fastmcp ddgs tbselenium lacale
-
Updated
Apr 17, 2026 - Python
Trafilatura API for html content info extract
python nlp docker flask rest-api text-extraction web-scraping content-extraction metadata-extraction news-scraper article-extraction trafilatura
-
Updated
Dec 2, 2025 - Python
A web scraper with an LLM-powered document suggestion system that combines web crawling, data extraction, and advanced AI capabilities to recommend relevant documents.
-
Updated
May 10, 2025 - Python
FastAPI service that classifies publisher websites for affiliate campaigns using an LLM pipeline (scrape → signals → RAG → scoring). Detects cashback, adult, gambling, scams. Supports OpenAI/Ollama, Redis cache, Docker.
docker redis openai python-asyncio affiliate-marketing rag pydantic httpx sentence-transformers llm website-scraping chromadb trafilatura ollama scam-detection publisher-website-classifier gambling-detection adult-content-filter cashback-detection
-
Updated
Mar 30, 2026 - Python
Improve this page
Add a description, image, and links to the trafilatura topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the trafilatura topic, visit your repo's landing page and select "manage topics."