#web-scraping (30 Repositories)
Ranked open-source repositories tagged with #web-scraping, scored by pull request acceptance likelihood and maintainer engagement velocity.
39.0%
33.6h
30 repositories tagged #web-scraping
konippi/servo-fetch
A self-contained browser engine that fetches, renders, and extracts web content as Markdown, JSON, or screenshots — no Chromium, no API key, no setup.
figranium/figranium
Build complex browser workflows visually and execute them via API.
ItamarZand88/CLI-Anything-WEB
Claude Code plugin that generates production-grade Python CLIs for any web app. 20 CLIs and counting.
microlinkhq/browserless
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
pinchtab/pinchtab
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
only-cli/oc
Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
justrach/kuri
Browser automation, web crawling, and iOS + Android device control for AI agents. Zig-native, token-efficient CDP snapshots, HAR recording, native adb wire-protocol client, and a standalone fetcher.
lexiforest/curl_cffi
Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
lorien/awesome-web-scraping
List of libraries, tools and APIs for web scraping and data processing.
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Anakin-Inc/anakin
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make up
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
oxylabs/oxylabs-ai-studio-py
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering.
Bin-Huang/camoufox-cli
Anti-detect browser automation CLI & Skills for AI agents — Camoufox-powered fingerprint spoofing, no bot-detectable Playwright leaks
go-rod/rod
A Chrome DevTools Protocol driver for web automation and scraping.
davidteather/everything-web-scraping
Learn everything web scraping with David Teather Codes on YouTube
jordantete/OddsHarvester
A python app designed to scrape and process sports betting data directly from oddsportal.com 🎯
Omarshraf/MangaVolt-Archive-Engine
Best MangaFox Downloader Script 2026: Batch Manga Grabber Tool
oxylabs/agent-skills
Official Agent skills of Oxylabs products
gildas-lormeau/single-file-cli
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
gosom/google-maps-scraper
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
crwlrsoft/crawler
Library for Rapid (Web) Crawler and Scraper Development
VIDA-NYU/ache
ACHE is a web crawler for domain-specific search.
z0m31en7/Uscrapper
Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.
oxylabs/oxylabs-ai-studio-js
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio JS SDK for intelligent web data gathering.
serpapi/nokolexbor
High-performance HTML5 parser for Ruby based on Lexbor, with support for both CSS selectors and XPath.
City-Bureau/city-scrapers
Scrape, standardize and share public meetings from local government websites