#spider (30 Repositories)
Ranked open-source repositories tagged with #spider, scored by pull request acceptance likelihood and maintainer engagement velocity.
43.4%
61.9h
30 repositories tagged #spider
alltheplaces/alltheplaces
A set of spiders and scrapers to extract location information from places that post their location on the internet.
BlessedRebuS/Krawl
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities using realistic, randomly generated decoy data and AI-generated HTML templates.
Norconex/crawler
Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.
eatmoreduck/boss-zhipin-scraper
Boss直聘爬虫 / BOSS直聘职位数据抓取工具，基于 Chrome CDP 协议复用真实登录态，绕过字体反爬，输出明文薪资 JSON/CSV + 薪资技能分析。A Chrome-CDP-based BOSS Zhipin job scraper/crawler.
fireinrain/asmr-downloader
A tool for download asmr media from asmr.one(Thanks for the asmr.one)
wujunwei928/parse-video
Golang短视频去水印：抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙,梨视频,acfun,好看视频...
JayBizzle/Crawler-Detect
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Evil0ctal/Douyin_TikTok_Download_API
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。
NaiboWang/EasySpider
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
jae-jae/QueryList
:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
DedSecInside/TorBot
Dark Web OSINT Tool
montaro2017/bili_novel_packer
轻小说打包器，采集哔哩轻小说网站(bilinovel.com linovelib.com)内容，打包成EPUB格式，支持封面、插图、目录，支持分卷合并。
tech-engine/goscrapy
GoScrapy: High perfomance webscraping framework in Go, inspired by Python's Scrapy.
JayBizzle/Laravel-Crawler-Detect
A Laravel wrapper for CrawlerDetect - the web crawler detection library
WuKongSecurity/SpiderBOX
SpiderBox - 虫盒 - 爬虫逆向资源导航站
willow-god/Friend-Circle-Lite
🐱一个精简版，无后端，且仅利用github action运行的精简版友链朋友圈程序，兼容fc的json格式信息，同时支持推送友圈更新，支持他人订阅个人站点并在更新时发送邮箱推送。
spider-rs/spider
Foundational low latency web data collecting in Rust
Boris-code/feapder
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度
Gerapy/Gerapy
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
hahaha108/copyBook
用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站
gsh199449/spider
A configurable web spider with a easy-to-use web console
crawlab-team/crawlab-lite
Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
fengzhizi715/NetDiscovery
NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。
tophubs/TopList
今日热榜，一个获取各大热门网站热门头条的聚合网站，使用Go语言编写，多协程异步快速抓取信息，预览:https://mo.fish
jaeles-project/gospider
Gospider - Fast web spider written in Go
skygazer42/DL-Hub
llms 大模型 笔记50篇 此仓库包含关于机器学习、深度学习、计算机视觉、自然语言处理、大模型 爬虫等领域 项目实战
ShunCai/QZoneExport
QQ空间导出助手，用于备份QQ空间的说说、日志、私密日记、相册、视频、留言板、QQ好友、收藏夹、分享、最近访客为文件，便于迁移与保存
l4rm4nd/LinkedInDumper
Python 3 script to dump/scrape/extract company employees from LinkedIn API
DoiiarX/NLCISBNPlugin
基于中国国家图书馆ISBN检索的calibre的source/metadata插件。https://doiiars.com/article/NLCISBNPlugin