🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

Transform Web Content into LLM-Ready Data

Flexible Node.js AI-assisted crawler library

抖音爬虫——采集账号主页、喜欢、收藏、音乐原声、话题、搜索、合集、作品、关注、粉丝等公开数据。

Run a high-fidelity browser-based web archiving crawler in a single Docker container

一个用于抓取和分析 X (Twitter) 用户数据和推文的工具。

js cookie逆向利器：js cookie变动监控可视化工具 & js cookie hook打条件断点

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

➖ Stripped down, stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are completely removed. Crawl and convert any website into LLM-ready markdown.

Crawler for Nintendo Switch eShop

Open-source, production-grade web scraping engine built for LLMs. Scrape and crawl the entire web, clean markdown, ready for your agents.

Lightweight scraper for Google News

Independent search engine. Includes web crawling, search indexing, dictionary API, and more. https://vyntr.com

一个自我托管的 JavBus API 服务

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

Official Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler

SEO/GEO toolkit to analyse, crawl, parse and optimise websites & logs (Nginx & Apache)