Categories: FAANG

Free Local RAG Scraper for GPTs and Assistants

We are currently testing this tool and will let you know when it is ready for full use!

This web scraper runs entirely in your browser and is perfect for creating training data for AI models. It works by reading the website’s sitemap.xml file, making it particularly well-suited for modern platforms like Squarespace and Shopify that automatically generate sitemaps.

The scraper preserves the structure of your content, including headings, paragraphs, lists, and tables, while removing unnecessary elements like navigation menus and footers. It also captures metadata, images, and PDF documents.

More technical details

This scraper uses a CORS proxy to access websites. Before using it:

  1. Visit CORS Anywhere Demo in a new tab
  2. Click the button to temporarily enable the demo server
  3. Return to this page and start scraping

The scraper will:

  • Read the website’s sitemap.xml to find all pages
  • Process each page while preserving content structure
  • Generate a markdown file with all content
  • Allow you to preview each page’s content before saving



Ready to start


AI Generated Robotic Content

Recent Posts

Does anyone else can’t stand ComfyUI and prefers classic Automatic/Forge UI or it’s just me?

EDIT: I can't believe how many great and useful replies I've got, and not a…

13 hours ago

Serving Multiple Users at Once: How Continuous Batching Keeps LLM Inference Efficient

This article is divided into four parts; they are: • The Problem with Static Batching…

13 hours ago

Everyone Has Their Targets Set on the MacBook Neo

Dell, Microsoft, and others are unveiling new laptops to compete directly with the Neo, but…

14 hours ago

Photon-driven synapse advances low-power neuromorphic systems

Modern artificial intelligence systems rely on moving large amounts of data between memory and processors,…

14 hours ago

Anima – Sharing Some Prompts and Results

Been experimenting with Anima lately and ended up spending way too much time refining prompts.…

2 days ago

Keychron K2 HE Concrete Edition Review: Rock-Solid Typing

Keychron's K2 HE Concrete Edition sounds like a cute gimmick, but as I discovered, there's…

2 days ago