Categories: FAANG

Free Local RAG Scraper for GPTs and Assistants

We are currently testing this tool and will let you know when it is ready for full use!

This web scraper runs entirely in your browser and is perfect for creating training data for AI models. It works by reading the website’s sitemap.xml file, making it particularly well-suited for modern platforms like Squarespace and Shopify that automatically generate sitemaps.

The scraper preserves the structure of your content, including headings, paragraphs, lists, and tables, while removing unnecessary elements like navigation menus and footers. It also captures metadata, images, and PDF documents.

More technical details

This scraper uses a CORS proxy to access websites. Before using it:

  1. Visit CORS Anywhere Demo in a new tab
  2. Click the button to temporarily enable the demo server
  3. Return to this page and start scraping

The scraper will:

  • Read the website’s sitemap.xml to find all pages
  • Process each page while preserving content structure
  • Generate a markdown file with all content
  • Allow you to preview each page’s content before saving



Ready to start


AI Generated Robotic Content

Recent Posts

Pirate VFX Breakdown | Made almost exclusively with SDXL and Wan!

In the past weeks, I've been tweaking Wan to get really good at video inpainting.…

3 hours ago

Try Deep Think in the Gemini app

Deep Think utilizes extended, parallel thinking and novel reinforcement learning techniques for significantly improved problem-solving.

3 hours ago

Introducing Amazon Bedrock AgentCore Browser Tool

At AWS Summit New York City 2025, Amazon Web Services (AWS) announced the preview of…

3 hours ago

New vision model from Cohere runs on two GPUs, beats top-tier VLMs on visual tasks

Cohere's Command A Vision can read graphs and PDFs to make enterprise research richer and…

4 hours ago

Anthropic Revokes OpenAI’s Access to Claude

OpenAI lost access to the Claude API this week after Anthropic claimed the company was…

4 hours ago

New AI tool learns to read medical images with far less data

A new artificial intelligence (AI) tool could make it much easier—and cheaper—for doctors and researchers…

4 hours ago