We are currently testing this tool and will let you know when it is ready for full use!
This web scraper runs entirely in your browser and is perfect for creating training data for AI models. It works by reading the website’s sitemap.xml file, making it particularly well-suited for modern platforms like Squarespace and Shopify that automatically generate sitemaps.
The scraper preserves the structure of your content, including headings, paragraphs, lists, and tables, while removing unnecessary elements like navigation menus and footers. It also captures metadata, images, and PDF documents.
More technical details
This scraper uses a CORS proxy to access websites. Before using it:
The scraper will:
Ready to start
What models or workflows are people using to generate these? submitted by /u/danikcara [link] [comments]
Ever felt like trying to find a needle in a haystack? That’s part of the…
We propose a distillation scaling law that estimates distilled model performance based on a compute…
How Alberta Health Services is using advanced AI to bolster its defenses as attackers increasingly…
A week after Disney and Universal filed a landmark lawsuit against Midjourney, the generative AI…
Imagine supercomputers that think with light instead of electricity. That s the breakthrough two European…