How to get data out of the Internet Archive's Wayback Machine: free ways first, then our Apify actor.
- How to scrape articles from the web archive (Wayback Machine): the text of one page or a whole site
Four ways to get the text of archived pages out of the Wayback Machine: by hand, the CDX API with the id_ snapshot address, the Availability API, or an Apify actor that returns one row per snapshot with the page text. What each costs and where it stops. - Wayback Machine scraper: how to list every snapshot of a URL or a whole domain, and keep only the changes
How to list every Wayback Machine capture of a page or a whole domain with its date and HTTP status, and keep only the captures where the content changed. The free CDX API with a worked example, the Availability API, and an Apify actor that returns the list as a spreadsheet.