Jun 9, 2018

Rendering and web scraping

  • Puppetron - Demo site that shows how to use Puppeteer and Headless Chrome to render pages. Inspired by GoogleChrome/rendertron.
  • Thal - Getting started with Puppeteer and Chrome Headless for Web Scraping.
  • pupperender - Express middleware that checks the User-Agent header of incoming requests, and if it matches one of a configurable set of bots, render the page using Puppeteer. Useful for PWA rendering.
  • headless-chrome-crawler - Crawler that provides simple APIs to manipulate Headless Chrome and allows you to crawl dynamic websites.
  • puppeteer-examples - Puppeteer Headless Chrome examples for real life use cases such as getting useful info from the web pages or common login scenarios.
  • browserless - Headless Chrome as a service letting you execute Puppeteer scripts remotely. Provides a docker image with configuration for concurrency, launch arguments and more.
  • Puppeteer Sandbox - Puppeteer sandbox environment as a service. Runs Puppeteer scripts and allows saving and embedding them in external sites and markdown files.