JS scrapers: examples
Here are just a few examples of using JavaScript scrapers, demonstrating the core functionality. Many more examples are published in the Scrapers Catalog
🔗 JS::TitleParser
Scraping the HTML title tag on an arbitrary page
🔗 JS::TextFromSE
Combining several built-in scrapers into one
🔗 Downloading files
Example of saving files of any size directly to disk
🔗 Working with CAPTCHA
Example of a JS scraper for working with captchas
🔗 Working with ReCaptcha2
Example of a JS scraper for working with recaptchas
🔗 Working with built-in scrapers
Example of getting results from a built-in scraper inside a JS scraper
🔗 Working with SQLite DB
Example of creating a table, inserting, and selecting data
🔗 Working with MySQL DB
Example of working with the mysql2 module; this scraper collects links from Google results and writes them to the DB
🔗 Pagination traversal
An example of scraping a website by query with navigation through pages is considered
🔗 Filtering by multiple criteria
Example of filtering pages by a large number of criteria, using fs to read the criteria file
🔗 Filtering images by their content
Example of using Google Vision to determine image content
🔗 Posting to Wordpress blogs
Posting messages to Wordpress blogs using xmlrpc technology
🔗 Sending emails
Example of working with nodemailer (sending emails from a JS scraper). Yandex SMTP is used.
🔗 Creating page screenshots
Demonstration of working with puppeteer (Chrome) using the example of creating website screenshots
🔗 Telegram notifications
Receiving notifications in Telegram about the expiration of domain registration