Improvements
- Implemented TLS fingerprint protection bypass, now A-Parser requests look 100% like Chrome
- NodeJS updated to version 16.15.0
- In
SE::Google added the Narrow results by region parameter, which allows you to narrow the search to a specific country - In the scraper
Maps::Google added automatic traversal of coordinates in a given region - Added the Start message number parameter to
Telegram::GroupScraper, which allows you to specify from which number to collect messages in telegram chat - In
SE::YouTube added collecting the number of channel subscribers - In
SE::Google::Trends added collection of topic ids for related topics - In
Shop::Wildberries::ProductsList scraper, collection of keys from "Still searching" and "Maybe you'll like it" has been added - Added Wildberries suggests scraper:
Shop::Wildberries::Suggest - Added collection of additional data to
Shop::Wildberries::ProductInfo - New types added to Usage rights in
SE::Google::Images - Added the ability to display raw data in Ahrefs scrapers
- Added Randomize TLS Fingerprint option to
Net::HTTP and JS scrapers - Improved work with Chromium from JavaScript scrapers
Net::HTTP and this.request() now use the latest Chrome user-agent by default
Corrections due to changes in the issue
- Many adaptations to changes of the layout in Google, Yandex, Bing
- Fixed scraping of turbo pages in Yandex scrapers
- The work of
SE::Yandex::WordStat has been fixed, there are plans to completely rewrite this scraper with the addition of new functionality - In
SE::Yandex::ByImage serp scraping fixed - Fixed username parsing in
Telegram::GroupScraper - In
SE::Google::ByImage the set of collected data and the default result format have been changed, and its work has been generally fixed - The work of the Bookvarix scrapers has been fixed, but due to the high load on their API, these scrapers may not work
- Fixed snippet scraping in
SE::Google::ByImage
SE::Google::TrustCheck,
Check::RosKomNadzor,
SE::DuckDuckGo,
SE::Seznam,
Shop::Wildberries::ProductsList,
SE::Baidu,
SE::Google::Images,
Rank::Ahrefs::KeywordGenerator
Bug fixes
- Fixed a bug in
Shop::Wildberries::ProductsList, due to which $totalcount was not displayed with some queries - Fixed captcha loading in
SE::Yandex::Images - Fixed a memory leak in the
DeepL::Translator and other bugs - Fixed work of proxy checker with socks5 IPv6 proxy
- Fixed work of the Don't verify TLS certs option in some cases
- Fixed a memory leak when running tasks frequently or using an API, the leak was related to an internal NodeJS bug
- Fixed freezing of Test parsing when editing the code of scrapers
- Fixed passing through redirects in
HTML::LinkExtractor - Fixed domain selection in
SE::Yahoo - Fixed log spamming with errors when using
Maps::Yandex - Fixed collection of $records results in
Net::DNS - Fixed a bug when the next page was not found with the Check next page option
