1.2.948 - new scrapers SecurityTrails IP and Domain, domain proxies support, many other improvements

[Support Vlad]

Improvements

• Added scraper Rank::MOZ.
  Collecting all information contained on the page.
• Added scrapers SecurityTrails::Domain and SecurityTrails::IP.
  It is necessary to specify preset Util::ReCaptcha2.
  • SecurityTrails::IP
    You should specify the ipv4 address as a request.
    Collects domains by IP and information about them.
    
  • SecurityTrails::Domain
    As a request you should specify the domain, for example a-parser.com.
    Collected data:
    • DNS data
    • List of technologies used on the website (engines etc.)
    • Open ports list
    • Alexa rank
    • Country
    • Hoster
    • Created and expires date
    • Whois status
    • Registrar
    • Historical DNS data
    • Subdomains
• Added ability to disable validation of TLS certificates.
• Added support for domain proxies.

Corrections due to changes in the issuance

• Fixed news collection in SE::Google.
• Fixed Social::Instagram::Profile.
• Fixes in SE::Yandex:
  • fixed turbo links collection;
  • fixed news snippets collection.
• Fixes in SE::Google, SE::Baidu, SE::Yandex::Direct, Shop::Yandex::Market.
• Fixes in SE::Yahoo - an error in selecting countries with the same domain, the snippet scraping has been restored.

Corrections

• Fixed an error in domain autoselection algorithm in SE::Yandex.
• Fixed Rank::MajesticSEO, SE::Bing::Translator.
• Fixed bug if config.txt file was saved in utf-8 encoding with BOM (parser read the file incorrectly).
• Solved the problem with option overrides in the HTML::LinkExtractor.
• NodeJS: new installed modules are now available before A-Parser rebooting.
• Fixed A-Parser crash when calling the getProxies method.