Skip to main content

SE::Yahoo - Yahoo Search Engine Scraper

Yahoo

Scraper overview​

The Yahoo search results scraper. Thanks to the Yahoo scraper, you can obtain large databases of links ready for further use. You can use queries in the same format as you enter them into the Yahoo search bar, including search operators (site, ip, etc.).

A-Parser functionality allows you to save Yahoo scraper settings for future use (presets), set scraping schedules, and much more. You can use automatic query multiplication, substitution of subqueries from files, and iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the format and structure you need, thanks to the built-in powerful Template Toolkit template engine, which allows applying additional logic to results and outputting data in various formats, including JSON, SQL, and CSV.

Collected data​

  • Links, anchors, and snippets from search results
  • List of related keywords
  • Paid search results (ads)
Collected data

Features​

  • Support for all Yahoo search operators (site:, ip:, etc.)
  • Scrapes the maximum number of results provided by Yahoo - 50 pages of 100 elements in search results
  • Can automatically scrape more than 1000 results per query - substitutes additional characters (option Parse all results)
  • Ability to scrape in depth by related keywords (Parse related to level)
  • Ability to search for related keywords
  • Supports specifying the search result time range

Use cases​

  • Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
  • Assessing competition for keywords
  • Finding backlinks (mentions) of websites
  • Checking website indexing
  • Finding websites on the same IP address
  • Finding vulnerable websites
  • Any other cases involving Yahoo scraping in one form or another

Queries​

As queries, you need to specify search phrases exactly as if you were entering them directly into the Yahoo search form, for example:

test
windows Moscow   
site:http://lenta.ru  
ip:222.36.12.12

Query substitutions​

You can use built-in macros for query multiplication; for example, if we want to get a very large database of forums, we specify several main queries in different languages:

forum
forum
foro
论坛

In the query format, we specify character iteration from a to zzzz; this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries—an impressive figure, but no problem at all for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Using operators​

You can use search operators in the query format, so they will be automatically added to each query from your list:

site:$query

Output results examples​

A-Parser supports flexible result formatting thanks to the built-in Template Toolkit template engine, which allows it to output results in any form, as well as structured formats like CSV or JSON.

Link list export​

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Keyword competition​

Same as in SE::Google.

Checking link indexing​

Same as in SE::Google.

Saving in SQL format​

Same as in SE::Google.

Dumping results to JSON​

Same as in SE::Google.

Results processing​

A-Parser allows processing results directly during scraping; in this section, we have listed the most popular cases for the Yahoo scraper.

Same as in SE::Google.

Same as in SE::Google.

Extracting domains​

Same as in SE::Google.

Removing tags from anchors and snippets​

Same as in SE::Google.

Same as in SE::Google.

Possible settings​

note

Common settings for all scrapers

Parameter nameDefault valueDescription
Pages count5Number of pages to scrape (from 1 to 50)
Serp timeAll timeSERP time (time-dependent search, "tbs=" parameter: All time / Past 24 hours / Past week / Past month)
Safe SearchModerateSafe search option selection (Off / Moderate / Strict)
Yahoo domainUnited States (English)Yahoo domain selection
Yahoo languageAnyYahoo language selection, allows choosing the search language
Yahoo countryAnyCountry selection, allows choosing the country for the search
Not found is error☐Whether to consider the absence of results as an error