Skip to main content

SE::Rambler - Rambler search engine scraper

img

Scraper Overview​

Rambler search results scraper. Thanks to the Rambler scraper, you will be able to obtain large databases of links ready for further use. You can use queries in the same form as you enter them in the Rambler search bar, including search operators (site, ip, etc.).

A-Parser functionality allows you to save Rambler scraper settings for future use (presets), set scraping schedules, and much more. You can use automatic query multiplication, substitution of subqueries from files, and iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure you need, thanks to the built-in powerful Template Toolkit template engine, which allows you to apply additional logic to results and output data in various formats, including JSON, SQL, and CSV.

Collected data​

  • Number of results in the SERP
  • Links, anchors, and snippets from the SERP
  • List of related keywords (hints)

what data the SE::Rambler scraper collects

Capabilities​

  • Support for Rambler search operators (url:, site:, inurl:, host:, rhost:, domain:.)
  • Scrapes up to 25 pages, from 10 to 50 results per page
  • Scrapes related keywords ($hints)
  • Ability to use solving services to bypass captchas
  • Choice of output device: regular desktop, mobile Android, or mobile iOS

Use Cases​

  • Collecting link databases
  • Assessing competition for keywords
  • Searching for site backlinks (mentions)
  • All cases where you need to scrape Rambler search results

Queries​

Specify queries just as in Rambler search. For example, if you only need links from one site. Enter in the query field:

"buy doors" site:http://kp.ru

Query substitutions​

You can use built-in macros for query multiplication; for example, if we want to get a very large database of forums, we specify several main queries in different languages:

forum
forum
foro
论坛

In the query format, we specify a character iteration from a to zzzz; this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries—an impressive figure, but no problem at all for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Using operators​

You can use search operators in the query format, so they will be automatically added to each query from your list:

site:$query

Output results examples​

A-Parser supports flexible result formatting thanks to the built-in Template Toolkit template engine, which allows it to output results in arbitrary form, as well as structured formats like CSV or JSON.

Exporting a list of links​

Same as in SE::Google.

Same as in SE::Google.

Same as in SE::Google.

Result format:

$hints.format('$hint\n')

Example result:

habrahabr
habr
habrahabr ru
xabra
livebusiness
eureka
electronic accountant
elba island
elba electronic accountant
habrahabr
...

Saving in SQL format​

Same as in SE::Google.

Dumping results to JSON​

Same as in SE::Google.

Results processing​

A-Parser allows processing results directly during scraping; in this section, we have listed the most popular cases for the Rambler scraper.

Same as in SE::Google.

Same as in SE::Google.

Extracting domains​

Same as in SE::Google.

Removing tags from anchors and snippets​

Same as in SE::Google.

Same as in SE::Google.

Possible settings​

note

General settings for all scrapers

Parameter nameDefault valueDescription
DeviceDesktopChoice of output device: regular desktop, mobile Android, or mobile iOS
Pages count5Number of pages to scrape (from 1 to 25)
Links per page10Number of results per page (10/15/30/50)
Rambler region IDAbility to set a region. You must specify the region ID. How to find the required region ID is described here
SortSites by relevanceChoice of result sorting option
Results filteringModerateChoice of result filtering option
Results languageAny languageChoice of result search language
Serp timeAnytimeChoice of result period
Results typeAny formatChoice of result type (mime type)
Exact match☐Strict match to query
Disable autocorrect☐Disables autocorrection, allows scraping the SERP exactly for the specified query
Use sessions☑Saves good sessions, which allows for even faster scraping with fewer errors
AntiGate presetdefaultDetermines whether to use Util::AntiGateUtil::AntiGate for bypassing captchas