SE::Yandex - Yandex Search Engine Results Scraper

Scraper overview

The Yandex search results scraper is rightfully among the TOP 5 most popular scrapers. One of the main competitive advantages of our scraper is its colossal performance. Thanks to A-Parser multithreaded operation, the request processing speed can reach 3000-7000 requests per minute, which on average allows obtaining up to 5,000,000 links per minute, while resource consumption is minimal; any office or home computer, as well as any entry-level VDS, is suitable for work. Our scraper supports all Yandex search operators, which significantly expands scraping capabilities. The stability and continuity of search results scraping are ensured by captcha recognition via AntiCaptcha or any other supported API (Anti-Captcha, RuCaptcha, CapMonster.cloud, 2captcha, and others).

Flexibility in settings allows you to specify the type of results (mobile/desktop), region, language, sorting of results by date, and much more. A-Parser functionality allows you to save scraping settings for future use (presets), set scraping schedules, and much more. You can use automatic query multiplication, substitution of subqueries from files, brute-forcing of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure you need, thanks to the built-in powerful Template Toolkit templating engine, which allows applying additional logic to results and outputting data in various formats, including JSON, SQL, and CSV.

Go to DEMO Buy A-Parser Pro ($299)

Scraper use cases

🔗 Yandex SERP and position number

Retrieves Yandex search results and the position number of the result. The result is saved to a csv file.

🔗 Scraping Yandex lite search results

This article provides an example of creating a scraper to collect information from the lite version of the Yandex search engine.

🔗 Scraping ads from Yandex

The preset parses the ad block in Yandex search results and saves the result in json format.

🔗 Competition assessment in Yandex

The preset determines competition in the Yandex search engine by keywords.

🔗 Yandex cache scraper

The preset implements the ability to obtain links to the Yandex search engine cache.

🔗 Scraping only snippets from Yandex

This preset takes a word or phrase as a parameter to search in Yandex.

Collected data

Number of results per query
Links, anchors, and snippets from search results
- Information about flags for each result is also collected; currently supported flags: Date, Image Preview, Video, Rich snippet, Featured snippet
Time of first and last caching (desktop results only)
List of related keywords
Determines whether Yandex considered the query a typo or not
Scraping the link to the page cache (desktop results only)
List of site icon names and their types, if present (including Turbo)
Top, middle, and bottom ad blocks from all pages
- Link, visible link, anchor, snippet, and domain
- Sitelinks and anchors
- Metro station
Missing words (words that may be displayed under each result in search results marked "Not found")
List of Quick answers: questions, answers, links to sources (enabled by a separate Parse Quick answers option)
AI answer (Search with Alice), its type, and list of sources

Capabilities

Choice of results type: mobile/desktop
Support for all Yandex search operators (site:, lang:, etc.)
Scrapes the maximum number of results provided by Yandex - 25 pages of 50 elements in search results
Ability to search for related keywords
Supports selection of search region and domain
Ability to sort results by date
Scrapes page indexing time, with the ability to filter results by this parameter
Browser emulation option (allows achieving high scraping speed and low captcha consumption)
Ability to bypass captcha using the AntiCaptcha service or any other supported API (it is recommended to enable the is_russian parameter in the anti-gate settings)
Ability to specify the number of Quick answers the scraper should collect by clicking deep into each question

The following scrapers work based on the Yandex scraper:

SE::Yandex::Position - checking website positions by keywords in Yandex

Usage scenarios

Collecting link databases - for A-Poster, XRumer, AllSubmitter, etc.
Assessing competition for keywords
Searching for backlinks (mentions) of sites
Checking site indexing
Searching for vulnerable sites
Any other scenarios involving Yandex scraping in one form or another

Queries

Search phrases should be specified as queries, exactly as if they were entered directly into the Yandex search form, for example:

windows Moscow
lang:en windows Moscow
url:a-parser.com
site:a-parser.com
"a-parser.com"

Query substitutions

You can use built-in macros for query multiplication; for example, if we want to get a very large database of forums, we specify several main queries in different languages:

forum
forum
foro
论坛

In the query format, we specify a character range from a to zzzz; this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries—an impressive figure, but no problem at all for A-Parser. At a speed of 2000 requests per minute, such a task will be processed in just 16 hours.

Using operators

You can use search operators in the query format, so they will be automatically added to each query from your list:

site:$query

Accounts

To use the SE::Yandex scraper, Yandex accounts may be required. Accounts can be registered using the SE::Yandex::Register scraper or simply by adding existing accounts to the files/SE-Yandex/accounts.txt file in the supported format.

Alternatively, you can enable account registration "on the fly".

To work using session authorization, the data string must be in this format: [email protected];MAQT78Z31Rinx4H;{"answer":"qmfhsxdcrk","proxy":"185.104.120.45:3128","session_id":"3:1748440908.5.0.1748440867459:ZXBxpg:47e4.1.2:1|2191075974.41.2.2:41.3:1748440908|3:10308131.797655.5pfkoRZWgLJGntKTlcUhYdysNfk"}

Result output options

A-Parser supports flexible result formatting thanks to the built-in templating engine Template Toolkit, which allows it to output results in arbitrary forms, as well as structured ones, such as CSV or JSON

Exporting a list of links

Result format:

$serp.format('$link\n')

Result example:

https://TestoMetrika.com/tests/
https://onlinetestpad.com/ru/tests
https://www.speedtest.net/
https://ustaliy.ru/testi/
https://yandex.ru/internet/
https://konstruktortestov.ru/popular
https://TestEdu.ru/test/
https://kto-chto-gde.ru/category/tests/
https://weekend.rambler.ru/tests/
https://GadalkinDom.ru/test
...

Links + anchors + snippets with position output

Result format:

[% FOREACH item IN serp;    loop.count _ ' - ' _ item.link _ ' - ' _ item.anchor _ ' - ' _ item.snippet _ "\n"; END %]

Result example:

- http://forum.r-rp.ru/ - <b>forum</b>.r-rp.ru - 
- https://forum.arizona-rp.com/ - <div class=a11y-hidden>Web result with sitelinks</div><b>Forum</b> – Arizona Role Play - Menu. Home. <b>Forums</b>. New messages. What's new? New messages. Users. Current visitors. <b>Forums</b>. Login. ... Statistics <b>of the forum</b>. Topics. 1,247,176. Messages. 5,225,340. Users. 623,675. 
- https://ru.wikipedia.org/wiki/%D0%A4%D0%BE%D1%80%D1%83%D0%BC - <b>Forum</b> — Wikipedia - <b>Fórum</b> (lat. <b>forum</b> — arch. vestibule of a tomb; area in a press for grapes to be processed; market square, city market; marketplace, central square):
- https://zen.yandex.ru/media/propromotion/chto-takoe-forum-i-vse-chto-s-nim-sviazano-5d65164c1d656a00ad52ba30 - What is a <b>forum</b> and everything related to it | Creative... - Today we will talk about what a <b>forum</b> is and everything related to it in accessible and simple words. Each of us strives to find such a person with whom one could build communication on common interests, share one's experience, give advice. In the modern world, all this has become possible to do without leaving home. Just one of the forms of such virtual contact is a web-<b>forum</b>. 
- https://forum.vimeworld.ru/ - VimeWorld - <b>Forum</b> - <b>Forum</b> of the project of ideal Minecraft game servers - VimeWorld...
...

Outputting links, anchors, and snippets to a CSV table

The built-in utility $tools.CSVLine allows creating correct tabular documents ready for import into Excel or Google Sheets.

General result format:

[%  FOREACH i IN p1.serp;    tools.CSVline(i.link, i.anchor, i.snippet); END  %]

File name:

$datefile.format().csv

Initial text:

Link,Anchor,Snippet

tip

In General result format the Template Toolkit templating engine is used to output the $serp array in a FOREACH.

In the result file name, you just need to change the file extension to csv.

To make the "Initial text" option available in the Task Editor, you need to activate "More options". In "Initial text", write the column names separated by commas and make the second line empty.

Outputting ad blocks

Result format:

$ads.format('$link - $anchor - $snippet\n')

Result example:

http://yabs.yandex.ru/count/WcOejI_zO3C2vH80P1zY-_ryBtnaD0K0CmCnZYWCO000000u109mhiMfd8qUW07CkUOvY07Kyz3GCP01vCcQhIwO0PgqhFigk06qZQ3m6C01NDW1gkAR5E01kAZK4-W1y06W0kYCvAl-Wue5-0Bmwl7WeSQurUK1c0FRc3lkh0Ju1Bpr48W5lFKGa0MxjssW1Qxa1QW5hkG5i0Mkv0Mu1OYr9S05eTt90SW5aFn4YkWqZwuhO8VP1W00012H0000gGVlTvJIyeiV0R07W82O3BW7W0Nn1tjIyvgFUtz-X8A0WSIqXdB92j4AXC7wM-4_u3nZJEzt003CKjw5aRa50DaBw0kyzRAxthu1gGn-j62AsN3cl-WCemBW3OE0W4293eDHIPs09kwAqTFvwFMAi8VO3WAX3zaFW13WszlG4DcTXo9ZI0HkD3-n4YxXl0bOc-q2u1E8jIMW58Yr9QWKkxTjl9wVx0Ne58m2q1Mydf_i1TWLmOhsxAEFlFnZyA0Mq92TW0R95l0_q1Qokzw-0O4N0F0_c1UwdvGKg1S9m1Uq0jWNm8GzcHYW60wm68UTi806q1WX-1Yf-9keZlxncYM06R3qkEBGlP6v890P0Q0PmWEm6RWP____0T8P4dbXOdDVSsLoTcLoBt8qEJSjCkWPWC83y1c0mWE16l__WxZFMxv27W2GPM2khLr2HGBSgKCU4fSjR_apLy29ToVZBSaX0K10aLK2xDc6HsxyWlx3mqOzRTCnV7G7IDvEXnY4YqauFXdHmHcIWrcJNGT1NfMC_8eB8q1m1-WEYbKFtWBTZHwcD4A80G00~1?from=yandex.ru%3Bsearch%26%23x2F%3B%3Bweb%3B%3B0%3B&amp;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Buy</b> <b>flight ticket</b> to / aviasales.ru - We offer: <b>Buy</b> <b>flight ticket</b> to here. Super offer! Hurry!
http://yabs.yandex.ru/count/WgGejI_zO5e2rHG092HY-_ryhkYFF0K0MWCnZYWCO000000u109mhDZpa8WGW07ZZm680U2f-wrQa07AywxRoe20W0AO0ShphjjAk06oWAxe8C01NDW1e8MobW7W0TJCXm_e0O01c0BImFq2e0BuQjW20l02g_w3YWNu0l3gyU2XnhZLvG600vF6eiSFY0FStR-O39W3cyKxYga3-0JJpWI81TFE1905Z-zGe0MPj06e1PMM0R05bPO1k0NInnJ01TF1ZG781PNz8uheD8-kAs27sGO0000GaG000Aa7xtUKqlAB7m6m1u20c0ou1u05yGTxKlEQZtj_VeI2l3M02W712l_aZtPQIU8_oGeJ5NRdMhxJFweB4E0yOqplTm00p5BUXP6v1G3P2-WBqyu4y0i6Y0ookzw-0QaCu_jL-Yu3zB_e3AC2u0s3W810YGwgO5HI9w3dYj7J-UZrYh27s0u2-0x7dPAe2Q4FFGhNet0zzZ_P3_0_W13GmC4Rm92GPpgqxC9xJZC_iHAuM7p6uhZj0k0JqiSKe1JInnIe58_lKB0KYw381hWKmAo0jQI04TWK-FpP_WNe58m2q1Nu_Dd-1TWLmOhsxAEFlFnZyA0Mq92TW0R95j0MihlUlW615vWNfwZz3wWN2S0Nj0BO5y24FPaOe1WAi1Z9fB201j0O8VWOgVYRg8x-yPebW1cmzBZYqBsHkI2G6G6W6S83i1cu6V___m7I6H9vOM9pNtDbSdPbSYzoD3atBJBe6O320_0PWC83WHh__oD2TTsApne0jsLWrgrT_2INXiZt8r8kcvWxe0SCE37tPBmjEYG0203ecjreFlD0AY-khXXjgcFZa190IXd9BOrkl3guMzzraExES_xHXH4WwWDq1xIoUlHJ6Y74~1?from=yandex.ru%3Bsearch%26%23x2F%3B%3Bweb%3B%3B0%3B&amp;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Buy</b> <b>flight ticket</b> online on Tutu.ru! Official site! - Plane tickets at a low price! Profitable flights worldwide! <b>Buy</b> ticket online!
http://yabs.yandex.ru/count/WcOejI_zO3C2vH80P1zY-_ryBtnaD0K0CmCnZYWCO000000u109mhiMfd8qUW07CkUOvY07Kyz3GCP01vCcQhIwO0PgqhFigk06qZQ3m6C01NDW1gkAR5E01kAZK4-W1y06W0kYCvAl-Wue5-0Bmwl7WeSQurUK1c0FRc3lkh0Ju1Bpr48W5lFKGa0MxjssW1Qxa1QW5hkG5i0Mkv0Mu1OYr9S05eTt90SW5aFn4YkWqZwuhO8VP1W00012H0000gGVlTvJIyeiV0R07W82O3BW7W0Nn1tjIyvgFUtz-X8A0WSIqXdB92j4AXC7wM-4_u3nZJEzt003CKjw5aRa50DaBw0kyzRAxthu1gGn-j62AsN3cl-WCemBW3OE0W4293eDHIPs09kwAqTFvwFMAi8VO3WAX3zaFW13WszlG4DcTXo9ZI0HkD3-n4YxXl0bOc-q2u1E8jIMW58Yr9QWKkxTjl9wVx0Ne58m2q1Mydf_i1TWLmOhsxAEFlFnZyA0Mq92TW0R95l0_q1Qokzw-0O4N0F0_c1UwdvGKg1S9m1Uq0jWNm8GzcHYW60wm68UTi806q1WX-1Yf-9keZlxncYM06R3qkEBGlP6v890P0Q0PmWEm6RWP____0T8P4dbXOdDVSsLoTcLoBt8qEJSjCkWPWC83y1c0mWE16l__WxZFMxv27W2GPM2khLr2HGBSgKCU4fSjR_apLy29ToVZBSaX0K10aLK2xDc6HsxyWlx3mqOzRTCnV7G7IDvEXnY4YqauFXdHmHcIWrcJNGT1NfMC_8eB8q1m1-WEYbKFtWBTZHwcD4A80G00~1?from=yandex.ru%3Bsearch%26%23x2F%3B%3Bweb%3B%3B0%3B&amp;q=%D0%BA%D1%83%D0%BF%D0%B8%D1%82%D1%8C+%D0%B0%D0%B2%D0%B8%D0%B0%D0%B1%D0%B8%D0%BB%D0%B5%D1%82%D1%8B&amp;etext=2202.wBaB7RlytlP_PRaU6jPhHD2nHBNT_4gSF6009OyJEfpWjUPoA5WrSJAqCfap0m9ReXhoaGRlaWhmYmllemF2ZA.f04b1271668949ea17ffcafcb11c72c2ab6454c5 - <b>Buy</b> <b>flight ticket</b> to / aviasales.ru - We offer: <b>Buy</b> <b>flight ticket</b> to here. Super offer! Hurry!
...

Result format:

$related.format('$key\n')

Result example:

<b>test</b> tanki online
tanki online
all <b>tests</b> dot ru
i exam ru testing
<b>test</b> internet speed
<b>tests</b>24.ru
speedtest
online <b>test</b> pc performance
online testing
my <b>test</b> student how to find out answers
...

tip

To automatically remove HTML tags in the result, you need to use the Results Builder, select the $related array and apply Remove HTML tags.

Keyword competition

Result format:

$query - $totalcount\n

Result example:

all tests dot ru - 25000000
online testing - 13000000
tanki online - 7000000
i exam ru testing - 27000000
tests24.ru - 238000000
online pc performance test - 16000000
speedtest - 2000000
internet speed test - 16000000
test tanki online - 19000000
my test student how to find out answers - 16000000

Determining keywords with errors

Result format:

$query - $misspell\n

Result example:

online testing - 0
internet speed test - 0
onliyn tastying - 1
intrenet sped test - 1

Checking link indexing

Query format:

site:$query

Result format:

$query.orig - $totalcount\n

Result example:

https://a-parser.com/pages/buy - 2
https://a-parser.com/wiki/parsers - 16
https://trjkjfkdf.bg.ky - 0
https://a-parser.com/resources - 1000
https://a-parser.com/forum - 499

tip

To check link indexing, insert the corresponding operator into the Query Format: site:.

The result format is output as "original url - number of pages in index".

As a result, we get the page address and its count in the search engine index.

If the page is missing, the result will be: 0.

Saving in SQL format

Result format:

[%  FOREACH serp;   "INSERT INTO serp VALUES('" _ query _ "', '";   link _ "', '";  anchor _ "')\n"; END  %]

Result example:

INSERT INTO serp VALUES('test', 'https://konstruktortestov.ru/popular', 'Popular online <b>tests</b>')
INSERT INTO serp VALUES('test', 'https://TestoMetrika.com/tests/', 'Online <b>tests</b> c with accurate result from psychological...')
INSERT INTO serp VALUES('test', 'https://ustaliy.ru/testi/', '<b>Tests</b> online: best, interesting and popular')
INSERT INTO serp VALUES('test', 'https://www.SunHome.ru/tests/Interesting_tests', 'Interesting <b>tests</b>. Take psychological interesting...')
INSERT INTO serp VALUES('test', 'https://onlinetestpad.com/ru/tests', '<b>Tests</b> online | Online Test Pad')
...

Dumping results to JSON

Common result format:

[% IF notFirst;
  ",\n";
ELSE;
  notFirst = 1;
END;

obj = {};
obj.totalcount = p1.totalcount;
obj.links = [];

FOREACH item IN p1.serp;
    obj.links.push(item.link);
END;

obj.json %]

Initial text:

Final text:

Result example:

[{"totalcount":113000000,"links":["https://TestoMetrika.com/tests/","https://konstruktortestov.ru/popular","https://ustaliy.ru/testi/","https://www.SunHome.ru/tests/Interesting_tests","https://GadalkinDom.ru/test","https://zen.yandex.ru/tes","https://onlinetestpad.com/ru/tests","https://kto-chto-gde.ru/category/tests/","https://psytests.org/top.html","https://MixTests.com/new/","https://TestEdu.ru/test/","https://testserver.pro/index","https://onedio.ru/tests","https://BankTestov.ru/","https://weekend.rambler.ru/tests/","https://edieta.org/testi","https://trikky.ru/?%21","https://BBF.ru/tests/","https://dropi.ru/c/tests/raznie","https://cadelta.ru/tests","https://www.Elle.ru/tests/","https://www.adme.ru/svoboda-psihologiya/polnyj-spisok-psihologicheskih-testov-dlya-poznaniya-sebya-kotorye-mozhno-projti-onlajn-2071715/","https://www.ellegirl.ru/tests/","https://test.tankionline.com/","https://vraki.net/onlajn-testy/","https://Lifehacker.ru/psixologicheskie-testy/","https://iq2u.ru/tests","https://www.b17.ru/tests/","https://ru.wikipedia.org/wiki/%D0%A2%D0%B5%D1%81%D1%82","https://gurutest.ru/test/","https://www.psychologies.ru/tests/","https://peopletalk.ru/category/tests/","https://obrazovaka.ru/testy","https://likeyou.io/category/test/","https://ProfTest.me/tests","https://TayniyMir.com/testy","https://psi-technology.net/psytest/","https://www.kp.ru/putevoditel/online-test/","https://tvoytest.ru/","https://twizz.ru/tests/","https://lunanews.net/testy/","https://www.ivi.ru/titr/tests","https://sntch.com/tests/","https://testy.online/","https://videouroki.net/tests/","https://www.speedtest.net/ru","https://rb.ru/tests/","https://aznaetelivy.ru/tests/","https://woman-psy.com/psihologicheskie-testy/testy_na_eruditsiyu_i_znaniya/interesnye_testy/","https://mamochka-club.com/psihologicheskie-testy/interesnye-testy/"]}]

tip

To make the "Initial text" and "Final text" options available in the Task Editor, you need to activate "More options".

Results processing

A-Parser allows processing results directly during scraping; in this section, we have provided the most popular cases for the Yandex scraper

Link deduplication

Add deduplication and select from the dropdown list $serp.$i.link - Link.

Download example

How to import an example into A-Parser

eJx9VE2P2jAQ/SurEYdWQqvQLpfcWFSqVnTZLuyhohy8ZIJcHNu1HQqK+O+dCUkc
ttVeoszze/NtVxCE3/tHhx6Dh3Rdga3/IYXlpzT9IXSGx5u51Ht/86zl7xJhCFY4
j47p6x6LDjLMRakCDCsIJ4vkxBzQOZmxSmZkW2eOJ4fBSfSEHYQqmTZKkgTOb8jE
Drem1CFqRm/QFedr0bEsKsZJT1K2xVC5nHK6BqrJEsBi2AwbxupC95Sx3kGLflbm
RShIgyvxvNm0XvzMuEJw8wZ2dNt0sjtcigOuDB3mUsXIfkbWgyg4yiATAfn0Nq8d
vXt/G47sQWSZDNJooS4RuPcxajOYFLQhLv1yd2fOFAQFrB0weGqzW8OgtmOV3y8a
SHOhPA7BU6ozQYlkr09kQCeCcQvL+RBegdETpeZ4QBVptf/7UqqMFmWSk+hLI/w/
ZfGPj3NXXj8UDfqPoxw6L7V1v/gWVZmZmx1Vnr3UwyxkINtP6+1JISFwj2i7nj1w
zwrjsAvTeG6i092wqHmr4sgmNkJXZVyN5RrcGp3L3aLZ1JZZ6hVdwIWemsIq5Lp0
qRSNxeNTXI+Jb8bARkzwtXhah+DS23sIwRjlvy4vqVonaf3GnGBBnexHbVxuhVLP
T/P+CcSVIuNnmdzdfeDvx3H9P7ogwNqAO0NbRZWdN90r0b0tVf+tSKszjeqXf7yQ
uC6mEEYN8jQHvuB/AYBymo0=

tip

Link deduplication by domain

Add deduplication and select from the dropdown list $serp.$i.link - Link. Select deduplication type: Domain.

Download example

How to import an example into A-Parser

eJx9VE2P2jAQ/SvI4tBKCIV2ueTG0lK1ost2YQ8V5WDIBLk4tms7FBTx33fGCXHY
VnuxPG/mzbddMc/dwT1acOAdS9cVM+HOUrb8nKY/ucrg1JsLdXC9ZyX+lNDbnnuf
dMGFYgNmuHVgibju2KMig5yX0rNBxfzZALrTR7BWZIBKkaFsrD6dLXgrwCF25LIk
s1GSJOzyBo3vYadL5SNn9Ia5pMwNWKJFxjjpUMpQFiqxcEo5XTOsySBAZLYZNBar
2jy71l6jX6TecslSb0u4bDZXL26mbcGpjX0zGjY9bZVLfoSVRmUuZIzsZig98IKi
9DPugbTDPDh6937oT+SBZ5nwQisu6wjU+xi1HhHylUZbvFJ3Z1YXCHkIDgg8X7Nb
s36QY5U/ag5Lcy4dDJjDVGccE8lea4QHy722C0P5IF4xrSZSzuEIMpoF//elkBku
yiRH0teG+H+TxT8+Lm153VA46L8Wc2i9BOl+8T2yMj3XexrZNgyzEB5lNw3bk7IE
wQOAaXv2QD0rtIU2TOO5iY6vxICirYojm5gI3ZRxM5ZbcKdVLvaLZlOvlqVa4VNc
qKkujASqS5VS4lgcPMX1mLhmDCTEBF+TpyEElX59h8xrLd23ZZ2qsQLXb0wJFtjJ
btTG5Y5L+fw072pYXCkUfpXJ3d0HOj+Ow31UI4y4HvYatworu2zaX6L9ZaruX5FW
FxzVb/dYG1FdZIIYNsjhHOiBvwAa7J3h

tip

Extracting domains

Add Results Builder and select the source from the dropdown list: $p1.serp.$i.link - Link. Select type: Extract Domain.

Download example

How to import an example into A-Parser

eJx9VEtv2zAM/isFkcMGBIGzNRff0qwBNmRN17SHIetBi+lAiyxpkpwlMPzfR8qO
7XZDL4b5+Eh+fKiCIPzB3zv0GDyk2wps/IcUNrdp+l3oDE9Xt6fgxC5cfTKFkNrD
GKxwHh0jtgNHMmSYi1IFGFcQzhYpjjmiczJDMsqMZOvM6ewwOIkc6ShUyW7TJEmg
fgMm9rgzpQ49ZvqGu5L64C06hvWIWVI/P4+BKFKNfmlcIZjqyE4nLe/OuBFHfDRk
zKXCXr0k6U4UnHKUiYBsneQx0Lv3k3DiCCLLZJBGC9Vk4Db1WZ+0/B0Za0O+9MuN
WDpTkCpgDMDK86W6LYyiDBSijNhvDQbSXCiPY/BU6lJQIdlriwzoRDBubbke0ldg
9FypFR5R9W4x/k0pVUYznecE+twC/++y/idG3dEbpqKZ/HFUQxclSjfrrz0qMyuz
J+bZT+KtZCEDyX4RB51CQsoDou16dsc9K4zDLk0buc1Om2xR8wL0I5vbXvWCxoux
DJQVeFO6HeXbJuMt0KLbWJw+AG9Pu3HYHEVzEzx158SZ9K17MO0WQk2YndG53K/b
Jb1kLvUjnd9aL0xhFXKfdKkUjdnjQ79uc9+OlYWe8GvwIqbgVl5OkGowyn/ZNNSt
k1TSjAkXNJlh1jbkTij19LAaWqBfURJ+lMn19Qf+fpzF/2mjAcYG3BvaUmLGhNsH
ontZquEzkVY1jf6Xv2+cmBe7kI4a5GmufNt/AaiMmIc=

tip

Removing tags from anchors and snippets

Add Results Builder and select the source from the dropdown list: $p1.serp.$i.anchor - Anchor. Select type: Remove HTML tags.

Add again Results Builder and select the source from the dropdown list: $p1.serp.$i.snippet - Snippet. Select type: Remove HTML tags.

Download example

How to import an example into A-Parser

eJyVVN9v2jAQ/lcii4dNQihs5SVvFA11Ey0dtA8T64NHLszDsT3bYaCI/313jknS
rqq0lyi+u++7X59dM8/d3t1bcOAdyzY1M+GfZWz9Kcu+cZXDMVlBqQ+Q3DzcLhLP
dy4prC6Tqdr+1NYlGJOslTCGKIbMcOvAEtmmx4GOHApeSc+GNfMnA5gCSa0VOaBT
5Hg2Vh9PFrwVQEwHLisKG6dpys5vwPgOtrpSvsOM3wiXQu2dAUuwDjFJz09PQ4bd
Y41urm3JaQoDMx7FkbTONT/Ag0ZnISR05jme7nhJKQc590DeURGI3r0f+SMx8DwX
XmjFZZOBxtRlfVTid+hYaYzFXxrEHGeNJg+BgIynS3UbNghnhhRVwH5tMCwruHQw
ZA5LnXMsJH/pER4s99ouDdWD9pppNZVyAQeQXVjgv66EzHGn0wJBnyPw9ZDlPxzn
tr1+KtzJH4s1tCzhdL287VC5Xugddp7/wL6lKIXHs5uFRWcsReMewLQzu6OZldpC
myYyx+wocgOKBNCtbGo607M2nq2lZ6yZ05XdYr5NOtwwFLqhrYabwEg/UXM23Jgb
X0pyW8tPaIzRnqQTISTT1xhdc5/+i/KCOSNmq1Uhdsuo/Es7lXrA675UM10aCTR8
VUmJ2nGw6jQ8dVErdOim+BI8CyloP5d7jWVo6b6sm3kaK7CqyTAW3s8aKbdcysfV
ou9hne7x8L1Kr64+0PfjJPyPGwsjrIedRuljZ9RwfHXal6zuvz1ZfUY9/XL3TRD1
RSFowwE5FAs9GH8BhLW+Jg==

tip

You can add the Results Builder as many times as you need.

Filtering links by inclusion

Add a filter and select from the dropdown list: $serp.$i.link - Link. Select type: Contains string. Next, in String specify the filtering criterion; for example, if we want only links containing .com, to be saved in the result, then write that in "String".

Download example

How to import an example into A-Parser

eJx9VE2P2jAQ/SvI4tBKCMFhL7mxqEit6LJd2EOFOHjxJHJxbNd2KCjKf++ME+Kw
rfaW+Xhv3nw4NQvcn/yzAw/Bs2xfMxu/Wca2X7LsJ9cCLqOVVAGc1MXo7TryIX5J
PVJSn9iEWe48OELvByAMCMh5pQKb1CxcLSCnOYNzUgAGpUDbOnO5OkBG8Og7c1VR
2nw2m7HmAxgv4GgqHRJm/kE66fQWHMES4mE2gOSxQwxi9yQ52zPsyaIjNnmYdBm7
Nv1odOBSDyRPj6ZE09ggjUbbg/asORxujH5lXMlprmM7n3ZD7oNbfoadaXVAcuPc
4YmXxD8WPABFp3kk+vR5Gi7EwIWQVJOrtgLtIVV91fJ31KcN5uInTXrlUGvGAkQC
cl5v6vZsHG3quIrYHy2GZTlXHibMo9QVRyHifUTifHgwbhNngP6aGb1Qag1nUCkt
8j9WUgk8mkWOoK8d8P8pm384mr69YSlc+h+HGnqWaD1uvieUMGtTYOfiLS62lAFt
v4yXlLEZOk8Atp/ZE82sNA76Mh1zVx2fjQVNF5ZWtrDJddfG3VrunXhNuSw23dXe
Miu9w7e50UtTWgXUl66UwrV4eEnnsfDdGshIAt+Dl7EEtX57kywYo/y3bSvVOonn
90ACS5zksGpHeeRKvb6shxGWTiqekyfaI95pYfCCsIvm0P8d+l9MPfxHZHWDa/nl
n9sk6oFS0IfD8PElzZu/Z6ed/A==

tip

Possible settings

note

General settings for all scrapers

Parameter name	Default value	Description
AntiGate preset	`default`	Choice of Util::AntiGate preset, more details on setting here
AntiGate preset for old captcha	`default`	Similar to AntiGate preset, but used only for regular (old, single image) captchas. If no preset is selected here, the preset selected in AntiGate preset will be used for such captchas.
Experimental img captcha max count	`5`	Maximum number of repeated captcha images per attempt
Preffered captcha type	`Click`	Choice of preferred captcha type: Click or Puzzle
Engine	`HTTP (Fast, JavaScript Disabled)`	Allows choosing the engine: HTTP (faster, but higher probability of captchas) or browser (slower, but lower probability of captchas)
Device	`Modern desktop computer (Windows 10, Chrome 84)`	Choice of results type (Desktop computer / Mobile device)
Pages count	`5`	Number of pages to scrape (from 1 to 25)
Sort serp by date	`☐`	Sorting results by date
Serp time	`All time`	Search period
Yandex domain	`www.yandex.ru`	Yandex domain for scraping; all domains are supported (.ru, .ua, .by, .kz, .com.tr, .com). Starting from version 1.1.345, it is selected automatically depending on the selected region.
Region of serp (lr=)	`Moscow`	Choice of search region (lr= parameter)
Custom region ID		Ability to specify a region ID not present in the selection field. This option takes precedence over the Region of serp (lr=) option. A corresponding domain must be set in Yandex domain.
Search sites from (rstr=)	`Not set`	Choice of regional site binding (rstr= parameter)
Language	`Any`	Search results language (Russian, English, Belorussian, French, German, Indonesian, Kazakh, Tatar, Turkish, Ukrainian)
Parse not found	`☑`	Determines whether to scrape results if zero results are found for the search query and results for another query are suggested
Not personalized	`☐`	Search personalization. More details here
Filter pages	`Moderate filter`	Filtering results from unwanted content (Family search / Moderate filter / Do not filter)
Use Accounts	`☐`	Working with existing accounts in the files/SE-Yandex/accounts.txt file. SE::Yandex::Register - Allows registering accounts in Yandex
Remove bad accounts	`☑`	Removing invalid accounts
Quick answers count	`0`	Maximum number of questions-answers (Quick answers) per query that the scraper should collect
Parse generative answer	`☐`	Whether to scrape the generative answer (this adds an additional subquery and accordingly slows down overall operation)
Accounts	`Only from "accounts.txt"`	Choice of account handling method: `Always auto register` - always automatically register accounts "on the fly", requires selecting a configured preset in the SE::Yandex::Register preset parameter. `Auto register if no more in "accounts.txt"` - first, existing accounts from accounts.txt are used, and if they run out, automatic registration "on the fly" is used, for which a configured preset must be selected in the SE::Yandex::Register preset parameter. `Only from "accounts.txt"` - use only existing accounts from accounts.txt, and if they run out, wait for the specified time (Wait new accounts in "accounts.txt" parameter) for new ones to appear. `Only by session_id from "accounts.txt"` - authorization by cookies.
Remove bad accounts	`Always, except wrong login/password`	Automatic removal of "bad" accounts: `Always` - always remove. `Always, except wrong login/password` - remove always, except when Yandex reported that an incorrect login/password was specified. The fact is that Yandex can give such a message when an IP is banned for a perfectly working account, so optionally such accounts can be left for reuse. `Never` - never remove. Regardless of the selected option, accounts are not removed in case of proxy/browser errors
Use sessions	`☑`	Using sessions
Wait new accounts in "accounts.txt"	`0`	Wait time for new accounts to appear in accounts.txt
SE::Yandex::Register preset	`default`	Choice of settings preset for SE::Yandex::Register
Force neuro	`☐`	Forced activation of neuro answer, similar to manual switching between "Search" and "Search with Alice"

Scraper overview​

Scraper use cases​

🔗 Yandex SERP and position number

🔗 Scraping Yandex lite search results

🔗 Scraping ads from Yandex

🔗 Competition assessment in Yandex

🔗 Yandex cache scraper

🔗 Scraping only snippets from Yandex

Collected data​

Capabilities​

Usage scenarios​

Queries​

Query substitutions​

Using operators​

Accounts​

Result output options​

Exporting a list of links​

Links + anchors + snippets with position output​

Outputting links, anchors, and snippets to a CSV table​

Outputting ad blocks​

Saving related keywords​

Keyword competition​

Determining keywords with errors​

Checking link indexing​

Saving in SQL format​

Dumping results to JSON​

Results processing​

Link deduplication​

Link deduplication by domain​

Extracting domains​

Removing tags from anchors and snippets​

Filtering links by inclusion​

Possible settings​

Scraper overview

Scraper use cases

Collected data

Capabilities

Usage scenarios

Queries

Query substitutions

Using operators

Accounts

Result output options

Exporting a list of links

Links + anchors + snippets with position output

Outputting links, anchors, and snippets to a CSV table

Outputting ad blocks

Saving related keywords

Keyword competition

Determining keywords with errors

Checking link indexing

Saving in SQL format

Dumping results to JSON

Results processing

Link deduplication

Link deduplication by domain

Extracting domains

Removing tags from anchors and snippets

Filtering links by inclusion

Possible settings