SE::Google::KeywordPlanner::SearchVolume - estimating search volume and other keyword metrics from Google Keyword Planner

Overview of the parser

SE::Google::KeywordPlanner::SearchVolume – a parser for estimating search volumes of keywords using Google Keyword Planner. A wide range of data is available for retrieval: average monthly searches, quarterly and yearly changes in volume, minimum and maximum bids, and competition level. In the parsing settings, you can specify the language, location, advertising network, and data sampling period. A batch mode is also available, allowing you to retrieve data for up to 10,000 keywords in a single request to the service, significantly speeding up the parsing process.

Thanks to the multi-threaded operation of A-Parser combined with bulk mode, the query processing speed can reach several tens of thousands of queries per minute.

The functionality of A-Parser allows you to save parsing settings for the SE::Google::KeywordPlanner::SearchVolume parser for future use (presets), set a parsing schedule, and much more.

Saving results is possible in the form and structure you need, thanks to the built-in powerful Template Toolkit which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

Go to DEMOBuy A-Parser Pro ($299)

Collected data

• Average monthly searches for the searched keyword
• Quarterly change
• Year-over-year change
• Competition level
• Minimum and maximum bids
• Search volume trends for the selected period
  • Month and year
  • Total number of queries
  • Number of queries from mobile devices only (not available in bulk mode)

Capabilities

• Support for authorization via login-password or by substituting cookies and headers
• Determination of $volume accuracy degree - exact/rounded value
• Bulk (packet) mode is supported and enabled by default
• Support for multi-accounts (to select the required account, its ocid(uscid) must be specified)

Use cases

• Collecting average monthly searches, minimum and maximum bids, competition assessment
• Trend analysis

Setting up

There are two options for configuring the parser:

• specify the email\password for the Keyword Planner account
• authorize in the browser and copy the required values

caution

Be cautious with the number of threads. It is recommended to specify a small number of threads; parsing without a proxy is quite possible.

Authorization via email and password

You need to override the E-mail and Password options by specifying the credentials for your Keyword Planner account. A campaign must be created on the account.

Spoiler: (Solution) Login failed TypeError: Cannot read property '1' of null

If this error occurs, you need to remove your Google account from the browser and log in again.

Browser authorization and header substitution in the parser

You need to authorize in the browser at https://ads.google.com/aw/keywordplanner/home, create the first campaign if it hasn't been created yet, take the following data and specify it in the parser settings:

Cookies can be specified in two ways:

• Specify all cookies in the All cookies option
• Specify values from cookies for the __Secure-3PSID, __Secure-3PSIDTS options (__Secure-3PSIDTS needs to be specified if authuser on the account is 0)

Other headers:

• Value of the x-framework-xsrf-token header
• Value of the ocid or uscid parameter from the URL
• Value of the authuser parameter from the URL

Spoiler: How to find the necessary parameters

Queries

Keywords should be specified as queries, one keyword per line. Example queries:

coca-cola
parsing
peace
starlink
test

Bulk (packet) mode is supported, which is activated by the Bulk (packet) mode option. In this mode, the parser will send packets of 10,000 keywords in a request to the service. In this mode, data on the number of queries from mobile devices ($trends.$i.mobile) is not collected. Bulk mode is enabled by default.

Query substitutions

You can use built-in macros for automatic substitution of subqueries from files; for example, if we want to add a list of other words to each query, let's specify several main queries:

fantasy
tower defense
rpg

In the query format, we will specify a substitution macro for additional words from the keywords.txt file; this method allows increasing the variety of queries manifold:

{subs:keywords} $query 

This macro will create as many additional queries as there are in the file for each original search query, which in total will give [number of original queries] x [number of queries in the Keywords file] = [total number of queries] as a result of the macro's work.

For example, if the keywords.txt file contains:

free
online

As a result, the substitution macro will turn 3 main queries into 6:

free fantasy
online fantasy
free tower defense
online tower defense
free rpg
online rpg

Output results examples

A-Parser supports flexible result formatting thanks to the built-in Template Toolkit engine, which allows it to output results in arbitrary form, as well as in structured form, such as CSV or JSON

Default output

Result format:

$query: $volume\n

Example result with the output of a keyword and its average monthly search volume:

coca-cola: 3350000
parsing: 165000
peace: 673000
starlink: 1830000
test: 7480000

Output to a CSV table

General result format:

[% tools.CSVline(query, p1.volume, p1.min_bid, p1.max_bid) %]

File name:

$datefile.format().csv

Initial text:

Keyword,Volume,"Min bid","Max bid"

tip

In the General result format, the Template Toolkit engine and the tools.CSVline tool are used for data output.

In the result file name, you just need to change the file extension to csv.

To make the "Initial text" option available in the Task Editor, you need to activate "More options". In "Initial text", write the column names separated by commas and make the second line empty.

Saving in SQL format

Result format:

[% "INSERT INTO volumes VALUES('" _ query _ "', '" _ volume _ "')\n" %]

Example result:

INSERT INTO volumes VALUES('perfume', '50000')
INSERT INTO volumes VALUES('eyfel perfume', '5000')
INSERT INTO volumes VALUES('memo marfa', '5000')

Dump results to JSON

General output format:

[% IF notFirst;
  ",\n";
ELSE;
  notFirst = 1;
END;

obj = {};
obj.keyword = query;
obj.volume = p1.volume;

obj.json %]

Initial text:

[

Final text:

]

Example result:

[{"keyword":"perfume","volume":"50000"},
{"keyword":"eyfel perfume","volume":"5000"},
{"keyword":"memo marfa","volume":"5000"}]

tip

More details about outputting results to JSON are described in this article.

Possible settings

note

Common settings for all parsers

Parameter	Default value	Description
All cookies		Specifying all cookies
Cookie "__Secure-3PSID"		Cookie "__Secure-3PSID"
Cookie "__Secure-3PSIDTS"		Cookie "__Secure-3PSIDTS"
Header "x-framework-xsrf-token"		Header "x-framework-xsrf-token"
Url parameter "ocid"("uscid")		Parameter "ocid"("uscid")
Url parameter "authuser"	0	Parameter "authuser"
E-mail		E-mail for authorization in Keyword Planner
Password		Password for authorization in Keyword Planner
Recovery e-mail		Recovery e-mail
Browser headless (debug auth)	☑	Headless mode for the browser used for login-password authorization
Log Login Screenshot (debug auth)	☐	Creating a screenshot of the authorization page and outputting it to the task log
Date from	Last 12 months	Date from
Date to	Last 12 months	Date to
Language	English	Language
Search networks	Google	Search network
Location code		Location (here you need to specify the location id, which can be taken from the first column of this table (copy))
Delete created plan	☑	Delete created plan
Bulk (packet) mode	☑	Enable bulk mode