Google Pagination...

[scrapefun]

Since Google has ended support of the "num=" operator, I need to scrape results by page but sometimes the results for one or several pages is not included in the results.

I'm just trying to save the full results page to a single html using format like:
$p1.pages.0.data\n$p1.pages.1.data\n$p1.pages.2.data\n$p1.pages.3.data\n$p1.pages.4.data\n

Also tried this format:
$pages.format('$data\n')

A good number of queries only have 2 or 3 pages instead of the full 5. These are queries that definitely have more than 5 pages of results.

What is the correct way to format results to ensure all pages are included?

I'm using version 1.2.2939

 

[bielawa]

Hi, did you tried to set to $p1.json ?
linksperpage looks abandoned right now.

 

[Support]

Send for diagnostics:
1) one or more sets (each set of pages is a separate archive) in which 2-3 pages were received, but there should be more
2) code for importing a task with included queries, by which this can be reproduced in the parser

 

[scrapefun]

I figured this out. Thanks.