AIO Not Loaded...

[scrapefun]

I download the full Google result page for custom parsing with an external script. Sometimes the AIO result does not load like in the screenshot below.

Is there a way I can automatically retry the a query if this happens?

I currently do a custom check and save these queries to a file to try again later manually but it would be great to do be able to do this real time and just try the query again automatically.

 

[Support]

Google does not always give the AI overview directly in the page source, so the original SE::Google can detect this and make additional subqueries to download it. But the source is not edited in any way, and by outputting $pages.0.data or $data you get a page without the loaded AI overview in such cases.

Spoiler: Screenshots

Is there a way I can automatically retry the a query if this happens?

In theory, you can find a sign of the absence of AI overview in the source code and put the query back in the queue using tools.query.add(). But how much this will work in practice and whether it will solve the task of obtaining sources with AI answers present in them - I can't say.