SE::Startpage::Images - Startpage images scraper

Scraper overview

Startpage search engine image scraper. Thanks to the SE::Startpage::Images scraper, you will be able to obtain image databases ready for further use. You can use queries in the same form as you enter them into the Startpage search bar.

The A-Parser functionality allows you to save the settings of the Startpage image scraper for future use (presets), set a scraping schedule, and much more. You can use automatic query multiplication, substitution of subqueries from files, and iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.

Saving results is possible in the form and structure you need, thanks to the built-in powerful Template Toolkit template engine, which allows you to apply additional logic to the results and output data in various formats, including JSON, SQL, and CSV.

Go to DEMO Buy A-Parser Pro ($299)

Collected data

Links, anchors, and image snippets
Links to source pages
Links to previews
Image width and height
Image types

Capabilities

Scrapes the maximum number of results provided - 50 pages with 10 elements per result page
Supports selection of search country, results language, and interface language
Ability to specify filters by size, color, type, and usage rights

Use cases

Collecting images for filling your blogs
Collecting avatar databases

Queries

Search phrases should be specified as queries, for example:

Waterfall  
Speak in english  
Cats and dogs  
cars

Query substitutions

You can use built-in macros for query multiplication; for example, if we want to get a very large database of forums, we will specify several main queries in different languages:

forum
forum
foro
论坛

In the query format, we will specify a character iteration from a to zzzz; this method allows for maximum rotation of search results and obtaining many new unique results:

$query {az:a:zzzz}

This macro will create 475254 additional queries for each original search query, which in total will give 4 x 475254 = 1901016 search queries—an impressive figure, but not a problem for A-Parser. At a speed of 2000 queries per minute, such a task will be processed in just 16 hours.

Output results examples

A-Parser supports flexible result formatting thanks to the built-in Template Toolkit template engine, which allows it to output results in an arbitrary form, as well as in a structured one, such as CSV or JSON.

Default output

Result format:

$serp.format('$link\n')

Result example:

https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg
https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943
https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg
https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg
https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO
https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc
https://code.org/shared/images/social-media/codeorg2020_social.png
https://miro.medium.com/max/12032/0*ghyQGW_ZCFN_afQ0
https://www.ionos.com/digitalguide/fileadmin/DigitalGuide/Teaser/code-editoren-t.jpg

Output in CSV table

Result format:

[% FOREACH item IN serp;
    tools.CSVline(query, item.link, item.width, item.height,  item.page, item.thumb);
END %]

Result example:

code,https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg,3888,2592,https://techcrunch.com/2016/05/10/please-dont-learn-to-code/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU,jpg
code,https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943,960,540,https://www.forbes.com/sites/enriquedans/2020/08/09/could-the-no-code-movement-put-programmers-out-of-ajob/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU,jpg
code,https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg,1200,630,https://www.educationandcareernews.com/stem-education/5-reasons-students-should-learn-to-code/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU,jpg
code,https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg,744,389,https://interestingengineering.com/the-best-ways-to-learn-how-to-code,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT90pAbc4h3wggUFQ20JsRJ_XeKAZ0wurtRsOO2rKsUaW4cKx61SntGDg8cpqZrZEcP-Ls&usqp=CAU,jpg
code,https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO,4000,2666,https://medium.com/dealeron-dev/how-to-write-readable-code-8434c58748a1,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcScySt4wtbwVa_vJIqVVmxkEF943I48xpvWVEbmqOd-PWFgpD1CBCyntWST12MKN7KLrnI&usqp=CAU,
code,https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc,980,659,https://www.welcometothejungle.com/en/collections/behind-the-code,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQOQYtxNtGy1qVMxUO42XWb75nYjE_w6fMlpJNaDxbro0ZorYIcfS0EPhmTEzhSyHBpvvo&usqp=CAU,
code,https://code.org/shared/images/social-media/codeorg2020_social.png,1200,630,https://code.org/,https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRB6y-6WBM9qNNlVnSnQtFPoHSLh-W0koeQvT_iKLmHasR-h7fPe7Y-U2327bZaRcZV1jk&usqp=CAU,png

Saving in SQL format

Result format:

[% FOREACH serp;
    "INSERT INTO serp VALUES('" _ query _ "', '"; link _ "', '";  page _ "', '";    thumb _ "')\n";
END %]

Result example:

INSERT INTO serp VALUES('code', 'https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://code.org/shared/images/social-media/codeorg2020_social.png', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRB6y-6WBM9qNNlVnSnQtFPoHSLh-W0koeQvT_iKLmHasR-h7fPe7Y-U2327bZaRcZV1jk&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://cdn-images.welcometothejungle.com/5DDbrp9_kdlw05Z0hzc7kYEpaaruHhUUWJqs-nW1o8k/rs:auto:980::/q:85/czM6Ly93dHRqLXByb2R1Y3Rpb24vdXBsb2Fkcy9jYXRlZ29yeS9jb3Zlci8yNjYwLzE1NDg4My9jb2xsZWN0aW9uX2NhdGVnb3J5X2JlaGluZF90aGVfY29kZS5qcGc', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQOQYtxNtGy1qVMxUO42XWb75nYjE_w6fMlpJNaDxbro0ZorYIcfS0EPhmTEzhSyHBpvvo&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://inteng-storage.s3.amazonaws.com/img/iea/9lwjAVnM6E/sizes/ocde_resize_md.jpg', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcT90pAbc4h3wggUFQ20JsRJ_XeKAZ0wurtRsOO2rKsUaW4cKx61SntGDg8cpqZrZEcP-Ls&usqp=CAU')
INSERT INTO serp VALUES('code', 'https://miro.medium.com/max/11520/0*Jy3heMl_yP_fQwMO', '', 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcScySt4wtbwVa_vJIqVVmxkEF943I48xpvWVEbmqOd-PWFgpD1CBCyntWST12MKN7KLrnI&usqp=CAU')

Dump results to JSON

Common result format:

[% IF notFirst;
  ",\n";
ELSE;
  notFirst = 1;
END;

obj = {};
obj.query = query;
obj.images = [];

FOREACH item IN p1.serp;
    obj.images.push({
        width = item.width
        height = item.height
        link = item.link
        pagelink = item.pagelink
        thumb = item.thumb
    });
END;

obj.json %]

Initial text:

Final text:

Result example:

[{
    "images": [
        {
            "link": "https://techcrunch.com/wp-content/uploads/2015/04/codecode.jpg",
            "width": 3888,
            "page": "https://techcrunch.com/2016/05/10/please-dont-learn-to-code/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTda13SHf3DRDIZLWnKwu1HLr0JHXzs8QtiQs98Qc5GyEdD9JuHtdk9B8Qko_n-PdDT39k&usqp=CAU",
            "height": 2592
        },
        {
            "link": "https://specials-images.forbesimg.com/imageserve/5f302109ffad89f9130e07db/960x0.jpg?cropX1=0&cropX2=4800&cropY1=243&cropY2=2943",
            "width": 960,
            "page": "https://www.forbes.com/sites/enriquedans/2020/08/09/could-the-no-code-movement-put-programmers-out-of-ajob/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSyHw0ZZT5tdDwR4UrT0YNui_R-hKHUCgEYBhNKmTRyKRyZ5XQppBVbYfxa9Tds8Zhx5CI&usqp=CAU",
            "height": 540
        },
        {
            "link": "https://victoria.mediaplanet.com/app/uploads/sites/102/2019/07/mainimage-26.jpg",
            "width": 1200,
            "page": "https://www.educationandcareernews.com/stem-education/5-reasons-students-should-learn-to-code/",
            "thumb": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQCPak-ygQMNGgIy404t7FticAgqunQe72andfS-QRpaRAno2oJ_qmOshBp85jhLuekQGs&usqp=CAU",
            "height": 630
        },
        
    ],
    "query": "code"
}]

tip

To make the "Prepend text" and "Append text" options available in the Task Editor, you need to activate "More options".

Possible settings

note

Common settings for all scrapers

Parameter	Default value	Description
Pages count	`5`	Number of pages to scrape
Family filter	`Filter depending on search`	Family filter
Search language	`English`	Selection of results language
Search country	`All`	Selection of the country from which the search is performed
Page language	`English`	Selection of interface language
Image size	`Any`	Selection of image size
Image color	`Any`	Selection of image color
Image type	`Any`	Selection of image type
License	`Any`	License for image usage

Scraper overview​

Collected data​

Capabilities​

Use cases​

Queries​

Query substitutions​

Output results examples​

Default output​

Output in CSV table​

Saving in SQL format​

Dump results to JSON​

Possible settings​

Scraper overview

Collected data

Capabilities

Use cases

Queries

Query substitutions

Output results examples

Default output

Output in CSV table

Saving in SQL format

Dump results to JSON

Possible settings