В последнее время часто стали поступать жалобы на плохой парсинг Гугла с использованием поисковых операторов. И действительно, Гугл начал еще жестче банить частые запросы с использованием операторов inurl, site и т.п.
Но благодаря этому посту от Master появилась идея парсить Гугл через сервис disconnect.me.
Но благодаря этому посту от Master появилась идея парсить Гугл через сервис disconnect.me.
Код:
eyJwcmVzZXQiOiJzZWFyY2guZGlzY29ubmVjdC5tZSIsInZhbHVlIjp7InByZXNl
dCI6InNlYXJjaC5kaXNjb25uZWN0Lm1lIiwicGFyc2VycyI6W1siTmV0OjpIVFRQ
IiwiZGVmYXVsdCIseyJ0eXBlIjoib3ZlcnJpZGUiLCJpZCI6Imdvb2RDb2RlIiwi
dmFsdWUiOjIwMH0seyJ0eXBlIjoib3ZlcnJpZGUiLCJpZCI6InF1ZXJ5Zm9ybWF0
IiwidmFsdWUiOiJodHRwczovL3NlYXJjaC5kaXNjb25uZWN0Lm1lL3NlYXJjaFRl
cm1zL3NlYXJjaD9zdGFydD1uYXYmb3B0aW9uPVdlYiZxdWVyeT0kcXVlcnkmc2Vz
PUdvb2dsZSZsb2NhdGlvbl9vcHRpb249UlUmbmV4dERERz0lMkZzZWFyY2glM0Zx
JTNEJTI2bmV3d2luZG93JTNEMSUyNmhsJTNEcnUlMjZzdGFydCUzRCRwYWdlbnVt
JTI2c2ElM0ROJnNob3dJY29ucz1mYWxzZSZmaWx0ZXJJY29ucz1ub25lJmpzX2Vu
YWJsZWQ9MCZzb3VyY2U9Tm9uZSJ9LHsidHlwZSI6Im92ZXJyaWRlIiwiaWQiOiJm
b3JtYXRyZXN1bHQiLCJ2YWx1ZSI6IiRzZXJwLmZvcm1hdCgnJGxpbmtcXG4nKSJ9
LHsidHlwZSI6ImN1c3RvbVJlc3VsdCIsInJlc3VsdCI6WyJwYWdlcyIsImRhdGEi
XSwicmVnZXgiOiI8dWwgaWQ9XCJub3JtYWwtcmVzdWx0c1wiKC4rPyk8XFwvdWw+
IiwicmVnZXhUeXBlIjoicyIsInJlc3VsdFR5cGUiOiJhcnJheSIsImFycmF5TmFt
ZSI6InNlcnBkYXRhIiwicmVzdWx0cyI6WyJkYXRhIl19LHsidHlwZSI6ImN1c3Rv
bVJlc3VsdCIsInJlc3VsdCI6WyJzZXJwZGF0YSIsImRhdGEiXSwicmVnZXgiOiI8
bGk+Lis/PGEgaHJlZj1cIiguKz8pXCIuKz8+KC4rPyk8Lis/PHA+KC4rPyk8Iiwi
cmVnZXhUeXBlIjoic2ciLCJyZXN1bHRUeXBlIjoiYXJyYXkiLCJhcnJheU5hbWUi
OiJzZXJwIiwicmVzdWx0cyI6WyJsaW5rIiwiYW5jaG9yIiwic25pcHBldCJdfSx7
InR5cGUiOiJvcHRpb25zIiwiaWQiOiJ1c2VQYWdlcyIsInZhbHVlIjoiMCIsImFk
ZGl0aW9uYWwiOnsicGFnZXNTdGVwIjoiMTAiLCJwYWdlY291bnQiOiI1In19LHsi
dHlwZSI6Im9wdGlvbnMiLCJpZCI6ImNoZWNrTmV4dFBhZ2UiLCJ2YWx1ZSI6Ijxk
aXYgY2xhc3M9XCJwYWdpbmF0aW9uXCI+XFxzKi4rPGEgaHJlZj0nLis/JyBkYXRh
LWluc3RhbnQ+Lis/PFxcL2E+XFxzKjxcXC9kaXY+In1dXSwicmVzdWx0c0Zvcm1h
dCI6IiRwMS5wcmVzZXQiLCJyZXN1bHRzU2F2ZVRvIjoiZmlsZSIsInJlc3VsdHNG
aWxlTmFtZSI6IiRkYXRlZmlsZS5mb3JtYXQoKS50eHQiLCJhZGRpdGlvbmFsRm9y
bWF0cyI6W10sInJlc3VsdHNVbmlxdWUiOiJubyIsInF1ZXJ5Rm9ybWF0IjpbIiRx
dWVyeSJdLCJ1bmlxdWVRdWVyaWVzIjpmYWxzZSwic2F2ZUZhaWxlZFF1ZXJpZXMi
OmZhbHNlLCJpdGVyYXRvck9wdGlvbnMiOnsib25BbGxMZXZlbHMiOmZhbHNlLCJx
dWVyeUJ1aWxkZXJzQWZ0ZXJJdGVyYXRvciI6ZmFsc2V9LCJyZXN1bHRzT3B0aW9u
cyI6eyJvdmVyd3JpdGUiOmZhbHNlfSwiZG9Mb2ciOiJkYiIsImtlZXBVbmlxdWUi
OiJObyIsIm1vcmVPcHRpb25zIjpmYWxzZSwicmVzdWx0c1ByZXBlbmQiOiIiLCJy
ZXN1bHRzQXBwZW5kIjoiIiwicXVlcnlCdWlsZGVycyI6W10sInJlc3VsdHNCdWls
ZGVycyI6W3sic291cmNlIjpbMCxbInNlcnAiLCJhbmNob3IiXV0sInR5cGUiOiJk
ZWNvZGVIdG1sIiwiYXJyYXkiOiJzZXJwIiwidG8iOiJhbmNob3IifSx7InNvdXJj
ZSI6WzAsWyJzZXJwIiwic25pcHBldCJdXSwidHlwZSI6ImRlY29kZUh0bWwiLCJh
cnJheSI6InNlcnAiLCJ0byI6InNuaXBwZXQifV0sImNvbmZpZ092ZXJyaWRlcyI6
W119fQ==- Парсятся ссылки ($link), анкоры ($anchor) и сниппеты ($snippet).
- Реализована воможность задавать кол-во страниц для парсинга (по-умолчанию - 5).
- Если в выдаче меньше страниц, чем задано для поиска - "пустые" запросы не делаются.
- Можно менять язык результатов (он же страна для поиска):
AF - Afghanistan
AL - Albania
DZ - Algeria
AS - American Samoa
AD - Andorra
AO - Angola
AI - Anguilla
AG - Antigua and Barbuda
AR - Argentina
AM - Armenia
0AC - Ascension Island
AU - Australia
AT - Austria
AZ - Azerbaijan
BS - Bahamas
BH - Bahrain
BD - Bangladesh
BY - Belarus
BE - Belgium
BZ - Belize
BJ - Benin
BT - Bhutan
BO - Bolivia
BA - Bosnia and Herzegovina
BW - Botswana
BR - Brazil
IO - British Indian Ocean Territory
VG - British Virgin Islands
BN - Brunei
BG - Bulgaria
BF - Burkina Faso
MM - Burma
BI - Burundi
KH - Cambodia
CM - Cameroon
CA - Canada
CV - Cape Verde
0CA - Catalonia Catalan Countries
CF - Central African Republic
TD - Chad
CL - Chile
CN - China
CC - Cocos (Keeling) Islands
CO - Colombia
CK - Cook Islands
CR - Costa Rica
HR - Croatia
CU - Cuba
CY - Cyprus
CZ - Czech Republic
CD - Democratic Republic of the Congo
DK - Denmark
DJ - Djibouti
DM - Dominica
DO - Dominican Republic
EC - Ecuador
EG - Egypt
SV - El Salvador
EE - Estonia
ET - Ethiopia
FM - Federated States of Micronesia
FJ - Fiji
FI - Finland
FR - France
GF - French Guiana
GA - Gabon
GM - Gambia
GE - Georgia
DE - Germany
GH - Ghana
GI - Gibraltar
GR - Greece
GL - Greenland
GP - Guadeloupe
GT - Guatemala
0GG - Guernsey
GY - Guyana
HT - Haiti
HN - Honduras
HK - Hong Kong
HU - Hungary
IS - Iceland
IN - India
ID - Indonesia
IR - Iran
IQ - Iraq
IE - Ireland
IM - Isle of Man
IL - Israel
IT - Italy
CI - Ivory Coast
JM - Jamaica
JP - Japan
0JE - Jersey
JO - Jordan
KZ - Kazakhstan
KE - Kenya
KI - Kiribati
KW - Kuwait
KG - Kyrgyzstan
LA - Laos
LV - Latvia
LB - Lebanon
LS - Lesotho
LY - Libya
LI - Liechtenstein
LT - Lithuania
LU - Luxembourg
MK - Macedonia
MG - Madagascar
MW - Malawi
MY - Malaysia
MV - Maldives
ML - Mali
MT - Malta
MU - Mauritius
MX - Mexico
MD - Moldova
MN - Mongolia
ME - Montenegro
MS - Montserrat
MA - Morocco
MZ - Mozambique
NA - Namibia
NR - Nauru
NP - Nepal
NL - Netherlands
NZ - New Zealand
NI - Nicaragua
NE - Niger
NG - Nigeria
NU - Niue
NF - Norfolk Island
NO - Norway
OM - Oman
PK - Pakistan
PS - Palestine
PA - Panama
PG - Papua New Guinea
PY - Paraguay
PE - Peru
PH - Philippines
PN - Pitcairn Islands
PL - Poland
PT - Portugal
PR - Puerto Rico
QA - Qatar
CG - Republic of the Congo
RO - Romania
RU - Russia
RW - Rwanda
SH - Saint Helena
LC - Saint Lucia
VC - Saint Vincent and the Grenadines
WS - Samoa
SM - San Marino
ST - Sao Tome and Principe
SA - Saudi Arabia
SN - Senegal
RS - Serbia
SC - Seychelles
SL - Sierra Leone
SG - Singapore
SK - Slovakia
SI - Slovenia
SB - Solomon Islands
SO - Somalia
ZA - South Africa
KR - South Korea
ES - Spain
LK - Sri Lanka
SE - Sweden
CH - Switzerland
TW - Taiwan
TJ - Tajikistan
TZ - Tanzania
TH - Thailand
TL - Timor-Leste
TG - Togo
TK - Tokelau
TO - Tonga
TT - Trinidad and Tobago
TN - Tunisia
TR - Turkey
TM - Turkmenistan
UG - Uganda
UA - Ukraine
AE - United Arab Emirates
UK - United Kingdom
US - United States
VI - United States Virgin Islands
UY - Uruguay
UZ - Uzbekistan
VU - Vanuatu
VE - Venezuela
VN - Vietnam
ZM - Zambia
ZW - Zimbabwe
AL - Albania
DZ - Algeria
AS - American Samoa
AD - Andorra
AO - Angola
AI - Anguilla
AG - Antigua and Barbuda
AR - Argentina
AM - Armenia
0AC - Ascension Island
AU - Australia
AT - Austria
AZ - Azerbaijan
BS - Bahamas
BH - Bahrain
BD - Bangladesh
BY - Belarus
BE - Belgium
BZ - Belize
BJ - Benin
BT - Bhutan
BO - Bolivia
BA - Bosnia and Herzegovina
BW - Botswana
BR - Brazil
IO - British Indian Ocean Territory
VG - British Virgin Islands
BN - Brunei
BG - Bulgaria
BF - Burkina Faso
MM - Burma
BI - Burundi
KH - Cambodia
CM - Cameroon
CA - Canada
CV - Cape Verde
0CA - Catalonia Catalan Countries
CF - Central African Republic
TD - Chad
CL - Chile
CN - China
CC - Cocos (Keeling) Islands
CO - Colombia
CK - Cook Islands
CR - Costa Rica
HR - Croatia
CU - Cuba
CY - Cyprus
CZ - Czech Republic
CD - Democratic Republic of the Congo
DK - Denmark
DJ - Djibouti
DM - Dominica
DO - Dominican Republic
EC - Ecuador
EG - Egypt
SV - El Salvador
EE - Estonia
ET - Ethiopia
FM - Federated States of Micronesia
FJ - Fiji
FI - Finland
FR - France
GF - French Guiana
GA - Gabon
GM - Gambia
GE - Georgia
DE - Germany
GH - Ghana
GI - Gibraltar
GR - Greece
GL - Greenland
GP - Guadeloupe
GT - Guatemala
0GG - Guernsey
GY - Guyana
HT - Haiti
HN - Honduras
HK - Hong Kong
HU - Hungary
IS - Iceland
IN - India
ID - Indonesia
IR - Iran
IQ - Iraq
IE - Ireland
IM - Isle of Man
IL - Israel
IT - Italy
CI - Ivory Coast
JM - Jamaica
JP - Japan
0JE - Jersey
JO - Jordan
KZ - Kazakhstan
KE - Kenya
KI - Kiribati
KW - Kuwait
KG - Kyrgyzstan
LA - Laos
LV - Latvia
LB - Lebanon
LS - Lesotho
LY - Libya
LI - Liechtenstein
LT - Lithuania
LU - Luxembourg
MK - Macedonia
MG - Madagascar
MW - Malawi
MY - Malaysia
MV - Maldives
ML - Mali
MT - Malta
MU - Mauritius
MX - Mexico
MD - Moldova
MN - Mongolia
ME - Montenegro
MS - Montserrat
MA - Morocco
MZ - Mozambique
NA - Namibia
NR - Nauru
NP - Nepal
NL - Netherlands
NZ - New Zealand
NI - Nicaragua
NE - Niger
NG - Nigeria
NU - Niue
NF - Norfolk Island
NO - Norway
OM - Oman
PK - Pakistan
PS - Palestine
PA - Panama
PG - Papua New Guinea
PY - Paraguay
PE - Peru
PH - Philippines
PN - Pitcairn Islands
PL - Poland
PT - Portugal
PR - Puerto Rico
QA - Qatar
CG - Republic of the Congo
RO - Romania
RU - Russia
RW - Rwanda
SH - Saint Helena
LC - Saint Lucia
VC - Saint Vincent and the Grenadines
WS - Samoa
SM - San Marino
ST - Sao Tome and Principe
SA - Saudi Arabia
SN - Senegal
RS - Serbia
SC - Seychelles
SL - Sierra Leone
SG - Singapore
SK - Slovakia
SI - Slovenia
SB - Solomon Islands
SO - Somalia
ZA - South Africa
KR - South Korea
ES - Spain
LK - Sri Lanka
SE - Sweden
CH - Switzerland
TW - Taiwan
TJ - Tajikistan
TZ - Tanzania
TH - Thailand
TL - Timor-Leste
TG - Togo
TK - Tokelau
TO - Tonga
TT - Trinidad and Tobago
TN - Tunisia
TR - Turkey
TM - Turkmenistan
UG - Uganda
UA - Ukraine
AE - United Arab Emirates
UK - United Kingdom
US - United States
VI - United States Virgin Islands
UY - Uruguay
UZ - Uzbekistan
VU - Vanuatu
VE - Venezuela
VN - Vietnam
ZM - Zambia
ZW - Zimbabwe
- Поддерживаются все поисковые операторы Гугла.
