1. Вступайте в наш Telegram чат: https://t.me/a_parser Нас уже 250 и мы растем!
    Скрыть объявление

Ошибка [1.2.482] SE::AOL не энкодит query, пустые результаты

Тема в разделе "1.2.484", создана пользователем siventes, 7 май 2019.

  1. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    Версия v1.2.480
    В случае с отсутствием энкодинга параметров запроса, добавить нечего.
    С пустыми результатами - скорее всего парсер неправильно "детектит" "плохой" прокси. По каким то причина пачка пустых результатов именно при старте парсинга, дальше реже.
    на 7К результатов, пришлось 1.3К "пустых", что совсем немало.
     
  2. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    Отредактировать первое сообщение не вышло, не понравилось движку форума.
    Пресет:
    eyJwcmVzZXQiOiJTRTpBT0wiLCJ2YWx1ZSI6eyJwcmVzZXQiOiJTRTpBT0wiLCJw
    YXJzZXJzIjpbWyJTRTo6QU9MIiwiZGVmYXVsdCIseyJ0eXBlIjoib3ZlcnJpZGUi
    LCJpZCI6InVzZXByb3h5IiwidmFsdWUiOnRydWV9LHsidHlwZSI6Im92ZXJyaWRl
    IiwiaWQiOiJwcm94eXJldHJpZXMiLCJ2YWx1ZSI6IjIwMCJ9LHsidHlwZSI6Im92
    ZXJyaWRlIiwiaWQiOiJwYWdlX2NvdW50IiwidmFsdWUiOjV9LHsidHlwZSI6Im92
    ZXJyaWRlIiwiaWQiOiJwcm94eWJhbm5lZGNsZWFudXAiLCJ2YWx1ZSI6IjAifSx7
    InR5cGUiOiJvdmVycmlkZSIsImlkIjoidGltZW91dCIsInZhbHVlIjoiMzAifSx7
    InR5cGUiOiJvdmVycmlkZSIsImlkIjoicmVxdWVzdGRlbGF5IiwidmFsdWUiOiIw
    In0seyJ0eXBlIjoib3ZlcnJpZGUiLCJpZCI6ImRvbWFpbiIsInZhbHVlIjoic2Vh
    cmNoLmFvbC5jb20ifSx7InR5cGUiOiJvdmVycmlkZSIsImlkIjoiZXh0cmFxdWVy
    eSIsInZhbHVlIjoiIn1dXSwicmVzdWx0c0Zvcm1hdCI6IlslXG5saW1pdD0yO1xu
    cmVzdWx0LmFkc19jb3VudD1wMS5hZHMuc2l6ZTtcbnJlc3VsdC5xdWVyeT1wMS5x
    dWVyeS5xdWVyeTtcbnJlc3VsdC5xdWVyeV9sdmw9cXVlcnkubHZsO1xucmVzdWx0
    LnNlcnA9W107XG5yZXN1bHQucmVsYXRlZD1bXTtcbnJlc3VsdC5hZHM9W107XG5y
    ZXN1bHQudG90YWxjb3VudD1wMS50b3RhbGNvdW50O1xuXG5GT1JFQUNIIGkgPSBw
    MS5yZWxhdGVkO1xuXHRrZXk9aS5yZWxhdGVkIHwgdHJpbTtcblx0cmVzdWx0LnJl
    bGF0ZWQucHVzaChrZXkpO1xuXHRJRiBxdWVyeS5sdmwgPCAgbGltaXQ7XG5cdFx0
    dG9vbHMucXVlcnkuYWRkKGtleSk7XG5cdEVORDtcbkVORDtcbkZPUkVBQ0ggaSA9
    IHAxLnNlcnA7XG5cdHNuaXBwZXQgPSBpLnNuaXBwZXQgfCB0cmltO1xuXHRzbmlw
    cGV0ID0gc25pcHBldC5yZXBsYWNlKCdcXG4nLCAnJyk7XG5cdGFuY2hvciA9IGku
    YW5jaG9yIHwgdHJpbTtcblx0YW5jaG9yID0gYW5jaG9yLnJlcGxhY2UoJ1xcbics
    ICcnKTtcblx0cmVzID0ge1xuXHQgICBzbmlwcGV0ID0+IHNuaXBwZXQsXG5cdCAg
    IGFuY2hvciA9PiBhbmNob3IsXG5cdCAgIGxpbmsgPT4gaS5saW5rLFxuICAgIH07
    XG5cdHJlc3VsdC5zZXJwLnB1c2gocmVzKTtcbkVORDtcbkZPUkVBQ0ggaSA9IHAx
    LmFkcztcblx0c25pcHBldCA9IGkuc25pcHBldCB8IHRyaW07XG5cdHNuaXBwZXQg
    PSBpLnNuaXBwZXQucmVwbGFjZSgnXFxuJywgJycpO1xuXHRhbmNob3IgPSBpLmFu
    Y2hvciB8IHRyaW07XG5cdGFuY2hvciA9IGFuY2hvci5yZXBsYWNlKCdcXG4nLCAn
    Jyk7XG4gICAgcmVzID0ge1xuXHQgICBzbmlwcGV0ID0+IHNuaXBwZXQsXG5cdCAg
    IGFuY2hvciA9PiBhbmNob3IsXG4gICAgICAgcG9zaXRpb24gPT5pLnBvc2l0aW9u
    LFxuICAgICAgIGxpbmsgPT4gaS5saW5rLFxuICAgIH07XG5cdHJlc3VsdC5hZHMu
    cHVzaChyZXMpO1xuRU5EO1xucmVzdWx0Lmpzb24oKSB8IHRyaW07XG5cIlxcblwi
    O1xuJV0iLCJyZXN1bHRzU2F2ZVRvIjoiZmlsZSIsInJlc3VsdHNGaWxlTmFtZSI6
    IiRkYXRlZmlsZS5mb3JtYXQoKS50eHQiLCJhZGRpdGlvbmFsRm9ybWF0cyI6W10s
    InJlc3VsdHNVbmlxdWUiOiJubyIsInF1ZXJpZXNGcm9tIjoiZmlsZSIsInF1ZXJ5
    Rm9ybWF0IjpbIiRxdWVyeSJdLCJ1bmlxdWVRdWVyaWVzIjp0cnVlLCJzYXZlRmFp
    bGVkUXVlcmllcyI6ZmFsc2UsIml0ZXJhdG9yT3B0aW9ucyI6eyJvbkFsbExldmVs
    cyI6ZmFsc2UsInF1ZXJ5QnVpbGRlcnNBZnRlckl0ZXJhdG9yIjpmYWxzZSwicXVl
    cnlCdWlsZGVyc09uQWxsTGV2ZWxzIjpmYWxzZX0sInJlc3VsdHNPcHRpb25zIjp7
    Im92ZXJ3cml0ZSI6ZmFsc2UsIndyaXRlQk9NIjpmYWxzZX0sImRvTG9nIjoibm8i
    LCJrZWVwVW5pcXVlIjoiTm8iLCJtb3JlT3B0aW9ucyI6ZmFsc2UsInJlc3VsdHNQ
    cmVwZW5kIjoiIiwicmVzdWx0c0FwcGVuZCI6IiIsInF1ZXJ5QnVpbGRlcnMiOltd
    LCJyZXN1bHRzQnVpbGRlcnMiOlt7InNvdXJjZSI6WzAsWyJyZWxhdGVkIiwicmVs
    YXRlZCJdXSwidHlwZSI6ImRlY29kZUh0bWwiLCJhcnJheSI6InJlbGF0ZWQiLCJ0
    byI6InJlbGF0ZWQifSx7InNvdXJjZSI6WzAsWyJyZWxhdGVkIiwicmVsYXRlZCJd
    XSwidHlwZSI6InJlbW92ZUh0bWwiLCJhcnJheSI6InJlbGF0ZWQiLCJ0byI6InJl
    bGF0ZWQifSx7InNvdXJjZSI6WzAsWyJzZXJwIiwic25pcHBldCJdXSwidHlwZSI6
    ImRlY29kZUh0bWwiLCJhcnJheSI6InNlcnAiLCJ0byI6InNuaXBwZXQifSx7InNv
    dXJjZSI6WzAsWyJzZXJwIiwic25pcHBldCJdXSwidHlwZSI6InJlbW92ZUh0bWwi
    LCJhcnJheSI6InNlcnAiLCJ0byI6InNuaXBwZXQifSx7InNvdXJjZSI6WzAsWyJz
    ZXJwIiwiYW5jaG9yIl1dLCJ0eXBlIjoiZGVjb2RlSHRtbCIsImFycmF5Ijoic2Vy
    cCIsInRvIjoiYW5jaG9yIn0seyJzb3VyY2UiOlswLFsic2VycCIsImFuY2hvciJd
    XSwidHlwZSI6InJlbW92ZUh0bWwiLCJhcnJheSI6InNlcnAiLCJ0byI6ImFuY2hv
    ciJ9XSwiY29uZmlnT3ZlcnJpZGVzIjpbXSwicnVuVGFza09uQ29tcGxldGUiOm51
    bGwsInVzZVJlc3VsdHNGaWxlQXNRdWVyaWVzRmlsZSI6ZmFsc2UsInJ1blRhc2tP
    bkNvbXBsZXRlQ29uZmlnIjoiZGVmYXVsdCIsInRvb2xzSlMiOiIiLCJwcmlvIjo1
    LCJyZW1vdmVPbkNvbXBsZXRlIjpmYWxzZSwiY2FsbFVSTE9uQ29tcGxldGUiOiIi
    LCJxdWVyaWVzRmlsZSI6WyJxdWVyaWVzL3NyYy9qb2JfMV9maWx0ZXJlZF9hbGwu
    dHh0Il0sImNhdGVnb3J5IjpbXX19
    Пустые результаты:
    {"ads":[],"related":[],"query":"find jobs in my area","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"best way to find jobs in los angeles","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"how to find remote jobs at startups","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"find tutoring jobs near me","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"find odd jobs near me","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"get jobs near me","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"find jobs close to me","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"find cash jobs near me","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
    {"ads":[],"related":[],"query":"find jobs near me for 15 year olds","ads_count":"","totalcount":"none","serp":[],"query_lvl":0}
     
    #2 siventes, 7 май 2019
    Последнее редактирование: 7 май 2019
  3. Support

    Support Administrator
    Команда форума A-Parser Enterprise

    Регистрация:
    16 мар 2012
    Сообщения:
    2.448
    Симпатии:
    1.551
    Запустите задание с включенным логом и покажите лог по одному из "пустых" результатов.
     
  4. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    А разделе "Parser Test" никак не удается пока воспроизвести проблему, чтобы дать исходный код страницы. Сейчас перепишу результаты, чтобы получить исходники html для понимания что это такое вообще.
    Несколько логов, что я просмотрел, выглядят примерно одинаково:
    Код:
    Thread #73 07/05 14:46:21 - 07/05 14:46:28 SE::AOL::0(Complete): find jobs applied linkedin
    07/05 14:46:21
    Parser SE::AOL::0 parse query find jobs applied linkedin
    07/05 14:46:21
    Use proxy socks://ip:port
    07/05 14:46:21
    Parse page 1
    07/05 14:46:27
    GET(1): https://search.aol.com/aol/search?q=find jobs applied linkedin&ei=UTF-8&btf= - 200 OK (23.16 KB)
    07/05 14:46:27
    Decode from utf-8(meta http-equiv)
    07/05 14:46:27
    Results count not found
    07/05 14:46:27
    Total found 0 results
    07/05 14:46:27
    No more results
    
     
    #4 siventes, 7 май 2019
    Последнее редактирование: 7 май 2019
  5. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    К сожалению, не могу предоставить нужные данные для дебага, так как в парсере куда то "пропал" раздел сырых данных - $p1.data.
    Прошу починить это в первую очередь в целях отладки.
    Result format:
    Код:
    $p1.json();
    
    Результат:
    {"related":[{"related":"jobs hiring near me"},{"related":"best job for you test"},{"related":"why are you looking for new opportunities"},{"related":"what job is right for you"},{"related":"find jobs near my location"},{"related":"finding employment opportunities"},{"related":"looking for a job online"},{"related":"post a job for free"}],"success":1,"info":{"success":1,"retries":1,"stats":{"success":1,"retries":1,"proxiesUsed":1,"requests":1,"queries":1}},"query":{"first":"find jobs for me","num":0,"lvl":0,"prev":"","threadId":"0","query":"find jobs for me","queryUid":"9e39a4","orig":"find jobs for me"},"serp":[{"link":"https://www.monster.com/jobs/search/","snippet":"Explore all the new jobs in the US from entry level to management roles. Browse by location, industry or company. Find the right job for you and build a career.","anchor":"Job Search: Find Job Openings | Monster.com"},{"link":"https://www.whatcareerisrightforme.com/","snippet":"Career Aptitude Test Take our free career test to determine what jobs are best suited to your skills and interests.. The test is composed of four brief sections covering your: SKILLS: What are your personal strengths and weakness? INTERESTS: What sorts of activities and subjects do you find interesting? STYLE: What style of work environment are you looking for?","anchor":"What Career is Right For Me?"},{"link":"https://www.monster.com/","snippet":"Monster is your source for jobs and career opportunities. Search for jobs, read career advice from Monster's job experts, and find hiring and recruiting advice.","anchor":"Monster Jobs - Job Search, Career Advice & Hiring ..."},{"link":"https://www.carecareers.com.au/page/career-quiz","snippet":"job search Will this suit me? Looking for more flexibility, greater challenges? More rewarding work? The care sector could be for you. Find out who it suits. Take our Career Quiz! Try our Career Quiz - it's a fun and easy way to see what career is right for you. Let Employers Find You ...","anchor":"Career Quiz | Care Careers"},{"link":"http://time.com/4343767/job-personality-work/","snippet":"Feeling the itch to change jobs, or even careers? Before you take the plunge, try this quiz to see which job best fits your personality. Find Out What Job Best Matches Your Personality","anchor":"Find Out What Job Best Matches Your Personality - time.com"},{"link":"https://www.roberthalf.com/blog/job-market/find-me-a-job-how-a-recruiting-agency-can-help","snippet":"Expert help in finding a job. Working with a recruiter is almost as simple as saying, \"Please find me a job.\" Once you reach out, a recruiter can act as your liaison to hiring managers, share industry information and career resources that support your job hunt, and even coach you on the finer points of assembling your job-search materials.","anchor":"Find Me a Job! How a Recruiting Agency Can Help | Robert Half"},{"link":"https://www.careerbuilder.com/jobs-full-time","snippet":"Search CareerBuilder for Full Time Jobs and browse our platform. Apply now for jobs that are hiring near you.","anchor":"Full Time Jobs - Apply Now | CareerBuilder"},{"link":"https://www.payscale.com/data-packages/best-jobs","snippet":"Basically, you're constantly trying to find out what job is best for you, no matter how old or how experienced you are. That's why PayScale created our Best Job for Me quiz. Learn About Jobs That ...","anchor":"PayScale's Best Jobs Tool - What Job Is Best For You?"},{"link":"https://www.indeed.com/find-jobs.jsp","snippet":"Find jobs. indeed.com one search. all jobs.","anchor":"Find jobs | Indeed.com"},{"link":"https://www.indeed.com/","snippet":"With Indeed, you can search millions of jobs online to find the next step in your career. With tools for job search, resumes, company reviews and more, we're with you every step of the way.","anchor":"Job Search | Indeed"}],"totalcount":728000000};
     
  6. Support

    Support Administrator
    Команда форума A-Parser Enterprise

    Регистрация:
    16 мар 2012
    Сообщения:
    2.448
    Симпатии:
    1.551
    Добавлена проверка контента, теперь при получении страницы, отличной от страницы с результатами (или сообщением об отсутствии результатов), парсер будет повторять попытку.
    Также добавлена возможность выводить контент страниц $pages.$i.data
    Запросы энкодились и ранее.
     
  7. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    Судя по логу в предыдущем моем сообщении, или запросы не энкодились или лог "врет".
    Код:
    GET(1): https://search.aol.com/aol/search?q=find jobs applied linkedin&ei=UTF-8&btf= - 200 OK (23.16 KB)
    
     
  8. Support

    Support Administrator
    Команда форума A-Parser Enterprise

    Регистрация:
    16 мар 2012
    Сообщения:
    2.448
    Симпатии:
    1.551
  9. siventes

    siventes A-Parser Pro License
    A-Parser Pro

    Регистрация:
    4 май 2019
    Сообщения:
    13
    Симпатии:
    0
    Понял, спасибо, в заблуждения я был введен потому что в других парсерах все строчки в логах с энкодом, в этом - нет.
    За обновление спасибо, потестирую.
     

Поделиться этой страницей