Enhancements
- Now for parsers
Net::HTTP,
HTML::TextExtractor,
HTML::LinkExtractor,
Rank::CMS cached same HTTP requests when using any combination of these parsers in the one task, which prevents the execution of repeated requests to the same URL, as a consequence of increasing speed, reducing the load on the target sites - Added an option Max connections per host allows you to limit the maximum number of simultaneous connections to the same domain/IP
- Added an option Global proxy ban allows share the statistics for ban of proxies between tasks
- In the parser
HTML::TextExtractor improved display of tabular data and lists - In the parser
SE::Yandex::WordStat added the ability to specify a minimum frequency for adding a request for parsing nested - In the parser
SE::Yandex added option Parse all results to automatically bypass the restriction on the size of the output and to collect more results - In the task editor added function Copy overrides allows you to copy the settings from one parser to another
- In the task editor added function Save overrides to preset allows you to create a new preset based on the redefined values
- When viewing logs with the option "Failed only" now displays only those threads that were failed queries
- When using xPath in conjunction with record individual queries into one array data are filled evenly
- In the task queue added last modified date when the task is displayed in a compact form
- In the task queue adds the ability to download the results file when an incomplete parsing
- When using option "Save failed queries" now additionally stored original queries for the entire task
- Added an option Report captcha - recognized by Google captchas are sent to our server for analysis, is enabled by default
Fixes
- Fixed problem with large memory consumption on Windows
- Fixed wrong saving unique option in some cases
- Did not work the replacement on the group in a regular expression in the Query Builder
- In the parser
HTML::LinkExtractor fixed parsing of links with spaces - Fixed a problem with the display of tasks in the queue with a large number of queries added through the text field