In this version was implemented a number of qualitative improvements, designed to increase parsing speed:
- Fast processing of UTF-8, a speed boost in some tasks up to 1.5x
- Implement a mechanism of analysis the variables used in the results, which allows to optimize the speed, extracting only the required data. The first optimized parser has become
SE::Google: two-time increase of limit speed of parsing, when collecting only links (7000-10000 requests per minute, 100 results per query, in other words, now you can get 1 million links for 1 - 1.5 minutes) - Now information about queries $response is generated only on demand
- Optimized work of template engine when using multiple parsers in the same task
- Now when you use the option "Run on complete" is used the preset name instead of the task number from queue
- In the Task Scheduler is also used preset name, that allows you to clean queue without the risk of losing the saved tasks for the scheduler
SE::Google - search in blogs has also increased the speed of parsing due to requests changes
SE::Yahoo,
SE::DuckDuckGo,
SE::AOL
- In the parser
Rank::Archive fixed a logic of the processing of error 403 Forbidden - Fixed crash when changing the number of threads in the task
- Fixed TLS support for some sites
- Fixed freezing of task when crashing worker process
- Now the task can be stopped if it is waiting slot
- Fixed option "Save window size"
- Fixed error in
HTML::LinkExtractor when working Parse to level option in cases, when the site gave a redirect to another page