Reddit::Comments - Reddit comment scraper
Overview of the Reddit::Comments scraper
Reddit::Comments - Reddit comments scraper.Collects a list of comments and a wealth of information for each of them from the service of the same name.
You can use automatic query multiplication, substitution of subqueries from files, iteration of alphanumeric combinations and lists to obtain the maximum possible number of results.
A-Parser functionality allows you to save Reddit::Posts scraper settings for future use (presets), set a scraping schedule, and much more.
Saving results is possible in any form and structure you need, thanks to the built-in powerful Template Toolkit which allows applying additional logic to results and outputting data in various formats, including JSON, SQL, and CSV.
Collected data
Array of comments:
- Link to the comment
- Comment content (in markdown)
- Rating and number of awards
- Comment creation date
- Comment author and their flair
- Link to the post the comment belongs to
- Post title and its flair
- Post rating, number of comments, and number of awards
- Post creation date
- Community where the post is published
- Post author and their flair
- Post content: text in markdown, link to media content, and link to an external resource
Features
- Specify the number of pages to scrape
- Specify the result sorting method
- Ability to scrape within a specific community
Use Cases
- Any scenarios where collecting comments left on Reddit posts is required
Queries
Two query options are supported:
Keywords
Example:
wordpress features
parser
The default result will output a list of links to comments, for example:
https://www.reddit.com/r/node/comments/14lmqbq/how_to_work_with_xlsx_files/jpy3r5a/
https://www.reddit.com/r/StardewValley/comments/14qidly/having_problems_installing_stardew_valley/jqnalwz/
https://www.reddit.com/r/elasticsearch/comments/14pr86i/how_to_parsing_this_lin_logstash/jqkstjw/
https://www.reddit.com/r/vexillology/comments/14fh5th/flag_of_riga_michigan/jp10w17/
https://www.reddit.com/r/Marvel/comments/14otc3t/hank_pym_is_a_really_humble_guy_the_mighty/jqf27xy/
https://www.reddit.com/r/math/comments/14p1lkg/from_the_perspective_of_you_mathematicians_what/jqgug4q/
https://www.reddit.com/r/Wordpress/comments/14okx06/help_looking_for_a_specific_plugin_for_booking/jqhwtu5/
https://www.reddit.com/r/osr/comments/13u8g7s/difference_between_whitebox_whitehack/jlzhthi/
...
Keywords and community links
The scraper supports searching by keyword within a specific community. To do this, specify the keyword and the community link separated by a space in the query. Example:
jesus https://www.reddit.com/r/atheism/
stage 3 https://www.reddit.com/r/Audi/
The default result will output a list of links to posts, for example:
https://www.reddit.com/r/atheism/comments/14dp1rv/sen_josh_hawley_shares_his_mindblowingly_stupid/jor20zd/
https://www.reddit.com/r/atheism/comments/14kt69e/why_do_my_christian_friends_view_my_atheism_as_an/jpsgbe5/
https://www.reddit.com/r/atheism/comments/14p6yir/finally_happened_the_one_babysitter_we_can_get/jqhk48s/
https://www.reddit.com/r/Audi/comments/14nyn9m/excuse_me_we_late/jqbdu2a/
https://www.reddit.com/r/Audi/comments/14oqxce/talk_me_inout_of_buying_this_gorgeous_audi_s5/jqev0p6/
https://www.reddit.com/r/Audi/comments/14pqr8a/is_this_a_good_deal_in_your_guys_opinions/jql4wnb/
...
Result output options
A-Parser supports flexible result formatting thanks to the built-in Template Toolkit, which allows it to output results in any form, as well as in structured formats like CSV or JSON.
Possible settings
| Parameter | Default value | Description |
|---|---|---|
| Pages count | 5 | Number of search result pages |
| Sort | Relevance | Result sorting |