SE::YouTube::Video - YouTube video data scraper
Scraper overview
YouTube video data scraper. Thanks to this scraper, you can scrape all basic video data, as well as subtitles and comments. You should use links to YouTube video pages as queries. You can collect video links using 
SE::YouTube. Using the YouTube video scraper, you can collect all video data in multithreaded mode.
A-Parser functionality allows you to save SE::YouTube::Video scraping settings for future use (presets), set up scraping schedules, and much more.
Results can be saved in any format and structure you need, thanks to the powerful built-in Template Toolkit which allows applying additional logic to results and outputting data in various formats, including JSON, SQL, and CSV.
Collected data
- Video title and description
- Video duration
- Number of views, likes, and comments
- Preview link
- Author name, links to their avatar and channel, and subscriber count
- Video subtitles (including timestamp information)
- List of tags
- List of comments (including replies to comments)
- Comment ID and parent comment ID (for replies)
- Author name, profile link, and avatar
- Comment text and publication time
- List of related videos
- Video link and title
- Author and date
- View count and video duration
- Information about video chapters ($chapters)
- Title, start time in seconds, and preview image link
Features
- Interface language selection
- Subtitle language selection
- Specifying the number of comment pages (approximately 20 comments per page)
- Specifying the maximum number of reply pages for each comment (approximately 10 replies on the first page, about 50 on subsequent pages)
- Specifying the number of related video pages (approximately 20 videos per page)
- Shorts support
Use Cases
- Gathering statistical data about YouTube videos
- Scraping subtitles and comments as a source of text
- Finding related videos
Operational details
Subtitle language selection logic
The scraper uses the following priority (in descending order): author-provided, author-provided translated, auto-generated, auto-generated translated.
For example, if the scraper is set to scrape English subtitles:
- if the video has author-provided English subtitles - author-provided will be scraped
- if the video has author-provided subtitles in another language - author-provided translated into English will be scraped
- if the video has no author-provided subtitles but has auto-generated in English - auto-generated will be scraped
- if the video has no author-provided subtitles and auto-generated are in another language (e.g. video is in another language) - auto-generated translated will be scraped
Parsing comments
Comments are collected in a single thread, so their scraping can be quite time-consuming, especially when scraping a large number of pages and replies. It is not recommended to set a large number of reply pages; usually 1-3 is enough, or you can disable reply scraping entirely - this will significantly speed up the process.
Queries
Video links must be specified as queries, for example:
https://www.youtube.com/watch?v=lWA2pjMjpBs
https://www.youtube.com/watch?v=EDwb9jOVRtU
https://www.youtube.com/watch?v=5NPBIwQyPWE
Output results examples
A-Parser supports flexible result formatting thanks to the built-in Template Toolkit, which allows it to output results in any form, as well as structured formats like CSV or JSON.
Default output
Result format:
$query - $title\nViews: $viewsCount, likes: $likesCount, comments: $commentsCount\n
The result will display the video link, its title, number of likes, views, and comments:
https://www.youtube.com/watch?v=5NPBIwQyPWE - Avril Lavigne - Complicated (Official Video)
Views: 571331713, likes: 3959948, comments: 143597
https://www.youtube.com/watch?v=EDwb9jOVRtU - Madonna - Hung Up (Official Video) [HD]
Views: 414662791, likes: 2153344, comments: 91895
https://www.youtube.com/watch?v=lWA2pjMjpBs - Rihanna - Diamonds
Views: 2104207258, likes: 10235971, comments: 394622
Subtitles output
Result format:
$query\n$subtitles.format('$text ')\n\n
The result will display the video link and subtitles in the specified language.
Output in CSV table
The built-in tools.CSVLine tool allows creating correct tabular documents ready for import into Excel or Google Sheets.
General result format:
[% tools.CSVline(query, p1.author, p1.date, p1.duration, p1.title, p1.viewsCount, p1.likesCount, p1.commentsCount, p1.tags.format('$tag,')) %]
File name:
$datefile.format().csv
Head text:
Link,Author,"Publish date",Duration,Title,"Views count","Likes count","Comments count",Tags
The Template Toolkit engine is used in the General result format.
In the result file name, you simply need to change the file extension to csv.
To make the "Prepend text" option available in the Task Editor, you need to activate "More options". In "Prepend text", write the column names separated by commas and make the second line empty.
Possible settings
| Parameter name | Default value | Description |
|---|---|---|
| Interface language | English | Interface language selection |
| Subtitles language | English | Subtitle language selection |
| Comments pages count | 5 | Number of comment pages |
| Pages count for replies | 3 | Number of reply pages for each comment |
| Pages count for related videos | 5 | Number of pages with related videos |
| Login required is error | ☑ | Tells the scraper to treat the authorization required message as an error and retry |