HTML Link Extractor: only crawl certain directories?

scrapefun

A-Parser Enterprise License
A-Parser Enterprise
Is it possible to only follow links in certain directories using the Link Extractor?

I want to scrape the Twitter profile directory and only want to follow links that contain "/i/directory/profiles/" in the URL.

I still want to collect all the links on those pages but only follow the links contain that url format (/i/directory/profiles/)
 
Nevermind I discovered the "$followlinks.$i.link - Link" filter and have it working as needed now.
 
Back
Top