Is it possible to only follow links in certain directories using the Link Extractor?
I want to scrape the Twitter profile directory and only want to follow links that contain "/i/directory/profiles/" in the URL.
I still want to collect all the links on those pages but only follow the links contain that url format (/i/directory/profiles/)
I want to scrape the Twitter profile directory and only want to follow links that contain "/i/directory/profiles/" in the URL.
I still want to collect all the links on those pages but only follow the links contain that url format (/i/directory/profiles/)