Automatically Control Size of Results Files

[scrapefun]

It would be great if there was a way to control the size of results files.

Sometimes they can contain over 10 million lines and get very large and hard to handle. So if there was a way to create a new result file when a certain number of lines is reached or a certain file size is reached that would be great.

 

[Forbidden]

Easy solution:

results splitted approximately by 1000 lines

 

[scrapefun]

Great! This is perfect. Thanks for the help.

 

[scrapefun]

How would I add "$datefile.format()" to the filename?

I've tried adding it to the solution you gave but the files are not split properly?

 

[Forbidden]

You can't mix together $datefile.format() and other variables because it will create new date each time when save new portion of results

 

[scrapefun]

ok, thanks for clarifying and for the help!