curl

curl

Brackets in curl

http://erik.silfversten.se/basic-web-scraping-with-curl/

You can also download alphanumeric sequences by using square brackets:

www.site.com/archive/2015/[01-10].html

www.site.com/archive/2015/[a-h].html

You can also use multiple brackets in one command:

www.site.com/archive/20[10-15]/[01-10].html

When using brackets to curl multiple files we can dynamically allocate file names using ‘#’ in the –o option, which will replace the ‘#’ with the current string in the URL being downloaded.

curl www.site.com/archive/20[10-15]/[01-10].html -o #1_#2.html

use cases:

• When I wanted to re-read Rice Boy, I was able to download the entire run with curl http://www.rice-boy.com/see/[001-440].gif -o "#1.gif"