wget and proxy

Posted under » Ubuntu » Linux » Raspberry Pi on 13 October 2013

Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

Normally we download stuff using the browser but the main drawback with this method is that if you close the browser, the download got cancelled and you have to redownload from the beginning. So what I normally do is get the download link from a browser, then a use wget to download it. Handy if the file is 2gb in size. The Pi can do this in the background while you do other stuff on the main computer. It will retry if there is a disconnection.

In some cases, wget will give up. So to continue where you left, use the '-c' or '--continue' option.

wget --limit-rate=100k -O /cruzer/WBA-ARS-EPL_H.mkv http://soccer.com/dl/WBA-ARS-EPL_H.mkv

In this example I set a limit rate to 100k and specify the download file location and name using the -O option.

You can RTFM.

Proxy

Sometimes it is useful to use http proxies to download stuff.

export http_proxy=http://proxy.anoneh.com:8080/

To check that everything is ok, run

env | grep proxy

To reset or undo the proxy

unset http_proxy;

If you need APIs for your wget, then you should try CURL.