IN GOD WE TRUST

It is a common practice to manage UNIX/Linux/BSD server remotely over ssh session. As you manage servers, you need to download the software or other files for installation or even download latest ISO of Linux distribution (or even MP3s). These days we have lots of GUI downloaders for X window such as:

* d4x: http://www.krasu.ru/soft/chuchelo
* kget: KDE download manager
* gwget2 – GNOME 2 wget front-end

However, when it comes to command line (shell prompt) wget the non-interactive downloader rules. It supports http, ftp, https protocols along with authentication facility, and tons of options. Here are some tips to get most out of it:

Télécharger un fichier avec wget:

$ wget http://www.cyberciti.biz/here/lsst.tar.gz
$ wget ftp://ftp.freebsd.org/pub/sys.tar.gz

Télécharger plusieurs fichiers avec wget

$ wget http://www.cyberciti.biz/download/lsst.tar.gz ftp://ftp.freebsd.org/pub/sys.tar.gz ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm

OU

i) Créez une variable qui enregistre tous les liens puis utilisez BASH boucle For’ pour télécharger tous les fichiers:

$ URLS=”http://www.cyberciti.biz/download/lsst.tar.gz ftp://ftp.freebsd.org/pub/sys.tar.gz ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm http://xyz.com/abc.iso"

ii) Utilisez la boucle For comme suit:

$ for u in $URLS; do wget $u; done

iii) Cependant, vous pouvez insérer tous les liens dans un fichier texte et utiliser l’option -i dans wget pour télécharger tous les fichiers:

(a) Créez le fichier texte en utilisant vi

$ vi /tmp/download.txt

Ajoutez la liste des liens:

http://www.cyberciti.biz/download/lsst.tar.gz
ftp://ftp.freebsd.org/pub/sys.tar.gz
ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm

http://xyz.com/abc.iso

(b) Executez wget comme suit:

$ wget -i /tmp/download.txt

(c) Forcez wget à reprendre les téléchargement

You can use -c option to wget. This is useful when you want to finish up a download started by a previous instance of wget and the net connection was lost. In such case you can add -c option as follows:

$ wget -c http://www.cyberciti.biz/download/lsst.tar.gz
$ wget -c -i /tmp/download.txt

Please note that all ftp/http server does not supports the download resume feature.

Force wget to download all files in background, and log the activity in a file:

$ wget -cb -o /tmp/download.log -i /tmp/download.txt

OR

$ nohup wget -c -o /tmp/download.log -i /tmp/download.txt &
nohup runs the given COMMAND (in this example wget) with hangup signals ignored, so that the command can continue running in the background after you log out.

Limit the download speed to amount bytes/kilobytes per seconds.

This is useful when you download a large file file, such as an ISO image. Recently one of admin started to download SuSe Linux DVD on one of production server for evaluation purpose. Soon wget started to eat up all bandwidth. No need to predict end result of such a disaster.

$ wget -c -o /tmp/susedvd.log --limit-rate=50k ftp://ftp.novell.com/pub/suse/dvd1.iso

Use m suffix for megabytes (–limit-rate=1m). Above command will limit the retrieval rate to 50KB/s. It is also possible to specify disk quota for automatic retrievals to avoid disk DoS attack. Following command will be aborted when the quota is (100MB+) exceeded.

$ wget -cb -o /tmp/download.log -i /tmp/download.txt --quota=100m

F) Use http username/password on an HTTP server:

$ wget –http-user=foo –http-password=bar http://cyberciti.biz/vivek/csits.tar.gz

G) Download all mp3 or pdf file from remote FTP server:
Generally you can use shell special character aka wildcards such as *, ?, [] to specify selection criteria for files. Same can be use with FTP servers while downloading files.

$ wget ftp://somedom.com/pub/downloads/*.pdf
$ wget ftp://somedom.com/pub/downloads/*.pdf

OR
$ wget -g on ftp://somedom.com/pub/downloads/*.pdf

H) Use aget when you need multithreaded http download:
aget fetches HTTP URLs in a manner similar to wget, but segments the retrieval into multiple parts to increase download speed. It can be many times as fast as wget in some circumstances( it is just like Flashget under MS Windows but with CLI):

$ aget -n=5 http://download.soft.com/soft1.tar.gz

Above command will download soft1.tar.gz in 5 segments.
Please note that wget command is available on Linux and UNIX/BSD like oses.
See man page of wget(1) for more advanced options.

Voici quelques astuces pour trouver des fichiers hébérgés sur les sites d’upload comme rapidshare.de , depositfiles.com , megaupload.com , mediafire.com ….
Puisque ces sites ne permettent pas de chercher directement les fichiers qu’ils hébèrgent, on va fouiller le site par un autre moteur de recherche, voici la methode:
Par exemple: je cherche une video du coup de tête de Zizou à Meterrazzi…
-Allons sur google.com ou .fr si vous voulez
-puis il suffit de mettre le debut de l’url de l’hebergeur entre des apostrophes suivi du nom du fichier recherché.

Voici quelques exemples:

rapidshare.de :	"rapidshare.de/files/" coup de boule
depositfiles.com :	"depositfiles.com/files/" coup de boule
mediafire.com :	"mediafire.com/?" coup de boule

P.S: la video de Zizou vous pouvez la trouver sur youtube.com sans se casser la tête.