Wget and robots.txt

because it is not part of the manpage, i want to mention it here:

wget -r -e robots=off