Archiving a password-protected site with wget

The combination of <a href="http://man.cx/wget%22 title="wget's man page">wget</a> and the Export Cookies add-on for Firefox is useful for creating offline, complete, static archives of websites that are only accessible with a password:

  1. First log in to the site and export cookies.txt,
  2. Then run
    wget \
    --recursive \
    --no-clobber \
    --page-requisites \
    --html-extension \
    --convert-links \
    --restrict-file-names=windows \
    --domains example.com \
    --no-parent \
    --load-cookies cookies.txt \
    --reject logout,admin* \
    example.com/sub/dir
    

The rejection of logout URLs is especially useful, because otherwise one will probably be logged out by wget accessing the logout link.


Retrieved from ‘https://samwilson.id.au/index.php?title=Archiving_a_password-protected_site_with_wget&oldid=87