[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wget
Subject:    Suggested wget feature - removing old files with -m (mirror) option
From:       Alan Kent <ajk () mds ! rmit ! edu ! au>
Date:       2002-05-27 8:44:06
[Download RAW message or body]

Hi.

wget is nice, useful, etc, but it appears that if you use the -m option
(to mirror a site) and the site changes its URL structure (eg: renames
a top level directory), then wget does not remove all the old files (at
the old URLs).

A suggestion is to add yet-another-option to go through all the files
in the directory tree and delete ones not referenced at the end of
a recursive fetch. Otherwise the mirror area will only get bigger and
bigger as old files no longer referenced from current HTML pages
(and no longer present on the site being mirrored) will never be
deleted from the local cache.

My current solution is to delete the old cache each time to make sure.
I could do it every 10th time or something, but it just makes it harder
to automate using cron files etc.

Sorry if this is a common issue. I am not a subscriber (and don't really
have time to subscribe etc). Just thought I would make a suggestion in
case someone is keen!

Alan
[prev in list] [next in list] [prev in thread] [next in thread]