[prev in list] [next in list] [prev in thread] [next in thread] 

List:       wget
Subject:    Workaround
From:       Dan Hartshorn <dharts () wpsee ! com>
Date:       1997-10-02 15:26:25
[Download RAW message or body]

I wrote earlier about running wget on HOHO, one of our linux boxes to
mirror school web sites. The challenge was then mirroring what was
collected to the NT web server TWINKIE which sits safely on a secure
server network. I couldn't run wget from TWINKIE itself because TWINKIE
cannot FTP out to anywhere due to the firewall and our security situation.

Thank you all for your suggestions and ensuing discussions. I won't weigh
in on the multi-threading debate, but I'll share with you what my solution
was. It did not include adding to or altering wget itself. I wrote a PERL
script that parses wget's transaction log to determine what files were
transferred. Then HOHO FTP's to TWINKIE and puts the appropriate files up
there. This way, my script could remain fairly dumb, quick to write, while
it uses wget's smarts in determining which files to transfer via the
transaction log. The end result meets our criteria: HOHO acts as an
intermediate agent between the school sites and our web server. I'd be
happy to share my PERL script if anyone's interested.

Dan Hartshorn
Intranet Technician
Kyrene School District
Tempe, AZ

[prev in list] [next in list] [prev in thread] [next in thread]