[prev in list] [next in list] [prev in thread] [next in thread]
List: webmin-l
Subject: Re: [webmin-l] Restricting file upload size
From: "Jamie Cameron" <jcameron () webmin ! com>
Date: 2006-11-28 17:35:36
Message-ID: 1164735336.31167 () www ! virtualmin ! com
[Download RAW message or body]
[Attachment #2 (text/html)]
Hi Dave,<br />I checked my code again, and found that only with forkcgis=1 does it \
not read the uploaded data into memory, sorry.<br />However, in that mode you need to \
make sure that all your .cgi scripts are executable (with chmod +x *.cgi).<br /><br \
/> - Jamie<br /><br />On 28/Nov/2006 08:02 Dave Isaacs wrote .. <blockquote \
type="cite">
<div><span class="146175715-28112006"><font size="2" face="Arial" color="#0000ff">I \
must be missing something. I've rerun my tests and they show the entire file
being uploaded before my module is being called. This is in the case when
forkcgis=0.</font></span></div>
<div><span class="146175715-28112006"><font size="2" face="Arial" \
color="#0000ff"></font></span> </div> <div><span class="146175715-28112006"><font \
size="2" face="Arial" color="#0000ff">When I set forkcgis=1, Webmin stops working \
completely. I just get a file permissions error showing up the in \
browser.</font></span></div> <div><span class="146175715-28112006"><font size="2" \
face="Arial" color="#0000ff"></font></span> </div> <div><span \
class="146175715-28112006"><font size="2" face="Arial" color="#0000ff">I am using \
Webmin version 1.290 on RedHat EL3.</font></span></div> <div><span \
class="146175715-28112006"><font size="2" face="Arial" color="#0000ff"></font></span> \
</div> <div><span class="146175715-28112006"><font size="2" face="Arial" \
color="#0000ff">Thanks</font></span></div> <div><span \
class="146175715-28112006"><font size="2" face="Arial" color="#0000ff"></font></span> \
</div> <div><span class="146175715-28112006"><font size="2" face="Arial" \
color="#0000ff">Dave I</font></span></div>
<blockquote style="margin-right: 0px;">
<div></div>
<div lang="en-us" align="left" dir="ltr" class="OutlookMessageHeader"><font \
size="2" face="Tahoma">-----Original Message-----<br /><b>From:</b> \
webadmin-list-bounces@lists.sourceforge.net \
[mailto:webadmin-list-bounces@lists.sourceforge.net] <b>On Behalf Of </b>Jamie \
Cameron<br /><b>Sent:</b> Monday, November 27, 2006 5:51 PM<br /><b>To:</b> Webmin \
users list<br /><b>Subject:</b> Re: [webmin-l] Restricting file upload size<br /><br \
/></font></div>Hi Dave,<br />Yes, for both forked and internally-executed Perl \
scripts it no longer reads the entire input into memory, at least not in \
miniserv.pl.<br /><br /> - Jamie<br /><br />On 27/Nov/2006 12:58 Dave Isaacs wrote \
.. <blockquote type="cite">
<div><span class="810325420-27112006"><font size="2" face="Arial" \
color="#0000ff">Thanks!</font></span></div> <div><span \
class="810325420-27112006"><font size="2" face="Arial" color="#0000ff"></font></span> \
</div> <div><span class="810325420-27112006"><font size="2" face="Arial" \
color="#0000ff">Did you also fix the forkcgis path? We now have the requirement to \
upload 500MB files to the server, so instead of limiting the file size we need to \
ensure that the file is not uploaded into memory first. If I remember correctly, \
the forkcgis path in miniserv.pl would do just that, but was not working \
correctly?</font></span></div> <div><span class="810325420-27112006"><font size="2" \
face="Arial" color="#0000ff"></font></span> </div> <div><span \
class="810325420-27112006"><font size="2" face="Arial" color="#0000ff">Forgive me if \
I am remembering this wrong. I am still working on dredging up all the details from \
my head.</font></span></div> <div><span class="810325420-27112006"><font size="2" \
face="Arial" color="#0000ff"></font></span> </div> <div><span \
class="810325420-27112006"><font size="2" face="Arial" \
color="#0000ff">Thanks</font></span></div> <div><span \
class="810325420-27112006"><font size="2" face="Arial" color="#0000ff"></font></span> \
</div> <div><span class="810325420-27112006"><font size="2" face="Arial" \
color="#0000ff">Dave I</font></span></div> <div><span \
class="810325420-27112006"><font size="2" face="Arial" color="#0000ff"></font></span> \
</div> <blockquote style="margin-right: 0px;">
<div></div>
<div lang="en-us" align="left" dir="ltr" class="OutlookMessageHeader"><font \
size="2" face="Tahoma">-----Original Message-----<br /><b>From:</b> \
webadmin-list-bounces@lists.sourceforge.net \
[mailto:webadmin-list-bounces@lists.sourceforge.net] <b>On Behalf Of </b>Jamie \
Cameron<br /><b>Sent:</b> Monday, November 27, 2006 3:52
PM<br /><b>To:</b> Webmin users list<br /><b>Subject:</b> Re: [webmin-l]
Restricting file upload size<br /><br /></font></div>Hi Dave,<br />Yes, this \
has
been fixed for a few versions now ..<br /><br /> - Jamie<br /><br />On
27/Nov/2006 12:42 Dave Isaacs wrote ..
<blockquote type="cite">
<p><font size="2">Jamie,</font> </p>
<p><font size="2">Almost a year ago I posted this question about
restricting file upload size. At the time you admitted to some
miniserv.pl limitations and said that you would take care of this.
</font></p>
<p><font size="2">Have you? If yes, great! If not, any
estimates on when this can get in?</font> </p>
<p><font size="2">Thanks</font> </p>
<p><font size="2">Dave I</font> </p>
<p><font size="2">From: Jamie Cameron <jcameron@webmin.com></font>
<br /><font size="2">To: webadmin-list@lists.sourceforge.net</font>
<br /><font size="2">Reply-To: webadmin-list@lists.sourceforge.net</font>
<br /><font size="2">Date: Feb 10 2006 - 6:38pm</font> </p>
<p><font size="2">On 11/Feb/2006 03:19 Dave Isaacs wrote ..</font>
<br /><font size="2">> My experience shows that this does not
work.</font> <br /><font size="2">></font> <br /><font size="2">> I put \
a 1000000 limit in my call to ReadParseMime then attempted to
upload</font> <br /><font size="2">> a</font> <br /><font size="2">> \
1GB
file. Using top, I watched the miniserv.pl process climb to about</font>
<br /><font size="2">> 600MB</font> <br /><font size="2">> before
crashing. ReadParseMime was never called because my module was</font>
<br /><font size="2">> never</font> <br /><font size="2">> \
invoked.</font>
<br /><font size="2">></font> <br /><font size="2">> If I look at
miniserv.pl, at around line 1740, I see</font> <br /><font \
size="2">></font> <br /><font size="2">> $clen =
$header{"content-length"};</font> <br /><font size="2">> if \
($method eq
"POST" && $clen_read < $clen) {</font> <br /><font \
size="2">>
# Still some more POST data to read</font> <br /><font size="2">>
while(length($postinput) < $clen) {</font> <br /><font size="2">> $buf
= &read_data($clen - length($postinput));</font> <br /><font \
size="2">> if (!length($buf)) {</font> <br /><font size="2">>
&http_error(500, "Failed to read ".</font> <br /><font \
size="2">>
"POST request");</font> <br /><font size="2">> }</font> <br \
/><font size="2">> $postinput .= $buf;</font> <br /><font \
size="2">> }</font>
<br /><font size="2">> }</font> <br /><font size="2">></font> <br \
/><font size="2">> This looks an awful lot like reading in the entire file \
upload. As a test,</font> <br /><font size="2">> I wrote the
length($postinput) value to a log file (right before the call</font>
<br /><font size="2">> to</font> <br /><font size="2">> read_data) and
found that miniserv.pl was looping in an attempt to read</font>
<br /><font size="2">> the</font> <br /><font size="2">> entire file
upload.</font> </p>
<p><font size="2">Hi Dave,</font> <br /><font size="2">You are absolutely
correct .. Webmin really does the whole posted input into</font>
<br /><font size="2">memory! Sorry, I totally forgot about that..</font>
</p>
<p><font size="2">> Then I stumbled upon the forkcgis configuration
setting, which appears</font> <br /><font size="2">> to</font> <br /><font \
size="2">> switch on a alternative method of invoking the webmin
modules. This method</font> <br /><font size="2">> has miniserv.pl
forwarding the file upload to the forked process as it</font> <br /><font \
size="2">> is</font> <br /><font size="2">> received. \
Unfortunately,
this does not work either. Now when I upload</font> <br /><font size="2">> \
a</font> <br /><font size="2">> large file, something goes
wrong and there is never a response. The log</font> <br /><font size="2">> \
messages I put in the miniserv loop shows that about 7500
bytes are read</font> <br /><font size="2">> in,</font> <br /><font \
size="2">> and then everything stops. Although this is better than
crashing the</font> <br /><font size="2">> server, it is still not
correct.</font> </p>
<p><font size="2">I looked into this too, and found that Webmin is
currently terminating the</font> <br /><font size="2">browser connection if
the uploaded data is more than the set limit. Unfortunately,</font>
<br /><font size="2">no browsers take kindly to this, and display an error
message about the connection</font> <br /><font size="2">being
closed.</font> </p>
<p><font size="2">In the next release of Webmin, it will handle this
better by reading all the data</font> <br /><font size="2">submitted by the
browser, but not actually storing it in memory if the limit is
exceeded.</font> <br /><font size="2">That is not quite ideal, but still
better than the current situation.</font> </p>
<p><font size="2">- Jamie</font>
</p></blockquote><br /></blockquote></blockquote><br \
/></blockquote></blockquote><br />
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
-
Forwarded by the Webmin mailing list at webadmin-list@lists.sourceforge.net
To remove yourself from this list, go to
http://lists.sourceforge.net/lists/listinfo/webadmin-list
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic