[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cgiapp
Subject:    Re: [cgiapp] large uploads handling
From:       Giannis Economou <gecon () di ! uoa ! gr>
Date:       2006-04-12 18:41:00
Message-ID: 443D49BC.6000506 () di ! uoa ! gr
[Download RAW message or body]



Michael Peters wrote:
> Giannis Economou wrote:
>   
>> Hello all...
>>
>> Ok this is a weird problem I'm facing. I installed a webapp based on
>> CGI::Application that I created on the production server. My development
>> server runs debian, the prod. server freebsd. I'm using SpeedyCGI and
>> application is running fine in my dev. server. In the production server
>> everything is running ok except one thing:
>> When trying to CGI upload files that are larger than about 100-150 Kb I
>> always get a 500 error. Apache error log empty. Not even produced by my
>> script as it seems.
>>     
>
> Do both servers have the same version of CGI.pm? 

Yes! Line by line compared...

> CGI.pm can also set the
> limit of the POSTs using $CGI::POST_MAX. Although the docs say that if
> it detects an upload larger than this amount it will produce an error.
>
>   

$CGI:POST_MAX is set by me in CGI.pm. Also uploads enabled in CGI.pm and 
tried /tmp and ~/tmp for saving temporary files.
Moreover, a simple CGI.pm script works without any problem (see below).

>> The same application under speedy on my dev. server handles these
>> uploads without problem. Tested uploads on prod. server without using my
>> application (just by using a simple CGI script I've created) and
>> everything went fine. I set my own TMPDIR for the uploads and in the
>> production server you can watch the upload as it is written in the
>> TMPDIR getting larger and suddently it stops getting bigger. 
>>     
>
> What size do the files end up being when they stop growing? Are they all
> the same or similar sizes?
>
>   
Uploading 800Kb size file.
Size of new file in TMPDIR (where the file ends up on server) varies. 
 From 80Kb to 320 Kb. Never more...
>> After a
>> while the browser gives the 500 error.
>> Production server is shared hosting and it seems they are using
>> setrlimit to put some limits on all processes, but I don't see anything
>> killed and I don't understand what limit could that be since all other
>> application requests are handled fine, except uploads that are larger
>> that the above mentioned size, only through my app.pl.
>>     
>
> setrlimit can restrict processes based on simply the amount of CPU time
> they use. Maybe since the request takes a long time to complete, you're
> hitting some timeout on their part. Have you contacted the hosting
> company about this?
>
>   

Asked hosting company. The said about resources and setrlimit, but in 
the logs nothings seems killed. No strict timeout limits by them. 
Anyway, this is not the case (I mean timeouts), because a simple CGI.pm 
script works always ok, regardless of the size of the upload! I created 
something simple for testing, like:
use CGI qw(:all); print header; my $f = upload('form_file_field'); 
#print the file
...
post to this one and it works _always_ fine, regardless of the size of 
the upload.
A perl script that doesn't even use CGI, (just dumps the STDIN) also 
works always fine.

>> I'm using CGI::Application::Dispatch and mod_rewrite rules in a
>> .htaccess and I start thinking that maybe it is a complication by the
>> dispatcher in the way?
>>     
>
> I don't think mod_rewrite or Dispatch have anything to do with this. I'm
> pretty sure that they would both just ignore any POST data. Besides, by
> the time CGI start's writing out the tmp files for the upload, both
> mod_rewrite and Dispatch have long since been forgotten.
>
>   
If Dispatch is forgotten then why don't I get any messages or error_log 
or in any log (I've put some print messages, plus everything in the C::A 
I use for logging) before the upload gets started? I've put print 
messages just after the shebang of the instance script. Nothing.

Yesterday moved the application to a new server, linux based and it is 
working without this strangeness. Guess I will never find out the reason 
for this...

Thank you!
Giannis
 

---------------------------------------------------------------------
Web Archive:  http://www.mail-archive.com/cgiapp@lists.erlbaum.net/
              http://marc.theaimsgroup.com/?l=cgiapp&r=1&w=2
To unsubscribe, e-mail: cgiapp-unsubscribe@lists.erlbaum.net
For additional commands, e-mail: cgiapp-help@lists.erlbaum.net

[prev in list] [next in list] [prev in thread] [next in thread]