[prev in list] [next in list] [prev in thread] [next in thread] 

List:       apache-httpd-bugs
Subject:    [Bug 55485] New: BRIGADE_NORMALIZE in ap_coure_input_filter causes 100% CPU consumption
From:       bugzilla () apache ! org
Date:       2013-08-26 18:27:29
Message-ID: bug-55485-7868 () https ! issues ! apache ! org/bugzilla/
[Download RAW message or body]

https://issues.apache.org/bugzilla/show_bug.cgi?id=55485

            Bug ID: 55485
           Summary: BRIGADE_NORMALIZE in ap_coure_input_filter causes 100%
                    CPU consumption
           Product: Apache httpd-2
           Version: 2.2.15
          Hardware: HP
                OS: HP-UX
            Status: NEW
          Severity: critical
          Priority: P2
         Component: Core
          Assignee: bugs@httpd.apache.org
          Reporter: vamsi.kvv@gmail.com

Created attachment 30765
  --> https://issues.apache.org/bugzilla/attachment.cgi?id=30765&action=edit
top output and trace of system calls(tusc)

Our customer runs Apache 2.2.15 with worker module on HP Itanium server. also
they use 3rd party module "Oracle webgate"(Please find the attached
configuration files httpd.conf and httpd-mpm.conf)

Once in a while, they observe ~100% CPU consumption by the httpd processes.
Please find the attached files:
1. stack trace : bt_all
2. top and syscall trace (top_syscall_trace) 
3. Mapping between stack trace, source code and itanium assembly
code(trace_srccode_assemblyCode)

syscall trace shows that main thread waits for the use space thread that run
ap_core_input_filter()

trace:
Thread 22 (system thread 1947121):
#0  0x40000000000977f0:2 in ap_core_input_filter () at core_filters.c:136
#1  0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489
#2  0xc000000000ac76c0:0 in logio_in_filter () at mod_logio.c:129
#3  0x40000000000c3f50:0 in ap_get_brigade () at util_filter.c:489
#4  0x4000000000074980:0 in ap_rgetline_core () at protocol.c:231
#5  0xc000000004a33580:0 in ap_proxygetline () at mod_proxy_http.c:1658
#6  0xc000000004a338c0:0 in ap_proxy_http_process_response () at
mod_proxy_http.c:1733
#7  0xc000000004a35ee0:0 in proxy_http_handler () at mod_proxy_http.c:2434
#8  0xc000000004929a10:0 in proxy_run_scheme_handler () at mod_proxy.c:2675
#9  0xc000000004928b60:0 in proxy_handler () at mod_proxy.c:1024
#10 0x400000000009d2d0:0 in ap_run_handler ()
#11 0x400000000009e9e0:0 in ap_invoke_handler () at config.c:381
#12 0x40000000000cdec0:0 in ap_process_request () at http_request.c:282
#13 0x40000000000c5a40:0 in ap_process_http_connection () at http_core.c:190
#14 0x40000000000b8d20:0 in ap_process_connection () at connection.c:189
#15 0x40000000000de4c0:0 in process_socket () at worker.c:590
#16 0x40000000000df860:0 in worker_thread () at worker.c:974
#17 0xc000000006f95200:0 in dummy_worker () at threadproc/unix/thread.c:160
#18 0xc00000000013fb20:0 in __pthread_bound_body () at
/ux/core/libs/threadslibs/src/common/pthreads/pthread.c:4875

Eventhough it shows Line:136, actual loop happens @ line 141:
   140      /* ### This is bad. */
   141      BRIGADE_NORMALIZE(ctx->b);

Due to compiler optimzation and macro @ line 141, this line no is not shown in
trace.


As shown in trace_srccode_assemblyCode attachment, linked list has become
circular because address and content of (ctx->b)->list))->next are same

(gdb) info register r44
   gr44: 0x6000000000777de8

(gdb) x/gx0x6000000000777de8
0x6000000000777de8:     0x6000000000777de8

Prev's value also contains the same.
(gdb) x/gx 0x6000000000774508
0x6000000000774508:     0x6000000000777de8

I think we need to add appropriate checks to prevent this issue. 

Please help to resolve this.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

[prev in list] [next in list] [prev in thread] [next in thread]