[prev in list] [next in list] [prev in thread] [next in thread]
List: varnish-misc
Subject: Re: [For Help] Varnish error information
From: Per Buer <perbu () varnish-software ! com>
Date: 2012-10-30 9:34:20
Message-ID: CAOXZevDhTwJiD8_bY0fgfy_j-HxC7_36h0HgQ+4jm2Ho7ZetnA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi Yao,
You are seriously low on virtual memory. And probably low on physical
memory as well.
Per.
On Tue, Oct 30, 2012 at 8:18 AM, 杨尧 <yao.yang@autonavi.com> wrote:
> **
> There are some error on another server.
>
> ------------------------------
> Oct 28 03:13:17 kernel: imklog 4.6.2, log source = /proc/kmsg started.
>
> Oct 28 03:13:17 rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" \
> x-pid="1811" x-info=" http://www.rsyslog.com"] (re)start
>
> Oct 28 03:13:20 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information.
> Oct 29 03:11:23 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information.
> Oct 30 00:26:19 varnishd[29228]: Platform: \
> Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit Oct 30 00:26:19 \
> varnishd[29228]: child (29229) Started Oct 30 00:26:19 varnishd[29228]: Child \
> (29229) said Child starts
> Oct 30 03:38:26 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information.
> Oct 30 11:22:48 abrt[10036]: abrt daemon is not running. If it crashed, \
> /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to \
> 'core' Oct 30 11:22:50 varnishd[29228]: Child (29229) died signal=6 (core dumped)
> Oct 30 11:22:50 varnishd[29228]:
> Child (29229) Panic message: Missing errorhandling code in SMS_Makesynth(), \
> storage_synth.c line 88:#012 Condition((sto) != 0) not true.errno = 12 (Cannot \
> allocate memory)#012thread = (cache-worker)#012ident = \
> Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012sp = \
> 0x7f7440300008 {#012 fd = 1194, id = 1194, xid = 844468737,#012 client = \
> 202.98.152.243 59144,#012 step = STP_ERROR,#012 handling = deliver,#012 err_code \
> = 503, err_reason = (null),#012 restarts = 0, esi_level = 0#012 ws = \
> 0x7f7440300080 { #012 id = "sess",#012 {s,f,r,e} = \
> {0x7f7440300cf0,+608,(nil),+5000000},#012 },#012 http[req] = {#012 ws = \
> 0x7f7440300080[sess]#012 "GET",#012 \
> "/mapabc/maptile?v=w2.61&zoom=2&x=26040&y=13438",#012 "HTTP/1.1",#012 \
> "Accept: */*",#012 "Referer: \
> http://www.aibang.com/youbian/a-%E5%8D%97%E5%85%85%E5%B8%82%E5%8C%BA%E9%A1%BA%E5%BA%86%E7%96%BE%E6%8E%A7%E4%B8%AD%E5%BF%83
> ",#012 "Accept-Language: zh-cn",#012 "User-Agent: Mozilla/4.0 \
> (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; msn OptimizedIE8;ZHCN)",#012 \
> "Host: emap2.mapabc.com
> ",#012 "Connection: Keep-Alive",#012 "X-Forwarded-For: 202.98.152.243, \
> 202.98.152.243",#012 "Accept-Encoding: gzip",#012 },#012 worker = \
> 0x7f755c3f3b60 {#012 ws = 0x7f755c3f3d00 { #012 id = "wrk",#012 \
> {s,f,r,e} = {0x7f755c3e1af0,+4592,(nil),+65536},#012 },#012 },#012 vcl = \
> {#012 srcname = {#012 "input",#012 "Default",#012 },#012 \
> },#012 obj = 0x7f689ea4f200 {#012 xid = 844468737,#012 ws = 0x7f689ea4f218 { \
> #012 id = "obj",#012 {s,f,r,e} = {0x7f689ea4f770,+152,(nil),+1024},#012 \
> },#012 http[obj] = {#012 ws = 0x7f689ea4f218[obj]#012 \
> "HTTP/1.1",#012 "Service Unavailable",#012 "Date: Tue, 30 Oct 2012 \
> 03:22:48 GMT",#012 "Server: Varnish",#012 "Content-Type: text/html; \
> charset=utf-8",#012 "Retry-After: 5",#012 },#012 len = 0,#012 store \
> = {#012 Oct 30 11:22:50 varnishd[29228]: child (10037) Started
> Oct 30 11:22:50 varnishd[29228]: Child (10037) said Child starts
> Oct 30 12:01:27 wall[11761]: wall: user root broadcasted 1 lines (24 chars)
>
> Oct 30 14:47:55 abrt[14718]: abrt daemon is not running. If it crashed, \
> /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to \
> 'core'
> Oct 30 14:47:56 varnishd[29228]: Child (10037) not responding to CLI, killing it.
> Oct 30 14:47:56 varnishd[29228]: Child (10037) died signal=6 (core dumped)
> Oct 30 14:47:56 varnishd[29228]:
> Child (10037) Panic message: Missing errorhandling code in HSH_Prealloc(), \
> cache_hash.c line 87:#012 Condition((oc) != 0) not true.errno = 12 (Cannot \
> allocate memory)#012thread = (cache-worker)#012ident = \
> Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012sp = \
> 0x7f7239d00008 {#012 fd = 4444, id = 4444, xid = 167112905,#012 client = \
> 113.140.2.186 2107,#012 step = STP_LOOKUP,#012 handling = hash,#012 restarts = \
> 0, esi_level = 0#012 ws = 0x7f7239d00080 { #012 id = "sess",#012 {s,f,r,e} = \
> {0x7f7239d00cf0,+528,(nil),+5000000},#012 },#012 http[req] = {#012 ws = \
> 0x7f7239d00080[sess]#012 "GET",#012 \
> "/mapabc/maptile?v=w2.61&zoom=1&x=52603&y=26105",#012 "HTTP/1.1",#012 \
> "Accept: */*",#012 "Referer: \
> http://bus.aibang.com/xian/busxid-1231359865-423759482 ",#012 \
> "Accept-Language: zh-cn",#012 "User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; \
> Windows NT 5.1; Trident/4.0; GTB6; QQDownload 687; .NET CLR 2.0.50727)",#012 \
> "Host: emap2.mapabc.com
> ",#012 "Connection: Keep-Alive",#012 "X-Forwarded-For: 113.140.2.186, \
> 113.140.2.186",#012 "Accept-Encoding: gzip",#012 },#012 worker = \
> 0x7f74ce7feb60 {#012 ws = 0x7f74ce7fed00 { #012 id = "wrk",#012 \
> {s,f,r,e} = {0x7f74ce7ecaf0,0x7f74ce7ecaf0,(nil),+65536},#012 },#012 },#012 \
> vcl = {#012 srcname = {#012 "input",#012 "Default",#012 \
> },#012 },#012},#012 Oct 30 14:47:56 varnishd[29228]: child (14729) Started
> Oct 30 14:47:56 varnishd[29228]: Child (14729) said Child starts
>
> *发件人：* 杨尧 <yao.yang@autonavi.com>
> *发送时间：* 2012-10-30 15:04
> *收件人：* varnish-misc@varnish-cache.org
> *抄送：* 杨尧 <yao.yang@autonavi.com>
> *主题：* [For Help] Varnish error information
> ****
>
> Hi, All,
> I used Varnish for half a year, these days some errors puzzled me.
> And I find some error informations from /var/log/message file.
> Is my fault or the software's bug?
>
> Thanks.
> Yangyao
> ------------------------------
>
> Oct 28 03:47:01 kernel: imklog 4.6.2, log source = /proc/kmsg started.
>
> Oct 28 03:47:01 rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" \
> x-pid="1721" x-info=" http://www.rsyslog.com"] (re)start
>
> Oct 28 03:47:39 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information. Oct \
> 28 20:20:01
> kernel: varnishncsa[17385]: segfault at 7f2a06670f58 ip 00007f2a06687005 sp \
> 00007fff242d0720 error 4 in libvarnishapi.so.1.0.0[7f2a0667f000+11000]
> Oct 28 20:20:01 abrt[17387]: abrt daemon is not running. If it crashed, \
> /proc/sys/kernel/core_pattern contains a stale value, consider resetting it to \
> 'core'
> Oct 29 03:13:03 varnishd[2591]: Child (2592) not responding to CLI, killing it.
>
> Oct 29 03:13:13 varnishd[2591]: Child (2592) not responding to CLI, killing it.
>
> Oct 29 03:13:15 varnishd[2591]: Child (2592) not responding to CLI, killing it.
> Oct 29 03:13:15 varnishd[2591]: Child (2592) died signal=3 (core dumped)
> Oct 29 03:13:15 varnishd[2591]: child (28563) Started
> Oct 29 03:13:15 varnishd[2591]: Child (28563) said Child starts
>
> Oct 29 03:45:46 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information.
> Oct 29 17:23:16 varnishd[2591]: Child (28563) not responding to CLI, killing it.
>
> Oct 29 17:23:26 varnishd[2591]: Child (28563) not responding to CLI, killing it.
>
> Oct 29 17:23:34 varnishd[2591]: Child (28563) not responding to CLI, killing it.
>
> Oct 29 17:23:34 varnishd[2591]: Child (28563) not responding to CLI, killing it.
> Oct 29 17:23:34 varnishd[2591]: Child (28563) died signal=3 (core dumped)
> Oct 29 17:23:34 varnishd[2591]: child (15919) Started
> Oct 29 17:23:34 varnishd[2591]: Child (15919) said Child starts
>
> Oct 29 20:56:39 varnishd[2591]: Child (15919) not responding to CLI, killing it.
>
> Oct 29 20:56:43 varnishd[2591]: Child (15919) not responding to CLI, killing it.
>
> Oct 29 20:56:43 varnishd[2591]: Child (15919) not responding to CLI, killing it.
> Oct 29 20:56:43 varnishd[2591]: Child (15919) died signal=3 (core dumped)
> Oct 29 20:56:43 varnishd[2591]: child (22868) Started
> Oct 29 20:56:43 varnishd[2591]: Child (22868) said Child starts
>
> Oct 30 03:32:46 rhsm-complianced: This system is missing one or more valid \
> entitlement certificates. Please run subscription-manager for more information.
> Oct 30 10:30:40 kernel: varnishncsa invoked oom-killer: gfp_mask=0x201da, order=0, \
> oom_adj=0 Oct 30 10:30:40 kernel: varnishncsa cpuset=/ mems_allowed=0-1
>
> Oct 30 10:30:40 kernel: Pid: 15396, comm: varnishncsa Not tainted \
> 2.6.32-131.0.15.el6.x86_64 #1 Oct 30 10:30:40 kernel: Call Trace:
>
> Oct 30 10:30:40 kernel: [<ffffffff810c0101>] ? \
> cpuset_print_task_mems_allowed+0x91/0xb0 Oct 30 10:30:40 kernel: \
> [<ffffffff811102bb>] ? oom_kill_process+0xcb/0x2e0
> Oct 30 10:30:40 kernel: [<ffffffff81110880>] ? select_bad_process+0xd0/0x110
> Oct 30 10:30:40 kernel: [<ffffffff81110918>] ? __out_of_memory+0x58/0xc0
> Oct 30 10:30:40 kernel: [<ffffffff81110b19>] ? out_of_memory+0x199/0x210
>
> Oct 30 10:30:40 kernel: [<ffffffff811202dd>] ? __alloc_pages_nodemask+0x80d/0x8b0
>
> Oct 30 10:30:40 kernel: [<ffffffff8115464a>] ? alloc_pages_current+0xaa/0x110
> Oct 30 10:30:40 kernel: [<ffffffff8110d717>] ? __page_cache_alloc+0x87/0x90
>
> Oct 30 10:30:40 kernel: [<ffffffff81122c4b>] ? __do_page_cache_readahead+0xdb/0x210
> Oct 30 10:30:40 kernel: [<ffffffff81122da1>] ? ra_submit+0x21/0x30
> Oct 30 10:30:40 kernel: [<ffffffff8110e9e3>] ? filemap_fault+0x4c3/0x500
> Oct 30 10:30:40 kernel: [<ffffffff812628a9>] ? cpumask_next_and+0x29/0x50
> Oct 30 10:30:40 kernel: [<ffffffff81137114>] ? __do_fault+0x54/0x510
> Oct 30 10:30:40 kernel: [<ffffffff811376c7>] ? handle_pte_fault+0xf7/0xb50
> Oct 30 10:30:40 kernel: [<ffffffff8100987e>] ? __switch_to+0x26e/0x320
>
> Oct 30 10:30:40 kernel: [<ffffffff81057f99>] ? find_busiest_queue+0x69/0x150
> Oct 30 10:30:40 kernel: [<ffffffff811382f8>] ? handle_mm_fault+0x1d8/0x2a0
>
> Oct 30 10:30:40 kernel: [<ffffffff81092ff3>] ? __hrtimer_start_range_ns+0x1a3/0x460
> Oct 30 10:30:40 kernel: [<ffffffff81041529>] ? __do_page_fault+0x139/0x480
>
> Oct 30 10:30:40 kernel: [<ffffffff8109332f>] ? hrtimer_try_to_cancel+0x3f/0xd0
> Oct 30 10:30:40 kernel: [<ffffffff810933e2>] ? hrtimer_cancel+0x22/0x30
> Oct 30 10:30:40 kernel: [<ffffffff814dccc3>] ? do_nanosleep+0x93/0xc0
> Oct 30 10:30:40 kernel: [<ffffffff810934b4>] ? hrtimer_nanosleep+0xc4/0x180
> Oct 30 10:30:40 kernel: [<ffffffff810922a0>] ? hrtimer_wakeup+0x0/0x30
> Oct 30 10:30:40 kernel: [<ffffffff814e0c3e>] ? do_page_fault+0x3e/0xa0
> Oct 30 10:30:40 kernel: [<ffffffff814ddfe5>] ? page_fault+0x25/0x30
> Oct 30 10:30:40 kernel: Mem-Info:
> Oct 30 10:30:40 kernel: Node 0 DMA per-cpu:
> Oct 30 10:30:40 kernel: CPU 0: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 1: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 2: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 3: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 4: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 5: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 6: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 7: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 8: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 9: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 10: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 11: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 12: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 13: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 14: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: CPU 15: hi: 0, btch: 1 usd: 0
> Oct 30 10:30:40 kernel: Node 0 DMA32 per-cpu:
> Oct 30 10:30:40 kernel: CPU 0: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 2: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 3: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 4: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 5: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 6: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 7: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 8: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 9: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 10: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 11: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 12: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 13: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 14: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 15: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: Node 0 Normal per-cpu:
> Oct 30 10:30:40 kernel: CPU 0: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 2: hi: 186, btch: 31 usd: 11
> Oct 30 10:30:40 kernel: CPU 3: hi: 186, btch: 31 usd: 2
> Oct 30 10:30:40 kernel: CPU 4: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 5: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 6: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 7: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 8: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 9: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 10: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 11: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 12: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 13: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 14: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 15: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: Node 1 Normal per-cpu:
> Oct 30 10:30:40 kernel: CPU 0: hi: 186, btch: 31 usd: 5
> Oct 30 10:30:40 kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 2: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 3: hi: 186, btch: 31 usd: 1
> Oct 30 10:30:40 kernel: CPU 4: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 5: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 6: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 7: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 8: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 9: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 10: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 11: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 12: hi: 186, btch: 31 usd: 8
> Oct 30 10:30:40 kernel: CPU 13: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 14: hi: 186, btch: 31 usd: 0
> Oct 30 10:30:40 kernel: CPU 15: hi: 186, btch: 31 usd: 0
>
> Oct 30 10:30:40 kernel: active_anon:15180189 inactive_anon:868880 isolated_anon:0
> Oct 30 10:30:40 kernel: active_file:496 inactive_file:603 isolated_file:0
> Oct 30 10:30:40 kernel: unevictable:0 dirty:22 writeback:20 unstable:0
>
> Oct 30 10:30:40 kernel: free:55664 slab_reclaimable:10223 slab_unreclaimable:284822
> Oct 30 10:30:40 kernel: mapped:386 shmem:0 pagetables:54263 bounce:0
>
> Oct 30 10:30:40 kernel: Node 0 DMA free:15616kB min:20kB low:24kB high:28kB \
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB \
> isolated(anon):0kB isolated(file):0kB present:15224kB mlocked:0kB dirty:0kB \
> writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB \
> kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB \
> pages_scanned:0 all_unreclaimable? no Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 \
> 2991 32281 32281
> Oct 30 10:30:40 kernel: Node 0 DMA32 free:121196kB min:4168kB low:5208kB \
> high:6252kB active_anon:1856648kB inactive_anon:457828kB active_file:124kB \
> inactive_file:372kB unevictable:0kB isolated(anon):0kB isolated(file):0kB \
> present:3063392kB mlocked:0kB dirty:12kB writeback:16kB mapped:156kB shmem:0kB \
> slab_reclaimable:3732kB slab_unreclaimable:2508kB kernel_stack:0kB \
> pagetables:5060kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:352 \
> all_unreclaimable? no Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 29290 29290
>
> Oct 30 10:30:40 kernel: Node 0 Normal free:40808kB min:40844kB low:51052kB \
> high:61264kB active_anon:27839988kB inactive_anon:1739808kB active_file:1104kB \
> inactive_file:1240kB unevictable:0kB isolated(anon):128kB isolated(file):0kB \
> present:29992960kB mlocked:0kB dirty:52kB writeback:40kB mapped:332kB shmem:0kB \
> slab_reclaimable:19552kB slab_unreclaimable:584592kB kernel_stack:10656kB \
> pagetables:107120kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1528 \
> all_unreclaimable? no Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 0 0
>
> Oct 30 10:30:40 kernel: Node 1 Normal free:45036kB min:45072kB low:56340kB \
> high:67608kB active_anon:31024120kB inactive_anon:1277884kB active_file:756kB \
> inactive_file:800kB unevictable:0kB isolated(anon):0kB isolated(file):0kB \
> present:33095680kB mlocked:0kB dirty:24kB writeback:24kB mapped:1056kB shmem:0kB \
> slab_reclaimable:17608kB slab_unreclaimable:552188kB kernel_stack:5456kB \
> pagetables:104872kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1267 \
> all_unreclaimable? no Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 0 0
>
> Oct 30 10:30:40 kernel: Node 0 DMA: 2*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB \
> 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15616kB
> Oct 30 10:30:40 kernel: Node 0 DMA32: 1357*4kB 979*8kB 689*16kB 1413*32kB 447*64kB \
> 66*128kB 15*256kB 10*512kB 7*1024kB 0*2048kB 0*4096kB = 122684kB
> Oct 30 10:30:40 kernel: Node 0 Normal: 10120*4kB 45*8kB 7*16kB 3*32kB 1*64kB \
> 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 42776kB
> Oct 30 10:30:40 kernel: Node 1 Normal: 2241*4kB 1460*8kB 766*16kB 273*32kB 51*64kB \
> 2*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 46692kB Oct 30 10:30:40 \
> kernel: 10994 total pagecache pages Oct 30 10:30:40 kernel: 9447 pages in swap \
> cache
> Oct 30 10:30:40 kernel: Swap cache stats: add 10177329, delete 10167882, find \
> 6787957/7512667 Oct 30 10:30:40 kernel: Free swap = 0kB
> Oct 30 10:30:40 kernel: Total swap = 4194296kB
> Oct 30 10:30:40 kernel: 16777215 pages RAM
> Oct 30 10:30:40 kernel: 285063 pages reserved
> Oct 30 10:30:40 kernel: 2192 pages shared
> Oct 30 10:30:40 kernel: 16161719 pages non-shared
> Oct 30 10:30:40 kernel:
> Out of memory: kill process 2591 (varnishd) score 164119553 or a child
>
> Oct 30 10:30:40 kernel: Killed process 22868 (varnishd) vsz:147678356kB, \
> anon-rss:63571052kB, file-rss:616kB
> Oct 30 10:30:50 varnishd[2591]: Child (22868) not responding to CLI, killing it.
>
> Oct 30 10:30:53 varnishd[2591]: Child (22868) not responding to CLI, killing it.
> Oct 30 10:30:53 varnishd[2591]: Child (22868) died signal=9
> Oct 30 10:30:53 varnishd[2591]: child (15405) Started
> Oct 30 10:30:53 varnishd[2591]: Child (15405) said Child starts
>
> Oct 30 14:44:27 varnishd[21828]: Platform: \
> Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit Oct 30 14:44:27 \
> varnishd[21828]: Child start failed: could not open sockets
> Oct 30 14:44:32 varnishd[21855]: Platform: \
> Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit Oct 30 14:44:32 \
> varnishd[21855]: child (21856) Started Oct 30 14:44:32 varnishd[21855]: Child \
> (21856) said Child starts
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
--
<http://www.varnish-software.com/> *Per Buer*
CEO | Varnish Software AS
Phone: +47 958 39 117 | Skype: per.buer
We Make Websites Fly!
[Attachment #5 (text/html)]
<div>Hi Yao,</div><div><br></div>You are seriously low on virtual memory. And \
probably low on physical memory as well.<br><br>Per.<br><div><br><div \
class="gmail_quote">On Tue, Oct 30, 2012 at 8:18 AM, 杨尧 <span dir="ltr"><<a \
href="mailto:yao.yang@autonavi.com" \
target="_blank">yao.yang@autonavi.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><u></u>
<div style="MARGIN:10px">
<div>There are some error on another server.</div>
<div> </div>
<div>
<hr>
</div>
<div>
<div style="FONT-SIZE:9pt">Oct 28 03:13:17 kernel: imklog 4.6.2, log source = /proc/kmsg started.</div>
<div style="FONT-SIZE:9pt">Oct 28 03:13:17 rsyslogd: [origin  \
;software="rsyslogd" swVersion="4.6.2" x-pid="1811" x-info="<a \
href="http://www.rsyslog.com" \
target="_blank">http://www.rsyslog.com</a>"] (re)start</div>
<div style="FONT-SIZE:9pt">Oct 28 03:13:20 rhsm-complianced: This& \
nbsp;system is missing one or more valid entitlemen \
t certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 29 03:11:23 rhsm-complianced: This \
system is missing one or more valid entitleme \
nt certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 30 00:26:19 varnishd[29228]: Platf \
orm: Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit</div> \
<div style="FONT-SIZE:9pt">Oct 30 00:26:19 varnishd[29228]: child (29229) Started</div>
<div style="FONT-SIZE:9pt">Oct 30 00:26:19 varnishd[29228]: Child (29229) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 30 03:38:26 rhsm-complianced: This \
system is missing one or more valid entitleme \
nt certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 30 11:22:48 abrt[10036]: abrt  \
;daemon is not running. If it crashed, /proc/sys/ke \
rnel/core_pattern contains a stale value, consider resetting it to 'core'</div>
<div style="FONT-SIZE:9pt">Oct 30 11:22:50 varnishd[29228]: Child (29229) died signal=6 (core dumped)</div>
<div style="FONT-SIZE:9pt">Oct 30 11:22:50 varnishd[29228]: <span \
style="COLOR:#ff0000">Child (29229) Panic message: Missing er \
rorhandling code in SMS_Makesynth(), storage_synth.c line&nbs \
p;88:#012 Condition((sto) != 0) not true.errno =&n \
bsp;12 (Cannot allocate memory)#012thread = (cache-worker)#01 \
2ident = Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit \
,epoll#012sp = 0x7f7440300008 </span>{#012 fd = 11 \
94, id = 1194, xid = 844468737,#012 client&nb \
sp;= 202.98.152.243 59144,#012 step = STP_ERROR,#012&nb \
sp; handling = deliver,#012 err_code = 503, e \
rr_reason = (null),#012 restarts = 0, esi_level&nb \
sp;= 0#012 ws = 0x7f7440300080 { #012 & \
nbsp; id = "sess",#012 {s,f,r,e} \
= {0x7f7440300cf0,+608,(nil),+5000000},#012 },#012 http[req \
] = {#012 ws = 0x7f7440300080[sess]#012&nbs \
p; "GET",#012 &nbs \
p;"/mapabc/maptile?v=w2.61&zoom=2&x=26040&y=13438",#012 &nb \
sp; "HTTP/1.1",#012 &nbs \
p;"Accept: */*",#012 "Referer: <a \
href="http://www.aibang.com/youbian/a-%E5%8D%97%E5%85%85%E5%B8%82%E5%8C%BA%E9%A1%BA%E5%BA%86%E7%96%BE%E6%8E%A7%E4%B8%AD%E5%BF%83" \
target="_blank">http://www.aibang.com/youbian/a-%E5%8D%97%E5%85%85%E5%B8%82%E5%8C%BA%E \
9%A1%BA%E5%BA%86%E7%96%BE%E6%8E%A7%E4%B8%AD%E5%BF%83</a>",#012 & \
nbsp; "Accept-Language: zh-cn",#012 & \
nbsp; "User-Agent: Mozilla/4.0 (compatible; MSIE 8.0;&nb \
sp;Windows NT 5.1; Trident/4.0; msn OptimizedIE8;ZHCN)",#012 "Host: <a \
href="http://emap2.mapabc.com" \
target="_blank">emap2.mapabc.com</a>",#012 &qu \
ot;Connection: Keep-Alive",#012 "X-F \
orwarded-For: 202.98.152.243, 202.98.152.243",#012 &nb \
sp; "Accept-Encoding: gzip",#012 },#012 &nbs \
p;worker = 0x7f755c3f3b60 {#012 ws = 0 \
x7f755c3f3d00 { #012 id = " \
;wrk",#012 {s,f,r,e} = {0x7f755c3e1a \
f0,+4592,(nil),+65536},#012 },#012 },#01 \
2 vcl = {#012 srcn \
ame = {#012 "input" \
,#012 "Default",#012 &nb \
sp; },#012 },#012 obj =& \
nbsp;0x7f689ea4f200 {#012 xid = 844468737,#012&n \
bsp; ws = 0x7f689ea4f218 { #012 \
id = "obj",#012 &nb \
sp;{s,f,r,e} = {0x7f689ea4f770,+152,(nil),+1024},#012   \
;},#012 http[obj] = {#012 &nbs \
p; ws = 0x7f689ea4f218[obj]#012 &nbs \
p; "HTTP/1.1",#012 " \
;Service Unavailable",#012 &q \
uot;Date: Tue, 30 Oct 2012 03:22:48 GMT",#012 \
"Server: Varnish",#012 & \
nbsp; "Content-Type: text/html; char \
set=utf-8",#012 "Retry-After: \
5",#012 },#012 len = 0,#012 store = {#012 </div>
<div style="FONT-SIZE:9pt">Oct 30 11:22:50 varnishd[29228]: child (10037) Started</div>
<div style="FONT-SIZE:9pt">Oct 30 11:22:50 varnishd[29228]: Child (10037) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 30 12:01:27 wall[11761]: wall: user root broadcasted 1 lines (24 chars)</div>
<div style="FONT-SIZE:9pt">Oct 30 14:47:55 abrt[14718]: abrt  \
;daemon is not running. If it crashed, /proc/sys/ke \
rnel/core_pattern contains a stale value, consider resetting it to 'core'</div>
<div style="FONT-SIZE:9pt">Oct 30 14:47:56 varnishd[29228]: Child \
(10037) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 30 14:47:56 varnishd[29228]: Child (10037) died signal=6 (core dumped)</div>
<div style="FONT-SIZE:9pt">Oct 30 14:47:56 varnishd[29228]:<span \
style="COLOR:#ff0000"> Child (10037) Panic message: Missing&n \
bsp;errorhandling code in HSH_Prealloc(), cache_hash.c line&n \
bsp;87:#012 Condition((oc) != 0) not true.errno =& \
nbsp;12 (Cannot allocate memory)#012thread = (cache-worker)#0 \
12ident = Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbi \
t,epoll#012sp = 0x7f7239d00008 </span>{#012 fd = 4 \
444, id = 4444, xid = 167112905,#012 client&n \
bsp;= 113.140.2.186 2107,#012 step = STP_LOOKUP,#012&nb \
sp; handling = hash,#012 restarts = 0, esi_le \
vel = 0#012 ws = 0x7f7239d00080 { #012 & \
nbsp; id = "sess",#012 {s,f,r,e} \
= {0x7f7239d00cf0,+528,(nil),+5000000},#012 },#012 ht \
tp[req] = {#012 ws = 0x7f7239d00080[sess]#0 \
12 "GET",#012 &nbs \
p; "/mapabc/maptile?v=w2.61&zoom=1&x=52603&y=26105",#012&nb \
sp; "HTTP/1.1",#012 &nbs \
p; "Accept: */*",#012 "Referer: <a \
href="http://bus.aibang.com/xian/busxid-1231359865-423759482" \
target="_blank">http://bus.aibang.com/xian/busxid-1231359865-423759482</a>",#012& \
nbsp; "Accept-Language: zh-cn",#012 & \
nbsp; "User-Agent: Mozilla/4.0 (compatible;  \
;MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6; QQDow \
nload 687; .NET CLR 2.0.50727)",#012 "Host: <a \
href="http://emap2.mapabc.com" \
target="_blank">emap2.mapabc.com</a>",#012 &qu \
ot;Connection: Keep-Alive",#012 "X-F \
orwarded-For: 113.140.2.186, 113.140.2.186",#012   \
; "Accept-Encoding: gzip",#012 },#012 \
worker = 0x7f74ce7feb60 {#012 ws = 0x7 \
f74ce7fed00 { #012 id = "w \
rk",#012 {s,f,r,e} = {0x7f74ce7ecaf0 \
,0x7f74ce7ecaf0,(nil),+65536},#012 },#012 &nbs \
p;},#012 vcl = {#012 &nb \
sp;srcname = {#012 "inpu \
t",#012 "Default",#012&n \
bsp; },#012 },#012},#012</div>
<div style="FONT-SIZE:9pt">Oct 30 14:47:56 varnishd[29228]: child (14729) Started</div>
<div style="FONT-SIZE:9pt">Oct 30 14:47:56 varnishd[29228]: Child (14729) said Child starts</div>
</div>
<div> </div>
<div style="BORDER-BOTTOM:medium none;BORDER-LEFT:medium \
none;PADDING-BOTTOM:0cm;PADDING-LEFT:0cm;PADDING-RIGHT:0cm;BORDER-TOP:#b5c4df 1pt \
solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt"> <div \
style="padding-right:8px;padding-left:8px;padding-top:8px;font-size:12px;background:#efefef;padding-bottom:8px">
<div><b>发件人：</b> <a href="mailto:yao.yang@autonavi.com" \
target="_blank">杨尧</a></div> <div><b>发送时间：</b> <a \
href="tel:2012-10-30%C2%A015" value="+12012103015" \
target="_blank">2012-10-30 15</a>:04</div> <div><b>收件人：</b> <a \
href="mailto:varnish-misc@varnish-cache.org" \
target="_blank">varnish-misc@varnish-cache.org</a></div> <div><b>抄送：</b> <a \
href="mailto:yao.yang@autonavi.com" target="_blank">杨尧</a></div> \
<div><b>主题：</b> [For Help] Varnish error information</div> </div>
</div><div><div class="h5">
<div>
<div><u></u><u></u>
<div style="FONT-SIZE:9pt" dir="ltr"> </div>
<div style="FONT-SIZE:9pt" dir="ltr">Hi, All,</div>
<div style="TEXT-INDENT:2em;FONT-SIZE:9pt" dir="ltr">I used Varnish for half a year, \
these days some errors puzzled me.</div> <div style="TEXT-INDENT:2em;FONT-SIZE:9pt" \
dir="ltr">And I find some error informations from /var/log/message file.</div> \
<div style="TEXT-INDENT:2em;FONT-SIZE:9pt" dir="ltr">Is my fault or the \
software's bug?</div> <div style="TEXT-INDENT:2em;FONT-SIZE:9pt" \
dir="ltr"> </div> <div style="TEXT-INDENT:2em;FONT-SIZE:9pt" \
dir="ltr">Thanks.</div> <div style="TEXT-INDENT:2em;FONT-SIZE:9pt" \
dir="ltr">Yangyao</div> <div style="TEXT-INDENT:2em;FONT-SIZE:9pt" dir="ltr">
<hr>
</div>
<blockquote style="MARGIN-RIGHT:0px" dir="ltr">
<div style="FONT-SIZE:9pt">Oct 28 03:47:01 kernel: imklog 4.6.2, log source = /proc/kmsg started.</div>
<div style="FONT-SIZE:9pt">Oct 28 03:47:01 rsyslogd: [origin  \
;software="rsyslogd" swVersion="4.6.2" x-pid="1721" x-info="<a \
href="http://www.rsyslog.com" \
target="_blank">http://www.rsyslog.com</a>"] (re)start</div>
<div style="FONT-SIZE:9pt">Oct 28 03:47:39 rhsm-complianced: This& \
nbsp;system is missing one or more valid entitlemen \
t certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 28 20:20:01<span \
style="COLOR:#ff0000"> kernel: varnishncsa[17385]: segfault at&nbs \
p;7f2a06670f58 ip 00007f2a06687005 sp 00007fff242d0720 error 4 in libvarnishapi.so.1.0.0[7f2a0667f000+11000]</span></div>
<div style="FONT-SIZE:9pt">Oct 28 20:20:01 abrt[17387]: abrt \
daemon is not running. If it crashed, /proc/sys/ker \
nel/core_pattern contains a stale value, consider resetting it to 'core'</div>
<div style="FONT-SIZE:9pt">Oct 29 03:13:03 varnishd[2591]: Child& \
nbsp;(2592) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 03:13:13 varnishd[2591]: Child&n \
bsp;(2592) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 03:13:15 varnishd[2591]: Child&n \
bsp;(2592) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 03:13:15 varnishd[2591]: Child (2592) died signal=3 (core dumped)</div>
<div style="FONT-SIZE:9pt">Oct 29 03:13:15 varnishd[2591]: child (28563) Started</div>
<div style="FONT-SIZE:9pt">Oct 29 03:13:15 varnishd[2591]: Child (28563) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 29 03:45:46 rhsm-complianced: This \
system is missing one or more valid entitleme \
nt certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 29 17:23:16 varnishd[2591]: Child& \
nbsp;(28563) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 17:23:26 varnishd[2591]: Child&n \
bsp;(28563) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 17:23:34 varnishd[2591]: Child&n \
bsp;(28563) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 17:23:34 varnishd[2591]: Child&n \
bsp;(28563) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 17:23:34 varnishd[2591]: Child (28563) died signal=3 (core dumped)</div>
<div style="FONT-SIZE:9pt">Oct 29 17:23:34 varnishd[2591]: child (15919) Started</div>
<div style="FONT-SIZE:9pt">Oct 29 17:23:34 varnishd[2591]: Child (15919) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 29 20:56:39 varnishd[2591]: Child& \
nbsp;(15919) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 20:56:43 varnishd[2591]: Child&n \
bsp;(15919) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 20:56:43 varnishd[2591]: Child&n \
bsp;(15919) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 29 20:56:43 varnishd[2591]: Child (15919) died signal=3 (core dumped)</div>
<div style="FONT-SIZE:9pt">Oct 29 20:56:43 varnishd[2591]: child (22868) Started</div>
<div style="FONT-SIZE:9pt">Oct 29 20:56:43 varnishd[2591]: Child (22868) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 30 03:32:46 rhsm-complianced: This \
system is missing one or more valid entitleme \
nt certificates. Please run subscription-manager for more information.</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: varnishncsa&nb \
sp;invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0</div> \
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: varnishncsa cpuset=/ mems_allowed=0-1</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Pid: 1539 \
6, comm: varnishncsa Not tainted 2.6.32-131.0.15.el6.x86_64 #1</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Call Trace:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff810c0101>] ? cpuset_print_task_mems_allowed+0x91/0xb0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff811102bb>] ? oom_kill_process+0xcb/0x2e0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81110880>] ? select_bad_process+0xd0/0x110</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81110918>] ? __out_of_memory+0x58/0xc0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81110b19>] ? out_of_memory+0x199/0x210</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff811202dd>] ? __alloc_pages_nodemask+0x80d/0x8b0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff8115464a>] ? alloc_pages_current+0xaa/0x110</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff8110d717>] ? __page_cache_alloc+0x87/0x90</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81122c4b>] ? __do_page_cache_readahead+0xdb/0x210</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81122da1>] ? ra_submit+0x21/0x30</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff8110e9e3>] ? filemap_fault+0x4c3/0x500</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff812628a9>] ? cpumask_next_and+0x29/0x50</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81137114>] ? __do_fault+0x54/0x510</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff811376c7>] ? handle_pte_fault+0xf7/0xb50</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff8100987e>] ? __switch_to+0x26e/0x320</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81057f99>] ? find_busiest_queue+0x69/0x150</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff811382f8>] ? handle_mm_fault+0x1d8/0x2a0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81092ff3>] ? __hrtimer_start_range_ns+0x1a3/0x460</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff81041529>] ? __do_page_fault+0x139/0x480</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff8109332f>] ? hrtimer_try_to_cancel+0x3f/0xd0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff810933e2>] ? hrtimer_cancel+0x22/0x30</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff814dccc3>] ? do_nanosleep+0x93/0xc0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff810934b4>] ? hrtimer_nanosleep+0xc4/0x180</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff810922a0>] ? hrtimer_wakeup+0x0/0x30</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff814e0c3e>] ? do_page_fault+0x3e/0xa0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: [<ffffffff814ddfe5>] ? page_fault+0x25/0x30</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Mem-Info:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0 DMA per-cpu:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 0: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 1: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 2: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 3: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 4: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 5: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 6: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 7: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 8: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 9: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 10: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 11: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 12: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 13: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 14: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 15: hi: 0, btch: 1 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0 DMA32 per-cpu:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 0: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 1: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 2: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 3: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 4: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 5: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 6: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 7: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 8: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 9: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 10: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 11: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 12: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 13: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 14: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 15: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0 Normal per-cpu:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 0: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 1: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 2: hi: 186, btch: 31 usd: 11</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 3: hi: 186, btch: 31 usd: 2</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 4: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 5: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 6: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 7: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 8: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 9: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 10: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 11: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 12: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 13: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 14: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 15: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 1 Normal per-cpu:</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 0: hi: 186, btch: 31 usd: 5</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 1: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 2: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 3: hi: 186, btch: 31 usd: 1</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 4: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 5: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 6: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 7: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 8: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 9: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 10: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 11: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 12: hi: 186, btch: 31 usd: 8</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 13: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
; 14: hi: 186, btch: 31 usd: 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: CPU   \
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: active_anon:15180189 inactive_anon:868880 isolated_anon:0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: active_file:496 inactive_file:603 isolated_file:0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: unevictable:0 dirty:22 writeback:20 unstable:0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: free:55664 slab_reclaimable:10223 slab_unreclaimable:284822</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: mapped:386 shmem:0 pagetables:54263 bounce:0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;DMA free:15616kB min:20kB low:24kB high:28kB active_anon:0 \
kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable \
:0kB isolated(anon):0kB isolated(file):0kB present:15224kB mlocked \
:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclai \
mable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB u \
nstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 2991 32281 32281</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;DMA32 free:121196kB min:4168kB low:5208kB high:6252kB acti \
ve_anon:1856648kB inactive_anon:457828kB active_file:124kB inactive_fil \
e:372kB unevictable:0kB isolated(anon):0kB isolated(file):0kB pres \
ent:3063392kB mlocked:0kB dirty:12kB writeback:16kB mapped:156kB&n \
bsp;shmem:0kB slab_reclaimable:3732kB slab_unreclaimable:2508kB kernel_ \
stack:0kB pagetables:5060kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:352 all_unreclaimable? no</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 29290 29290</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;Normal free:40808kB min:40844kB low:51052kB high:61264kB a \
ctive_anon:27839988kB inactive_anon:1739808kB active_file:1104kB inacti \
ve_file:1240kB unevictable:0kB isolated(anon):128kB isolated(file):0kB& \
nbsp;present:29992960kB mlocked:0kB dirty:52kB writeback:40kB mapp \
ed:332kB shmem:0kB slab_reclaimable:19552kB slab_unreclaimable:584592kB \
kernel_stack:10656kB pagetables:107120kB unstable:0kB bounce:0kB& \
nbsp;writeback_tmp:0kB pages_scanned:1528 all_unreclaimable? no</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 0 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 1&nb \
sp;Normal free:45036kB min:45072kB low:56340kB high:67608kB a \
ctive_anon:31024120kB inactive_anon:1277884kB active_file:756kB inactiv \
e_file:800kB unevictable:0kB isolated(anon):0kB isolated(file):0kB  \
;present:33095680kB mlocked:0kB dirty:24kB writeback:24kB mapped:1 \
056kB shmem:0kB slab_reclaimable:17608kB slab_unreclaimable:552188kB&nb \
sp;kernel_stack:5456kB pagetables:104872kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1267 all_unreclaimable? no</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: lowmem_reserve[]: 0 0 0 0</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;DMA: 2*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 0* \
256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15616kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;DMA32: 1357*4kB 979*8kB 689*16kB 1413*32kB 447*64kB 6 \
6*128kB 15*256kB 10*512kB 7*1024kB 0*2048kB 0*4096kB = 122684kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 0&nb \
sp;Normal: 10120*4kB 45*8kB 7*16kB 3*32kB 1*64kB 1*128kB \
0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 42776kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Node 1&nb \
sp;Normal: 2241*4kB 1460*8kB 766*16kB 273*32kB 51*64kB 2 \
*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 46692kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 10994 total pagecache pages</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 9447 pages in swap cache</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Swap cach \
e stats: add 10177329, delete 10167882, find 6787957/7512667</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Free swap = 0kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Total swap = 4194296kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 16777215 pages RAM</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 285063 pages reserved</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 2192 pages shared</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: 16161719 pages non-shared</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: <span \
style="COLOR:#ff0000">Out of memory: kill process 2591 (varnishd) score 164119553 or a child</span></div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:40 kernel: Killed pr \
ocess 22868 (varnishd) vsz:147678356kB, anon-rss:63571052kB, file-rss:616kB</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:50 varnishd[2591]: Child& \
nbsp;(22868) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 30 10:30:53 varnishd[2591]: Child&n \
bsp;(22868) not responding to CLI, killing it.</div> \
<div style="FONT-SIZE:9pt">Oct 30 10:30:53 varnishd[2591]: Child (22868) died signal=9</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:53 varnishd[2591]: child (15405) Started</div>
<div style="FONT-SIZE:9pt">Oct 30 10:30:53 varnishd[2591]: Child (15405) said Child starts</div>
<div style="FONT-SIZE:9pt">Oct 30 14:44:27 varnishd[21828]: Platf \
orm: Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit</div> \
<div style="FONT-SIZE:9pt">Oct 30 14:44:27 varnishd[21828]: Child start failed: could not open sockets</div>
<div style="FONT-SIZE:9pt">Oct 30 14:44:32 varnishd[21855]: Platf \
orm: Linux,2.6.32-131.0.15.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit</div> \
<div style="FONT-SIZE:9pt">Oct 30 14:44:32 varnishd[21855]: child (21856) Started</div>
<div style="FONT-SIZE:9pt">Oct 30 14:44:32 varnishd[21855]: Child (21856) said Child starts</div>
</blockquote>
</div>
</div>
</div></div></div>
<br>_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" \
target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br></blockquote></div><br><br \
clear="all"><div><br></div>-- <br>
<div style="font-size:12.727272033691406px;margin:5px 15px 0px \
0px;padding-bottom:5px;color:rgb(34,34,34);font-family:arial,sans-serif;background-color:rgb(255,255,255)"><table \
border="0" cellpadding="0" cellspacing="0" \
style="text-align:start;font-size:12px;line-height:1.5em;font-family:'Helvetica \
Neue',Arial,sans-serif;color:rgb(102,102,102);width:550px;border-top-width:1px;bor \
der-top-style:solid;border-top-color:rgb(238,238,238);border-bottom-width:1px;border-b \
ottom-style:solid;border-bottom-color:rgb(238,238,238);margin-top:20px;padding-top:5px;padding-bottom:5px">
<tbody><tr><td width="100" style="font-family:arial,sans-serif;margin:0px"><a \
href="http://www.varnish-software.com/" style="color:rgb(17,85,204)" \
target="_blank"><img \
src="http://www.varnish-software.com/static/media/logo-email.png" \
style="float:left;margin-left:10px"></a></td>
<td style="font-family:arial,sans-serif;margin:0px"><b \
style="font-size:14px;color:rgb(34,34,34)">Per Buer</b><br>CEO | Varnish Software \
AS<br>Phone: +47 958 39 117 | Skype: per.buer<br><span \
style="font-weight:bold">We Make Websites Fly!<br>
<br></span></td></tr></tbody></table><div></div></div><div \
style="font-size:12.727272033691406px;margin:15px \
0px;clear:both;color:rgb(34,34,34);font-family:arial,sans-serif;background-color:rgb(255,255,255)"></div><br>
</div>
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic