[prev in list] [next in list] [prev in thread] [next in thread] 

List:       xen-devel
Subject:    Re: [Xen-devel] phy disks and vifs timing out in DomU
From:       Anthony Wright <anthony () overnetdata ! com>
Date:       2011-07-30 17:05:24
Message-ID: 4E3439D4.6010502 () overnetdata ! com
[Download RAW message or body]

On 29/07/2011 21:01, Konrad Rzeszutek Wilk wrote:
> [Ian, I copied you on this b/c of the netbk issue - read on]
> 
> > > > > > > On Thu, Jul 28, 2011 at 7:24 AM, Anthony Wright \
> > > > > > > <anthony@overnetdata.com> wrote:
> > > > > > > > I have a 32 bit 3.0 Dom0 kernel running Xen 4.1. I am trying to run a \
> > > > > > > > 32 bit PV DomU with two tap:aio disks, two phy disks & 1 vif. The two \
> > > > > > > > tap:aio disks are working fine, but the phy disks and the vif don't \
> > > > > > > > work and I get the following error messages from the DomU kernel \
> > > > > > > > during boot: 
> > > > > > > > [    1.783658] Using IPI No-Shortcut mode
> > > > > > > > [   11.880061] XENBUS: Timeout connecting to device: device/vbd/51729 \
> > > > > > > > (state 3) [   11.880072] XENBUS: Timeout connecting to device: \
> > > > > > > > device/vbd/51745 (state 3)
> Hm, which version of DomU were these? I wonder if this is related to the \
> 'feature-barrier' that is not supported with 3.0. Do you see anything in the DomU \
> about the disks? or xen-blkfront? Can you run the guests with 'initcall_debug \
> loglevel=8 debug' to see if if the blkfront is actually running on those disks.
I have attached the domU console output with these options set.

I have also spent a fair amount of time trying to narrow down the
conditions that cause it, with lots of hardware switching & disk
imaging. The conclusion I came to was that it's not hardware related,
but there's a subtle interaction going on with LVM that's causing the
problem, but I'm struggling to work out how to narrow it down any
further than that.

I started with a setup that works: Machine 1 with HDD 1 (IDE), and a
setup that didn't: Machine 2 with HDD 2 (SATA). Machine 2 has an IDE
port so I unplugged HDD 2 and put HDD 1 in Machine 2 and that setup
worked thus excluding most of the hardware. Next I imaged HDD 3 (SATA)
from HDD 1 (IDE), unplugged HDD 1 and put HDD 3 in Machine 2, and that
setup worked, thus excluding an IDE/SATA issue, and giving me a disk I
could safely play with. The disks are organised into two partitions,
partition 1 is for Dom0, partition 2 is an LVM volume group and is used
for the DomUs. One LV (called Main) in this volume group is used by Dom0
to hold the DomU kernels, config information and other static data &
executables, the rest of the VG is issued as LVs to the various DomUs as
needed with a fair amount of free space left in the VG. I took the Main
LV from HDD 2 (didn't work) and imaged it onto HDD 3 and by judicious LV
renaming booted against this image and the setup failed - great I
thought - it's looks like a very subtle config issue. Next I created a
third LV this time imaged from the Main LV that worked, giving me three
Main LVs (I called them Main-Works, Main-Broken & Main-Testing) and I
simply use lvrename to select the one I wanted as active. However now I
couldn't get the setup to work with any of these three Main LVs
including the one that originally worked. Removing the LVs I had
recently created, and going back to the original Main LV, the setup
started working again.

I'm going to try an up to date version of LVM (the one I'm using is a
little out of date), and see if that makes any difference, but the
version I have at the moment has worked without problem in the past.
> Any idea where the source for those DomU's is? If it is an issue with \
> 'feature-barrier' it looks like it can't handle not having that option visible \
> which it should. 
We build the DomUs with a tightly controlled internal build sysem, so I
have a full manifest for the DomU.
> > > What device does that correspond to (hint: run xl block-list or xm block-list)?
> > > 
> > The output from block-list is:
> > 
> > Vdev  BE  handle state evt-ch ring-ref BE-path
> > 51729 0   764    3     10     10       /local/domain/0/backend/vbd/764/51729
> > 51745 0   764    3     11     11       /local/domain/0/backend/vbd/764/51745
> > 51713 0   764    4     8      8       
> > /local/domain/0/backend/qdisk/764/51713
> > 51714 0   764    4     9      9       
> > /local/domain/0/backend/qdisk/764/51714
> > 
> > The two vbds map to two LVM logical volumes in two different volume groups.
> qdisk.. ok so it does swap over to QEMU internal AIO path. From the output it looks
> like the ones that hang are the 'phy' types? Is that right?
> 
The ones that hang are phy and are the first two, with vdev numbers of
51729 & 51745.
> > On 29/07/2011 17:06, Konrad Rzeszutek Wilk wrote:
> > > > > I have installed virtually identical systems on two physical machines -
> > > > > identical (and I mean identical) xen, dom0, domU with possibly a
> > > md5sum match?
> > Yes - md5sum match on all the key components, i.e. xen, dom0 kernel,
> > 99.9% of the root filesystem, the domU kernel & 99.9% of the domU
> > filesystem. Where there isn't a precise match is on some of the config
> > files. I don't think these should have any effect, but I will have a go
> > at mirroring the disks (I can't swap disks since one is SATA & the other
> > IDE).
> > 
> > I also was having problems with the vif device, and got a kernel bug
> > report that could potentially relate to it. I've attached two syslogs.
> Yeah, that is bad. I actually see a similar issue if I kill forcibly the guests.
> I hadn't yet narrowed it down - .. you are looking to be using 4.1.. But not 
> 4.1.1 right?
I started with 4.1.0, but upgraded to 4.1.1 in the hope that it might
fix the problem. The vif timeouts have happened with both versions, but
I think the kernel errors have only been happening since I upgraded to
xen 4.1.1, however I'm not sure. I've also had a number of kernel Oops
in place of the kernel errors as well.
> Can you describe to me how you get the netbk crash?
The DomU when it realises it has a problem with one of it's disks issues
a warning message and then shuts itself down. The netbk crash happens
partway through that shutdown process, but not when the DomU is touching
the network (as far as I know) - it's issuing SIG KILLs to all
processes. It's always at the same point in the shutdown process, but
the shutdown process pauses at that point for quite a while and since it
doesn't touch the network, I'm not convinced it's triggered by something
that DomU is doing. The netbk crash only happens the first time the DomU
starts up & shuts down, it doesn't happen on subsequent DomU
startup/shutdown cycles. It also doesn't happen if the disks work
correctly. I do have a setup that consistently produces it.
> > 2011 Jul 29 07:02:10 kernel: [   33.242680] vbd vbd-1-51745: 1 mapping ring-ref \
> > 11 port 11                                                                 2011 \
> > Jul 29 07:02:10 kernel: [   33.253038] vif vif-1-0: vif1.0: failed to map tx \
> > ring. err=-12 status=-1                                                  2011 Jul \
> > 29 07:02:10 kernel: [   33.253065] vif vif-1-0: 1 mapping shared-frames 768/769 \
> > port 12                                                           2011 Jul 29 \
> > 07:02:43 kernel: [   66.103514] vif vif-1-0: 2 reading script                     \
> >  2011 Jul 29 07:02:43 kernel: [   66.106265] br-internal: port 1(vif1.0) entering \
> > disabled state                                                            2011 \
> > Jul 29 07:02:43 kernel: [   66.106309] libfcoe_device_notification: \
> > NETDEV_UNREGISTER vif1.0                                                          \
> > 2011 Jul 29 07:02:43 kernel: [   66.106333] br-internal: port 1(vif1.0) entering \
> > disabled state                                                            2011 \
> > Jul 29 07:02:43 kernel: [   66.106372] br-internal: mixed no checksumming and \
> > other settings.                                                         2011 Jul \
> > 29 07:02:43 kernel: [   66.114097] ------------[ cut here ]------------           \
> >  2011 Jul 29 07:02:43 kernel: [   66.114878] kernel BUG at mm/vmalloc.c:2164!     \
> >  2011 Jul 29 07:02:43 kernel: [   66.115058] invalid opcode: 0000 [#1] SMP        \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376] Modules linked in:                   \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]                                      \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376] Pid: 20, comm: xenwatch Not tainted \
> > 3.0.0 #1 MSI MS-7309/MS-7309                                               2011 \
> > Jul 29 07:02:43 kernel: [   66.115376] EIP: 0061:[<c0494bff>] EFLAGS: 00010203 \
> > CPU: 1                                                                 2011 Jul \
> > 29 07:02:43 kernel: [   66.115376] EIP is at free_vm_area+0xf/0x19                \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376] EAX: 00000000 EBX: cf866480 ECX: \
> > 00000018 EDX: 00000000                                                        \
> > 2011 Jul 29 07:02:43 kernel: [   66.115376] ESI: cfa06800 EDI: d076c400 EBP: \
> > cfa06c00 ESP: d0ce7eb4                                                        \
> > 2011 Jul 29 07:02:43 kernel: [   66.115376]  DS: 007b ES: 007b FS: 00d8 GS: 0000 \
> > SS: 0069                                                                  2011 \
> > Jul 29 07:02:43 kernel: [   66.115376] Process xenwatch (pid: 20, ti=d0ce6000 \
> > task=d0c55140 task.ti=d0ce6000)                                         2011 Jul \
> > 29 07:02:43 kernel: [   66.115376] Stack:                                         \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  cfa06c00 c09e87aa fffc6e63 c0c4bd65 \
> > d0ce7ecc cfa06844 d0ce7ecc d0ce7ecc                                       2011 \
> > Jul 29 07:02:43 kernel: [   66.115376]  cfa06c00 cfa06800 d076c400 cfa06c94 \
> > c09eace0 d04cd380 00000000 fffffffe                                       2011 \
> > Jul 29 07:02:43 kernel: [   66.115376]  d0ce7f9c c061fe74 d04cd2e0 d076c420 \
> > d076c400 d0ce7f9c c09e9f8c d076c400                                       2011 \
> > Jul 29 07:02:43 kernel: [   66.115376] Call Trace:                                \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c09e87aa>] ? \
> > xen_netbk_unmap_frontend_rings+0xbf/0xd3                                          \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c0c4bd65>] ? \
> > netdev_run_todo+0x1b7/0x1cc                                                       \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c09eace0>] ? \
> > xenvif_disconnect+0xd0/0xe4                                                       \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c061fe74>] ? xenbus_rm+0x37/0x3e  \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c09e9f8c>] ? \
> > netback_remove+0x40/0x5d                                                          \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c062075d>] ? \
> > xenbus_dev_remove+0x2c/0x3d                                                       \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c06620e6>] ? \
> > __device_release_driver+0x42/0x79                                                 \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c06621ac>] ? \
> > device_release_driver+0xf/0x17                                                    \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c0661818>] ? \
> > bus_remove_device+0x75/0x84                                                       \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c0660693>] ? \
> > device_del+0xe6/0x125                                                             \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c06606da>] ? \
> > device_unregister+0x8/0x10                                                        \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c06205f0>] ? \
> > xenbus_dev_changed+0x71/0x129                                                     \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c0405394>] ? check_events+0x8/0xc \
> >  2011 Jul 29 07:02:43 kernel: [   66.115376]  [<c061f711>] ? \
> > xenwatch_thread+0xeb/0x113                                                        \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624]  [<c044792c>] ? \
> > wake_up_bit+0x53/0x53                                                             \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624]  [<c061f626>] ? \
> > xenbus_thread+0x1cc/0x1cc                                                         \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624]  [<c0447616>] ? kthread+0x63/0x68    \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624]  [<c04475b3>] ? \
> > kthread_worker_fn+0x122/0x122                                                     \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624]  [<c0e0f036>] ? \
> > kernel_thread_helper+0x6/0x10                                                     \
> >  2011 Jul 29 07:02:43 kernel: [   66.129624] Code: c1 00 00 00 01 89 f0 e8 a1 ff \
> > ff ff 81 6b 08 00 10 00 00 eb 02 31 db 89 d8 5b 5e c3 53 89 c3 8b 40 04 e8 9b ff \
> > ff ff 39 d8 74 04 <0f> 0b eb fe 5b e9 73 95 00 00 57 89 d7 56 31 f6 53 89 c3 eb \
> > 09                                                                  2011 Jul 29 \
> > 07:02:43 kernel: [   66.129624] EIP: [<c0494bff>] free_vm_area+0xf/0x19 SS:ESP \
> > 0069:d0ce7eb4                                                   2011 Jul 29 \
> > 07:02:43 kernel: [   66.129624] ---[ end trace 7bb110af96f32256 ]---              \
> > 


["bootlog-debug" (text/plain)]

[    3.226316] Reserving virtual address space above 0xf5800000
[    3.226316] Linux version 2.6.30.1 (root@linux-krq0) (gcc version 4.3.2 (GCC) ) #2 \
SMP Thu Jun 24 22:18:52 GMT 2010 [    3.226316] KERNEL supported cpus:                \
 [    3.226316]   Intel GenuineIntel                                                  \
 [    3.226316]   AMD AuthenticAMD                                                    \
 [    3.226316]   NSC Geode by NSC                                                    \
 [    3.226316]   Cyrix CyrixInstead                                                  \
 [    3.226316]   Centaur CentaurHauls                                                \
 [    3.226316]   Transmeta GenuineTMx86                                              \
 [    3.226316]   Transmeta TransmetaCPU                                              \
 [    3.226316]   UMC UMC UMC UMC                                                     \
 [    3.226316] BIOS-provided physical RAM map:                                       \
 [    3.226316]  Xen: 0000000000000000 - 00000000000a0000 (usable)                    \
 [    3.226316]  Xen: 00000000000a0000 - 0000000000100000 (reserved)                  \
 [    3.226316]  Xen: 0000000000100000 - 000000000065e000 (usable)                    \
 [    3.226316]  Xen: 000000000065e000 - 000000000075b000 (reserved)                  \
 [    3.226316]  Xen: 000000000075b000 - 000000003e800000 (usable)                    \
 [    3.226316] DMI not present or invalid.                                           \
 [    3.226316] last_pfn = 0x3e800 max_arch_pfn = 0x1000000                           \
 [    3.226316] init_memory_mapping: 0000000000000000-00000000229fe000                \
 [    3.226316] NX (Execute Disable) protection: active                               \
 [    3.226316]  0000000000 - 00229fe000 page 4k                                      \
 [    3.226316] kernel direct mapping tables up to 229fe000 @ 539000-652000           \
 [    3.226316] 446MB HIGHMEM available.                                              \
 [    3.226316] 553MB LOWMEM available.                                               \
 [    3.226316]   mapped low ram: 0 - 229fe000                                        \
 [    3.226316]   low ram: 0 - 229fe000                                               \
 [    3.226316]   node 0 low ram: 00000000 - 229fe000                                 \
 [    3.226316]   node 0 bootmap 00007000 - 0000b540                                  \
 [    3.226316] (7 early reservations) ==> bootmem [0000000000 - 00229fe000]          \
 [    3.226316]   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - \
0000001000]                           [    3.226316]   #1 [000075b000 - 0000761000]   \
XEN PAGETABLES ==> [000075b000 - 0000761000]                           [    3.226316] \
#2 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]           \
 [    3.226316]   #3 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - \
0000007000]                           [    3.226316]   #4 [0000100000 - 00005386f4]   \
TEXT DATA BSS ==> [0000100000 - 00005386f4]                           [    3.226316]  \
#5 [0000539000 - 0000647000]          PGTABLE ==> [0000539000 - 0000647000]           \
 [    3.226316]   #6 [0000007000 - 000000c000]          BOOTMAP ==> [0000007000 - \
000000c000]                           [    3.226316] Zone PFN ranges:                 \
 [    3.226316]   DMA      0x00000000 -> 0x00001000                                   \
 [    3.226316]   Normal   0x00001000 -> 0x000229fe                                   \
 [    3.226316]   HighMem  0x000229fe -> 0x0003e800                                   \
 [    3.226316] Movable zone start PFN for each node                                  \
 [    3.226316] early_node_map[3] active PFN ranges                                   \
 [    3.226316]     0: 0x00000000 -> 0x000000a0                                       \
 [    3.226316]     0: 0x00000100 -> 0x0000065e                                       \
 [    3.226316]     0: 0x0000075b -> 0x0003e800                                       \
 [    3.226316] On node 0 totalpages: 255651                                          \
 [    3.226316] free_area_init_node: node 0, pgdat c046a580, node_mem_map c1000000    \
 [    3.226316]   DMA zone: 32 pages used for memmap                                  \
 [    3.226316]   DMA zone: 0 pages reserved                                          \
 [    3.226316]   DMA zone: 3715 pages, LIFO batch:0                                  \
 [    3.226316]   Normal zone: 1076 pages used for memmap                             \
 [    3.226316]   Normal zone: 136650 pages, LIFO batch:31                            \
 [    3.226316]   HighMem zone: 893 pages used for memmap                             \
 [    3.226316]   HighMem zone: 113285 pages, LIFO batch:31                           \
 [    3.226316] Using APIC driver default                                             \
 [    3.226316] SMP: Allowing 1 CPUs, 0 hotplug CPUs                                  \
 [    3.226316] Local APIC disabled by BIOS -- you can enable it with "lapic"         \
 [    3.226316] nr_irqs_gsi: 16                                                       \
 [    3.226316] Allocating PCI resources starting at 40000000 (gap: \
3e800000:c1800000)                                  [    3.226316] NR_CPUS:8 \
nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1                                          \
 [    3.226316] PERCPU: Allocated 6 4k pages, static data 22940 bytes                 \
 [    4.118093] trying to map vcpu_info 0 at e31ff00c, mfn 124a32, offset 12          \
 [    3.810521] cpu 0 using vcpu_info at e31ff00c                                     \
 [    3.810521] Xen: using vcpu_info placement                                        \
 [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: \
253650                             [    0.000000] Kernel command line: \
root=/dev/xvda1 initcall_debug loglevel=8 debug                                    [  \
0.000000] Enabling fast FPU save and restore... done.                                 \
 [    0.000000] Enabling unmasked SIMD FPU exception support... done.                 \
 [    0.000000] Initializing CPU#0                                                    \
 [    0.000000] NR_IRQS:512                                                           \
 [    0.000000] PID hash table entries: 4096 (order: 12, 16384 bytes)                 \
 [    0.000000] Detected 2712.512 MHz processor.                                      \
 [    0.010000] Console: colour dummy device 80x25                                    \
 [    0.010000] console [tty0] enabled                                                \
 [    0.010000] console [hvc0] enabled                                                \
 [    0.010000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)      \
 [    0.010000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)        \
 [    0.010000] Initializing HighMem for node 0 (000229fe:0003e800)                   \
 [    0.010000] Memory: 1007440k/1024000k available (2539k kernel code, 14300k \
reserved, 1071k data, 244k init, 456712k highmem) [    0.010000] virtual kernel \
memory layout:                                                                        \
 [    0.010000]     fixmap  : 0xf574f000 - 0xf57ff000   ( 704 kB)                     \
 [    0.010000]     pkmap   : 0xf5200000 - 0xf5400000   (2048 kB)                     \
 [    0.010000]     vmalloc : 0xe31fe000 - 0xf51fe000   ( 288 MB)                     \
 [    0.010000]     lowmem  : 0xc0000000 - 0xe29fe000   ( 553 MB)                     \
 [    0.010000]       .init : 0xc0492000 - 0xc04cf000   ( 244 kB)                     \
 [    0.010000]       .data : 0xc037af15 - 0xc0486e18   (1071 kB)                     \
 [    0.010000]       .text : 0xc0100000 - 0xc037af15   (2539 kB)                     \
 [    0.010000] Checking if this processor honours the WP bit even in supervisor \
mode...Ok.                                      [    0.010000] Xen: using vcpuop \
timer interface                                                                       \
 [    0.010000] installing Xen timer for CPU 0                                        \
 [    0.010000] Calibrating delay loop (skipped), value calculated using timer \
frequency.. 5425.02 BogoMIPS (lpj=27125120)       [    0.010000] Mount-cache hash \
table entries: 512                                                                    \
 [    0.010000] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)     \
 [    0.010000] CPU: L2 Cache: 512K (64 bytes/line)                                   \
 [    0.010000] CPU: Physical Processor ID: 0                                         \
 [    0.010000] CPU: Processor Core ID: 1                                             \
 [    0.010000] SMP alternatives: switching to UP code                                \
 [    0.030000] Freeing SMP alternatives: 28k freed                                   \
 [    0.030064] calling  migration_init+0x0/0x4b @ 1                                  \
 [    0.030096] initcall migration_init+0x0/0x4b returned 1 after 0 usecs             \
 [    0.030107] initcall migration_init+0x0/0x4b returned with error code 1           \
 [    0.030114] calling  spawn_ksoftirqd+0x0/0x47 @ 1                                 \
 [    0.030143] initcall spawn_ksoftirqd+0x0/0x47 returned 0 after 0 usecs            \
 [    0.030153] calling  init_call_single_data+0x0/0x70 @ 1                           \
 [    0.030160] initcall init_call_single_data+0x0/0x70 returned 0 after 0 usecs      \
 [    0.030167] calling  tracer_alloc_buffers+0x0/0x12a @ 1                           \
 [    0.030250] initcall tracer_alloc_buffers+0x0/0x12a returned 0 after 0 usecs      \
 [    0.030260] calling  init_trace_printk+0x0/0xf @ 1                                \
 [    0.030267] initcall init_trace_printk+0x0/0xf returned 0 after 0 usecs           \
 [    0.030273] Brought up 1 CPUs                                                     \
 [    0.030494] calling  net_ns_init+0x0/0x12d @ 1                                    \
 [    0.030502] net_namespace: 452 bytes                                              \
 [    0.030509] initcall net_ns_init+0x0/0x12d returned 0 after 0 usecs               \
 [    0.030518] calling  reboot_init+0x0/0x11 @ 1                                     \
 [    0.030525] initcall reboot_init+0x0/0x11 returned 0 after 0 usecs                \
 [    0.030532] calling  print_banner+0x0/0xd @ 1                                     \
 [    0.030539] Booting paravirtualized kernel on Xen                                 \
 [    0.030544] Xen version: 4.1.1 (preserve-AD)                                      \
 [    0.030550] initcall print_banner+0x0/0xd returned 0 after 0 usecs                \
 [    0.030557] calling  init_smp_flush+0x0/0x1e @ 1                                  \
 [    0.030564] initcall init_smp_flush+0x0/0x1e returned 0 after 0 usecs             \
 [    0.030571] calling  sysctl_init+0x0/0x13 @ 1                                     \
 [    0.030579] initcall sysctl_init+0x0/0x13 returned 0 after 0 usecs                \
 [    0.030586] calling  ksysfs_init+0x0/0x96 @ 1                                     \
 [    0.030597] initcall ksysfs_init+0x0/0x96 returned 0 after 0 usecs                \
 [    0.030604] calling  async_init+0x0/0x50 @ 1                                      \
 [    0.030627] initcall async_init+0x0/0x50 returned 0 after 0 usecs                 \
 [    0.030635] calling  init_jiffies_clocksource+0x0/0xf @ 1                         \
 [    0.030643] initcall init_jiffies_clocksource+0x0/0xf returned 0 after 0 usecs    \
 [    0.030651] calling  filelock_init+0x0/0x27 @ 1                                   \
 [    0.030661] initcall filelock_init+0x0/0x27 returned 0 after 0 usecs              \
 [    0.030668] calling  init_script_binfmt+0x0/0x11 @ 1                              \
 [    0.030674] initcall init_script_binfmt+0x0/0x11 returned 0 after 0 usecs         \
 [    0.030681] calling  init_elf_binfmt+0x0/0x11 @ 1                                 \
 [    0.030687] initcall init_elf_binfmt+0x0/0x11 returned 0 after 0 usecs            \
 [    0.030694] calling  debugfs_init+0x0/0x4a @ 1                                    \
 [    0.030702] initcall debugfs_init+0x0/0x4a returned 0 after 0 usecs               \
 [    0.030709] calling  calibrate_xor_blocks+0x0/0x13b @ 1                           \
 [    0.030715] xor: automatically using best checksumming function: pIII_sse         \
 [    0.080004]    pIII_sse  :  2292.400 MB/sec                                       \
 [    0.080010] xor: using function: pIII_sse (2292.400 MB/sec)                       \
 [    0.080018] initcall calibrate_xor_blocks+0x0/0x13b returned 0 after 48828 usecs  \
 [    0.080026] calling  random32_init+0x0/0xa8 @ 1                                   \
 [    0.080032] initcall random32_init+0x0/0xa8 returned 0 after 0 usecs              \
 [    0.080039] calling  gnttab_init+0x0/0x16c @ 1                                    \
 [    0.080083] Grant table initialized                                               \
 [    0.080090] initcall gnttab_init+0x0/0x16c returned 0 after 0 usecs               \
 [    0.080097] calling  sock_init+0x0/0x51 @ 1                                       \
 [    0.080129] initcall sock_init+0x0/0x51 returned 0 after 0 usecs                  \
 [    0.080137] calling  netlink_proto_init+0x0/0x150 @ 1                             \
 [    0.080150] NET: Registered protocol family 16                                    \
 [    0.080164] initcall netlink_proto_init+0x0/0x150 returned 0 after 0 usecs        \
 [    0.080171] calling  bdi_class_init+0x0/0x35 @ 1                                  \
 [    0.080230] initcall bdi_class_init+0x0/0x35 returned 0 after 0 usecs             \
 [    0.080239] calling  kobject_uevent_init+0x0/0x4e @ 1                             \
 [    0.080247] initcall kobject_uevent_init+0x0/0x4e returned 0 after 0 usecs        \
 [    0.080254] calling  xenbus_probe_init+0x0/0xa3 @ 1                               \
 [    0.081134] initcall xenbus_probe_init+0x0/0xa3 returned 0 after 0 usecs          \
 [    0.081143] calling  tty_class_init+0x0/0x28 @ 1                                  \
 [    0.081182] initcall tty_class_init+0x0/0x28 returned 0 after 0 usecs             \
 [    0.081190] calling  vtconsole_class_init+0x0/0xa1 @ 1                            \
 [    0.081270] initcall vtconsole_class_init+0x0/0xa1 returned 0 after 0 usecs       \
 [    0.081279] calling  arch_kdebugfs_init+0x0/0x1e @ 1                              \
 [    0.081287] initcall arch_kdebugfs_init+0x0/0x1e returned 0 after 0 usecs         \
 [    0.081294] calling  init_pit_clocksource+0x0/0x91 @ 1                            \
 [    0.081301] initcall init_pit_clocksource+0x0/0x91 returned 0 after 0 usecs       \
 [    0.081308] calling  setup_vcpu_hotplug_event+0x0/0x1f @ 1                        \
 [    0.081427] initcall setup_vcpu_hotplug_event+0x0/0x1f returned 0 after 0 usecs   \
 [    0.081435] calling  dmi_id_init+0x0/0x2b4 @ 1                                    \
 [    0.081441] initcall dmi_id_init+0x0/0x2b4 returned -19 after 0 usecs             \
 [    0.081448] calling  topology_init+0x0/0x36 @ 1                                   \
 [    0.081487] initcall topology_init+0x0/0x36 returned 0 after 0 usecs              \
 [    0.081495] calling  param_sysfs_init+0x0/0x1d2 @ 1                               \
 [    0.083160] initcall param_sysfs_init+0x0/0x1d2 returned 0 after 0 usecs          \
 [    0.083172] calling  default_bdi_init+0x0/0x2f @ 1                                \
 [    0.083222] initcall default_bdi_init+0x0/0x2f returned 0 after 0 usecs           \
 [    0.083230] calling  init_bio+0x0/0xf1 @ 1                                        \
 [    0.083250] bio: create slab <bio-0> at 0                                         \
 [    0.083261] initcall init_bio+0x0/0xf1 returned 0 after 0 usecs                   \
 [    0.083269] calling  cryptomgr_init+0x0/0x29 @ 1                                  \
 [    0.083279] initcall cryptomgr_init+0x0/0x29 returned 0 after 0 usecs             \
 [    0.083286] calling  blk_settings_init+0x0/0x1d @ 1                               \
 [    0.083293] initcall blk_settings_init+0x0/0x1d returned 0 after 0 usecs          \
 [    0.083299] calling  blk_ioc_init+0x0/0x24 @ 1                                    \
 [    0.083308] initcall blk_ioc_init+0x0/0x24 returned 0 after 0 usecs               \
 [    0.083316] calling  blk_softirq_init+0x0/0x5b @ 1                                \
 [    0.083322] initcall blk_softirq_init+0x0/0x5b returned 0 after 0 usecs           \
 [    0.083329] calling  genhd_device_init+0x0/0x61 @ 1                               \
 [    0.083400] initcall genhd_device_init+0x0/0x61 returned 0 after 0 usecs          \
 [    0.083408] calling  setup_shutdown_event+0x0/0x11 @ 1                            \
 [    0.083541] initcall setup_shutdown_event+0x0/0x11 returned 0 after 0 usecs       \
 [    0.083549] calling  balloon_init+0x0/0x1ff @ 1                                   \
 [    0.083553] xen_balloon: Initialising balloon driver.                             \
 [    0.085085] initcall balloon_init+0x0/0x1ff returned 0 after 0 usecs              \
 [    0.085093] calling  misc_init+0x0/0x84 @ 1                                       \
[    0.085161] calling  serio_init+0x0/0x73 @ 1                                       \
 [    0.085258] initcall serio_init+0x0/0x73 returned 0 after 0 usecs                 \
 [    0.085265] calling  input_init+0x0/0x106 @ 1                                     \
 [    0.085313] initcall input_init+0x0/0x106 returned 0 after 0 usecs                \
 [    0.085320] calling  raid6_select_algo+0x0/0x168 @ 1                              \
 [    0.250034] raid6: int32x1    693 MB/s                                            \
 [    0.420029] raid6: int32x2   1283 MB/s                                            \
 [    0.590013] raid6: int32x4    624 MB/s                                            \
 [    0.760034] raid6: int32x8    765 MB/s                                            \
 [    0.930018] raid6: mmxx1     2116 MB/s                                            \
 [    1.100008] raid6: mmxx2     3931 MB/s                                            \
 [    1.270035] raid6: sse1x1    1734 MB/s                                            \
 [    1.440019] raid6: sse1x2    2423 MB/s                                            \
 [    1.610017] raid6: sse2x1    3027 MB/s                                            \
 [    1.780008] raid6: sse2x2    4285 MB/s                                            \
 [    1.780023] raid6: using algorithm sse2x2 (4285 MB/s)                             \
 [    1.780033] initcall raid6_select_algo+0x0/0x168 returned 0 after 1660156 usecs   \
 [    1.780041] calling  md_init+0x0/0xb7 @ 1                                         \
 [    1.780053] initcall md_init+0x0/0xb7 returned 0 after 0 usecs                    \
 [    1.780061] calling  proto_init+0x0/0xf @ 1                                       \
 [    1.780069] initcall proto_init+0x0/0xf returned 0 after 0 usecs                  \
 [    1.780076] calling  net_dev_init+0x0/0x142 @ 1                                   \
 [    1.780202] initcall net_dev_init+0x0/0x142 returned 0 after 0 usecs              \
 [    1.780211] calling  neigh_init+0x0/0x66 @ 1                                      \
 [    1.780218] initcall neigh_init+0x0/0x66 returned 0 after 0 usecs                 \
 [    1.780225] calling  genl_init+0x0/0xb0 @ 1                                       \
 [    1.780237] initcall genl_init+0x0/0xb0 returned 0 after 0 usecs                  \
 [    1.780245] calling  sysctl_init+0x0/0x3b @ 1                                     \
 [    1.780251] initcall sysctl_init+0x0/0x3b returned 0 after 0 usecs                \
 [    1.780258] calling  pci_iommu_init+0x0/0xc @ 1                                   \
 [    1.780265] initcall pci_iommu_init+0x0/0xc returned 0 after 0 usecs              \
 [    1.780272] calling  print_all_ICs+0x0/0x47d @ 1                                  \
 [    1.780279] initcall print_all_ICs+0x0/0x47d returned 0 after 0 usecs             \
 [    1.780286] calling  clocksource_done_booting+0x0/0x11 @ 1                        \
 [    1.780293] initcall clocksource_done_booting+0x0/0x11 returned 0 after 0 usecs   \
 [    1.780301] calling  rb_init_debugfs+0x0/0x38 @ 1                                 \
 [    1.780311] initcall rb_init_debugfs+0x0/0x38 returned 0 after 0 usecs            \
 [    1.780318] calling  tracer_init_debugfs+0x0/0x344 @ 1                            \
 [    1.780373] initcall tracer_init_debugfs+0x0/0x344 returned 0 after 0 usecs       \
 [    1.780382] calling  init_trace_printk_function_export+0x0/0x39 @ 1               \
 [    1.780391] initcall init_trace_printk_function_export+0x0/0x39 returned 0 after \
0 usecs                                     [    1.780399] calling  \
init_pipe_fs+0x0/0x3d @ 1                                                             \
 [    1.780411] initcall init_pipe_fs+0x0/0x3d returned 0 after 0 usecs               \
 [    1.780419] calling  init_mnt_writers+0x0/0x4a @ 1                                \
 [    1.780425] initcall init_mnt_writers+0x0/0x4a returned 0 after 0 usecs           \
 [    1.780432] calling  eventpoll_init+0x0/0xa5 @ 1                                  \
 [    1.780447] initcall eventpoll_init+0x0/0xa5 returned 0 after 0 usecs             \
 [    1.780454] calling  anon_inode_init+0x0/0xe3 @ 1                                 \
 [    1.780466] initcall anon_inode_init+0x0/0xe3 returned 0 after 0 usecs            \
 [    1.780473] calling  chr_dev_init+0x0/0x89 @ 1                                    \
 [    1.780803] initcall chr_dev_init+0x0/0x89 returned 0 after 0 usecs               \
 [    1.780813] calling  firmware_class_init+0x0/0x61 @ 1                             \
 [    1.780853] initcall firmware_class_init+0x0/0x61 returned 0 after 0 usecs        \
 [    1.780862] calling  sysctl_core_init+0x0/0x2d @ 1                                \
 [    1.780872] initcall sysctl_core_init+0x0/0x2d returned 0 after 0 usecs           \
 [    1.780879] calling  inet_init+0x0/0x199 @ 1                                      \
 [    1.780901] NET: Registered protocol family 2                                     \
 [    1.780940] IP route cache hash table entries: 32768 (order: 5, 131072 bytes)     \
 [    1.781034] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)  \
 [    1.781498] TCP bind hash table entries: 65536 (order: 7, 524288 bytes)           \
 [    1.781727] TCP: Hash tables configured (established 131072 bind 65536)           \
 [    1.781736] TCP reno registered                                                   \
 [    1.781789] initcall inet_init+0x0/0x199 returned 0 after 0 usecs                 \
 [    1.781798] calling  af_unix_init+0x0/0x47 @ 1                                    \
 [    1.781810] NET: Registered protocol family 1                                     \
 [    1.781820] initcall af_unix_init+0x0/0x47 returned 0 after 0 usecs               \
 [    1.781828] calling  default_rootfs+0x0/0x65 @ 1                                  \
 [    1.781844] initcall default_rootfs+0x0/0x65 returned 0 after 0 usecs             \
 [    1.781852] calling  i8259A_init_sysfs+0x0/0x1d @ 1                               \
 [    1.781945] initcall i8259A_init_sysfs+0x0/0x1d returned 0 after 0 usecs          \
 [    1.781954] calling  sbf_init+0x0/0xdd @ 1                                        \
 [    1.781961] initcall sbf_init+0x0/0xdd returned 0 after 0 usecs                   \
 [    1.781968] calling  i8237A_init_sysfs+0x0/0x1d @ 1                               \
 [    1.782035] initcall i8237A_init_sysfs+0x0/0x1d returned 0 after 0 usecs          \
 [    1.782045] calling  add_rtc_cmos+0x0/0x31 @ 1                                    \
 [    1.782089] platform rtc_cmos: registered platform RTC device (no PNP device \
found)                                          [    1.782099] initcall \
add_rtc_cmos+0x0/0x31 returned 0 after 0 usecs                                        \
 [    1.782106] calling  cache_sysfs_init+0x0/0x55 @ 1                                \
 [    1.782293] initcall cache_sysfs_init+0x0/0x55 returned 0 after 0 usecs           \
 [    1.782302] calling  cpuid_init+0x0/0x10e @ 1                                     \
 [    1.782385] initcall cpuid_init+0x0/0x10e returned 0 after 0 usecs                \
 [    1.782394] calling  ioapic_init_sysfs+0x0/0x84 @ 1                               \
 [    1.782434] initcall ioapic_init_sysfs+0x0/0x84 returned 0 after 0 usecs          \
 [    1.782442] calling  add_pcspkr+0x0/0x24 @ 1                                      \
 [    1.782483] initcall add_pcspkr+0x0/0x24 returned 0 after 0 usecs                 \
 [    1.782491] calling  proc_execdomains_init+0x0/0x1c @ 1                           \
 [    1.782500] initcall proc_execdomains_init+0x0/0x1c returned 0 after 0 usecs      \
 [    1.782508] calling  ioresources_init+0x0/0x31 @ 1                                \
 [    1.782517] initcall ioresources_init+0x0/0x31 returned 0 after 0 usecs           \
 [    1.782524] calling  uid_cache_init+0x0/0x7b @ 1                                  \
 [    1.782538] initcall uid_cache_init+0x0/0x7b returned 0 after 0 usecs             \
 [    1.782545] calling  init_posix_timers+0x0/0xa9 @ 1                               \
 [    1.782556] initcall init_posix_timers+0x0/0xa9 returned 0 after 0 usecs          \
 [    1.782563] calling  init_posix_cpu_timers+0x0/0x8d @ 1                           \
 [    1.782569] initcall init_posix_cpu_timers+0x0/0x8d returned 0 after 0 usecs      \
 [    1.782577] calling  nsproxy_cache_init+0x0/0x27 @ 1                              \
 [    1.782586] initcall nsproxy_cache_init+0x0/0x27 returned 0 after 0 usecs         \
 [    1.782594] calling  create_proc_profile+0x0/0x18b @ 1                            \
 [    1.782600] initcall create_proc_profile+0x0/0x18b returned 0 after 0 usecs       \
 [    1.782608] calling  timekeeping_init_device+0x0/0x1d @ 1                         \
 [    1.782677] initcall timekeeping_init_device+0x0/0x1d returned 0 after 0 usecs    \
 [    1.782687] calling  init_clocksource_sysfs+0x0/0x43 @ 1                          \
 [    1.782758] initcall init_clocksource_sysfs+0x0/0x43 returned 0 after 0 usecs     \
 [    1.782767] calling  init_timer_list_procfs+0x0/0x27 @ 1                          \
 [    1.782775] initcall init_timer_list_procfs+0x0/0x27 returned 0 after 0 usecs     \
 [    1.782783] calling  futex_init+0x0/0x81 @ 1                                      \
 [    1.782793] initcall futex_init+0x0/0x81 returned 0 after 0 usecs                 \
 [    1.782800] calling  proc_dma_init+0x0/0x1c @ 1                                   \
 [    1.782808] initcall proc_dma_init+0x0/0x1c returned 0 after 0 usecs              \
 [    1.782816] calling  proc_modules_init+0x0/0x1c @ 1                               \
 [    1.782824] initcall proc_modules_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.782831] calling  kallsyms_init+0x0/0x1f @ 1                                   \
 [    1.782839] initcall kallsyms_init+0x0/0x1f returned 0 after 0 usecs              \
 [    1.782846] calling  utsname_sysctl_init+0x0/0x11 @ 1                             \
 [    1.782853] initcall utsname_sysctl_init+0x0/0x11 returned 0 after 0 usecs        \
 [    1.782862] calling  init_tracepoints+0x0/0xf @ 1                                 \
 [    1.782869] initcall init_tracepoints+0x0/0xf returned 0 after 0 usecs            \
 [    1.782876] calling  init_events+0x0/0x5a @ 1                                     \
 [    1.782883] initcall init_events+0x0/0x5a returned 0 after 0 usecs                \
 [    1.782890] calling  init_per_zone_pages_min+0x0/0x77 @ 1                         \
 [    1.782971] initcall init_per_zone_pages_min+0x0/0x77 returned 0 after 0 usecs    \
 [    1.782981] calling  pdflush_init+0x0/0x1b @ 1                                    \
 [    1.783026] initcall pdflush_init+0x0/0x1b returned 0 after 0 usecs               \
 [    1.783034] calling  kswapd_init+0x0/0x1d @ 1                                     \
 [    1.783062] initcall kswapd_init+0x0/0x1d returned 0 after 0 usecs                \
 [    1.783071] calling  init_tmpfs+0x0/0xb9 @ 1                                      \
 [    1.783095] initcall init_tmpfs+0x0/0xb9 returned 0 after 0 usecs                 \
 [    1.783104] calling  setup_vmstat+0x0/0xa8 @ 1                                    \
 [    1.783118] initcall setup_vmstat+0x0/0xa8 returned 0 after 0 usecs               \
 [    1.783125] calling  mm_sysfs_init+0x0/0x22 @ 1                                   \
 [    1.783134] initcall mm_sysfs_init+0x0/0x22 returned 0 after 0 usecs              \
 [    1.783142] calling  proc_vmalloc_init+0x0/0x1f @ 1                               \
 [    1.783150] initcall proc_vmalloc_init+0x0/0x1f returned 0 after 0 usecs          \
 [    1.783157] calling  init_emergency_pool+0x0/0x57 @ 1                             \
 [    1.783188] highmem bounce pool size: 64 pages                                    \
 [    1.783196] initcall init_emergency_pool+0x0/0x57 returned 0 after 0 usecs        \
 [    1.783203] calling  procswaps_init+0x0/0x1c @ 1                                  \
 [    1.783211] initcall procswaps_init+0x0/0x1c returned 0 after 0 usecs             \
 [    1.783218] calling  hugetlb_init+0x0/0x277 @ 1                                   \
 [    1.783224] HugeTLB registered 2 MB page size, pre-allocated 0 pages              \
 [    1.783237] initcall hugetlb_init+0x0/0x277 returned 0 after 0 usecs              \
 [    1.783244] calling  slab_proc_init+0x0/0x1f @ 1                                  \
 [    1.783251] initcall slab_proc_init+0x0/0x1f returned 0 after 0 usecs             \
 [    1.783258] calling  cpucache_init+0x0/0x36 @ 1                                   \
 [    1.783264] initcall cpucache_init+0x0/0x36 returned 0 after 0 usecs              \
 [    1.783270] calling  fasync_init+0x0/0x24 @ 1                                     \
 [    1.783281] initcall fasync_init+0x0/0x24 returned 0 after 0 usecs                \
 [    1.783288] calling  proc_filesystems_init+0x0/0x1c @ 1                           \
 [    1.783296] initcall proc_filesystems_init+0x0/0x1c returned 0 after 0 usecs      \
 [    1.783303] calling  dnotify_init+0x0/0x24 @ 1                                    \
 [    1.783313] initcall dnotify_init+0x0/0x24 returned 0 after 0 usecs               \
 [    1.783319] calling  inotify_setup+0x0/0x11 @ 1                                   \
 [    1.783325] initcall inotify_setup+0x0/0x11 returned 0 after 0 usecs              \
 [    1.783332] calling  inotify_user_setup+0x0/0x98 @ 1                              \
 [    1.783352] initcall inotify_user_setup+0x0/0x98 returned 0 after 0 usecs         \
 [    1.783360] calling  aio_setup+0x0/0x60 @ 1                                       \
 [    1.783394] initcall aio_setup+0x0/0x60 returned 0 after 0 usecs                  \
 [    1.783402] calling  proc_locks_init+0x0/0x1c @ 1                                 \
 [    1.783409] initcall proc_locks_init+0x0/0x1c returned 0 after 0 usecs            \
 [    1.783416] calling  init_mbcache+0x0/0x11 @ 1                                    \
 [    1.783422] initcall init_mbcache+0x0/0x11 returned 0 after 0 usecs               \
 [    1.783429] calling  dquot_init+0x0/0xe4 @ 1                                      \
 [    1.783434] VFS: Disk quotas dquot_6.5.2                                          \
 [    1.783446] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)            \
 [    1.783458] initcall dquot_init+0x0/0xe4 returned 0 after 0 usecs                 \
 [    1.783465] calling  proc_cmdline_init+0x0/0x1c @ 1                               \
 [    1.783472] initcall proc_cmdline_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783479] calling  proc_cpuinfo_init+0x0/0x1c @ 1                               \
 [    1.783487] initcall proc_cpuinfo_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783494] calling  proc_devices_init+0x0/0x1c @ 1                               \
 [    1.783501] initcall proc_devices_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783508] calling  proc_interrupts_init+0x0/0x1c @ 1                            \
 [    1.783516] initcall proc_interrupts_init+0x0/0x1c returned 0 after 0 usecs       \
 [    1.783523] calling  proc_loadavg_init+0x0/0x1c @ 1                               \
 [    1.783531] initcall proc_loadavg_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783538] calling  proc_meminfo_init+0x0/0x1c @ 1                               \
 [    1.783545] initcall proc_meminfo_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783552] calling  proc_stat_init+0x0/0x1c @ 1                                  \
 [    1.783560] initcall proc_stat_init+0x0/0x1c returned 0 after 0 usecs             \
 [    1.783567] calling  proc_uptime_init+0x0/0x1c @ 1                                \
 [    1.783575] initcall proc_uptime_init+0x0/0x1c returned 0 after 0 usecs           \
 [    1.783582] calling  proc_version_init+0x0/0x1c @ 1                               \
 [    1.783590] initcall proc_version_init+0x0/0x1c returned 0 after 0 usecs          \
 [    1.783597] calling  proc_kcore_init+0x0/0x40 @ 1                                 \
 [    1.783605] initcall proc_kcore_init+0x0/0x40 returned 0 after 0 usecs            \
 [    1.783612] calling  proc_kmsg_init+0x0/0x1f @ 1                                  \
 [    1.783619] initcall proc_kmsg_init+0x0/0x1f returned 0 after 0 usecs             \
 [    1.783626] calling  proc_page_init+0x0/0x37 @ 1                                  \
 [    1.783635] initcall proc_page_init+0x0/0x37 returned 0 after 0 usecs             \
 [    1.783642] calling  init_devpts_fs+0x0/0x32 @ 1                                  \
 [    1.783656] initcall init_devpts_fs+0x0/0x32 returned 0 after 0 usecs             \
 [    1.783663] calling  init_ext3_fs+0x0/0x64 @ 1                                    \
 [    1.783679] initcall init_ext3_fs+0x0/0x64 returned 0 after 0 usecs               \
 [    1.783686] calling  init_ext2_fs+0x0/0x64 @ 1                                    \
 [    1.783700] initcall init_ext2_fs+0x0/0x64 returned 0 after 0 usecs               \
 [    1.783707] calling  init_ext4_fs+0x0/0xa5 @ 1                                    \
 [    1.783734] initcall init_ext4_fs+0x0/0xa5 returned 0 after 0 usecs               \
 [    1.783743] calling  journal_init+0x0/0x85 @ 1                                    \
 [    1.783760] initcall journal_init+0x0/0x85 returned 0 after 0 usecs               \
 [    1.783767] calling  journal_init+0x0/0x9a @ 1                                    \
[    1.783795] calling  init_squashfs_fs+0x0/0x5d @ 1                                 \
 [    1.783804] squashfs: version 4.0 (2009/01/31) Phillip Lougher                    \
 [    1.783811] initcall init_squashfs_fs+0x0/0x5d returned 0 after 0 usecs           \
 [    1.783818] calling  init_ramfs_fs+0x0/0xf @ 1                                    \
 [    1.783824] initcall init_ramfs_fs+0x0/0xf returned 0 after 0 usecs               \
 [    1.783831] calling  init_hugetlbfs_fs+0x0/0x82 @ 1                               \
 [    1.783850] initcall init_hugetlbfs_fs+0x0/0x82 returned 0 after 0 usecs          \
 [    1.783859] calling  init_iso9660_fs+0x0/0x64 @ 1                                 \
 [    1.783883] initcall init_iso9660_fs+0x0/0x64 returned 0 after 0 usecs            \
 [    1.783890] calling  init_nfs_fs+0x0/0x116 @ 1                                    \
 [    1.783969] initcall init_nfs_fs+0x0/0x116 returned 0 after 0 usecs               \
 [    1.783977] calling  init_nfsd+0x0/0xca @ 1                                       \
 [    1.783982] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).              \
 [    1.784275] initcall init_nfsd+0x0/0xca returned 0 after 0 usecs                  \
 [    1.784283] calling  init_nlm+0x0/0x1c @ 1                                        \
 [    1.784291] initcall init_nlm+0x0/0x1c returned 0 after 0 usecs                   \
 [    1.784298] calling  init_cifs+0x0/0x422 @ 1                                      \
 [    1.784386] initcall init_cifs+0x0/0x422 returned 0 after 0 usecs                 \
 [    1.784394] calling  init_autofs_fs+0x0/0xf @ 1                                   \
 [    1.784401] initcall init_autofs_fs+0x0/0xf returned 0 after 0 usecs              \
 [    1.784408] calling  init_autofs4_fs+0x0/0x1e @ 1                                 \
 [    1.784415] initcall init_autofs4_fs+0x0/0x1e returned -16 after 0 usecs          \
 [    1.784423] initcall init_autofs4_fs+0x0/0x1e returned with error code -16        \
 [    1.784430] calling  ipc_init+0x0/0x20 @ 1                                        \
 [    1.784439] msgmni has been set to 1107                                           \
 [    1.784449] initcall ipc_init+0x0/0x20 returned 0 after 0 usecs                   \
 [    1.784455] calling  ipc_sysctl_init+0x0/0x11 @ 1                                 \
 [    1.784463] initcall ipc_sysctl_init+0x0/0x11 returned 0 after 0 usecs            \
 [    1.784470] calling  key_proc_init+0x0/0x2c @ 1                                   \
 [    1.784478] initcall key_proc_init+0x0/0x2c returned 0 after 0 usecs              \
 [    1.784485] calling  crypto_wq_init+0x0/0x29 @ 1                                  \
 [    1.784511] initcall crypto_wq_init+0x0/0x29 returned 0 after 0 usecs             \
 [    1.784519] calling  crypto_algapi_init+0x0/0xc @ 1                               \
 [    1.784527] initcall crypto_algapi_init+0x0/0xc returned 0 after 0 usecs          \
 [    1.784534] calling  chainiv_module_init+0x0/0xf @ 1                              \
 [    1.784541] initcall chainiv_module_init+0x0/0xf returned 0 after 0 usecs         \
 [    1.784548] calling  eseqiv_module_init+0x0/0xf @ 1                               \
 [    1.784554] initcall eseqiv_module_init+0x0/0xf returned 0 after 0 usecs          \
 [    1.784561] calling  md5_mod_init+0x0/0xf @ 1                                     \
 [    1.784613] initcall md5_mod_init+0x0/0xf returned 0 after 0 usecs                \
 [    1.784622] calling  crypto_cbc_module_init+0x0/0xf @ 1                           \
 [    1.784628] initcall crypto_cbc_module_init+0x0/0xf returned 0 after 0 usecs      \
 [    1.784636] calling  des_generic_mod_init+0x0/0x33 @ 1                            \
 [    1.784719] initcall des_generic_mod_init+0x0/0x33 returned 0 after 0 usecs       \
 [    1.784727] calling  cast5_mod_init+0x0/0xf @ 1                                   \
 [    1.784775] initcall cast5_mod_init+0x0/0xf returned 0 after 0 usecs              \
 [    1.784785] calling  krng_mod_init+0x0/0xf @ 1                                    \
 [    1.784810] alg: No test for stdrng (krng)                                        \
 [    1.784823] initcall krng_mod_init+0x0/0xf returned 0 after 0 usecs               \
 [    1.784830] calling  async_tx_init+0x0/0x11 @ 1                                   \
 [    1.784835] async_tx: api initialized (sync-only)                                 \
 [    1.784841] initcall async_tx_init+0x0/0x11 returned 0 after 0 usecs              \
 [    1.784848] calling  async_memcpy_init+0x0/0x7 @ 1                                \
 [    1.784855] initcall async_memcpy_init+0x0/0x7 returned 0 after 0 usecs           \
 [    1.784862] calling  async_xor_init+0x0/0x7 @ 1                                   \
 [    1.784868] initcall async_xor_init+0x0/0x7 returned 0 after 0 usecs              \
 [    1.784875] calling  proc_genhd_init+0x0/0x31 @ 1                                 \
 [    1.784884] initcall proc_genhd_init+0x0/0x31 returned 0 after 0 usecs            \
 [    1.784891] calling  noop_init+0x0/0x11 @ 1                                       \
 [    1.784897] io scheduler noop registered                                          \
 [    1.784902] initcall noop_init+0x0/0x11 returned 0 after 0 usecs                  \
 [    1.784909] calling  as_init+0x0/0x11 @ 1                                         \
 [    1.784914] io scheduler anticipatory registered (default)                        \
 [    1.784921] initcall as_init+0x0/0x11 returned 0 after 0 usecs                    \
 [    1.784927] calling  deadline_init+0x0/0x11 @ 1                                   \
 [    1.784933] io scheduler deadline registered                                      \
 [    1.784938] initcall deadline_init+0x0/0x11 returned 0 after 0 usecs              \
 [    1.784946] calling  cfq_init+0x0/0x89 @ 1                                        \
 [    1.784962] io scheduler cfq registered                                           \
 [    1.784968] initcall cfq_init+0x0/0x89 returned 0 after 0 usecs                   \
 [    1.784975] calling  init_kmp+0x0/0xf @ 1                                         \
 [    1.784981] initcall init_kmp+0x0/0xf returned 0 after 0 usecs                    \
 [    1.784988] calling  init_bm+0x0/0xf @ 1                                          \
 [    1.784994] initcall init_bm+0x0/0xf returned 0 after 0 usecs                     \
 [    1.785001] calling  init_fsm+0x0/0xf @ 1                                         \
 [    1.785007] initcall init_fsm+0x0/0xf returned 0 after 0 usecs                    \
 [    1.785013] calling  percpu_counter_startup+0x0/0x35 @ 1                          \
 [    1.785020] initcall percpu_counter_startup+0x0/0x35 returned 0 after 0 usecs     \
 [    1.785028] calling  xenfs_init+0x0/0x27 @ 1                                      \
 [    1.785035] initcall xenfs_init+0x0/0x27 returned 0 after 0 usecs                 \
 [    1.785043] calling  rand_initialize+0x0/0x25 @ 1                                 \
 [    1.785066] initcall rand_initialize+0x0/0x25 returned 0 after 0 usecs            \
 [    1.785074] calling  tty_init+0x0/0xd5 @ 1                                        \
 [    1.787762] initcall tty_init+0x0/0xd5 returned 0 after 0 usecs                   \
 [    1.787771] calling  pty_init+0x0/0x4ab @ 1                                       \
 [    1.789129] initcall pty_init+0x0/0x4ab returned 0 after 0 usecs                  \
 [    1.789138] calling  sysrq_init+0x0/0x1f @ 1                                      \
 [    1.789147] initcall sysrq_init+0x0/0x1f returned 0 after 0 usecs                 \
 [    1.789155] calling  xen_init+0x0/0x7f @ 1                                        \
 [    1.789542] initcall xen_init+0x0/0x7f returned 0 after 0 usecs                   \
 [    1.789552] calling  topology_sysfs_init+0x0/0x51 @ 1                             \
 [    1.789563] initcall topology_sysfs_init+0x0/0x51 returned 0 after 0 usecs        \
 [    1.789571] calling  loop_init+0x0/0x16c @ 1                                      \
 [    1.790303] loop: module loaded                                                   \
 [    1.790311] initcall loop_init+0x0/0x16c returned 0 after 10068 usecs             \
 [    1.790319] calling  xlblk_init+0x0/0x6d @ 1                                      \
 [    1.802828] initcall xlblk_init+0x0/0x6d returned 0 after 12212 usecs             \
 [    1.802841] calling  net_olddevs_init+0x0/0x82 @ 1                                \
 [    1.802851] initcall net_olddevs_init+0x0/0x82 returned 0 after 3 usecs           \
 [    1.802859] calling  netif_init+0x0/0x47 @ 1                                      \
 [    1.802863] Initialising Xen virtual ethernet driver.                             \
 [    1.803339] blkfront: xvda1: barriers enabled                                     \
 [    1.804268] blkfront: xvda2: barriers enabled                                     \
 [    1.804636] initcall netif_init+0x0/0x47 returned 0 after 1824 usecs              \
 [    1.804644] calling  i8042_init+0x0/0xfb @ 1                                      \
 [    1.805466] i8042.c: No controller found.                                         \
 [    1.805473] initcall i8042_init+0x0/0xfb returned -19 after 803 usecs             \
 [    1.805480] calling  serport_init+0x0/0x2b @ 1                                    \
 [    1.805487] initcall serport_init+0x0/0x2b returned 0 after 0 usecs               \
 [    1.805494] calling  mousedev_init+0x0/0x51 @ 1                                   \
 [    1.805569] mice: PS/2 mouse device common for all mice                           \
 [    1.805578] initcall mousedev_init+0x0/0x51 returned 0 after 76 usecs             \
 [    1.805585] calling  atkbd_init+0x0/0x20 @ 1                                      \
 [    1.805664] initcall atkbd_init+0x0/0x20 returned 0 after 70 usecs                \
 [    1.805672] calling  linear_init+0x0/0xf @ 1                                      \
 [    1.805677] md: linear personality registered for level -1                        \
 [    1.805684] initcall linear_init+0x0/0xf returned 0 after 5 usecs                 \
 [    1.805690] calling  raid0_init+0x0/0xf @ 1                                       \
 [    1.805695] md: raid0 personality registered for level 0                          \
 [    1.805701] initcall raid0_init+0x0/0xf returned 0 after 5 usecs                  \
 [    1.805707] calling  raid_init+0x0/0xf @ 1                                        \
 [    1.805712] md: raid1 personality registered for level 1                          \
 [    1.805719] initcall raid_init+0x0/0xf returned 0 after 5 usecs                   \
 [    1.805725] calling  raid_init+0x0/0xf @ 1                                        \
 [    1.805730] md: raid10 personality registered for level 10                        \
 [    1.805736] initcall raid_init+0x0/0xf returned 0 after 5 usecs                   \
 [    1.805743] calling  raid5_init+0x0/0x25 @ 1                                      \
 [    1.805748] md: raid6 personality registered for level 6                          \
 [    1.805754] md: raid5 personality registered for level 5                          \
 [    1.805759] md: raid4 personality registered for level 4                          \
 [    1.805766] initcall raid5_init+0x0/0x25 returned 0 after 16 usecs                \
 [    1.805772] calling  multipath_init+0x0/0xf @ 1                                   \
 [    1.805778] md: multipath personality registered for level -4                     \
 [    1.805785] initcall multipath_init+0x0/0xf returned 0 after 5 usecs              \
 [    1.805792] calling  raid_init+0x0/0xf @ 1                                        \
 [    1.805796] md: faulty personality registered for level -5                        \
 [    1.805803] initcall raid_init+0x0/0xf returned 0 after 5 usecs                   \
 [    1.805810] calling  dm_init+0x0/0x38 @ 1                                         \
 [    1.805842] device-mapper: uevent: version 1.0.3                                  \
 [    1.805957] device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: \
dm-devel@redhat.com                                 [    1.805968] initcall \
dm_init+0x0/0x38 returned 0 after 148 usecs                                           \
 [    1.805975] calling  dm_crypt_init+0x0/0x5a @ 1                                   \
 [    1.805990] initcall dm_crypt_init+0x0/0x5a returned 0 after 8 usecs              \
 [    1.805997] calling  dm_delay_init+0x0/0xa7 @ 1                                   \
 [    1.806042] initcall dm_delay_init+0x0/0xa7 returned 0 after 36 usecs             \
 [    1.806050] calling  dm_multipath_init+0x0/0x106 @ 1                              \
 [    1.806141] device-mapper: multipath: version 1.0.5 loaded                        \
 [    1.806149] initcall dm_multipath_init+0x0/0x106 returned 0 after 89 usecs        \
 [    1.806156] calling  dm_rr_init+0x0/0x32 @ 1                                      \
 [    1.806161] device-mapper: multipath round-robin: version 1.0.0 loaded            \
 [    1.806169] initcall dm_rr_init+0x0/0x32 returned 0 after 6 usecs                 \
 [    1.806175] calling  dm_snapshot_init+0x0/0x203 @ 1                               \
 [    1.806242] initcall dm_snapshot_init+0x0/0x203 returned 0 after 58 usecs         \
 [    1.806251] calling  dm_mirror_init+0x0/0x69 @ 1                                  \
 [    1.806268] initcall dm_mirror_init+0x0/0x69 returned 0 after 10 usecs            \
 [    1.806275] calling  dm_dirty_log_init+0x0/0x49 @ 1                               \
 [    1.806282] initcall dm_dirty_log_init+0x0/0x49 returned 0 after 0 usecs          \
 [    1.806289] calling  dm_zero_init+0x0/0x28 @ 1                                    \
 [    1.806296] initcall dm_zero_init+0x0/0x28 returned 0 after 0 usecs               \
 [    1.806302] calling  hid_init+0x0/0x26 @ 1                                        \
 [    1.806368] initcall hid_init+0x0/0x26 returned 0 after 57 usecs                  \
 [    1.806376] calling  oprofile_init+0x0/0x47 @ 1                                   \
 [    1.806381] oprofile: using timer interrupt.                                      \
 [    1.806392] initcall oprofile_init+0x0/0x47 returned 0 after 9 usecs              \
 [    1.806399] calling  llc_init+0x0/0x1b @ 1                                        \
 [    1.806406] initcall llc_init+0x0/0x1b returned 0 after 0 usecs                   \
 [    1.806413] calling  snap_init+0x0/0x31 @ 1                                       \
 [    1.806419] initcall snap_init+0x0/0x31 returned 0 after 0 usecs                  \
 [    1.806426] calling  nfnetlink_init+0x0/0x51 @ 1                                  \
 [    1.806431] Netfilter messages via NETLINK v0.30.                                 \
 [    1.806441] initcall nfnetlink_init+0x0/0x51 returned 0 after 9 usecs             \
 [    1.806448] calling  nfnetlink_queue_init+0x0/0x87 @ 1                            \
 [    1.806459] initcall nfnetlink_queue_init+0x0/0x87 returned 0 after 4 usecs       \
 [    1.806466] calling  nfnetlink_log_init+0x0/0xb3 @ 1                              \
 [    1.806479] initcall nfnetlink_log_init+0x0/0xb3 returned 0 after 6 usecs         \
 [    1.806486] calling  xt_init+0x0/0xf6 @ 1                                         \
 [    1.806492] initcall xt_init+0x0/0xf6 returned 0 after 0 usecs                    \
 [    1.806498] calling  tcpudp_mt_init+0x0/0x14 @ 1                                  \
 [    1.806505] initcall tcpudp_mt_init+0x0/0x14 returned 0 after 0 usecs             \
 [    1.806512] calling  classify_tg_init+0x0/0xf @ 1                                 \
 [    1.806518] initcall classify_tg_init+0x0/0xf returned 0 after 0 usecs            \
 [    1.806524] calling  hl_tg_init+0x0/0x14 @ 1                                      \
 [    1.806530] initcall hl_tg_init+0x0/0x14 returned 0 after 0 usecs                 \
 [    1.806536] calling  mark_tg_init+0x0/0x14 @ 1                                    \
 [    1.806542] initcall mark_tg_init+0x0/0x14 returned 0 after 0 usecs               \
 [    1.806549] calling  nfqueue_tg_init+0x0/0x14 @ 1                                 \
 [    1.806555] initcall nfqueue_tg_init+0x0/0x14 returned 0 after 0 usecs            \
 [    1.806562] calling  xt_rateest_tg_init+0x0/0x31 @ 1                              \
 [    1.806571] initcall xt_rateest_tg_init+0x0/0x31 returned 0 after 3 usecs         \
 [    1.806577] calling  comment_mt_init+0x0/0xf @ 1                                  \
 [    1.806583] initcall comment_mt_init+0x0/0xf returned 0 after 0 usecs             \
 [    1.806590] calling  dccp_mt_init+0x0/0x7f @ 1                                    \
 [    1.806596] initcall dccp_mt_init+0x0/0x7f returned 0 after 0 usecs               \
 [    1.806603] calling  dscp_mt_init+0x0/0x14 @ 1                                    \
 [    1.806609] initcall dscp_mt_init+0x0/0x14 returned 0 after 0 usecs               \
 [    1.806616] calling  esp_mt_init+0x0/0x14 @ 1                                     \
 [    1.806622] initcall esp_mt_init+0x0/0x14 returned 0 after 0 usecs                \
 [    1.806628] calling  hashlimit_mt_init+0x0/0x8d @ 1                               \
 [    1.806647] initcall hashlimit_mt_init+0x0/0x8d returned 0 after 12 usecs         \
 [    1.806655] calling  hl_mt_init+0x0/0x14 @ 1                                      \
 [    1.806660] initcall hl_mt_init+0x0/0x14 returned 0 after 0 usecs                 \
[    1.806673] initcall iprange_mt_init+0x0/0x14 returned 0 after 0 usecs             \
 [    1.806680] calling  length_mt_init+0x0/0x14 @ 1                                  \
 [    1.806686] initcall length_mt_init+0x0/0x14 returned 0 after 0 usecs             \
 [    1.806693] calling  limit_mt_init+0x0/0xf @ 1                                    \
 [    1.806698] initcall limit_mt_init+0x0/0xf returned 0 after 0 usecs               \
 [    1.806705] calling  mac_mt_init+0x0/0xf @ 1                                      \
 [    1.806711] initcall mac_mt_init+0x0/0xf returned 0 after 0 usecs                 \
 [    1.806717] calling  mark_mt_init+0x0/0x14 @ 1                                    \
 [    1.806723] initcall mark_mt_init+0x0/0x14 returned 0 after 0 usecs               \
 [    1.806730] calling  multiport_mt_init+0x0/0x14 @ 1                               \
 [    1.806736] initcall multiport_mt_init+0x0/0x14 returned 0 after 0 usecs          \
 [    1.806743] calling  owner_mt_init+0x0/0x14 @ 1                                   \
 [    1.806749] initcall owner_mt_init+0x0/0x14 returned 0 after 0 usecs              \
 [    1.806755] calling  physdev_mt_init+0x0/0xf @ 1                                  \
 [    1.806761] initcall physdev_mt_init+0x0/0xf returned 0 after 0 usecs             \
 [    1.806768] calling  pkttype_mt_init+0x0/0xf @ 1                                  \
 [    1.806774] initcall pkttype_mt_init+0x0/0xf returned 0 after 0 usecs             \
 [    1.806781] calling  quota_mt_init+0x0/0xf @ 1                                    \
 [    1.806787] initcall quota_mt_init+0x0/0xf returned 0 after 0 usecs               \
 [    1.806793] calling  xt_rateest_mt_init+0x0/0xf @ 1                               \
 [    1.806800] initcall xt_rateest_mt_init+0x0/0xf returned 0 after 0 usecs          \
 [    1.806806] calling  realm_mt_init+0x0/0xf @ 1                                    \
 [    1.806812] initcall realm_mt_init+0x0/0xf returned 0 after 0 usecs               \
 [    1.806818] calling  recent_mt_init+0x0/0x82 @ 1                                  \
 [    1.806826] initcall recent_mt_init+0x0/0x82 returned 0 after 1 usecs             \
 [    1.806833] calling  sctp_mt_init+0x0/0x14 @ 1                                    \
 [    1.806839] initcall sctp_mt_init+0x0/0x14 returned 0 after 0 usecs               \
 [    1.806845] calling  statistic_mt_init+0x0/0xf @ 1                                \
 [    1.806852] initcall statistic_mt_init+0x0/0xf returned 0 after 0 usecs           \
 [    1.806858] calling  string_mt_init+0x0/0x14 @ 1                                  \
 [    1.806864] initcall string_mt_init+0x0/0x14 returned 0 after 0 usecs             \
 [    1.806871] calling  tcpmss_mt_init+0x0/0x14 @ 1                                  \
 [    1.806877] initcall tcpmss_mt_init+0x0/0x14 returned 0 after 0 usecs             \
 [    1.806884] calling  time_mt_init+0x0/0x47 @ 1                                    \
 [    1.806889] xt_time: kernel timezone is -0000                                     \
 [    1.806895] initcall time_mt_init+0x0/0x47 returned 0 after 5 usecs               \
 [    1.806902] calling  u32_mt_init+0x0/0xf @ 1                                      \
 [    1.806907] initcall u32_mt_init+0x0/0xf returned 0 after 0 usecs                 \
 [    1.806914] calling  sysctl_ipv4_init+0x0/0x3f @ 1                                \
 [    1.806926] initcall sysctl_ipv4_init+0x0/0x3f returned 0 after 5 usecs           \
 [    1.806934] calling  ipv4_netfilter_init+0x0/0xf @ 1                              \
 [    1.806941] initcall ipv4_netfilter_init+0x0/0xf returned 0 after 0 usecs         \
 [    1.806949] calling  ip_tables_init+0x0/0x92 @ 1                                  \
 [    1.806961] ip_tables: (C) 2000-2006 Netfilter Core Team                          \
 [    1.806968] initcall ip_tables_init+0x0/0x92 returned 0 after 13 usecs            \
 [    1.806975] calling  iptable_filter_init+0x0/0x5d @ 1                             \
 [    1.806987] initcall iptable_filter_init+0x0/0x5d returned 0 after 5 usecs        \
 [    1.806994] calling  iptable_mangle_init+0x0/0x38 @ 1                             \
 [    1.807003] initcall iptable_mangle_init+0x0/0x38 returned 0 after 2 usecs        \
 [    1.807010] calling  iptable_raw_init+0x0/0x38 @ 1                                \
 [    1.807019] initcall iptable_raw_init+0x0/0x38 returned 0 after 2 usecs           \
 [    1.807026] calling  addrtype_mt_init+0x0/0x14 @ 1                                \
 [    1.807033] initcall addrtype_mt_init+0x0/0x14 returned 0 after 0 usecs           \
 [    1.807039] calling  ah_mt_init+0x0/0xf @ 1                                       \
 [    1.807045] initcall ah_mt_init+0x0/0xf returned 0 after 0 usecs                  \
 [    1.807052] calling  ecn_mt_init+0x0/0xf @ 1                                      \
 [    1.807057] initcall ecn_mt_init+0x0/0xf returned 0 after 0 usecs                 \
 [    1.807064] calling  ecn_tg_init+0x0/0xf @ 1                                      \
 [    1.807070] initcall ecn_tg_init+0x0/0xf returned 0 after 0 usecs                 \
 [    1.807077] calling  log_tg_init+0x0/0x24 @ 1                                     \
 [    1.807083] initcall log_tg_init+0x0/0x24 returned 0 after 0 usecs                \
 [    1.807090] calling  reject_tg_init+0x0/0xf @ 1                                   \
 [    1.807096] initcall reject_tg_init+0x0/0xf returned 0 after 0 usecs              \
 [    1.807103] calling  arp_tables_init+0x0/0x78 @ 1                                 \
 [    1.807113] arp_tables: (C) 2002 David S. Miller                                  \
 [    1.807119] initcall arp_tables_init+0x0/0x78 returned 0 after 10 usecs           \
 [    1.807126] calling  arpt_mangle_init+0x0/0xf @ 1                                 \
 [    1.807132] initcall arpt_mangle_init+0x0/0xf returned 0 after 0 usecs            \
 [    1.807139] calling  arptable_filter_init+0x0/0x38 @ 1                            \
 [    1.807150] initcall arptable_filter_init+0x0/0x38 returned 0 after 3 usecs       \
 [    1.807157] calling  cubictcp_register+0x0/0x7f @ 1                               \
 [    1.807163] TCP cubic registered                                                  \
 [    1.807168] initcall cubictcp_register+0x0/0x7f returned 0 after 5 usecs          \
 [    1.807175] calling  packet_init+0x0/0x39 @ 1                                     \
 [    1.807181] NET: Registered protocol family 17                                    \
 [    1.807190] initcall packet_init+0x0/0x39 returned 0 after 8 usecs                \
 [    1.807197] calling  br_init+0x0/0xbe @ 1                                         \
 [    1.807217] Bridge firewalling registered                                         \
 [    1.807225] initcall br_init+0x0/0xbe returned 0 after 21 usecs                   \
 [    1.807232] calling  init_sunrpc+0x0/0x53 @ 1                                     \
 [    1.807322] RPC: Registered udp transport module.                                 \
 [    1.807331] RPC: Registered tcp transport module.                                 \
 [    1.807338] initcall init_sunrpc+0x0/0x53 returned 0 after 97 usecs               \
 [    1.807345] calling  init_rpcsec_gss+0x0/0x3f @ 1                                 \
 [    1.807360] initcall init_rpcsec_gss+0x0/0x3f returned 0 after 8 usecs            \
 [    1.807367] calling  init_kerberos_module+0x0/0x26 @ 1                            \
 [    1.807377] initcall init_kerberos_module+0x0/0x26 returned 0 after 3 usecs       \
 [    1.807384] calling  init_spkm3_module+0x0/0x26 @ 1                               \
 [    1.807392] initcall init_spkm3_module+0x0/0x26 returned 0 after 1 usecs          \
 [    1.807399] calling  update_mp_table+0x0/0x59b @ 1                                \
 [    1.807406] initcall update_mp_table+0x0/0x59b returned 0 after 0 usecs           \
 [    1.807412] calling  lapic_insert_resource+0x0/0x45 @ 1                           \
 [    1.807420] initcall lapic_insert_resource+0x0/0x45 returned 0 after 0 usecs      \
 [    1.807426] calling  print_ipi_mode+0x0/0x26 @ 1                                  \
 [    1.807432] Using IPI No-Shortcut mode                                            \
 [    1.807437] initcall print_ipi_mode+0x0/0x26 returned 0 after 4 usecs             \
 [    1.807444] calling  ioapic_insert_resources+0x0/0x4b @ 1                         \
 [    1.807450] initcall ioapic_insert_resources+0x0/0x4b returned 0 after 0 usecs    \
 [    1.807458] calling  io_apic_bug_finalize+0x0/0x1a @ 1                            \
 [    1.807465] initcall io_apic_bug_finalize+0x0/0x1a returned 0 after 0 usecs       \
 [    1.807472] calling  check_early_ioremap_leak+0x0/0x5e @ 1                        \
 [    1.807479] initcall check_early_ioremap_leak+0x0/0x5e returned 0 after 0 usecs   \
 [    1.807487] calling  init_oops_id+0x0/0x3a @ 1                                    \
 [    1.807497] initcall init_oops_id+0x0/0x3a returned 0 after 3 usecs               \
 [    1.807504] calling  disable_boot_consoles+0x0/0x36 @ 1                           \
 [    1.807511] initcall disable_boot_consoles+0x0/0x36 returned 0 after 0 usecs      \
 [    1.807519] calling  pm_qos_power_init+0x0/0xae @ 1                               \
 [    1.807715] initcall pm_qos_power_init+0x0/0xae returned 0 after 185 usecs        \
 [    1.807724] calling  clear_boot_tracer+0x0/0x27 @ 1                               \
 [    1.807730] initcall clear_boot_tracer+0x0/0x27 returned 0 after 0 usecs          \
 [    1.807738] calling  max_swapfiles_check+0x0/0x7 @ 1                              \
 [    1.807745] initcall max_swapfiles_check+0x0/0x7 returned 0 after 0 usecs         \
 [    1.807752] calling  random32_reseed+0x0/0x83 @ 1                                 \
 [    1.807764] initcall random32_reseed+0x0/0x83 returned 0 after 5 usecs            \
 [    1.807771] calling  boot_wait_for_devices+0x0/0x18 @ 1                           \
 [   11.900071] XENBUS: Timeout connecting to device: device/vbd/51729 (state 3)      \
 [   11.900084] XENBUS: Timeout connecting to device: device/vbd/51745 (state 3)      \
 [   11.900091] XENBUS: Timeout connecting to device: device/vif/0 (state 1)          \
 [   11.900103] initcall boot_wait_for_devices+0x0/0x18 returned 0 after 9855678 \
usecs                                           [   11.900112] calling  \
seqgen_init+0x0/0xe @ 1                                                               \
 [   11.900131] initcall seqgen_init+0x0/0xe returned 0 after 13 usecs                \
 [   11.900138] calling  memmap_init+0x0/0x89 @ 1                                     \
 [   11.900171] initcall memmap_init+0x0/0x89 returned 0 after 25 usecs               \
 [   11.900178] calling  tcp_congestion_default+0x0/0xf @ 1                           \
 [   11.900186] initcall tcp_congestion_default+0x0/0xf returned 0 after 1 usecs      \
 [   11.900193] calling  ip_auto_config+0x0/0x633 @ 1                                 \
 [   11.900205] initcall ip_auto_config+0x0/0x633 returned 0 after 6 usecs            \
 [   11.900212] calling  initialize_hashrnd+0x0/0x16 @ 1                              \
 [   11.900222] initcall initialize_hashrnd+0x0/0x16 returned 0 after 3 usecs         \
 [   11.900238] async_waiting @ 1                                                     \
 [   11.900242] async_continuing @ 1 after 0 usec                                     \
 [   11.900261] md: Waiting for all devices to be available before autodetect         \
 [   11.900266] md: If you don't use raid, use raid=noautodetect                      \
 [   11.900272] async_waiting @ 1                                                     \
 [   11.900276] async_continuing @ 1 after 0 usec                                     \
 [   11.900446] md: Autodetecting RAID arrays.                                        \
 [   11.900452] md: Scanned 0 and added 0 devices.                                    \
 [   11.900457] md: autorun ...                                                       \
 [   11.900460] md: ... autorun DONE.                                                 \
 [   11.904602] VFS: Mounted root (squashfs filesystem) readonly on device 202:1.     \
 [   11.904624] async_waiting @ 1                                                     \
 [   11.904629] async_continuing @ 1 after 0 usec                                     \
 [   11.904648] Freeing unused kernel memory: 244k freed                              \




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel


[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic