[prev in list] [next in list] [prev in thread] [next in thread]
List: dpdk-users
Subject: [dpdk-users] Using DPDK for contiguous physical memory allocation
From: alain () edicogenome ! com (Alain Gautherot)
Date: 2016-01-25 22:51:01
Message-ID: BY2PR07MB0576FBFD8C466E68EEF8E8BB1C70 () BY2PR07MB057 ! namprd07 ! prod ! outlook ! com
[Download RAW message or body]
Sergio,
Yes, I made that change and am now using the following:
size_t i;
for (i = 1; i <= 200; ++i) {
size_t allocsize = (i << 30) / 10U;
printf(" Allocating %3.1fGB: ", ((float )i)/10.0f);
fflush(stdout);
void* ptr = rte_malloc(NULL, allocsize, 0U);
if (ptr != NULL) {
printf("PASS\n");
rte_free(ptr);
} else {
printf("fail\n");
}
}
printf("Done\n");
That seems to be running fine now with DPDK 2.2.0 (was not with 2.0.0).
Thanks,
Alain
-----Original Message-----
From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.monroy at intel.com]
Sent: Monday, January 25, 2016 2:27 PM
To: Alain Gautherot <alain at edicogenome.com>
Cc: users at dpdk.org
Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation
Hi Alain,
On 25/01/2016 21:02, Alain Gautherot wrote:
> [resend with enclosed log instead of attachment]
>
> Hello Sergio,
>
> I'm running the following command
>
> $ ./build/helloworld -c fff -n 1
>
> And get the attached log (hope it goes through). Using "-n 2" (I'm not sure how \
> many channels) gives the same SIGSEGV error.
> Here's the configuration:
>
> $ numactl -H
> available: 1 nodes (0)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
> node 0 size: 65431 MB
> node 0 free: 62040 MB
> node distances:
> node 0
> 0: 10
>
> $ cat /proc/meminfo
> MemTotal: 65867360 kB
> MemFree: 63529276 kB
> Buffers: 93996 kB
> Cached: 562160 kB
> SwapCached: 0 kB
> Active: 314816 kB
> Inactive: 483752 kB
> Active(anon): 144372 kB
> Inactive(anon): 28 kB
> Active(file): 170444 kB
> Inactive(file): 483724 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> SwapTotal: 0 kB
> SwapFree: 0 kB
> Dirty: 12 kB
> Writeback: 0 kB
> AnonPages: 144184 kB
> Mapped: 49004 kB
> Shmem: 280 kB
> Slab: 77572 kB
> SReclaimable: 31580 kB
> SUnreclaim: 45992 kB
> KernelStack: 2904 kB
> PageTables: 7744 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 32421680 kB
> Committed_AS: 383316 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed: 378992 kB
> VmallocChunk: 34359352736 kB
> HardwareCorrupted: 0 kB
> AnonHugePages: 73728 kB
> HugePa ges_Total: 500
> HugePages_Free: 9
> HugePages_Rsvd: 9
> HugePages_Surp: 0
> Hugepagesize: 2048 kB
> DirectMap4k: 4096 kB
> DirectMap2M: 2027520 kB
> DirectMap1G: 65011712 kB
>
>
> Log:
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 0 on socket 0
> EAL: Detected lcore 7 as core 1 on socket 0
> EAL: Detected lcore 8 as core 2 on socket 0
> EAL: Detected lcore 9 as core 3 on socket 0
> EAL: Detected lcore 10 as core 4 on socket 0
> EAL: Detected lcore 11 as core 5 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 12 lcore(s)
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7fd26c800000 (size = 0x200000)
> EAL: Ask a virtual area of 0x35800000 bytes
> EAL: Virtual area found at 0x7fd236e00000 (size = 0x35800000)
> EAL: Requesting 429 pages of size 2MB from socket 0
> EAL: TSC frequency is ~2400001 KHz
> EAL: Master lcore 0 is ready (tid=6cd40880;cpuset=[0])
> PMD: ENICPMD trace: rte_enic_pmd_init
> EAL: lcore 6 is ready (tid=331f7700;cpuset=[6])
> EAL: lcore 5 is ready (tid=33bf8700;cpuset=[5])
> EAL: lcore 9 is ready (tid=313f4700;cpuset=[9])
> EAL: lcore 11 is ready (tid=2fff2700;cpuset=[11])
> EAL: lcore 4 is ready (tid=345f9700;cpuset=[4])
> EAL: lcore 8 is ready (tid=31df5700;cpuset=[8])
> EAL: lcore 1 is ready (tid=363fc700;cpuset=[1])
> EAL: lcore 10 is ready (tid=309f3700;cpuset=[10])
> EAL: lcore 3 is ready (tid=34ffa700;cpuset=[3])
> EAL: lcore 2 is ready (tid=359fb700;cpuset=[2])
> EAL: lcore 7 is ready (tid=327f6700;cpuset=[7])
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:03:00.0 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:03:00.1 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: Not managed by a supported kernel driver, skipped
> Allocating 0.1GB: PASS
> Allocating 0.2GB: PASS
> Allocating 0.3GB: PASS
> Allocating 0.4GB: fail
> Allocating 0.5GB: fail
> Allocating 0.6GB: fail
> Allocating 0.7GB: fail
> Allocating 0.8GB: fail
> Allocating 0.9GB: fail
> Allocating 1.0GB: fail
> Allocating 1.1GB: fail
> Allocating 1.2GB: fail
> Allocating 1.3GB: fail
> Allocating 1.4GB: fail
> Allocating 1.5GB: fail
> Allocating 1.6GB: fail
> Allocating 1.7GB: fail
> Allocating 1.8GB: fail
> Allocating 1.9GB: fail
> Allocating 2.0GB: fail
> Allocating 2.1GB: fail
> Allocating 2.2GB:
>
> Thanks,
> Alain
>
> -----Original Message-----
> From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.monroy at intel.com]
> Sent: Monday, January 25, 2016 5:50 AM
> To: Alain Gautherot <alain at edicogenome.com>; users at dpdk.org
> Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory
> allocation
>
> On 23/01/2016 00:20, Alain Gautherot wrote:
> > Hello,
> >
> > I came across DPDK in a thread @ \
> > http://stackoverflow.com/questions/4401912/linux-contiguous-physical-memory-from-userspace \
> > (bottom reply from mrsmith) and wanted to see if I can use rte_malloc() to \
> > allocate large blocks of contiguous physical memory (16GB or even 32GB at some \
> > point).
> > The platform I'm working on has an FPGA that shares host memory with the x86_64 \
> > cores via a QPI link. The FPGA crunches data directly from host memory and uses \
> > physical addresses (mostly a QPI limitation, but it is also dictated by \
> > performance considerations and the ability to make the best possible use of \
> > multiple memory controllers). The data shared is 16GB or up to 32GB and could be \
> > provided as multiple descriptors to the FPGA, but that still means that each \
> > descriptor is in the order of several GBytes each. I understand that allocation \
> > may fail, but is ok for now, since I'm still in the proof-of-concept stage, \
> > trying to rule things out.
> > My sample application attempts to allocate memory by chunks of 100MB like so:
> >
> > int main(int argc, char **argv)
> > {
> > int ret;
> >
> > ret = rte_eal_init(argc, argv);
> > if (ret < 0) {
> > rte_panic("Cannot init EAL\n");
> > }
> >
> > int i;
I get warning with this code. It warns of undefined behavior because of signed \
integer overflow. Could you change the above 'int i' to 'size_t i' and run it again?
Sergio
> > for (i = 1; i <= 100; ++i) {
> > size_t allocsize = i * 100*1000*1000;
> >
> > printf(" Allocating %3.1fGB: ", ((float )i)/10.0f);
> > fflush(stdout);
> > void* ptr = rte_malloc(NULL, allocsize, 0U);
> > if (ptr != NULL) {
> > printf("PASS\n");
> > rte_free(ptr);
> > } else {
> > printf("fail\n");
> > }
> > }
> >
> > printf("Done\n");
> > return 0;
> > }
> >
> > I get a consistent crash @ the 2.2GB mark:
> > (gdb) r -c f -n 4
> > ...
> > EAL: PCI device 0000:06:00.1 on NUMA socket 0
> > EAL: probe driver: 8086:1521 rte_igb_pmd
> > EAL: Not managed by a supported kernel driver, skipped
> > Allocating 0.1GB: fail
> > Allocating 0.2GB: fail
> > ...
> > Allocating 2.0GB: fail
> > Allocating 2.1GB: fail
> > Allocating 2.2GB:
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880, heap=0x7ffff7fe561c, \
> > mz=0x7ffff7fb2c1c, size=2200000064) at \
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61 61 \
> > elem->heap = heap; Missing separate debuginfos, use: debuginfo-install
> > glibc-2.12-1.149.el6_6.5.x86_64
> > (gdb) bt
> > ...
> > #0 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880, \
> > heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
> > #1 0x00000000004c694e in split_elem (elem=0x7ffff3e00000,
> > split_pt=0x800070eaa880) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:121
> > #2 0x00000000004c6bda in malloc_elem_alloc (elem=0x7ffff3e00000, \
> > size=18446744071614584320, align=64) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:223
> > #3 0x00000000004c736e in malloc_heap_alloc (heap=0x7ffff7fe561c, type=0x0, \
> > size=18446744071614584320, align=64) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_heap.c:167
> > #4 0x00000000004c0aa1 in rte_malloc_socket (type=0x0, size=18446744071614584320, \
> > align=0, socket_arg=-1) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:89
> > #5 0x00000000004c0b5b in rte_malloc (type=0x0,
> > size=18446744071614584320, align=0) at
> > /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:115
> > #6 0x000000000041ca6e in main (argc=5, argv=0x7fffffffdd48) at
> > /home/alaing/INTEL/dpdk-2.0.0/examples/hugephymem/main.c:66
> >
> >
> > Has anybody seen such an issue?
> > Could I be misusing RTE somehow?
> >
> What options are you running your DPDK app with?
>
> Can you also provide the full initialization log and hugepage info?
>
> Sergio
> > Thanks for your time,
> > Alain
> >
> >
> > --
> > Alain Gautherot
> > Edico Genome
> >
> -------------- next part -------------- An embedded and
> charset-unspecified text was scrubbed...
> Name: dpdk_log.txt
> URL:
> <http://dpdk.org/ml/archives/users/attachments/20160125/3fa0b05c/attac
> hment.txt>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic