[prev in list] [next in list] [prev in thread] [next in thread] 

List:       mjpeg-developer
Subject:    Re: [Mjpeg-developer] Memory leaks in y4mdenoise with 64-bit Linux
From:       Bernhard Praschinger <shadowlord () utanet ! at>
Date:       2010-10-24 4:48:37
Message-ID: 4CC3BAA5.2070408 () utanet ! at
[Download RAW message or body]

Hallo

Sorry for the long silence.

> I've been digging through y4mdenoise's code today, and so far, I haven't
> found any obvious reason why it should be running out of memory. I ran
> it with a fake 720x576 progressive PAL image stream, generated from a
> 720x480 progressive NTSC-sized image stream, but nothing is going wrong.
>
> I have assertions that make sure dynamically-allocated memory is cleared
> out regularly, so the only possibility I have left is that there's
> something wrong with my memory allocation itself. Luckily, all memory
> allocation is done in Allocator classes.
>
> Since this problem started with y4mdenoise version 3, the first thing I
> would like you to do is to undefine SET_REGION_IMPLEMENTED_WITH_VECTOR
> and BORDEREXTENTBOUNDARYSET_IMPLEMENTED_WITH_VECTOR in MotionSearcher.hh
> -- that should factor out whether it's some weird issue with the
> VariableSizeAllocator class. (Do this with the latest code, not version 3.)
With that I did always get the message that it was not able to denoise 
frame No 1:
> lav2yuv eddi.eli |
/home/bernhard/download/cvs/mjpeg_play/y4mdenoise/y4mdenoise -p 0 | yuvplay
    INFO: [lav2yuv] chroma '422' recommended with this input
    INFO: [lav2yuv] set default chroma '420jpeg'
Frame 1: 0.0%+0.0%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+100.0% new
Frame 1: 0.0%+0.0%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+100.0% new
**ERROR: [y4mdenoise] Could not denoise frame 1
    INFO: [yuvplay] Played 0000 frames (0:00:00.00)

I got that at some other circumstances too. Is it possible to get more 
information what was wrong ?

> In the letter you sent back on September 6, I noticed that the number of
> leaked bytes, divided by the number of leaked blocks, is ~256KB.
> Multiple allocators use that size, i.e. 262144 bytes. So if the above
> hack doesn't magically make the problem go away, please modify
> SearchBorder.hh so that m_oMovedRegionSetAllocator is initialized with
> 393216 (i.e. 384KB) and m_oMovedRegionAllocator is initialized with
> 524288 (i.e. 512KB), and modify SearchWindow.hh so that m_oPSBNAllocator
> is initialized with 786432 (i.e. 768KB). Then tell me what the leak
> summary says. Hopefully that'll narrow down which allocator is keeping
> memory after it's supposed to be purged.
That didn't help. For some strange reason it always stopped working 
after frame no 5. And it did not depend on the -f option (it seemed to 
me that way)

> So far, I suspect that I'm doing something wrong that doesn't manifest
> under 32-bit Linux, but manifests under 64-bit Linux. I'm doing some
> strange pointer-arithmetic in my allocator classes in order to divide a
> single system-allocated block into pieces, but as far as I can tell,
> none of it should break just because the pointer/word size has changed
> from 32 to 64 bits.
I have tested it  on my 32bit Linux (also OpenSuse 11.3 and in a very 
old Opensuse 11.1 32Bit)

And it has a similar weird behaviour :(
According to top the Virtual mem is after 20 frames about 600MB and 
after 45Frames 800MB, 70 Frames 1,1GB
I used that command: lav2yuv file.avi | \
y4mdenoise -r 4 -f 2 -v 2 | yuvplay

When I wanted to use larger values for -r >4 or -f >4 y4mdenosie did 
crash more likely on both systems. But not in a way I can reproduce, it 
happens sometime after a few frames and at other time is can denoise 
300-500 frames. One thing I have noticed on the suse 11.1 is that it did 
alway say: [y4mdenoise] Could not denoise frame 1
when I gave the virtual machine had only 768MB Ram.

It seems that the 64Bit system is able to handle -r 8 or -f 6 also reliable.
( gcc (SUSE Linux) 4.5.0 20100604 [gcc-4_5-branch revision 160292
uname -a output: 2.6.34.7-0.3-desktop #1 SMP PREEMPT 2010-09-20 15:27:38 
+0200 x86_64 x86_64 x86_64 GNU/Linux)

> Thanks for helping me track this down! Sorry for the hassle.
No problem.

I'm not sure if the problem is in y4mdenoise.

I have a strange problem with yuv2lav. (I did some tests for a RC)
and it seem that the two chroma planes are not handeld correct. It seems 
to me that they are put together into one frame. Where the left pat of 
the picture is red an the right part of the picture is green, and the 
whole picture is B/W :(

I did a simple: lav2yuv -f 300 file.avi | yuv2lav -f a -o test.avi

I have the feeling that the Suse gcc 4.5.0 is doing here doing some 
things in a very own way.

auf hoffentlich bald,

Berni the Chaos of Woodquarter

Email: shadowlord@utanet.at
www: http://www.lysator.liu.se/~gz/bernhard

------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in  U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store 
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Mjpeg-developer mailing list
Mjpeg-developer@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mjpeg-developer
[prev in list] [next in list] [prev in thread] [next in thread]