[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensolaris-dtrace-discuss
Subject:    Re: [dtrace-discuss] dtrace and kmem heap corruption ?
From:       Jonathan Adams <jonathan.adams () oracle ! com>
Date:       2011-02-18 23:42:13
Message-ID: 20110218234213.GB615160 () eng ! sun ! com
[Download RAW message or body]

On Fri, Feb 18, 2011 at 05:51:35PM -0500, Peter Shoults wrote:
> Hi,
> 
> I wrote a dtrace script that is tracking a couple of routines that
> decrement a reference count, trying to catch the thread that
> de-references the count down to -1.  The script works fine, except for
> some reason, I suspect it is not recording the last few decrements. 
> Probably best to show what I mean:
> 
> from msgbuf of corefile:
> 
> kernel memory allocator: buffer modified after being freed
> modification occurred at offset 0x0 (0xdeadbeefdeadbeef replaced by
> 0xdeadbeeedeadbeef)
> buffer=3004ec26af8  bufctl=3004ec65d20  cache: kmem_alloc_56
                             ^^^^^^^^^^^

Have you done:

	3004ec65d20::bufctl -v

and printed the "CONTENTS" field as a credgrp_t?

> Now, grepping for that bufaddr on my dtrace script output...
> 
> 
>   1  20287  crgrprele:entry cred_ptr is 3004ec26af8, cred ref = 5
>   1  20287  crgrprele:entry cred_ptr is 3004ec26af8, cred ref = 5
>   1  20287  crgrprele:entry cred_ptr is 3004ec26af8, cred ref = 5
>   1  20287  crgrprele:entry cred_ptr is 3004ec26af8, cred ref = 4
>   1  20287  crgrprele:entry cred_ptr is 3004ec26af8, cred ref = 3
> #
> 
> I have to believe we decremented down to zero and freed, then again,
> maybe we did not decrement down to zero and still freed, and then when
> buffer was on free list, we tried to decrement thinking we still had to
> and we were still active.  Either way - what can I do to catch what I
> believe must have been at least one more decrement - it is seems as if
> the decrement occurred, and we went belly up so quickly dtrace never had
> a chance to record the decrement.
> 
> Anyway - if there is a way to boost what dtrace can capture, or ensure
> it actually records when an event is triggered...that would be nice.

There are DCMDs in mdb to dump the in-kernel dtrace state:

	::dtrace_state

lists the active dtrace consumers, and

	addr::dtrace

will process the in-kernel state for a particular consumer.

Cheers,
- jonathan
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
[prev in list] [next in list] [prev in thread] [next in thread]