[prev in list] [next in list] [prev in thread] [next in thread] 

List:       opensolaris-dtrace-discuss
Subject:    Re: [dtrace-discuss] Enabling anonymous probes to fire immediately
From:       John Rice <John.Rice () Sun ! COM>
Date:       2006-12-19 17:58:02
Message-ID: 4588282A.8060704 () sun ! com
[Download RAW message or body]

Thanks Sean for all the suggestions.

Darren and I got it sorted and got the stack traces :)

Turns out to be a race condition between the fc-cache setting up the 
font cash and any Gnome app accessing the fonts before the cache was 
ready. The fix is just to introduce a dependency on the fc-cache in 
DTlogin's SMF manifest file.

We did the following:
Created a simplified appcrash.d script [below]
Ran on my own nevada b54 machine:
dtrace -AFs appcrash.d
saved:
/kernel/drv/dtrace.conf
/etc/system

Got the test system to a point where we had just finished install.
Log'ed into Failsafe mode - mounted local file system to /a
Mounted usb stick with the above files on it:
Copied over the files to the mounted /a drive
/a/kernel/drv/dtrace.conf
/a/etc/system

Modified the Gnome settings to stop it intercepting the segv on reboot:
Edit /a/usr/dt/config/Xinitrc.jds and add:
export GNOME_DISABLE_CRASH_DIALOG=1

Reboot.

Gnome-session crashed and the logon hangs. Just killed the X Server to 
get back to the login prompt, logged in and got the dtrace anonymous 
ouput. Odd thing here is that the system("prun ... didn't work for us to 
cleanly close down gnome-session. It was stopped when we logged in. No 
big deal as we still got the stack data - is this a difference when 
running anonymous scripts I wonder? Didn't see this effect when running 
the scripts on my own machine.

dtrace -ae -o dtrace.log

Happy days as they say.

JR


appcrash.d
-------------
#!/usr/sbin/dtrace -qws
#pragma D option bufpolicy=ring
#pragma D option bufsize=4m

proc:::signal-send
/(args[2] == SIGBUS || args[2] == SIGSEGV) &&
pid == args[1]->pr_pid/
{
@[execname, pid, ustack(100,1000)] = count();

stop();
printf("\n=================\n");
printa(@);
trunc(@);
printf("=================\n");
system("prun %d", pid); <--- Didn't seem to be fired ??
}

Log:
-------
0 | sigtoproc:signal-send
=================

gnome-session 3215
libfontconfig.so.1`FcPatternPosition+0x6b
libfontconfig.so.1`FcPatternFindElt+0x2e
libfontconfig.so.1`FcPatternGet+0x2e
libfontconfig.so.1`FcFontSetSort+0x498
libfontconfig.so.1`FcFontSort+0xaa
libpangoft2-1.0.so.0.1400.7`pango_fc_font_map_get_patterns+0x1ba
libpangoft2-1.0.so.0.1400.7`pango_fc_font_map_load_fontset+0x36


Sean McGrath - Sun Microsystems Ireland wrote:
> John Rice stated:
> < Hi,
> < 
> < We have a tricky intermittent gnome-session crash we need to debug in 
> < Solaris nevada b55, that we are only seeing after an initial install 
> < from DVD media and the first logon by root. Not on subsequent logons by 
> < root or any other created user.
> < 
> < What I'd like to do is use the app_crash.d script in the article below 
> < to catch the sigsegv signal from gnome-session and dump out a pstack. 
> < However to setup anonymous tracing I seem to need to login first, enable 
> < it for a given script and then reboot. This will be too late unfortunately.
> < 
> < Would it be possible to just replace the config and system files created 
> < by running "dtrace -A app_crash.d" in the install image, to have 
> < anonymous tracing for these probes enabled on install and first login by 
> < root? Has anyone done this before?
>
>   You can't use dtrace directly from an install image as it isn't there :)
>   Theres an old thread in either the internal dtrace archives or the 
>   opensolaris one where there's talk of how to get it in though, adding
>   the dtrace packages to the install boot image.  Though not necessarily
>   needed here.
>
>   You should be able to use take an already generated dtrace.conf and
>   /etc/system settings to get what you want.  You'll just need to copy
>   them down the /a root dir on the install partition before you reboot
>   after install.
>
> < # dtrace -AFs app_crash.d
>
>   dtrace -o d.out -AFs app_crash.d
>
>   The d.out would then  be appended to the /a/kernel/drv/dtrace.conf
>   Similarly for the /etc/system settings.
>
>   How you'd do this from install off a DVD ?  Perhaps have the d.out and
>   the generated /etc/system files on a perserved partition or before reboot
>   plumb a nic etc (unless already plumbed :) and get it from another machine.
>
> < dtrace: saved anonymous enabling in /kernel/drv/dtrace.conf
> < dtrace: added forceload directives to /etc/system
> < dtrace: run update_drv(1M) or reboot to enable changes
> < #
> < 
> < Here is the article describing the app_crash.d script:
> < http://developers.sun.com/solaris/articles/app_crash/app_crash.html#DTsolution 
> < 
> < DTrace script: 
> < http://developers.sun.com/solaris/articles/app_crash/app_crash.d.txt
> < User Script: 
> < http://developers.sun.com/solaris/articles/app_crash/runme_on_app_crash.txt
>
>  Interesting script to try and run just after an install :)
>
>  You've tried this the usual way already and it works ?  i.e. not from
>  an install, but the usual 'dtrace -AFs app_crash.d' and reboot ?
>
> < 
> < JR
> < 
> < _______________________________________________
> < dtrace-discuss mailing list
> < dtrace-discuss@opensolaris.org
>
>
> Regards,
>   

_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
[prev in list] [next in list] [prev in thread] [next in thread]