[prev in list] [next in list] [prev in thread] [next in thread] 

List:       freebsd-hackers
Subject:    Re: How do I debug this?
From:       Konstantin Belousov <kostikbel () gmail ! com>
Date:       2016-08-22 16:14:37
Message-ID: 20160822161437.GC83214 () kib ! kiev ! ua
[Download RAW message or body]

On Mon, Aug 22, 2016 at 04:54:22PM +0100, Matthew Seaman wrote:
> 
> We've got an intermittent crash bug with postgresql92-server-9.2.17_2
> which seems to be occurring in process startup:
> 
> # gdb -c /usr/local/pgsql/data/postgres.core /usr/local/bin/postgres
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
> Core was generated by `postgres'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /lib/libthr.so.3...done.
> Loaded symbols for /lib/libthr.so.3
> Reading symbols from /usr/local/lib/libxml2.so.2...done.
> Loaded symbols for /usr/local/lib/libxml2.so.2
> Reading symbols from /usr/local/lib/libicuuc.so.55...done.
> Loaded symbols for /usr/local/lib/libicuuc.so.55
> Reading symbols from /usr/local/lib/libicui18n.so.55...done.
> Loaded symbols for /usr/local/lib/libicui18n.so.55
> Reading symbols from /usr/local/lib/libssl.so.8...done.
> Loaded symbols for /usr/local/lib/libssl.so.8
> Reading symbols from /usr/local/lib/libcrypto.so.8...done.
> Loaded symbols for /usr/local/lib/libcrypto.so.8
> Reading symbols from /lib/libm.so.5...done.
> Loaded symbols for /lib/libm.so.5
> Reading symbols from /lib/libc.so.7...done.
> Loaded symbols for /lib/libc.so.7
> Reading symbols from /lib/libz.so.6...done.
> Loaded symbols for /lib/libz.so.6
> Reading symbols from /usr/lib/liblzma.so.5...done.
> Loaded symbols for /usr/lib/liblzma.so.5
> Reading symbols from /usr/local/lib/libicudata.so.55...done.
> Loaded symbols for /usr/local/lib/libicudata.so.55
> Reading symbols from /usr/lib/libc++.so.1...done.
> Loaded symbols for /usr/lib/libc++.so.1
> Reading symbols from /lib/libcxxrt.so.1...done.
> Loaded symbols for /lib/libcxxrt.so.1
> Reading symbols from /lib/libgcc_s.so.1...done.
> Loaded symbols for /lib/libgcc_s.so.1
> Reading symbols from /usr/local/lib/postgresql/auto_explain.so...done.
> Loaded symbols for /usr/local/lib/postgresql/auto_explain.so
> Reading symbols from /usr/local/lib/postgresql/plpgsql.so...done.
> Loaded symbols for /usr/local/lib/postgresql/plpgsql.so
> Reading symbols from /libexec/ld-elf.so.1...done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0  0x0000000800d1cd0a in _r_debug_postinit () from /libexec/ld-elf.so.1
> [New Thread 805806400 (LWP 100362/)]
> (gdb) bt full
> #0  0x0000000800d1cd0a in _r_debug_postinit () from /libexec/ld-elf.so.1
> No symbol table info available.
> #1  0x0000000800d1a46d in .text () from /libexec/ld-elf.so.1
> No symbol table info available.
> #2  0x0000000804ffe174 in ?? () from /lib/libgcc_s.so.1
> No symbol table info available.
> #3  0x00007fffffffe410 in ?? ()
> No symbol table info available.
> #4  0x00000008050078f1 in ?? () from /lib/libgcc_s.so.1
> No symbol table info available.
> #5  0x00007fffffffe410 in ?? ()
> No symbol table info available.
> #6  0x0000000800d1d9a8 in dlclose () from /libexec/ld-elf.so.1
> No symbol table info available.
> Previous frame inner to this frame (corrupt stack?)
> 
> Application is:
> 
> postgresql92-server-9.2.17_2
> 
> On:
> 
> % freebsd-version -ku
> 10.3-RELEASE-p4
> 10.3-RELEASE-p5
> 
> This is apparently a postgres child process crashing on startup
> somewhere before where main() gets called.  Doesn't happen often -- only
> when the server is reasonably busy and then it will take somewhere
> between a day and a week between incidents.  We can't correlate it with
> any particular SQL being run.  This was a production server so we've had
> to move the customers assigned to it back to some slightly older but
> very hardware but running the same version of postgresql (same package
> compiled on our own pkgrepo) and same 10.3-RELEASE-p4/p5 -- where they
> are running quite happily.
> 
Compile rtld/libc/libthr and postgres itself with debugging symbols
and re-obtain the backtrace, for start.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
[prev in list] [next in list] [prev in thread] [next in thread]