[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ipfilter
Subject:    Re: IPFilter 4.1.1  caused SUN V240 to panic and reboot
From:       "Mathew Thomas" <mathew.thomas () rmit ! edu ! au>
Date:       2004-07-22 3:44:16
Message-ID: s0ffc4cd.062 () its-mn-inet1 ! its ! rmit ! edu ! au
[Download RAW message or body]

Hi Darren,

Looks like the problem moved to different stack. I am now using IPFilter-4.1.3 \
compiled with Sun Studio 8 ( This a DNS server running Solaris 9). The following are \
from the dump analysis:

ts-du-ns2[/var/crash/its-du-ns2]
$>mdb -k unix.2 vmcore.2
Loading modules: [ unix krtld genunix ip ufs_log usba cpc ipc random nfs ]
> $c
mutex_enter+4(30004009180, 0, 780e24e8, 0, 300055dc1ac, 0)
fr_updatestate+0x5c(2a1005d68e8, 30004009080, 780e24e8, 2a1005d694c, 
30004009180, 1)
fr_checkstate+0x32c(2a1005d68e8, 2a1005d68d0, 40004702, 3, 30002ec4a18, 
30004009080)
fr_check+0x5e4(300055dc198, 30006fe2f80, 0, 1, 2a1005d68e8, 2a1005d6d30)
pfil_precheck+0xd28(e, 30006fe2f80, 2, 30001245ed8, 0, 300055dc198)
pfilmodwput+0x17c(300011f8898, 30006fe2f80, 20, 300055dc1ac, de010800, 
30001245ed8)
putnext+0x21c(0, 30006fe2f80, ffffffffffffffff, 0, 579b40000, 8)
ip_wput_ire+0xacc(10000, 300000727b0, 3, ffff, 83aa0201, 30006fe2f80)
ip_wput+0x4d4(30006cc3088, 0, c9ebf91, 300055dc198, 0, 21683a1)
putnext+0x21c(0, 300010e5f80, 20, 0, 0, c0)
udp_wput+0x5b0(38, 14, 38, 300055dc1ac, 300055dc198, c0)
putnext+0x21c(0, 30006f4d680, 20, 30006fe2f80, 0, 98)
strput+0x270(30003a28018, 0, 0, 2a1005d7598, 0, 0)
kstrputmsg+0x36c(300011e58f8, 30004392b88, 0, 0, 0, 0)
sosend_dgram+0x25c(0, 3000135f9e8, 10, 2a1005d78d0, 8000, 2337d78)
sosendmsg+0x3f4(0, 3000135f9e8, 7, 20, 428aa4, 2337d78)
sendit+0x15c(2a1005d78d0, 8000, 300011e58f8, 8000, 2a1005d7900, 21683a1)
sendmsg+0x2e8(18, fef7f820, 8000, fef7f840, fef7f83c, c0)
syscall_trap32+0xa8(18, fef7f820, 8000, fef7f840, fef7f83c, c0)


> $<msgbuf
0x300010ac8a3:  mc-us3i1 at root: SAFARI 0x1 0x0 ...
0x300010ac4e3:  mc-us3i1 is /memory-controller@1,0
0x300010ac260:  power0 at ebus0: offset 0,800
0x30001093ea3:  power0 is /pci@1e,600000/isa@7/power@0,800
0x30001093c20:  rmc_comm0 at ebus0: offset 0,3e8
0x300010939a3:  rmc_comm0 is /pci@1e,600000/isa@7/rmc-comm@0,3e8
0x30001093722:  PCI-device: pmu@6, pmubus0
0x300010934a2:  PCI-device: gpio@8a, pmugpio0
0x30001093223:  pmugpio0 is /pci@1e,600000/pmu@6/gpio@8a
0x30001092fa2:  pseudo-device: rmclomv0
0x30001092d23:  rmclomv0 is /pseudo/rmclomv@0
0x30001092aa3:  Hardware watchdog enabled
0x30001092820:  su0 at ebus0: offset 0,3f8
0x300010925a3:  su0 is /pci@1e,600000/isa@7/serial@0,3f8
0x30001092320:  su1 at ebus0: offset 0,2e8
0x300010920a3:  su1 is /pci@1e,600000/isa@7/serial@0,2e8
0x300010dbda0:  cpu1: UltraSPARC-IIIi (portid 1 impl 0x16 ver 0x24 clock 1002 
MHz)
0x300010dbb20:  cpu0: UltraSPARC-IIIi (portid 0 impl 0x16 ver 0x24 clock 1002 
MHz)
0x300010db8a0:  cpu 0 initialization complete - online
0x300010db622:  PCI-device: usb@a, ohci0
0x300010db3a3:  ohci0 is /pci@1e,600000/usb@a
0x300010db11f:  bge0: BCM570x driver v0.37: type "ether" mac address 
00:03:ba:5f:de:01
0x300010daea2:  PCI-device: network@2, bge0
0x300010dac23:  bge0 is /pci@1f,700000/network@2
0x300010da99f:  bge1: BCM570x driver v0.37: type "ether" mac address 
00:03:ba:5f:de:02
0x300010da722:  PCI-device: network@2,1, bge1
0x300010da4a3:  bge1 is /pci@1f,700000/network@2,1
0x300010da21f:  bge2: BCM570x driver v0.37: type "ether" mac address 
00:03:ba:5f:de:03
0x30001253f22:  PCI-device: network@2, bge2
0x30001253ca3:  bge2 is /pci@1d,700000/network@2
0x30001253a1f:  bge3: BCM570x driver v0.37: type "ether" mac address 
00:03:ba:5f:de:04
0x300012537a2:  PCI-device: network@2,1, bge3
0x30001253523:  bge3 is /pci@1d,700000/network@2,1
0x30001252edf:  NOTICE: bge0: link down (initialised)
0x30001252c63:  dump on /dev/md/dsk/d20 size 2048 MB
0x300012529df:  NOTICE: bge0: link up 1000Mbps Full-Duplex (initialised)
0x30001252622:  pseudo-device: devinfo0
0x300012523a3:  devinfo0 is /pseudo/devinfo@0
0x30001252122:  pseudo-device: fssnap0
0x3000241dd63:  fssnap0 is /pseudo/fssnap@0
0x3000241d9a2:  pseudo-device: ramdisk1024
0x3000241d723:  ramdisk1024 is /pseudo/ramdisk@1024
0x3000241d4a2:  pseudo-device: winlock0
0x3000241d223:  winlock0 is /pseudo/winlock@0
0x3000241cfa2:  pseudo-device: lockstat0
0x3000241cd23:  lockstat0 is /pseudo/lockstat@0
0x3000241caa2:  pseudo-device: vol0
0x3000241c823:  vol0 is /pseudo/vol@0
0x3000241c6e2:  pseudo-device: llc10
0x3000241c463:  llc10 is /pseudo/llc1@0
0x3000241c1e2:  pseudo-device: pm0
0x30002573ee3:  pm0 is /pseudo/pm@0
0x30002573c62:  pseudo-device: tod0
0x300025739e3:  tod0 is /pseudo/tod@0
0x30002573622:  pseudo-device: lofi0
0x300025733a3:  lofi0 is /pseudo/lofi@0
0x30002573122:  pseudo-device: fcp0
0x30002572ea3:  fcp0 is /pseudo/fcp@0
0x30002572c21:  /kernel/drv/sparcv9/fcsm: undefined symbol
0x30002572861:   'fc_ulp_get_port_login_params'
0x300025725e1:  WARNING: mod_load: cannot load module 'fcsm'
0x30002572362:  pseudo-device: rsm0
0x300025720e3:  rsm0 is /pseudo/rsm@0
0x30002561ca2:  pseudo-device: trapstat0
0x30002561a23:  trapstat0 is /pseudo/trapstat@0
0x300025618e2:  pseudo-device: rmcadm0
0x30002561663:  rmcadm0 is /pseudo/rmcadm@0
0x3000256129f:  IP Filter: v4.1.3, running.
0x30002560b1f:  IP Filter: v4.1.3 detached.
0x30002560edf:  IP Filter: v4.1.3, running.
0x300061d4762:  pseudo-device: devinfo0
0x300061c53a3:  devinfo0 is /pseudo/devinfo@0
0x300061c0620:  
panic[cpu0]/thread=30004d517e0: 
0x300000824a0:  BAD TRAP: type=31 rp=2a1005d6490 addr=0 mmu_fsr=0 occurred in 
module "unix"
 due to a NULL pointer dereference
0x300061c6820:  

0x300061c5da0:  named: 
0x300061c65a0:  trap type = 0x31
0x300061bb360:  pid=364, pc=0x1035ba4, sp=0x2a1005d5d31, tstate=0x9980001605, 
context=0x8a0
0x300061c6d20:  g1-g7: ffffffffffffffd0, 1, 1388, 0, 300055dc1ac, 0, 30004d517e0
0x300061c79a0:  
0x300061c03a3:  000002a1005d61c0 unix:die+80 (31, 2a1005d6490, 0, 0, de010800, 
30001245ed8)
0x300061c5623:    %l0-3: 0000000000000000 0000000001413590 000002a1005d6490 
000002a1005d638
8
  %l4-7: 0000000000000031 000002a1005d78d0 0000000000000000 00000300070ef4c0
0x300061c74a3:  000002a1005d62a0 unix:trap+874 (2a1005d6490, 0, 10000, 10200, 0, 
8)
0x30001252263:    %l0-3: 0000000000000001 0000000000000000 0000030004d37488 
000000000000003
1
  %l4-7: 0000000000000005 0000000000000001 0000000000000000 0000000000000000
0x300061c58a3:  000002a1005d63e0 unix:ktl0+48 (0, 30004d517e0, 3, ffff, 
83aa0201, 300055dc1
ac)
0x300061c7723:    %l0-3: 0000000000000006 0000000000001400 0000009980001605 
000000000102c90
c
  %l4-7: 0000000000000000 0000000000000000 0000000000000000 000002a1005d6490
0x30002560763:  000002a1005d6530 ipf:fr_movequeue+30 (30004009180, 0, 780e24e8, 
0, 300055dc
1ac, 0)
0x300061c1a23:    %l0-3: 0000000000000000 0000000000000002 0000000000000000 
00000300040090c
8                                     
  %l4-7: 0000030004009080 0000030004009080 0000030006cc3088 000000000000800f
0x300061c5123:  000002a1005d6600 ipf:fr_updatestate+5c (2a1005d68e8, 
30004009080, 780e24e8,
 2a1005d694c, 30004009180, 1)
0x300061c4c23:    %l0-3: 00000000780e24e8 0000030004009080 0000000000000000 
000003000400908
0
  %l4-7: 0000030004a6a000 00000000904941e7 000002a1005d68f0 0000000000000000
0x30000082363:  000002a1005d66f0 ipf:fr_checkstate+32c (2a1005d68e8, 
2a1005d68d0, 40004702,
 3, 30002ec4a18, 30004009080)
0x30002572223:    %l0-3: 0000000000000000 0000000000000011 000000000000ffff 
000000000000000
3
  %l4-7: 000002a1005d68e8 0000030006f4d680 00000000014dad2c 00000300055dc1ac
0x3000241c5a3:  000002a1005d67e0 ipf:fr_check+5e4 (300055dc198, 30006fe2f80, 0, 
1, 2a1005d6
8e8, 2a1005d6d30)
0x300010db9e3:    %l0-3: 0000000000000000 000002a1005d68e8 0000000040000000 
00000000780e2fe
0
  %l4-7: 000000000000001c 0000030006fe2f80 000000000000001c 0000000000000004
0x300010926e3:  000002a1005d69f0 pfil:pfil_precheck+d28 (e, 30006fe2f80, 2, 
30001245ed8, 0,
 300055dc198)
0x300025613e3:    %l0-3: 0000000000000014 0000000078161558 0000000000000000 
0000030006fe2f8
0
  %l4-7: 000000000000004c 0000030006fe2fa0 000000000000004c 0000000000000000
0x300025617a3:  000002a1005d6c40 pfil:pfilmodwput+17c (300011f8898, 30006fe2f80, 
20, 300055
dc1ac, de010800, 30001245ed8)
0x30003210fa3:    %l0-3: 00000300000727b0 0000030001245ed8 0000030003a28018 
00000300011f889
8
  %l4-7: 0000000000000083 000002a1005d78d0 0000000000000000 00000300070ef4c0
0x30003210463:  000002a1005d6d40 unix:putnext+21c (0, 30006fe2f80, 
ffffffffffffffff, 0, 579
b40000, 8)
0x30000083da3:    %l0-3: 00000000013fbe10 00000300011f8988 00000300011f8898 
000000000000000
0
  %l4-7: 0000000000000000 0000000078020930 00000300011f9540 0000030006fe2f80
0x30003211223:  000002a1005d6df0 ip:ip_wput_ire+acc (10000, 300000727b0, 3, 
ffff, 83aa0201,
 30006fe2f80)                         
0x30003210d23:    %l0-3: 0000030007041540 000003000102e000 0000030006cc3088 
00000300055dc19
8
  %l4-7: 0000000000000000 0000000000000000 00000300011f9540 000000000000000e
0x300061d5b63:  000002a1005d6ff0 ip:ip_wput+4d4 (30006cc3088, 0, c9ebf91, 
300055dc198, 0, 2
1683a1)
0x300003a79e3:    %l0-3: 0000030006fe2f80 0000000000000000 000000000c9ebf91 
00000000014a940
0
  %l4-7: 0000030004392900 0000030002fba7b8 0000030006cc3088 0000030002fba7c8
0x300000839e3:  000002a1005d70a0 unix:putnext+21c (0, 300010e5f80, 20, 0, 0, c0)
0x30003211363:    %l0-3: 00000000011fee0c 00000300010e9ee0 0000030004392900 
000000000000000
0
  %l4-7: 00000300010e5f88 00000000014a9b50 0000030004392678 0000030006fe2f80
0x30003211ae3:  000002a1005d7150 udp:udp_wput+5b0 (38, 14, 38, 300055dc1ac, 
300055dc198, c0
)
0x300025608a3:    %l0-3: 000000000c9ebf91 0000030006fe2f80 000000000000ffff 
000000000000003
5
  %l4-7: 0000030004392678 0000030006f4d680 00000000014dad2c 0000030004301498
0x30001092463:  000002a1005d7220 unix:putnext+21c (0, 30006f4d680, 20, 
30006fe2f80, 0, 98)
0x3000241dc23:    %l0-3: 00000000012d9060 0000030004392768 0000030004392678 
000000000000000
0
  %l4-7: 0000000000000000 00000000014d9350 0000030004392b88 0000030006f4d680
0x300061d5f23:  000002a1005d72d0 genunix:strput+270 (30003a28018, 0, 0, 
2a1005d7598, 0, 0)
0x300061573a3:    %l0-3: 0000000000000004 0000030004392b88 0000000000000000 
0000030003a2801
8
  %l4-7: 0000030006f4d680 000002a1005d78d0 0000000000000000 0000000000000000
0x30002560da3:  000002a1005d74c0 genunix:kstrputmsg+36c (300011e58f8, 
30004392b88, 0, 0, 0,
 0)
0x3000241d363:    %l0-3: 0000030003a28098 00000300070ef4c0 0000030003a28018 
000000000000000
4
  %l4-7: 0000000000000083 000002a1005d78d0 0000000000000000 00000300070ef4c0
0x30002561f23:  000002a1005d75a0 sockfs:sosend_dgram+25c (0, 3000135f9e8, 10, 
2a1005d78d0, 
8000, 2337d78)
0x30000c3f3e3:    %l0-3: 0000000000000010 000000000144f000 0000000000000030 
00000300011e58f
8
  %l4-7: 00000300070ef4c0 00000000000003c0 0000000000000000 0000000000000000
0x300003c6963:  000002a1005d7690 sockfs:sosendmsg+3f4 (0, 3000135f9e8, 7, 20, 
428aa4, 2337d
78)
0x300010921e3:    %l0-3: 0000000000000010 0000000000008000 000003000135f9e8 
000000000000000
0
  %l4-7: 00000300011e58f8 000002a1005d7a80 000002a1005d78d0 0000000000000000
0x30002573da3:  000002a1005d7750 sockfs:sendit+15c (2a1005d78d0, 8000, 
300011e58f8, 8000, 2
a1005d7900, 21683a1)
0x30001252763:    %l0-3: 000003000135f9e8 0000000000000000 0000000000000000 
000000000000000
0
  %l4-7: 00000300041bea80 0000000000000018 0000000000000030 000002a1005d7a80
0x300010dbee3:  000002a1005d7820 sockfs:sendmsg+2e8 (18, fef7f820, 8000, 
fef7f840, fef7f83c
, c0)
0x300010db263:    %l0-3: 0000000000000001 0000000000008000 00000000004bea90 
000000000010000
0
  %l4-7: 0000000000008000 000000000140f5b8 0000000000000001 00000000001555c0
0x3000241d0e0:  
0x30001253663:  syncing file systems...
0x300061bafa3:   4
0x300061bb4a3:   done
0x300061bb9a3:  dumping to /dev/md/dsk/d20, offset 429588480, content: kernel + 
curproc
> 


Thanks
Mathew



> > > Darren Reed <darrenr@reed.wattle.id.au> 20/07/04 7:54:47 >>>
In some email I received from Laurent Blume, sie wrote:
> Jeff A. Earickson wrote:
> 
> > 2) Sun software engineers will **always** blame ipfilter for the problem
> > when the see it in the stack trace.  While the problem may actually be
> > due to ipfilter, don't believe Sun.
> [snip]
> 
> Well, since ipfilter is integrated in Solaris 10, it's not like they're 
> always going to get away with that -- so you'll be able to believe them 
> pretty soon :-)

The problem reported was fixed for 4.1.2.

Darren


[prev in list] [next in list] [prev in thread] [next in thread]