On this system we have: # dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/parity Savecore enabled: no Save compressed: on So savecore is disabled - dumpadm -n is in effect. We will solve the case immediately upon receipt of a "dump pending" event, but can't include pointers to an extracted crash dump. We force a panic with reboot -d. On reboot we see the following to the console: SUNW-MSG-ID: SUNOS-8000-KL, TYPE: Defect, VER: 1, SEVERITY: Major EVENT-TIME: Sat May 22 16:43:23 PDT 2010 PLATFORM: Sun-Fire-V40z, CSN: XG051535088, HOSTNAME: parity SOURCE: software-diagnosis, REV: 0.1 EVENT-ID: 9e174ce3-ef35-62e2-b4fb-ca2487078cfa DESC: The system has rebooted after a kernel panic. Refer to http://sun.com/msg/SUNOS-8000-KL for more information. AUTO-RESPONSE: The failed system image was dumped to the dump device. If savecore is enabled (see dumpadm(1M)) a copy of the dump will be written to the savecore directory . IMPACT: There may be some performance impact while the panic is copied to the savecore directory. Disk space usage by panics can be substantial. REC-ACTION: Please log a call with you support vendor and provide them with this information. If savecore is not enabled then please take steps to preserve the crash image. # fmadm faulty --------------- ------------------------------------ -------------- --------- TIME EVENT-ID MSG-ID SEVERITY --------------- ------------------------------------ -------------- --------- May 22 16:43:23 9e174ce3-ef35-62e2-b4fb-ca2487078cfa SUNOS-8000-KL Major Host : parity Platform : Sun-Fire-V40z Chassis_id : XG051535088 Product_sn : Fault class : defect.sunos.kernel.panic Affects : sw:///:path=/var/crash/parity/.9e174ce3-ef35-62e2-b4fb-ca2487078cfa faulted but still in service Problem in : sw:///:path=/var/crash/parity/.9e174ce3-ef35-62e2-b4fb-ca2487078cfa faulted but still in service Description : The system has rebooted after a kernel panic. Refer to http://sun.com/msg/SUNOS-8000-KL for more information. Response : The failed system image was dumped to the dump device. If savecore is enabled (see dumpadm(1M)) a copy of the dump will be written to the savecore directory . Impact : There may be some performance impact while the panic is copied to the savecore directory. Disk space usage by panics can be substantial. Action : Please log a call with you support vendor and provide them with this information. If savecore is not enabled then please take steps to preserve the crash image. The notification email: Date: Sat, 22 May 2010 16:43:23 -0700 (PDT) From: No Access User Message-Id: <201005222343.o4MNhNwU100766@parity.sfbay.sun.com> X-FMEV-HOSTNAME: parity X-FMEV-CLASS: list.suspect X-FMEV-UUID: 9e174ce3-ef35-62e2-b4fb-ca2487078cfa X-FMEV-CODE: SUNOS-8000-KL X-FMEV-SEVERITY: Major Reply-To: root@parity.sfbay.sun.com Subject: Fault Management Event: parity:SUNOS-8000-KL X-Source-IP: parity.SFBay.Sun.COM [10.4.193.172] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BF86CB5.004B,ss=1,fgs=0 SUNW-MSG-ID: SUNOS-8000-KL, TYPE: Defect, VER: 1, SEVERITY: Major EVENT-TIME: 20 PLATFORM: Sun-Fire-V40z, CSN: XG051535088, HOSTNAME: parity SOURCE: software-diagnosis, REV: 0.1 EVENT-ID: 9e174ce3-ef35-62e2-b4fb-ca2487078cfa DESC: The system has rebooted after a kernel panic. Refer to http://sun.com/msg/SUNOS-8000-KL for more information. AUTO-RESPONSE: The failed system image was dumped to the dump device. If savecore is enabled (see dumpadm(1M)) a copy of the dump will be written to the savecore directory . IMPACT: There may be some performance impact while the panic is copied to the savecore directory. Disk space usage by panics can be substantial. REC-ACTION: Please log a call with you support vendor and provide them with this information. If savecore is not enabled then please take steps to preserve the crash image. In the messages file there is also: May 22 16:43:12 parity savecore: [ID 837958 auth.error] Panic crashdump pending on dump device but dumpadm -n in effect; run savecore(1M) manually to extract. Image UUID 9e174ce3-ef35-62e2-b4fb-ca2487078cfa. The full list.suspect: # fmdump -u 9e174ce3-ef35-62e2-b4fb-ca2487078cfa -Vp TIME UUID SUNW-MSG-ID May 22 2010 16:43:23.318344000 9e174ce3-ef35-62e2-b4fb-ca2487078cfa SUNOS-8000-KL TIME CLASS ENA May 22 16:43:12.3618 ireport.os.sunos.panic.dump_pending_on_device 0x0000000000000000 nvlist version: 0 version = 0x0 class = list.suspect uuid = 9e174ce3-ef35-62e2-b4fb-ca2487078cfa code = SUNOS-8000-KL diag-time = 1274571803 272876 de = fmd:///module/software-diagnosis fault-list-sz = 0x1 fault-list = (array of embedded nvlists) (start fault-list[0]) nvlist version: 0 version = 0x0 class = defect.sunos.kernel.panic certainty = 0x64 asru = sw:///:path=/var/crash/parity/.9e174ce3-ef35-62e2-b4fb-ca2487078cfa resource = sw:///:path=/var/crash/parity/.9e174ce3-ef35-62e2-b4fb-ca2487078cfa savecore-succcess = 0 os-instance-uuid = 9e174ce3-ef35-62e2-b4fb-ca2487078cfa panicstr = forced crash dump initiated at user request panicstack = genunix:kadmin+16e () | genunix:uadmin+10f () | unix:brand_sys_syscall32+272 () | crashtime = 1274571727 panic-time = Sat May 22 16:42:07 2010 PDT (end fault-list[0]) fault-status = 0x1 severity = Major __ttl = 0x1 __tod = 0x4bf86c1b 0x12f98b40 And the original event: # fmdump -IVp -c ireport.os.sunos.panic.dump_pending_on_device TIME UUID May 22 2010 16:43:12.361855092 85f43ab3-3df6-cab0-a288-bcd583e58d1b nvlist version: 0 version = 0x0 class = ireport.os.sunos.panic.dump_pending_on_device uuid = 85f43ab3-3df6-cab0-a288-bcd583e58d1b detector = sw:///:path=/usr/bin/amd64/savecore#:file=../savecore.c:func=raise_event:line=1603 pri = high attr = (embedded nvlist) nvlist version: 0 dumpdir = /var/crash/parity savecore-enabled = 0 os-instance-uuid = 9e174ce3-ef35-62e2-b4fb-ca2487078cfa crashtime = 1274571727 panicstr = forced crash dump initiated at user request panicstack = genunix:kadmin+16e () | genunix:uadmin+10f () | unix:brand_sys_syscall32+272 () | fm-panic = 0 will-attempt-savecore = 0 panic-time = Sat May 22 16:42:07 2010 PDT (end attr) __ttl = 0x1 __tod = 0x4bf86c10 0x15917874