From sherry.moore@Sun.COM Tue Aug 11 23:28:54 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C6SsCC014946 for ; Tue, 11 Aug 2009 23:28:54 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7C6SkCA009718; Wed, 12 Aug 2009 07:28:51 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO9007053C3W100@brm-avmta-1.central.sun.com>; Wed, 12 Aug 2009 00:28:51 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO90017S3C27L80@brm-avmta-1.central.sun.com>; Wed, 12 Aug 2009 00:28:50 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n7C6Soh8302459 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 11 Aug 2009 23:28:50 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id n7C6Sows302458; Tue, 11 Aug 2009 23:28:50 -0700 (PDT) Date: Tue, 11 Aug 2009 23:28:50 -0700 From: Sherry Moore Subject: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] To: psarc-ext@Sun.COM Cc: David Plauger , Steven Sistare Message-id: <20090812062850.GF285030@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 X-Authentication-warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 11176 I am sponsoring this fasttrack for Dave Plauger and Steve Sistare. The timer is set for next Tuesday August 18, 2009. Requesting Micro/Patch binding. The one-pager, project specification and man page diffs are available in the materials directory. Sherry Moore Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Fast Crash Dump 1.2. Name of Document Author/Supplier: Dave Plauger Steve Sistare 1.3. Date of This Document: August 11, 2009 4. Technical Description: 4.1. Details: New command line flags to dumpadm(1M) control how core files are to be saved. The setting is saved across reboots in /etc/dumpadm.conf. The new default behavior is to save files in compressed format instead of always uncompressing them, as it does currently. If saving compressed, savecore(1M) copies the core file from the dump device to vmdump.N, where N is the usual dump integer. Copying a core file is much faster than uncompressing it into unix.N and vmcore.N images; and it takes up much less disk space. On systems that dump to the swap area there is less risk that the core image will be over-written by swap activity before it can be extracted. savecore(1M) performance has been improved by reading the dump file with fread(3C) instead of pread(2). The compressed core dump format is largely unchanged. The dump header and the dump version number are unchanged. However, there are changes to the way memory pages are saved within the compressed file. A compressed dump file must be uncompressed first, by manually running savecore(1M) a second time, before it can be used by other tools. Therefore, there is no impact on mdb(1) due to the changes in compression methods. Once the core dump has been uncompressed, the resulting unix.N and vmcore.N files are in the same format as before. This project has introduced a bzip2 compression library [2] into Solaris common code, where it is shared by savecore(1M) and the kernel. The current lzjb compression algorithm is much faster, but also much weaker. The bzip2 library requires much more memory and compute resources. If these resources are available during panic, the kernel will save memory pages with bzip2 instead of lzjb, and savecore(1M) will use the same bzip2 library in order to uncompress the pages. The kernel function dumpsys() does most of the work in creating core dump images. The section that saves memory pages has been expanded to support parallelism. Most kernel services are not available during panic. Instead, CPUs spinning in panic_idle() call into dumpsys() and coordinate via memory. These helper CPUs copy pages, compress them, and produce streams of compression data that savecore(1M) can uncompress. The panic CPU acts as the master and does all page mapping and I/O operations. There are two compression modes in this implementation. The older method, lzjb is the default on smaller systems. With up to 4 CPUs, it usually speeds up dump by 2-4 times. The new bzip2 library is employed on large systems with many spare CPUs and memory. This can speed up dumps by 4-10 times. The mode is chosen at crash time based on processor type, number of CPUs, and available free memory for buffers. The existing savecore -L (live dump) option creates a dump image on a running system. This option is available only when there is a dedicated dump device. In this case, the dump helpers in the kernel run as system tasks. file(1) can detect the new compressed format. For example, # file vmcore.0 vmcore.0: SunOS 5.11 snv_81 64-bit SPARC crash dump from 'oaf415' # file vmdump.0 vmdump.0: SunOS 5.11 snv_81 64-bit SPARC compressed crash dump from 'oaf415' For more information, see the project specification in the materials directory. 4.2. Bug/RFE Number(s): RFE 6828976 Fast Crash Dump 4.5. Interfaces: New command line flags to dumpadm(1M). And additions to the meaning of existing flags to savecore(1M). Micro/patch binding requested. INTERFACE COMMITMENT LEVEL COMMENT dumpadm -z (1M) Committed Enables save compressed, or not. 4.6. Doc Impact: Man page changes for dumpadm(1M) and savecore(1M). See appendix A for diffs. 6. Resources and Schedule: 6.4. Product Approval Committee requested information: 6.4.1. Consolidation or Component Name: OS/Net 6.5. ARC review type: Fasttrack 6.6. ARC Exposure: Open. A. Man pages A.1 dumpadm(1M) A.2 savecore(1M) A.1 Man pages dumpadm(1M) System Administration Commands dumpadm(1M) NAME dumpadm - configure operating system crash dump SYNOPSIS /usr/sbin/dumpadm [-nuy] [-c content-type] [-d dump-device] [-m mink | minm | min%] [-s savecore-dir] [-r root-dir] [-z y | n] | DESCRIPTION The dumpadm program is an administrative command that manages the configuration of the operating system crash dump facility. A crash dump is a disk copy of the physical memory of the computer at the time of a fatal system error. When a fatal operating system error occurs, a message describing the error is printed to the console. The operating system then generates a crash dump by writing the contents of phy- sical memory to a predetermined dump device, which is typi- cally a local disk partition. The dump device can be config- ured by way of dumpadm. Once the crash dump has been written to the dump device, the system will reboot. Fatal operating system errors can be caused by bugs in the | operating system, its associated device drivers and loadable | modules, or by faulty hardware. Whatever the cause, the | crash dump itself provides invaluable information to your | support engineer to aid in diagnosing the problem. As such, | it is vital that the crash dump be retrieved and given to | your support provider. Following an operating system crash, | the savecore(1M) utility is executed automatically during | boot to retrieve the crash dump from the dump device, and | write it to your file system in compressed form to a file | name vmdump.X, where X is an integer identifying the dump. | Afterwards, savecore(1M) can be invoked on the same or | another system to expand the compressed crash dump to a pair | of files named unix.X and vmcore.X. The directory in which | the crash dump is saved on reboot can also be configured | using dumpadm. For systems with a UFS root file system, the default dump | device is configured to be an appropriate swap partition. | Swap partitions are disk partitions reserved as virtual | memory backing store for the operating system. Thus, no per- | manent information resides in swap to be overwritten by the | dump. See swap(1M). For systems with a ZFS root file system, | dedicated ZFS volumes are used for swap and dump areas. For | further information about setting up a dump area with ZFS, | see the ZFS Administration Guide. To view the current dump | configuration, use the dumpadm command with no arguments: example# dumpadm Dump content: kernel pages Dump device: /dev/dsk/c0t0d0s1 (swap) Savecore directory: /var/crash/saturn Savecore enabled: yes Save compressed: yes | When no options are specified, dumpadm prints the current | crash dump configuration. The example shows the set of | default values: the dump content is set to kernel memory | pages only, the dump device is a swap disk partition, the | directory for savecore files is set to /var/crash/hostname. | savecore(1M) is set to run automatically on reboot and save | the crash dump in a compressed format. -z y | n | Modify the dump configuration to control the operation | of savecore on reboot. The options are y (yes) to enable | saving core files in a compressed format, and n (no) | automatically uncompress the crash dump file. The | default is yes, because crash dump files can be very | large and will require less file system space if saved | in a compressed format. | EXAMPLES Example 1 Reconfiguring The Dump Device To A Dedicated Dump Device: The following command reconfigures the dump device to a dedicated dump device: example# dumpadm -d /dev/dsk/c0t2d0s2 Dump content: kernel pages Dump device: /dev/dsk/c0t2d0s2 (dedicated) Savecore directory: /var/crash/saturn Savecore enabled: yes Save compressed: yes | A.2 Man pages for savecore(1M) System Administration Commands savecore(1M) NAME savecore - save a crash dump of the operating system SYNOPSIS /usr/bin/savecore [-Lvd] [-f dumpfile] [directory] DESCRIPTION The savecore utility saves a crash dump of the kernel (assuming that one was made) and writes a reboot message in the shutdown log. It is invoked by the dumpadm service each time the system boots. savecore can be configured by dumpadm(1M) to save crash dump | data in either a compressed or uncompressed format. For the | compressed format, savecore saves the crash dump data in the | file directory/vmdump.N, where N in the name is replaced by | a number which grows every time savecore is run in that | directory. The compressed file can be uncompressed in a | separate step using the -f dumpfile option. For the | uncompressed format, savecore saves the crash dump data in | the file directory/vmcore.N and the kernel's namelist in | directory/unix.N. OPTIONS -f dumpfile Save a crash dump from the specified file | instead of from the system's current dump | device. When given directory/vmdump.N, | uncompress the file to vmcore.N and unix.N, | where N is the same number as in the | compressed name. | This option may also be useful if the infor- | mation stored on the dump device has been | copied to an on-disk file by means of the | dd(1M) command. FILES directory/vmdump.n | directory/vmcore.n directory/unix.n -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From gdamore@sun.com Tue Aug 11 23:37:48 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C6blNo014984 for ; Tue, 11 Aug 2009 23:37:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7C6bdEg027949 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 14:37:46 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900F013QYY400@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 11 Aug 2009 23:37:46 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO9009BP3QXQF40@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 11 Aug 2009 23:37:45 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7C6bjk2015024 for ; Tue, 11 Aug 2009 23:37:45 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO9009003EZ9Q00@fe-sfbay-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 11 Aug 2009 23:37:45 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KO9002GD3QWAU40@fe-sfbay-10.sun.com>; Tue, 11 Aug 2009 23:37:44 -0700 (PDT) Date: Tue, 11 Aug 2009 23:37:44 -0700 From: "Garrett D'Amore" Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <4A826338.9070608@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 12050 +1. (I cheated -- I've seen the case materials ahead of time. :-) This enhancement (and the need to perform subsequent actions to decompress the crash dump) probably deserves special mention in the Release Notes. I'd also add, in retrospect, it seems like perhaps mdb ought to have its man page updated with at least a passing reference to compressed crash dumps (and the step required to decompress them.) -- Garrett Sherry Moore wrote: > I am sponsoring this fasttrack for Dave Plauger and Steve Sistare. The > timer is set for next Tuesday August 18, 2009. Requesting Micro/Patch > binding. > > The one-pager, project specification and man page diffs are available > in the materials directory. > > Sherry Moore > > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > > 1. Introduction > 1.1. Project/Component Working Name: > Fast Crash Dump > 1.2. Name of Document Author/Supplier: > Dave Plauger > Steve Sistare > 1.3. Date of This Document: > August 11, 2009 > > 4. Technical Description: > 4.1. Details: > New command line flags to dumpadm(1M) control how core files > are to be saved. The setting is saved across reboots in > /etc/dumpadm.conf. The new default behavior is to save files > in compressed format instead of always uncompressing them, as > it does currently. > > If saving compressed, savecore(1M) copies the core file from > the dump device to vmdump.N, where N is the usual dump > integer. Copying a core file is much faster than uncompressing > it into unix.N and vmcore.N images; and it takes up much less > disk space. On systems that dump to the swap area there is > less risk that the core image will be over-written by swap > activity before it can be extracted. > > savecore(1M) performance has been improved by reading the dump > file with fread(3C) instead of pread(2). > > The compressed core dump format is largely unchanged. The dump > header and the dump version number are unchanged. However, > there are changes to the way memory pages are saved within the > compressed file. A compressed dump file must be uncompressed > first, by manually running savecore(1M) a second time, before > it can be used by other tools. Therefore, there is no impact > on mdb(1) due to the changes in compression methods. Once the > core dump has been uncompressed, the resulting unix.N and > vmcore.N files are in the same format as before. > > This project has introduced a bzip2 compression library [2] > into Solaris common code, where it is shared by savecore(1M) > and the kernel. The current lzjb compression algorithm is much > faster, but also much weaker. The bzip2 library requires much > more memory and compute resources. If these resources are > available during panic, the kernel will save memory pages with > bzip2 instead of lzjb, and savecore(1M) will use the same > bzip2 library in order to uncompress the pages. > > The kernel function dumpsys() does most of the work in > creating core dump images. The section that saves memory pages > has been expanded to support parallelism. Most kernel services > are not available during panic. Instead, CPUs spinning in > panic_idle() call into dumpsys() and coordinate via > memory. These helper CPUs copy pages, compress them, and > produce streams of compression data that savecore(1M) can > uncompress. The panic CPU acts as the master and does all page > mapping and I/O operations. > > There are two compression modes in this implementation. The > older method, lzjb is the default on smaller systems. With up > to 4 CPUs, it usually speeds up dump by 2-4 times. The new > bzip2 library is employed on large systems with many spare > CPUs and memory. This can speed up dumps by 4-10 times. The > mode is chosen at crash time based on processor type, number > of CPUs, and available free memory for buffers. > > The existing savecore -L (live dump) option creates a dump > image on a running system. This option is available only when > there is a dedicated dump device. In this case, the dump > helpers in the kernel run as system tasks. > > file(1) can detect the new compressed format. For example, > # file vmcore.0 > vmcore.0: SunOS 5.11 snv_81 64-bit SPARC crash dump from 'oaf415' > # file vmdump.0 > vmdump.0: SunOS 5.11 snv_81 64-bit SPARC compressed crash dump from 'oaf415' > > For more information, see the project specification in the > materials directory. > > 4.2. Bug/RFE Number(s): > > RFE 6828976 Fast Crash Dump > > 4.5. Interfaces: > > New command line flags to dumpadm(1M). And additions to the > meaning of existing flags to savecore(1M). > > Micro/patch binding requested. > > INTERFACE COMMITMENT LEVEL COMMENT > > dumpadm -z (1M) Committed Enables save compressed, or not. > > > 4.6. Doc Impact: > > Man page changes for dumpadm(1M) and savecore(1M). > > See appendix A for diffs. > > 6. Resources and Schedule: > > 6.4. Product Approval Committee requested information: > 6.4.1. Consolidation or Component Name: > OS/Net > > 6.5. ARC review type: Fasttrack > > 6.6. ARC Exposure: Open. > > A. Man pages > A.1 dumpadm(1M) > A.2 savecore(1M) > > A.1 Man pages dumpadm(1M) > > > System Administration Commands dumpadm(1M) > > NAME > dumpadm - configure operating system crash dump > > SYNOPSIS > /usr/sbin/dumpadm [-nuy] [-c content-type] [-d dump-device] > [-m mink | minm | min%] [-s savecore-dir] > [-r root-dir] [-z y | n] | > > DESCRIPTION > The dumpadm program is an administrative command that > manages the configuration of the operating system crash dump > facility. A crash dump is a disk copy of the physical memory > of the computer at the time of a fatal system error. When a > fatal operating system error occurs, a message describing > the error is printed to the console. The operating system > then generates a crash dump by writing the contents of phy- > sical memory to a predetermined dump device, which is typi- > cally a local disk partition. The dump device can be config- > ured by way of dumpadm. Once the crash dump has been written > to the dump device, the system will reboot. > > Fatal operating system errors can be caused by bugs in the | > operating system, its associated device drivers and loadable | > modules, or by faulty hardware. Whatever the cause, the | > crash dump itself provides invaluable information to your | > support engineer to aid in diagnosing the problem. As such, | > it is vital that the crash dump be retrieved and given to | > your support provider. Following an operating system crash, | > the savecore(1M) utility is executed automatically during | > boot to retrieve the crash dump from the dump device, and | > write it to your file system in compressed form to a file | > name vmdump.X, where X is an integer identifying the dump. | > Afterwards, savecore(1M) can be invoked on the same or | > another system to expand the compressed crash dump to a pair | > of files named unix.X and vmcore.X. The directory in which | > the crash dump is saved on reboot can also be configured | > using dumpadm. > > > For systems with a UFS root file system, the default dump | > device is configured to be an appropriate swap partition. | > Swap partitions are disk partitions reserved as virtual | > memory backing store for the operating system. Thus, no per- | > manent information resides in swap to be overwritten by the | > dump. See swap(1M). For systems with a ZFS root file system, | > dedicated ZFS volumes are used for swap and dump areas. For | > further information about setting up a dump area with ZFS, | > see the ZFS Administration Guide. To view the current dump | > configuration, use the dumpadm command with no arguments: > > example# dumpadm > > Dump content: kernel pages > Dump device: /dev/dsk/c0t0d0s1 (swap) > Savecore directory: /var/crash/saturn > Savecore enabled: yes > Save compressed: yes | > > When no options are specified, dumpadm prints the current | > crash dump configuration. The example shows the set of | > default values: the dump content is set to kernel memory | > pages only, the dump device is a swap disk partition, the | > directory for savecore files is set to /var/crash/hostname. | > savecore(1M) is set to run automatically on reboot and save | > the crash dump in a compressed format. > > -z y | n | > Modify the dump configuration to control the operation | > of savecore on reboot. The options are y (yes) to enable | > saving core files in a compressed format, and n (no) | > automatically uncompress the crash dump file. The | > default is yes, because crash dump files can be very | > large and will require less file system space if saved | > in a compressed format. | > > > EXAMPLES > Example 1 Reconfiguring The Dump Device To A Dedicated Dump > Device: > > The following command reconfigures the dump device to a > dedicated dump device: > > example# dumpadm -d /dev/dsk/c0t2d0s2 > > Dump content: kernel pages > Dump device: /dev/dsk/c0t2d0s2 (dedicated) > Savecore directory: /var/crash/saturn > Savecore enabled: yes > Save compressed: yes | > > A.2 Man pages for savecore(1M) > > System Administration Commands savecore(1M) > > NAME > savecore - save a crash dump of the operating system > > SYNOPSIS > /usr/bin/savecore [-Lvd] [-f dumpfile] [directory] > > > DESCRIPTION > The savecore utility saves a crash dump of the kernel > (assuming that one was made) and writes a reboot message in > the shutdown log. It is invoked by the dumpadm service each > time the system boots. > > savecore can be configured by dumpadm(1M) to save crash dump | > data in either a compressed or uncompressed format. For the | > compressed format, savecore saves the crash dump data in the | > file directory/vmdump.N, where N in the name is replaced by | > a number which grows every time savecore is run in that | > directory. The compressed file can be uncompressed in a | > separate step using the -f dumpfile option. For the | > uncompressed format, savecore saves the crash dump data in | > the file directory/vmcore.N and the kernel's namelist in | > directory/unix.N. > > OPTIONS > > -f dumpfile Save a crash dump from the specified file | > instead of from the system's current dump | > device. When given directory/vmdump.N, | > uncompress the file to vmcore.N and unix.N, | > where N is the same number as in the | > compressed name. | > > This option may also be useful if the infor- | > mation stored on the dump device has been | > copied to an on-disk file by means of the | > dd(1M) command. > > FILES > directory/vmdump.n | > directory/vmcore.n > directory/unix.n > > From Alan.Hargreaves@sun.com Tue Aug 11 23:50:13 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C6oCkZ015084 for ; Tue, 11 Aug 2009 23:50:13 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7C6o0WI023034 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 07:50:12 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900D1N4BL4L00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 11 Aug 2009 23:50:09 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO900BKC4BJQXD0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 11 Aug 2009 23:50:08 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7C6o7lS019693 for ; Wed, 12 Aug 2009 06:50:07 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO90080041V1S00@mail-apac.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 14:50:07 +0800 (SGT) Received: from [129.158.12.60] ([unknown] [129.158.12.60]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KO900AO84BGMJI0@mail-apac.sun.com>; Wed, 12 Aug 2009 14:50:07 +0800 (SGT) Date: Wed, 12 Aug 2009 16:48:42 +1000 From: Alan Hargreaves Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Alan.Hargreaves@sun.com To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <4A8265CA.1020907@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.22 (X11/20090706) Status: RO Content-Length: 12105 It's not completely clear to me from the notes whether or not to uncompress a vmdump.N needs to be done on the machine that generated the vmdump.N, or if it can be done anywhere else. From a support perspective, the latter would be nice. i.e. Customer uploads a vmcore.N to us and we uncompress it. Regards, Alan Hargreaves Sherry Moore wrote: > I am sponsoring this fasttrack for Dave Plauger and Steve Sistare. The > timer is set for next Tuesday August 18, 2009. Requesting Micro/Patch > binding. > > The one-pager, project specification and man page diffs are available > in the materials directory. > > Sherry Moore > > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > > 1. Introduction > 1.1. Project/Component Working Name: > Fast Crash Dump > 1.2. Name of Document Author/Supplier: > Dave Plauger > Steve Sistare > 1.3. Date of This Document: > August 11, 2009 > > 4. Technical Description: > 4.1. Details: > New command line flags to dumpadm(1M) control how core files > are to be saved. The setting is saved across reboots in > /etc/dumpadm.conf. The new default behavior is to save files > in compressed format instead of always uncompressing them, as > it does currently. > > If saving compressed, savecore(1M) copies the core file from > the dump device to vmdump.N, where N is the usual dump > integer. Copying a core file is much faster than uncompressing > it into unix.N and vmcore.N images; and it takes up much less > disk space. On systems that dump to the swap area there is > less risk that the core image will be over-written by swap > activity before it can be extracted. > > savecore(1M) performance has been improved by reading the dump > file with fread(3C) instead of pread(2). > > The compressed core dump format is largely unchanged. The dump > header and the dump version number are unchanged. However, > there are changes to the way memory pages are saved within the > compressed file. A compressed dump file must be uncompressed > first, by manually running savecore(1M) a second time, before > it can be used by other tools. Therefore, there is no impact > on mdb(1) due to the changes in compression methods. Once the > core dump has been uncompressed, the resulting unix.N and > vmcore.N files are in the same format as before. > > This project has introduced a bzip2 compression library [2] > into Solaris common code, where it is shared by savecore(1M) > and the kernel. The current lzjb compression algorithm is much > faster, but also much weaker. The bzip2 library requires much > more memory and compute resources. If these resources are > available during panic, the kernel will save memory pages with > bzip2 instead of lzjb, and savecore(1M) will use the same > bzip2 library in order to uncompress the pages. > > The kernel function dumpsys() does most of the work in > creating core dump images. The section that saves memory pages > has been expanded to support parallelism. Most kernel services > are not available during panic. Instead, CPUs spinning in > panic_idle() call into dumpsys() and coordinate via > memory. These helper CPUs copy pages, compress them, and > produce streams of compression data that savecore(1M) can > uncompress. The panic CPU acts as the master and does all page > mapping and I/O operations. > > There are two compression modes in this implementation. The > older method, lzjb is the default on smaller systems. With up > to 4 CPUs, it usually speeds up dump by 2-4 times. The new > bzip2 library is employed on large systems with many spare > CPUs and memory. This can speed up dumps by 4-10 times. The > mode is chosen at crash time based on processor type, number > of CPUs, and available free memory for buffers. > > The existing savecore -L (live dump) option creates a dump > image on a running system. This option is available only when > there is a dedicated dump device. In this case, the dump > helpers in the kernel run as system tasks. > > file(1) can detect the new compressed format. For example, > # file vmcore.0 > vmcore.0: SunOS 5.11 snv_81 64-bit SPARC crash dump from 'oaf415' > # file vmdump.0 > vmdump.0: SunOS 5.11 snv_81 64-bit SPARC compressed crash dump from 'oaf415' > > For more information, see the project specification in the > materials directory. > > 4.2. Bug/RFE Number(s): > > RFE 6828976 Fast Crash Dump > > 4.5. Interfaces: > > New command line flags to dumpadm(1M). And additions to the > meaning of existing flags to savecore(1M). > > Micro/patch binding requested. > > INTERFACE COMMITMENT LEVEL COMMENT > > dumpadm -z (1M) Committed Enables save compressed, or not. > > > 4.6. Doc Impact: > > Man page changes for dumpadm(1M) and savecore(1M). > > See appendix A for diffs. > > 6. Resources and Schedule: > > 6.4. Product Approval Committee requested information: > 6.4.1. Consolidation or Component Name: > OS/Net > > 6.5. ARC review type: Fasttrack > > 6.6. ARC Exposure: Open. > > A. Man pages > A.1 dumpadm(1M) > A.2 savecore(1M) > > A.1 Man pages dumpadm(1M) > > > System Administration Commands dumpadm(1M) > > NAME > dumpadm - configure operating system crash dump > > SYNOPSIS > /usr/sbin/dumpadm [-nuy] [-c content-type] [-d dump-device] > [-m mink | minm | min%] [-s savecore-dir] > [-r root-dir] [-z y | n] | > > DESCRIPTION > The dumpadm program is an administrative command that > manages the configuration of the operating system crash dump > facility. A crash dump is a disk copy of the physical memory > of the computer at the time of a fatal system error. When a > fatal operating system error occurs, a message describing > the error is printed to the console. The operating system > then generates a crash dump by writing the contents of phy- > sical memory to a predetermined dump device, which is typi- > cally a local disk partition. The dump device can be config- > ured by way of dumpadm. Once the crash dump has been written > to the dump device, the system will reboot. > > Fatal operating system errors can be caused by bugs in the | > operating system, its associated device drivers and loadable | > modules, or by faulty hardware. Whatever the cause, the | > crash dump itself provides invaluable information to your | > support engineer to aid in diagnosing the problem. As such, | > it is vital that the crash dump be retrieved and given to | > your support provider. Following an operating system crash, | > the savecore(1M) utility is executed automatically during | > boot to retrieve the crash dump from the dump device, and | > write it to your file system in compressed form to a file | > name vmdump.X, where X is an integer identifying the dump. | > Afterwards, savecore(1M) can be invoked on the same or | > another system to expand the compressed crash dump to a pair | > of files named unix.X and vmcore.X. The directory in which | > the crash dump is saved on reboot can also be configured | > using dumpadm. > > > For systems with a UFS root file system, the default dump | > device is configured to be an appropriate swap partition. | > Swap partitions are disk partitions reserved as virtual | > memory backing store for the operating system. Thus, no per- | > manent information resides in swap to be overwritten by the | > dump. See swap(1M). For systems with a ZFS root file system, | > dedicated ZFS volumes are used for swap and dump areas. For | > further information about setting up a dump area with ZFS, | > see the ZFS Administration Guide. To view the current dump | > configuration, use the dumpadm command with no arguments: > > example# dumpadm > > Dump content: kernel pages > Dump device: /dev/dsk/c0t0d0s1 (swap) > Savecore directory: /var/crash/saturn > Savecore enabled: yes > Save compressed: yes | > > When no options are specified, dumpadm prints the current | > crash dump configuration. The example shows the set of | > default values: the dump content is set to kernel memory | > pages only, the dump device is a swap disk partition, the | > directory for savecore files is set to /var/crash/hostname. | > savecore(1M) is set to run automatically on reboot and save | > the crash dump in a compressed format. > > -z y | n | > Modify the dump configuration to control the operation | > of savecore on reboot. The options are y (yes) to enable | > saving core files in a compressed format, and n (no) | > automatically uncompress the crash dump file. The | > default is yes, because crash dump files can be very | > large and will require less file system space if saved | > in a compressed format. | > > > EXAMPLES > Example 1 Reconfiguring The Dump Device To A Dedicated Dump > Device: > > The following command reconfigures the dump device to a > dedicated dump device: > > example# dumpadm -d /dev/dsk/c0t2d0s2 > > Dump content: kernel pages > Dump device: /dev/dsk/c0t2d0s2 (dedicated) > Savecore directory: /var/crash/saturn > Savecore enabled: yes > Save compressed: yes | > > A.2 Man pages for savecore(1M) > > System Administration Commands savecore(1M) > > NAME > savecore - save a crash dump of the operating system > > SYNOPSIS > /usr/bin/savecore [-Lvd] [-f dumpfile] [directory] > > > DESCRIPTION > The savecore utility saves a crash dump of the kernel > (assuming that one was made) and writes a reboot message in > the shutdown log. It is invoked by the dumpadm service each > time the system boots. > > savecore can be configured by dumpadm(1M) to save crash dump | > data in either a compressed or uncompressed format. For the | > compressed format, savecore saves the crash dump data in the | > file directory/vmdump.N, where N in the name is replaced by | > a number which grows every time savecore is run in that | > directory. The compressed file can be uncompressed in a | > separate step using the -f dumpfile option. For the | > uncompressed format, savecore saves the crash dump data in | > the file directory/vmcore.N and the kernel's namelist in | > directory/unix.N. > > OPTIONS > > -f dumpfile Save a crash dump from the specified file | > instead of from the system's current dump | > device. When given directory/vmdump.N, | > uncompress the file to vmcore.N and unix.N, | > where N is the same number as in the | > compressed name. | > > This option may also be useful if the infor- | > mation stored on the dump device has been | > copied to an on-disk file by means of the | > dd(1M) command. > > FILES > directory/vmdump.n | > directory/vmcore.n > directory/unix.n > > -- Alan Hargreaves - http://blogs.sun.com/tpenta Principal Field Technologist (Kernel/VOSJEC/Performance) Asia Pacific/Emerging Markets Sun Microsystems From iszczesniak@gmail.com Tue Aug 11 23:59:18 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C6xItT015509 for ; Tue, 11 Aug 2009 23:59:18 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7C6xGkr021944; Tue, 11 Aug 2009 23:59:16 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900I094QSWU00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 11 Aug 2009 23:59:16 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO9009TA4QRQ960@nwk-avmta-1.sfbay.Sun.COM>; Tue, 11 Aug 2009 23:59:15 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7C6oLJ6010791; Wed, 12 Aug 2009 06:59:14 +0000 (GMT) Received: from mmp41es.mmp.us.syntegra.com ([160.41.221.10] [160.41.221.10]) by relay42i.sun.com with ESMTP id BT-MMP-134805; Wed, 12 Aug 2009 06:59:14 +0000 (Z) Received: from relay41i.sun.com (relay41i.sun.com [192.5.209.70]) by mmp41es.mmp.us.syntegra.com with ESMTP id BT-MMP-53619372; Wed, 12 Aug 2009 06:59:14 +0000 (Z) Received: from mail-fx0-f223.google.com ([209.85.220.223] [209.85.220.223]) by relay4i.sun.com with ESMTP id BT-MMP-16326412; Wed, 12 Aug 2009 06:59:14 +0000 (Z) Received: by fxm23 with SMTP id 23so3687881fxm.6 for ; Tue, 11 Aug 2009 23:58:24 -0700 (PDT) Received: by 10.204.100.11 with SMTP id w11mr6051612bkn.110.1250060304567; Tue, 11 Aug 2009 23:58:24 -0700 (PDT) Date: Wed, 12 Aug 2009 08:58:24 +0200 From: "I. Szczesniak" Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> To: Sherry Moore Cc: psarc-ext@sun.com, Steven Sistare , David Plauger Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=scQUNc37YRML3YOsd6q5+0HDPjZKj+lOiGQv5W6vVgo=; b=meK3QHK5qXsOLMRdOXHL3VtJ8YtF/9iVBdq+wpLtCRCxfP7rC5yfk0fVF8E9olEA29 dDS0l4CB3ql0PdZuU764VxPkZhPMkTFNpvf/M6oNs0Ts3JC11Aacfw1K7imOEWXy4oCd H3flVAxtThX2f2t8r1QEkgt+YrZ+2Og8mmEHg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=MRnriXqFdjpw5PFGZuzrEUnOBCNNo66/lW9QBfoPh2APHEXgrVWcMuEFrhC2OM1RtG oZwDRPn6kj7H1bOR2roE48u2yg3lu6f2zdJoL3Oz712iHX681x5sgBBy3/L6L6BmocGI 7Wd0mt0WlsiY/kHtEipsOSc9BdiwnCgv4la9k= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=-1.1/5.0, scanned in 0.070sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> Status: RO Content-Length: 1880 On 8/12/09, Sherry Moore wrote: > I am sponsoring this fasttrack for Dave Plauger and Steve Sistare. The > timer is set for next Tuesday August 18, 2009. Requesting Micro/Patch > binding. > > The one-pager, project specification and man page diffs are available > in the materials directory. > > Sherry Moore > > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > > 1. Introduction > 1.1. Project/Component Working Name: > Fast Crash Dump > 1.2. Name of Document Author/Supplier: > Dave Plauger > Steve Sistare > 1.3. Date of This Document: > August 11, 2009 > > 4. Technical Description: > 4.1. Details: > New command line flags to dumpadm(1M) control how core files > are to be saved. The setting is saved across reboots in > /etc/dumpadm.conf. The new default behavior is to save files > in compressed format instead of always uncompressing them, as > it does currently. > > If saving compressed, savecore(1M) copies the core file from > the dump device to vmdump.N, where N is the usual dump > integer. Copying a core file is much faster than uncompressing > it into unix.N and vmcore.N images; and it takes up much less > disk space. On systems that dump to the swap area there is > less risk that the core image will be over-written by swap > activity before it can be extracted. > > savecore(1M) performance has been improved by reading the dump > file with fread(3C) instead of pread(2). Sherry, the stdio implementation in libc is among the slowest stdio versions out there. If you want to archive better performance you should use the stdio implementation in libast or use mmap(2). Irek From cyril.plisko@gmail.com Wed Aug 12 00:02:50 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C72o5B008162 for ; Wed, 12 Aug 2009 00:02:50 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7C72nQD019100; Wed, 12 Aug 2009 00:02:49 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900E014WP3S00@nwk-avmta-2.sfbay.sun.com>; Wed, 12 Aug 2009 00:02:49 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO900BU34WOR3D0@nwk-avmta-2.sfbay.sun.com>; Wed, 12 Aug 2009 00:02:48 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7C6sZZV007096; Wed, 12 Aug 2009 07:02:48 +0000 (GMT) Received: from mmp11es.mmp.us.syntegra.com ([160.41.208.11] [160.41.208.11]) by relay11i.sun.com with ESMTP id BT-MMP-57942; Wed, 12 Aug 2009 07:02:48 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp11es.mmp.us.syntegra.com with ESMTP id BT-MMP-46931230; Wed, 12 Aug 2009 07:02:30 +0000 (Z) Received: from mail-ew0-f223.google.com ([209.85.219.223] [209.85.219.223]) by relay1i.sun.com with ESMTP id BT-MMP-4656554; Wed, 12 Aug 2009 07:02:30 +0000 (Z) Received: by mail-ew0-f223.google.com with SMTP id 23so4431150ewy.8 for ; Wed, 12 Aug 2009 00:02:25 -0700 (PDT) Received: by 10.216.90.6 with SMTP id d6mr1422253wef.95.1250060545272; Wed, 12 Aug 2009 00:02:25 -0700 (PDT) Date: Wed, 12 Aug 2009 10:02:05 +0300 From: Cyril Plisko Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: cyril.plisko@gmail.com To: Sherry Moore Cc: psarc-ext@sun.com, Steven Sistare , David Plauger Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:from:date:x-google-sender-auth:message-id:subject:to:cc :content-type:content-transfer-encoding; bh=9ovVw/xc7Bne4xP4t7/IRQCr7YaY/+1egH8QxTHD0Iw=; b=m/5TnTkOiiT+oTCuOfHdxr9qFqayMArMmwybcQ6sSXShDD7B+N+asFaFhp51oJlFJa 7s86TN1Hp4soBk3ureNycqPkU7PFyaX++d04l1gbVVu7Qknb2hKT3Rj0IjlZu4lZahbB HVJe/MeWbzl0ttJ189Xf1qWfuZsdDKsp2vmb4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=RhfmhKFVe1322HavKTSDsaqNZjrYyXaOypHYrAosy0iKe1V7EnGAkcDPBA08cYyAGR e9Df4H02SOvuTrPiYw0FXzGAHwWwwMvs5X1AxowFd/wmz7COb7BP1cKCDXO1l5iTdbPl 4mnYtjTcJ2zWkoADlyx2cgdoOPs509hUL6PLw= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Google-Sender-Auth: 46bab0536336de9f X-Antispam: No, score=0.0/5.0, scanned in 8.931sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sac.sfbay.sun.com id n7C72o5B008162 Status: RO Content-Length: 604 On Wed, Aug 12, 2009 at 9:28 AM, Sherry Moore wrote: > I am sponsoring this fasttrack for Dave Plauger and Steve Sistare.  The > timer is set for next Tuesday August 18, 2009.  Requesting Micro/Patch > binding. > > The one-pager, project specification and man page diffs are available > in the materials directory. > > Sherry Moore > Hello, I see that case materials (design document) mention other compression algorithms. - " Several compression methods were compared before bzip2 was chosen. " Can you please share the results of that comparison ? Regards, Cyril From iszczesniak@gmail.com Wed Aug 12 00:15:39 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C7FcfQ019896 for ; Wed, 12 Aug 2009 00:15:38 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7C7FSIN017926; Wed, 12 Aug 2009 15:15:33 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900L0R5HT3Q00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 12 Aug 2009 00:15:29 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO9009PJ5HSQ980@nwk-avmta-1.sfbay.Sun.COM>; Wed, 12 Aug 2009 00:15:29 -0700 (PDT) Received: from relay41i.sun.com ([192.5.209.70]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7C7DprZ018836; Wed, 12 Aug 2009 07:15:28 +0000 (GMT) Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay41i.sun.com with ESMTP id BT-MMP-2387763; Wed, 12 Aug 2009 07:15:28 +0000 (Z) Received: from relay41i.sun.com (relay41i.sun.com [192.5.209.70]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-53534872; Wed, 12 Aug 2009 07:15:27 +0000 (Z) Received: from mail-bw0-f211.google.com ([209.85.218.211] [209.85.218.211]) by relay4i.sun.com with ESMTP id BT-MMP-16345787; Wed, 12 Aug 2009 07:15:27 +0000 (Z) Received: by bwz7 with SMTP id 7so3533998bwz.8 for ; Wed, 12 Aug 2009 00:14:36 -0700 (PDT) Received: by 10.204.112.16 with SMTP id u16mr7871787bkp.121.1250061276550; Wed, 12 Aug 2009 00:14:36 -0700 (PDT) Date: Wed, 12 Aug 2009 09:14:36 +0200 From: "I. Szczesniak" Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A8265CA.1020907@Sun.COM> To: Alan Hargreaves Cc: Sherry Moore , Steven Sistare , psarc-ext@sun.com, David Plauger Message-id: MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7BIT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=f7DJsLIqUEFH5L5fDAraroMP+UuJnjV8cGzim8MEJMY=; b=L2cBBT0Yu/JIYKyVvmNhidscuZEKgdjEXbOGLTIKOn/Zk8hK9Yk7Vw9sEJZ1gb4Xct Qre4mpr2TtKWNpIHDvul2xaO4mRqoxTxLriFF3OacnNjv0Yvwcqr4WaB6qxOdCLULA0I 5m1qXfRzFUZLwKoP0qZFxuIVOrMYE3VpWI8m8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ZcXhUL8qCmqi0WpV+hPSN90k54x9gHyiggFBC4JobkuRkPqIhPT7oTUk4fyyYwTIHj SXRwu8RZl64kzsprNYOHEz6YKjfssmVgU3wZd5Pey0TmgGkSKgTtEr99r+8NTIBRMMqK dEQynAL/qza7X/E38W4GqXjHZWH/LSD/JPSe0= X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 0.055sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> <4A8265CA.1020907@Sun.COM> Status: RO Content-Length: 636 On 8/12/09, Alan Hargreaves wrote: > It's not completely clear to me from the notes whether or not to uncompress > a vmdump.N needs to be done on the machine that generated the vmdump.N, or > if it can be done anywhere else. From a support perspective, the latter > would be nice. i.e. Customer uploads a vmcore.N to us and we uncompress it. Storing the file as compressed data is not as easy as you think. You will need a specialized unpack command as the stock version of /usr/bin/bunzip does not handle sparse files, vmdump.N is sparse and mdb will no longer be able to access the files via mmap(2). Irek From Alan.Hargreaves@sun.com Wed Aug 12 00:19:17 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C7JHtR019933 for ; Wed, 12 Aug 2009 00:19:17 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7C7JE9i000328 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 00:19:16 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900D075O4I900@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 01:19:16 -0600 (MDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO9001XC5O27VA0@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 01:19:15 -0600 (MDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7C7JDnG003248 for ; Wed, 12 Aug 2009 07:19:13 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO9002005KM4O00@mail-apac.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 15:19:13 +0800 (SGT) Received: from [129.158.12.60] ([unknown] [129.158.12.60]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KO900HDP5NX2UH0@mail-apac.sun.com>; Wed, 12 Aug 2009 15:19:12 +0800 (SGT) Date: Wed, 12 Aug 2009 17:17:46 +1000 From: Alan Hargreaves Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: Sender: Alan.Hargreaves@sun.com To: "I. Szczesniak" Cc: Sherry Moore , Steven Sistare , psarc-ext@sun.com, David Plauger Message-id: <4A826C9A.4000402@Sun.COM> MIME-version: 1.0 Content-type: text/html; CHARSET=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A8265CA.1020907@Sun.COM> User-Agent: Thunderbird 2.0.0.22 (X11/20090706) Status: RO Content-Length: 1636

What I was getting at was, do I ned to run savecore a second time on the machine that ghenerated the dump; or could I run savecore elsewhere.

alan.


I. Szczesniak wrote: 
On 8/12/09, Alan Hargreaves <Alan.Hargreaves@sun.com> wrote:
  
It's not completely clear to me from the notes whether or not to uncompress
a vmdump.N needs to be done on the machine that generated the vmdump.N, or
if it can be done anywhere else. From a support perspective, the latter
would be nice. i.e. Customer uploads a vmcore.N to us and we uncompress it.
    

Storing the file as compressed data is not as easy as you think. You
will need a specialized unpack command as the stock version of
/usr/bin/bunzip does not handle sparse files, vmdump.N is sparse and
mdb will no longer be able to access the files via mmap(2).

Irek
  

-- 
Alan Hargreaves - http://blogs.sun.com/tpenta
Principal Field Technologist (Kernel/VOSJEC/Performance)
Asia Pacific/Emerging Markets
Sun Microsystems
From Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com Wed Aug 12 02:01:14 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C91Ciq022192 for ; Wed, 12 Aug 2009 02:01:13 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7C916WG012828 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 10:01:12 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900M01ADZAL00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 02:01:11 -0700 (PDT) Received: from sca-ea-mail-4.sun.com ([192.18.43.22]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO900FUYADYUB70@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 02:01:11 -0700 (PDT) Received: from relay15i.sun.com (ip125.net129179-4.block1.us.syntegra.com [129.179.4.125]) by sca-ea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7C8v69K015321 for ; Wed, 12 Aug 2009 09:01:10 +0000 (GMT) Received: from mmp13es.mmp.us.syntegra.com ([160.41.208.13] [160.41.208.13]) by relay15i.sun.com with ESMTP id BT-MMP-155663 for psarc-ext@sun.com; Wed, 12 Aug 2009 09:01:10 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp13es.mmp.us.syntegra.com with ESMTP id BT-MMP-51151317 for psarc-ext@sun.com; Wed, 12 Aug 2009 09:01:07 +0000 (Z) Received: from relay02-haj2.antispameurope.com ([83.246.65.52] [83.246.65.52]) by relay1i.sun.com with ESMTP id BT-MMP-2078466 for psarc-ext@sun.com; Wed, 12 Aug 2009 09:00:56 +0000 (Z) Received: by relay02-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id BD7B26F0558; Wed, 12 Aug 2009 11:00:53 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay02-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 5C95D6F054E; Wed, 12 Aug 2009 11:00:51 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.2/8.14.2) with SMTP id n7C90pcE024505; Wed, 12 Aug 2009 11:00:51 +0200 (MEST) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Wed, 12 Aug 2009 11:00:51 +0200 Date: Wed, 12 Aug 2009 11:00:17 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: Sender: Joerg.Schilling9ab33xy531fokus.fraunhofer.de@bounce.antispameurope.com To: iszczesniak@gmail.com, Alan.Hargreaves@sun.com Cc: Steve.Sistare@sun.com, Sherry.Moore@sun.com, psarc-ext@sun.com, Dave.Plauger@sun.com Message-id: <4a8284a1.+z9CbTbSdzNjazZg%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-Antispam: No, score=0.0/5.0, scanned in 3.037sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> <4A8265CA.1020907@Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 12 Aug 2009 09:00:51.0920 (UTC) FILETIME=[649F8500:01CA1B2B] Status: RO Content-Length: 1382 "I. Szczesniak" wrote: > On 8/12/09, Alan Hargreaves wrote: > > It's not completely clear to me from the notes whether or not to uncompress > > a vmdump.N needs to be done on the machine that generated the vmdump.N, or > > if it can be done anywhere else. From a support perspective, the latter > > would be nice. i.e. Customer uploads a vmcore.N to us and we uncompress it. > > Storing the file as compressed data is not as easy as you think. You > will need a specialized unpack command as the stock version of > /usr/bin/bunzip does not handle sparse files, vmdump.N is sparse and > mdb will no longer be able to access the files via mmap(2). There us bunzip2 or gunzip, which rogran are you talking about? bzip2 compresses null bytes in an efficient way and it would not be hard to add support for doing a lseek() instead of a write if a block of uncompressed data appears to contain only nulls. BTW: this is the approach I used as the first attempt to support sparse files with star 16 years ago (see option -force-hole). Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From Darren.Moffat@sun.com Wed Aug 12 02:21:47 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7C9LkC2022485 for ; Wed, 12 Aug 2009 02:21:47 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7C9LdBg021527 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 17:21:45 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900301BC7FX00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 03:21:43 -0600 (MDT) Received: from gmp-eb-inf-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO900MGKBC6GF60@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 03:21:43 -0600 (MDT) Received: from fe-emea-10.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7C9LgmL014829 for ; Wed, 12 Aug 2009 09:21:42 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO900J00AY19500@fe-emea-10.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 10:21:31 +0100 (BST) Received: from [192.168.1.105] (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150]) by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KO900J85BBTI480@fe-emea-10.sun.com>; Wed, 12 Aug 2009 10:21:29 +0100 (BST) Date: Wed, 12 Aug 2009 10:21:21 +0100 From: Darren J Moffat Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Darren.Moffat@sun.com To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <4A828991.3020405@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090623) Status: RO Content-Length: 946 Sherry Moore wrote: > -z y | n | > Modify the dump configuration to control the operation | > of savecore on reboot. The options are y (yes) to enable | > saving core files in a compressed format, and n (no) | > automatically uncompress the crash dump file. The | > default is yes, because crash dump files can be very | > large and will require less file system space if saved | > in a compressed format. | That is a very clunky interface. I suspect it is partly that way because "-y and -n" are already used in dumpadm(1M) to determine if savecore should run or not and -u is already used to mean update. Instead of "-z y" what about -Z Instead of "-z n" what about -U Or "-S compress" "-S uncompress" "-S" denoting "savecore options". -- Darren J Moffat From Dave.Plauger@sun.com Wed Aug 12 04:42:28 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7CBgSfo027678 for ; Wed, 12 Aug 2009 04:42:28 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7CBgRrP039085 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 12 Aug 2009 05:42:27 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KO900A09HUQXH00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 04:42:26 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KO9006PLHUQEKA0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 04:42:26 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7CBgQST015293 for ; Wed, 12 Aug 2009 11:42:26 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KO900400HSIHG00@mail-amer.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 12 Aug 2009 05:42:26 -0600 (MDT) Received: from [129.148.131.222] ([unknown] [129.148.131.222]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KO900KMIHUP6B60@mail-amer.sun.com>; Wed, 12 Aug 2009 05:42:26 -0600 (MDT) Date: Wed, 12 Aug 2009 07:42:25 -0400 From: Dave.Plauger@sun.com Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A8265CA.1020907@Sun.COM> Sender: Dave.Plauger@sun.com To: Alan Hargreaves Cc: Sherry Moore , psarc-ext@sun.com, Steven Sistare Message-id: <4A82AAA1.8090605@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A8265CA.1020907@Sun.COM> User-Agent: Thunderbird 2.0.0.14 (X11/20080531) Status: RO Content-Length: 14295 On 08/12/09 02:48, Alan Hargreaves wrote: > It's not completely clear to me from the notes whether or not to > uncompress a vmdump.N needs to be done on the machine that generated > the vmdump.N, or if it can be done anywhere else. From a support > perspective, the latter would be nice. i.e. Customer uploads a > vmcore.N to us and we uncompress it. > > Regards, > Alan Hargreaves Thanks for pointing that out. The notes omitted some of the description. The idea is to be able to move compressed dump files over the network where they can be uncompressed with savecore(1M) on another machine. This is from the the 1-pager: Newer systems have ever larger memory sizes. Some systems are now 1TB and larger. One consequence is that it takes a much longer time to save crash images, and it takes a greater amount of disk space in order to record them. This affects availability: should a panic occur, it takes a longer time to save a memory dump before the system restarts. It also affects serviceability, for two reasons: 1) the larger core files require more reserved disk space, and 2) it takes a longer time to transfer core files through the network for off site analysis. The second is more important, because most core file analysis usually takes place off site, especially for Sun customers. This project approaches the problem in three dimensions: First, the time factor is reduced by introducing parallelism. In the existing implementation, during a crash, all other CPUs except for the one processing the panic are idled. This means there is a huge resource available in untapped processing power. These otherwise idle processors will be employed during the memory compression phase of a core dump. Second, the time to save a crash dump is limited by the I/O rate. This I/O time is reduced by increasing the compression factor. Less bytes to record means less time to save them. Third, crash dumps will be recorded in a compressed format. Compressed dumps require less disk space, and they can be transmitted through the network in shorter time. > > > Sherry Moore wrote: >> I am sponsoring this fasttrack for Dave Plauger and Steve Sistare. The >> timer is set for next Tuesday August 18, 2009. Requesting Micro/Patch >> binding. >> >> The one-pager, project specification and man page diffs are available >> in the materials directory. >> >> Sherry Moore >> >> Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI >> This information is Copyright 2009 Sun Microsystems >> >> 1. Introduction >> 1.1. Project/Component Working Name: Fast Crash Dump >> 1.2. Name of Document Author/Supplier: >> Dave Plauger >> Steve Sistare >> 1.3. Date of This Document: >> August 11, 2009 >> >> 4. Technical Description: >> 4.1. Details: >> New command line flags to dumpadm(1M) control how core files >> are to be saved. The setting is saved across reboots in >> /etc/dumpadm.conf. The new default behavior is to save files >> in compressed format instead of always uncompressing them, as >> it does currently. >> >> If saving compressed, savecore(1M) copies the core file from >> the dump device to vmdump.N, where N is the usual dump >> integer. Copying a core file is much faster than uncompressing >> it into unix.N and vmcore.N images; and it takes up much less >> disk space. On systems that dump to the swap area there is >> less risk that the core image will be over-written by swap >> activity before it can be extracted. >> >> savecore(1M) performance has been improved by reading the dump >> file with fread(3C) instead of pread(2). >> >> The compressed core dump format is largely unchanged. The dump >> header and the dump version number are unchanged. However, >> there are changes to the way memory pages are saved within the >> compressed file. A compressed dump file must be uncompressed >> first, by manually running savecore(1M) a second time, before >> it can be used by other tools. Therefore, there is no impact >> on mdb(1) due to the changes in compression methods. Once the >> core dump has been uncompressed, the resulting unix.N and >> vmcore.N files are in the same format as before. >> >> This project has introduced a bzip2 compression library [2] >> into Solaris common code, where it is shared by savecore(1M) >> and the kernel. The current lzjb compression algorithm is much >> faster, but also much weaker. The bzip2 library requires much >> more memory and compute resources. If these resources are >> available during panic, the kernel will save memory pages with >> bzip2 instead of lzjb, and savecore(1M) will use the same >> bzip2 library in order to uncompress the pages. >> >> The kernel function dumpsys() does most of the work in >> creating core dump images. The section that saves memory pages >> has been expanded to support parallelism. Most kernel services >> are not available during panic. Instead, CPUs spinning in >> panic_idle() call into dumpsys() and coordinate via >> memory. These helper CPUs copy pages, compress them, and >> produce streams of compression data that savecore(1M) can >> uncompress. The panic CPU acts as the master and does all page >> mapping and I/O operations. >> >> There are two compression modes in this implementation. The >> older method, lzjb is the default on smaller systems. With up >> to 4 CPUs, it usually speeds up dump by 2-4 times. The new >> bzip2 library is employed on large systems with many spare >> CPUs and memory. This can speed up dumps by 4-10 times. The >> mode is chosen at crash time based on processor type, number >> of CPUs, and available free memory for buffers. >> >> The existing savecore -L (live dump) option creates a dump >> image on a running system. This option is available only when >> there is a dedicated dump device. In this case, the dump >> helpers in the kernel run as system tasks. >> >> file(1) can detect the new compressed format. For example, >> # file vmcore.0 >> vmcore.0: SunOS 5.11 snv_81 64-bit SPARC crash dump from 'oaf415' >> # file vmdump.0 >> vmdump.0: SunOS 5.11 snv_81 64-bit SPARC compressed crash dump >> from 'oaf415' >> >> For more information, see the project specification in the >> materials directory. >> >> 4.2. Bug/RFE Number(s): >> >> RFE 6828976 Fast Crash Dump >> 4.5. Interfaces: >> >> New command line flags to dumpadm(1M). And additions to the >> meaning of existing flags to savecore(1M). >> >> Micro/patch binding requested. >> >> INTERFACE COMMITMENT LEVEL COMMENT >> >> dumpadm -z (1M) Committed Enables save compressed, or not. >> >> 4.6. Doc Impact: >> Man page changes for dumpadm(1M) and savecore(1M). >> >> See appendix A for diffs. >> >> 6. Resources and Schedule: >> >> 6.4. Product Approval Committee requested information: >> 6.4.1. Consolidation or Component Name: >> OS/Net >> >> 6.5. ARC review type: Fasttrack >> >> 6.6. ARC Exposure: Open. >> >> A. Man pages >> A.1 dumpadm(1M) >> A.2 savecore(1M) >> >> A.1 Man pages dumpadm(1M) >> >> >> System Administration Commands dumpadm(1M) >> >> NAME >> dumpadm - configure operating system crash dump >> >> SYNOPSIS >> /usr/sbin/dumpadm [-nuy] [-c content-type] [-d dump-device] >> [-m mink | minm | min%] [-s savecore-dir] >> [-r root-dir] [-z y | n] | >> >> DESCRIPTION >> The dumpadm program is an administrative command that >> manages the configuration of the operating system crash dump >> facility. A crash dump is a disk copy of the physical memory >> of the computer at the time of a fatal system error. When a >> fatal operating system error occurs, a message describing >> the error is printed to the console. The operating system >> then generates a crash dump by writing the contents of phy- >> sical memory to a predetermined dump device, which is typi- >> cally a local disk partition. The dump device can be config- >> ured by way of dumpadm. Once the crash dump has been written >> to the dump device, the system will reboot. >> >> Fatal operating system errors can be caused by bugs in the | >> operating system, its associated device drivers and loadable | >> modules, or by faulty hardware. Whatever the cause, the | >> crash dump itself provides invaluable information to your | >> support engineer to aid in diagnosing the problem. As such, | >> it is vital that the crash dump be retrieved and given to | >> your support provider. Following an operating system crash, | >> the savecore(1M) utility is executed automatically during | >> boot to retrieve the crash dump from the dump device, and | >> write it to your file system in compressed form to a file | >> name vmdump.X, where X is an integer identifying the dump. | >> Afterwards, savecore(1M) can be invoked on the same or | >> another system to expand the compressed crash dump to a pair | >> of files named unix.X and vmcore.X. The directory in which | >> the crash dump is saved on reboot can also be configured | >> using dumpadm. >> >> >> For systems with a UFS root file system, the default dump | >> device is configured to be an appropriate swap partition. | >> Swap partitions are disk partitions reserved as virtual | >> memory backing store for the operating system. Thus, no per- | >> manent information resides in swap to be overwritten by the | >> dump. See swap(1M). For systems with a ZFS root file system, | >> dedicated ZFS volumes are used for swap and dump areas. For | >> further information about setting up a dump area with ZFS, | >> see the ZFS Administration Guide. To view the current dump | >> configuration, use the dumpadm command with no arguments: >> >> example# dumpadm >> >> Dump content: kernel pages >> Dump device: /dev/dsk/c0t0d0s1 (swap) >> Savecore directory: /var/crash/saturn >> Savecore enabled: yes >> Save compressed: yes | >> >> When no options are specified, dumpadm prints the current | >> crash dump configuration. The example shows the set of | >> default values: the dump content is set to kernel memory | >> pages only, the dump device is a swap disk partition, the | >> directory for savecore files is set to /var/crash/hostname. | >> savecore(1M) is set to run automatically on reboot and save | >> the crash dump in a compressed format. >> >> -z y | n | >> Modify the dump configuration to control the operation | >> of savecore on reboot. The options are y (yes) to enable | >> saving core files in a compressed format, and n (no) | >> automatically uncompress the crash dump file. The | >> default is yes, because crash dump files can be very | >> large and will require less file system space if saved | >> in a compressed format. | >> >> >> EXAMPLES >> Example 1 Reconfiguring The Dump Device To A Dedicated Dump >> Device: >> >> The following command reconfigures the dump device to a >> dedicated dump device: >> >> example# dumpadm -d /dev/dsk/c0t2d0s2 >> >> Dump content: kernel pages >> Dump device: /dev/dsk/c0t2d0s2 (dedicated) >> Savecore directory: /var/crash/saturn >> Savecore enabled: yes >> Save compressed: yes | >> >> A.2 Man pages for savecore(1M) >> >> System Administration Commands savecore(1M) >> >> NAME >> savecore - save a crash dump of the operating system >> >> SYNOPSIS >> /usr/bin/savecore [-Lvd] [-f dumpfile] [directory] >> >> >> DESCRIPTION >> The savecore utility saves a crash dump of the kernel >> (assuming that one was made) and writes a reboot message in >> the shutdown log. It is invoked by the dumpadm service each >> time the system boots. >> >> savecore can be configured by dumpadm(1M) to save crash dump | >> data in either a compressed or uncompressed format. For the | >> compressed format, savecore saves the crash dump data in the | >> file directory/vmdump.N, where N in the name is replaced by | >> a number which grows every time savecore is run in that | >> directory. The compressed file can be uncompressed in a | >> separate step using the -f dumpfile option. For the | >> uncompressed format, savecore saves the crash dump data in | >> the file directory/vmcore.N and the kernel's namelist in | >> directory/unix.N. >> >> OPTIONS >> >> -f dumpfile Save a crash dump from the specified file | >> instead of from the system's current dump | >> device. When given directory/vmdump.N, | >> uncompress the file to vmcore.N and unix.N, | >> where N is the same number as in the | >> compressed name. | >> >> This option may also be useful if the infor- | >> mation stored on the dump device has been | >> copied to an on-disk file by means of the | >> dd(1M) command. >> >> FILES >> directory/vmdump.n | >> directory/vmcore.n >> directory/unix.n >> >> > From Dave.Plauger@Sun.COM Thu Aug 13 13:11:07 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7DKB7si012096 for ; Thu, 13 Aug 2009 13:11:07 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7DKB7w8021840 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 13 Aug 2009 13:11:07 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC0040102JQQ00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Thu, 13 Aug 2009 13:11:07 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC00H8Z02IRZB0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Thu, 13 Aug 2009 13:11:06 -0700 (PDT) Received: from fe-amer-10.sun.com ([192.18.109.80]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7DKB6vA004230 for ; Thu, 13 Aug 2009 20:11:06 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KOB00E00ZA97800@mail-amer.sun.com> for psarc-ext@Sun.COM (ORCPT psarc-ext@Sun.COM); Thu, 13 Aug 2009 14:11:06 -0600 (MDT) Received: from [129.150.178.82] ([unknown] [129.150.178.82]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KOC00DY1026A380@mail-amer.sun.com>; Thu, 13 Aug 2009 14:10:55 -0600 (MDT) Date: Thu, 13 Aug 2009 16:10:52 -0400 From: Dave Plauger Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Dave.Plauger@Sun.COM To: Sherry Moore Cc: psarc-ext@Sun.COM, Steven Sistare Message-id: <4A84734C.9000303@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Status: RO Content-Length: 5485 Thanks to everyone who took the time to comment on our proposal. We have replied to each one in private discussion, and now present this summary of our responses. The first 2 items result in some changes or additions to our proposal. Garrett D'Amore wrote: > +1. (I cheated -- I've seen the case materials ahead of time. :-) > This enhancement (and the need to perform subsequent actions to > decompress the crash dump) probably deserves special mention in the > Release Notes. I'd also add, in retrospect, it seems like perhaps mdb > ought to have its man page updated with at least a passing reference > to compressed crash dumps (and the step required to decompress them.) We agree that the mdb man page should describe vmdump.X, as it already mentions vmcore.X and unix.X. We will provide an updated mdb man page shortly. We'll follow up during integration and make sure that the Release Notes are updated as well. Darren J Moffat wrote: >> -z y | n | >> Modify the dump configuration to control the operation | >> of savecore on reboot. The options are y (yes) to enable | >> saving core files in a compressed format, and n (no) | >> automatically uncompress the crash dump file. The | >> default is yes, because crash dump files can be very | >> large and will require less file system space if saved | >> in a compressed format. | > > That is a very clunky interface. I suspect it is partly that way > because "-y and -n" are already used in dumpadm(1M) to determine if > savecore should run or not and -u is already used to mean update. > > Instead of "-z y" what about -Z > Instead of "-z n" what about -U > > Or "-S compress" "-S uncompress" "-S" denoting "savecore options". We discussed a few alternatives with Darren. We agreed to keep "-z *", because z implies compression in the Unix world, and defining two different flags, with different mnemonics, to enable or disable it is harder to remember for the user. It also consumes 2 flags for every new binary option added, leaving less room for future expansion. We noted that the zfs(1M) utility uses "on | off". We will change our proposal to "-z on | off", and update the dumpadm man page. Alan Hargreaves wrote: > It's not completely clear to me from the notes whether or not to > uncompress a vmdump.N needs to be done on the machine that generated > the vmdump.N, or if it can be done anywhere else. From a support > perspective, the latter would be nice. i.e. Customer uploads a > vmcore.N to us and we uncompress it. > > What I was getting at was, do I ned to run savecore a second time on > the machine that ghenerated the dump; or could I run savecore elsewhere. > Yes, vmdump.N can definitely be moved to another machine and uncompressed there. That is a main goal of the project. The fast track notes did not include some of the description of the project, as the emphasis here is on interfaces. Please see the 1-pager and design spec in the supporting materials at http://arc.opensolaris.org/caselog/PSARC/2009/330/materials. Ivek Szczesniak wrote (same thread): > Storing the file as compressed data is not as easy as you think. You > will need a specialized unpack command as the stock version of > /usr/bin/bunzip does not handle sparse files, vmdump.N is sparse and > mdb will no longer be able to access the files via mmap(2). > savecore(1m) is the specialized unpack command. It can be run on the same machine, or vmdump.N can be uploaded to another machine, and savecore run there. mdb(1) can access the files after they are uncompressed. Jörg Schilling wrote (same thread): > There us bunzip2 or gunzip, which rogran are you talking about? > > bzip2 compresses null bytes in an efficient way and it would not be hard > to add support for doing a lseek() instead of a write if a block of > uncompressed data appears to contain only nulls. The bzip2 library is built into the kernel and savecore. A dump image has several sections, not all of them compressed. So, there is no outside compress/uncompress utility involved. It does indeed compress zeros remarkably well. In addition, savecore does not write out zero pages. This leaves holes in vmcore.N that mdb reads as zeros. Ivek Szczesniak wrote: > The stdio implementation in libc is among the slowest stdio > versions out there. If you want to archive better performance you > should use the stdio implementation in libast or use mmap(2). This is an interesting implementation suggestion, but is outside the scope of PSARC because it does not affect the interfaces being proposed. We did achieve quite a speedup over the old method. We'll take another look. Cyril Plisko wrote: > " Several compression methods were compared before bzip2 was > chosen. > " > > Can you please share the results of that comparison ? > The best resource is here: http://en.wikipedia.org/wiki/Comparison_of_file_archivers We wrote several little compress utilities and ran them against some large core files. Our results showed that lzjb < gzip < bzip2 < 7zip, for compression ratio and CPU time, but 7zip was far too CPU expensive, so we chose bzip2. With enough CPUs, bzip2 can be parallelized such that compression CPU time is not the bottleneck, and we reap the benefits of higher bzip2 compression. From gdamore@sun.com Thu Aug 13 14:23:40 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7DLNd8v013937 for ; Thu, 13 Aug 2009 14:23:39 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7DLNRYY003201 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 14 Aug 2009 05:23:38 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC009033FAPM00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 14:23:34 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC00HK93FARUD0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 14:23:34 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7DLNYue017535 for ; Thu, 13 Aug 2009 14:23:34 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOC00B0036T7400@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 14:23:34 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOC00E9Q3F9W5E0@fe-sfbay-09.sun.com>; Thu, 13 Aug 2009 14:23:34 -0700 (PDT) Date: Thu, 13 Aug 2009 14:23:33 -0700 From: "Garrett D'Amore" Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A84734C.9000303@sun.com> Sender: Garrett.Damore@sun.com To: Dave Plauger Cc: Sherry Moore , psarc-ext@sun.com, Steven Sistare Message-id: <4A848455.8040102@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 810 Dave Plauger wrote: > > > > > Ivek Szczesniak wrote: >> The stdio implementation in libc is among the slowest stdio >> versions out there. If you want to archive better performance you >> should use the stdio implementation in libast or use mmap(2). > This is an interesting implementation suggestion, but is outside the > scope of PSARC because it does not affect the interfaces > being proposed. We did achieve quite a speedup over the old method. > We'll take another look. Using libast might well incur extra PSARC oversight -- are the libast interfaces public? Consolidation private? If they are *project private* then you'll need to get a contract for them. Using mmap() would be free of those issues, and is likely to be the fastest without imposing any new interdependencies. -- Garrett From jonathan.adams@sun.com Thu Aug 13 14:51:48 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7DLpmGr014093 for ; Thu, 13 Aug 2009 14:51:48 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7DLpkdN000868; Thu, 13 Aug 2009 14:51:47 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC00B054QARN00@nwk-avmta-2.sfbay.sun.com>; Thu, 13 Aug 2009 14:51:46 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC00B9K4QACX00@nwk-avmta-2.sfbay.sun.com>; Thu, 13 Aug 2009 14:51:46 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n7DLpkQn869259 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 13 Aug 2009 14:51:46 -0700 (PDT) Received: (from jwadams@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id n7DLpkBX869257; Thu, 13 Aug 2009 14:51:46 -0700 (PDT) Date: Thu, 13 Aug 2009 14:51:46 -0700 From: Jonathan Adams Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A848455.8040102@sun.com> To: "Garrett D'Amore" Cc: Dave Plauger , Sherry Moore , psarc-ext@sun.com, Steven Sistare Message-id: <20090813215146.GN197133@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> <4A848455.8040102@sun.com> X-Authentication-warning: jurassic-x4600.sfbay.sun.com: jwadams set sender to jonathan.adams@sun.com using -f User-Agent: Mutt/1.5.14 (2007-02-12) Status: RO Content-Length: 1262 On Thu, Aug 13, 2009 at 02:23:33PM -0700, Garrett D'Amore wrote: > Dave Plauger wrote: > > > > > > > > > >Ivek Szczesniak wrote: > >>The stdio implementation in libc is among the slowest stdio > >>versions out there. If you want to archive better performance you > >>should use the stdio implementation in libast or use mmap(2). > >This is an interesting implementation suggestion, but is outside the > >scope of PSARC because it does not affect the interfaces > >being proposed. We did achieve quite a speedup over the old method. > >We'll take another look. > > Using libast might well incur extra PSARC oversight -- are the libast > interfaces public? Consolidation private? If they are *project > private* then you'll need to get a contract for them. Using mmap() > would be free of those issues, and is likely to be the fastest without > imposing any new interdependencies. Without knowing about this project, yesterday I prototyped a simple change which buffered input and output from savecore(), and it achieved a 2.5x speedup on an pessimal setup (/var/crash on UFS, dump device on the same disk). I'm codereviewing this wad, so I'll talk to the developer about this. I agree that this doesn't seem architectural to me. Cheers, - jonathan From Brian.Ruthven@sun.com Thu Aug 13 14:53:56 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7DLrtFS014128 for ; Thu, 13 Aug 2009 14:53:55 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7DLrq4w016556 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 14 Aug 2009 05:53:54 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC00B034TTX500@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 14:53:53 -0700 (PDT) Received: from gmp-eb-inf-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC00BDP4TSCX00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 14:53:52 -0700 (PDT) Received: from fe-emea-09.sun.com (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged)) by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7DLrpIa018431 for ; Thu, 13 Aug 2009 21:53:51 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOC006004SY7B00@fe-emea-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 22:53:51 +0100 (BST) Received: from [192.168.0.109] ([unknown] [86.7.210.90]) by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOC00CYU4TQR860@fe-emea-09.sun.com>; Thu, 13 Aug 2009 22:53:51 +0100 (BST) Date: Thu, 13 Aug 2009 22:53:49 +0100 From: Brian Ruthven - Solaris Network Sustaining - Sun UK Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Brian.Ruthven@sun.com To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <4A848B6D.4060403@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.21 (X11/20090623) Status: RO Content-Length: 398 Sherry Moore wrote: > compressed file. A compressed dump file must be uncompressed > first, by manually running savecore(1M) a second time, before > it can be used by other tools. Will vmdump.X be removed automatically after executing the uncompression step? Brian -- Brian Ruthven Solaris Revenue Product Engineering Sun Microsystems UK Sparc House, Guillemont Park, Camberley, GU17 9QG From roland.mainz@nrubsig.org Thu Aug 13 15:44:04 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7DMi4bg015861 for ; Thu, 13 Aug 2009 15:44:04 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7DMi2N1008586; Thu, 13 Aug 2009 16:44:02 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC00B0775D9200@nwk-avmta-1.sfbay.Sun.COM>; Thu, 13 Aug 2009 15:44:01 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC0064375DP520@nwk-avmta-1.sfbay.Sun.COM>; Thu, 13 Aug 2009 15:44:01 -0700 (PDT) Received: from relay42i.sun.com ([192.5.209.72]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7DMaccF011287; Thu, 13 Aug 2009 22:44:01 +0000 (GMT) Received: from mmp42es.mmp.us.syntegra.com ([160.41.221.11] [160.41.221.11]) by relay42i.sun.com with ESMTP id BT-MMP-312051; Thu, 13 Aug 2009 22:44:00 +0000 (Z) Received: from relay41i.sun.com (relay41i.sun.com [192.5.209.70]) by mmp42es.mmp.us.syntegra.com with ESMTP id BT-MMP-56549823; Thu, 13 Aug 2009 22:44:00 +0000 (Z) Received: from mail-in-16.arcor-online.net ([151.189.21.56] [151.189.21.56]) by relay4i.sun.com with ESMTP id BT-MMP-19675738; Thu, 13 Aug 2009 22:44:00 +0000 (Z) Received: from mail-in-19-z2.arcor-online.net (mail-in-19-z2.arcor-online.net [151.189.8.36]) by mx.arcor.de (Postfix) with ESMTP id ED582256D50; Fri, 14 Aug 2009 00:43:58 +0200 (CEST) Received: from mail-in-14.arcor-online.net (mail-in-14.arcor-online.net [151.189.21.54]) by mail-in-19-z2.arcor-online.net (Postfix) with ESMTP id D26A16BD8D; Fri, 14 Aug 2009 00:43:58 +0200 (CEST) Received: from jupiterb48.nrubsig.org (dslb-188-097-010-030.pools.arcor-ip.net [188.97.10.30]) by mail-in-14.arcor-online.net (Postfix) with ESMTPS id 149B128AE1B; Fri, 14 Aug 2009 00:43:56 +0200 (CEST) Received: from nrubsig.org (localhost [127.0.0.1]) by jupiterb48.nrubsig.org (8.13.8+Sun/8.13.8) with ESMTP id n7DMhs15001731; Fri, 14 Aug 2009 00:43:55 +0200 (CEST) Date: Fri, 14 Aug 2009 00:43:54 +0200 From: Roland Mainz Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] Sender: gisburn@jupiterb48.nrubsig.org To: "Garrett D'Amore" Cc: Dave Plauger , Steven Sistare , psarc-ext@sun.com, Sherry Moore Message-id: <4A84972A.1DC4BB3D@nrubsig.org> MIME-version: 1.0 X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.11 sun4u) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-14.arcor-online.net 149B128AE1B X-Antispam: No, score=0.0/5.0, scanned in 0.069sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> <4A848455.8040102@sun.com> Status: RO Content-Length: 1273 Garrett D'Amore wrote: > Dave Plauger wrote: > > Ivek Szczesniak wrote: > >> The stdio implementation in libc is among the slowest stdio > >> versions out there. If you want to archive better performance you > >> should use the stdio implementation in libast or use mmap(2). > > This is an interesting implementation suggestion, but is outside the > > scope of PSARC because it does not affect the interfaces > > being proposed. We did achieve quite a speedup over the old method. > > We'll take another look. > > Using libast might well incur extra PSARC oversight -- are the libast > interfaces public? Consolidation private? If they are *project > private* then you'll need to get a contract for them. Using mmap() > would be free of those issues, and is likely to be the fastest without > imposing any new interdependencies. Erm... I think it shouldn't be a problem to make the |malloc()| and parts of libast "consolidation private" for use within OS/Net (e.g. this are the most stable APIs of libast and very unlikely to change) ... ... question is: How do I do that ? ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz@nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) From gdamore@sun.com Thu Aug 13 21:45:53 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7E4jruG001413 for ; Thu, 13 Aug 2009 21:45:53 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7E4jqSM006761 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 13 Aug 2009 21:45:52 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC00201NWGQP00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 22:45:52 -0600 (MDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC00L9LNWF5P60@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 22:45:52 -0600 (MDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n7E4jpAa017574 for ; Thu, 13 Aug 2009 21:45:51 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOC00G00NVX5500@fe-sfbay-09.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 13 Aug 2009 21:45:51 -0700 (PDT) Received: from [192.168.251.11] ([unknown] [76.93.15.33]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOC00MRONWFWBF0@fe-sfbay-09.sun.com>; Thu, 13 Aug 2009 21:45:51 -0700 (PDT) Date: Thu, 13 Aug 2009 21:45:50 -0700 From: "Garrett D'Amore" Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A84972A.1DC4BB3D@nrubsig.org> Sender: Garrett.Damore@sun.com To: Roland Mainz Cc: Dave Plauger , Steven Sistare , psarc-ext@sun.com, Sherry Moore Message-id: <4A84EBFE.9010001@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> <4A848455.8040102@sun.com> <4A84972A.1DC4BB3D@nrubsig.org> User-Agent: Thunderbird 2.0.0.18 (X11/20081201) Status: RO Content-Length: 1394 Roland Mainz wrote: > Garrett D'Amore wrote: > >> Dave Plauger wrote: >> >>> Ivek Szczesniak wrote: >>> >>>> The stdio implementation in libc is among the slowest stdio >>>> versions out there. If you want to archive better performance you >>>> should use the stdio implementation in libast or use mmap(2). >>>> >>> This is an interesting implementation suggestion, but is outside the >>> scope of PSARC because it does not affect the interfaces >>> being proposed. We did achieve quite a speedup over the old method. >>> We'll take another look. >>> >> Using libast might well incur extra PSARC oversight -- are the libast >> interfaces public? Consolidation private? If they are *project >> private* then you'll need to get a contract for them. Using mmap() >> would be free of those issues, and is likely to be the fastest without >> imposing any new interdependencies. >> > > Erm... I think it shouldn't be a problem to make the |malloc()| and > parts of libast "consolidation private" for use within OS/Net > (e.g. this are the most stable APIs of libast and very unlikely to > change) ... > > ... question is: How do I do that ? > Lets find out if this is even necessary. At this point in time, I'd prefer to avoid creating this dependency; mmap() is IMO a potentially superior choice. - Garrett > ---- > > Bye, > Roland > > From roland.mainz@nrubsig.org Thu Aug 13 22:17:07 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7E5H6CE001991 for ; Thu, 13 Aug 2009 22:17:07 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7E5GtOD014857; Fri, 14 Aug 2009 13:17:00 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC00M03PCAD200@nwk-avmta-2.sfbay.sun.com>; Thu, 13 Aug 2009 22:16:58 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC009DIPCANA60@nwk-avmta-2.sfbay.sun.com>; Thu, 13 Aug 2009 22:16:58 -0700 (PDT) Received: from relay11i.sun.com (ip121.net129179-4.block1.us.syntegra.com [129.179.4.121]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7E5GqHH028059; Fri, 14 Aug 2009 05:16:58 +0000 (GMT) Received: from mmp12es.mmp.us.syntegra.com ([160.41.208.12] [160.41.208.12]) by relay11i.sun.com with ESMTP id BT-MMP-230198; Fri, 14 Aug 2009 05:16:53 +0000 (Z) Received: from relay15i.sun.com (relay15i.sun.com [129.179.4.125]) by mmp12es.mmp.us.syntegra.com with ESMTP id BT-MMP-51011222; Fri, 14 Aug 2009 05:16:52 +0000 (Z) Received: from mail-in-02.arcor-online.net ([151.189.21.42] [151.189.21.42]) by relay1i.sun.com with ESMTP id BT-MMP-1362236; Fri, 14 Aug 2009 05:15:03 +0000 (Z) Received: from mail-in-11-z2.arcor-online.net (mail-in-11-z2.arcor-online.net [151.189.8.28]) by mx.arcor.de (Postfix) with ESMTP id A9C213FE020; Fri, 14 Aug 2009 07:14:30 +0200 (CEST) Received: from mail-in-14.arcor-online.net (mail-in-14.arcor-online.net [151.189.21.54]) by mail-in-11-z2.arcor-online.net (Postfix) with ESMTP id 909D334832A; Fri, 14 Aug 2009 07:14:30 +0200 (CEST) Received: from jupiterb48.nrubsig.org (dslb-188-097-010-030.pools.arcor-ip.net [188.97.10.30]) by mail-in-14.arcor-online.net (Postfix) with ESMTPS id 3BA1928AE09; Fri, 14 Aug 2009 07:14:28 +0200 (CEST) Received: from nrubsig.org (localhost [127.0.0.1]) by jupiterb48.nrubsig.org (8.13.8+Sun/8.13.8) with ESMTP id n7E5EQR0001819; Fri, 14 Aug 2009 07:14:27 +0200 (CEST) Date: Fri, 14 Aug 2009 07:14:26 +0200 From: Roland Mainz Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] Sender: gisburn@jupiterb48.nrubsig.org To: "Garrett D'Amore" Cc: Dave Plauger , Steven Sistare , psarc-ext@sun.com, Sherry Moore Message-id: <4A84F2B2.9A8A2F@nrubsig.org> MIME-version: 1.0 X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.11 sun4u) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en X-PMX-Version: 5.4.1.325704 X-Brightmail-Tracker: AAAAAA== X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-14.arcor-online.net 3BA1928AE09 X-Antispam: No, score=0.0/5.0, scanned in 0.179sec at (localhost [127.0.0.1]) by smf-spamd v1.3.1 - http://smfs.sf.net/ References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> <4A848455.8040102@sun.com> <4A84972A.1DC4BB3D@nrubsig.org> <4A84EBFE.9010001@sun.com> Status: RO Content-Length: 1705 Garrett D'Amore wrote: > Roland Mainz wrote: > > Garrett D'Amore wrote: > >> Dave Plauger wrote: > >>> Ivek Szczesniak wrote: > >>>> The stdio implementation in libc is among the slowest stdio > >>>> versions out there. If you want to archive better performance you > >>>> should use the stdio implementation in libast or use mmap(2). > >>>> > >>> This is an interesting implementation suggestion, but is outside the > >>> scope of PSARC because it does not affect the interfaces > >>> being proposed. We did achieve quite a speedup over the old method. > >>> We'll take another look. > >>> > >> Using libast might well incur extra PSARC oversight -- are the libast > >> interfaces public? Consolidation private? If they are *project > >> private* then you'll need to get a contract for them. Using mmap() > >> would be free of those issues, and is likely to be the fastest without > >> imposing any new interdependencies. > > > > Erm... I think it shouldn't be a problem to make the |malloc()| and > > parts of libast "consolidation private" for use within OS/Net > > (e.g. this are the most stable APIs of libast and very unlikely to > > change) ... > > > > ... question is: How do I do that ? > > Lets find out if this is even necessary. Erm... the question above is more generic (since we had the idea to make libast::stdio "consolidation private" several times in the past and never had the time to do it) ... ... what's the general procedure to make a "project private" API "consolidation private" ? ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz@nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) From casper@holland.sun.com Fri Aug 14 01:00:55 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7E80tV4008700 for ; Fri, 14 Aug 2009 01:00:55 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7E80pgI000390; Fri, 14 Aug 2009 01:00:55 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOC0010LWXI9C00@brm-avmta-1.central.sun.com>; Fri, 14 Aug 2009 02:00:54 -0600 (MDT) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOC009ZKWXHCV70@brm-avmta-1.central.sun.com>; Fri, 14 Aug 2009 02:00:53 -0600 (MDT) Received: from holland (room101.Holland.Sun.COM [10.16.117.40]) by dm-holland-02.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7E80nGY042725; Fri, 14 Aug 2009 09:00:49 +0100 (BST) Date: Fri, 14 Aug 2009 10:00:49 +0200 From: Casper.Dik@sun.com Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A848455.8040102@sun.com> Sender: casper@holland.sun.com To: "Garrett D'Amore" Cc: Dave Plauger , Sherry Moore , psarc-ext@sun.com, Steven Sistare Message-id: <200908140800.n7E80nGY042725@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <4A84734C.9000303@sun.com> <4A848455.8040102@sun.com> Status: RO Content-Length: 982 >> Ivek Szczesniak wrote: >>> The stdio implementation in libc is among the slowest stdio >>> versions out there. If you want to archive better performance you >>> should use the stdio implementation in libast or use mmap(2). >> This is an interesting implementation suggestion, but is outside the >> scope of PSARC because it does not affect the interfaces >> being proposed. We did achieve quite a speedup over the old method. >> We'll take another look. > >Using libast might well incur extra PSARC oversight -- are the libast >interfaces public? Consolidation private? If they are *project >private* then you'll need to get a contract for them. Using mmap() >would be free of those issues, and is likely to be the fastest without >imposing any new interdependencies. I would suggest that you first measure before using libast. Typical, actual I/O operations takes a lot of time and the "ineffective fread implementation" time waste wouldn't be measurable. Casper From Dave.Plauger@sun.com Fri Aug 14 10:57:33 2009 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7EHvW3O023920 for ; Fri, 14 Aug 2009 10:57:33 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7EHvSds016492 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 14 Aug 2009 18:57:32 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOD00607OJVYQ00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Fri, 14 Aug 2009 10:57:31 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOD00M59OJVY390@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@Sun.COM); Fri, 14 Aug 2009 10:57:31 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7EHvVjp008710 for ; Fri, 14 Aug 2009 17:57:31 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOD00400OBXNK00@mail-amer.sun.com> for psarc-ext@Sun.COM (ORCPT psarc-ext@Sun.COM); Fri, 14 Aug 2009 11:57:31 -0600 (MDT) Received: from [129.150.64.230] ([unknown] [129.150.64.230]) by mail-amer.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOD00KO2OJIWNA0@mail-amer.sun.com>; Fri, 14 Aug 2009 11:57:18 -0600 (MDT) Date: Fri, 14 Aug 2009 13:57:16 -0400 From: Dave Plauger Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> Sender: Dave.Plauger@sun.com To: Sherry Moore Cc: PSARC-ext@sun.com, Steven Sistare Message-id: <4A85A57C.5040305@sun.com> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) Status: RO Content-Length: 2316 Thanks again for your comments. Here is a summary of our responses since yesterday. Garrett D'Amore wrote: > Ivek Szczesniak wrote: >> The stdio implementation in libc is among the slowest stdio >> versions out there. If you want to archive better performance you >> should use the stdio implementation in libast or use mmap(2). > This is an interesting implementation suggestion, but is outside the > scope of PSARC because it does not affect the interfaces > being proposed. We did achieve quite a speedup over the old method. > We'll take another look. > > Using libast might well incur extra PSARC oversight -- are the libast > interfaces public? Consolidation private? If they are *project > private* then you'll need to get a contract for them. Using mmap() > would be free of those issues, and is likely to be the fastest without > imposing any new interdependencies. Jonathan Adams wrote (same thread): > Without knowing about this project, yesterday I prototyped a simple change > which buffered input and output from savecore(), and it achieved a 2.5x speedup > on an pessimal setup (/var/crash on UFS, dump device on the same disk). > > I'm codereviewing this wad, so I'll talk to the developer about this. > > I agree that this doesn't seem architectural to me. Roland Mainz wrote (same thread): > Erm... I think it shouldn't be a problem to make the |malloc()| and > parts of libast "consolidation private" for use within OS/Net > (e.g. this are the most stable APIs of libast and very unlikely to > change) ... > > ... question is: How do I do that ? > After this, thread becomes a discussion about libasm. This project does not use libasm. Since it is still an uncommitted interface, it is unlikely that we would choose to use it right now. Brian Ruthven wrote: > Will vmdump.X be removed automatically after executing the > uncompression step? No, it is not removed. One reason is that vmdump.X files may be copied from some remote system (or accessed remotely) and then uncompressed into a temp directory. There are many different usage scenarios, and so it seems wiser to leave the choice to the user. The user can always do an "rm", with permissions. Discussed this with Brian, and he agrees that *not* automatically removing the file is the right behavior. From sherry.moore@sun.com Mon Aug 17 11:17:11 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7HIHAJg000275 for ; Mon, 17 Aug 2009 11:17:10 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7HIH2lc023687 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Tue, 18 Aug 2009 02:17:09 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOJ008079GKJA00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 17 Aug 2009 11:17:08 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOJ004ZW9GJIPC0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Mon, 17 Aug 2009 11:17:07 -0700 (PDT) Received: from too-cool.sfbay.sun.com (too-cool.SFBay.Sun.COM [129.146.226.229]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n7HIH7A7025264; Mon, 17 Aug 2009 11:17:07 -0700 (PDT) Received: from too-cool.SFBay.Sun.COM (too-cool [127.0.0.1]) by too-cool.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n7HIK02M002074; Mon, 17 Aug 2009 11:20:00 -0700 (PDT) Received: (from sherrym@localhost) by too-cool.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n7HIK0iw002073; Mon, 17 Aug 2009 11:20:00 -0700 (PDT) Date: Mon, 17 Aug 2009 11:20:00 -0700 From: Sherry Moore Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <20090817182000.GG1745@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> X-Authentication-warning: too-cool.SFBay.Sun.COM: sherrym set sender to sherry.moore@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 383 The latest versions of all the documentation are available in the final-materials directory at http://arc.opensolaris.org/caselog/PSARC/2009/330/ One pager: fast_crash_dump_1-pager.txt Design Spec: fast_crash_dump_design.txt Man pages: savecore.1m.diffmk.txt dumpadm.1m.diffmk.txt mdb.1.diffmk.txt Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From Alan.Hargreaves@sun.com Wed Aug 19 02:03:27 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7J93QUt015757 for ; Wed, 19 Aug 2009 02:03:27 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n7J93NFJ006970 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 19 Aug 2009 17:03:25 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOM00I0195PAF00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 19 Aug 2009 03:03:25 -0600 (MDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOM00A4095NCQ70@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 19 Aug 2009 03:03:24 -0600 (MDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id n7J93MGA027090 for ; Wed, 19 Aug 2009 09:03:22 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) id <0KOM00D0091XDY00@mail-apac.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 19 Aug 2009 17:03:22 +0800 (SGT) Received: from [129.158.12.60] ([unknown] [129.158.12.60]) by mail-apac.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul 2 2009)) with ESMTPSA id <0KOM008W995K51H0@mail-apac.sun.com>; Wed, 19 Aug 2009 17:03:22 +0800 (SGT) Date: Wed, 19 Aug 2009 19:01:49 +1000 From: Alan Hargreaves Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090817182000.GG1745@sun.com> Sender: Alan.Hargreaves@sun.com To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <4A8BBF7D.8080902@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <20090817182000.GG1745@sun.com> User-Agent: Thunderbird 2.0.0.22 (X11/20090803) Status: RO Content-Length: 836 I have to go back through the specs myself, but as a result of some training I'm in the middle of, I have to ask the question about how this fits in with the Xen "domain dumps" initiated form the hypervisor? Does anything eed to be done to ensure this continues to function as expected? Regards, Alan Hargreaves Sherry Moore wrote: > The latest versions of all the documentation are available in the > final-materials directory at > > http://arc.opensolaris.org/caselog/PSARC/2009/330/ > > One pager: fast_crash_dump_1-pager.txt > Design Spec: fast_crash_dump_design.txt > Man pages: savecore.1m.diffmk.txt > dumpadm.1m.diffmk.txt > mdb.1.diffmk.txt > > Sherry > -- Alan Hargreaves - http://blogs.sun.com/tpenta Principal Field Technologist (Kernel/VOSJEC/Performance) Asia Pacific/Emerging Markets Sun Microsystems From sherry.moore@sun.com Wed Aug 19 09:57:59 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7JGvxwc007818 for ; Wed, 19 Aug 2009 09:57:59 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7JGvuXO028621; Wed, 19 Aug 2009 09:57:56 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOM00705V4J5800@nwk-avmta-1.sfbay.Sun.COM>; Wed, 19 Aug 2009 09:57:55 -0700 (PDT) Received: from too-cool.local ([129.146.226.229]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOM00LQLV4JKB90@nwk-avmta-1.sfbay.Sun.COM>; Wed, 19 Aug 2009 09:57:55 -0700 (PDT) Received: from too-cool.local (too-cool [127.0.0.1]) by too-cool.local (8.14.3+Sun/8.14.3) with ESMTP id n7JH0kh6101167; Wed, 19 Aug 2009 10:00:46 -0700 (PDT) Received: (from sherrym@localhost) by too-cool.local (8.14.3+Sun/8.14.3/Submit) id n7JH0kg8101166; Wed, 19 Aug 2009 10:00:46 -0700 (PDT) Date: Wed, 19 Aug 2009 10:00:46 -0700 From: Sherry Moore Subject: Re: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <4A8BBF7D.8080902@Sun.COM> To: Alan Hargreaves Cc: Sherry Moore , psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <20090819170046.GB101077@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> <20090817182000.GG1745@sun.com> <4A8BBF7D.8080902@Sun.COM> X-Authentication-warning: too-cool.local: sherrym set sender to sherry.moore@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 1222 Hi Alan, The project team please correct me if I am wrong, but I believe that if the current "domain dumps" mechanism invokes the domain's dump algorithm, nothing additional needs to be done. Thanks, Sherry On Wed, Aug 19, 2009 at 07:01:49PM +1000, Alan Hargreaves wrote: > I have to go back through the specs myself, but as a result of some > training I'm in the middle of, I have to ask the question about how this > fits in with the Xen "domain dumps" initiated form the hypervisor? Does > anything eed to be done to ensure this continues to function as expected? > > Regards, > Alan Hargreaves > > > Sherry Moore wrote: >> The latest versions of all the documentation are available in the >> final-materials directory at >> >> http://arc.opensolaris.org/caselog/PSARC/2009/330/ >> >> One pager: fast_crash_dump_1-pager.txt >> Design Spec: fast_crash_dump_design.txt >> Man pages: savecore.1m.diffmk.txt >> dumpadm.1m.diffmk.txt >> mdb.1.diffmk.txt >> >> Sherry >> > > -- > Alan Hargreaves - http://blogs.sun.com/tpenta > Principal Field Technologist (Kernel/VOSJEC/Performance) > Asia Pacific/Emerging Markets > Sun Microsystems -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Wed Aug 19 10:07:48 2009 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n7JH7mrs009094 for ; Wed, 19 Aug 2009 10:07:48 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n7JH7iFe003262; Wed, 19 Aug 2009 10:07:45 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KOM00J0PVKX9Z00@nwk-avmta-2.sfbay.sun.com>; Wed, 19 Aug 2009 10:07:45 -0700 (PDT) Received: from too-cool.local ([129.146.226.229]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KOM009IYVKXHCD0@nwk-avmta-2.sfbay.sun.com>; Wed, 19 Aug 2009 10:07:45 -0700 (PDT) Received: from too-cool.local (too-cool [127.0.0.1]) by too-cool.local (8.14.3+Sun/8.14.3) with ESMTP id n7JHAaNN101235; Wed, 19 Aug 2009 10:10:36 -0700 (PDT) Received: (from sherrym@localhost) by too-cool.local (8.14.3+Sun/8.14.3/Submit) id n7JHAaLi101234; Wed, 19 Aug 2009 10:10:36 -0700 (PDT) Date: Wed, 19 Aug 2009 10:10:36 -0700 From: Sherry Moore Subject: Re: Fast Crash Dump [PSARC/2009/330 FastTrack timeout 08/18/2009] In-reply-to: <20090812062850.GF285030@sun.com> To: Sherry Moore Cc: psarc-ext@sun.com, David Plauger , Steven Sistare Message-id: <20090819171036.GC101077@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <20090812062850.GF285030@sun.com> X-Authentication-warning: too-cool.local: sherrym set sender to sherry.moore@sun.com using -f User-Agent: Mutt/1.5.19 (2009-01-05) Status: RO Content-Length: 118 This case has been approved by PSARC today. Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym