From Thu Jul 3 14:34:43 2008 Date: Thu, 3 Jul 2008 14:34:43 -0700 (PDT) From: Postmaster Subject: Message from mail server Content-Length: 95 Mime-Version: 1.0 Status: RO X-IMAP: 1215120883 67 Delete. This is a system message. --END+PSEUDO-- From sacadmin Fri Jun 13 17:06:00 2008 Received: from tethys.sfbay.sun.com (tethys.SFBay.Sun.COM [129.146.226.92]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E060Og024295; Fri, 13 Jun 2008 17:06:00 -0700 (PDT) Received: from tethys.sfbay.sun.com (localhost [127.0.0.1]) by tethys.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m5E02SYk007089; Fri, 13 Jun 2008 17:02:28 -0700 (PDT) Received: (from jg@localhost) by tethys.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5E02RCY007085; Fri, 13 Jun 2008 17:02:27 -0700 (PDT) Date: Fri, 13 Jun 2008 17:02:27 -0700 (PDT) From: Jerry Gilliam Message-Id: <200806140002.m5E02RCY007085@tethys.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Subject: Fast Reboot [PSARC/2008/382 FastTrack timeout 06/20/2008] Content-Length: 544 Status: RO X-Status: $$$$ X-UID: 0000000001 Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Fast Reboot 1.2. Name of Document Author/Supplier: Author: Sherry Moore 1.3 Date of This Document: 13 June, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From jg@jurassic.sfbay.Sun.COM Fri Jun 13 17:21:55 2008 Received: from jurassic.eng.sun.com (jurassic-226-b.SFBay.Sun.COM [129.146.226.130]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E0LtTu025314 for ; Fri, 13 Jun 2008 17:21:55 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id m5E0LslG232387; Fri, 13 Jun 2008 17:21:54 -0700 (PDT) Message-Id: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Date: Fri, 13 Jun 2008 17:18:22 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam Subject: Fast Reboot PSARC/2008/382 To: psarc-ext@sac.sfbay.sun.com Cc: seth.goldberg@sun.com, darrin.johnson@sun.com, neal.pollack@sun.com, sherry.moore@sun.com, jerry.gilliam@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: 45IPhUcm9RT3ymYU90XdOA== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_78 SunOS 5.11 sun4u sparc Content-Length: 24912 Status: RO X-Status: $$$$ X-UID: 0000000002 I am sponsoring the following fast-track on behalf of Sherry Moore, with timeout set for one week, June 20 2008. Supplemental documents in the materials directory include more detailed design and implementation specs and man page updates including diffs. The project requests minor binding only. -jg 1. Introduction 1.1. Project/Component Working Name: Solaris Fast Reboot 1.2. Name of Document Author/Supplier: Sherry Q. Moore 1.3. Date of This Document: 5/29/2008 4. Technical Description 4.1 Introduction Solaris has always strived to be the most reliable and available operating system. Many technologies have been invented to achieve this goal, the notable ones include Dynamic Reconfiguration (DR), Fault Management Architecture (FMA), SMF, ZFS, just to name a few. The objectives of all these projects are to keep the systems up and running correctly in the face of unexpected hardware and software failures with as little down time as possible. System boot/reboot time is considered system down time. The less time a system spends in the boot phase, the more useful work it can do. High availability is extremely important to most, if not all, of our customers. Shorter reboot time also reduces the test turnaround time, thus improves developers' productivity. 4.2 Background The Solaris boot and reboot path involves the following basic steps: On x86 systems: (Hardware reset) -> BIOS -> grub -> dboot -> kernel On SPARC systems: (Hardware reset) -> POST -> OBP -> dboot -> kernel On x86 systems, upon startup or reset, the BIOS code performs hardware testing and initialization, then jumps to "grub" the boot loader. Grub loads dboot, unix text, data and the boot archive into memory, then calls dboot. dboot does necessary initialization, such as building the initial page tables, or loading kernel text and data to a different location, then jumps to the kernel. As computer systems become more complex, the time they spend in the BIOS/POST phase to test and initialize hardware gets longer. In the next 12 months we expect to see x86 systems with 1TB of memory. Memory initialization alone will take over 1/2 hour. It becomes more and more desirable to short circuit the reboot path so that the firmware and bootloaders can be bypassed. The fast reboot code will act as an in-kernel boot loader that loads the kernel into memory and switches to it. The new kernel in the context of this write-up includes the dboot that gets tacked on during build time. The goal of the Solaris Fast Reboot project is to get to login prompt from "rebooting..." within seconds (assuming boot archive has been updated). The Solaris implementation will support systems with arbitrarily large amount of memory, and provide flexibility to reboot to 32-bit or 64-bit kernels. 4.3 Interface Table INTERFACE COMMITMENT LEVEL COMMENT reboot -f (1M) Committed To initiate a fast reboot. reboot -f -e (1M) Committed To fast reboot to a different BE. uadmin(2) Committed Added AD_FASTBOOT and AD_FASTBOOT_DRYRUN to facilitate fast reboot. quiesce(9E) Committed To quiesce a device. ddi_no_quiesce(9F) Committed Returns DDI_SUCCESS. No need to quiesce. ddi_quiesce_not_\ Committed Returns DDI_FAILURE. Quiesce needed supported(9F) but not implemented. dev_ops(9S) Committed Added devo_quiesce ops for quiescing devices. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open A. Man pages A.1 quiesce(9E): new man page A.2 reboot(1M) A.3 uadmin(2) A.4 dev_ops(9S) A.1 Man page for quiesce(9E) Man pages for ddi_no_quiesce(9F) and ddi_quiesce_not_supported(9F) will be links to the man page for quiesce(9E). Driver Entry Points quiesce(9E) NAME quiesce - quiesce a device SYNOPSIS #include #include int prefix quiesce(dev_info_t dip, ddi_quiesce_cmd_t cmd, void *arg); int ddi_no_quiesce(dev_info_t *dip, ddi_quiesce_cmd_t cmd, void *arg); int ddi_quiesce_not_supported(dev_info_t *dip, ddi_quiesce_cmd_t cmd, void *arg); INTERFACE LEVEL Solaris DDI specific (Solaris DDI) PARAMETERS dip A pointer to the device's dev_info structure. cmd Type of quiesce operation. Currently only DDI_QUIESCE is supported. arg Argument to the quiesce routine if needed. Can be set to NULL. DESCRIPTION The quiesce() function quiesces a device so that it will no longer generate interrupts or modify or access memory. It should reset the device to a hardware state from which it can be correctly configured by the driver's attach() routine without a system power cycle or being configured by the firmware. For devices that come with factory default settings, drivers must also restore such settings in its quiesce() routine. DDI_QUIESCE If cmd is set to DDI_QUIESCE, quiesce() is used to stop devices from generating interrupts or modify or access memory. One such use case is Fast Reboot where firmware is bypassed when booting to a new OS image. The quiesce() function will be called once for each instance of the device for which there has been a successful attach(). The system guarantees that the function will only be called for a particular dev_info node after a successful attach(9E) of that device. The system is not single-threaded when quiesce() is called, so the driver must ensure that concurrent accesses to the device when quiesce() is invoked is correctly coordinated. The driver can choose to drop outstanding I/O instead of waiting for them to complete as long as it can guarantee on disk data integrity. The driver must cancel any outstanding timeouts and remove outstanding tasks from taskqs before returning successfully from quiesce(). If quiesce() determines a particular instance of the device cannot be quiesced when requested because of some exceptional condition, quiesce() must return DDI_FAILURE. This should almost never happen. For the fast reboot case, if DDI_FAILURE is returned for the DDI_QUIESCE cmd, regular reboot path will be taken. If a driver has previously implemented the obsolete reset() interface, its functionality must be merged into quiesce(). The driver's reset() routine will no longer be called if an implementation of quiesce() is present. The ddi_no_quiesce() function always returns DDI_SUCCESS. It is used to indicate that a device does not need to be quiesced for fast reboot. The ddi_quiesce_not_supported() always returns DDI_FAILURE. It is used to indicate that the device needs to be quiesced but the device driver has not implemented the function yet. RETURN VALUES DDI_SUCCESS For DDI_QUIESCE, the device has been successfully quiesced. DDI_FAILURE The operation failed or the request was not understood. CONTEXT This function is called from kernel context only. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Interface Stability | Committed | |_____________________________|_____________________________| SEE ALSO attach(9E), detach(9E), ddi_add_intr(9F), ddi_map_regs(9F), pci_config_setup(9F), ddi_no_quiesce(9F), ddi_quiesce_not_supported(9F), timeout(9F), reboot(1M), uadmin(1M), uadmin(2) A.2 Man page for reboot(1M) System Administration Commands reboot(1M) NAME reboot - restart the operating system SYNOPSIS /usr/sbin/reboot [-dlnqf] [-e BE] [boot_arguments] | DESCRIPTION The reboot utility restarts the kernel. The kernel is loaded into memory by the PROM monitor, which transfers control to the loaded kernel. On x86 systems, when the -f flag is specified, the running | kernel will load the next kernel into memory, then transfers | control to the loaded kernel. | Although reboot can be run by the super-user at any time, shutdown(1M) is normally used first to warn all users logged in of the impending loss of service. See shutdown(1M) for details. The reboot utility performs a sync(1M) operation on the disks, and then a multi-user reboot is initiated. See init(1M) for details. On x86 systems, reboot may also update the boot archive as needed to ensure a successful reboot. The reboot utility normally logs the reboot to the system log daemon, syslogd(1M), and places a shutdown record in the login accounting file /var/adm/wtmpx. These actions are inhibited if the -n or -q options are present. Normally, the system reboots itself at power-up or after crashes. OPTIONS The following options are supported: -d Force a system crash dump before rebooting. See dumpadm(1M) for information on configuring system crash dumps. -e If -f is present, reboot to the specified boot | environment. | | | -f Fast reboot bypassing firmware and boot loader. The | new kernel will be loaded into memory by the running | kernel, and control will be transferred to the loaded | kernel. If disk or kernel arguments are specified, | they must be specified before other boot arguments. | See Example 3 for details. | | Currently only available on x86 system. | | -l Suppress sending a message to the system log daemon, syslogd(1M) about who executed reboot. -n Avoid calling sync(2) and do not log the reboot to syslogd(1M) or to /var/adm/wtmpx. The kernel still attempts to sync filesystems prior to reboot, except if the -d option is also present. If -d is used with -n, the kernel does not attempt to sync filesystems. -q Quick. Reboot quickly and ungracefully, without shut- ting down running processes first. OPERANDS The following operands are supported: boot_arguments An optional boot_arguments specifies argu- ments to the uadmin(2) function that are passed to the boot program and kernel upon restart. The form and list of arguments is described in the boot(1M) and kernel(1M) man pages.. If the arguments are speci- fied, whitespace between them is replaced by single spaces unless the whitespace is quoted for the shell. If the boot_arguments begin with a hyphen, they must be preceded by the -- delimiter (two hyphens) to denote the end of the reboot argument list. EXAMPLES Example 1 Passing the -r and -v Arguments to boot In the following example, the delimiter -- (two hyphens) must be used to separate the options of reboot from the arguments of boot(1M). example# reboot -dl -- -rv Example 2 Rebooting Using a Specific Disk and Kernel The following example reboots using a specific disk and ker- nel. example# reboot disk1 kernel.test/unix Example 3 Fast reboot | | Check if all the drivers on the system are fast reboot capable. | | example# reboot -f dryrun | | Rebooting to another UFS root disk. | | example# reboot -f -- '/dev/dsk/c1d0s0' | | Rebooting to another ZFS root pool. | | example# reboot -f -- 'rootpool/root1' | | Rebooting to "mykernel" on the same disk with "-k" option. | | example# reboot -f -- '/platform/i86pc/mykernel/amd64/unix -k' | | Rebooting to "mykernel" off another root disk mounted on /mnt. | | example# reboot -f -- '/mnt/platform/i86pc/mykernel/amd64/unix -k' | | Rebooting to "/platform/i86pc/kernel/$ISADIR/unix" on another boot | environment named "second_root". | | example# reboot -f -e second_root | | Rebooting to the same kernel with "-kv" options. | | example# reboot -f -- '-kv' | FILES /var/adm/wtmpx login accounting file ATTRIBUTES System Administration Commands reboot(1M) See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Availability | SUNWcsu | |_____________________________|_____________________________| SEE ALSO mdb(1), boot(1M), dumpadm(1M), fsck(1M), halt(1M), init(1M), kernel(1M), shutdown(1M), sync(1M), syslogd(1M), sync(2), uadmin(2), reboot(3C), attributes(5) NOTES The reboot utility does not execute the scripts in /etc/rcnum.d or execute shutdown actions in inittab(4). To ensure a complete shutdown of system services, use shutdown(1M) or init(1M) to reboot a Solaris system. A.3 Man page for uadmin(2) System Calls uadmin(2) NAME uadmin - administrative control SYNOPSIS #include int uadmin(int cmd, int fcn, uintptr_t mdep); DESCRIPTION The uadmin() function provides control for basic administra- tive functions. This function is tightly coupled to the sys- tem administrative procedures and is not intended for gen- eral use. The argument mdep is provided for machine- dependent use and is not defined here. It should be initial- ized to NULL if not used. As specified by cmd, the following commands are available: A_SHUTDOWN The system is shut down. All user processes are killed, the buffer cache is flushed, and the root file system is unmounted. The action to be taken after the system has been shut down is specified by fcn. The functions are generic; the hardware capabilities vary on specific machines. AD_HALT Halt the processor(s). AD_POWEROFF Halt the processor(s) and turn off the power. AD_BOOT Reboot the system, using the kernel file. AD_IBOOT Interactive reboot; user is prompted for bootable program name. AD_FASTREBOOT Bypass BIOS and boot loader | | AD_FASTREBOOT_DRYRUN Fast reboot dry run to | check whether a system supports | fast reboot. | A_REBOOT The system stops immediately without any further processing. The action to be taken next is specified by fcn as above. A_DUMP The system is forced to panic immediately without any further processing and a crash dump is written to the dump device (see dumpadm(1M)). The action to be taken next is specified by fcn, as above. A_REMOUNT The root file system is mounted again after having been fixed. This should be used only during the startup process. A_FREEZE Suspend the whole system. The system state is preserved in the state file. The following subcommands, specified by fcn, are available. AD_SUSPEND_TO_DISK Save the system state to the state file. This subcom- mand is equivalent to ACPI state S4. AD_CHECK_SUSPEND_TO_DISK Check if your sys- tem supports suspend to disk. Without performing a system suspend/resume, this subcommand checks if this feature is currently avail- able on your sys- tem. AD_SUSPEND_TO_RAM Save the system state to memory This subcommand is equivalent to ACPI state S3. AD_CHECK_SUSPEND_TO_RAM Check if your sys- tem supports suspend to memory. Without performing a system suspend/resume, this subcommand checks if this feature is currently available on your system. The following subcommands, specified by fcn, are obsolete and might be removed in a subse- quent release: AD_COMPRESS Save the system state to the state file with compression of data. This subcommand has been replaced by AD_SUSPEND_TO_DISK, which should be used instead. AD_CHECK Check if your system supports suspend and resume. Without performing a system suspend/resume, this command checks if this feature is currently available on your system. This subcommand has been replaced by AD_CHECK_SUSPEND_TO_DISK, which should be used instead. AD_FORCE Force AD_COMPRESS even when threads of user applications are not suspendable. This sub- command should never be used, as it might result in undefined behavior. RETURN VALUES Upon successful completion, the value returned depends on cmd as follows: A_SHUTDOWN Never returns. A_REBOOT Never returns. A_FREEZE 0 upon resume. A_REMOUNT 0. Otherwise, -1 is returned and errno is set to indicate the error. ERRORS The uadmin() function will fail if: EBUSY Suspend is already in progress. EINVAL The cmd argument is invalid. ENOMEM Suspend/resume ran out of physical memory. ENOSPC Suspend/resume could not allocate enough space on the root file system to store system information. ENOTSUP Suspend/resume is not supported on this platform or the command specified by cmd is not allowed. ENXIO Unable to successfully suspend system. EPERM The {PRIV_SYS_CONFIG} privilege is not asserted in the effective set of the calling process. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Interface Stability | See below. | |_____________________________|_____________________________| The A_FREEZE command and its subcommands are Committed. SEE ALSO dumpadm(1M), kernel(1M), uadmin(1M), attributes(5), privileges(5) A.4 Man page for dev_ops(9S) Data Structures for Drivers dev_ops(9S) NAME dev_ops - device operations structure SYNOPSIS #include #include INTERFACE LEVEL Solaris DDI specific (Solaris DDI). DESCRIPTION dev_ops contains driver common fields and pointers to the bus_ops and cb_ops(9S). Following are the device functions provided in the device operations structure. All fields must be set at compile time. devo_rev Driver build version. Set this to DEVO_REV. devo_refcnt Driver reference count. Set this to 0. devo_getinfo Get device driver information (see getinfo(9E)). devo_identify This entry point is obsolete. Set to nulldev. devo_probe Probe device. See probe(9E). devo_attach Attach driver to dev_info. See attach(9E). devo_detach Detach/prepare driver to unload. See detach(9E). devo_reset Reset device. (Not supported in this release.) Set this to nodev. devo_cb_ops Pointer to cb_ops(9S) structure for leaf drivers. devo_bus_ops Pointer to bus operations structure for nexus drivers. Set this to NULL if this is for a leaf driver. devo_power Power a device attached to system. See power(9E). devo_quiesce Quiesce a device attached to system. See | quiesce(9E). Can be set to ddi_no_quiesce if | the device does not generate interrupts or | perform DMA. | STRUCTURE MEMBERS int devo_rev; int devo_refcnt; int (*devo_getinfo)(dev_info_t *dip, ddi_info_cmd_t infocmd, void *arg, void **result); int (*devo_identify)(dev_info_t *dip); int (*devo_probe)(dev_info_t *dip); int (*devo_attach)(dev_info_t *dip, ddi_attach_cmd_t cmd); int (*devo_detach)(dev_info_t *dip, ddi_detach_cmd_t cmd); int (*devo_reset)(dev_info_t *dip, ddi_reset_cmd_t cmd); struct cb_ops *devo_cb_ops; struct bus_ops *devo_bus_ops; int (*devo_power)(dev_info_t *dip, int component, int level); int (*devo_quiesce)(dev_info_t *dip, | ddi_quiesce_cmd_t cmd, void *arg); | SEE ALSO attach(9E), detach(9E), getinfo(9E), probe(9E), power(9E), | quiesce(9E), nodev(9F) | From Terry.Whatley@Sun.COM Fri Jun 13 17:49:06 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E0n6V1025603 for ; Fri, 13 Jun 2008 17:49:06 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5E0n69W062533 for ; Fri, 13 Jun 2008 17:49:06 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5E0n1jf006698 for ; Fri, 13 Jun 2008 17:49:01 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2F00601GTA5100@fe-sfbay-09.sun.com> (original mail from Terry.Whatley@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Fri, 13 Jun 2008 17:49:01 -0700 (PDT) Received: from [10.211.55.3] ([67.123.136.98]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2F004UFGXO30F0@fe-sfbay-09.sun.com>; Fri, 13 Jun 2008 17:49:01 -0700 (PDT) Date: Fri, 13 Jun 2008 17:09:58 -0700 From: "Terry (Sarito) Whatley" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Sender: Terry.Whatley@Sun.COM To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Jerry.Gilliam@Sun.COM, Seth.Goldberg@Sun.COM, Sherry.Moore@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM Message-id: <48530C56.5000507@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.9 (X11/20080213) Content-Length: 1047 Status: RO X-Status: $$$$ X-UID: 0000000003 Jerry Gilliam wrote: > I am sponsoring the following fast-track on behalf of Sherry > Moore, with timeout set for one week, June 20 2008. > > Supplemental documents in the materials directory include more > detailed design and implementation specs and man page updates > including diffs. > > The project requests minor binding only. > > > -jg > > > ... > > Driver Entry Points quiesce(9E) > > NAME > quiesce - quiesce a device > > SYNOPSIS > #include > #include > > int prefix quiesce(dev_info_t dip, ddi_quiesce_cmd_t cmd, void *arg); > > int ddi_no_quiesce(dev_info_t *dip, ddi_quiesce_cmd_t cmd, void *arg); > > int ddi_quiesce_not_supported(dev_info_t *dip, ddi_quiesce_cmd_t cmd, > void *arg); > ... > Historically, I believe that interfaces provided in Solaris with "no" in their names are ones that return failure. Null is the name usually included to indicate "didn't do anything but returns success". C.f. nulldev(9F), nodev(9F). -sarito From sommerfeld@sun.com Fri Jun 13 17:55:21 2008 Received: from localhost.SFBay.Sun.COM (dhcp-umpk17-109-127.SFBay.Sun.COM [129.146.109.127]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E0tLjn025907 for ; Fri, 13 Jun 2008 17:55:21 -0700 (PDT) Received: from localhost.SFBay.Sun.COM (localhost [127.0.0.1]) by localhost.SFBay.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id m5E0tJ7d002525; Fri, 13 Jun 2008 17:55:19 -0700 (PDT) Received: (from sommerfeld@localhost) by localhost.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id m5E0tIfT002524; Fri, 13 Jun 2008 17:55:18 -0700 (PDT) X-Authentication-Warning: localhost.SFBay.Sun.COM: sommerfeld set sender to sommerfeld@sun.com using -f Subject: Re: Fast Reboot PSARC/2008/382 From: Bill Sommerfeld To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com In-Reply-To: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Fri, 13 Jun 2008 17:55:17 -0700 Message-Id: <1213404917.974.47.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.12.2 Content-Length: 1098 Status: RO X-Status: $$$$ X-UID: 0000000004 On Fri, 2008-06-13 at 17:18 -0700, Jerry Gilliam wrote: > reboot -f (1M) Committed To initiate a fast reboot. > > reboot -f -e (1M) Committed To fast reboot to a different BE. ... > For the fast reboot case, if DDI_FAILURE is returned for the > DDI_QUIESCE cmd, regular reboot path will be taken. if reboot -f -e is used, does this mean that a different BE than the desired BE will be booted? Could we decouple -e and -f? --- Is there a meaningful difference between DDI_QUIESCE and DDI_SUSPEND? The proposed quiesce/DDI_QUIESCE says: If cmd is set to DDI_QUIESCE, quiesce() is used to stop devices from generating interrupts or modify or access memory. the existing detach/DDI_SUSPEND is documented as: If cmd is set to DDI_SUSPEND, detach() is used to suspend all activity of a device before power is (possibly) removed from the device. The steps associated with suspension must include putting the underlying device into a quiescent state so that it will not generate interrupts or modify or access memory. From gdamore@sun.com Fri Jun 13 18:33:26 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E1XQf0026213 for ; Fri, 13 Jun 2008 18:33:26 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5E1XQhB011233 for ; Fri, 13 Jun 2008 18:33:26 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5E1XLHZ008045 for ; Fri, 13 Jun 2008 18:33:21 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2F00401IV8L600@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Fri, 13 Jun 2008 18:33:21 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2F0089SIZKGA70@fe-sfbay-09.sun.com>; Fri, 13 Jun 2008 18:33:21 -0700 (PDT) Date: Fri, 13 Jun 2008 18:32:14 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Sender: Garrett.Damore@sun.com To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Message-id: <48531F9E.8000507@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 26993 Status: RO X-Status: $$$$ X-UID: 0000000005 So one thing that is slightly confusing to me is, why do we need both ddi_no_quiesce(), and ddi_quiesce_not_supported()? And how is the entry point being NULL interpreted? It *seems* (and maybe I'm being naive here), that ddi_quiesce_not_supported() may not have much value -- drivers that have a bug where quiesce is necessary but lack the current can just implement a trivial "return DDI_FAILURE" quiesce in the meantime -- not much more effort than stubbing in the ddi_quiesce_not_supported(). Additionally, unlike the ddi_no_quiesce() case, the driver is going to need to be modified at some point anyway, right? Finally, it may be informative/helpful to list some example drivers in the case materials that need to have quiesce implemented, as well as perhaps a couple that don't. -- Garrett Jerry Gilliam wrote: > I am sponsoring the following fast-track on behalf of Sherry > Moore, with timeout set for one week, June 20 2008. > > Supplemental documents in the materials directory include more > detailed design and implementation specs and man page updates > including diffs. > > The project requests minor binding only. > > > -jg > > > > > 1. Introduction > 1.1. Project/Component Working Name: > Solaris Fast Reboot > > 1.2. Name of Document Author/Supplier: > Sherry Q. Moore > > 1.3. Date of This Document: > 5/29/2008 > > 4. Technical Description > > 4.1 Introduction > > Solaris has always strived to be the most reliable and available > operating system. Many technologies have been invented to achieve > this goal, the notable ones include Dynamic Reconfiguration (DR), > Fault Management Architecture (FMA), SMF, ZFS, just to name a few. > The objectives of all these projects are to keep the systems up and > running correctly in the face of unexpected hardware and software > failures with as little down time as possible. > > System boot/reboot time is considered system down time. The less > time a system spends in the boot phase, the more useful work it can > do. High availability is extremely important to most, if not all, > of our customers. Shorter reboot time also reduces the test > turnaround time, thus improves developers' productivity. > > 4.2 Background > > The Solaris boot and reboot path involves the following basic > steps: > > On x86 systems: > (Hardware reset) -> BIOS -> grub -> dboot -> kernel > > On SPARC systems: > (Hardware reset) -> POST -> OBP -> dboot -> kernel > > On x86 systems, upon startup or reset, the BIOS code performs > hardware testing and initialization, then jumps to "grub" the boot > loader. Grub loads dboot, unix text, data and the boot archive > into memory, then calls dboot. dboot does necessary > initialization, such as building the initial page tables, or > loading kernel text and data to a different location, then jumps to > the kernel. > > As computer systems become more complex, the time they spend in the > BIOS/POST phase to test and initialize hardware gets longer. In > the next 12 months we expect to see x86 systems with 1TB of > memory. Memory initialization alone will take over 1/2 hour. It > becomes more and more desirable to short circuit the reboot path so > that the firmware and bootloaders can be bypassed. > > The fast reboot code will act as an in-kernel boot loader that > loads the kernel into memory and switches to it. The new kernel in > the context of this write-up includes the dboot that gets tacked on > during build time. > > The goal of the Solaris Fast Reboot project is to get to login > prompt from "rebooting..." within seconds (assuming boot archive > has been updated). > > The Solaris implementation will support systems with arbitrarily > large amount of memory, and provide flexibility to reboot to 32-bit > or 64-bit kernels. > > 4.3 Interface Table > > INTERFACE COMMITMENT LEVEL COMMENT > > reboot -f (1M) Committed To initiate a fast reboot. > > reboot -f -e (1M) Committed To fast reboot to a different BE. > > uadmin(2) Committed Added AD_FASTBOOT and > AD_FASTBOOT_DRYRUN to facilitate > fast reboot. > > quiesce(9E) Committed To quiesce a device. > > ddi_no_quiesce(9F) Committed Returns DDI_SUCCESS. No need to quiesce. > > ddi_quiesce_not_\ Committed Returns DDI_FAILURE. Quiesce needed > supported(9F) but not implemented. > > dev_ops(9S) Committed Added devo_quiesce ops for > quiescing devices. > > 6. Resources and Schedule > > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > > 6.5. ARC review type: FastTrack > > 6.6. ARC Exposure: open > > > A. Man pages > A.1 quiesce(9E): new man page > A.2 reboot(1M) > A.3 uadmin(2) > A.4 dev_ops(9S) > > A.1 Man page for quiesce(9E) > > Man pages for ddi_no_quiesce(9F) and ddi_quiesce_not_supported(9F) > will be links to the man page for quiesce(9E). > > > Driver Entry Points quiesce(9E) > > NAME > quiesce - quiesce a device > > SYNOPSIS > #include > #include > > int prefix quiesce(dev_info_t dip, ddi_quiesce_cmd_t cmd, void *arg); > > int ddi_no_quiesce(dev_info_t *dip, ddi_quiesce_cmd_t cmd, void *arg); > > int ddi_quiesce_not_supported(dev_info_t *dip, ddi_quiesce_cmd_t cmd, > void *arg); > > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI) > > PARAMETERS > dip A pointer to the device's dev_info structure. > > > cmd Type of quiesce operation. Currently only DDI_QUIESCE > is supported. > > > arg Argument to the quiesce routine if needed. Can be set > to NULL. > > > DESCRIPTION > The quiesce() function quiesces a device so that it will no longer > generate interrupts or modify or access memory. It should reset > the device to a hardware state from which it can be correctly > configured by the driver's attach() routine without a system power > cycle or being configured by the firmware. For devices that come > with factory default settings, drivers must also restore such > settings in its quiesce() routine. > > DDI_QUIESCE > If cmd is set to DDI_QUIESCE, quiesce() is used to stop devices > from generating interrupts or modify or access memory. One such > use case is Fast Reboot where firmware is bypassed when booting to > a new OS image. > > The quiesce() function will be called once for each instance of > the device for which there has been a successful attach(). The > system guarantees that the function will only be called for a > particular dev_info node after a successful attach(9E) of that > device. The system is not single-threaded when quiesce() is > called, so the driver must ensure that concurrent accesses to the > device when quiesce() is invoked is correctly coordinated. The > driver can choose to drop outstanding I/O instead of waiting for > them to complete as long as it can guarantee on disk data > integrity. The driver must cancel any outstanding timeouts and > remove outstanding tasks from taskqs before returning successfully > from quiesce(). > > If quiesce() determines a particular instance of the device cannot > be quiesced when requested because of some exceptional condition, > quiesce() must return DDI_FAILURE. This should almost never > happen. > > For the fast reboot case, if DDI_FAILURE is returned for the > DDI_QUIESCE cmd, regular reboot path will be taken. > > If a driver has previously implemented the obsolete reset() > interface, its functionality must be merged into quiesce(). The > driver's reset() routine will no longer be called if an > implementation of quiesce() is present. > > The ddi_no_quiesce() function always returns DDI_SUCCESS. It is > used to indicate that a device does not need to be quiesced for > fast reboot. > > The ddi_quiesce_not_supported() always returns DDI_FAILURE. It > is used to indicate that the device needs to be quiesced but > the device driver has not implemented the function yet. > > > RETURN VALUES > DDI_SUCCESS For DDI_QUIESCE, the device has been successfully > quiesced. > > DDI_FAILURE The operation failed or the request was not > understood. > > > CONTEXT > This function is called from kernel context only. > > ATTRIBUTES > See attributes(5) for descriptions of the following attri- > butes: > > > > ____________________________________________________________ > | ATTRIBUTE TYPE | ATTRIBUTE VALUE | > |_____________________________|_____________________________| > | Interface Stability | Committed | > |_____________________________|_____________________________| > > > SEE ALSO > attach(9E), detach(9E), ddi_add_intr(9F), ddi_map_regs(9F), > pci_config_setup(9F), ddi_no_quiesce(9F), > ddi_quiesce_not_supported(9F), timeout(9F), reboot(1M), > uadmin(1M), uadmin(2) > > > A.2 Man page for reboot(1M) > > System Administration Commands reboot(1M) > > > > NAME > reboot - restart the operating system > > SYNOPSIS > /usr/sbin/reboot [-dlnqf] [-e BE] [boot_arguments] | > > > DESCRIPTION > The reboot utility restarts the kernel. The kernel is loaded > into memory by the PROM monitor, which transfers control to > the loaded kernel. > > > On x86 systems, when the -f flag is specified, the running | > kernel will load the next kernel into memory, then transfers | > control to the loaded kernel. | > > > Although reboot can be run by the super-user at any time, > shutdown(1M) is normally used first to warn all users logged > in of the impending loss of service. See shutdown(1M) for > details. > > > The reboot utility performs a sync(1M) operation on the > disks, and then a multi-user reboot is initiated. See > init(1M) for details. On x86 systems, reboot may also update > the boot archive as needed to ensure a successful reboot. > > > The reboot utility normally logs the reboot to the system > log daemon, syslogd(1M), and places a shutdown record in the > login accounting file /var/adm/wtmpx. These actions are > inhibited if the -n or -q options are present. > > > Normally, the system reboots itself at power-up or after > crashes. > > OPTIONS > The following options are supported: > > -d Force a system crash dump before rebooting. See > dumpadm(1M) for information on configuring system > crash dumps. > > > -e If -f is present, reboot to the specified boot | > environment. | > | > | > -f Fast reboot bypassing firmware and boot loader. The | > new kernel will be loaded into memory by the running | > kernel, and control will be transferred to the loaded | > kernel. If disk or kernel arguments are specified, | > they must be specified before other boot arguments. | > See Example 3 for details. | > | > Currently only available on x86 system. | > | > > -l Suppress sending a message to the system log daemon, > syslogd(1M) about who executed reboot. > > > -n Avoid calling sync(2) and do not log the reboot to > syslogd(1M) or to /var/adm/wtmpx. The kernel still > attempts to sync filesystems prior to reboot, except > if the -d option is also present. If -d is used with > -n, the kernel does not attempt to sync filesystems. > > > -q Quick. Reboot quickly and ungracefully, without shut- > ting down running processes first. > > > OPERANDS > The following operands are supported: > > boot_arguments An optional boot_arguments specifies argu- > ments to the uadmin(2) function that are > passed to the boot program and kernel upon > restart. The form and list of arguments is > described in the boot(1M) and kernel(1M) > man pages.. If the arguments are speci- > fied, whitespace between them is replaced > by single spaces unless the whitespace is > quoted for the shell. If the > boot_arguments begin with a hyphen, they > must be preceded by the -- delimiter (two > hyphens) to denote the end of the reboot > argument list. > > > EXAMPLES > Example 1 Passing the -r and -v Arguments to boot > > > In the following example, the delimiter -- (two hyphens) > must be used to separate the options of reboot from the > arguments of boot(1M). > > > example# reboot -dl -- -rv > > > > Example 2 Rebooting Using a Specific Disk and Kernel > > > The following example reboots using a specific disk and ker- > nel. > > > example# reboot disk1 kernel.test/unix > > > Example 3 Fast reboot | > | > Check if all the drivers on the system are fast reboot capable. | > | > example# reboot -f dryrun | > | > Rebooting to another UFS root disk. | > | > example# reboot -f -- '/dev/dsk/c1d0s0' | > | > Rebooting to another ZFS root pool. | > | > example# reboot -f -- 'rootpool/root1' | > | > Rebooting to "mykernel" on the same disk with "-k" option. | > | > example# reboot -f -- '/platform/i86pc/mykernel/amd64/unix -k' | > | > Rebooting to "mykernel" off another root disk mounted on /mnt. | > | > example# reboot -f -- '/mnt/platform/i86pc/mykernel/amd64/unix -k' | > | > Rebooting to "/platform/i86pc/kernel/$ISADIR/unix" on another boot | > environment named "second_root". | > | > example# reboot -f -e second_root | > | > Rebooting to the same kernel with "-kv" options. | > | > example# reboot -f -- '-kv' | > > > FILES > /var/adm/wtmpx login accounting file > > > ATTRIBUTES > > System Administration Commands reboot(1M) > > > See attributes(5) for descriptions of the following attri- > butes: > > ____________________________________________________________ > | ATTRIBUTE TYPE | ATTRIBUTE VALUE | > |_____________________________|_____________________________| > | Availability | SUNWcsu | > |_____________________________|_____________________________| > > > SEE ALSO > mdb(1), boot(1M), dumpadm(1M), fsck(1M), halt(1M), init(1M), > kernel(1M), shutdown(1M), sync(1M), syslogd(1M), sync(2), > uadmin(2), reboot(3C), attributes(5) > > NOTES > The reboot utility does not execute the scripts in > /etc/rcnum.d or execute shutdown actions in inittab(4). To > ensure a complete shutdown of system services, use > shutdown(1M) or init(1M) to reboot a Solaris system. > > > A.3 Man page for uadmin(2) > > > System Calls uadmin(2) > > > NAME > uadmin - administrative control > > SYNOPSIS > #include > > int uadmin(int cmd, int fcn, uintptr_t mdep); > > > DESCRIPTION > The uadmin() function provides control for basic administra- > tive functions. This function is tightly coupled to the sys- > tem administrative procedures and is not intended for gen- > eral use. The argument mdep is provided for machine- > dependent use and is not defined here. It should be initial- > ized to NULL if not used. > > > As specified by cmd, the following commands are available: > > A_SHUTDOWN The system is shut down. All user processes > are killed, the buffer cache is flushed, and > the root file system is unmounted. The action > to be taken after the system has been shut > down is specified by fcn. The functions are > generic; the hardware capabilities vary on > specific machines. > > AD_HALT Halt the processor(s). > > > AD_POWEROFF Halt the processor(s) and turn > off the power. > > > AD_BOOT Reboot the system, using the > kernel file. > > > AD_IBOOT Interactive reboot; user is > prompted for bootable program > name. > > > AD_FASTREBOOT Bypass BIOS and boot loader | > | > AD_FASTREBOOT_DRYRUN Fast reboot dry run to | > check whether a system supports | > fast reboot. | > > > A_REBOOT The system stops immediately without any > further processing. The action to be taken > next is specified by fcn as above. > > > A_DUMP The system is forced to panic immediately > without any further processing and a crash > dump is written to the dump device (see > dumpadm(1M)). The action to be taken next is > specified by fcn, as above. > > > A_REMOUNT The root file system is mounted again after > having been fixed. This should be used only > during the startup process. > > > A_FREEZE Suspend the whole system. The system state is > preserved in the state file. The following > subcommands, specified by fcn, are available. > > AD_SUSPEND_TO_DISK Save the system > state to the state > file. This subcom- > mand is equivalent > to ACPI state S4. > > > AD_CHECK_SUSPEND_TO_DISK Check if your sys- > tem supports > suspend to disk. > Without performing > a system > suspend/resume, > this subcommand > checks if this > feature is > currently avail- > able on your sys- > tem. > > > AD_SUSPEND_TO_RAM Save the system > state to memory > This subcommand is > equivalent to ACPI > state S3. > > > AD_CHECK_SUSPEND_TO_RAM Check if your sys- > tem supports > suspend to memory. > Without performing > a system > suspend/resume, > this subcommand > checks if this > feature is > currently > available on your > system. > > The following subcommands, specified by fcn, > are obsolete and might be removed in a subse- > quent release: > > AD_COMPRESS Save the system state to the > state file with compression of > data. This subcommand has been > replaced by AD_SUSPEND_TO_DISK, > which should be used instead. > > > AD_CHECK Check if your system supports > suspend and resume. Without > performing a system > suspend/resume, this command > checks if this feature is > currently available on your > system. This subcommand has > been replaced by > AD_CHECK_SUSPEND_TO_DISK, which > should be used instead. > > > AD_FORCE Force AD_COMPRESS even when > threads of user applications > are not suspendable. This sub- > command should never be used, > as it might result in undefined > behavior. > > > > RETURN VALUES > Upon successful completion, the value returned depends on > cmd as follows: > > A_SHUTDOWN Never returns. > > > A_REBOOT Never returns. > > > A_FREEZE 0 upon resume. > > > A_REMOUNT 0. > > Otherwise, -1 is returned and errno is set to indicate the > error. > > ERRORS > The uadmin() function will fail if: > > EBUSY Suspend is already in progress. > > > EINVAL The cmd argument is invalid. > > > ENOMEM Suspend/resume ran out of physical memory. > > > ENOSPC Suspend/resume could not allocate enough space on > the root file system to store system information. > > > ENOTSUP Suspend/resume is not supported on this platform > or the command specified by cmd is not allowed. > > > ENXIO Unable to successfully suspend system. > > > EPERM The {PRIV_SYS_CONFIG} privilege is not asserted > in the effective set of the calling process. > > > ATTRIBUTES > See attributes(5) for descriptions of the following attri- > butes: > > > > ____________________________________________________________ > | ATTRIBUTE TYPE | ATTRIBUTE VALUE | > |_____________________________|_____________________________| > | Interface Stability | See below. | > |_____________________________|_____________________________| > > > > The A_FREEZE command and its subcommands are Committed. > > SEE ALSO > dumpadm(1M), kernel(1M), uadmin(1M), attributes(5), > privileges(5) > > > > A.4 Man page for dev_ops(9S) > > > Data Structures for Drivers dev_ops(9S) > > > > NAME > dev_ops - device operations structure > > SYNOPSIS > #include > #include > > > INTERFACE LEVEL > Solaris DDI specific (Solaris DDI). > > DESCRIPTION > dev_ops contains driver common fields and pointers to the > bus_ops and cb_ops(9S). > > > Following are the device functions provided in the device > operations structure. All fields must be set at compile > time. > > devo_rev Driver build version. Set this to > DEVO_REV. > > > devo_refcnt Driver reference count. Set this to 0. > > > devo_getinfo Get device driver information (see > getinfo(9E)). > > > devo_identify This entry point is obsolete. Set to > nulldev. > > > devo_probe Probe device. See probe(9E). > > > devo_attach Attach driver to dev_info. See attach(9E). > > > devo_detach Detach/prepare driver to unload. See > detach(9E). > > > devo_reset Reset device. (Not supported in this > release.) Set this to nodev. > > > devo_cb_ops Pointer to cb_ops(9S) structure for leaf > drivers. > > > devo_bus_ops Pointer to bus operations structure for > nexus drivers. Set this to NULL if this is > for a leaf driver. > > > devo_power Power a device attached to system. See > power(9E). > > > devo_quiesce Quiesce a device attached to system. See | > quiesce(9E). Can be set to ddi_no_quiesce if | > the device does not generate interrupts or | > perform DMA. | > > STRUCTURE MEMBERS > int devo_rev; > int devo_refcnt; > int (*devo_getinfo)(dev_info_t *dip, > ddi_info_cmd_t infocmd, void *arg, void **result); > int (*devo_identify)(dev_info_t *dip); > int (*devo_probe)(dev_info_t *dip); > int (*devo_attach)(dev_info_t *dip, > ddi_attach_cmd_t cmd); > int (*devo_detach)(dev_info_t *dip, > ddi_detach_cmd_t cmd); > int (*devo_reset)(dev_info_t *dip, ddi_reset_cmd_t cmd); > struct cb_ops *devo_cb_ops; > struct bus_ops *devo_bus_ops; > int (*devo_power)(dev_info_t *dip, int component, int > level); > int (*devo_quiesce)(dev_info_t *dip, | > ddi_quiesce_cmd_t cmd, void *arg); | > > > SEE ALSO > attach(9E), detach(9E), getinfo(9E), probe(9E), power(9E), | > quiesce(9E), nodev(9F) | > > > From jek3@sun.com Fri Jun 13 18:44:48 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E1imMj026278 for ; Fri, 13 Jun 2008 18:44:48 -0700 (PDT) Received: from [129.150.13.200] (vpn-129-150-13-200.SFBay.Sun.COM [129.150.13.200]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5E1ikgV721874; Fri, 13 Jun 2008 18:44:46 -0700 (PDT) Message-ID: <48532335.6040007@sun.com> Date: Fri, 13 Jun 2008 15:47:33 -1000 From: Joseph Kowalski User-Agent: Thunderbird 2.0.0.9 (X11/20080225) MIME-Version: 1.0 To: Jerry Gilliam CC: psarc-ext@sac.sfbay.sun.com, seth.goldberg@sun.com, darrin.johnson@sun.com, neal.pollack@sun.com, sherry.moore@sun.com, jerry.gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> In-Reply-To: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 732 Status: RO X-Status: $$$$ X-UID: 0000000006 > -e If -f is present, reboot to the specified boot | > environment. | > "boot environment" doesn't seem to have a clear definition here. > | > | > -f Fast reboot bypassing firmware and boot loader. The | > new kernel will be loaded into memory by the running | > kernel, and control will be transferred to the loaded | > kernel. If disk or kernel arguments are specified, | > they must be specified before other boot arguments. | > See Example 3 for details. | > | > Currently only available on x86 system. | > The x86 disclaimer seems worthy of a place in the actual proposal and a short justification. - jek3 From Nicolas.Williams@sun.com Fri Jun 13 19:08:53 2008 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E28rp6026868 for ; Fri, 13 Jun 2008 19:08:53 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m5E28f62020878; Fri, 13 Jun 2008 21:08:42 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m5E28flC020877; Fri, 13 Jun 2008 21:08:41 -0500 (CDT) X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Fri, 13 Jun 2008 21:08:41 -0500 From: Nicolas Williams To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080614020841.GQ2735@Sun.COM> Mail-Followup-To: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Sherry.Moore@Sun.COM, Jerry.Gilliam@Sun.COM References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> User-Agent: Mutt/1.5.7i Content-Length: 229 Status: RO X-Status: $$$$ X-UID: 0000000007 Will it be possible to configure a system to do a fast reboot after a panic? (Also, it might be possible to save less than a full crash dump and let the system handle the rest of the crash dump after the fast reboot.) Nico -- From gdamore@sun.com Fri Jun 13 19:35:42 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E2Zgfe026987 for ; Fri, 13 Jun 2008 19:35:42 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5E2ZggK007335 for ; Fri, 13 Jun 2008 19:35:42 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5E2Zb4G009312 for ; Fri, 13 Jun 2008 19:35:37 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2F00401LQ0KA00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Fri, 13 Jun 2008 19:35:37 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2F008CCLVCGAF0@fe-sfbay-09.sun.com>; Fri, 13 Jun 2008 19:35:37 -0700 (PDT) Date: Fri, 13 Jun 2008 19:34:31 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080614020841.GQ2735@Sun.COM> Sender: Garrett.Damore@sun.com To: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Message-id: <48532E37.3020805@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 622 Status: RO X-Status: $$$$ X-UID: 0000000008 Nicolas Williams wrote: > Will it be possible to configure a system to do a fast reboot after a > panic? (Also, it might be possible to save less than a full crash dump > and let the system handle the rest of the crash dump after the fast > reboot.) > > Nico > The last I checked, the copy of the crash dump is already done in parallel to normal multi-user boot. I think the dump of physical ram to core has to be done before the reboot (fast or otherwise) in order to prevent loss of data. (The contents of RAM are not guaranteed to be the same after the reboot cycle, even for a fast reboot.) -- Garrett From jek3@sun.com Fri Jun 13 19:40:46 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E2ek63027003 for ; Fri, 13 Jun 2008 19:40:46 -0700 (PDT) Received: from [129.150.13.200] (vpn-129-150-13-200.SFBay.Sun.COM [129.150.13.200]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5E2ei9T725373; Fri, 13 Jun 2008 19:40:44 -0700 (PDT) Message-ID: <48533053.1000901@sun.com> Date: Fri, 13 Jun 2008 16:43:31 -1000 From: Joseph Kowalski User-Agent: Thunderbird 2.0.0.9 (X11/20080225) MIME-Version: 1.0 To: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> In-Reply-To: <20080614020841.GQ2735@Sun.COM> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 456 Status: RO X-Status: $$$$ X-UID: 0000000009 Nicolas Williams wrote: > Will it be possible to configure a system to do a fast reboot after a > panic? (Also, it might be possible to save less than a full crash dump > and let the system handle the rest of the crash dump after the fast > reboot.) > > Nico > When I was working on Whitesmoke (Motorola Interaction - Telco) this was one of their requirements. Their idea of "less than a full crash dump" was "only kernel space". Just FYI. - jek3 From dan.mick@sun.com Fri Jun 13 19:42:17 2008 Received: from dm-eng-02.sfbay.sun.com (dm-eng-02.SFBay.Sun.COM [129.146.11.32]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E2gHOP027035 for ; Fri, 13 Jun 2008 19:42:17 -0700 (PDT) Received: from [10.1.48.101] (angus.West.Sun.COM [10.1.48.101]) by dm-eng-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5E2frla031096; Fri, 13 Jun 2008 19:41:53 -0700 (PDT) Message-ID: <48532FF0.1070601@sun.com> Date: Fri, 13 Jun 2008 19:41:52 -0700 From: Dan Mick User-Agent: Thunderbird 2.0.0.0 (X11/20070419) MIME-Version: 1.0 To: "Garrett D'Amore" CC: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> <48532E37.3020805@sun.com> In-Reply-To: <48532E37.3020805@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 1097 Status: RO X-Status: $$$$ X-UID: 0000000010 Garrett D'Amore wrote: > Nicolas Williams wrote: >> Will it be possible to configure a system to do a fast reboot after a >> panic? (Also, it might be possible to save less than a full crash dump >> and let the system handle the rest of the crash dump after the fast >> reboot.) >> >> Nico >> > The last I checked, the copy of the crash dump is already done in > parallel to normal multi-user boot. Indeed; 'savecore'. > I think the dump of physical ram to core has to be done before the > reboot (fast or otherwise) in order to prevent loss of data. (The > contents of RAM are not guaranteed to be the same after the reboot > cycle, even for a fast reboot.) Actually, yes they are; that's the whole trick of fast reboot, is that you write the kernel into memory and then jump to it without going thru anything that would potentially clear the memory. That's exactly the whole leverage of the fast reboot mechanism. Saving pieces of the old kernel for later dumping would be trickier, but certainly possible; however I'm pretty sure it's not part of the current implementation. From gdamore@sun.com Fri Jun 13 19:51:04 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5E2p43X027074 for ; Fri, 13 Jun 2008 19:51:04 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5E2p4VF010921 for ; Fri, 13 Jun 2008 19:51:04 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5E2owEX009493 for ; Fri, 13 Jun 2008 19:50:59 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2F00601MK4AY00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Fri, 13 Jun 2008 19:50:58 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2F004LAMKYO600@fe-sfbay-10.sun.com>; Fri, 13 Jun 2008 19:50:58 -0700 (PDT) Date: Fri, 13 Jun 2008 19:49:52 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <48532FF0.1070601@sun.com> Sender: Garrett.Damore@sun.com To: Dan Mick Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Message-id: <485331D0.7010000@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> <48532E37.3020805@sun.com> <48532FF0.1070601@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 2207 Status: RO X-Status: $$$$ X-UID: 0000000011 Dan Mick wrote: > Garrett D'Amore wrote: >> Nicolas Williams wrote: >>> Will it be possible to configure a system to do a fast reboot after a >>> panic? (Also, it might be possible to save less than a full crash dump >>> and let the system handle the rest of the crash dump after the fast >>> reboot.) >>> >>> Nico >>> >> The last I checked, the copy of the crash dump is already done in >> parallel to normal multi-user boot. > > Indeed; 'savecore'. > >> I think the dump of physical ram to core has to be done before the >> reboot (fast or otherwise) in order to prevent loss of data. (The >> contents of RAM are not guaranteed to be the same after the reboot >> cycle, even for a fast reboot.) > > Actually, yes they are; that's the whole trick of fast reboot, is that > you write the kernel into memory and then jump to it without going > thru anything that would potentially clear the memory. That's exactly > the whole leverage of the fast reboot mechanism. > > Saving pieces of the old kernel for later dumping would be trickier, > but certainly possible; however I'm pretty sure it's not part of the > current implementation. One possible concern I think I raised has to do with the fact that memory may not be cleared. I worry that going from one boot to another, you could have random bits in memory which you'd rather not expose. This is especially true if you fast reboot from a BE with one set of permissions to another one that might have different permissions. I'd feel a lot better if a "fast zero" of memory could be done, perhaps conditionally. I realize that this represents both additional complexity and increased time to boot.... but I believe that a 1-2 second cycle of clearing physical memory would give a much better level of assurance that data isn't being exposed. Hopefully it would be possible to do this far more quickly than a typical BIOS POST run. (No testing of memory requested, just clobber preexisting data is all.) Hmm. and its not really perfect even then, because you could have stale user data left out in device space (e.g. in framebuffer memory). Still, the idea of zeroing memory may not be totally worthless. -- Garrett From sherry.moore@sun.com Sat Jun 14 13:05:20 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5EK5KOv011318 for ; Sat, 14 Jun 2008 13:05:20 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5EK5JuB780749 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Jun 2008 13:05:19 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5EK5Jrc780748; Sat, 14 Jun 2008 13:05:19 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Sat, 14 Jun 2008 13:05:19 -0700 From: Sherry Moore To: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080614200519.GG709115@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080614020841.GQ2735@Sun.COM> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 1532 Status: RO X-Status: $$$$ X-UID: 0000000012 Hi Nico, > Will it be possible to configure a system to do a fast reboot after a > panic? (Also, it might be possible to save less than a full crash dump > and let the system handle the rest of the crash dump after the fast > reboot.) Not yet, because the current Fast Reboot implementation relies on devices to quiesce. Depending where the panic occurred, it might not be possible to call the device quiesce code. Being able to fast reboot post panic is one of the project's goals, so we are actively investigating and experimenting with various implementations to allow us to fast reboot post panic. For instance, we have been looking into leveraging capabilities of the PCI framework to save and restore the PCI config space and resetting devices. With regard to how to handle crash dump, one of the ideas being considered is to always load a good kernel into memory (if there's sufficient memory, which will be released if freemem is low). On panic, do the regular crash dump, then remap the range of memory that the good kernel is in to be the usual kernel virtual memory. Another idea we are experimenting with is to keep track of the physical memory ranges being used, which we can already do, then pass that information to the new kernel via the memory list. Those ranges of memory will not be used until the content has been saved. We are still at experimental stage with post panic fast reboot. The final implementation might differ. Thanks, -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Sat Jun 14 13:56:22 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5EKuMWg012546 for ; Sat, 14 Jun 2008 13:56:22 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5EKuMYA788428 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Jun 2008 13:56:22 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5EKuM0F788427; Sat, 14 Jun 2008 13:56:22 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Sat, 14 Jun 2008 13:56:22 -0700 From: Sherry Moore To: "Garrett D'Amore" Cc: Dan Mick , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080614205622.GB781958@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> <48532E37.3020805@sun.com> <48532FF0.1070601@sun.com> <485331D0.7010000@sun.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <485331D0.7010000@sun.com> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 1567 Status: RO X-Status: $$$$ X-UID: 0000000013 Hi Garrett, Thanks for raising the concern. I talked with Gary Winiger and he said that zero'ing memory on boot/reboot is not a security requirement. > One possible concern I think I raised has to do with the fact that memory > may not be cleared. I worry that going from one boot to another, you could > have random bits in memory which you'd rather not expose. This is > especially true if you fast reboot from a BE with one set of permissions to > another one that might have different permissions. 1. With non-malicious new kernel It probably won't do anything with whatever random bits are exposed from the previous boot. From the perspective of correctness, kernel text and data are reloaded, BSS cleared. When the kernel allocates memory, it either explicitly requests for zero'ed memory, or it zeros memory itself. The VM layer can guarantee user processes are given zero-filled memory. The new kernel does not use anything left in memory from the old kernel, so if file permissions have changed, the new kernel honors the new permission. As a matter of fact, the new kernel has not recollection of the existence of a previous kernel. 2. With malicious new kernel For a system to be fast rebooted to a malicious new kernel, the malicious user would have to have gained privilege control of the system. If the malicious user already had privilege control of the system, all bets are off anyway. Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Sat Jun 14 14:13:27 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5ELDRL8012780 for ; Sat, 14 Jun 2008 14:13:27 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5ELDQtV790298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Jun 2008 14:13:26 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5ELDQLx790297; Sat, 14 Jun 2008 14:13:26 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Sat, 14 Jun 2008 14:13:26 -0700 From: Sherry Moore To: Joseph Kowalski Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, seth.goldberg@sun.com, darrin.johnson@sun.com, neal.pollack@sun.com, sherry.moore@sun.com, jerry.gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080614211326.GC781958@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48532335.6040007@sun.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48532335.6040007@sun.com> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 2148 Status: RO X-Status: $$$$ X-UID: 0000000014 Hi Joe, >> -e If -f is present, reboot to the specified boot | >> environment. | >> > > "boot environment" doesn't seem to have a clear definition here. How about If -f is present, reboot to the specified boot environment (BE) as created by lucreate(1M). >> -f Fast reboot bypassing firmware and boot loader. The | >> new kernel will be loaded into memory by the running | >> kernel, and control will be transferred to the loaded | >> kernel. If disk or kernel arguments are specified, | >> they must be specified before other boot arguments. | >> See Example 3 for details. | >> | >> Currently only available on x86 system. | >> > > The x86 disclaimer seems worthy of a place in the actual proposal and a > short justification. This is what I put in section 3.1 of http://sac/Archives/CaseLog/arc/PSARC/2008/382/materials/onepager.txt ============================================================================ To make the project more manageable and still deliver value on their own, the problem described above can be solved in the following phases: Phase I: Fast Reboot on x86 platforms. Phase II: Fast Reboot post panic on x86 platforms. Fast Reboot (normal and post panic) on SPARC platforms. Phase II of the project will likely require some form of memory DR where a minimum amount of clean memory is used to bring up the kernel while the rest of the memory is tested and brought in dynamically. This case delivers phase I. ============================================================================ Does that seem sufficient to you? Garrett asked a similar question before, "Why can't the SPARC support be done in parallel?" The answer is basically that, "It can, but I am kind of single-threaded". :) Thanks much, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From gdamore@sun.com Sat Jun 14 14:50:08 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5ELo8h6013082 for ; Sat, 14 Jun 2008 14:50:08 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5ELo7Ru011004 for ; Sat, 14 Jun 2008 14:50:08 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5ELo2oL002723 for ; Sat, 14 Jun 2008 14:50:02 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2H00G0133APY00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sat, 14 Jun 2008 14:50:02 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2H003UV3BDBGC0@fe-sfbay-09.sun.com>; Sat, 14 Jun 2008 14:50:02 -0700 (PDT) Date: Sat, 14 Jun 2008 14:48:53 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080614205622.GB781958@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: Dan Mick , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Message-id: <48543CC5.2030601@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <20080614020841.GQ2735@Sun.COM> <48532E37.3020805@sun.com> <48532FF0.1070601@sun.com> <485331D0.7010000@sun.com> <20080614205622.GB781958@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1935 Status: RO X-Status: $$$$ X-UID: 0000000015 Sherry Moore wrote: > Hi Garrett, > > Thanks for raising the concern. I talked with Gary Winiger and he said > that zero'ing memory on boot/reboot is not a security requirement. > > >> One possible concern I think I raised has to do with the fact that memory >> may not be cleared. I worry that going from one boot to another, you could >> have random bits in memory which you'd rather not expose. This is >> especially true if you fast reboot from a BE with one set of permissions to >> another one that might have different permissions. >> > > 1. With non-malicious new kernel > > It probably won't do anything with whatever random bits are exposed > from the previous boot. > > From the perspective of correctness, kernel text and data are > reloaded, BSS cleared. When the kernel allocates memory, it either > explicitly requests for zero'ed memory, or it zeros memory itself. > The VM layer can guarantee user processes are given zero-filled > memory. > > The new kernel does not use anything left in memory from the old > kernel, so if file permissions have changed, the new kernel honors > the new permission. As a matter of fact, the new kernel has not > recollection of the existence of a previous kernel. > > 2. With malicious new kernel > > For a system to be fast rebooted to a malicious new kernel, the > malicious user would have to have gained privilege control of the > system. If the malicious user already had privilege control of the > system, all bets are off anyway. > Its case 2 that I worry about, and really the concern is booting to an alternate BE. (Obviously rebooting the same BE should not be a problem.) I wonder if a small NOTE in the man page indicating that memory is not cleared, and therefore fast reboot should not be used to boot to alternative and untrusted environments is appropriate. -- Garrett From gdamore@sun.com Sat Jun 14 14:53:37 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5ELrb4x013114 for ; Sat, 14 Jun 2008 14:53:37 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5ELrbV6011745 for ; Sat, 14 Jun 2008 14:53:37 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5ELrWND002780 for ; Sat, 14 Jun 2008 14:53:32 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2H00I013G48H00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sat, 14 Jun 2008 14:53:32 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2H003813H7BGD0@fe-sfbay-09.sun.com>; Sat, 14 Jun 2008 14:53:31 -0700 (PDT) Date: Sat, 14 Jun 2008 14:52:22 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080614211326.GC781958@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: Joseph Kowalski , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Message-id: <48543D96.8040309@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48532335.6040007@sun.com> <20080614211326.GC781958@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 2742 Status: RO X-Status: $$$$ X-UID: 0000000016 So my question here about SPARC versus x86 is, can we pre-approve the SPARC project in this ARC case, and allow for phased delivery to deliver phase II? (Just thinking about minimizing the number of times the project team has to come back to ARC, especially when adding in SPARC support should be a no-brainer at least as far as ARC is concerned.) I'll grant you that I *suspect* the post-panic fast reboot case probably is more complicated, and probably will need a case of its own. -- Garrett Sherry Moore wrote: > Hi Joe, > > >>> -e If -f is present, reboot to the specified boot | >>> environment. | >>> >>> >> "boot environment" doesn't seem to have a clear definition here. >> > > How about > > If -f is present, reboot to the specified boot environment (BE) > as created by lucreate(1M). > > >>> -f Fast reboot bypassing firmware and boot loader. The | >>> new kernel will be loaded into memory by the running | >>> kernel, and control will be transferred to the loaded | >>> kernel. If disk or kernel arguments are specified, | >>> they must be specified before other boot arguments. | >>> See Example 3 for details. | >>> | >>> Currently only available on x86 system. | >>> >>> >> The x86 disclaimer seems worthy of a place in the actual proposal and a >> short justification. >> > > This is what I put in section 3.1 of > > http://sac/Archives/CaseLog/arc/PSARC/2008/382/materials/onepager.txt > > ============================================================================ > To make the project more manageable and still deliver value on > their own, the problem described above can be solved in the > following phases: > > Phase I: Fast Reboot on x86 platforms. > Phase II: Fast Reboot post panic on x86 platforms. > Fast Reboot (normal and post panic) on SPARC > platforms. > > Phase II of the project will likely require > some form of memory DR where a minimum amount > of clean memory is used to bring up the kernel > while the rest of the memory is tested and > brought in dynamically. > > This case delivers phase I. > ============================================================================ > > Does that seem sufficient to you? > > Garrett asked a similar question before, "Why can't the SPARC support > be done in parallel?" The answer is basically that, "It can, but I am > kind of single-threaded". :) > > Thanks much, > Sherry > From sherry.moore@sun.com Sat Jun 14 15:21:06 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5EML6td013500 for ; Sat, 14 Jun 2008 15:21:06 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5EML6dV794246 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Jun 2008 15:21:06 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5EML6LN794245; Sat, 14 Jun 2008 15:21:06 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Sat, 14 Jun 2008 15:21:06 -0700 From: Sherry Moore To: Bill Sommerfeld Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080614222106.GD781958@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <1213404917.974.47.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1213404917.974.47.camel@localhost> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 3842 Status: RO X-Status: $$$$ X-UID: 0000000017 Hi Bill, > > reboot -f (1M) Committed To initiate a fast reboot. > > > > reboot -f -e (1M) Committed To fast reboot to a different BE. > ... > > > For the fast reboot case, if DDI_FAILURE is returned for the > > DDI_QUIESCE cmd, regular reboot path will be taken. > > if reboot -f -e is used, does this mean that a different BE than the > desired BE will be booted? It means a different BE than the *default* BE will be booted. > Could we decouple -e and -f? Technically yes. It will basically be "luactivate BE; init 6", or manually setting the default in menu.lst, or finding out the corresponding menu item number in menu.lst for the desired BE, and do "reboot number". However, luactivate explicitly disassociates itself from reboot. ********************************************************************** The target boot environment has been activated. It will be used when you reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You MUST USE either the init or the shutdown command when you reboot. If you do not use either init or shutdown, the system will not boot using the target BE. ********************************************************************** I don't know the history behind the requirement stated above. I will contact the Live Upgrade team to see if's OK to support -e for regular reboot as well. If anybody knows the answer, feel free to chime in. > Is there a meaningful difference between DDI_QUIESCE and DDI_SUSPEND? > > The proposed quiesce/DDI_QUIESCE says: > > If cmd is set to DDI_QUIESCE, quiesce() is used to stop devices > from generating interrupts or modify or access memory. > > the existing detach/DDI_SUSPEND is documented as: > > If cmd is set to DDI_SUSPEND, detach() is used to suspend > all activity of a device before power is (possibly) removed > from the device. The steps associated with suspension must > include putting the underlying device into a quiescent state > so that it will not generate interrupts or modify or access > memory. That's an excellent question! The meaningful differences are in the parts that you didn't quote. For DDI_SUSPEND - "It will be issued only to devices which have a reg property or which export a pm-hardware- state property with the value needs-suspend-resume." Some of us are not convinced that devices meeting the above conditions are the only ones that need to be quiesced because drivers without a reg property can still do in-bound DMA. We felt that it's better to leave the decision to the driver authors. A driver that does not manage devices that need to be quiesed must explicitly set its devo_quiesce op to ddi_no_quiesce() (or whatever name we settle on) to indicate that there's no need to quiesce. - "It must save the hardware state of the device to memory and block incoming or existing requests until attach() is called with DDI_RESUME." Saving hardware state is not a requirement for DDI_QUIESCE. With some devices, the hardware is powered off by the driver's implementation for DDI_SUSPEND. For DDI_QUIESCE, power needs to stay on. The semantics for DDI_QUIESCE are much more relaxed than that for DDI_SUSPEND. Not being able to detach from a software framework is not a condition to fail DDI_QUIESCE. All the driver needs to guarantee is that no more memory access will occur, and no more interrupts will be generated post devo_quiesce(). That said, to avoid unnecessary code duplication, the current DDI_QUIESCE implementation we have done for most drivers are the same as that for DDI_SUSPEND, but minus the device powering off part if exists. Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From John.Plocher@Sun.COM Sat Jun 14 15:37:20 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5EMbKJ5013553 for ; Sat, 14 Jun 2008 15:37:20 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5EMbJOa022543 for ; Sat, 14 Jun 2008 15:37:19 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5EMbE6k003441 for ; Sat, 14 Jun 2008 15:37:14 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2H005015GDVY00@fe-sfbay-09.sun.com> (original mail from John.Plocher@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Sat, 14 Jun 2008 15:37:14 -0700 (PDT) Received: from [192.168.168.4] ([208.74.177.212]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2H001LN5I2DY10@fe-sfbay-09.sun.com>; Sat, 14 Jun 2008 15:37:14 -0700 (PDT) Date: Sat, 14 Jun 2008 15:37:14 -0700 From: John Plocher Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080614222106.GD781958@sun.com> Sender: John.Plocher@Sun.COM To: Sherry Moore Cc: Bill Sommerfeld , Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Jerry.Gilliam@Sun.COM, Jerry Gilliam , Seth.Goldberg@Sun.COM, psarc-ext@sac.sfbay.sun.com Message-id: <4854481A.3050105@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <1213404917.974.47.camel@localhost> <20080614222106.GD781958@sun.com> User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) Content-Length: 1386 Status: RO X-Status: $$$$ X-UID: 0000000018 Garrett D'Amore wrote: > I wonder if a small NOTE in the man page indicating that memory is not > cleared, and therefore fast reboot should not be used to boot to > alternative and untrusted environments is appropriate. I would consider this advice or TCA at most If I have access to a system, I can compromise it anyways, and if there are trojan horses out there, a note in a manpage can't even hope to address the problem. "fast reboot doesn't zero out memory" is a good note, "fast reboot... alt BE...malicious trojan..." begs the question of why we don't point out all the *other* places trojans might bite you, as well as all the other potential social engineering, worm and virus risks that might or might not exist... Might as well stick a label on it that says "might contain nuts" just to be safe :-) Sherry Moore said: > I don't know the history behind the requirement stated above. I will > contact the Live Upgrade team to see if's OK to support -e for regular > reboot as well. If anybody knows the answer, feel free to chime in. lu uses init.d K* scripts to copy volatile files and perform the magic needed to switch things under the covers so that the system actually reboots elsewhere (eeprom boot-env changes, grub menu updates, whatever) If you bypass those scripts with commands like reboot, halt and uadmin, lu can't switch BEs for you. -John From gdamore@sun.com Sat Jun 14 18:06:09 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5F169GZ015796 for ; Sat, 14 Jun 2008 18:06:09 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5F168hI058165 for ; Sat, 14 Jun 2008 18:06:08 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5F163UL005578 for ; Sat, 14 Jun 2008 18:06:03 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2H00I01CAVAI00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sat, 14 Jun 2008 18:06:03 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2H000C7CE25M70@fe-sfbay-10.sun.com>; Sat, 14 Jun 2008 18:06:03 -0700 (PDT) Date: Sat, 14 Jun 2008 18:04:52 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <4854481A.3050105@Sun.Com> Sender: Garrett.Damore@sun.com To: John Plocher Cc: Sherry Moore , Bill Sommerfeld , Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com, Jerry Gilliam , Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Message-id: <48546AB4.2080900@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <1213404917.974.47.camel@localhost> <20080614222106.GD781958@sun.com> <4854481A.3050105@Sun.Com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1766 Status: RO X-Status: $$$$ X-UID: 0000000019 John Plocher wrote: > Garrett D'Amore wrote: > > I wonder if a small NOTE in the man page indicating that memory is not > > cleared, and therefore fast reboot should not be used to boot to > > alternative and untrusted environments is appropriate. > > I would consider this advice or TCA at most I agree. I never meant to imply anything stronger than that. Sorry if that wasn't clear. > > If I have access to a system, I can compromise it anyways, and if > there are trojan horses out there, a note in a manpage can't even > hope to address the problem. > > "fast reboot doesn't zero out memory" is a good note, > > "fast reboot... alt BE...malicious trojan..." begs the question of why > we don't point out all the *other* places trojans might bite you, as > well as all the other potential social engineering, worm and virus > risks that might or might not exist... Might as well stick a label > on it that says "might contain nuts" just to be safe :-) Heh. Okay, I agree that the first comment (doesn't zero out memory) is good enough. Smart people can figure out the corollaries, if they care. (And admittedly, most people won't care. :-) -- Garrett > > > Sherry Moore said: > > I don't know the history behind the requirement stated above. I will > > contact the Live Upgrade team to see if's OK to support -e for regular > > reboot as well. If anybody knows the answer, feel free to chime in. > > lu uses init.d K* scripts to copy volatile files and perform the > magic needed to switch things under the covers so that the system > actually reboots elsewhere (eeprom boot-env changes, grub menu > updates, whatever) > > If you bypass those scripts with commands like reboot, halt and > uadmin, lu can't switch BEs for you. > > -John > > From sherry.moore@sun.com Sat Jun 14 18:35:37 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5F1Zb7U015869 for ; Sat, 14 Jun 2008 18:35:37 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5F1ZXXm800915 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Jun 2008 18:35:34 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5F1ZXSk800914; Sat, 14 Jun 2008 18:35:33 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Sat, 14 Jun 2008 18:35:33 -0700 From: Sherry Moore To: "Garrett D'Amore" Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080615013533.GA800431@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48531F9E.8000507@sun.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48531F9E.8000507@sun.com> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 1744 Status: RO X-Status: $$$$ X-UID: 0000000020 Hi Garrett, > So one thing that is slightly confusing to me is, why do we need both > ddi_no_quiesce(), and ddi_quiesce_not_supported()? And how is the entry > point being NULL interpreted? > > It *seems* (and maybe I'm being naive here), that > ddi_quiesce_not_supported() may not have much value -- drivers that have a > bug where quiesce is necessary but lack the current can just implement a > trivial "return DDI_FAILURE" quiesce in the meantime -- not much more > effort than stubbing in the ddi_quiesce_not_supported(). Additionally, > unlike the ddi_no_quiesce() case, the driver is going to need to be > modified at some point anyway, right? NULL entry point will be treated the same as ddi_quiesce_not_supported() (or whatever function name we eventually settle on). These functions are introduced so that when new driver developers start writing drivers by making copies of existing ones, it is clear that devo_quiesce is a dev_op that needs to be implemented, or set to ddi_no_quiesce() or ddi_quiesce_not_supported. If it is set to ddi_quiesce_not_supported, I would like to see someone on the ARC committee asking, "Uh, why is quiesce not supported?" :) In addition, we have a clear picture on the number of drivers that don't require quiesce, and those that do but haven't implemented it yet. In other words, one can simply look for ddi_no_quiesce and ddi_quiesce_not_supported in cscope instead of dev_ops and manually sort through them. > Finally, it may be informative/helpful to list some example drivers in the > case materials that need to have quiesce implemented, as well as perhaps a > couple that don't. Will do. I have write that up and have Jerry copy to the materials directory. Thanks, Sherry From gdamore@sun.com Sat Jun 14 19:44:08 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5F2i8g5016406 for ; Sat, 14 Jun 2008 19:44:08 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5F2i8HX014477 for ; Sat, 14 Jun 2008 19:44:08 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5F2i30n006736 for ; Sat, 14 Jun 2008 19:44:03 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2H00401GMP6Z00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sat, 14 Jun 2008 19:44:03 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2H00IUFGXEEJ30@fe-sfbay-09.sun.com>; Sat, 14 Jun 2008 19:44:03 -0700 (PDT) Date: Sat, 14 Jun 2008 19:42:52 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080615013533.GA800431@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Message-id: <485481AC.4070302@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48531F9E.8000507@sun.com> <20080615013533.GA800431@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 2270 Status: RO X-Status: $$$$ X-UID: 0000000021 Sherry Moore wrote: > Hi Garrett, > > >> So one thing that is slightly confusing to me is, why do we need both >> ddi_no_quiesce(), and ddi_quiesce_not_supported()? And how is the entry >> point being NULL interpreted? >> >> It *seems* (and maybe I'm being naive here), that >> ddi_quiesce_not_supported() may not have much value -- drivers that have a >> bug where quiesce is necessary but lack the current can just implement a >> trivial "return DDI_FAILURE" quiesce in the meantime -- not much more >> effort than stubbing in the ddi_quiesce_not_supported(). Additionally, >> unlike the ddi_no_quiesce() case, the driver is going to need to be >> modified at some point anyway, right? >> > > NULL entry point will be treated the same as ddi_quiesce_not_supported() > (or whatever function name we eventually settle on). > > These functions are introduced so that when new driver developers start > writing drivers by making copies of existing ones, it is clear that > devo_quiesce is a dev_op that needs to be implemented, or set to > ddi_no_quiesce() or ddi_quiesce_not_supported. If it is set to > ddi_quiesce_not_supported, I would like to see someone on the ARC > committee asking, "Uh, why is quiesce not supported?" :) > So, rather than creating a new entry point for this, can't we instead just make this a new question to add to the 20 questions? > In addition, we have a clear picture on the number of drivers that > don't require quiesce, and those that do but haven't implemented it > yet. In other words, one can simply look for ddi_no_quiesce and > ddi_quiesce_not_supported in cscope instead of dev_ops and manually > sort through them. > Hmm... that seems like a temporary problem to solve. I don't mind having that as an implementation detail, but I question whether ddi_not_supported() really needs to be part of the official DDI. I'd rather not have it be a documented interface. > >> Finally, it may be informative/helpful to list some example drivers in the >> case materials that need to have quiesce implemented, as well as perhaps a >> couple that don't. >> > > Will do. I have write that up and have Jerry copy to the materials > directory. > Thanks. -- Garrett > Thanks, > Sherry > From jg@jurassic.sfbay.Sun.COM Sun Jun 15 14:55:40 2008 Received: from jurassic.eng.sun.com (jurassic-226-a.SFBay.Sun.COM [129.146.226.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5FLte4p002598 for ; Sun, 15 Jun 2008 14:55:40 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id m5FLtbMn294877; Sun, 15 Jun 2008 14:55:37 -0700 (PDT) Message-Id: <200806152155.m5FLtbMn294877@jurassic.eng.sun.com> Date: Sun, 15 Jun 2008 14:52:04 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 To: gdamore@sun.com Cc: Sherry.Moore@sun.com, psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: iVeWBWSMQhEmja1BihQkqA== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_78 SunOS 5.11 sun4u sparc Content-Length: 1747 Status: RO X-Status: $$$$ X-UID: 0000000022 >> >> These functions are introduced so that when new driver developers start >> writing drivers by making copies of existing ones, it is clear that >> devo_quiesce is a dev_op that needs to be implemented, or set to >> ddi_no_quiesce() or ddi_quiesce_not_supported. If it is set to >> ddi_quiesce_not_supported, I would like to see someone on the ARC >> committee asking, "Uh, why is quiesce not supported?" :) >> > >So, rather than creating a new entry point for this, can't we instead >just make this a new question to add to the 20 questions? > >> In addition, we have a clear picture on the number of drivers that >> don't require quiesce, and those that do but haven't implemented it >> yet. In other words, one can simply look for ddi_no_quiesce and >> ddi_quiesce_not_supported in cscope instead of dev_ops and manually >> sort through them. >> > >Hmm... that seems like a temporary problem to solve. I don't mind >having that as an implementation detail, but I question whether >ddi_not_supported() really needs to be part of the official DDI. I'd >rather not have it be a documented interface. > The thought was that providing functions that drivers can reference for common default behaviors allows a number of ways to answer questions like "does this driver fail quiesce?" or "what drivers are preventing fast reboot?". With symbols, these questions can be easily answered via source, nm or kadb. There would be no way to enforce the use of these functions of course, but providing and using handlers for common defaults such as ddi_getinfo_1to1() and ddi_no_info() are convenient and obvious and seem like a practice we should encourage. Even if we don't quite promote these to the level of "official DDI". -jg From gdamore@sun.com Sun Jun 15 20:58:08 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5G3w8Sw008128 for ; Sun, 15 Jun 2008 20:58:08 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5G3w8WB007192 for ; Sun, 15 Jun 2008 20:58:08 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5G3w3r9028798 for ; Sun, 15 Jun 2008 20:58:03 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2J00H01EZQCU00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sun, 15 Jun 2008 20:58:03 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2J004XJF0QBL50@fe-sfbay-10.sun.com>; Sun, 15 Jun 2008 20:58:02 -0700 (PDT) Date: Sun, 15 Jun 2008 20:56:47 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806152155.m5FLtbMn294877@jurassic.eng.sun.com> Sender: Garrett.Damore@sun.com To: Jerry Gilliam Cc: Sherry.Moore@sun.com, psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Message-id: <4855E47F.2000505@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806152155.m5FLtbMn294877@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1778 Status: RO X-Status: $$$$ X-UID: 0000000023 A further explanation, which I'd like to have in the case log, for why a separate quiesce routine, and suspend was deemed insufficient follows. These details have previously been discussed with the project team before this case was submitted. Note that none of the following is a request for any action on anyone's part, I just want to record the rationale more clearly here for posterity. -- Garrett It is interesting to note that for many drivers there may be a great deal of shared implementation between quiesce and suspend. However, we should consider why a separate quiesce is required: 1) suspend requires device drivers be able to suspend without loss of functionality, which makes it much, much harder to guarantee correct function. quiesce is far far simpler in this regard. (If device drivers never "failed" suspend -- e.g. because the device is in use, then they could probably use their suspend logic.) 2) quiesce requires a device be placed in a state where it can be reinitialized as if it were in a more-or-less default state, without a power cycle or bus reset, and without any soft state being passed from one environment to the next. To put a concrete example, think of a NIC device which initializes with a default MAC address, but which can be temporarily changed in software by the driver, e.g. in response to an end user using ifconfig to change the mac address. The problem now is that the original factory mac address is "lost" unless the device gets a full reset. (And in fact, this particular behavior for NICs is not uncommon.) In this regard, the ability to go back to an initial "power on" or "default" state is actually a *new* requirement, and one where suspend semantics might not be sufficient. From gdamore@sun.com Sun Jun 15 20:58:28 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5G3wSER008140 for ; Sun, 15 Jun 2008 20:58:28 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5G3wSRj043880 for ; Sun, 15 Jun 2008 20:58:28 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5G3wNJl028804 for ; Sun, 15 Jun 2008 20:58:23 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2J00H01EZQCU00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Sun, 15 Jun 2008 20:58:23 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2J004Z5F1ABL50@fe-sfbay-10.sun.com>; Sun, 15 Jun 2008 20:58:22 -0700 (PDT) Date: Sun, 15 Jun 2008 20:57:08 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806152155.m5FLtbMn294877@jurassic.eng.sun.com> Sender: Garrett.Damore@sun.com To: Jerry Gilliam Cc: Sherry.Moore@sun.com, psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Message-id: <4855E494.7090106@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806152155.m5FLtbMn294877@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 2878 Status: RO X-Status: $$$$ X-UID: 0000000024 Jerry Gilliam wrote: > >>> These functions are introduced so that when new driver developers start >>> writing drivers by making copies of existing ones, it is clear that >>> devo_quiesce is a dev_op that needs to be implemented, or set to >>> ddi_no_quiesce() or ddi_quiesce_not_supported. If it is set to >>> ddi_quiesce_not_supported, I would like to see someone on the ARC >>> committee asking, "Uh, why is quiesce not supported?" :) >>> >>> >> So, rather than creating a new entry point for this, can't we instead >> just make this a new question to add to the 20 questions? >> Yeah, I'd rather have that too. >> >>> In addition, we have a clear picture on the number of drivers that >>> don't require quiesce, and those that do but haven't implemented it >>> yet. In other words, one can simply look for ddi_no_quiesce and >>> ddi_quiesce_not_supported in cscope instead of dev_ops and manually >>> sort through them. >>> >>> >> Hmm... that seems like a temporary problem to solve. I don't mind >> having that as an implementation detail, but I question whether >> ddi_not_supported() really needs to be part of the official DDI. I'd >> rather not have it be a documented interface. >> >> > > The thought was that providing functions that drivers can reference > for common default behaviors allows a number of ways to answer > questions like "does this driver fail quiesce?" or "what drivers > are preventing fast reboot?". With symbols, these questions can > be easily answered via source, nm or kadb. > > There would be no way to enforce the use of these functions of course, > but providing and using handlers for common defaults such as > ddi_getinfo_1to1() and ddi_no_info() are convenient and obvious and > seem like a practice we should encourage. Even if we don't quite > promote these to the level of "official DDI". > So I have no concern with ddi_no_quiesce(), which is a reasonable default behavior (the device doesn't require quiesce.) My concern was with ddi_quiesce_not_supported(), which it sounds like even the project team thinks nobody should use. (It sounded like this may have been useful to the project team to identify the drivers they have looked at, and make cscope easier to deal with, but that's still, IMO, an implementation detail, and probably doesn't warrant promoting ddi_quiesce_not_supported() into the public DDI. (Note that I'm specifically not asking the team not to create the routine or use it, I'm just saying lets not document or bless this as a valid approach. It sounds like device drivers really need to take one of two paths: 1) declare that they don't require a quiesce by using ddi_no_quiesce() 2) implement a functional quiesce() All other cases are just, if I understand properly, a short term (hopefully!) bug. - Garrett > > -jg > > From dmk@noho.sfbay.sun.com Mon Jun 16 01:11:52 2008 Received: from noho.sfbay.sun.com (noho.SFBay.Sun.COM [10.6.92.101]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5G8BqMU013538 for ; Mon, 16 Jun 2008 01:11:52 -0700 (PDT) Received: from noho.sfbay.sun.com (localhost [127.0.0.1]) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7) with ESMTP id m5G8BqS6022464; Mon, 16 Jun 2008 01:11:52 -0700 (PDT) Received: (from dmk@localhost) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7/Submit) id m5G8Bpqg022463; Mon, 16 Jun 2008 01:11:52 -0700 (PDT) Date: Mon, 16 Jun 2008 01:11:52 -0700 (PDT) From: David Kahn Message-Id: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> To: jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 Cc: darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com, sherry.moore@sun.com Content-Length: 2440 Status: RO X-Status: $$$$ X-UID: 0000000025 >NAME > quiesce - quiesce a device > > The quiesce() function will be called once for each instance of > the device for which there has been a successful attach(). The > system guarantees that the function will only be called for a > particular dev_info node after a successful attach(9E) of that > device. The system is not single-threaded when quiesce() is > called, so the driver must ensure that concurrent accesses to the > device when quiesce() is invoked is correctly coordinated. The > driver can choose to drop outstanding I/O instead of waiting for > them to complete as long as it can guarantee on disk data > integrity. The driver must cancel any outstanding timeouts and > remove outstanding tasks from taskqs before returning successfully > from quiesce(). I think the definition for quiesce(9e) needs some more work. What I'm reading above is fairly vague. Specifically, can open(9e) or any other entry point be called while the driver is in quiesce(9e)? I would think we would want to guard against that, and if not, the driver needs to know that. If the intent is that nothing else will be called while the driver is in quiesce(9e), please state that guarantee in the man page. Second, are there open instances of the driver when quiesce(9e) is called, or does the framework guarantee to close all open instances of the device first? Also, the phrase "guarantee on disk data integrety" is not well-defined from a driver perspective. If I put my driver hat on, I really have no idea what that requirement is. Do you expect the driver writer to do something specific in that case, or just reset the controller, dropping any scheduled or in-flight transactions? How does it know if the disk data integrety requirement can be met. Finally, I'm not sure if you want to say anything about DMA mappings, etc. I guess they can be thrown away because the driver for any iommu in the path (yes, x86 has them too) will end up being reset by its driver? How will console output work during this process? Also, which ddi services is the driver permitted to call and not permitted to call as part of it's quiesce(9e) implementation? I guess what you are doing is loading the new ramdisk or kernel into memory and then calling quiesce(9e)? How would that work in a panic situation? (Yes, I saw that's part of phase 2, but it still needs to fit these interfaces.) -David From David.Kahn@sun.com Mon Jun 16 01:23:47 2008 Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5G8NlD0013941 for ; Mon, 16 Jun 2008 01:23:47 -0700 (PDT) Received: from [192.168.0.9] (noho.SFBay.Sun.COM [10.6.92.101]) by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5G8Nj2s016594; Mon, 16 Jun 2008 01:23:45 -0700 (PDT) Message-ID: <48562311.40306@sun.com> Date: Mon, 16 Jun 2008 01:23:45 -0700 From: David Kahn User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: jg@jurassic.sfbay.sun.com CC: psarc-ext@sac.sfbay.sun.com, darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com, sherry.moore@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 341 Status: RO X-Status: $$$$ X-UID: 0000000026 Also, I suppose we could use a detach_cmd_t value for DDI_QUIESCE instead of expanding struct dev_ops. int prefix detach(dev_info_t *dip, ddi_detach_cmd_t cmd); This is really a form of detach, isn't it? Right now cmd must be either DDI_DETACH or DDI_SUSPEND. The driver returns DDI_FAILURE if it's not one of those two values. -David From David.Kahn@sun.com Mon Jun 16 01:31:53 2008 Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5G8VrPq013964 for ; Mon, 16 Jun 2008 01:31:53 -0700 (PDT) Received: from [192.168.0.9] (noho.SFBay.Sun.COM [10.6.92.101]) by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5G8VmGb016612; Mon, 16 Jun 2008 01:31:48 -0700 (PDT) Message-ID: <485624F4.40708@sun.com> Date: Mon, 16 Jun 2008 01:31:48 -0700 From: David Kahn User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: jg@jurassic.sfbay.sun.com CC: psarc-ext@sac.sfbay.sun.com, darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com, sherry.moore@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <48562311.40306@sun.com> In-Reply-To: <48562311.40306@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 442 Status: RO X-Status: $$$$ X-UID: 0000000027 The other question I have (but I'm admittedly out of my area of expertise here) is: zfod pages will still be zero filled, I assume. And can I assume that there will be no way for old data laying around in memory from the old instance of the OS to be mapped and read by the new instance of the OS (specifically, applications), since we aren't clearing memory before the restart? Are we meeting security requirements in this area? -David From carlsonj@phorcys.east.sun.com Mon Jun 16 03:55:32 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GAtW8J016672 for ; Mon, 16 Jun 2008 03:55:32 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5GAtR62022399; Mon, 16 Jun 2008 06:55:27 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m5GAtRuW022396; Mon, 16 Jun 2008 06:55:27 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18518.18079.767601.202029@gargle.gargle.HOWL> Date: Mon, 16 Jun 2008 06:55:27 -0400 From: James Carlson To: Sherry Moore Cc: "Garrett D'Amore" , Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com, Jerry Gilliam , Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 In-Reply-To: <20080615013533.GA800431@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48531F9E.8000507@sun.com> <20080615013533.GA800431@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Content-Length: 1719 Status: RO X-Status: $$$$ X-UID: 0000000028 Sherry Moore writes: > NULL entry point will be treated the same as ddi_quiesce_not_supported() > (or whatever function name we eventually settle on). > > These functions are introduced so that when new driver developers start > writing drivers by making copies of existing ones, ... we hunt them down and eject them from the source base. > it is clear that > devo_quiesce is a dev_op that needs to be implemented, or set to Oh. That sentence didn't end the way I was expecting it to. ;-} > In addition, we have a clear picture on the number of drivers that > don't require quiesce, and those that do but haven't implemented it > yet. In other words, one can simply look for ddi_no_quiesce and > ddi_quiesce_not_supported in cscope instead of dev_ops and manually > sort through them. That doesn't work, for exactly the reason you stated above: you've got to support NULL so that you can support old drivers, and that means NULL should work on new ones as well. In fact, for drivers that are written to support multiple releases of Solaris (e.g., those in the CS consolidation or in external open source repositories), the most likely case is that they'd leave that structure member uninitialized on purpose rather than deal with a new DDI function that might not be present. Finding the non-quiesce-compliant drivers is just going to be harder than find|xargs grep. I don't think expanding dev_ops works well here. As David Kahn mentioned, this is just another form of detach(9E). -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From carlsonj@phorcys.east.sun.com Mon Jun 16 04:06:31 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GB6VOs016820 for ; Mon, 16 Jun 2008 04:06:31 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5GB6QGw022429; Mon, 16 Jun 2008 07:06:26 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m5GB6QFj022426; Mon, 16 Jun 2008 07:06:26 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18518.18738.4703.615559@gargle.gargle.HOWL> Date: Mon, 16 Jun 2008 07:06:26 -0400 From: James Carlson To: Sherry Moore Cc: Bill Sommerfeld , Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com, Jerry Gilliam , Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 In-Reply-To: <20080614222106.GD781958@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <1213404917.974.47.camel@localhost> <20080614222106.GD781958@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Content-Length: 1933 Status: RO X-Status: $$$$ X-UID: 0000000029 Sherry Moore writes: > > > reboot -f (1M) Committed To initiate a fast reboot. > > > > > > reboot -f -e (1M) Committed To fast reboot to a different BE. [...] > > Could we decouple -e and -f? > > Technically yes. It will basically be "luactivate BE; init 6", or > manually setting the default in menu.lst, or finding out the > corresponding menu item number in menu.lst for the desired BE, and do > "reboot number". [...] > I don't know the history behind the requirement stated above. I will > contact the Live Upgrade team to see if's OK to support -e for regular > reboot as well. If anybody knows the answer, feel free to chime in. I didn't notice that in the original case, or I would have commented sooner. "reboot -e" doesn't make sense to me. To do that right with the existing LU, you'd have to reach out to the undocumented /etc/init.d/lu and run that script. That's why the existing "reboot" does *not* switch BEs, and why you have to do "init 6" or "shutdown" -- it requires that the legacy services be shut down for the switch to occur. I really don't see a point to that. We're going to chuck LU out the same open window through which most of the Install gate is being thrown by the Indiana team. Doing an upgrade on an OpenSolaris 2008.05 system is already quite a bit different ("beadm") from the old LU. If this is really necessary (special support for an S10 patch?), then I think you're going to need a contract on the Install consolidation's 'lu' script so that you can invoke it directly from the reboot command. And the Indiana team will likely need to know about this so that they can rip the code out of their own delivery, because it just won't work there. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Joep.Vesseur@Sun.COM Mon Jun 16 05:13:42 2008 Received: from sunnl.Holland.Sun.COM (sunnl.Holland.Sun.COM [129.159.237.90]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GCDf6I018025 for ; Mon, 16 Jun 2008 05:13:41 -0700 (PDT) Received: from [10.16.117.32] (enigma.Holland.Sun.COM [10.16.117.32]) by sunnl.Holland.Sun.COM (8.13.8+Sun/8.13.8) with ESMTP id m5GCDcvT019745; Mon, 16 Jun 2008 14:13:38 +0200 (MEST) Message-ID: <485658F1.6010700@Sun.COM> Date: Mon, 16 Jun 2008 14:13:37 +0200 From: Joep Vesseur User-Agent: Thunderbird 2.0.0.14 (X11/20080519) MIME-Version: 1.0 To: Jerry Gilliam CC: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Sherry.Moore@Sun.COM, Jerry.Gilliam@Sun.COM Subject: Re: Fast Reboot PSARC/2008/382 References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> In-Reply-To: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> X-Enigmail-Version: 0.95.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Length: 1150 Status: RO X-Status: $$$$ X-UID: 0000000030 At first reading, it seems to me that this case and PSARC 2008/195 "Validated Execution" are mutually exclusive. One of the purposes of that case (which has not integrated yet) is to ensure that, from system-boot up to the running userland, all code has been checked and validated before it is started. On systems that contain a Trusted Platform Module, the TPM registers reflect the checksums of all validated pieces of the boot chain. The registers can be "extended" (hash over hash with particular values) to lead to certain values that can be used to unlock registers containing sensitive data. Restarting the kernel as proposed by this case will either run unverified code (at the vary least, not every step of the boot-process is checked sequentially anymore) or the registers used to record the validation will no longer unlock the registers containing the sensitive data needed to continue the boot process. I'm not sure what the regular process here is; either PSARC declares the functionality defined by these cases as excluding each other (which would be bad), or the teams need get together to see if the issues can be resolved. Joep From Nicolas.Williams@sun.com Mon Jun 16 08:11:28 2008 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GFBRKn021213 for ; Mon, 16 Jun 2008 08:11:28 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m5GFBMWN022010; Mon, 16 Jun 2008 10:11:23 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m5GFBMNW022009; Mon, 16 Jun 2008 10:11:22 -0500 (CDT) X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Mon, 16 Jun 2008 10:11:22 -0500 From: Nicolas Williams To: Joep Vesseur Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080616151121.GX2735@Sun.COM> Mail-Followup-To: Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Sherry.Moore@Sun.COM, Jerry.Gilliam@Sun.COM References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <485658F1.6010700@Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <485658F1.6010700@Sun.COM> User-Agent: Mutt/1.5.7i Content-Length: 725 Status: RO X-Status: $$$$ X-UID: 0000000031 On Mon, Jun 16, 2008 at 02:13:37PM +0200, Joep Vesseur wrote: > Restarting the kernel as proposed by this case will either run unverified code > (at the vary least, not every step of the boot-process is checked sequentially > anymore) or the registers used to record the validation will no longer unlock > the registers containing the sensitive data needed to continue the boot > process. Well, the TPM need not know (because its driver might not actually fully reset it on quiesce?) that a new kernel is replacing the old one. The old kernel was trusted and it can do signature verification of the new kernel. And the old kernel could pass to the new kernel any data the new kernel will need to access the TPM. Nico -- From gdamore@sun.com Mon Jun 16 08:14:48 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GFEmGo021249 for ; Mon, 16 Jun 2008 08:14:48 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GFEm5S062761 for ; Mon, 16 Jun 2008 08:14:48 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GFEhpL013994 for ; Mon, 16 Jun 2008 08:14:43 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00301A3BFK00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 08:14:43 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K000T1ACHG3A0@fe-sfbay-09.sun.com>; Mon, 16 Jun 2008 08:14:42 -0700 (PDT) Date: Mon, 16 Jun 2008 08:13:26 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com Message-id: <48568316.5080501@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 2867 Status: RO X-Status: $$$$ X-UID: 0000000032 David Kahn wrote: >> NAME >> quiesce - quiesce a device >> >> > > >> The quiesce() function will be called once for each instance of >> the device for which there has been a successful attach(). The >> system guarantees that the function will only be called for a >> particular dev_info node after a successful attach(9E) of that >> device. The system is not single-threaded when quiesce() is >> called, so the driver must ensure that concurrent accesses to the >> device when quiesce() is invoked is correctly coordinated. The >> driver can choose to drop outstanding I/O instead of waiting for >> them to complete as long as it can guarantee on disk data >> integrity. The driver must cancel any outstanding timeouts and >> remove outstanding tasks from taskqs before returning successfully >> from quiesce(). >> > > I think the framework should guarantee that the kernel is single threaded at the point that quiesce is called, and that there will be no subsequent IO performed to the device. I suspect that this guarantee is already in place, but maybe the project team can clarify. -- Garrett > I think the definition for quiesce(9e) needs some more work. > > What I'm reading above is fairly vague. > > Specifically, can open(9e) or any other entry point be called > while the driver is in quiesce(9e)? I would think we would want > to guard against that, and if not, the driver needs to know that. > If the intent is that nothing else will be called while the > driver is in quiesce(9e), please state that guarantee in the > man page. > > Second, are there open instances of the driver when quiesce(9e) > is called, or does the framework guarantee to close all open > instances of the device first? > > Also, the phrase "guarantee on disk data integrety" is not > well-defined from a driver perspective. If I put my driver hat > on, I really have no idea what that requirement is. Do you expect > the driver writer to do something specific in that case, or just > reset the controller, dropping any scheduled or in-flight > transactions? How does it know if the disk data integrety requirement > can be met. > > Finally, I'm not sure if you want to say anything about DMA > mappings, etc. I guess they can be thrown away because the > driver for any iommu in the path (yes, x86 has them too) will > end up being reset by its driver? > > How will console output work during this process? > > Also, which ddi services is the driver permitted to call and not > permitted to call as part of it's quiesce(9e) implementation? > > I guess what you are doing is loading the new ramdisk or kernel > into memory and then calling quiesce(9e)? How would that work > in a panic situation? (Yes, I saw that's part of phase 2, but > it still needs to fit these interfaces.) > > -David > > From gdamore@sun.com Mon Jun 16 08:17:02 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GFH2VB021292 for ; Mon, 16 Jun 2008 08:17:02 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GFH2kJ030794 for ; Mon, 16 Jun 2008 08:17:02 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GFGv72014211 for ; Mon, 16 Jun 2008 08:16:57 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00C01ADVGR00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 08:16:57 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K000U5AG2G3B0@fe-sfbay-09.sun.com>; Mon, 16 Jun 2008 08:16:51 -0700 (PDT) Date: Mon, 16 Jun 2008 08:15:34 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <48562311.40306@sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com Message-id: <48568396.5050704@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <48562311.40306@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 972 Status: RO X-Status: $$$$ X-UID: 0000000033 I requested that the team not create a new DDI_QUIESCE command. The expansion of detach(9e) and attach(9e) with new interfaces was, IMO, a big mistake. The biggest problem is that it is impossible to tell (at least easily) if a driver supports the new interfaces or not. By adding a new devops entry point, it becomes nothing more than a matter of routine inspection. (The current design is one of the major limiting factors in suspend-to-ram. There is no predictability about whether suspend will work or not -- the only way to know is to actually try it.) -- Garrett David Kahn wrote: > > Also, I suppose we could use a detach_cmd_t > value for DDI_QUIESCE instead of expanding > struct dev_ops. > > int prefix detach(dev_info_t *dip, ddi_detach_cmd_t cmd); > > This is really a form of detach, isn't it? > > Right now cmd must be either DDI_DETACH or > DDI_SUSPEND. The driver returns DDI_FAILURE > if it's not one of those two values. > > -David > From gdamore@sun.com Mon Jun 16 08:18:14 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GFIESH021306 for ; Mon, 16 Jun 2008 08:18:14 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GFIENn000209 for ; Mon, 16 Jun 2008 08:18:14 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GFI9Yg014327 for ; Mon, 16 Jun 2008 08:18:09 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00J019SLPY00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 08:18:09 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K002Z8AI3T5B0@fe-sfbay-10.sun.com>; Mon, 16 Jun 2008 08:18:03 -0700 (PDT) Date: Mon, 16 Jun 2008 08:16:47 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <485624F4.40708@sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com Message-id: <485683DF.60107@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <48562311.40306@sun.com> <485624F4.40708@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 676 Status: RO X-Status: $$$$ X-UID: 0000000034 David Kahn wrote: > > The other question I have (but I'm admittedly out of my > area of expertise here) is: > > zfod pages will still be zero filled, I assume. > > And can I assume that there will be no way for > old data laying around in memory from the old instance > of the OS to be mapped and read by the new instance of > the OS (specifically, applications), since we aren't > clearing memory before the restart? > > Are we meeting security requirements in this area? This goes the question/concern I raised. If the new OS is assumed to be "friendly", then yes, user pages are cleared properly. The only risk comes if a "hostile" kernel is rebooted. - Garrett From gdamore@sun.com Mon Jun 16 08:24:28 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GFOSfw021436 for ; Mon, 16 Jun 2008 08:24:28 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GFOSWk004666 for ; Mon, 16 Jun 2008 08:24:28 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GFONoW015092 for ; Mon, 16 Jun 2008 08:24:23 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00C01ADVGR00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 08:24:23 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K000JKASHG3F0@fe-sfbay-09.sun.com>; Mon, 16 Jun 2008 08:24:18 -0700 (PDT) Date: Mon, 16 Jun 2008 08:23:02 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080616151121.GX2735@Sun.COM> Sender: Garrett.Damore@sun.com To: Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Message-id: <48568556.6030105@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <485658F1.6010700@Sun.COM> <20080616151121.GX2735@Sun.COM> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1271 Status: RO X-Status: $$$$ X-UID: 0000000035 Nicolas Williams wrote: > On Mon, Jun 16, 2008 at 02:13:37PM +0200, Joep Vesseur wrote: > >> Restarting the kernel as proposed by this case will either run unverified code >> (at the vary least, not every step of the boot-process is checked sequentially >> anymore) or the registers used to record the validation will no longer unlock >> the registers containing the sensitive data needed to continue the boot >> process. >> > > Well, the TPM need not know (because its driver might not actually fully > reset it on quiesce?) that a new kernel is replacing the old one. The > old kernel was trusted and it can do signature verification of the new > kernel. And the old kernel could pass to the new kernel any data the > new kernel will need to access the TPM. > Yes, this passing of state seems the right approach here. I'd like to see a response from the project team as to how this can be achieved -- at present there is no state sharing. (One possible implementation, which may or may not work for TPM, is to scribble whatever state needs to be shared into a device scratch register somewhere. Of course if TPM has no such scratch registers, or if they aren't wide enough, then some other approach will be required.) -- Garrett > Nico > From jg@jurassic.sfbay.Sun.COM Mon Jun 16 09:14:36 2008 Received: from jurassic.eng.sun.com (jurassic-226-a.SFBay.Sun.COM [129.146.226.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GGEauP024227 for ; Mon, 16 Jun 2008 09:14:36 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id m5GGEZCT326671; Mon, 16 Jun 2008 09:14:36 -0700 (PDT) Message-Id: <200806161614.m5GGEZCT326671@jurassic.eng.sun.com> Date: Mon, 16 Jun 2008 09:11:01 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 To: gdamore@sun.com Cc: psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Vk1BK6Ye5iOjZiS5euXMQQ== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_78 SunOS 5.11 sun4u sparc Content-Length: 426 Status: RO X-Status: $$$$ X-UID: 0000000036 > >I requested that the team not create a new DDI_QUIESCE command. The >expansion of detach(9e) and attach(9e) with new interfaces was, IMO, a >big mistake. > Garrett, I agree, and doing this would make providing functions that implement common behaviors that drivers can simply plug into the devops cleaner. Simply jettisoning the ddi_quiesce_t command and the for now NULL argument would be a cleaner design. -jg From sherry.moore@sun.com Mon Jun 16 09:59:35 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GGxZqL026120 for ; Mon, 16 Jun 2008 09:59:35 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5GGxYiO012155; Mon, 16 Jun 2008 09:59:34 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5GGxYDx012154; Mon, 16 Jun 2008 09:59:34 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Mon, 16 Jun 2008 09:59:34 -0700 From: Sherry Moore To: Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, seth.goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080616165933.GA11878@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <485658F1.6010700@Sun.COM> <20080616151121.GX2735@Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080616151121.GX2735@Sun.COM> User-Agent: Mutt/1.4.1i Content-Length: 1499 Status: RO X-Status: $$$$ X-UID: 0000000037 Nico is correct: the running kernel can easily validate the new kernel and boot archive by explicitly passing O_VERIFY when opening the files. Once entering the new kernel, the execution is no different from a regular boot. Nico, since the initial manifest itself (used before the the validation daemon is fully initialized) is included in the boot archive, I don't think there is any data that the running kernel needs to pass to the new kernel to access the TPM. If you believe that there is information that needs to be passed, let's chat offline to see how to best achieve the purpose. Thanks, Sherry On Mon, Jun 16, 2008 at 10:11:22AM -0500, Nicolas Williams wrote: > On Mon, Jun 16, 2008 at 02:13:37PM +0200, Joep Vesseur wrote: > > Restarting the kernel as proposed by this case will either run unverified code > > (at the vary least, not every step of the boot-process is checked sequentially > > anymore) or the registers used to record the validation will no longer unlock > > the registers containing the sensitive data needed to continue the boot > > process. > > Well, the TPM need not know (because its driver might not actually fully > reset it on quiesce?) that a new kernel is replacing the old one. The > old kernel was trusted and it can do signature verification of the new > kernel. And the old kernel could pass to the new kernel any data the > new kernel will need to access the TPM. > > Nico > -- -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From gdamore@sun.com Mon Jun 16 10:45:38 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GHjcDb028101 for ; Mon, 16 Jun 2008 10:45:38 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GHjcIB040680 for ; Mon, 16 Jun 2008 10:45:38 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GHjWQs011092 for ; Mon, 16 Jun 2008 10:45:32 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00I01GIELF00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 10:45:32 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K000UZHBNDUD0@fe-sfbay-09.sun.com>; Mon, 16 Jun 2008 10:45:24 -0700 (PDT) Date: Mon, 16 Jun 2008 10:44:06 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806161614.m5GGEZCT326671@jurassic.eng.sun.com> Sender: Garrett.Damore@sun.com To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com Message-id: <4856A666.2070108@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806161614.m5GGEZCT326671@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 646 Status: RO X-Status: $$$$ X-UID: 0000000038 Jerry Gilliam wrote: > >> I requested that the team not create a new DDI_QUIESCE command. The >> expansion of detach(9e) and attach(9e) with new interfaces was, IMO, a >> big mistake. >> >> > > > Garrett, I agree, and doing this would make providing functions > that implement common behaviors that drivers can simply plug into > the devops cleaner. Simply jettisoning the ddi_quiesce_t command > and the for now NULL argument would be a cleaner design. > I actually agree with you wholeheartedly. I think they used the command for possible future expansion, following precedent set by other DDI entry points. -- Garrett From jek3@sun.com Mon Jun 16 10:48:57 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GHmvCa028144 for ; Mon, 16 Jun 2008 10:48:57 -0700 (PDT) Received: from [129.150.13.200] (vpn-129-150-13-200.SFBay.Sun.COM [129.150.13.200]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5GHma2S955168; Mon, 16 Jun 2008 10:48:36 -0700 (PDT) Message-ID: <4856A821.4060301@sun.com> Date: Mon, 16 Jun 2008 07:51:29 -1000 From: Joseph Kowalski User-Agent: Thunderbird 2.0.0.9 (X11/20080225) MIME-Version: 1.0 To: Sherry Moore CC: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, seth.goldberg@sun.com, darrin.johnson@sun.com, neal.pollack@sun.com, jerry.gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48532335.6040007@sun.com> <20080614211326.GC781958@sun.com> In-Reply-To: <20080614211326.GC781958@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 748 Status: RO X-Status: $$$$ X-UID: 0000000039 Sherry Moore wrote: > Hi Joe, > > >>> -e If -f is present, reboot to the specified boot | >>> environment. | >>> >>> >> "boot environment" doesn't seem to have a clear definition here. >> > > How about > > If -f is present, reboot to the specified boot environment (BE) > as created by lucreate(1M). > Which *still* doesn't provide a good definition, but that's an lucreate "bug". How about: If -f is present, reboot to the specified boot environment (BE) as specified by live_upgrade(5). Anyway, lets not get into the noise, either is OK. > This is what I put in section 3.1 of > > http://sac/Archives/CaseLog/arc/PSARC/2008/382/materials/onepager.txt > Great! Thanks. - jek3 From sherry.moore@sun.com Mon Jun 16 10:58:43 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GHwhDx028625 for ; Mon, 16 Jun 2008 10:58:43 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5GHwhtU012626; Mon, 16 Jun 2008 10:58:43 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5GHwhhQ012625; Mon, 16 Jun 2008 10:58:43 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Mon, 16 Jun 2008 10:58:43 -0700 From: Sherry Moore To: "Garrett D'Amore" Cc: Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Sherry.Moore@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080616175841.GG11878@sun.com> References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <48531F9E.8000507@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48531F9E.8000507@sun.com> User-Agent: Mutt/1.4.1i Content-Length: 385 Status: RO X-Status: $$$$ X-UID: 0000000040 > Finally, it may be informative/helpful to list some example drivers in > the case materials that need to have quiesce implemented, as well as > perhaps a couple that don't. List of example drivers is available at http://sac.eng/Archives/CaseLog/arc/PSARC/2008/382/materials/sample-driver-list.txt Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Mon Jun 16 13:37:50 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GKboMp007585 for ; Mon, 16 Jun 2008 13:37:50 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5GKbnxJ014045; Mon, 16 Jun 2008 13:37:49 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5GKbncY014044; Mon, 16 Jun 2008 13:37:49 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Mon, 16 Jun 2008 13:37:49 -0700 From: Sherry Moore To: David Kahn Cc: jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com, sherry.moore@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080616203746.GT11878@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> User-Agent: Mutt/1.4.1i Content-Length: 2855 Status: RO X-Status: $$$$ X-UID: 0000000041 Hi David, > Specifically, can open(9e) or any other entry point be called > while the driver is in quiesce(9e)? I would think we would want > to guard against that, and if not, the driver needs to know that. > If the intent is that nothing else will be called while the > driver is in quiesce(9e), please state that guarantee in the > man page. Technically, yes; architecturally no. Since we're basically at the same place as we would be when the reset() entry points are called, the same sort of assumption hold here also. I will add the following to the man page: "open(9E) and other entry points should not be called while the driver is in quiesce(9E)." > Second, are there open instances of the driver when quiesce(9e) > is called, or does the framework guarantee to close all open > instances of the device first? Yes, as long as the driver can guarantee that no more asynchronous writes occur to system memory as operation to the open instances. I will add that to the man page as well. > Also, the phrase "guarantee on disk data integrety" is not > well-defined from a driver perspective. If I put my driver hat > on, I really have no idea what that requirement is. Do you expect > the driver writer to do something specific in that case, or just > reset the controller, dropping any scheduled or in-flight > transactions? How does it know if the disk data integrety requirement > can be met. Fair enough. I will remove the data integrity clause. From Fast Reboot's perspective, it's preferred that the driver drop any scheduled or in-flight transactions. > Finally, I'm not sure if you want to say anything about DMA > mappings, etc. I guess they can be thrown away because the > driver for any iommu in the path (yes, x86 has them too) will > end up being reset by its driver? OK. How about "asynchronous writes to system memory" instead? > How will console output work during this process? Console output will continue to work until the chain of drivers for the console are quiesced, which will be performed last. > Also, which ddi services is the driver permitted to call and not > permitted to call as part of it's quiesce(9e) implementation? All DDI interfaces can be called except those prevented by devtree_freeze(). > I guess what you are doing is loading the new ramdisk or kernel > into memory and then calling quiesce(9e)? How would that work > in a panic situation? (Yes, I saw that's part of phase 2, but > it still needs to fit these interfaces.) We will probably require drivers to implement quiesce() in a lock-free, no-memory-allocation, do-the-bare-minimum manner to poke only the few registers to stop all asynchronous writes to system memory and interrupt generate. That's why the quisce() interface has an "arg" field... Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Mon Jun 16 13:46:53 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GKkr7q007664 for ; Mon, 16 Jun 2008 13:46:53 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5GKkqAE014348; Mon, 16 Jun 2008 13:46:52 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5GKkqoF014347; Mon, 16 Jun 2008 13:46:52 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Mon, 16 Jun 2008 13:46:52 -0700 From: Sherry Moore To: "Garrett D'Amore" Cc: David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com, Sherry.Moore@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080616204651.GU11878@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <48568316.5080501@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48568316.5080501@sun.com> User-Agent: Mutt/1.4.1i Content-Length: 976 Status: RO X-Status: $$$$ X-UID: 0000000042 Hi Garrett, > I think the framework should guarantee that the kernel is single > threaded at the point that quiesce is called, and that there will be no > subsequent IO performed to the device. I suspect that this guarantee is > already in place, but maybe the project team can clarify. We would like to call quiesce in the single-threaded environment. That was original design that you reviewed. However, once we are in a single-threaded environment, we must not grab locks. When I contacted various driver owners, they all said that it's very difficult to implement quiesce() in a lock-free manner. As a result, I have moved the quiesce() invocation to immediately before the system becomes single-threaded. If driver owners can step up and provide lock-free, no-memory-allocation, do-the-bare-minimal quiesce() implementations, that might solve the panic reboot problem as well. Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From gdamore@sun.com Mon Jun 16 14:04:39 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GL4dYW008860 for ; Mon, 16 Jun 2008 14:04:39 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5GL4dhr005595 for ; Mon, 16 Jun 2008 14:04:39 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5GL4Yv1007519 for ; Mon, 16 Jun 2008 14:04:34 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00I01QF26M00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 14:04:34 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K00GOWQJLISE0@fe-sfbay-10.sun.com>; Mon, 16 Jun 2008 14:04:33 -0700 (PDT) Date: Mon, 16 Jun 2008 14:03:15 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080616204651.GU11878@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com Message-id: <4856D513.6080005@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <48568316.5080501@sun.com> <20080616204651.GU11878@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1766 Status: RO X-Status: $$$$ X-UID: 0000000043 Sherry Moore wrote: > Hi Garrett, > > >> I think the framework should guarantee that the kernel is single >> threaded at the point that quiesce is called, and that there will be no >> subsequent IO performed to the device. I suspect that this guarantee is >> already in place, but maybe the project team can clarify. >> > > We would like to call quiesce in the single-threaded environment. That > was original design that you reviewed. However, once we are in a > single-threaded environment, we must not grab locks. When I contacted > various driver owners, they all said that it's very difficult to > implement quiesce() in a lock-free manner. As a result, I have moved > the quiesce() invocation to immediately before the system becomes > single-threaded. If driver owners can step up and provide lock-free, > no-memory-allocation, do-the-bare-minimal quiesce() implementations, > that might solve the panic reboot problem as well. > > Thanks, > Sherry > Wow. I'm surprised driver developers made that assertion. I wonder if the problem is really that device driver authors expect to be able to use interrupts to quiesce, and hence need condvars to synchronize with? It seems (to me at least), that if interrupts are disabled and the kernel is truly single threaded, then device drivers should just be able to poll during quiesce if they need to wait for a device to respond, and should just poll for completion of other things. Perhaps the driver folks didn't understand that at the point of quiesce, no inflight operations are going to complete on their own. (Other than perhaps ones that have already been submitted to the device.) Locks in a single threaded non-preemptable environment seem pointless to me. -- Garrett From David.Kahn@sun.com Mon Jun 16 16:55:36 2008 Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5GNtaZI020005 for ; Mon, 16 Jun 2008 16:55:36 -0700 (PDT) Received: from [192.168.0.9] (noho.SFBay.Sun.COM [10.6.92.101]) by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5GNtR8w025880; Mon, 16 Jun 2008 16:55:28 -0700 (PDT) Message-ID: <4856FD70.60901@sun.com> Date: Mon, 16 Jun 2008 16:55:28 -0700 From: David Kahn User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Sherry Moore CC: David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> In-Reply-To: <20080616203746.GT11878@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 8163 Status: RO X-Status: $$$$ X-UID: 0000000044 A couple of replies in one email: Garrett D'Amore wrote: > I requested that the team not create a new DDI_QUIESCE command. The > expansion of detach(9e) and attach(9e) with new interfaces was, IMO, a > big mistake. The biggest problem is that it is impossible to tell (at > least easily) if a driver supports the new interfaces or not. By adding > a new devops entry point, it becomes nothing more than a matter of > routine inspection. (The current design is one of the major limiting > factors in suspend-to-ram. There is no predictability about whether > suspend will work or not -- the only way to know is to actually try it.) I guess inspection is a good reason to create a new entry in dev_ops, but it's still not a guarantee that quiesce(9e) will succeed. You could require the driver to export a property if it supports quiesce(9e) and get the same effect you are looking for without expanding dev_ops if you wanted to. There's nothing wrong with expanding dev_ops, but we don't have to expand it if we don't want to. Sherry Moore wrote: > Hi David, > >> Specifically, can open(9e) or any other entry point be called >> while the driver is in quiesce(9e)? I would think we would want >> to guard against that, and if not, the driver needs to know that. >> If the intent is that nothing else will be called while the >> driver is in quiesce(9e), please state that guarantee in the >> man page. > > Technically, yes; architecturally no. Since we're basically at the > same place as we would be when the reset() entry points are called, the > same sort of assumption hold here also. > > I will add the following to the man page: > > "open(9E) and other entry points should not be called while the driver > is in quiesce(9E)." I think the man page needs to provide a guarantee that the driver does not have to guard against open(9e) and close(9e) being called once the framework calls quiesce(9e), so that has to be worded appropriately. (Not a "should" for the implementation, but written as a normative requirement that the driver can rely on.) Something like: open(9e), close(9e), attach(9e), detach(9e) and other driver entry points shall not be called when quiesce(9e) is called. I'm not sure if that's enough. What if open, close, etc are in progress when the framework decides to call quiesce. Is that possible? So maybe it has to be worded even stronger such that the guarantee is that open, close, attach, detach and other driver entry points will not be active or called when the framework calls quiesce(9e). That still leaves the question of interrupt handlers, timers, callbacks, etc that could be called when the driver is in quiesce(9e) or might be active when the quiesce(9e) is called. This needs a bit of work, IMO. > >> Second, are there open instances of the driver when quiesce(9e) >> is called, or does the framework guarantee to close all open >> instances of the device first? > > Yes, as long as the driver can guarantee that no more asynchronous > writes occur to system memory as operation to the open instances. > > I will add that to the man page as well. Bear with me on this. A driver has scheduled the device to do a bunch of io by handing off a list of requests to the device. The driver probably uses completion interrupts or doorbells to the application directly to signal completions or errors from the device. Nothing has been done to tell the driver to stop doing that when we call quiesce(9e), so the device may still be writing to memory and sending interrupts. Will those interrupts be taken when the driver is in quiesce(9e)? If so, that's why the driver needs some sort of lock to guard against race conditions with it's interrupt handler. (Also, timers, callbacks, etc. Basically any interface that can asynchronously call into the driver while any other entry point is active.) There's always the possibility of a race inside the driver if any interrupt can be taken when we start to call quiesce(9e). Also, there's timer interrupts, soft interrupts, etc. Basically any sort of async event that the driver setup before quiesce(9e) was called. This needs some work also. >> Also, the phrase "guarantee on disk data integrety" is not >> well-defined from a driver perspective. If I put my driver hat >> on, I really have no idea what that requirement is. Do you expect >> the driver writer to do something specific in that case, or just >> reset the controller, dropping any scheduled or in-flight >> transactions? How does it know if the disk data integrety requirement >> can be met. > > Fair enough. I will remove the data integrity clause. From Fast > Reboot's perspective, it's preferred that the driver drop any scheduled > or in-flight transactions. Removing it is ok, but I'm not sure that's sufficient. We really have to be very explicit about what we expect the device to do and what it can expect in quiesce(9e). > >> Finally, I'm not sure if you want to say anything about DMA >> mappings, etc. I guess they can be thrown away because the >> driver for any iommu in the path (yes, x86 has them too) will >> end up being reset by its driver? > > OK. How about "asynchronous writes to system memory" instead? Not sure. Basically, I think what you are trying to create is an interface to the driver that gets the device into a quiet reset state. But doing that in the driver may be more complicated than you think if it has to worry about taking interrupts, soft ints, timers, callbacks or any other async event while it's in quiesce(9e). > >> How will console output work during this process? > > Console output will continue to work until the chain of drivers for the > console are quiesced, which will be performed last. What happens after that and before we're ready to do console output again? What if the process fails? There's no way to display a message on the console? What is the behavior of this stuff if the process fails? I suppose the fallback could be to do a normal reset, as long as we end up booting the same thing that would have been booted had this process succeeded. Is that part of the interface? >> Also, which ddi services is the driver permitted to call and not >> permitted to call as part of it's quiesce(9e) implementation? > > All DDI interfaces can be called except those prevented by > devtree_freeze(). So I can create timers, add interrupts, wait for locks, etc? > >> I guess what you are doing is loading the new ramdisk or kernel >> into memory and then calling quiesce(9e)? How would that work >> in a panic situation? (Yes, I saw that's part of phase 2, but >> it still needs to fit these interfaces.) > > We will probably require drivers to implement quiesce() in a lock-free, > no-memory-allocation, do-the-bare-minimum manner to poke only the few > registers to stop all asynchronous writes to system memory and > interrupt generate. That's why the quisce() interface has an "arg" > field... The quiesce(9e) man page is one of the interfaces that PSARC is being asked to approve with this case, right? The man page doesn't say that. I think this project needs a bit of work before it can just time out as a fast-track. It should be possible for you to do this, but the interfaces, especially quiesce(9e) and what the driver can do and is supposed to do needs some more thought, IMO. It seems like what you want to do is stop interrupt dispatch, stop callback threads, timer callbacks, soft ints, etc, and then just call into the driver to tell it to stop whatever it might be doing and get the device into a quiet/reset state. You have to guarantee in the interface all of the above conditions in order to implement something like that and show the driver writer that they don't need to use locks or worry about any other part of the driver becoming active when it's in quiesce(9e). Once the driver returns successfully from quiesce(9e), it basically guarantees that the device will no longer access system memory (we need more there.) I think we also have to guarantee that once quiesce(9e) is called, we won't call into the driver in any e.p. or callback, interrupt, timer, etc anymore. -David From gdamore@sun.com Mon Jun 16 17:10:47 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H0AkC8020642 for ; Mon, 16 Jun 2008 17:10:46 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5H0AkRI050757 for ; Mon, 16 Jun 2008 17:10:46 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5H0AfBH017686 for ; Mon, 16 Jun 2008 17:10:41 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2K00301Z2OXT00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 17:10:41 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2K00962Z5AE660@fe-sfbay-09.sun.com>; Mon, 16 Jun 2008 17:10:23 -0700 (PDT) Date: Mon, 16 Jun 2008 17:09:03 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <4856FD70.60901@sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: Sherry Moore , David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com Message-id: <4857009F.8080206@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 6089 Status: RO X-Status: $$$$ X-UID: 0000000045 David Kahn wrote: > A couple of replies in one email: > > > > Garrett D'Amore wrote: >> I requested that the team not create a new DDI_QUIESCE command. The >> expansion of detach(9e) and attach(9e) with new interfaces was, IMO, >> a big mistake. The biggest problem is that it is impossible to tell >> (at least easily) if a driver supports the new interfaces or not. By >> adding a new devops entry point, it becomes nothing more than a >> matter of routine inspection. (The current design is one of the >> major limiting factors in suspend-to-ram. There is no predictability >> about whether suspend will work or not -- the only way to know is to >> actually try it.) > > I guess inspection is a good reason to create a new entry in dev_ops, > but it's still not a guarantee that quiesce(9e) will succeed. Yes. But in the case of suspend/resume, we could have been a lot *smarter* here. I wouldn't mind have a new flag that drivers could supply in devops that says "quiesce *always* succeeds". In fact, IMO, there should be *no* reason for quiesce to *ever* fail -- short of a fatal problem in the hardware. Maybe it should have been specified this way? That does raise a possible question, which is that in the event of an FMA detected fault condition, the hardware may not be able to quiesce properly, and for sanity's sake, a full power-cycle is probably required. Does the project team take hardware FMA status into consideration when deciding whether to fast reboot or not? > > You could require the driver to export a property if it supports > quiesce(9e) and get the same effect you are looking for without > expanding dev_ops if you wanted to. Yes, you could. Properties (as opposed to dev ops flags) are a PITA. > > There's nothing wrong with expanding dev_ops, but we don't have > to expand it if we don't want to. I've (for a long time now) wanted to break SUSPEND/RESUME out of attach/detach processing -- its entirely unpredictable right now. My ideal world is: * all drivers export (suspend/resume/quiesce). * those entry points are assumed to always succeed/never fail * a driver that actually may have a failure condition in suspend (and possibly in quiesce) can set a dev ops flag indicating this The point is, I would rather not have to look beyond the dev_ops routine to learn whether a function is likely to succeed or not. > > Sherry Moore wrote: >> Hi David, >> >>> Specifically, can open(9e) or any other entry point be called >>> while the driver is in quiesce(9e)? I would think we would want >>> to guard against that, and if not, the driver needs to know that. >>> If the intent is that nothing else will be called while the >>> driver is in quiesce(9e), please state that guarantee in the >>> man page. >> >> Technically, yes; architecturally no. Since we're basically at the >> same place as we would be when the reset() entry points are called, the >> same sort of assumption hold here also. >> >> I will add the following to the man page: >> >> "open(9E) and other entry points should not be called while the driver >> is in quiesce(9E)." > > I think the man page needs to provide a guarantee that the driver > does not have to guard against open(9e) and close(9e) being called > once the framework calls quiesce(9e), so that has to be worded > appropriately. (Not a "should" for the implementation, but written > as a normative requirement that the driver can rely on.) > > Something like: > > open(9e), close(9e), attach(9e), detach(9e) and other driver entry > points shall not be called when quiesce(9e) is called. > > I'm not sure if that's enough. What if open, close, etc are in > progress when the framework decides to call quiesce. Is that > possible? So maybe it has to be worded even stronger such that > the guarantee is that open, close, attach, detach and other driver > entry points will not be active or called when the framework calls > quiesce(9e). That still leaves the question of interrupt handlers, > timers, callbacks, etc that could be called when the driver is > in quiesce(9e) or might be active when the quiesce(9e) is called. > > This needs a bit of work, IMO. This sound less architectural, and more wordsmithing. I won't disagree that the docs can be cleaned up, made clearer.... > > >> >>> Second, are there open instances of the driver when quiesce(9e) >>> is called, or does the framework guarantee to close all open >>> instances of the device first? >> >> Yes, as long as the driver can guarantee that no more asynchronous >> writes occur to system memory as operation to the open instances. >> >> I will add that to the man page as well. > > Bear with me on this. A driver has scheduled the device to do > a bunch of io by handing off a list of requests to the device. > The driver probably uses completion interrupts or doorbells to > the application directly to signal completions or errors from > the device. Nothing has been done to tell the driver to stop > doing that when we call quiesce(9e), so the device may still be > writing to memory and sending interrupts. Will those interrupts > be taken when the driver is in quiesce(9e)? If so, that's why > the driver needs some sort of lock to guard against race conditions > with it's interrupt handler. (Also, timers, callbacks, etc. Basically > any interface that can asynchronously call into the driver while > any other entry point is active.) Ah, but what I'm saying is that the kernel should have disabled all interrupts from occurring at the point that quiesce is called. So those events, which may occur in hardware, won't trigger an ISR execution. No locking is required. > > There's always the possibility of a race inside the driver if any > interrupt can be taken when we start to call quiesce(9e). See above. Interrupts should be disabled. > > Also, there's timer interrupts, soft interrupts, etc. Basically > any sort of async event that the driver setup before quiesce(9e) > was called. > > This needs some work also. Again, no interrupts, totally single threaded... etc. -- Garrett From David.Kahn@sun.com Mon Jun 16 17:42:19 2008 Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H0gJTS021364 for ; Mon, 16 Jun 2008 17:42:19 -0700 (PDT) Received: from [192.168.0.9] (noho.SFBay.Sun.COM [10.6.92.101]) by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5H0gHuI024535; Mon, 16 Jun 2008 17:42:18 -0700 (PDT) Message-ID: <4857086A.2050108@sun.com> Date: Mon, 16 Jun 2008 17:42:18 -0700 From: David Kahn User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: "Garrett D'Amore" CC: Sherry Moore , David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <4857009F.8080206@sun.com> In-Reply-To: <4857009F.8080206@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 1586 Status: RO X-Status: $$$$ X-UID: 0000000046 Garrett D'Amore wrote: > Yes, you could. Properties (as opposed to dev ops flags) are a PITA. A flag is fine. Same idea. I'm not really hung up on expanding dev_ops vs adding a new detach command with a flag/property. I was just pointing out that if the project team doesn't want to expand dev_ops, there is a decent alternative available. >> This needs a bit of work, IMO. > > This sound less architectural, and more wordsmithing. I won't disagree > that the docs can be cleaned up, made clearer.... No, this is the specification of the interface. It isn't wordsmithing. The specification of the interface is what's important here and it must be crystal clear and not subject to different interpretations. > Ah, but what I'm saying is that the kernel should have disabled all > interrupts from occurring at the point that quiesce is called. So those > events, which may occur in hardware, won't trigger an ISR execution. No > locking is required. It must be specified that way. Not the single threaded part, but the guarantee that timers, callbacks, interrupt service routines and all other async callbacks and other driver entry points will not be called or active when the driver is in quiesce(9e). If that's the interface, then it must be specified that way as part of the case materials. PSARC is reviewing the interface specification. The interface specification is not sufficient, so it needs work before it can be approved so all these things that have been mentioned in the replies are addressed and made part of the interface specification. Thanks, David From Darren.Reed@Sun.COM Mon Jun 16 17:47:05 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H0l5UF021515 for ; Mon, 16 Jun 2008 17:47:05 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5H0l4IZ002057 for ; Mon, 16 Jun 2008 17:47:04 -0700 (PDT) Received: from fe-apac-06.sun.com (fe-apac-06.sun.com [192.18.19.177] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m5H0ls2B001111 for ; Tue, 17 Jun 2008 00:47:54 GMT Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0K2L007010NOEZ00@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Tue, 17 Jun 2008 08:45:11 +0800 (SGT) Received: from [129.146.106.55] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0K2L00FP60R3QYEH@mail-apac.sun.com>; Tue, 17 Jun 2008 08:45:06 +0800 (SGT) Date: Mon, 16 Jun 2008 17:46:50 -0700 From: Darren Reed Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> Sender: Darren.Reed@Sun.COM To: Jerry Gilliam Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Sherry.Moore@Sun.COM, Jerry.Gilliam@Sun.COM Message-id: <4857097A.9000906@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-au, en References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20060120 Content-Length: 4568 Status: RO X-Status: $$$$ X-UID: 0000000047 Jerry Gilliam wrote: >... > >OPTIONS > The following options are supported: > > -d Force a system crash dump before rebooting. See > dumpadm(1M) for information on configuring system > crash dumps. > > > -e If -f is present, reboot to the specified boot | > environment. | > | > | > -f Fast reboot bypassing firmware and boot loader. The | > new kernel will be loaded into memory by the running | > kernel, and control will be transferred to the loaded | > kernel. If disk or kernel arguments are specified, | > they must be specified before other boot arguments. | > See Example 3 for details. | > | > Currently only available on x86 system. | > | > > -l Suppress sending a message to the system log daemon, > syslogd(1M) about who executed reboot. > > > -n Avoid calling sync(2) and do not log the reboot to > syslogd(1M) or to /var/adm/wtmpx. The kernel still > attempts to sync filesystems prior to reboot, except > if the -d option is also present. If -d is used with > -n, the kernel does not attempt to sync filesystems. > > > -q Quick. Reboot quickly and ungracefully, without shut- > ting down running processes first. > > >OPERANDS > The following operands are supported: > > boot_arguments An optional boot_arguments specifies argu- > ments to the uadmin(2) function that are > passed to the boot program and kernel upon > restart. The form and list of arguments is > described in the boot(1M) and kernel(1M) > man pages.. If the arguments are speci- > fied, whitespace between them is replaced > by single spaces unless the whitespace is > quoted for the shell. If the > boot_arguments begin with a hyphen, they > must be preceded by the -- delimiter (two > hyphens) to denote the end of the reboot > argument list. > >... > Example 3 Fast reboot | > | > Check if all the drivers on the system are fast reboot capable. | > | > example# reboot -f dryrun | > | > Rebooting to another UFS root disk. | > | > example# reboot -f -- '/dev/dsk/c1d0s0' | > | > Rebooting to another ZFS root pool. | > | > example# reboot -f -- 'rootpool/root1' | > | > Rebooting to "mykernel" on the same disk with "-k" option. | > | > example# reboot -f -- '/platform/i86pc/mykernel/amd64/unix -k' | > | > Rebooting to "mykernel" off another root disk mounted on /mnt. | > | > example# reboot -f -- '/mnt/platform/i86pc/mykernel/amd64/unix -k' | > | > Rebooting to "/platform/i86pc/kernel/$ISADIR/unix" on another boot | > environment named "second_root". | > | > example# reboot -f -e second_root | > | > Rebooting to the same kernel with "-kv" options. | > | > example# reboot -f -- '-kv' | > Hmmm... My only comment from the above is that the architecture of reboot(1m) doesn't make it clear which of the sub commands come from uadmin(2), boot(1m) or kernel(1m). This leads to a number of comments that are documentation related, and perhaps not architectural. For example, if the "reboot -f dryrun" wasn't present, how would one even guess that this is valid? And how would I know to use "dryrun" rather than "DRYRUN" or some other magic string? Does its presence meant that I can also do "reboot -f remount"? It may be that the faults in this man page aren't all the problem of this project but it would be nice if reboot(1M) became more meaningful as a result of this update and documented at least which of the uadmin(2) options were supported by which args to reboot(1m). The examples where you've used '--' are also confusing as it conflicts with the documentation about that says they only need to be used when the next arg start with a '-'. e.g. why isn't it # reboot -f /mnt/platform/i86pc/mykernel/amd64/unix -- -k Given the situation is now (or always was) more complex, I'd suggest either updating the man page further or keeping your examples in line with the existing docs. Darren From Darren.Reed@Sun.COM Mon Jun 16 17:54:54 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H0srA2022075 for ; Mon, 16 Jun 2008 17:54:53 -0700 (PDT) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5H0sqdQ039939 for ; Mon, 16 Jun 2008 17:54:53 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m5H0tBGL025805 for ; Tue, 17 Jun 2008 00:55:11 GMT Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0K2L00I0113OG700@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Tue, 17 Jun 2008 08:54:23 +0800 (SGT) Received: from [129.146.106.55] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0K2L003VR16LJPO2@mail-apac.sun.com>; Tue, 17 Jun 2008 08:54:23 +0800 (SGT) Date: Mon, 16 Jun 2008 17:54:44 -0700 From: Darren Reed Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080616165933.GA11878@sun.com> Sender: Darren.Reed@Sun.COM To: Sherry Moore Cc: Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Jerry.Gilliam@Sun.COM Message-id: <48570B54.80706@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-au, en References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <485658F1.6010700@Sun.COM> <20080616151121.GX2735@Sun.COM> <20080616165933.GA11878@sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.7) Gecko/20060120 Content-Length: 2085 Status: RO X-Status: $$$$ X-UID: 0000000048 Sherry, In the examples given on the man pages, you have reboot activating a kernel on a completely different disk. In such a situation, there is no reason why the current kernel would have a hash stored in its "database" that covers this new kernel - thus passing the O_VERIFY flag would be meaningless. But this isn't enough. The hash only verifies that the file on disk that you're going to execute matches a previously recorded hash for it. It doesn't engage the TPM in any way and nor can you make any assertions about the trustworthiness of the new binary. Darren Sherry Moore wrote: >Nico is correct: the running kernel can easily validate the new kernel >and boot archive by explicitly passing O_VERIFY when opening the >files. Once entering the new kernel, the execution is no different >from a regular boot. > >Nico, since the initial manifest itself (used before the the validation >daemon is fully initialized) is included in the boot archive, I don't >think there is any data that the running kernel needs to pass to the >new kernel to access the TPM. If you believe that there is information >that needs to be passed, let's chat offline to see how to best achieve >the purpose. > >Thanks, >Sherry > >On Mon, Jun 16, 2008 at 10:11:22AM -0500, Nicolas Williams wrote: > > >>On Mon, Jun 16, 2008 at 02:13:37PM +0200, Joep Vesseur wrote: >> >> >>>Restarting the kernel as proposed by this case will either run unverified code >>>(at the vary least, not every step of the boot-process is checked sequentially >>>anymore) or the registers used to record the validation will no longer unlock >>>the registers containing the sensitive data needed to continue the boot >>>process. >>> >>> >>Well, the TPM need not know (because its driver might not actually fully >>reset it on quiesce?) that a new kernel is replacing the old one. The >>old kernel was trusted and it can do signature verification of the new >>kernel. And the old kernel could pass to the new kernel any data the >>new kernel will need to access the TPM. >> >>Nico >>-- >> >> > > > From gdamore@sun.com Mon Jun 16 18:38:41 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H1cfdU022801 for ; Mon, 16 Jun 2008 18:38:41 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5H1cbqd057392 for ; Mon, 16 Jun 2008 18:38:41 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5H1cWUF004162 for ; Mon, 16 Jun 2008 18:38:32 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2L00J0134AKT00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Mon, 16 Jun 2008 18:38:32 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2L00MUP386BI60@fe-sfbay-10.sun.com>; Mon, 16 Jun 2008 18:38:31 -0700 (PDT) Date: Mon, 16 Jun 2008 18:37:11 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <4857086A.2050108@sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: Sherry Moore , David Kahn , jg@jurassic.sfbay.sun.com, psarc-ext@sac.sfbay.sun.com, Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, Seth.Goldberg@sun.com Message-id: <48571547.3030509@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <4857009F.8080206@sun.com> <4857086A.2050108@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1978 Status: RO X-Status: $$$$ X-UID: 0000000049 David Kahn wrote: > > > Garrett D'Amore wrote: > >> Yes, you could. Properties (as opposed to dev ops flags) are a PITA. > > A flag is fine. Same idea. I'm not really hung up on expanding dev_ops > vs adding a new detach command with a flag/property. I was just pointing > out that if the project team doesn't want to expand dev_ops, there is a > decent alternative available. > >>> This needs a bit of work, IMO. >> >> This sound less architectural, and more wordsmithing. I won't >> disagree that the docs can be cleaned up, made clearer.... > > No, this is the specification of the interface. It isn't wordsmithing. > The specification of the interface is what's important here and it > must be crystal clear and not subject to different interpretations. > >> Ah, but what I'm saying is that the kernel should have disabled all >> interrupts from occurring at the point that quiesce is called. So >> those events, which may occur in hardware, won't trigger an ISR >> execution. No locking is required. > > It must be specified that way. Not the single threaded part, > but the guarantee that timers, callbacks, interrupt service routines > and all other async callbacks and other driver entry points will not be > called or active when the driver is in quiesce(9e). If that's the > interface, > then it must be specified that way as part of the case materials. > > PSARC is reviewing the interface specification. The interface > specification is not sufficient, so it needs work before it can > be approved so all these things that have been mentioned in the > replies are addressed and made part of the interface specification. I agree they should be part of the interface specification. But anyway, it sounds like the current implementation (for better or worse -- I think worse) isn't implemented this way. I'd frankly rather have this kind of wording, and implementation to match, to keep life simple for everyone. -- Garrett > > Thanks, > David > > From Nicolas.Williams@sun.com Mon Jun 16 19:53:26 2008 Received: from binky.Central.Sun.COM (binky.Central.Sun.COM [129.153.128.104]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5H2rPLW024954 for ; Mon, 16 Jun 2008 19:53:26 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id m5H2rOLo003093; Mon, 16 Jun 2008 21:53:24 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id m5H2rNtK003092; Mon, 16 Jun 2008 21:53:23 -0500 (CDT) X-Authentication-Warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f Date: Mon, 16 Jun 2008 21:53:23 -0500 From: Nicolas Williams To: Darren Reed Cc: Sherry Moore , Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080617025323.GC2735@Sun.COM> Mail-Followup-To: Darren Reed , Sherry Moore , Joep Vesseur , Jerry Gilliam , psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@Sun.COM, Darrin.Johnson@Sun.COM, Neal.Pollack@Sun.COM, Jerry.Gilliam@Sun.COM References: <200806140021.m5E0LslG232387@jurassic.eng.sun.com> <485658F1.6010700@Sun.COM> <20080616151121.GX2735@Sun.COM> <20080616165933.GA11878@sun.com> <48570B54.80706@Sun.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48570B54.80706@Sun.COM> User-Agent: Mutt/1.5.7i Content-Length: 611 Status: RO X-Status: $$$$ X-UID: 0000000050 On Mon, Jun 16, 2008 at 05:54:44PM -0700, Darren Reed wrote: > But this isn't enough. The hash only verifies that the file on disk that > you're going to execute matches a previously recorded hash for it. > It doesn't engage the TPM in any way and nor can you make any > assertions about the trustworthiness of the new binary. So? That's true of *every* bit of executable code loaded and run by the kernel after crypto services are running, and that includes kernel loadable modules and user-land programs and libraries. I don't see why the new kernel must be verified by the TPM instead of the old kernel. From carlsonj@phorcys.east.sun.com Tue Jun 17 05:20:09 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HCK94t008368 for ; Tue, 17 Jun 2008 05:20:09 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5HCK65X027006; Tue, 17 Jun 2008 08:20:06 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m5HCK6dx027003; Tue, 17 Jun 2008 08:20:06 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18519.44022.398055.521491@gargle.gargle.HOWL> Date: Tue, 17 Jun 2008 08:20:06 -0400 From: James Carlson To: David Kahn Cc: Sherry Moore , Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com, jg@jurassic.sfbay.sun.com, David Kahn , Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 In-Reply-To: <4856FD70.60901@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Content-Length: 2251 Status: RO X-Status: $$$$ X-UID: 0000000051 David Kahn writes: > I'm not sure if that's enough. What if open, close, etc are in > progress when the framework decides to call quiesce. Is that > possible? So maybe it has to be worded even stronger such that Not just possible, but likely. The serial drivers all do this normally; a blocking open(9E) waiting for carrier is the norm. It also shouldn't be a problem there. The open(9E) is blocked, but the hardware is idle. It's harmless to shut it down. > the guarantee is that open, close, attach, detach and other driver > entry points will not be active or called when the framework calls > quiesce(9e). That still leaves the question of interrupt handlers, > timers, callbacks, etc that could be called when the driver is > in quiesce(9e) or might be active when the quiesce(9e) is called. That means the system can never call quiesce(9E) on (at least) serial ports. That doesn't sound like a good answer to me. > This needs a bit of work, IMO. Yes. I'm not trying to design it here, but one possible answer to consider would be: quiesce(9E) may be called while other threads are active in a driver's other entry points. However, the framework guarantees that no new calls to open(9E), close(9E), or the other entry points will be made during the execution of quiesce(9E). > It seems like what you want to do is stop interrupt dispatch, > stop callback threads, timer callbacks, soft ints, etc, and then > just call into the driver to tell it to stop whatever it might be I don't see how that's possible. There's nothing in the system that knows about existing timeout(9F) callout records on a per-driver basis, so there's no way to stop them before entry. I suspect that all of that, if it's necessary, must be the driver's responsibility. (My understanding of this quiesce proposal is that it would _not_ be necessary to disable callbacks and the like. All that's necessary is that DMA and interrupts on the hardware are shut down so that nothing outside of the CPU itself will be doing work.) -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From gdamore@sun.com Tue Jun 17 09:37:58 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HGbwjT021991 for ; Tue, 17 Jun 2008 09:37:58 -0700 (PDT) Received: from sca-es-mail-2.sun.com (sca-es-mail-2.Sun.COM [192.18.43.133]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5HGbvxA054008 for ; Tue, 17 Jun 2008 09:37:57 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5HGbqnW026661 for ; Tue, 17 Jun 2008 09:37:52 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2M00F018KNLU00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Tue, 17 Jun 2008 09:37:52 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2M008OE8UTICB0@fe-sfbay-09.sun.com>; Tue, 17 Jun 2008 09:37:42 -0700 (PDT) Date: Tue, 17 Jun 2008 09:36:19 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <18519.44022.398055.521491@gargle.gargle.HOWL> Sender: Garrett.Damore@sun.com To: James Carlson Cc: David Kahn , Sherry Moore , Darrin.Johnson@sun.com, Neal.Pollack@sun.com, Jerry.Gilliam@sun.com, jg@jurassic.sfbay.sun.com, David Kahn , Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Message-id: <4857E803.4090902@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1347 Status: RO X-Status: $$$$ X-UID: 0000000052 James Carlson wrote: > David Kahn writes: > > > >> the guarantee is that open, close, attach, detach and other driver >> entry points will not be active or called when the framework calls >> quiesce(9e). That still leaves the question of interrupt handlers, >> timers, callbacks, etc that could be called when the driver is >> in quiesce(9e) or might be active when the quiesce(9e) is called. >> > > That means the system can never call quiesce(9E) on (at least) serial > ports. That doesn't sound like a good answer to me. > Why? It seems (to me at least), that if there is something that needs a timeout, the quiesce routine should be able to discover that a timeout was pending, and poll for completion on its own. Maybe I'm just not understanding your concern here. >> It seems like what you want to do is stop interrupt dispatch, >> stop callback threads, timer callbacks, soft ints, etc, and then >> just call into the driver to tell it to stop whatever it might be >> > > I don't see how that's possible. There's nothing in the system that > knows about existing timeout(9F) callout records on a per-driver > basis, so there's no way to stop them before entry. > Not individually, but *all* interrupts would be stopped. The system should be running with all interrupts on the CPU disabled. -- Garrett From sherry.moore@sun.com Tue Jun 17 10:31:15 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HHVFET027214 for ; Tue, 17 Jun 2008 10:31:15 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5HHVFrh017034; Tue, 17 Jun 2008 10:31:15 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5HHVFOh017033; Tue, 17 Jun 2008 10:31:15 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Tue, 17 Jun 2008 10:31:15 -0700 From: Sherry Moore To: psarc-ext@sac.sfbay.sun.com, david.kahn@sun.com Cc: darrin.johnson@sun.com, jerry.gilliam@sun.com, neal.pollack@sun.com, seth.goldberg@sun.com, sherry.moore@sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080617173114.GB16403@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> User-Agent: Mutt/1.4.1i Content-Length: 4314 Status: RO X-Status: $$$$ X-UID: 0000000053 David, Jerry and the project team worked offline and we have revised the man pages for quiesce(9E). ============================================================================ Driver Entry Points quiesce(9E) NAME quiesce - quiesce a device SYNOPSIS #include #include int prefix quiesce(dev_info_t dip); int ddi_no_quiesce(dev_info_t *dip); int ddi_quiesce_not_supported(dev_info_t *dip); INTERFACE LEVEL Solaris DDI specific (Solaris DDI) PARAMETERS dip A pointer to the device's dev_info structure. DESCRIPTION The quiesce() function quiesces a device so that the device will no longer generate interrupts or modify or access memory. The driver should reset the device to a hardware state from which the device can be correctly configured by the driver's attach() routine without a system power cycle or being configured by the firmware. For devices with a defined reset state configuration, the driver should return that device to that state as part of the quiesce operation. One such use case is Fast Reboot where firmware is bypassed when booting to a new OS image. Quiesce() is only called for an attached device instance as one of the final operations of a reboot sequence, and no other thread can be active for this device. The system guarantees that no other driver entry point will be active or invoked quiesce() is invoked. The system also guarantees that no timeout or taskq will be invoked. The system is single-threaded and not preemptable or interrupted, therefore the driver's quiesce() implementation must not use locks or timeouts or rely on them being called. The driver must discard all outstanding I/O instead of waiting for completion. By conclusion of the quiesce() operation, the driver must guarantee that device will not generate further access to memory or interrupts. The only DDI interfaces that can be called by the quiesce() implementation are the ddi_get*(9F) and ddi_put*(9F) functions. The driver must not create or tear down mappings, or call FMA functions, or create/cancel callbacks. If quiesce() determines a particular instance of the device cannot be quiesced when requested because of some exceptional condition, quiesce() must return DDI_FAILURE. This should almost never happen. If a driver has previously implemented the obsolete reset() interface, its functionality must be merged into quiesce(). The driver's reset() routine will no longer be called if an implementation of quiesce() is present. ddi_no_quiesce() always returns DDI_SUCCESS. A driver can set its devo_quiesce device function to ddi_no_quiesce() to indicate that the device it manages does not need to be quiesced. ddi_quiesce_not_supported() always returns DDI_FAILURE. A driver can set its devo_quiesce device function to ddi_quiesce_not_supported() to indicate that either the device cannot be quiesced, or quiesce() has not been implemented. RETURN VALUES DDI_SUCCESS For quiesce(), the device has been successfully quiesced. DDI_FAILURE The operation failed. CONTEXT This function is called from kernel context only. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Interface Stability | Committed | |_____________________________|_____________________________| SEE ALSO attach(9E), detach(9E), ddi_add_intr(9F), ddi_map_regs(9F), pci_config_setup(9F), ddi_get8(9F), ddi_put8(9F), ddi_no_quiesce(9F), ddi_quiesce_not_supported(9F), dev_ops(9S), timeout(9F), reboot(1M), uadmin(1M), uadmin(2), ============================================================================ Please review and send me additional comments, and I will revise it further. Thanks, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From carlsonj@phorcys.east.sun.com Tue Jun 17 11:31:56 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HIVtWZ000144 for ; Tue, 17 Jun 2008 11:31:55 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5HIVlWs028800; Tue, 17 Jun 2008 14:31:47 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m5HIVlGx028797; Tue, 17 Jun 2008 14:31:47 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18520.787.176785.832316@gargle.gargle.HOWL> Date: Tue, 17 Jun 2008 14:31:47 -0400 From: James Carlson To: "Garrett D'Amore" Cc: David Kahn , Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, David Kahn , Sherry Moore , jg@jurassic.sfbay.sun.com, Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 In-Reply-To: <4857E803.4090902@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> <4857E803.4090902@sun.com> X-Mailer: VM 7.01 under Emacs 21.3.1 Content-Length: 1891 Status: RO X-Status: $$$$ X-UID: 0000000054 Garrett D'Amore writes: > James Carlson wrote: > > David Kahn writes: > >> the guarantee is that open, close, attach, detach and other driver ^^^^ > >> entry points will not be active or called when the framework calls ^^^^^^^^^^^^^^^^^^ > >> quiesce(9e). That still leaves the question of interrupt handlers, > >> timers, callbacks, etc that could be called when the driver is > >> in quiesce(9e) or might be active when the quiesce(9e) is called. > >> > > > > That means the system can never call quiesce(9E) on (at least) serial > > ports. That doesn't sound like a good answer to me. > > > > Why? It seems (to me at least), that if there is something that needs a > timeout, the quiesce routine should be able to discover that a timeout > was pending, and poll for completion on its own. The problem is indicated above. open(9E) is almost always "active" on a serial port, because there are threads there snoozing away, waiting for someone to assert DCD. > >> It seems like what you want to do is stop interrupt dispatch, > >> stop callback threads, timer callbacks, soft ints, etc, and then > >> just call into the driver to tell it to stop whatever it might be > >> > > > > I don't see how that's possible. There's nothing in the system that > > knows about existing timeout(9F) callout records on a per-driver > > basis, so there's no way to stop them before entry. > > > > Not individually, but *all* interrupts would be stopped. The system > should be running with all interrupts on the CPU disabled. It didn't sound to me like that's what the original project was proposing, but ok. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From jek3@sun.com Tue Jun 17 11:38:34 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HIcY4n000177 for ; Tue, 17 Jun 2008 11:38:34 -0700 (PDT) Received: from [129.150.13.200] (vpn-129-150-13-200.SFBay.Sun.COM [129.150.13.200]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5HIcSG3244166; Tue, 17 Jun 2008 11:38:32 -0700 (PDT) Message-ID: <48580554.1050401@sun.com> Date: Tue, 17 Jun 2008 08:41:24 -1000 From: Joseph Kowalski User-Agent: Thunderbird 2.0.0.9 (X11/20080225) MIME-Version: 1.0 To: James Carlson CC: "Garrett D'Amore" , David Kahn , Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, David Kahn , Sherry Moore , jg@jurassic.sfbay.sun.com, Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> <4857E803.4090902@sun.com> <18520.787.176785.832316@gargle.gargle.HOWL> In-Reply-To: <18520.787.176785.832316@gargle.gargle.HOWL> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 218 Status: RO X-Status: $$$$ X-UID: 0000000055 I'm seeing a lot of good, productive discussion on this, but the discussion is getting rather long. Should this case be moved to "waiting need spec" while the "discussion group" gets this solved off-line? - jek3 From David.Kahn@sun.com Tue Jun 17 15:41:39 2008 Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HMfdsZ010333 for ; Tue, 17 Jun 2008 15:41:39 -0700 (PDT) Received: from [192.168.0.9] (noho.SFBay.Sun.COM [10.6.92.101]) by dtmail.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5HMfbY5018471; Tue, 17 Jun 2008 15:41:38 -0700 (PDT) Message-ID: <48583DA1.3040107@sun.com> Date: Tue, 17 Jun 2008 15:41:37 -0700 From: David Kahn User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: James Carlson CC: "Garrett D'Amore" , Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, David Kahn , Sherry Moore , jg@jurassic.sfbay.sun.com, Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> <4857E803.4090902@sun.com> <18520.787.176785.832316@gargle.gargle.HOWL> In-Reply-To: <18520.787.176785.832316@gargle.gargle.HOWL> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 764 Status: RO X-Status: $$$$ X-UID: 0000000056 James Carlson wrote: > The problem is indicated above. open(9E) is almost always "active" on > a serial port, because there are threads there snoozing away, waiting > for someone to assert DCD. If that's the case, it's impossible to do this safely without locks. There's no way for the driver to guard against both threads being active at the same time. I believe that the framework must guarantee that there will only be a single thread active in the driver at a time for this to work they way they want it to work. In that case, if quiesce(9e) is active, even if DCD is asserted, that thread won't become active at the same time that the driver is in quiesce(9e) and there won't be two threads competing over the registers/resources of the device. -David From sherry.moore@sun.com Tue Jun 17 16:44:49 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HNinG7013643 for ; Tue, 17 Jun 2008 16:44:49 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5HNinAI017980; Tue, 17 Jun 2008 16:44:49 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5HNinUx017979; Tue, 17 Jun 2008 16:44:49 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Tue, 17 Jun 2008 16:44:49 -0700 From: Sherry Moore To: David Kahn Cc: James Carlson , "Garrett D'Amore" , Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, David Kahn , Sherry Moore , jg@jurassic.sfbay.sun.com, Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080617234447.GK16403@sun.com> References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> <4857E803.4090902@sun.com> <18520.787.176785.832316@gargle.gargle.HOWL> <48583DA1.3040107@sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48583DA1.3040107@sun.com> User-Agent: Mutt/1.4.1i Content-Length: 1081 Status: RO X-Status: $$$$ X-UID: 0000000057 Yes the framework will guarantee when quiesce() is invoked there will be only a single thread running at LOCK_LEVEL on a single CPU. Thanks, Sherry On Tue, Jun 17, 2008 at 03:41:37PM -0700, David Kahn wrote: > > > James Carlson wrote: > > >The problem is indicated above. open(9E) is almost always "active" on > >a serial port, because there are threads there snoozing away, waiting > >for someone to assert DCD. > > If that's the case, it's impossible to do this safely without > locks. There's no way for the driver to guard against both > threads being active at the same time. > > I believe that the framework must guarantee that there will > only be a single thread active in the driver at a time for > this to work they way they want it to work. In that case, > if quiesce(9e) is active, even if DCD is asserted, that thread > won't become active at the same time that the driver is in > quiesce(9e) and there won't be two threads competing over > the registers/resources of the device. > > -David -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From sherry.moore@sun.com Tue Jun 17 16:59:37 2008 Received: from geralyn.sfbay.sun.com (geralyn.SFBay.Sun.COM [129.146.226.228]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5HNxboT014602 for ; Tue, 17 Jun 2008 16:59:37 -0700 (PDT) Received: from geralyn.sfbay.sun.com (localhost [127.0.0.1]) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.1) with ESMTP id m5HNxbv4018385; Tue, 17 Jun 2008 16:59:37 -0700 (PDT) Received: (from sherrym@localhost) by geralyn.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m5HNxb1V018384; Tue, 17 Jun 2008 16:59:37 -0700 (PDT) X-Authentication-Warning: geralyn.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Tue, 17 Jun 2008 16:59:37 -0700 From: Sherry Moore To: psarc-ext@sac.sfbay.sun.com Cc: Seth Goldberg , Darrin Johnson , Neal Pollack , Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080617235936.GL16403@sun.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="zhXaljGHf11kAtnf" Content-Disposition: inline User-Agent: Mutt/1.4.1i Content-Length: 14339 Status: RO X-Status: $$$$ X-UID: 0000000058 --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi folks, The project team has resolved most issues offline with various parties. I have attached the updated versions of the specs reflecting the resolution. Please send me mail if you don't believe issues you raised have been addressed. Scott Rotondo will be posting summary with regard to interaction between this project and PSARC 2008/195 "Validated Execution". Thanks much, Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="quiesce.man.txt" Driver Entry Points quiesce(9E) NAME quiesce - quiesce a device SYNOPSIS #include #include int prefix quiesce(dev_info_t dip); int ddi_quiesce_not_needed(dev_info_t *dip); int ddi_quiesce_not_supported(dev_info_t *dip); INTERFACE LEVEL Solaris DDI specific (Solaris DDI) PARAMETERS dip A pointer to the device's dev_info structure. DESCRIPTION The quiesce() function quiesces a device so that the device will no longer generate interrupts or modify or access memory. The driver should reset the device to a hardware state from which the device can be correctly configured by the driver's attach() routine without a system power cycle or being configured by the firmware. For devices with a defined reset state configuration, the driver should return that device to that state as part of the quiesce operation. One such use case is Fast Reboot where firmware is bypassed when booting to a new OS image. Quiesce is only called for an attached device instance as one of the final operations of a reboot sequence, and no other thread can be active for this device. The system guarantees that no other driver entry point will be active or invoked quiesce(9E) is invoked. The system also guarantees that no timeout or taskq will be invoked. The system is single-threaded and not preemptable or interrupted, therefore the driver's quiesce() implementation must not use locks or timeouts or rely on them being called. The driver must discard all outstanding I/O instead of waiting for completion. By conclusion of the quiesce() operation, the driver must guarantee that device will not generate further access to memory or interrupts. The only DDI interfaces that can be called by the quiesce() implementation are non-blocking functions, such as ddi_get*(9F) and ddi_put*(9F). If quiesce() determines a particular instance of the device cannot be quiesced when requested because of some exceptional condition, quiesce() must return DDI_FAILURE. This should almost never happen. If a driver has previously implemented the obsolete reset() interface, its functionality must be merged into quiesce(). The driver's reset() routine will no longer be called if an implementation of quiesce() is present. ddi_quiesce_not_needed() always returns DDI_SUCCESS. A driver can set its devo_quiesce device function to ddi_quiesce_not_needed() to indicate that the device it manages does not need to be quiesced. ddi_quiesce_not_supported() always returns DDI_FAILURE. A driver can set its devo_quiesce device function to ddi_quiesce_not_supported() to indicate that either the device cannot be quiesced, or quiesce() has not been implemented. RETURN VALUES DDI_SUCCESS For quiesce(), the device has been successfully quiesced. DDI_FAILURE The operation failed. CONTEXT This function is called from kernel context only. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Interface Stability | Committed | |_____________________________|_____________________________| SEE ALSO attach(9E), detach(9E), ddi_add_intr(9F), ddi_map_regs(9F), pci_config_setup(9F), timeout(9F), reboot(1M), uadmin(1M), uadmin(2), ddi_quiesce_not_needed(9F), ddi_quiesce_not_supported(9F), dev_ops(9S) NOTES When quiesce() is called, the system is single-threaded, therefore the driver's quiesce() implementation must not block. For example, the implementation must not create or tear down mappings, or call FMA functions, or create/cancel callbacks. --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dev_ops.man.txt" Data Structures for Drivers dev_ops(9S) NAME dev_ops - device operations structure SYNOPSIS #include #include INTERFACE LEVEL Solaris DDI specific (Solaris DDI). DESCRIPTION dev_ops contains driver common fields and pointers to the bus_ops and cb_ops(9S). Following are the device functions provided in the device operations structure. All fields must be set at compile time. devo_rev Driver build version. Set this to DEVO_REV. devo_refcnt Driver reference count. Set this to 0. devo_getinfo Get device driver information (see getinfo(9E)). devo_identify This entry point is obsolete. Set to nulldev. devo_probe Probe device. See probe(9E). devo_attach Attach driver to dev_info. See attach(9E). devo_detach Detach/prepare driver to unload. See detach(9E). devo_reset Reset device. (Not supported in this release.) Set this to nodev. devo_cb_ops Pointer to cb_ops(9S) structure for leaf drivers. SunOS 5.11 Last change: 11 Apr 2003 1 Data Structures for Drivers dev_ops(9S) devo_bus_ops Pointer to bus operations structure for nexus drivers. Set this to NULL if this is for a leaf driver. devo_power Power a device attached to system. See power(9E). devo_quiesce Quiesce a device attached to system. See quiesce(9E). Can be set to ddi_quiesce_not_needed(9F) if the driver does not need to implement quiesce, or set to ddi_quiesce_not_supported(9F) if the driver cannot quiesce device to support fast-reboot. STRUCTURE MEMBERS int devo_rev; int devo_refcnt; int (*devo_getinfo)(dev_info_t *dip, ddi_info_cmd_t infocmd, void *arg, void **result); int (*devo_identify)(dev_info_t *dip); int (*devo_probe)(dev_info_t *dip); int (*devo_attach)(dev_info_t *dip, ddi_attach_cmd_t cmd); int (*devo_detach)(dev_info_t *dip, ddi_detach_cmd_t cmd); int (*devo_reset)(dev_info_t *dip, ddi_reset_cmd_t cmd); struct cb_ops *devo_cb_ops; struct bus_ops *devo_bus_ops; int (*devo_power)(dev_info_t *dip, int component, int level); int (*devo_quiesce)(dev_info_t *dip); SEE ALSO attach(9E), detach(9E), getinfo(9E), probe(9E), power(9E), quiesce(9E), nodev(9F), ddi_quiesce_not_needed(9F), ddi_quiesce_not_supported(9F) --zhXaljGHf11kAtnf Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="reboot.man.txt" System Administration Commands reboot(1M) NAME reboot - restart the operating system SYNOPSIS /usr/sbin/reboot [-dlnqf] [-e BE] [boot_arguments] DESCRIPTION The reboot utility restarts the kernel. The kernel is loaded into memory by the PROM monitor, which transfers control to the loaded kernel. On x86 systems, when the -f flag is specified, the running kernel will load the next kernel into memory, then transfers control to the loaded kernel. Although reboot can be run by the super-user at any time, shutdown(1M) is normally used first to warn all users logged in of the impending loss of service. See shutdown(1M) for details. The reboot utility performs a sync(1M) operation on the disks, and then a multi-user reboot is initiated. See init(1M) for details. On x86 systems, reboot may also update the boot archive as needed to ensure a successful reboot. The reboot utility normally logs the reboot to the system log daemon, syslogd(1M), and places a shutdown record in the login accounting file /var/adm/wtmpx. These actions are inhibited if the -n or -q options are present. Normally, the system reboots itself at power-up or after crashes. OPTIONS The following options are supported: -d Force a system crash dump before rebooting. See dumpadm(1M) for information on configuring system crash dumps. -e If -f is present, reboot to the specified boot environment (BE) as specified by live_upgrade(5). -f Fast reboot bypassing firmware and boot loader. The new kernel will be loaded into memory by the running kernel, and control will be transferred to the loaded kernel. If disk or kernel arguments are specified, they must be specified before other boot arguments. When -f is present, reboot(1M) accepts a special argument "dryrun" to check whether all the drivers in the system have implemented quiesce(9E). Currently only available on x86 system. See Example 3 for details. -l Suppress sending a message to the system log daemon, syslogd(1M) about who executed reboot. -n Avoid calling sync(2) and do not log the reboot to syslogd(1M) or to /var/adm/wtmpx. The kernel still attempts to sync filesystems prior to reboot, except if the -d option is also present. If -d is used with -n, the kernel does not attempt to sync filesystems. SunOS 5.11 Last change: 11 Apr 2005 1 System Administration Commands reboot(1M) -q Quick. Reboot quickly and ungracefully, without shut- ting down running processes first. OPERANDS The following operands are supported: boot_arguments An optional boot_arguments specifies argu- ments to the uadmin(2) function that are passed to the boot program and kernel upon restart. The form and list of arguments is described in the boot(1M) and kernel(1M) man pages.. If the arguments are speci- fied, whitespace between them is replaced by single spaces unless the whitespace is quoted for the shell. If the boot_arguments begin with a hyphen, they must be preceded by the -- delimiter (two hyphens) to denote the end of the reboot argument list. EXAMPLES Example 1 Passing the -r and -v Arguments to boot In the following example, the delimiter -- (two hyphens) must be used to separate the options of reboot from the arguments of boot(1M). example# reboot -dl -- -rv Example 2 Rebooting Using a Specific Disk and Kernel The following example reboots using a specific disk and ker- nel. example# reboot disk1 kernel.test/unix Example 3 Fast reboot Check if all the drivers on the system are fast reboot capable. example# reboot -f dryrun Rebooting to another UFS root disk. example# reboot -f -- '/dev/dsk/c1d0s0' Rebooting to another ZFS root pool. example# reboot -f -- 'rootpool/root1' Rebooting to "mykernel" on the same disk with "-k" option. example# reboot -f -- '/platform/i86pc/mykernel/amd64/unix -k' Rebooting to "mykernel" off another root disk mounted on /mnt. example# reboot -f -- '/mnt/platform/i86pc/mykernel/amd64/unix -k' Rebooting to "/platform/i86pc/kernel/$ISADIR/unix" on another boot environment named "second_root". example# reboot -f -e second_root Rebooting to the same kernel with "-kv" options. example# reboot -f -- '-kv' FILES /var/adm/wtmpx login accounting file ATTRIBUTES SunOS 5.11 Last change: 11 Apr 2005 2 System Administration Commands reboot(1M) See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Availability | SUNWcsu | |_____________________________|_____________________________| SEE ALSO mdb(1), boot(1M), dumpadm(1M), fsck(1M), halt(1M), init(1M), kernel(1M), shutdown(1M), sync(1M), syslogd(1M), sync(2), uadmin(2), reboot(3C), attributes(5) NOTES The reboot utility does not execute the scripts in /etc/rcnum.d or execute shutdown actions in inittab(4). To ensure a complete shutdown of system services, use shutdown(1M) or init(1M) to reboot a Solaris system. SunOS 5.11 Last change: 11 Apr 2005 3 --zhXaljGHf11kAtnf-- From gdamore@sun.com Tue Jun 17 23:36:47 2008 Received: from dm-sfbay-02.sfbay.sun.com (dm-sfbay-02.SFBay.Sun.COM [129.146.11.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5I6alCB021775 for ; Tue, 17 Jun 2008 23:36:47 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5I6alhR048887 for ; Tue, 17 Jun 2008 23:36:47 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5I6agpZ029590 for ; Tue, 17 Jun 2008 23:36:42 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2N00A01BM2ZO00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Tue, 17 Jun 2008 23:36:42 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2N00A9KBP5W900@fe-sfbay-09.sun.com>; Tue, 17 Jun 2008 23:36:42 -0700 (PDT) Date: Tue, 17 Jun 2008 23:35:18 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <48583DA1.3040107@sun.com> Sender: Garrett.Damore@sun.com To: David Kahn Cc: James Carlson , Darrin.Johnson@sun.com, Jerry.Gilliam@sun.com, Neal.Pollack@sun.com, David Kahn , Sherry Moore , jg@jurassic.sfbay.sun.com, Seth.Goldberg@sun.com, psarc-ext@sac.sfbay.sun.com Message-id: <4858ACA6.10202@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806160811.m5G8Bpqg022463@noho.sfbay.sun.com> <20080616203746.GT11878@sun.com> <4856FD70.60901@sun.com> <18519.44022.398055.521491@gargle.gargle.HOWL> <4857E803.4090902@sun.com> <18520.787.176785.832316@gargle.gargle.HOWL> <48583DA1.3040107@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 1298 Status: RO X-Status: $$$$ X-UID: 0000000059 David Kahn wrote: > > > James Carlson wrote: > >> The problem is indicated above. open(9E) is almost always "active" on >> a serial port, because there are threads there snoozing away, waiting >> for someone to assert DCD. > > If that's the case, it's impossible to do this safely without > locks. There's no way for the driver to guard against both > threads being active at the same time. > > I believe that the framework must guarantee that there will > only be a single thread active in the driver at a time for > this to work they way they want it to work. In that case, > if quiesce(9e) is active, even if DCD is asserted, that thread > won't become active at the same time that the driver is in > quiesce(9e) and there won't be two threads competing over > the registers/resources of the device. All this is moot, IMO. The framework should be fully and completely single threaded at the time quiesce() is called, IMO. There are no compelling reasons why locking, multiple threads, or any thing much beyond ddi_putXX, ddi_getXX are required. All the driver should be doing is shutting down or resetting the device to initial power on state. We should be well past the point where anything that might have been happening in other threads is interesting. -- Garrett > > -David From gdamore@sun.com Tue Jun 17 23:39:15 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5I6dEKl021791 for ; Tue, 17 Jun 2008 23:39:14 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5I6dE7I005190 for ; Tue, 17 Jun 2008 23:39:14 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5I6d9nx029658 for ; Tue, 17 Jun 2008 23:39:09 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K2N00901BPYT200@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for psarc-ext@sac.sfbay.sun.com; Tue, 17 Jun 2008 23:39:09 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K2N00GWIBT8HLE0@fe-sfbay-10.sun.com>; Tue, 17 Jun 2008 23:39:09 -0700 (PDT) Date: Tue, 17 Jun 2008 23:37:45 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080617235936.GL16403@sun.com> Sender: Garrett.Damore@sun.com To: Sherry Moore Cc: psarc-ext@sac.sfbay.sun.com, Seth Goldberg , Darrin Johnson , Neal Pollack , Jerry Gilliam Message-id: <4858AD39.6080909@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <20080617235936.GL16403@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 726 Status: RO X-Status: $$$$ X-UID: 0000000060 Sherry Moore wrote: > Hi folks, > > The project team has resolved most issues offline with various > parties. I have attached the updated versions of the specs reflecting > the resolution. Please send me mail if you don't believe issues you > raised have been addressed. > > Scott Rotondo will be posting summary with regard to interaction > between this project and PSARC 2008/195 "Validated Execution". > > Thanks much, > Sherry > I'd still prefer ddi_quiesce_not_supported were removed. I don't think it has any merit as a documented API. (Again, if the project team wants to use it as a private API to aid during the transition, that's fine. But I'd rather not see new drivers adopt this API.) -- Garrett From scott.rotondo@sun.com Wed Jun 18 00:11:39 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5I7Bdhh023137 for ; Wed, 18 Jun 2008 00:11:39 -0700 (PDT) Received: from viaggio.local (punchin-rotondo.SFBay.Sun.COM [10.7.251.213]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5I7BcTE328785; Wed, 18 Jun 2008 00:11:38 -0700 (PDT) Message-ID: <4858B58A.6010805@sun.com> Date: Wed, 18 Jun 2008 00:13:14 -0700 From: Scott Rotondo User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Sherry Moore CC: psarc-ext@sac.sfbay.sun.com, Seth Goldberg , Darrin Johnson , Neal Pollack , Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 References: <20080617235936.GL16403@sun.com> In-Reply-To: <20080617235936.GL16403@sun.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 2168 Status: RO X-Status: $$$$ X-UID: 0000000061 Sherry Moore wrote: > > Scott Rotondo will be posting summary with regard to interaction > between this project and PSARC 2008/195 "Validated Execution". We need to worry about interactions with Validated Execution and with the state of the Trusted Platform Module (TPM), even in the absence of validation. A hardware reset is the only way to set the TPM's Platform Configuration Registers (PCR's) back to their initial value of 0. Because Fast Reboot is a "warm start," the PCR's will continue to contain values that reflect the sequence of code that was executed during the original system boot. Since the PCR's will not be reset to their starting value, it is important for the reboot code *not* to extend code measurements into the PCR's as it would during a normal boot. [1] If it did, the PCR's would contain a different set of values every time the system was rebooted without a hardware reset. As others have suggested, it makes sense to treat the reboot as an extension of the original boot and require the kernel to perform the validation that would normally be done by the boot loader. The kernel will still need to validate the initial manifest, and we want to maintain the property that successful validation of the initial manifest requires administrative action using the TPM Owner password. Therefore, we probably want to seal the new initial manifest using the current PCR contents (and the TPM Owner password) at the time that the reboot is requested. Upon reboot, the kernel must recognize that it should not extend PCR values for bootstrapping code, and it needs to validate the code it loads using the appropriate new initial manifest. The two project teams will need to work together on some details, but we are convinced that Validated Execution and Fast Reboot are not incompatible. Scott [1] PCR values are never overwritten. They are "extended" by concatenating the existing value and the new measurement and then hashing the concatenation to produce the new value. This allows a single fixed-size register to contain a value that is uniquely determined by an arbitrarily long sequence of measurements. From jg@jurassic.sfbay.Sun.COM Wed Jun 18 08:18:01 2008 Received: from jurassic.eng.sun.com (jurassic-226-a.SFBay.Sun.COM [129.146.226.31]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5IFI1mg002397 for ; Wed, 18 Jun 2008 08:18:01 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id m5IFI03c511115; Wed, 18 Jun 2008 08:18:01 -0700 (PDT) Message-Id: <200806181518.m5IFI03c511115@jurassic.eng.sun.com> Date: Wed, 18 Jun 2008 08:14:26 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 To: gdamore@sun.com Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: B9Oe0xfEctk+ZJnOx5IfIg== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_78 SunOS 5.11 sun4u sparc Content-Length: 380 Status: RO X-Status: $$$$ X-UID: 0000000062 >> >I'd still prefer ddi_quiesce_not_supported were removed. I don't think >it has any merit as a documented API. (Again, if the project team wants >to use it as a private API to aid during the transition, that's fine. >But I'd rather not see new drivers adopt this API.) > > -- Garrett This is fine with me. Sorry, I had meant to respond to this earlier. -jg From jg@jurassic.sfbay.Sun.COM Tue Jun 24 19:05:21 2008 Received: from jurassic.eng.sun.com (jurassic-226-b.SFBay.Sun.COM [129.146.226.130]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5P25Lar006644 for ; Tue, 24 Jun 2008 19:05:21 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id m5P25KqG814666; Tue, 24 Jun 2008 19:05:21 -0700 (PDT) Message-Id: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> Date: Tue, 24 Jun 2008 19:01:42 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 To: sherry.moore@sun.com Cc: psarc-ext@sac.sfbay.sun.com, Seth.Goldberg@sun.com, Darrin.Johnson@sun.com, Neal.Pollack@sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Ygq3+ZoxzVEmS7Yyb7AFGQ== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_78 SunOS 5.11 sun4u sparc Content-Length: 229 Status: RO X-Status: $$$$ X-UID: 0000000063 Sherry has updated the spec as discussed and agreed upon here and these documents have been placed in the final.materials directory. The case has timed out with no further comments so am marking it closed/approved. thx, -jg From sherry.moore@sun.com Tue Jun 24 23:02:57 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5P62vSC012015 for ; Tue, 24 Jun 2008 23:02:57 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5P62vll286969 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Jun 2008 23:02:57 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5P62vAA286968; Tue, 24 Jun 2008 23:02:57 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Tue, 24 Jun 2008 23:02:57 -0700 From: Sherry Moore To: psarc-ext@sac.sfbay.sun.com Cc: Jerry Gilliam , "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080625060257.GA286308@sun.com> References: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 275 Status: RO X-Status: $$$$ X-UID: 0000000064 This is an e-mail note indicating that the dry-run information will be suppressed from man pages, and are reclassified as Project Private per disscussions between the project team, Jerry and Garrett. Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From Garrett.Damore@Sun.COM Wed Jun 25 04:58:08 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5PBw8Ei021281 for ; Wed, 25 Jun 2008 04:58:08 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5PBw8fs024793 for ; Wed, 25 Jun 2008 04:58:08 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5PBw3oO007868 for ; Wed, 25 Jun 2008 04:58:03 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K3000H01P872G00@fe-sfbay-10.sun.com> (original mail from Garrett.Damore@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Wed, 25 Jun 2008 04:58:03 -0700 (PDT) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K3000CHAP8QLKB0@fe-sfbay-10.sun.com>; Wed, 25 Jun 2008 04:58:03 -0700 (PDT) Date: Wed, 25 Jun 2008 04:56:06 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080625060257.GA286308@sun.com> Sender: Garrett.Damore@Sun.COM To: Sherry Moore Cc: psarc-ext@sac.sfbay.sun.com, Jerry Gilliam Reply-to: Garrett.Damore@Sun.COM Message-id: <48623256.4030807@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT References: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> <20080625060257.GA286308@sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Content-Length: 794 Status: RO X-Status: $$$$ X-UID: 0000000065 Sherry Moore wrote: > This is an e-mail note indicating that the dry-run information will be > suppressed from man pages, and are reclassified as Project Private per > disscussions between the project team, Jerry and Garrett. > > Sherry > Thank you. Just for the clarity of the record, I think this means that the project team agrees that the dry run options to reboot, as well as to uadmin(2), are project private. Btw, given that the dry run option to uadmin is private, it seems that you probably could just skip modifying reboot for dry-run. For the internal testing purposes for which I think this is intended, uadmin(1M) should be adequate for testing. See the work done by the CPR project, where they use various different subcommands to uadmin for testing. -- Garrett From sherry.moore@sun.com Wed Jun 25 07:12:25 2008 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.59]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5PECPGL025898 for ; Wed, 25 Jun 2008 07:12:25 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com (localhost [127.0.0.1]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m5PECPOj395922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Jun 2008 07:12:25 -0700 (PDT) Received: (from sherrym@localhost) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3/Submit) id m5PECPET395921; Wed, 25 Jun 2008 07:12:25 -0700 (PDT) X-Authentication-Warning: jurassic-x4600.sfbay.sun.com: sherrym set sender to sherry.moore@sun.com using -f Date: Wed, 25 Jun 2008 07:12:25 -0700 From: Sherry Moore To: "Garrett D'Amore" Cc: Sherry Moore , psarc-ext@sac.sfbay.sun.com, Jerry Gilliam Subject: Re: Fast Reboot PSARC/2008/382 Message-ID: <20080625141225.GA394681@sun.com> References: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> <20080625060257.GA286308@sun.com> <48623256.4030807@sun.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48623256.4030807@sun.com> User-Agent: Mutt/1.5.17 (2007-11-01) Content-Length: 1035 Status: RO X-Status: $$$$ X-UID: 0000000066 On Wed, Jun 25, 2008 at 04:56:06AM -0700, Garrett D'Amore wrote: > Sherry Moore wrote: >> This is an e-mail note indicating that the dry-run information will be >> suppressed from man pages, and are reclassified as Project Private per >> disscussions between the project team, Jerry and Garrett. >> >> Sherry >> > Thank you. Just for the clarity of the record, I think this means that the > project team agrees that the dry run options to reboot, as well as to > uadmin(2), are project private. Yes. > Btw, given that the dry run option to uadmin is private, it seems that you > probably could just skip modifying reboot for dry-run. For the internal > testing purposes for which I think this is intended, uadmin(1M) should be > adequate for testing. See the work done by the CPR project, where they use > various different subcommands to uadmin for testing. I believe that's implementation detail that the project team can choose to implement. Sherry -- Sherry Moore, Solaris Core Kernel http://blogs.sun.com/sherrym From Garrett.Damore@Sun.COM Wed Jun 25 07:41:46 2008 Received: from dm-sfbay-01.sfbay.sun.com (dm-sfbay-01.SFBay.Sun.COM [129.145.155.118]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m5PEfkIb026165 for ; Wed, 25 Jun 2008 07:41:46 -0700 (PDT) Received: from sca-es-mail-1.sun.com (sca-es-mail-1.Sun.COM [192.18.43.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m5PEff2k049918 for ; Wed, 25 Jun 2008 07:41:41 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m5PEfaEF017393 for ; Wed, 25 Jun 2008 07:41:36 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K3000401WN0G100@fe-sfbay-09.sun.com> (original mail from Garrett.Damore@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Wed, 25 Jun 2008 07:41:36 -0700 (PDT) Received: from [129.153.2.5] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K30004V0WTAHPB0@fe-sfbay-09.sun.com>; Wed, 25 Jun 2008 07:41:36 -0700 (PDT) Date: Wed, 25 Jun 2008 07:41:34 -0700 From: "Garrett D'Amore" Subject: Re: Fast Reboot PSARC/2008/382 In-reply-to: <20080625141225.GA394681@sun.com> Sender: Garrett.Damore@Sun.COM To: Sherry Moore Cc: psarc-ext@sac.sfbay.sun.com, Jerry Gilliam Message-id: <4862591E.8060207@sun.com> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_kP918fTT9weoBlSXFBWQAg)" References: <200806250205.m5P25KqG814666@jurassic.eng.sun.com> <20080625060257.GA286308@sun.com> <48623256.4030807@sun.com> <20080625141225.GA394681@sun.com> User-Agent: Thunderbird 2.0.0.12 (X11/20080228) Content-Length: 3383 Status: RO X-Status: $$$$ X-UID: 0000000067 This is a multi-part message in MIME format. --Boundary_(ID_kP918fTT9weoBlSXFBWQAg) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Sherry Moore wrote: > On Wed, Jun 25, 2008 at 04:56:06AM -0700, Garrett D'Amore wrote: > >> Sherry Moore wrote: >> >>> This is an e-mail note indicating that the dry-run information will be >>> suppressed from man pages, and are reclassified as Project Private per >>> disscussions between the project team, Jerry and Garrett. >>> >>> Sherry >>> >>> >> Thank you. Just for the clarity of the record, I think this means that the >> project team agrees that the dry run options to reboot, as well as to >> uadmin(2), are project private. >> > > Yes. > Thank you. > >> Btw, given that the dry run option to uadmin is private, it seems that you >> probably could just skip modifying reboot for dry-run. For the internal >> testing purposes for which I think this is intended, uadmin(1M) should be >> adequate for testing. See the work done by the CPR project, where they use >> various different subcommands to uadmin for testing. >> > > I believe that's implementation detail that the project team can choose > to implement. > Agreed. I was just offering some friendly implementation advice, not architectural guidance. -- Garrett --Boundary_(ID_kP918fTT9weoBlSXFBWQAg) Content-type: text/html; charset=ISO-8859-1 Content-transfer-encoding: 7BIT Sherry Moore wrote: 
On Wed, Jun 25, 2008 at 04:56:06AM -0700, Garrett D'Amore wrote:
  
Sherry Moore wrote:
    
This is an e-mail note indicating that the dry-run information will be
suppressed from man pages, and are reclassified as Project Private per
disscussions between the project team, Jerry and Garrett.

Sherry
  
      
Thank you.  Just for the clarity of the record, I think this means that the 
project team agrees that the dry run options to reboot, as well as to 
uadmin(2), are project private.
    

Yes.

Thank you.

  
Btw, given that the dry run option to uadmin is private, it seems that you 
probably could just skip modifying reboot for dry-run.  For the internal 
testing purposes for which I think this is intended, uadmin(1M) should be 
adequate for testing.  See the work done by the CPR project, where they use 
various different subcommands to uadmin for testing.
    

I believe that's implementation detail that the project team can choose
to implement.

Agreed.  I was just offering some friendly implementation advice, not architectural guidance.

    -- Garrett

--Boundary_(ID_kP918fTT9weoBlSXFBWQAg)--