From sacadmin Tue Jul 31 16:42:56 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l6VNguwi016264 for ; Tue, 31 Jul 2007 16:42:56 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l6VNegs4025603; Tue, 31 Jul 2007 16:40:42 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM200G07HRRP900@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 16:40:39 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.106.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM200E28HRQZI10@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 16:40:38 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l6VNecq4641952; Tue, 31 Jul 2007 16:40:38 -0700 (PDT) Date: Tue, 31 Jul 2007 16:39:30 -0700 (PDT) From: Jerry Gilliam Subject: 2006/525: New Boot Sparc To: psarc@sun.com Cc: jerry.gilliam@sun.com, jan.setje-eilers@sun.com, john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: TiTtkXVx2KE3aBLLqoPgqw== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 19853 I am sponsoring this project as a fast-track on behalf of Jan Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. The project desires patch/update binding. thx, -jg #ident "@(#)design 1.2 07/07/31 SMI" New Solaris SPARC Boot Architecture Summary The Solaris SPARC bootstrap process is being redesigned, both to increase the commonality with Solaris x86, and to enable ITUs (install time updates) on SPARC. Secondary goals are to combine inetboot and wanboot into a single network boot architecture, and to provide a simplified architecture for disk filesystems other than UFS (e.g, ZFS). The Solaris x86 newboot project has already been delivered to Nevada and S10u1. That project was described by PSARC 2004/454. This project is a follow-on to that project. This design specification only covers the architecture for Solaris SPARC. For details of the Solaris x86 design, see PSARC 2004/454. 1. Introduction --------------- The Solaris boot process was designed in early 1990's for desktops which were tiny by today's standards. On the one hand, the design was constrained by a relatively small amount of system memory. On the other hand, the design took advantage of the presence of Open Boot Prom (OBP) on all Sun platforms. The resulting implementation involves a complex sequence of control handoff between kernel and OBP to load a minimum amount of text and data from the root device into memory. 2. Motivation ------------- A change in the Solaris boot architecture is motivated by the following problems: 2.1 Supporting new hardware Solaris SPARC does not support ITUs, so a platform that requires any Solaris changes requires a full Solaris update release. This requirement causes pain for both the Solaris and SPARC HW groups, as the two must fight over Update schedules for almost every HW platform. 2.2 Commonality with Solaris x86 The newboot project on Solaris x86 has made both the administrative and boot processes visibly different between Solaris SPARC and Solaris x86. This is in part due to the addition of a boot archive to Solaris x86. It is in Sun's interest that the look and feel of Solaris on different platforms be as common as feasible. 2.3 Ease of booting different FS types The current boot architecture requires multiple filesystem readers, making adding new ones a difficult process. The goal of this project is to only require one phase to read the root FS before the kernel mounts it. This will enable new filesystem types like ZFS to be more easily supported as root filesystems. 2.5 Common network boot process The Solaris SPARC network boot process is very different between booting over a LAN versus booting over a WAN. The WAN process boots via a miniroot that looks much like how Solaris x86 installs over a network, but there is no commonality in how the miniroots are created or administered. This project will unify SPARC network booting around a single architecture. 3. Proposed Architecture ------------------------ 3.1 Boot Phase Independence The primary design center is to make the phases of the boot process be independent of each other. This allows the addition of new features (e.g., new filesystem types) without requiring changes to multiple parts of the boot chain. This project envisions 4 phases of booting on SPARC machines: OBP The OBP phase of boot is unchanged. In fact, it's a requirement that SPARC newboot not require new OBP functionality. OBP will continue to load and execute a booter from a disk or network device. booter The booter phase is responsible for reading in the boot archive and executing it. This is the only phase of the boot process that requires knowledge of the root filesystem format. ramdisk The ramdisk is a boot archive containing either kernel modules or an install miniroot. This boot archive is the same boot archive as is used on Solaris x86. Its FS format is private to itself. i.e., neither the booter nor the kernel needs to know whether the archive is HSFS or UFS (or ZFS for that matter). The ramdisk will extract the kernel image from the boot archive and execute it. In order to minimize the size of the ramdisk, in particular the install miniroot, which must reside in memory, the contents of the miniroot will be compressed. This compression is on a per file level and is implemented within the filesystem. In order to create compressed files a userland utility is used that simply compresses the file in place, the file is then marked as compressed via the _FIO_COMPRESSED (private) ioctl in the filesystem metadata. kernel The final stage is the kernel. The kernel extracts the rest of the primary modules from the boot archive, initializes itself, mounts the real root file system, and throws away the boot archive. This process is also the same as on Solaris x86. 3.2 Removal of bootops (and ufsboot) The bootops vector on SPARC originally existed to support platforms with either OBP or SUNMON FW. The second stage booter (e.g., ufsboot) presented a common set of operations to the kernel so the kernel didn't need to know either what prom version was running or what filesystem type root was. The last Solaris release to support a SUNMON platform was Solaris 2.4, and the last one to support pre-ieee1275 OBP was Solaris 7, so it's time for the bootops to go gently into that good night. 3.2.3 unix and krtld combined The Xen project (PSARC 2006/260) has combined the unix and krtld modules in order to enable booting from either BIOS or the Xen hypervisor. Since combining these two makes the unix ELF header far simpler to parse and load, this project has adapted this change for SPARC. 3.2.4 boot properties There are three properties retrieved via the current bootops that neither OBP nor the kernel have any knowledge of. These are: "fstype" name of root filesystem type (e.g., "ufs") "impl-arch-name" platform name (aka `uname -i`) "whoami" file system name of booted kernel A new node (/packages/boot-properties) will be added to the OBP device tree by the booter which contains these properties. In addition to the above compatibility properties, a couple of new properties are needed: "bootarchive" boot archive path (as opposed to "bootpath", the root file system path) "elfheader-address" address of kernel ELF header, used by krtld in lieu of the bootaux vector passed up via the second-stage booter "elfheader-length" length of kernel ELF header "archive-fstype" fstype of boot archive 3.3 Differences from Solaris x86 The primary difference from PSARC 2004/454 is that there is no dependence on grub. This decision was made for both practical and functional reasons. The practical reason is that grub0.95 - which was used by Solaris x86 - is not available on SPARC. The functional reason is that most of the reasons grub was used for Solaris x86 (e.g., eliminate real-mode, eliminate the boot shell, 3rd party device support) are not applicable to Solaris SPARC. Making grub work in the network boot case on SPARC is not a trivial exercise, as grub does not currently support NFS or HTTP, and adding wanboot's non-exportable cryptography code is problematic with respect to the GPL. The boot menu provided by grub would be an interesting feature to provide Solaris SPARC users. If grub2 were to become available on SPARC in a stable supportable release it could be incorporated in booter phase above. This should not require much change to existing code since the booter retains its ability to load alternate secondary booters such as cprboot. 4 Interfaces ------------ 4.1 Interface Exported ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- Boot files Evolving platform//boot_archive default name of boot archive (initially may be sun4u/sun4v only) Boot args kernel args Evolving see kernel(1M) -F Evolving alternate booter or boot archive usr/sbin/bootadm(1M) Stable usr/sbin/root_archive(1M) Stable boot/solaris/bin create_ramdisk Proj. Priv. Install-time Update different user prompts boot/solaris filelist.ramdisk Proj. Priv. boot archive content filestat.ramdisk Proj. Priv. boot archive file status kernel/fs/dcfs Proj. Priv. compression file system sbin/fiocompress Proj. Priv. file compression utility usr/sbin/fiocompress Proj. Priv. link to sbin/fiocompress usr/include/sys/fs/decomp.h Proj. Priv. dcfs header file _FIO_COMPRESSED Proj. Priv. file compression ioctl usr/platform/sun4[uv]/lib/fs hsfs/bootblk Proj. Priv. filesystem readers zfs/bootblk Proj. Priv. filesystem readers platform/sun4[uv]/ufsboot Proj. Priv. removed kernel/misc/sparcv9/krtld Proj. Priv. merged into unix ----------------------------------------------------------------------- 4.2 Interfaces Reimplemented ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- reboot(1M) Stable? update boot archive as needed halt(1M) poweroff(1M) shutdown(1M) init(1M) pkgadd(1M) patchadd(1M) add_install_client Evolving modified server setup smdiskless(1M) Evolving setup /tftpboot area Install consume bootadm and installgrub Upgrade Live Upgrade Flash Install Net Install Jumpstart Release engineering tools ------------------------- modified miniroot construction 5. User Experience ------------------ 5.1 System startup The system startup changes are not visible to the user. 5.2 Installation and Upgrade The ITU menu which currently only exists on x86 may be exposed on sparc as well. Customers who rely on undocumented non-interfaced implementation details of add_install_client may need to amend their procedures. If we deliver the wanboot component, wanboot customers will see a simplification of the deployment procedures. 5.3 Internal tools The bfu scripts will be updated to permit developers to transition from old boot to new boot. For a system initially installed with old boot, it will be possible to bfu back and forth across the boundary. For a system initially installed with new boot, bfu back to old boot is not supported. Booting a glommed kernel is supported, with the same restrictions regarding the compatibility of userland and kernel. If the root file system goes into a non-bootable state, a user may boot the failsafe archive to perform manual recovery operations. The failsafe archives contains files normally present in a CD or netinstall miniroot. If the boot archive goes into a non-bootable state, a user may bypass the boot archive and directly boot the kernel with the -F option. 5.4 Coexistence with other OSes The only other OS of note for SPARC is Linux, and they currently have no grub plans. SPARC / Linux boots from a loader called SILO, which already has a grub-like menu facility. In general OBP is considered well suited to loading multiple OS's or versions of OS's from various devices on the system, so no additional work beyond what is provided by OBP/the system is desirable. 6. Technical Details -------------------- 6.1 OBP phase This project does not change the OBP phase of SPARC boot; this section is included for reference only. When a user types "boot" on an OBP-based system, the device selected - either from the command line or via the "boot-device" nvram variable - has its "open" and "load" methods called. The program loaded by this process is then executed, and the boot process enters the booter phase. 6.1.1 Disk For disk devices, the FW driver usually uses the OBP label package's "load" method, which parses the VTOC label at the beginning of the disk to locate the specified partition, then reads sectors 1-15 of that partition into memory. This area is commonly called the boot block and usually contains a filesystem reader. 6.1.2 Net For network devices, the process is slightly different between booting over a LAN versus booting over a WAN. In both cases, however, the prom will download a booter from a boot or install server (inetboot in this case). 6.1.2.1 LAN boot When booting over a LAN, the FW uses either RARP and BOOTP or DHCP to discover its boot or install server. It then uses TFTP to download the booter (inetboot in this case). 6.1.2.2 WAN boot When booting over a WAN, the FW uses either DHCP or nvram properties to discover its install server, and the router and proxies needed to connect to it. It then uses HTTP to download the booter, and may optionally check the booter's signature with a predefined private key. For more details, see PSARC 2001/009. 6.2 Booter phase This phase is derived from the SPARC wanboot ramdisk process (see PSARC 2001/009), and is responsible for reading the boot archive from the root file system (or install server in wanboot's case) into a ramdisk device. It does this by: 1) opening the boot-device (which it found as the "bootpath" property in the OBP "/chosen" node) 2) using its file system specific reader to read the boot archive (by default, /platform/`uname -m`/boot_archive) 3) creating a ramdisk device in "/ramdisk" 4) creating "bootarchive" and "fstype" properties in "/packages/boot-properties" 5) booting the archive (a ramdisk is just another type of disk, so executing the boot block area serves this purpose) 6.3 Ramdisk phase The ramdisk is self-describing in the same sense any disk image is by virtue of having a filesystem reader in its boot block. This reader is over 90% the same as the disk boot block for a given filesystem so the same program is re-used. Its job is to load and execute the kernel from the archive by: 1) opening the boot archive (the "bootarchive" property from the previous phase) 2) using its file system specific reader to read the kernel (by default, /platform/`uname -i`/kernel/unix) 3) creating "impl-arch-name", "whoami" and "elfheader" properties 4) executing the kernel 6.4 Kernel phase When krtld gains control, it mounts the boot archive and loads additional kernel modules from the boot archive via the ramdisk. Subsequent kernel initialization procedures remain the same until after the kernel mounts the root file system. At that point, the kernel throws away the boot archive and reclaims the memory it occupies. Note that in the install case, the ramdisk actually contains the root file system, and is not thrown away. The kernel ramdisk driver simply takes over control of the ramdisk image. 6.5 Chained booters When booting from a disk, the booter will support chained booters both for cprboot (see PSARC 1992/201) and for situations where the file system reader cannot fit in the boot block. In a future project, grub2 can use this facility to add a graphical user menu to the booter phase. This facility will not be available when booting from a network, since the chained booter usually uses the same virtual address space as the original booter. 6.6 Boot archive management There are two kinds of boot archive: failsafe and normal. A failsafe archive is self-sufficient and bootable by itself. It is created at install time and requires no maintenance. A normal archive shadows a root filesystem, so it contains all kernel modules, driver.conf files, and a few configuration files in /etc which are read by the kernel before root is mounted. Once the root filesystem is mounted, the kernel discards the boot archive from memory and file I/O will be performed against the root device. By default, the normal archive contains the following files and directories: etc/system etc/name_to_major etc/driver_aliases etc/name_to_sysnum etc/dacf.conf etc/driver_classes etc/path_to_inst etc/devices/devid_cache etc/devices/mdi_scsi_vhci_cache etc/devices/mdi_ib_cache etc/cluster/nodeid etc/zfs/zpool.cache kernel platform The contents under the platform directory will be segregated into those needed for a sun4u boot archive and those needed for a sun4v boot archive. Further per-platform differentiated boot archives may be considered if that helps us gain faster booting via faster archive load, trading off a more complicated archive construction process. If any files in this list (or under directories listed) is updated, the boot archive must be rebuilt prior to the next reboot for the modification to take effect. The package and patch tools are updated to update the boot archive whenever needed. In addition, the boot archive is updated as necessary on an orderly system shutdown to catch files modified manually. The boot archive could be out of sync with the root filesystem if the system panics in the middle of an update, but before archive update is completed. We check for such conditions on every boot before root filesystem is mounted writeable. If an inconsistency is detected, the system will stop in single-user mode, similar to current behavior when fsck fails on the root filesystem. The recommended recovery method is to boot the failsafe archive and recreate the boot_archive. An expert user may decide to continue booting if the out-of-sync files are not critical. As noted above, if the boot archive cannot be fixed, the kernel can be directly booted from the boot via the -F argument. This option will not work when booting via HTTP, since there is no underlying FS to read from. It will also be noticably slower than booting via the archive, since the relatively simple booters do not implement a buffer cache. The bootadm(1M) command will handle the details of archive update and verification. 6.7 Install and upgrade Normal install and upgrade is achieved by booting the miniroot from either CDROM/DVD or from the network. In both cases, the root filesystem of the miniroot is the ramdisk. This allows the Solaris boot CD to be ejected without rebooting the system. The boot archive contains the entire miniroot. The construction of the install CD is modified to use an hsfs boot block. The miniroot is packed into a single file in ufs format, to be loaded as the ramdisk image. The setup of the net boot server is also modified. The boot server will serve a boot strap as well as the ramdisk image which is downloaded and then booted from. The netinstall image will be packed using root_archive(1m). The process for installing the OS to disk remains the same except that the boot blocks are different and a boot archive must be constructed prior to booting the install target disk. The boot archives are created using bootadm(1M). The rest of code changes comes with packages and patches and no special treatment is required. 6.8 Diskless clients Diskless boot is similar to booting the miniroot for net install except that the root filesystem is on NFS instead of on UFS. 6.9 Install-time Update (ITU) New-boot (phase 1: x86) reduced the x86 ITU mechanism described in PSARC 1997/059 to simply adding Solaris binaries (drivers, kernel modules, commands, libraries, symlinks and the like) to the running miniroot and then the target install environment. It also extended the supported media that an ITU could be supplied on to include CD/DVD, memory sticks and similar devices. This mechanism will now be made available on sparc as well to allow platform support to be delivered out of band of a regularly scheduled release. If a core kernel component needs to be updated, say unix or something else that needs to be loaded before an ITU can be added, then a pre-patched miniroot image needs to be made available (50Mb download) along with the patch. References ---------- 1. Shudong Zhou PSARC 2004/454 Solaris Boot Architecture 2. Carl Smith PSARC 2001/009 WAN-boot 3. Clark Dong PSARC 1992/201 Checkpoint Resume (Reanimator) 4. Allan McKillop PSARC 2006/260 Solaris on Xen From sacadmin Tue Jul 31 17:39:11 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l710dA2s019572 for ; Tue, 31 Jul 2007 17:39:10 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l710apAR014942; Wed, 1 Aug 2007 08:36:53 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM200J0BKDFKG00@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 17:36:51 -0700 (PDT) Received: from eastmail2bur.East.Sun.COM ([129.148.13.40]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM200E63KDDZO30@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 17:36:50 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l710aiEr017117; Tue, 31 Jul 2007 20:36:44 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l710ai7K021250; Tue, 31 Jul 2007 20:36:44 -0400 (EDT) Date: Tue, 31 Jul 2007 20:36:43 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> To: Jerry Gilliam Cc: psarc@Sun.COM, Jerry.Gilliam@Sun.COM, Jan.Setje-Eilers@Sun.COM, John.Johnson@Sun.COM, Lori.Alt@Sun.COM, Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM Message-id: <1185928603.18965.111.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> Status: RO Content-Length: 2100 Is there a reason why this wasn't filed as an open case? I don't see anything that would require this to remain closed. On Tue, 2007-07-31 at 16:39 -0700, Jerry Gilliam wrote: > The package and patch tools are updated to update the boot archive > whenever needed. In addition, the boot archive is updated as necessary on an > orderly system shutdown to catch files modified manually. > > The boot archive could be out of sync with the root filesystem if the system > panics in the middle of an update, but before archive update is completed. We > check for such conditions on every boot before root filesystem is mounted > writeable. If an inconsistency is detected, the system will stop in single-user > mode, similar to current behavior when fsck fails on the root filesystem. The > recommended recovery method is to boot the failsafe archive and recreate the > boot_archive. An expert user may decide to continue booting if the out-of-sync > files are not critical. Boot stalls requiring manual intervention due to inconsistencies between the boot archive and the root filesystem are, in my experience, a major dissatisfier with the x86 boot archive and are also a frightening thing to present to a new solaris admin. Several of the files included in the boot archive (the files in /etc, for instance) have dynamic content which is expected to change under normal system operation in the presence of hardware changes (for instance, USB hotplug or disk replacement). If it's possible to continue to bypass the boot archive during boot, it seems to me that it should be possibile to omit dynamic files from the boot archive entirely and instead always read them from the root filesystem; if we limit cases where we report inconsistencies to cases where we crash in the middle of a non-alternate-root upgrade, I think we can make this dissatisfier just vanish. perhaps one approach would be to have a list of files to read from the root filesystem in the boot archive, and read them all in to kernel memory after reading the boot archive. I'm sure there are other ways to do this. From sacadmin Tue Jul 31 17:43:55 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l710htCS019605 for ; Tue, 31 Jul 2007 17:43:55 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l710fepN061017; Tue, 31 Jul 2007 18:41:40 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM200J01KLFS900@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 17:41:39 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.228.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM200E22KLFZC40@nwk-avmta-2.sfbay.sun.com>; Tue, 31 Jul 2007 17:41:39 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l710fdAj642961; Tue, 31 Jul 2007 17:41:39 -0700 (PDT) Date: Tue, 31 Jul 2007 17:40:30 -0700 (PDT) From: Jerry Gilliam Subject: Re: 2006/525: New Boot Sparc To: sommerfeld@sun.com Cc: psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200708010041.l710fdAj642961@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: F9XZ8uRBZaI9vriJ3pFpFA== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 281 > >Is there a reason why this wasn't filed as an open case? I don't see >anything that would require this to remain closed. > Not that I'm aware of. The case had previously been submitted via a one-pager and I just left the exposure field untouched. I'll correct this. -jg From sacadmin Tue Jul 31 21:32:22 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l714WMGf028091 for ; Tue, 31 Jul 2007 21:32:22 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l714U2wh040405; Tue, 31 Jul 2007 22:30:07 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM200509V662Q00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Jul 2007 21:30:06 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM200FAHV66JBC0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 31 Jul 2007 21:30:06 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l714TwgQ008727; Tue, 31 Jul 2007 21:29:58 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l713nS6p027586; Tue, 31 Jul 2007 20:49:28 -0700 (PDT) Date: Tue, 31 Jul 2007 20:49:28 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Tue, 31 Jul 2007 17:40:30 PDT." <200708010041.l710fdAj642961@jurassic.eng.sun.com> To: Jerry Gilliam Cc: sommerfeld@sun.com, psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708010349.l713nS6p027586@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 572 <200708010041.l710fdAj642961@jurassic.eng.sun.com>Jerry Gilliam writes: > > > > > >Is there a reason why this wasn't filed as an open case? I thought I had attempted to create it as open, so I suppose the reason is operator error. Thanks for catching it! > > I don't see anything that would require this to remain closed. We're touching a couple of things in usr/closed, but that shouldn't drag the whole case there. Come to think of it I'm very confused as to why some random platmods are closed and others aren't. > I'll correct this. Thanks Jerry! -jan From sacadmin Wed Aug 1 04:22:14 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71BMDs6011417 for ; Wed, 1 Aug 2007 04:22:13 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l71BJmCi002598 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Wed, 1 Aug 2007 19:19:58 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300203E584D00@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 05:19:56 -0600 (MDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM3002E7E56SGE0@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 05:19:55 -0600 (MDT) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l71BJsxZ024812 for ; Wed, 01 Aug 2007 11:19:54 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM300M01E38LF00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 12:19:54 +0100 (BST) Received: from [129.156.173.21] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM3008QKE54BR30@d1-emea-09.sun.com>; Wed, 01 Aug 2007 12:19:53 +0100 (BST) Date: Wed, 01 Aug 2007 12:19:52 +0100 From: Darren J Moffat Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> Sender: Darren.Moffat@sun.com To: Jerry Gilliam Cc: psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B06C58.3040108@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 441 There is a comment in this case about the wanboot cryptographic support being non exportable. That isn't true. There is not US export reason why that crypto for wanboot can't be available in source form - which it already is. In fact parts of wanboot actually use OpenSSL (which is why OpenSSL is in ON not SFW). However there could be licensing or technical issues but there are not any export control issues. -- Darren J Moffat From sacadmin Wed Aug 1 09:25:38 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71GPcuZ018864 for ; Wed, 1 Aug 2007 09:25:38 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71GNLrc033180; Wed, 1 Aug 2007 10:23:22 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300H0DS6YVR00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 09:23:22 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM300F8MS6XWD40@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 09:23:21 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71GNKsd019343; Wed, 01 Aug 2007 09:23:20 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71GCq5W029991; Wed, 01 Aug 2007 09:12:52 -0700 (PDT) Date: Wed, 01 Aug 2007 09:12:52 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Tue, 31 Jul 2007 20:36:43 EDT." <1185928603.18965.111.camel@thunk> To: Bill Sommerfeld Cc: Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708011612.l71GCq5W029991@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2193 > Boot stalls requiring manual intervention due to inconsistencies between > the boot archive and the root filesystem are, in my experience, a major > dissatisfier with the x86 boot archive and are also a frightening thing > to present to a new solaris admin. Correct. Since the initial introduction of new-boot x86 there have been a number of improvements to this check that have all but eliminated it coming up. > Several of the files included in the boot archive (the files in /etc, > for instance) have dynamic content which is expected to change under > normal system operation in the presence of hardware changes (for > instance, USB hotplug or disk replacement). If it's possible to > continue to bypass the boot archive during boot, it seems to me that it > should be possibile to omit dynamic files from the boot archive entirely > and instead always read them from the root filesystem; if we limit cases > where we report inconsistencies to cases where we crash in the middle of > a non-alternate-root upgrade, I think we can make this dissatisfier > just vanish. That's exactly the approach that's been implemented if you take a look at /boot/solaris/filelist.safe (shipping now on x86), it contains a list of files that are either just caches or just grow. They are all safe to ignore changes in and the check now does just that. > perhaps one approach would be to have a list of files to read from the > root filesystem in the boot archive, and read them all in to kernel > memory after reading the boot archive. I'm sure there are other ways to > do this. This may help us drag more files into the safe list. At the time I had to re-order something to make etc/rtc_config safe. We may identify more files, on x86 and since it's new more likely on sparc, that qualify for this sort of treatment and will apply it there. Lastly, along these lines the sparc implementation is slightly superior to x86 as the boot block only loads one object, the boot_archive, rather than the kernel and the boot_archive, eliminating any chance that that kernel and archive are out of sync. While not part of this case, that's something I'd still like to see improved on x86. -jan From sacadmin Wed Aug 1 09:31:15 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71GVFVL018934 for ; Wed, 1 Aug 2007 09:31:15 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71GSw7I007665 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Wed, 1 Aug 2007 09:29:00 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300I13SGB1W00@nwk-avmta-2.sfbay.sun.com> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 09:28:59 -0700 (PDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM300FPNSGBWB40@nwk-avmta-2.sfbay.sun.com> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 09:28:59 -0700 (PDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71GSvnY005794; Wed, 01 Aug 2007 09:28:57 -0700 (PDT) Received: from [10.7.251.244] (punchin-jgj [10.7.251.244]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71GSuq5070145; Wed, 01 Aug 2007 09:28:57 -0700 (PDT) Date: Wed, 01 Aug 2007 09:21:41 -0700 From: John Johnson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B06C58.3040108@Sun.COM> To: Darren J Moffat Cc: Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B0B315.9090504@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> <46B06C58.3040108@Sun.COM> User-Agent: Thunderbird 2.0b1pre (X11/20061022) Status: RO Content-Length: 649 Darren J Moffat wrote: > There is a comment in this case about the wanboot cryptographic support > being non exportable. That isn't true. There is not US export reason > why that crypto for wanboot can't be available in source form - which it > already is. In fact parts of wanboot actually use OpenSSL (which is why > OpenSSL is in ON not SFW). However there could be licensing or > technical issues but there are not any export control issues. > The comment was based on wanboot being removed from the ON export build. The comment really isn't germane to the technical content of the case, so it certainly can be changed. JJ From sacadmin Wed Aug 1 09:36:28 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71GaRvS018999 for ; Wed, 1 Aug 2007 09:36:28 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l71GY81n000090 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Thu, 2 Aug 2007 00:34:12 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300K0RSOZDD00@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 10:34:11 -0600 (MDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM3005NDSOXY9B0@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 10:34:10 -0600 (MDT) Received: from d1-emea-10.sun.com (d1-emea-10.sun.com [192.18.2.120]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l71GY9re027509 for ; Wed, 01 Aug 2007 16:34:09 +0000 (GMT) Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM300901SN8AM00@d1-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 17:34:09 +0100 (BST) Received: from [129.156.173.21] by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM300HDTSOW7S00@d1-emea-10.sun.com>; Wed, 01 Aug 2007 17:34:08 +0100 (BST) Date: Wed, 01 Aug 2007 17:34:08 +0100 From: Darren J Moffat Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B0B315.9090504@sun.com> Sender: Darren.Moffat@Sun.COM To: John Johnson Cc: Jerry Gilliam , psarc@Sun.COM, Jerry.Gilliam@Sun.COM, Jan.Setje-Eilers@Sun.COM, Lori.Alt@Sun.COM, Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM Message-id: <46B0B600.9000905@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> <46B06C58.3040108@Sun.COM> <46B0B315.9090504@sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 1073 John Johnson wrote: > Darren J Moffat wrote: >> There is a comment in this case about the wanboot cryptographic >> support being non exportable. That isn't true. There is not US >> export reason why that crypto for wanboot can't be available in source >> form - which it already is. In fact parts of wanboot actually use >> OpenSSL (which is why OpenSSL is in ON not SFW). However there could >> be licensing or technical issues but there are not any export control >> issues. >> > > The comment was based on wanboot being removed from the ON export > build. The comment really isn't germane to the technical content of the > case, so it certainly can be changed. The reason it is not in the ON export_src build has nothing to do with current US export rules and more to do with the fact that there are no engineers left to look after wanboot to fix this. You may not think it important to the architecture but they way I read the original comment it looked like it was an important part of the reason why that path wasn't taken. -- Darren J Moffat From sacadmin Wed Aug 1 09:55:00 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71Gt05i019963 for ; Wed, 1 Aug 2007 09:55:00 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71Gqjp5014683 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Wed, 1 Aug 2007 09:52:45 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300007TJWII00@nwk-avmta-1.sfbay.Sun.COM> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 09:52:44 -0700 (PDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM300D3TTJV7DF0@nwk-avmta-1.sfbay.Sun.COM> for psarc@sun.com (ORCPT psarc@sun.com); Wed, 01 Aug 2007 09:52:43 -0700 (PDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l71GqcHT008809; Wed, 01 Aug 2007 09:52:38 -0700 (PDT) Received: from [10.7.251.244] (punchin-jgj [10.7.251.244]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71Gqbbe071123; Wed, 01 Aug 2007 09:52:37 -0700 (PDT) Date: Wed, 01 Aug 2007 09:45:22 -0700 From: John Johnson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B0B600.9000905@Sun.COM> To: Darren J Moffat Cc: Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B0B8A2.3080807@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200707312340.l6VNecq4641952@jurassic.eng.sun.com> <46B06C58.3040108@Sun.COM> <46B0B315.9090504@sun.com> <46B0B600.9000905@Sun.COM> User-Agent: Thunderbird 2.0b1pre (X11/20061022) Status: RO Content-Length: 432 Darren J Moffat wrote: > > You may not think it important to the architecture but they way I read > the original comment it looked like it was an important part of the > reason why that path wasn't taken. > It was not. The primary reasons for not using grub 0.95 are explained earlier in the same paragraph. The decision to wait for grub2 would not have changed if wanboot were part of the ON export src build. JJ From sacadmin Wed Aug 1 10:50:58 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71How1I024795 for ; Wed, 1 Aug 2007 10:50:58 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71HmfSN004611 for <@sunmail2sca.sfbay.sun.com:psarc@Sun.COM>; Wed, 1 Aug 2007 10:48:43 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM30010XW57Q500@brm-avmta-1.central.sun.com> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 11:48:43 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM30052BW55Y7F0@brm-avmta-1.central.sun.com> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 11:48:42 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71HmXGB003266; Wed, 01 Aug 2007 10:48:33 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71H84IZ000229; Wed, 01 Aug 2007 10:08:04 -0700 (PDT) Date: Wed, 01 Aug 2007 10:08:01 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 17:34:08 BST." <46B0B600.9000905@Sun.COM> To: Darren J Moffat Cc: John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708011708.l71H84IZ000229@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1630 <46B0B600.9000905@Sun.COM>Darren J Moffat writes: > John Johnson wrote: > > Darren J Moffat wrote: > >> There is a comment in this case about the wanboot cryptographic > >> support being non exportable. That isn't true. There is not US > >> export reason why that crypto for wanboot can't be available in source > >> form - which it already is. In fact parts of wanboot actually use > >> OpenSSL (which is why OpenSSL is in ON not SFW). However there could > >> be licensing or technical issues but there are not any export control > >> issues. > >> > > > > The comment was based on wanboot being removed from the ON export > > build. The comment really isn't germane to the technical content of the > > case, so it certainly can be changed. > > The reason it is not in the ON export_src build has nothing to do with > current US export rules and more to do with the fact that there are no > engineers left to look after wanboot to fix this. > > You may not think it important to the architecture but they way I read > the original comment it looked like it was an important part of the > reason why that path wasn't taken. It was not. GRUB on sparc comes up very short, particularly in terms of netboot support. In light of already having OBP which provides a supportable interactive moment before boot it's also not really clear why we would even want GRUB here. If we were to introduce a menu on sparc, I could see pulling in GRUB in order to provide a consistent look and feel, but we're not proposing any new interactive features during boot (which on average are a bug) with this project. -jan From sacadmin Wed Aug 1 11:09:41 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71I9evN025622 for ; Wed, 1 Aug 2007 11:09:41 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l71I7FL2017756 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Thu, 2 Aug 2007 02:07:25 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM30021PX0CQW00@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 12:07:24 -0600 (MDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM30025MX09BC10@brm-avmta-1.central.sun.com> for psarc@sun.com (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 12:07:22 -0600 (MDT) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l71I7LVx000734 for ; Wed, 01 Aug 2007 18:07:21 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM300301WXJUM00@d1-emea-09.sun.com> (original mail from Paul.Jakma@Sun.COM) for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 19:07:21 +0100 (BST) Received: from dhcp-ellg01-80-219.UK.Sun.COM ([129.159.80.219]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM3005PKX08I300@d1-emea-09.sun.com>; Wed, 01 Aug 2007 19:07:21 +0100 (BST) Date: Wed, 01 Aug 2007 19:06:45 +0100 (IST) From: Paul Jakma Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708011708.l71H84IZ000229@smack.Eng.Sun.COM> Sender: Paul.Jakma@sun.com To: Jan Setje-Eilers Cc: Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; format=flowed; charset=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708011708.l71H84IZ000229@smack.Eng.Sun.COM> Status: RO Content-Length: 1249 On Wed, 1 Aug 2007, Jan Setje-Eilers wrote: > GRUB on sparc comes up very short, particularly in terms of netboot > support. While acknowledging the current reality of the above point... > In light of already having OBP which provides a supportable > interactive moment before boot it's also not really clear why we would > even want GRUB here. Hmm, the reason why is stated as motivation in the submitted document, in the motivation section: 2.2 Commonality with Solaris x86 Pre-boot interactive session is most important when sysadmins are trying to diagnose/fix some significant problem. That's *precisely* the time when platform inconsistencies are the most annoying and disruptive. I.e. Divergence makes our users lives harder. So there needs to be a really good reason for it - "I can't see why we'd want GRUB on SPARC" surely isn't it ;). We should minimise divergence for it's /own sake/ - no other justification should be needed. Whether it's possible to get GRUB or GRUB2 working at a reasonable time/resource cost, I of course don't know though. ;) regards, -- Paul Jakma, Solaris Networking Sun Microsystems, Scotland http://opensolaris.org/os/project/quagga tel: EMEA x73150 / +44 15066 73150 From sacadmin Wed Aug 1 12:05:42 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71J5g3v027360 for ; Wed, 1 Aug 2007 12:05:42 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71J3OCI002168; Wed, 1 Aug 2007 12:03:26 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM30061DZLP3Z00@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 13:03:25 -0600 (MDT) Received: from localhost.east.sun.com ([129.148.19.3]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM3002ECZLNB760@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 13:03:24 -0600 (MDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71J11GV010109; Wed, 01 Aug 2007 15:01:01 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l71J11i1010108; Wed, 01 Aug 2007 15:01:01 -0400 (EDT) Date: Wed, 01 Aug 2007 15:01:01 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708011612.l71GCq5W029991@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1185994861.9479.13.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708011612.l71GCq5W029991@smack.Eng.Sun.COM> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 1058 On Wed, 2007-08-01 at 09:12 -0700, Jan Setje-Eilers wrote: > > if we limit cases > > where we report inconsistencies to cases where we crash in the middle of > > a non-alternate-root upgrade, I think we can make this dissatisfier > > just vanish. > > That's exactly the approach that's been implemented if you take a look > at /boot/solaris/filelist.safe (shipping now on x86), it contains a > list of files that are either just caches or just grow. They are all safe > to ignore changes in and the check now does just that. So, I don't think we're there yet -- this case identifies 12 files in /etc as present in the boot archive and I only see half of them in filelist.safe; moreover, there's also dynamic configuration in *.conf files in /kernel (the one I'm most familiar with is md.conf); what's more, /etc/system can be updated outside an upgrade and is not in filelist.safe. Where I think we need to get to: only files packaged immutably (type "f") are present in the boot archive; all others are read "a la carte" from the root. - Bill From sacadmin Wed Aug 1 12:13:13 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71JDDDW027549 for ; Wed, 1 Aug 2007 12:13:13 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71JAt36034136 for <@sunmail2sca.sfbay.sun.com:psarc@Sun.COM>; Wed, 1 Aug 2007 13:10:58 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM300C2PZY8IE00@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 12:10:56 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM3006AQZY5X360@nwk-avmta-1.sfbay.Sun.COM> for psarc@Sun.COM (ORCPT psarc@Sun.COM); Wed, 01 Aug 2007 12:10:53 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71JAnVs013335; Wed, 01 Aug 2007 12:10:49 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71IqlhM001462; Wed, 01 Aug 2007 11:52:47 -0700 (PDT) Date: Wed, 01 Aug 2007 11:52:47 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 19:06:45 BST." To: Paul Jakma Cc: Jan Setje-Eilers , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708011852.l71IqlhM001462@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2809 > On Wed, 1 Aug 2007, Jan Setje-Eilers wrote: > > > GRUB on sparc comes up very short, particularly in terms of netboot > > support. > > While acknowledging the current reality of the above point... In that case this discussion should probably just end here. However for the sake of argument: > > In light of already having OBP which provides a supportable > > interactive moment before boot it's also not really clear why we would > > even want GRUB here. > > Hmm, the reason why is stated as motivation in the submitted document, > in the motivation section: > > 2.2 Commonality with Solaris x86 > > Pre-boot interactive session is most important when sysadmins are trying > to diagnose/fix some significant problem. That's *precisely* the time > when platform inconsistencies are the most annoying and disruptive. If you're trying to sort out some connectivity or device naming problem during pre-boot, anything that abstracts the firmwares view of the system from you is only going to aid in the confusion. Attempting to make x86 look like sparc or vice versa, or attempting to make both look like Solaris is the wrong approach. Systems tend to come with documentation that explains them in terms of their firmware. In the case of x86 that means first BIOS boot device (0x80) and the like. In the case of Solaris this means /pci@1e,600000/ide@d/disk@0,0. If there's a connectivity issue these names need to be exposed in terms that appear in the documentation for the hardware and on the labels on the system itself. If there's a name mapping issue, then the interfaces that map these firmware names into the Solaris device namespace need to be solid. On sparc this works pretty well. On x86 there are some inherent problems with the scheme used by BIOS not being entirely deterministic, particularly across different systems. EFI at least partially addresses this, at least for the (disk) devices we care most about. > I.e. Divergence makes our users lives harder. So there needs to be a > really good reason for it - "I can't see why we'd want GRUB on SPARC" > surely isn't it ;). We should minimise divergence for it's /own sake/ - > no other justification should be needed. We need GRUB (or something like it or the DCA) on x86 because there is no uniform pre-boot environment to pass things like boot options. This exists on sparc, so there's no need to add more layers. > Whether it's possible to get GRUB or GRUB2 working at a reasonable > time/resource cost, I of course don't know though. ;) I've looked into it and don't believe it is, at least not at this point in time. A couple of other folks have pursued it more seriously. All of those folks have left the company at this point. I hope it wasn't causal, but I can't help but wonder. -jan From sacadmin Wed Aug 1 12:55:33 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71JtXSH028383 for ; Wed, 1 Aug 2007 12:55:33 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71JrGoS049936; Wed, 1 Aug 2007 13:53:17 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM40040P1WQH800@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 12:53:14 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM40030B1WNB550@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 12:53:11 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71Jr418018216; Wed, 01 Aug 2007 12:53:04 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71JCgfG001736; Wed, 01 Aug 2007 12:12:42 -0700 (PDT) Date: Wed, 01 Aug 2007 12:12:42 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 15:01:01 EDT." <1185994861.9479.13.camel@localhost> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708011912.l71JCgfG001736@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2104 > On Wed, 2007-08-01 at 09:12 -0700, Jan Setje-Eilers wrote: > > > if we limit cases > > > where we report inconsistencies to cases where we crash in the middle of > > > a non-alternate-root upgrade, I think we can make this dissatisfier > > > just vanish. > > > > That's exactly the approach that's been implemented if you take a look > > at /boot/solaris/filelist.safe (shipping now on x86), it contains a > > list of files that are either just caches or just grow. They are all safe > > to ignore changes in and the check now does just that. > > So, I don't think we're there yet -- this case identifies 12 files > in /etc as present in the boot archive and I only see half of them in > filelist.safe; The ones that aren't listed aren't totally provably safe to ignore changes in. They don't however on average end up changing without some sort of explicit (and on production systems very rare) act. > moreover, there's also dynamic configuration in *.conf > files in /kernel (the one I'm most familiar with is md.conf); The correct solution to .conf files is actually to suck them into smf and make that data and other data available during early boot. That's however another project. > what's more, /etc/system can be updated outside an upgrade and is > not in filelist.safe. But if you set something like say physmem in etc/system and didn't get it picked up early in boot (from the archive) you'd probably want to know about it. So I don't think it should go into filelist.safe. That said, it's a horrible interface and we should continue to work on either eliminating tunables entirely, or moving the ones that make sense to expose into other interfaces (like say smf). > Where I think we need to get to: only files packaged immutably (type > "f") are present in the boot archive; all others are read "a la carte" > from the root. That however defeats the archive. We need to be able to do a single read until we mount root. That does involve pulling a couple of editable files into the archive. However we need to continue to work on making them cache only. -jan From sacadmin Wed Aug 1 13:07:27 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71K7Re2028558 for ; Wed, 1 Aug 2007 13:07:27 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71K576K025717; Wed, 1 Aug 2007 13:05:11 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM4004832GNW300@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 13:05:11 -0700 (PDT) Received: from eastmail2bur.East.Sun.COM ([129.148.13.40]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM4003P92GKB780@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 13:05:08 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71K4iaE013222; Wed, 01 Aug 2007 16:04:44 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71K4iP8024857; Wed, 01 Aug 2007 16:04:44 -0400 (EDT) Date: Wed, 01 Aug 2007 16:04:43 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708011912.l71JCgfG001736@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , psarc@Sun.COM, Jerry.Gilliam@Sun.COM, Jan.Setje-Eilers@Sun.COM, John.Johnson@Sun.COM, Lori.Alt@Sun.COM, Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM Message-id: <1185998683.24794.6.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708011912.l71JCgfG001736@smack.Eng.Sun.COM> Status: RO Content-Length: 659 On Wed, 2007-08-01 at 12:12 -0700, Jan Setje-Eilers wrote: > That however defeats the archive. We need to be able to do a single > read until we mount root. But why is this desirable? What problems does this solve that are worse than a recurring risk that after a power outage boot will be interrupted while unanswerable questions are posed on the console? The booter is reading one file from the filesystem already; your materials state that, on sparc, it will be possible to bypass the archive entirely at the cost of a significant performance hit; why can't it also read a small set of configuration files to supplement the archive? - Bill From sacadmin Wed Aug 1 13:48:55 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71KmtaG029226 for ; Wed, 1 Aug 2007 13:48:55 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71Kkcne010118; Wed, 1 Aug 2007 13:46:39 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM4005054DQUL00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 13:46:38 -0700 (PDT) Received: from athyra.sfbay.sun.com ([129.146.228.66]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM4005RG4DPND10@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 13:46:37 -0700 (PDT) Received: from athyra (localhost [127.0.0.1]) by athyra.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71KkVsC018754; Wed, 01 Aug 2007 13:46:31 -0700 (PDT) Date: Wed, 01 Aug 2007 13:46:30 -0700 From: Mike Kupfer Subject: Re: 2006/525: New Boot Sparc In-reply-to: Message from Jan Setje-Eilers "of Tue, 31 Jul 2007 20:49:28 PDT." <200708010349.l713nS6p027586@smack.Eng.Sun.COM> Sender: kupfer@athyra.sfbay.sun.com To: Jan Setje-Eilers Cc: Jerry Gilliam , sommerfeld@sun.com, psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <18753.1186001190@athyra> X-Mailer: MH-E 8.0.3; nmh 1.0.4; XEmacs 21.4 (patch 18) Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 496 >>>>> "Jan" == Jan Setje-Eilers writes: Jan> Come to think of it I'm very confused as to why some random Jan> platmods are closed and others aren't. There are 3 likely reasons; one or more may apply. - the code is encumbered for some reason - we don't have a clear answer on whether the code is encumbered - the code has a compile-time dependency on something else in the closed tree For questions on specific kernel modules, check with me or Bonnie Corwin. mike From sacadmin Wed Aug 1 14:55:01 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71Lt0JZ001390 for ; Wed, 1 Aug 2007 14:55:00 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71LqcSV019789; Wed, 1 Aug 2007 22:52:39 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM4002017FQBD00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 14:52:38 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400K8Q7FQMF50@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 14:52:38 -0700 (PDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71LqQl6306998; Wed, 01 Aug 2007 14:52:27 -0700 (PDT) Date: Wed, 01 Aug 2007 11:49:56 -1000 From: Joseph Kowalski Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708011852.l71IqlhM001462@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Paul Jakma , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, joseph.bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B10004.9010107@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708011852.l71IqlhM001462@smack.Eng.Sun.COM> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 648 Jan Setje-Eilers wrote: > Systems tend to come with documentation that explains them in terms > of their firmware. In the case of x86 that means first BIOS boot > device (0x80) and the like. In the case of Solaris this means > /pci@1e,600000/ide@d/disk@0,0. > Huh? When was the last time a user opened a box containing either a SPARC machine or an x86 machine, from any vendor and read anything like this? That person is already in the area of "Damn, it doesn't work. Now I need to look at firmware. Damn, damn, damn,..." The concept is the same. It's the primary boot device, or the sequence of boot devices. What am I missing? - jek3 From sacadmin Wed Aug 1 15:20:31 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71MKUIq001795 for ; Wed, 1 Aug 2007 15:20:31 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71MICZH004977; Wed, 1 Aug 2007 23:18:14 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400H038MCEB00@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 16:18:12 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400GBG8MADQ10@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 16:18:11 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71MIAma003720; Wed, 01 Aug 2007 15:18:10 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71LbeC6003694; Wed, 01 Aug 2007 14:37:40 -0700 (PDT) Date: Wed, 01 Aug 2007 14:37:40 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 16:04:43 EDT." <1185998683.24794.6.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708012137.l71LbeC6003694@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1477 > On Wed, 2007-08-01 at 12:12 -0700, Jan Setje-Eilers wrote: > > That however defeats the archive. We need to be able to do a single > > read until we mount root. > > But why is this desirable? What problems does this solve that are worse > than a recurring risk that after a power outage boot will be > interrupted while unanswerable questions are posed on the console? Having a single item (or small set of items) to load means we can manage their state on disk and don't have to roll the ufs log or commit the intent log in the boot-block. The latter in particular has been identified as near-insurmountable be the zfs folks. > The booter is reading one file from the filesystem already; your > materials state that, on sparc, it will be possible to bypass the > archive entirely at the cost of a significant performance hit; It's also potentially very unsafe to do so due to the log issue. The last couple of times I've thought about this I've concluded that being able to boot from underneath the archive is neat and may be useful in some situations, so we'll leave it in. However it's not something we can expose in a supportable manner. > why can't > it also read a small set of configuration files to supplement the > archive? If I already have to manage the state of those files, in order to ensure that they are in a fully committed state in the fs, then they might as well just be in the archive which is managed to contain those files. -jan From sacadmin Wed Aug 1 16:03:46 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71N3kmk003078 for ; Wed, 1 Aug 2007 16:03:46 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71N1Ta2029121; Wed, 1 Aug 2007 16:01:29 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400901AMFJ800@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:01:27 -0700 (PDT) Received: from eastmail1bur.East.Sun.COM ([129.148.9.49]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400KRZAMEMF90@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:01:27 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail1bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71N1EsN005567; Wed, 01 Aug 2007 19:01:14 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71N1Eu4025586; Wed, 01 Aug 2007 19:01:14 -0400 (EDT) Date: Wed, 01 Aug 2007 19:01:13 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012137.l71LbeC6003694@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186009273.25504.25.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012137.l71LbeC6003694@smack.Eng.Sun.COM> Status: RO Content-Length: 1959 On Wed, 2007-08-01 at 14:37 -0700, Jan Setje-Eilers wrote: > > On Wed, 2007-08-01 at 12:12 -0700, Jan Setje-Eilers wrote: > > > That however defeats the archive. We need to be able to do a single > > > read until we mount root. > > > > But why is this desirable? What problems does this solve that are worse > > than a recurring risk that after a power outage boot will be > > interrupted while unanswerable questions are posed on the console? > > Having a single item (or small set of items) to load means we can > manage their state on disk and don't have to roll the ufs log or > commit the intent log in the boot-block. This makes no sense to me. A system can run stably for MONTHS with /etc/system out of date with its boot archive, and then fail to boot cleanly after a power failure because the boot-archive service notices an inconsistency. > The latter in particular has > been identified as near-insurmountable be the zfs folks. In the case of ZFS root, my understanding is that the worst that can happen if we don't commit the intent log before reading is that we will read /etc/system contents which doesn't contain edits made during the last few seconds before a crash. If we get /etc/system out of the boot archive, it may be months out of date. > It's also potentially very unsafe to do so due to the log issue. Huh? Not with zfs root -- the on-disk state will be self-consistent as of the last time an uberblock update committed -- at most a few seconds old. > > why can't > > it also read a small set of configuration files to supplement the > > archive? > > If I already have to manage the state of those files, in order to > ensure that they are in a fully committed state in the fs, then they > might as well just be in the archive which is managed to contain > those files. But we're not managing their state! We're not automatically regenerating the boot archive when these files are updated. - Bill From sacadmin Wed Aug 1 16:08:12 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71N8BhP003362 for ; Wed, 1 Aug 2007 16:08:12 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71N5lSX029618; Thu, 2 Aug 2007 00:05:51 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400K03ATPFN00@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 17:05:49 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400GSKATODN40@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 17:05:48 -0600 (MDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l71N5ipe319417; Wed, 01 Aug 2007 16:05:45 -0700 (PDT) Date: Wed, 01 Aug 2007 13:03:15 -1000 From: Joseph Kowalski Subject: Re: 2006/525: New Boot Sparc In-reply-to: <1185998683.24794.6.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B11133.9050109@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708011912.l71JCgfG001736@smack.Eng.Sun.COM> <1185998683.24794.6.camel@thunk> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 126 This still seems to be an open case. (Sent to psarc, not psarc-ext.) If there is a problem, contact John Plocher. - jek3 From sacadmin Wed Aug 1 16:11:01 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71NB1NW003550 for ; Wed, 1 Aug 2007 16:11:01 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71N8iCa055445; Wed, 1 Aug 2007 17:08:45 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400C01AYK8J00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 16:08:44 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM4008EUAYKBF70@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 16:08:44 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71N8gth008857; Wed, 01 Aug 2007 16:08:42 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71MRvWh004513; Wed, 01 Aug 2007 15:27:57 -0700 (PDT) Date: Wed, 01 Aug 2007 15:27:54 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 11:49:56 -1000." <46B10004.9010107@sun.com> To: Joseph Kowalski Cc: Jan Setje-Eilers , Paul Jakma , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, joseph.bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708012227.l71MRvWh004513@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1087 > Jan Setje-Eilers wrote: > > Systems tend to come with documentation that explains them in terms > > of their firmware. In the case of x86 that means first BIOS boot > > device (0x80) and the like. In the case of Solaris this means > > /pci@1e,600000/ide@d/disk@0,0. > > > Huh? > > When was the last time a user opened a box containing either a SPARC > machine or an x86 machine, from any vendor and read anything like this? http://panacea/twiki/bin/view/Products/ProdDocumentationDeviceMapSunFirex8x0 > That person is already in the area of "Damn, it doesn't work. Now I need > to look at firmware. Damn, damn, damn,..." > > The concept is the same. It's the primary boot device, or the sequence > of boot devices. Yes, and we ship devaliases for that. > What am I missing? When things beyond the aliases break down you need to know whey they actually are. I suppose alternatively you could just accept that if it doesn't work at that level you replace the entire box, and many folks do, but if you care to sort things out you get to look at these paths. -jan From sacadmin Wed Aug 1 16:27:48 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71NRlUr003778 for ; Wed, 1 Aug 2007 16:27:48 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71NPK7e007110; Thu, 2 Aug 2007 00:25:30 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400C03BQB0F00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:25:23 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400KFZBQBMFA0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:25:23 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l71NPMBI010319; Wed, 01 Aug 2007 16:25:22 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l71NF4O7005126; Wed, 01 Aug 2007 16:15:04 -0700 (PDT) Date: Wed, 01 Aug 2007 16:15:04 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 19:01:13 EDT." <1186009273.25504.25.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708012315.l71NF4O7005126@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2961 > On Wed, 2007-08-01 at 14:37 -0700, Jan Setje-Eilers wrote: > > > On Wed, 2007-08-01 at 12:12 -0700, Jan Setje-Eilers wrote: > > > > That however defeats the archive. We need to be able to do a single > > > > read until we mount root. > > > > > > But why is this desirable? What problems does this solve that are worse > > > than a recurring risk that after a power outage boot will be > > > interrupted while unanswerable questions are posed on the console? > > > > Having a single item (or small set of items) to load means we can > > manage their state on disk and don't have to roll the ufs log or > > commit the intent log in the boot-block. > > This makes no sense to me. A system can run stably for MONTHS > with /etc/system out of date with its boot archive, and then fail to > boot cleanly after a power failure because the boot-archive service > notices an inconsistency. So you prefer to load corrupt data in the case of a reboot soon after an update to an archive check warning? > > The latter in particular has > > been identified as near-insurmountable be the zfs folks. > > In the case of ZFS root, my understanding is that the worst that can > happen if we don't commit the intent log before reading is that we will > read /etc/system contents which doesn't contain edits made during the > last few seconds before a crash. I can't confirm or deny that, but even if that's the case you then have no way to know that the old copy was loaded. I suppose we could check-sum everything on load, but this really doesn't change the situation from the boot_archive. > If we get /etc/system out of the boot archive, it may be months out of > date. In which case we catch this when the archive contents is verified. > > It's also potentially very unsafe to do so due to the log issue. > > Huh? Not with zfs root -- the on-disk state will be self-consistent as > of the last time an uberblock update committed -- at most a few seconds > old. But it is with ufs. Yes, I'd like to just totally dismiss it as well at this point, but that's not going to be a realistic option for a while. Or should I infer that you agree to let me amend this case with ufs root obsolescence, with a patch binding. :) > > > why can't > > > it also read a small set of configuration files to supplement the > > > archive? > > > > If I already have to manage the state of those files, in order to > > ensure that they are in a fully committed state in the fs, then they > > might as well just be in the archive which is managed to contain > > those files. > > But we're not managing their state! We're not automatically > regenerating the boot archive when these files are updated. If they aren't in the archive, then their state would have to be managed to ensure that they aren't unrolled or uncommitted. I'm suggesting that managing that is as much work as ensuring they get placed in the archive at all the right times. -jan From jg@jurassic.sfbay.sun.com Wed Aug 1 16:53:05 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l71Nr4mp003948 for ; Wed, 1 Aug 2007 16:53:05 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l71Noc6S023872; Thu, 2 Aug 2007 00:50:44 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400E01CWIMV00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:50:42 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.224.31]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400K1MCWIMFC0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 16:50:42 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l71Nofph699895; Wed, 01 Aug 2007 16:50:42 -0700 (PDT) Date: Wed, 01 Aug 2007 16:49:32 -0700 (PDT) From: Jerry Gilliam Subject: 2006/525: New Boot Sparc To: psarc-ext@sun.com Cc: jerry.gilliam@sun.com, jan.setje-eilers@sun.com, john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200708012350.l71Nofph699895@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: oqq6wNsQjOqlZxD+QkIfKg== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 19963 I am sponsoring this project as a fast-track on behalf of Jan Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. The project desires patch/update binding. (For those who have already seen this, this re-introduction is to transition the case from closed to open.) thx, -jg #ident "@(#)design 1.2 07/07/31 SMI" New Solaris SPARC Boot Architecture Summary The Solaris SPARC bootstrap process is being redesigned, both to increase the commonality with Solaris x86, and to enable ITUs (install time updates) on SPARC. Secondary goals are to combine inetboot and wanboot into a single network boot architecture, and to provide a simplified architecture for disk filesystems other than UFS (e.g, ZFS). The Solaris x86 newboot project has already been delivered to Nevada and S10u1. That project was described by PSARC 2004/454. This project is a follow-on to that project. This design specification only covers the architecture for Solaris SPARC. For details of the Solaris x86 design, see PSARC 2004/454. 1. Introduction --------------- The Solaris boot process was designed in early 1990's for desktops which were tiny by today's standards. On the one hand, the design was constrained by a relatively small amount of system memory. On the other hand, the design took advantage of the presence of Open Boot Prom (OBP) on all Sun platforms. The resulting implementation involves a complex sequence of control handoff between kernel and OBP to load a minimum amount of text and data from the root device into memory. 2. Motivation ------------- A change in the Solaris boot architecture is motivated by the following problems: 2.1 Supporting new hardware Solaris SPARC does not support ITUs, so a platform that requires any Solaris changes requires a full Solaris update release. This requirement causes pain for both the Solaris and SPARC HW groups, as the two must fight over Update schedules for almost every HW platform. 2.2 Commonality with Solaris x86 The newboot project on Solaris x86 has made both the administrative and boot processes visibly different between Solaris SPARC and Solaris x86. This is in part due to the addition of a boot archive to Solaris x86. It is in Sun's interest that the look and feel of Solaris on different platforms be as common as feasible. 2.3 Ease of booting different FS types The current boot architecture requires multiple filesystem readers, making adding new ones a difficult process. The goal of this project is to only require one phase to read the root FS before the kernel mounts it. This will enable new filesystem types like ZFS to be more easily supported as root filesystems. 2.5 Common network boot process The Solaris SPARC network boot process is very different between booting over a LAN versus booting over a WAN. The WAN process boots via a miniroot that looks much like how Solaris x86 installs over a network, but there is no commonality in how the miniroots are created or administered. This project will unify SPARC network booting around a single architecture. 3. Proposed Architecture ------------------------ 3.1 Boot Phase Independence The primary design center is to make the phases of the boot process be independent of each other. This allows the addition of new features (e.g., new filesystem types) without requiring changes to multiple parts of the boot chain. This project envisions 4 phases of booting on SPARC machines: OBP The OBP phase of boot is unchanged. In fact, it's a requirement that SPARC newboot not require new OBP functionality. OBP will continue to load and execute a booter from a disk or network device. booter The booter phase is responsible for reading in the boot archive and executing it. This is the only phase of the boot process that requires knowledge of the root filesystem format. ramdisk The ramdisk is a boot archive containing either kernel modules or an install miniroot. This boot archive is the same boot archive as is used on Solaris x86. Its FS format is private to itself. i.e., neither the booter nor the kernel needs to know whether the archive is HSFS or UFS (or ZFS for that matter). The ramdisk will extract the kernel image from the boot archive and execute it. In order to minimize the size of the ramdisk, in particular the install miniroot, which must reside in memory, the contents of the miniroot will be compressed. This compression is on a per file level and is implemented within the filesystem. In order to create compressed files a userland utility is used that simply compresses the file in place, the file is then marked as compressed via the _FIO_COMPRESSED (private) ioctl in the filesystem metadata. kernel The final stage is the kernel. The kernel extracts the rest of the primary modules from the boot archive, initializes itself, mounts the real root file system, and throws away the boot archive. This process is also the same as on Solaris x86. 3.2 Removal of bootops (and ufsboot) The bootops vector on SPARC originally existed to support platforms with either OBP or SUNMON FW. The second stage booter (e.g., ufsboot) presented a common set of operations to the kernel so the kernel didn't need to know either what prom version was running or what filesystem type root was. The last Solaris release to support a SUNMON platform was Solaris 2.4, and the last one to support pre-ieee1275 OBP was Solaris 7, so it's time for the bootops to go gently into that good night. 3.2.3 unix and krtld combined The Xen project (PSARC 2006/260) has combined the unix and krtld modules in order to enable booting from either BIOS or the Xen hypervisor. Since combining these two makes the unix ELF header far simpler to parse and load, this project has adapted this change for SPARC. 3.2.4 boot properties There are three properties retrieved via the current bootops that neither OBP nor the kernel have any knowledge of. These are: "fstype" name of root filesystem type (e.g., "ufs") "impl-arch-name" platform name (aka `uname -i`) "whoami" file system name of booted kernel A new node (/packages/boot-properties) will be added to the OBP device tree by the booter which contains these properties. In addition to the above compatibility properties, a couple of new properties are needed: "bootarchive" boot archive path (as opposed to "bootpath", the root file system path) "elfheader-address" address of kernel ELF header, used by krtld in lieu of the bootaux vector passed up via the second-stage booter "elfheader-length" length of kernel ELF header "archive-fstype" fstype of boot archive 3.3 Differences from Solaris x86 The primary difference from PSARC 2004/454 is that there is no dependence on grub. This decision was made for both practical and functional reasons. The practical reason is that grub0.95 - which was used by Solaris x86 - is not available on SPARC. The functional reason is that most of the reasons grub was used for Solaris x86 (e.g., eliminate real-mode, eliminate the boot shell, 3rd party device support) are not applicable to Solaris SPARC. Making grub work in the network boot case on SPARC is not a trivial exercise, as grub does not currently support NFS or HTTP, and adding wanboot's non-exportable cryptography code is problematic with respect to the GPL. The boot menu provided by grub would be an interesting feature to provide Solaris SPARC users. If grub2 were to become available on SPARC in a stable supportable release it could be incorporated in booter phase above. This should not require much change to existing code since the booter retains its ability to load alternate secondary booters such as cprboot. 4 Interfaces ------------ 4.1 Interface Exported ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- Boot files Evolving platform//boot_archive default name of boot archive (initially may be sun4u/sun4v only) Boot args kernel args Evolving see kernel(1M) -F Evolving alternate booter or boot archive usr/sbin/bootadm(1M) Stable usr/sbin/root_archive(1M) Stable boot/solaris/bin create_ramdisk Proj. Priv. Install-time Update different user prompts boot/solaris filelist.ramdisk Proj. Priv. boot archive content filestat.ramdisk Proj. Priv. boot archive file status kernel/fs/dcfs Proj. Priv. compression file system sbin/fiocompress Proj. Priv. file compression utility usr/sbin/fiocompress Proj. Priv. link to sbin/fiocompress usr/include/sys/fs/decomp.h Proj. Priv. dcfs header file _FIO_COMPRESSED Proj. Priv. file compression ioctl usr/platform/sun4[uv]/lib/fs hsfs/bootblk Proj. Priv. filesystem readers zfs/bootblk Proj. Priv. filesystem readers platform/sun4[uv]/ufsboot Proj. Priv. removed kernel/misc/sparcv9/krtld Proj. Priv. merged into unix ----------------------------------------------------------------------- 4.2 Interfaces Reimplemented ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- reboot(1M) Stable? update boot archive as needed halt(1M) poweroff(1M) shutdown(1M) init(1M) pkgadd(1M) patchadd(1M) add_install_client Evolving modified server setup smdiskless(1M) Evolving setup /tftpboot area Install consume bootadm and installgrub Upgrade Live Upgrade Flash Install Net Install Jumpstart Release engineering tools ------------------------- modified miniroot construction 5. User Experience ------------------ 5.1 System startup The system startup changes are not visible to the user. 5.2 Installation and Upgrade The ITU menu which currently only exists on x86 may be exposed on sparc as well. Customers who rely on undocumented non-interfaced implementation details of add_install_client may need to amend their procedures. If we deliver the wanboot component, wanboot customers will see a simplification of the deployment procedures. 5.3 Internal tools The bfu scripts will be updated to permit developers to transition from old boot to new boot. For a system initially installed with old boot, it will be possible to bfu back and forth across the boundary. For a system initially installed with new boot, bfu back to old boot is not supported. Booting a glommed kernel is supported, with the same restrictions regarding the compatibility of userland and kernel. If the root file system goes into a non-bootable state, a user may boot the failsafe archive to perform manual recovery operations. The failsafe archives contains files normally present in a CD or netinstall miniroot. If the boot archive goes into a non-bootable state, a user may bypass the boot archive and directly boot the kernel with the -F option. 5.4 Coexistence with other OSes The only other OS of note for SPARC is Linux, and they currently have no grub plans. SPARC / Linux boots from a loader called SILO, which already has a grub-like menu facility. In general OBP is considered well suited to loading multiple OS's or versions of OS's from various devices on the system, so no additional work beyond what is provided by OBP/the system is desirable. 6. Technical Details -------------------- 6.1 OBP phase This project does not change the OBP phase of SPARC boot; this section is included for reference only. When a user types "boot" on an OBP-based system, the device selected - either from the command line or via the "boot-device" nvram variable - has its "open" and "load" methods called. The program loaded by this process is then executed, and the boot process enters the booter phase. 6.1.1 Disk For disk devices, the FW driver usually uses the OBP label package's "load" method, which parses the VTOC label at the beginning of the disk to locate the specified partition, then reads sectors 1-15 of that partition into memory. This area is commonly called the boot block and usually contains a filesystem reader. 6.1.2 Net For network devices, the process is slightly different between booting over a LAN versus booting over a WAN. In both cases, however, the prom will download a booter from a boot or install server (inetboot in this case). 6.1.2.1 LAN boot When booting over a LAN, the FW uses either RARP and BOOTP or DHCP to discover its boot or install server. It then uses TFTP to download the booter (inetboot in this case). 6.1.2.2 WAN boot When booting over a WAN, the FW uses either DHCP or nvram properties to discover its install server, and the router and proxies needed to connect to it. It then uses HTTP to download the booter, and may optionally check the booter's signature with a predefined private key. For more details, see PSARC 2001/009. 6.2 Booter phase This phase is derived from the SPARC wanboot ramdisk process (see PSARC 2001/009), and is responsible for reading the boot archive from the root file system (or install server in wanboot's case) into a ramdisk device. It does this by: 1) opening the boot-device (which it found as the "bootpath" property in the OBP "/chosen" node) 2) using its file system specific reader to read the boot archive (by default, /platform/`uname -m`/boot_archive) 3) creating a ramdisk device in "/ramdisk" 4) creating "bootarchive" and "fstype" properties in "/packages/boot-properties" 5) booting the archive (a ramdisk is just another type of disk, so executing the boot block area serves this purpose) 6.3 Ramdisk phase The ramdisk is self-describing in the same sense any disk image is by virtue of having a filesystem reader in its boot block. This reader is over 90% the same as the disk boot block for a given filesystem so the same program is re-used. Its job is to load and execute the kernel from the archive by: 1) opening the boot archive (the "bootarchive" property from the previous phase) 2) using its file system specific reader to read the kernel (by default, /platform/`uname -i`/kernel/unix) 3) creating "impl-arch-name", "whoami" and "elfheader" properties 4) executing the kernel 6.4 Kernel phase When krtld gains control, it mounts the boot archive and loads additional kernel modules from the boot archive via the ramdisk. Subsequent kernel initialization procedures remain the same until after the kernel mounts the root file system. At that point, the kernel throws away the boot archive and reclaims the memory it occupies. Note that in the install case, the ramdisk actually contains the root file system, and is not thrown away. The kernel ramdisk driver simply takes over control of the ramdisk image. 6.5 Chained booters When booting from a disk, the booter will support chained booters both for cprboot (see PSARC 1992/201) and for situations where the file system reader cannot fit in the boot block. In a future project, grub2 can use this facility to add a graphical user menu to the booter phase. This facility will not be available when booting from a network, since the chained booter usually uses the same virtual address space as the original booter. 6.6 Boot archive management There are two kinds of boot archive: failsafe and normal. A failsafe archive is self-sufficient and bootable by itself. It is created at install time and requires no maintenance. A normal archive shadows a root filesystem, so it contains all kernel modules, driver.conf files, and a few configuration files in /etc which are read by the kernel before root is mounted. Once the root filesystem is mounted, the kernel discards the boot archive from memory and file I/O will be performed against the root device. By default, the normal archive contains the following files and directories: etc/system etc/name_to_major etc/driver_aliases etc/name_to_sysnum etc/dacf.conf etc/driver_classes etc/path_to_inst etc/devices/devid_cache etc/devices/mdi_scsi_vhci_cache etc/devices/mdi_ib_cache etc/cluster/nodeid etc/zfs/zpool.cache kernel platform The contents under the platform directory will be segregated into those needed for a sun4u boot archive and those needed for a sun4v boot archive. Further per-platform differentiated boot archives may be considered if that helps us gain faster booting via faster archive load, trading off a more complicated archive construction process. If any files in this list (or under directories listed) is updated, the boot archive must be rebuilt prior to the next reboot for the modification to take effect. The package and patch tools are updated to update the boot archive whenever needed. In addition, the boot archive is updated as necessary on an orderly system shutdown to catch files modified manually. The boot archive could be out of sync with the root filesystem if the system panics in the middle of an update, but before archive update is completed. We check for such conditions on every boot before root filesystem is mounted writeable. If an inconsistency is detected, the system will stop in single-user mode, similar to current behavior when fsck fails on the root filesystem. The recommended recovery method is to boot the failsafe archive and recreate the boot_archive. An expert user may decide to continue booting if the out-of-sync files are not critical. As noted above, if the boot archive cannot be fixed, the kernel can be directly booted from the boot via the -F argument. This option will not work when booting via HTTP, since there is no underlying FS to read from. It will also be noticably slower than booting via the archive, since the relatively simple booters do not implement a buffer cache. The bootadm(1M) command will handle the details of archive update and verification. 6.7 Install and upgrade Normal install and upgrade is achieved by booting the miniroot from either CDROM/DVD or from the network. In both cases, the root filesystem of the miniroot is the ramdisk. This allows the Solaris boot CD to be ejected without rebooting the system. The boot archive contains the entire miniroot. The construction of the install CD is modified to use an hsfs boot block. The miniroot is packed into a single file in ufs format, to be loaded as the ramdisk image. The setup of the net boot server is also modified. The boot server will serve a boot strap as well as the ramdisk image which is downloaded and then booted from. The netinstall image will be packed using root_archive(1m). The process for installing the OS to disk remains the same except that the boot blocks are different and a boot archive must be constructed prior to booting the install target disk. The boot archives are created using bootadm(1M). The rest of code changes comes with packages and patches and no special treatment is required. 6.8 Diskless clients Diskless boot is similar to booting the miniroot for net install except that the root filesystem is on NFS instead of on UFS. 6.9 Install-time Update (ITU) New-boot (phase 1: x86) reduced the x86 ITU mechanism described in PSARC 1997/059 to simply adding Solaris binaries (drivers, kernel modules, commands, libraries, symlinks and the like) to the running miniroot and then the target install environment. It also extended the supported media that an ITU could be supplied on to include CD/DVD, memory sticks and similar devices. This mechanism will now be made available on sparc as well to allow platform support to be delivered out of band of a regularly scheduled release. If a core kernel component needs to be updated, say unix or something else that needs to be loaded before an ITU can be added, then a pre-patched miniroot image needs to be made available (50Mb download) along with the patch. References ---------- 1. Shudong Zhou PSARC 2004/454 Solaris Boot Architecture 2. Carl Smith PSARC 2001/009 WAN-boot 3. Clark Dong PSARC 1992/201 Checkpoint Resume (Reanimator) 4. Allan McKillop PSARC 2006/260 Solaris on Xen From sommerfeld@sun.com Wed Aug 1 17:31:36 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l720VZdM004515 for ; Wed, 1 Aug 2007 17:31:36 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l720TIEf027567; Thu, 2 Aug 2007 08:29:19 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400I07EOU0T00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 17:29:18 -0700 (PDT) Received: from eastmail1bur.East.Sun.COM ([129.148.9.49]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400KZZEOJMGE0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 17:29:07 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail1bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l720Sq4M014666; Wed, 01 Aug 2007 20:28:52 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l720Sqwj025922; Wed, 01 Aug 2007 20:28:52 -0400 (EDT) Date: Wed, 01 Aug 2007 20:28:50 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012315.l71NF4O7005126@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186014530.25504.82.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012315.l71NF4O7005126@smack.Eng.Sun.COM> Status: RO Content-Length: 3580 (earlier messages in this thread should be visible in a day or so at http://www.opensolaris.org/os/community/arc/caselog/2006/525/mail; sorry for the confusion). On Wed, 2007-08-01 at 16:15 -0700, Jan Setje-Eilers wrote: > So you prefer to load corrupt data in the case of a reboot soon > after an update to an archive check warning? Not corrupt data, slightly old data. I realize that there are lingering nightmares of boots interrupted by corruption caused by a long-standing bug in logging UFS, but that bug -- 4782952 -- was fixed late in s10. During s10 development I had to manually recover a bunch of systems, typically due to munched md.conf files. logging ufs before the fix to 4782952 permitted blocks belonging to a file stably stored to be freed and then overwritten before the transaction which freed the block committed. (now, as is typical for changes to solaris UFS, that fix needed some followup work, but as best I can tell, things have damped out..) I saw boot failures on mirrored root sparc systems on a regular basis before this bug was fixed and haven't seen them since. I don't want to start seeing them again if this project integrates. > > In the case of ZFS root, my understanding is that the worst that can > > happen if we don't commit the intent log before reading is that we will > > read /etc/system contents which doesn't contain edits made during the > > last few seconds before a crash. > > I can't confirm or deny that, You don't need to; Neil Perrin confirmed this recently; see the mail log for 2007/171 (ZFS Separate Intent Log). I asked: > As I understand it, loss of the information in the intent log means that > the last few seconds of changes to a pool have been lost, but the pool > is otherwise intact. His response was "True" > but even if that's the case you then have no way to know that the old copy was loaded. I'm not sure that's a problem -- you're booting a point-in-time consistent config, just not necessarily the up-to-the-millisecond config at the time of the crash. Now, that's not good enough if we crash in the middle of a pkgadd -- but then the boot archive doesn't help very much then, either (because once we come up and discard the boot archive we may still load a mix of old and new kernel modules) For cases where there isn't a need for consistent updates to multiple files we should be fine. > > If we get /etc/system out of the boot archive, it may be months out of > > date. > > In which case we catch this when the archive contents is verified. And essentially crash/hang until an expert comes along to rescue the system, which is IMHO unacceptable behavior. > > > It's also potentially very unsafe to do so due to the log issue. > > > > Huh? Not with zfs root -- the on-disk state will be self-consistent as > > of the last time an uberblock update committed -- at most a few seconds > > old. > > But it is with ufs. As best as I can tell, it's not been unsafe for ufs since 4782952 was fixed. I saw lots of problems on pre-FCS s10, but I've never seen problems of lufs corruption breaking boot on sparc systems running s10 FCS or nevada. > If they aren't in the archive, then their state would have to be > managed to ensure that they aren't unrolled or uncommitted. But that's not how (working) lufs and zfs actually work. If the code which updates these files does the usual copy-edit-fsync-rename dance, there should never be a window where the on-disk structure even *without* the log contains something other than either the old or the new version of the file. From setje@smack.eng.sun.com Wed Aug 1 17:44:42 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l720ifM9004572 for ; Wed, 1 Aug 2007 17:44:42 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l720g7l7002694; Thu, 2 Aug 2007 08:42:25 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400303FAN7400@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 18:42:23 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400GM9FAMDW90@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 18:42:22 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l720gIS7017768; Wed, 01 Aug 2007 17:42:18 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l720Vkk9006181; Wed, 01 Aug 2007 17:31:46 -0700 (PDT) Date: Wed, 01 Aug 2007 17:31:46 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 20:28:50 EDT." <1186014530.25504.82.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 4241 You seem to be arguing that we can come up with old data without rolling/committing logs. If that's acceptable, then we can some up with the old archive as well and can just eliminate the check. I get the impression that you're placing some value on how old the data is. However in the case of a non-interfaced binary kernel component that really doesn't matter. It only matters if it's compatible with the rest of the bits or not. So either way we'd need the check. -jan > (earlier messages in this thread should be visible in a day or so at > http://www.opensolaris.org/os/community/arc/caselog/2006/525/mail; > sorry for the confusion). > > On Wed, 2007-08-01 at 16:15 -0700, Jan Setje-Eilers wrote: > > So you prefer to load corrupt data in the case of a reboot soon > > after an update to an archive check warning? > > Not corrupt data, slightly old data. > > I realize that there are lingering nightmares of boots interrupted by > corruption caused by a long-standing bug in logging UFS, but that bug -- > 4782952 -- was fixed late in s10. > > During s10 development I had to manually recover a bunch of systems, > typically due to munched md.conf files. logging ufs before the fix to > 4782952 permitted blocks belonging to a file stably stored to be freed > and then overwritten before the transaction which freed the block > committed. > > (now, as is typical for changes to solaris UFS, that fix needed some > followup work, but as best I can tell, things have damped out..) > > I saw boot failures on mirrored root sparc systems on a regular basis > before this bug was fixed and haven't seen them since. I don't want to > start seeing them again if this project integrates. > > > > In the case of ZFS root, my understanding is that the worst that can > > > happen if we don't commit the intent log before reading is that we will > > > read /etc/system contents which doesn't contain edits made during the > > > last few seconds before a crash. > > > > I can't confirm or deny that, > > You don't need to; Neil Perrin confirmed this recently; see the mail log > for 2007/171 (ZFS Separate Intent Log). I asked: > > > As I understand it, loss of the information in the intent log means that > > the last few seconds of changes to a pool have been lost, but the pool > > is otherwise intact. > > His response was "True" > > > but even if that's the case you then have no way to know that the old copy was loaded. > > I'm not sure that's a problem -- you're booting a point-in-time > consistent config, just not necessarily the up-to-the-millisecond config > at the time of the crash. > > Now, that's not good enough if we crash in the middle of a pkgadd -- but > then the boot archive doesn't help very much then, either (because once > we come up and discard the boot archive we may still load a mix of old > and new kernel modules) > > For cases where there isn't a need for consistent updates to multiple > files we should be fine. > > > > If we get /etc/system out of the boot archive, it may be months out of > > > date. > > > > In which case we catch this when the archive contents is verified. > > And essentially crash/hang until an expert comes along to rescue the > system, which is IMHO unacceptable behavior. > > > > > It's also potentially very unsafe to do so due to the log issue. > > > > > > Huh? Not with zfs root -- the on-disk state will be self-consistent as > > > of the last time an uberblock update committed -- at most a few seconds > > > old. > > > > But it is with ufs. > > As best as I can tell, it's not been unsafe for ufs since 4782952 was > fixed. I saw lots of problems on pre-FCS s10, but I've never seen > problems of lufs corruption breaking boot on sparc systems running s10 > FCS or nevada. > > > If they aren't in the archive, then their state would have to be > > managed to ensure that they aren't unrolled or uncommitted. > > But that's not how (working) lufs and zfs actually work. If the code > which updates these files does the usual copy-edit-fsync-rename dance, > there should never be a window where the on-disk structure even > *without* the log contains something other than either the old or the > new version of the file. > > From sommerfeld@sun.com Wed Aug 1 18:31:55 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l721Vshm005371 for ; Wed, 1 Aug 2007 18:31:54 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l721TT1L025264; Thu, 2 Aug 2007 09:29:38 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400I2HHHAT000@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 18:29:34 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400IQTHH70U40@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 18:29:32 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l721TDM0002158; Wed, 01 Aug 2007 21:29:13 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l721TDBc026062; Wed, 01 Aug 2007 21:29:13 -0400 (EDT) Date: Wed, 01 Aug 2007 21:29:12 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186018152.25504.125.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> Status: RO Content-Length: 1580 On Wed, 2007-08-01 at 17:31 -0700, Jan Setje-Eilers wrote: > I get the impression that you're placing some value on how old the > data is. I'm placing a great deal of value on a system in normal operation never requiring manual intervention to come back up after an asynchronous interruption (power hit, panic, etc.,). > However in the case of a non-interfaced binary kernel > component that really doesn't matter. And that's why I conceded in the first message I sent to this case that it was okay to interrupt boot in the event an upgrade of the active root was interrupted. You'd be just as screwed without the boot archive. The only way to ensure you always run with a a matched set of kernel modules is via live upgrade or equivalent, where you upgrade a copy of the boot environment and atomicly swap the new one in once it's complete. To repeat what I wrote earlier today: Several of the files included in the boot archive (the files in /etc, for instance) have dynamic content which is expected to change under normal system operation in the presence of hardware changes (for instance, USB hotplug or disk replacement). If it's possible to continue to bypass the boot archive during boot, it seems to me that it should be possibile to omit dynamic files from the boot archive entirely and instead always read them from the root filesystem; if we limit cases where we report inconsistencies to cases where we crash in the middle of a non-alternate-root upgrade, I think we can make this dissatisfier just vanish. -- I feel TCR-strong about this. - Bill From sacadmin Wed Aug 1 21:08:21 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7248Knx007737 for ; Wed, 1 Aug 2007 21:08:21 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7245tX8000824; Thu, 2 Aug 2007 12:05:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400E05OPU6200@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 21:05:54 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400IZLOPU9A50@nwk-avmta-1.sfbay.Sun.COM>; Wed, 01 Aug 2007 21:05:54 -0700 (PDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l7245jnM349913; Wed, 01 Aug 2007 21:05:45 -0700 (PDT) Date: Wed, 01 Aug 2007 18:03:14 -1000 From: Joseph Kowalski Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012227.l71MRvWh004513@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Paul Jakma , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, joseph.bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B15782.4060903@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012227.l71MRvWh004513@smack.Eng.Sun.COM> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 1617 Jan Setje-Eilers wrote: >> Jan Setje-Eilers wrote: >> >>> Systems tend to come with documentation that explains them in terms >>> of their firmware. In the case of x86 that means first BIOS boot >>> device (0x80) and the like. In the case of Solaris this means >>> /pci@1e,600000/ide@d/disk@0,0. >>> >>> >> Huh? >> >> When was the last time a user opened a box containing either a SPARC >> machine or an x86 machine, from any vendor and read anything like this? >> > > > http://panacea/twiki/bin/view/Products/ProdDocumentationDeviceMapSunFirex8x0 > I'm not getting your point. Yes, big servers are complex. What does that have to do with SPARC/x86? If your point is that big servers are more likely to be SPARC based, would you be willing to place a bet that this assertion will be true in a year? Two years? 5 years? >> That person is already in the area of "Damn, it doesn't work. Now I need >> to look at firmware. Damn, damn, damn,..." >> >> The concept is the same. It's the primary boot device, or the sequence >> of boot devices. >> > > Yes, and we ship devaliases for that. > > >> What am I missing? >> > > When things beyond the aliases break down you need to know whey they > actually are. > Sure. This is true of SPARC or a similarly complex x86 machine. Sorry, I just can't see that this design is ISA dependent. > I suppose alternatively you could just accept that if it doesn't > work at that level you replace the entire box, and many folks do, but > if you care to sort things out you get to look at these paths. > :-) > -jan > > - jek3 From setje@smack.eng.sun.com Wed Aug 1 22:55:57 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l725tumE009086 for ; Wed, 1 Aug 2007 22:55:57 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l725raS0018172; Thu, 2 Aug 2007 13:53:40 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400007TPE3X00@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 23:53:38 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM400GA1TPDM830@brm-avmta-1.central.sun.com>; Wed, 01 Aug 2007 23:53:37 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l725rY24024937; Wed, 01 Aug 2007 22:53:34 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l725DFD8010082; Wed, 01 Aug 2007 22:13:15 -0700 (PDT) Date: Wed, 01 Aug 2007 22:13:14 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 21:29:12 EDT." <1186018152.25504.125.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708020513.l725DFD8010082@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 3190 > > I get the impression that you're placing some value on how old the > > data is. > > I'm placing a great deal of value on a system in normal operation never > requiring manual intervention to come back up after an asynchronous > interruption (power hit, panic, etc.,). > > However in the case of a non-interfaced binary kernel > > component that really doesn't matter. > > And that's why I conceded in the first message I sent to this case that > it was okay to interrupt boot in the event an upgrade of the active root > was interrupted. You'd be just as screwed without the boot archive. > > The only way to ensure you always run with a a matched set of kernel > modules is via live upgrade or equivalent, where you upgrade a copy of > the boot environment and atomicly swap the new one in once it's > complete. > > To repeat what I wrote earlier today: > > Several of the files included in the boot archive (the files in /etc, > for instance) have dynamic content which is expected to change under > normal system operation in the presence of hardware changes (for OK, lets say you edited etc/system and didn't sync the archive and encounter a power failure. With the archive we come up with the old etc/system and stop to tell you about this. Without the archive we come up and get the old non-rolled etc/system and hopefully still run the check and stop to tell you about this. Granted the log gets rolled faster and more dynamically than the archive is updated. If you'd like to discuss ways to update the archive more aggressively (I'm have some interest in FEM, a cron job would just make things worse), I'm open to that. > instance, USB hotplug or disk replacement). You keep mentioning USB hotplug. That updates path_to_inst which since it only grows has been part of filelist.safe since snv_44 (also in u4), which means that action, which was a significant issue is a thing of the past. > If it's possible to > continue to bypass the boot archive during boot, It's possible, but has the following catches: 1) This only works on OBP platforms, x86 will need to be documented as different and doesn't benefit from any of the effort invested in this. That means we'll have to continue to solve that problem some other way there. 2) You may still pick up old (hopefully not corrupt) versions of the files. That means that this doesn't actually solve the problem as we still need the check. 3) We continue calling back into OBP to do do this which leaves us with one less place we managed to let go of the firmware. > it seems to me that it > should be possibile to omit dynamic files from the boot archive entirely > and instead always read them from the root filesystem; if we limit cases > where we report inconsistencies to cases where we crash in the middle of > a non-alternate-root upgrade, I think we can make this dissatisfier > just vanish. > > -- > > I feel TCR-strong about this. In that case we need another way to address this. Booting underneath the archive is a neat trick we happen to be able to pull off on sparc. However it doesn't solve the issue of wanting to avoid the up-to-datedness check. -jan From sacadmin Wed Aug 1 22:59:24 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l725xN7f009100 for ; Wed, 1 Aug 2007 22:59:23 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l725v2hV019594; Thu, 2 Aug 2007 13:57:05 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM400A01TV30E00@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 22:57:03 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM4006DLTV3GS20@nwk-avmta-2.sfbay.sun.com>; Wed, 01 Aug 2007 22:57:03 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l725v0De025243; Wed, 01 Aug 2007 22:57:00 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l725cvcN010482; Wed, 01 Aug 2007 22:38:57 -0700 (PDT) Date: Wed, 01 Aug 2007 22:38:57 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 01 Aug 2007 18:03:14 -1000." <46B15782.4060903@sun.com> To: Joseph Kowalski Cc: Jan Setje-Eilers , Paul Jakma , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, joseph.bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708020538.l725cvcN010482@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1808 > I'm not getting your point. Yes, big servers are complex. What does > that have to do > with SPARC/x86? Sorry, I've clearly confused you. I thought you had asked for an example of documentation that used the firmware device names to explain a system. My point is that we shouldn't try to make a SPARC system look and act like an x86 system or vice-versa. We did this back in the days of the DCA making x86 early boot look a lot like 1275. This was really rather useless when a Solaris customer looked at their Compaq documentation. By the same notion I don't think we should try to make SPARC systems look like our current x86 experience just to make the two appear the same. As, when something goes wrong in this space, it really does come down to the details. So, while having GRUB on SPARC might make the pictures in the manuals look nice and uniform, it does little to nothing in terms of making dealing with the systems equivalent. > > When things beyond the aliases break down you need to know whey they > > actually are. > > > Sure. This is true of SPARC or a similarly complex x86 machine. > > Sorry, I just can't see that this design is ISA dependent. Not ISA, but platform dependent. A BIOS system is very different from an OBP system in terms of pre-boot interaction. Side note: Considering what's going on with GRUB development (GRUB 0.9x development is frozen and no one can put back anything and GRUB 2 unfinished making glacial progress with no distributions willing to even consider adopting) I'm hesitant to commit another platform to it as the x86 Linux world may very well abandon GRUB in favor of elilo. A move, that if we want to continue to derive benefits from code exposure, testing and debugging on other platforms, we will want to follow. -jan From sacadmin Thu Aug 2 04:17:48 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72BHlBk014514 for ; Thu, 2 Aug 2007 04:17:47 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l72BFVIa012248 for <@sunmail2sca.sfbay.sun.com:psarc@sun.com>; Thu, 2 Aug 2007 19:15:32 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM50080D8LU5U00@nwk-avmta-1.sfbay.Sun.COM> for psarc@sun.com (ORCPT psarc@sun.com); Thu, 02 Aug 2007 04:15:30 -0700 (PDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM5000K28LSHA40@nwk-avmta-1.sfbay.Sun.COM> for psarc@sun.com (ORCPT psarc@sun.com); Thu, 02 Aug 2007 04:15:30 -0700 (PDT) Received: from d1-emea-09.sun.com ([192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l72BFR5v002868 for ; Thu, 02 Aug 2007 11:15:27 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM500E018HMOZ00@d1-emea-09.sun.com> (original mail from Paul.Jakma@Sun.COM) for psarc@sun.com (ORCPT psarc@sun.com); Thu, 02 Aug 2007 12:15:27 +0100 (BST) Received: from dhcp-ellg01-81-120.UK.Sun.COM ([129.159.81.120]) by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM5005L88LQI320@d1-emea-09.sun.com>; Thu, 02 Aug 2007 12:15:27 +0100 (BST) Date: Thu, 02 Aug 2007 12:14:51 +0100 (IST) From: Paul Jakma Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708020538.l725cvcN010482@smack.Eng.Sun.COM> Sender: Paul.Jakma@sun.com To: Jan Setje-Eilers Cc: Joseph Kowalski , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; format=flowed; charset=US-ASCII Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708020538.l725cvcN010482@smack.Eng.Sun.COM> Status: RO Content-Length: 878 FWIW, My point was not about how OBP and BIOS interact with early kernel boot, but about human interaction. As per Joseph's point, the human-friendly concepts ("boot device", "{first,second,..} disk") are platform-independent, never mind system-independent. The GRUB UI has its own namespace for that - the sysadmin always knows how to find his disks in a GRUB UI. You don't need to know the details of the PCI topology in your machine, and how your boot disk fits into that to fix your machine with the GRUB UI. If we expect users to know that level of detail to fix things in fecal-fan-impact situations, we're going to find those users will buy other systems next time around.. ;) But anyway.. regards, -- Paul Jakma, Solaris Networking Sun Microsystems, Scotland http://opensolaris.org/os/project/quagga tel: EMEA x73150 / +44 15066 73150 From Joerg.Schilling@fokus.fraunhofer.de Thu Aug 2 06:06:51 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72D6oBi016129 for ; Thu, 2 Aug 2007 06:06:51 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72D4Vjp003797; Thu, 2 Aug 2007 14:04:31 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500H2DDNJO000@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 06:04:31 -0700 (PDT) Received: from sca-ea-mail-1.sun.com ([192.18.43.24]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM5000UODNIHAC0@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 06:04:30 -0700 (PDT) Received: from relay13i.sun.com (ip123.net129179-4.block1.us.syntegra.com [129.179.4.123]) by sca-ea-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l72D4Tvc022461; Thu, 02 Aug 2007 13:04:29 +0000 (GMT) Received: from mmp12es.sun.com ([160.41.209.22] [160.41.209.22]) by relay13i.sun.com with ESMTP id BT-MMP-258944; Thu, 02 Aug 2007 13:04:28 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp12es.sun.com with ESMTP id BT-MMP-2875637; Thu, 02 Aug 2007 13:04:28 +0000 (Z) Received: from mailgwb1.fraunhofer.de ([153.96.87.18] [153.96.87.18]) by relay1ib.sun.com with ESMTP id BT-MMP-1136965; Thu, 02 Aug 2007 13:04:28 +0000 (Z) Received: from mailgwb1.fraunhofer.de (localhost [127.0.0.1]) by mailgwb1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l72D4PpJ028976; Thu, 02 Aug 2007 15:04:25 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) by mailgwb1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l72D4NG7028796 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 02 Aug 2007 15:04:24 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.13.7/8.13.7) with SMTP id l72D0WiX017786; Thu, 02 Aug 2007 15:00:33 +0200 (MEST) Received: from burner ([10.147.65.166]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Thu, 02 Aug 2007 15:00:33 +0200 Date: Thu, 02 Aug 2007 14:57:48 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012350.l71Nofph699895@jurassic.eng.sun.com> To: psarc-ext@sun.com, jg@jurassic.sfbay.sun.com Cc: Lori.Alt@sun.com, lin.ling@sun.com, Joseph.Bonasera@sun.com, john.johnson@sun.com, jerry.gilliam@sun.com, Jeff.Cheeney@sun.com, jan.setje-eilers@sun.com Message-id: <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.2.0.264296 X-Fraunhofer-Email-Policy: accepted References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 02 Aug 2007 13:00:33.0066 (UTC) FILETIME=[1C5B80A0:01C7D505] Status: RO Content-Length: 1644 Jerry Gilliam wrote: > ramdisk > > The ramdisk is a boot archive containing either kernel modules or an install > miniroot. This boot archive is the same boot archive as is used on Solaris x86. > Its FS format is private to itself. i.e., neither the booter nor the kernel > needs to know whether the archive is HSFS or UFS (or ZFS for that matter). The > ramdisk will extract the kernel image from the boot archive and execute it. > > In order to minimize the size of the ramdisk, in particular the > install miniroot, which must reside in memory, the contents of the > miniroot will be compressed. This compression is on a per file level > and is implemented within the filesystem. In order to create > compressed files a userland utility is used that simply compresses the > file in place, the file is then marked as compressed via the > _FIO_COMPRESSED (private) ioctl in the filesystem metadata. Do you like to implement the zisofs extension for Rock Ridge or does this apply to ZFS only? > 5.4 Coexistence with other OSes > > The only other OS of note for SPARC is Linux, and they currently have no grub > plans. SPARC / Linux boots from a loader called SILO, which already has a > grub-like menu facility. I have no idea about SILO on harddisks, on CDs/DVDs, it is imcompatible to the Sun boot. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From carlsonj@phorcys.east.sun.com Thu Aug 2 06:42:04 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72Dg3f4016257 for ; Thu, 2 Aug 2007 06:42:04 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l72DdhDs021773; Thu, 2 Aug 2007 21:39:44 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500707FA6SQ00@nwk-avmta-2.sfbay.sun.com>; Thu, 02 Aug 2007 06:39:42 -0700 (PDT) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM5006JLFA5RH60@nwk-avmta-2.sfbay.sun.com>; Thu, 02 Aug 2007 06:39:42 -0700 (PDT) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l72DdU51029836; Thu, 02 Aug 2007 09:39:30 -0400 (EDT) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l72DdUOa029833; Thu, 02 Aug 2007 09:39:30 -0400 (EDT) Date: Thu, 02 Aug 2007 09:39:30 -0400 From: James Carlson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Bill Sommerfeld , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <18097.56978.420493.786461@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <1186014530.25504.82.camel@thunk> <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> Status: RO Content-Length: 2934 Jan Setje-Eilers writes: > You seem to be arguing that we can come up with old data without > rolling/committing logs. If that's acceptable, then we can some up > with the old archive as well and can just eliminate the check. I don't think so. The effects of the boot archive are quite surprising to administrators because they have a much longer delay time. If I edit /etc/system (or, worse, perform some action that has the side-effect of editing that file), then my expectation is that unless the system goes down unexpectedly *right away*, those bits will be on the disk "soon." I have an expectation that after a few seconds (and perhaps a few mumbled superstitious 'sync' invocations), everything that I've changed administratively is stable again. User data in flight may be discarded if the system crashes while it's in flight, but the OS itself is stable. That's true on the current SPARC systems, but not true once we have a boot archive containing volatile files. In that case, I've got an extra step to perform: regenerating the boot archive after doing some edits. On x86, because I can't easily know when these sorts of changes happen, I've taken to uttering "bootadm update-archive -v" every now and then, just on spec. Every once in a while, it catches something surprising. (Particularly so in the first reboot after an upgrade -- something about the boot process almost always tweaks /etc/system or some famous file, meaning that right after upgrade, my system is just _always_ in an unstable state.) I suspect that some customers are using cron jobs for similar effect. It's like the bad old days with "sync ; sync ; sync" ... wait for it ... "reboot." It causes users to distrust the system. > I get the impression that you're placing some value on how old the > data is. However in the case of a non-interfaced binary kernel > component that really doesn't matter. It only matters if it's > compatible with the rest of the bits or not. So either way we'd need > the check. Yes, part of it is a concern over how old the data in the archive are. The other part is the effect of failure: when this happens, the machine is stuck in boot. Unless you've got access to the console, and realize what's happened, the machine is just a warm brick. Moving the volatile files out of the archive limits the scope of the problem. At that point, it's _only_ intentional packaging changes that could affect the consistency of the boot archive, and we could devise some simple way to make sure that those changes get committed to the archive. With volatile files in the archive, it's much more wide-open, and more exotic (and I think unlikely) schemes such as FEM would be needed. -- James Carlson, Solaris Networking Sun Microsystems / 1 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sommerfeld@sun.com Thu Aug 2 07:47:26 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72ElQhN017422 for ; Thu, 2 Aug 2007 07:47:26 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72Ej7nE023467; Thu, 2 Aug 2007 08:45:07 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500B05IB93900@nwk-avmta-2.sfbay.sun.com>; Thu, 02 Aug 2007 07:45:09 -0700 (PDT) Received: from localhost.east.sun.com ([129.148.19.3]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500AKSIB7PA00@nwk-avmta-2.sfbay.sun.com>; Thu, 02 Aug 2007 07:45:08 -0700 (PDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l72EgUFk010571; Thu, 02 Aug 2007 10:42:30 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l72EgTSa010570; Thu, 02 Aug 2007 10:42:30 -0400 (EDT) Date: Thu, 02 Aug 2007 10:42:29 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <18097.56978.420493.786461@gargle.gargle.HOWL> To: James Carlson Cc: Jan Setje-Eilers , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186065749.10179.30.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <1186014530.25504.82.camel@thunk> <200708020031.l720Vkk9006181@smack.Eng.Sun.COM> <18097.56978.420493.786461@gargle.gargle.HOWL> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 1315 On Thu, 2007-08-02 at 09:39 -0400, James Carlson wrote: > I have an expectation that after a few seconds (and perhaps a few > mumbled superstitious 'sync' invocations), everything that I've > changed administratively is stable again. I have somewhat higher expectations. I have an expectation that after a few seconds (and perhaps a few mumbled superstitious 'sync' invocations), everything that I changed will still be there after reboot. I have an expectation that if the system crashes at any time while I am in the middle of applying a series of correct changes to files in the correct order using tools that update files atomically, the system will be able to boot back to multiuser mode and will come back with a prefix of those changes active and a possibly longer prefix of those changes visible in the filesystem. I do not like sitting there in fear of a power failure during the large chunk of a minute that "bootadm update-archive" takes to run to completion on an X4600. (when I timed it, it took 26.5 seconds, but it felt like several minutes). I realize that newboot on x86 shipped before it was complete -- there were compelling reasons why it was significantly better than what it replaced in certain dimensions. But as best as I can tell, this is not the case on sparc. - Bill From sommerfeld@Sun.COM Thu Aug 2 09:33:51 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72GXo2v019910 for ; Thu, 2 Aug 2007 09:33:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72GVX8T002723; Thu, 2 Aug 2007 10:31:34 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500B0RN8N3H00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 09:31:35 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500KYVN8NV3B0@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 09:31:35 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l72GVLg1036472; Thu, 02 Aug 2007 12:31:21 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l72GVL2H029222; Thu, 02 Aug 2007 12:31:21 -0400 (EDT) Date: Thu, 02 Aug 2007 12:31:20 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708020513.l725DFD8010082@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Jerry Gilliam , PSARC-ext@Sun.COM, Jerry.Gilliam@Sun.COM, Jan.Setje-Eilers@Sun.COM, John.Johnson@Sun.COM, Lori.Alt@Sun.COM, Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM Message-id: <1186072280.29012.14.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708020513.l725DFD8010082@smack.Eng.Sun.COM> Status: RO Content-Length: 496 On Wed, 2007-08-01 at 22:13 -0700, Jan Setje-Eilers wrote: > (I'm have some interest in FEM, a cron job would just make > things worse), I'm open to that. At best, FEM-driven update of the boot archive could shrink the window of vulnerability to perhaps a few minutes. While this is a significant improvement over current behavior, it's still unacceptably large. It's clear to me that there are simpler approaches which completely eliminate the window of vulnerability I'm concerned about. From sacadmin Thu Aug 2 11:15:58 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72IFvYB024245 for ; Thu, 2 Aug 2007 11:15:57 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l72IDRxF023212; Fri, 3 Aug 2007 02:13:40 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500L0DRYREM00@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 12:13:39 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500DLJRYQ8D90@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 12:13:38 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l72IDaIh017961; Thu, 02 Aug 2007 11:13:36 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l72I3IDl015194; Thu, 02 Aug 2007 11:03:18 -0700 (PDT) Date: Thu, 02 Aug 2007 11:03:18 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Thu, 02 Aug 2007 12:14:51 BST." To: Paul Jakma Cc: Jan Setje-Eilers , Joseph Kowalski , Darren J Moffat , John Johnson , Jerry Gilliam , psarc@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708021803.l72I3IDl015194@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1768 So lets suppose you have an x86 system with GRUB that's booting off of some internal disk. Now you're sitting at the GRUB prompt and want to boot off a dvd. How do you find out what device it is? To fint out tou type "root (" and hit the tab key. Then GRUB will list a bunch of bios device numbers and you try them one by one until you find the one with the hsfs filesystem on it that makes the light on the dvd drive light up. Conversely on sparc, if you're sitting at the ok prompt, you simply type boot cdrom and typically the devalias is set up correctly. In the rare case that it's not you will have to run showdevs and find the actual path. The bottom line is that the level of pain is similar, but ultimately the way in which these devices are presented is totally dependent on the firmware doing the presentation. -jan > FWIW, My point was not about how OBP and BIOS interact with early kernel > boot, but about human interaction. > > As per Joseph's point, the human-friendly concepts ("boot device", > "{first,second,..} disk") are platform-independent, never mind > system-independent. The GRUB UI has its own namespace for that - the > sysadmin always knows how to find his disks in a GRUB UI. > > You don't need to know the details of the PCI topology in your machine, > and how your boot disk fits into that to fix your machine with the GRUB > UI. If we expect users to know that level of detail to fix things in > fecal-fan-impact situations, we're going to find those users will buy > other systems next time around.. ;) > > But anyway.. > > regards, > -- > Paul Jakma, > Solaris Networking Sun Microsystems, Scotland > http://opensolaris.org/os/project/quagga tel: EMEA x73150 / +44 15066 73150 From jek3@sun.com Thu Aug 2 11:24:22 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72IOMTr024624 for ; Thu, 2 Aug 2007 11:24:22 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72IM65u026391; Thu, 2 Aug 2007 11:22:07 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500L1RSCUUY00@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 12:22:06 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500DWASCT8D90@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 12:22:06 -0600 (MDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l72ILuvj443888; Thu, 02 Aug 2007 11:21:56 -0700 (PDT) Date: Thu, 02 Aug 2007 08:19:25 -1000 From: Joseph Kowalski Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg Schilling Cc: psarc-ext@sun.com, jg@jurassic.sfbay.sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Joseph.Bonasera@sun.com, John.Johnson@sun.com, Jerry.Gilliam@sun.com, Jeff.Cheeney@sun.com, Jan.Setje-Eilers@sun.com Message-id: <46B2202D.9090409@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Thunderbird 1.5.0.5 (X11/20060925) Status: RO Content-Length: 493 Joerg Schilling wrote: > Jerry Gilliam wrote: > ... >> 5.4 Coexistence with other OSes >> >> The only other OS of note for SPARC is Linux, and they currently have no grub >> plans. SPARC / Linux boots from a loader called SILO, which already has a >> grub-like menu facility. >> > > I have no idea about SILO on harddisks, on CDs/DVDs, it is imcompatible to the > Sun boot. > Speaking of SILO, I assume the Ubuntu UltraSPARC T1 port uses SILO? - jek3 From john.johnson@sun.com Thu Aug 2 11:32:22 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72IWLib024936 for ; Thu, 2 Aug 2007 11:32:22 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72IU1qv000493; Thu, 2 Aug 2007 19:30:05 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500L0DSQ3K200@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 11:30:03 -0700 (PDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500FAZSQ1QF30@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 11:30:01 -0700 (PDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l72ITxfb010513; Thu, 02 Aug 2007 11:29:59 -0700 (PDT) Received: from [129.146.96.101] (dvorak [129.146.96.101]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72ITxmm034147; Thu, 02 Aug 2007 11:29:59 -0700 (PDT) Date: Thu, 02 Aug 2007 11:29:59 -0700 From: John Johnson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B2202D.9090409@sun.com> To: Joseph Kowalski Cc: Joerg Schilling , psarc-ext@sun.com, jg@jurassic.sfbay.sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Joseph.Bonasera@sun.com, Jerry.Gilliam@sun.com, Jeff.Cheeney@sun.com, Jan.Setje-Eilers@sun.com Message-id: <46B222A7.7090408@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> <46B2202D.9090409@sun.com> User-Agent: Thunderbird 2.0.0.5pre (X11/20070721) Status: RO Content-Length: 266 Joseph Kowalski wrote: >> > Speaking of SILO, I assume the Ubuntu UltraSPARC T1 port uses SILO? > Yes, same as previous SPARC/Linux versions. The SPARC/Linux lead is not a big grub fan, so the grub team would have to do the grub work to boot it. JJ From setje@smack.eng.sun.com Thu Aug 2 11:40:31 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72IeVYu025121 for ; Thu, 2 Aug 2007 11:40:31 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72IcEKN050991 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 2 Aug 2007 12:38:14 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM500M03T3S2P00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 02 Aug 2007 11:38:16 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM500FQTT3SQL30@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 02 Aug 2007 11:38:16 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l72IcCmr021538; Thu, 02 Aug 2007 11:38:12 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l72ICojQ015404; Thu, 02 Aug 2007 11:12:50 -0700 (PDT) Date: Thu, 02 Aug 2007 11:12:50 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Thu, 02 Aug 2007 14:57:48 +0200." <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> To: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Cc: PSARC-ext@sun.com, jg@jurassic.sfbay.sun.com, Lori.Alt@sun.com, lin.ling@sun.com, Joseph.Bonasera@sun.com, john.johnson@sun.com, jerry.gilliam@sun.com, Jeff.Cheeney@sun.com, jan.setje-eilers@sun.com, setje@smack.eng.sun.com Message-id: <200708021812.l72ICojQ015404@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=iso-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1507 > > In order to minimize the size of the ramdisk, in particular the > > install miniroot, which must reside in memory, the contents of the > > miniroot will be compressed. This compression is on a per file level > > and is implemented within the filesystem. In order to create > > compressed files a userland utility is used that simply compresses the > > file in place, the file is then marked as compressed via the > > _FIO_COMPRESSED (private) ioctl in the filesystem metadata. > > Do you like to implement the zisofs extension for Rock Ridge or does this > apply to ZFS only? Funny you should ask. We'd like to, but we're still waiting for legal review (let's please not discuss that here as neither you nor I can do anything to sway them one way or another). If we get the green light we'll do this, which will involve the corresponding change to mkisofs. > > 5.4 Coexistence with other OSes > > > > The only other OS of note for SPARC is Linux, and they currently have no grub > > plans. SPARC / Linux boots from a loader called SILO, which already has a > > grub-like menu facility. > > I have no idea about SILO on harddisks, on CDs/DVDs, it is imcompatible to the > Sun boot. I'd expect it lives in the boot-block of the slice, so assuming one OS per slice it can co-exist as closely as it reasonably needs to. More than one OS per slice assumes they can use the same FS as well, which other than iso/hsfs is typically not something people would expect to be able to do. -jan From setje@smack.eng.sun.com Thu Aug 2 16:40:48 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72NemmO003220 for ; Thu, 2 Aug 2007 16:40:48 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72NcX8K023388; Thu, 2 Aug 2007 16:38:33 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM60020T709XB00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 16:38:33 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM6001UY7097H80@nwk-avmta-1.sfbay.Sun.COM>; Thu, 02 Aug 2007 16:38:33 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l72NcNkP003071; Thu, 02 Aug 2007 16:38:23 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l72Mw0Zf003023; Thu, 02 Aug 2007 15:58:00 -0700 (PDT) Date: Thu, 02 Aug 2007 15:58:00 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Thu, 02 Aug 2007 09:39:30 EDT." <18097.56978.420493.786461@gargle.gargle.HOWL> To: James Carlson Cc: Jan Setje-Eilers , Bill Sommerfeld , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 X-Exmh-Isig-CompType: repl X-Exmh-Isig-Folder: inbox Status: RO Content-Length: 2919 Thanks to everyone who commented so far. Here's an attempt summarize what I've gathered so far on the archive The primary concern is the editable files in the archive, not the ones delivered and managed by the install and patching tools. Further, the most serious concern is the system stopping during boot and requiring the administrator to sign off on its state. This is followed by the concern about the effect of using old data. It sounds like it would be acceptable for whatever tool is used to update these files to do whatever sync is needed atomically as part of the update. Sadly for most of these files this tool is still a text editor and we don't get to fix that in a patch release. :( However it seems like the files of concern fall into a category of their own that is currently being dealt with in a manner that should actually be reserved for out of sync kernel binaries. This suggest we have the following three types of files in the archive: 1) Kernel binaries (including modules and drivers) If any of these are out of sync and not just new, the concern is that things with miss-matched interfaces may be running and the kernel may act unpredictably. So, for these we stop hard in a panic like fashion and refuse to mount root until someone active takes responsibility for doing so. This is the classic check. 2) Files that are either caches or only grow or can be safely re-read later. These are what's currently in filelist.safe. These files do not cause the system to stop during boot and trigger an archive update later in boot. This is the refinement that went back into nv44 and u4. 3) Files that etc/system that contain information that can't always be usefully processed later during boot, but if out of date _do not_ leave the system in a dangerously unstable state. These files are currently being treated just like the kernel binaries. However since the system is not dangerously unstable at this point, it is reasonable to drive on and mount root read-write. This means they should really get their own check. If this check fails, I propose the following: Print a warning to console. Leave a service (which won't block multi-user) in maintenance mode so the state is communicated via svcs -x. Drive on and mount root read-write. Update the archive. And potentially reboot immediately to the device we just booted from now that the archive is updated. The auto-reboot still makes us a little nervous, so it may be something that needs to be explicitly enabled based on site policy, but at least on sparc we have a solid idea of what boot device we booted from, so it may turn out to a reasonable default action to take. Clearly the exact service dependencies will differ depending on whether or not the system will automatically reboot to pick up the changed files. Ideas, thoughts, comments? -jan From sacadmin Thu Aug 2 16:47:05 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l72Nl5EG003278 for ; Thu, 2 Aug 2007 16:47:05 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l72NioTK024484 for <@sunmail2sca.sfbay.sun.com:fwarc@sun.com>; Thu, 2 Aug 2007 16:44:50 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM60030J7AQJD00@nwk-avmta-1.sfbay.Sun.COM> for fwarc@sun.com (ORCPT fwarc@Sun.COM); Thu, 02 Aug 2007 16:44:50 -0700 (PDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM6001WN7AP7HB0@nwk-avmta-1.sfbay.Sun.COM> for fwarc@sun.com (ORCPT fwarc@Sun.COM); Thu, 02 Aug 2007 16:44:49 -0700 (PDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l72NinVa008466 for ; Thu, 02 Aug 2007 23:44:49 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM6000016U6TP00@mail-amer.sun.com> (original mail from Hitendra.Zhangada@Sun.COM) for fwarc@Sun.COM (ORCPT fwarc@Sun.COM); Thu, 02 Aug 2007 17:44:49 -0600 (MDT) Received: from [129.150.33.103] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM600C6S7AOGRG6@mail-amer.sun.com> for fwarc@Sun.COM (ORCPT fwarc@Sun.COM); Thu, 02 Aug 2007 17:44:49 -0600 (MDT) Date: Thu, 02 Aug 2007 16:46:10 -0700 From: Hitendra Zhangada Subject: [Fwd: 2006/525: New Boot Sparc] Sender: Hitendra.Zhangada@Sun.COM To: Firmware Arch Message-id: <46B26CC2.5010908@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 User-Agent: Thunderbird 2.0.0.5 (Windows/20070716) Status: RO Content-Length: 20783 FYI. See ongoing discussions on this case in the mail file, http://sac.eng/PSARC/2006/525/mail -------- Original Message -------- Subject: 2006/525: New Boot Sparc Date: Wed, 01 Aug 2007 16:49:32 -0700 (PDT) From: Jerry Gilliam Reply-To: Jerry Gilliam To: psarc-ext@sun.com CC: John.Johnson@Sun.COM, Jerry.Gilliam@Sun.COM, Lori.Alt@Sun.COM, Lin.Ling@Sun.COM, Jan.Setje-Eilers@Sun.COM, Joseph.Bonasera@Sun.COM, Jeff.Cheeney@Sun.COM I am sponsoring this project as a fast-track on behalf of Jan Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. The project desires patch/update binding. (For those who have already seen this, this re-introduction is to transition the case from closed to open.) thx, -jg #ident "@(#)design 1.2 07/07/31 SMI" New Solaris SPARC Boot Architecture Summary The Solaris SPARC bootstrap process is being redesigned, both to increase the commonality with Solaris x86, and to enable ITUs (install time updates) on SPARC. Secondary goals are to combine inetboot and wanboot into a single network boot architecture, and to provide a simplified architecture for disk filesystems other than UFS (e.g, ZFS). The Solaris x86 newboot project has already been delivered to Nevada and S10u1. That project was described by PSARC 2004/454. This project is a follow-on to that project. This design specification only covers the architecture for Solaris SPARC. For details of the Solaris x86 design, see PSARC 2004/454. 1. Introduction --------------- The Solaris boot process was designed in early 1990's for desktops which were tiny by today's standards. On the one hand, the design was constrained by a relatively small amount of system memory. On the other hand, the design took advantage of the presence of Open Boot Prom (OBP) on all Sun platforms. The resulting implementation involves a complex sequence of control handoff between kernel and OBP to load a minimum amount of text and data from the root device into memory. 2. Motivation ------------- A change in the Solaris boot architecture is motivated by the following problems: 2.1 Supporting new hardware Solaris SPARC does not support ITUs, so a platform that requires any Solaris changes requires a full Solaris update release. This requirement causes pain for both the Solaris and SPARC HW groups, as the two must fight over Update schedules for almost every HW platform. 2.2 Commonality with Solaris x86 The newboot project on Solaris x86 has made both the administrative and boot processes visibly different between Solaris SPARC and Solaris x86. This is in part due to the addition of a boot archive to Solaris x86. It is in Sun's interest that the look and feel of Solaris on different platforms be as common as feasible. 2.3 Ease of booting different FS types The current boot architecture requires multiple filesystem readers, making adding new ones a difficult process. The goal of this project is to only require one phase to read the root FS before the kernel mounts it. This will enable new filesystem types like ZFS to be more easily supported as root filesystems. 2.5 Common network boot process The Solaris SPARC network boot process is very different between booting over a LAN versus booting over a WAN. The WAN process boots via a miniroot that looks much like how Solaris x86 installs over a network, but there is no commonality in how the miniroots are created or administered. This project will unify SPARC network booting around a single architecture. 3. Proposed Architecture ------------------------ 3.1 Boot Phase Independence The primary design center is to make the phases of the boot process be independent of each other. This allows the addition of new features (e.g., new filesystem types) without requiring changes to multiple parts of the boot chain. This project envisions 4 phases of booting on SPARC machines: OBP The OBP phase of boot is unchanged. In fact, it's a requirement that SPARC newboot not require new OBP functionality. OBP will continue to load and execute a booter from a disk or network device. booter The booter phase is responsible for reading in the boot archive and executing it. This is the only phase of the boot process that requires knowledge of the root filesystem format. ramdisk The ramdisk is a boot archive containing either kernel modules or an install miniroot. This boot archive is the same boot archive as is used on Solaris x86. Its FS format is private to itself. i.e., neither the booter nor the kernel needs to know whether the archive is HSFS or UFS (or ZFS for that matter). The ramdisk will extract the kernel image from the boot archive and execute it. In order to minimize the size of the ramdisk, in particular the install miniroot, which must reside in memory, the contents of the miniroot will be compressed. This compression is on a per file level and is implemented within the filesystem. In order to create compressed files a userland utility is used that simply compresses the file in place, the file is then marked as compressed via the _FIO_COMPRESSED (private) ioctl in the filesystem metadata. kernel The final stage is the kernel. The kernel extracts the rest of the primary modules from the boot archive, initializes itself, mounts the real root file system, and throws away the boot archive. This process is also the same as on Solaris x86. 3.2 Removal of bootops (and ufsboot) The bootops vector on SPARC originally existed to support platforms with either OBP or SUNMON FW. The second stage booter (e.g., ufsboot) presented a common set of operations to the kernel so the kernel didn't need to know either what prom version was running or what filesystem type root was. The last Solaris release to support a SUNMON platform was Solaris 2.4, and the last one to support pre-ieee1275 OBP was Solaris 7, so it's time for the bootops to go gently into that good night. 3.2.3 unix and krtld combined The Xen project (PSARC 2006/260) has combined the unix and krtld modules in order to enable booting from either BIOS or the Xen hypervisor. Since combining these two makes the unix ELF header far simpler to parse and load, this project has adapted this change for SPARC. 3.2.4 boot properties There are three properties retrieved via the current bootops that neither OBP nor the kernel have any knowledge of. These are: "fstype" name of root filesystem type (e.g., "ufs") "impl-arch-name" platform name (aka `uname -i`) "whoami" file system name of booted kernel A new node (/packages/boot-properties) will be added to the OBP device tree by the booter which contains these properties. In addition to the above compatibility properties, a couple of new properties are needed: "bootarchive" boot archive path (as opposed to "bootpath", the root file system path) "elfheader-address" address of kernel ELF header, used by krtld in lieu of the bootaux vector passed up via the second-stage booter "elfheader-length" length of kernel ELF header "archive-fstype" fstype of boot archive 3.3 Differences from Solaris x86 The primary difference from PSARC 2004/454 is that there is no dependence on grub. This decision was made for both practical and functional reasons. The practical reason is that grub0.95 - which was used by Solaris x86 - is not available on SPARC. The functional reason is that most of the reasons grub was used for Solaris x86 (e.g., eliminate real-mode, eliminate the boot shell, 3rd party device support) are not applicable to Solaris SPARC. Making grub work in the network boot case on SPARC is not a trivial exercise, as grub does not currently support NFS or HTTP, and adding wanboot's non-exportable cryptography code is problematic with respect to the GPL. The boot menu provided by grub would be an interesting feature to provide Solaris SPARC users. If grub2 were to become available on SPARC in a stable supportable release it could be incorporated in booter phase above. This should not require much change to existing code since the booter retains its ability to load alternate secondary booters such as cprboot. 4 Interfaces ------------ 4.1 Interface Exported ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- Boot files Evolving platform//boot_archive default name of boot archive (initially may be sun4u/sun4v only) Boot args kernel args Evolving see kernel(1M) -F Evolving alternate booter or boot archive usr/sbin/bootadm(1M) Stable usr/sbin/root_archive(1M) Stable boot/solaris/bin create_ramdisk Proj. Priv. Install-time Update different user prompts boot/solaris filelist.ramdisk Proj. Priv. boot archive content filestat.ramdisk Proj. Priv. boot archive file status kernel/fs/dcfs Proj. Priv. compression file system sbin/fiocompress Proj. Priv. file compression utility usr/sbin/fiocompress Proj. Priv. link to sbin/fiocompress usr/include/sys/fs/decomp.h Proj. Priv. dcfs header file _FIO_COMPRESSED Proj. Priv. file compression ioctl usr/platform/sun4[uv]/lib/fs hsfs/bootblk Proj. Priv. filesystem readers zfs/bootblk Proj. Priv. filesystem readers platform/sun4[uv]/ufsboot Proj. Priv. removed kernel/misc/sparcv9/krtld Proj. Priv. merged into unix ----------------------------------------------------------------------- 4.2 Interfaces Reimplemented ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- reboot(1M) Stable? update boot archive as needed halt(1M) poweroff(1M) shutdown(1M) init(1M) pkgadd(1M) patchadd(1M) add_install_client Evolving modified server setup smdiskless(1M) Evolving setup /tftpboot area Install consume bootadm and installgrub Upgrade Live Upgrade Flash Install Net Install Jumpstart Release engineering tools ------------------------- modified miniroot construction 5. User Experience ------------------ 5.1 System startup The system startup changes are not visible to the user. 5.2 Installation and Upgrade The ITU menu which currently only exists on x86 may be exposed on sparc as well. Customers who rely on undocumented non-interfaced implementation details of add_install_client may need to amend their procedures. If we deliver the wanboot component, wanboot customers will see a simplification of the deployment procedures. 5.3 Internal tools The bfu scripts will be updated to permit developers to transition from old boot to new boot. For a system initially installed with old boot, it will be possible to bfu back and forth across the boundary. For a system initially installed with new boot, bfu back to old boot is not supported. Booting a glommed kernel is supported, with the same restrictions regarding the compatibility of userland and kernel. If the root file system goes into a non-bootable state, a user may boot the failsafe archive to perform manual recovery operations. The failsafe archives contains files normally present in a CD or netinstall miniroot. If the boot archive goes into a non-bootable state, a user may bypass the boot archive and directly boot the kernel with the -F option. 5.4 Coexistence with other OSes The only other OS of note for SPARC is Linux, and they currently have no grub plans. SPARC / Linux boots from a loader called SILO, which already has a grub-like menu facility. In general OBP is considered well suited to loading multiple OS's or versions of OS's from various devices on the system, so no additional work beyond what is provided by OBP/the system is desirable. 6. Technical Details -------------------- 6.1 OBP phase This project does not change the OBP phase of SPARC boot; this section is included for reference only. When a user types "boot" on an OBP-based system, the device selected - either from the command line or via the "boot-device" nvram variable - has its "open" and "load" methods called. The program loaded by this process is then executed, and the boot process enters the booter phase. 6.1.1 Disk For disk devices, the FW driver usually uses the OBP label package's "load" method, which parses the VTOC label at the beginning of the disk to locate the specified partition, then reads sectors 1-15 of that partition into memory. This area is commonly called the boot block and usually contains a filesystem reader. 6.1.2 Net For network devices, the process is slightly different between booting over a LAN versus booting over a WAN. In both cases, however, the prom will download a booter from a boot or install server (inetboot in this case). 6.1.2.1 LAN boot When booting over a LAN, the FW uses either RARP and BOOTP or DHCP to discover its boot or install server. It then uses TFTP to download the booter (inetboot in this case). 6.1.2.2 WAN boot When booting over a WAN, the FW uses either DHCP or nvram properties to discover its install server, and the router and proxies needed to connect to it. It then uses HTTP to download the booter, and may optionally check the booter's signature with a predefined private key. For more details, see PSARC 2001/009. 6.2 Booter phase This phase is derived from the SPARC wanboot ramdisk process (see PSARC 2001/009), and is responsible for reading the boot archive from the root file system (or install server in wanboot's case) into a ramdisk device. It does this by: 1) opening the boot-device (which it found as the "bootpath" property in the OBP "/chosen" node) 2) using its file system specific reader to read the boot archive (by default, /platform/`uname -m`/boot_archive) 3) creating a ramdisk device in "/ramdisk" 4) creating "bootarchive" and "fstype" properties in "/packages/boot-properties" 5) booting the archive (a ramdisk is just another type of disk, so executing the boot block area serves this purpose) 6.3 Ramdisk phase The ramdisk is self-describing in the same sense any disk image is by virtue of having a filesystem reader in its boot block. This reader is over 90% the same as the disk boot block for a given filesystem so the same program is re-used. Its job is to load and execute the kernel from the archive by: 1) opening the boot archive (the "bootarchive" property from the previous phase) 2) using its file system specific reader to read the kernel (by default, /platform/`uname -i`/kernel/unix) 3) creating "impl-arch-name", "whoami" and "elfheader" properties 4) executing the kernel 6.4 Kernel phase When krtld gains control, it mounts the boot archive and loads additional kernel modules from the boot archive via the ramdisk. Subsequent kernel initialization procedures remain the same until after the kernel mounts the root file system. At that point, the kernel throws away the boot archive and reclaims the memory it occupies. Note that in the install case, the ramdisk actually contains the root file system, and is not thrown away. The kernel ramdisk driver simply takes over control of the ramdisk image. 6.5 Chained booters When booting from a disk, the booter will support chained booters both for cprboot (see PSARC 1992/201) and for situations where the file system reader cannot fit in the boot block. In a future project, grub2 can use this facility to add a graphical user menu to the booter phase. This facility will not be available when booting from a network, since the chained booter usually uses the same virtual address space as the original booter. 6.6 Boot archive management There are two kinds of boot archive: failsafe and normal. A failsafe archive is self-sufficient and bootable by itself. It is created at install time and requires no maintenance. A normal archive shadows a root filesystem, so it contains all kernel modules, driver.conf files, and a few configuration files in /etc which are read by the kernel before root is mounted. Once the root filesystem is mounted, the kernel discards the boot archive from memory and file I/O will be performed against the root device. By default, the normal archive contains the following files and directories: etc/system etc/name_to_major etc/driver_aliases etc/name_to_sysnum etc/dacf.conf etc/driver_classes etc/path_to_inst etc/devices/devid_cache etc/devices/mdi_scsi_vhci_cache etc/devices/mdi_ib_cache etc/cluster/nodeid etc/zfs/zpool.cache kernel platform The contents under the platform directory will be segregated into those needed for a sun4u boot archive and those needed for a sun4v boot archive. Further per-platform differentiated boot archives may be considered if that helps us gain faster booting via faster archive load, trading off a more complicated archive construction process. If any files in this list (or under directories listed) is updated, the boot archive must be rebuilt prior to the next reboot for the modification to take effect. The package and patch tools are updated to update the boot archive whenever needed. In addition, the boot archive is updated as necessary on an orderly system shutdown to catch files modified manually. The boot archive could be out of sync with the root filesystem if the system panics in the middle of an update, but before archive update is completed. We check for such conditions on every boot before root filesystem is mounted writeable. If an inconsistency is detected, the system will stop in single-user mode, similar to current behavior when fsck fails on the root filesystem. The recommended recovery method is to boot the failsafe archive and recreate the boot_archive. An expert user may decide to continue booting if the out-of-sync files are not critical. As noted above, if the boot archive cannot be fixed, the kernel can be directly booted from the boot via the -F argument. This option will not work when booting via HTTP, since there is no underlying FS to read from. It will also be noticably slower than booting via the archive, since the relatively simple booters do not implement a buffer cache. The bootadm(1M) command will handle the details of archive update and verification. 6.7 Install and upgrade Normal install and upgrade is achieved by booting the miniroot from either CDROM/DVD or from the network. In both cases, the root filesystem of the miniroot is the ramdisk. This allows the Solaris boot CD to be ejected without rebooting the system. The boot archive contains the entire miniroot. The construction of the install CD is modified to use an hsfs boot block. The miniroot is packed into a single file in ufs format, to be loaded as the ramdisk image. The setup of the net boot server is also modified. The boot server will serve a boot strap as well as the ramdisk image which is downloaded and then booted from. The netinstall image will be packed using root_archive(1m). The process for installing the OS to disk remains the same except that the boot blocks are different and a boot archive must be constructed prior to booting the install target disk. The boot archives are created using bootadm(1M). The rest of code changes comes with packages and patches and no special treatment is required. 6.8 Diskless clients Diskless boot is similar to booting the miniroot for net install except that the root filesystem is on NFS instead of on UFS. 6.9 Install-time Update (ITU) New-boot (phase 1: x86) reduced the x86 ITU mechanism described in PSARC 1997/059 to simply adding Solaris binaries (drivers, kernel modules, commands, libraries, symlinks and the like) to the running miniroot and then the target install environment. It also extended the supported media that an ITU could be supplied on to include CD/DVD, memory sticks and similar devices. This mechanism will now be made available on sparc as well to allow platform support to be delivered out of band of a regularly scheduled release. If a core kernel component needs to be updated, say unix or something else that needs to be loaded before an ITU can be added, then a pre-patched miniroot image needs to be made available (50Mb download) along with the patch. References ---------- 1. Shudong Zhou PSARC 2004/454 Solaris Boot Architecture 2. Carl Smith PSARC 2001/009 WAN-boot 3. Clark Dong PSARC 1992/201 Checkpoint Resume (Reanimator) 4. Allan McKillop PSARC 2006/260 Solaris on Xen _______________________________________________ opensolaris-arc mailing list opensolaris-arc@opensolaris.org -- Hitendra Zhangada ============================================= SPS Common SW Features Engineering Systems Group, Sun Microsystems, Inc. Work Ph# (858) 625 3757, Ext. x53757 SUN Internal homepage http://esp.west/~hitu From Lori.Alt@sun.com Thu Aug 2 17:21:28 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l730LRWm004008 for ; Thu, 2 Aug 2007 17:21:28 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l730J5Fe026928; Fri, 3 Aug 2007 08:19:12 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM600J0N8VZWX00@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 18:19:11 -0600 (MDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM600J3W8VYTI00@brm-avmta-1.central.sun.com>; Thu, 02 Aug 2007 18:19:10 -0600 (MDT) Received: from fe-amer-03.sun.com ([192.18.108.177]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l730J9l8018600; Fri, 03 Aug 2007 00:19:09 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM600A017TNFM00@mail-amer.sun.com> (original mail from Lori.Alt@Sun.COM) ; Thu, 02 Aug 2007 18:19:09 -0600 (MDT) Received: from sun.com ([129.150.33.53]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM6007138VWLEB0@mail-amer.sun.com>; Thu, 02 Aug 2007 18:19:09 -0600 (MDT) Date: Thu, 02 Aug 2007 18:19:09 -0600 From: Lori Alt Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> Sender: Lori.Alt@sun.com To: Jan Setje-Eilers Cc: James Carlson , Bill Sommerfeld , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B2747D.5040508@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 Status: RO Content-Length: 3267 Jan Setje-Eilers wrote: > Thanks to everyone who commented so far. > > Here's an attempt summarize what I've gathered so far on the archive > > The primary concern is the editable files in the archive, not the ones >delivered and managed by the install and patching tools. > > Further, the most serious concern is the system stopping during boot >and requiring the administrator to sign off on its state. This is >followed by the concern about the effect of using old data. > > It sounds like it would be acceptable for whatever tool is used to >update these files to do whatever sync is needed atomically as part of >the update. Sadly for most of these files this tool is still a text >editor and we don't get to fix that in a patch release. :( > > However it seems like the files of concern fall into a category of >their own that is currently being dealt with in a manner that should >actually be reserved for out of sync kernel binaries. This suggest we >have the following three types of files in the archive: > > 1) Kernel binaries (including modules and drivers) > > If any of these are out of sync and not just new, the concern > is that things with miss-matched interfaces may be running and > the kernel may act unpredictably. > > So, for these we stop hard in a panic like fashion and refuse > to mount root until someone active takes responsibility for > doing so. > > This is the classic check. > > 2) Files that are either caches or only grow or can be safely re-read > later. > > These are what's currently in filelist.safe. These files do > not cause the system to stop during boot and trigger an > archive update later in boot. This is the refinement that went > back into nv44 and u4. > > 3) Files that etc/system that contain information that can't always > be usefully processed later during boot, but if out of date _do > not_ leave the system in a dangerously unstable state. > > These files are currently being treated just like the kernel > binaries. However since the system is not dangerously unstable > at this point, it is reasonable to drive on and mount root > read-write. > > This means they should really get their own check. > > If this check fails, I propose the following: > > Print a warning to console. > > Leave a service (which won't block multi-user) in > maintenance mode so the state is communicated via svcs > -x. > > Drive on and mount root read-write. > If the root is zfs, it will already have been mounted read-write (there is no need to do a read-only mount since zfs has no fsck.) so no need for a remount at this point. I don't think this changes the overall logic here though. > > Update the archive. > > And potentially reboot immediately to the device we > just booted from now that the archive is updated. > > The auto-reboot still makes us a little nervous, so it may be > something that needs to be explicitly enabled based on site > policy, but at least on sparc we have a solid idea of what > boot device we booted from, so it may turn out to a reasonable > default action to take. > > Clearly the exact service dependencies will differ depending > on whether or not the system will automatically reboot to > pick up the changed files. > > Ideas, thoughts, comments? > >-jan > > > > From Darren.Moffat@sun.com Fri Aug 3 02:46:53 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l739kqQq011567 for ; Fri, 3 Aug 2007 02:46:52 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l739iRx2007188; Fri, 3 Aug 2007 17:44:36 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM600907Z2A8B00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 03:44:34 -0600 (MDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM6008JNZ27BI00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 03:44:33 -0600 (MDT) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l739iUBM019066; Fri, 03 Aug 2007 09:44:30 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM600901YX5CS00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM); Fri, 03 Aug 2007 10:44:30 +0100 (BST) Received: from [129.156.173.21] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM6003C1Z1ZKC30@d1-emea-09.sun.com>; Fri, 03 Aug 2007 10:44:24 +0100 (BST) Date: Fri, 03 Aug 2007 10:44:23 +0100 From: Darren J Moffat Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> Sender: Darren.Moffat@sun.com To: Jan Setje-Eilers Cc: James Carlson , Bill Sommerfeld , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B2F8F7.2030206@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 2000 Jan Setje-Eilers wrote: > 3) Files that etc/system that contain information that can't always > be usefully processed later during boot, but if out of date _do > not_ leave the system in a dangerously unstable state. > > These files are currently being treated just like the kernel > binaries. However since the system is not dangerously unstable > at this point, it is reasonable to drive on and mount root > read-write. > > This means they should really get their own check. > > If this check fails, I propose the following: > > Print a warning to console. > > Leave a service (which won't block multi-user) in > maintenance mode so the state is communicated via svcs > -x. I like this. > Drive on and mount root read-write. Up to this point I'm fine with this. > Update the archive. If you do that you are you possibly removing information that the admin may need to diagnose why the system was in this state ? Also it will make the service in main look strange - it said the boot archive was out of date but and admin will find that it isn't. I'd personally leave this to one of the "later" normal cases that causes an automatic update of the archive. > And potentially reboot immediately to the device we > just booted from now that the archive is updated. I really don't like the auto reboot it could cause application problems and will almost certainly surprise admins. I don't think we should ever autoreboot after getting this far up. > The auto-reboot still makes us a little nervous, so it may be > something that needs to be explicitly enabled based on site > policy, but at least on sparc we have a solid idea of what > boot device we booted from, so it may turn out to a reasonable > default action to take. and since we don't on x86 I don't think we should do this since one of the stated goals of this project is removing differences! Also consider that there will be other platforms eventually. -- Darren J Moffat From Joerg.Schilling@fokus.fraunhofer.de Fri Aug 3 03:27:22 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73ARMKe012328 for ; Fri, 3 Aug 2007 03:27:22 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73AOxQx022448; Fri, 3 Aug 2007 11:25:00 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700B0D0XNLQ00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 04:24:59 -0600 (MDT) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM7008VQ0XLBA10@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 04:24:58 -0600 (MDT) Received: from relay24.sun.com (ip192-12-251-74.block6.us.syntegra.com [192.12.251.74]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l73AOvHk029850; Fri, 03 Aug 2007 10:24:57 +0000 (GMT) Received: from mms24es.sun.com ([150.143.232.74] [150.143.232.74]) by relay24.sun.com with ESMTP id BT-MMP-963780; Fri, 03 Aug 2007 10:24:57 +0000 (Z) Received: from mms23bas.mms.us.syntegra.com (mms23bas.mms.us.syntegra.com [192.12.251.50]) by mms24es.sun.com with ESMTP id BT-MMP-1729533; Fri, 03 Aug 2007 10:24:56 +0000 (Z) Received: from mailgw1.fraunhofer.de ([153.96.1.18] [153.96.1.18]) by relay23.sun.com with ESMTP id BT-MMP-13926288; Fri, 03 Aug 2007 10:24:56 +0000 (Z) Received: from mailgw1.fraunhofer.de (localhost [127.0.0.1]) by mailgw1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73AOsJj014209; Fri, 03 Aug 2007 12:24:55 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) by mailgw1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73AOsFN014165 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 03 Aug 2007 12:24:54 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.13.7/8.13.7) with SMTP id l73AOqmA006544; Fri, 03 Aug 2007 12:24:53 +0200 (MEST) Received: from burner ([10.147.65.166]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Fri, 03 Aug 2007 12:24:52 +0200 Date: Fri, 03 Aug 2007 12:22:09 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708021812.l72ICojQ015404@smack.Eng.Sun.COM> To: setje@smack.eng.sun.com Cc: Frank.Hofmann@sun.com, setje@smack.eng.sun.com, psarc-ext@sun.com, Lori.Alt@sun.com, lin.ling@sun.com, Joseph.Bonasera@sun.com, john.johnson@sun.com, jg@jurassic.sfbay.sun.com, jerry.gilliam@sun.com, Jeff.Cheeney@sun.com, jan.setje-eilers@sun.com Message-id: <46b301d1.B60X4zq+vhSNgupu%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.2.0.264296 X-Fraunhofer-Email-Policy: accepted References: <200708021812.l72ICojQ015404@smack.Eng.Sun.COM> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 03 Aug 2007 10:24:52.0493 (UTC) FILETIME=[875AEBD0:01C7D5B8] Status: RO Content-Length: 2945 Jan Setje-Eilers wrote: > > > and is implemented within the filesystem. In order to create > > > compressed files a userland utility is used that simply compresses the > > > file in place, the file is then marked as compressed via the > > > _FIO_COMPRESSED (private) ioctl in the filesystem metadata. > > > > Do you like to implement the zisofs extension for Rock Ridge or does this > > apply to ZFS only? > > Funny you should ask. We'd like to, but we're still waiting for legal > review (let's please not discuss that here as neither you nor I can do > anything to sway them one way or another). Well, I did propose to support this 1-2 years ago (not 100% sure abut the date). > If we get the green light we'll do this, which will involve the > corresponding change to mkisofs. Mkisofs by default includes support for "-z": -z, -transparent-compression Enable transparent compression of files We only need to add the program "mkzftree" which is currently non-portable (Linux only) available in source from "H. Peter Anvin". I did already talk with him about integrating the program into cdrtools a year ago and he did give me the permission. For license details, please contact me offline. Did you read the file README.compression from cdrtools? > > > 5.4 Coexistence with other OSes > > > > > > The only other OS of note for SPARC is Linux, and they currently have no grub > > > plans. SPARC / Linux boots from a loader called SILO, which already has a > > > grub-like menu facility. > > > > I have no idea about SILO on harddisks, on CDs/DVDs, it is imcompatible to the > > Sun boot. > > I'd expect it lives in the boot-block of the slice, so assuming one > OS per slice it can co-exist as closely as it reasonably needs to. > More than one OS per slice assumes they can use the same FS as well, > which other than iso/hsfs is typically not something people would > expect to be able to do. While the official Sparc boot (as expected by the boot prom) asumes that there is a Sun disk label on the CD/DVD media and the boot for a specific architecture is on a specific partition, SILO works on unlabelled media and needs a hand crafted command line to boot from CD/DVD. SILO also needs to be patched to know where it's second stage is located. For this reason, SILO is not userfriendly and it is not possible to auto-create a SILO enabled ISO-9660 filesystem using mkisofs. If you create a new boot, please make it compatible to what mkisofs supports. See also the description in the files: README.sparcboot README.sunx86boot README.eltorito in cdrtools. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From Joerg.Schilling@fokus.fraunhofer.de Fri Aug 3 03:30:06 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73AU6EB012345 for ; Fri, 3 Aug 2007 03:30:06 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73ARlCJ032678; Fri, 3 Aug 2007 04:27:47 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700F0112DYK00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 03:27:49 -0700 (PDT) Received: from sca-ea-mail-3.sun.com ([192.18.43.21]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM700A0M12CBW80@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 03:27:48 -0700 (PDT) Received: from relay16i.sun.com (ip126.net129179-4.block1.us.syntegra.com [129.179.4.126]) by sca-ea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l73ARlR8026854; Fri, 03 Aug 2007 10:27:48 +0000 (GMT) Received: from mmp11es.sun.com ([160.41.209.21] [160.41.209.21]) by relay16i.sun.com with ESMTP id BT-MMP-310330; Fri, 03 Aug 2007 10:27:47 +0000 (Z) Received: from relay11i.sun.com (relay11i.sun.com [129.179.4.121]) by mmp11es.sun.com with ESMTP id BT-MMP-359103; Fri, 03 Aug 2007 10:27:47 +0000 (Z) Received: from mailgw12.fraunhofer.de ([153.96.1.24] [153.96.1.24]) by relay1i.sun.com with ESMTP id BT-MMP-1587448; Fri, 03 Aug 2007 10:27:47 +0000 (Z) Received: from mailgw12.fraunhofer.de (localhost [127.0.0.1]) by mailgw12.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73ARjb6001929; Fri, 03 Aug 2007 12:27:45 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) by mailgw12.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73ARjQB001926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 03 Aug 2007 12:27:45 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.13.7/8.13.7) with SMTP id l73ARi3c006691; Fri, 03 Aug 2007 12:27:44 +0200 (MEST) Received: from burner ([10.147.65.166]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Fri, 03 Aug 2007 12:27:44 +0200 Date: Fri, 03 Aug 2007 12:25:01 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B222A7.7090408@sun.com> To: john.johnson@sun.com, jek3@sun.com Cc: psarc-ext@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Joseph.Bonasera@sun.com, jg@jurassic.sfbay.sun.com, Jerry.Gilliam@sun.com, Jeff.Cheeney@sun.com, Jan.Setje-Eilers@sun.com Message-id: <46b3027d.Tfmu+CJcB5k/wHWP%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.2.0.264296 X-Fraunhofer-Email-Policy: accepted References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> <46b1d4cc.50FGxJdt5x0Yo8KQ%Joerg.Schilling@fokus.fraunhofer.de> <46B2202D.9090409@sun.com> <46B222A7.7090408@sun.com> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 03 Aug 2007 10:27:44.0841 (UTC) FILETIME=[EE152390:01C7D5B8] Status: RO Content-Length: 827 John Johnson wrote: > Joseph Kowalski wrote: > >> > > Speaking of SILO, I assume the Ubuntu UltraSPARC T1 port uses SILO? > > > > Yes, same as previous SPARC/Linux versions. The SPARC/Linux lead is not > a big grub fan, so the grub team would have to do the grub work to boot it. I would not try to rely on SILO as it does not work with mkisofs. I did try to help the authors of SILO to make it compliant 8 years ago to no avail. They have not been interested to change even 10-20 lines of code. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From Frank.Hofmann@sun.com Fri Aug 3 03:46:03 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73Ak3SQ012415 for ; Fri, 3 Aug 2007 03:46:03 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73Ahjmg028460 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Fri, 3 Aug 2007 11:43:47 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM700H0X1SXHW00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 03:43:45 -0700 (PDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.6]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM700AQX1SUBUB0@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 03:43:44 -0700 (PDT) Received: from d1-emea-10.sun.com ([192.18.2.120]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l73AhgMD015888 for ; Fri, 03 Aug 2007 10:43:42 +0000 (GMT) Received: from conversion-daemon.d1-emea-10.sun.com by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JM700H011SGMH00@d1-emea-10.sun.com> (original mail from Frank.Hofmann@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Fri, 03 Aug 2007 11:43:42 +0100 (BST) Received: from dhcp-elon07-138-84.UK.Sun.COM ([129.156.138.84]) by d1-emea-10.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JM7006121STT2XE@d1-emea-10.sun.com>; Fri, 03 Aug 2007 11:43:42 +0100 (BST) Date: Fri, 03 Aug 2007 11:43:06 +0100 (BST) From: Frank Hofmann Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46b301d1.B60X4zq+vhSNgupu%Joerg.Schilling@fokus.fraunhofer.de> Sender: Frank.Hofmann@sun.com X-X-Sender: frankho@dhcp-elon07-138-84.UK.Sun.COM To: Joerg Schilling Cc: setje@smack.eng.sun.com, Frank.Hofmann@sun.com, psarc-ext@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Joseph.Bonasera@sun.com, John.Johnson@sun.com, jg@jurassic.sfbay.sun.com, Jerry.Gilliam@sun.com, Jeff.Cheeney@sun.com, Jan.Setje-Eilers@sun.com Message-id: Organization: Sun Microsystems OP/N1 RPE MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_9q3LlfzaAqBMWjFsABCX4Q)" X-PMX-Version: 5.2.0.264296 References: <200708021812.l72ICojQ015404@smack.Eng.Sun.COM> <46b301d1.B60X4zq+vhSNgupu%Joerg.Schilling@fokus.fraunhofer.de> Status: RO Content-Length: 3560 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --Boundary_(ID_9q3LlfzaAqBMWjFsABCX4Q) Content-type: TEXT/PLAIN; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 8BIT On Fri, 3 Aug 2007, Joerg Schilling wrote: > Jan Setje-Eilers wrote: > >>>> and is implemented within the filesystem. In order to create >>>> compressed files a userland utility is used that simply compresses the >>>> file in place, the file is then marked as compressed via the >>>> _FIO_COMPRESSED (private) ioctl in the filesystem metadata. >>> >>> Do you like to implement the zisofs extension for Rock Ridge or does this >>> apply to ZFS only? >> >> Funny you should ask. We'd like to, but we're still waiting for legal >> review (let's please not discuss that here as neither you nor I can do >> anything to sway them one way or another). > > Well, I did propose to support this 1-2 years ago (not 100% sure abut the date). zisofs ? The proposal I've seen was logged a year ago by the Xen group, strangely enough, here: 6449443 the zisofs filesystem format should be supported Feel free :) FrankH. > > >> If we get the green light we'll do this, which will involve the >> corresponding change to mkisofs. > > Mkisofs by default includes support for "-z": > > -z, -transparent-compression > Enable transparent compression of files > > We only need to add the program "mkzftree" which is currently non-portable > (Linux only) available in source from "H. Peter Anvin". > > I did already talk with him about integrating the program into cdrtools a year > ago and he did give me the permission. For license details, please contact me > offline. > > Did you read the file README.compression from cdrtools? > > >>>> 5.4 Coexistence with other OSes >>>> >>>> The only other OS of note for SPARC is Linux, and they currently have no grub >>>> plans. SPARC / Linux boots from a loader called SILO, which already has a >>>> grub-like menu facility. >>> >>> I have no idea about SILO on harddisks, on CDs/DVDs, it is imcompatible to the >>> Sun boot. >> >> I'd expect it lives in the boot-block of the slice, so assuming one >> OS per slice it can co-exist as closely as it reasonably needs to. >> More than one OS per slice assumes they can use the same FS as well, >> which other than iso/hsfs is typically not something people would >> expect to be able to do. > > While the official Sparc boot (as expected by the boot prom) asumes that > there is a Sun disk label on the CD/DVD media and the boot for a specific > architecture is on a specific partition, SILO works on unlabelled media and > needs a hand crafted command line to boot from CD/DVD. SILO also needs to be > patched to know where it's second stage is located. > > For this reason, SILO is not userfriendly and it is not possible to auto-create > a SILO enabled ISO-9660 filesystem using mkisofs. > > If you create a new boot, please make it compatible to what mkisofs supports. > See also the description in the files: > > README.sparcboot > README.sunx86boot > README.eltorito > > in cdrtools. > > Jörg > > -- > EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin > js@cs.tu-berlin.de (uni) > schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ > URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily > --Boundary_(ID_9q3LlfzaAqBMWjFsABCX4Q)-- From Joerg.Schilling@fokus.fraunhofer.de Fri Aug 3 05:19:18 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73CJIIm014105 for ; Fri, 3 Aug 2007 05:19:18 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73CGv6s058835; Fri, 3 Aug 2007 06:16:57 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM70080764BG100@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 05:16:59 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM7004ZV64BML20@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 05:16:59 -0700 (PDT) Received: from relay43i.sun.com ([192.5.209.74]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l73CGwTr004827; Fri, 03 Aug 2007 12:16:58 +0000 (GMT) Received: from mms49es.sun.com ([160.41.221.233] [160.41.221.233]) by relay43i.sun.com with ESMTP id BT-MMP-270825; Fri, 03 Aug 2007 12:16:58 +0000 (Z) Received: from relay44i.sun.com ([192.5.209.118] [192.5.209.118]) by mms49es.sun.com with ESMTP id BT-MMP-528311; Fri, 03 Aug 2007 12:16:58 +0000 (Z) Received: from mailgw1.fraunhofer.de ([153.96.1.18] [153.96.1.18]) by relay4i.sun.com with ESMTP id BT-MMP-2317917; Fri, 03 Aug 2007 12:16:57 +0000 (Z) Received: from mailgw1.fraunhofer.de (localhost [127.0.0.1]) by mailgw1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73CGumf023285; Fri, 03 Aug 2007 14:16:56 +0200 (CEST) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) by mailgw1.fraunhofer.de (8.13.5+/8.13.4) with ESMTP id l73CGsss023173 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 03 Aug 2007 14:16:55 +0200 (CEST) Received: from EXCHSRV.fokus.fraunhofer.de (bohr [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.13.7/8.13.7) with SMTP id l73CGrmA011796; Fri, 03 Aug 2007 14:16:54 +0200 (MEST) Received: from burner ([10.147.65.166]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.3959); Fri, 03 Aug 2007 14:16:54 +0200 Date: Fri, 03 Aug 2007 14:14:10 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) Subject: Re: 2006/525: New Boot Sparc In-reply-to: To: Frank.Hofmann@Sun.COM Cc: setje@smack.eng.sun.com, psarc-ext@Sun.COM, Lori.Alt@Sun.COM, Lin.Ling@Sun.COM, Joseph.Bonasera@Sun.COM, John.Johnson@Sun.COM, jg@jurassic.sfbay.sun.com, Jerry.Gilliam@Sun.COM, Jeff.Cheeney@Sun.COM, Jan.Setje-Eilers@Sun.COM, Frank.Hofmann@Sun.COM Message-id: <46b31c12.LCm8NFkM0TNyS4uK%Joerg.Schilling@fokus.fraunhofer.de> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 8BIT X-PMX-Version: 5.2.0.264296 X-Fraunhofer-Email-Policy: accepted References: <200708021812.l72ICojQ015404@smack.Eng.Sun.COM> <46b301d1.B60X4zq+vhSNgupu%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: nail 11.22 3/20/05 X-OriginalArrivalTime: 03 Aug 2007 12:16:54.0056 (UTC) FILETIME=[2DB80A80:01C7D5C8] Status: RO Content-Length: 960 Frank Hofmann wrote: > >>> Do you like to implement the zisofs extension for Rock Ridge or does this > >>> apply to ZFS only? > >> > >> Funny you should ask. We'd like to, but we're still waiting for legal > >> review (let's please not discuss that here as neither you nor I can do > >> anything to sway them one way or another). > > > > Well, I did propose to support this 1-2 years ago (not 100% sure abut the date). > > zisofs ? > > The proposal I've seen was logged a year ago by the Xen group, strangely > enough, here: > > 6449443 the zisofs filesystem format should be supported It has been discussed some time ago with FritS Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From setje@smack.eng.sun.com Fri Aug 3 14:37:57 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73LbvvY026011 for ; Fri, 3 Aug 2007 14:37:57 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73LZekU029693; Fri, 3 Aug 2007 14:35:41 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM70050BVZHPW00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 14:35:41 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM700FXVVZFM260@nwk-avmta-1.sfbay.Sun.COM>; Fri, 03 Aug 2007 14:35:39 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l73LZb5o022211; Fri, 03 Aug 2007 14:35:37 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l73LPHGO018922; Fri, 03 Aug 2007 14:25:17 -0700 (PDT) Date: Fri, 03 Aug 2007 14:25:17 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Fri, 03 Aug 2007 10:44:23 BST." <46B2F8F7.2030206@Sun.COM> To: Darren J Moffat Cc: Jan Setje-Eilers , James Carlson , Bill Sommerfeld , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708032125.l73LPHGO018922@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2264 > > Update the archive. > > If you do that you are you possibly removing information that the admin > may need to diagnose why the system was in this state ? Also it will > make the service in main look strange - it said the boot archive was out > of date but and admin will find that it isn't. > > I'd personally leave this to one of the "later" normal cases that causes > an automatic update of the archive. The idea behind leaving that check in maintenance mode is that it communicates what state the machine is running in, so I think that's correct. Since the archive will most likely just get updated by system/ boot-archive-update, I don't see a reason to disable that. However I understand the desire for a copy of the files that the system is currently running with to be available so decisions can be made based on their state. > > And potentially reboot immediately to the device we > > just booted from now that the archive is updated. > > I really don't like the auto reboot it could cause application problems > and will almost certainly surprise admins. I don't think we should ever > autoreboot after getting this far up. It makes me nervous as well, hence "potentially". OTOH I don't want to entirely dismiss this approach without fully exploring its pros and cons. Not that it's available yet, but would you feel any differently if this was a kexec? I know I would. In fact I'd even consider attacking the general boot-archive case with a kexec of a tiny fail-safe that updated the archive and then re-exec'd the intended kernel. > > The auto-reboot still makes us a little nervous, so it may be > > something that needs to be explicitly enabled based on site > > policy, but at least on sparc we have a solid idea of what > > boot device we booted from, so it may turn out to a reasonable > > default action to take. > > and since we don't on x86 I don't think we should do this since one of > the stated goals of this project is removing differences! Also consider > that there will be other platforms eventually. It's something we will be able to do on efi based systems, and is really something I regard as a huge gaping bug in BIOS systems, so I'm not willing to dismiss based on that. -jan From sacadmin Fri Aug 3 16:35:40 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73NZeS5028736 for ; Fri, 3 Aug 2007 16:35:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73NXBlb051176; Fri, 3 Aug 2007 17:33:11 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800G011FDC000@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8004B11FDMOA0@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l73NXDnC026086; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JM800F01187JW00@fe-sfbay-09.sun.com> (original mail from Hitendra.Zhangada@Sun.COM); Fri, 03 Aug 2007 16:33:12 -0700 (PDT) Received: from [129.153.85.31] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JM8007S91FBM1D0@fe-sfbay-09.sun.com>; Fri, 03 Aug 2007 16:33:12 -0700 (PDT) Date: Fri, 03 Aug 2007 16:33:11 -0700 From: Hitendra Zhangada Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012350.l71Nofph699895@jurassic.eng.sun.com> Sender: Hitendra.Zhangada@Sun.COM To: Jerry Gilliam Cc: psarc-ext@Sun.COM, John.Johnson@Sun.COM, Jerry.Gilliam@Sun.COM, Lori.Alt@Sun.COM, Lin.Ling@Sun.COM, Jan.Setje-Eilers@Sun.COM, Joseph.Bonasera@Sun.COM, Jeff.Cheeney@Sun.COM, Firmware Arch Message-id: <46B3BB37.4070707@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.5pre (X11/20070721) Status: RO Content-Length: 1248 Jerry Gilliam wrote: > > I am sponsoring this project as a fast-track on behalf of Jan > Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. FYI. Added FWARC to the mailing list. One comment below. > > A new node (/packages/boot-properties) will be added to the OBP device tree by > the booter which contains these properties. Is there a reason to create new node under /packages instead of under /chosen node? /chosen is defined by IEEE 1275 standard to contain properties such as "bootpath", "bootargs" etc. It seems like /chosen node is a better place to add the new node. > > In addition to the above compatibility properties, a couple of new properties > are > needed: > > "bootarchive" boot archive path (as opposed to "bootpath", the root > file system path) > "elfheader-address" address of kernel ELF header, used by krtld in lieu of > the bootaux vector passed up via the second-stage booter > "elfheader-length" length of kernel ELF header > "archive-fstype" fstype of boot archive > > -- Hitendra Zhangada ==================================== SPS Common SW Features Engineering Systems Group, Sun Microsystems, Inc. Sun Ph# (858) 625 3757, Sun Ext. x53757 Internal homepage http://esp.west/~hitu From Hitendra.Zhangada@Sun.COM Fri Aug 3 16:35:40 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l73NZeBs028738 for ; Fri, 3 Aug 2007 16:35:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l73NXBlb051176; Fri, 3 Aug 2007 17:33:11 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800G011FDC000@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8004B11FDMOA0@nwk-avmta-2.sfbay.sun.com>; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l73NXDnC026086; Fri, 03 Aug 2007 16:33:13 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JM800F01187JW00@fe-sfbay-09.sun.com> (original mail from Hitendra.Zhangada@Sun.COM); Fri, 03 Aug 2007 16:33:12 -0700 (PDT) Received: from [129.153.85.31] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JM8007S91FBM1D0@fe-sfbay-09.sun.com>; Fri, 03 Aug 2007 16:33:12 -0700 (PDT) Date: Fri, 03 Aug 2007 16:33:11 -0700 From: Hitendra Zhangada Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708012350.l71Nofph699895@jurassic.eng.sun.com> Sender: Hitendra.Zhangada@Sun.COM To: Jerry Gilliam Cc: psarc-ext@Sun.COM, John.Johnson@Sun.COM, Jerry.Gilliam@Sun.COM, Lori.Alt@Sun.COM, Lin.Ling@Sun.COM, Jan.Setje-Eilers@Sun.COM, Joseph.Bonasera@Sun.COM, Jeff.Cheeney@Sun.COM, Firmware Arch Message-id: <46B3BB37.4070707@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> User-Agent: Thunderbird 2.0.0.5pre (X11/20070721) Status: RO Content-Length: 1248 Jerry Gilliam wrote: > > I am sponsoring this project as a fast-track on behalf of Jan > Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. FYI. Added FWARC to the mailing list. One comment below. > > A new node (/packages/boot-properties) will be added to the OBP device tree by > the booter which contains these properties. Is there a reason to create new node under /packages instead of under /chosen node? /chosen is defined by IEEE 1275 standard to contain properties such as "bootpath", "bootargs" etc. It seems like /chosen node is a better place to add the new node. > > In addition to the above compatibility properties, a couple of new properties > are > needed: > > "bootarchive" boot archive path (as opposed to "bootpath", the root > file system path) > "elfheader-address" address of kernel ELF header, used by krtld in lieu of > the bootaux vector passed up via the second-stage booter > "elfheader-length" length of kernel ELF header > "archive-fstype" fstype of boot archive > > -- Hitendra Zhangada ==================================== SPS Common SW Features Engineering Systems Group, Sun Microsystems, Inc. Sun Ph# (858) 625 3757, Sun Ext. x53757 Internal homepage http://esp.west/~hitu From sacadmin Fri Aug 3 17:12:45 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l740Cixt029517 for ; Fri, 3 Aug 2007 17:12:45 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l740AR7v020558; Sat, 4 Aug 2007 08:10:27 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800E0335EYT00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:10:26 -0600 (MDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8005DX35C5860@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:10:24 -0600 (MDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l740AJ80005199; Fri, 03 Aug 2007 17:10:19 -0700 (PDT) Received: from [10.7.251.244] (punchin-jgj [10.7.251.244]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l740AIab098981; Fri, 03 Aug 2007 17:10:19 -0700 (PDT) Date: Fri, 03 Aug 2007 17:02:57 -0700 From: John Johnson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B3BB37.4070707@sun.com> To: Hitendra Zhangada Cc: Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Jeff.Cheeney@sun.com, Firmware Arch Message-id: <46B3C231.1030808@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> <46B3BB37.4070707@sun.com> User-Agent: Thunderbird 2.0b1pre (X11/20061022) Status: RO Content-Length: 1032 Hitendra Zhangada wrote: > Jerry Gilliam wrote: >> >> I am sponsoring this project as a fast-track on behalf of Jan >> Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. > > FYI. Added FWARC to the mailing list. One comment below. > > >> >> A new node (/packages/boot-properties) will be added to the OBP device >> tree by >> the booter which contains these properties. > > Is there a reason to create new node under /packages instead of > under /chosen node? /chosen is defined by IEEE 1275 standard > to contain properties such as "bootpath", "bootargs" etc. > It seems like /chosen node is a better place to add the new node. > That was my first choice, but there doesn't seem to be a way for a client to add properties to /chosen. The 'setprop' client i/f only allows you to modify an existing property, and /chosen doesn't have an open method, so you can't do $open-package/property. Internally, obp does 'chosen-variable', but it's headerless, so escaping to forth doesn't work either. JJ From john.johnson@sun.com Fri Aug 3 17:12:47 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l740CkIi029521 for ; Fri, 3 Aug 2007 17:12:46 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l740AR7v020558; Sat, 4 Aug 2007 08:10:27 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JM800E0335EYT00@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:10:26 -0600 (MDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JM8005DX35C5860@brm-avmta-1.central.sun.com>; Fri, 03 Aug 2007 18:10:24 -0600 (MDT) Received: from dtmail.sfbay.sun.com (pkg.SFBay.Sun.COM [129.146.90.56]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l740AJ80005199; Fri, 03 Aug 2007 17:10:19 -0700 (PDT) Received: from [10.7.251.244] (punchin-jgj [10.7.251.244]) by dtmail.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l740AIab098981; Fri, 03 Aug 2007 17:10:19 -0700 (PDT) Date: Fri, 03 Aug 2007 17:02:57 -0700 From: John Johnson Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B3BB37.4070707@sun.com> To: Hitendra Zhangada Cc: Jerry Gilliam , psarc-ext@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Jeff.Cheeney@sun.com, Firmware Arch Message-id: <46B3C231.1030808@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708012350.l71Nofph699895@jurassic.eng.sun.com> <46B3BB37.4070707@sun.com> User-Agent: Thunderbird 2.0b1pre (X11/20061022) Status: RO Content-Length: 1032 Hitendra Zhangada wrote: > Jerry Gilliam wrote: >> >> I am sponsoring this project as a fast-track on behalf of Jan >> Setje-Eilers and John Johnson. The case timer is set to 8/7/2007. > > FYI. Added FWARC to the mailing list. One comment below. > > >> >> A new node (/packages/boot-properties) will be added to the OBP device >> tree by >> the booter which contains these properties. > > Is there a reason to create new node under /packages instead of > under /chosen node? /chosen is defined by IEEE 1275 standard > to contain properties such as "bootpath", "bootargs" etc. > It seems like /chosen node is a better place to add the new node. > That was my first choice, but there doesn't seem to be a way for a client to add properties to /chosen. The 'setprop' client i/f only allows you to modify an existing property, and /chosen doesn't have an open method, so you can't do $open-package/property. Internally, obp does 'chosen-variable', but it's headerless, so escaping to forth doesn't work either. JJ From sommerfeld@SUN.COM Mon Aug 6 11:31:41 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l76IVdhi018506 for ; Mon, 6 Aug 2007 11:31:40 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l76ITErT005051; Mon, 6 Aug 2007 19:29:19 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMD00K077CT3F00@nwk-avmta-2.sfbay.sun.com>; Mon, 06 Aug 2007 11:29:18 -0700 (PDT) Received: from eastmail2bur.East.Sun.COM ([129.148.13.40]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMD00HFO7CTO340@nwk-avmta-2.sfbay.sun.com>; Mon, 06 Aug 2007 11:29:17 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l76IT1oc003122; Mon, 06 Aug 2007 14:29:01 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l76IT0vF016611; Mon, 06 Aug 2007 14:29:00 -0400 (EDT) Date: Mon, 06 Aug 2007 14:28:59 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: James Carlson , John.Johnson@SUN.COM, Jerry.Gilliam@SUN.COM, Lori.Alt@SUN.COM, Jan.Setje-Eilers@SUN.COM, psarc-ext@SUN.COM, Jerry Gilliam , Joseph.Bonasera@SUN.COM, Lin.Ling@SUN.COM, Jeff.Cheeney@SUN.COM Message-id: <1186424939.14296.40.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708022258.l72Mw0Zf003023@smack.Eng.Sun.COM> Status: RO Content-Length: 3198 On Thu, 2007-08-02 at 15:58 -0700, Jan Setje-Eilers wrote: > It sounds like it would be acceptable for whatever tool is used to > update these files to do whatever sync is needed atomically as part of > the update. Sadly for most of these files this tool is still a text > editor and we don't get to fix that in a patch release. :( I don't think it's necessary to use this case to fix *this* problem -- solaris administrators have had to deal with this all along, and you're not going to make things worse if solaris continues to read these manually-edited files individually at boot time. > However it seems like the files of concern fall into a category of > their own that is currently being dealt with in a manner that should > actually be reserved for out of sync kernel binaries. This suggest we > have the following three types of files in the archive: > > 1) Kernel binaries (including modules and drivers) ... > 2) Files that are either caches or only grow or can be safely re-read > later. .... > 3) Files that etc/system that contain information that can't always > be usefully processed later during boot, but if out of date _do > not_ leave the system in a dangerously unstable state. > Ideas, thoughts, comments? First, I don't believe we have proof that editable/volatile files present in the boot archive fall only into categories (2) and (3), nor sufficient controls to prevent files outside (2) & (3) from being added. I believe it's unreasonable to have files of type (3) in the boot archive used when a solaris system boots from its own root file system. I believe that a change to force the administrator to run "bootadm update-archive" after editing files of type (3) is a change not compatible with the Patch binding requested for this case. While it is true that this exact change was inflicted on solaris x86 in solaris 10 update 1, there were mitigating business reasons why this appeared to be the least bad option open to us at the time. These reasons do not appear to be present for sparc. The main architectural benefit of newboot is that it reads in all files needed to mount the root filesystem before the kernel is started. This benefit would still be present if these files weren't packed into a single boot archive on disk but were read individually from the root filesystem. I believe most if not all of the operational problems with the boot archive would go away if, in a default install of solaris booting from its own root filesystem, a list of critical configuration files of type (2) and (3) were read individually from the root filesystem into memory to supplement the boot archive before the kernel starts. (I specifically do not want a system where the kernel reads them in on demand). With 20:20 hindsight, I can now say that the operational instability inherent in putting volatile/editable files in the boot archive should never have been inflicted on administrators of solaris on x86. I will not stand by and see the instability inherent in the current monolithic boot archive system inflicted on sparc systems as well. This must change or else I will derail so I can vote against this proposal. - Bill From setje@smack.eng.sun.com Mon Aug 6 19:09:56 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7729uEb029980 for ; Mon, 6 Aug 2007 19:09:56 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7727bFS014909; Mon, 6 Aug 2007 19:07:37 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMD00K0TSKMAC00@brm-avmta-1.central.sun.com>; Mon, 06 Aug 2007 20:07:34 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMD00GIMSKMU220@brm-avmta-1.central.sun.com>; Mon, 06 Aug 2007 20:07:34 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7727WSO010048; Mon, 06 Aug 2007 19:07:32 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l771v9Vl028485; Mon, 06 Aug 2007 18:57:09 -0700 (PDT) Date: Mon, 06 Aug 2007 18:57:09 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Mon, 06 Aug 2007 14:28:59 EDT." <1186424939.14296.40.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 4781 For the purposes of this project (new boot SPARC), we can avoid this problem by not placing editable files into the boot_archive. kobj_open() would be modified to read read any files it fails to find in the archive (if they exist at all) from the filesystem via the firmware. This has the up-side that it keeps all these files from invalidating the archive. The down-side is that it's one more place the kernel will continue to call into OBP. It's also a SPARC only solution. It should however fully address these concerns for the sake of this project on SPARC. Beyond this project we can investigate more drastic approaches that eliminate this issue on all architectures. We could have the boot-loader (a modified and extended GRUB on x86 and the bootblock or more likely a standalone on SPARC) read all the files that would ordinarily make up the archive into memory at the time of boot, and then boot that data-structure (which basically contains everything the archive contains today). This not only avoids the out of sync issues, but also saves the work currently being done to commit the archive to the filesystem when it is updated. This approach has potential performance implications (many seeks as opposed to one big read) that need to be investigated, but could be addressed with a hybrid approach. The hybrid places only the static files into a maintained, on-disk archive loading all editable files dynamically. -jan > On Thu, 2007-08-02 at 15:58 -0700, Jan Setje-Eilers wrote: > > It sounds like it would be acceptable for whatever tool is used to > > update these files to do whatever sync is needed atomically as part of > > the update. Sadly for most of these files this tool is still a text > > editor and we don't get to fix that in a patch release. :( > > I don't think it's necessary to use this case to fix *this* problem -- > solaris administrators have had to deal with this all along, and you're > not going to make things worse if solaris continues to read these > manually-edited files individually at boot time. > > > However it seems like the files of concern fall into a category of > > their own that is currently being dealt with in a manner that should > > actually be reserved for out of sync kernel binaries. This suggest we > > have the following three types of files in the archive: > > > > 1) Kernel binaries (including modules and drivers) > ... > > 2) Files that are either caches or only grow or can be safely re-read > > later. > .... > > 3) Files that etc/system that contain information that can't always > > be usefully processed later during boot, but if out of date _do > > not_ leave the system in a dangerously unstable state. > > > Ideas, thoughts, comments? > > First, I don't believe we have proof that editable/volatile files > present in the boot archive fall only into categories (2) and (3), nor > sufficient controls to prevent files outside (2) & (3) from being added. > > I believe it's unreasonable to have files of type (3) in the boot > archive used when a solaris system boots from its own root file system. > > I believe that a change to force the administrator to run "bootadm > update-archive" after editing files of type (3) is a change not > compatible with the Patch binding requested for this case. While it is > true that this exact change was inflicted on solaris x86 in solaris 10 > update 1, there were mitigating business reasons why this appeared to be > the least bad option open to us at the time. These reasons do not > appear to be present for sparc. > > The main architectural benefit of newboot is that it reads in all files > needed to mount the root filesystem before the kernel is started. This > benefit would still be present if these files weren't packed into a > single boot archive on disk but were read individually from the root > filesystem. > > I believe most if not all of the operational problems with the boot > archive would go away if, in a default install of solaris booting from > its own root filesystem, a list of critical configuration files of type > (2) and (3) were read individually from the root filesystem into memory > to supplement the boot archive before the kernel starts. (I > specifically do not want a system where the kernel reads them in on > demand). > > With 20:20 hindsight, I can now say that the operational instability > inherent in putting volatile/editable files in the boot archive should > never have been inflicted on administrators of solaris on x86. I will > not stand by and see the instability inherent in the current monolithic > boot archive system inflicted on sparc systems as well. > > This must change or else I will derail so I can vote against this > proposal. > > - Bill > > > > > > From sommerfeld@sun.com Mon Aug 6 19:59:26 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l772xPAM001782 for ; Mon, 6 Aug 2007 19:59:25 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l772v1pJ026109; Tue, 7 Aug 2007 10:57:05 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMD00M0RUV24800@nwk-avmta-2.sfbay.sun.com>; Mon, 06 Aug 2007 19:57:02 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMD006BYUV1OQF0@nwk-avmta-2.sfbay.sun.com>; Mon, 06 Aug 2007 19:57:01 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l772uoPi031089; Mon, 06 Aug 2007 22:56:50 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l772uoFP019065; Mon, 06 Aug 2007 22:56:50 -0400 (EDT) Date: Mon, 06 Aug 2007 22:56:49 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186455409.18708.19.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> Status: RO Content-Length: 1460 On Mon, 2007-08-06 at 18:57 -0700, Jan Setje-Eilers wrote: > For the purposes of this project (new boot SPARC), we can avoid this > problem by not placing editable files into the boot_archive. > kobj_open() would be modified to read read any files it fails to find > in the archive (if they exist at all) from the filesystem via the > firmware. This has the up-side that it keeps all these files from > invalidating the archive. The down-side is that it's one more place > the kernel will continue to call into OBP. It's also a SPARC only > solution. It should however fully address these concerns for the sake > of this project on SPARC. I think this addresses my fears in the short term, though we do need to see a revised spec which nails down specifics of how files may be excluded from the archive. > Beyond this project we can investigate more drastic approaches that > eliminate this issue on all architectures. We could have the > boot-loader (a modified and extended GRUB on x86 and the bootblock or > more likely a standalone on SPARC) read all the files that would > ordinarily make up the archive into memory at the time of boot, and > then boot that data-structure (which basically contains everything the > archive contains today). ... This also sounds like the right answer for the long run; if it would help make it happen we could derail and issue an opinion with advice that this approach should be investigated. - Bill From setje@smack.eng.sun.com Mon Aug 6 20:56:23 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l773uMxg002483 for ; Mon, 6 Aug 2007 20:56:22 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l773s3sR005873; Mon, 6 Aug 2007 20:54:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMD00N05XI30P00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 20:54:03 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMD00D2QXI351A0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 06 Aug 2007 20:54:03 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l773s13g021178; Mon, 06 Aug 2007 20:54:01 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l773ZqEN000963; Mon, 06 Aug 2007 20:35:52 -0700 (PDT) Date: Mon, 06 Aug 2007 20:35:52 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Mon, 06 Aug 2007 22:56:49 EDT." <1186455409.18708.19.camel@thunk> To: Bill Sommerfeld Cc: Jan Setje-Eilers , James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708070335.l773ZqEN000963@smack.Eng.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1913 <1186455409.18708.19.camel@thunk>Bill Sommerfeld writes: > On Mon, 2007-08-06 at 18:57 -0700, Jan Setje-Eilers wrote: > > For the purposes of this project (new boot SPARC), we can avoid this > > problem by not placing editable files into the boot_archive. > > kobj_open() would be modified to read read any files it fails to find > > in the archive (if they exist at all) from the filesystem via the > > firmware. This has the up-side that it keeps all these files from > > invalidating the archive. The down-side is that it's one more place > > the kernel will continue to call into OBP. It's also a SPARC only > > solution. It should however fully address these concerns for the sake > > of this project on SPARC. > > I think this addresses my fears in the short term, though we do need to > see a revised spec which nails down specifics of how files may be > excluded from the archive. I suspect it should be all editable files as well as all driver.conf files (editable or not) in the archive. I'll gather some data on this and will update the spec accordingly. > > Beyond this project we can investigate more drastic approaches that > > eliminate this issue on all architectures. We could have the > > boot-loader (a modified and extended GRUB on x86 and the bootblock or > > more likely a standalone on SPARC) read all the files that would > > ordinarily make up the archive into memory at the time of boot, and > > then boot that data-structure (which basically contains everything the > > archive contains today). > > ... > > This also sounds like the right answer for the long run; if it would > help make it happen we could derail and issue an opinion with advice > that this approach should be investigated. > Since this project is required for zfs boot I'd rather not slip out any more than we have to, so long as we have a reasonable interim solution, which, it seems, we now have. -jan From dmk@noho.sfbay.sun.com Tue Aug 7 05:05:18 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l77C5H1Z009699 for ; Tue, 7 Aug 2007 05:05:18 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l77C2mHP024248; Tue, 7 Aug 2007 20:02:49 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JME00D0DK4O8000@brm-avmta-1.central.sun.com>; Tue, 07 Aug 2007 06:02:48 -0600 (MDT) Received: from noho.sfbay.sun.com ([10.6.92.101]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JME000DVK4MRZE0@brm-avmta-1.central.sun.com>; Tue, 07 Aug 2007 06:02:47 -0600 (MDT) Received: from noho.sfbay.sun.com (localhost [127.0.0.1]) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l77C2CNO008669; Tue, 07 Aug 2007 05:02:12 -0700 (PDT) Received: (from dmk@localhost) by noho.sfbay.sun.com (8.13.7+Sun/8.13.7/Submit) id l77C2BIe008668; Tue, 07 Aug 2007 05:02:11 -0700 (PDT) Date: Tue, 07 Aug 2007 05:02:11 -0700 (PDT) From: David Kahn Subject: Re: 2006/525: New Boot Sparc To: setje@smack.eng.sun.com, sommerfeld@sun.com Cc: Jan.Setje-Eilers@sun.com, Jeff.Cheeney@sun.com, Jerry.Gilliam@sun.com, John.Johnson@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Lori.Alt@sun.com, james.d.carlson@sun.com, jg@jurassic.sfbay.sun.com, psarc-ext@sun.com Message-id: <200708071202.l77C2BIe008668@noho.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 341 > Since this project is required for zfs boot I'd rather not slip out >any more than we have to, so long as we have a reasonable interim >solution, which, it seems, we now have. If you aren't going to derail this case, at least extend the timer so we have a chance to evaluate the updated materials. Have they been submitted yet? -David From jg@jurassic.sfbay.sun.com Tue Aug 7 13:40:54 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l77KerfI024006 for ; Tue, 7 Aug 2007 13:40:53 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l77KcFeG018514; Wed, 8 Aug 2007 04:38:27 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMF00H07802YC00@nwk-avmta-1.sfbay.Sun.COM>; Tue, 07 Aug 2007 13:38:26 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.226.130]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMF003DL802QEC0@nwk-avmta-1.sfbay.Sun.COM>; Tue, 07 Aug 2007 13:38:26 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l77KcJln958241; Tue, 07 Aug 2007 13:38:19 -0700 (PDT) Date: Tue, 07 Aug 2007 13:37:07 -0700 (PDT) From: Jerry Gilliam Subject: Re: 2006/525: New Boot Sparc To: dmk@noho.sfbay.sun.com, psarc-ext@sun.com Cc: setje@smack.eng.sun.com, Jeff.Cheeney@sun.com, Jerry.Gilliam@sun.com, John.Johnson@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Lori.Alt@sun.com Reply-to: Jerry Gilliam Message-id: <200708072038.l77KcJln958241@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: M8+F3BfaW/pDT4VkE44xDg== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 137 > >Have they been submitted yet? > >-David No, the team will provide one soon. I'm placing the case on hold in the meantime. -jg From scott.rotondo@sun.com Tue Aug 7 14:12:25 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l77LCO4V024882 for ; Tue, 7 Aug 2007 14:12:24 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l77LA2EO001382; Wed, 8 Aug 2007 05:10:02 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMF00K019GPUL00@brm-avmta-1.central.sun.com>; Tue, 07 Aug 2007 15:10:01 -0600 (MDT) Received: from sfbaymail2sca.sfbay.sun.com ([129.145.155.42]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMF00KJY9GN3V00@brm-avmta-1.central.sun.com>; Tue, 07 Aug 2007 15:10:00 -0600 (MDT) Received: from domus.sfbay.sun.com (domus.SFBay.Sun.COM [10.6.64.11]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l77L9wAM005609; Tue, 07 Aug 2007 14:09:58 -0700 (PDT) Received: from [129.146.108.62] (vinifera.SFBay.Sun.COM [129.146.108.62]) by domus.sfbay.sun.com (Trusted Solaris (8.11.7)/8.11.6) with ESMTP id l77L9vE25572; Tue, 07 Aug 2007 14:09:57 -0700 (PDT) Date: Tue, 07 Aug 2007 14:09:56 -0700 From: Scott Rotondo Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Bill Sommerfeld , James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B8DFA4.6000602@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> User-Agent: Thunderbird 2.0b2 (X11/20070326) Status: RO Content-Length: 1855 Jan Setje-Eilers wrote: > For the purposes of this project (new boot SPARC), we can avoid this > problem by not placing editable files into the boot_archive. > kobj_open() would be modified to read read any files it fails to find > in the archive (if they exist at all) from the filesystem via the > firmware. This has the up-side that it keeps all these files from > invalidating the archive. The down-side is that it's one more place > the kernel will continue to call into OBP. It's also a SPARC only > solution. It should however fully address these concerns for the sake > of this project on SPARC. > > Beyond this project we can investigate more drastic approaches that > eliminate this issue on all architectures. We could have the > boot-loader (a modified and extended GRUB on x86 and the bootblock or > more likely a standalone on SPARC) read all the files that would > ordinarily make up the archive into memory at the time of boot, and > then boot that data-structure (which basically contains everything the > archive contains today). This not only avoids the out of sync issues, > but also saves the work currently being done to commit the archive to > the filesystem when it is updated. This approach has potential > performance implications (many seeks as opposed to one big read) that > need to be investigated, but could be addressed with a hybrid > approach. The hybrid places only the static files into a maintained, > on-disk archive loading all editable files dynamically. > The idea for a long-term solution (having the boot loader read multiple files into memory instead of a single archive) sounds like a good one. I'm a little concerned that implementing the short-term solution (having the kernel read individual files via bootops) only for Sparc will reduce the urgency of implementing the long-term solution. Scott From Darren.Moffat@sun.com Wed Aug 8 02:07:56 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7897tTt010403 for ; Wed, 8 Aug 2007 02:07:56 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7895Sui019549 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 8 Aug 2007 17:05:32 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00I036L6AE00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 02:05:30 -0700 (PDT) Received: from gmp-ea-fw-1.sun.com ([129.156.42.5]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00GD06L3L430@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 02:05:29 -0700 (PDT) Received: from d1-emea-09.sun.com (d1-emea-09.sun.com [192.18.2.119]) by gmp-ea-fw-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l7895RA0018417 for ; Wed, 08 Aug 2007 09:05:27 +0000 (GMT) Received: from conversion-daemon.d1-emea-09.sun.com by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMG00K016GEFZ00@d1-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 10:05:27 +0100 (BST) Received: from [129.156.173.21] by d1-emea-09.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMG0028X6L0JQ30@d1-emea-09.sun.com>; Wed, 08 Aug 2007 10:05:26 +0100 (BST) Date: Wed, 08 Aug 2007 10:05:24 +0100 From: Darren J Moffat Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B8DFA4.6000602@sun.com> Sender: Darren.Moffat@sun.com To: Scott Rotondo Cc: Jan Setje-Eilers , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jerry Gilliam , James Carlson , psarc-ext@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B98754.8000201@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> <46B8DFA4.6000602@sun.com> User-Agent: Thunderbird 2.0.0.0 (X11/20070605) Status: RO Content-Length: 1280 Scott Rotondo wrote: > The idea for a long-term solution (having the boot loader read multiple > files into memory instead of a single archive) sounds like a good one. I'm not sure that is the best solution in the network boot case. Consider that one of the reasons we designed wanboot with a downloadable archive was to *reduce* the number of files transferred over the network and to reduce the number of files we had to verify with an hmac. This is important not just for wanboot (which should get ported to x86) and also the netboot for NFS and iSCSI root cases as well. The root cause here seems to be that there are multiple copies of a file, one in the root filesystem and one in the archive filesystem. It isn't that the boot system loads an archive. If instead we could fix the problem so that there is never the possibility of the root filesystem being out of sync with the archive that would help. There are possible ways to do this - I shared some of my thoughts privately with Jan yesterday and he had already looked a similar way of solving it but it is too complex for now. I just don't want the outcome of this fast-track to imply that the future *better* solution requires the boot loader to be reading multiple files. -- Darren J Moffat From sommerfeld@sun.com Wed Aug 8 07:24:17 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78EOG8J014636 for ; Wed, 8 Aug 2007 07:24:16 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l78ELmTI012395; Wed, 8 Aug 2007 22:21:49 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00E27L8BW600@brm-avmta-1.central.sun.com>; Wed, 08 Aug 2007 08:21:47 -0600 (MDT) Received: from localhost.east.sun.com ([129.148.19.3]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00EVRL848B00@brm-avmta-1.central.sun.com>; Wed, 08 Aug 2007 08:21:45 -0600 (MDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l78ELLGk004683; Wed, 08 Aug 2007 10:21:21 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.1+Sun/8.14.1/Submit) id l78ELJnt004682; Wed, 08 Aug 2007 10:21:19 -0400 (EDT) Date: Wed, 08 Aug 2007 10:21:19 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708070335.l773ZqEN000963@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <1186582879.3838.47.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070335.l773ZqEN000963@smack.Eng.Sun.COM> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 904 On Mon, 2007-08-06 at 20:35 -0700, Jan Setje-Eilers wrote: > > I think this addresses my fears in the short term, though we do need to > > see a revised spec which nails down specifics of how files may be > > excluded from the archive. > > I suspect it should be all editable files as well as all driver.conf > files (editable or not) in the archive. I'll gather some data on this > and will update the spec accordingly. You mentioned in a side conversation that you had heard that there was a good reason why we packaged many driver.conf files as non-editable but still occasionally told people to edit them anyway. If at all possible it would be very helpful to chase down the origin of that folklore, as what I'm hearing about likely changes coming in the packaging/patching/upgrade space suggests that the ground underneath that folklore is about to shift fairly dramatically. - Bill From scott.rotondo@sun.com Wed Aug 8 09:52:01 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78Gq12P017977 for ; Wed, 8 Aug 2007 09:52:01 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l78GnfPT005133 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 8 Aug 2007 09:49:41 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00M2ZS2SQQ00@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 10:49:40 -0600 (MDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00ERUS2R8390@brm-avmta-1.central.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 10:49:39 -0600 (MDT) Received: from domus.sfbay.sun.com (domus.SFBay.Sun.COM [10.6.64.11]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l78Gnanw029105; Wed, 08 Aug 2007 09:49:36 -0700 (PDT) Received: from [129.146.108.62] (vinifera.SFBay.Sun.COM [129.146.108.62]) by domus.sfbay.sun.com (Trusted Solaris (8.11.7)/8.11.6) with ESMTP id l78GnZE03999; Wed, 08 Aug 2007 09:49:35 -0700 (PDT) Date: Wed, 08 Aug 2007 09:49:35 -0700 From: Scott Rotondo Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B98754.8000201@Sun.COM> To: Darren J Moffat Cc: Jan Setje-Eilers , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jerry Gilliam , James Carlson , psarc-ext@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46B9F41F.8090006@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> <46B8DFA4.6000602@sun.com> <46B98754.8000201@Sun.COM> User-Agent: Thunderbird 2.0b2 (X11/20070326) Status: RO Content-Length: 1825 Darren J Moffat wrote: > Scott Rotondo wrote: >> The idea for a long-term solution (having the boot loader read >> multiple files into memory instead of a single archive) sounds like a >> good one. > > I'm not sure that is the best solution in the network boot case. > Consider that one of the reasons we designed wanboot with a downloadable > archive was to *reduce* the number of files transferred over the network > and to reduce the number of files we had to verify with an hmac. This > is important not just for wanboot (which should get ported to x86) and > also the netboot for NFS and iSCSI root cases as well. > > The root cause here seems to be that there are multiple copies of a > file, one in the root filesystem and one in the archive filesystem. It > isn't that the boot system loads an archive. If instead we could fix > the problem so that there is never the possibility of the root > filesystem being out of sync with the archive that would help. There > are possible ways to do this - I shared some of my thoughts privately > with Jan yesterday and he had already looked a similar way of solving it > but it is too complex for now. > > I just don't want the outcome of this fast-track to imply that the > future *better* solution requires the boot loader to be reading multiple > files. > I agree that nothing in this case should determine a required architecture or design for a future better solution. I also had a side conversation with Jan yesterday about some other alternatives. My point was simply to express the concern that implementing the partial, sparc-only solution might reduce the incentive to pursue a better solution in the near future. Though not committing to a specific solution, Jan did assure me that he has plans for future work in this area. Scott From Nicolas.Williams@Sun.COM Wed Aug 8 10:11:37 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78HBaLx019681 for ; Wed, 8 Aug 2007 10:11:37 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l78H8vkE009932; Thu, 9 Aug 2007 01:09:07 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00J05SZ51Y00@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:09:05 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00G1ESZ4KL50@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:09:04 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l78H92ZV007557; Wed, 08 Aug 2007 12:09:02 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l78H9246007556; Wed, 08 Aug 2007 12:09:02 -0500 (CDT) Date: Wed, 08 Aug 2007 12:09:01 -0500 From: Nicolas Williams Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B98754.8000201@Sun.COM> To: Darren J Moffat Cc: Scott Rotondo , Jerry.Gilliam@Sun.COM, John.Johnson@Sun.COM, Lori.Alt@Sun.COM, Jan.Setje-Eilers@Sun.COM, James Carlson , psarc-ext@Sun.COM, Jerry Gilliam , Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM, Jan Setje-Eilers Message-id: <20070808170901.GG1490@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> <46B8DFA4.6000602@sun.com> <46B98754.8000201@Sun.COM> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 2943 On Wed, Aug 08, 2007 at 10:05:24AM +0100, Darren J Moffat wrote: > Scott Rotondo wrote: > > The idea for a long-term solution (having the boot loader read multiple > > files into memory instead of a single archive) sounds like a good one. > > I'm not sure that is the best solution in the network boot case. > Consider that one of the reasons we designed wanboot with a downloadable > archive was to *reduce* the number of files transferred over the network > and to reduce the number of files we had to verify with an hmac. This > is important not just for wanboot (which should get ported to x86) and > also the netboot for NFS and iSCSI root cases as well. In the boot-from-local-media case the boot loader knows how to read from that media, and teaching it about the filesystem on that media isn't all that difficult (compared to writing realmode device drivers, I imagine). In the network boot case the stack is much larger, particularly when IPsec or RPCSEC_GSS are thrown into the picture. Expecting the boot loader to be able to read files from the real root in this case seems like asking a lot. Now, the good news is that in the network boot case we can rely on the archive being up to date: the boot server can see to it. Though it means teaching the boot server how to read from the client's real root FS; if the boot server is Linux and the client's root is on ZFS then this might be a problem (FUSE to the rescue!). So I think we might just want to do something like: - if / is on local media then the archive contains no editable files, with the kernel accessing them through the boot loader until / is mounted - if / is remote then the server SHOULD/MUST update the boot archive and the client SHALL depend on it, and the archive will contain editable files IOW, from the kernel's p.o.v. it just looks first in the archive, then if ENOENT it asks the boot loader. From the boot loader's p.o.v. the boot loader must know how to access root filesystems on local media, but not how to access remote filesystems. This is Jan's proposal, as I read it, with the extra req that the boot server keep the boot archive updated. > The root cause here seems to be that there are multiple copies of a > file, one in the root filesystem and one in the archive filesystem. It > isn't that the boot system loads an archive. If instead we could fix > the problem so that there is never the possibility of the root > filesystem being out of sync with the archive that would help. There > are possible ways to do this - I shared some of my thoughts privately > with Jan yesterday and he had already looked a similar way of solving it > but it is too complex for now. The above (Jan's) proposal does just that. > I just don't want the outcome of this fast-track to imply that the > future *better* solution requires the boot loader to be reading multiple > files. I don't think it does. Nico -- From sommerfeld@Sun.COM Wed Aug 8 10:24:14 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78HODpT020524 for ; Wed, 8 Aug 2007 10:24:13 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l78HLgFD018533; Wed, 8 Aug 2007 18:21:52 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00107TKEBJ00@brm-avmta-1.central.sun.com>; Wed, 08 Aug 2007 11:21:50 -0600 (MDT) Received: from eastmail2bur.East.Sun.COM ([129.148.13.40]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00EAPTKD8EC0@brm-avmta-1.central.sun.com>; Wed, 08 Aug 2007 11:21:50 -0600 (MDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l78HLWXr013432; Wed, 08 Aug 2007 13:21:32 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l78HLWgc011575; Wed, 08 Aug 2007 13:21:32 -0400 (EDT) Date: Wed, 08 Aug 2007 13:21:31 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <46B9F41F.8090006@sun.com> To: Scott Rotondo Cc: Darren J Moffat , Jerry.Gilliam@Sun.COM, John.Johnson@Sun.COM, Lori.Alt@Sun.COM, Jan.Setje-Eilers@Sun.COM, James Carlson , psarc-ext@Sun.COM, Jerry Gilliam , Joseph.Bonasera@Sun.COM, Lin.Ling@Sun.COM, Jeff.Cheeney@Sun.COM, Jan Setje-Eilers Message-id: <1186593691.25465.22.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> <46B8DFA4.6000602@sun.com> <46B98754.8000201@Sun.COM> <46B9F41F.8090006@sun.com> Status: RO Content-Length: 491 On Wed, 2007-08-08 at 09:49 -0700, Scott Rotondo wrote: > My point was simply to express the concern that implementing the > partial, sparc-only solution might reduce the incentive to pursue a > better solution in the near future. I disagree. By clearly establishing precedent that it's okay to boot from something other than a monolithic archive when you're booting from your own root file system, this encourages experimentation about the right way to do this on x86. - Bill From scott.rotondo@sun.com Wed Aug 8 10:39:43 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78Hdgbu021006 for ; Wed, 8 Aug 2007 10:39:43 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l78HbBOw026342; Thu, 9 Aug 2007 01:37:21 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00K01UA7E000@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:37:19 -0700 (PDT) Received: from sfbaymail1sca.SFBay.Sun.COM ([129.145.154.35]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00G35UA7KN60@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:37:19 -0700 (PDT) Received: from domus.sfbay.sun.com (domus.SFBay.Sun.COM [10.6.64.11]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l78HbGIF027692; Wed, 08 Aug 2007 10:37:16 -0700 (PDT) Received: from [129.146.108.62] (vinifera.SFBay.Sun.COM [129.146.108.62]) by domus.sfbay.sun.com (Trusted Solaris (8.11.7)/8.11.6) with ESMTP id l78HbGE04469; Wed, 08 Aug 2007 10:37:16 -0700 (PDT) Date: Wed, 08 Aug 2007 10:37:16 -0700 From: Scott Rotondo Subject: Re: 2006/525: New Boot Sparc In-reply-to: <1186593691.25465.22.camel@thunk> To: Bill Sommerfeld Cc: Darren J Moffat , Jerry.Gilliam@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, James Carlson , psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, Jan Setje-Eilers Message-id: <46B9FF4C.4020003@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070157.l771v9Vl028485@smack.Eng.Sun.COM> <46B8DFA4.6000602@sun.com> <46B98754.8000201@Sun.COM> <46B9F41F.8090006@sun.com> <1186593691.25465.22.camel@thunk> User-Agent: Thunderbird 2.0b2 (X11/20070326) Status: RO Content-Length: 736 Bill Sommerfeld wrote: > On Wed, 2007-08-08 at 09:49 -0700, Scott Rotondo wrote: >> My point was simply to express the concern that implementing the >> partial, sparc-only solution might reduce the incentive to pursue a >> better solution in the near future. > > I disagree. By clearly establishing precedent that it's okay to boot > from something other than a monolithic archive when you're booting from > your own root file system, this encourages experimentation about the > right way to do this on x86. > Agreed. It's also possible that the partial solution could help drive the better solution. In any case, "concern" != "certainty" and "might" != "will". As far as I'm concerned, my concern has been satisfied. Scott From setje@smack.eng.sun.com Wed Aug 8 10:58:37 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78HwaWv021800 for ; Wed, 8 Aug 2007 10:58:37 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l78Hu45s006830; Thu, 9 Aug 2007 01:56:16 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00L0BV5O8B00@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:56:12 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00GXTV5NKL70@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 10:56:11 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l78HuAqF026552; Wed, 08 Aug 2007 10:56:10 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l78HjjAr015232; Wed, 08 Aug 2007 10:45:45 -0700 (PDT) Date: Wed, 08 Aug 2007 10:45:45 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 08 Aug 2007 10:21:19 EDT." <1186582879.3838.47.camel@localhost> To: Bill Sommerfeld Cc: Jan Setje-Eilers , James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708081745.l78HjjAr015232@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1702 > On Mon, 2007-08-06 at 20:35 -0700, Jan Setje-Eilers wrote: > > > I think this addresses my fears in the short term, though we do need to > > > see a revised spec which nails down specifics of how files may be > > > excluded from the archive. > > > > I suspect it should be all editable files as well as all driver.conf > > files (editable or not) in the archive. I'll gather some data on this > > and will update the spec accordingly. > > You mentioned in a side conversation that you had heard that there was a > good reason why we packaged many driver.conf files as non-editable but > still occasionally told people to edit them anyway. > > If at all possible it would be very helpful to chase down the origin of > that folklore, as what I'm hearing about likely changes coming in the > packaging/patching/upgrade space suggests that the ground underneath > that folklore is about to shift fairly dramatically. I'm inclined to question that it was good reason, but as far as I can tell the hope is that as few .conf files as possible are actually used for customization. So they are by default of type f and are treated as though nothing in them needs to be preserved. This is obviously dangerous and I know of at least one recent example where we promoted a .conf file to e via a(n internal) bug. I've also been told that it's a similarly dubious upgrade performance optimization to upgrade as ON packages not calling out all their cross dependencies; something which is causing problems and is being addressed. That said, since we'd really like to eliminate driver.conf files in favor of moving all their data into smf, I'm inclined not to promote them further at this time. -jan From jek3@sun.com Wed Aug 8 11:03:30 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78I3TJV022038 for ; Wed, 8 Aug 2007 11:03:30 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l78I11Ai040156; Wed, 8 Aug 2007 12:01:01 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00L59VDVEE00@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 11:01:07 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.56.144]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00G2ZVDTKO70@nwk-avmta-2.sfbay.sun.com>; Wed, 08 Aug 2007 11:01:05 -0700 (PDT) Received: from [129.150.12.124] (vpn-129-150-12-124.SFBay.Sun.COM [129.150.12.124]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78I13ji126190; Wed, 08 Aug 2007 11:01:03 -0700 (PDT) Date: Wed, 08 Aug 2007 08:01:43 -1000 From: Joseph Kowalski Subject: Re: 2006/525: New Boot Sparc In-reply-to: <1186582879.3838.47.camel@localhost> To: Bill Sommerfeld Cc: Jan Setje-Eilers , James Carlson , John.Johnson@sun.com, Jerry.Gilliam@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <46BA0507.30804@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708070335.l773ZqEN000963@smack.Eng.Sun.COM> <1186582879.3838.47.camel@localhost> User-Agent: Thunderbird 1.5.0.5 (X11/20060911) Status: RO Content-Length: 1628 Bill Sommerfeld wrote: > On Mon, 2007-08-06 at 20:35 -0700, Jan Setje-Eilers wrote: > >>> I think this addresses my fears in the short term, though we do need to >>> see a revised spec which nails down specifics of how files may be >>> excluded from the archive. >>> >> I suspect it should be all editable files as well as all driver.conf >> files (editable or not) in the archive. I'll gather some data on this >> and will update the spec accordingly. >> > > You mentioned in a side conversation that you had heard that there was a > good reason why we packaged many driver.conf files as non-editable but > still occasionally told people to edit them anyway. > > If at all possible it would be very helpful to chase down the origin of > that folklore, as what I'm hearing about likely changes coming in the > packaging/patching/upgrade space suggests that the ground underneath > that folklore is about to shift fairly dramatically. > > - Bill > Oh boy, did you just turn over a slimy rock. The reason that driver.conf files were separated out was so they could be edited. Otherwise we could just hardwire the information in the driver itself. However, there was a *big* discussion about where they should be. One side was that they should be in the same filesystem as the driver (root vs. usr). The other side (mine) was that they should all be in /root so they could be editted. I recall this was a bigger issue than one might think. Of course, the real issue is "editable or not", rather than "where". The real question should be "if they aren't editable, why do we have them"? - jek3 From setje@smack.eng.sun.com Wed Aug 8 11:22:48 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78IMlgV022775 for ; Wed, 8 Aug 2007 11:22:47 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l78IKEWj020822 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 9 Aug 2007 02:20:26 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG00M13WA1IZ00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 11:20:25 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG00GDSWA1KE80@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 08 Aug 2007 11:20:25 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l78IKMvv002846; Wed, 08 Aug 2007 11:20:22 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l78He05l015147; Wed, 08 Aug 2007 10:40:00 -0700 (PDT) Date: Wed, 08 Aug 2007 10:39:59 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 08 Aug 2007 12:09:01 CDT." <20070808170901.GG1490@Sun.COM> To: Nicolas Williams Cc: Darren J Moffat , Scott Rotondo , Jerry.Gilliam@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, James Carlson , psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, Jan Setje-Eilers , setje@smack.eng.sun.com Message-id: <200708081740.l78He05l015147@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1487 > On Wed, Aug 08, 2007 at 10:05:24AM +0100, Darren J Moffat wrote: > > Scott Rotondo wrote: > > > The idea for a long-term solution (having the boot loader read multiple > > > files into memory instead of a single archive) sounds like a good one. > > > > I'm not sure that is the best solution in the network boot case. > > Consider that one of the reasons we designed wanboot with a downloadable > > archive was to *reduce* the number of files transferred over the network > > and to reduce the number of files we had to verify with an hmac. This > > is important not just for wanboot (which should get ported to x86) and > > also the netboot for NFS and iSCSI root cases as well. > > In the boot-from-local-media case the boot loader knows how to read from > that media, and teaching it about the filesystem on that media isn't all > that difficult (compared to writing realmode device drivers, I imagine). > > In the network boot case the stack is much larger, particularly when > IPsec or RPCSEC_GSS are thrown into the picture. Expecting the boot > loader to be able to read files from the real root in this case seems > like asking a lot. In the network boot case the boot loader also needs to know how to access and read from the remote source. Anything else would basically be magic. :) What is different is that we don't expose the entire client OS image to the booter anymore. We could go back to this, but as you point out it's not really necessary. -jan From Nicolas.Williams@sun.com Wed Aug 8 11:57:54 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l78Ivsxv024483 for ; Wed, 8 Aug 2007 11:57:54 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l78ItHK0057586; Wed, 8 Aug 2007 12:55:25 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMG0001HXWIN900@nwk-avmta-1.sfbay.Sun.COM>; Wed, 08 Aug 2007 11:55:30 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMG000TXXWH5I00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 08 Aug 2007 11:55:29 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l78ItLI6007693; Wed, 08 Aug 2007 13:55:21 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l78ItLeO007692; Wed, 08 Aug 2007 13:55:21 -0500 (CDT) Date: Wed, 08 Aug 2007 13:55:21 -0500 From: Nicolas Williams Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708081740.l78He05l015147@smack.Eng.Sun.COM> To: Jan Setje-Eilers Cc: Darren J Moffat , Scott Rotondo , Jerry.Gilliam@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Jan.Setje-Eilers@sun.com, James Carlson , psarc-ext@sun.com, Jerry Gilliam , Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <20070808185520.GR1490@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <20070808170901.GG1490@Sun.COM> <200708081740.l78He05l015147@smack.Eng.Sun.COM> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 523 On Wed, Aug 08, 2007 at 10:39:59AM -0700, Jan Setje-Eilers wrote: > In the network boot case the boot loader also needs to know how to > access and read from the remote source. Anything else would basically > be magic. :) But it already does. Though only with WANboot is it secure. Teaching grub to do HTTP and TLS won't suffice for x86: we need sort of key storage, and short of a BIOS (or LinuxBIOS) extension to match the OBP's then TPMs will have to be the answer. But at least we can defer that to another day. From jg@jurassic.sfbay.sun.com Thu Aug 9 08:53:36 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l79FrZ6U020050 for ; Thu, 9 Aug 2007 08:53:35 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l79Fp2db029675; Thu, 9 Aug 2007 16:51:09 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMI00K0NK16ZY00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 09 Aug 2007 08:51:06 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.68.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMI00FRDK15Q560@nwk-avmta-1.sfbay.Sun.COM>; Thu, 09 Aug 2007 08:51:05 -0700 (PDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l79Fp5D5187272; Thu, 09 Aug 2007 08:51:05 -0700 (PDT) Date: Thu, 09 Aug 2007 08:49:51 -0700 (PDT) From: Jerry Gilliam Subject: Re: 2006/525: New Boot Sparc To: psarc-ext@sun.com Cc: jerry.gilliam@sun.com, jan.setje-eilers@sun.com, john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200708091551.l79Fp5D5187272@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: IRNtyKLsXAuoE86y/GQShw== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 22395 Here are the updated materials for this fast-track. All updates are marked with change bars. The updates include input from FWARC. I've also bumped the timeout for a week, to 8/16/2007. -jg #ident "@(#)design 1.4 07/08/09 SMI" New Solaris SPARC Boot Architecture Summary The Solaris SPARC bootstrap process is being redesigned, both to increase the commonality with Solaris x86, and to enable ITUs (install time updates) on SPARC. Secondary goals are to combine inetboot and wanboot into a single network boot architecture, and to provide a simplified architecture for disk filesystems other than UFS (e.g, ZFS). The Solaris x86 newboot project has already been delivered to Nevada and S10u1. That project was described by PSARC 2004/454. This project is a follow-on to that project. This design specification only covers the architecture for Solaris SPARC. For details of the Solaris x86 design, see PSARC 2004/454. 1. Introduction --------------- The Solaris boot process was designed in early 1990's for desktops which were tiny by today's standards. On the one hand, the design was constrained by a relatively small amount of system memory. On the other hand, the design took advantage of the presence of Open Boot Prom (OBP) on all Sun platforms. The resulting implementation involves a complex sequence of control handoff between kernel and OBP to load a minimum amount of text and data from the root device into memory. 2. Motivation ------------- A change in the Solaris boot architecture is motivated by the following problems: 2.1 Supporting new hardware Solaris SPARC does not support ITUs, so a platform that requires any Solaris changes requires a full Solaris update release. This requirement causes pain for both the Solaris and SPARC HW groups, as the two must fight over Update schedules for almost every HW platform. 2.2 Commonality with Solaris x86 The newboot project on Solaris x86 has made both the administrative and boot processes visibly different between Solaris SPARC and Solaris x86. This is in part due to the addition of a boot archive to Solaris x86. It is in Sun's interest that the look and feel of Solaris on different platforms be as common as feasible. 2.3 Ease of booting different FS types The current boot architecture requires multiple filesystem readers, making adding new ones a difficult process. The goal of this project is to only require one phase to read the root FS before the kernel mounts it. This will enable new filesystem types like ZFS to be more easily supported as root filesystems. 2.5 Common network boot process The Solaris SPARC network boot process is very different between booting over a LAN versus booting over a WAN. The WAN process boots via a miniroot that looks much like how Solaris x86 installs over a network, but there is no commonality in how the miniroots are created or administered. This project will unify SPARC network booting around a single architecture. 3. Proposed Architecture ------------------------ 3.1 Boot Phase Independence The primary design center is to make the phases of the boot process be independent of each other. This allows the addition of new features (e.g., new filesystem types) without requiring changes to multiple parts of the boot chain. This project envisions 4 phases of booting on SPARC machines: OBP The OBP phase of boot is unchanged. In fact, it's a requirement that SPARC newboot not require new OBP functionality. OBP will continue to load and execute a booter from a disk or network device. booter The booter phase is responsible for reading in the boot archive and executing it. This is the only phase of the boot process that requires knowledge of the root filesystem format. ramdisk The ramdisk is a boot archive containing either kernel modules or an install miniroot. This boot archive is the same boot archive as is used on Solaris x86. Its FS format is private to itself. i.e., neither the booter nor the kernel needs to know whether the archive is HSFS or UFS (or ZFS for that matter). The ramdisk will extract the kernel image from the boot archive and execute it. In order to minimize the size of the ramdisk, in particular the install miniroot, which must reside in memory, the contents of the miniroot will be compressed. This compression is on a per file level and is implemented within the filesystem. In order to create compressed files a userland utility is used that simply compresses the file in place, the file is then marked as compressed via the _FIO_COMPRESSED (private) ioctl in the filesystem metadata. kernel The final stage is the kernel. The kernel extracts the rest of the primary modules from the boot archive, initializes itself, mounts the real root file system, and throws away the boot archive. This process is also the same as on Solaris x86. 3.2 Removal of bootops (and ufsboot) The bootops vector on SPARC originally existed to support platforms with either OBP or SUNMON FW. The second stage booter (e.g., ufsboot) presented a common set of operations to the kernel so the kernel didn't need to know either what prom version was running or what filesystem type root was. The last Solaris release to support a SUNMON platform was Solaris 2.4, and the last one to support pre-ieee1275 OBP was Solaris 7, so it's time for the bootops to go gently into that good night. 3.2.3 unix and krtld combined The Xen project (PSARC 2006/260) has combined the unix and krtld modules in order to enable booting from either BIOS or the Xen hypervisor. Since combining these two makes the unix ELF header far simpler to parse and load, this project has adapted this change for SPARC. 3.2.4 boot properties | | The booter will publish properties previously passed to the kernel via | bootops in the OBP /chosen node. | | Properties for both disk and network boot: | | "archive-fstype" | String, encoded with encode-string, which contains the name of | the boot archive filesystem type (e.g., "hsfs") | | "bootarchive" | String, encoded with encode-string, which contains the path to | the boot archive ramdisk device (e.g., "/ramdisk-root") | | "bootfs" | Integer, encoded with encode-int, which contains an ihandle | of the package whence the kernel was loaded. | | "elfheader-address" | Integer, encoded with encode-int, which contains the virtual | address of the kernel ELF image. | | "elfheader-length" | Integer, encoded with encode-int, which contains the size of | the kernel ELF image. | | "fstype" | String, encoded with encode-string, which contains the name | of the root filesystem type (e.g., "ufs") | | "fs-package" | String, encoded with encode-string, which contains the name | of the root file system reader package. | (e.g., "ufs-file-system") | | "impl-arch-name" | String, encoded with encode-string, which contains the | platform name. (e.g., "SUNW,Sun-Blade-1000") | | "whoami" | String, encoded with encode-string, which contains the file | system name of the kernel. | e.g., "/platform/sun4u/kernel/unix") | | Property only for wanboot: | | "netdev-path" | String, encoded with encode-string, which contains the path | to the newtwork device the boot archive is loaded from. | (e.g., /pci@8,700000/network@5,1") | | 3.3 Differences from Solaris x86 The primary difference from PSARC 2004/454 is that there is no dependence on grub. This decision was made for both practical and functional reasons. The practical reason is that grub0.95 - which was used by Solaris x86 - is not available on SPARC. The functional reason is that most of the reasons grub was used for Solaris x86 (e.g., eliminate real-mode, eliminate the boot shell, 3rd party device support) are not applicable to Solaris SPARC. Making grub work in the network boot case on SPARC is not a trivial exercise, as grub does not currently support NFS or HTTP. | The boot menu provided by grub would be an interesting feature to provide Solaris SPARC users. If grub2 were to become available on SPARC in a stable supportable release it could be incorporated in booter phase above. This should not require much change to existing code since the booter retains its ability to load alternate secondary booters such as cprboot. On SPARC the ability to read potentially volatile files directly from | the filesystem rather than the archive is exploited to avoid | triggering a boot archive check failure if those files get out of | date. This mechanism does not exist on x86. | 4 Interfaces ------------ 4.1 Interface Exported ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- Boot files Evolving platform//boot_archive default name of boot archive (initially may be sun4u/sun4v only) Boot args kernel args Evolving see kernel(1M) -F Evolving alternate booter or boot archive usr/sbin/bootadm(1M) Stable usr/sbin/root_archive(1M) Stable boot/solaris/bin create_ramdisk Proj. Priv. Install-time Update different user prompts boot/solaris filelist.ramdisk Proj. Priv. boot archive content filestat.ramdisk Proj. Priv. boot archive file status kernel/fs/dcfs Proj. Priv. compression file system sbin/fiocompress Proj. Priv. file compression utility usr/sbin/fiocompress Proj. Priv. link to sbin/fiocompress usr/include/sys/fs/decomp.h Proj. Priv. dcfs header file _FIO_COMPRESSED Proj. Priv. file compression ioctl usr/platform/sun4[uv]/lib/fs hsfs/bootblk Proj. Priv. filesystem readers zfs/bootblk Proj. Priv. filesystem readers platform/sun4[uv]/ufsboot Proj. Priv. removed kernel/misc/sparcv9/krtld Proj. Priv. merged into unix ----------------------------------------------------------------------- 4.2 Interfaces Reimplemented ----------------------------------------------------------------------- Name Level Comments ----------------------------------------------------------------------- reboot(1M) Stable? update boot archive as needed halt(1M) poweroff(1M) shutdown(1M) init(1M) pkgadd(1M) patchadd(1M) add_install_client Evolving modified server setup smdiskless(1M) Evolving setup /tftpboot area Install consume bootadm and installgrub Upgrade Live Upgrade Flash Install Net Install Jumpstart Release engineering tools ------------------------- modified miniroot construction 5. User Experience ------------------ 5.1 System startup The system startup changes are not visible to the user. 5.2 Installation and Upgrade The ITU menu which currently only exists on x86 may be exposed on sparc as well. Customers who rely on undocumented non-interfaced implementation details of add_install_client may need to amend their procedures. If we deliver the wanboot component, wanboot customers will see a simplification of the deployment procedures. 5.3 Internal tools The bfu scripts will be updated to permit developers to transition from old boot to new boot. For a system initially installed with old boot, it will be possible to bfu back and forth across the boundary. For a system initially installed with new boot, bfu back to old boot is not supported. Booting a glommed kernel is supported, with the same restrictions regarding the compatibility of userland and kernel. If the root file system goes into a non-bootable state, a user may boot the failsafe archive to perform manual recovery operations. The failsafe archives contains files normally present in a CD or netinstall miniroot. If the boot archive goes into a non-bootable state, a user may bypass the boot archive and directly boot the kernel with the -F option. 5.4 Coexistence with other OSes The only other OS of note for SPARC is Linux, and they currently have no grub plans. SPARC / Linux boots from a loader called SILO, which already has a grub-like menu facility. In general OBP is considered well suited to loading multiple OS's or versions of OS's from various devices on the system, so no additional work beyond what is provided by OBP/the system is desirable. 6. Technical Details -------------------- 6.1 OBP phase This project does not change the OBP phase of SPARC boot; this section is included for reference only. When a user types "boot" on an OBP-based system, the device selected - either from the command line or via the "boot-device" nvram variable - has its "open" and "load" methods called. The program loaded by this process is then executed, and the boot process enters the booter phase. 6.1.1 Disk For disk devices, the FW driver usually uses the OBP label package's "load" method, which parses the VTOC label at the beginning of the disk to locate the specified partition, then reads sectors 1-15 of that partition into memory. This area is commonly called the boot block and usually contains a filesystem reader. 6.1.2 Net For network devices, the process is slightly different between booting over a LAN versus booting over a WAN. In both cases, however, the prom will download a booter from a boot or install server (inetboot in this case). 6.1.2.1 LAN boot When booting over a LAN, the FW uses either RARP and BOOTP or DHCP to discover its boot or install server. It then uses TFTP to download the booter (inetboot in this case). 6.1.2.2 WAN boot When booting over a WAN, the FW uses either DHCP or nvram properties to discover its install server, and the router and proxies needed to connect to it. It then uses HTTP to download the booter, and may optionally check the booter's signature with a predefined private key. For more details, see PSARC 2001/009. 6.2 Booter phase This phase is derived from the SPARC wanboot ramdisk process (see PSARC 2001/009), and is responsible for reading the boot archive from the root file system (or install server in wanboot's case) into a ramdisk device. It does this by: 1) opening the boot-device (which it found as the "bootpath" property in the OBP "/chosen" node) 2) using its file system specific reader to read the boot archive (by default, /platform/`uname -m`/boot_archive) 3) creating a ramdisk device in "/ramdisk" 4) creating "bootarchive" and "fstype" properties in | "/chosen" | 5) booting the archive (a ramdisk is just another type of disk, so executing the boot block area serves this purpose) 6.3 Ramdisk phase The ramdisk is self-describing in the same sense any disk image is by virtue of having a filesystem reader in its boot block. This reader is over 90% the same as the disk boot block for a given filesystem so the same program is re-used. Its job is to load and execute the kernel from the archive by: 1) opening the boot archive (the "bootarchive" property from the previous phase) 2) using its file system specific reader to read the kernel (by default, /platform/`uname -i`/kernel/unix) 3) creating "impl-arch-name", "whoami" and "elfheader-address" properties | 4) executing the kernel 6.4 Kernel phase When krtld gains control, it mounts the boot archive and loads additional kernel modules from the boot archive via the ramdisk. Subsequent kernel initialization procedures remain the same until after the kernel mounts the root file system. At that point, the kernel throws away the boot archive and reclaims the memory it occupies. Note that in the install case, the ramdisk actually contains the root file system, and is not thrown away. The kernel ramdisk driver simply takes over control of the ramdisk image. 6.5 Chained booters When booting from a disk, the booter will support chained booters both for cprboot (see PSARC 1992/201) and for situations where the file system reader cannot fit in the boot block. In a future project, grub2 can use this facility to add a graphical user menu to the booter phase. This facility will not be available when booting from a network, since the chained booter usually uses the same virtual address space as the original booter. 6.6 Boot archive management There are two kinds of boot archive: failsafe and normal. A failsafe archive is self-sufficient and bootable by itself. It is created at install time and requires no maintenance. A normal archive shadows a root filesystem, so it contains all kernel modules, driver.conf files, and a few configuration files in /etc which are read by the kernel before root is mounted. Once the root filesystem is mounted, the kernel discards the boot archive from memory and file I/O will be performed against the root device. By default, the normal archive contains the following files and | directories: | | etc/dacf.conf | etc/devices/devid_cache | etc/devices/mdi_scsi_vhci_cache | etc/devices/mdi_ib_cache | etc/cluster/nodeid | etc/zfs/zpool.cache | kernel | platform | | Excluding any editable driver.conf files. | | While on x86 it also contains the following editable files: | | etc/system | etc/name_to_major | etc/driver_aliases | etc/name_to_sysnum | etc/driver_classes | etc/path_to_inst | | As well as a number of editable driver.conf files. | The contents under the platform directory will be segregated into those needed for a sun4u boot archive and those needed for a sun4v boot archive. Further per-platform differentiated boot archives may be considered if that helps us gain faster booting via faster archive load, trading off a more complicated archive construction process. If any files in this list (or under directories listed) is updated, the boot archive must be rebuilt prior to the next reboot for the modification to take effect. The package and patch tools are updated to update the boot archive whenever needed. In addition, the boot archive is updated as necessary on an orderly system shutdown to catch files modified manually. The boot archive could be out of sync with the root filesystem if the | system panics in the middle of an update, but before archive update is | completed. We check for such conditions on every boot before root | filesystem is mounted read/write. If an inconsistency is detected, the | system will stop in single-user mode, similar to current behavior when | fsck fails on the root filesystem. The recommended recovery method is | to boot the failsafe archive and recreate the boot_archive. An expert | user may decide to continue booting if the out-of-sync files are not | critical. | | There are a number of files in the archive (as listed in | oot/solaris/filelist.safe): | | etc/devices/devid_cache | etc/devices/mdi_scsi_vhci_cache | etc/devices/mdi_ib_cache | etc/path_to_inst | etc/rtc_config | etc/zfs/zpool.cache | | Each represents a pure cache, and while being up-to-date can improve | boot-time, can simply be re-built with no ill-effects on the system. | These files being out of sync does not cause an archive check failure. | | Further, an out-of-sync state due to editable files that are | updated during system administration tasks in the archive is avoided | by not placing such files in the archive. These files are read | directly from the filesystem by kobj_open() if it fails to find them | in the archive. This mechanism only exists on SPARC. | | The bootadm(1M) command will handle the details of archive update and verification. 6.7 Install and upgrade Normal install and upgrade is achieved by booting the miniroot from either CDROM/DVD or from the network. In both cases, the root filesystem of the miniroot is the ramdisk. This allows the Solaris boot CD to be ejected without rebooting the system. The boot archive contains the entire miniroot. The construction of the install CD is modified to use an hsfs boot block. The miniroot is packed into a single file in ufs format, to be loaded as the ramdisk image. The setup of the net boot server is also modified. The boot server will serve a boot strap as well as the ramdisk image which is downloaded and then booted from. The netinstall image will be packed using root_archive(1m). The process for installing the OS to disk remains the same except that the boot blocks are different and a boot archive must be constructed prior to booting the install target disk. The boot archives are created using bootadm(1M). The rest of code changes comes with packages and patches and no special treatment is required. 6.8 Diskless clients Diskless boot is similar to booting the miniroot for net install except that the root filesystem is on NFS instead of on UFS. 6.9 Install-time Update (ITU) New-boot (phase 1: x86) reduced the x86 ITU mechanism described in PSARC 1997/059 to simply adding Solaris binaries (drivers, kernel modules, commands, libraries, symlinks and the like) to the running miniroot and then the target install environment. It also extended the supported media that an ITU could be supplied on to include CD/DVD, memory sticks and similar devices. This mechanism will now be made available on sparc as well to allow platform support to be delivered out of band of a regularly scheduled release. If a core kernel component needs to be updated, say unix or something else that needs to be loaded before an ITU can be added, then a pre-patched miniroot image needs to be made available (50Mb download) along with the patch. References ---------- 1. Shudong Zhou PSARC 2004/454 Solaris Boot Architecture 2. Carl Smith PSARC 2001/009 WAN-boot 3. Clark Dong PSARC 1992/201 Checkpoint Resume (Reanimator) 4. Allan McKillop PSARC 2006/260 Solaris on Xen From jg@jurassic.sfbay.sun.com Tue Aug 14 14:56:57 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7ELuuig025932 for ; Tue, 14 Aug 2007 14:56:56 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7ELsN35004268; Wed, 15 Aug 2007 05:54:25 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMS00307A6NY800@brm-avmta-1.central.sun.com>; Tue, 14 Aug 2007 15:54:23 -0600 (MDT) Received: from jurassic.eng.sun.com ([129.146.56.144]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMS00L2OA6LYP60@brm-avmta-1.central.sun.com>; Tue, 14 Aug 2007 15:54:22 -0600 (MDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l7ELsKfY177777; Tue, 14 Aug 2007 14:54:21 -0700 (PDT) Date: Tue, 14 Aug 2007 14:53:05 -0700 (PDT) From: Jerry Gilliam Subject: Re: 2006/525: New Boot Sparc To: psarc-ext@sun.com, jan.setje-eilers@sun.com Cc: john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: 7xD/2Y5l+i/4rE8JtkLgiA== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 630 Peter Memishian has pointed out that the statement that /etc/path_to_inst is a "pure cache" is not correct. It is true that a functional instance file can be rebuilt but solaris does rely on the instance number ordering in the instance file not to change, otherwise devices could be renumbered. Additions to the instance file aren't necessary to boot, because the earlier version must have been bootable too. And since changes are cumulative, any additions in the updated file can be merged without needing manual intervention. Hence a discrepancy between the two instance files is not cause to invalidate the archive. -jg From casper@holland.sun.com Tue Aug 14 15:03:35 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7EM3YJs026318 for ; Tue, 14 Aug 2007 15:03:34 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7EM17Oc006905 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 15 Aug 2007 06:01:08 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMS00M03AHVDK00@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 14 Aug 2007 15:01:07 -0700 (PDT) Received: from dm-holland-02.uk.sun.com ([129.156.101.225]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMS00BT4AHTAEA0@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Tue, 14 Aug 2007 15:01:06 -0700 (PDT) Received: from holland (room101.Holland.Sun.COM [129.159.130.93]) by dm-holland-02.uk.sun.com (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7EM0rBs005106; Tue, 14 Aug 2007 23:00:53 +0100 (BST) Date: Wed, 15 Aug 2007 00:00:53 +0200 From: Casper.Dik@sun.com Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> Sender: casper@holland.sun.com To: Jerry Gilliam Cc: psarc-ext@sun.com, Jan.Setje-Eilers@sun.com, John.Johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com Message-id: <200708142200.l7EM0rBs005106@dm-holland-02.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> Status: RO Content-Length: 1093 > >Peter Memishian has pointed out that the statement that >/etc/path_to_inst is a "pure cache" is not correct. >It is true that a functional instance file can be rebuilt >but solaris does rely on the instance number ordering >in the instance file not to change, otherwise devices >could be renumbered. Additions to the instance file >aren't necessary to boot, because the earlier version >must have been bootable too. And since changes are >cumulative, any additions in the updated file can be merged >without needing manual intervention. Hence a discrepancy >between the two instance files is not cause to invalidate >the archive. With the proviso that under no circumstances the version in the archive can be used to assign NEW device instances. I.e., when you have a new USB device plugged in at boot and the /etc/path_to_inst file has a record of old USB devices not yet reflected in the boot archive, there must not be overlap in device IDs. In other words: we must be using the non-bootarchive version of /etc/path_to_inst before assigning new device instance numbers. Casper From sommerfeld@sun.com Tue Aug 14 15:09:28 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7EM9Sb7026527 for ; Tue, 14 Aug 2007 15:09:28 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7EM72ls018910; Tue, 14 Aug 2007 15:07:03 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMS00M07ARPMS00@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Aug 2007 15:07:01 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMS00BIUARNAOB0@nwk-avmta-2.sfbay.sun.com>; Tue, 14 Aug 2007 15:06:59 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l7EM6ok8063323; Tue, 14 Aug 2007 18:06:50 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l7EM6ocs020006; Tue, 14 Aug 2007 18:06:50 -0400 (EDT) Date: Tue, 14 Aug 2007 18:06:49 -0400 From: Bill Sommerfeld Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> To: Jerry Gilliam Cc: psarc-ext@sun.com, jan.setje-eilers@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com, Lori.Alt@sun.com, john.johnson@sun.com Message-id: <1187129209.18736.38.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> Status: RO Content-Length: 1255 On Tue, 2007-08-14 at 14:53 -0700, Jerry Gilliam wrote: > Peter Memishian has pointed out that the statement that > /etc/path_to_inst is a "pure cache" is not correct. > It is true that a functional instance file can be rebuilt > but solaris does rely on the instance number ordering > in the instance file not to change, otherwise devices > could be renumbered. Additions to the instance file > aren't necessary to boot, because the earlier version > must have been bootable too. And since changes are > cumulative, any additions in the updated file can be merged > without needing manual intervention. Hence a discrepancy > between the two instance files is not cause to invalidate > the archive. I think it would be helpful to come up with a revision to section 6.6 which captures the higher-level requirement more crisply. I suspect that zpool.cache may be in much the same boat; as I understand it, it's not, strictly speaking a cache (if you blow it away, you have to re-import pools). I'd hope that it would be sufficient for the zpool.cache in the archive to describe the root pool (which at present is limited to a single mirror set), and that configuration for other pools could be pulled out of the actual copy in the root filesystem. From setje@smack.eng.sun.com Wed Aug 15 11:49:50 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7FIno65018123 for ; Wed, 15 Aug 2007 11:49:50 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7FIlCSI008919 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Wed, 15 Aug 2007 19:47:22 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMT0009BW6WLT00@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 15 Aug 2007 11:47:20 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMT00HUFW6R5360@nwk-avmta-1.sfbay.Sun.COM> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Wed, 15 Aug 2007 11:47:15 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7FIlFsV023263; Wed, 15 Aug 2007 11:47:15 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7FIaisl029309; Wed, 15 Aug 2007 11:36:44 -0700 (PDT) Date: Wed, 15 Aug 2007 11:36:44 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Tue, 14 Aug 2007 14:53:05 PDT." <200708142154.l7ELsKfY177777@jurassic.eng.sun.com> To: Jerry Gilliam Cc: psarc-ext@sun.com, jan.setje-eilers@sun.com, john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708151836.l7FIaisl029309@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1653 > > Peter Memishian has pointed out that the statement that > /etc/path_to_inst is a "pure cache" is not correct. > It is true that a functional instance file can be rebuilt > but solaris does rely on the instance number ordering > in the instance file not to change, otherwise devices > could be renumbered. The only place these instance numbers are actually exposes is in network drivers. The fact that they are exposed there is a long standing miss-architecture that will eventually (no immediate cut- over due to compat. issues) be addressed by clearview. I had in fact prototyped a re-merge of an out of date path_to_inst during boot, but was encouraged not to put that back with the archive check changes since the only exposed issue this addresses is dr-ing in more than one nic of the same flavor and then rebooting without syncing the archive while caring which nic was which (not ipmp or an aggr). While this situation is real, it is at least somewhat contrived and the additional complexity of the re-merge didn't seem like something we'd want to take give that another project was going to eliminate the exposure entirely. I'm still open to putting this back in, but we should feel like it's something we'll want to keep around moving forward or just a temporary bug fix. > Additions to the instance file > aren't necessary to boot, because the earlier version > must have been bootable too. And since changes are > cumulative, any additions in the updated file can be merged > without needing manual intervention. Hence a discrepancy > between the two instance files is not cause to invalidate > the archive. Correct. -jan From setje@smack.eng.sun.com Wed Aug 15 12:01:39 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7FJ1caQ018578 for ; Wed, 15 Aug 2007 12:01:39 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7FIwoJn012311; Wed, 15 Aug 2007 19:59:11 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMT00185WQKI000@nwk-avmta-1.sfbay.Sun.COM>; Wed, 15 Aug 2007 11:59:08 -0700 (PDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMT00HNZWQI5A70@nwk-avmta-1.sfbay.Sun.COM>; Wed, 15 Aug 2007 11:59:06 -0700 (PDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7FIx5Y9025493; Wed, 15 Aug 2007 11:59:05 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7FIf51h029378; Wed, 15 Aug 2007 11:41:05 -0700 (PDT) Date: Wed, 15 Aug 2007 11:41:05 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Tue, 14 Aug 2007 18:06:49 EDT." <1187129209.18736.38.camel@thunk> To: Bill Sommerfeld Cc: Jerry Gilliam , PSARC-EXT@sun.com, jan.setje-eilers@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com, Lori.Alt@sun.com, john.johnson@sun.com, setje@smack.eng.sun.com Message-id: <200708151841.l7FIf51h029378@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1625 > On Tue, 2007-08-14 at 14:53 -0700, Jerry Gilliam wrote: > > Peter Memishian has pointed out that the statement that > > /etc/path_to_inst is a "pure cache" is not correct. > > It is true that a functional instance file can be rebuilt > > but solaris does rely on the instance number ordering > > in the instance file not to change, otherwise devices > > could be renumbered. Additions to the instance file > > aren't necessary to boot, because the earlier version > > must have been bootable too. And since changes are > > cumulative, any additions in the updated file can be merged > > without needing manual intervention. Hence a discrepancy > > between the two instance files is not cause to invalidate > > the archive. > > I think it would be helpful to come up with a revision to section 6.6 > which captures the higher-level requirement more crisply. > > I suspect that zpool.cache may be in much the same boat; as I understand > it, it's not, strictly speaking a cache (if you blow it away, you have > to re-import pools). I'd hope that it would be sufficient for the > zpool.cache in the archive to describe the root pool (which at present > is limited to a single mirror set), and that configuration for other > pools could be pulled out of the actual copy in the root filesystem. I was lead to believe that zpool.cache will auto-magically be re-built for non-root filesystems. For root filesystems, the kernel will need to learn how to mount root without it (explicitly hunt for the device), otherwise we'll have a lot of ugly failed to mount root failures, which are obviously a serious issue. -jan From Lori.Alt@sun.com Wed Aug 15 12:20:23 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7FJKNFX018862 for ; Wed, 15 Aug 2007 12:20:23 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7FJHer0062538; Wed, 15 Aug 2007 13:17:43 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMT00315XLV2K00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 15 Aug 2007 12:17:55 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMT00HAKXLR5BA0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 15 Aug 2007 12:17:52 -0700 (PDT) Received: from fe-amer-02.sun.com ([192.18.108.176]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l7FJHpvL000954; Wed, 15 Aug 2007 19:17:51 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMT00M01XDD0F00@mail-amer.sun.com> (original mail from Lori.Alt@Sun.COM) ; Wed, 15 Aug 2007 13:17:50 -0600 (MDT) Received: from esquivel.Central.Sun.COM ([172.20.25.206]) by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMT00E4YXLPLRF7@mail-amer.sun.com>; Wed, 15 Aug 2007 13:17:50 -0600 (MDT) Date: Wed, 15 Aug 2007 13:17:26 -0600 From: Lori Alt Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708151841.l7FIf51h029378@smack.Eng.Sun.COM> Sender: Lori.Alt@sun.com To: Jan Setje-Eilers Cc: Bill Sommerfeld , Jerry Gilliam , psarc-ext@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Lin.Ling@sun.com, Jeff.Cheeney@sun.com, John.Johnson@sun.com Reply-to: Lori.Alt@sun.com Message-id: <46C35146.5050801@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708151841.l7FIf51h029378@smack.Eng.Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 2059 Jan Setje-Eilers wrote: >> On Tue, 2007-08-14 at 14:53 -0700, Jerry Gilliam wrote: >> >>> Peter Memishian has pointed out that the statement that >>> /etc/path_to_inst is a "pure cache" is not correct. >>> It is true that a functional instance file can be rebuilt >>> but solaris does rely on the instance number ordering >>> in the instance file not to change, otherwise devices >>> could be renumbered. Additions to the instance file >>> aren't necessary to boot, because the earlier version >>> must have been bootable too. And since changes are >>> cumulative, any additions in the updated file can be merged >>> without needing manual intervention. Hence a discrepancy >>> between the two instance files is not cause to invalidate >>> the archive. >>> >> I think it would be helpful to come up with a revision to section 6.6 >> which captures the higher-level requirement more crisply. >> >> I suspect that zpool.cache may be in much the same boat; as I understand >> it, it's not, strictly speaking a cache (if you blow it away, you have >> to re-import pools). I'd hope that it would be sufficient for the >> zpool.cache in the archive to describe the root pool (which at present >> is limited to a single mirror set), and that configuration for other >> pools could be pulled out of the actual copy in the root filesystem. >> > > I was lead to believe that zpool.cache will auto-magically be > re-built for non-root filesystems. > > For root filesystems, the kernel will need to learn how to mount root > without it (explicitly hunt for the device), otherwise we'll have a > lot of ugly failed to mount root failures, which are obviously a > serious issue. > > > Although we rely on zpool.cache now, we need to modify the early kernel code to cope if there isn't one. The identity of the pool being booted from is known to the booter (both GRUB and the OBP zfs booter) and is passed to the kernel. From that information, and the vdev label on the disk we're booted from, we should be able to initialize the root pool. Lori From Sebastien.Roy@sun.com Wed Aug 15 13:10:09 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7FKA9E5020019 for ; Wed, 15 Aug 2007 13:10:09 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7FK7duS029126 for <@sunmail2sca.sfbay.sun.com:PSARC-EXT@sun.com>; Wed, 15 Aug 2007 13:07:42 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMT00F05ZWTP100@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 15 Aug 2007 14:07:41 -0600 (MDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMT0087UZWS9W60@brm-avmta-1.central.sun.com> for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 15 Aug 2007 14:07:41 -0600 (MDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l7FK7edq020365 for ; Wed, 15 Aug 2007 20:07:40 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JMT00401ZKE5K00@mail-amer.sun.com> (original mail from Sebastien.Roy@Sun.COM) for PSARC-EXT@sun.com (ORCPT PSARC-EXT@sun.com); Wed, 15 Aug 2007 14:07:40 -0600 (MDT) Received: from [129.148.174.103] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JMT00DUPZWREWF7@mail-amer.sun.com>; Wed, 15 Aug 2007 14:07:40 -0600 (MDT) Date: Wed, 15 Aug 2007 16:07:37 -0400 From: Sebastien Roy Subject: Re: 2006/525: New Boot Sparc In-reply-to: <200708151836.l7FIaisl029309@smack.Eng.Sun.COM> Sender: Sebastien.Roy@sun.com To: Jan Setje-Eilers Cc: Jerry Gilliam , John.Johnson@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, PSARC-EXT@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Jeff.Cheeney@sun.com Message-id: <46C35D09.6080907@sun.com> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708151836.l7FIaisl029309@smack.Eng.Sun.COM> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 2456 Jan Setje-Eilers wrote: >> Peter Memishian has pointed out that the statement that >> /etc/path_to_inst is a "pure cache" is not correct. >> It is true that a functional instance file can be rebuilt >> but solaris does rely on the instance number ordering >> in the instance file not to change, otherwise devices >> could be renumbered. > > The only place these instance numbers are actually exposes is in > network drivers. The fact that they are exposed there is a long > standing miss-architecture that will eventually (no immediate cut- > over due to compat. issues) be addressed by clearview. The Clearview project doesn't completely remove that particular bit of mis-architecture. It just makes the impact less disastrous to administrators. The Clearview UV component legitimizes the administrative object which is the datalink, and allows the administrator to give that object a meaningful name (e.g., please give the name "isp3" to the datalink over the bge0 device) There is a device under that datalink, and that device name is still at the mercy of path_to_inst. If path_to_inst is munged in such a way that what used to be "bge0" is now "bge2", then an administrator will need to re-associate the "isp3" datalink with the proper device by renaming links (i.e., manually using dladm(1M) rename-link commands). As a result I would say that path_to_inst is still a crucial piece of system state during boot even after Clearview. > I had in fact prototyped a re-merge of an out of date path_to_inst > during boot, but was encouraged not to put that back with the archive > check changes since the only exposed issue this addresses is dr-ing in > more than one nic of the same flavor and then rebooting without > syncing the archive while caring which nic was which (not ipmp or an > aggr). While this situation is real, it is at least somewhat contrived > and the additional complexity of the re-merge didn't seem like > something we'd want to take give that another project was going to > eliminate the exposure entirely. Would it always be safe to use the /etc/path_to_inst file in the / filesystem as is being proposed for editable files and driver.conf files? > I'm still open to putting this back in, but we should feel like it's > something we'll want to keep around moving forward or just a temporary > bug fix. If we can read the file that's in the filesystem, then I don't see the need for the re-merge. -Seb From setje@smack.eng.sun.com Wed Aug 15 13:53:02 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7FKr1jt020921 for ; Wed, 15 Aug 2007 13:53:02 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7FKoMTm022797 for <@sunmail2sca.sfbay.sun.com:PSARC-EXT@Sun.COM>; Wed, 15 Aug 2007 21:50:34 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JMU00I5V1W71J00@brm-avmta-1.central.sun.com> for PSARC-EXT@Sun.COM (ORCPT PSARC-EXT@Sun.COM); Wed, 15 Aug 2007 14:50:31 -0600 (MDT) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JMU008RI1W69Z80@brm-avmta-1.central.sun.com> for PSARC-EXT@Sun.COM (ORCPT PSARC-EXT@Sun.COM); Wed, 15 Aug 2007 14:50:30 -0600 (MDT) Received: from smack.Eng.Sun.COM (smack.SFBay.Sun.COM [129.146.226.114]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7FKoSmX018347; Wed, 15 Aug 2007 13:50:28 -0700 (PDT) Received: from smack (localhost [127.0.0.1]) by smack.Eng.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7FKA0L5000949; Wed, 15 Aug 2007 13:10:00 -0700 (PDT) Date: Wed, 15 Aug 2007 13:10:00 -0700 From: Jan Setje-Eilers Subject: Re: 2006/525: New Boot Sparc In-reply-to: "Your message of Wed, 15 Aug 2007 16:07:37 EDT." <46C35D09.6080907@sun.com> To: Sebastien Roy Cc: Jan Setje-Eilers , Jerry Gilliam , John.Johnson@sun.com, Lori.Alt@sun.com, Lin.Ling@sun.com, PSARC-EXT@sun.com, Jan.Setje-Eilers@sun.com, Joseph.Bonasera@sun.com, Jeff.Cheeney@sun.com, setje@smack.eng.sun.com Message-id: <200708152010.l7FKA0L5000949@smack.Eng.Sun.COM> MIME-version: 1.0 X-Mailer: exmh version 2.6.3 04/04/2003 with version: MH 6.8.4 #31[UCI] Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2709 > Jan Setje-Eilers wrote: > >> Peter Memishian has pointed out that the statement that > >> /etc/path_to_inst is a "pure cache" is not correct. > >> It is true that a functional instance file can be rebuilt > >> but solaris does rely on the instance number ordering > >> in the instance file not to change, otherwise devices > >> could be renumbered. > > > > The only place these instance numbers are actually exposes is in > > network drivers. The fact that they are exposed there is a long > > standing miss-architecture that will eventually (no immediate cut- > > over due to compat. issues) be addressed by clearview. > > The Clearview project doesn't completely remove that particular bit of > mis-architecture. It just makes the impact less disastrous to > administrators. > > The Clearview UV component legitimizes the administrative object which is > the datalink, and allows the administrator to give that object a > meaningful name (e.g., please give the name "isp3" to the datalink over > the bge0 device) There is a device under that datalink, and that device > name is still at the mercy of path_to_inst. If path_to_inst is munged in > such a way that what used to be "bge0" is now "bge2", then an > administrator will need to re-associate the "isp3" datalink with the > proper device by renaming links (i.e., manually using dladm(1M) > rename-link commands). > > As a result I would say that path_to_inst is still a crucial piece of > system state during boot even after Clearview. > > > I had in fact prototyped a re-merge of an out of date path_to_inst > > during boot, but was encouraged not to put that back with the archive > > check changes since the only exposed issue this addresses is dr-ing in > > more than one nic of the same flavor and then rebooting without > > syncing the archive while caring which nic was which (not ipmp or an > > aggr). While this situation is real, it is at least somewhat contrived > > and the additional complexity of the re-merge didn't seem like > > something we'd want to take give that another project was going to > > eliminate the exposure entirely. > > Would it always be safe to use the /etc/path_to_inst file in the / > filesystem as is being proposed for editable files and driver.conf files? > > > I'm still open to putting this back in, but we should feel like it's > > something we'll want to keep around moving forward or just a temporary > > bug fix. > > If we can read the file that's in the filesystem, then I don't see the > need for the re-merge. If clearview continues to expose the internal instance number in the device name, then we should really re-merge path_to_inst on both sparc and x86. -jan From jg@jurassic.sfbay.sun.com Mon Aug 20 18:26:59 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7L1QxbT005507 for ; Mon, 20 Aug 2007 18:26:59 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7L1ORtX014657; Mon, 20 Aug 2007 18:24:27 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JN300303NWRI400@brm-avmta-1.central.sun.com>; Mon, 20 Aug 2007 19:24:27 -0600 (MDT) Received: from jurassic.eng.sun.com ([129.146.228.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JN300DKHNWP5ME0@brm-avmta-1.central.sun.com>; Mon, 20 Aug 2007 19:24:26 -0600 (MDT) Received: from tethys (tethys.SFBay.Sun.COM [129.146.226.92]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with SMTP id l7L1OPtW813816; Mon, 20 Aug 2007 18:24:25 -0700 (PDT) Date: Mon, 20 Aug 2007 18:23:06 -0700 (PDT) From: Jerry Gilliam Subject: Re: 2006/525: New Boot Sparc To: PSARC-EXT@sun.com Cc: jerry.gilliam@sun.com, jan.setje-eilers@sun.com, john.johnson@sun.com, Lori.Alt@sun.com, Joseph.Bonasera@sun.com, lin.ling@sun.com, Jeff.Cheeney@sun.com Reply-to: Jerry Gilliam Message-id: <200708210124.l7L1OPtW813816@jurassic.eng.sun.com> MIME-version: 1.0 X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.7_64 SunOS 5.11 sun4u sparc Content-type: TEXT/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-MD5: Otdc24YHelrvf4lz0i82FQ== X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 1760 Last week we worked out offline an update to the spec to clarify the /etc/path_to_inst issue. Diffs to the spec included below. The complete updated spec has been placed in the final.materials directory. Resolving this has addressed all outstanding requests, bringing this case to a successful conclusion. Thanks everyone. -jg ------- etc/devices/devid_cache etc/devices/mdi_scsi_vhci_cache etc/devices/mdi_ib_cache - etc/path_to_inst etc/rtc_config etc/zfs/zpool.cache Each represents a pure cache, and while being up-to-date can improve boot-time, can simply be re-built with no ill-effects on the system. These files being out of sync does not cause an archive check failure. + etc/path_to_inst is also included in the list of safe files. The + system does rely on the persistence of entries added to this + file to preserve device ordering. A system can always boot with + an out-of-date path_to_inst, as long as newer entries from the + root filesystem are merged with that of the boot archive and that + no new device instances are assigned before the corrected + path_to_inst is constructed. + + The circumstances that require special handling for the instance + file are rather unique and the team hopes that future projects + will not contribute additional requirements in this direction. +