Copyright 2007 Sun Microsystems Authors:Margot Hackett Miller margot.miller@sun.com Ken Erickson ken.erickson@sun.com Jens Deppe jens.deppe@sun.com Problem: When NSG preinstalls Solaris on their boxes, they do so by disk cloning the Solaris image, modifying an rc script to create a hostid and then sys-unconfiging the system. When the customer boots the preinstalled Solaris box an rc script is run that creates the hostid and a reboot is necessary. What we would like would be an integrated software solution to the X86/64 hostid generation that doesn't necessitate a reboot. During the project we learned about the PSARC/2005/295 case which signs all executables and does runtime verification of object signatures before they are executed. As a result, that team asked us to remove kernel/misc/sysinit, the ELF file where the Solaris x86/64 hostid is stored and instead use a non ELF file. This was one of the reasons to discard kernel/misc/sysinit and instead use /etc/hostid for storage of x86/64 hostid in ASCII, similar to Linux systems. Currently the hostid is generated by Suninstall and loaded into the kernel by sysinit module. To simplify the situation, we would like to move the X86/64 hostid generation code from Install consolidation and move into ON. And with Solaris being open sourced and the many blogs detailing how to modify the hostid, having the overly complex and obscure sysinit module no longer makes sense; it is an example of security by obscurity. The major problem with simplifying hostid and making it more "Linux" like is the whole licensing issue with hostid. Hostid is used by external products for licensing schemes, though the man page for gethostid states that hostid is "not garanteed to be unique". We have polled numerous aliases about this issue and the impact on the ISV's which resulted in two opposing camps - ISV's should never have used hostid in this way or Sun shouldn't "break" hostid licensing and anger our ISV's. We originally thought we would like to follow the Linux model and store hostid in a protected file /etc/hostid and also provide the "generate the hostid" option to the existing hostid command as well as an option to set hostid to a particular value (options would be restricted to privileged users). However, there was much angst to this approach from MDE stating that even though hostid has never been a secure mechanism, many Solaris ISV's are basing their licensing shemes on it. As a result, we will not provide an option to the hostid to generate a hostid or to set the hostid to a particular value. As a result no changes will be made to the hostid command. Consequently, there will be a /etc/hostid file owned by root and furthermore, after numerous discussions with MDE we have agreed to encrypt the file with ROT47 encryption. This is by no means increases the security of the file and is only done so to appease the ISV's. The file /etc/hostid will be a project private file and will not be documented in any of the man pages; the end users should do nothing with this file. We will also put in the file the comment "Do not edit this file." This proposal is targeting those platforms that do not support Solaris-compatible hostid values (which is currently Solaris X86/64) and not Solaris Sparc. The Solaris Sparc implementation will not be modified and the documentation for hostid will state that it is limited to those non-Sparc platforms. In addition, we will take this opportunity to clean up the man pages wrt hostid and X86/64. For example, the man page for gethostid() states that hostid is "taken from the CPU board's ID PROM." This of course, isn't true on X86/64 boxes. The SPARC and X86/64 implementations will always be different as hostid on SPARC is tied to the NVRAM and on X86/64 it really is just an instance of a software installation. We can't really get rid of hostid as our investigation has shown that it is widely used in licensing. The approach we would like to take for this fast track is to limit our scope and fix our immediate problems: Eliminate the reboot needed on the preinstalled NSG systems Remove ELF binary of sysinit File RFE's against daemons and libraries that are using gethostid() as a seed to random number generator and instead encourage use /dev/random or /dev/urandom. (See below future work that this is could be preparation for) Clean up man pages to document hostid behavior differences for those systems that have hostid in hardware vs. hostid generated via software Optionally state in man pages to not use hostid for licensing purpose as it is not secure (need to work with MDE and legal on appropriate wording for that) Retain current behavior of Solaris upgrade that preserves existing hostid EXPORTED INTERFACES ------------------- /etc/hostid Contracted Project Private (Install consolidation will use this to preserve hostid upon upgrade) /lib/svc/method/sethostid-system Project Private svc:system/hostid FMRI Project Private /kernel/misc/sysinit Will be removed from ON consolidation /kernel/misc/amd64/sysinit Will be removed from ON consolidation (Still used by Install consolidation to preserve hostid upon upgrade on older systems) Technical Description: Unlike Sparc systems, X86/64 boxes do not have a unique hardware hostid. As a result, during a suninstall the function setser() from svc_updateserial.c is called to generate a software serial number, based on some algorithm involving gettimeofday(). We would no longer generate the hostid during a Suninstall. When the system boots up the kernel will read from /etc/hostid file if it exists. If /etc/hostid does not exist or is corrupted, the kernel will generate one. Currently in usr/src/uts/i86pc/os/startup.c, the X86 hostid is loaded into the kernel by modloading the sysinit kernel module. Note that loading this value into the kernel in Solaris is different than how it is done on Linux systems. On Linux systems the hostid is not loaded into the kernel and is just read from the /etc/hostid file; if nothing in the file, the hostid is created by basing it on the machine's IP address with some bit fiddling done on that. To minimize changes and therefore decrease risk of breakage we decided to keep loading the hostid into the kernel; it is used in a few places in the kernel (see below section detailing where HW_SERIAL is used in kernel). Also since Sparc loads the hostid into the kernel we thought it best to also continue loading hostid into kernel on X86/64. When determining how to load it into the kernel a few options were explored: creating a new system call, adding an ioctl in a new pseudo driver, and creating a new ioctl in some existing pseudo driver. After discussion with PSARC, a method similar to rtc_config was investigated and seemed straight forward, easy to implement, and minimum chance of breakage. In usr/src/uts/i86pc/os/startup.c, the rtc_config file is read by the kernel in startup_modules(), the same function that currently modloads sysinit. We would just replace the modloading of the sysinit module in this routine with a function to read /etc/hostid file. If there is no /etc/hostid file or it is corrupted, the kernel will generate a new hostid else it will read the hostid value from /etc hostid. We debated whether we should store the hostid in SMF repository or in a file. We decided to store it in a file as we load hostid early on in the boot process and the SMF repository isn't available until much later in booting. We confirmed with the SMF team that there are no kernel routines to read the SMF repository early in boot. However, this might be possible in the future for the kernel to read SMF repository early in the boot- 6545020 "kernel access to repository data." However we will still make use of an SMF service that will check to see if there is a /etc/hostid on system. If not, query the kernel and write the hostid out to /etc/hostid. We deleay the writing of /etc/hostid to disk until the root file system is mounted read/write, which is when SMF is run. The SMF service will also log the hostid to /var/log/syslog to maintain a very elementary audit log. Besides ON changes there are changes needed in the Install consolidation. When you boot the Solaris CD/DVD, the atconfig binary is run that searches the disk for kernel/misc/sysinit file and copies this to /tmp/root/kernel/misc/sysinit so the svc_serial.c code from Suninstall can try to preserve the hostid during an upgrade and even an install. We need to modify this code to also search for /etc/hostid and preserve it. However, for older systems we also need to search for kernel/misc/sysinit for those systems that have hostid stored there, and convert the hostid in sysinit into a hostid in /etc/hostid. There was no contract between ON and Install for sysinit; we now have a contract in place between the two consolidations for Install's use of /kernel/misc/sysinit and /etc/hostid. Possible future work: 1) Zones Currently, there is one hostid per machine and we did not want to break that with this new /etc/hostid implementation. The global zone on an X86/64 box will have a /etc/hostid file but not the other non-global zones. Since there is one kernel shared among all the zones and the hostid on X86/64 boxes is currently tied to the kernel, all zones on a box will have the same hostid. Note on Sparc systems the hostid is the same on all zones on a box. There has been requests for allowing different hostids per zone due to zone consolidation and once again, for licensing reasons. That will not be addressed in this project as we will not change the current behavior of hostid wrt zones. However, our design of a global /etc/hostid is easy to extend if a later project wants to allow per-zone hostid by having a separate /etc/hostid file in a non-global zone's root file system. 2) Moving hostid into SMF As Solaris moves towards a goal of read only root file system, alot of the configuration files in /etc are being moved into SMF. There are at least two directions on how to move to storing hostid as an SMF property instead of in the file /etc/hostid: 1) Move hostid out of the kernel 2) Keep hostid in the kernel if RFE 6545020 is implemented If removing hostid from kernel we would need to do the following: a) Don't load hostid into kernel upon bootup on Sparc AND X86/64 b) Change all occurances of HW_SERIAL in kernel that use it as a seed for a random number or might be using it as a system identifier c) Make sure that sysinfo(HW_SERIAL) still works if hostid not in kernel d) Change all userland libraries and daemons that use hostid for seed for a random number to not do so or make sure the dependencies in SMF are correct so hostid is available early enough for these userland processes More serious thought needs to be done to determine if hostid should still be in the kernel. Using it as seed to a random number generator can be done easier in other ways (like /dev/uradom) but is using hostid as a "moderately" unique number to identify a system in an enterprise an appropriate use? Talking to zfs team and they are proposing using hostid as a label to zfs device label to determine which system accessed the pool last. Note we will be making changes to ON and Install consolidations. We now have a contract with ON and Install regarding Install's usage of sysinit and future use of /etc/hostid in the atconfig program. ON consolidation changes: Modify startup.c in the kernel to longer modload sysinit but to read /etc/hostid. No longer need usr/src/uts/common/sysinit.c. Even though it is in a common code location, it only is built for x86/64. Remove reference to modloading sysinit in usr/src/uts/i86pc/os/startup.c Remove /kernel/misc/sysinit Remove /kernel/misc/amd64/sysinit Install/Admin consolidation changes: Remove hostid generation code in admin/install consolidation, src/common/lib/libspmisvc/svc_updateserial.c and src/bundled/app/wbem/solaris/provider/com/sun/wbem/ solarisprovide/libsmoss/libsmoss/setser.c Modify atconfig program to look for and save /etc/hostid in addtion to /kernel/misc/sysinit. RELATED HOSTID BUGIDS: 4762207 Fix the hostid generation in x86 6306078 All shipping V20z's have identical hostid 4415508 X86: flash install set same hostid of clone as master 4421570 x86: hardware serial number not based on hardware 4856440 hostid on Solaris X86 needs to setable early in the boot process 6306078 all shipping v20z's have identical hostid 4160584 Support use of something other than hostid to lock software RELATED ARC CASES: PSARC/2002/614 Solaris x86 support for Grizzly platform PSARC/2002/711 Claymore PSARC/2005/745 Sneep KERNEL CODE USING HW_SERIAL: /usr/src/uts/common/fs/nfs/nfs3_srv.c rfs3_srvrinit() /usr/src/uts/common/fs/nfs/nfs3_vnops.c nfs3create() /usr/src/uts/common/os/sunddi.c ddi_devid_init() /usr/src/uts/common/io/lvm/md/md_mddb.c set_dtag() init_set() REFERENCE DOCUMENTS: NSG website for preinstalls: http://swie.west Frank Hofmann's blog about X86 hostids: http://blogs.sun.com/ambiguous/category/OpenSolaris Linux source code: http://www.devdaily.com/scw/c/cygwin/src/newlib/libc/sys/linux/ gethostid.c.shtml ADDTIONAL INFO WITH ZFS: 6282725 hostname/hostid should be stored in the label integrated April 2007 Following kernel files use HW_SERIAL: usr/src/uts/common/fs/zfs/spa.c spa_load() usr/src/uts/common/fs/zfs/spa_config.c spa_config_generate()