From Mark.Maybee@Sun.COM Wed Apr 4 11:28:35 2007 Received: from sfbaymail1sca.SFBay.Sun.COM (sfbaymail1sca.SFBay.Sun.COM [129.145.154.35]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l34ISZYH028194 for ; Wed, 4 Apr 2007 11:28:35 -0700 (PDT) Received: from brmea-mail-3.sun.com (brmea-mail-3.Sun.COM [192.18.98.34]) by sfbaymail1sca.SFBay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l34IS2XV004309 for ; Wed, 4 Apr 2007 11:28:02 -0700 (PDT) Received: from fe-amer-02.sun.com ([192.18.108.176]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l34IS2qv026159 for ; Wed, 4 Apr 2007 18:28:02 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JFZ00201KJMBS00@mail-amer.sun.com> (original mail from Mark.Maybee@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Wed, 04 Apr 2007 12:28:02 -0600 (MDT) Received: from [172.20.25.54] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JFZ00CU0KMP7D50@mail-amer.sun.com>; Wed, 04 Apr 2007 12:28:02 -0600 (MDT) Date: Wed, 04 Apr 2007 12:27:43 -0600 From: Mark Maybee Subject: ZFS hotplug [PSARC/2007/197 Timeout: 04/11/2007] Sender: Mark.Maybee@Sun.COM To: psarc-ext@sac.sfbay.sun.com Cc: Eric Schrock Message-id: <4613EE1F.3050604@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT User-Agent: Thunderbird 1.5.0.5 (X11/20060730) Status: RO Content-Length: 17091 Subject: PSARC FastTrack [04/11/2007]: ZFS hotplug Template Version: @(#)sac_nextcase 1.57 03/26/07 SMI This information is Copyright 2007 Sun Microsystems, Inc. 1. Introduction 1.1. Project/Component Working Name: ZFS hotplug 1.2. Name of Document Author/Supplier: Author: Eric Schrock 1.3 Date of This Document: 03 April, 2007 4. Technical Description The requested changes introduced in this case have a stability of committed and a release binding of patch/micro. Also, see the following thread for the Opensolaris discussion on this topic: http://www.opensolaris.org/jive/thread.jspa?threadID=26720 4.1. INTRODUCTION Currently, ZFS supports what is affectionately known as "poor man's hotplug". If a device is removed from the system, then it is assumed that upon I/O failure, an attempt to reopen the same device will fail. This will trigger a FMA fault, substituting a hot spare if available. This is undesirable for two reasons: - There is no distinction between device removal and arbitrary failure. If a device is removed from the system, it should be treated as a deliberate action different from normal failure. - There is no support for automatic response to device insertion. For a server configured with a ZFS pool, the administrator should be able to walk up, remove any drive (preferably a faulted one), insert a new drive, and not have to issue any ZFS commands to reconfigure the pool. This is particularly true for the appliance space, where hardware reconfiguration should "just work". This case enhances ZFS to respond to device removal and provides a mechanism to automatically deal with device insertion. While the framework is generic, the primary target is devices supported by the SATA framework. The only device-specific portion of this proposal concerns determining if a device is in the same "physical location" as a previously known device, and involves correlating a transport's enumeration of the device with the device's physical location within the chassis. 4.2. DEVICE REMOVAL There are two types of device removal within Solaris. Coordinated device removal involves stopping all consumers of the device, using the appropriate cfgadm(1M) command (PSARC 1996/285), and then physically removing the device. Uncoordinated removal (also known as "surprise removal") is when a device is physically removed while still in active use by the system. The latter increasingly common as more I/O protocols support hotplug and higher level software (ZFS) becomes more capable. There are several ways to detect device removal within Solaris. Fibre channel drivers generate the NDI events FCAL_INSERT_EVENT and FCAL_REMOVE_EVENT. USB and 1394 drivers generate the NDI events DDI_DEVI_INSERT_EVENT and DDI_DEVI_REMOVE_EVENT. In addition to these event channels, there is also the DKIOCSTATE ioctl() which returns (on capable drivers) DKIO_DEV_GONE if the device has been removed. Of these, the ioctl() is the most widely supported, and is the mechanism used as part of this case. Since this is an implementation detail of the current architecture, it does not preclude using alternate mechanisms in the future. When an I/O to a disk fails, ZFS will query the media state by the DKIOCSTATE ioctl. If the device is any state other than DKIO_INSERTED, ZFS will transition the device to a new REMOVED state. No FMA fault will be triggered, and a hot spare (if any) will be substituted if available. Note that the DKIO_DEV_GONE can be returned for a variety of reasons (pulling cables, external chassis being powered off, etc). In the absence of additional FMA information, it is assumed that this is intentional administrative action. As part of this work, lofiadm(1M) will be expanded to include a new force (-f) flag when removing devices. Combined with the upcoming lofi devfs events (PSARC 2006/709), this will provide a much simpler testing framework without the need for physical hardware interaction. When this flag is used, the underlying file will be closed, any further I/O or attempts to open the device will fail, and DKIOCSTATE will return DKIO_DEV_GONE. This flag will remain private for testing only, and will not be documented. An example of this in action: # lofiadm -a /disk/a /dev/lofi/1 # lofiadm -a /disk/b /dev/lofi/2 # lofiadm -a /disk/c /dev/lofi/3 # zpool create -f test mirror /dev/lofi/1 /dev/lofi/2 spare /dev/lofi/3 # while :; do touch /test/foo; sync; sleep 1; done & [1] 100662 # zpool status pool: test state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 mirror ONLINE 0 0 0 /dev/lofi/1 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 AVAIL errors: No known data errors # lofiadm -d /disk/a -f # zpool status pool: test state: DEGRADED scrub: resilver completed with 0 errors on Mon Mar 12 10:57:43 2007 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 mirror DEGRADED 0 0 0 spare DEGRADED 0 0 0 /dev/lofi/1 REMOVED 0 0 0 /dev/lofi/3 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 INUSE currently in use errors: No known data errors This behavior is universal for all pools, and cannot be disabled. If a device doesn't support DKIOCSTATE, then it will be diagnosed as faulty through the standard FMA mechanisms. The 'REMOVED' state is persistent, but only if the device is not present when the system is rebooted. If a new device is inserted, then the vdev will appear in the FAULTED state when the system comes up. 4.3. DEVICE INSERTION When a device is inserted, there are two possible outcomes of interest to ZFS: - If a previously known device is inserted, then we want to online the device. - If a new device is inserted into a physical location that previously contained a ZFS device, then we want to format the device and replace the original device. The former is applicable to any pool, and is always enabled. The latter is potentially damaging, as it will automatically overwrite any data present on newly inserted devices. To protect against this, a new pool property (PSARC 2006/577), 'autoreplace', will be defined. This boolean property will be off by default to minimize the impact on existing systems or unknown hardware. If unset, the current behavior remains the same, and any replacement operation must be initiated by the administrator via zpool(1M). When set, it indicates that any new device found in the same physical location as a device previously belonging to the pool will be automatically formatted and replaced. To ensure consistent behavior, ZFS must behave in the same manner when the device is replaced (via hotplug) while the system is running, as well as when the device is replaced while the system is powered off. 4.4. ONLINE DEVICE INSERTION A new syseventd module will be introduced that listens for EC_DEV_ADD events of subclass ESC_DISK or ESC_LOFI. This event is triggered when the device node for the disk or lofi device is created, not necessarily when a disk is inserted. Currently, the USB framework auto-configures drives on insertion, while the SATA framework does not. Modifying the SATA framework behavior is outside the scope of this case. In the meantime, these SATA events will be triggered only by an explicit 'cfgadm -c configure' by the user. When one of these events is received, the corresponding device path is derived from the sysevent payload. For disks, this will be the device node, while for lofi it will be a particular minor node. If the device has a devid, then we first search all pools for a vdev with a matching devid. If none is found, or the device does not have a devid, then we search all pools for vdevs with the specified device path. As part of this work, the ZFS configuration will be expanded to store the physical device path as part of the vdev label. This will also have the benefit of allowing ZFS to boot from devices which don't support devids. Currently, ZFS only identifies by devid or /dev path, neither of which may be available when mounting the root filesystem. This simplistic mechanism will only work for devices which have the behavior that the device path identifies a physical location, which may not be true for FC, SAS, SCSI, or iSCSI devices, or for devices plumbed under MPxIO. This logic can be altered or expanded in the future, and may leverage libtopo for this purpose. If no matching vdevs are found, then the event is ignored and nothing is done. Otherwise, the device is onlined to determine if it is a known ZFS device. This online operation will automatically remove any attached spare when the resilver is complete. To continue the above example: # zpool status pool: test state: DEGRADED scrub: resilver completed with 0 errors on Mon Mar 12 10:57:43 2007 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 mirror DEGRADED 0 0 0 spare DEGRADED 0 0 0 /dev/lofi/1 REMOVED 0 0 0 /dev/lofi/3 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 INUSE currently in use errors: No known data errors # lofiadm -a /disk/a /dev/lofi/1 # zpool status pool: test state: ONLINE scrub: resilver completed with 0 errors on Mon Mar 12 10:58:22 2007 config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 mirror ONLINE 0 0 0 /dev/lofi/1 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 AVAIL errors: No known data errors If the online attempt failed, then we are dealing with a new device inserted into the same physical slot. If the 'autoreplace' property is unset, then the event is ignored. If the original event was ESC_DISK and the vdev is not a whole disk, then the event is also ignored. Otherwise, the disk is labeled with an EFI label in the same manner as when the pool is initially created. If that succeeds, then the corresponding 'zpool replace' command is automatically invoked. To continue the above example: # lofiadm -d /disk/a -f # zpool status pool: test state: DEGRADED scrub: resilver completed with 0 errors on Mon Mar 12 10:57:43 2007 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 mirror DEGRADED 0 0 0 spare DEGRADED 0 0 0 /dev/lofi/1 REMOVED 0 0 0 /dev/lofi/3 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 INUSE currently in use errors: No known data errors # lofiadm -a /disk/d /dev/lofi/1 # zpool status pool: test state: DEGRADED scrub: resilver completed with 0 errors on Mon Mar 12 17:31:06 2007 config: NAME STATE READ WRITE CKSUM test DEGRADED 0 0 0 mirror DEGRADED 0 0 0 spare DEGRADED 0 0 0 replacing DEGRADED 0 0 0 /dev/lofi/1/old FAULTED 0 0 0 corrupted data /dev/lofi/1 ONLINE 0 0 0 /dev/lofi/3 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 INUSE currently in use errors: No known data errors # zpool status pool: test state: ONLINE scrub: resilver completed with 0 errors on Mon Mar 12 17:31:06 2007 config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 mirror ONLINE 0 0 0 /dev/lofi/1 ONLINE 0 0 0 /dev/lofi/2 ONLINE 0 0 0 spares /dev/lofi/3 AVAIL errors: No known data errors In this case, the device was automatically replaced with the next contents. If the device replace fails (because the device was too small, for example), the original device will remain in the faulted state and a FMA fault will be generated. 4.5. OFFLINE DEVICE INSERTION If a device is replaced while the system is powered off, then ZFS should behave in a similar manner. If devices change attachment points (i.e. swapped) while the system is powered off, ZFS already handles this case for devices which support devids. If a device can be opened but the devid doesn't match, then ZFS will treat this as a disk insertion event. If the 'autoreplace' property is set, then ZFS will label the disk and perform the appropriate 'zpool replace' operation to resilver the device. 4.6. HOT SPARES Currently, ZFS does not do any I/O to inactive hot spares, so it is incapable of detecting when a hot spare is removed from the system. This case will modify ZFS to periodically attempt to read from all hot spares and make sure they are online and available. If a hot spare is removed, then when this I/O fails it will trigger the normal remove path. This case will also allow offline hot spares to be replaced. With these changes, hot spares will be treated as normal devices with respect to hotplug. If an active hot spare is removed, then the hot spare will be detached and marked removed. If another hot spare is available, then it will be substituted in its place. If a hot spare is inserted, and there is a faulted device with no current hot spare, then inserting the device will automatically trigger a hot spare. 4.7. MANPAGE DIFFS The exact changes for pool properties have not yet been integrated, so it's difficult to generate exact diffs. A new section, "Hot Plug Behavior", will be added: Hot Plug Behavior ZFS recognizes device removal and insertion for hotplug-capable hardware. In the event of device removal, ZFS will mark the corresponding vdev as removed, substituting a hot spare if available. If the same device is re-inserted, either in the same location or elsewhere in the system, then ZFS will automatically online the device, removing any hot spare when complete. If a new device is inserted into the same physical location, then by default the vdev will transition to the faulted state. Alternatively, the administrator can set the 'autoreplace' pool property. If this property is set, then ZFS will automatically label the new disk and attempt to replace the original device with the newly inserted device, regardless of the contents of the device (unless the device has already been identified as a known ZFS device as described above). The ability to identify the physical location of the device is platform-dependent, and may not be supported on all platforms or devices. The 'autoreplace' behavior also applies to device replacement while the system is powered off. If a device is replaced while the system is powered down, and the 'autoreplace' property is set, then device replacement will automatically be triggered when the system powers on. Both forms of autoreplacement only work for whole disks (those labeled by ZFS and displayed without a corresponding slice). Automatic formatting and replacement of particular disk slices is not supported. The following text will be added under the new "Properties" section: autoreplace= Controls whether ZFS attempts to automatically replace devices inserted to the same physical slot as a previous vdev. For more information, see the section "Hot Plug Behavior", above. Defaults to 'off'. 4.8. REFERENCES PSARC 1996/285 Dynamic Attach/Detach of CPU/Memory Boards PSARC 2002/240 ZFS PSARC 2006/223 ZFS Hot Spares PSARC 2006/577 zpool property to disable delegation PSARC 2006/709 lofi devfs events 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack From glenn.skinner@sun.com Wed Apr 4 16:40:57 2007 Received: from ivrel.sfbay.sun.com (ivrel [129.146.74.76]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l34Nev7O006897 for ; Wed, 4 Apr 2007 16:40:57 -0700 (PDT) Received: from ivrel (ivrel [129.146.74.76]) by ivrel.sfbay.sun.com (8.13.8+Sun/8.13.8) with SMTP id l34NcIO5022296; Wed, 4 Apr 2007 16:38:18 -0700 (PDT) Message-Id: <200704042338.l34NcIO5022296@ivrel.sfbay.sun.com> Date: Wed, 4 Apr 2007 16:38:18 -0700 (PDT) From: Glenn Skinner Reply-To: Glenn Skinner Subject: Re: 2007/197 [ZFS hotplug] To: psarc-ext@sac.sfbay.sun.com, Mark.Maybee@Sun.COM Cc: eric.schrock@Sun.COM MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Daq9UCX+0vQKG5B+oR1Irg== X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.6_36 SunOS 5.11 sun4u sparc Status: RO Content-Length: 1188 Date: Wed, 04 Apr 2007 12:27:43 -0600 From: Mark Maybee Subject: ZFS hotplug [PSARC/2007/197 Timeout: 04/11/2007] ... 4.4. ONLINE DEVICE INSERTION ... When one of these events is received, the corresponding device path is derived from the sysevent payload. For disks, this will be the device node, while for lofi it will be a particular minor node. If the device has a devid, then we first search all pools for a vdev with a matching devid. If none is found, or the device does not have a devid, then we search all pools for vdevs with the specified device path. As part of this work, the ZFS configuration will be expanded to store the physical device path as part of the vdev label. This will also have the benefit of allowing ZFS to boot from devices which don't support devids. Currently, ZFS only identifies by devid or /dev path, neither of which may be available when mounting the root filesystem. Does expanding the stored ZFS configuration information imply that the ZFS on disk version number must be bumped? I didn't see that mentioned as an affected interface... -- Glenn From eschrock@zion.eng.sun.com Wed Apr 4 18:29:49 2007 Received: from zion.eng.sun.com (zion [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l351TnEg008997 for ; Wed, 4 Apr 2007 18:29:49 -0700 (PDT) Received: from zion.eng.sun.com (localhost [127.0.0.1]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l351TI5T001334; Wed, 4 Apr 2007 18:29:18 -0700 (PDT) Received: (from eschrock@localhost) by zion.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id l351TInT001333; Wed, 4 Apr 2007 18:29:18 -0700 (PDT) Date: Wed, 4 Apr 2007 18:29:18 -0700 From: Eric Schrock To: Glenn Skinner Cc: psarc-ext@sac.sfbay.sun.com, Mark.Maybee@sun.com Subject: Re: 2007/197 [ZFS hotplug] Message-ID: <20070405012918.GA1273@eng.sun.com> References: <200704042338.l34NcIO5022296@ivrel.sfbay.sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200704042338.l34NcIO5022296@ivrel.sfbay.sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1890 No, this doesn't require the on-disk version number to be increased. This doesn't introduce any new objects outside of the existing vdev label, and the newly added properties (persistently removed and device path) are benign on older systems. If you import a newer pool onto an old system, the end result is that these new properties will be ignored (and subsequently omitted when written out on the old host). Since ignoring them has no ill effect, there is no need to bump the on-disk version. - Eric On Wed, Apr 04, 2007 at 04:38:18PM -0700, Glenn Skinner wrote: > Date: Wed, 04 Apr 2007 12:27:43 -0600 > From: Mark Maybee > Subject: ZFS hotplug [PSARC/2007/197 Timeout: 04/11/2007] > > ... > 4.4. ONLINE DEVICE INSERTION > > ... > When one of these events is received, the corresponding device path is > derived from the sysevent payload. For disks, this will be the device > node, while for lofi it will be a particular minor node. If the device > has a devid, then we first search all pools for a vdev with a matching > devid. If none is found, or the device does not have a devid, then we > search all pools for vdevs with the specified device path. As part of > this work, the ZFS configuration will be expanded to store the physical > device path as part of the vdev label. This will also have the benefit > of allowing ZFS to boot from devices which don't support devids. > Currently, ZFS only identifies by devid or /dev path, neither of which > may be available when mounting the root filesystem. > > Does expanding the stored ZFS configuration information imply that the > ZFS on disk version number must be bumped? > > I didn't see that mentioned as an affected interface... > > -- Glenn -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock From sommerfeld@sun.com Thu Apr 5 12:50:18 2007 Received: from eastmail2bur.East.Sun.COM (eastmail2bur.East.Sun.COM [129.148.13.40]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l35JoItN029958 for ; Thu, 5 Apr 2007 12:50:18 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by eastmail2bur.East.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l35JniAe000646; Thu, 5 Apr 2007 15:49:44 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l35JniC9005403; Thu, 5 Apr 2007 15:49:44 -0400 (EDT) Subject: Re: ZFS hotplug [PSARC/2007/197 Timeout: 04/11/2007] From: Bill Sommerfeld To: Mark Maybee Cc: psarc-ext@sac.sfbay.sun.com, Eric Schrock In-Reply-To: <4613EE1F.3050604@Sun.COM> References: <4613EE1F.3050604@Sun.COM> Content-Type: text/plain Date: Thu, 05 Apr 2007 15:49:42 -0400 Message-Id: <1175802582.4423.54.camel@thunk> Mime-Version: 1.0 X-Mailer: Evolution 2.8.1.1 Content-Transfer-Encoding: 7bit Status: RO Content-Length: 884 I asked a variant of this question when the case was proposed on zfs-discuss and don't see any discussion of this issue in the case spec so I'll ask a hopefully more focussed question for the case record. As I understand it, zfs will in some cases automatically grow a vdev when the underlying devices get bigger. At present this is an irreversable action, as pool capacity cannot be reduced. What's the interaction between automatic growing, automatic device removal, and automatic device replacement? What I'd assume: - automatic removal leaves behind a placeholder which remembers the size of the removed disk, preventing automatic growing from happening. - automatic replacement will grow the vdev and the pool if the new device is bigger than the old device and the other devices making up the vdev are all bigger than the old device. Is this correct? - Bill From eschrock@zion.eng.sun.com Thu Apr 5 12:58:10 2007 Received: from zion.eng.sun.com (zion [129.146.17.75]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l35JwAAe000584 for ; Thu, 5 Apr 2007 12:58:10 -0700 (PDT) Received: from zion.eng.sun.com (localhost [127.0.0.1]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id l35JvbHN022493; Thu, 5 Apr 2007 12:57:37 -0700 (PDT) Received: (from eschrock@localhost) by zion.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id l35JvbpP022492; Thu, 5 Apr 2007 12:57:37 -0700 (PDT) Date: Thu, 5 Apr 2007 12:57:37 -0700 From: Eric Schrock To: Bill Sommerfeld Cc: Mark Maybee , psarc-ext@sac.sfbay.sun.com Subject: Re: ZFS hotplug [PSARC/2007/197 Timeout: 04/11/2007] Message-ID: <20070405195736.GA22276@eng.sun.com> References: <4613EE1F.3050604@Sun.COM> <1175802582.4423.54.camel@thunk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1175802582.4423.54.camel@thunk> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1164 On Thu, Apr 05, 2007 at 03:49:42PM -0400, Bill Sommerfeld wrote: > I asked a variant of this question when the case was proposed on > zfs-discuss and don't see any discussion of this issue in the case spec > so I'll ask a hopefully more focussed question for the case record. > > As I understand it, zfs will in some cases automatically grow a vdev > when the underlying devices get bigger. At present this is an > irreversable action, as pool capacity cannot be reduced. > > What's the interaction between automatic growing, automatic device > removal, and automatic device replacement? > > What I'd assume: > > - automatic removal leaves behind a placeholder which remembers the > size of the removed disk, preventing automatic growing from happening. > > - automatic replacement will grow the vdev and the pool if the new > device is bigger than the old device and the other devices making up the > vdev are all bigger than the old device. > > Is this correct? Yes, both of the above statements are correct. Sorry for not clarifying this in the case materials. - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock From Mark.Maybee@Sun.COM Wed Apr 11 10:26:41 2007 Received: from sfbaymail2sca.sfbay.sun.com (sfbaymail2sca.SFBay.Sun.COM [129.145.155.42]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l3BHQfo6023153 for ; Wed, 11 Apr 2007 10:26:41 -0700 (PDT) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by sfbaymail2sca.sfbay.sun.com (8.13.6+Sun/8.12.10/ENSMAIL,v2.2) with ESMTP id l3BHQ4mS011571 for ; Wed, 11 Apr 2007 10:26:04 -0700 (PDT) Received: from fe-amer-04.sun.com ([192.18.108.178]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id l3BHQ3Kl024223 for ; Wed, 11 Apr 2007 17:26:03 GMT Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JGC00G01GELBL00@mail-amer.sun.com> (original mail from Mark.Maybee@Sun.COM) for psarc-ext@sac.sfbay.sun.com; Wed, 11 Apr 2007 11:26:03 -0600 (MDT) Received: from [172.20.25.54] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JGC00JD0GFF3TG1@mail-amer.sun.com>; Wed, 11 Apr 2007 11:26:03 -0600 (MDT) Date: Wed, 11 Apr 2007 11:25:39 -0600 From: Mark Maybee Subject: PSARC FastTrack 2007/197: approved Sender: Mark.Maybee@Sun.COM To: Eric Schrock , psarc-ext@sac.sfbay.sun.com Message-id: <461D1A13.4020201@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT User-Agent: Thunderbird 1.5.0.5 (X11/20060730) Status: RO Content-Length: 42 Approved during PSARC business 04/11/2007