From ap25164@jansky.sfbay.sun.com Mon Nov 19 11:39:57 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJJduwj007225 for ; Mon, 19 Nov 2007 11:39:56 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lAJJdq5C025243 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@Sun.Com>; Tue, 20 Nov 2007 03:39:55 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00F03QMIVO00@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Mon, 19 Nov 2007 12:39:54 -0700 (MST) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00EBHQMHLH40@brm-avmta-1.central.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Mon, 19 Nov 2007 12:39:53 -0700 (MST) Received: from jansky.sfbay.sun.com (jansky.SFBay.Sun.COM [192.29.76.72]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAJJdrP4015906 for ; Mon, 19 Nov 2007 11:39:53 -0800 (PST) Received: from jansky.sfbay.sun.com (localhost [127.0.0.1]) by jansky.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lAJJdr97539925; Mon, 19 Nov 2007 11:39:53 -0800 (PST) Received: (from ap25164@localhost) by jansky.sfbay.sun.com (8.14.1+Sun/8.14.1/Submit) id lAJJdqbi539921; Mon, 19 Nov 2007 11:39:52 -0800 (PST) Date: Mon, 19 Nov 2007 11:39:52 -0800 (PST) From: Alan Perry Subject: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] To: PSARC-ext@Sun.Com Message-id: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 5671 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Host Protected Area (HPA) on x86 ATA drives 1.2. Name of Document Author/Supplier: Author: Lawrence Lee 1.3 Date of This Document: 19 November, 2007 4. Technical Description 4.1. Overview This project provides limited support for the Host Protected Area (HPA) feature of ATA disk drives. The intent is to provide enough support for Solaris to co-exist on drives that are either shipped with this feature enabled or set by other OS's. 4.2. Problem Description The Host Protected Area (HPA) is a feature that is optionally implemented in the ATA drive firmware. The HPA feature was defined in ATA revision 5 specification and subsequently modified in the ATA 6 specification so that it would also support 48 bit addresses. The HPA uses a feature in the ATA specification which sets the maximum address on the ATA drive. Any attempt to access disk blocks beyond that address returns an invalid address. Thus creating an area on the disk that is normally safe even from operating systems. Dell has shipped systems with this feature enabled. I have been unable to determine what data is in HPA area, and I have been unable to find anyone at Dell who can explain further. Linux restores the HPA address to the maximum address on the drive, effectively disabling this feature until the next power cycle. We had reports the Windows Vista also behaved this way, but our tests have demonstrated that WinVT does NOT alter or ignore the HPA. The problem for Solaris users is that the HPA feature is restored at the next power cycle and that the disk labels appear to extend beyond the end of the disk. Which Solaris treats as an invalid label and refuses to use the disk until it contains a correct label. Which at present means deleting either the Linux partition or the entire fdisk extended partition. Two additional areas where HPA may be used are: 1) protecting metadata, either for encryption or RAIDs. 2) laptop manufactures may use HPA to protect Recover CD images. 4.3. Proposal This proposal would add support for a driver property HPA to the x86 ata hba driver and implement configuration of this new capability through the ata.conf file. The proposed commitment level of this interface is Committed. The proposed release binding is minor/patch. The proposed driver property defined in the ata.conf file provides a means to specify 1) a default HPA setting which applies to all ATA drives and 2) to selectively assign a setting to individual drives. Drives are selected using the device path found under the /devices tree. This is something that a user can easily obtain and something that will not change as hardware is added/removed from the system. The settings allow any of the following 1) Use the HPA setting that is already in the drive. 2) specify the amount of space in the protected area. 3) specify the amount of space in the unprotected area. 4) The setting will persist across reboots of the OS, and may be set to reset or persist at the next drive power cycle. The HPA property is specified using a property string array 'HPA=' default_value ( ',' device_path ',' device_value )* ';' values are '*' | [ '-' ] number [ 'P' ] '*' means don't change the HPA setting, use exactly as booted. if '-' is specified, the number is disk capacity in sectors and all sector address above number are assigned to the HPA if '-' is omitted, number is size of protected area in sectors. 'P' means keep setting across power cycles Examples: - Set the host protected area to 0 permanently for all ATA disks. Permanently means the setting will persist across power cycles. So Solaris will have access to every sector of the disk. HPA="0P"; - Don't change the HPA setting on all but two disks. Make one disk a 200GB drive (390625000*512 = 200,000,000,000) Give the other disk a 1MB Protected area HPA="*", "pci@0,0/pci-ide@6/ide@0", "-390625000", "pci@0,0/pci-ide@6/ide@1", "2000"; More details/examples are available in the Evaluation section of CR 5044205. Solaris's default behavior will match Windows and will be set in the ata.conf file that is shipped with Solaris. In extraordinary cases the uses may modify the ata.conf properties using a standard text editor. 4.5. References The HPA feature was introduced in the ATA revision 5 specification and modified in the ATA revision 6 to accomodate 48 bit addressing. These are available on the internet or at: /net/hs-uego02-03.west/export/home1/03/lclee ATA5-d1321r3-ATA-ATAPI.pdf ATA6-d1410r3-ATA-ATAPI.pdf ATA7-d1532v1r4b-ATA-ATAPI.pdf 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From gdamore@sun.com Mon Nov 19 11:57:42 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJJvfj7007955 for ; Mon, 19 Nov 2007 11:57:42 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAJJvdL6016230 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 19 Nov 2007 19:57:40 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00907RG3QB00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 11:57:39 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00MRSRG3T4C0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 11:57:39 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAJJvdZV005608 for ; Mon, 19 Nov 2007 11:57:39 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRR00E01R26FH00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 11:57:39 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRR00EWGRG2PLC0@fe-sfbay-09.sun.com>; Mon, 19 Nov 2007 11:57:38 -0800 (PST) Date: Mon, 19 Nov 2007 11:53:05 -0800 From: "Garrett D'Amore" Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Alan Perry Cc: PSARC-ext@sun.com Message-id: <4741E9A1.7050303@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070827) Status: RO Content-Length: 6511 So my questions with this are, how does the end-user who has drives with HPA on them configure them. My main fear here is that we are creating yet another knob, with little explanation, and no automation behind it. But maybe I'm missing something here. If indeed there is no automatic selection behind it, then I fear the end result will just be more confusion and trouble amongst users. It does seem to me that Linux's behavior here is itself a bug. If the HPA has been explicitly configured, then it seems a poor choice to silently clobber it. -- Garrett Alan Perry wrote: > Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > This information is Copyright 2007 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > Host Protected Area (HPA) on x86 ATA drives > 1.2. Name of Document Author/Supplier: > Author: Lawrence Lee > 1.3 Date of This Document: > 19 November, 2007 > 4. Technical Description > > 4.1. Overview > > This project provides limited support for the Host Protected Area > (HPA) feature of ATA disk drives. The intent is to provide enough > support for Solaris to co-exist on drives that are either shipped > with this feature enabled or set by other OS's. > > 4.2. Problem Description > > The Host Protected Area (HPA) is a feature that is optionally > implemented in the ATA drive firmware. The HPA feature was > defined in ATA revision 5 specification and subsequently > modified in the ATA 6 specification so that it would also > support 48 bit addresses. > > The HPA uses a feature in the ATA specification which sets the > maximum address on the ATA drive. Any attempt to access disk > blocks beyond that address returns an invalid address. Thus > creating an area on the disk that is normally safe even from > operating systems. > > Dell has shipped systems with this feature enabled. I have been > unable to determine what data is in HPA area, and I have been > unable to find anyone at Dell who can explain further. > > Linux restores the HPA address to the maximum address on the > drive, effectively disabling this feature until the next > power cycle. We had reports the Windows Vista also behaved > this way, but our tests have demonstrated that WinVT does NOT > alter or ignore the HPA. > > The problem for Solaris users is that the HPA feature is restored > at the next power cycle and that the disk labels appear to extend > beyond the end of the disk. Which Solaris treats as an invalid > label and refuses to use the disk until it contains a correct > label. Which at present means deleting either the Linux partition > or the entire fdisk extended partition. > > Two additional areas where HPA may be used are: > 1) protecting metadata, either for encryption or RAIDs. > 2) laptop manufactures may use HPA to protect Recover CD images. > > 4.3. Proposal > > This proposal would add support for a driver property HPA to the > x86 ata hba driver and implement configuration of this new > capability through the ata.conf file. > > The proposed commitment level of this interface is Committed. > > The proposed release binding is minor/patch. > > The proposed driver property defined in the ata.conf file provides > a means to specify 1) a default HPA setting which applies to all > ATA drives and 2) to selectively assign a setting to individual > drives. > > Drives are selected using the device path found under the > /devices tree. This is something that a user can easily obtain > and something that will not change as hardware is added/removed > from the system. > > The settings allow any of the following > 1) Use the HPA setting that is already in the drive. > 2) specify the amount of space in the protected area. > 3) specify the amount of space in the unprotected area. > 4) The setting will persist across reboots of the OS, and may be > set to reset or persist at the next drive power cycle. > > The HPA property is specified using a property string array > > 'HPA=' default_value ( ',' device_path ',' device_value )* ';' > > values are '*' | [ '-' ] number [ 'P' ] > > '*' means don't change the HPA setting, use exactly as booted. > > if '-' is specified, the number is disk capacity in sectors and > all sector address above number are assigned to the HPA > > if '-' is omitted, number is size of protected area in sectors. > > 'P' means keep setting across power cycles > > Examples: > > - Set the host protected area to 0 permanently for all ATA disks. > Permanently means the setting will persist across power cycles. > So Solaris will have access to every sector of the disk. > > HPA="0P"; > > - Don't change the HPA setting on all but two disks. > Make one disk a 200GB drive (390625000*512 = 200,000,000,000) > Give the other disk a 1MB Protected area > > HPA="*", > "pci@0,0/pci-ide@6/ide@0", "-390625000", > "pci@0,0/pci-ide@6/ide@1", "2000"; > > More details/examples are available in the Evaluation section of > CR 5044205. > > Solaris's default behavior will match Windows and will be set > in the ata.conf file that is shipped with Solaris. In > extraordinary cases the uses may modify the ata.conf > properties using a standard text editor. > > 4.5. References > > The HPA feature was introduced in the ATA revision 5 specification > and modified in the ATA revision 6 to accomodate 48 bit > addressing. > > These are available on the internet or at: > > /net/hs-uego02-03.west/export/home1/03/lclee > ATA5-d1321r3-ATA-ATAPI.pdf > ATA6-d1410r3-ATA-ATAPI.pdf > ATA7-d1532v1r4b-ATA-ATAPI.pdf > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open > > From sommerfeld@sun.com Mon Nov 19 12:10:26 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJKAP7n008426 for ; Mon, 19 Nov 2007 12:10:25 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAJKAOrh021219; Mon, 19 Nov 2007 13:10:25 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00A03S1CAH00@nwk-avmta-2.sfbay.sun.com>; Mon, 19 Nov 2007 12:10:24 -0800 (PST) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00MHAS1CTJE0@nwk-avmta-2.sfbay.sun.com>; Mon, 19 Nov 2007 12:10:24 -0800 (PST) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAJKAN6x064863; Mon, 19 Nov 2007 15:10:23 -0500 (EST) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lAJKAM79006820; Mon, 19 Nov 2007 15:10:23 -0500 (EST) Date: Mon, 19 Nov 2007 15:10:22 -0500 From: Bill Sommerfeld Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <4741E9A1.7050303@sun.com> To: "Garrett D'Amore" Cc: Alan Perry , PSARC-ext@sun.com Message-id: <1195503022.5267.56.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.12.1 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <4741E9A1.7050303@sun.com> Status: RO Content-Length: 861 On Mon, 2007-11-19 at 11:53 -0800, Garrett D'Amore wrote: > My main fear here is that we are creating yet another knob, with little > explanation, and no automation behind it. But maybe I'm missing > something here. If indeed there is no automatic selection behind it, > then I fear the end result will just be more confusion and trouble > amongst users. This is a "let's pretend the disk is smaller" interface. I think this needs to be visible, and probably also settable, through at least format(1m); ata.conf is not the right granularity for an administrative interface. one other data point: if I'm not mistaken at least some TiVo DVRs shipped with this value (or at least some very similar drive-shrinking parameter) set on its internal drive. I had to code up an implementation for NetBSD to back up my ancient TiVo Series 1 hard drive). From Alan.Perry@sun.com Mon Nov 19 12:25:26 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJKPQa0008809 for ; Mon, 19 Nov 2007 12:25:26 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAJKPN7X005502; Mon, 19 Nov 2007 12:25:26 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00J09SQDLG00@brm-avmta-1.central.sun.com>; Mon, 19 Nov 2007 13:25:25 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00J0KSQCLE00@brm-avmta-1.central.sun.com>; Mon, 19 Nov 2007 13:25:24 -0700 (MST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAJKPO0P029687; Mon, 19 Nov 2007 12:25:24 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRR00601SGE8P00@fe-sfbay-09.sun.com> (original mail from Alan.Perry@Sun.COM); Mon, 19 Nov 2007 12:25:24 -0800 (PST) Received: from [10.0.0.2] ([75.165.11.54]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRR00BU9SQ80090@fe-sfbay-09.sun.com>; Mon, 19 Nov 2007 12:25:21 -0800 (PST) Date: Mon, 19 Nov 2007 12:25:11 -0800 From: Alan Perry Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <1195503022.5267.56.camel@thunk> Sender: Alan.Perry@sun.com To: Bill Sommerfeld Cc: "Garrett D'Amore" , Alan Perry , PSARC-ext@sun.com, Larry Lee Message-id: <4741F127.6000906@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <4741E9A1.7050303@sun.com> <1195503022.5267.56.camel@thunk> User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) Status: RO Content-Length: 1017 Nothing to add to the discussion. Just wanted to get the submitter on the Reply-all list. Bill Sommerfeld wrote: >On Mon, 2007-11-19 at 11:53 -0800, Garrett D'Amore wrote: > > >>My main fear here is that we are creating yet another knob, with little >>explanation, and no automation behind it. But maybe I'm missing >>something here. If indeed there is no automatic selection behind it, >>then I fear the end result will just be more confusion and trouble >>amongst users. >> >> > >This is a "let's pretend the disk is smaller" interface. > >I think this needs to be visible, and probably also settable, through at >least format(1m); ata.conf is not the right granularity for an >administrative interface. > >one other data point: if I'm not mistaken at least some TiVo DVRs >shipped with this value (or at least some very similar drive-shrinking >parameter) set on its internal drive. I had to code up an >implementation for NetBSD to back up my ancient TiVo Series 1 hard >drive). > > > > > From gdamore@sun.com Mon Nov 19 12:37:37 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJKbadt009773 for ; Mon, 19 Nov 2007 12:37:36 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAJKbS6v010260; Mon, 19 Nov 2007 20:37:35 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00K03TAMK100@brm-avmta-1.central.sun.com>; Mon, 19 Nov 2007 13:37:34 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00J09TALLC30@brm-avmta-1.central.sun.com>; Mon, 19 Nov 2007 13:37:33 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAJKbXnb001022; Mon, 19 Nov 2007 12:37:33 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRR00M01T9PWG00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) ; Mon, 19 Nov 2007 12:37:33 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRR00BRYTAGQ7D0@fe-sfbay-10.sun.com>; Mon, 19 Nov 2007 12:37:29 -0800 (PST) Date: Mon, 19 Nov 2007 12:32:55 -0800 From: "Garrett D'Amore" Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <4741F127.6000906@sun.com> Sender: Garrett.Damore@sun.com To: Alan Perry Cc: Bill Sommerfeld , Alan Perry , PSARC-ext@sun.com, Larry Lee Message-id: <4741F2F7.4050802@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <4741E9A1.7050303@sun.com> <1195503022.5267.56.camel@thunk> <4741F127.6000906@sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070827) Status: RO Content-Length: 2096 Alan Perry wrote: > Nothing to add to the discussion. Just wanted to get the submitter on > the Reply-all list. > > Bill Sommerfeld wrote: > >> On Mon, 2007-11-19 at 11:53 -0800, Garrett D'Amore wrote: >> >> >>> My main fear here is that we are creating yet another knob, with >>> little explanation, and no automation behind it. But maybe I'm >>> missing something here. If indeed there is no automatic selection >>> behind it, then I fear the end result will just be more confusion >>> and trouble amongst users. >>> >> >> This is a "let's pretend the disk is smaller" interface. >> I think this needs to be visible, and probably also settable, through at >> least format(1m); ata.conf is not the right granularity for an >> administrative interface. >> one other data point: if I'm not mistaken at least some TiVo DVRs >> shipped with this value (or at least some very similar drive-shrinking >> parameter) set on its internal drive. I had to code up an >> implementation for NetBSD to back up my ancient TiVo Series 1 hard >> drive). >> >> I still believe that Linux's behavior in this regard is fundamentally a bug. Is there a way to inquire/detect the fact that the HPA has been temporarily disabled and will be restored on reboot? If so, then perhaps the ata driver can refuse to report the HPA area to the operating system, regardless of whether it is enabled or not? Since the details of the HPA are recorded in flash, then I believe if we did the above thing (detect temporary disabling of HPA and treat it as if the HPA were still active), then we'd be in good shape across reboots. Now it may also be true that there is a desire to manage the presence of the persistent HPA as well. I believe that should, as Bill points out, be handled in format(1m), ideally. I do not believe that retaining state in driver.conf files for this is ever likely to be the best answer, since that is really just a second copy of the state that the firmware is already keeping (and the potential for problems when the two copies disagree then arises.) -- Garrett From James.McPherson@sun.com Mon Nov 19 13:07:50 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJL7nMf011350 for ; Mon, 19 Nov 2007 13:07:49 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAJL7kfI028510 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 19 Nov 2007 21:07:48 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR00C01UP0YM00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 13:07:48 -0800 (PST) Received: from sineb-mail-1.sun.com ([192.18.19.6]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00B24UOY9S20@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 13:07:47 -0800 (PST) Received: from fe-apac-03.sun.com (fe-apac-03.sun.com [192.18.19.174] (may be forged)) by sineb-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id lAJL7kCi014446 for ; Mon, 19 Nov 2007 21:07:46 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0JRR00F01RZEVM00@mail-apac.sun.com> (original mail from James.McPherson@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 05:07:46 +0800 (SGT) Received: from [10.7.251.162] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0JRR008PJUOVXD83@mail-apac.sun.com>; Tue, 20 Nov 2007 05:07:46 +0800 (SGT) Date: Tue, 20 Nov 2007 07:07:41 +1000 From: "James C. McPherson" Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> Sender: James.McPherson@sun.com To: Alan Perry Cc: PSARC-ext@sun.com, Larry Lee Reply-to: James.McPherson@sun.com Message-id: <4741FB1D.7030400@Sun.COM> Organization: Sun Microsystems MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=UTF-8 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071023) Status: RO Content-Length: 1097 Alan Perry wrote: > Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > This information is Copyright 2007 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > Host Protected Area (HPA) on x86 ATA drives > 1.2. Name of Document Author/Supplier: > Author: Lawrence Lee > 1.3 Date of This Document: > 19 November, 2007 ... > Examples: ... > - Don't change the HPA setting on all but two disks. > Make one disk a 200GB drive (390625000*512 = 200,000,000,000) > Give the other disk a 1MB Protected area > > HPA="*", > "pci@0,0/pci-ide@6/ide@0", "-390625000", > "pci@0,0/pci-ide@6/ide@1", "2000"; > > More details/examples are available in the Evaluation section of > CR 5044205. Hi Alan and Larry, how is the customer supposed to know that they should use a signed integer for the length/size field? James C. McPherson -- Senior Kernel Software Engineer, Solaris Sun Microsystems http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog From Alan.Perry@sun.com Mon Nov 19 13:23:06 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAJLN6ig011586 for ; Mon, 19 Nov 2007 13:23:06 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAJLN30q028967 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 19 Nov 2007 13:23:06 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRR0010JVEG7V00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 19 Nov 2007 14:23:04 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRR00JSDVEFLCC0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 19 Nov 2007 14:23:04 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAJLN3ZF006198 for ; Mon, 19 Nov 2007 13:23:03 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRR00501V9HVK00@fe-sfbay-10.sun.com> (original mail from Alan.Perry@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Mon, 19 Nov 2007 13:23:03 -0800 (PST) Received: from [10.0.0.2] ([75.165.11.54]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRR007MAVE5E800@fe-sfbay-10.sun.com>; Mon, 19 Nov 2007 13:22:54 -0800 (PST) Date: Mon, 19 Nov 2007 13:22:44 -0800 From: Alan Perry Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <4741FB1D.7030400@Sun.COM> Sender: Alan.Perry@sun.com To: James.McPherson@sun.com Cc: Alan Perry , PSARC-ext@sun.com, Larry Lee Message-id: <4741FEA4.1040605@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=UTF-8 Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <4741FB1D.7030400@Sun.COM> User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) Status: RO Content-Length: 1463 James C. McPherson wrote: > Alan Perry wrote: > >> Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI >> This information is Copyright 2007 Sun Microsystems >> 1. Introduction >> 1.1. Project/Component Working Name: >> Host Protected Area (HPA) on x86 ATA drives >> 1.2. Name of Document Author/Supplier: >> Author: Lawrence Lee >> 1.3 Date of This Document: >> 19 November, 2007 > > ... > >> Examples: > > ... > >> - Don't change the HPA setting on all but two disks. >> Make one disk a 200GB drive (390625000*512 = >> 200,000,000,000) >> Give the other disk a 1MB Protected area >> >> HPA="*", >> "pci@0,0/pci-ide@6/ide@0", "-390625000", >> "pci@0,0/pci-ide@6/ide@1", "2000"; >> >> More details/examples are available in the Evaluation section of >> CR 5044205. > > > Hi Alan and Larry, > how is the customer supposed to know that they should use a > signed integer for the length/size field? The value is not signed. The syntax description of the HPA property appears just above where it says "Examples:" in the Technical Description. It says "if '-' is specified, the number is disk capacity in sectors and all sector address[es] above [the] number are assigned to the HPA." The details of the HPA property would be added to the ata(7D) man page, which is how the customer would find out how to use it. alan From Lawrence.Lee@sun.com Mon Nov 19 18:54:41 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAK2sfkc022493 for ; Mon, 19 Nov 2007 18:54:41 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAK2seQh001330 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 19 Nov 2007 18:54:41 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRS00703AR5UD00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 18:54:41 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRS006KMAR400D0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 18:54:40 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAK2seYK007097 for ; Mon, 19 Nov 2007 18:54:40 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRS00J01ALOJS00@fe-sfbay-10.sun.com> (original mail from Lawrence.Lee@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 19 Nov 2007 18:54:40 -0800 (PST) Received: from [10.1.48.100] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRS00ADOAR4TDD0@fe-sfbay-10.sun.com>; Mon, 19 Nov 2007 18:54:40 -0800 (PST) Date: Mon, 19 Nov 2007 18:52:45 -0800 From: Lawrence Lee Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> Sender: Lawrence.Lee@sun.com To: Alan Perry Cc: PSARC-ext@sun.com Message-id: <47424BFD.2020709@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 5465 I do appreciate all of your input. Please see my responses below. Larry gdamore@sun.com > So my questions with this are, how does the end-user who has drives with HPA on them configure them. > > My main fear here is that we are creating yet another knob, with little explanation, and no automation behind it. > But maybe I'm missing something here. If indeed there is no automatic selection behind it, then I fear the end > result will just be more confusion and trouble amongst users. > > It does seem to me that Linux's behavior here is itself a bug. > If the HPA has been explicitly configured, then it seems a poor choice to silently clobber it. I agree that the the Linux behavior should be regarded as a bug, but the Linux folks obviously went to some effort to add the code to disable the feature and removing that code would cripple some of their installed user base. I doubt that they would remove it for us. The bug was originally reported in 2004, at that time a workaround was provided which was to use a utility (DLG toolkit) that was downloadable from one of the disk manufacturers (see bugster for details) that ran on DOS/Windows that allowed the user to set/control the feature. That utility is no longer available, so the workaround no longer works. Recently the install group has run into problem again and raised the priority to P1. So we needed to address it. It was initially believed that WinVT had the same behavior as Linux. The WinVT problems actually turned out to be other errors in fdisk. It has been demonstrated that WinVT uses the HPA as is. sommerfeld@sun.com > This is a "let's pretend the disk is smaller" interface. > > I think this needs to be visible, and probably also settable, through at > least format(1m); ata.conf is not the right granularity for an > administrative interface. See section 4.1 and 4.2 What you are suggesting is certainly possible, but there would be significant impact and risk with changing a drives capacity after the disk labels have been read and capacity is not stored in both the hba, target drivers, and filesystems on that drive. Controlling this feature at attach time has far less risk and the effort is more in line with the frequency of the problems occurence. This is a workaround for a real, but rarely scene problem. I don't see that is sufficient justification for the amount of work required to implement a new ioctl and implement the functionality in format. > > Is there a way to inquire/detect the fact that the HPA has been temporarily > disabled and will be restored on reboot? If so, then perhaps the ata driver > can refuse to report the HPA area to the operating system, regardless of > whether it is enabled or not? We can certainly tell when the drives currently reported capacity is different from it's native capacity. There isn't any way to tell what the setting will be after the next power cycle (except by power cycling the disk). We are not necessarily just switching between a zero value and a non zero value. There is a current setting (which may be any value) and a power on setting which may be any value. So the current setting might be HPA=100 and the power on might be 200. > > Since the details of the HPA are recorded in flash, then I believe if we did the > above thing (detect temporary disabling of HPA and treat it as if the HPA were still active), > then we'd be in good shape across reboots. Yes but.... The power on setting can't be determined, only the current setting. There are other consequences/impacts of not having all OS's on a disk agree on the size of a disk. Example GPT labels write the backup label in the last 33 sectors of the disk if our perceived end of disk was actually in the middle of a linux partition we wouldn't be very nice neighbors. 1.5 TB drives are due in 2008, fdisk has a max capacity of 1TB unless you're really brave about ignoring the sign it in the LBA field. so expect an increase the usage of GPT labels over fdisk labels. There is also an impact on the use of partitions in an fdisk extended partition, when the extended partition extends beyond the end of the disk. > Now it may also be true that there is a desire to manage the presence of the persistent HPA as well. Please keep in mind that this is a rarely seen problem. I do not expect most users to use or even know of the feature, but there are messages written by the driver in /var/adm/messages and there is sufficient documentation in our (proposed) manpage and on the internet in general. If a user is confronted with the problem, they should be able to easily find the documentation and change the drives setting. My expectation is that when a user actually uses this feature they will almost always use HPA="0P"; to permanently disable the HPA and recover normal disk function. Once they have booted the system with that setting, it can be removed from the ata.conf file. Would you like me to emphasize that in the documentation? I am not proposing that we develop Solaris features based on HPA, only that when we are confronted with a disk that has the feature set, that we provide our users with a method of dealing with it. James.McPherson@Sun.COM > Hi Alan and Larry, > how is the customer supposed to know that they should use a > signed integer for the length/size field? See the proposed manpage (bug 6625517) See the limited comments in ata.conf file Larry From Alan.Perry@sun.com Wed Nov 28 10:11:38 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lASIBbxY029651 for ; Wed, 28 Nov 2007 10:11:37 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lASIBGB8007185 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 29 Nov 2007 02:11:36 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JS800M0RAJ8QI00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 28 Nov 2007 10:11:32 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JS800M92AJ78010@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Wed, 28 Nov 2007 10:11:31 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lASIBVce010245 for ; Wed, 28 Nov 2007 10:11:31 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JS800601AFTTH00@fe-sfbay-09.sun.com> (original mail from Alan.Perry@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 28 Nov 2007 10:11:31 -0800 (PST) Received: from [10.0.0.7] ([75.165.11.54]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JS800E35AJ6HXF0@fe-sfbay-09.sun.com> for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Wed, 28 Nov 2007 10:11:31 -0800 (PST) Date: Wed, 28 Nov 2007 10:11:17 -0800 From: Alan Perry Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <47424BFD.2020709@sun.com> Sender: Alan.Perry@sun.com To: Lawrence Lee Cc: PSARC-ext@sun.com Message-id: <474DAF45.3030800@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <47424BFD.2020709@sun.com> User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) Status: RO Content-Length: 5782 This was approved at today's PSARC meeting. Lawrence Lee wrote: > I do appreciate all of your input. > Please see my responses below. > > Larry > > gdamore@sun.com > > So my questions with this are, how does the end-user who has drives > with HPA on them configure them. > > > > My main fear here is that we are creating yet another knob, with > little explanation, and no automation behind it. > > But maybe I'm missing something here. If indeed there is no > automatic selection behind it, then I fear the end > > result will just be more confusion and trouble amongst users. > > > > It does seem to me that Linux's behavior here is itself a bug. > > If the HPA has been explicitly configured, then it seems a poor > choice to silently clobber it. > > I agree that the the Linux behavior should be regarded as a bug, but > the Linux folks > obviously went to some effort to add the code to disable the feature > and removing that code would cripple some of their installed user base. > I doubt that they would remove it for us. > > The bug was originally reported in 2004, at that time a workaround was > provided > which was to use a utility (DLG toolkit) that was downloadable from > one of the > disk manufacturers (see bugster for details) that ran on DOS/Windows > that allowed > the user to set/control the feature. > That utility is no longer available, so the workaround no longer works. > > Recently the install group has run into problem again and raised the > priority to P1. > So we needed to address it. It was initially believed that WinVT had > the same behavior as Linux. > The WinVT problems actually turned out to be other errors in fdisk. > It has been demonstrated that WinVT uses the HPA as is. > > > sommerfeld@sun.com > > This is a "let's pretend the disk is smaller" interface. > > > > I think this needs to be visible, and probably also settable, > through at > > least format(1m); ata.conf is not the right granularity for an > > administrative interface. > > See section 4.1 and 4.2 > > What you are suggesting is certainly possible, but there would be > significant > impact and risk with changing a drives capacity after the disk labels > have been > read and capacity is not stored in both the hba, target drivers, and > filesystems > on that drive. Controlling this feature at attach time has far less risk > and the effort is more in line with the frequency of the problems > occurence. > > This is a workaround for a real, but rarely scene problem. I don't > see that is > sufficient justification for the amount of work required to implement > a new ioctl > and implement the functionality in format. > > > > > Is there a way to inquire/detect the fact that the HPA has been > temporarily > > disabled and will be restored on reboot? If so, then perhaps the > ata driver > > can refuse to report the HPA area to the operating system, > regardless of > > whether it is enabled or not? > > We can certainly tell when the drives currently reported capacity is > different from > it's native capacity. > There isn't any way to tell what the setting will be after the next > power cycle > (except by power cycling the disk). > > We are not necessarily just switching between a zero value and a non > zero value. > There is a current setting (which may be any value) and a power on > setting which > may be any value. So the current setting might be HPA=100 and the > power on might be 200. > > > > > Since the details of the HPA are recorded in flash, then I believe > if we did the > > above thing (detect temporary disabling of HPA and treat it as if > the HPA were still active), > > then we'd be in good shape across reboots. > > Yes but.... > The power on setting can't be determined, only the current setting. > > There are other consequences/impacts of not having all OS's on a disk > agree on the size of a disk. > > Example GPT labels write the backup label in the last 33 sectors of > the disk > if our perceived end of disk was actually in the middle of a linux > partition > we wouldn't be very nice neighbors. > > 1.5 TB drives are due in 2008, fdisk has a max capacity of 1TB > unless you're really brave about ignoring the sign it in the LBA field. > so expect an increase the usage of GPT labels over fdisk labels. > > There is also an impact on the use of partitions in an fdisk extended > partition, > when the extended partition extends beyond the end of the disk. > > > Now it may also be true that there is a desire to manage the > presence of the persistent HPA as well. > > Please keep in mind that this is a rarely seen problem. > I do not expect most users to use or even know of the feature, > but there are messages written by the driver in /var/adm/messages > and there is sufficient documentation in our (proposed) manpage > and on the internet in general. If a user is confronted with the > problem, they should be able to easily find the documentation > and change the drives setting. > > My expectation is that when a user actually uses this feature > they will almost always use HPA="0P"; to permanently disable > the HPA and recover normal disk function. Once they have booted > the system with that setting, it can be removed from the ata.conf file. > Would you like me to emphasize that in the documentation? > > I am not proposing that we develop Solaris features based on HPA, > only that when we are confronted with a disk that has the feature > set, that we provide our users with a method of dealing with it. > > James.McPherson@Sun.COM > > Hi Alan and Larry, > > how is the customer supposed to know that they should use a > > signed integer for the length/size field? > > See the proposed manpage (bug 6625517) > See the limited comments in ata.conf file > > Larry From Lawrence.Lee@Sun.COM Wed Nov 28 10:56:33 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lASIuW91002795 for ; Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lASIuWpO002047 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JS800K05CM8VM00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JS8009BMCM86X60@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lASIuWln017320 for ; Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JS800H01C9K1W00@fe-sfbay-10.sun.com> (original mail from Lawrence.Lee@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 28 Nov 2007 10:56:32 -0800 (PST) Received: from [192.168.1.2] ([75.84.8.184]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JS800G5NCLZICD0@fe-sfbay-10.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 28 Nov 2007 10:56:23 -0800 (PST) Date: Wed, 28 Nov 2007 10:57:18 -0800 From: Lawrence Lee Subject: Re: Host Protected Area (HPA) on x86 ATA drives [PSARC/2007/660 FastTrack timeout 11/28/2007] In-reply-to: <474DAF45.3030800@sun.com> Sender: Lawrence.Lee@Sun.COM To: Alan Perry Cc: PSARC-ext@Sun.COM Message-id: <474DBA0E.7030606@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711191939.lAJJdqbi539921@jansky.sfbay.sun.com> <47424BFD.2020709@sun.com> <474DAF45.3030800@sun.com> User-Agent: Thunderbird 1.5.0.5 (X11/20060828) Status: RO Content-Length: 6013 Thanks, I had called in and was listening. Larry Alan Perry wrote: > This was approved at today's PSARC meeting. > > Lawrence Lee wrote: > >> I do appreciate all of your input. >> Please see my responses below. >> >> Larry >> >> gdamore@sun.com >> > So my questions with this are, how does the end-user who has drives >> with HPA on them configure them. >> > >> > My main fear here is that we are creating yet another knob, with >> little explanation, and no automation behind it. >> > But maybe I'm missing something here. If indeed there is no >> automatic selection behind it, then I fear the end >> > result will just be more confusion and trouble amongst users. >> > >> > It does seem to me that Linux's behavior here is itself a bug. >> > If the HPA has been explicitly configured, then it seems a poor >> choice to silently clobber it. >> >> I agree that the the Linux behavior should be regarded as a bug, but >> the Linux folks >> obviously went to some effort to add the code to disable the feature >> and removing that code would cripple some of their installed user base. >> I doubt that they would remove it for us. >> >> The bug was originally reported in 2004, at that time a workaround >> was provided >> which was to use a utility (DLG toolkit) that was downloadable from >> one of the >> disk manufacturers (see bugster for details) that ran on DOS/Windows >> that allowed >> the user to set/control the feature. >> That utility is no longer available, so the workaround no longer works. >> >> Recently the install group has run into problem again and raised the >> priority to P1. >> So we needed to address it. It was initially believed that WinVT had >> the same behavior as Linux. >> The WinVT problems actually turned out to be other errors in fdisk. >> It has been demonstrated that WinVT uses the HPA as is. >> >> >> sommerfeld@sun.com >> > This is a "let's pretend the disk is smaller" interface. >> > >> > I think this needs to be visible, and probably also settable, >> through at >> > least format(1m); ata.conf is not the right granularity for an >> > administrative interface. >> >> See section 4.1 and 4.2 >> >> What you are suggesting is certainly possible, but there would be >> significant >> impact and risk with changing a drives capacity after the disk labels >> have been >> read and capacity is not stored in both the hba, target drivers, and >> filesystems >> on that drive. Controlling this feature at attach time has far less >> risk >> and the effort is more in line with the frequency of the problems >> occurence. >> >> This is a workaround for a real, but rarely scene problem. I don't >> see that is >> sufficient justification for the amount of work required to implement >> a new ioctl >> and implement the functionality in format. >> >> > >> > Is there a way to inquire/detect the fact that the HPA has been >> temporarily >> > disabled and will be restored on reboot? If so, then perhaps the >> ata driver >> > can refuse to report the HPA area to the operating system, >> regardless of >> > whether it is enabled or not? >> >> We can certainly tell when the drives currently reported capacity is >> different from >> it's native capacity. >> There isn't any way to tell what the setting will be after the next >> power cycle >> (except by power cycling the disk). >> >> We are not necessarily just switching between a zero value and a non >> zero value. >> There is a current setting (which may be any value) and a power on >> setting which >> may be any value. So the current setting might be HPA=100 and the >> power on might be 200. >> >> > >> > Since the details of the HPA are recorded in flash, then I believe >> if we did the >> > above thing (detect temporary disabling of HPA and treat it as if >> the HPA were still active), >> > then we'd be in good shape across reboots. >> >> Yes but.... >> The power on setting can't be determined, only the current setting. >> >> There are other consequences/impacts of not having all OS's on a disk >> agree on the size of a disk. >> >> Example GPT labels write the backup label in the last 33 sectors of >> the disk >> if our perceived end of disk was actually in the middle of a linux >> partition >> we wouldn't be very nice neighbors. >> >> 1.5 TB drives are due in 2008, fdisk has a max capacity of 1TB >> unless you're really brave about ignoring the sign it in the LBA field. >> so expect an increase the usage of GPT labels over fdisk labels. >> >> There is also an impact on the use of partitions in an fdisk extended >> partition, >> when the extended partition extends beyond the end of the disk. >> >> > Now it may also be true that there is a desire to manage the >> presence of the persistent HPA as well. >> >> Please keep in mind that this is a rarely seen problem. >> I do not expect most users to use or even know of the feature, >> but there are messages written by the driver in /var/adm/messages >> and there is sufficient documentation in our (proposed) manpage >> and on the internet in general. If a user is confronted with the >> problem, they should be able to easily find the documentation >> and change the drives setting. >> >> My expectation is that when a user actually uses this feature >> they will almost always use HPA="0P"; to permanently disable >> the HPA and recover normal disk function. Once they have booted >> the system with that setting, it can be removed from the ata.conf file. >> Would you like me to emphasize that in the documentation? >> >> I am not proposing that we develop Solaris features based on HPA, >> only that when we are confronted with a disk that has the feature >> set, that we provide our users with a method of dealing with it. >> >> James.McPherson@Sun.COM >> > Hi Alan and Larry, >> > how is the customer supposed to know that they should use a >> > signed integer for the length/size field? >> >> See the proposed manpage (bug 6625517) >> See the limited comments in ata.conf file >> >> Larry > >