From tpm@lp64.sfbay.sun.com Tue Nov 20 20:40:01 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAL4e1wh023253 for ; Tue, 20 Nov 2007 20:40:01 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAL4e0HR007075 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 20 Nov 2007 20:40:00 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRU00K05AAOBD00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 21:40:00 -0700 (MST) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRU00GL8AAOWBA0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 21:40:00 -0700 (MST) Received: from lp64.sfbay.sun.com (lp64.SFBay.Sun.COM [129.146.228.105]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lAL4dxiL064570; Tue, 20 Nov 2007 20:39:59 -0800 (PST) Received: from lp64.sfbay.sun.com (localhost [127.0.0.1]) by lp64.sfbay.sun.com (8.14.0+Sun/8.14.0) with ESMTP id lAL4RTMZ008761; Tue, 20 Nov 2007 20:27:29 -0800 (PST) Received: (from tpm@localhost) by lp64.sfbay.sun.com (8.14.0+Sun/8.14.0/Submit) id lAL4RT1u008757; Tue, 20 Nov 2007 20:27:29 -0800 (PST) Date: Tue, 20 Nov 2007 20:27:29 -0800 (PST) From: Tim Marsland Subject: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] To: PSARC-ext@Sun.COM Cc: nils.nieuwejaar@Sun.COM Message-id: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 9344 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI 1. Introduction 1.1. Project/Component Working Name: Paravirtualized Drivers for Fully Virtualized xVM Domains 1.2. Name of Document Author/Supplier: Author: Nils Nieuwejaar 1.3 Date of This Document: 04 November, 2007 4. Technical Description 4.1 Summary The Xen hypervisor and the port of Solaris to that hypervisor are described in PSARC/2006/260. That case is considered required pre-reading for this. The Solaris port to the Xen hypervisor included disk and network drivers for paravirtualized (PV) domUs. These PV drivers provide standard disk and network interfaces to Solaris, but communicate with back-end drivers in the control domain (dom0) rather than interacting directly with hardware. When running in a fully virtualized (HVM) domain, Solaris continues to use standard device drivers. These drivers believe they are interacting with hardware, but are actually interacting with emulated devices that run in dom0. These emulated physical devices deliver significantly lower performance than the corresponding virtual devices in a PV domain. For example, the emulated network device may run at 1/10th the speed of a virtual device. This project proposes to provide the infrastructure necessary to allow the existing Solaris PV drivers to function in HVM domains, delivering significantly higher performance to applications running in those domains. We intend to backport these PV drivers to Solaris 10, enabling it to be a performant guest operating system. Therefore, we are requesting micro/patch binding for this project. 4.2 Proposal The project is comprised of several components. There is one new driver that provides the infrastructure necessary to allow software in an HVM domain to communicate with the hypervisor and the control domain. There are also minor modifications to the existing drivers, that will allow them to plug into the new infrastructure. 4.2.1 xpv xpv is a new leaf driver that binds to Xen's PCI ID (pci5853,1). This driver provides all the plumbing needed for PV drivers to drill through the bottom of a fully virtualized domain, and interact directly with the hypervisor. The driver uses HVM-specific hypercalls to: o map in the 'Xen shared info' page, which is used for communication between the hypervisor and OS. o map pages for grant tables, which are used to enable temporary transfers of page ownership between domains o register a single interrupt with the hypervisor, which it will trigger when there are events pending for the OS to process. After initialization, the driver's primary responsibility is to receive emulated 'physical' interrupts from the hypervisor when events are pending, and then forward those event notifications to the correct PV driver(s). 4.2.2 Modified Drivers cmdk is an existing Solaris driver that provides a common framework for block devices. This driver presents a simple high-level disk model to Solaris while abstracting away the distinctions between disks, CD drives, etc. This project modifies cmdk to be PV-aware. When loaded in an HVM domain, this driver uses the Layered Driver framework to forward all disk requests to the PV xdf driver rather than to the standard driver(s) for emulated physical devices. This forwarding mechanism allows us to add the PV drivers after a domain has already been installed, without having to modify the path to the boot device. In addition, the paravirtualized xdf, xnf, xpvd, and xenbus drivers must all be modified to register event handlers with a new 'xpv' driver, rather than registering interrupt handlers with Solaris. Other than this event delivery mechanism, all of a driver's interaction with the hypervisor and control plane is exactly the same in both PV and HVM domains. The scope of these changes is on the order of about a dozen lines per driver. 4.3 User Experience 4.3.1 Background Xen has adopted two very different models for supporting PV devices in HVM domains. These models are built into the architecture of Xen itself, and are not specific to Sun's implementation. For block devices, each device configured in a domU has two interfaces: an emulated ATA device on the PCI bus, and an explicit PV interface accessible through the Xenbus. The two interfaces provide access to the same storage on the back end. For network devices, each NIC is configured in dom0 to be either PV or HVM. If a device is configured to be HVM, Xen provides an emulated RealTek NIC on the PCI bus. If it is configured to be PV, then it provides an explicit PV interface on the Xenbus. A single NIC cannot be configured in both modes at once. 4.3.2 Initial Installation When the PV drivers are integrated into the WOS, they will function as any other block or network driver. When a domU is installed, the disks will automatically and transparently be accessed using the PV interfaces. Users will install to /dev/dsk/cXdYsZ as they do now. All the normal tools, such as format, prtvtoc, and fdisk will work as expected. When the installer probes for network devices, any PV NICs configured into the domain will be made available as xnf0 through xnf, just as (for example) Intel NICS are made available as e1000g0 through e1000g. 4.3.3 Upgrades or post-install additions When the PV drivers are added to an existing domain, our modifications to the cmdk driver will allow the PV disk drivers to fall into place transparently. When the driver is first installed, the system will continue to access disks using the existing emulated metal interfaces. When the domain is rebooted, the disks will automatically be accessed through the PV interfaces. The domain's bootpath remains unchanged, so the transition from HVM to PV should be completely transparent to the user. For network devices, however, the user experience will be less satisfying. As a NIC must be explicitly configured in dom0 to be either PV or HVM, simply installing the PV drivers will not deliver PV performance to the user. The domU must be reconfig- ured to contain PV NICs rather than HVM. When the domain is reconfigured, the 'brand' of the NIC changes from rtls (i.e. "RealTek") to xnf. Since Solaris binds the identity of a system to its NIC name, the user will have to sys-unconfig and reconfigure the domain. A natural question is: can't we transparently remap the rtls devices to xnf devices under the hood (as we do for disks), making this reconfiguration step go away. Unfortunately, no. Whether a NIC is configured to be PV or HVM is not simply a cosmetic issue. When Xen instantiates an HVM NIC for a domain, it does not create a back-end device that is capable of interacting with a front-end PV driver. Even if the domU had sufficient information about the back-end configuration to understand how this remapping should be done, there is simply nothing in the dom0 for our PV drivers to talk to. While this upgrade experience is certainly less smooth than one would hope, it is not dissimilar to the experience of upgrading physical hardware. Replacing a disk in a physical machine is a simple process, and the device naming is not affected by the brand of the hard drive. Replacing an Intel NIC with a Broadcom NIC, on other hand, requires exactly the same sys-unconfig as that required by replacing a fully virtualized NIC with a paravirtualized NIC. 4.4 Interface Table Imported Interfaces --------------------------------------------------------------------- | Interface Level Comments |-------------------------------------------------------------------- | Layered Driver Evolving | Framework | | Xen Interfaces External | Hypercall interface | Shared page format | Xenbus protocol | | PCI ID pci5853,1 External Xen's registered | PCI ID |-------------------------------------------------------------------- Exported Interfaces --------------------------------------------------------------------- | Interface Level Comments |-------------------------------------------------------------------- | ec_bind_evtchn_to_handler() Project Used by the PV | ec_unbind_evtchn() private drivers to plug into | hypervisor_mask_event() the new PV-on-HVM | hypervisor_unmask_event() framework. |-------------------------------------------------------------------- 4.4 References [1] PSARC 2006/260: Solaris on Xen [2] PSARC 2001/769: LDF: Layered Driver Framework 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From tim.marsland@sun.com Tue Nov 20 20:40:36 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAL4eah6023265 for ; Tue, 20 Nov 2007 20:40:36 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAL4eZYJ004340; Tue, 20 Nov 2007 20:40:35 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRU00K01ABMER00@brm-avmta-1.central.sun.com>; Tue, 20 Nov 2007 21:40:34 -0700 (MST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRU00G27ABMW9B0@brm-avmta-1.central.sun.com>; Tue, 20 Nov 2007 21:40:34 -0700 (MST) Received: from [10.0.1.6] (vpn-129-150-16-35.SFBay.Sun.COM [129.150.16.35]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lAL4eXn6573404; Tue, 20 Nov 2007 20:40:34 -0800 (PST) Date: Tue, 20 Nov 2007 20:40:31 -0800 From: Tim Marsland Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> To: PSARC-ext@sun.com Cc: Tim Marsland , Nils Nieuwejaar Message-id: <71C4C2B2-4151-4A2F-BB15-B0336C09C128@sun.com> MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> Status: RO Content-Length: 435 (Fumbled sac_nextcase, sorry) I'm sponsoring the following fast-track for Nils Nieuwejaar. It's basically to allow Solaris running as an otherwise fully virtualized guest on the xVM hypervisor to have higher performance disk and network device drivers. Release binding is 'patch' (because we want to deliver these drivers changes for S10 HVM guests) The timeout is 11/30/07 to allow for mpk17 shutdown and the holidays. tim From gdamore@sun.com Tue Nov 20 21:30:27 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAL5URoY023946 for ; Tue, 20 Nov 2007 21:30:27 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lAL5UQim062268 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 20 Nov 2007 22:30:26 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRU00K09CMQ0700@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 21:30:26 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRU00BTQCMQAO80@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 21:30:26 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lAL5UPib016531 for ; Tue, 20 Nov 2007 21:30:25 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRU00401CJO5B00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Tue, 20 Nov 2007 21:30:25 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRU0012OCMPYUB0@fe-sfbay-10.sun.com>; Tue, 20 Nov 2007 21:30:25 -0800 (PST) Date: Tue, 20 Nov 2007 21:25:46 -0800 From: "Garrett D'Amore" Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> Sender: Garrett.Damore@sun.com To: Tim Marsland Cc: PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <4743C15A.8030008@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070827) Status: RO Content-Length: 10520 Hmm... I wonder if paravirtualization would work better with something like my blk2scsa framework (see PSARC 2007/654). Alternatively, maybe a separate block driver rather than modifying cmdk? The idea that we have to touch each driver that needs to support paravirtualization seems kind of clumsy to me. A lot of the complexity in extant drivers is to deal with various hardware bugs and workarounds, and I fear that this approach just further complicates already complex drivers, and maybe also robs PV drivers of some their performance as well? But it sounds, from the materials below, like maybe for cmdk's specific case this isn't too bad, and there maybe upgrade issues that are avoided by this approach? -- Garrett Tim Marsland wrote: > Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > > 1. Introduction > 1.1. Project/Component Working Name: > Paravirtualized Drivers for Fully Virtualized xVM Domains > > > 1.2. Name of Document Author/Supplier: > Author: Nils Nieuwejaar > > 1.3 Date of This Document: > 04 November, 2007 > > 4. Technical Description > > 4.1 Summary > > The Xen hypervisor and the port of Solaris to that hypervisor are > described in PSARC/2006/260. That case is considered required > pre-reading for this. > > The Solaris port to the Xen hypervisor included disk and network > drivers for paravirtualized (PV) domUs. These PV drivers provide > standard disk and network interfaces to Solaris, but communicate > with back-end drivers in the control domain (dom0) rather than > interacting directly with hardware. > > When running in a fully virtualized (HVM) domain, Solaris continues > to use standard device drivers. These drivers believe they are > interacting with hardware, but are actually interacting with > emulated devices that run in dom0. These emulated physical devices > deliver significantly lower performance than the corresponding > virtual devices in a PV domain. For example, the emulated network > device may run at 1/10th the speed of a virtual device. > > This project proposes to provide the infrastructure necessary to > allow the existing Solaris PV drivers to function in HVM domains, > delivering significantly higher performance to applications running > in those domains. > > We intend to backport these PV drivers to Solaris 10, enabling it > to be a performant guest operating system. Therefore, we are > requesting micro/patch binding for this project. > > 4.2 Proposal > > The project is comprised of several components. There is one new > driver that provides the infrastructure necessary to allow software > in an HVM domain to communicate with the hypervisor and the control > domain. There are also minor modifications to the existing > drivers, that will allow them to plug into the new infrastructure. > > 4.2.1 xpv > > xpv is a new leaf driver that binds to Xen's PCI ID > (pci5853,1). This driver provides all the plumbing needed for > PV drivers to drill through the bottom of a fully virtualized > domain, and interact directly with the hypervisor. > > The driver uses HVM-specific hypercalls to: > > o map in the 'Xen shared info' page, which is used for > communication between the hypervisor and OS. > > o map pages for grant tables, which are used to enable > temporary transfers of page ownership between domains > > o register a single interrupt with the hypervisor, which it > will trigger when there are events pending for the OS to > process. > > After initialization, the driver's primary responsibility is to > receive emulated 'physical' interrupts from the hypervisor when > events are pending, and then forward those event notifications > to the correct PV driver(s). > > 4.2.2 Modified Drivers > > cmdk is an existing Solaris driver that provides a common > framework for block devices. This driver presents a simple > high-level disk model to Solaris while abstracting away the > distinctions between disks, CD drives, etc. > > This project modifies cmdk to be PV-aware. When loaded in an > HVM domain, this driver uses the Layered Driver framework to > forward all disk requests to the PV xdf driver rather than to > the standard driver(s) for emulated physical devices. This > forwarding mechanism allows us to add the PV drivers after a > domain has already been installed, without having to modify the > path to the boot device. > > In addition, the paravirtualized xdf, xnf, xpvd, and xenbus > drivers must all be modified to register event handlers with a > new 'xpv' driver, rather than registering interrupt handlers > with Solaris. Other than this event delivery mechanism, all of > a driver's interaction with the hypervisor and control plane is > exactly the same in both PV and HVM domains. > > The scope of these changes is on the order of about a dozen > lines per driver. > > 4.3 User Experience > > 4.3.1 Background > > Xen has adopted two very different models for supporting PV > devices in HVM domains. These models are built into the > architecture of Xen itself, and are not specific to Sun's > implementation. > > For block devices, each device configured in a domU has two > interfaces: an emulated ATA device on the PCI bus, and an > explicit PV interface accessible through the Xenbus. The two > interfaces provide access to the same storage on the back end. > > For network devices, each NIC is configured in dom0 to be > either PV or HVM. If a device is configured to be HVM, Xen > provides an emulated RealTek NIC on the PCI bus. If it is > configured to be PV, then it provides an explicit PV interface > on the Xenbus. A single NIC cannot be configured in both modes > at once. > > 4.3.2 Initial Installation > > When the PV drivers are integrated into the WOS, they will > function as any other block or network driver. > > When a domU is installed, the disks will automatically and > transparently be accessed using the PV interfaces. Users will > install to /dev/dsk/cXdYsZ as they do now. All the normal > tools, such as format, prtvtoc, and fdisk will work as > expected. > > When the installer probes for network devices, any PV NICs > configured into the domain will be made available as xnf0 > through xnf, just as (for example) Intel NICS are made > available as e1000g0 through e1000g. > > 4.3.3 Upgrades or post-install additions > > When the PV drivers are added to an existing domain, our > modifications to the cmdk driver will allow the PV disk drivers > to fall into place transparently. When the driver is first > installed, the system will continue to access disks using the > existing emulated metal interfaces. When the domain is > rebooted, the disks will automatically be accessed through the > PV interfaces. The domain's bootpath remains unchanged, so the > transition from HVM to PV should be completely transparent to > the user. > > For network devices, however, the user experience will be less > satisfying. As a NIC must be explicitly configured in dom0 to > be either PV or HVM, simply installing the PV drivers will not > deliver PV performance to the user. The domU must be reconfig- > ured to contain PV NICs rather than HVM. When the domain is > reconfigured, the 'brand' of the NIC changes from rtls (i.e. > "RealTek") to xnf. Since Solaris binds the identity of a > system to its NIC name, the user will have to sys-unconfig and > reconfigure the domain. > > A natural question is: can't we transparently remap the rtls > devices to xnf devices under the hood (as we do for disks), > making this reconfiguration step go away. Unfortunately, no. > > Whether a NIC is configured to be PV or HVM is not simply a > cosmetic issue. When Xen instantiates an HVM NIC for a domain, > it does not create a back-end device that is capable of > interacting with a front-end PV driver. Even if the domU had > sufficient information about the back-end configuration to > understand how this remapping should be done, there is simply > nothing in the dom0 for our PV drivers to talk to. > > While this upgrade experience is certainly less smooth than one > would hope, it is not dissimilar to the experience of > upgrading physical hardware. Replacing a disk in a physical > machine is a simple process, and the device naming is not > affected by the brand of the hard drive. Replacing an Intel > NIC with a Broadcom NIC, on other hand, requires exactly the > same sys-unconfig as that required by replacing a fully > virtualized NIC with a paravirtualized NIC. > > 4.4 Interface Table > > Imported Interfaces > --------------------------------------------------------------------- > | Interface Level Comments > |-------------------------------------------------------------------- > | Layered Driver Evolving > | Framework > | > | Xen Interfaces External > | Hypercall interface > | Shared page format > | Xenbus protocol > | > | PCI ID pci5853,1 External Xen's registered > | PCI ID > |-------------------------------------------------------------------- > > Exported Interfaces > --------------------------------------------------------------------- > | Interface Level Comments > |-------------------------------------------------------------------- > | ec_bind_evtchn_to_handler() Project Used by the PV > | ec_unbind_evtchn() private drivers to plug into > | hypervisor_mask_event() the new PV-on-HVM > | hypervisor_unmask_event() framework. > |-------------------------------------------------------------------- > > 4.4 References > > [1] PSARC 2006/260: Solaris on Xen > > [2] PSARC 2001/769: LDF: Layered Driver Framework > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open > > From John.Plocher@sun.com Wed Nov 21 02:50:52 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALAoqf0029055 for ; Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALAoq0T022726 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRU00909RGSTZ00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRU00BGZRGSQLB0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALAoq4c023135 for ; Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRU00H01RE3FQ00@fe-sfbay-09.sun.com> (original mail from John.Plocher@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 02:50:52 -0800 (PST) Received: from wp668.local ([208.74.177.212]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRU00BSCRGRP0C0@fe-sfbay-09.sun.com>; Wed, 21 Nov 2007 02:50:52 -0800 (PST) Date: Wed, 21 Nov 2007 02:50:51 -0800 From: John Plocher Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> Sender: John.Plocher@sun.com To: Tim Marsland Cc: PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <47440D8B.8010908@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Status: RO Content-Length: 644 Tim Marsland wrote: > Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > > 1. Introduction > 1.1. Project/Component Working Name: > Paravirtualized Drivers for Fully Virtualized xVM Domains > For network devices, however, the user experience will be less > satisfying. ... Since Solaris binds the identity of a > system to its NIC name, the user will have to sys-unconfig and > reconfigure the domain. sys_unconfig is a pretty big hammer to use for this - isn't it sufficient to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that the user can determine the mapping between the two? -John From carlsonj@phorcys.east.sun.com Wed Nov 21 06:48:23 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALEmNHf001890 for ; Wed, 21 Nov 2007 06:48:23 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lALEm2di026535; Wed, 21 Nov 2007 14:48:19 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0030H2GGLS00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 06:48:16 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV00KA22GFJ690@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 06:48:15 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALElPlj010101; Wed, 21 Nov 2007 09:47:25 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALElP0R010098; Wed, 21 Nov 2007 09:47:25 -0500 (EST) Date: Wed, 21 Nov 2007 09:47:25 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47440D8B.8010908@Sun.Com> To: John Plocher Cc: Tim Marsland , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <18244.17661.406079.436599@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> Status: RO Content-Length: 1403 John Plocher writes: > Tim Marsland wrote: > > Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > > > > 1. Introduction > > 1.1. Project/Component Working Name: > > Paravirtualized Drivers for Fully Virtualized xVM Domains > > > For network devices, however, the user experience will be less > > satisfying. ... Since Solaris binds the identity of a > > system to its NIC name, the user will have to sys-unconfig and > > reconfigure the domain. > > sys_unconfig is a pretty big hammer to use for this - isn't it sufficient > to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that the > user can determine the mapping between the two? Renaming a driver is a serious pain. It's not just /etc/hostname.* that's affected, it's /etc/dhcp.* and potentially many other files such as /etc/ipf/ipf.conf and /etc/ipf/ipnat.conf, /etc/gateways, /etc/sfw/smb.conf, /etc/inet/dhcpsvc.conf, /etc/inet/ndpd.conf, /etc/quagga/*.conf, /etc/zones/*.xml, and probably dozens of others that I'm too lazy to locate at the moment. Basically, as we said with the ipge transition before (2004/782), you can't do it in any non-trivial configuration. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Glenn.Brunette@sun.com Wed Nov 21 07:01:03 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALF12GN002325 for ; Wed, 21 Nov 2007 07:01:02 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lALF03L5007073 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 23:01:01 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0026Z31JY800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:00:55 -0700 (MST) Received: from brmea-mail-3.sun.com ([192.18.98.34]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV0021N31AXS00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:00:46 -0700 (MST) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-3.sun.com (8.13.6+Sun/8.12.9) with ESMTP id lALF0kIN006803 for ; Wed, 21 Nov 2007 15:00:46 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV00J012DWAK00@mail-amer.sun.com> (original mail from Glenn.Brunette@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:00:46 -0700 (MST) Received: from [10.7.250.215] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV00CV3315T280@mail-amer.sun.com>; Wed, 21 Nov 2007 08:00:43 -0700 (MST) Date: Wed, 21 Nov 2007 10:00:54 -0500 From: Glenn Brunette Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <18244.17661.406079.436599@gargle.gargle.HOWL> Sender: Glenn.Brunette@sun.com To: James Carlson Cc: John Plocher , Tim Marsland , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <47444826.1020409@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.4 (X11/20070814) Status: RO Content-Length: 1699 James Carlson wrote: > John Plocher writes: >> Tim Marsland wrote: >>> Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI >>> >>> 1. Introduction >>> 1.1. Project/Component Working Name: >>> Paravirtualized Drivers for Fully Virtualized xVM Domains >>> For network devices, however, the user experience will be less >>> satisfying. ... Since Solaris binds the identity of a >>> system to its NIC name, the user will have to sys-unconfig and >>> reconfigure the domain. >> sys_unconfig is a pretty big hammer to use for this - isn't it sufficient >> to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that the >> user can determine the mapping between the two? > > Renaming a driver is a serious pain. It's not just /etc/hostname.* > that's affected, it's /etc/dhcp.* and potentially many other files > such as /etc/ipf/ipf.conf and /etc/ipf/ipnat.conf, /etc/gateways, > /etc/sfw/smb.conf, /etc/inet/dhcpsvc.conf, /etc/inet/ndpd.conf, > /etc/quagga/*.conf, /etc/zones/*.xml, and probably dozens of others > that I'm too lazy to locate at the moment. > > Basically, as we said with the ipge transition before (2004/782), you > can't do it in any non-trivial configuration. While I agree with you on these points, sys-unconfig will not help with many of these either, right? It sounds like we need a better option than just sys-unconfig - one that could (ideally) understand these relationships or at the very least reset them to default values (saving the original content)? Today, I would argue it is just too easy to do sys-unconfig - follow the rules - and still get fowled up because of other settings that sys-unconfig does not know about. g From carlsonj@phorcys.east.sun.com Wed Nov 21 07:04:05 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALF454c002380 for ; Wed, 21 Nov 2007 07:04:05 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALF43f1038314; Wed, 21 Nov 2007 08:04:04 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0060N36R9G00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 07:04:03 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV00K3P36PJ6E0@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 07:04:02 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALF3CSs010225; Wed, 21 Nov 2007 10:03:12 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALF3CBr010222; Wed, 21 Nov 2007 10:03:12 -0500 (EST) Date: Wed, 21 Nov 2007 10:03:12 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47444826.1020409@sun.com> To: Glenn Brunette Cc: John Plocher , Tim Marsland , PSARC-ext@sun.com, nils.nieuwejaar@sun.com Message-id: <18244.18608.359279.818337@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47444826.1020409@sun.com> Status: RO Content-Length: 1037 Glenn Brunette writes: > While I agree with you on these points, sys-unconfig will not help with > many of these either, right? I would say that ought to, but does not. > It sounds like we need a better option > than just sys-unconfig - one that could (ideally) understand these > relationships or at the very least reset them to default values (saving > the original content)? Today, I would argue it is just too easy to > do sys-unconfig - follow the rules - and still get fowled up because > of other settings that sys-unconfig does not know about. Agreed; sys-unconfig is incomplete. The only good advice we have now is that users need to be extremely cautious when changing driver names, as they may well end up damaging the system and even compromising security (due to non-matching filtering rules). -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From nn35248@negril.east.sun.com Wed Nov 21 07:36:15 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALFaEhs002845 for ; Wed, 21 Nov 2007 07:36:15 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALFaB9l021021; Wed, 21 Nov 2007 07:36:14 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV006094OD2U00@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 08:36:13 -0700 (MST) Received: from negril.east.sun.com ([129.148.183.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV002O24OAXS10@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 08:36:10 -0700 (MST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALFavwF011083; Wed, 21 Nov 2007 10:36:57 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALFavwN011082; Wed, 21 Nov 2007 10:36:57 -0500 (EST) Date: Wed, 21 Nov 2007 10:36:56 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47440D8B.8010908@Sun.Com> To: John Plocher Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <20071121153656.GA11061@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1303 On Wed 11/21/07 at 02:50 AM, John.Plocher@Sun.COM wrote: > Tim Marsland wrote: > >Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > > > >1. Introduction > > 1.1. Project/Component Working Name: > > Paravirtualized Drivers for Fully Virtualized xVM Domains > > > For network devices, however, the user experience will be less > > satisfying. ... Since Solaris binds the identity of a > > system to its NIC name, the user will have to sys-unconfig and > > reconfigure the domain. > > sys_unconfig is a pretty big hammer to use for this - isn't it sufficient > to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that the > user can determine the mapping between the two? Unfortunately, it's not that simple. Under Xen, a network device must be explicitly configured to be either PV or HVM. There is no mapping between PV and HVM devices. A single domain could have both an xnf0 and an rtls0 simultaneously. In such a domain, the two devices could connect to different physical NICs on the back end, and they could be on different subnets. Within the domU, we don't have enough information to draw any inferences about network topology. Any guesses we made about a sensible renaming scheme would likely go wrong under all but the simplest scenarios. Nils From nn35248@negril.east.sun.com Wed Nov 21 07:52:11 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALFqASg002953 for ; Wed, 21 Nov 2007 07:52:10 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lALFkoSp004874; Wed, 21 Nov 2007 23:52:07 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0071J5EJ9T00@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 08:51:55 -0700 (MST) Received: from negril.east.sun.com ([129.148.183.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV0021F5EIXN30@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 08:51:54 -0700 (MST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALFqeHk011124; Wed, 21 Nov 2007 10:52:40 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALFqeaa011123; Wed, 21 Nov 2007 10:52:40 -0500 (EST) Date: Wed, 21 Nov 2007 10:52:40 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <4743C15A.8030008@sun.com> To: "Garrett D'Amore" Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <20071121155240.GB11061@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <4743C15A.8030008@sun.com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 2061 On Tue 11/20/07 at 21:25 PM, gdamore@sun.com wrote: > Hmm... I wonder if paravirtualization would work better with something > like my blk2scsa framework (see PSARC 2007/654). Alternatively, maybe > a separate block driver rather than modifying cmdk? I took a quick look at that case, but it doesn't seem to address the specific issue this was meant to solve. Our decision to modify cmdk was driven by our desire to be able drop these PV drivers onto an existing domain with the least amount of fuss and risk possible. We also wanted to be able to remove the drivers without disabling the domain. When we install an HVM domU without the PV drivers, Solaris believes it is installing to a physical ata device. Thus, the 'cmdk' device name gets hardcoded into the bootpath for that domain. Replacing the cmdk driver with one that is PV-aware allows us to slot the PV disk driver in place, without modifying any delicate files such as /etc/path_to_inst or /boot/solaris/bootenv.rc. It also lets us remove the driver in the future without causing any damage. The old 'metal' driver simply takes over on the next boot. > The idea that we have to touch each driver that needs to support > paravirtualization seems kind of clumsy to me. A lot of the complexity > in extant drivers is to deal with various hardware bugs and workarounds, > and I fear that this approach just further complicates already complex > drivers, and maybe also robs PV drivers of some their performance as well? It's really only the disk device that runs into this problem. This is the only device that simply can't change its name, type, or location. If the path to the boot device isn't _exactly_ the same through the life of the domain, you're dead. Changing the name or type of a NIC is more awkward than one would hope, but it can be done without reinstalling the domain. > But it sounds, from the materials below, like maybe for cmdk's specific > case this isn't too bad, and there maybe upgrade issues that are avoided > by this approach? Yes, exactly. Nils From gdamore@sun.com Wed Nov 21 08:04:14 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALG4EcL003396 for ; Wed, 21 Nov 2007 08:04:14 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALG47OK002820 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 09:04:14 -0700 (MST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0082F5Z02H00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:04:12 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV002RI5YYXN30@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:04:10 -0700 (MST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALG4An0015964 for ; Wed, 21 Nov 2007 08:04:10 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV008015SZH800@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:04:10 -0800 (PST) Received: from [192.168.251.106] ([76.174.83.55]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV00KD65YX5IF0@fe-sfbay-09.sun.com>; Wed, 21 Nov 2007 08:04:09 -0800 (PST) Date: Wed, 21 Nov 2007 07:59:29 -0800 From: "Garrett D'Amore" Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <18244.18608.359279.818337@gargle.gargle.HOWL> Sender: Garrett.Damore@sun.com To: James Carlson Cc: Glenn Brunette , John Plocher , Tim Marsland , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <474455E1.1020603@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47444826.1020409@sun.com> <18244.18608.359279.818337@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.4 (X11/20070827) Status: RO Content-Length: 1160 James Carlson wrote: > Glenn Brunette writes: > >> While I agree with you on these points, sys-unconfig will not help with >> many of these either, right? >> > > I would say that ought to, but does not. > > >> It sounds like we need a better option >> than just sys-unconfig - one that could (ideally) understand these >> relationships or at the very least reset them to default values (saving >> the original content)? Today, I would argue it is just too easy to >> do sys-unconfig - follow the rules - and still get fowled up because >> of other settings that sys-unconfig does not know about. >> > > Agreed; sys-unconfig is incomplete. The only good advice we have now > is that users need to be extremely cautious when changing driver > names, as they may well end up damaging the system and even > compromising security (due to non-matching filtering rules). > I think vanity naming can help here. With vanity naming, the mapping between physical device and logical name could be changed, without changing any of the other files. Of course, that requires the configuration to actually *use* the vanity names. -- Garrett From carlsonj@phorcys.east.sun.com Wed Nov 21 08:10:31 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALGAUDv003411 for ; Wed, 21 Nov 2007 08:10:31 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lALGAOO1016501; Thu, 22 Nov 2007 00:10:26 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0050D69CYU00@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 08:10:24 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV00HUF69CUDC0@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 08:10:24 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALG9Yb9010768; Wed, 21 Nov 2007 11:09:34 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALG9Yfs010765; Wed, 21 Nov 2007 11:09:34 -0500 (EST) Date: Wed, 21 Nov 2007 11:09:34 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <474455E1.1020603@sun.com> To: "Garrett D'Amore" Cc: John Plocher , Glenn Brunette , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com, Tim Marsland Message-id: <18244.22590.865238.478463@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47444826.1020409@sun.com> <18244.18608.359279.818337@gargle.gargle.HOWL> <474455E1.1020603@sun.com> Status: RO Content-Length: 626 Garrett D'Amore writes: > I think vanity naming can help here. With vanity naming, the mapping > between physical device and logical name could be changed, without > changing any of the other files. Of course, that requires the > configuration to actually *use* the vanity names. Yes. This replacement problem is exactly the sort of problem it was designed to fix. It's just not yet available. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Darren.Moffat@sun.com Wed Nov 21 08:43:27 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALGhR1J003651 for ; Wed, 21 Nov 2007 08:43:27 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lALGhMK7005241 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 16:43:26 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV007017SD8C00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:43:25 -0800 (PST) Received: from gmp-eb-mail-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV00H5I7SBUKE0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 08:43:24 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe1.eu.sun.com [192.18.6.10]) by gmp-eb-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALGhN04020008 for ; Wed, 21 Nov 2007 16:43:23 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV00D015NHEE00@fe-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 16:43:23 +0000 (GMT) Received: from [129.156.173.21] by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV000KY7S5NU20@fe-emea-09.sun.com>; Wed, 21 Nov 2007 16:43:18 +0000 (GMT) Date: Wed, 21 Nov 2007 16:43:17 +0000 From: Darren J Moffat Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> Sender: Darren.Moffat@sun.com To: Tim Marsland Cc: PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <47446025.5070002@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20070924) Status: RO Content-Length: 575 What are the actually binary deliverables from this case ? I think they are: Updated /kernel/drv/$ISA/cmdk New /kernel/drv/$ISA/xpv New /kernel/drv/$ISA/xnf Is that correct ? I don't think the network device name changing is too big an issue, because as you pointed out the configuration of the domain had to be explicitly changed in dom0 first. Out of interest how much performance improvement should be expected for these new drivers ? [ Not architectural and I'm happy to see the answer sent to xen-discuss@opensolaris.org instead of here ]. -- Darren J Moffat From nn35248@negril.east.sun.com Wed Nov 21 09:12:00 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHC0tY004452 for ; Wed, 21 Nov 2007 09:12:00 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALHBwf8033619; Wed, 21 Nov 2007 10:11:58 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0081D93Y9Q00@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 09:11:58 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV007JW93XTQ10@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 09:11:57 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALHCio1011512; Wed, 21 Nov 2007 12:12:44 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALHCima011511; Wed, 21 Nov 2007 12:12:44 -0500 (EST) Date: Wed, 21 Nov 2007 12:12:44 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47446025.5070002@Sun.COM> To: Darren J Moffat Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <20071121171244.GF11061@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47446025.5070002@Sun.COM> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1493 On Wed 11/21/07 at 16:43 PM, Darren.Moffat@Sun.COM wrote: > What are the actually binary deliverables from this case ? > > I think they are: > > Updated /kernel/drv/$ISA/cmdk > New /kernel/drv/$ISA/xpv > New /kernel/drv/$ISA/xnf > > Is that correct ? It is for cmdk. The other drivers are specific to i86pc, so they go under /platform: /platform/i86pc/kernel/drv/$ISA/xpv PV plumbing /platform/i86pc/kernel/drv/$ISA/xpvd PV nexus driver /platform/i86pc/kernel/drv/$ISA/xdf PV disk driver /platform/i86pc/kernel/drv/$ISA/xnv PV network driver 'xpv' is really the only new driver. The other three are built from the same source as the fully PV versions that live under /platform/i86xpv. There are a handful of #ifdefs in each driver's source that encapsulate the differences between the two platforms. > I don't think the network device name changing is too big an issue, > because as you pointed out the configuration of the domain had to be > explicitly changed in dom0 first. > > Out of interest how much performance improvement should be expected for > these new drivers ? [ Not architectural and I'm happy to see the answer > sent to xen-discuss@opensolaris.org instead of here ]. Network performance improvement is around 6x. I haven't actually measured disk performance yet. I was hoping to get to it before Thanksgiving, but that probably won't happen. Just based on the time it takes to rebuild the boot archives, it feels like the improvement is about 2-3x. Nils From John.Plocher@sun.com Wed Nov 21 09:31:59 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHVwem004801 for ; Wed, 21 Nov 2007 09:31:58 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lALHVq7s002398 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 17:31:57 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV00913A185J00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:31:56 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV007T8A18TT30@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:31:56 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALHVtkV013679 for ; Wed, 21 Nov 2007 09:31:55 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV00H019THZJ00@fe-sfbay-10.sun.com> (original mail from John.Plocher@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:31:55 -0800 (PST) Received: from wp668.local ([208.74.177.212]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV00G6BA0ZH050@fe-sfbay-10.sun.com>; Wed, 21 Nov 2007 09:31:47 -0800 (PST) Date: Wed, 21 Nov 2007 09:31:47 -0800 From: John Plocher Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <18244.17661.406079.436599@gargle.gargle.HOWL> Sender: John.Plocher@sun.com To: James Carlson Cc: Tim Marsland , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com Message-id: <47446B83.1060106@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Status: RO Content-Length: 1108 James Carlson wrote: > John Plocher writes: >> Tim Marsland wrote: >>> Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI >>> >>> 1. Introduction >>> 1.1. Project/Component Working Name: >>> Paravirtualized Drivers for Fully Virtualized xVM Domains >>> For network devices, however, the user experience will be less >>> satisfying. ... Since Solaris binds the identity of a >>> system to its NIC name, the user will have to sys-unconfig and >>> reconfigure the domain. >> sys_unconfig is a pretty big hammer to use for this - isn't it sufficient >> to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that the >> user can determine the mapping between the two? > > Renaming a driver is a serious pain. Are they changing a driver name or are they simply changing the network device they are using? Yes they are similar things... This is a common enuf task on systems with multiple network interfaces that there should be some easy (easier?) way to do this - changing from qfe2 to qfe3 for my sunray network at home didn't require all that pain... -John From John.Plocher@sun.com Wed Nov 21 09:38:26 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHcQ4O004856 for ; Wed, 21 Nov 2007 09:38:26 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALHcPsL011677 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 09:38:26 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV00F0BAC16600@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 10:38:25 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV002KKAC0XZA0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 10:38:24 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALHcOeg026924 for ; Wed, 21 Nov 2007 09:38:24 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV00401A5NDR00@fe-sfbay-10.sun.com> (original mail from John.Plocher@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 09:38:24 -0800 (PST) Received: from wp668.local ([208.74.177.212]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV0052NABZTK10@fe-sfbay-10.sun.com>; Wed, 21 Nov 2007 09:38:23 -0800 (PST) Date: Wed, 21 Nov 2007 09:38:22 -0800 From: John Plocher Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071121153656.GA11061@east.sun.com> Sender: John.Plocher@sun.com To: Nils Nieuwejaar Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <47446D0E.8050807@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <20071121153656.GA11061@east.sun.com> User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Status: RO Content-Length: 841 Nils Nieuwejaar wrote: > Unfortunately, it's not that simple. > Within the domU, we don't have enough information to draw any inferences > about network topology. Any guesses we made about a sensible renaming > scheme would likely go wrong under all but the simplest scenarios. So if I do a sys-unconfig and reboot, I get presented with a bunch of questions (which network do you want to use...) that /I/ don't have any clue about which to choose either... I'm not saying to automate this completely, but to provide tools that the user can use to manage the complexity. In the simple case, where I have a domU with a single iface, a tool that could do change_network_device --from rtls0 --to xnf34 is what I'm thinking of. My question above is how the user would figure out that xfn34 would be the right new device name. -John From nn35248@negril.east.sun.com Wed Nov 21 09:44:11 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHiArk004914 for ; Wed, 21 Nov 2007 09:44:10 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALHi9SV013458; Wed, 21 Nov 2007 09:44:09 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0070FALJDW00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 09:44:07 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV00101ALJ2H20@nwk-avmta-1.sfbay.Sun.COM>; Wed, 21 Nov 2007 09:44:07 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALHirML011623; Wed, 21 Nov 2007 12:44:53 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALHirNC011622; Wed, 21 Nov 2007 12:44:53 -0500 (EST) Date: Wed, 21 Nov 2007 12:44:53 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47446B83.1060106@Sun.Com> To: John Plocher Cc: James Carlson , Tim Marsland , PSARC-ext@sun.com Message-id: <20071121174453.GH11061@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47446B83.1060106@Sun.Com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1445 On Wed 11/21/07 at 09:31 AM, John.Plocher@Sun.COM wrote: > James Carlson wrote: > >John Plocher writes: > >>Tim Marsland wrote: > >>>Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI > >>> > >>>1. Introduction > >>> 1.1. Project/Component Working Name: > >>> Paravirtualized Drivers for Fully Virtualized xVM Domains > >>> For network devices, however, the user experience will be less > >>> satisfying. ... Since Solaris binds the identity of a > >>> system to its NIC name, the user will have to sys-unconfig and > >>> reconfigure the domain. > >>sys_unconfig is a pretty big hammer to use for this - isn't it sufficient > >>to just rename /etc/hostname.rtls# to /etc/hostname.xnf#, assuming that > >>the > >>user can determine the mapping between the two? > > > >Renaming a driver is a serious pain. > > Are they changing a driver name or are they simply changing the > network device they are using? Yes they are similar things... In the network case, we are adding a new driver that allows domains to access a new kind of network device. Currently, domains can only communicate to the world at large by using the rtls driver to access an emulated RealTek 8139. We are adding an xnf driver that will also allow domains to access an explicitly virtualized Xen network device. There is absolutely no connection or relationship between the RealTek devices configured into a domain and the Xen PV devices. Nils From carlsonj@phorcys.east.sun.com Wed Nov 21 09:47:12 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHlCfc004958 for ; Wed, 21 Nov 2007 09:47:12 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALHl8Bq013945; Wed, 21 Nov 2007 09:47:10 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV0091JAQMWW00@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 09:47:10 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV007EYAQLTO50@nwk-avmta-2.sfbay.sun.com>; Wed, 21 Nov 2007 09:47:10 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALHkK25014462; Wed, 21 Nov 2007 12:46:20 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALHkKBg014459; Wed, 21 Nov 2007 12:46:20 -0500 (EST) Date: Wed, 21 Nov 2007 12:46:20 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47446B83.1060106@Sun.Com> To: John Plocher Cc: Tim Marsland , Nils.Nieuwejaar@sun.com, PSARC-ext@sun.com Message-id: <18244.28396.59568.201360@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47446B83.1060106@Sun.Com> Status: RO Content-Length: 1723 John Plocher writes: > James Carlson wrote: > > Renaming a driver is a serious pain. > > Are they changing a driver name or are they simply changing the > network device they are using? Yes they are similar things... There's really no difference in terms of networking configuration breakage. > This is a common enuf task on systems with multiple network interfaces > that there should be some easy (easier?) way to do this - changing from > qfe2 to qfe3 for my sunray network at home didn't require all that pain... Sure. For trivial cases, it's not too hard to get it right or to "luck into" a working configuration. However, the names of network interfaces (including, yes, "qfe2") are administrative units that are available to and used by *dozens* of applications and configuration components throughout the system. When you change one, there simply are no tools (other than perhaps find/xargs/grep -- good luck if you have configuration systems based on something other than plain text files) that can help you deal with the aftermath. You have to "remember" where you or any other administrator or user of the system may have used the name. That's why I referenced the previous ipge case. It's the same inherent problem that caused us to deny that case. Yes, vanity naming solves the problem pretty neatly. No, vanity naming doesn't exist yet and this case isn't dependent on it. The best we can do is to warn users that such a name change is *in the general case* infeasible. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From carlsonj@phorcys.east.sun.com Wed Nov 21 09:49:32 2007 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALHnWtx005015 for ; Wed, 21 Nov 2007 09:49:32 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALHnUh0015714; Wed, 21 Nov 2007 09:49:31 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV00G0HAUI0400@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 10:49:30 -0700 (MST) Received: from phorcys.east.sun.com ([129.148.174.143]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV002XTAUHY280@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 10:49:29 -0700 (MST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALHmex3014505; Wed, 21 Nov 2007 12:48:40 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALHmeji014502; Wed, 21 Nov 2007 12:48:40 -0500 (EST) Date: Wed, 21 Nov 2007 12:48:40 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47446D0E.8050807@Sun.Com> To: John Plocher Cc: Nils Nieuwejaar , Tim Marsland , PSARC-ext@sun.com Message-id: <18244.28536.615170.680326@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <20071121153656.GA11061@east.sun.com> <47446D0E.8050807@Sun.Com> Status: RO Content-Length: 699 John Plocher writes: > In the simple case, where I have a domU with a single iface, a tool > that could do > > change_network_device --from rtls0 --to xnf34 No such tool is feasible. It would have to have some sort of "magic" that allows it to find and edit the configuration of an arbitrary number of subsystems and applications, including ones that aren't even shipped with Solaris. Again, I refer you to the ipge case (2004/782). We've been over this ground before. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From nn35248@negril.east.sun.com Wed Nov 21 10:02:54 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALI2rGQ005857 for ; Wed, 21 Nov 2007 10:02:54 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lALI2nSb018086; Wed, 21 Nov 2007 18:02:51 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV00H05BGQ1400@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 11:02:50 -0700 (MST) Received: from negril.east.sun.com ([129.148.183.104]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV002XCBGPXNA0@brm-avmta-1.central.sun.com>; Wed, 21 Nov 2007 11:02:50 -0700 (MST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lALI3aNu011866; Wed, 21 Nov 2007 13:03:36 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.1+Sun/8.14.1/Submit) id lALI3KVh011865; Wed, 21 Nov 2007 13:03:20 -0500 (EST) Date: Wed, 21 Nov 2007 13:03:20 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <47446D0E.8050807@Sun.Com> To: John Plocher Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <20071121180320.GI11061@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <20071121153656.GA11061@east.sun.com> <47446D0E.8050807@Sun.Com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1668 On Wed 11/21/07 at 09:38 AM, John.Plocher@Sun.COM wrote: > Nils Nieuwejaar wrote: > >Unfortunately, it's not that simple. > > >Within the domU, we don't have enough information to draw any inferences > >about network topology. Any guesses we made about a sensible renaming > >scheme would likely go wrong under all but the simplest scenarios. > > So if I do a sys-unconfig and reboot, I get presented with a bunch > of questions (which network do you want to use...) that /I/ don't have > any clue about which to choose either... I'm not sure I understand why you wouldn't. This is exactly the same as if you swap out your Broadcom NIC for an Intel NIC in a physical system. > I'm not saying to automate this completely, but to provide tools that > the user can use to manage the complexity. I understand this request, but I believe it is beyond the scope of this project. From some of the other messages in this thread, it sounds like this is one of the goals of the vanity naming project. > In the simple case, where I have a domU with a single iface, a tool > that could do > > change_network_device --from rtls0 --to xnf34 > > is what I'm thinking of. My question above is how the user would figure > out that xfn34 would be the right new device name. Both before and after the change, the domain will only have a single network device. Initially it will have rtls0. After the domain is halted, reconfigured, and rebooted, it will have xnf0. If you are using DHCP and NWAM, then I'm pretty sure the transition from HVM to PV devices should be seamless. As soon as I can find an HVM-capable system on a network with DHCP, I'll verify this. Nils From John.Plocher@sun.com Wed Nov 21 11:24:11 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lALJOBh1008017 for ; Wed, 21 Nov 2007 11:24:11 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lALJOAN9018845 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 21 Nov 2007 12:24:11 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JRV00E0HF8ACV00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 11:24:10 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JRV0072DF89TQF0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 11:24:09 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lALJO9Bs027434 for ; Wed, 21 Nov 2007 11:24:09 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JRV00101F49BY00@fe-sfbay-10.sun.com> (original mail from John.Plocher@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 21 Nov 2007 11:24:09 -0800 (PST) Received: from wp668.local ([208.74.177.212]) by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JRV00EO5F89NEB0@fe-sfbay-10.sun.com>; Wed, 21 Nov 2007 11:24:09 -0800 (PST) Date: Wed, 21 Nov 2007 11:24:08 -0800 From: John Plocher Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071121180320.GI11061@east.sun.com> Sender: John.Plocher@sun.com To: Nils Nieuwejaar Cc: Tim Marsland , PSARC-ext@sun.com Message-id: <474485D8.5020909@Sun.Com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <20071121153656.GA11061@east.sun.com> <47446D0E.8050807@Sun.Com> <20071121180320.GI11061@east.sun.com> User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Status: RO Content-Length: 347 Nils Nieuwejaar wrote: > If you are using DHCP and NWAM, then I'm pretty sure the transition from > HVM to PV devices should be seamless. As soon as I can find an HVM-capable > system on a network with DHCP, I'll verify this. OK. I'm satisfied - not this case for the complicated scenario and just works for the simple, common case. -John From jek3@sun.com Mon Nov 26 18:38:40 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lAR2cdG4025128 for ; Mon, 26 Nov 2007 18:38:39 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lAR2cW4o002494; Tue, 27 Nov 2007 10:38:34 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JS500G038O9X800@nwk-avmta-2.sfbay.sun.com>; Mon, 26 Nov 2007 18:38:33 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JS5001FK8O8KBD0@nwk-avmta-2.sfbay.sun.com>; Mon, 26 Nov 2007 18:38:32 -0800 (PST) Received: from [129.150.12.137] (vpn-129-150-12-137.SFBay.Sun.COM [129.150.12.137]) by jurassic-x4600.sfbay.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lAR2V1Bu794293; Mon, 26 Nov 2007 18:31:02 -0800 (PST) Date: Mon, 26 Nov 2007 16:30:29 -1000 From: Joseph Kowalski Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <18244.28396.59568.201360@gargle.gargle.HOWL> To: James Carlson Cc: John Plocher , Tim Marsland , nils.nieuwejaar@sun.com, PSARC-ext@sun.com Message-id: <474B8145.3000703@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47446B83.1060106@Sun.Com> <18244.28396.59568.201360@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0b2 (X11/20070227) Status: RO Content-Length: 507 James Carlson wrote: > Yes, vanity naming solves the problem pretty neatly. No, vanity > naming doesn't exist yet and this case isn't dependent on it. The > best we can do is to warn users that such a name change is *in the > general case* infeasible. > Gee, I must have slept through something. What is "vanity naming" and where can I get some? (Pointers that is...) Remember the days when all you needed was a major and minor number, and then we could call it anything you wanted? 8^) - jek3 From carlsonj@phorcys.east.sun.com Tue Nov 27 04:26:00 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lARCQ0Tg011216 for ; Tue, 27 Nov 2007 04:26:00 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lARCPwm6015287; Tue, 27 Nov 2007 05:25:58 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JS500403ZV9VP00@nwk-avmta-2.sfbay.sun.com>; Tue, 27 Nov 2007 04:25:57 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JS5002MLZV9BU40@nwk-avmta-2.sfbay.sun.com>; Tue, 27 Nov 2007 04:25:57 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lARCHZug017078; Tue, 27 Nov 2007 07:17:35 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lARCHZ1f017075; Tue, 27 Nov 2007 07:17:35 -0500 (EST) Date: Tue, 27 Nov 2007 07:17:35 -0500 From: James Carlson Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <474B8145.3000703@sun.com> To: Joseph Kowalski Cc: John Plocher , PSARC-ext@sun.com, Nils.Nieuwejaar@sun.com, Tim Marsland Message-id: <18252.2783.271549.909908@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <47440D8B.8010908@Sun.Com> <18244.17661.406079.436599@gargle.gargle.HOWL> <47446B83.1060106@Sun.Com> <18244.28396.59568.201360@gargle.gargle.HOWL> <474B8145.3000703@sun.com> Status: RO Content-Length: 1636 Joseph Kowalski writes: > James Carlson wrote: > > Yes, vanity naming solves the problem pretty neatly. No, vanity > > naming doesn't exist yet and this case isn't dependent on it. The > > best we can do is to warn users that such a name change is *in the > > general case* infeasible. > > > Gee, I must have slept through something. What is "vanity naming" and > where can I get some? (Pointers that is...) Sorry; that was too terse. It's a reference to "Clearview Nemo unification and vanity naming" (PSARC 2006/499; also known as "UV"), which will provide the ability to rename network interfaces. One of the most important usage cases for this feature (despite the "vanity naming" title for the project) is to be able to keep the name of an interface the same. Say, for example, you pull out a "qfe" card and replace with a "ce" card. UV will allow you to rename the new ce0 interface back to qfe0, so that all those configuration files that refer to qfe0 won't have to change. It's an aid for DR and zone migration, among other things. > Remember the days when all you needed was a major and minor number, and > then we could call it anything you wanted? 8^) You still can, but the name (and typically not the number, other than for Sun Cluster) ends up getting embedded in software that refers to the node, and thus you can't _change_ the name without hunting down the affected consumers. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From nn35248@negril.east.sun.com Thu Nov 29 09:53:38 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATHrcYO005689 for ; Thu, 29 Nov 2007 09:53:38 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lATHrPq5007599; Thu, 29 Nov 2007 17:53:35 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00J0D4D9EK00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 09:53:33 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA00GTJ4D9UK40@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 09:53:33 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATHrWQ8000281; Thu, 29 Nov 2007 12:53:32 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lATHrWGY000280; Thu, 29 Nov 2007 12:53:32 -0500 (EST) Date: Thu, 29 Nov 2007 12:53:32 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> To: tim.marsland@sun.com Cc: PSARC-ext@sun.com Message-id: <20071129175332.GA29689@east.sun.com> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_z1fN9aJBnT854BpDE6Cb4Q)" Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 13650 --Boundary_(ID_z1fN9aJBnT854BpDE6Cb4Q) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline When this proposal was originally sent out, there was a bit of concern expressed on this list about our intention to modify cmdk and possibly some future drivers. Offline, I received some rather stronger objections to this idea, so I have updated the proposal to address those concerns. Rather than modifying the existing cmdk, we are now proposing to create a brand new cmdk that will only be used to access paravirtualized disk drivers. This approach will leave the existing driver completely untouched. To ensure that this new PV-aware cmdk is only loaded in HVM domains, we propose to install it in a new directory: /platform/i86hvm/kernel/drv. In HVM domains, this new directory will be prepended to the default module search path. Just to be clear: we are not proposing to create a whole new 'implementation' called i86hvm. This is just for the PV drivers. I am attaching a modified version of the fast track proposal with changebars. I know we're very close to the original timeout for this case, so if anybody has concerns with this change, I'm happy to have that timeout extended. Nils --Boundary_(ID_z1fN9aJBnT854BpDE6Cb4Q) Content-type: text/plain; NAME=pv_driver_fasttrack; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: attachment; filename=pv_driver_fasttrack 1. Introduction 1.1. Project/Component Working Name: Paravirtualized Drivers for Fully Virtualized xVM Domains 1.2. Name of Document Author/Supplier: Author: Nils Nieuwejaar 1.3 Date of This Document: 28 November, 2007 | 4. Technical Description 4.1 Summary The Xen hypervisor and the port of Solaris to that hypervisor are described in PSARC/2006/260. That case is considered required pre-reading for this. The Solaris port to the Xen hypervisor included disk and network drivers for paravirtualized (PV) domUs. These PV drivers provide standard disk and network interfaces to Solaris, but communicate with back-end drivers in the control domain (dom0) rather than interacting directly with hardware. When running in a fully virtualized (HVM) domain, Solaris continues to use standard device drivers. These drivers believe they are interacting with hardware, but are actually interacting with emulated devices that run in dom0. These emulated physical devices deliver significantly lower performance than the corresponding virtual devices in a PV domain. For example, the emulated network device may run at 1/10th the speed of a virtual device. This project proposes to provide the infrastructure necessary to allow the existing Solaris PV drivers to function in HVM domains, delivering significantly higher performance to applications running in those domains. We intend to backport these PV drivers to Solaris 10, enabling it to be a performant guest operating system. Therefore, we are requesting micro/patch binding for this project. 4.2 Proposal The project is comprised of several components. There is one new driver that provides the infrastructure necessary to allow software in an HVM domain to communicate with the hypervisor and the control domain. There are also minor modifications to the existing drivers, that will allow them to plug into the new infrastructure. 4.2.1 xpv xpv is a new leaf driver that binds to Xen's PCI ID (pci5853,1). This driver provides all the plumbing needed for PV drivers to drill through the bottom of a fully virtualized domain, and interact directly with the hypervisor. The driver uses HVM-specific hypercalls to: o map in the 'Xen shared info' page, which is used for communication between the hypervisor and OS. o map pages for grant tables. Grant tables allow the guest | domain to grant temporary access to a page by another | domain. Bulk data transfer between the domains is | performed using these temporarily shared pages. | o register a single interrupt with the hypervisor, which it will trigger when there are events pending for the OS to process. After initialization, the driver's primary responsibility is to receive emulated 'physical' interrupts from the hypervisor when events are pending, and then forward those event notifications to the correct PV driver(s). 4.2.2 Modified Drivers The existing paravirtualized xdf, xnf, xpvd, and xenbus drivers | must all be modified to register event handlers with a new 'xpv' driver, rather than registering interrupt handlers with Solaris. Other than this event delivery mechanism, all of a driver's interaction with the hypervisor and control plane is exactly the same in both PV and HVM domains. The scope of these changes is on the order of about a dozen lines per driver. 4.2.3 New Drivers | | We will be adding a PV-aware versions of the existing 'metal' | drivers 'cmdk'. cmdk is an existing Solaris driver that | provides a common framework for block devices. This driver presents a simple high-level disk model to Solaris while abstracting away the distinctions between disks, CD drives, etc. This project creates a new version of cmdk that is PV-aware, | and which will appear in the same location in the device | hierarchy as the standard cmdk driver. This new version of cmdk will use the Layered Driver framework to forward all disk requests to the PV xdf driver rather than to the standard driver(s) for emulated physical devices. This forwarding mechanism allows us to add the PV drivers after a domain has already been installed, without having to modify the path to the boot device. 4.2.4 i86hvm | To ensure that Solaris attempts to load the PV drivers only in | HVM Xen domains, we will create a new directory to hold PV | drivers: | /platform/i86hvm/kernel/drv | This new directory will be added as the first item on the | module search path when an HVM domain is booted. This allows | us to deliver two versions of the cmdk driver (PV and 'metal') | and ensure that the proper driver is loaded when the Solaris | instance is booted. | Note: we do not propose to create an entirely new x86 | implementation called i86hvm, so 'uname -i' will continue to | return i86pc. The difference between an HVM domain and an x86 | instance running on bare metal is so insignificant that | introducing a new implementation seems like overkill. In | particular, no user-level code should be aware of the | distinction. The only elements of Solaris that should be aware | that this is an HVM domain are these PV drivers, which are | explicitly intended to penetrate the boundaries of that domain. | 4.3 User Experience 4.3.1 Background Xen has adopted two very different models for supporting PV devices in HVM domains. These models are built into the architecture of Xen itself, and are not specific to Sun's implementation. For block devices, each device configured in a domU has two interfaces: an emulated ATA device on the PCI bus, and an explicit PV interface accessible through the Xenbus. The two interfaces provide access to the same storage on the back end. For network devices, each NIC is configured in dom0 to be either PV or HVM. If a device is configured to be HVM, Xen provides an emulated RealTek NIC on the PCI bus. If it is configured to be PV, then it provides an explicit PV interface on the Xenbus. A single NIC cannot be configured in both modes at once. 4.3.2 Initial Installation When the PV drivers are integrated into the WOS, they will function as any other block or network driver. When a domU is installed, the disks will automatically and transparently be accessed using the PV interfaces. Users will install to /dev/dsk/cXdYsZ as they do now. All the normal tools, such as format, prtvtoc, and fdisk will work as expected. When the installer probes for network devices, any PV NICs configured into the domain will be made available as xnf0 through xnf, just as (for example) Intel NICS are made available as e1000g0 through e1000g. 4.3.3 Upgrades or post-install additions When the PV drivers are added to an existing domain, our modifications to the cmdk driver will allow the PV disk drivers to fall into place transparently. When the driver is first installed, the system will continue to access disks using the existing emulated metal interfaces. When the domain is rebooted, the disks will automatically be accessed through the PV interfaces. The domain's bootpath remains unchanged, so the transition from HVM to PV should be completely transparent to the user. For network devices, however, the user experience will be less satisfying. As a NIC must be explicitly configured in dom0 to be either PV or HVM, simply installing the PV drivers will not deliver PV performance to the user. The domU must be reconfig- ured to contain PV NICs rather than HVM. When the domain is reconfigured, the 'brand' of the NIC will then change from rtls (i.e. "RealTek") to xnf. In the simplest case (a domU with a single NIC that receives | its IP address via DHCP), this transition will still be | seamless. Upon reboot, the NWAM framework will plumb the new | NIC, and will request an IP address for xnf0 rather than rtls0. | In more complicated cases, the user will likely have to | sys-unconfig and reconfigure the domain. | A natural question is: can't we transparently remap the rtls devices to xnf devices under the hood (as we do for disks), making this reconfiguration step go away. Unfortunately, no. Whether a NIC is configured to be PV or HVM is not simply a cosmetic issue. When Xen instantiates an HVM NIC for a domain, it does not create a back-end device that is capable of interacting with a front-end PV driver. Even if the domU had sufficient information about the back-end configuration to understand how this remapping should be done, there is simply nothing in the dom0 for our PV drivers to talk to. While this upgrade experience is certainly less smooth than one would hope, it is not dissimilar to the experience of upgrading physical hardware. Replacing a disk in a physical machine is a simple process, and the device naming is not affected by the brand of the hard drive. Replacing an Intel NIC with a Broadcom NIC, on other hand, requires exactly the same sys-unconfig as that required by replacing a fully virtualized NIC with a paravirtualized NIC. 4.4 Futures | The Xen I/O model has changed since we snapshotted the version we | are currently shipping. Specifically, network devices now appear | as hybrid PV/HVM devices just as disks do. When we move forward to | this newer version of Xen, we will be able to transparently replace | emulated network devices with PV devices. This change will remove | all of the user-visible complications discussed in section 4.3. | To achieve this transparency, we will have to deliver a shell | version of rtls that forwards requests to the xnf device, just as | cmdk forwards requests to xdf. | 4.5 Interface Table Imported Interfaces --------------------------------------------------------------------- | Interface Level Comments |-------------------------------------------------------------------- | Layered Driver Evolving | Framework | | Xen Interfaces External | Hypercall interface | Shared page format | Xenbus protocol | | PCI ID pci5853,1 External Xen's registered | PCI ID |-------------------------------------------------------------------- Exported Interfaces --------------------------------------------------------------------- | Interface Level Comments |-------------------------------------------------------------------- | ec_bind_evtchn_to_handler() Project Used by the PV | ec_unbind_evtchn() private drivers to plug into | hypervisor_mask_event() the new PV-on-HVM | hypervisor_unmask_event() framework. | /platform/i86hvm | |-------------------------------------------------------------------- 4.6 References [1] PSARC 2006/260: Solaris on Xen [2] PSARC 2001/769: LDF: Layered Driver Framework 5. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open --Boundary_(ID_z1fN9aJBnT854BpDE6Cb4Q)-- From Darren.Moffat@sun.com Thu Nov 29 10:11:58 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATIBwVv006840 for ; Thu, 29 Nov 2007 10:11:58 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lATIBtHP004157 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 29 Nov 2007 10:11:57 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA0040157WP500@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 11:11:56 -0700 (MST) Received: from gmp-eb-mail-2.sun.com ([192.18.6.24]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA00BP857V01D0@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 11:11:56 -0700 (MST) Received: from fe-emea-10.sun.com (gmp-eb-lb-2-fe1.eu.sun.com [192.18.6.10]) by gmp-eb-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lATIBtMo020534 for ; Thu, 29 Nov 2007 18:11:55 +0000 (GMT) Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JSA00E014YPPA00@fe-emea-10.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 18:11:55 +0000 (GMT) Received: from [129.156.173.21] by fe-emea-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JSA00LDR579UY30@fe-emea-10.sun.com>; Thu, 29 Nov 2007 18:11:34 +0000 (GMT) Date: Thu, 29 Nov 2007 18:11:33 +0000 From: Darren J Moffat Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071129175332.GA29689@east.sun.com> Sender: Darren.Moffat@sun.com To: Nils Nieuwejaar Cc: Tim.Marsland@sun.com, PSARC-ext@sun.com Message-id: <474F00D5.5060308@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071105) Status: RO Content-Length: 803 Nils Nieuwejaar wrote: > When this proposal was originally sent out, there was a bit of concern > expressed on this list about our intention to modify cmdk and possibly some > future drivers. Offline, I received some rather stronger objections to > this idea, so I have updated the proposal to address those concerns. > > Rather than modifying the existing cmdk, we are now proposing to create a > brand new cmdk that will only be used to access paravirtualized disk > drivers. This approach will leave the existing driver completely > untouched. Please tell me you don't intend to take a copy of the cmdk source to do this ? Is this going to be a very simple driver that happens to have then name cmdk to take advantage of the fact that the driver name is baked in places ? -- Darren J Moffat From nn35248@negril.east.sun.com Thu Nov 29 10:28:23 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATISMcC007275 for ; Thu, 29 Nov 2007 10:28:23 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lATISBG1012659; Fri, 30 Nov 2007 02:28:18 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA001035Z3OV00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 10:28:16 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA00G7L5Z2UIC0@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 10:28:15 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATISEbb000356; Thu, 29 Nov 2007 13:28:14 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lATISENk000355; Thu, 29 Nov 2007 13:28:14 -0500 (EST) Date: Thu, 29 Nov 2007 13:28:14 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <474F00D5.5060308@Sun.COM> To: Darren J Moffat Cc: Tim.Marsland@Sun.COM, PSARC-ext@Sun.COM Message-id: <20071129182814.GB29689@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <474F00D5.5060308@Sun.COM> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1092 On Thu 11/29/07 at 18:11 PM, Darren.Moffat@Sun.COM wrote: > Nils Nieuwejaar wrote: > >When this proposal was originally sent out, there was a bit of concern > >expressed on this list about our intention to modify cmdk and possibly some > >future drivers. Offline, I received some rather stronger objections to > >this idea, so I have updated the proposal to address those concerns. > > > >Rather than modifying the existing cmdk, we are now proposing to create a > >brand new cmdk that will only be used to access paravirtualized disk > >drivers. This approach will leave the existing driver completely > >untouched. > > Please tell me you don't intend to take a copy of the cmdk source to do > this ? > Is this going to be a very simple driver that happens to have then name > cmdk to take advantage of the fact that the driver name is baked in places ? Exactly. This is a thin shell driver that does nothing but make LDI calls to the paravirtual 'xdf' driver. It will be incapable of managing physical disks, which is why we need to sequester it in the new i86hvm directory. Nils From tim.marsland@Sun.COM Thu Nov 29 11:35:00 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATJYxXv010316 for ; Thu, 29 Nov 2007 11:35:00 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lATJYUHL018162; Fri, 30 Nov 2007 03:34:55 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00C019258W00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 11:34:53 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA0027F925XR40@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 11:34:53 -0800 (PST) Received: from [10.0.1.6] (vpn-129-150-21-199.SFBay.Sun.COM [129.150.21.199]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATJYrkN670704; Thu, 29 Nov 2007 11:34:53 -0800 (PST) Date: Thu, 29 Nov 2007 11:34:51 -0800 From: Tim Marsland Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071129175332.GA29689@east.sun.com> To: Nils Nieuwejaar Cc: Tim Marsland , PSARC-ext@Sun.COM Message-id: MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> Status: RO Content-Length: 435 > To ensure that this new PV-aware cmdk is only loaded in HVM > domains, we > propose to install it in a new directory: /platform/i86hvm/kernel/ > drv. In > HVM domains, this new directory will be prepended to the default > module > search path. Just to be clear: we are not proposing to create a > whole new > 'implementation' called i86hvm. This is just for the PV drivers. How will that prepending happen, exactly? tim From Darren.Moffat@sun.com Thu Nov 29 11:43:33 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATJhXK8010960 for ; Thu, 29 Nov 2007 11:43:33 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lATJhUQd009872 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 29 Nov 2007 12:43:33 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA0050J9GJM900@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 11:43:31 -0800 (PST) Received: from gmp-eb-mail-1.sun.com ([192.18.6.21]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA0022I9GH8M60@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 11:43:30 -0800 (PST) Received: from fe-emea-09.sun.com (gmp-eb-lb-2-fe2.eu.sun.com [192.18.6.11]) by gmp-eb-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id lATJhTnH025859 for ; Thu, 29 Nov 2007 19:43:29 +0000 (GMT) Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0JSA00H019BG4500@fe-emea-09.sun.com> (original mail from Darren.Moffat@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Thu, 29 Nov 2007 19:43:29 +0000 (GMT) Received: from [129.156.173.21] by fe-emea-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0JSA00HPS9GGXE30@fe-emea-09.sun.com>; Thu, 29 Nov 2007 19:43:29 +0000 (GMT) Date: Thu, 29 Nov 2007 19:43:28 +0000 From: Darren J Moffat Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071129182814.GB29689@east.sun.com> Sender: Darren.Moffat@sun.com To: Nils Nieuwejaar Cc: Tim.Marsland@sun.com, PSARC-ext@sun.com Message-id: <474F1660.8060604@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <474F00D5.5060308@Sun.COM> <20071129182814.GB29689@east.sun.com> User-Agent: Thunderbird 2.0.0.6 (X11/20071105) Status: RO Content-Length: 1263 Nils Nieuwejaar wrote: > On Thu 11/29/07 at 18:11 PM, Darren.Moffat@Sun.COM wrote: >> Nils Nieuwejaar wrote: >>> When this proposal was originally sent out, there was a bit of concern >>> expressed on this list about our intention to modify cmdk and possibly some >>> future drivers. Offline, I received some rather stronger objections to >>> this idea, so I have updated the proposal to address those concerns. >>> >>> Rather than modifying the existing cmdk, we are now proposing to create a >>> brand new cmdk that will only be used to access paravirtualized disk >>> drivers. This approach will leave the existing driver completely >>> untouched. >> Please tell me you don't intend to take a copy of the cmdk source to do >> this ? >> Is this going to be a very simple driver that happens to have then name >> cmdk to take advantage of the fact that the driver name is baked in places ? > > Exactly. This is a thin shell driver that does nothing but make LDI calls > to the paravirtual 'xdf' driver. It will be incapable of managing physical > disks, which is why we need to sequester it in the new i86hvm directory. Good, and the only reason for calling it cmdk is because the driver name gets baked into bootenv.rc, right ? -- Darren J Moffat From edward.pilatowicz@sun.com Thu Nov 29 11:46:15 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATJkFCO011149 for ; Thu, 29 Nov 2007 11:46:15 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id lATJkDsK010356 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 29 Nov 2007 11:46:15 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00D9B9L2ZT00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 29 Nov 2007 11:46:14 -0800 (PST) Received: from dm-eng-01.sfbay.sun.com ([129.145.155.198]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA002RF9KXY150@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 29 Nov 2007 11:46:10 -0800 (PST) Received: from mcescher.eng.sun.com (mcescher.SFBay.Sun.COM [129.146.224.55]) by dm-eng-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lATJk8Jo043925; Thu, 29 Nov 2007 11:46:08 -0800 (PST) Received: from mcescher.eng.sun.com (localhost [127.0.0.1]) by mcescher.eng.sun.com (8.14.1+Sun/8.14.1) with ESMTP id lATJk8vp496532; Thu, 29 Nov 2007 11:46:08 -0800 (PST) Received: (from edp@localhost) by mcescher.eng.sun.com (8.14.1+Sun/8.14.1/Submit) id lATJk8LI496531; Thu, 29 Nov 2007 11:46:08 -0800 (PST) Date: Thu, 29 Nov 2007 11:46:08 -0800 From: Edward Pilatowicz Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <474F1660.8060604@Sun.COM> To: Darren J Moffat Cc: Nils Nieuwejaar , Tim.Marsland@sun.com, PSARC-ext@sun.com Message-id: <20071129194608.GD496372@eng.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <474F00D5.5060308@Sun.COM> <20071129182814.GB29689@east.sun.com> <474F1660.8060604@Sun.COM> X-Authentication-warning: mcescher.eng.sun.com: edp set sender to edward.pilatowicz@sun.com using -f User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1389 On Thu, Nov 29, 2007 at 07:43:28PM +0000, Darren J Moffat wrote: > Nils Nieuwejaar wrote: > >On Thu 11/29/07 at 18:11 PM, Darren.Moffat@Sun.COM wrote: > >>Nils Nieuwejaar wrote: > >>>When this proposal was originally sent out, there was a bit of concern > >>>expressed on this list about our intention to modify cmdk and possibly > >>>some > >>>future drivers. Offline, I received some rather stronger objections to > >>>this idea, so I have updated the proposal to address those concerns. > >>> > >>>Rather than modifying the existing cmdk, we are now proposing to create a > >>>brand new cmdk that will only be used to access paravirtualized disk > >>>drivers. This approach will leave the existing driver completely > >>>untouched. > >>Please tell me you don't intend to take a copy of the cmdk source to do > >>this ? > >>Is this going to be a very simple driver that happens to have then name > >>cmdk to take advantage of the fact that the driver name is baked in > >>places ? > > > >Exactly. This is a thin shell driver that does nothing but make LDI calls > >to the paravirtual 'xdf' driver. It will be incapable of managing physical > >disks, which is why we need to sequester it in the new i86hvm directory. > > Good, and the only reason for calling it cmdk is because the driver name > gets baked into bootenv.rc, right ? > basically. it is also baked in /dev links. ed From nn35248@negril.east.sun.com Thu Nov 29 11:55:16 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATJtF4C012025 for ; Thu, 29 Nov 2007 11:55:16 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lATJtBk5015468; Thu, 29 Nov 2007 19:55:13 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00F03A01JT00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 11:55:13 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA002RCA00Y050@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 11:55:12 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATJtCnu000658; Thu, 29 Nov 2007 14:55:12 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lATJtCDu000657; Thu, 29 Nov 2007 14:55:12 -0500 (EST) Date: Thu, 29 Nov 2007 14:55:12 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: To: Tim Marsland Cc: PSARC-ext@sun.com Message-id: <20071129195512.GA534@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1194 On Thu 11/29/07 at 11:34 AM, Tim.Marsland@Sun.COM wrote: > >To ensure that this new PV-aware cmdk is only loaded in HVM > >domains, we > >propose to install it in a new directory: /platform/i86hvm/kernel/ > >drv. In > >HVM domains, this new directory will be prepended to the default > >module > >search path. Just to be clear: we are not proposing to create a > >whole new > >'implementation' called i86hvm. This is just for the PV drivers. > > How will that prepending happen, exactly? I'm still working on the 'exactly'. It will probably involve a change in either startup_modules() or kobj_init(), but I just started looking into this. The 'generally' is that Xen provides a cpuid instruction that allows us to determine whether this is an HVM domain and, if so, which version of Xen we are running on. On non-Xen platforms, this cpuid instruction returns an error. If we determine we are running in an hvm domain on a supported version of Xen, we will add /platform/i86hvm/kernel to the beginning of the default search path. Otherwise, we will leave the search path untouched, and Solaris will access all devices through the (possibly emulated) physical interfaces. Nils From Tim.Marsland@sun.com Thu Nov 29 12:01:25 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATK1Otl012240 for ; Thu, 29 Nov 2007 12:01:25 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lATK1Dll002339; Fri, 30 Nov 2007 04:01:20 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00617AA7EK00@nwk-avmta-2.sfbay.sun.com>; Thu, 29 Nov 2007 12:01:19 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA002Q1AA28I60@nwk-avmta-2.sfbay.sun.com>; Thu, 29 Nov 2007 12:01:15 -0800 (PST) Received: from [10.0.1.6] (vpn-129-150-21-199.SFBay.Sun.COM [129.150.21.199]) by jurassic-x4600.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATK1EWS676148; Thu, 29 Nov 2007 12:01:14 -0800 (PST) Date: Thu, 29 Nov 2007 12:01:13 -0800 From: Tim Marsland Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071129195512.GA534@east.sun.com> To: Nils Nieuwejaar Cc: Tim Marsland , PSARC-ext@sun.com Message-id: MIME-version: 1.0 X-Mailer: Apple Mail (2.752.3) Content-type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <20071129195512.GA534@east.sun.com> Status: RO Content-Length: 1612 On Nov 29, 2007, at 11:55 AM, Nils Nieuwejaar wrote: > On Thu 11/29/07 at 11:34 AM, Tim.Marsland@Sun.COM wrote: >>> To ensure that this new PV-aware cmdk is only loaded in HVM >>> domains, we >>> propose to install it in a new directory: /platform/i86hvm/kernel/ >>> drv. In >>> HVM domains, this new directory will be prepended to the default >>> module >>> search path. Just to be clear: we are not proposing to create a >>> whole new >>> 'implementation' called i86hvm. This is just for the PV drivers. >> >> How will that prepending happen, exactly? > > I'm still working on the 'exactly'. It will probably involve a > change in > either startup_modules() or kobj_init(), but I just started looking > into > this. > > The 'generally' is that Xen provides a cpuid instruction that > allows us to > determine whether this is an HVM domain and, if so, which version > of Xen we > are running on. On non-Xen platforms, this cpuid instruction > returns an > error. > > If we determine we are running in an hvm domain on a supported > version of > Xen, we will add /platform/i86hvm/kernel to the beginning of the > default > search path. Otherwise, we will leave the search path untouched, and > Solaris will access all devices through the (possibly emulated) > physical > interfaces. But up until now (my understanding was) we didn't have to touch the kernel to make these PV drivers work. There's considerable advantage in the perception that we're just adding drivers, rather than patching the kernel .. (Were you *already* patching the kernel and I just didn't realize it?) tim From nn35248@negril.east.sun.com Thu Nov 29 12:17:50 2007 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATKHmnB012935 for ; Thu, 29 Nov 2007 12:17:49 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id lATKHeld011567; Fri, 30 Nov 2007 04:17:46 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA00J03B1L6700@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 12:17:45 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA002KFB1KXP60@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 12:17:45 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATKHivH000707; Thu, 29 Nov 2007 15:17:44 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lATKHip3000706; Thu, 29 Nov 2007 15:17:44 -0500 (EST) Date: Thu, 29 Nov 2007 15:17:44 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: To: Tim Marsland Cc: PSARC-ext@sun.com Message-id: <20071129201744.GB534@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <20071129195512.GA534@east.sun.com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1103 On Thu 11/29/07 at 12:01 PM, Tim.Marsland@Sun.COM wrote: > >If we determine we are running in an hvm domain on a supported version > >of Xen, we will add /platform/i86hvm/kernel to the beginning of the > >default search path. Otherwise, we will leave the search path > >untouched, and Solaris will access all devices through the (possibly > >emulated) physical interfaces. > > But up until now (my understanding was) we didn't have to touch the > kernel to make these PV drivers work. There's considerable advantage in > the perception that we're just adding drivers, rather than patching the > kernel .. > > (Were you *already* patching the kernel and I just didn't realize it?) We can deliver the PV drivers without making any kernel modifications. Providing tranparent access to those drivers through the existing physical interfaces does require some minor kernel modifications. Since we're already going to depend on a kernel patch to deliver these in an update, we decided to use this cleaner i86hvm approach, rather than hanging a PV bag off the side of the existing cmdk driver. Nils From nn35248@negril.east.sun.com Thu Nov 29 13:21:25 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id lATLLOmV014549 for ; Thu, 29 Nov 2007 13:21:25 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id lATLLJor000873; Thu, 29 Nov 2007 21:21:23 GMT Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JSA0060FDZLYK00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 13:21:21 -0800 (PST) Received: from negril.east.sun.com ([129.148.183.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JSA002NNDZKXRB0@nwk-avmta-1.sfbay.Sun.COM>; Thu, 29 Nov 2007 13:21:21 -0800 (PST) Received: from negril.east.sun.com (localhost [127.0.0.1]) by negril.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id lATLLKvG000899; Thu, 29 Nov 2007 16:21:20 -0500 (EST) Received: (from nn35248@localhost) by negril.east.sun.com (8.14.2+Sun/8.14.2/Submit) id lATLLKSH000898; Thu, 29 Nov 2007 16:21:20 -0500 (EST) Date: Thu, 29 Nov 2007 16:21:20 -0500 From: Nils Nieuwejaar Subject: Re: Paravirtualized Drivers for Fully Virtualized xVM Domains [PSARC/2007/664 FastTrack timeout 11/30/2007] In-reply-to: <20071129201744.GB534@east.sun.com> To: Tim Marsland Cc: PSARC-ext@sun.com Message-id: <20071129212120.GA887@east.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200711210427.lAL4RT1u008757@lp64.sfbay.sun.com> <20071129175332.GA29689@east.sun.com> <20071129195512.GA534@east.sun.com> <20071129201744.GB534@east.sun.com> User-Agent: Mutt/1.4.2i Status: RO Content-Length: 1774 On Thu 11/29/07 at 15:17 PM, Nils.Nieuwejaar@Sun.COM wrote: > On Thu 11/29/07 at 12:01 PM, Tim.Marsland@Sun.COM wrote: > > >If we determine we are running in an hvm domain on a supported version > > >of Xen, we will add /platform/i86hvm/kernel to the beginning of the > > >default search path. Otherwise, we will leave the search path > > >untouched, and Solaris will access all devices through the (possibly > > >emulated) physical interfaces. > > > > But up until now (my understanding was) we didn't have to touch the > > kernel to make these PV drivers work. There's considerable advantage in > > the perception that we're just adding drivers, rather than patching the > > kernel .. > > > > (Were you *already* patching the kernel and I just didn't realize it?) > > We can deliver the PV drivers without making any kernel modifications. > Providing tranparent access to those drivers through the existing physical > interfaces does require some minor kernel modifications. Since we're > already going to depend on a kernel patch to deliver these in an update, we > decided to use this cleaner i86hvm approach, rather than hanging a PV bag > off the side of the existing cmdk driver. Upon further reflection, by including new versions of cmdk and cmlb that install in /platform/i86pc rather than /kernel, we should be able to deliver all of this as a simple pkgadd for an update release. These modifications to cmdk and cmlb would not be something we want integrated into ON. They would just be a stopgap measure to simplify the end-user experience for update releases that are already in the field. Upgrading domU to the next update release would remove these modified drivers, and would install a kernel with the necessary i86hvm support built in. Nils From carlsonj@phorcys.east.sun.com Wed Jan 9 09:59:42 2008 Received: from phorcys.east.sun.com (phorcys.East.Sun.COM [129.148.174.143]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m09Hxf5N028878 for ; Wed, 9 Jan 2008 09:59:42 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m09HwMcp013939 for ; Wed, 9 Jan 2008 12:58:22 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.2+Sun/8.14.2/Submit) id m09HwMQZ013936; Wed, 9 Jan 2008 12:58:22 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18309.2877.751750.67959@gargle.gargle.HOWL> Date: Wed, 9 Jan 2008 12:58:21 -0500 From: James Carlson To: psarc-ext@sac.sfbay.sun.com Subject: 2007/664 Paravirtualized Drivers for Fully Virtualized xVM Domains X-Mailer: VM 7.01 under Emacs 21.3.1 Status: RO Content-Length: 365 This fast-track request was approved back on November 28th of last year, but wasn't closed by the owner. I'm closing it now to clean it up. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677