From jhd@xenbld.sfbay.sun.com Wed Aug 29 11:01:05 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7TI15bg019455 for ; Wed, 29 Aug 2007 11:01:05 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7THwNqZ024841; Wed, 29 Aug 2007 10:58:26 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNJ0031NR9CP400@brm-avmta-1.central.sun.com>; Wed, 29 Aug 2007 11:58:24 -0600 (MDT) Received: from jurassic.eng.sun.com ([129.146.58.166]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNJ00FDOR9B7BD0@brm-avmta-1.central.sun.com>; Wed, 29 Aug 2007 11:58:23 -0600 (MDT) Received: from xenbld.SFBay.Sun.COM (xenbld.SFBay.Sun.COM [192.168.79.55]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7THwCYu548236 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 29 Aug 2007 10:58:12 -0700 (PDT) Received: from xenbld.SFBay.Sun.COM (localhost [127.0.0.1]) by xenbld.SFBay.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7THw3rc005404; Wed, 29 Aug 2007 10:58:03 -0700 (PDT) Received: (from jhd@localhost) by xenbld.SFBay.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l7THw3bs005401; Wed, 29 Aug 2007 10:58:03 -0700 (PDT) Date: Wed, 29 Aug 2007 10:58:03 -0700 (PDT) From: John Danielson Subject: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] To: PSARC-ext@sun.com Cc: matrix-eng@sun.com Message-id: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 Status: RO Content-Length: 2565 Template Version: @(#)sac_nextcase 1.64 07/13/07 SMI This information is Copyright 2007 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: SMF services for Xen addendum 1.2. Name of Document Author/Supplier: Author: John Levon 1.3 Date of This Document: 29 August, 2007 4. Technical Description SMF services for Xen addendum 1. Introduction PSARC 2007/127 introduced the new SMF services provided for running a Solaris-based domain 0 when running on Xen or a Xen-compatible hypervisor. Further work has generated the need for an additional service, described below. The "platform profiles" problem described in PSARC 2007/127 applies equally well to this service. As before, a patch binding is requested. 2. Domains service Solaris zones provides a simple transient service that allows autobooting of zones when the control plane (global zone) is booted, and graceful shutdown at shutdown/reboot time. In a similar fashion this amendment is for the transient service: svc:/system/xctl/domains:default Behaviour is very similar to that of system/zones:default; guest domains marked as autoboot are booted at dom0 boot time, and are shut down cleanly at shutdown. This is achieved by having the 'domains' service depend upon xend:default. When the service is started, we ask xend to autoboot all domains as needed. On shutdown, we ask xend to shutdown all running domains and wait for this to complete (timing out after 6 minutes if no further progress is being made). Note that a restart of xend should not cause guest domains to be restarted. This is the current behaviour of the upstream community version of Xen. We have avoided this behaviour by making sure the domains service does not restart on a restart of its dependent services, xend and xenconsoled. We intend to work with the community to fix this defect in the upstream version in a similar fashion. 3. Interface table It is unclear that the interfaces provided in this document will remain stable as upstream community development continues. In particular, forthcoming releases are providing a radical new base for administrative remote access to the control tools. As a result, we feel that the interfaces described here are best represented by a classification of Volatile: svc:/system/xctl/domains:default Volatile 4. References PSARC/2006/260 Solaris on Xen PSARC/2007/128 SMF services for Xen From sommerfeld@sun.com Wed Aug 29 11:11:56 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7TIBtbF019835 for ; Wed, 29 Aug 2007 11:11:55 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7TI94Mf009040; Thu, 30 Aug 2007 02:09:15 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNJ00E0FRREMT00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 11:09:14 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNJ005RTRRCMA50@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 11:09:13 -0700 (PDT) Received: from thunk.east.sun.com (thunk.East.Sun.COM [129.148.174.66]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id l7TI9CQF027053; Wed, 29 Aug 2007 14:09:12 -0400 (EDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.1+Sun/8.14.1) with ESMTP id l7TI9CFs004568; Wed, 29 Aug 2007 14:09:12 -0400 (EDT) Date: Wed, 29 Aug 2007 14:09:11 -0400 From: Bill Sommerfeld Subject: Re: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] In-reply-to: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> To: John Danielson Cc: PSARC-ext@sun.com, matrix-eng@sun.com Message-id: <1188410951.3850.65.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.10.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 References: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> Status: RO Content-Length: 801 On Wed, 2007-08-29 at 10:58 -0700, John Danielson wrote: > On shutdown, we ask xend to shutdown all running domains and wait for this > to complete (timing out after 6 minutes if no further progress is being > made). where did "6 minutes" come from? existing practice in the open-source hypervisor code? > As a result, we feel that the > interfaces described here are best represented by a classification of > Volatile: I'd think the interface that needs to be stable is the one for marking domains as autoboot; exactly which SMF service makes this happen shouldn't matter (and IMHO inter-domain dependencies should be treated like inter-host dependencies in networking: services must be able to cope with an unexpected disappearance or restart of a peer). - Bill From johnlev@barman.uk.sun.com Wed Aug 29 11:22:26 2007 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7TIMQkq020324 for ; Wed, 29 Aug 2007 11:22:26 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7TIJJYD039050; Wed, 29 Aug 2007 12:19:22 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNJ00F0DS8XLB00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 11:19:45 -0700 (PDT) Received: from dm-uk-02.uk.sun.com ([129.156.101.196]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNJ005ZMS8WMH60@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 11:19:45 -0700 (PDT) Received: from barman.uk.sun.com (barman.UK.Sun.COM [129.156.132.12]) by dm-uk-02.uk.sun.com (8.13.7+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7TIJhKT014806; Wed, 29 Aug 2007 19:19:43 +0100 (BST) Received: from johnlev by barman.uk.sun.com with local (Exim 4.42) id 1IQSAd-0001iM-FB; Wed, 29 Aug 2007 19:21:39 +0100 X-URL: http://jurassic.eng/~johnlev/ Date: Wed, 29 Aug 2007 19:21:39 +0100 From: John Levon Subject: Re: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] In-reply-to: <1188410951.3850.65.camel@thunk> Sender: John Levon To: Bill Sommerfeld Cc: John Danielson , PSARC-ext@sun.com, matrix-eng@sun.com Message-id: <20070829182139.GC1682@barman.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> <1188410951.3850.65.camel@thunk> User-Agent: Mutt/1.5.6i Status: RO Content-Length: 956 On Wed, Aug 29, 2007 at 02:09:11PM -0400, Bill Sommerfeld wrote: > > On shutdown, we ask xend to shutdown all running domains and wait for this > > to complete (timing out after 6 minutes if no further progress is being > > made). > > where did "6 minutes" come from? existing practice in the open-source > hypervisor code? After 5 minutes, xend will log a message that the domain is unresponsive and mark it as such in the domain store. Previous/other versions of Xen would kill the domain at the 5-minute mark. So the 6 minutes is previous behaviour plus a little bit of slop. > > As a result, we feel that the > > interfaces described here are best represented by a classification of > > Volatile: > > I'd think the interface that needs to be stable is the one for marking > domains as autoboot This interface comes from upstream and is thus subject to the same rules as described in the umbrella Xen case. regards john From boyd-adamson@usa.net Wed Aug 29 15:33:39 2007 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7TMXcmC028209 for ; Wed, 29 Aug 2007 15:33:38 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7TMUvPl008926 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 29 Aug 2007 23:30:58 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNK00J0F3VK8800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 29 Aug 2007 16:30:56 -0600 (MDT) Received: from sca-ea-mail-2.sun.com ([192.18.43.25]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNK00J303VI6I00@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 29 Aug 2007 16:30:55 -0600 (MDT) Received: from relay21.sun.com (relay21.sun.com [192.12.251.14] (may be forged)) by sca-ea-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id l7TM5oho020113 for ; Wed, 29 Aug 2007 22:30:54 +0000 (GMT) Received: from mms23es.sun.com ([150.143.232.54] [150.143.232.54]) by relay25i.sun.com with ESMTP id BT-MMP-290043; Wed, 29 Aug 2007 22:30:50 +0000 (Z) Received: from relay24.sun.com (relay24.sun.com [192.12.251.74]) by mms23es.sun.com with ESMTP id BT-MMP-219630; Wed, 29 Aug 2007 22:30:50 +0000 (Z) Received: from ipmail03.adl2.internode.on.net ([203.16.214.135] [203.16.214.135]) by relay24i.sun.com with ESMTP id BT-MMP-1719499; Wed, 29 Aug 2007 22:30:49 +0000 (Z) Received: from ppp157-138.static.internode.on.net (HELO whirlpool) ([150.101.157.138]) by ipmail03.adl2.internode.on.net with ESMTP; Thu, 30 Aug 2007 08:00:47 +0930 Received: from eddy.tactio.lan ([10.11.12.2] ident=brontitall) by whirlpool with esmtp (Exim 3.36 #1 (Debian)) id 1IQW3i-0002nB-00; Thu, 30 Aug 2007 08:30:46 +1000 Date: Thu, 30 Aug 2007 08:30:46 +1000 From: Boyd Adamson Subject: Re: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] In-reply-to: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> To: John Danielson Cc: PSARC-ext@sun.com, matrix-eng@sun.com Message-id: <87ejhmug6h.fsf@eddy.tactio.lan> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.2.0.264296 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ao8CAC+O1UaWZZ2K/2dsb2JhbAA X-IronPort-AV: E=Sophos;i="4.19,323,1183300200"; d="scan'208";a="139953010" References: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux) Status: RO Content-Length: 1495 John Danielson writes: > 2. Domains service > > Solaris zones provides a simple transient service that allows autobooting > of zones when the control plane (global zone) is booted, and graceful > shutdown at shutdown/reboot time. In a similar fashion this amendment is > for the transient service: > > svc:/system/xctl/domains:default > > Behaviour is very similar to that of system/zones:default; guest domains > marked as autoboot are booted at dom0 boot time, and are shut down cleanly > at shutdown. > > This is achieved by having the 'domains' service depend upon xend:default. > When the service is started, we ask xend to autoboot all domains as needed. > > On shutdown, we ask xend to shutdown all running domains and wait for this > to complete (timing out after 6 minutes if no further progress is being > made). One thing I've heard a few times from customers using zones is that sometimes they would like to be able to offer ordering and dependency of zone startup, so that zone A has to finish starting before zone B is started. It seems that a similar thing may be of interest with Xen domains. Perhaps an implementation where each domain is an instance of svc:/system/xctl/domains could allow dependencies? e.g.: svc:/system/xctl/domains:domain-a svc:/system/xctl/domains:domain-b On the other hand, I don't know how you determine from dom0 that a domU domain has "started" completely. Boyd From Nicolas.Williams@sun.com Wed Aug 29 15:46:58 2007 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7TMkwLB028458 for ; Wed, 29 Aug 2007 15:46:58 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id l7TMiIFi018547; Wed, 29 Aug 2007 15:44:18 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNK00E014HUOA00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 15:44:18 -0700 (PDT) Received: from binky.Central.Sun.COM ([129.153.128.104]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNK00A6E4HUF940@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 15:44:18 -0700 (PDT) Received: from binky.Central.Sun.COM (localhost [127.0.0.1]) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1) with ESMTP id l7TMiI4G019553; Wed, 29 Aug 2007 17:44:18 -0500 (CDT) Received: (from nw141292@localhost) by binky.Central.Sun.COM (8.14.1+Sun/8.14.1/Submit) id l7TMiHl0019552; Wed, 29 Aug 2007 17:44:17 -0500 (CDT) Date: Wed, 29 Aug 2007 17:44:17 -0500 From: Nicolas Williams Subject: Re: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] In-reply-to: <87ejhmug6h.fsf@eddy.tactio.lan> To: Boyd Adamson Cc: John Danielson , PSARC-ext@sun.com, matrix-eng@sun.com Message-id: <20070829224417.GH16963@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> <87ejhmug6h.fsf@eddy.tactio.lan> X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to Nicolas.Williams@sun.com using -f User-Agent: Mutt/1.5.7i Status: RO Content-Length: 1072 On Thu, Aug 30, 2007 at 08:30:46AM +1000, Boyd Adamson wrote: > One thing I've heard a few times from customers using zones is that > sometimes they would like to be able to offer ordering and dependency of > zone startup, so that zone A has to finish starting before zone B is > started. > > It seems that a similar thing may be of interest with Xen > domains. Perhaps an implementation where each domain is an instance of > svc:/system/xctl/domains could allow dependencies? e.g.: > > svc:/system/xctl/domains:domain-a > svc:/system/xctl/domains:domain-b > > On the other hand, I don't know how you determine from dom0 that a domU > domain has "started" completely. Prior to the advent of virtualization technologies one simply had to have apps hosted on different systems, but with inter-dependencies, just deal. ISTM that the feature you request wouldn't get you past application restart-related issues after boot, so I don't see the value in moving remote service monitoring from where it belongs (in the zones/domain) to the global zone. Just my 2c, Nico -- From johnlev@barman.uk.sun.com Wed Aug 29 18:03:31 2007 Received: from sunmail4.Singapore.Sun.COM (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id l7U13UDw001918 for ; Wed, 29 Aug 2007 18:03:31 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.Singapore.Sun.COM (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id l7U10jPL025649; Thu, 30 Aug 2007 09:00:50 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0JNK0041DATBL600@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 18:00:47 -0700 (PDT) Received: from dm-uk-02.uk.sun.com ([129.156.101.196]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0JNK0017VAT8UO90@nwk-avmta-1.sfbay.Sun.COM>; Wed, 29 Aug 2007 18:00:45 -0700 (PDT) Received: from barman.uk.sun.com (barman.UK.Sun.COM [129.156.132.12]) by dm-uk-02.uk.sun.com (8.13.7+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id l7U10hOZ003776; Thu, 30 Aug 2007 02:00:43 +0100 (BST) Received: from johnlev by barman.uk.sun.com with local (Exim 4.42) id 1IQYQh-0004OL-6D; Thu, 30 Aug 2007 02:02:39 +0100 X-URL: http://jurassic.eng/~johnlev/ Date: Thu, 30 Aug 2007 02:02:39 +0100 From: John Levon Subject: Re: SMF services for Xen addendum [PSARC/2007/497 FastTrack timeout 09/05/2007] In-reply-to: <87ejhmug6h.fsf@eddy.tactio.lan> Sender: John Levon To: Boyd Adamson Cc: John Danielson , PSARC-ext@Sun.COM, matrix-eng@Sun.COM Message-id: <20070830010238.GC16347@barman.uk.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.2.0.264296 References: <200708291758.l7THw3bs005401@xenbld.SFBay.Sun.COM> <87ejhmug6h.fsf@eddy.tactio.lan> User-Agent: Mutt/1.5.6i Status: RO Content-Length: 929 On Thu, Aug 30, 2007 at 08:30:46AM +1000, Boyd Adamson wrote: > It seems that a similar thing may be of interest with Xen > domains. Perhaps an implementation where each domain is an instance of > svc:/system/xctl/domains could allow dependencies? e.g.: > > svc:/system/xctl/domains:domain-a > svc:/system/xctl/domains:domain-b This is a somewhat obviously useful direction to go in, but not necessarily for ordering problems, as Nicholas quite rightly points out. The current implementation is careful to leave this option open, as is the current zones implementation. > On the other hand, I don't know how you determine from dom0 that a domU > domain has "started" completely. There's no good answer to that. What does "started completely" even mean? It entirely depends on what you're asking and why. In such circumstances, a feature-based test (even if it must be a polling loop) is still the best option. regards john