From sacadmin Thu May 29 11:23:59 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4TINwE8010615 for ; Thu, 29 May 2008 11:23:59 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m4TINe11019724 for <@sunmail2sca.sfbay.sun.com:psarc-members@Sun.COM>; Fri, 30 May 2008 02:23:57 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K1N0090Z73WF300@brm-avmta-1.central.sun.com> for psarc-members@Sun.COM (ORCPT psarc-members@Sun.COM); Thu, 29 May 2008 12:23:56 -0600 (MDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K1N006SK73VTA10@brm-avmta-1.central.sun.com> for psarc-members@Sun.COM (ORCPT psarc-members@Sun.COM); Thu, 29 May 2008 12:23:55 -0600 (MDT) Received: from [129.146.11.145] (sr1-jurassic-02.SFBay.Sun.COM [129.146.11.145]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m4TINtXY918626 for ; Thu, 29 May 2008 11:23:55 -0700 (PDT) Date: Thu, 29 May 2008 11:23:55 -0700 From: Kais Belgaied Subject: Materials for the pre-inception of Crossbow PSARC/2006/357 on their way To: psarc-members@sun.com Reply-to: Kais.Belgaied@sun.com Message-id: <483EF4BB.9070206@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.12 (X11/20080228) Status: RO Content-Length: 216 The materials for the pre-inception of Crossbow PSARC/2006/357 are being placed gradually in the pre-inception.materials directory. The complete set will be there by COB today. Apologies for the delay. Kais. From owner-sac-advocates Thu May 29 13:06:35 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4TK6ZBs016245 for ; Thu, 29 May 2008 13:06:35 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4TK5pYn012774; Thu, 29 May 2008 13:05:51 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K1N0070LBTQOJ00@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 13:05:50 -0700 (PDT) Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K1N006G0BTO1750@nwk-avmta-2.sfbay.sun.com> (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 13:05:48 -0700 (PDT) Received: from sac.sfbay.sun.com (new-sac.SFBay.Sun.COM [129.146.175.65]) by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4TK5mj1062128; Thu, 29 May 2008 13:05:48 -0700 (PDT) Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4TK4xHS016195 for ; Thu, 29 May 2008 13:04:59 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4TK4wKe022087 for <@sunmail2sca.sfbay.sun.com:psarc-agenda-announce@sun.com>; Thu, 29 May 2008 14:04:59 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K1N00701BS9MG00@nwk-avmta-2.sfbay.sun.com> for psarc-agenda-announce@sun.com (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 13:04:57 -0700 (PDT) Received: from brmea-mail-2.sun.com ([192.18.98.43]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K1N0068IBS81750@nwk-avmta-2.sfbay.sun.com> for psarc-agenda-announce@sun.com (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 13:04:56 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m4TK4uXS006800 for ; Thu, 29 May 2008 20:04:56 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K1N00801BLXGC00@mail-amer.sun.com> (original mail from Aarti.Pai@Sun.COM) for psarc-agenda-announce@Sun.COM (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 14:04:56 -0600 (MDT) Received: from [129.145.154.55] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K1N00ICFBRZQNC0@mail-amer.sun.com> for psarc-agenda-announce@Sun.COM (ORCPT psarc-agenda-announce@Sun.COM); Thu, 29 May 2008 14:04:48 -0600 (MDT) Date: Thu, 29 May 2008 13:04:47 -0700 From: Aarti Pai Subject: PSARC Agenda - 06/04/2008 - [2008/318], [2007/425], [2006/357] Sender: Aarti.Pai@sun.com To: psarc-agenda-announce@sun.com Cc: Yong Colin Zou , Jennifer Truong Reply-to: Aarti.Pai@sun.com Message-id: <483F0C5F.9070401@Sun.COM> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_PLOcYZymKMDOjCUTej4akg)" X-PMX-Version: 5.4.1.325704 X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.12 (X11/20080228) Status: RO Content-Length: 6445 This is a multi-part message in MIME format. --Boundary_(ID_PLOcYZymKMDOjCUTej4akg) Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ NOTE: We WILL NOT be bridging the internal and external numbers - you *must* call in on the proper number, and you *will* need to hang up and dial in on the other number when we switch between the open and closed sections. Please see agenda below for details. $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ ============================================================================ PSARC - Platform Software Architecture Review Committee PSARC meets weekly on Wednesday: 10am-1pm (Pacific) except on the 3rd Wednesday each month 4pm-7pm (Pacific) on the 3rd Wednesday each month Where: MPK 17 Presidio Conference room, 3rd floor, room 3507 Video: Realtime: http://mpk17-presidio-camera.sfbay.sun.com Low Bandwidth: http://sac.sfbay.sun.com/Reports/PresidioCamera/room.jpg (refresh every 5 minutes) Anyone from Engineering may attend PSARC meetings (unless otherwise noted in the agenda); in addition, most meetings have segments that are open to various open source communities outside of Sun. PSARC meetings are recorded. ============================================================================ TELECONFERENCE NUMBERS: OPEN PSARC MEETINGS: Open Dial in - (866)545-5223 (Within US) - (215)446-3661 (International Access) - x44409 (direct dial inside Sun) Participant Code: 939 5 586# =========================================================================== ARCHITECTURE REVIEW SCHEDULE: 06/04/2008 (Tim, Aarti, Plocher Out; Kais to record) 10:00-10:10 Closed ARC Business (use closed dial in above) 10:10-10:55 Closed Preinception: OSS Audio for Solaris (2008/318 ) Submitter: Garrett D'Amore Owner: Garrett D'Amore Exposure: closed 11:00-11:10 Open ARC Business (use open dial in above) 11:10-11:55 Open Commitment: Wireless USB Support (2007/425 ) Submitter: Colin Zou Owner: Kais Belgaied Intern: Frank Che Exposure: open 11:55-12:30 Open Preinception: Crossbow - Network Virtualization and Resource Management (2006/357 ) Submitter: Kais Belgaied Owner: Kais Belgaied Intern: Michael Speer Exposure: open --Boundary_(ID_PLOcYZymKMDOjCUTej4akg) Content-type: text/html; charset=ISO-8859-1 Content-transfer-encoding: 7BIT   
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
NOTE: We WILL NOT be bridging the internal and external numbers
- you *must* call in on the proper number, and you *will* need to hang
up and dial in on the other number when we switch between the open and
closed sections. Please see agenda below for details.
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
============================================================================

	PSARC - Platform Software Architecture Review Committee

        PSARC meets weekly on Wednesday:
		10am-1pm (Pacific) except on the 3rd Wednesday each month
		 4pm-7pm (Pacific) on the 3rd Wednesday each month

        Where: MPK 17 Presidio Conference room, 3rd floor, room 3507
        Video: Realtime:      http://mpk17-presidio-camera.sfbay.sun.com
               Low Bandwidth: http://sac.sfbay.sun.com/Reports/PresidioCamera/room.jpg
                              (refresh every 5 minutes)

	Anyone from Engineering may attend PSARC meetings (unless otherwise
	noted in the agenda); in addition, most meetings have segments that
	are open to various open source communities outside of Sun.
	PSARC meetings are recorded.

============================================================================
        TELECONFERENCE NUMBERS:

	    OPEN PSARC MEETINGS:

	    Open Dial in - (866)545-5223 (Within US)
		         - (215)446-3661 (International Access)
			 - x44409 (direct dial inside Sun)
	    Participant Code: 939 5 586#

===========================================================================
ARCHITECTURE REVIEW SCHEDULE:

06/04/2008 (Tim, Aarti, Plocher Out; Kais to record)	
    10:00-10:10 Closed ARC Business  (use closed dial in above)
    10:10-10:55	Closed Preinception: OSS Audio for Solaris (2008/318)
		Submitter: Garrett D'Amore
		Owner: Garrett D'Amore
		Exposure: closed
    11:00-11:10 Open ARC Business (use open dial in above)
    11:10-11:55 Open Commitment: Wireless USB Support (2007/425) 
		Submitter: Colin Zou
		Owner: Kais Belgaied
		Intern: Frank Che
		Exposure: open
    11:55-12:30 Open Preinception:  Crossbow - Network Virtualization 
			and Resource Management (2006/357)
		Submitter: Kais Belgaied
		Owner: Kais Belgaied
		Intern: Michael Speer
		Exposure: open
--Boundary_(ID_PLOcYZymKMDOjCUTej4akg)-- From sacadmin Thu Aug 14 10:01:31 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7EH1Ufc023925 for ; Thu, 14 Aug 2008 10:01:30 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m7EH11Xs025864; Thu, 14 Aug 2008 11:01:05 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K5L0050TOLSGB00@nwk-avmta-1.sfbay.Sun.COM>; Thu, 14 Aug 2008 10:01:04 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K5L00G84OLRR6B0@nwk-avmta-1.sfbay.Sun.COM>; Thu, 14 Aug 2008 10:01:04 -0700 (PDT) Received: from [129.146.11.146] (sr1-jurassic-03.SFBay.Sun.COM [129.146.11.146]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m7EH13AP576927; Thu, 14 Aug 2008 10:01:03 -0700 (PDT) Date: Thu, 14 Aug 2008 10:01:03 -0700 From: Kais Belgaied Subject: Commitment Materials for Crossbow - Network Virtualization and Resource Control (PSARC/2006/357) To: PSARC@sun.com Cc: crossbow-interest@sun.com Reply-to: Kais.Belgaied@sun.com Message-id: <48A464CF.8050102@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.14 (X11/20080505) Status: RO Content-Length: 587 ls -l /shared/sac/Archives/CaseLog/arc/PSARC/2006/357/commitment.materials total 1195 -r--r--r-- 1 kais sac 7955 Aug 13 14:40 acctadm.1m.txt -r--r--r-- 1 kais sac 308414 Aug 14 09:26 Crossbow_Design_Doc.pdf -rw-r--r-- 1 kais sac 108371 Aug 14 07:27 Crossbow_Interfaces.pdf -r--r--r-- 1 kais sac 68781 Aug 13 23:40 dladm.1m.diffmk.txt -r--r--r-- 1 kais sac 22699 Aug 14 01:34 flowadm.1m.txt -rw-r--r-- 1 kais sac 837 Aug 14 07:28 README.txt drwxr-sr-x 2 kais sac 8 Aug 14 07:26 References From sac-owner Fri Aug 22 13:44:10 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7MKiAGM010324 for ; Fri, 22 Aug 2008 13:44:10 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m7MKiAOE017663 for <@sunmail2sca.sfbay.sun.com:All-ARCs@sun.com>; Fri, 22 Aug 2008 13:44:10 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K6000203S9M8V00@nwk-avmta-2.sfbay.sun.com> for All-ARCs@sun.com (ORCPT All-ARCs@Sun.COM); Fri, 22 Aug 2008 13:44:10 -0700 (PDT) Received: from brmea-mail-1.sun.com ([192.18.98.31]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K6000CB8S9LQIF0@nwk-avmta-2.sfbay.sun.com> for All-ARCs@sun.com (ORCPT All-ARCs@Sun.COM); Fri, 22 Aug 2008 13:44:09 -0700 (PDT) Received: from fe-amer-09.sun.com ([192.18.109.79]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m7MKi9GX026112 for ; Fri, 22 Aug 2008 20:44:09 +0000 (GMT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K6000501ROBWI00@mail-amer.sun.com> (original mail from Aarti.Pai@Sun.COM) for All-ARCs@Sun.COM (ORCPT All-ARCs@Sun.COM); Fri, 22 Aug 2008 14:44:09 -0600 (MDT) Received: from [129.145.154.87] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K6000BW6S96XQ20@mail-amer.sun.com> for All-ARCs@Sun.COM (ORCPT All-ARCs@Sun.COM); Fri, 22 Aug 2008 14:43:56 -0600 (MDT) Date: Fri, 22 Aug 2008 13:43:54 -0700 From: Aarti Pai Subject: PSARC approved 08/20/2008 - (2006/357) Sender: Aarti.Pai@sun.com To: Aarti Pai Reply-to: Aarti.Pai@sun.com Message-id: <48AF250A.1050102@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.14 (X11/20080505) Status: RO Content-Length: 191 PSARC approved 08/20/2008 Case: Crossbow (2006/357) Incompatible Changes: none Precedent: none If more information is needed, please contact the case owner/intern. From Kais.Belgaied@sun.com Wed Nov 26 19:54:18 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mAR3sIIB018726 for ; Wed, 26 Nov 2008 19:54:18 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mAR3s9RZ012439; Thu, 27 Nov 2008 03:54:15 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KAZ00C0146CFK00@nwk-avmta-2.sfbay.sun.com>; Wed, 26 Nov 2008 19:54:12 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KAZ008RF46B5490@nwk-avmta-2.sfbay.sun.com>; Wed, 26 Nov 2008 19:54:11 -0800 (PST) Received: from [129.146.11.145] (sr1-jurassic-02.SFBay.Sun.COM [129.146.11.145]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mAR3rlkL351812; Wed, 26 Nov 2008 19:53:53 -0800 (PST) Date: Wed, 26 Nov 2008 19:53:44 -0800 From: Kais Belgaied Subject: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning To: PSARC-ext@sun.com Reply-to: Kais.Belgaied@sun.com Message-id: <492E19C8.2010706@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_BQZRgt9R5L77RlKalbxcoA)" X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 8710 This is a multi-part message in MIME format. --Boundary_(ID_BQZRgt9R5L77RlKalbxcoA) Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT Attached is the opinion of the Crossbow project, submitted for PSARC review by December 2nd 2008. Since the commitment review, the project team needed to make four minor changes. The changes were motivated by internal and external feed-back from early adopters and Beta customers, and by the integration with other consumers of the project interfaces. The changes do not constitute any architectural depart from the original specifications voted on, therefore, I'm including them in this opinion. In the case directory, final.materials has the exact specifications, taking into account the commitment review TCR and spec updates. The revised.materials directory there includes the four changes below. I can file a separate fasttrack to cover these changes if members believe it is necessary. - Phased delivery of the some of the resource controls. Initially, maxbw (maximum bandwidth), priority, cpus and fanout properties were approved for flows and datalinks. The support of the cpus property for flows and the fanout property for flows and datalinks are now targeted after the first integration. All other properties, are still supported, and provide sufficient added value for the first phase. - Add a -H option to dladm show-phys and dladm create-vnic. One of the projects added value is exposing a CLI to control the assignment of some of the NICs hardware resources to MAC clients (i.e. VNICs). At commitment time, factory MAC addresses were the only such hardware resource. The need for controlling the assignment of Receive Rings became increasingly obvious during the implementation and Beta testing, thus the -H option. - The integration with new features of LDOMs required the ability to allow an exclusive MAC client to set the interface's MTU, thus a new Consolidation Private MAC client function: mac_set_mtu(). - A limitation in the multi-threadedness of a major NIC device driver necessitated the addition of a flag to request the serialization of transmit operations submitted to that driver. A new flag needed to be added to the Consolidation Private MAC provider interface (MAC_VIRT_SERIALIZE flag of the mac_register_t's m_v12n). Kais. --Boundary_(ID_BQZRgt9R5L77RlKalbxcoA) Content-type: text/plain; name=opinion.ascii Content-transfer-encoding: 7BIT Content-disposition: inline; filename=opinion.ascii # ident "@(#)opinion.ascii 1.4 08/11/26 SMI" sun microsystems Systems Architecture Committee _________________________________________________________________ Copyright 2007 Sun Microsystems, Inc. Subject: Crossbow - Network Virtualization and Resource Partitioning Submitted by: Sunay Tripathi et al. File: PSARC/2006/357/opinion.ascii Date: August 20th, 2008 Committee: Kais Belgaied, Mark Carlson, Garrett D'Amore, Richard Matthiews Product Approval Committee: Solaris PAC solaris-pac@sun.com 1. Summary OpenSolaris Project Crossbow provides the building blocks for network virtualization and resource control by creating virtual stacks around any service (HTTP, HTTPS, FTP, NFS, etc.), protocol (TCP, UDP, SCTP, etc.), or Virtual machines such as Containers and xVM (Xen, and logical domains based hypervisor). Each virtual stack can be assigned its own priority and bandwidth on a shared NIC without causing any performance degradation. The architecture dynamically manages priority and bandwidth resources, and can provide better defense against denial-of-service attacks directed at a particular service or virtual machine by isolating the impact just to that entity. The virtual stacks are separated by means of hardware classification engine such that traffic for one stack does not impact other virtual stacks. 2. Decision & Precedence Information The project is approved as specified in references [1] through [10], but as modified by the required technical changes listed in Appendix A below. The project may be delivered in a minor release of OpenSolaris: 3. Interfaces The project exports the following interfaces. +-----------------------+-------------------------+-----------------------+ |Interface | Classification | Comments | +-----------------------+-------------------------+-----------------------+ |MAC Client Interface | Consolidation Private | See [2],[3],[5] | +-----------------------+-------------------------+-----------------------+ |MAC Provider Interface | Consolidation Private | See [2],[4],[6] | +-----------------------+-------------------------+-----------------------+ |flowadm(1m) | Committed | See [7] | +-----------------------+-------------------------+-----------------------+ |dladm(1m) | Committed | See [2] and [8]. | | | | New subcommands +-----------------------+-------------------------+-----------------------+ |acctadm(1m) | Committed | See [9] and [8]. | | | | New resource type | +-----------------------+-------------------------+-----------------------+ The project obsoletes the following interfaces: . Former Consolidation Private GLDv3 interface for NIC drivers (see [2]): - mc_resources callback. - MAC_CAPAB_MULTIADDRESS capability. . Use of the PPA hack for VLANs designation (see [10]). 4. Opinion 4.1 Attributes vs Properties Some of the reviewers discussed the possible confusion of the meaning of attributes of vs properties for links and flows. During the discussions, a member pointed out that a customers and ISVs may encounter the same ambiguity. The Project team indicated that they are following the same convention as the existing dladm(1m) choices, which were deemed acceptable for the UIRB review. They also clarified that 'attributes' are used for characteristics that define a datalink or a flow, such as selectors for traffic that makes up the flow, whereas properties are assigned to the link or the flow to express a policy on the resources associated with it. 4.2 Output Format One of the reviewers pointed out that the of the metered information produced for flows and always follows the raw format intended for gnuplot. That choice can limit future extentions to support other formats. The issue was resolved by accepting a format specifier option of dladm and flowadm, as captured in TCR#1 below. 4.3 EOF of the VLAN PPA Hack Although the committee members and reviewers agreed on the benefit of obsoleting the VLAN PPA hack, they debated the proper timing of the removal of the hack from OpenSolaris. A member noted the possible disruption to users of this visible feature, even though the release binding is minor. He suggested considering a phased approach, with an announcement of the EOF as part of this project, followed by another fast-track to finish the actual removal of the feature. The project team, while open to the idea of a two-step approach, argued that the co-existence of the old hack with the new architecture was problematic. The project re-designed the VLANs to be created as VNICS, which are MAC level datalinks, so that obeservability features and resource controls can be applied to them. Keeping the old hack will perpetuate the ambiguous behavior when plumbing a non existing interface, result in a duplication of code in the MAC and DLS layers. The answer was satisfactory to the committee. 5. Minority Opinion(s) none. 6. Advisory Information none. 7. Appendices 7.1. Appendix A: Technical Changes Required 1. Change the flowadm(1m) to take a -F for enabling future extentions to more output formats. 7.2. Appendix B: Technical Changes Advised none. 7.3. Appendix C: Reference Material All paths are relative to the revised.materials folder of the case directory. [1] Crossbow_Design_Doc.pdf, Main Architecture & Design document. [2] Crossbow_Interfaces.pdf, Crossbow Interfaces. [3] References/crossbow-virt.pdf, MAC virtualization. Reference for the MAC Client API [4] References/mac_client.h, header file for the MAC Client API [5] References/provider_model.pdf, Reference for the driver (provider) API. [6] References/mac_provider.h, header file for the MAC driver API [7] flowadm.1m.txt, flowadm(1m) draft man page [8] dladm.1m.diffmk.txt, diff-marked dladm(1m) draft man page. [9] acctadm.1m.txt, diff-marked accadm(1m) draft man page. [10] dlpi.7p.txt --Boundary_(ID_BQZRgt9R5L77RlKalbxcoA)-- From gdamore@sun.com Thu Nov 27 07:59:45 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mARFxj5W002874 for ; Thu, 27 Nov 2008 07:59:45 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mARFxdfq063431 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Thu, 27 Nov 2008 08:59:44 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB000H051RJ9500@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 27 Nov 2008 07:59:43 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB0007UB1RJ9OA0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 27 Nov 2008 07:59:43 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mARFxhao021667 for ; Thu, 27 Nov 2008 07:59:43 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KB0005011NWTI00@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Thu, 27 Nov 2008 07:59:42 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KB0006UY1RI3E60@fe-sfbay-09.sun.com>; Thu, 27 Nov 2008 07:59:42 -0800 (PST) Date: Thu, 27 Nov 2008 07:53:16 -0800 From: "Garrett D'Amore" Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <492E19C8.2010706@Sun.COM> Sender: Garrett.Damore@sun.com To: Kais.Belgaied@sun.com Cc: PSARC-ext@sun.com Message-id: <492EC26C.3080608@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 2409 The first sentence in section 4.2 doesn't parse properly. Otherwise it looks OK to me. -- Garrett Kais Belgaied wrote: > Attached is the opinion of the Crossbow project, submitted for PSARC > review by December 2nd 2008. > > Since the commitment review, the project team needed to make four minor > changes. The changes were motivated by internal and external feed-back > from > early adopters and Beta customers, and by the integration with other > consumers of the project interfaces. > > The changes do not constitute any architectural depart from the original > specifications voted on, therefore, I'm including them in this opinion. > In the case directory, final.materials has the exact specifications, > taking into account the commitment review TCR and spec updates. > The revised.materials directory there includes the four changes below. > I can file a separate fasttrack to cover these changes if members > believe it > is necessary. > > - Phased delivery of the some of the resource controls. > Initially, maxbw (maximum bandwidth), priority, cpus and fanout > properties > were approved for flows and datalinks. > The support of the cpus property for flows and the fanout property for > flows and datalinks are now targeted after the first integration. > All other properties, are still supported, and provide sufficient added > value for the first phase. > > - Add a -H option to dladm show-phys and dladm create-vnic. > One of the projects added value is exposing a CLI to control the > assignment > of some of the NICs hardware resources to MAC clients (i.e. VNICs). > At commitment time, factory MAC addresses were the only such hardware > resource. > The need for controlling the assignment of Receive Rings became > increasingly > obvious during the implementation and Beta testing, thus the -H option. > > - The integration with new features of LDOMs required the ability to > allow an exclusive MAC client to set the interface's MTU, > thus a new Consolidation Private MAC client function: mac_set_mtu(). > > - A limitation in the multi-threadedness of a major NIC device driver > necessitated the addition of a flag to request the serialization > of transmit operations submitted to that driver. > A new flag needed to be added to the Consolidation Private MAC provider > interface (MAC_VIRT_SERIALIZE flag of the mac_register_t's m_v12n). > > Kais. From Kais.Belgaied@sun.com Thu Nov 27 11:50:56 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mARJouNJ027598 for ; Thu, 27 Nov 2008 11:50:56 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mARJosrL005443; Thu, 27 Nov 2008 11:50:55 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB000A01CGV3O00@brm-avmta-1.central.sun.com>; Thu, 27 Nov 2008 12:50:55 -0700 (MST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB0008XLCGRAIA0@brm-avmta-1.central.sun.com>; Thu, 27 Nov 2008 12:50:55 -0700 (MST) Received: from [129.146.11.145] (sr1-jurassic-02.SFBay.Sun.COM [129.146.11.145]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mARJoXxl424068; Thu, 27 Nov 2008 11:50:39 -0800 (PST) Date: Thu, 27 Nov 2008 11:50:33 -0800 From: Kais Belgaied Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <492EC26C.3080608@sun.com> To: "Garrett D'Amore" Cc: PSARC-ext@sun.com Reply-to: Kais.Belgaied@sun.com Message-id: <492EFA09.6000102@Sun.COM> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <492EC26C.3080608@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 409 Thanks for review Garrett. Fixed in the case directory. It now reads "One of the reviewers pointed out that the metering information produced for flows and datalinks always follows the raw format intended for gnuplot." Kais. On 11/27/08 07:53, Garrett D'Amore wrote: > The first sentence in section 4.2 doesn't parse properly. > > Otherwise it looks OK to me. > > -- Garrett From carlsonj@phorcys.east.sun.com Mon Dec 1 05:31:57 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1DVudn028235 for ; Mon, 1 Dec 2008 05:31:57 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mB1DVcxQ004188; Mon, 1 Dec 2008 21:31:54 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB7004099L4RM00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 05:31:52 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB70057M9L32TB0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 05:31:51 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB1DVoPp001853; Mon, 01 Dec 2008 08:31:50 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB1DVoSE001850; Mon, 01 Dec 2008 08:31:50 -0500 (EST) Date: Mon, 01 Dec 2008 08:31:50 -0500 From: James Carlson Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <492E19C8.2010706@Sun.COM> To: Kais.Belgaied@sun.com Cc: PSARC-ext@sun.com Message-id: <18739.59206.800639.176726@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> Status: RO Content-Length: 3223 Kais Belgaied writes: > 4.3 EOF of the VLAN PPA Hack > Although the committee members and reviewers agreed on the benefit of > obsoleting the VLAN PPA hack, they debated the proper timing of the > removal of the hack from OpenSolaris. > A member noted the possible disruption to users of this visible > feature, even though the release binding is minor. He suggested > considering a phased approach, with an announcement of the EOF as part > of this project, followed by another fast-track to finish the actual > removal of the feature. The project team, while open to the idea of a > two-step approach, argued that the co-existence of the old hack with > the new architecture was problematic. The project re-designed the > VLANs to be created as VNICS, which are MAC level datalinks, so that > obeservability features and resource controls can be applied to them. > Keeping the old hack will perpetuate the ambiguous behavior when > plumbing a non existing interface, result in a duplication of code in > the MAC and DLS layers. The answer was satisfactory to the committee. I wasn't available for the review, and this is obviously a late time to bring this up, but the answers above sound insufficient to me. Removal of support for the VLAN PPA hack in the kernel itself sounds feasible, but I don't agree that this should be taken as license to be deliberately incompatible on upgrade. Doing so will break machines like jurassic.sfbay and many of our internal boot servers unless the administrators of those systems take deliberate -- and, per the opinion, apparently undocumented -- action prior to upgrade. Alternative plans that would allow removal of the kernel components and still preserve compatibility include at least these two approaches: a) Running a one-time conversion that checks for /etc/hostname.* and /etc/hostname6.* names that specify a VLAN, and automatically running the necessary dladm command to create the VLAN first. The scripting work in the net-physical script to accomplish this is trivial, and looks something like: while [ $# -gt 0 ]; do case $1 in *[0-9][0-9][0-9][0-9]) base=`expr $1 : '\(.*[^0-9]\)'` ppa=`expr $1 : '.*\([0-9][0-9][0-9]\)$' + 0` vid=`expr $1 : '.*[^0-9]\([0-9]*\)[0-9][0-9][0-9]$'` dladm show-link $1 >/dev/null 2>&1 || \ dladm create-vlan -l $base$ppa -v $vid $1 esac /sbin/ifconfig $1 plumb (There are other places this could be done, and it's unclear to me how this interacts with exclusive IP stack zones.) b) Modify ifconfig itself to recognize the VLAN naming pattern and, if the device named doesn't exist, use libdladm to create it first. These are cheap, easy to specify, implement, and test, and would avoid breaking most running systems using VLANs today. It would leave out only ad-hoc snooping on unplumbed VLANs and non-IP protocols run over VLANs. Is there any reason to avoid at least that level of compatibility? -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From gdamore@sun.com Mon Dec 1 08:23:41 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1GNevO018093 for ; Mon, 1 Dec 2008 08:23:40 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mB1GNcPs025530 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 2 Dec 2008 00:23:39 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB700C03HJDL600@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 01 Dec 2008 08:23:37 -0800 (PST) Received: from sca-es-mail-2.sun.com ([192.18.43.133]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB7008Y6HJDAHA0@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Mon, 01 Dec 2008 08:23:37 -0800 (PST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB1GNbDc008886 for ; Mon, 01 Dec 2008 08:23:37 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KB700G01HD9O000@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Mon, 01 Dec 2008 08:23:37 -0800 (PST) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KB700CPLHJ6A720@fe-sfbay-10.sun.com>; Mon, 01 Dec 2008 08:23:30 -0800 (PST) Date: Mon, 01 Dec 2008 08:16:49 -0800 From: "Garrett D'Amore" Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <18739.59206.800639.176726@gargle.gargle.HOWL> Sender: Garrett.Damore@sun.com To: James Carlson Cc: Kais.Belgaied@sun.com, PSARC-ext@sun.com Message-id: <49340DF1.1050505@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 3343 James Carlson wrote: > Kais Belgaied writes: > >> 4.3 EOF of the VLAN PPA Hack >> Although the committee members and reviewers agreed on the benefit of >> obsoleting the VLAN PPA hack, they debated the proper timing of the >> removal of the hack from OpenSolaris. >> A member noted the possible disruption to users of this visible >> feature, even though the release binding is minor. He suggested >> considering a phased approach, with an announcement of the EOF as part >> of this project, followed by another fast-track to finish the actual >> removal of the feature. The project team, while open to the idea of a >> two-step approach, argued that the co-existence of the old hack with >> the new architecture was problematic. The project re-designed the >> VLANs to be created as VNICS, which are MAC level datalinks, so that >> obeservability features and resource controls can be applied to them. >> Keeping the old hack will perpetuate the ambiguous behavior when >> plumbing a non existing interface, result in a duplication of code in >> the MAC and DLS layers. The answer was satisfactory to the committee. >> > > I wasn't available for the review, and this is obviously a late time > to bring this up, but the answers above sound insufficient to me. > > Removal of support for the VLAN PPA hack in the kernel itself sounds > feasible, but I don't agree that this should be taken as license to be > deliberately incompatible on upgrade. Doing so will break machines > like jurassic.sfbay and many of our internal boot servers unless the > administrators of those systems take deliberate -- and, per the > opinion, apparently undocumented -- action prior to upgrade. > > Alternative plans that would allow removal of the kernel components > and still preserve compatibility include at least these two > approaches: > > a) Running a one-time conversion that checks for /etc/hostname.* and > /etc/hostname6.* names that specify a VLAN, and automatically > running the necessary dladm command to create the VLAN first. > The scripting work in the net-physical script to accomplish this > is trivial, and looks something like: > > while [ $# -gt 0 ]; do > case $1 in > *[0-9][0-9][0-9][0-9]) > base=`expr $1 : '\(.*[^0-9]\)'` > ppa=`expr $1 : '.*\([0-9][0-9][0-9]\)$' + 0` > vid=`expr $1 : '.*[^0-9]\([0-9]*\)[0-9][0-9][0-9]$'` > dladm show-link $1 >/dev/null 2>&1 || \ > dladm create-vlan -l $base$ppa -v $vid $1 > esac > /sbin/ifconfig $1 plumb > > (There are other places this could be done, and it's unclear to > me how this interacts with exclusive IP stack zones.) > > b) Modify ifconfig itself to recognize the VLAN naming pattern and, > if the device named doesn't exist, use libdladm to create it > first. > > These are cheap, easy to specify, implement, and test, and would avoid > breaking most running systems using VLANs today. It would leave out > only ad-hoc snooping on unplumbed VLANs and non-IP protocols run over > VLANs. > > Is there any reason to avoid at least that level of compatibility? > > ISTR that we had agreed the team would use an upgrade path akin to option a, during review. I know that this topic was discussed for more than just a minute or two during the meeting. -- Garrett From carlsonj@phorcys.east.sun.com Mon Dec 1 08:35:39 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1GZcL1018218 for ; Mon, 1 Dec 2008 08:35:38 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB1GZcrn024132; Mon, 1 Dec 2008 08:35:38 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB700361I3EIJ00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 08:35:38 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700358I3DE500@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 08:35:37 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB1GZbV2002905; Mon, 01 Dec 2008 11:35:37 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB1GZbqf002902; Mon, 01 Dec 2008 11:35:37 -0500 (EST) Date: Mon, 01 Dec 2008 11:35:37 -0500 From: James Carlson Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <49340DF1.1050505@sun.com> To: "Garrett D'Amore" Cc: Kais.Belgaied@sun.com, PSARC-ext@sun.com Message-id: <18740.4697.195363.101205@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49340DF1.1050505@sun.com> Status: RO Content-Length: 580 Garrett D'Amore writes: > ISTR that we had agreed the team would use an upgrade path akin to > option a, during review. I know that this topic was discussed for more > than just a minute or two during the meeting. OK, then, that seems worthy of a mention as part of the opinion text. Otherwise, it looks incomplete -- issue raised, but no resolution. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From erik.nordmark@sun.com Mon Dec 1 11:45:27 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1JjQJ2024584 for ; Mon, 1 Dec 2008 11:45:26 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mB1Jj84K005026; Tue, 2 Dec 2008 03:45:21 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB70050LQVLA200@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 11:45:21 -0800 (PST) Received: from jurassic.eng.sun.com ([129.146.228.50]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700KPJQVKEX50@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 11:45:20 -0800 (PST) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1JjFb1568988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 01 Dec 2008 11:45:20 -0800 (PST) Date: Mon, 01 Dec 2008 11:45:15 -0800 From: Erik Nordmark Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <18739.59206.800639.176726@gargle.gargle.HOWL> To: James Carlson Cc: Kais.Belgaied@sun.com, PSARC-ext@sun.com Message-id: <49343ECB.1040807@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.17 (X11/20081023) Status: RO Content-Length: 2665 James Carlson wrote: > Removal of support for the VLAN PPA hack in the kernel itself sounds > feasible, but I don't agree that this should be taken as license to be > deliberately incompatible on upgrade. Doing so will break machines > like jurassic.sfbay and many of our internal boot servers unless the > administrators of those systems take deliberate -- and, per the > opinion, apparently undocumented -- action prior to upgrade. I agree it would be good to have an automatic compatibility story on upgrade. > Alternative plans that would allow removal of the kernel components > and still preserve compatibility include at least these two > approaches: > > a) Running a one-time conversion that checks for /etc/hostname.* and > /etc/hostname6.* names that specify a VLAN, and automatically > running the necessary dladm command to create the VLAN first. > The scripting work in the net-physical script to accomplish this > is trivial, and looks something like: > > while [ $# -gt 0 ]; do > case $1 in > *[0-9][0-9][0-9][0-9]) > base=`expr $1 : '\(.*[^0-9]\)'` > ppa=`expr $1 : '.*\([0-9][0-9][0-9]\)$' + 0` > vid=`expr $1 : '.*[^0-9]\([0-9]*\)[0-9][0-9][0-9]$'` > dladm show-link $1 >/dev/null 2>&1 || \ > dladm create-vlan -l $base$ppa -v $vid $1 > esac > /sbin/ifconfig $1 plumb > > (There are other places this could be done, and it's unclear to > me how this interacts with exclusive IP stack zones.) FWIW if you have an exclusive-IP zone which uses bge1003 there wouldn't be a /etc/hostname.bge1003 in the global zone; the zonecfg xml file is the only place the global zone knows about the VLAN. (The non-global zone is likely to have a /etc/hostname.bge1003, but the ngz doesn't have the privileges to create the vlan.) > b) Modify ifconfig itself to recognize the VLAN naming pattern and, > if the device named doesn't exist, use libdladm to create it > first. That one also has issues with exclusive-IP zones since the ifconfig plumb is run in the ngz which doesn't have the privilege to create the vlan. For exclusive-IP zones zoneadm(d) does issue an ioctl to tell the kernel that a datalink name will be used by a zone, but I don't know exactly what gld does with the ioctl and whether than can be extended to create the vlan. Erik > These are cheap, easy to specify, implement, and test, and would avoid > breaking most running systems using VLANs today. It would leave out > only ad-hoc snooping on unplumbed VLANs and non-IP protocols run over > VLANs. > > Is there any reason to avoid at least that level of compatibility? > From carlsonj@phorcys.east.sun.com Mon Dec 1 12:05:24 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1K5Ojq009683 for ; Mon, 1 Dec 2008 12:05:24 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB1K5MDs001574; Mon, 1 Dec 2008 12:05:23 -0800 (PST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB70030LRSZ0S00@nwk-avmta-2.sfbay.sun.com>; Mon, 01 Dec 2008 12:05:23 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700LT0RSX7K50@nwk-avmta-2.sfbay.sun.com>; Mon, 01 Dec 2008 12:05:22 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB1K5Lb6004400; Mon, 01 Dec 2008 15:05:21 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB1K5Lf0004397; Mon, 01 Dec 2008 15:05:21 -0500 (EST) Date: Mon, 01 Dec 2008 15:05:21 -0500 From: James Carlson Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <49343ECB.1040807@sun.com> To: Erik Nordmark Cc: Kais.Belgaied@sun.com, PSARC-ext@sun.com Message-id: <18740.17281.337743.489731@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> Status: RO Content-Length: 2097 Erik Nordmark writes: > > (There are other places this could be done, and it's unclear to > > me how this interacts with exclusive IP stack zones.) > > FWIW if you have an exclusive-IP zone which uses bge1003 there wouldn't > be a /etc/hostname.bge1003 in the global zone; the zonecfg xml file is > the only place the global zone knows about the VLAN. (The non-global > zone is likely to have a /etc/hostname.bge1003, but the ngz doesn't have > the privileges to create the vlan.) I wasn't sure if the VLAN was created in the global zone in that case, or if it was just generated ad-hoc by zoneadmd. If it's the latter, then having the upgrade (or subsequent after-boot but before-zones) script seek out those /etc/zones/*.xml files and do what's necessary via dladm would be the obvious solution. It shouldn't be hard. > > b) Modify ifconfig itself to recognize the VLAN naming pattern and, > > if the device named doesn't exist, use libdladm to create it > > first. > > That one also has issues with exclusive-IP zones since the ifconfig > plumb is run in the ngz which doesn't have the privilege to create the vlan. I didn't expect so. > For exclusive-IP zones zoneadm(d) does issue an ioctl to tell the kernel > that a datalink name will be used by a zone, but I don't know exactly > what gld does with the ioctl and whether than can be extended to create > the vlan. Yes; that'd be the equivalent of (for the global zone) modifying ifconfig. I don't really care which one we pick, but given that the VLAN hack first appeared over 8 years ago and was backported to S8 and S9, I think use is pretty widespread at this point. Even if we think the VLAN hack is ugly (and it _is_ ugly), compatibility -- particularly in administrative interfaces and system configuration, if not kernel behavior -- shouldn't be an optional feature. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From Kais.Belgaied@sun.com Mon Dec 1 13:08:56 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1L8uTk011872 for ; Mon, 1 Dec 2008 13:08:56 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB1L8rBu013726; Mon, 1 Dec 2008 14:08:54 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB700F3FUQUZF00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 13:08:54 -0800 (PST) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.59]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700KROUQTEU90@nwk-avmta-1.sfbay.Sun.COM>; Mon, 01 Dec 2008 13:08:53 -0800 (PST) Received: from [129.146.11.145] (sr1-jurassic-02.SFBay.Sun.COM [129.146.11.145]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB1L8rF2912330; Mon, 01 Dec 2008 13:08:53 -0800 (PST) Date: Mon, 01 Dec 2008 13:08:53 -0800 From: Kais Belgaied Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <18740.17281.337743.489731@gargle.gargle.HOWL> To: James Carlson Cc: Erik Nordmark , PSARC-ext@sun.com, Eric Cheng , Michael Lim Reply-to: Kais.Belgaied@sun.com Message-id: <49345265.2070409@Sun.COM> MIME-version: 1.0 Content-type: multipart/alternative; boundary="Boundary_(ID_lJknSVRbyLlt7bn/nmBAZQ)" X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 6353 This is a multi-part message in MIME format. --Boundary_(ID_lJknSVRbyLlt7bn/nmBAZQ) Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT Here's what the project team (work done by Eric & Mike Cc'ed) implemented to address the upgrade : In both bfu and the postinstall script of SUNWcnetr: - We parse /etc/hostname*.* and come up with list of linknames & VIDs configured in the global zone. - From the output of zonecfg info net of each configured zone, we get the liist of PPA hack VLANs for the zones. - With these two lists, we add the appropriate dladm create-vlan commands to /var/svc/profile/upgrade_datalink Upon reboot, the right vlan data links get created. I'll capture that in the section 4.3 of the opinion. Thanks, Kais & Eric. On 12/01/08 12:05, James Carlson wrote: > Erik Nordmark writes: > >>> (There are other places this could be done, and it's unclear to >>> me how this interacts with exclusive IP stack zones.) >>> >> FWIW if you have an exclusive-IP zone which uses bge1003 there wouldn't >> be a /etc/hostname.bge1003 in the global zone; the zonecfg xml file is >> the only place the global zone knows about the VLAN. (The non-global >> zone is likely to have a /etc/hostname.bge1003, but the ngz doesn't have >> the privileges to create the vlan.) >> > > I wasn't sure if the VLAN was created in the global zone in that case, > or if it was just generated ad-hoc by zoneadmd. > > If it's the latter, then having the upgrade (or subsequent after-boot > but before-zones) script seek out those /etc/zones/*.xml files and do > what's necessary via dladm would be the obvious solution. It > shouldn't be hard. > > >>> b) Modify ifconfig itself to recognize the VLAN naming pattern and, >>> if the device named doesn't exist, use libdladm to create it >>> first. >>> >> That one also has issues with exclusive-IP zones since the ifconfig >> plumb is run in the ngz which doesn't have the privilege to create the vlan. >> > > I didn't expect so. > > >> For exclusive-IP zones zoneadm(d) does issue an ioctl to tell the kernel >> that a datalink name will be used by a zone, but I don't know exactly >> what gld does with the ioctl and whether than can be extended to create >> the vlan. >> > > Yes; that'd be the equivalent of (for the global zone) modifying > ifconfig. > > I don't really care which one we pick, but given that the VLAN hack > first appeared over 8 years ago and was backported to S8 and S9, I > think use is pretty widespread at this point. Even if we think the > VLAN hack is ugly (and it _is_ ugly), compatibility -- particularly in > administrative interfaces and system configuration, if not kernel > behavior -- shouldn't be an optional feature. > > --Boundary_(ID_lJknSVRbyLlt7bn/nmBAZQ) Content-type: text/html; charset=us-ascii Content-transfer-encoding: 7BIT Here's what the project team (work done by Eric & Mike Cc'ed) implemented to address the upgrade :

In both bfu and the postinstall script of SUNWcnetr:
- We parse /etc/hostname*.* and come up with list of linknames & VIDs configured in the global zone.
- From the output of zonecfg info net of each configured zone, we get the liist of PPA hack VLANs for the zones.
- With  these two lists, we add the appropriate  dladm create-vlan commands to /var/svc/profile/upgrade_datalink
Upon reboot, the right vlan data links get created.

I'll capture that in the  section 4.3 of the opinion.

Thanks,

    Kais & Eric.

On 12/01/08 12:05, James Carlson wrote: 
Erik Nordmark writes:
  
     (There are other places this could be done, and it's unclear to
     me how this interacts with exclusive IP stack zones.)
      
FWIW if you have an exclusive-IP zone which uses bge1003 there wouldn't 
be a /etc/hostname.bge1003 in the global zone; the zonecfg xml file is 
the only place the global zone knows about the VLAN. (The non-global 
zone is likely to have a /etc/hostname.bge1003, but the ngz doesn't have 
the privileges to create the vlan.)
    

I wasn't sure if the VLAN was created in the global zone in that case,
or if it was just generated ad-hoc by zoneadmd.

If it's the latter, then having the upgrade (or subsequent after-boot
but before-zones) script seek out those /etc/zones/*.xml files and do
what's necessary via dladm would be the obvious solution.  It
shouldn't be hard.

  
  b) Modify ifconfig itself to recognize the VLAN naming pattern and,
     if the device named doesn't exist, use libdladm to create it
     first.
      
That one also has issues with exclusive-IP zones since the ifconfig 
plumb is run in the ngz which doesn't have the privilege to create the vlan.
    

I didn't expect so.

  
For exclusive-IP zones zoneadm(d) does issue an ioctl to tell the kernel 
that a datalink name will be used by a zone, but I don't know exactly 
what gld does with the ioctl and whether than can be extended to create 
the vlan.
    

Yes; that'd be the equivalent of (for the global zone) modifying
ifconfig.

I don't really care which one we pick, but given that the VLAN hack
first appeared over 8 years ago and was backported to S8 and S9, I
think use is pretty widespread at this point.  Even if we think the
VLAN hack is ugly (and it _is_ ugly), compatibility -- particularly in
administrative interfaces and system configuration, if not kernel
behavior -- shouldn't be an optional feature.

--Boundary_(ID_lJknSVRbyLlt7bn/nmBAZQ)-- From Alan.Coopersmith@sun.com Mon Dec 1 13:12:46 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1LCkfj011964 for ; Mon, 1 Dec 2008 13:12:46 -0800 (PST) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB1LCjEj017255 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 1 Dec 2008 14:12:46 -0700 (MST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB700G03UX7JO00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 01 Dec 2008 13:12:43 -0800 (PST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700KRCUX6ETA0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 01 Dec 2008 13:12:42 -0800 (PST) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB1LCgbG006837 for ; Mon, 01 Dec 2008 13:12:42 -0800 (PST) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KB700L01UL7AN00@fe-sfbay-09.sun.com> (original mail from Alan.Coopersmith@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 01 Dec 2008 13:12:42 -0800 (PST) Received: from almas.sfbay.sun.com ([129.146.106.93]) by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KB7008FEUX5K4E0@fe-sfbay-09.sun.com>; Mon, 01 Dec 2008 13:12:42 -0800 (PST) Date: Mon, 01 Dec 2008 13:12:41 -0800 From: Alan Coopersmith Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <49345265.2070409@Sun.COM> Sender: Alan.Coopersmith@sun.com To: Kais.Belgaied@sun.com Cc: James Carlson , Erik Nordmark , PSARC-ext@sun.com, Eric Cheng , Michael Lim Message-id: <49345349.5030809@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=UTF-8 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Enigmail-Version: 0.95.1 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> <49345265.2070409@Sun.COM> User-Agent: Thunderbird 2.0.0.17 (X11/20081014) Status: RO Content-Length: 824 Kais Belgaied wrote: > Here's what the project team (work done by Eric & Mike Cc'ed) > implemented to address the upgrade : > > In both bfu and the postinstall script of SUNWcnetr: > - We parse /etc/hostname*.* and come up with list of linknames & VIDs > configured in the global zone. > - From the output of zonecfg info net of each configured zone, we get > the liist of PPA hack VLANs for the zones. > - With these two lists, we add the appropriate dladm create-vlan > commands to /var/svc/profile/upgrade_datalink > Upon reboot, the right vlan data links get created. And what about the IPS case, where there is no postinstall script? (I know, not officially ARC'ed, just shipped & deployed & used.) -- -Alan Coopersmith- alan.coopersmith@sun.com Sun Microsystems, Inc. - X Window System Engineering From carlsonj@phorcys.east.sun.com Mon Dec 1 13:21:31 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB1LLUEs012022 for ; Mon, 1 Dec 2008 13:21:31 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id mB1LL9or002224; Tue, 2 Dec 2008 05:21:26 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB70070RVBNT500@nwk-avmta-2.sfbay.sun.com>; Mon, 01 Dec 2008 13:21:23 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB700LPTVBM7BB0@nwk-avmta-2.sfbay.sun.com>; Mon, 01 Dec 2008 13:21:23 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB1LLMmP005133; Mon, 01 Dec 2008 16:21:22 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB1LLMal005130; Mon, 01 Dec 2008 16:21:22 -0500 (EST) Date: Mon, 01 Dec 2008 16:21:22 -0500 From: James Carlson Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <49345265.2070409@Sun.COM> To: Kais.Belgaied@sun.com Cc: Erik Nordmark , PSARC-ext@sun.com, Eric Cheng , Michael Lim Message-id: <18740.21842.592367.452504@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> <49345265.2070409@Sun.COM> Status: RO Content-Length: 1160 Kais Belgaied writes: > Here's what the project team (work done by Eric & Mike Cc'ed) > implemented to address the upgrade : > > In both bfu and the postinstall script of SUNWcnetr: > - We parse /etc/hostname*.* and come up with list of linknames & VIDs > configured in the global zone. > - From the output of zonecfg info net of each configured zone, we get > the liist of PPA hack VLANs for the zones. > - With these two lists, we add the appropriate dladm create-vlan > commands to /var/svc/profile/upgrade_datalink > Upon reboot, the right vlan data links get created. > > I'll capture that in the section 4.3 of the opinion. Sounds good; thanks. Just for my curiosity, what are you doing on the OpenSolaris distribution? Those 'postinstall' scripts don't do anything there. (Is the assumption that the VLAN hack was just never used there because it's more of a server-oriented thing? If so, that may well be sufficient.) -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From erik.nordmark@sun.com Mon Dec 1 16:41:57 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB20funX003634 for ; Mon, 1 Dec 2008 16:41:57 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB20fpWA018410; Tue, 2 Dec 2008 00:41:52 GMT Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB800G0T4LQAK00@brm-avmta-1.central.sun.com>; Mon, 01 Dec 2008 17:41:50 -0700 (MST) Received: from jurassic.eng.sun.com ([129.146.224.130]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB80022B4LQ1MD0@brm-avmta-1.central.sun.com>; Mon, 01 Dec 2008 17:41:50 -0700 (MST) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB20fmGT574553 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 01 Dec 2008 16:41:49 -0800 (PST) Date: Mon, 01 Dec 2008 16:41:48 -0800 From: Erik Nordmark Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <49345265.2070409@Sun.COM> To: Kais.Belgaied@sun.com Cc: James Carlson , PSARC-ext@sun.com, Eric Cheng , Michael Lim Message-id: <4934844C.5020703@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> <49345265.2070409@Sun.COM> User-Agent: Thunderbird 2.0.0.17 (X11/20081023) Status: RO Content-Length: 873 Kais Belgaied wrote: > Here's what the project team (work done by Eric & Mike Cc'ed) > implemented to address the upgrade : > > In both bfu and the postinstall script of SUNWcnetr: > - We parse /etc/hostname*.* and come up with list of linknames & VIDs > configured in the global zone. > - From the output of zonecfg info net of each configured zone, we get > the liist of PPA hack VLANs for the zones. Please verify that you can run zonecfg info for an alternate root which is a different OS release than what is currently booted. > - With these two lists, we add the appropriate dladm create-vlan > commands to /var/svc/profile/upgrade_datalink > Upon reboot, the right vlan data links get created. > > I'll capture that in the section 4.3 of the opinion. Makes sense in the opinion. Project team needs to work out the implementation detail above. Erik From Michael.Lim@sun.com Tue Dec 2 01:25:00 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB29P0Qw013618 for ; Tue, 2 Dec 2008 01:25:00 -0800 (PST) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id mB29Ot4Y027596 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Tue, 2 Dec 2008 01:24:59 -0800 (PST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB80020PSTNC800@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Tue, 02 Dec 2008 02:24:59 -0700 (MST) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB800LJ9STL0V30@brm-avmta-1.central.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.COM); Tue, 02 Dec 2008 02:24:57 -0700 (MST) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id mB29Ovef001045 for ; Tue, 02 Dec 2008 01:24:57 -0800 (PST) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0KB800901ST2KH00@fe-sfbay-10.sun.com> (original mail from Michael.Lim@Sun.COM) for PSARC-ext@Sun.COM (ORCPT PSARC-ext@Sun.COM); Tue, 02 Dec 2008 01:24:57 -0800 (PST) Received: from [129.146.11.144] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0KB80049ASTKJ230@fe-sfbay-10.sun.com>; Tue, 02 Dec 2008 01:24:57 -0800 (PST) Date: Tue, 02 Dec 2008 01:24:56 -0800 From: Michael Lim Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <4934844C.5020703@sun.com> Sender: Michael.Lim@sun.com To: Erik Nordmark Cc: Kais.Belgaied@sun.com, James Carlson , PSARC-ext@sun.com, Eric Cheng Message-id: <4934FEE8.4060605@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> <49345265.2070409@Sun.COM> <4934844C.5020703@sun.com> User-Agent: Thunderbird 2.0.0.16 (X11/20080807) Status: RO Content-Length: 1273 On 12/01/08 16:41, Erik Nordmark wrote: > Kais Belgaied wrote: >> Here's what the project team (work done by Eric & Mike Cc'ed) >> implemented to address the upgrade : >> >> In both bfu and the postinstall script of SUNWcnetr: >> - We parse /etc/hostname*.* and come up with list of linknames & VIDs >> configured in the global zone. >> - From the output of zonecfg info net of each configured zone, we get >> the liist of PPA hack VLANs for the zones. > > Please verify that you can run zonecfg info for an alternate root > which is a different OS release than what is currently booted. > I just verified this with our latest crossbow build running zonecfg against an alternate root with a pre-ip-instances zone. ns-t2000-17: crossbow nemo2: s10 (01/2005) with zone1 # zonecfg -z zone1 -R /net/nemo2 info zonepath zonepath: /net/nemo2/export/zone1 # zonecfg -z zone1 -R /net/nemo2 info ip-type ip-type: shared -Mike >> - With these two lists, we add the appropriate dladm create-vlan >> commands to /var/svc/profile/upgrade_datalink >> Upon reboot, the right vlan data links get created. >> >> I'll capture that in the section 4.3 of the opinion. > > Makes sense in the opinion. Project team needs to work out the > implementation detail above. > > Erik From carlsonj@phorcys.east.sun.com Tue Dec 2 06:23:49 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id mB2ENmQk004399 for ; Tue, 2 Dec 2008 06:23:49 -0800 (PST) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id mB2ENiGd019662; Tue, 2 Dec 2008 14:23:44 GMT Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KB9007016NJIS00@nwk-avmta-2.sfbay.sun.com>; Tue, 02 Dec 2008 06:23:43 -0800 (PST) Received: from phorcys.east.sun.com ([129.148.174.143]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KB9006UD6NI5250@nwk-avmta-2.sfbay.sun.com>; Tue, 02 Dec 2008 06:23:43 -0800 (PST) Received: from phorcys.east.sun.com (localhost [127.0.0.1]) by phorcys.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id mB2ENgjH007774; Tue, 02 Dec 2008 09:23:42 -0500 (EST) Received: (from carlsonj@localhost) by phorcys.east.sun.com (8.14.3+Sun/8.14.3/Submit) id mB2ENgRx007771; Tue, 02 Dec 2008 09:23:42 -0500 (EST) Date: Tue, 02 Dec 2008 09:23:42 -0500 From: James Carlson Subject: Re: Opinion for review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning In-reply-to: <4934844C.5020703@sun.com> To: Erik Nordmark Cc: Kais.Belgaied@sun.com, PSARC-ext@sun.com, Eric Cheng , Michael Lim Message-id: <18741.17646.666825.744803@gargle.gargle.HOWL> MIME-version: 1.0 X-Mailer: VM 7.01 under Emacs 21.3.1 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <492E19C8.2010706@Sun.COM> <18739.59206.800639.176726@gargle.gargle.HOWL> <49343ECB.1040807@sun.com> <18740.17281.337743.489731@gargle.gargle.HOWL> <49345265.2070409@Sun.COM> <4934844C.5020703@sun.com> Status: RO Content-Length: 580 Erik Nordmark writes: > Kais Belgaied wrote: > > - From the output of zonecfg info net of each configured zone, we get > > the liist of PPA hack VLANs for the zones. > > Please verify that you can run zonecfg info for an alternate root which > is a different OS release than what is currently booted. Yes. I added that feature in CR 6436841 for Zulu. -- James Carlson, Solaris Networking Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677 From sac-owner Thu Jul 2 11:06:51 2009 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n62I6pV0018824 for ; Thu, 2 Jul 2009 11:06:51 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n62I6p5t001812 for <@sunmail2sca.sfbay.sun.com:sac-review@Sun.COM>; Thu, 2 Jul 2009 11:06:51 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KM600J032BFZV00@nwk-avmta-1.sfbay.Sun.COM> for sac-review@Sun.COM (ORCPT sac-review@Sun.COM); Thu, 02 Jul 2009 11:06:51 -0700 (PDT) Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KM600JJU2BEN300@nwk-avmta-1.sfbay.Sun.COM> for sac-review@Sun.COM (ORCPT sac-review@Sun.COM); Thu, 02 Jul 2009 11:06:50 -0700 (PDT) Received: from [129.146.11.146] (sr1-jurassic-03.SFBay.Sun.COM [129.146.11.146]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n62I6owA509951 for ; Thu, 02 Jul 2009 11:06:50 -0700 (PDT) Date: Thu, 02 Jul 2009 11:06:50 -0700 From: Kais Belgaied Subject: Opinion for SAC review: 2006/357 Crossbow - Network Virtualization and Resource Partitioning To: sac-review@Sun.COM Reply-to: Kais.Belgaied@Sun.COM Message-id: <4A4CF73A.3030304@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_Zzaun+N4nvsJfDYQ5RK5Fg)" X-PMX-Version: 5.4.1.325704 User-Agent: Thunderbird 2.0.0.19 (X11/20090110) Status: RO Content-Length: 7088 This is a multi-part message in MIME format. --Boundary_(ID_Zzaun+N4nvsJfDYQ5RK5Fg) Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT The comments brought up during the PSARC review round for this opinion have been added. attached is the updated opinion, for review by 07/09/2009 Thanks, Kais. --Boundary_(ID_Zzaun+N4nvsJfDYQ5RK5Fg) Content-type: text/plain; name=opinion.ascii Content-transfer-encoding: 7BIT Content-disposition: inline; filename=opinion.ascii # ident "@(#)opinion.ascii 1.5 08/11/27 SMI" sun microsystems Systems Architecture Committee _________________________________________________________________ Copyright 2007 Sun Microsystems, Inc. Subject: Crossbow - Network Virtualization and Resource Partitioning Submitted by: Sunay Tripathi et al. File: PSARC/2006/357/opinion.ascii Date: August 20th, 2008 Committee: Kais Belgaied, Mark Carlson, Garrett D'Amore, Richard Matthiews Product Approval Committee: Solaris PAC solaris-pac@sun.com 1. Summary OpenSolaris Project Crossbow provides the building blocks for network virtualization and resource control by creating virtual stacks around any service (HTTP, HTTPS, FTP, NFS, etc.), protocol (TCP, UDP, SCTP, etc.), or Virtual machines such as Containers and xVM (Xen, and logical domains based hypervisor). Each virtual stack can be assigned its own priority and bandwidth on a shared NIC without causing any performance degradation. The architecture dynamically manages priority and bandwidth resources, and can provide better defense against denial-of-service attacks directed at a particular service or virtual machine by isolating the impact just to that entity. The virtual stacks are separated by means of hardware classification engine such that traffic for one stack does not impact other virtual stacks. 2. Decision & Precedence Information The project is approved as specified in references [1] through [10], but as modified by the required technical changes listed in Appendix A below. The project may be delivered in a minor release of OpenSolaris: 3. Interfaces The project exports the following interfaces. +-----------------------+-------------------------+-----------------------+ |Interface | Classification | Comments | +-----------------------+-------------------------+-----------------------+ |MAC Client Interface | Consolidation Private | See [2],[3],[5] | +-----------------------+-------------------------+-----------------------+ |MAC Provider Interface | Consolidation Private | See [2],[4],[6] | +-----------------------+-------------------------+-----------------------+ |flowadm(1m) | Committed | See [7] | +-----------------------+-------------------------+-----------------------+ |dladm(1m) | Committed | See [2] and [8]. | | | | New subcommands +-----------------------+-------------------------+-----------------------+ |acctadm(1m) | Committed | See [9] and [8]. | | | | New resource type | +-----------------------+-------------------------+-----------------------+ The project obsoletes the following interfaces: . Former Consolidation Private GLDv3 interface for NIC drivers (see [2]): - mc_resources callback. - MAC_CAPAB_MULTIADDRESS capability. . Use of the PPA hack for VLANs designation (see [10]). 4. Opinion 4.1 Attributes vs Properties Some of the reviewers discussed the possible confusion of the meaning of attributes of vs properties for links and flows. During the discussions, a member pointed out that a customers and ISVs may encounter the same ambiguity. The Project team indicated that they are following the same convention as the existing dladm(1m) choices, which were deemed acceptable for the UIRB review. They also clarified that 'attributes' are used for characteristics that define a datalink or a flow, such as selectors for traffic that makes up the flow, whereas properties are assigned to the link or the flow to express a policy on the resources associated with it. 4.2 Output Format One of the reviewers pointed out that the metering information produced for flows and datalinks always follows the raw format intended for gnuplot. That choice can limit future extentions to support other formats. The issue was resolved by accepting a format specifier option of dladm and flowadm, as captured in TCR#1 below. 4.3 EOF of the VLAN PPA Hack Although the committee members and reviewers agreed on the benefit of obsoleting the VLAN PPA hack, they debated the proper timing of the removal of the hack from OpenSolaris. A member noted the possible disruption to users of this visible feature, even though the release binding is minor. He suggested considering a phased approach, with an announcement of the EOF as part of this project, followed by another fast-track to finish the actual removal of the feature. The project team, while open to the idea of a two-step approach, argued that the co-existence of the old hack with the new architecture was problematic. The project re-designed the VLANs to be created as VNICS, which are MAC level datalinks, so that obeservability features and resource controls can be applied to them. During the opnion review, the ARC and the project team converged to the following change captured in TCR-1 below to address the upgrade from VLANs created the old way: Bfu and the postinstall script of the SUNWcnetr package will be changed to add the necessary lines in upgrade_datalink, so that all VLANs configured for the global zone (/etc/hostname*.*) or for exclusive zones get re-created upon first reboot after the upgrade. 5. Minority Opinion(s) none. 6. Advisory Information none. 7. Appendices 7.1. Appendix A: Technical Changes Required 1. Change the flowadm(1m) to take a -F for enabling future extentions to more output formats. 2. Change bfu and the postinstall script of SUNWcnetr to ensure the re-creation of the existing VLAN interfaces in all zones as vlan-datalinks after the upgrade. 7.2. Appendix B: Technical Changes Advised none. 7.3. Appendix C: Reference Material All paths are relative to the revised.materials folder of the case directory. [1] Crossbow_Design_Doc.pdf, Main Architecture & Design document. [2] Crossbow_Interfaces.pdf, Crossbow Interfaces. [3] References/crossbow-virt.pdf, MAC virtualization. Reference for the MAC Client API [4] References/mac_client.h, header file for the MAC Client API [5] References/provider_model.pdf, Reference for the driver (provider) API. [6] References/mac_provider.h, header file for the MAC driver API [7] flowadm.1m.txt, flowadm(1m) draft man page [8] dladm.1m.diffmk.txt, diff-marked dladm(1m) draft man page. [9] acctadm.1m.txt, diff-marked accadm(1m) draft man page. [10] dlpi.7p.txt --Boundary_(ID_Zzaun+N4nvsJfDYQ5RK5Fg)-- From sac-owner Sun Jul 19 12:16:08 2009 Received: from jurassic-x4600.sfbay.sun.com (cretaceous.SFBay.Sun.COM [129.146.17.63]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n6JJG8BC024309 for ; Sun, 19 Jul 2009 12:16:08 -0700 (PDT) Received: from [10.5.238.201] (sr2-opensolaris.SFBay.Sun.COM [10.5.238.201]) by jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n6JJG8d6362797; Sun, 19 Jul 2009 12:16:08 -0700 (PDT) Message-ID: <4A6370F1.5000901@Sun.COM> Date: Sun, 19 Jul 2009 12:16:01 -0700 From: Kais Belgaied Reply-To: Kais.Belgaied@Sun.COM User-Agent: Thunderbird 2.0.0.19 (X11/20090110) MIME-Version: 1.0 To: sac-opinion@sac.sfbay.sun.com CC: solaris-pac@Sun.COM Subject: Opinion for archiving: 2006/357 Crossbow - Network Virtualization and Resource Partitioning] Content-Type: multipart/mixed; boundary="------------040701070604080302050307" Status: RO Content-Length: 6926 This is a multi-part message in MIME format. --------------040701070604080302050307 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit --------------040701070604080302050307 Content-Type: text/plain; name="opinion.ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="opinion.ascii" # ident "@(#)opinion.ascii 1.5 08/11/27 SMI" sun microsystems Systems Architecture Committee _________________________________________________________________ Copyright 2007 Sun Microsystems, Inc. Subject: Crossbow - Network Virtualization and Resource Partitioning Submitted by: Sunay Tripathi et al. File: PSARC/2006/357/opinion.ascii Date: August 20th, 2008 Committee: Kais Belgaied, Mark Carlson, Garrett D'Amore, Richard Matthiews Product Approval Committee: Solaris PAC solaris-pac@sun.com 1. Summary OpenSolaris Project Crossbow provides the building blocks for network virtualization and resource control by creating virtual stacks around any service (HTTP, HTTPS, FTP, NFS, etc.), protocol (TCP, UDP, SCTP, etc.), or Virtual machines such as Containers and xVM (Xen, and logical domains based hypervisor). Each virtual stack can be assigned its own priority and bandwidth on a shared NIC without causing any performance degradation. The architecture dynamically manages priority and bandwidth resources, and can provide better defense against denial-of-service attacks directed at a particular service or virtual machine by isolating the impact just to that entity. The virtual stacks are separated by means of hardware classification engine such that traffic for one stack does not impact other virtual stacks. 2. Decision & Precedence Information The project is approved as specified in references [1] through [10], but as modified by the required technical changes listed in Appendix A below. The project may be delivered in a minor release of OpenSolaris: 3. Interfaces The project exports the following interfaces. +-----------------------+-------------------------+-----------------------+ |Interface | Classification | Comments | +-----------------------+-------------------------+-----------------------+ |MAC Client Interface | Consolidation Private | See [2],[3],[5] | +-----------------------+-------------------------+-----------------------+ |MAC Provider Interface | Consolidation Private | See [2],[4],[6] | +-----------------------+-------------------------+-----------------------+ |flowadm(1m) | Committed | See [7] | +-----------------------+-------------------------+-----------------------+ |dladm(1m) | Committed | See [2] and [8]. | | | | New subcommands +-----------------------+-------------------------+-----------------------+ |acctadm(1m) | Committed | See [9] and [8]. | | | | New resource type | +-----------------------+-------------------------+-----------------------+ The project obsoletes the following interfaces: . Former Consolidation Private GLDv3 interface for NIC drivers (see [2]): - mc_resources callback. - MAC_CAPAB_MULTIADDRESS capability. . Use of the PPA hack for VLANs designation (see [10]). 4. Opinion 4.1 Attributes vs Properties Some of the reviewers discussed the possible confusion of the meaning of attributes of vs properties for links and flows. During the discussions, a member pointed out that a customers and ISVs may encounter the same ambiguity. The Project team indicated that they are following the same convention as the existing dladm(1m) choices, which were deemed acceptable for the UIRB review. They also clarified that 'attributes' are used for characteristics that define a datalink or a flow, such as selectors for traffic that makes up the flow, whereas properties are assigned to the link or the flow to express a policy on the resources associated with it. 4.2 Output Format One of the reviewers pointed out that the metering information produced for flows and datalinks always follows the raw format intended for gnuplot. That choice can limit future extentions to support other formats. The issue was resolved by accepting a format specifier option of dladm and flowadm, as captured in TCR#1 below. 4.3 EOF of the VLAN PPA Hack Although the committee members and reviewers agreed on the benefit of obsoleting the VLAN PPA hack, they debated the proper timing of the removal of the hack from OpenSolaris. A member noted the possible disruption to users of this visible feature, even though the release binding is minor. He suggested considering a phased approach, with an announcement of the EOF as part of this project, followed by another fast-track to finish the actual removal of the feature. The project team, while open to the idea of a two-step approach, argued that the co-existence of the old hack with the new architecture was problematic. The project re-designed the VLANs to be created as VNICS, which are MAC level datalinks, so that obeservability features and resource controls can be applied to them. During the opnion review, the ARC and the project team converged to the following change captured in TCR-1 below to address the upgrade from VLANs created the old way: Bfu and the postinstall script of the SUNWcnetr package will be changed to add the necessary lines in upgrade_datalink, so that all VLANs configured for the global zone (/etc/hostname*.*) or for exclusive zones get re-created upon first reboot after the upgrade. 5. Minority Opinion(s) none. 6. Advisory Information none. 7. Appendices 7.1. Appendix A: Technical Changes Required 1. Change the flowadm(1m) to take a -F for enabling future extentions to more output formats. 2. Change bfu and the postinstall script of SUNWcnetr to ensure the re-creation of the existing VLAN interfaces in all zones as vlan-datalinks after the upgrade. 7.2. Appendix B: Technical Changes Advised none. 7.3. Appendix C: Reference Material All paths are relative to the revised.materials folder of the case directory. [1] Crossbow_Design_Doc.pdf, Main Architecture & Design document. [2] Crossbow_Interfaces.pdf, Crossbow Interfaces. [3] References/crossbow-virt.pdf, MAC virtualization. Reference for the MAC Client API [4] References/mac_client.h, header file for the MAC Client API [5] References/provider_model.pdf, Reference for the driver (provider) API. [6] References/mac_provider.h, header file for the MAC driver API [7] flowadm.1m.txt, flowadm(1m) draft man page [8] dladm.1m.diffmk.txt, diff-marked dladm(1m) draft man page. [9] acctadm.1m.txt, diff-marked accadm(1m) draft man page. [10] dlpi.7p.txt --------------040701070604080302050307--