From kais@sac.sfbay.sun.com Tue Jul 7 11:49:25 2009 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n67InPc8023480 for ; Tue, 7 Jul 2009 11:49:25 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n67InOcW049521; Tue, 7 Jul 2009 12:49:25 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMF00307DMCKA00@nwk-avmta-2.sfbay.sun.com>; Tue, 07 Jul 2009 11:49:24 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMF00I1EDMABM90@nwk-avmta-2.sfbay.sun.com>; Tue, 07 Jul 2009 11:49:22 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n67InLeR060853; Tue, 07 Jul 2009 11:49:21 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n67InKgl023475; Tue, 07 Jul 2009 11:49:20 -0700 (PDT) Received: (from kais@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n67InKRb023471; Tue, 07 Jul 2009 11:49:20 -0700 (PDT) Date: Tue, 07 Jul 2009 11:49:20 -0700 (PDT) From: Kais Belgaied Subject: Crossbow Import of Interrupt Affinity Interfaces [PSARC/2009/382 Self Review] To: PSARC-ext@sun.com Cc: alan.slivensky@sun.com, govinda.tatti@sun.com, markus.flierl@sun.com, rajagopal.kunhappan@sun.com Message-id: <200907071849.n67InKRb023471@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 2336 Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI This information is Copyright 2009 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: Crossbow Import of Interrupt Affinity Interfaces 1.2. Name of Document Author/Supplier: Author: Rajagopal Kunhappan 1.3 Date of This Document: 07 July, 2009 4. Technical Description This fasttrack covers the changes required for the Crossbow architecture to import the DDI interfaces below, introduced by PSARC 2009/340. The interface taxonomy is Contracted Project Private. A copy of the contract is placed in this case's directory. Since the architectural impact of the interrupt affinity DDI interfaces and its consumers has already been reviewed and approved with PSARC 2009/340, I am marking this case as approved, pending the manager's emails accepting the contracts terms. I'll be happy to set a timer if needed. typedef processorid_t ddi_intr_target_t; int ddi_intr_get_affinity(ddi_intr_handle_t h, ddi_intr_target_t *tgt_p); int ddi_intr_set_affinity(ddi_intr_handle_t h, ddi_intr_target_t tgt); Crossbow framework will be a consumer of these interfaces. More details on Crossbow requirements: 1) Crossbow provides a framework by which NIC resources such as Rx and Tx rings are exposed to the MAC layer. The MAC layer doles out these resources to VNICs when they get created while reserving a fixed amount for the primary NIC. CPUs, on which the processing of packets take place, can be specified at VNIC creation time or later. If they are specified, the interrupts associated with the Rx/Tx rings need to be re-targeted to the specified CPUs. A mechanism by which a specific MSI-X interrupt can be re-targeted to a different CPU is needed. This is for the virtualization part of Crossbow. 2) For optimal performance of regular NICs (as well as VNICs), the poll thread associated with an Rx ring should be bound to the same CPU as the interrupt CPU. So given an interrupt handle and a CPU, a mechanism is needed to re-target the interrupt to the specified CPU. The above 2 requirements are addressed by the interfaces introduced in PSARC 2009/340. 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: Automatic 6.6. ARC Exposure: open From Markus.Flierl@sun.com Tue Jul 7 13:53:01 2009 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n67Kr0EW028693 for ; Tue, 7 Jul 2009 13:53:00 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n67Kquki024630 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 8 Jul 2009 04:52:59 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0KMF00B07JC9JY00@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 07 Jul 2009 13:52:57 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0KMF009QIJC8O420@nwk-avmta-2.sfbay.sun.com> for PSARC-ext@sun.com (ORCPT PSARC-ext@Sun.Com); Tue, 07 Jul 2009 13:52:57 -0700 (PDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n67Kquhr013930 for ; Tue, 07 Jul 2009 13:52:56 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) id <0KMF00M00J5MLC00@fe-sfbay-09.sun.com> for PSARC-ext@Sun.Com (ORCPT PSARC-ext@Sun.Com); Tue, 07 Jul 2009 13:52:56 -0700 (PDT) Received: from [129.145.154.53] ([unknown] [129.145.154.53]) by fe-sfbay-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.02 64bit (built Apr 16 2009)) with ESMTPSA id <0KMF00M0TJC7JY10@fe-sfbay-09.sun.com>; Tue, 07 Jul 2009 13:52:56 -0700 (PDT) Date: Tue, 07 Jul 2009 13:52:55 -0700 From: Markus Flierl Subject: Re: Crossbow Import of Interrupt Affinity Interfaces [PSARC/2009/382 Self Review] In-reply-to: <200907071849.n67InKRb023471@sac.sfbay.sun.com> Sender: Markus.Flierl@sun.com To: Kais Belgaied Cc: PSARC-ext@sun.com, Alan.Slivensky@sun.com, Govinda.Tatti@sun.com, Rajagopal.Kunhappan@sun.com Reply-to: Markus.Flierl@sun.com Message-id: <4A53B5A7.9000506@Sun.COM> MIME-version: 1.0 Content-type: text/plain; CHARSET=US-ASCII; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200907071849.n67InKRb023471@sac.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.19 (X11/20090110) Status: RO Content-Length: 2658 I accept. Markus On 07/07/09 11:49, Kais Belgaied wrote: > Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI > This information is Copyright 2009 Sun Microsystems > 1. Introduction > 1.1. Project/Component Working Name: > Crossbow Import of Interrupt Affinity Interfaces > 1.2. Name of Document Author/Supplier: > Author: Rajagopal Kunhappan > 1.3 Date of This Document: > 07 July, 2009 > 4. Technical Description > This fasttrack covers the changes required for the Crossbow architecture > to import the DDI interfaces below, introduced by PSARC 2009/340. > The interface taxonomy is Contracted Project Private. A copy of > the contract is placed in this case's directory. > > Since the architectural impact of the interrupt affinity DDI interfaces and > its consumers has already been reviewed and approved with PSARC 2009/340, > I am marking this case as approved, pending the manager's emails > accepting the contracts terms. I'll be happy to set a timer if > needed. > > typedef processorid_t ddi_intr_target_t; > > int ddi_intr_get_affinity(ddi_intr_handle_t h, ddi_intr_target_t > *tgt_p); > int ddi_intr_set_affinity(ddi_intr_handle_t h, ddi_intr_target_t tgt); > > Crossbow framework will be a consumer of these interfaces. > > More details on Crossbow requirements: > 1) Crossbow provides a framework by which NIC resources such as Rx and > Tx rings are exposed to the MAC layer. The MAC layer doles out these > resources to VNICs when they get created while reserving a fixed amount > for the primary NIC. CPUs, on which the processing of packets take > place, can be specified at VNIC creation time or later. If they are > specified, the interrupts associated with the Rx/Tx rings need to be > re-targeted to the specified CPUs. A mechanism by which a specific MSI-X > interrupt can be re-targeted to a different CPU is needed. This is for > the virtualization part of Crossbow. > > 2) For optimal performance of regular NICs (as well as VNICs), the poll > thread associated with an Rx ring should be bound to the same CPU as the > interrupt CPU. So given an interrupt handle and a CPU, a mechanism is > needed to re-target the interrupt to the specified CPU. > > The above 2 requirements are addressed by the interfaces introduced in > PSARC 2009/340. > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: Automatic > 6.6. ARC Exposure: open > > -- Markus Flierl Sr. Manager, Solaris Core OS 17 Network Circle, MS UMPK17-202 Menlo Park, CA 94025 phone: 650-786-2056 http://blogs.sun.com/markusflierl