From tedk@sac.sfbay.sun.com Wed May 19 16:38:57 2010 Received: from sunmail6brm.central.sun.com (sunmail6brm.Central.Sun.COM [129.147.4.169]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4JNcufl018107 for ; Wed, 19 May 2010 16:38:56 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail6brm.central.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4JNct4L000300; Wed, 19 May 2010 18:38:56 -0500 (CDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2O00005XOWMR00@nwk-avmta-1.sfbay.Sun.COM>; Wed, 19 May 2010 16:38:56 -0700 (PDT) Received: from dm-sfbay-01.sfbay.sun.com ([129.145.155.118]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2O003GAXOVO760@nwk-avmta-1.sfbay.Sun.COM>; Wed, 19 May 2010 16:38:55 -0700 (PDT) Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132]) by dm-sfbay-01.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4JNctvF020465; Wed, 19 May 2010 16:38:55 -0700 (PDT) Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4JNcsQM018101; Wed, 19 May 2010 16:38:54 -0700 (PDT) Received: (from tedk@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id o4JNcswV018097; Wed, 19 May 2010 16:38:54 -0700 (PDT) Date: Wed, 19 May 2010 16:38:54 -0700 (PDT) From: Ted Kim Subject: Multiple IPoIB instances per PKEY [PSARC/2010/182 FastTrack timeout 05/26/2010] To: PSARC-ext@sun.com Cc: ted.h.kim@oracle.com Message-id: <201005192338.o4JNcswV018097@sac.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 2466 Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. 1. Introduction 1.1. Project/Component Working Name: Multiple IPoIB instances per PKEY 1.2. Name of Document Author/Supplier: Author: Sudhakar Dindukurti 1.3 Date of This Document: 19 May, 2010 4. Technical Description 4.1 Acronyms P_Key : Partition Key IPoIB : IP over InfiniBand 4.2 Problem Today the IPoIB driver allows you to create only one IPoIB instance (aka IB partition) per P_Key per port. As a result, on a multi-zone environment, it is not possible to assign one IPoIB instance per "Exclusive-IP stack" zone with the same P_Key. The work around is to use "Shared-IP" mode when creating the zone which could potentially lead to performance /scaling issues. Historically, this limitation has existed since IPoIB was first implemented on Solaris. But it is unnecessary, and now it is becoming a hindrance. 4.2 Proposal A micro/patch binding is asserted for this proposal. This case proposes to enhance IPoIB to support multiple IPoIB instances per P_Key per port. With PSARC 2010/085 administration enhancements, removing the limitation can be easily fixed as the system administrator creates IPoIB instances as needed using dladm(1M). Before PSARC 2010/085, IPoIB instances were automatically instantiated by the IBTF framework and so removing the limitation was not easy to fix. The IPoIB instance after this enhancement acts semantically the same as before in sending and receiving unicast, multicast, and broadcast traffic. For example, consider two IB systems, each system with one HCA and two ports. On the first system, two IPoIB instances (part0 and part1) are created on the same port with P_Key1. On the second system, two IPoIB instances are created (part3 and part4) on different ports with the same P_Key1. In this example, all the IPoIB instances act semantically same. Note: Man page does not make reference to the one part per port/pkey limitation and so no edit is necessary for the man page. 4.4 References PSARC 2001/289 IP over InfiniBand PSARC 2010/085 IPoIB Administration Enhancement 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From garrett.damore@oracle.com Wed May 19 20:39:55 2010 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4K3dsgs021214 for ; Wed, 19 May 2010 20:39:54 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id o4K3ds1X002576 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Wed, 19 May 2010 20:39:54 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L2P00K038UIIZ00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 20:39:54 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L2P009CQ8UITV60@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Wed, 19 May 2010 20:39:54 -0700 (PDT) Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4K3dqWQ002951; Thu, 20 May 2010 03:39:52 +0000 (GMT) Received: from acsmt355.oracle.com (acsmt355.oracle.com [141.146.40.155]) by rcsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4K2e3jQ004242; Thu, 20 May 2010 03:39:51 +0000 (GMT) Received: from abhmt017.oracle.com by acsmt353.oracle.com with ESMTP id 283112781274326764; Wed, 19 May 2010 20:39:24 -0700 Received: from [10.7.251.172] (/10.7.251.172) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 May 2010 20:39:23 -0700 Date: Wed, 19 May 2010 20:39:22 -0700 From: "Garrett D'Amore" Subject: Re: Multiple IPoIB instances per PKEY [PSARC/2010/182 FastTrack timeout 05/26/2010] In-reply-to: <201005192338.o4JNcswV018097@sac.sfbay.sun.com> To: Ted Kim Cc: PSARC-ext@sun.com, ted.h.kim@oracle.com Message-id: <4BF4AEEA.4090205@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt355.oracle.com [141.146.40.155] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090203.4BF4AF07.01B4:SCFMA4539814,ss=1,fgs=0 References: <201005192338.o4JNcswV018097@sac.sfbay.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1.8) Gecko/20100412 Lightning/1.0b1 Thunderbird/3.0.3 Status: RO Content-Length: 2650 +1. - Garrett On 05/19/10 04:38 PM, Ted Kim wrote: > Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI > This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. > 1. Introduction > 1.1. Project/Component Working Name: > Multiple IPoIB instances per PKEY > 1.2. Name of Document Author/Supplier: > Author: Sudhakar Dindukurti > 1.3 Date of This Document: > 19 May, 2010 > 4. Technical Description > > 4.1 Acronyms > > P_Key : Partition Key > IPoIB : IP over InfiniBand > > 4.2 Problem > > Today the IPoIB driver allows you to create only one IPoIB > instance (aka IB partition) per P_Key per port. As a result, > on a multi-zone environment, it is not possible to assign > one IPoIB instance per "Exclusive-IP stack" zone with the > same P_Key. The work around is to use "Shared-IP" mode when > creating the zone which could potentially lead to performance > /scaling issues. Historically, this limitation has existed > since IPoIB was first implemented on Solaris. But it is > unnecessary, and now it is becoming a hindrance. > > 4.2 Proposal > > A micro/patch binding is asserted for this proposal. > > This case proposes to enhance IPoIB to support multiple > IPoIB instances per P_Key per port. With PSARC 2010/085 > administration enhancements, removing the limitation can be > easily fixed as the system administrator creates IPoIB > instances as needed using dladm(1M). Before PSARC 2010/085, > IPoIB instances were automatically instantiated by the IBTF > framework and so removing the limitation was not easy to fix. > > The IPoIB instance after this enhancement acts semantically > the same as before in sending and receiving unicast, multicast, > and broadcast traffic. For example, consider two IB systems, > each system with one HCA and two ports. On the first system, > two IPoIB instances (part0 and part1) are created on the same > port with P_Key1. On the second system, two IPoIB instances > are created (part3 and part4) on different ports with the > same P_Key1. In this example, all the IPoIB instances act > semantically same. > > Note: Man page does not make reference to the one part per > port/pkey limitation and so no edit is necessary for the man > page. > > 4.4 References > > PSARC 2001/289 IP over InfiniBand > PSARC 2010/085 IPoIB Administration Enhancement > > 6. Resources and Schedule > 6.4. Steering Committee requested information > 6.4.1. Consolidation C-team Name: > ON > 6.5. ARC review type: FastTrack > 6.6. ARC Exposure: open > > From ted.h.kim@oracle.com Thu May 27 16:10:11 2010 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id o4RNABG3029548 for ; Thu, 27 May 2010 16:10:11 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.4) with ESMTP id o4RNA8mw013700 for <@sunmail2sca.sfbay.sun.com:psarc-ext@sun.com>; Thu, 27 May 2010 17:10:10 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0L3300K1BPOXS700@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 27 May 2010 16:10:09 -0700 (PDT) Received: from brmea-mail-4.sun.com ([192.18.98.36]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0L3300JXIPOXNW60@nwk-avmta-2.sfbay.sun.com> for psarc-ext@sun.com (ORCPT psarc-ext@sun.com); Thu, 27 May 2010 16:10:09 -0700 (PDT) Received: from acsinet15.oracle.com (acsinet15.oracle.com [141.146.126.227]) by brmea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id o4RNA8on003829 for ; Thu, 27 May 2010 23:10:09 +0000 (GMT) Received: from acsmt353.oracle.com (acsmt353.oracle.com [141.146.40.153]) by acsinet15.oracle.com (Switch-3.4.2/Switch-3.4.1) with ESMTP id o4RM9Qw4008529 for ; Thu, 27 May 2010 23:10:08 +0000 (GMT) Received: from abhmt007.oracle.com by acsmt353.oracle.com with ESMTP id 304167821275001788; Thu, 27 May 2010 16:09:48 -0700 Received: from [129.153.3.74] (/129.153.3.74) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 27 May 2010 16:09:48 -0700 Date: Thu, 27 May 2010 16:09:20 -0700 From: "Ted H. Kim" Subject: Re: Multiple IPoIB instances per PKEY [PSARC/2010/182 FastTrack timeout, 05/26/2010] To: psarc-ext@sun.com Message-id: <4BFEFBA0.4050701@oracle.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 X-Source-IP: acsmt353.oracle.com [141.146.40.153] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090204.4BFEFBD0.00E7:SCFMA4539814,ss=1,fgs=0 User-Agent: Thunderbird 2.0.0.24 (X11/20100302) Status: RO Content-Length: 302 The timer has expired, and there were no comments except the +1. I am closing the case as approved. -ted -- Ted H. Kim, PhD Sun/Oracle America Inc. ted.h.kim@oracle.com 222 North Sepulveda Blvd., 10th Floor (310) 341-1116 El Segundo, CA 90245 (310) 341-1115 FAX