Subject: Multiple IPoIB instances per PKEY [PSARC/2010/182 FastTrack timeout 05/26/2010]
To:      PSARC-ext@Sun.Com
Cc:      ted.h.kim@oracle.com
Bcc:     one-pager-list@sac.sfbay one-pager-log@sac.sfbay sac-bar@sac.sfbay


Template Version: @(#)sac_nextcase 1.70 03/30/10 SMI
This information is Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
1. Introduction
    1.1. Project/Component Working Name:
	 Multiple IPoIB instances per PKEY
    1.2. Name of Document Author/Supplier:
	 Author:  Sudhakar Dindukurti
    1.3  Date of This Document:
	19 May, 2010
4. Technical Description

4.1 Acronyms

P_Key             : Partition Key
IPoIB             : IP over InfiniBand

4.2 Problem

Today the IPoIB driver allows you to  create only one  IPoIB
instance (aka IB partition) per P_Key per port.  As a result,
on a  multi-zone  environment, it is not  possible  to assign
one IPoIB  instance per  "Exclusive-IP  stack" zone  with the 
same P_Key. The work  around is to use  "Shared-IP" mode when
creating the zone which could potentially lead to performance
/scaling issues.  Historically,  this limitation has  existed
since IPoIB was first implemented on Solaris. But it is 
unnecessary, and now it is becoming a hindrance. 

4.2 Proposal

A micro/patch binding is asserted for this proposal.

This  case  proposes to  enhance  IPoIB to  support  multiple
IPoIB  instances  per  P_Key per port.  With  PSARC  2010/085
administration  enhancements,  removing the limitation can be
easily  fixed  as  the  system  administrator  creates  IPoIB
instances  as needed using  dladm(1M).  Before PSARC 2010/085,
IPoIB  instances were automatically  instantiated by the IBTF
framework and so removing the limitation was not easy to fix. 

The IPoIB  instance after this  enhancement acts semantically
the same as before in sending and receiving unicast, multicast,
and broadcast traffic.  For example, consider two IB systems,
each system  with one HCA and two ports. On the first system,
two IPoIB instances (part0 and part1) are created on the same
port with  P_Key1. On the second system,  two IPoIB instances
are created  (part3  and  part4) on  different ports with the
same  P_Key1.  In this example,  all the  IPoIB instances act
semantically same.

Note:  Man page  does not  make reference to the one part per 
port/pkey limitation and  so no edit is necessary for the man
page.

4.4 References

PSARC 2001/289 IP over InfiniBand
PSARC 2010/085 IPoIB Administration Enhancement 

6. Resources and Schedule
    6.4. Steering Committee requested information
   	6.4.1. Consolidation C-team Name:
		ON
    6.5. ARC review type: FastTrack
    6.6. ARC Exposure: open