.de Sc
\\s-1\\$1\\s0\\$2
..
.ds cA 2007/272
.ds aR \s-1PSARC\s0
.LP
.so ../../amac
.Co
.ds LF \fI\*(aR/\*(cA\fP
.ds RF \fICopyright 2007 Sun Microsystems\fP
.if n .ds CF
.IP \fBSubject:\fP 15
Project Clearview: IPMP Rearchitecture
.IP "\fBSubmitted by:\fP" 15
Peter Memishian
.IP \fBFile:\fP 15
\*(aR/\*(cA/opinion.ms
.IP \fBDate:\fP 15
November 19th, 2008
.IP "\fBCommittee:\fP" 15
Kais Belgaied, James Carlson, Glenn Skinner.  Abstain: Gary Winiger.  (opinion written by Sebastien Roy)
.IP "\fBProduct Approval Committee:\fP" 15

Solaris PAC
.br
solaris-pac-opinion@sun.com

.pn 2
.NH
Summary
.LP
This is one of a series of projects under the PSARC/2005/132 umbrella
case, "Clearview: Network Interface Coherence".  This project
rearchitects the existing Solaris IP Multipathing (IPMP) technology so
that it can work transparently with all IP-based applications,
allowing it to be deployed much more widely by customers.  As a side
effect, it also enables core technologies such as DHCP to work
seamlessly with IPMP.  It also provides an improved administrative
model and new diagnostic facilities which should greatly ease
troubleshooting in IPMP environments -- and reduce our support costs.
Finally, it massively simplifies the IPMP kernel implementation,
shrinking it by over 35% (3000 lines less code) and isolating the
majority of the remaining code behing a well-defined internal API.
.NH
Decision & Precedence Information
.LP
The project is approved as specified in references [1-6].
.LP
The project may be delivered in a Minor release of Solaris as part of
the ON consolidation.
.NH
Interfaces
.LP
The project exports the following interfaces.
.if n .ne 8
.if t .ne 3
.TS H
box;
c s s
l | l | l.
Interfaces Exported
_
Interface	Classification	Comments
_
.TH
IPMP IP Interface	Committed	[1] section 3.1
outbount load spreading behavior	Volatile	[1] section 3.2
source address selection behavior	Volatile	[1] section 3.3
ifconfig "ipmp" subcommand	Committed	[1] section 4.1.5
ifconfig "ipmp" subcommand	Committed	[1] section 4.1.5
"ipmp" in /etc/hostname[6].*	Committed	[1] section 4.3
networking commands on IPMP and	Committed	[1] sections 4.1,
underlying IP interfaces		4.4, 4.5, 4.8-4.10
IPMP packet monitoring semantics	Committed	[1] section 4.11
IPMP packet filtering semantics	Committed	[1] section 4.12
DHCP for IPMP data/test addresses	Committed	[1] section 4.13
kstats for IPMP IP interfaces	Committed[4]	[1] section 4.8
MIBII stats for IPMP IP interfaces	Uncommitted	[1] section 5.18
IPv6 link-local IPMP interaction	Committed	[1] section 4.6
IPMP bring-up at boot	Project Private	[1] section 4.3
		
/usr/sbin/ipmpstat	Committed	[1] section 4.2
/sbin/ipmpstat alternate location	Volatile	[1] section 4.3.4
/sbin/in.mpathd alternate location	Volatile	[1] section 4.3.4
ipmpstat output modes	Committed	[1] section 4.2
(-a, -g, -i, -p, -t)
ipmpstat normal output format	Not-an-Interface	[1] section 4.2
ipmpstat -P output format	Committed	[1] section 4.2
		
SIOCG[L]IFCONF and SIOCG[L]IFNUM	Committed	[1] section 5.1
IPMP interaction
LIFC_UNDER_IPMP	Committed	[1] section 5.1
SIOC[GS]LIFFLAGS IPMP interaction	Committed	[1] section 5.2
IFF_IPMP	Committed	[1] section 5.3
IFF_RUNNING on IPMP IP interfaces	Committed	[1] section 5.3
Visibility of IPMP and underlying	Committed	[1] section 5.4
IP interfaces via PF_ROUTE
SOL_ROUTE socket level	Committed	[1] section 5.4.2
RT_AWARE / RTAW_* socket option	Committed	[1] section 5.4.2
Assorted "set" SIOC* and PF_ROUTE	Volatile	[1] section 5.*
ops on underlying IP interfaces
SIOC*ARP IPMP behavior	Committed	[1] section 5.11
SIOC*ND IPMP behavior	Committed	[1] section 5.12
SIOC[GS]LIFZONE/DLDIOC_SETZID	Cons. Private	[1] section 5.13
SIOC*LIF*SRC* IPMP interaction	Volatile	[1] section 5.14
IPMP data address multicast API	Committed	[1] section 5.15
behavior and semantics
SIOCGLIFBINDING	Project Private	[1] section 5.17
SIOCGLIFGROUPINFO	Project Private	[1] section 5.17
lifr_binding lifreq member	Project Private	[1] section 5.17
struct lifgroupinfo	Project Private	[1] section 5.17
EXPER_IP_AND_TESTHIDDEN MIB level	Cons. Private	[1] section 5.18
dlpistub kernel module	Cons. Private	[1] section 5.20
/dev/ipmpstub	Project Private	[1] section 5.20
		
libipmp APIs	Contracted Cons.	[1] section 5.21
in.mpathd/libipmp communication	Project Private	[1] section 5.21
/usr/include/ipmp_admin.h	Contracted Cons.	[1] section 5.21
IPMP async events	Contracted Cons.	[1] section 5.22
if_indextoname() IPMP behavior	Committed	[1] section 5.23.1
if_nametoindex() IPMP behavior	Committed	[1] section 5.23.1
if_nameindex() IPMP behavior	Committed	[1] section 5.23.2
ifaddrlist() enhancements	Cons. Private	[1] section 5.23.3
ifaddrlistx(), ifaddrlistx_free()	Cons. Private	[1] section 5.23.4
sockaddrcmp()	Cons. Private	[1] section 5.23.5
.TE
.LP
The project imports the following interfaces.
.if n .ne 8
.if t .ne 3
.TS H
box;
c s s
l | l | l.
Interfaces Imported
_
Interface	Classification	Comments
_
.TH
vni device driver	Cons. Private	[1] section 5.20
NIC kstats	Committed	[1] section 4.8
sysevent API	Committed	For PSARC/2002/137
IP module IPSQ framework	Project Private	[2] question 14
ARP/IP message-passing API	Cons. Private	[2] question 14
modhash kernel API	Cons. Private	Impl. artifact
/etc/hostname[6].*	Committed	[1] section 4.3
.TE
.NH
Opinion
.LP
.NH 2
IPMP is a Network Layer Mechanism
.LP
During the case review, concern was raised that an IPMP group
represented by a virtual IP interface could be confusing to users.
Specifically, IP interfaces that have no one-to-one relationship with
an underlying link require special handling by observability tools,
and thus require additional architecture such as PSARC/2006/475.

The discussion did not result in any technical change nor advice.
IPMP is by definition a network layer mechanism, and thus cannot be
represented at the link layer.
.NH 2
Reverse ARP Autoconfiguration and IPMP
.LP
Section 4.1.11 of [1] states that the ifconfig auto-revarp subcommand
will fail on an IPMP interface.  The group interface does not exist as
a DLPI device, and DLPI is needed for ifconfig to send and receive ARP
packets.

One PSARC member noted that while it is true that the group interface
does not exist as a DLPI device, implementing auto-revarp should still
be possible by interacting directly with one of the underlying
interface's DLPI devices.  All agreed that implementing such a
mechanism is likely not worth the effort.
.NH 2
Incompatible Change to Networking MIB
.LP
Section 5.18 of [1] describes that data associated with test addresses
on underlying interfaces will be invisible to consumers of the
Networking MIB unless the new EXPER_IP_AND_TESTHIDDEN MIB II level is
specified in the request, under the assumption that only in.mpathd
should be concerned with those details.

The committee noted that this constituted a slight regression in
functionality, as SNMP applications will no longer have observability
into IPMP configuration.  This resulted in advice to management
described in section 6.
.NH
Minority Opinion(s)
.LP
None
.NH
Advisory Information
.LP
This project's management should fund a project to add SNMP
observability into IPMP configuration, perhaps using a new private
MIB.  See section 4.3 for details.
.NH
Appendices
.NH 2
Appendix A: Technical Changes Required
.LP
None
.NH 2
Appendix B: Technical Changes Advised
.LP
None
.NH 2
Appendix C: Reference Material
.LP
Unless stated otherwise, path names are relative to the case
directory \*(aR/\*(cA/final.materials/.
.RS
.IP 1.
High-Level Design Document
.br
File: ipmp-highlevel-design.pdf
.IP 2.
PSARC 20 Questions
.br
File: ipmp-20q.txt
.IP 3.
Specification for ifaddrlistx() and ifaddrlistx_free()
.br
File: ifaddrlistx.txt
.IP 4.
IPMP Async Events Specification
.br
File: ipmp-async.txt
.IP 5.
libipmp Specification
.br
File: libipmp.txt
.IP 6.
Draft Manual Pages
.br
File: manpages/index.html
.RE