1. What specifically is the proposal that we are reviewing? This project is a component of the Clearview project. It falls under its umbrella case - PSARC/2005/132 (Clearview: Network Interface Coherence). Specifically, this project aims to provide a consistent model to administer all network interfaces (Nemo unification). In addition, it will allow all network interfaces to be given administratively-chosen names (vanity naming). - What is the technical content of the project? See section 4.1 of one pager document. This project will focus on the support of DL_ETHER interfaces (including wireless interfaces). Providing GLDv3 support for interfaces of DL_IB type and other media types is out of the scope of this project (see more in question 4). - Is this a new product, or a change to a pre-existing one? If it is a change, would you consider it a "major", "minor", or "micro" change? This project will make some changes to the existing administrative command: dladm(1M). Further, while it will keep the current network devices nodes under /dev, it will introduce a /dev/net namespace to hold all vanity-named nodes for network interfaces. Because below changes will be incompatible changes unsuitable for a patch or micro release of Solaris, this project is targeting a minor release: * Third-party DLPI applications need to be altered to open the /dev/net devices nodes to access network devices, in order to use vanity naming feature and be consistent with other bundled DLPI applications. * Today, some applications enumerate the names of network interfaces on the system by traversing the devinfo tree and looking for /devices nodes of type DDI_NT_NET (see di_walk_minor(3DEVINFO)). After this project, interface will be able to be (re)named to any random names, therefore the traditional way to enumerate network interface will not work. This project will provide a library function dladm_walk() to enumerate network interfaces. - If your project is an evolution of a previous project, what changed from one version to another? This project is based on the GLDv3 framework (Nemo). It will enhance GLDv3 to provide the vanity naming support for network interfaces, and it will make all legacy devices managed by GLDv3. - What is the motivation for it, in general as well as specific terms? * Nemo unification While GLDv3 provides a new network driver framework with enhanced features such as link aggregation, it also introduces a confusing administrative model, since only GLDv3-based devices can be managed by dladm(1M). This project will "unify" all the legacy devices into GLDv3, hence provide a consistent administrative model and feature sets to all network devices. Specifically, we will provide link aggregation and generic VLAN support for Ethernet legacy devices. Other than the existing feature sets GLDv3 current has, there are other ongoing projects aim to make further enhancement to GLDv3 (for example, the Crossbow project which is going to support virtualization and resource management of network devices). With Nemo unification, those enhancements can potentially work for non-GLDv3 network devices. * Vanity naming Today, the network names are tied to the underlying network hardware (e.g., bge0, ce0). Because configuring the system requires network interface names to be referenced in various administrative tasks and a wide range of configuration files, being able to give a meaningful vanity name to a network interface (e.g., netmgt0 for an interface dedicated to network management, or upstream2 for an ISP-facing interface) will help to make network configuration much clearer, hence will make network administration easier. More importantly, vanity naming will prove especially useful for machine migration and Dynamic Reconfiguration. - What are the expected benefits for Sun? The customer will have a more consistent and flexible model to administer network interfaces. Specifically, the customer will have the ability to configure aggregations and VLANs on all Ethernet devices, and use vanity names with all network interfaces. - By what criteria will you judge its success? The project will be complete once the following requirements have been met: * Must provide a consistent model for network interface administration: - All network interfaces are administered by the same set of commands. - All network interfaces of a given type (e.g. Ethernet) will support a uniform set of administrative features. * Must be able to configure a vanity name for any network interface: - Administrators are able to name network interfaces, and also are able to administer the interfaces based on their vanity names. - There is a vanity-name DLPI device node for each network interface, allowing applications to interact with interfaces by their vanity names. 2. Describe how your project changes the user experience, upon installation and during normal operation. we will make configuring network interface names to be an optional step of the installation, therefore allowing network vanity naming to be used by default. Under normal operation, administrators will be able to: * administer network interfaces using the same set of administrative interfaces. * create aggregations/VLANs on all Ethernet interfaces. * (re)name network interfaces. With administratively-chosen names, network configurations will not be tied to the driver names. This will ease network administration and make some formerly impossible administrative operations possible (for example, replace an interface with one of another driver using dynamic configuration). See details in the design document. - What does the user perceive when the system is upgraded from a previous release? If there are any legacy network devices on the system, these legacy devices will have the same set of features as other GLDv3 devices, and administrators will be able to manage them using the same dladm utility. 3. What is its plan? - What is its current status? Has a design review been done? Are there multiple delivery phases? It is currently in development phase. The Nemo unification support is mostly done, and we just started the vanity naming support. A design review was conducted on opensolaris.org in the networking community. We will integrate both the Nemo unification and vanity naming components at the same time. 4. Are there related projects in Sun? * devname (PSARC/2003/246) The /dev/net namespace will be managed using a new general-purpose filesystem mechanism (Filesystem Driven Device Naming, devname). The devname project team is aware of this project and recognize the /dev/net FS as a potential user of the devname framework. * libdlpi (PSARC/2006/436) The libdlpi library is a public library which will be used by all DLPI applications. Therefore, the details of accessing DLPI nodes under /dev/net will be hidden from applications. This project and the libdlpi project are both part of Clearview, and both are under the same umbrella case PSARC/2005/132 (Clearview, Network Interface Coherence). * VLAN Observability Enhancement (PSARC/2006/358) The project depends on the correct behavior proposed by PSARC/2006/358 to receive all tagged packets when binding to ETHERTYPE_VLAN sap. We are changing GLDv2 and GLDv3 to comply to PSARC/2006/358, and SSG is aware of the issue and has agreed to change the Cassini driver. * Sun Trunking After Nemo unification, all Ethernet devices will be able to be aggregated using dladm create-aggr. Therefore, we proposed to obsolete Sun Trunking, which provides the same functionality as dladm create-aggr, but only works on particular drivers. A discussion is underway with the SSG group about the potential possibility of obsoleting Sun Trunking. We agreed that a configuration conversion tool to convert Sun Trunking configuration to dladm configuration needs to be provided in order to make the transition smoother. Note that neither the obsolescence of Sun Trunking nor the configuration conversion tool fall into the scope of this project. * GLDv3 IPoIB (IP over InfiniBand) driver Currently, the IPoIB driver (ibd) is written in GLDv2. Because of the peculiarities of ibd (in particular, packets are not passed upstream fully formatted even in raw mode), we decided that it is too much effort to work around all of its peculiarities to make it work with Nemo unification, so the alternative is to port the ibd driver to GLDv3 directly. Note that the IPoIB driver is not the driver for the InfiniBand host adapters, it is a service driver which talks to the underlying adapter using the IBTF (InfiniBand Transport Framework) interfaces. Therefore only one IPoIB driver is required, so that the porting work is limited. We already prototyped the GLDv3 IPoIB driver and the DL_IB MAC type plugin and proved this alternative should work. While the vanity naming support should work with the new GLDv3 IPoIB driver, the work to have a GLDv3 IPoIB driver does not fall into the scope of this project. * GLDv3 MAC plugins for other media types GLDv3 MAC type plugins are required for devices of other media types (such as FDDI and Token Ring) to work with Nemo unification. Implementing such plugins can be done as part of follow-up work provided that hardware required by the development and testing work become available. * Stack instances (PSARC/2006/366) Currently the Stack Instances project can not provide support for network interfaces that only have style-2 devices nodes (for example, ce and VLANs). The DLPI style-1 devices nodes created by vanity naming under /dev/net will allow such interfaces be exported by Stack Instances to individual zones * Crossbow (PSARC/2006/357) Crossbow is a network virtualization project which allows effective sharing of physical networking resources among multiple user. It allows administrator to create multiple data devices (VNICs) to map to a single physical MAC instance. Although Crossbow can be implemented independently from this project, with Nemo unification, Crossbow will be able to support VNICs on non-GLDv3 devices. Further, this project's design will allow vanity naming of VNICs as well. 5. How is the project delivered into the system? - Identify packages, directories, libraries, databases, etc. * softmac SUNWckr (Core Solaris Kernel (Root)) kernel/drv/$ARCH/softmac * net_dacf SUNWckr (Core Solaris Kernel (Root)) kernel/dacf/$ARCH/net_dacf 6. Describe the project's hardware platform dependencies. None. 7. System administration - How will the project's deliverables be installed and (re)configured? The deliverables from this project will be installed using the standard Solaris package utilities. - How will the project's deliverables be uninstalled? The deliverables are part of the base system and cannot be uninstalled. - Does it use inetd to start itself? No. - Does it need installation within any global system tables? No. - Does it use a naming service such as NIS, NIS+ or LDAP? No. - What are its on-going maintenance requirements (e.g. Keeping global tables up to date, trimming files)? None. - How does this project's administrative mechanisms fit into Sun's system administration strategies? E.g., how does it fit under the Solaris Management Console (SMC) and Web-Based Enterprise Management (WBEM), how does it make use of roles, authorizations and rights profiles? Additionally, how does it provide for administrative audit in support of the Solaris BSM configuration? N/A Note that dladm is currently part of Network Management execution profile, and users must be granted access to a role with that profile in order to successfully invoke its subcommands. The dladm rename-link subcommand will need the same profile. We don't think it needs additional privilege as renaming an interface is not much different from specifying a name when creating an interface (a tunnels, a VLAN or an aggregation). - What tunable parameters are exported? Can they be changed without rebooting the system? Examples include, but are not limited to, entries in /etc/system and ndd(8) parameters. What ranges are appropriate for each tunable? What are the commitment levels associated with each tunable (these are interfaces)? None. 8. Reliability, Availability, Serviceability (RAS) - Does the project make any material improvement to RAS? The aggregation support for legacy Ethernet devices will provide redundancy at layer 2 where none currently exists. - How can users/administrators diagnose failures or determine operational state? (For example, how could a user tell the difference between a failure and very slow performance?) The normal network tools (such as netstat, kstat, snoop, traceroute, ping) can be used to analyze the network problem. - What are the project's effects on boot time requirements? TBD - How does the project handle dynamic reconfiguration (DR) events? Before this project, during a DR operation, both the failed interface and the repaired interface were required to have the same network driver. This is because the network interface name was tied to the driver name. After this project, there will be no such restriction: an interface can be replaced with another using a different driver, and a renaming operation will make it possible for the new interface to inherit all of the existing configuration related to the failed interface. - What mechanisms are provided for continuous availability of service? N/A - Does the project call panic()? Explain why these panics cannot be avoided. No. - How are significant administrative or error conditions transmitted? SNMP traps? Email notification? Errors detected by dladm are reported, the same as before. - How does the project deal with failure and recovery? N/A - Does it ever require reboot? If so, explain why this situation cannot be avoided. No. - How does your project deal with network failures (including partition and re- integration)? How do you handle the failure of hardware that your project depends on? N/A - Can it save/restore or checkpoint and recover? N/A - Can its files be corrupted by failures? Does it clean up any locks /files after crashes? The file (/etc/datalink.conf) which keeps all dladm configuration (including vanity naming configuration) will be automatically updated and must not be directly edited. If this file is corrupted, the system and all network devices on the system should still work, under the expectation that some dladm configuration might get lost. 9. Observability - Does the project export status, either via observable output (e.g., netstat) or via internal data structures (kstats)? The kstats of each interface will be observed using the name of "net/ifname", or the "dladm show-link -s" subcommand. The interface name of the interface and its associated device can be seen from the output of various "dladm show-XXX" subcommands. - How would a user or administrator tell that this subsystem is or is not behaving as anticipated? Traditional network interface tools (kstats, netstat, ping, snoop, etc.) can be used. - What statistics does the subsystem export, and by what mechanism? None. - What state information is logged? None. - In principle, would it be possible for a program to tune the activity of your project? None. 10. What are the security implications of this project? - What security issues do you address in your project? None. - The Solaris BSM configuration carries a Common Criteria (CC) Controlled Access Protection Profile (CAPP) -- Orange Book C2 -- and a Role Based Access Control Protection Profile (RBAC) -- rating, does the addition of your project effect this rating? E.g., does it introduce interfaces that make access or privilege decisions that are not audited, does it introduce removable media support that is not managed by the allocate subsystem, does it provide administration mechanisms that are not audited? No. - Is system or subsystem security compromised in any way if your project's configuration files are corrupt or missing? No. - Please justify the introduction of any (all) new setuid executables. N/A - Include a thorough description of the security assumptions, capabilities and any potential risks (possible attack points) being introduced by your project. A separate Security Questionnaire http://sac.sfbay/cgi-bin/bp.cgi?NAME=Security.bp is provided for more detailed guidance on the necessary information. Cases are encouraged to fill out and include the Security questionnaire (leveraging references to existing documentation) in the case materials. Projects must highlight information for the following important areas: - What features are newly visible on the network and how are they protected from exploitation (e.g. unauthorized access, eavesdropping) None. - If the project makes decisions about which users, hosts, services, ... are allowed to access resources it manages, how is the requestor's identity determined and what data is used to determine if the access granted. Also how this data is protected from tampering. N/A - What privileges beyond what a common user (e.g. 'noaccess') can perform does this project require and why those are necessary. The dladm utility currently requires both PRIV_SYS_NET_CONFIG and PRIV_NET_RAWACCESS to successfully invoke its subcommands. This will not be changed in this project. - What parts of the project are active upon default install and how it can be turned off. N/A 11. What is its UNIX operational environment: - Which Solaris release(s) does it run on? Solaris Nevada - Environment variables? Exit status? Signals issued? Signals caught? (See signal(3HEAD).) N/A - Device drivers directly used (e.g. /dev/audio)? .rc/defaults or other resource/configuration files or databases? None. - Does it use any "hidden" (filename begins with ".") or temp files? No. - Does it use any locking files? No. - Command line or calling syntax: What options are supported? (please include man pages if available) This project will introduce several new subcommands and options to dladm(1M). See details in section 4 of [2]. Does it conform to getopt() parsing requirements? Yes. - Is there support for standard forms, e.g. "-display" for X programs? Are these propagated to sub-environments? N/A - What shared libraries does it use? (Hint: if you have code use "ldd" and "dump -Lv")? The dladm utility will not depend on any additional libraries. The libdladm library will be updated to support vanity naming. - Identify and justify the requirement for any static libraries. None. - Does it depend on kernel features not provided in your packages and not in the default kernel (e.g. Berkeley compatibility package, /usr/ccs, /usr/ucblib, optional kernel loadable modules)? No. - Is your project 64-bit clean/ready? If not, are there any architectural reasons why it would not work in a 64-bit environment? Does it interoperate with 64-bit versions? Yes. - Does the project depend on particular versions of supporting software (especially Java virtual machines)? If so, do you deliver a private copy? What happens if a conflicting or incompatible version is already or subsequently installed on the system? No. - Is the project internationalized and localized? Yes. - Is the project compatible with IPV6 interfaces and addresses? Yes. 12. What is its window/desktop operational environment? N/A -- no graphical components are provided by this project. 13. What interfaces does your project import and export? IMPORTED INTERFACES: Interface Classification Comments -------------------- ------------------ ------------------ libdlpi Public/Committed PSARC/2006/436 GLDv3 MAC interfaces Consolidation Private PSARC/2006/249 devname APIs (TBD) Consolidation Private PSARC/2003/246 EXPORTED INTERFACES: Interface Classification Comments -------------------- ------------------ ------------------- MAC capabilities MAC layer capabilities. MAC_CAPAB_NOZCOPY Project Private Defined in MAC_CAPAB_TX_LOOPBACK Project Private See Section 7.1 of [2]. MAC_CAPAB_LIMITED_VLAN Project Private MAC_CAPAB_PUTNEXT_TX Project Private MAC_CAPAB_NOLINK_UPDATE Project Private MAC_CAPAB_PERSTREAM Project Private MAC_CAPAB_MDT Project Private MAC_CAPAB_IPSEC Consolidation Private libdladm enhancements Consolidation Private See section 5 of [2]. dacf_get_dev() Consolidation Private Obtain the dev_t of a given dacf_infohdl_t. Defined in DL_IOC_VLAN_CAPAB Consolidation Private Set by network drivers that are compatible with generic VLAN support. See section 6.1.4 of [2]. dladm(1M) enhancement Public/Uncommitted dladm show-link Changed. dladm show-dev See section 4.1 of [2]. dladm show-aggr dladm show-phys New dladm show-vlan See section 4.1 of [2]. dladm create-aggr Changed dladm delete-aggr See section 4.2 of [2]. dladm modify-aggr dladm add-aggr dladm remove-aggr dladm rename-link New dladm rename-link See section 4.3 of [2]. dladm create-vlan dladm delete-vlan dladm autopush dladm delete-phys dladm show-dev Obsolete See section 4.1.3 of [2]. dladm aggr keys Obsolete See section 4.2 of [2]. dladm -d support Obsolete See section 4 of [2]. The /dev/net namespace Stable See section 6.2.1 of [2]. - What other applications should it interoperate with? How will it do so? N/A - Is it "pipeable"? How does it use stdin, stdout, stderr? dladm(1M) will behave the same as today. - Explain the significant file formats, names, syntax, and semantics. N/A - Is there a public namespace? (Can third parties create names in your namespace?) How is this administered? Yes, the /dev/net namespace. It will be used to keep all of the available interfaces on the system. Each interface will have a DLPI style-1 /dev/net node with the same name as its interface name. Interface names will be administered using dladm(1M) utility. Details are discussed in section 3.1.5 of [2]. - Are the externally visible interfaces documented clearly enough for a non-Sun client to use them successfully? The dladm(1M) manpage will be updated to include all the changes made to dladm. Further, we will add a vanity naming chapter to the sysadmin guide document, explaining the use model, the best practice surrounding vanity naming. 14. What are its other significant internal interfaces inter-subsystem and inter-invocation)? - Protocols (public or private) DLPI - Private ToolTalk usage N/A - Files N/A - Other N/A - Are the interfaces re-entrant? Yes. 15. Is the interface extensible? How will the interface evolve? - How is versioning handled? The interfaces exported by this project (including libdladm and other kernel interfaces) are either project-private or consolidation-private, they will be revised together with their consumers when they evolve. The dladm(1M) command will evolve in accordance with that specified in the Interface Taxonomy policy. - What was the commitment level of the previous version? N/A - Can this version co-exist with existing standards and with earlier and later versions or with alternative implementations (perhaps by other vendors)? N/A - What are the clients over which a change should be managed? N/A - How is transition to a new version to be accomplished? What are the consequences to ISV's and their customers? N/A 16. How do the interfaces adapt to a changing world? As discussed in question 4, this project allows the future development of MAC and link-layer features, like Stack Instances or Crossbow, to apply to all network interfaces, not just those were written directly using GLDv3. 17. Interoperability - If applicable, explain your project's interoperability with the other major implementations in the industry. In particular, does it interoperate with Microsoft's implementation, if one exists? N/A - What would be different about installing your project in a heterogeneous site instead of a homogeneous one (such as Sun)? None. - Does your project assume that a Solaris-based system must be in control of the primary administrative node? No. 18. Performance - How will the project contribute (positively or negatively) to "system load" and "perceived performance"? The project aims to no noticeable degrade network performance on GLDv3 devices and on the fast data-path of non-GLDv3 devices. We did the netperf tests on our current Nemo unification bits. The result shows that UDP performance degrades %1, whereas the TCP performance improves %1 on a softmac interface, compared to the performance of the corresponding legacy driver. - What are the performance goals of the project? How were they evaluated? What is the test or reference platform? See answer to above question. We will run benchmark tests such as libmicro, netperf and specweb99 to make sure we meet this requirement. The test platforms will include SPARC, x86 and x64. - Does the application pause for significant amounts of time? Can the user interact with the application while it is performing long-duration tasks? No. - What is your project's MT model? How does it use threads internally? How does it expect its client to use threads? If it uses callbacks, can the called entity create a thread and recursively call back? The project will be fully MT. - What is the impact on overall system performance? What is the average working set of this component? How much of this is shared/sharable by other apps? N/A - Does this application "wake up" periodically? How often and under what conditions? What is the working set associated with this behavior? N/A - Will it require large files/databases (for example, new fonts)? No. - Do files, databases or heap space tend to grow with time/load? What mechanisms does the user have to use to control this? What happens to performance/system load? No. 19. Please identify any issues that you would like the ARC to address. - Are there issues or related projects that the ARC should advise the appropriate steering committees? * One proposal is under consideration that during fresh installation, network devices will be renamed automatically to some default names like net0, net1, etc., unless are specified not to do so. While this will enable widespread deployment of vanity naming, it may also impact existing third-party software which has assumed that the hardware device names (e.g., retrieved using libdevinfo) can be mapped directly to DLPI links that can be opened in /dev. We'd appreciate input from the ARC on this issue. * Today, all network drivers (including physical network device drivers and pseudo drivers like aggr) have the same device policy - net_rawaccess for both read and write. However, Solaris allows the device policy to be changed on the per-device-driver basis using add_drv(1m). After vanity naming, every network configuration will be separated from the underlying hardware. But allowing specifying the per-device-driver policy would leave part of the security configuration tied to the hardware, which deviates from what we'd like to see. We've consulted Casper Dik and Darren Moffat about the possibility of removing the per-device-driver policy for network devices, and they agreed that a single privilege policy (and perhaps the current default one) for the network devices should suffice the real-world usage. Again, we'd like to hear more advice on this from the ARC. 20. Appendices to include [1] One pager. [2] Vanity Naming and Nemo Unification High-Level Design Specification: uv-design.pdf