1. What specifically is the proposal that we are reviewing? This project is a component of the Clearview project. It falls under its umbrella case - PSARC/2005/132 (Clearview: Network Interface Coherence). Specifically, this project aims to provide a consistent model to administer all network interfaces (Nemo unification). In addition, it will allow all network interfaces to be given administratively-chosen names (vanity naming). - What is the technical content of the project? See section 4.1 of one pager document. This project will focus on the support of DL_ETHER interfaces (including wireless interfaces). Providing GLDv3 support for interfaces of DL_IB type and other media types is out of the scope of this project (see more in question 4). - Is this a new product, or a change to a pre-existing one? If it is a change, would you consider it a "major", "minor", or "micro" change? This project will make some changes to the existing administrative command: dladm(1M). Further, while it will keep the current network devices nodes under /dev, it will introduce a /dev/net namespace to hold all vanity-named nodes for network interfaces. Because below changes will be incompatible changes unsuitable for a patch or micro release of Solaris, this project is targeting a minor release: * Third-party DLPI applications need to be altered to open the /dev/net devices nodes to access network devices, in order to use vanity naming feature and be consistent with other bundled DLPI applications. * Today, some applications enumerate the names of network devices and * datalinks on the system by traversing the devinfo tree and looking * for /devices nodes of type DDI_NT_NET (see di_walk_minor(3DEVINFO)). * After this project, datalinks can be renamed by the administrator, but * the network device names (in the devinfo tree) will be unaffected, * impacting applications that were walking the devinfo tree to enumerate * datalink names. These applications will need to be ported to use the * new dlpi_walk() library function; see section 3.1.6 of [2] for details. * - If your project is an evolution of a previous project, what changed from one version to another? This project is based on the GLDv3 framework (Nemo). It will enhance GLDv3 to provide the vanity naming support for network interfaces, and it will make all legacy devices managed by GLDv3. - What is the motivation for it, in general as well as specific terms? * Nemo unification While GLDv3 provides a new network driver framework with enhanced features such as link aggregation, it also introduces a confusing administrative model, since only GLDv3-based devices can be managed by dladm(1M). This project will "unify" all the legacy devices into GLDv3, hence provide a consistent administrative model and feature sets to all network devices. Specifically, we will provide link aggregation and generic VLAN support for Ethernet legacy devices. Other than the existing feature sets GLDv3 current has, there are other ongoing projects aim to make further enhancement to GLDv3 (for example, the Crossbow project which is going to support virtualization and resource management of network devices). With Nemo unification, those enhancements can potentially work for non-GLDv3 network devices. * Vanity naming Today, the network names are tied to the underlying network hardware (e.g., bge0, ce0). Because configuring the system requires network interface names to be referenced in various administrative tasks and a wide range of configuration files, being able to give a meaningful vanity name to a network interface (e.g., netmgt0 for an interface dedicated to network management, or upstream2 for an ISP-facing interface) will help to make network configuration much clearer, hence will make network administration easier. More importantly, vanity naming will prove especially useful for machine migration and Dynamic Reconfiguration. - What are the expected benefits for Sun? The customer will have a more consistent and flexible model to administer network interfaces. Specifically, the customer will have the ability to configure aggregations and VLANs on all Ethernet devices, and use vanity names with all network interfaces. - By what criteria will you judge its success? The project will be complete once the following requirements have been met: * Must provide a consistent model for network interface administration: - All network interfaces are administered by the same set of commands. - All network interfaces of a given type (e.g. Ethernet) will support a uniform set of administrative features. * Must be able to configure a vanity name for any network interface: - Administrators are able to name network interfaces, and also are able to administer the interfaces based on their vanity names. - There is a vanity-name DLPI device node for each network interface, allowing applications to interact with interfaces by their vanity names. 2. Describe how your project changes the user experience, upon installation and during normal operation. Under normal operation, administrators will be able to: * administer network interfaces using the same set of administrative interfaces. * create aggregations/VLANs on all Ethernet interfaces. * (re)name network interfaces. With administratively-chosen names, network configurations will not be tied to the driver names. This will ease network administration and make some formerly impossible administrative operations possible (for example, replace an interface with one of another driver using dynamic configuration). See details in the design document. Note that no changes to the installation process are proposed at this * time, and thus vanity naming for physical links will not be used by * default. In the future, we might make network link name configuration * an optional installation step, allowing network vanity naming be to * used by default. However, to minimize the risks discussed above, this * will not be done until we have more experience with the impact vanity * naming has on the system. * - What does the user perceive when the system is upgraded from a previous release? If there are any legacy network devices on the system, these legacy devices will have the same set of features as other GLDv3 devices, and administrators will be able to manage them using the same dladm utility. 3. What is its plan? - What is its current status? Has a design review been done? Are there multiple delivery phases? Currently, core development has completed and testing is ongoing. * A design review was conducted on opensolaris.org in the networking community. We will integrate both the Nemo unification and vanity naming components at the same time. 4. Are there related projects in Sun? * devname (PSARC/2003/246) The /dev/net namespace will be managed using a new general-purpose filesystem mechanism (Filesystem Driven Device Naming, devname). The devname project team is aware of this project and recognize the /dev/net FS as a potential user of the devname framework. We have discussed the devname changes we made with the devname iteam. * * libdlpi (PSARC/2006/436) The libdlpi library is a public library which will be used by all DLPI applications. Therefore, the details of accessing DLPI nodes under /dev/net will be hidden from applications. This project and the libdlpi project are both part of Clearview, and both are under the same umbrella case PSARC/2005/132 (Clearview, Network Interface Coherence). Currently, all relevant DLPI applications * in ON were ported to use libdlpi. * * VLAN Observability Enhancement (PSARC/2006/358) The project depends on the correct behavior proposed by PSARC/2006/358 to receive all tagged packets when binding to ETHERTYPE_VLAN sap. We are changing GLDv2 and GLDv3 to comply to PSARC/2006/358, and SSG is aware of the issue and already changed the Cassini driver. This project was integrated on Sep. 2006. * * Sun Trunking After Nemo unification, all Ethernet devices will be able to be aggregated using dladm create-aggr. Although the Nemo aggregation * solution will provide similiar functionality as Sun Trunking, we * expect some performance regression of Nemo aggregations over legacy * devices as compared to Sun Trunking (e.g, because Nemo aggregation * does not support MDT). Therefore, Sun Trunking will not be able to * be obsoleted until a thorough performance analysis has been done * and any gating performance issues have been resolved. * * GLDv3 IPoIB (IP over InfiniBand) driver Currently, the IPoIB driver (ibd) is written in GLDv2. Because of the peculiarities of ibd (in particular, packets are not passed upstream fully formatted even in raw mode), we decided that it is too much effort to work around all of its peculiarities to make it work with Nemo unification, so the alternative is to port the ibd driver to GLDv3 directly. Note that the IPoIB driver is not the driver for the InfiniBand host adapters, it is a service driver which talks to the underlying adapter using the IBTF (InfiniBand Transport Framework) interfaces. Therefore only one IPoIB driver is required, so that the porting work is limited. We already prototyped the GLDv3 IPoIB driver and the DL_IB MAC type plugin and proved this alternative should work. While the vanity naming support should work with the new GLDv3 IPoIB driver, the work to have a GLDv3 IPoIB driver does not fall into the scope of this project. * GLDv3 MAC plugins for other media types GLDv3 MAC type plugins are required for devices of other media types (such as FDDI and Token Ring) to work with Nemo unification. Implementing such plugins can be done as part of follow-up work provided that hardware required by the development and testing work become available. * Stack instances (PSARC/2006/366) Currently the Stack Instances project can not provide support for * network interfaces that are not written to the Nemo framework * (for example, ce interfaces and VLANs created over the ce device). * The Nemo unification component will allow such interfaces be * exported by Stack Instances to individual zones. * * Crossbow (PSARC/2006/357) Crossbow is a network virtualization project which allows effective sharing of physical networking resources among multiple user. It allows administrator to create multiple data devices (VNICs) to map to a single physical MAC instance. Although Crossbow can be implemented independently from this project, with Nemo unification, Crossbow will be able to support VNICs on non-GLDv3 devices. Further, this project's design will allow vanity naming of VNICs as well. 5. How is the project delivered into the system? - Identify packages, directories, libraries, databases, etc. * SUNWckr (Core Solaris Kernel (Root)) kernel/drv/$ARCH/softmac kernel/dacf/$ARCH/net_dacf * SUNWcsd (Core Solaris Devices) * dev/net (The /dev/net namespace) * * SUNWcsr (Core Solaris, (Root)) * sbin/dlmgmtd (dlmgmtd daemon binary) * etc/.dlmgmtd_door (door file for dlmgmtd) * lib/svc/method/svc-dlmgmtd (network/datalink-management service method) * var/svc/manifest/network/datalink-management.xml * (network/datalink-management service manifest) * * SUNWcsu (Core Solaris, (Usr)) * usr/lib/rcm/modules/SUNW_vlan_rcm.so (The RCM plugin module for vlan) * usr/lib/rcm/modules/SUNW_aggr_rcm.so (The RCM plugin module for aggr) * * SUNWcnetr (Core Solaris Network Infrastructure (Root)) * etc/dladm/datalink.conf (The private datalink configuration file) * 6. Describe the project's hardware platform dependencies. None. 7. System administration - How will the project's deliverables be installed and (re)configured? The deliverables from this project will be installed using the standard Solaris package utilities. - How will the project's deliverables be uninstalled? The deliverables are part of the base system and cannot be uninstalled. - Does it use inetd to start itself? No. - Does it need installation within any global system tables? No. - Does it use a naming service such as NIS, NIS+ or LDAP? No. - What are its on-going maintenance requirements (e.g. Keeping global tables up to date, trimming files)? None. - How does this project's administrative mechanisms fit into Sun's system administration strategies? E.g., how does it fit under the Solaris Management Console (SMC) and Web-Based Enterprise Management (WBEM), how does it make use of roles, authorizations and rights profiles? Additionally, how does it provide for administrative audit in support of the Solaris BSM configuration? N/A Note that dladm is currently part of Network Management execution profile, and users must be granted access to a role with that profile in order to successfully invoke its subcommands. The dladm rename-link subcommand will need the same profile. We don't think it needs additional privilege as renaming an interface is not much different from specifying a name when creating an interface (a tunnels, a VLAN or an aggregation). - What tunable parameters are exported? Can they be changed without rebooting the system? Examples include, but are not limited to, entries in /etc/system and ndd(8) parameters. What ranges are appropriate for each tunable? What are the commitment levels associated with each tunable (these are interfaces)? An undocumented "DLPI_DEVONLY" environment variable can be used to cause * libdlpi to skip /dev/net DLPI devices and instead only open those under * /dev. This variable should never be needed, but is provided for diagnostic * use and to facilitate possible workarounds in the field. * 8. Reliability, Availability, Serviceability (RAS) - Does the project make any material improvement to RAS? The aggregation support for legacy Ethernet devices will provide redundancy at layer 2 where none currently exists. - How can users/administrators diagnose failures or determine operational state? (For example, how could a user tell the difference between a failure and very slow performance?) The normal network tools (such as netstat, kstat, snoop, traceroute, ping) can be used to analyze the network problem. - What are the project's effects on boot time requirements? No noticeable effect on boot time are expected. - How does the project handle dynamic reconfiguration (DR) events? Before this project, during a DR operation, both the failed interface and the repaired interface were required to have the same network driver. This is because the network interface name was tied to the driver name. After this project, there will be no such restriction: an interface can be replaced with another using a different driver, and a renaming operation will make it possible for the new interface to inherit all of the existing configuration related to the failed interface. Further, today, a network device cannot be disconnected if there is * aggregation created over it. This restriction will also be removed * by this project. * - What mechanisms are provided for continuous availability of service? N/A - Does the project call panic()? Explain why these panics cannot be avoided. No. - How are significant administrative or error conditions transmitted? SNMP traps? Email notification? Errors detected by dladm are reported, the same as before. - How does the project deal with failure and recovery? N/A - Does it ever require reboot? If so, explain why this situation cannot be avoided. No. - How does your project deal with network failures (including partition and re- integration)? How do you handle the failure of hardware that your project depends on? N/A - Can it save/restore or checkpoint and recover? N/A - Can its files be corrupted by failures? Does it clean up any locks /files after crashes? A system crash when dlmgmtd updates the datalink configuration may * corrupt the configuration file. The window of risk will be kept to * be absolute minimum. The code that updates the data file will write to * a temporary file first. Once the temporary file is written, the change * will be committed to the real datalink.conf file. * We are also talking with the NWAM team and the SMF team about the * future datalink SMF service model, and agreed that as a future effort * of the Clearview project, each datalink should be modeled as a single * datalink service instance. At that time, configuration information * will be placed in the SMF repository to leverage any recover mechanism * that it provides. * 9. Observability - Does the project export status, either via observable output (e.g., netstat) or via internal data structures (kstats)? The kstats of each interface will be observed using the link's * vanity name, or the "dladm show-link -s" subcommand. * The interface name of the interface and its associated device can be seen from the output of various "dladm show-XXX" subcommands. - How would a user or administrator tell that this subsystem is or is not behaving as anticipated? Traditional network interface tools (kstats, netstat, ping, snoop, etc.) can be used. - What statistics does the subsystem export, and by what mechanism? None. - What state information is logged? None. - In principle, would it be possible for a program to tune the activity of your project? None. 10. What are the security implications of this project? - What security issues do you address in your project? None. - The Solaris BSM configuration carries a Common Criteria (CC) Controlled Access Protection Profile (CAPP) -- Orange Book C2 -- and a Role Based Access Control Protection Profile (RBAC) -- rating, does the addition of your project effect this rating? E.g., does it introduce interfaces that make access or privilege decisions that are not audited, does it introduce removable media support that is not managed by the allocate subsystem, does it provide administration mechanisms that are not audited? No. - Is system or subsystem security compromised in any way if your project's configuration files are corrupt or missing? No. - Please justify the introduction of any (all) new setuid executables. N/A - Include a thorough description of the security assumptions, capabilities and any potential risks (possible attack points) being introduced by your project. A separate Security Questionnaire http://sac.sfbay/cgi-bin/bp.cgi?NAME=Security.bp is provided for more detailed guidance on the necessary information. Cases are encouraged to fill out and include the Security questionnaire (leveraging references to existing documentation) in the case materials. Projects must highlight information for the following important areas: - What features are newly visible on the network and how are they protected from exploitation (e.g. unauthorized access, eavesdropping) None. - If the project makes decisions about which users, hosts, services, ... are allowed to access resources it manages, how is the requestor's identity determined and what data is used to determine if the access granted. Also how this data is protected from tampering. N/A - What privileges beyond what a common user (e.g. 'noaccess') can perform does this project require and why those are necessary. The dladm utility currently requires both PRIV_SYS_NET_CONFIG and PRIV_NET_RAWACCESS to successfully invoke its subcommands. This will not be changed in this project. - What parts of the project are active upon default install and how it can be turned off. N/A 11. What is its UNIX operational environment: - Which Solaris release(s) does it run on? Solaris Nevada - Environment variables? Exit status? Signals issued? Signals caught? (See signal(3HEAD).) N/A - Device drivers directly used (e.g. /dev/audio)? .rc/defaults or other resource/configuration files or databases? None. - Does it use any "hidden" (filename begins with ".") or temp files? The /etc/.dlmgmtd_door file, which is used by dlmgmtd to provide datalink * linkid management and dladm configuration management. * The /var/run/datalink.conf file, which is used by dlmgmtd to store * temporary link information. * As covered in question 8, a temporary file is used when updating either * datalink.conf change to prevent data loss or corruption. * - Does it use any locking files? No. * - Command line or calling syntax: What options are supported? (please include man pages if available) This project will introduce several new subcommands and options to dladm(1M). See details in section 4 of [2]. Does it conform to getopt() parsing requirements? Yes. - Is there support for standard forms, e.g. "-display" for X programs? Are these propagated to sub-environments? N/A - What shared libraries does it use? (Hint: if you have code use "ldd" and "dump -Lv")? The libdladm library will depend on the librcm and libnvpair libraries * in addition to the libraries it already uses. * The dlmgmtd daemon will use libdladm, libdlpi, libavl and libscf. * - Identify and justify the requirement for any static libraries. None. - Does it depend on kernel features not provided in your packages and not in the default kernel (e.g. Berkeley compatibility package, /usr/ccs, /usr/ucblib, optional kernel loadable modules)? No. - Is your project 64-bit clean/ready? If not, are there any architectural reasons why it would not work in a 64-bit environment? Does it interoperate with 64-bit versions? Yes. - Does the project depend on particular versions of supporting software (especially Java virtual machines)? If so, do you deliver a private copy? What happens if a conflicting or incompatible version is already or subsequently installed on the system? No. - Is the project internationalized and localized? Yes. - Is the project compatible with IPV6 interfaces and addresses? Yes. 12. What is its window/desktop operational environment? N/A -- no graphical components are provided by this project. 13. What interfaces does your project import and export? IMPORTED INTERFACES: Interface Classification Comments -------------------- ------------------ ------------------ libdlpi Committed PSARC/2006/436 devname APIs Consolidation Private PSARC/2003/246 dacf_get_dev() Consolidation Private Obtain the dev_t of a given dacf_infohdl_t. Defined in zone_datalink_walk() Consolidation Private Walk the datalinks in * a Specific zone. * EXPORTED INTERFACES: Interface Classification Comments -------------------- ------------------ ------------------- dlpi_walk() Committed libdlpi interface * to walk the datalinks * MAC capabilities MAC layer capabilities. MAC_CAPAB_NOZCOPY Project Private Defined in MAC_CAPAB_TX_LOOPBACK Project Private See Section 7.1 of [2]. MAC_CAPAB_LIMITED_VLAN Project Private MAC_CAPAB_PUTNEXT_TX Project Private MAC_CAPAB_NOLINK_UPDATE Project Private MAC_CAPAB_MDT Project Private MAC_CAPAB_PERSTREAM Project Private mc_open() callback Project Private New MAC callbacks. See * mc_close() callback section 7.1.3 of [2]. * mac_open() Consolidation Private Signature change. NO ddi_instance argument * is required. See 7.2.4 * of [2]. * mac_perstream_open() Project Private Open the (soft)mac in * the perstream mode. * Exchange the handles of * upper and lower streams.* See 7.1.4 of [2]. * softmac_create() Project Private The softmac API. * softmac_destroy() * softmac_hold_link() * softmac_rele_link() * DLIOCVLANINFO Consolidation Private Set by network drivers that are compatible with generic VLAN support. See section 6.1.4 of [2]. DLPI_DEVONLY flag Consolidation Private Used by the dlpi_open() * function to not open * the /dev/net nodes * DLPI_DEVONLY system Project Private With this variable * variable exported, applications * will not use the * /dev/net nodes. * RCM_RESOURCE_LINK_NEW Project Private New RCM resource name * to register for new * Links. See 6.2.7 of [2].* /etc/.dlmgmtd_door Project Private Door file for dlmgmtd. * type datalink_id_t Consolidation Private See [3]. * & datalink linkid * and configuration * management API * libdladm enhancements Consolidation Private dladm_hold_link() Removed. Changed to * dladm_rele_link() dladm_set/getzid(). * dladm_setzid() New interfaces to * dladm_getzid() set/get zoneid of a * given link. * dladm_aggr_walk() Removed. Replaced by * dladm_wlan_walk() dladm_walk_datalink_id()* dladm_wlan_is_valid() Removed. * dladm_wlan_set_prop() Removed. Generalized to * dladm_wlan_get_prop() dladm_set/get_prop() * dladm_wlan_walk_prop() * dladm_valid_linkname() New interfaces. * dladm_vlan_create() See section 5 of [2]. * dladm_vlan_delete() * dladm_vlan_up() * dladm_vlan_info() * dladm_aggr_info() * dladm_rename_link() * dladm_phys_delete() * dladm_key2linkid() * dladm_dev2linkid() * dladm_linkid2legacyname() * dladm_class2str() * dladm_meida2str() * dladm_walk() * dladm(1M) enhancement Committed dladm show-link Changed. dladm show-dev See section 4.1 of [2]. dladm show-aggr dladm show-phys New dladm show-vlan See section 4.1 of [2]. dladm create-aggr Changed dladm delete-aggr See section 4.2 of [2]. dladm modify-aggr dladm add-aggr dladm remove-aggr dladm rename-link New dladm rename-link See section 4.3 of [2]. dladm create-vlan dladm delete-vlan dladm set-linkprop -p autopush dladm delete-phys dladm show-dev Obsolete See section 4.1.3 of [2]. dladm aggr keys Obsolete See section 4.2 of [2]. dladm -d support Obsolete See section 4 of [2]. The /dev/net namespace Committed See section 6.2.1 of [2]. librcm.so Consolidation Private Moved from /usr/lib to * /lib. See section * 6.2.8 of [2]. * - What other applications should it interoperate with? How will it do so? N/A - Is it "pipeable"? How does it use stdin, stdout, stderr? dladm(1M) will behave the same as today. - Explain the significant file formats, names, syntax, and semantics. N/A - Is there a public namespace? (Can third parties create names in your namespace?) How is this administered? Yes, the /dev/net namespace. It will be used to keep all of the available interfaces on the system. Each interface will have a DLPI style-1 /dev/net node with the same name as its interface name. Interface names will be administered using dladm(1M) utility. Details are discussed in section 3.1.5 of [2]. - Are the externally visible interfaces documented clearly enough for a non-Sun client to use them successfully? The dladm(1M) manpage will be updated to include all the changes made to dladm. Further, we will add a vanity naming chapter to the sysadmin guide document, explaining the use model, the best practice surrounding vanity naming. 14. What are its other significant internal interfaces inter-subsystem and inter-invocation)? - Protocols (public or private) DLPI - Private ToolTalk usage N/A - Files N/A - Other Datalink ID and persistent configuration management API; see [3]. * - Are the interfaces re-entrant? Yes. 15. Is the interface extensible? How will the interface evolve? - How is versioning handled? The interfaces exported by this project (including libdladm and other kernel interfaces) are either project-private or consolidation-private, they will be revised together with their consumers when they evolve. The dladm(1M) command will evolve in accordance with that specified in the Interface Taxonomy policy. - What was the commitment level of the previous version? N/A - Can this version co-exist with existing standards and with earlier and later versions or with alternative implementations (perhaps by other vendors)? N/A - What are the clients over which a change should be managed? N/A - How is transition to a new version to be accomplished? What are the consequences to ISV's and their customers? N/A 16. How do the interfaces adapt to a changing world? As discussed in question 4, this project allows the future development of MAC and link-layer features, like Stack Instances or Crossbow, to apply to all network interfaces, not just those were written directly using GLDv3. 17. Interoperability - If applicable, explain your project's interoperability with the other major implementations in the industry. In particular, does it interoperate with Microsoft's implementation, if one exists? N/A - What would be different about installing your project in a heterogeneous site instead of a homogeneous one (such as Sun)? None. - Does your project assume that a Solaris-based system must be in control of the primary administrative node? No. 18. Performance - How will the project contribute (positively or negatively) to "system load" and "perceived performance"? The project aims to no noticeable degrade network performance on GLDv3 devices and on the fast data-path of non-GLDv3 devices. - What are the performance goals of the project? How were they evaluated? What is the test or reference platform? See answer to above question. We will run benchmark tests such as netperf and specweb99 to make sure we meet this requirement. The test platforms will include SPARC, x86 andx64. - Does the application pause for significant amounts of time? Can the user interact with the application while it is performing long-duration tasks? No. - What is your project's MT model? How does it use threads internally? How does it expect its client to use threads? If it uses callbacks, can the called entity create a thread and recursively call back? The project will be fully MT. - What is the impact on overall system performance? What is the average working set of this component? How much of this is shared/sharable by other apps? N/A - Does this application "wake up" periodically? How often and under what conditions? What is the working set associated with this behavior? N/A - Will it require large files/databases (for example, new fonts)? No. - Do files, databases or heap space tend to grow with time/load? What mechanisms does the user have to use to control this? What happens to performance/system load? No. 19. Please identify any issues that you would like the ARC to address. - Are there issues or related projects that the ARC should advise the appropriate steering committees? None. * 20. Appendices to include [1] One pager. [2] Vanity Naming and Nemo Unification High-Level Design Specification: uv-design.pdf [3] Vanity Naming Link ID and Persistent Configuration Management: * link_id_management.pdf *