Brussels - II - enhanced network interface configuration via ipadm 1. Introduction ---------------- A full description of the overall architecture of the Brussels - II project is described in [1]. Note, however, that this PSARC case (2009/306) will only deliver a subset of the design in [1]. 2. Brussels - II Overview -------------------------- Network configuration on Solaris has always been needlessly complex. Tools like ifconfig(1M) are known to be severely overloaded, confusing and complex. This has been recorded in several places and to quote a few (a) the ON SAC opinion for PSARC 1997/184 stated, " .. the already complex ifconfig utility. The committee feels this project is making an already serious ease of use problem even worse. The steering committees are advised to initiate a project to create a new easier to use network interface configuration utility." (b) one of the comments in ifconfig.c, reads as below, " .. ifconfig(1M) is too overloaded and the code is so convoluted that it is "safer" not to re-architect the code to fix the above issue, hence this "hack." We may be better off to have a new command with better syntax for configuring network interface parameters..." PSARC 2001/544 made an attempt to provide a clean API for managing network interfaces through the libinetcfg library, but those interfaces were not extended over time to accommodate features such as IPMP and Stateful IPv6 Address Configuration. An additional drawback of libinetcfg was that the API did not solve the problem of the growing complexity of the ifconfig command itself. More recently, several projects have made steps towards shrinking ifconfig bloat by providing more intuitive and elegant interfaces for configuring ipsec and tunnel management through commands like ipssecconf(1M) and dladm(1M). This project completes that trend by migrating the remaining features into ipadm(1M) while providing a full set of parallel library interfaces as defined in the design document [1]. A related issue, not addressed by PSARC 2001/544, but recognized as a problem by customers, is the lack of simple administrative interfaces for persistent configuration of TCP/IP tunables. Specifically, the ndd(1M) tool lacks Stable interfaces and a well-defined mechanism for applying settings persistently across reboot. This project will introduce a tool, ipadm(1M) that will allow system administrators to persistently set TCP/IP tunables and will have a user- and library-interface that is consistent with corresponding dladm(1M) interfaces that are in use for Layer 2 of the networking stack. The set of tunables supported by ipadm will be constructed to include meaningful semantics that are competitive with, or improve upon, those available on other Operating Systems. The interfaces provided by this project will supersede the Consolidation Private interfaces defined in libinetcfg. Existing invocations to functions in libinetcfg will be converted to appropriate invocations of libipadm. 3. Brussels - II components ---------------------------- The components of the Brussels - II project are * provision of network configuration tool (/sbin/ipadm) and associated library libipadm to support interface, address and tunable management with persistence on reboot * support for IPMP in ipadm, * transition tools, including conversion scripts to convert existing ad-hoc persistence mechanisms like /etc/hostname.intf to the format used for the ipadm(1M) repository * conversion of several ON network configuration tools to link into libipadm.so.1 Each of these components will be separately delivered and will be represented by separate PSARC cases under this umbrella. This PSARC case (PSARC 2009/306) will deliver the changes pertaining to network interface configuration. All features described in [1] other than those in Section 12, 14 and 15 would be delivered in the initial phase. Until the delivery of the last component, projects that need to adapt dynamically to the network configuration (e.g., by querying /modifying i /etc/hostname.intf files) should be aware of the changes being proposed by this project. Minimally, these should use libipadm interfaces to also query persistent ipadm configuration. In addition, the first component will deliver interfaces for obtaining interface and address configuration, including porting the BSD getifaddrs() function. These interfaces should be the preferred interface for managing interfaces and addresses. 3.1 Network interface configuration ------------------------------------- Architectural details of the CLI are available in [1]. This component will deliver a tool, /sbin/ipadm that can do all of the following ** interface creation/deletion ** address management (add,delete,show) for static IPv4 & IPv6 addresses, DHCP, stateless/stateful IPv6 Address Configuration ** tunable management (set, get, show) global (ndd) tunables, as well as (to be added) per-interface tunables ** persistent configuration of all of the above parameters on reboot, following a model similar to that used for other network configuration tools like dladm and flowadm ** modify a critical subset of network configuration tools to link into the newly created library as proof of concept. This subset will minimally include nwamd, pppd & ifconfig. ** Enhance the existing net-physical service and network_rcm to recognize and instantiate any configuration created using ipadm(1M) NOTE: ----- * Where possible, support for obsolete or meaningless ndd tunables will be removed from the system. Typical examples of such tunables are those pertaining to status reports (see CR 4616660). * Note that all the ndd tunables will *not* be converted blindly into ipadm tunables. Some may be consolidated, e.g., the various ndd tunables for "ip_respond_to_" tunables will be collapsed into one ipadm tunable that takes as input the icmp type, a "yes/no" action indicating whether to respond to the icmp type, and a "unicast/bcast/mcast" option qualifying the type of packets targeted. Section 4.4 for set of ndd tunables that would be supported by ipadm(1M). * However, since ndd(1m) has some history of usage for providing a means of tuning "customer-specials" (corner case tweaks for some subset of customers), and this needs to be done on a per-stack-instance basis, we will move all the 'other (not in section 4.4)' tunables (ip/tcp/udp/sctp) as 'private' tunables with Volatile stability that may be used with ipadm set/get prop commands. The Project team believes that taking this course of action is more beneficial as it would ensure usage of only one tool for tunable management, going forward, which is ipadm(1m), as against two tools. 3.2 IPMP subcommand support --------------------------- This component will provide support for IPMP configuration using ipadm and is described in Section 12 of [1]. In conjunction with ipmpstat (delivered by PSARC 2007/272) ipadm and ipmpstat will provide a user-friendly API for configuring IPMP and querying IPMP state in the system. 3.3 Transition existing /etc/hostname.intf configuration to ipadm ------------------------------------------------------------------- The first component of this umbrella case will initiate the adoption of ipadm by modifying interface start-up services like svc:/network/phyiscal:default and network_rcm to first parse the /etc/hostname.intf files, followed by a query of the ipadm repository (using ipadm_if_info()) to instantiate existing ipadm configuration. However, since the ultimate objective is to eradicate /etc/hotname.intf usage, the "Transition" component of ipadm will provide the necessary scripts to convert /etc/hostname.intf configuration to ipadm configuration, along with appropriate methods for alerting the administrator about the modification. 4. Interfaces delivered by PSARC 2009/306 ----------------------------------------- The following interfaces will be delivered as part of the first putback: 4.1: /sbin/ipadm(1M) -------------------- Section 2, 3, 4, 5, 6, 7, 8, 9, 10 & 11 of [1] talks extensively on each of the subcommand supported by ipadm(1M) and all the options each subcommand takes. 4.2: /lib/libipadm.so.1 ------------------------ All of the following library interfaces will be declared in libipadm.h and will be defined in libipadm.so.1, to be delivered by this project. All of these interfaces are 'Consolidation Private', unless specified otherwise. Every interface below returns ipadm_status_t, unless specified otherwise. The enumeration ipadm_status_t is defined after the table below. An interface ipadm_status2str() will be provided to get an human readable error message for a given error code. The first argument to every interface will be ipadm_handle_t, unless specified otherwise. This handle is opaque and reference to the handle is obtained by calling ipadm_open() and the handle is closed using ipadm_close(). Handle creation/destruction ------------------------------------------------------------------------------ Interface Classification Comments ------------------------------------------------------------------------------ ipadm_open Consolidation Private Opens a library handle ipadm_close Consolidation Private Closes a library handle The above two functions opens and closes an opaque library handle (struct ipadm_handle). This handle will hold socket descriptors, interface name and other implementation specific flags. Interface Management ------------------------------------------------------------------------------ Interface Classification Comments ------------------------------------------------------------------------------ ipadm_create_if Consolidation Private Section 3.1 of [1] ipadm_delete_if Consolidation Private Section 3.2 of [1] ipadm_up_if Consolidation Private Section 3.1.1 of [1] ipadm_down_if Consolidation Private Section 3.2.1 of [1] ipadm_if_info Consolidation Private Section 3.4 of [1] Address Management ------------------------------------------------------------------------------ Interface Classification Comments ------------------------------------------------------------------------------ ipadm_create_addr Consolidation Private Section 4.1 of [1] ipadm_delete_addr Consolidation Private Section 4.2 of [1] ipadm_addr_info Consolidation Private Section 4.3 & 7 of [1] ipadm_create_ipv6addr Consolidation Private Section 5.1 of [1] ipadm_delete_ipv6addr Consolidation Private Section 5.2 of [1] ipadm_v6addr_info Consolidation Private Section 5.3 of [1] ipadm_create_dhcp Consolidation Private Section 6.1 of [1] ipadm_delete_dhcp Consolidation Private Section 6.2 of [1] ipadm_dhcp_info Consolidation Private Section 6.3 of [1] Property Management ------------------------------------------------------------------------------ Interface Classification Comments ------------------------------------------------------------------------------ ipadm_set_prop Consolidation Private Section 3.3, 10 & 11 of [1] ipadm_set_ifprop Consolidation Private Same as above ipadm_get_prop Consolidation Private Same as above ipadm_get_ifprop Consolidation Private Same as above 4.3: getifaddrs ---------------- As part of this project, we will port getifaddrs() function to Solaris. ------------------------------------------------------------------------------ Interface Classification Comments ------------------------------------------------------------------------------ getifaddrs Committed Section 8 of [1], [2] This function would be exact replica of the one in *BSD and the data returned would be a linked list of the network interfaces (struct ifaddrs) on the local machine. The data returned by getifaddrs() is dynamically allocated and should be freed by the caller using freeifaddrs(). A new header file ifaddrs.h would be created, which will hold function declarations and struct ifaddrs definition. 4.4: SIOCSMODPROP/SIOCGMODPROP ------------------------------ Property interaction between libipadm and the kernel will be done through two new Consolidation Private socket ioctls SIOCSMODPROP/SIOCGMODPROP. ----------------------------------------------------------------------------- Interface Classification Comments ----------------------------------------------------------------------------- SIOCSMODPROP Consolidation Private (modified) SIOCGMODPROP Consolidation Private (modified) mod_ioc_prop_t Consolidation Private (modified) The list of tunables that would be supported in the initial release and their stability levels follows. ------------------------------------------------------------------------------- Property Scope Classification Comments ------------------------------------------------------------------------------- mtu per-if Committed see [4] metric per-if Committed see [4] state per-if Committed see [4] usesrc per-if Committed see [4] resolver per-if Committed see [4] rtexchg per-if Committed see [4] routing per-if Committed see [4] fragment_timeout per-if Committed see [4] fragment6_timeout per-if Committed see [4] default_ttl global Committed see [4] multicast_ttl global Committed see [4] brodcast_ttl global Committed see [4] pathmtu_interval global Committed see [4] redirect_interval global Committed see [4] icmp_respond global Committed see [4] forward_src_routed global Committed see [4] forward6_src_routed global Committed see [4] icmp_err_interval global Committed see [4] icmp_err_burst global Committed see [4] multidata_outbound global Committed see [4] lso_outbound global Committed see [4] All the tcp/sctp/udp tunables will be supported 'as is' in ipadm(1m). 4.5: inittab(4) changes ----------------------- We need to reapply tunables early-on during system initialization, i.e. before any network applications open sockets. Since Networking applications first need to open sockets to be affected by TCP/IP tunable settings, and the ability to open sockets is controlled by soconfig(1m), we propose to create a new process netstart(1M) that will be started from inittab(4) before svc.startd and will do the following: 1. Execute ipadm init-prop to restore globally persistent settings that apply to all interfaces 2. Start soconfig netstart(1M) will take an optional '-f file' option which represents the soconfig(1M) configuration file. The specified file will be passed to soconfig(1M) from within netstart(1M). 5. Recap of Brussels - II components -------------------------------- (1) Basic network interface configuration delivery (PSARC 2009/306) (2) supporting set of ipadm(1M) subcommands to configure IPMP (3) transition tool, perhaps a SMF service, to convert old persistent store (/etc/hostname.*), into ipadm(1M) repository. (4) conversion of several ON network configuration tools to link into libipadm.so.1 (2), (3), and (4) will depend on (1). References (enclosed in case directory) ---------------------------------------- [1] "Brussels II - ipadm and libipadm" design document - brussels2_design.pdf [2] BSD getifaddrs() manpage - freebsd_getifaddr_manpage.txt [3] one-pager - http://arc.opensolaris.org/caselog/PSARC/2009/306/20090515_girish.moodalbail.2 [4] Summary of Global or Per-interface properties supported by 'ipadm(1M)' - brussels2_prop.txt