|
|
Network Auto-Magic DesignVersion 1.0.5, 2007-Mar-05
There are eight focus areas described below: 1. Overview & Component Interaction2. Information Gathering3. Event Handler4. Profiles5. Network Service Model6. Processes & Threads7. User Interface8. Dependencies with the rest of the SystemThere are also two appendices:Appendix A: GlossaryAppendix B: Revision History1. Overview & Component Interaction1.1 IntroductionNetwork Profiles, the primary component of the Network Auto-Magic project, are a way to simplify network configuration management. They work by allowing users to specify profiles (i.e. collections of various properties) which determine how things work in different circumstances. The profiles and their respective properties include:
1.2 OverviewLet us begin with an architectural overview. The primary components are:
1.3 InteractionsHow they interact is as follows:
1.4 Examples
2. Information GatheringBefore NWAM can select a profile to activate, it needs to learn about the available network links and the connected networks. This information gathering is done at different levels of the network stack.For example, at the data link level, NWAM might detect that 802.1X is being used to do authentication and need to consult a link layer profile for authentication information. Suppose there are two Ethernet interface cards available on a Solaris box and both of them are connected to a network. NWAM might use the Link Aggregation Control Protocol (LACP) to find out if the two NICs can be grouped together to form an aggregation. If it is possible, NWAM will remember this information when selecting a profile. Another example is when a machine has a WiFi network interface card, NWAM will need to find out all the available WiFi access points before it can decide which one is preferred. At the IP level, NWAM will also need to find out several pieces of information. Again suppose there are two NICs available on a given box. NWAM will need to find out if the two NICs are connected to the same IP network. If they are, there is a possibility to set up an IPMP group on them to increase network availability. NWAM will also need to find out if IPv6 can be used with the attached IP network. It will use this information later to decide if an IPv6 interface needs to be plumbed over a network link. Another example is that NWAM will need to find out if DHCP service is available on the attached IP network. If it is, NWAM will need to find out what the available DHCP server(s) is/are providing. NWAM can use this information to select which server to talk to or if it should use DHCP to configure an interface at all. 2.1 DHCP2.1.1 BackgroundWhen NWAM detects that an IP interface is available (the IFF_RUNNING flag is set), it needs to decide how to configure the interface by consulting an NCP. An NCP may specify that this interface is to have a static IP address. Or it may specify that this interface is to be configured using DHCP. Or it may specify that depending on different "conditions," this interface is to be configured using different methods. Or it may not specify any method to configure the new interface, and NWAM needs to decide what to do with it.In the last two cases above, NWAM needs to discover what is available using the new interface. For example, it needs to find out if a DHCP server is available through the new interface. Currently, Solaris' dhcpagent(1M) does not provide a detection method which a caller can request the dhcpagent to find out the DHCP information on a given interface. When DHCP is enabled on an interface, the dhcpagent will try to complete the full DHCP lease processing and bind the interface. This document is to introduce a discover method to the dhcpagent without doing the actual lease processing. 2.1.2 Proposal
2.2 IPMP GroupsInterfaces which are connected to the same Ethernet broadcast domain should be IPMP'd together. Interfaces that are on the same IP subnet will be in the same broadcast domain; thus interfaces that are known to have an IP subnet in common should be placed in an IPMP group together. The simple subnet test will not catch all interfaces that are connected to the same Ethernet broadcast domain, however. Jim suggests an additional test to discover interfaces that should be in the same IPMP group: Instead of pinging (and all the IP and ARP headaches that involves), I'd suggest using a multicast or broadcast message sent to ethertype 9000 (loopback). If the other interfaces can hear your message, then they're in the same Ethernet broadcast domain, and can (and probably should) be IPMP'd together. 2.3 How to configure data-link and IP interfacesThis section discusses the mechanisms NWAM uses to create and configure data-links and IP interfaces. 2.3.1 OverviewIn order for a communication pipe to be useful to the user a data-link portion has to be created and configured and an IP interface needs to be created and configured on top of that. It should be noted that a richer structure can be built using tunnels over IP to create another data-link over which IP can be configured and so on. This section is closely related to Section 3 (Event Handler). That section covers how the mechanisms described in this section are used to manage the lifetime of the data-link and IP interfaces. Until Clearview has been completely integrated, aggregations, VLANs, and tunnels will not be data-links. This document will discuss the issues surrounding these objects as if they were data-links. It is expected that the necessary Clearview components will integrate before NWAM is completed. If that does not turn out to be true then an alternate design will be fleshed out and implemented. It is not expected to be difficult. 2.3.2 Creation of data-linksMany data-links are created when the associated driver is attached. We will not discuss those further in this document. For aggregations, VLANs, and tunnels data-links are created using dladm(1M). The dladm functionality is built upon the consolidation private libraries libdladm (management of link properties), libaadm (configuration of link aggregations), and libiptun (management of tunnels). Configuration of link layer attributes will be done via these libraries. 2.3.2.1 AggregationIf the link we are bringing up is actually an aggregation of data-links then we need to use the laadm_create() entry point in the laadm library. 2.3.2.2 VLANIn order to create a VLAN data-link the libdladm function dladm_datalink_create() will be used. Related functions will be used to manipulate the VLAN. See the Clearview "UV Design Specification" and "Link management API design" documents for further information. 2.3.2.3 TunnelTunnel creation will be done with the libiptun function iptun_create(). Related functions will be used to manipulate the tunnel. See the Clearview "IP Tunneling Device Driver Design Specification" for more information. In addition the documents from §2.3.2.2 are relevant. 2.3.3 Configuring IPv4The following sections will discuss various mechanisms used to apply IPv4 configuration. When creating and configuring an IP interface at the command level ifconfig(1M) is used. libinetcfg (PSARC 2001/677, 202/247, 2003/427) was built to provide a rich set of consolidation private routines for configuring interfaces. It is currently not being used by anything within ON. NWAM's needs for configuring IP will be built upon libinetcfg. Due to libinetcfg's lack of use this may cause the need to extend libinetcfg. 2.3.3.1 DHCPThe first mechanism is for the dynamic configuration of IP using DHCP. In order to communicate with dhcpagent the project private library libdhcpagent (PSARC 1999/040) is used. Per §2.1 of this document an additional state is exported from dhcpagent allowing the user to inspect potential leases before accepting one. Minor changes will need to be made to libdhcpagent to generate the new requests. Depending on the information needs of the NWAM profile being executed the appropriate DHCP command will be sent via the communication channel. If only information is being requested then that will be placed in the appropriate state structures for other components to consume. At a later point the user will either choose to accept one of the leases offered or not continue using DHCP to configure the interface. 2.3.3.2 Static ConfigurationIf interface configuration parameters are supplied by the user then libinetcfg will be use to configure the interface. Currently libinetcfg does not support the creation of logical interfaces. That functionality will be added to libinetcfg. Since the removal of logical interfaces might happen in the future, the abstraction used should be consistent with either the use of logical interfaces or the ability to apply multiple addresses to a single link. Once a logical interfaces (or equivalent abstraction) is obtained then application of the static config is done via icfg_set_{flags, addr, ...}. 2.3.4 Configuring IPv6The following sections will discuss the mechanisms used to apply IPv6 configuration. A fundamental difference between IPv4 and IPv6 is that the decision to use DHCP is not managed from the host but from the network. Additionally link-local addresses are always available on interfaces and are managed without NWAM's intervention. 2.3.4.1 DHCPSince stateful address management is controlled from the network the choice to use DHCP is not in the local user's hands. We allow the user to reject DHCP configuration but he cannot request it if the network does not offer it. Unless the user wants to reject DHCP configuration we will allow the dhcpagent to control the interface without our intervention. The file 2.3.4.2 Static ConfigurationThe library libinetcfg will be used to apply static IPv6 configuration to IP interfaces. Since the addresses are too long for most people to remember, users rarely do this. 2.4 Suspend/ResumeAfter a suspend/resume or other operation which causes a link flap, we need to re-evaluate our network environment to determine if our configuration is still valid or not. Fortunately, DHCP takes care of this for us, so there is little to do in the general case. There may be some corner cases in which the use of DNA (see RFC 4436 and I-D dna-protocol-03 for details) might provide some degree of optimization, which we may look into in the future if the need arises, but that will be part of a later phase of this project.3. Event HandlerThe NWAM daemon needs to handle many different events triggered both externally by the system or user and internally by its different threads. For example, it needs to handle the event of an IP interface becomes unavailable because the underlying link is gone. It needs to handle internally generated events such as the thread responsible to gather interface information has completed. This section explains the different events related to data-links and how they are handled by the NWAM daemon.3.1 EventsThe following are the explanations of the events the daemon needs to handle with data-link and IP interface.3.1.1 Link and Interface Events3.1.1.1 EV_LINK_CREATE / EV_LINK_DESTROYThese events are triggered externally when the daemon detects that a new data-link is created/destroyed. With the Clearview project, a new kind of sysevents will (need to be confirmed) be generated for data-link creation and removal. The daemon will listen to those sysevents and generate the appropriate internal event, EV_LINK_CREATE/EV_LINK_DESTROY. This new kind of sysevents correctly reflects all the link create/ destroy events the daemon is interested in. The EV_LINK_CREATE event can also be triggered internally when NWAM starts up. At that time, the daemon detects all existing links and may generate this event depending on the start up method. This is explained in the event handling section later. 3.1.1.2 EV_LINK_UP / EV_LINK_DOWNThe EV_LINK_UP event is triggered externally when the daemon detects that a link is now available and the link flag can be marked NWAM_LINK_UP. Similarly, if the link is down, the EV_LINK_DOWN event is triggered and the link flag NWAM_LINK_UP is cleared. The daemon will use the DLPI interface to catch DL_NOTE_LINK_{UP,DOWN} notifications. For those device drivers which do not send DL_NOTE_LINK_UP, the daemon can try periodically polling the kstat of the underlying device of a physically link to check if its link is up. For wireless interfaces, this event is generated when the link is connected to a wireless network successfully. This depends on the device driver to correctly send the DL_NOTE_LINK_UP notification to IP. 3.1.1.3 EV_IF_CREATE / EV_IF_DESTROYThese events should be triggered externally when IP interfaces are plumbed/unplumbed. But currently there is no standard way to catch such events. The NWAM daemon only knows about interface plumbing/ unplumbing done by itself. This mechanism needs to be added, possibly by using routing socket. The EV_IF_CREATE event can also be triggered internally when NWAM starts up. At that time, NWAM detects all existing interfaces and may generate this event depending on the start up method. This is explained in the event handling section later. 3.1.1.4 EV_IF_INFO_KNOWNThis event is generated internally when the daemon finishes gathering network information of an interface. The NWAM daemon may need to find out network information of an interface before it can make the decision on how to configure that interface. For example, it may want to know about the DHCP information before deciding if the interface should be configured using DHCP. 3.1.1.5 EV_IF_UPThis event is generated internally when an IP interface is configured and up. For example, suppose an interface is to be configured using DHCP. After the dhcpagent(1M) sets the interface address and marks the interface up, the daemon will get the appropriate routing socket messages (RTM_IFINFO and RTM_NEWADDR). Then it will generate the EV_IF_UP event to tell the other parts of the daemon that an interface is up and running. As indicated above, this event depends on an IP interface to have an address (RTM_NEWADDR) and is marked up (RTM_IFINFO). Both of these conditions must be true for this event to be generated. The daemon will remember if an IP interface has an address set or not and if it is marked up or not. 3.1.1.6 EV_IF_DOWNThis event is generated internally when an interface is unconfigured and down. For example, suppose an interface is configured using DHCP. After the dhcpagent(1M) successfully releases the lease, the daemon will get the appropriate routing socket messages (RTM_IFINFO and RTM_DELADDR). Then it will generate the EV_IF_DOWN event to tell the other parts of the daemon that an interface is down. This event will be generated when either an IP interface is marked down or the address is deleted. 3.1.2 Event handlingWhen starting up, the NWAM daemon reads in the Network Configuration Profile (NCP) as described in Section 4.1 "Network Configuration Profile Contents"). It then creates an internal representation of the NCP and all the Network Configuration Units (NCU). All the link structures are marked with the NWAM_LINK_NOT_PRESENT flag. After the event handler is up, the NWAM daemon will detect the existing network configuration of the system. Then it will perform either one of the following.1. If the start up method is to clean up everything, the daemon will unplumb all detected interfaces and remove all virtual links. Then for each detected physical link, the daemon will follow the contingency rules as specified in the NCP and NCU(s) to decide if it needs to generate the EV_LINK_CREATE event to trigger the creation of the associated links and interfaces as described in the NCP. 2. If the start up method is not to change the existing set up, then for each detected links and interfaces, the daemon will compare it with the internal representation of NCP and note the difference. If a link and an interface does not exist in the NCP, the daemon will trigger a EV_LINK_CREATE or EV_IF_CREATE event. 3.1.2.1 EV_LINK_CREATEThe daemon will create an internal structure for this link if it is not already created when the daemon starts up. The newly structure does not have the NWAM_LINK_NOT_PRESENT flag set. Based on the information it gets with this event, the daemon will try to find a correct position in the graph of links to place this new link. For example, if this link is a physical link, the daemon will place it as a root link. But if this link is a VLAN link, the daemon will place it as a child of the link it is based on. The link flag of the newly created representation is marked NWAM_LINK_EXTERNAL. The daemon will also check if the new link is up. If it is up, the daemon will trigger a EV_LINK_UP event. If the structure of this link is already there, the daemon will clear the NWAM_LINK_NOT_PRESENT flag and check if the link flag NWAM_LINK_ENABLED is set. If yes, the daemon will check if the link is UP and if yes, it will trigger a EV_LINK_UP event. If the NWAM_LINK_ENABLED flag is not set and the daemon gets this event, it means that an external entity has created this link. The daemon will stop the event processing. The daemon will also create a new SMF service instance of the link as explained in Section 4.1.3. 3.1.2.2 EV_LINK_DESTROYThe daemon will first start the interface NCU handling, as explained later, of this event for all the associated interface NCUs of the link. And then it will mark the link flag NWAM_LINK_NOT_PRESENT. It will walk through all its children link and mark their flag NWAM_LINK_NOT_PRESENT. The exception is for an aggregation link which still has at least one link belonging to that aggregation without being marked NWAM_LINK_NOT_PRESENT. The daemon will also destroy the corresponding SMF service instance of the link, as explained in §4.1.3. 3.1.2.3 EV_LINK_UPThe daemon will check the link flag of the corresponding link structure. If it is already set, the daemon stops the event processing. If the flag is not set, the daemon will set the flag. Then it will walk through all the children of the link and trigger a EV_LINK_UP event for each of them if it is marked NWAM_LINK_ENABLED. If there is no child link, then the daemon will start the interface NCU handling as described below for all the associated interface NCUs. The daemon will also create a new SMF service instance of all the associated interface NCUs as explained in Section 4.1.3. 3.1.2.4 EV_LINK_DOWNIn getting the EV_LINK_DOWN event, the daemon will clear the NWAM_LINK_UP flag of the corresponding link. It will then walk through all the children of that link clearing the NWAM_LINK_UP flag. If a child link is an aggregation and there is another link belonging to that aggregation which still has the NWAM_LINK_UP flag set, the aggregation link flag is not touched and the traversal of link will be stopped for that particular branch. For all the children links which have the NWAM_LINK_UP flag cleared but have the NWAM_LINK_ENABLED flag set, the daemon will start the interface NCU handling of this event as described below in "IP interface NCU handling" for all the associated interface NCU(s). 3.1.3 IP interface NCU handlingEach interface NCU is represented by a structure and the daemon runs a state machine for it. The following are the possible states. 1. NWAN_IF_INIT 2. NWAM_IF_QUERYING 3. NWAM_IF_READY 4. NWAM_IF_RUNNING For every structure, the initial state is NWAM_IF_INIT. For each interface NCU, the daemon creates it when it reads in the NCP. For externally created interface, the daemon triggers the EV_IF_CREATE event and a structure will be created. This is a brief state transition diagram of the interface NCU structure. Note that the EV_IF_DESTROY transition is not shown. The reason is that the EV_IF_DESTROY transition is also dependent on the EV_IF_EXTERNAL flag. This flag is set for externally created interfaces not in the NCP. Instead of adding a new set of states just for those interfaces, the flag is used to differentiate them. So the states are the same for both types of interface. An EV_IF_DESTROY event will remove the structure for externally created interface (meaning going to CLOSED state). But for an interface in a NCP, the structure stays and the state will transition to NWAM_IF_INIT.
The interface state transition diagram is as follows; 3.1.3.1 CLOSEDEV_IF_CREATE: the daemon will create the structure representing this interface. The flag of this newly created interface is set to NWAM_IF_EXTERNAL. The state of this interface structure is set to NWAM_IF_RUNNING. And the daemon will find out information about this interface and use the gathered information to fill in the structure. 3.1.3.2 NWAM_IF_INITEV_LINK_DESTROY: the daemon will unplumb the IP interface. EV_LINK_UP: if the flag does not have NWAM_IF_EXTERNAL set, the IP interface is plumbed if it is not plumbed yet and the state will be changed to NWAM_IF_QUERYING. The state machine will start a thread to collect information on the network. EV_IF_DESTROY: if the flag has NWAM_IF_EXTERNAL set, the daemon will free up the interface structure. The daemon will also destroy the corresponding SMF service instance of the interface, as explained in §4.1.3. All other events: do nothing. 3.1.3.3 NWAM_IF_QUERYINGEV_IF_INFO_KNOWN: the state will be changed to NWAM_IF_READY. The state machine will use the information stored in the interface NCU to determine how to configure the IP interface. If only static IP addresses are needed to be set, the IP is configured and the state will be changed to NWAM_IF_RUNNING immediately. If further processing are needed, such as finishing up the DHCP processing, the state machine will start a thread to do the job. EV_LINK_DESTROY: the state machine will unplumb the IP interface and change the state to NWAM_IF_INIT. EV_LINK_DOWN: the state will be changed to NWAM_IF_INIT. Note that the interface is not unplumbed. EV_IF_DESTROY: the state will be changed to NWAM_IF_INIT. All other events: do nothing 3.1.3.4 NWAM_IF_READYEV_IF_UP: the state will be changed to NWAM_IF_RUNNING. EV_LINK_DESTROY: if a thread has been spawned to configure the interface, the state machine will make a note and the thread will de-configure the interface. If there is no configuration in progress, the state machine will unplumb the IP interface. The state will be changed to NWAM_IF_INIT. EV_LINK_DOWN: if configuration is in progress, the state machine will make a note and the thread will not configure the interface. The state of the link will be changed to NWAM_IF_INIT. Note that the interface is not unplumbed. EV_IF_DESTROY: the state will be changed to NWAM_IF_INIT. All other events: do nothing 3.1.3.5 NWAM_IF_RUNNINGEV_IF_DOWN: if NWAM_IF_EXTERNAL flag is not set, the state will be changed to NWAM_IF_READY. EV_LINK_DOWN: if NWAM_IF_EXTERNAL flag is not set, the daemon will de-configure the IP interface. The state will be changed to NWAM_IF_INIT. EV_LINK_DESTROY: if the flag NWAM_IF_EXTERNAL is set, the daemon free up the structure. Otherwise, the daemon will unplumb the IP interface and the state is changed to NWAM_IF_INIT. EV_IF_DESTROY: if the flag has NWAM_IF_EXTERNAL set, the daemon will free up the interface structure. The daemon will also destroy the corresponding SMF service instance of the interface, as explained in Section 4.1.3. If the flag is not set, the state will be changed to NWAM_IF_INIT. 4. Profiles4.1 Network Configuration Profile ContentsSee Architecture §4.1 for background and §5.1.1, item 1 for a related discussion of how this will be stored.4.1.1 Previously Considered AlternativesOriginally we had N separate Link-Layer Profiles (LLPs), one per link, but these were perceived to be too complex to be workable when putting the whole system together. So we came up with the idea of a single Link & Interface Specification (link-spec), but this seemed to be too inflexible, and people had a hard time understanding it.4.1.2 ProposalTo solve these problems, we returned to the idea of LLPs, using them as a starting point. We then added some attributes to make it easier to build these units into a complete picture. The result is something akin to LLPs which we have dubbed NCUs (network configuration units), which when all put together comprise something akin to a link-spec: an NCP (network configuration profile). An NCP may also contain information about external network applications that may be registered with NWAM, allowing them to be enabled/disabled by NWAM according to rules defined by the user. This registration interface is discussed in §4.5.2. 4.1.3 DetailsThere are several attributes which make up an NCU, each of which can take one of several values, some of which have sub-values. In the internal representation of the configuration, we will have a subset of this structure per address. Thus the attributes in this structure possible describe a set of NCUs:
4.2 Multiple Active LinksThe primary design goal here is that, whenever possible, things should Just Work™, with minimal manual intervention. Since we have the technology to plumb all interfaces at boot, and bring them all up (`ifconfig
IF dhcp` as needed, possibly preceded by `wificonfig -i
IF autoconf` or equivalent), the key question is when we would
want to do that, or more to the point, when we would not want to do
that. In other words, we can already implement the mechanisms, but we need
to be able to implement various policies.
So we will need the UI to allow users to specify policies, but we also need to have a default policy (i.e., for when the user has failed to express his/her preferences for a given scenario). The proposal is that by default all interfaces are plumbed and brought up, though wireless networks will only be connected to after explicit authorization from the user. If multiple IP interfaces get brought up, then IPMP will be used as possible. IPMP does not work with DHCP, though this is being fixed by Clearview's IPMP Rearchitecture project. So until this is fixed, when multiple IP interfaces are detected to be in the same broadcast domain, only one will be brought up, and it will be brought up using DHCP but not IPMP. Once the IPMP Rearchitecture project is complete, all such interfaces will be brought up using DHCP and IPMP. This should allow us to do the Right Thing™ in the horizontal scaling and large server scenarios, yet still be flexible for the laptop scenario. The rationale for this set of defaults is Requirement #17: For security reasons, bridging networks must be not permitted by default. (Note that we tend to think of this internally as "thou shalt not have a SWAN interface and a non-SWAN interface up at the same time".) It is fairly safe to assume that if a cable is plugged into an Ethernet port of a machine, then the user wants the machine to bring up the network to which the cable is attached, whereas a wireless interface could pick up an AP from just about anywhere, and choosing from among multiple networks (and even worse, multiple unknown networks) without data is a potential security hazard. 4.3 Environment ContentSee Architecture §4.2 for background and §5.1.1 for a related discussion of how this will be stored.An Environment (previously called an Upper Layer Profile or ULP) contains information to configure the system after IP service is available. As opposed to NCP attributes which are per-link or per-interface, each of these Environment attributes affect the entire system.
4.3.1 DuckwaterThe Duckwater (aka Simplified Name Service Management) project will introduce name service profiles, which are collections of name service configuration data. These collections will include both name service switch (which name service back-end to use for different types of lookups) information and configuration details for specific back-ends (e.g. DNS, NIS, LDAP). The general rule we are following for NWAM is that the NWAM Environment specifies the enabled/disabled state of certain upper-layer network services, but the detailed configuration and policy rules for those services belong in the services themselves. Thus for name service configuration, NWAM will allow the user to specify a name service profile as part of each Environment: when setting up an NWAM Environment, "Name Services" will be a tab, menu item, whatever (depending on GUI details). When you choose this item, you will be presented with a single list of options: Automatic The Automatic option would be the default and would map to a Duckwater profile with all properties set to "automatic." These are described in the Duckwater design draft auto value for nsswitch/* properties means that the name service switch (nscd(1M)) should decide which source types to use based on the back-end availability information provided to it by back- ends and delivered using nsutil(1M) [nsutil].[Note that NSBEC stands for Name Service Back End Configuration.] The link to the Duckwater/name service configuration GUI could be a menu item, or it could be a button in the Name Services window. 4.3.1.1 Alternatives Considered
4.3.1.2 Concerns
4.3.2 Enhanced SMF Profile FrameworkNWAM profiles will be instances of Enhanced SMF profiles. The Enhanced SMF Profile model allows SMF service properties to be grouped into profiles, which may then be layered on the running system. In this model, NWAM profiles will be layered above the base and the system default (generic_open, for example) profiles. The
local profile will include changes introduced by the user,
and will overlay any NWAM profiles.
4.3.2.1 Secure By Default and Enhanced SMF ProfilesThe Enhanced SMF Profile team is planning to introduce new system profiles to reflect Secure By Default requirements. The current proposal is to breakgeneric_open and generic_limited_net into four new
profiles:
system_services and local_services profiles
will be applied by default, layered below any NWAM profiles. The
secure_net_services and network_services profiles may be
used as templates for the creation of new NWAM profiles.
4.3.3 host namesCurrently if the user wants to change the hostname of the system, s/he has to either sys-unconfig then configure the system, or update a bunch of files then reboot the machine.NWAM can introduce an interface for setting the hostname and subscribing to hostname changes. To achieve this, the following steps need to be taken:
4.4 Environment PolicyThis section describes the way a user can create Environments for NWAM and also how the policy for selecting the appropriate Environment will work.4.4.1 Creating an EnvironmentThe creation of a new Environment would be explicitly initiated by the user via the User Interface (UI). When the user indicates that s/he wishes to create a new Environment, the UI would enable the user to provide the list of attributes associated with that Environment. This set of attributes (or outputs) would be available when the Environment is active. See §4.3 just above for a list of these attributes.Due to the system-wide nature of these attributes, only one Environment may be active at any time. Based on each attribute selected, a detailed description of services/rules and/or configurations applicable to that particular attribute will be presented. So, for example, if the user wants to specify a particular node name for the machine, and so selects the attribute "Node Name", the UI will prompt him/her for the node name s/he wants to give the machine. The user will also need to specify some sort of selection criteria (or input) such that the NWAM daemon can find the best-fit Environment, given the information available. These selection criteria include IP address, domain name, ESSID and/or BSSID to connect to (in case of wireless) and possibly the active link available. 4.4.2 Selecting an EnvironmentBy default, NWAM will provide two Environments - a "No Network" Environment and an "Automatic" Environment At the time of selection of Environments, the NWAM daemon will first check for the number of active links available. If there are no active links available, then the daemon will not try to apply any Environment, neither user-defined nor "Automatic". Instead, it will assume that the "No Network" Environment should apply, and that the attributes for the machine in the stand-alone mode are required. However, if an active link is discovered, the daemon will examine the Conditions of each Environment and select the one which matches. If none match, the "Automatic" profile will be available as a fall-back, allowing the NWAM daemon to connect to some available network (per Requirement #7). If more than one Environment has Conditions which are met, the Priority of the matching Environments will be used as a tie-breaker.4.5 External Network ModifiersThe NWAM solution must be able to coexist with external entities, in particular those provided by third parties, which may perform their own link/IP configuration. We will henceforth use the term External Network Modifiers (ENM). Configuration performed by an ENM cannot be stored within the NWAM framework; this would require that the user understand the details of how the service is configured, or that NWAM be designed with that knowledge, for all possible services. Neither of these alternatives is practical. §4.5.1 describes the coexistence of NWAM and ENMs when they have no knowledge of each other. §s 4.5.2 and 4.5.3 discuss an interface provided by NWAM which will allow better integration of NWAM and ENMs. 4.5.1 Externally-Initiated ConfigurationThe model described in §4.1 allows externally-initiated configuration; if an ENM (such as a VPN application) does any new plumbing, NWAM's view of the system will be updated to reflect the changes. This will be done by creating in-memory representations of these new links and/or IP interfaces; as such, these changes are considered transient and not part of the NCP stored in the repository. When the NWAM service is restarted or refreshed, transient changes will not be recreated. It is not feasible for NWAM to attempt to recreate a given configuration for which it does not have a complete description; nor is it reasonable to extract the NWAM-controlled part of the configuration and only reset that part. Rather, the restart/refresh request must be treated as a request from the administrator for NWAM to restore a specific configuration, i.e. the current NCP stored in the repository.
If the ENM is an SMF service, it can be restarted automatically in this case
with a dependency (with 4.5.2 Service Registry InterfaceENMs may be registered with NWAM, allowing NWAM to correctly enable and disable them as needed. The registration may be performed by the ENM itself via an NWAM-provided API, or by the user via the NWAM UI. An "unregister" operation will also be provided, removing a registered ENM. Registered ENMs will be instances of the NWAM service. Attributes will include:
When an ENM is registered with NWAM, its instance will be created; it will not be tied to the current NCP. Thus, if the NCP is changed, the ENM registration will not be affected. 4.5.3 API for External Network ModifiersIn addition to the service registry, NWAM will offer several control hooks to ENMs. ENMs need not be registered with NWAM to take advantage of these control hooks.4.5.3.1 Environment Selection EngineENMs may request that the Environment selection engine be run; this may be desired if the ENM makes changes which may not automatically trigger the engine (such as bump-in-stack VPNs, which do not create any new links or system changes detectable by NWAM).An additional hook will allow ENMs to request that the selection engine not run until a further request to run it is received. This may be desirable if the ENM is making a series of configuration changes, and it wants to be sure that NWAM does not react until all of its changes have completed. A timeout must be included with this request. 4.5.3.2 Environment SpecificationsNWAM will also provide an interface for external entities, including ENMs, to load complete Environment specifications, including conditions under which the Environment should be applied. Thus, an NWAM-aware ENM would be able to specify the Environment that should be activated when the ENM is enabled.These Environment specifications will be visible through the NWAM UI, and may be modified by the user, as with any other Environment specification. The intention here is to allow an application to install itself such that minimal user intervention is required; but a user who wishes to further customize may certainly do so. 4.6 Wireless4.6.1 Prioritizing of WLANsThe NWAM daemon maintains a "Preferred WLANs List" which contains the information of all the WLANs that the system has connected to and the used has asked NWAM to remember. This information includes the ESSID, BSSID, the priority of each WLAN, and the type of encryption/authentication used (if any). When a system needs to connect to a network, the daemon scans for the available WLANs. This list of available WLANs is henceforth referred to as the "available WLAN List".Since it is quite possible to have multiple known WLANs available at the time the system needs to connect to the network, it is required that some kind of prioritization be applied to the "Preferred WLANs List". NWAM will take a simple approach to the prioritization - it will allow the user to specify the order in which the known WLANs should be tried in order to connect, and, in the event that the user has not explicitly specified any order, it will assume that the known WLANs should be tried in the order in which they appear on the "Preferred WLANs List". The NWAM daemon will use the following policies to create and maintain the "Preferred WLANs List" -
Non-broadcast WLANs are those that don't show up in the `wificonfig scan`. They require that user inputs the ESSID at the very first time. The NWAM daemon keeps track of all non-broadcast wireless LANs which a user has specified, as described in the discussion above about the Preferred WLANs List. This would include both the non-broadcast WLANs that it has successfully connected to, and also those that the user has provided but may not have been connected to yet, unless the user specifically deletes the entry from the "Preferred WLANs List". Only one "Preferred WLANs List" is maintained for the system instead of a per NCU list, which will include both the broadcast as well as the non- broadcast WLANs. The biggest reason for maintaining a single list is to avoid the deadlock situation at boot time. The NWAM daemon has to know which Profile would need to get activated when the system boots up. However, to decide which Profile gets activated, the daemon would have to know which WLANs are available, which makes it a deadlock if the "Preferred WLANs List" is kept per Profile. The "Preferred WLANs List" will be stored in the SMF repository (libscf.3LIB). The "available WLAN List", however, will be created per NIC, but all the respective "available WLAN Lists" for all NICs will be rolled into one master "available WLAN List" to be presented to the user. As mentioned above, the user decides the order in which WLANs appear in the "Preferred WLANs List". Whichever WLAN has been assigned the highest ordering by the user, will be tried first (irrespective of whether it is a broadcast WLAN or a non-broadcast WLAN). Note: during design, it was suggested that "many users would like to choose between available known networks at connect time rather than having to make a persistent decision which will often be wrong". As we did not have sufficient data about which model would be better, we chose the one which we thought was more "auto-magic". 4.6.2 Periodic Scan of Wireless Access PointsThe daemon would have to perform periodic scan of all available wireless APs to ensure completeness of information about the network. The scan needs to be done periodically so as to make sure that any WLANs that were not visible at boot time (or connection establishment time), but have become available later, do not get ignored. This is especially important in situations where the WLAN which has become available later is actually present in the "known WLAN List" with a higher priority than the one with which connection is currently established.As of now, the scan will be done on a fixed interval, the exact duration of which will be determined later. Apart from this, scan should also happen every time the user uses the UI option to list the available WLANs. Effect of frequent scans on the system:
4.7 RoutingGenerally speaking, routing is an area where complex configuration and policy details should remain part of the subsystem, and not be subsumed by NWAM. NWAM's role should be that of administrator, specifying the circumstances under which the subsystem (or components of the subsystem) should be enabled; §4.7.1 will discuss this aspect of NWAM's relationship with the routing subsystem.Unfortunately, it is difficult to make such a clean distinction between NWAM and the routing subsystem. There are a couple additional areas in which NWAM must interact with the routing subsystem; these are discussed in §4.7.2 and §4.7.3. Finally, the interfaces between NWAM and the routing subsystem's Routing Information Base (RIB), for the setting of both routes and routing policy, are defined in §4.7.4. 4.7.1 NWAM as Routing AdministratorOne component of the Upper Layer Profile/Environment will be the specification of which routing daemons to run. NWAM will build on the functionality introduced by the Quagga project to convert routing services to the SMF framework. As SMF services, the different routing service options can easily be enabled/disabled by NWAM as specified by the user.
The NWAM interface will offer the user all the functionality that is
available through the 4.7.2 NWAM as Route ProviderThe user may wish to set static routes for a particular environment. Thus, a list of static routes will be an additional component of the Upper Layer Profile/Environment. When the ULP/Environment is enabled, NWAM will pass these routes to the RIB, which will apply policy and send routes to the kernel's Forwarding Information Base (FIB) as needed. The interface between NWAM and the RIB is specified in §4.7.4.4.7.3 NWAM as Routing Policy ProviderOn multi-homed systems, it is possible that automatic configuration could result in multiple routes to a given destination. The most common example of this is the case where a system with two interfaces uses DHCP on both interfaces, and is given the address of a different "default" router (i.e. a route to destination 0.0.0.0 or ::0) on each interface. In that case, the user may specify a preferred interface as part of the NCU. NWAM will pass this preference to the RIB, which will take it into account when determining which routes to send to the FIB. Routes learned via DHCP and IPv6 Router Advertisements will be sent to the RIB for policy filtering, rather than directly to the kernel's FIB, as is currently the case.4.7.4 Interfaces Between NWAM and the RIB
The zebra daemon will serve as the RIB, receiving routes from various
sources (routing daemons,
The interface by which routes are specified will be based on the existing
The interface by which routing policy is specified will be extensible, so that in the future different types of policy may be passed using the interface; but initially, interface preferences will be the only type of policy specified. It should be noted that this design requires changes beyond the NWAM project:
4.8 Static ConfigurationStatic (link & IP) configuration means everything has a fixed value, which also implies that the conditions are static and thus whatever Environment is selected is also static. This is all just a degenerate case of the rest of this document; this section is only included for the sake of completeness.4.9 Legacy Support4.9.1
One of the most vile interfaces (in the |
| service name | current dependency | future default disposition |
|---|---|---|
| milestone/single-user:default | network/loopback (require-any) | enabled, not controlled by any NWAM profiles |
| system/identity:node | network/loopback (require_any) and network/physical (optional_all). | The NWAM daemon will set the node name and domain after it decides the value for each, then the daemon will enable the services. The services should be refreshed if the network environment changes and the change causes the nodename / domain to change. |
| system/identity:domain | ||
| network/ipfilter:default | network/physical (require_all). | disabled in standalone Environment |
| network/ssh:default | network/loopback (require_all) and network/physical (require_all). | disabled in standalone profile |
| network/smtp:sendmail | network/service (require_all). | enabled in standalone profile as local-only mode and needs to be refreshed if the network environment changes. |
| network/inetd:default | network/loopback (require_any) | enabled in standalone profile |
| network/iscsi_initiator:default | network/loopback (require_any) | enabled in standalone profile |
| network/rarp:default | network/loopback (require_any) | disabled in standalone profile |
| network/slp:default | network/loopback (require_any) | disabled in standalone profile |
| network/shell:kshell | network/loopback (require_any) | disabled in standalone profile |
| network/shell:default | network/loopback (require_any) | disabled in standalone profile |
| network/ntp:default | network/service (require_any) | disabled in standalone profile |
| network/dns/server:default | network/loopback (require_any) | disabled in standalone profile |
| network/dns/client:default | network/loopback (require_any) and network/service (require_any) | disabled in standalone profile |
| network/ldap/client:default | network/initial(require_all) | disabled in standalone profile |
6. Processes & Threads
6.1 Breakdown
NWAM will be broken down into a set of processes and threads. This section specifies how this will be done. This section is coupled with §8.1 (Internal IPC) and §8.2 (External IPC).
Along with this section are diagrams which show how the nwam daemon interacts with the system and other processes and another diagram which shows some of its internal organization into threads. The diagram is available at nwam_process.pdf.
At this time there is no room for a legend on the diagrams so here is help in interpreting the diagrams. In the first diagram (NWAM sys interaction) the diagram is broken up by a horizontal red bar which indicates the user space/kernel space split. Circles below the bar indicate conceptual sources of events. In some cases those will actually be connected through the kernel to another user space process or might actually be multiple sources of information. Boxes above the line indicate user space processes. Ovals above the line indicate shared objects or families of shared objects. A line connecting a box to an oval indicates that the process (box) uses the shared object (oval). Arrow between boxes indicate primary data flow. There is also a vertical line which indicates the separate between two zones. With shared networking stacks the configuration information will be provided by the nwam daemon in the global zone. For zone exclusive stacks the configuration information will be provided by a nwam daemon running in the zone the stack is attached to.
In the second diagram (NWAM proc design) the same kernel/user space horizontal bar exists. Below the bar are event sources. Above the bar boxes indicate threads within the NWAM process (instead of separate processes as they did on the previous diagram). Ovals indicate shared objects again. Lines indicate a uses relationship and arrows indicate data flow. Rectangles with horizontal lines through them indicate a queue used for inter-thread communication.
6.1.1 Core functionality
The core functionality of NWAM will be contained in a single daemon. Within that daemon will exist a set of threads used to manage various asynchronous needs.
6.1.2 main event loop (MEL)
A single thread will be used to manage the main event loop (section 3) The other threads will gather up events and feed to them to queue that this thread retrieves its work from.
6.1.3 event collection
A set of threads will be used to collect events. Each of these threads will collect events and place them on the main event queue for the main event loop to consume.
A single thread will be used to turn signals into synchronous events.
Signals will be blocked in all threads. sigwait(2) is used
to wait for threads and that that thread will determine how to manage that
signal. This allows for the synchronous management of signals.
A thread will be used to collect file based events. At this time these are the events from the routing socket.
A thread will be used to wait for sys events.
6.1.4 Integration with Zones
Post stack instances (Nevada build 57) the administrative model for interfaces in zones with exclusive stacks is similar enough to that of a global zone and rich enough to be able to manage interfaces from within a zone. We will have a separate NWAM daemon running in the each zone to effect this. Shared stacks are administered from the NWAM daemon running in the global zone. The user can assign use of shared stacks to non global zones via zonecfg(1m).
6.1.5 Future integration with Crossbow
As more of Crossbow is integrated into Solaris the administrative model of Zones will become richer. In section 6.1 there is a description of how configuration will be applied across zones with different types of networking stacks.
6.1.6 Integration with CLI
The cli will be a single command modeled after zoneadm(1M).
6.1.7 Integration with GUI
Integration with the GUI will be done via the NWAM API defined in §7.3.
6.1.8 Integration with Gnome panel application
The API mentioned in §6.1.7 immediately above will be used to integrate with the Gnome panel application.
6.2 Internal IPC
The current design for NWAM is as a single process. Since Stack Instances integrated further work will be done to have the nwam daemons managing zone exclusive networking stacks communicating with the daemon in the global zone if necessary.
7. User Interface
7.1 Command Line Interface
See our separate page for this.7.2 Graphical User Interface
See our separate page for this.7.3 Data Structures & APIs
See our separate page for this.8. Dependencies with the rest of the System
8.1 Privileges
8.1.1 Introduction
Least Privilege for Solaris (PSARC 2002/188) was introduced in Solaris 10 build 30 (2003-Feb-27). It provides a mechanism to divide up the privileges normally given to root so that processes can execute with a minimal set.
8.1.2 Process requirements
The NWAM processes will need the following privileges. Depending on how NWAM is broken into processes only a subset of these might be needed by any specific part of NWAM. The privileges needed by external processes (like dhcpagent, §2.1) are not listed here.
| privilege | requirement |
|---|---|
| PRIV_SYS_NET_CONFIG | §2.3 needed to configure network interfaces |
| PRIV_NET_RAWACCESS | §3, 4.7 (needed for routing socket) |
| PRIV_PROC_EXEC | §4.3 Environment selection script |
| PRIV_PROC_FORK | §4.3 Environment selection script |
Two types of user hook are available. The first (§4.4.2) allows the user to write code to query the state of the machine and select the appropriate environment. The second (§4.5) allows the user to provide a hook to invoke functionality not directly supported by NWAM.
The selection scripts do not need to be run with any special privileges. They are not allowed to modify their environment. This script will be run as a non-root user with basic privileges.
The configuration scripts will need to be run with privileges. Several
designs have been discussed from exec'ng with all privileges to pfexec'ng
from basic privileges. Since it is going to take root privileges to install
one of these scripts (at least one that wants additional privileges) it is
not seen as a burden for the user to have to create exec_attr
entries for each script. Thus the configuration scripts should assume that
they will be pfexec'd with a default of basic privileges.
8.1.3 Startup considerations
Nominally NWAM will be started by SMF. In order to allow an administrator to start NWAM by hand when debugging configuration privileges will be assigned to the NWAM processes (section 6) via exec_attr(4). This will be referenced via use_profile and profile in the smf_method(5) manifest.
8.2 External IPC
There are multiple external processes which NWAM must communicate with. They are listed in the following table with information about the type of information exchanged and the method of IPC.| External Entity | Purpose of | Kind of IPC interaction | Notes |
|---|---|---|---|
| RIB (Quagga) | interface preference | undefined | §4.7.1 |
| kernel | obtain state changes | routing sockets | |
| DHCP | DNS server, default route, IP address, etc. | socket via libdhcpagent | |
| Link Status change | LINK_UP/DOWN info sys, *_RUNNING flag changes | sysevents, DLPI | |
| SMF | system management state changes | door via libscf | |
| CLI, GUI | allow for user interaction | door | §7.3 |
| NWAM in other Zones | coordinate NWAM across zones | undefined |
8.3 Install
Between the Solaris installation program and the sysidtool(1m) suite of programs, many questions are asked of the user, several of which will no longer be needed once this project makes them obsolete. I.e., NWAM will do as its name suggests, and automatically determine network configuration for use both during installation and subsequently. If any permanent static network configuration is desired, it will be deferred; post-installation, the NWAM UI can be used for this.Specifically, there are several programs in the sysidtool(1M) suite which will need to be modified not to ask questions for which NWAM will provide the answer. In particular:
- sysidnet will no longer need to query for network interface or IP address or IPv6 usage (IPv6 will be enabled by default)
- sysidns will no longer need to query for anything; in fact, this may go away completely
- sysidsys will no longer need to query for netmask (as no IP address will be queried for)
- sysidnfs4 will no longer need to query for the NFSv4 domain (it will be left undefined)
Looking ahead to implementation, this will require multiple putbacks, as most of NWAM will go in the ON consolidation, whereas the sysidtools are in the Install consolidation, unless Visual Panels should happen to come along soon enough to save us the trouble.
Appendix A: Glossary
Appendix B: Revision History
| Revision | Date | Changes |
|---|---|---|
| 0.1 | 2006-Apr-11 | initial draft, cloned from Version 1.0.3 of Architecture |
| 0.1.1 | 2006-Apr-18 | changed section 4 name from Configuration to Profiles |
| 0.1.2 | 2006-Apr-20 | split sections into child pages |
| 0.2 | 2006-Aug-30 | re-merged child pages, updated with months of discussions |
| 0.2.1 | 2006-Aug-31 | updated diagram, filled in §2.2 |
| 0.2.2 | 2006-Sep-01 | updated §2.2 |
| 0.2.3 | 2006-Sep-13 | Added introduction to §2 |
| 0.2.4 | 2006-Sep-14 | First draft of §4.8 |
| 0.2.5 | 2006-Sep-26 | First draft of §5.4 |
| 0.2.6 | 2006-Nov-03 | First draft of §4.6 |
| 0.3 | 2006-Nov-10 | Filled in §4.1 & §4.3, merged §5.4.4 into §4.8.1, moved §5.4.5 to §4.3.3, rewrote §5.4 |
| 0.3.1 | 2006-Nov-21 | Minor corrections to §5.4 |
| 0.3.2 | 2006-Nov-21 | First draft of §4.3.1 |
| 0.3.3 | 2006-Nov-28 | Filled in §4.4, minor tweaks to §4.3, added §8.3 |
| 0.3.4 | 2006-Dec-01 | Revised §4.8 |
| 0.3.5 | 2006-Dec-04 | First draft of §4.3.2 |
| 0.3.6 | 2006-Dec-05 | Added §2.4 |
| 0.4 | 2006-Dec-08 | Filled in §2.3, §6.0, §6.1, §8.1 |
| 0.4.1 | 2006-Dec-10 | Fixed broken hierarchy in §6 |
| 0.5 | 2006-Dec-15 | Major revision to §4.1 |
| 0.5.1 | 2006-Dec-19 | Updated §s 4.8.2 and most of 5 to match §4.1 |
| 0.5.2 | 2006-Dec-21 | Added §8.2 |
| 0.5.3 | 2006-Dec-21 | Minor edits (orange text removal) |
| 0.5.4 | 2006-Dec-21 | Filled in §4.7, more orange text clean-up |
| 0.5.5 | 2006-Dec-21 | Updated §4.2 & §4.8.1 to match §4.1 |
| 0.6 | 2006-Dec-21 | Updated §1 to match §4.1 |
| 0.7 | 2006-Dec-22 | Inserted new §4.5 (shifting §4.5 to §4.6, etc.), updated §4.1.2, §4.3. & §5.1.1 to match |
| 0.7.1 | 2006-Dec-23 | Edited §8.1.2 |
| 0.7.2 | 2007-Jan-03 | Minor edits to §s 1.3 & 1.4 |
| 0.7.3 | 2007-Jan-04 | Minor edits to many §s |
| 0.7.4 | 2007-Jan-04 | Revised §2.4 |
| 0.7.5 | 2007-Jan-09 | Revised §8.1.2 |
| 0.8 | 2007-Jan-11 | Minor update to §4.1.3, significant updates to §s 4.5 & 5.4.2 |
| 0.8.1 | 2007-Jan-12 | Fix typos from 0.8 updates |
| 0.8.2 | 2007-Feb-02 | Added §7.3: link to APIs doc |
| 0.8.3 | 2007-Feb-13 | Edit §6, §7.1: remove orange text |
| 0.9 | 2007-Feb-16 | Update §3: rewrite |
| 0.9.1 | 2007-Feb-16 | Update §6.1.4, §6.1.5: for stack instances |
| 0.9.2 | 2007-Feb-16 | Replace ASCII art in §3.1.3 with diagram |
| 0.9.3 | 2007-Feb-16 | Minor cleanup in §3.1 |
| 0.9.4 | 2007-Feb-16 | Minor tweaks to §s4.1 & 5.1 |
| 1.0 | 2007-Feb-16 | Link to CLIs page from §7.1, Design Complete |
| 1.0.1 | 2007-Feb-28 | Fix a number of minor issues noticed by David Bustos |
| 1.0.2 | 2007-Mar-01 | More minor issues noticed by David Bustos |
| 1.0.3 | 2007-Mar-02 | Minor issues in §5 |
| 1.0.4 | 2007-Mar-05 | Updates due to issues noticed by David Bustos, §6.14, §8.1.2, §8.2 |
| 1.0.5 | 2007-Mar-05 | Updates due to issues noticed by David Bustos, §2.3.3.1, §4.1.3 |