Overview ======== This document specifies the sysevent definitions for the IPMP subsystem. These events are being defined for consumption for the IPMP test suite, inside ON consolidation, and by contract with other consolidations (e.g., Sun Cluster). The events are generated by the IPMP daemon process (in.mpathd), and delivered to the user-level consumers via the sysevent_evc_publish() interface defined by PSARC/2002/321. The interfaces defined here assume a model where applications first use a set of "query" interfaces to retrieve the initial state of the IPMP subsystem, and then subsequently rely on the asynchronous events defined within to learn of and react to changes to that state. The query interfaces are specified separately. Details ======= Following the conventions in PSARC/2000/189, we specify the following subclasses to uts/common/sys/sysevent/eventdefs.h: /* Sysevent Class definitions */ #define EC_IPMP "EC_ipmp" /* event class for IPMP */ ... /* * EC_IPMP subclass definitions */ #define ESC_IPMP_GROUP_STATE "ESC_ipmp_group_state" #define ESC_IPMP_GROUP_CHANGE "ESC_ipmp_group_change" #define ESC_IPMP_GROUP_MEMBER_CHANGE "ESC_ipmp_group_member_change" #define ESC_IPMP_IF_CHANGE "ESC_ipmp_if_change" #define ESC_IPMP_PROBE_STATE "ESC_ipmp_probe_state" The header file contains all IPMP event attribute definitions (name/value pairs). See "Sysevent Definitions" below for details. Of course, it is hard to make much use of the aforementioned events without knowing additional context, such as the list of IPMP groups in the system, which interfaces belong which IPMP groups, and what the states of those interfaces are; this functionality is provided by the aforementioned IPMP query API. Unfortunately, since querying the current IPMP state is distinct from the delivery of sysevents, this introduces a synchronization problem. For instance, if an application queries the IPMP state associated with an interface and then receives a sysevent stating that the interface has changed states, the application has no way to determine which mechanism is providing the current information (since it's possible the sysevent was generated prior to the application retrieving the interface state). Furthermore, while uncommon, it is possible for sysevents to be lost, which will cause the application to have an incorrect view of the multipathing subsystem. To address these issues, in.mpathd associates a 64-bit signature with each IPMP group. Each time an observable change is made to an IPMP group or one of its members, this signature is incremented. By including the current signature of the relevant IPMP group in each IPMP sysevent and IPMP query interface, it becomes possible for the application to synchronize between the two interfaces. Furthermore, since signatures are always monotonically incremented, applications can easily detect lost sysevents by noting gaps in the signature space, at which point they can resynchronize their state through the query APIs if need be. To address similar issues in the list of IPMP groups itself, there is also a signature associated with the list of IPMP groups. When in.mpathd is started, it creates a signature for each existing IPMP group and for the list of IPMP groups. Thereafter, in.mpathd creates new signatures for any new IPMP groups that are created and destroys any signatures for any IPMP groups that are removed. If (for some reason) in.mpathd exits, all signatures are destroyed. To handle the "lost update" problem that could otherwise occur when the a group of the same name is deleted and then recreated, the signature itself is composed of a 16-bit generation number and a 48-bit sequence number. Upon signature creation, a the generation number is chosen at random and the sequence number is set to 1. Sysevent Definitions ==================== All IPMP sysevents are published on sysevent channel "com.sun:ipmp:events" (aka IPMP_EVENT_CHAN in ). IPMP sysevent definitions follow. Note that attributes that appear in more than one event are described after the first event they appear in. IPMP group state event (ESC_IPMP_GROUP_STATE) Description - This event indicates that an IP multipathing group has changed states. Currently defined states are "ok", "degraded", and "failed". See the group-state text below for state definitions. /* * Event type EC_IPMP/ESC_IPMP_GROUP_STATE event schema */ Event Class - EC_IPMP Event Sub-Class - ESC_IPMP_GROUP_STATE Event Vendor - com.sun Event Publisher - in.mpathd Attribute Name - IPMP_EVENT_VERSION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_GROUP_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_GROUP_SIGNATURE Attribute Type - SE_DATA_TYPE_UINT64 Attribute Value - Attribute Name - IPMP_GROUP_STATE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - where: is the version of this event format (currently 2). is the name of the IPMP group. is the current signature of the IPMP group. has one of the following values: IPMP_GROUP_OK: all group interfaces are functioning IPMP_GROUP_DEGRADED: some group interfaces are functioning IPMP_GROUP_FAILED: no group interfaces are functioning IPMP group change event (ESC_IPMP_GROUP_CHANGE) Description - This event indicates that an IPMP group has been created or removed. /* * Event type EC_IPMP/ESC_IPMP_GROUP_CHANGE event schema */ Event Class - EC_IPMP Event Sub-Class - ESC_IPMP_GROUP_CHANGE Event Vendor - com.sun Event Publisher - in.mpathd Attribute Name - IPMP_GROUP_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_EVENT_VERSION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_GROUPLIST_SIGNATURE Attribute Type - SE_DATA_TYPE_UINT64 Attribute Value - Attribute Name - IPMP_GROUP_SIGNATURE Attribute Type - SE_DATA_TYPE_UINT64 Attribute Value - Attribute Name - IPMP_GROUP_OPERATION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - where: has one of the following values: IPMP_GROUP_ADD: a new IPMP group has been created IPMP_GROUP_REMOVE: an existing IPMP group has been removed is the current signature on the list of IPMP groups. IPMP group member event (ESC_IPMP_GROUP_MEMBER_CHANGE) Description - This event indicates that an interface has been added to or removed from an IP multipathing group. /* * Event type EC_IPMP/ESC_IPMP_GROUP_MEMBER event schema */ Event Class - EC_IPMP Event Sub-Class - ESC_IPMP_GROUP_MEMBER_CHANGE Event Vendor - com.sun Event Publisher - in.mpathd Attribute Name - IPMP_GROUP_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_EVENT_VERSION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_GROUP_SIGNATURE Attribute Type - SE_DATA_TYPE_UINT64 Attribute Value - Attribute Name - IPMP_IF_OPERATION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_IF_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_IF_TYPE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_IF_STATE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - where: has one of the following values: IPMP_IF_ADD: a new interface has joined the group. IPMP_IF_REMOVE: an existing interface has left the group. is the name of the interface. is one of the following: IPMP_IF_STANDBY: the interface is a standby. IPMP_IF_NORMAL: the interface is not a standby. is one of the following: IPMP_IF_OK: the interface is functional. IPMP_IF_FAILED: the interface is in a failed state. IPMP_IF_OFFLINE: the interface is offline. IPMP_IF_UNKNOWN: in.mpathd has not yet been able to ascertain whether the interface is functional (e.g., not enough probes have been sent yet). IPMP network interface change event (ESC_IPMP_IF_CHANGE) Description - This event indicates that an interface in a IP multipathing group has changed state or type. /* * Event type EC_IPMP/ESC_IPMP_IF_CHANGE event schema */ Event Class - EC_IPMP Event Sub-Class - ESC_IPMP_IF_CHANGE Event Vendor - com.sun Event Publisher - in.mpathd Attribute Name - IPMP_GROUP_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_EVENT_VERSION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_GROUP_SIGNATURE Attribute Type - SE_DATA_TYPE_UINT64 Attribute Value - Attribute Name - IPMP_IF_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_IF_STATE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_IF_TYPE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - where: is the name of the IPMP group to which the interface belongs, or "" if the interface does not belong to any group (this assumes that the IPMP daemon is running with TRACK_INTERFACES_ONLY_WITH_GROUPS set to "no"). is the current signature for the group. For signature purposes, "" is considered a group and thus has its own signature. IPMP probe state event (ESC_IPMP_PROBE_STATE) Description- This event indicates that an IPMP probe has changed state. Currently defined states are "sent", "acked" and "lost". See the probe-state text below for state definitions. /* * Event type EC_IPMP/ESC_IPMP_PROBE_STATE event schema */ Event Class - EC_IPMP Event Sub-Class - ESC_IPMP_PROBE_STATE Event Vendor - com.sun Event Publisher - in.mpathd Attribute Name - IPMP_PROBE_ID Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_EVENT_VERSION Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_IF_NAME Attribute Type - SE_DATA_TYPE_STRING Attribute Value - Attribute Name - IPMP_PROBE_STATE Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_PROBE_START_TIME Attribute Type - SE_DATA_TYPE_TIME Attribute Value - Attribute Name - IPMP_PROBE_SENT_TIME Attribute Type - SE_DATA_TYPE_TIME Attribute Value - Attribute Name - IPMP_PROBE_ACKRECV_TIME Attribute Type - SE_DATA_TYPE_TIME Attribute Value - Attribute Name - IPMP_PROBE_ACKPROC_TIME Attribute Type - SE_DATA_TYPE_TIME Attribute Value - Attribute Name - IPMP_PROBE_TARGET Attribute Type - SE_DATA_TYPE_BYTES Attribute Value - Attribute Name - IPMP_PROBE_TARGET_RTTAVG Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - Attribute Name - IPMP_PROBE_TARGET_RTTDEV Attribute Type - SE_DATA_TYPE_UINT32 Attribute Value - where: is the sequence number included in the ICMP probe packet, used to associated the probe request with any response. Combined with and the IP address family, this uniquely identifies a probe. is one of the following: IPMP_PROBE_SENT: the probe has been sent IPMP_PROBE_ACKED: the probe has been acked IPMP_PROBE_LOST: the probe has been lost is when in.mpathd started executing the code to send the probe. is when in.mpathd asked the kernel to send the probe. is when the IP module received the ack, or zero if the ack has not been received. is when in.mpathd processed the ack, or zero if the ack has not been received. is the target IP address of the probe. is the current RTT average for probe-target-ip, or zero if unknown. is the current RTT deviation for probe-target-ip, or zero if unknown. Exported Interface Table ======================== Contracted Cons. Private com.sun:ipmp:events channel Contracted Cons. Private EC_ipmp sysevent class Contracted Cons. Private ESC_ipmp_group_state event Contracted Cons. Private ESC_ipmp_group_change event Contracted Cons. Private ESC_ipmp_if_change event Contracted Cons. Private ESC_ipmp_group_member_change event Contracted Cons. Private ESC_ipmp_probe_state event Contracted Cons. Private