# ident "%Z%%M% %I% %E% SMI" System Administration Commands flowadm(1M) NAME flowadm - administer bandwidth resource control and priority for protocols, services, Containers and Virtual machines SYNOPSIS flowadm show-flow [-pP] [-S] [-s [-i interval]] [-l link] [-o field,...] [flow] flowadm add-flow [-t] [-R root-dir] -l link -a attr=value[,...] -p prop=value[,...] flow flowadm remove-flow [-t] [-R root-dir] {-l link | flow} flowadm set-flowprop [-t] [-R root-dir] -p prop=value[,...] flow flowadm reset-flowprop [-t] [-R root-dir] [-p prop,...] flow flowadm show-flowprop [-cP] [-l link] [-o field,...] [-p prop,...] [flow] flowadm show-usage [-d|-p] [-s time] [-e time] -f [-l link] [flow] DESCRIPTION The flowadm command is used to create, modify, remove and show networking bandwidth and associated resources for a type of traffic on a particular link. CONCEPTS The flowadm command allows users to manage networking bandwidth resource for a transport, service, or a subnet. The service is specified as a combination of transport and local port. The subnet is specified by its IP address and subnet mask. The command can be used on any type of data link, including physical links, virtual NICs, and link aggregations. A flow is defined as a set of attributes based on Layer3 and 4 headers which can be used to identify a protocol, service, or a virtual machine. When a flow is identified based on flow attributes, separate kernel resources including layer 2, 3, and 4 queues, their processing threads, etc are uniquely created for it such that other traffic has minimal or zero impact on it. Inbound and outbound packet are matched to flows in a very fast and scalable way, so that limits can be enforced with minimal performance impact. The flowadm command can be used to identify a flow without imposing any bandwidth resource control. This would result in the traffic type getting its own resources and queues so that it is isolated from rest of the networking traffic for more obserbale and deterministic behavior. SUBCOMMANDS The following subcommands are supported: flowadm show-flow [-pP] [-S] [-s [-i interval]] [-o field,...] [-l link] [flow] Show flow configuration information (the default) or statistics, either for all flows, all flows on a link or for the specified name. -o field, --output=field A case-insensitive, comma-separated list of output fields to display. The field name must be one of the fields listed below, or a special value all, to display all fields. For each flow found, the following fields can be displayed: flow The name of the flow. link The name of the link the flow is on. ipaddr IP address of the flow. This can be either local or remote depending on how the flow was defined. transport The name of the layer for protocol to be used. port Local port of service for flow. dsfield Differentiated services value for flow and mask used with DSFIELD value to state the bits of interest in the differentiated services field of the IP header -p, --parseable Display using a stable machine-parseable format. -P, --persistent Display persistent flow property information. -S, --continuous Continuously display network utilization by flow in a manner similar to the way that prstat(1m) displays cpu utilization by process. -s, --statistics Displays flow statistics. -i interval, --interval=interval Used with the -s option to specify an interval, in seconds, at which statistics should be displayed. If this option is not specified, statistics will only be displayed once. -l link, --link=link | flow Display information for all flows on the named link or information for the named flow. flowadm add-flow [-t] [-R root-dir] -l link -a attr=value[,...] -p prop=value[,...] flow Adds a flow to the system identified by its flow attributes and properties. As part of identifying a particular flow, its bandwidth resource can be limited and its relative priority to other traffic can be specified. If no bandwidth limit or priority is specified, the traffic still gets its unique layer 2, 3, and 4 queues and processing threads including NIC hardware resources (when supported) so that the selected traffic can be separated from others and can flow with minimal impact from other traffic. -t, --temporary The changes are temporary and will not persist across reboots. Persistence is the dedault. -R root-dir, --root-dir=root-dir Specifies an alternate root directory where flowadm should apply persistent creation. -l , --link= Specify the link to which the flow will be added. -a attr=value,..., --attr attr=value,... A comma-separated list of attributes to be set to the specified values. -p prop=value,..., --prop prop=value,... A comma-separated list of properties to be set to the specified values. flowadm remove-flow [-t] [-R root-dir] {-l link | flow} Remove an existing flows identified by its link or name. -t, --temporary The changes are temporary and will not persist across reboots. -R root-dir, --root-dir=root-dir Specifies an alternate root directory where flowadm should apply persistent removal. -l , --link= | flow If a link is specified, remove all flows from that link. If a single flow is specified, remove only that flow. flowadm set-flowprop [-t] [-R root-dir] -p prop=value[,...] flow Set values of one or more properties on the flow specifed by name. The complete list of properties can be retrieved using show-flow -t, --temporary The changes are temporary and will not persist across reboots. -R root-dir, --root-dir=root-dir Specifies an alternate root directory where flowadm should apply persistent creation. -p prop=value,..., --prop prop=value,... A comma-separated list of properties to be set to the specified values. flowadm reset-flowprop [-t] [-R root-dir] [-p prop, ...] flow Resets one or more properties to their default values on the specified flow. If no properties are specified, all pro- perties are reset. See show-flowprop for a description of properties which includes the default values -t, --temporary Specifies that the resets are temporary. Temporary resets last until the next reboot. -R root-dir, --root-dir=root-dir Specifies an alternate root directory where flowadm should apply persistent creation. -p prop, ..., --prop=prop, ... A comma-separated list of properties to reset. flowadm show-flowprop [-cP] [-l link] [-p prop, ...] [flow] Show the current or persistent values of one or more properties, either for all flows, flows on a specified link, or for the specified flow. By default, current values are shown. If no properties are specified, all available flow properties are displayed. For each property, the following fields are displayed: FLOW The name of the flow. PROPERTY The name of the property. VALUE The current (or persistent) property value. The value is shown as "--", if it is not set, and "?", if the value is unknown. Per- sistent values that are not set or have been reset will be shown as "--" and will use the system DEFAULT value (if any). DEFAULT The default value of the property. If the property has no default value, "--" is shown. POSSIBLE A comma-separated list of the values the property may have. If the values span a numeric range, min - max may be shown as shorthand. If the possible values are unknown or unbounded, "--" is shown. Flow properties are documented in the FLOW PROPERTIES section. -c, --parseable Display using a stable machine-parseable format. -P, --persistent Display persistent flow property information -p prop, ..., --prop=prop, ... A comma-separated list of properties to show. flowadm show-usage [-d|-p] [-s time] [-e time]] -f filename [-l link] [flow] Show the historical network flow usage from a stored extended accounting file. Configuration and enabling of net accounting through acctadm(1m) is required. The default output will be the summary of flow usage for the entire period of time in which extended accounting was enabled. -d Display the dates for which there is logging information. The date is in the format MM/DD/YYYY. -p When specified with -s and/or -e, outputs flow usage data in a form suitable for input to gnuplot. -s time, -e time Start and stop times for data display. Time is in the format MM/DD/YYYY,hh:mm:ss. -f filename Read extended accounting records of network flow usage from the filename. -l link If specified, only print the network flow usage from the named link. Otherwise, display network usage from all links. flow If specified, only print the network flow usage from the named flow. Otherwise, display network usage from all flows. FLOW ATTRIBUTES The flow attribute that identify a flow is a comma separated list of one or more keyword,value pair from below local_ip[/prefix_len] Identifies a network flow by the local IP address. must be a IPv4 address in dotted-decimal notation or an IPv6 address in colon-separated notation. prefix_len is optional. If specified, it describes the netmask for a subnet address, following the same notation convention of ifconfig(1M) and route(1M) addresses. If unspecified, given IP address will be considered as host address for which default prefix length for a IPv4 address is /32 and for IPv6 it is /128. remote_ip[/prefix_len] Identifies a network flow by the remote IP address. The syntax is the same as local_ip attribute's transport={tcp|udp|sctp|icmp|icmpv6} Identifies a layer 4 protocol to be used. Is typically used | in combination with local_port or remote_port to identify the | local or remote service which needs special attention. local_port Identifies a service specified by the local port. + remote_port + Identifies a traffic specified by the remote port. + dsfield[:dsfield_mask] Identifies the 8-bit differentiated services field (as defined in RFC 2474). The optional dsfield_mask is used to state the bits of interest in the differentiated services field when comparing with the dsfield value. A '0' in a bit position indicates that the bit value needs to be ignored and a '1' indicates otherwise. The mask can range from 0x01 to 0xff. If dsfield_mask is not specified, the default mask 0xff is used. Both dsfield value and mask must be in hexadecimal. | The following six types of combinations of attributes are supported: local_ip[/prefixlen]=address remote_ip[/prefixlen]=address transport={tcp|udp|sctp|icmp|icmpv6} transport={tcp|udp|sctp},local_port=port + transport={tcp|udp|sctp},remote_port=port dsfield=val[:dsfield_mask] On a given link, the combinations above are mutually exclusive. An attempt to create flows of different types will fail. RESTRICTIONS Individual flow restrictions Restrictions on individual flows do not require knowledge of other flows that have been added to the link. Duplicate attributes An attribute may only be listed once for each flow. Not Valid: #flowadm add-flow -l vnic1 -a local_port=80,local_port=8080 httpflow | transport and local_port or transport and remote_port | TCP, UDP, or SCTP flows can be specified with a local | port or with a remote port. ICMP or ICMPv6 flow which | specifies a port will not be allowed. | If a either local_port or remote_port is specifed, the | transport must be either TCP, UDP or SCTP. Valid: #flowadm add-flow -l e1000g0 -a transport=udp udpflow #flowadm add-flow -l e1000g0 -a transport=tcp,local_port=80 udp80flow Not valid: | #flowadm add-flow -l e1000g0 -a remote_port=25 flow25 | #flowadm add-flow -l e1000g0 -a transport=icmpv6,remote_port=16 \ flow16 Flow restrictions per zone Flow names Within a zone, no 2 flows may have the same name. Conversely, after adding the flow with the link specified, the link will not be required for display, modifcation or deletion of the flow. FLOW PROPERTIES The following flow properties are supported. Note that the ability to set a given property to a given value depends on the driver and hardware. maxbw Sets the full duplex bandwidth for the flow. The bandwidth is specified as an integer with one of the scale suffixes(K, M, or G for Kbps, Mbps, and Gbps). If no units are specified, the input value will be read as Mbps. Default is no bandwidth limit. priority Sets the relative priority for the flow. The value may be given as one of the tokens high, medium, or low. Default is medium. EXAMPLES Example 1 Create a policy around mission critical port 443 traffic which is https service. To create a policy around inbound https traffic on a https server so that https gets it dedicated NIC hardware and kernel TCP/IP resources. The name specified is https-1 which is used to later modify of delete the policy. # flowadm add-flow -l bge0 -a transport=TCP,local_port=443 https-1 # flowadm show-flow -l bge0 | FLOW LINK IP ADDR PROTO PORT RPORT DSFLD | https1 bge0 -- tcp 443 -- -- Example 2 Modify an existing policy to add bandwidth resource control To modify https-1 policy to add bandwidth control and give it a high priority. # flowadm set-flowprop -p maxbw=500M,priority=high https-1 # flowadm show-flow https-1 | FLOW LINK IP ADDR PROTO PORT RPORT DSFLD | https1 bge0 -- tcp 443 -- -- # flowadm show-flowprop https-1 FLOW PROPERTY VALUE DEFAULT POSSIBLE https-1 maxbw 500 -- -- https-1 priority HIGH -- LOW,NORMAL,HIGH Example 3 Limit the bandwidth usage of UDP protocol To create a policy for UDP protocol so that it can not consume more than 100Mbps of available bandwidth. The flow is named limit-udp-1. # flowadm add-flow -l bge0 -a transport=UDP -p maxbw=100M, priority=low limit-udp-1 Example 4 Show flow usage Flow usage statistics can be stored using the extended accounting facility acctadm(1m). # acctadm -e extended -f /var/log/net.log net # acctadm net Network accounting: active Network accounting file: /var/log/net.log Tracked Network resources: extended Untracked Network resources: none The historical data which was saved can be retrieved in summary form using the show-usage option of flowadm. Example 5 Set a policy for EF PHB (DSCP value of 101110 from RFC 2598) with bandwidth of 500 Mbps and high priority. The dsfield value for this flow will be 0x2e (101110) with the dsfield_mask being 0xfc (since we want to ignore the 2 least significant bits). # flowadm add-flow -l bge0 -a dsfield=0x2e:0xfc -p maxbw=500M,priority=high efphb-flow Display summary information # flowadm show-usage -f /var/log/net.log FLOW DURATION IPACKETS RBYTES OPACKETS OBYTES BANDWIDTH flowtcp 100 1031 546908 0 0 43.76 Kbps flowudp 0 0 0 0 0 0.00 Mbps Display dates for which logging information is available # flowadm show-usage -d -f /var/log/net.log 02/19/2008 Display logging information for 'flowtcp' starting at 02/19/2008,10:38:46 and ending at 02/19/2008,10:40:06 # flowadm show-usage -s 02/19/2008,10:39:06 -e 02/19/2008,10:40:06 -f /var/log/net.log flowtcp FLOW TIME IPACKETS RBYTES OPACKETS OBYTES BANDWIDTH flowtcp 10:39:06 1 1546 4 6539 3.23 Kbps flowtcp 10:39:26 2 3586 5 9922 5.40 Kbps flowtcp 10:39:46 1 240 1 216 182.40 bps flowtcp 10:40:06 0 0 0 0 0.00 bps Output the same information as above as a plotfile # flowadm show-usage -s 02/19/2008,12:28:04 -e 02/19/2008,12:32:04 -p -f /var/log/net.log flowtcp # Time tcp-flow 10:39:06 3.23 10:39:26 5.40 10:39:46 0.18 10:40:06 0.00 EXIT STATUS The following exit values are returned: 0 All actions were performed successfully. >0 An error occurred. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| |_____________________________|_____________________________| | Interface Stability | Evolving | |_____________________________|_____________________________| SEE ALSO dladm(1M), attributes(5), dlpi(7P), ifconfig(1M), route(1M), acctadm(1m)