From ahl@zion.sfbay.sun.com Wed May 7 18:03:05 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4813504025095 for ; Wed, 7 May 2008 18:03:05 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m48134aR005911; Wed, 7 May 2008 18:03:05 -0700 (PDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0I00703YX46X00@brm-avmta-1.central.sun.com>; Wed, 07 May 2008 19:03:04 -0600 (MDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0I002CEYX4T850@brm-avmta-1.central.sun.com>; Wed, 07 May 2008 19:03:04 -0600 (MDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m48133JX014261; Thu, 08 May 2008 01:03:03 +0000 (GMT) Received: (from ahl@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m48133fp014257; Wed, 07 May 2008 18:03:03 -0700 (PDT) Date: Wed, 07 May 2008 18:03:03 -0700 (PDT) From: Adam Leventhal Subject: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] To: PSARC-ext@sun.com Cc: ahl@eng.sun.com, brendan.gregg@sun.com Message-id: <200805080103.m48133fp014257@zion.sfbay.sun.com> Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 9985 Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: DTrace IP Provider 1.2. Name of Document Author/Supplier: Author: Brendan Gregg 1.3 Date of This Document: 07 May, 2008 4. Technical Description A. INTRODUCTION The following specification describes the ip provider for DTrace. It has been reviewed and approved by the DTrace community; the PSARC case is closed approved automatic to record the interface. The stability is Committed and the binding is Patch. Adam --8<-- This case adds a DTrace 'ip' provider with probes for send and receive for both IPv4 and IPv6 protocols. This is intended for use by customers for network observability and troubleshooting, and is the first component of a suite of planned providers for the network stack. These providers have previously been discussed on public mailing lists: see section D for references. B. DESCRIPTION This will introduce the following probes for the 'ip' provider: ip:::send ip:::receive The arguments to these probes are: args[0] pktinfo_t * packet info args[1] csinfo_t * connection state info args[2] ipinfo_t * generic IP info args[3] ifinfo_t * interface info args[4] ipv4info_t * IPv4 header args[5] ipv6info_t * IPv6 header The order and content has been chosen for consistency with other planned network providers, and to also leave room to accommodate future enhancements to the network stack. The arguments contain: /* * pktinfo is where packet ID info can be made available for deeper * analysis if packet IDs become supported by the kernel in the future. * The pkt_addr member is currently always NULL. */ typedef struct pktinfo { uintptr_t pkt_addr; } pktinfo_t; /* * csinfo is where connection state info can be made available if * connection IDs become supported by the kernel in the future. * The cs_addr member is currently always NULL. */ typedef struct csinfo { uintptr_t cs_addr; } csinfo_t; /* * ipinfo contains common IP info for both IPv4 and IPv6. */ typedef struct ipinfo { uint8_t ip_ver; /* IP version (4, 6) */ uint16_t ip_plength; /* payload length */ string ip_saddr; /* source address */ string ip_daddr; /* destination address */ } ipinfo_t; /* * ifinfo contains network interface info. */ typedef struct ifinfo { string if_name; /* interface name */ int8_t if_local; /* is delivered locally */ netstackid_t if_ipstack; /* ipstack ID */ uintptr_t if_addr; /* pointer to raw ill_t */ } ifinfo_t; /* * ipv4info is a translated version of the IPv4 header (with raw pointer). * These values are NULL if the packet is not IPv4. */ typedef struct ipv4info { uint8_t ipv4_ver; /* IP version (4) */ uint8_t ipv4_ihl; /* header length, bytes */ uint8_t ipv4_tos; /* type of service field */ uint16_t ipv4_length; /* length (header + payload) */ uint16_t ipv4_ident; /* identification */ uint8_t ipv4_flags; /* IP flags */ uint16_t ipv4_offset; /* fragment offset */ uint8_t ipv4_ttl; /* time to live */ uint8_t ipv4_protocol; /* next level protocol */ string ipv4_protostr; /* next level protocol, as a string */ uint16_t ipv4_checksum; /* header checksum */ ipaddr_t ipv4_src; /* source address */ ipaddr_t ipv4_dst; /* destination address */ string ipv4_saddr; /* source address, string */ string ipv4_daddr; /* destination address, string */ ipha_t *ipv4_hdr; /* pointer to raw header */ } ipv4info_t; /* * ipv6info is a translated version of the IPv6 header (with raw pointer). * These values are NULL if the packet is not IPv6. */ typedef struct ipv6info { uint8_t ipv6_ver; /* IP version (6) */ uint8_t ipv6_tclass; /* traffic class */ uint32_t ipv6_flow; /* flow label */ uint16_t ipv6_plen; /* payload length */ uint8_t ipv6_nexthdr; /* next header protocol */ string ipv6_nextstr; /* next header protocol, as a string */ uint8_t ipv6_hlim; /* hop limit */ in6_addr_t *ipv6_src; /* source address */ in6_addr_t *ipv6_dst; /* destination address */ string ipv6_saddr; /* source address, string */ string ipv6_daddr; /* destination address, string */ ip6_t *ipv6_hdr; /* pointer to raw header */ } ipv6info_t; C. EXAMPLES This DTrace one-liner counts received packets by host: # dtrace -n 'ip:::receive { @[args[2]->ip_saddr] = count(); }' dtrace: description 'ip:::receive ' matched 4 probes ^C 192.168.1.5 1 192.168.1.185 4 fe80::214:4fff:fe3b:76c8 9 127.0.0.1 14 192.168.1.109 28 This DTrace one-liner prints distribution plots of sent payload size by destination: # dtrace -n 'ip:::send { @[args[2]->ip_daddr] = quantize(args[2]->ip_plength); }' dtrace: description 'ip:::send ' matched 11 probes ^C 192.168.2.27 value ------------- Distribution ------------- count 8 | 0 16 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 7 32 |@@@@ 1 64 |@@@@ 1 128 | 0 192.168.1.109 value ------------- Distribution ------------- count 8 | 0 16 |@@@@@ 5 32 |@@@ 3 64 |@@@@@@@@@@@@@@@@@@@@@@@@@@ 24 128 |@ 1 256 |@ 1 512 |@@ 2 1024 |@ 1 2048 | 0 This DTrace script uses the ip provider to show packets as they pass in and out of tunnels: # ./ipio.d CPU DELTA(us) SOURCE DEST INT BYTES 1 598913 10.1.100.123 -> 192.168.10.75 ip.tun0 68 1 73 192.168.1.108 -> 192.168.5.1 nge0 140 1 18325 192.168.1.108 <- 192.168.5.1 nge0 140 1 69 10.1.100.123 <- 192.168.10.75 ip.tun0 68 0 102921 10.1.100.123 -> 192.168.10.75 ip.tun0 20 0 79 192.168.1.108 -> 192.168.5.1 nge0 92 This DTrace script provides a neat summary for both send and receive IP traffic: # ./ipproto.d Tracing... Hit Ctrl-C to end. ^C SADDR DADDR PROTO COUNT 192.168.1.108 192.168.155.32 UDP 1 192.168.1.108 192.168.17.55 UDP 1 192.168.1.108 192.168.228.54 UDP 1 192.168.1.108 192.168.1.5 UDP 1 192.168.1.108 192.168.2.27 ICMP 1 192.168.1.200 192.168.3.255 UDP 1 192.168.1.5 192.168.1.108 UDP 1 192.168.2.27 192.168.1.108 ICMP 1 fe80::214:4fff:fe3b:76c8 ff02::1 ICMPV6 1 fe80::2e0:81ff:fe5e:8308 fe80::214:4fff:fe3b:76c8 ICMPV6 1 fe80::2e0:81ff:fe5e:8308 ff02::1:2 UDP 1 192.168.1.185 192.168.1.255 UDP 2 192.168.1.211 192.168.1.255 UDP 3 192.168.1.109 192.168.1.108 TCP 428 192.168.1.108 192.168.1.109 TCP 789 D. REFERENCES The suite of planned providers is described on the following website, which includes demonstrations and source from previous prototypes: http://www.opensolaris.org/os/community/dtrace/NetworkProvider These providers have also been discussed in the past on both dtrace-discuss and networking-discuss: http://www.opensolaris.org/jive/thread.jspa?messageID=57666 http://www.opensolaris.org/jive/thread.jspa?messageID=128518😆 E. DOCUMENTATION A new chapter has been added to the current Solaris Dynamic Tracing Guide for this proposed provider: http://wikis.sun.com/display/DTrace/Documentation # DTrace Guide http://wikis.sun.com/display/DTrace/ip+Provider # ip Provider chapter F. STABILITY The DTrace internal stability table is described below: Element Name stability Data stability Dependency class Provider Evolving Evolving ISA Module Private Private Unknown Function Private Private Unknown Name Evolving Evolving ISA Arguments Evolving Evolving ISA 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: OS/Net 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From erik.nordmark@sun.com Sun May 11 09:34:32 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4BGYVTF012473 for ; Sun, 11 May 2008 09:34:31 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m4BGY84R013676; Mon, 12 May 2008 00:34:28 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0P00A03Q1DK900@nwk-avmta-1.sfbay.Sun.COM>; Sun, 11 May 2008 09:34:25 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.104.45]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0P00AVAQ1CVIE0@nwk-avmta-1.sfbay.Sun.COM>; Sun, 11 May 2008 09:34:25 -0700 (PDT) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4BGYNar856724 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 11 May 2008 09:34:24 -0700 (PDT) Date: Sun, 11 May 2008 09:34:20 -0700 From: Erik Nordmark Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <200805080103.m48133fp014257@zion.sfbay.sun.com> To: Adam Leventhal Cc: PSARC-ext@sun.com, ahl@eng.sun.com, brendan.gregg@sun.com Message-id: <4827200C.1020703@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 1115 Adam, Overall this looks good. One nit though. > /* > * ipinfo contains common IP info for both IPv4 and IPv6. > */ > typedef struct ipinfo { > uint8_t ip_ver; /* IP version (4, 6) */ > uint16_t ip_plength; /* payload length */ > string ip_saddr; /* source address */ > string ip_daddr; /* destination address */ > } ipinfo_t; I don't think "payload length" has a natural definition when trying to look at what IP actually carries because of the IPv6 extension headers. For instance, for a 100 byte IPv6 packet with a 8 byte hop-by-hop options header, a fragment header, and a TCP header, what is the payload length? The IPv6 header field for payload length is just 40 less than the size of the IPv6 packet. But the payload that IP delivers to TCP is less all the extension headers. And some might think that "payload" refers to the TCP payload. Thus I think it would be more clear to have the IP provider expose the IP packet length instead of payload length. Sorry for not catching this earlier. Erik From Darren.Reed@sun.com Mon May 12 06:51:09 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4CDp8gK004292 for ; Mon, 12 May 2008 06:51:09 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4CDp7gO000855 for <@sunmail2sca.sfbay.sun.com:PSARC-ext@sun.com>; Mon, 12 May 2008 07:51:08 -0600 (MDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0R00I0FD580E00@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 12 May 2008 06:51:08 -0700 (PDT) Received: from sineb-mail-2.sun.com ([192.18.19.7]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0R00BVSD56TUD0@nwk-avmta-1.sfbay.Sun.COM> for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 12 May 2008 06:51:07 -0700 (PDT) Received: from fe-apac-05.sun.com (fe-apac-05.sun.com [192.18.19.176] (may be forged)) by sineb-mail-2.sun.com (8.13.6+Sun/8.12.9) with ESMTP id m4CDpmO7023248 for ; Mon, 12 May 2008 13:51:48 +0000 (GMT) Received: from conversion-daemon.mail-apac.sun.com by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0K0R00001CZA7J00@mail-apac.sun.com> (original mail from Darren.Reed@Sun.COM) for PSARC-ext@sun.com (ORCPT PSARC-ext@sun.com); Mon, 12 May 2008 21:50:52 +0800 (SGT) Received: from [10.13.21.205] by mail-apac.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0K0R00MTMD4Q9U0W@mail-apac.sun.com>; Mon, 12 May 2008 21:50:51 +0800 (SGT) Date: Mon, 12 May 2008 21:50:08 +0800 From: Darren Reed Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <4827200C.1020703@sun.com> Sender: Darren.Reed@sun.com To: Erik Nordmark Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com, Brendan.Gregg@sun.com Message-id: <48284B10.4050500@Sun.COM> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) Status: RO Content-Length: 1423 Erik Nordmark wrote: > Adam, > > Overall this looks good. One nit though. > >> /* >> * ipinfo contains common IP info for both IPv4 and IPv6. >> */ >> typedef struct ipinfo { >> uint8_t ip_ver; /* IP version (4, 6) */ >> uint16_t ip_plength; /* payload length */ >> string ip_saddr; /* source address */ >> string ip_daddr; /* destination address */ >> } ipinfo_t; > > I don't think "payload length" has a natural definition when trying to > look at what IP actually carries because of the IPv6 extension headers. > For instance, for a 100 byte IPv6 packet with a 8 byte hop-by-hop > options header, a fragment header, and a TCP header, what is the > payload length? > The IPv6 header field for payload length is just 40 less than the size > of the IPv6 packet. But the payload that IP delivers to TCP is less > all the extension headers. And some might think that "payload" refers > to the TCP payload. > > Thus I think it would be more clear to have the IP provider expose the > IP packet length instead of payload length. > > Sorry for not catching this earlier. Hmmm, what position should be taken with respect to IPv6 jumbograms? We don't support them now, but should implementing them mean this interface (the dtrace provider) needs to be changed? Otherwise, I can't see why ip_plength shouldn't be 32bits. Darren From brendan@zion.sfbay.sun.com Mon May 12 11:44:32 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4CIiWsa015194 for ; Mon, 12 May 2008 11:44:32 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4CIiVZb000650; Mon, 12 May 2008 11:44:31 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0R00103QQ7TZ00@nwk-avmta-1.sfbay.Sun.COM>; Mon, 12 May 2008 11:44:31 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0R00G0UQQ648D0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 12 May 2008 11:44:30 -0700 (PDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4CIiPOq020650; Mon, 12 May 2008 18:44:25 +0000 (GMT) Received: (from brendan@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m4CIiOaF020649; Mon, 12 May 2008 11:44:24 -0700 (PDT) Date: Mon, 12 May 2008 11:44:24 -0700 From: Brendan Gregg - Sun Microsystems Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <4827200C.1020703@sun.com> To: Erik Nordmark Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <20080512184424.GA18735@zion.sfbay.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 2385 G'Day Erik, On Sun, May 11, 2008 at 09:34:20AM -0700, Erik Nordmark wrote: > Adam, > > Overall this looks good. One nit though. > > >/* > > * ipinfo contains common IP info for both IPv4 and IPv6. > > */ > >typedef struct ipinfo { > > uint8_t ip_ver; /* IP version (4, 6) */ > > uint16_t ip_plength; /* payload length */ > > string ip_saddr; /* source address */ > > string ip_daddr; /* destination address */ > >} ipinfo_t; > > I don't think "payload length" has a natural definition when trying to > look at what IP actually carries because of the IPv6 extension headers. > For instance, for a 100 byte IPv6 packet with a 8 byte hop-by-hop > options header, a fragment header, and a TCP header, what is the payload > length? > The IPv6 header field for payload length is just 40 less than the size > of the IPv6 packet. But the payload that IP delivers to TCP is less all > the extension headers. And some might think that "payload" refers to the > TCP payload. > Thus I think it would be more clear to have the IP provider expose the > IP packet length instead of payload length. ipinfo_t is to provide basic IP details across all protocols: TCP, UDP, ICMP, etc - which isn't obvious from this case alone. We can't assume that a complete IP header will be available at these times - it may be as an implementation artifact but this isn't stable. What we can provide is an "IP payload length at time of tracing", and document that this is provided for convenience. The IP provider ipv4info_t and ipv6info_t structs are to provide the best understanding of what IP really does. The ipv4info_t and ipv6info_t structs are based on the protocol fields; but can include additional members if they are stable and useful. This is the best location for exposing the actual IP packet length, as these structs are only used by the IP provider when the headers should be available. IPv4 does have this member (ipv4_length), but IPv6 does not (it would be calculated using ipv6_plen + IPV6_HDR_LEN). Exporting the packet length easily from the IP provider would indeed be useful and has a clear definition that should reduce confusion. We can add this to the ipv6info_t as ipv6_length, with the ipv4 companion as ipv4_length. How does that sound? cheers, Brendan -- Brendan [CA, USA] From brendan@zion.sfbay.sun.com Mon May 12 12:32:55 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4CJWsMh016290 for ; Mon, 12 May 2008 12:32:54 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4CJWdUj001271; Mon, 12 May 2008 20:32:50 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0R0070LSYN2400@nwk-avmta-1.sfbay.Sun.COM>; Mon, 12 May 2008 12:32:47 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0R004FHSYM9OB0@nwk-avmta-1.sfbay.Sun.COM>; Mon, 12 May 2008 12:32:46 -0700 (PDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4CJWe59021725; Mon, 12 May 2008 19:32:40 +0000 (GMT) Received: (from brendan@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m4CJWeRU021724; Mon, 12 May 2008 12:32:40 -0700 (PDT) Date: Mon, 12 May 2008 12:32:40 -0700 From: Brendan Gregg - Sun Microsystems Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <48284B10.4050500@Sun.COM> To: Darren Reed Cc: Erik Nordmark , Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <20080512193240.GA21482@zion.sfbay.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> <48284B10.4050500@Sun.COM> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1787 G'Day Darren, On Mon, May 12, 2008 at 09:50:08PM +0800, Darren Reed wrote: > Erik Nordmark wrote: > >Adam, > > > >Overall this looks good. One nit though. > > > >>/* > >> * ipinfo contains common IP info for both IPv4 and IPv6. > >> */ > >>typedef struct ipinfo { > >> uint8_t ip_ver; /* IP version (4, 6) */ > >> uint16_t ip_plength; /* payload length */ > >> string ip_saddr; /* source address */ > >> string ip_daddr; /* destination address */ > >>} ipinfo_t; > > > >I don't think "payload length" has a natural definition when trying to > >look at what IP actually carries because of the IPv6 extension headers. > >For instance, for a 100 byte IPv6 packet with a 8 byte hop-by-hop > >options header, a fragment header, and a TCP header, what is the > >payload length? > >The IPv6 header field for payload length is just 40 less than the size > >of the IPv6 packet. But the payload that IP delivers to TCP is less > >all the extension headers. And some might think that "payload" refers > >to the TCP payload. > > > >Thus I think it would be more clear to have the IP provider expose the > >IP packet length instead of payload length. > > > >Sorry for not catching this earlier. > > Hmmm, what position should be taken with respect to IPv6 jumbograms? > > We don't support them now, but should implementing them mean this > interface (the dtrace provider) needs to be changed? > > Otherwise, I can't see why ip_plength shouldn't be 32bits. Good point, the uint16_t ip_plength came from IPv4, and as a translator (/usr/lib/dtrace/ip.d) derived member it can be whatever we like. Sounds like 32 bits is a safer choice - I'll make the tiny change. cheers, Brendan -- Brendan [CA, USA] From erik.nordmark@sun.com Thu May 15 10:03:08 2008 Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FH388L018900 for ; Thu, 15 May 2008 10:03:08 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail2sca.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4FH35sB003865; Thu, 15 May 2008 10:03:07 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00D216150300@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 10:03:05 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.228.50]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X008XO614IS80@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 10:03:04 -0700 (PDT) Received: from [10.7.251.248] (punchin-nordmark.SFBay.Sun.COM [10.7.251.248]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FH30Zx149156 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 15 May 2008 10:03:01 -0700 (PDT) Date: Thu, 15 May 2008 10:03:00 -0700 From: Erik Nordmark Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <20080512184424.GA18735@zion.sfbay.sun.com> To: Brendan Gregg - Sun Microsystems Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <482C6CC4.7010506@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> <20080512184424.GA18735@zion.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 1611 Brendan Gregg - Sun Microsystems wrote: > ipinfo_t is to provide basic IP details across all protocols: TCP, UDP, > ICMP, etc - which isn't obvious from this case alone. We can't assume > that a complete IP header will be available at these times - it may be as > an implementation artifact but this isn't stable. What we can provide is > an "IP payload length at time of tracing", and document that this is > provided for convenience. The IP provider ipv4info_t and ipv6info_t > structs are to provide the best understanding of what IP really does. I think you are missing the point. It should be "IP packet length" instead of "IP payload length", since the latter term isn't even well-defined. For the implementation this means that the number is a bit larger: - no need to subtract the IPv4 header length from ipha_length - for IPv6 add in the constant 40 byte size of the IPv6 header > Exporting the packet length easily from the IP provider would indeed be > useful and has a clear definition that should reduce confusion. We can > add this to the ipv6info_t as ipv6_length, with the ipv4 companion as > ipv4_length. How does that sound? I think you should replace the uint16_t ip_plength; /* payload length */ with uint16_t ip_length; /* packet length */ The ip version specific ones should best reflect the actual IP headers, which are different for IPv4 and IPv6. Thus having an ipv4_length (which is the packet length - the ipha_length in the IPv4 header) and ipv6_plength (which is the ipv6 payload length field in the IPv6 header.) Erik From brendan@zion.sfbay.sun.com Thu May 15 10:25:45 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4FHPj2A019499 for ; Thu, 15 May 2008 10:25:45 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4FHPiD9028223; Thu, 15 May 2008 10:25:44 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00D0172VSM00@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 10:25:43 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X008FG72UISA0@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 10:25:42 -0700 (PDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4FHPasi029742; Thu, 15 May 2008 17:25:36 +0000 (GMT) Received: (from brendan@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m4FHPaRC029741; Thu, 15 May 2008 10:25:36 -0700 (PDT) Date: Thu, 15 May 2008 10:25:35 -0700 From: Brendan Gregg - Sun Microsystems Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <482C6CC4.7010506@sun.com> To: Erik Nordmark Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <20080515172535.GA29679@zion.sfbay.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> <20080512184424.GA18735@zion.sfbay.sun.com> <482C6CC4.7010506@sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 936 G'Day Erik, On Thu, May 15, 2008 at 10:03:00AM -0700, Erik Nordmark wrote: > Brendan Gregg - Sun Microsystems wrote: > > >ipinfo_t is to provide basic IP details across all protocols: TCP, UDP, > >ICMP, etc - which isn't obvious from this case alone. We can't assume > >that a complete IP header will be available at these times - it may be as > >an implementation artifact but this isn't stable. What we can provide is > >an "IP payload length at time of tracing", and document that this is > >provided for convenience. The IP provider ipv4info_t and ipv6info_t > >structs are to provide the best understanding of what IP really does. > > I think you are missing the point. It should be "IP packet length" > instead of "IP payload length", since the latter term isn't even > well-defined. Ok, so you are suggesting we commit to providing "IP packet length" in the TCP layer? UDP? SCTP? etc? Brendan -- Brendan [CA, USA] From erik.nordmark@sun.com Thu May 15 17:21:15 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4G0LEQq002686 for ; Thu, 15 May 2008 17:21:15 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m4G0L6Vd022244; Fri, 16 May 2008 01:21:12 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00725QBBKR00@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 17:21:11 -0700 (PDT) Received: from jurassic.eng.sun.com ([129.146.17.57]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X004GOQBAHS60@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 17:21:10 -0700 (PDT) Received: from [129.146.229.7] (dhcp-umpk17-229-7.SFBay.Sun.COM [129.146.229.7]) by jurassic.eng.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4G0L96L160384 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 15 May 2008 17:21:10 -0700 (PDT) Date: Thu, 15 May 2008 17:21:04 -0700 From: Erik Nordmark Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <20080515172535.GA29679@zion.sfbay.sun.com> To: Brendan Gregg - Sun Microsystems Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <482CD370.6050706@sun.com> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> <20080512184424.GA18735@zion.sfbay.sun.com> <482C6CC4.7010506@sun.com> <20080515172535.GA29679@zion.sfbay.sun.com> User-Agent: Thunderbird 2.0.0.4 (X11/20070723) Status: RO Content-Length: 519 Brendan Gregg - Sun Microsystems wrote: > Ok, so you are suggesting we commit to providing "IP packet length" in the > TCP layer? UDP? SCTP? etc? For the IP layer. More specifically, for ipinfo_t change from: uint16_t ip_plength; /* payload length */ to uint32_t ip_length; /* IP packet length */ (the 16->32 change was what Darren suggested.) When doing providers for TCP, UDP, SCTP it makes sense that they define their own notion of length. But that isn't part of this case. Erik From brendan@zion.sfbay.sun.com Thu May 15 18:23:24 2008 Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m4G1NOnG008462 for ; Thu, 15 May 2008 18:23:24 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail3mpk.sfbay.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m4G1NNs6029651; Thu, 15 May 2008 18:23:23 -0700 (PDT) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K0X00A09T6YG600@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 18:23:22 -0700 (PDT) Received: from zion.sfbay.sun.com ([129.146.17.75]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K0X004UPT6XHSB0@nwk-avmta-2.sfbay.sun.com>; Thu, 15 May 2008 18:23:21 -0700 (PDT) Received: from zion.sfbay.sun.com (localhost [127.0.0.1]) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2) with ESMTP id m4G1NGYh010373; Fri, 16 May 2008 01:23:16 +0000 (GMT) Received: (from brendan@localhost) by zion.sfbay.sun.com (8.14.2+Sun/8.14.2/Submit) id m4G1NFf3010372; Thu, 15 May 2008 18:23:15 -0700 (PDT) Date: Thu, 15 May 2008 18:23:15 -0700 From: Brendan Gregg - Sun Microsystems Subject: Re: DTrace IP Provider [PSARC/2008/302 FastTrack timeout 05/07/2008] In-reply-to: <482CD370.6050706@sun.com> To: Erik Nordmark Cc: Adam Leventhal , PSARC-ext@sun.com, ahl@eng.sun.com Message-id: <20080516012315.GA9625@zion.sfbay.sun.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT Content-disposition: inline X-PMX-Version: 5.4.1.325704 References: <200805080103.m48133fp014257@zion.sfbay.sun.com> <4827200C.1020703@sun.com> <20080512184424.GA18735@zion.sfbay.sun.com> <482C6CC4.7010506@sun.com> <20080515172535.GA29679@zion.sfbay.sun.com> <482CD370.6050706@sun.com> User-Agent: Mutt/1.4.2.1i Status: RO Content-Length: 1108 On Thu, May 15, 2008 at 05:21:04PM -0700, Erik Nordmark wrote: > Brendan Gregg - Sun Microsystems wrote: > > >Ok, so you are suggesting we commit to providing "IP packet length" in the > >TCP layer? UDP? SCTP? etc? > > For the IP layer. More specifically, for ipinfo_t change from: > uint16_t ip_plength; /* payload length */ > to > uint32_t ip_length; /* IP packet length */ > > (the 16->32 change was what Darren suggested.) > > When doing providers for TCP, UDP, SCTP it makes sense that they define > their own notion of length. But that isn't part of this case. After speaking to Erik on the phone, we have agreed to keep ip_plength as the payload length and to make sure the documentation describes it as the payload length at time of tracing, and that it is an approximation. IP packet length can be fetched in the IP provider using args[4]->ipv4_length and args[5]->ipv6_plen + IPV6_HDR_LEN; and if this becomes a nuisance for customers, we can later add an additional members to the IP provider in args[4] and args[5] to make it easier. Brendan -- Brendan [CA, USA]