From sacadmin Mon May 24 09:36:49 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Mon, 24 May 2004 12:36:24 -0400 From: James Carlson To: psarc@sac.sfbay.sun.com cc: Phil.Kirk@Sun.COM Subject: 2004/426 Checksum Support for Snoop Content-Length: 2262 I'm sponsoring this fast-track request for Philip Kirk. The timer is set to 06/01/2004 due to the holiday on Monday. Problem ------- Currently snoop does not verify checksums for IP, ICMP, UDP or TCP packets. When debugging certain network problems this ability would prove very helpful. The output of snoop is classified as "Unstable." This change request is for a Minor release binding. Solution and example output --------------------------- This change will enable verification of checksums for IP, ICMP, UDP and TCP packets. In the case of UDP and TCP checksum verification will only be done for non-fragmented packets. The RFE associated with this case is: "1196134 snoop should verify ip, tcp, and udp checksums" In summary form the keyword ERR will preceed the summary line: 32 0.00006 a -> b ERR TELNET C port=49277 In verbose summary mode the keyword ERR will be printed before the summary line of the layer that has a problem. For the case where multiple layers have a problem ERR will be printed for all layers that have a problem. Examples of a single layer having a problem and multiple layers having problems are below: Single layer: 32 0.00000 a -> b ETHER Type=0800 (IP), size = 54 bytes 32 0.00000 a -> b IP D=129.156.195.7 S=129.156.195.73 LEN=40, ID=32485, TOS=0x0, TTL=64 32 0.00000 a -> b ERR TCP D=23 S=49277 Ack=1265953414 Seq=3907929187 Len=0 Win=49640 32 0.00000 a -> b TELNET C port=49277 Multiple layers: 5 0.00000 a -> b ETHER Type=0800 (IP), size = 66 bytes 5 0.00000 a -> b ERR IP D=144.86.195.7 S=129.156.195.73 LEN=52, ID=32471, TOS=0x0, TTL=64 5 0.00000 a -> b ERR TCP D=23 S=49277 Syn Seq=3907929068 Len=0 Win=49640 Options= 5 0.00000 a -> b TELNET C port=49277 In verbose mode the string (Correct) will be printed in the case of the checksum been correct. If the checksum is incorrect the string (Wrong should be ) will be printed. In the case where it has not been possible to calculate the checksum then the string (Not verified) will be printed. Examples of all these forms are below: IP: Header checksum = 3261 (Correct) TCP: Checksum = 0x89a4 (Wrong should be 0x4080) TCP: Checksum = 0x04a9 (Not verified) From sacadmin Mon May 24 09:44:29 2004 Date: Mon, 24 May 2004 17:40:40 +0100 From: Jeremy Harris User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.3) Gecko/20030314 X-Accept-Language: en-us, en MIME-Version: 1.0 To: James Carlson CC: psarc@sac.sfbay.sun.com, Phil.Kirk@sun.com Subject: Re: 2004/426 Checksum Support for Snoop Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Length: 391 James Carlson wrote: > I'm sponsoring this fast-track request for Philip Kirk. The timer is > set to 06/01/2004 due to the holiday on Monday. > The RFE associated with this case is: > > "1196134 snoop should verify ip, tcp, and udp checksums" What about link-layer checksums, for various link types? Other transports (SCTP)? Higher-layer protocol checksums (are there any)? - Jeremy From sacadmin Mon May 24 10:09:40 2004 Date: Mon, 24 May 2004 18:09:16 +0100 (BST) From: Philip Kirk - Solaris Sustaining Subject: Re: 2004/426 Checksum Support for Snoop To: james.d.carlson@Sun.COM, jeremy.harris@Sun.COM Cc: psarc@sac.sfbay.sun.com, Phil.Kirk@Sun.COM MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: 2XqTjg9OEQpyrkybqjj6Mg== Content-Length: 512 >What about link-layer checksums, for various link types? >Other transports (SCTP)? >Higher-layer protocol checksums (are there any)? Right now I've implemented this for the protocols described in rfe 1196134 (IP,UDP and TCP). I did consider other link types, SCTP, Higher-layer protocol checksums but for now don't plan on implementing them. The protocols in rfe 1196134 seem like a good starting point for improving snoop and if people do want other checksums verified then new rfe's can be raised. Phil From sacadmin Mon May 24 11:00:00 2004 To: James Carlson cc: psarc@sac.sfbay.sun.com, Phil.Kirk@sun.com Subject: Re: 2004/426 Checksum Support for Snoop Date: Mon, 24 May 2004 19:59:36 +0200 From: Casper Dik Content-Length: 347 >Currently snoop does not verify checksums for >IP, ICMP, UDP or TCP packets. When debugging >certain network problems this ability would >prove very helpful. Isn't it the case that for hardware which supports hw checksumming the loopback packets do not show the correct checksum because the checksum won't be filled in by the software? Casper From sacadmin Mon May 24 11:27:09 2004 Date: Mon, 24 May 2004 11:23:22 -0700 From: Frank Dimambro Subject: Re: 2004/426 Checksum Support for Snoop To: Casper Dik Cc: James Carlson , psarc@sac.sfbay.sun.com, Phil.Kirk@sun.com MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 Content-Length: 420 Hi Casper That's correct. Frank Casper Dik wrote: >>Currently snoop does not verify checksums for >>IP, ICMP, UDP or TCP packets. When debugging >>certain network problems this ability would >>prove very helpful. > > > Isn't it the case that for hardware which supports hw checksumming > the loopback packets do not show the correct checksum because the > checksum won't be filled in by the software? > > Casper From sacadmin Mon May 24 11:40:26 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Mon, 24 May 2004 14:40:01 -0400 From: James Carlson To: Frank.Dimambro@Sun.COM Cc: Casper Dik , psarc@sac.sfbay.sun.com, Phil.Kirk@Sun.COM Subject: Re: 2004/426 Checksum Support for Snoop Content-Length: 1203 Frank Dimambro writes: > Casper Dik wrote: > > Isn't it the case that for hardware which supports hw checksumming > > the loopback packets do not show the correct checksum because the > > checksum won't be filled in by the software? > > That's correct. The problem there, though, is that the raw DLPI user is seeing data formatted in a way that it didn't ask for and doesn't know about. There's no way for snoop or any other listener to know that the checksum field is "sometimes" unreliable. I'd argue that since it's not implemented in a per-stream manner, or at least in a manner that's invisible to unknowing streams (i.e., doesn't switch off when there are checksum-acceleration-ignorant listeners or simply generate the checksum [slowly] when snoop is present), this is a bug in either the checksum acceleration architecture or the existing implementations. Note that this problem breaks ethereal and tcpdump on Solaris as well, since they already validate checksums. -- James Carlson, IP Systems Group Sun Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 From sacadmin Mon May 24 14:40:57 2004 From: Bill Sommerfeld To: James Carlson cc: psarc@sac.sfbay.sun.com, Phil.Kirk@Sun.COM Subject: Re: 2004/426 Checksum Support for Snoop Date: Mon, 24 May 2004 17:40:34 -0400 Content-Length: 754 > In verbose summary mode the keyword ERR will be printed > before the summary line of the layer that has a problem. > For the case where multiple layers have a problem > ERR will be printed for all layers that have a problem. I'd suggest something more indicative of the problem: "BAD SUM:", perhaps? (trivia note: I implemented this feature in tcpdump a long time ago, though I'm not sure if it was my implementation which got sucked back into the main distribution or someone else's..) There are actually three states: good sum, bad sum, and "can't tell"; the latter occurs for fragmented packets or if you're using -s to only read a prefix of each packet; seems like that would also apply in the event of checksum offload on TX.. - Bill From sacadmin Tue May 25 03:09:52 2004 Date: Tue, 25 May 2004 11:09:26 +0100 (BST) From: Philip Kirk - Solaris Sustaining Subject: Re: 2004/426 Checksum Support for Snoop To: James.D.Carlson@Sun.COM, sommerfeld@east.sun.com Cc: psarc@sac.sfbay.sun.com, Phil.Kirk@Sun.COM MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: iX58Myw7AqrbBXwbzzWe2w== Content-Length: 631 >I'd suggest something more indicative of the problem: "BAD SUM:", >perhaps? I did discuss this with James. "ERR" is proposed as it's intended to be generic. Any header error triggers this output including checksum errors and future cases might extend the meaning of "ERR". Header errors are rare and so fine grained distinctions seem unnecessary. If there are header errors present the user will probably want to run with "-v" and find out what the problem really is. >seems like that would also apply in the event of checksum offload on TX.. I'd agree though right now I don't believe there's a way to implement this. Phil From sacadmin Tue May 25 03:17:15 2004 To: Philip Kirk - Solaris Sustaining cc: James.D.Carlson@sun.com, sommerfeld@east.sun.com, psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Date: Tue, 25 May 2004 12:16:48 +0200 From: Casper Dik Content-Length: 1414 >>I'd suggest something more indicative of the problem: "BAD SUM:", >>perhaps? > >I did discuss this with James. "ERR" is proposed as it's intended to be generic. >Any header error triggers this output including checksum errors and future >cases might extend the meaning of "ERR". Header errors are rare and so fine >grained distinctions seem unnecessary. If there are header errors present the >user will probably want to run with "-v" and find out what the problem really >is. > >>seems like that would also apply in the event of checksum offload on TX.. > >I'd agree though right now I don't believe there's a way to implement this. I've seen some other request where a customer wanted to be able to distinguish between packets received from the network and packets send over the network; in his particular case because he didn't want to see the packets he was sending. But I fear that if we don't allow snoop to distinguish between packets send w/ HW checksuming on and packets received with bad checksums that that would be a high call generator. Note that there is a history of customers caring about this; the in-kernel rlogin/telnet modify packets in place in ldterm; these used to be shared with the packets snooped and tcpdump would report bad checksums; similarly for nfs. We fixed this by duping the packet in IP if it is shared. And thanks for guessing the intend behind my question :-) Casper From sacadmin Tue May 25 04:06:56 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 25 May 2004 07:06:29 -0400 From: James Carlson To: Casper Dik Cc: Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Content-Length: 3579 Casper Dik writes: > I've seen some other request where a customer wanted to be able > to distinguish between packets received from the network and packets > send over the network; in his particular case because he didn't want to > see the packets he was sending. Yes, I think it'd be nice if our raw DLPI implementation could distinguish between transmitted and received packets, particularly so for point-to-point links, where there's no MAC address that could hint at the actual sender. (Snoop is _so_ Ethernet-centric.) Even if it did, though, there's no way to save TX versus RX in the current snoop on-disk format, so that's a fair amount of work (updating drivers and on-disk formats, as well as talking to other folks who import snoop format). And even if we fixed all of that, it's really just a hack around an underlying issue: the DLPI data are corrupted and there's no way for snoop or any other DLPI user to know that. Note that "fixing" the problem in this way would create a brand new (and perhaps unfixable) problem: snooping transmitted packets that are the result of IP forwarding would be broken. We would have to "assume" that because it's transmitted, the checksums are right, even though we didn't generate or check the transport layer checksums in transit, and they may be wrong. Not all transmitted packets are internally-generated! Transmit != error free. > But I fear that if we don't allow > snoop to distinguish between packets send w/ HW checksuming on > and packets received with bad checksums that that would be a high call > generator. I don't agree. First of all, we already *have* this problem. The bug isn't in snoop, but rather in the garbled data that snoop sees from the underlying DLPI provider. For customers who've realized that snoop isn't the best tool -- and have downloaded either ethereal or tcpdump -- they're in the same boat. Even customers who think that we're supplying a usable tool are in the lurch, since "-x 0" displays the raw data -- bogus checksum and all. (Yes, I've been bitten by this myself in looking at snoop -x 0 output, and I've had to respond to customers who've sent me private email about the issue.) I think the right fix here, rather than distinguishing between transmitted and received data, is to assert that raw data sent up via DLPI *must* be in a standard format, even if that means that using snoop disables certain optimizations. In other words, if the driver and network layer want to conspire in a game where the bytes sent down by the network layer aren't the same as those put on the wire, then the driver has a responsibility to make it look as though nothing is wrong to consumers ignorant of this game -- that is, it must do the checksums in software for snoop's benefit. > Note that there is a history of customers caring about this; the > in-kernel rlogin/telnet modify packets in place in ldterm; Not sure what that's about; ldterm isn't rlmod or telmod. > these used to be > shared with the packets snooped and tcpdump would report bad checksums; > similarly for nfs. We fixed this by duping the packet in IP if it is > shared. That's just a STREAMS programming error. If you touch the contents of a dblk where db_ref is greater than 1, then you've made a big mistake. Trying to design around programming errors sounds like a bad plan to me. -- James Carlson, IP Systems Group Sun Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 From sacadmin Tue May 25 04:38:00 2004 To: James Carlson cc: Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 25 May 2004 13:37:36 +0200 From: Casper Dik Content-Length: 832 >> Note that there is a history of customers caring about this; the >> in-kernel rlogin/telnet modify packets in place in ldterm; > >Not sure what that's about; ldterm isn't rlmod or telmod. Packets where moved directly from ethernet into rlmod/telmod into ldterm. So ldterm modified the packets. >> these used to be >> shared with the packets snooped and tcpdump would report bad checksums; >> similarly for nfs. We fixed this by duping the packet in IP if it is >> shared. > >That's just a STREAMS programming error. If you touch the contents of >a dblk where db_ref is greater than 1, then you've made a big mistake. >Trying to design around programming errors sounds like a bad plan to >me. Try, but IP never checked and so the errors happened (it now checks and dups the packet before any processing is done) Casper From sacadmin Tue May 25 10:10:15 2004 Date: Tue, 25 May 2004 10:09:44 -0700 From: Mike Ditto X-Accept-Language: en MIME-Version: 1.0 To: James Carlson CC: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 1075 James Carlson wrote: > Even if it did, though, there's no way to save TX versus RX in the > current snoop on-disk format, so that's a fair amount of work > (updating drivers and on-disk formats, as well as talking to other > folks who import snoop format). And issuing an update to RFC1761. And in case anybody's keeping track, the other noted shortcoming of both the disk format and the kernel interface to snoop is that there is no way to indicate which interface carried a packet. This makes it difficult to create a kernel "snoop multiplexer" that could capture packets across multiple interfaces, as well as a unified capture file that could record a time-ordered history of a complex multi-link protocol interaction. It would also be nice to be able to use the same file format for consolidated packet logs across a whole cluster of related notes, with interfaces uniquely identified (host1:hme0 vs host2:hme0). Finally, an optional "reason" code would enable use for anomaly/policy violation logs for firewalls and intrusion detection systems. -=] Mike [=- From sacadmin Tue May 25 10:40:16 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 25 May 2004 13:39:49 -0400 From: James Carlson To: Mike Ditto Cc: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Content-Length: 1736 Mike Ditto writes: > And issuing an update to RFC1761. That wouldn't really be _required_, though it'd be nice to do. That was Informational (non-standards-track) anyway. > And in case anybody's keeping > track, the other noted shortcoming of both the disk format and the > kernel interface to snoop is that there is no way to indicate > which interface carried a packet. This makes it difficult to > create a kernel "snoop multiplexer" that could capture packets > across multiple interfaces, as well as a unified capture file that > could record a time-ordered history of a complex multi-link > protocol interaction. It would also be nice to be able to use > the same file format for consolidated packet logs across a whole > cluster of related notes, with interfaces uniquely identified > (host1:hme0 vs host2:hme0). There's a lot more than that missing. For example, being able to represent error conditions (frames with bad FCS, collisions) and special link events (carrier loss, flow control, beacons). > Finally, an optional "reason" code > would enable use for anomaly/policy violation logs for firewalls > and intrusion detection systems. That's part of a wider notion of tagging the contents of the log with metainformation, such as partial decodes, flow identifiers, comments and the like. Not that this is really the right list for redesigning snoop. The point I was making was that distinguishing TX from RX doesn't really fix the underlying problem and can't generally be done anyway. -- James Carlson, IP Systems Group Sun Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 From sacadmin Tue May 25 11:25:08 2004 Date: Tue, 25 May 2004 11:24:44 -0700 From: Gary Morton Subject: Re: 2004/426 Checksum Support for Snoop To: James Carlson Cc: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.4) Gecko/20040414 Content-Length: 2360 James Carlson wrote: > >> But I fear that if we don't allow >>snoop to distinguish between packets send w/ HW checksuming on >>and packets received with bad checksums that that would be a high call >>generator. > > > I don't agree. First of all, we already *have* this problem. The bug > isn't in snoop, but rather in the garbled data that snoop sees from > the underlying DLPI provider. For customers who've realized that > snoop isn't the best tool -- and have downloaded either ethereal or > tcpdump -- they're in the same boat. Even customers who think that > we're supplying a usable tool are in the lurch, since "-x 0" displays > the raw data -- bogus checksum and all. (Yes, I've been bitten by > this myself in looking at snoop -x 0 output, and I've had to respond > to customers who've sent me private email about the issue.) > > I think the right fix here, rather than distinguishing between > transmitted and received data, is to assert that raw data sent up via > DLPI *must* be in a standard format, even if that means that using > snoop disables certain optimizations. In other words, if the driver > and network layer want to conspire in a game where the bytes sent down > by the network layer aren't the same as those put on the wire, then > the driver has a responsibility to make it look as though nothing is > wrong to consumers ignorant of this game -- that is, it must do the > checksums in software for snoop's benefit. > I agree - we have a similar issue with inline IPsec hardware acceleration - when snoop is enabled we disable inline IPsec inorder to not present unencrypted data to the "snooper". It would seem that the same thing could be done for hardware checksum offload - disable when snoop is on. > >>Note that there is a history of customers caring about this; the >>in-kernel rlogin/telnet modify packets in place in ldterm; > > > Not sure what that's about; ldterm isn't rlmod or telmod. > > >>these used to be >>shared with the packets snooped and tcpdump would report bad checksums; >>similarly for nfs. We fixed this by duping the packet in IP if it is >>shared. > > > That's just a STREAMS programming error. If you touch the contents of > a dblk where db_ref is greater than 1, then you've made a big mistake. > Trying to design around programming errors sounds like a bad plan to > me. > From sacadmin Tue May 25 11:41:18 2004 Date: Tue, 25 May 2004 11:37:13 -0700 From: Frank Dimambro Subject: Re: 2004/426 Checksum Support for Snoop To: James Carlson Cc: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 Content-Length: 1796 Hi Jim > I think the right fix here, rather than distinguishing between > transmitted and received data, is to assert that raw data sent up via > DLPI *must* be in a standard format, even if that means that using > snoop disables certain optimizations. In other words, if the driver > and network layer want to conspire in a game where the bytes sent down > by the network layer aren't the same as those put on the wire, then > the driver has a responsibility to make it look as though nothing is > wrong to consumers ignorant of this game -- that is, it must do the > checksums in software for snoop's benefit. > Since snoop stream is the only stream which cares we could also add to the DLPI implentation a cksum calculator and correct the checksum field for packets going up. Switching off the feature is tricky because it is done on a per stream basis, and then each packet gets cksum info so Tx looks like if ((wq->sbp->flags.tcksum_on) && (mp->dblk_flags.cksum_enable)) { Update cksum info in descriptor. } Switching it off will require a notification to IP to tell it that it now needs to stop passing down cksums. We could add... if ((wq->sbp->flags.tcksum_on) && (mp->dblk_flags.cksum_enable) && (adapter->flags->promis_on)) { /* * You must copy as the hardware needs the * checksum initial packet. */ pmp = copymsg(mp); /* * On the copied packet mimic what the hardware * would do for the checksum caclulation, for the * packets going upstream. */ mimic_hwcksum(&pmp); /* * Do the same old pass packet up to all promisc * streams. */ for (each promisc tream, q) { tmp = dupmsg(&pmp); passup(q, tmp); } freemsg(pmp); } The above is for illustration only, actual code may not match this exactly. Frank From sacadmin Tue May 25 11:58:54 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 25 May 2004 14:58:28 -0400 From: James Carlson To: Frank.Dimambro@Sun.COM Cc: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com Subject: Re: 2004/426 Checksum Support for Snoop Content-Length: 1504 Frank Dimambro writes: > Since snoop stream is the only stream which cares [...] How exactly do we know that? The way DLPI negotiation "normally" works is that you start off with things in some sort of default state. For example, you're not registered for any multicast addresses. You then -- for *your* stream and no-one else's -- negotiate the features you'd like to have. It's the provider's (driver's) responsibility to keep track of the features that each stream is using. The underlying problem here is that we have a feature that modifies the behavior of all streams on the given PPA, and does so without any way of warning those other streams that they're not in Kansas anymore. > Switching it off will require a notification to IP to tell it that > it now needs to stop passing down cksums. That'd be pretty nasty to get right, as there'd undoubtedly be packets in flight. > We could add... > > if ((wq->sbp->flags.tcksum_on) && > (mp->dblk_flags.cksum_enable) && > (adapter->flags->promis_on)) { Unfortunately, it's not really related to promiscuous mode. It's related _only_ to whether that particular stream asked to enable the special checksum mode. > The above is for illustration only, actual code may not match this > exactly. No doubt. ;-} -- James Carlson, IP Systems Group Sun Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 From sacadmin Tue May 25 15:38:12 2004 Date: Tue, 25 May 2004 15:34:22 -0700 From: Frank Dimambro Subject: Re: 2004/426 Checksum Support for Snoop To: James Carlson Cc: Casper Dik , Philip Kirk - Solaris Sustaining , psarc@sac.sfbay.sun.com MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0.1) Gecko/20020920 Netscape/7.0 Content-Length: 2943 Hi Jim James Carlson wrote: > Frank Dimambro writes: > >>Since snoop stream is the only stream which cares > > [...] > > How exactly do we know that? > The DLPI driver knows that, as it keeps the state of promisc mode on a per stream basis as required by the DLPI spec. The Promisc state of the device is the logical or of all streams promisc state. When it comes to looping back packet (TX) or classification the driver has to do things differently if a stream is in promisc mode. As for the DLPI consumer it cannot know about the promisc state of the other consumers which is the problem > > The way DLPI negotiation "normally" works is that you start off with > things in some sort of default state. For example, you're not > registered for any multicast addresses. You then -- for *your* stream > and no-one else's -- negotiate the features you'd like to have. It's > the provider's (driver's) responsibility to keep track of the features > that each stream is using. > Yes that's correct > > The underlying problem here is that we have a feature that modifies > the behavior of all streams on the given PPA, and does so without any > way of warning those other streams that they're not in Kansas anymore. > That's somewhat true, there is the notification mechanism to tell the other streams. But the driver does insure that classification to correct streams remain intact. > > >>Switching it off will require a notification to IP to tell it that >>it now needs to stop passing down cksums. > > > That'd be pretty nasty to get right, as there'd undoubtedly be packets > in flight. > Yes, I can see that. > >>We could add... >> >> if ((wq->sbp->flags.tcksum_on) && >> (mp->dblk_flags.cksum_enable) && >> (adapter->flags->promis_on)) { > > > Unfortunately, it's not really related to promiscuous mode. > I see the encryption problem, but what's unique to promiscuous mode is that the problem of incorrect cksum only occurs for packets being sent back up (loopback from Tx), which only happens when enabling promisc mode, the regular Rx mode, the checksum passed up is the one received in the packet, and a second one received via hardware calc, which can simply be ingnored for streams not aware of hardware cksum. > > It's > related _only_ to whether that particular stream asked to enable the > special checksum mode. > For that stream, if it asked for it to be enabled, it will not have a problem because it will not look at the Tx packate again. It will only have a problem if it enabled cksum and then did promisc mode. If the checksum mode is enabled it's a contract between the driver and the guy enabling it, if a module is in the path that doesn't recognise cksum, then the negotiation fails, due to dl_cap negotiaion rules. Which leaves a class of application that's enabled promisc mode. > >>The above is for illustration only, actual code may not match this >>exactly. > > > No doubt. ;-} > From sacadmin Mon Jun 7 05:42:45 2004 Date: Mon, 7 Jun 2004 13:42:11 +0100 (BST) From: Philip Kirk - Solaris Sustaining Subject: Re: 2004/426 Checksum Support for Snoop To: psarc@sac.sfbay.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Span_of_Mules_613_000 Content-Length: 33799 --Span_of_Mules_613_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: y/+Nx6vm9G/ZCZmvlSatDg== Given the issues around hardware checksumming I'd like to propose some amendments to the case. The original case with amendments is attached. Phil --Span_of_Mules_613_000 Content-Type: TEXT/plain; name="snoop_checksum.txt.new"; charset=us-ascii; x-unix-mode=0644 Content-Description: snoop_checksum.txt.new Content-MD5: mlnBjoCWPOReT2YwjPjhng== Problem ------- Currently snoop does not verify checksums for IP, ICMP, UDP or TCP packets. When debugging certain network problems this ability would prove very helpful. The output of snoop is classified as "Unstable." This change request is for a Minor release binding. + Issues + ------ + Currently if a NIC is doing hardware checksumming then + for locally originating packets users can see + see garbled checksums with snoop and other + monitoring tools such as ethereal and tcpdump. + For ethereal and tcpdump which already do checksum + verification an incorrect checksum will be reported. + Right now there is no easy way to determine whether a NIC + is doing hardware checksumming and so no way of avoiding + the problem. This part of the problem is not been + addressed by this case. + + To address the issue described above the project + team requests that this case be derailed so that + advice can be given to the PAC, stating that the + current DLPI DL_PROMISC_PHYS semantics are broken + for all consumers by PSARC 1996/173 and 2003/264, + and that a new project is required to make network + interface monitoring usable on Solaris. + Solution and example output --------------------------- This change will enable verification of checksums for IP, ICMP, UDP and TCP packets. In the case of UDP and TCP checksum verification will only be done ! for non-fragmented packets. To address the issue ! described above the following workaround is ! proposed. Although there is no way for snoop to ! determine whether a device is doing hardware checksumming ! it can still send down the PSARC 2003/264 messages ! to determine whether the driver supports it. In the ! case where a driver is capable of hardware checksumming ! snoop will print the following warning: + # snoop + Using device /dev/eri (promiscuous mode) + WARNING: Checksum verification may be inaccurate + due to hardware checksum. + + This of course does not solve the problem of + saved files where there is no way to know whether + the device that the snoop file was captured + on was hardware checksum capable. To address this + the manpage documentation will also be updated + detailing the problem of checksum verification + when a device is doing h/w checksumming. An + updated man page is at the end of this document. + + Given the problem of garbled checksums when the + NIC is doing hardware checksumming I also + propose documenting this in an infodoc/srdb to + help avoid future customer questions. + The RFE associated with this case is: "1196134 snoop should verify ip, tcp, and udp checksums" In summary form the keyword ERR will preceed the summary line: 32 0.00006 a -> b ERR TELNET C port=49277 In verbose summary mode the keyword ERR will be printed before the summary line of the layer that has a problem. For the case where multiple layers have a problem ERR will be printed for all layers that have a problem. Examples of a single layer having a problem and multiple layers having problems are below: Single layer: 32 0.00000 a -> b ETHER Type=0800 (IP), size = 54 bytes 32 0.00000 a -> b IP D=129.156.195.7 S=129.156.195.73 LEN=40, ID=32485, TOS=0x0, TTL=64 32 0.00000 a -> b ERR TCP D=23 S=49277 Ack=1265953414 Seq=3907929187 Len=0 Win=49640 32 0.00000 a -> b TELNET C port=49277 Multiple layers: 5 0.00000 a -> b ETHER Type=0800 (IP), size = 66 bytes 5 0.00000 a -> b ERR IP D=144.86.195.7 S=129.156.195.73 LEN=52, ID=32471, TOS=0x0, TTL=64 5 0.00000 a -> b ERR TCP D=23 S=49277 Syn Seq=3907929068 Len=0 Win=49640 Options= 5 0.00000 a -> b TELNET C port=49277 In verbose mode the string (Correct) will be printed in the case of the checksum been correct. If the checksum is incorrect the string (Wrong should be ) will be printed. In the case where it has not been possible to calculate the checksum then the string (Not verified) will be printed. Examples of all these forms are below: IP: Header checksum = 3261 (Correct) TCP: Checksum = 0x89a4 (Wrong should be 0x4080) TCP: Checksum = 0x04a9 (Not verified) Updated man page ---------------- System Administration Commands snoop(1M) NAME snoop - capture and inspect network packets SYNOPSIS snoop [-aqrCDNPSvV] [ -t [r | a | d] ] [-c maxcount] [-d device] [-i filename] [-n filename] [-o filename] [ -p first [ , last]] [-s snaplen] [ -x offset [ , length]] [expres- sion] DESCRIPTION snoop captures packets from the network and displays their contents. snoop uses both the network packet filter and streams buffer modules to provide efficient capture of pack- ets from the network. Captured packets can be displayed as they are received, or saved to a file (which is RFC 1761- compliant) for later inspection. snoop can display packets in a single-line summary form or in verbose multi-line forms. In summary form, only the data pertaining to the highest level protocol is displayed. For example, an NFS packet will have only NFS information displayed. The underlying RPC, UDP, IP, and ethernet frame information is suppressed but can be displayed if either of the verbose options are chosen. In the absence of a name service, such as LDAP or NIS, snoop displays host names as numeric IP addresses. snoop requires an interactive interface. OPTIONS -C List the code generated from the filter expression for either the kernel packet filter, or snoop's own filter. -D Display number of packets dropped during capture on the summary line. -N Create an IP address-to-name file from a capture file. This must be set together with the -i option that names a capture file. The address-to-name file has the same name as the capture file with .names appended. This file records the IP address to hostname mapping at the capture site and increases the portability of the capture file. Generate a .names file if the cap- ture file is to be analyzed elsewhere. Packets are not displayed when this flag is used. -P Capture packets in non-promiscuous mode. Only broad- cast, multicast, or packets addressed to the host machine will be seen. -S Display size of the entire link layer frame in bytes on the summary line. -V Verbose summary mode. This is halfway between summary mode and verbose mode in degree of verbosity. Instead of displaying just the summary line for the highest level protocol in a packet, it displays a summary line for each protocol layer in the packet. For instance, for an NFS packet it will display a line each for the ETHER, IP, UDP, RPC and NFS layers. Verbose summary mode output may be easily piped through grep to extract packets of interest. For example, to view only RPC summary lines, enter the following: example# snoop -i rpc.cap -V | grep RPC -a Listen to packets on /dev/audio (warning: can be noisy). -c maxcount Quit after capturing maxcount packets. Otherwise keep capturing until there is no disk space left or until interrupted with Control-C. -d device Receive packets from the network using the interface specified by device, for example, eri0 or hme0. The program netstat(1M), when invoked with the -i flag, lists all the interfaces that a machine has. Normally, snoop will automatically choose the first non-loopback interface it finds. -i filename Display packets previously captured in filename. Without this option, snoop reads packets from the net- work interface. If a filename.names file is present, it is automatically loaded into the snoop IP address- to-name mapping table (See -N flag). -n filename Use filename as an IP address-to-name mapping table. This file must have the same format as the /etc/hosts file (IP address followed by the hostname). -o filename Save captured packets in filename as they are cap- tured. (This filename is referred to as the "capture file".) The format of the capture file is RFC 1761- compliant. During packet capture, a count of the number of packets saved in the file is displayed. If you wish just to count packets without saving to a file, name the file /dev/null. -p first [ , last ] Select one or more packets to be displayed from a cap- ture file. The first packet in the file is packet number 1. -q When capturing network packets into a file, do not display the packet count. This can improve packet cap- turing performance. -r Do not resolve the IP address to the symbolic name. This prevents snoop from generating network traffic while capturing and displaying packets. However, if the -n option is used, and an address is found in the mapping file, its corresponding name will be used. -s snaplen Truncate each packet after snaplen bytes. Usually the whole packet is captured. This option is useful if only certain packet header information is required. The packet truncation is done within the kernel giving better utilization of the streams packet buffer. This means less chance of dropped packets due to buffer overflow during periods of high traffic. It also saves disk space when capturing large traces to a capture file. To capture only IP headers (no options) use a snaplen of 34. For UDP use 42, and for TCP use 54. You can capture RPC headers with a snaplen of 80 bytes. NFS headers can be captured in 120 bytes. -t [ r | a | d ] Time-stamp presentation. Time-stamps are accurate to within 4 microseconds. The default is for times to be presented in d (delta) format (the time since receiv- ing the previous packet). Option a (absolute) gives wall-clock time. Option r (relative) gives time rela- tive to the first packet displayed. This can be used with the -p option to display time relative to any selected packet. -v Verbose mode. Print packet headers in lots of detail. This display consumes many lines per packet and should be used only on selected packets. -x offset [ , length] Display packet data in hexadecimal and ASCII format. The offset and length values select a portion of the packet to be displayed. To display the whole packet, use an offset of 0. If a length value is not provided, the rest of the packet is displayed. OPERANDS expression Select packets either from the network or from a cap- ture file. Only packets for which the expression is true will be selected. If no expression is provided it is assumed to be true. Given a filter expression, snoop generates code for either the kernel packet filter or for its own inter- nal filter. If capturing packets with the network interface, code for the kernel packet filter is gen- erated. This filter is implemented as a streams module, upstream of the buffer module. The buffer module accumulates packets until it becomes full and passes the packets on to snoop. The kernel packet filter is very efficient, since it rejects unwanted packets in the kernel before they reach the packet buffer or snoop. The kernel packet filter has some limitations in its implementation; it is possible to construct filter expressions that it cannot handle. In this event, snoop tries to split the filter and do as much filtering in the kernel as possible. The remain- ing filtering is done by the packet filter for snoop. The -C flag can be used to view generated code for either the packet filter for the kernel or the packet filter for snoop. If packets are read from a capture file using the -i option, only the packet filter for snoop is used. A filter expression consists of a series of one or more boolean primitives that may be combined with boolean operators (AND, OR, and NOT). Normal pre- cedence rules for boolean operators apply. Order of evaluation of these operators may be controlled with parentheses. Since parentheses and other filter expression characters are known to the shell, it is often necessary to enclose the filter expression in quotes. Refer to Setting Up A More Efficient Filter for information about setting up more efficient filters. The primitives are: host hostname True if the source or destination address is that of hostname. The hostname argument may be a literal address. The keyword host may be omit- ted if the name does not conflict with the name of another expression primitive. For example, "pinky" selects packets transmitted to or received from the host pinky, whereas "pinky and dinky" selects packets exchanged between hosts pinky AND dinky. The type of address used depends on the primi- tive which precedes the host primitive. The pos- sible qualifiers are "inet", "inet6", "ether", or none. These three primitives are discussed below. Having none of the primitives present is equivalent to "inet host hostname or inet6 host hostname". In other words, snoop tries to filter on all IP addresses associated with hostname. inet or inet6 A qualifier that modifies the host primitive that follows. If it is inet, then snoop tries to filter on all IPv4 addresses returned from a name lookup. If it is inet6, snoop tries to filter on all IPv6 addresses returned from a name lookup. ipaddr, atalkaddr , or etheraddr Literal addresses, IP dotted, AppleTalk dotted, and ethernet colon are recognized. For example, +o "172.16.40.13" matches all packets with that IP ; +o "2::9255:a00:20ff:fe73:6e35" matches all packets with that IPv6 address as source or destination; +o "65281.13" matches all packets with that AppleTalk address; +o "8:0:20:f:b1:51" matches all packets with the ethernet address as source or destina- tion. An ethernet address beginning with a letter is interpreted as a hostname. To avoid this, prepend a zero when specifying the address. For example, if the ethernet address is "aa:0:45:23:52:44", then specify it by add a leading zero to make it "0aa:0:45:23:52:44". from or src A qualifier that modifies the following host, net, ipaddr, atalkaddr, etheraddr, port or rpc primitive to match just the source address, port, or RPC reply. to or dst A qualifier that modifies the following host, net, ipaddr, atalkaddr, etheraddr, port or rpc primitive to match just the destination address, port, or RPC call. ether A qualifier that modifies the following host primitive to resolve a name to an ethernet address. Normally, IP address matching is per- formed. This option is not supported on media such as IPoIB (IP over InfiniBand). ethertype number True if the ethernet type field has value number. Equivalent to "ether[12:2] = number". ip, ip6, arp, rarp, pppoed, pppoes True if the packet is of the appropriate ether- type. pppoe True if the ethertype of the packet is either pppoed or pppoes. broadcast True if the packet is a broadcast packet. Equivalent to "ether[2:4] = 0xffffffff" for eth- ernet. This option is not supported on media such as IPoIB (IP over InfiniBand). multicast True if the packet is a multicast packet. Equivalent to "ether[0] & 1 = 1" on ethernet. This option is not supported on media such as IPoIB (IP over InfiniBand). bootp, dhcp True if the packet is an unfragmented UDP packet with either a source port of BOOTPS (67) and a destination port of BOOTPC (68), or a source port of BOOTPC (68) and a destination of BOOTPS (67). apple True if the packet is an Apple Ethertalk packet. Equivalent to "ethertype 0x809b or ethertype 0x80f3". decnet True if the packet is a DECNET packet. greater length True if the packet is longer than length. less length True if the packet is shorter than length. udp, tcp, icmp, icmp6, ah, esp True if the IP or IPv6 protocol is of the appropriate type. net net True if either the IP source or destination address has a network number of net. The from or to qualifier may be used to select packets for which the network number occurs only in the source or destination address. port port True if either the source or destination port is port. The port may be either a port number or name from /etc/services. The tcp or udp primi- tives may be used to select TCP or UDP ports only. The from or to qualifier may be used to select packets for which the port occurs only as the source or destination. rpc prog [ , vers [ , proc ] ] True if the packet is an RPC call or reply packet for the protocol identified by prog. The prog may be either the name of an RPC protocol from /etc/rpc or a program number. The vers and proc may be used to further qualify the program version and procedure number, for example, "rpc nfs,2,0" selects all calls and replies for the NFS null procedure. The to or from qualifier may be used to select either call or reply packets only. ldap True if the packet is an LDAP packet on port 389. gateway host True if the packet used host as a gateway, that is, the ethernet source or destination address was for host but not the IP address. Equivalent to "ether host host and not host host". nofrag True if the packet is unfragmented or is the first in a series of IP fragments. Equivalent to "ip[6:2] & 0x1fff = 0". expr relop expr True if the relation holds, where relop is one of >, <, >=, <=, =, !=, and expr is an arith- metic expression composed of numbers, packet field selectors, the length primitive, and arithmetic operators +, -, *, &, |, ^, and %. The arithmetic operators within expr are evaluated before the relational operator and normal precedence rules apply between the arith- metic operators, such as multiplication before addition. Parentheses may be used to control the order of evaluation. To use the value of a field in the packet use the following syntax: base[expr [: size ] ] where expr evaluates the value of an offset into the packet from a base offset which may be ether, ip, ip6, udp, tcp, or icmp. The size value specifies the size of the field. If not given, 1 is assumed. Other legal values are 2 and 4. For example, ether[0] & 1 = 1 is equivalent to multicast ether[2:4] = 0xffffffff is equivalent to broadcast. ip[ip[0] & 0xf * 4 : 2] = 2049 is equivalent to udp[0:2] = 2049 ip[0] & 0xf > 5 selects IP packets with options. ip[6:2] & 0x1fff = 0 eliminates IP fragments. udp and ip[6:2]&0x1fff = 0 and udp[6:2] != 0 finds all packets with UDP checksums. The length primitive may be used to obtain the length of the packet. For instance "length > 60" is equivalent to "greater 60", and "ether[length - 1]" obtains the value of the last byte in a packet. and Perform a logical AND operation between two boolean values. The AND operation is implied by the juxtaposition of two boolean expressions, for example "dinky pinky" is the same as "dinky AND pinky". or or , Perform a logical OR operation between two boolean values. A comma may be used instead, for example, "dinky,pinky" is the same as "dinky OR pinky". not or ! Perform a logical NOT operation on the following boolean value. This operator is evaluated before AND or OR. slp True if the packet is an SLP packet. sctp True if the packet is a SCTP packet. EXAMPLES Example 1: Using the snoop Command Capture all packets and display them as they are received: example# snoop Capture packets with host funky as either the source or des- tination and display them as they are received: example# snoop funky Capture packets between funky and pinky and save them to a file. Then inspect the packets using times (in seconds) relative to the first captured packet: example# snoop -o cap funky pinky example# snoop -i cap -t r | more To look at selected packets in another capture file: example# snoop -i pkts -p 99,108 99 0.0027 boutique -> sunroof NFS C GETATTR FH=8E6 100 0.0046 sunroof -> boutique NFS R GETATTR OK 101 0.0080 boutique -> sunroof NFS C RENAME FH=8E6C MTra00192 to .nfs08 102 0.0102 marmot -> viper NFS C LOOKUP FH=561E screen.r.13.i386 103 0.0072 viper -> marmot NFS R LOOKUP No such file or directory 104 0.0085 bugbomb -> sunroof RLOGIN C PORT=1023 h 105 0.0005 kandinsky -> sparky RSTAT C Get Statistics 106 0.0004 beeblebrox -> sunroof NFS C GETATTR FH=0307 107 0.0021 sparky -> kandinsky RSTAT R 108 0.0073 office -> jeremiah NFS C READ FH=2584 at 40960 for 8192 To look at packet 101 in more detail: example# snoop -i pkts -v -p101 ETHER: ----- Ether Header ----- ETHER: ETHER: Packet 101 arrived at 16:09:53.59 ETHER: Packet size = 210 bytes ETHER: Destination = 8:0:20:1:3d:94, Sun ETHER: Source = 8:0:69:1:5f:e, Silicon Graphics ETHER: Ethertype = 0800 (IP) ETHER: IP: ----- IP Header ----- IP: IP: Version = 4, header length = 20 bytes IP: Type of service = 00 IP: ..0. .... = routine IP: ...0 .... = normal delay IP: .... 0... = normal throughput IP: .... .0.. = normal reliability IP: Total length = 196 bytes IP: Identification 19846 IP: Flags = 0X IP: .0.. .... = may fragment IP: ..0. .... = more fragments IP: Fragment offset = 0 bytes IP: Time to live = 255 seconds/hops IP: Protocol = 17 (UDP) IP: Header checksum = 18DC IP: Source address = 172.16.40.222, boutique IP: Destination address = 172.16.40.200, sunroof IP: UDP: ----- UDP Header ----- UDP: UDP: Source port = 1023 UDP: Destination port = 2049 (Sun RPC) UDP: Length = 176 UDP: Checksum = 0 UDP: RPC: ----- SUN RPC Header ----- RPC: RPC: Transaction id = 665905 RPC: Type = 0 (Call) RPC: RPC version = 2 RPC: Program = 100003 (NFS), version = 2, procedure = 1 RPC: Credentials: Flavor = 1 (Unix), len = 32 bytes RPC: Time = 06-Mar-90 07:26:58 RPC: Hostname = boutique RPC: Uid = 0, Gid = 1 RPC: Groups = 1 RPC: Verifier : Flavor = 0 (None), len = 0 bytes RPC: NFS: ----- SUN NFS ----- NFS: NFS: Proc = 11 (Rename) NFS: File handle = 000016430000000100080000305A1C47 NFS: 597A0000000800002046314AFC450000 NFS: File name = MTra00192 NFS: File handle = 000016430000000100080000305A1C47 NFS: 597A0000000800002046314AFC450000 NFS: File name = .nfs08 NFS: To view just the NFS packets between sunroof and boutique: example# snoop -i pkts rpc nfs and sunroof and boutique 1 0.0000 boutique -> sunroof NFS C GETATTR FH=8E6C 2 0.0046 sunroof -> boutique NFS R GETATTR OK 3 0.0080 boutique -> sunroof NFS C RENAME FH=8E6C MTra00192 to .nfs08 To save these packets to a new capture file: example# snoop -i pkts -o pkts.nfs rpc nfs sunroof boutique To view encapsulated packets, there will be an indicator of encapsulation: example# snoop ip-in-ip sunroof -> boutique ICMP Echo request (1 encap) If -V is used on an encapsulated packet: example# snoop -V ip-in-ip sunroof -> boutique ETHER Type=0800 (IP), size = 118 bytes sunroof -> boutique IP D=172.16.40.222 S=172.16.40.200 LEN=104, ID=27497 sunroof -> boutique IP D=10.1.1.2 S=10.1.1.1 LEN=84, ID=27497 sunroof -> boutique ICMP Echo request Example 2: Setting Up A More Efficient Filter To set up a more efficient filter, the following filters should be used toward the end of the expression, so that the first part of the expression can be set up in the kernel: greater, less, port, rpc, nofrag, and relop. The presence of OR makes it difficult to split the filtering when using these primitives that cannot be set in the kernel. Instead, use parentheses to enforce the primitives that should be OR'd. To capture packets between funky and pinky of type tcp or udp on port 80: example# snoop funky and pinky and port 80 and tcp or udp Since the primitive port cannot be handled by the kernel filter, and there is also an OR in the expression, a more efficient way to filter is to move the OR to the end of the expression and to use parentheses to enforce the OR between tcp and udp: example# snoop funky and pinky and (tcp or udp) and port 80 EXIT STATUS 0 Successful completion. 1 An error occurred. FILES /dev/audio Symbolic link to the system's primary audio device. /dev/null The null file. /etc/hosts Host name database. /etc/rpc RPC program number data base. /etc/services Internet services and aliases. ATTRIBUTES See attributes(5) for descriptions of the following attri- butes: ____________________________________________________________ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | |_____________________________|_____________________________| | Availability | SUNWrcmdc | |_____________________________|_____________________________| SEE ALSO netstat(1M), hosts(4), rpc(4), services(4), attributes(5), audio(7I), bufmod(7M), dlpi(7P), pfmod(7M), tun(7M) Callaghan, B. and Gilligan, R. RFC 1761, Snoop Version 2 Packet Capture File Format. Network Working Group. February 1995. WARNINGS The processing overhead is much higher for realtime packet interpretation. Consequently, the packet drop count may be higher. For more reliable capture, output raw packets to a file using the -o option and analyze the packets off-line. Unfiltered packet capture imposes a heavy processing load on the host computer, particularly if the captured packets are interpreted realtime. This processing load further increases if verbose options are used. Since heavy use of snoop may deny computing resources to other processes, it should not be used on production servers. Heavy use of snoop should be restricted to a dedicated computer. snoop does not reassemble IP fragments. Interpretation of higher level protocol halts at the end of the first IP frag- ment. snoop may generate extra packets as a side-effect of its use. For example it may use a network name service (NIS or NIS+) to convert IP addresses to host names for display. Capturing into a file for later display can be used to post- pone the address-to-name mapping until after the capture session is complete. Capturing into an NFS-mounted file may also generate extra packets. Setting the snaplen (-s option) to small values may remove header information that is needed to interpret higher level protocols. The exact cutoff value depends on the network and protocols being used. For NFS Version 2 traffic using UDP on 10 Mb/s ethernet, do not set snaplen less than 150 bytes. For NFS Version 3 traffic using TCP on 100 Mb/s ethernet, snaplen should be 250 bytes or more. snoop requires information from an RPC request to fully interpret an RPC reply. If an RPC reply in a capture file or packet range does not have a request preceding it, then only the RPC reply header will be displayed. + snoop will attempt to verify that the checksums for IP, TCP, + UDP and ICMP are correct. If the device that snoop opens + is doing hardware checksumming then the checksum for packets + originating locally will be shown as incorrect. When snoop + opens a device for capturing if the device is capable of + doing harware checksumming then the following warning will + be displayed: + + WARNING: Checksum verification may be inaccurate due to + hardware checksumming. + + When using snoop to read a cpature file there is no way for + snoop to know whether the device the data was captured on + was capable of doing hardware checksumming and so apparent + checksum errors on packets originating locally may well be + a consequence of this. SunOS 5.10 Last change: 18 Nov 2003 13 + Acknowledgements + ---------------- + + Thanks to James Carlson for helping with the ammendments to + this case. --Span_of_Mules_613_000-- From sacadmin Wed Jun 9 13:17:55 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Wed, 9 Jun 2004 16:17:19 -0400 From: James Carlson To: psarc@sac.sfbay.sun.com cc: Phil.Kirk@Sun.COM Subject: 2004/426 Checksum Support for Snoop Content-Length: 386 This fast-track request was derailed at today's PSARC meeting, and a vote on the revised materials (including advice to the PAC) was held. The case was approved. -- James Carlson, IP Systems Group Sun Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 From sacadmin Mon Oct 11 10:50:00 2004 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16746.51073.290320.777051@gargle.gargle.HOWL> Date: Mon, 11 Oct 2004 13:48:49 -0400 From: James Carlson To: psarc@sac.sfbay.sun.com Subject: Opinion for review: 2004/426 Checksum Support for Snoop Content-Length: 5632 Please review and submit comments by 10/18/2004. sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Checksum Support for Snoop Submitted by: Philip Kirk File: PSARC/2004/426/opinion.ms Date: June 9th, 2004 Committee: James Carlson, Ralph Campbell, Bill Sommer- feld, Gary Winiger. Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary This project adds TCP/IP checksum validation to snoop, so that administrators can more easily understand why packets are dropped, and to achieve better functional parity with open source equivalents. 2. Decision & Precedence Information The project is approved as specified in reference [1]. The project may be delivered in a minor release of Solaris. 3. Interfaces The project exports the following interfaces. ________________________________________________ | Interfaces Exported | |___________|________________|_________________| |Interface | Classification| Comments | |___________|________________|_________________| |output text| Unstable | Add ERR summary| | | | and verbose| | | | text | |___________|________________|_________________| PSARC/2004/426 Copyright 2004 Sun Microsystems, Inc. - 2 - The project imports the following interfaces. ______________________________________________________ | Interfaces Imported | |___________|_______________________|________________| |Interface | Classification | Comments | |___________|_______________________|________________| |sys/pattr.h| Consolidation Private| PSARC 2003/264| |___________|_______________________|________________| 4. Opinion Although this project does represent a step forward for snoop, and the proposal is complete, it highlights a problem area in the existing architecture. Several ARC members felt that it was important to have these issues addressed in a more complete manner. This case was derailed in order to provide the PAC with advice in that direction. The issue is that when hardware checksum offloading (PSARC 2003/264) is used, IP's outbound packets do not have valid checksums. Because snoop and other monitoring (tcpdump) and raw packet (bpf, Packet Shell) applications receive packets via interfaces at the DLPI level, this means that they have several significant flaws. 4.1. Bad Checksums Exposed To Users The first and most obvious issue is that packets with bad checksums are exposed to users. This often leads to confu- sion and, potentially, to outright application failure. 4.2. No Means To Detect Problem If other applications were designed to handle the bad check- sum problem, they'd be faced with a separate problem in the design of hardware checksum offload: there is no way for a raw listener to find out if any other streams have hardware checksum enabled. At best, they can query if the device supports hardware checksum and, if it does, merely assume that all DLPI users that speak IP use the feature and that checksums are thus not reliable, as this project proposes doing for snoop. 4.3. Cannot Discriminate Input From Output The raw DLPI interfaces do not distinguish input from out- put. This means that if an application can detect the potential issue as above, it cannot necessarily determine which packets are affected. As the lack of input/output indication is a serious independent flaw in snoop's design, especially on non-Ethernet media, this is an issue that should be addressed separately. PSARC/2004/426 Copyright 2004 Sun Microsystems, Inc. - 3 - 4.4. Local Versus Forwarded Traffic If the application knew when hardware checksum was in use, and which packets were transmitted by IP, we would find that monitoring IP forwarding becomes a problem, as forwarded packets already have the correct checksums and must not be touched by hardware. Unfortunately, there is no way to know which these are among the transmitted packets. 4.5. Snoop File Format Cannot Accommodate Data The snoop file format cannot accommodate flags or other information about the data source. This means that even if the above measures can be used to mitigate the problem, saved snoop packet traces are still vulnerable to interpre- tation errors. 4.6. Inconsistent Overall Architecture Similar issues are present with IPsec acceleration, where the raw (unencrypted) data would ordinarily be seen by the snoop user rather than the encrypted data that are sent on the wire. To prevent this exposure, we turn off hardware IPsec acceleration when other DLPI listeners are present. Essentially the same issue is present for hardware checksum offload, but the solution is not the same. 5. Minority Opinion(s) None 6. Advisory Information The PAC is advised to fund an effort to resolve the observa- bility and DLPI functional problems caused by hardware checksum offloading. 7. Appendices 7.1. Appendix A: Technical Changes Required None 7.2. Appendix B: Technical Changes Advised None 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2004/426. 1. Specification File: spec.txt PSARC/2004/426 Copyright 2004 Sun Microsystems, Inc.