sun microsystems Systems Architecture Committee _________________________________________________________________ Subject: Checksum Support for Snoop Submitted by: Philip Kirk File: PSARC/2004/426/opinion.ms Date: June 9th, 2004 Committee: James Carlson, Ralph Campbell, Bill Sommer- feld, Gary Winiger. Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary This project adds TCP/IP checksum validation to snoop, so that administrators can more easily understand why packets are dropped, and to achieve better functional parity with open source equivalents. 2. Decision & Precedence Information The project is approved as specified in reference [1]. The project may be delivered in a minor release of Solaris. 3. Interfaces The project exports the following interfaces. ________________________________________________ | Interfaces Exported | |___________|________________|_________________| |Interface | Classification| Comments | |___________|________________|_________________| |output text| Unstable | Add ERR summary| | | | and verbose| | | | text | |___________|________________|_________________| PSARC/2004/426 Copyright 200r Sun Microsystems, Inc. - 2 - The project imports the following interfaces. ______________________________________________________ | Interfaces Imported | |___________|_______________________|________________| |Interface | Classification | Comments | |___________|_______________________|________________| |sys/pattr.h| Consolidation Private| PSARC 2003/264| |___________|_______________________|________________| 4. Opinion Although this project does represent a step forward for snoop, and the proposal is complete, it highlights a problem area in the existing architecture. Several ARC members felt that it was important to have these issues addressed in a more complete manner. This case was derailed in order to provide the PAC with advice in that direction. The issue is that when hardware checksum offloading (PSARC 2003/264) is used, IP's outbound packets do not have valid checksums. Because snoop and other monitoring (tcpdump) and raw packet (bpf, Packet Shell) applications receive packets via interfaces at the DLPI level, this means that they have several significant flaws. 4.1. Bad Checksums Exposed To Users The first and most obvious issue is that packets with bad checksums are exposed to users. This often leads to confu- sion and, potentially, to outright application failure. 4.2. No Means To Detect Problem If other applications were designed to handle the bad check- sum problem, they'd be faced with a separate problem in the design of hardware checksum offload: there is no way for a raw listener to find out if any other streams have hardware checksum enabled. At best, they can query if the device supports hardware checksum and, if it does, merely assume that all DLPI users that speak IP use the feature and that checksums are thus not reliable, as this project proposes doing for snoop. 4.3. Cannot Discriminate Input From Output The raw DLPI interfaces do not distinguish input from out- put. This means that if an application can detect the potential issue as above, it cannot necessarily determine which packets are affected. As the lack of input/output indication is a serious independent flaw in snoop's design, especially on non-Ethernet media, this is an issue that should be addressed separately. PSARC/2004/426 Copyright 200r Sun Microsystems, Inc. - 3 - 4.4. Local Versus Forwarded Traffic If the application knew when hardware checksum was in use, and which packets were transmitted by IP, we would find that monitoring IP forwarding becomes a problem, as forwarded packets already have the correct checksums and must not be touched by hardware. Unfortunately, there is no way to know which these are among the transmitted packets. 4.5. Snoop File Format Cannot Accommodate Data The snoop file format cannot accommodate flags or other information about the data source. This means that even if the above measures can be used to mitigate the problem, saved snoop packet traces are still vulnerable to interpre- tation errors. 4.6. Inconsistent Overall Architecture Similar issues are present with IPsec acceleration, where the raw (unencrypted) data would ordinarily be seen by the snoop user rather than the encrypted data that are sent on the wire. To prevent this exposure, we turn off hardware IPsec acceleration when other DLPI listeners are present. Essentially the same issue is present for hardware checksum offload, but the solution is not the same. 5. Minority Opinion(s) None 6. Advisory Information The PAC is advised to fund an effort to resolve the observa- bility and DLPI functional problems caused by hardware checksum offloading. 7. Appendices 7.1. Appendix A: Technical Changes Required None 7.2. Appendix B: Technical Changes Advised None 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2004/426. 1. Specification File: spec.txt PSARC/2004/426 Copyright 200r Sun Microsystems, Inc.