Sun Microsystems Systems Architecture Committee _________________________________________________________________ Subject: ILB: Integrated L3/L4 Load balancer Submitted by: Sangeeta Misra File: PSARC/2008/575/opinion.txt Date: July 20th, 2009. Committee: Sebastien Roy (opinion written by Mark Martin), Garrett D'Amore, Kais Belgaied, Richard Matthews, Glenn Skinner Product Approval Committee: Solaris PAC solaris-pac-opinion@sun.com 1. Summary ILB project will deliver the basic features needed for Solaris to be used as a L3/L4 load balancing solution on SPARC and X86 platforms. This solution can also be used by third party load balancing solutions. 2. Decision & Precedence Information The project is approved as specified in reference [1]. The project may be delivered in a minor release of Solaris. 3. Interfaces Imported Interfaces Interface Classification Comments ============================================================== port_create(3C) Committed Event ports PSARC 2003/462 port_associate (3C) Committed Event ports PSARC 2003/462 ucred_get(3C), pwd.h Committed User credential getpwuid_r() Committed User Credential priv_set(3C), priv.h Committed Solaris Privileges auth_attr.h,exec_attr.h Committed RBAC chkauthattr() Committed RBAC adt_put_event(),adt.h Contract Private Solaris audit PSARC 2000/517 adt_free_event() Contract Private Solaris audit PSARC 2000/517 adt_start_session() Contract Private Solaris audit PSARC 2000/517 posix-spawn(3C),spawn.h Committed Exported Interfaces Interface Classification Comments =============================================================== SIOCILB Project Private ilbd daemon uses this ioctl to configure the ILB component usr/include/libilb.h Committed libilb functions Committed See Appendix C of design doc ilbadm(1M) Committed ILB CLI. See Appendix B of design doc 4. Opinion 4.1. Continuing networking path additions Members of the committee noted this project, like many others related to networking, are continuing to add processing to the processing path of packets through the network stack. Each project that delivers packet processing functionality, as this project does, has the onus of ensuring the no serious regression of performance occurs. Members noted that there does not exist any project tasked with ensuring architectural integrity. This integrity would ensure that as future projects continue to deliver additional packet processing features, the risks of severe or significant performance degradation would be minimized. The member's concern is that these features are adding branches in the data-path that intercept packets when the feature is enabled. Even when the features are disabled, each project's data-path "hooks" adds a tiny bit of latency to the data-path, and each is in its own location in the data-path due to individual design constraints. When taken individually, these features may not have much impact, but the aggregate impact of these features may be observably negative on the system as a whole. The committee members offer advice to the project management organization that it appears absolutely prudent to investigate creating a project to address this architectural concern. This project would define an architecture that allows projects to intercept packets in well-defined locations in that data-path without adding incremental latency. This advice is noted in the Advisory Section (section 6.1). 4.2. Providing diagnostic aid on troubleshooting configuration Additionally, members of the committee noted that this case causes an overlap in functionality with another network function, specifically sharing NAT type functionality with IPFilter. Although the team notes that the NAT functionality provided by ILB is much smaller in scope than IPFilter, the opportunity does exist for both to be enabled at the same time. The members advised the team that this should be noted in the documentation, preferably via some description that indicates the ordering of processing of packets, contrasting the NAT translation occurring earlier in ILB than later with IPFilter. This seemed sufficient as a minimal step to reduce possible configuration issues for administrators who may choose to enable both tools. The members also advise the same organization as mentioned in the preceding paragraph that it seems that some sort of diagnostic tool seems warranted here. The members cautioned against creating too much coupling between subsystems, especially where subsystem functionality may overlap, and by extension, configuration may overlap or be contentious. Embedding knowledge of each tool's capability for overlap in each other tool would cause a mushrooming amount of complexity and coupling. The members of the committee also advise the project management team that it should consider investigating a project that could be used as a diagnostic aid to administrators that could provide insights and information on packet processing on the system as a whole, including all possible paths through integrated subsystems. This advice is noted in the Advisory Section (section 6.2). 4.3. The "persist" netmask should be optional Finally, as noted in the advisory information section, members of the committee noted that the use of the "persist=" would more useful if it did not use any netmask as a default. The members advised the team that the "persist" option should allow a bare netmask, meaning "any host". This advice is noted in the Technical Changes Advised Section (section 7.2). 5. Minority Opinion(s) None. 6. Advisory Information 6.1. Assess Networking Architecture Scalability. As previously mentioned in the Opinion section 4.1, the committee members advise that the project management organization should consider project a project that investigates and potentially addresses the ongoing concern of adding additional network processing in the networking architecture. It is the committee's concern that as teams continue to come forward adding incremental overhead to network packet processing, maintaining a high performance network processing profile will continue to be a challenge. This advice is not intended for the project team presenting this case, but rather to the project management organization. 6.2. Provide Ways for Diagnosing Network Configurations in a Systemic Way As mentioned in the Opinion section 4.2, the committee members also advise the project management organization that the potential for network related subsystems such as this project to cause configuration overlap with other present and future projects continues to grow. Rather than provide diagnostic aids into each subsystem that addresses the individual overlapping configuration and functionality, it would be prudent to develop tools or infrastructure that could address diagnosing these overlaps (and the potential misconfiguration that could result). Such architecture would prevent requiring each project to build in knowledge of every other potentially conflicting subsystem, thus also preventing the unnecessary coupling that result from such subsystem centric knowledge. 7. Appendices 7.1. Appendix A: Technical Changes Required None. 7.2. Appendix B: Technical Changes Advised As mentioned in the Opinion section 4.3, the committee members advised the project team that it should change the specification so that the "persist" option should allow the netmask parameter to be optional, thereby implying all hosts by default. 7.3. Appendix C: Reference Material Unless stated otherwise, path names are relative to the case directory PSARC/2008/575. 1 Onepager File: 20080409_bill.sommerfeld 2 Inception minutes File: inception.materials/20081203.2008.575.inception 3 Commmitment minutes File: commitment.materials/20090708.2008.575.commitment 5 PSARC 20 Questions. File: inception.materials/ILB-PSARC20q.txt PSARC/2008/575 Copyright 2009 Sun Microsystems