From Mon Mar 19 14:15:26 2001 Date: Mon, 19 Mar 2001 14:15:26 -0800 (PST) From: Postmaster Subject: Message from mail server Content-Length: 95 Mime-Version: 1.0 Status: RO X-IMAP: 985040125 17 Delete. This is a system message. --END+PSEUDO-- From sac-list-owner Mon Feb 5 15:59:17 2001 Date: Mon, 05 Feb 2001 16:00:54 -0800 From: David Robinson X-Accept-Language: en MIME-Version: 1.0 To: psarc@sac.eng.sun.com CC: Jack.Bochsler@west.sun.com Subject: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) Content-Type: multipart/mixed; boundary="------------34E70F3386422AB4EC150615" Content-Length: 49569 Status: RO X-Status: $$$$ X-UID: 0000000001 This is a multi-part message in MIME format. --------------34E70F3386422AB4EC150615 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I am sponsoring the following for fast track approval. The timer expires 02/12/01. This case is really two in one, the first is a description of the generic interface and the second is the first instantiation. Given the closeness and it is easier to evaluate an infrastructure with an example I merged the two. -David --- DL_IOC_INFO_REQ/SET extensible interface for detecting/enabling lower level driver capabilities. jack.bochsler@west.sun.com 26 January 2001 COMMITMENT LEVEL DL_IOC_INFO_REQ Request and Set Primitive(s) Consolidation-Private RELEASE Minor. 1. Introduction 1.1. Project/Component Working Name: TCP/IP support for adapter/driver specific features 1.2. Name of Document Author/Supplier: Jack Bochsler, jack.bochsler@sun.com 1.3. Date of This Document: 01/02/01 1.4. Name of Major Document Customer(s)/Consumer(s): IE, PSARC 2. Description There is an expanding array of hardware optimization capabilities | being implemented in Network Interface Cards (NICs) that need to | be discovered by IP so that IP knows when and how to exploit them. There is an existing interface for IP to discover hardware checksumming capability but this interface is not extensible to cover the various requirements of other capabilities such as encryption and authentication. This project is a proposal to define an extensible request/response framework for upper layer software to determine device specific characteristics provided by specific hardware adapters and their drivers. This project proposes to introduce a generic Consolidation-Private Interface which is a simple but extensible framework for | communications between upper layer software to assess and control | hardware capabilities. Separate Contract-Private Interfaces between the producers (NIC device drivers) and consumers (IP) of the data would be necessary prior to the actual use of this interface. The advantage of this mechanism is that it would support response of multiple capabilities of arbitrary length. Note: PSARC/1996/173/ defined an interface for implementing a mechanism for a Contract Private Interface between IP and the SMCC SAHI3 155/622 ATM network interface device driver (ba) for the negotiation of inetcksuming when the device is opened by IP and to transport inetcksum state for the IP M_DATA fast-path. This implementation is not extensible and will not accommodate different data lengths. 3. External Requester N/A 4. Opportunity Window/Exposure Enable a uniform interface for communicating device specific info between IP and lower level software. 5. Business Opportunity/Exposure There are an increasing number of NICs that have hardware acceleration for multiple reasons. A mechanism needs to be in place for IP to detect and exploit these capabilities. 6. Technical Description This proposal creates the following two interfaces: |-----------------------|-----------------------|---------------- |Interface | Classification | Comments | |-----------------------|-----------------------|---------------- |DL_IOC_INFO_REQ | Consolidation Private | | |DL_IOC_INFO_SET | Consolidation Private | | | |DL_IOC_INFO_REQ_NONE | Consolidation Private | | |-----------------------|-----------------------|---------------- The following changes are recommended to the existing header file in the "Sun additions" area: #define DL_IOC_INFO_REQ (DLIOC|11) /* device specific\ info request */ #define DL_IOC_INFO_SET (DLIOC|12) /* device specific control */ The following #defines with proposed initial primitives and the generic response data structures format would be added to the bottom of the file: #define DL_IOC_INFO_REQ_NONE 0x00 /* no capabilities */ typedef struct { t_uscalar_t dl_cap; /* driver capability primitive */ t_uscalar_t dl_length; /* length of data following */ t_uscalar_t dl_data[1]; /* payload */ } dl_ioc_info_t; It is expected that additional capability primitives will be added to over time. Potential future capability primitives would | be for hardware checksum, IPsec AH and ESP acceleration and zero copy. | In order for these primitives to be implemented, separate ARC cases | would be necessary to establish the specific data content, format and error handling. There are two components to this proposal, DL_IOC_INFO_REQ and DL_IOC_INFO_SET. The first ioctl, DL_IOC_INFO_REQ, allows upper layer software to determine device specific capabilities provided by the layers below it. The second component is DL_IOC_INFO_SET which | is used by the upper layer software to selectively set (enable or | disable) the state of those capabilities. The expectation is that all hardware capabilities are off by default, and will only be enabled specifically at the request of the upper level software. If the lower level driver recognizes the DL_IOC_INFO_REQ primitive, | it will allocate an M_DATA buffer for each of the capabilities that | it supports, attach it to the b_cont field of the M_IOCTL and M_IOCAK the reply. The M_DATA buffer would contain a response of the form of dl_ioc_info_t. The actual length and format of the data would be a Contract-Private format agreed upon between the producer and consumer, per capability type. The elements of the dl_ioc_info_t structure are to be interpreted as follows: dl_cap - driver capability that is being enumerated as a result of a DL_IOC_INFO_REQ or controlled by a DL_IOC_INFO_SET request. dl_length - length of data following dl_data - payload field that will contain the Contract Private definition and use of this field. The expected error response to a DL_IOC_INFO_REQ are M_IOCNAK with the following error codes: EINVAL if the driver doesn't understand DL_IOC_INFO_REQ; ENOTSUP if the driver does support DL_IOC_INFO_REQ in general, but doesn't support the requested primitive. The primitive DL_IOC_INFO_REQ_NONE is allocated for any drivers that elect to implement and support the DL_IOC_INFO_REQ interface but have no hardware specific capabilities to report. They would set dl_cap to DL_IOC_INFO_REQ_NONE, dl_length to 0 and have no further data. When multiple capabilities are returned from the driver to the upper level software, it is mandatory that all the capabilities can be | exercised on the packet, if appropriate. E.g. if hardware checksum | and IPsec encryption acceleration are both supported by the underlying | hardware, then both DL_IOC_INFO_REQ_HCKSUM and DL_IOC_INFO_REQ_ESP | would both be returned. The expectation is that both checksum support and encryption support will apply to the packet. Note for example that if the ESP processing fails (due to no SA state in the hardware), the packet can still be passed up for software decryption, but there is no point in performing the hardware checksum assist. | The second ioctl, DL_IOC_INFO_SET, allows upper layer software to | control the capabilities discovered as a result of the DL_IOC_INFO_REQ M_IOCTL command. The format of the DL_IOC_INFO_SET command is a M_IOCTL mblk with ioc_cmd set to DL_IOC_INFO_SET and one or more mblks chained from the b_cont of the M_IOCTL mblk, each containing dl_ioc_info_t data, with the dl_cap field set to the capability that is to be enabled. On receipt of an DL_IOC_INFO_SET command, the leaf driver should | set the state of all capabilities listed in the dl_cap elements of the attached mblk chain. The driver should then send an M_IOCACK back up | the stream. Note that the actual content format of the mblks is | Contract-Private and could potentially extend the granularity of the set operation. For example, in the case of DL_IOC_INFO_REQ_ESP primitive above, it may be desirable to increase the granularity to control a specific cipher or even (cipher,keylength). | The expected error response to a DL_IOC_INFO_SET are M_IOCNAK with | the following error codes: | EINVAL if the driver doesn't understand DL_IOC_INFO_SET; | ENOTSUP if the driver does support DL_IOC_INFO_SET in general, | but doesn't support the requested primitive. Note that when if a driver supports the DL_IOC_INFO_SET command, it should send a M_FLUSH prior to response in order to flush any mblks in-flight. This will remove packets from the stream while the driver and IP are coordinating their processing. 7. Reference Documents PSARC/1996/173/ "TCP/IP support for SAHI3 hardware checksuming", 03/27/96, Bruce Curtis, brutus@Eng 8. Projected Availability Now. 9. Cost of Effort | 1.0 man month. 10. Prototype Availability | 03/30/01 11. Prototype Cost | 0.5 man month. 12. Cost of Capital Resources N/A --- DL_IOC_INFO_REQ_ESP interface for detecting/enabling lower level encryption capabilities. jack.bochsler@west.sun.com 01/02/02 COMMITMENT LEVEL DL_IOC_INFO_REQ_ESP Encryption Primitive(s) Contract-Private DL_IOC_INFO_REQ_AH Encryption Primitive(s) Contract-Private RELEASE Minor. BACKGROUND The Venus project provides hardware acceleration of encryption algorithms. In order for IPsec to exploit this capability, it needs to detect and enable this functionality. PSARC/2001/070/ defined a generic request/response mechanism to discover and control lower level hardware features. This proposal is to describe a specific implementation to detect and enable hardware encryption. PROBLEM In order for IPsec to take advantage of hardware acceleration in the Venus card, a method must be employed to determine specifically which algorithms may be accelerated in hardware. PSARC/2001/070/ defines a method for polling NICs to determine their specific hardware capabilities. This proposal defines the interfaces and behaviors to be layered on PSARC/2001/XXX/ to provide feature detection. This describes two similar primitives and data structures used to pass information between the driver and IP. PROPOSAL This proposal creates the following two interfaces, layered on the Consolidation Private interface as defined in PSARC/2001/070: |-----------------------|-----------------------|---------------- |Interface | Classification | Comments | |-----------------------|-----------------------|---------------- |DL_IOC_INFO_REQ_AH | Contract Private | | |DL_IOC_INFO_REQ_ESP | Contract Private | | |-----------------------|-----------------------|---------------- The following changes are recommended to the existing DLPI interfaces and are to be incorporated into : /* * Note that numbers correspond to the types in the IPsec DOI document. */ #define DL_IOC_INFO_REQ_AH 0x02 /* IPsec AH acceleration */ #define DL_IOC_INFO_REQ_ESP 0x03 /* IPsec ESP acceleration */ typedef struct { t_uscalar_t cip_version; /* interface version */ t_uscalar_t cip_nciphers; /* number of ciphers supported */ t_uscalar_t cip_flag; /* flags */ dl_ioc_info_alg_t cip_data[]; /* data */ } dl_ioc_info_cip_t; /* cip_flag values */ #define DL_IOC_INFO_CIP_ENABLE 0x01 /* enable */ typedef struct { t_uscalar_t alg_prim; /* algorithm primitive */ uint16_t alg_thruput; /* throughput metric */ uint16_t alg_nkeylen; /* num supported key lengths */ t_uscalar_t alg_flag; /* flags */ t_uscalar_t alg_skey[1]; /* supported key length */ } dl_ioc_info_alg_t; /* alg_flag values */ #define DL_IOC_INFO_ALG_ENABLE 0x01 /* enable this algorithm */ /* alg_prim ciphers */ #define DL_IOC_INFO_ESP_NONE 0x00 /* no cipher */ #define DL_IOC_INFO_ESP_DES 0x02 #define DL_IOC_INFO_ESP_3DES 0x03 #define DL_IOC_INFO_ESP_BLOWFISH 0x07 /* alg_prim authentications */ #define DL_IOC_INFO_AH_NONE 0x00 /* no authentication */ #define DL_IOC_INFO_AH_MD5HMAC 0x02 #define DL_IOC_INFO_AH_SHA1HMAC 0x03 The above structures define the payload (dl_data) area of the response to a DL_IOC_INFO_REQ and the input and response to a DL_IOC_INFO_SET command as defined in PSARC/2001/070. The request/response mechanism is initiated by the upper layer software issuing a dl_ioc_info_t embedded in the ioc_blk as part of a M_IOCTL issued down to the driver. The ioc_blk will contain a dl_ioc_info_t with dl_cap set to DL_IOC_INFO_REQ or DL_IOC_INFO_SET. On receipt of a DL_IOC_INFO_REQ, the driver will allocate an mblk and chain it to the b_cont field of the request (as per PSARC/2001/070/) with the data referenced by the mblk containing a dl_ioc_info_cip_t which will include information from the driver based on the capabilities supported by the hardware. Note that if the hardware device supports acceleration for both encryption and authentication, two separate dl_ioc_info_cip_t's would follow, one set to DL_IOC_INFO_REQ_AH and the other set to DL_IOC_INFO_REQ_ESP. The elements in the dl_ioc_info_cip_t have the following meaning: cip_version - set to 1 to reflect the current version of this interface; cip_nciphers - number of encryption or authentication ciphers supported. This number corresponds to the number of dl_ioc_info_alg_t structures following the dl_ioc_info_cip_t; cip_flag - not used for DL_IOC_INFO_REQ; cip_data - contains a series of dl_ioc_info_alg_t structures which define the actual hardware algorithms supported by the device. Note that each dl_ioc_info_alg_t may have a different number of alg_skey elements. The dl_ioc_info_alg_t fields have the following meaning: alg_prim - the ESP or AH algorithm supported and described by this structure; alg_thruput - approx. throughput in Mbit/sec; alg_nkeylen - number of supported key lengths; alg_flag - not used for DL_IOC_INFO_REQ; alg_skey - variable number of elements reflecting the supported key lengths for this algorithm. As described in PSARC/2001/070/, a DL_IOC_INFO_SET command is used by upper layer software to enable the capabilities discovered by DL_IOC_INFO_REQ. In constructing a DL_IOC_INFO_SET command, upper layer software will send down to the driver a M_IOCTL/ioc_blk with ioc_cmd set to DL_IOC_INFO_SET. Chained from the b_cont field of the M_IOCTL will be an mblk with the data containing a dl_ioc_info_t with the dl_cap field set to DL_IOC_INFO_REQ_AH if this feature is to be enabled. The dl_data element of the dl_ioc_info_t will contain a dl_ioc_info_cip_t which will be filled in as above, except that the cip_flag flag of dl_ioc_info_cip_t will be set to DL_IOC_INFO_CIP_ENABLE for each feature to be enabled. Additionally, for each dl_ioc_info_alg_t (embedded in the cip_data), the alg_flag will be set to DL_IOC_INFO_ALG_ENABLE for each specific algorithm to be enabled. On the receipt of a DL_IOC_INFO_SET M_IOCTL, the lower level driver will walk the chain of mblks and enable the features with a corresponding dl_ioc_info_cip_t with the cip_flag set to DL_IOC_INFO_CIP_ENABLE. Note that further granularity is provided by the alg_flag and only those AH and ESP cipher algorithms with the alg_flag set to DL_IOC_INFO_ALG_ENABLE will be enabled. All other features will remain off until explicitly enabled. It is likely that IP, after receiving the response to the DL_IOC_INFO_REQ will select the desired features, convert the response into a DL_IOC_INFO_SET, set the cip_flag element to DL_IOC_INFO_SET_ENABLE and send this back down to the driver to enable the desired features. REFERENCE DOCUMENTS PSARC/2001/070/ DL_IOC_INFO_REQ/SET extensible interface for detecting and/or enabling lower level driver capabilities. --------------34E70F3386422AB4EC150615 Content-Type: application/pdf; name="dl_ioc_info_req.pdf" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="dl_ioc_info_req.pdf" JVBERi0xLjIKJeLjz9MNCjIgMCBvYmoKPDwKL0xlbmd0aCAxMDM5MwovRmlsdGVyIC9GbGF0 ZURlY29kZQo+PgpzdHJlYW0NCkiJ3JdLj9w2Esfv/Sn6OLOAOxIlUdR1kzhIkMcmnpthNPxe xy2PMRmssd9+q1hvSmNfFtkHDHvq5xGLRdafxWJ/7OHP3dvDV9896Y9v/zj0x3fHwzQe87Kc +nQc8vFR3/WnbjzevT68+cuhO+IfGDCn47TMJ/hgPUzDaVK8OGRKZUPl1BXEcWOjWZ07kx3D /GH8yojB2sRbKiO5b21YINqnObVoQcV4MQL1sx6W7jQWc2uYiea0Q3WapWvsctLAnMmOYdow fmWUXaWJtzQQNjavpSxhaYQuqBBvjcD23fv0GXKJ489LMys46kV0uMSxx2mrziacaiuznKbT OFKy5ywoqUFcksN86jNvX2MX1lO02Ztqi0eptiDA2cmpxcT+zagLUl01qCHFaFlXYzJRjckU JZNyUraYeJuj3Z0mWX6wNaFuuEhK9xMwjbY/6DDYHDnLpkELI0RIKtId9j59LnyO+Htvy44F Hc3daUiiI/ixoyNIQao6GsW+iKYAx2JZ3eLsJLbFRbY22NMpi8qCzVOb4miUKm6WXNHsWywW eEO1UtP0wa6boyJsUIOO62FNkp9V9CBuBTs+4VWnG5qcTLc4SPXzdubqPQZTpjXZ0hiVrSSF 597iWCxqQ0sH24tTcYMWZVgAi1qS6H36dHsZ1G+8bVv+15vDV48T6PrmzaHvsGaigsmqe1IX dLOCtG9eHp5evTi/vH5Urm4/3F8/u/kB/vdRf8o5z8ebbw4wCEbRZ9fl6gx/X8Hf5/fPP9rH 4wTh4cfi7+7jPXq8q598exOPGR3Fz5brngXDeu1Fh4rJLlbEKQWcJeXBTpSw6t/bJovgQc9S Cj3JDur94EwuN1ayPfZ2AfiI+bRUN1rDk7+QdXIptA1+oeBtLjTybQeCBumBkJ3lyXuryIyd qL6xZyf5Bi2uEDKfANl679MnySevfuNt20Y4AXvC72f0AsJHjX4CGXuZdsdyGqa5b1S/ouwv 71n79xtF9wmLZsLFoKTBSptGtx+xL11oaT1cUbW3XShc5tx1DLhLDdAi61QRICNog39vq2+8 4YKTVRkCXfz0Gx7IHWC9FjZYr1lafYBEVROASlzDFn9cXA1VnUGcOdHtIM6NE2O9IBqUVeQk s7c88/cbsBW0C8PYgoNVmRLFs3uw/OYUwbYg9zG/zC6uEDSF4VIZPPu8B0HUId52Ox5UPA54 nTxYmKtTKBn12SJCILZ0pkI3muMpy35GWKiakNsAsz5ZohtT7zhSFRO17rAqMtgD3ZiixpYt whi+qLP6WjXh4tqw92rbsMyWN6CnL7dAUzgJ0kCToG45nxBNSU4tyOpYaC27kEK8IjzNQvDs UxaTWT8LYNu5U6LreocBPVlz8urFuRbpf358zWV6PHVzX/sN/O0Ff/n+TIUZRFvdQfOD0w09 xg++HrmC/tMZR3x/DSK/+uX6Ub76+gbs+epHLe0pW2vYp+oj56FelX1f36DSVrpPL3ZWfvsO jB/A+P1Ye8VPsL7jT8enz7rjK3pVFTrAZl/qRBBynqnkPHlog6YZI7H9eVdbt+vuqvZwZ96P 3Wtqgqjh/vvsNTUuGWfSA8588awHHE7aHvsTtMdyfUV7pl6DggjAEVg5oIFaDXBdk4thj2dJ 1wa60c5/yxZhCJ6rAbvScmCujSdXyPe4k8PZgJbR3IK72oMbLQi25RTGHvuCscedHNkNuIuq Qb+CsDwuH5q14NZnOOa+fhbAdj/Ieqyn7LP3FjxQTqNrvpLcCYrZ3WJbrn45Ay1o4WvACnfw YrIdXSs00zq07m8geZm2bCHFeEWo5EybKnPOmdJIjJNqMcCgRys3ID6dMGmkCtO2lc/HDutE wZblsdBadhGG8Fl4tvHesctRTB6PKJvpN8Kzevqw8PCNZ+WU0BI1af+hvOiORpC9qk4DkEsn NBr4J9RHF2KM/7+7Qtq283mY9BWxsRcvtJa/UPEsEd6xy1lMZv0ogNvNIDx49kJjycKDH/8v Fzmsy5flLWpVnvMuFwljA/w8INk2/IVrnlxZ9VTXxqGM7bAXyRblkdBCT70gBRGAI3ASp5H/ +SbArSAs79/fBDzUIE/4BTbI1vO/u70uVy/P+M/6ip8Rj+AdMYJs4B2x30rf/eM5/rzI95AA uDDo3dF8CsZrNO7ubu92HiJT7lAR8hBJNN03P8K7o786w1Nkufrla/r582N8kvxy/u3b+stf /aPEylhfyyQ9Bex8zNm1DK4U4TNjGKkamI3fDXhrTTNzfXPACjFy+JG6HmvLDG87qEISfI/B s/HpAIs4f4+x//wYY/71ePzb3e3H2z+eX65vfse4J0jmUMvYBOHAng9yTVpFW8pxmHu9Srsq pKakJRBR0vXDiKS1F+Qx0y/poUly2WNSEXAqu0xX09LCzN1RjSGAr/fBzSqMC9Mw4BfI6nkD vKCCJ6dlF1MMuE6uztZDgvxiPyzOkdlfjcRx9Q/cAr9oyW2AXueMblZh3VfkseyyhoEqc2wb KsALGMKCmX3MYUEUnKUievZ5ixnFzwL4HQ5q7fH6frDvq14hSYPoAOVHfPHsk9li4SO67IDK L9js30mRxpkUIerRnRjkRTPbQnJKjOjiicGKEHvuekSI4lryJHE4TiK9YCd+lVSnAawxCk5M hbyfIrotDlmyHIHXIppr2MXnQxfFyY5Hvz49IW88pLSTb/Q2fkFtA/QxvahtVr54lsyD+Dzi ogBzw0X3OgLvY53T29lu+eDE1DdqAimGHZ5VgA04LQaw2GLgokRys0re1a3j0ZfELYsWnaUF vbTA3p0qR6dJ3WaZnbMgx2SLvchyA8VrsuUxtUtgfWpOok+fwJDZ+pW3bXv3mi/cg6G2e9h9 UZPz4lw7pNsP99ZG5Zzn2nZZg/biGruocvUK/j6/f/7R9Vyp+B7txfnu431ttbQz8ieFjtPn K3NJ2CZZZSa2ylyk2lF122LWYhxhPOmdEmz27yozjVutgja9zKK9SWmByoUrxR4tnhgsnwd2 pZVZXWvVXMKN+sVCtXM9sUc7AzTQKnOxCjqWLfJm1Wki8FqkMrds8fnQpTLLjke/Pj0hb/Ur b9tGPqD8VAr6AeWjSD+Bjr1Ou2M5DdPcN7JfUfeX9yz++42koR0Bxws3G3N9mbWShhMJoeNv YHXUbDFfPFNdo2Zrj2kPcL4IM+9ynSYAz4E7HNysyhD56MMAzjItynyHqVHgZQdgmzQUUSNu loOhmav1MHQ1ZnXtuMaBPJbIPF1dh2Me3tiFt7VOGmDRhiU4WRUlJRLDDqdRJmqhrnboprB6 YYvQB0+RWM6C35DgkHr6LIBtdtQtPHrxMnmwFqPbsT4IV1UAsWUS2Ot0nLg+of8W6G1QSgvs 0umURppOIdLsdbrDRVPZQp+dElvWGJsFiDLJ2apZFueOs1feDvdZMhFgkrOH0wSQOZwWaaSJ kbddzodsYXUcgdcjWmvZhRTiFfVJJoLnkLaQUBkTQDZ0pzDX9S49tt7Wkry6YF0+v6t9yUuz P7y5PWuXkk65X7Ba6/f139f1w7e1sv/ddTRLNzWlHTuZy3Xnu5qn8OPZF2o8HBX4sXdU5oEe lCJK4otnXwO3WFQBEfiI1Tm8PeiDMjr58+u7xtss5n+pvnM+JIQtDqrpFoo/US1/vrpLxoLf kF6fd/rK27bRtbvPBcsP9q31qTPkJA/HetwGyM9ArbR+XAcvHTweprofal7MC191T5wWUGSp HmHd58yN3XRqZSPPmw43dXW2G9VTZE/s4E3wVe4x8/XgdfUp15w8HErFc8XvtZBOQxX0xEIC nNIeVlFM1KA2KKUQ3TZ2Lc9zirZMDeEFDysjrsXNjijTzXmLSWeMds8LpNdjRIszLgFDUj+Y 4vqAFbeCHADgWALyRDV4wzovqiXY9dccqreTRhI8rIy6+TT7DsrsafRo2764274sYfmELs6w BJK/Jsv79Gn16V66jS2rC+qtGn+wwarloL4nV1EkoQq0FMtltOnIoRyDTcfN1ZnZOTVh0hgV ZsdVhKXX8c1eV+BtOrGsvEAWRQyQhddxF8lKE6ecCJneMLOavJlou6s7b1v36MerrjhHdK14 mzcF0xdsjpwlFNEC8pGygGRT/8V9uezWkRtheD9P4aW0kNDNZpPsZeAkQJCLkcxyEBzY48kF Pm0bgpBB3j4k617kkbLIDJJsdPqDuotVrL+KRW1Rb7/KyrHYR9ktI58tPMYX5bPV2X5H+bTM AVIWtnoNWKLBQNtmnrGDqQe0w4rB91kxG2UAWpnHrbkLxs1zi4cFZIkdsr6igMDKSb2JjBIG KkUy7JC6k36kZhzdM5oWPfWPSE+8pyhl2qlm0jyj/6gbh+yHdhCPYdppbVHnhBPVXyhuvfmY 3QILoX0vU/aHSx+tv3x+VmNyStmNyR/qTKxG5K/ycgwl9ZfJ3tPXPnY/DRM0t6abA3QOMj9j 76P5R1AdbANB/4ruGSalblw9ytxsvj+5eckMoYdmHClGPIK0X/0cVHs0lKXfumlZrFCzZKOC SbexQx/L7PphxvQb7WZyehxG+jTpHTzlJ1l5oI1aqHvusRV7MSh8PwmDt9hOKSnaok6fyuqx 2EfZz3kthHzgjbPJ9seqbK3c5U153Pa8ukI4WyVcP2E5PA8iX0NSKs+hZcupfK25CWu/8tQR vQ7ZTV7AV809I43bHk346CN8W89CBi3AMgZwjXbAGDMnMymH3CBhNdayV9zFQmEbwGedVULx 2IbTXUui/jUFI3/F4EdlXQCNdQUoxs/dc6FtTR6kDIyRk5FTgj5MOERayANEn1abTGTx0FWE +vp0dnWCber7awbUZhvdwgB7c9ToZiMOcaSAiJMXZTJCBTLucFB2+x76QARWDcgwZ82ITqun Set0woVT6WFNSomexUcbACkTjJ2cZTYunLTyJgzrJQ87115yQGsoLdI9hZi2HeuDs5KCB4oP teZZuWT8JfVxJoxlnTabUPzGwHK7L0PwS7MkQ8rHD5femf/59QfszfFxyWtsvbn999r++ekC 3bjqtpsLb2C50vyvth5aF89g8PeX+4d895v7hzXevbt/2O/e/uLtb7mXc/uqUa6leaX6Xh8S w7GPrY4mt5osGtyWR/qoenKEZur6zbe34l6P3nMk7r/30ex+uesz2gXjnJ4569Er9sVDZ63D H4imfQDArbyxqqAJJlUWM+Yac4DnMHhgIIi0jBmu8xZVUl6MWKIKQbhBgu1Wu8NVPrB4bMPB qkdjoHqxDMUmy0ILmLGqqAmuVJ0O4qNEo59xfWkH+B23A0kPujDhqCIQVLpIel+gOwys/DXB YLeQfBrLOvlWFv01AyoRRu8ptR7x4lm17n0q5SEL+ao5q7Nr5JWGluH5oE7nAFdQku4fiqIT XuDIh8rapZTliHSQ6KAFyXpmB43vJGAwxceWmCZOj2p+Ei+F+QhxQCWZ3HOWc9MYEZHydqMP E6aBygPGiip0qPwzzpMoKSPGrM6ezSt+YoDM3WjBtyWZ+5WHxyfkq+Y9KD1MOKvBeeCyUAL0 E+5rc8AAra7kWj/7+dsv+qpC+F9tvJwPdGHCu7rIzDiwzA3AHqHEDbzWhCGlxqBOvpUFfGBg pvWtNHdR6/Xn/2Pc2Io9ICrzgZDTlA/ueB5CVAr3/NrAAcZO1gUbFza9UjOuF4tlvn86yNRh koci90Rj5uefM5THJpyfYs64NZ2X/kqbzutFouCE/uW+3H1/aX/Oj3g3eaiXk1jVUS8n8zn+ 6R/v2++V3l8fYygJLjPu1frwQ3t4evryNLndrKWriW43AZb75e/qxWa9u9T7zXH37i38/uHX 9w/p7t3lT7/q//yjvu1IFwtLvz4udGmtaq0SCMumBhHVXuolJ9SmGOG+A4/trdgvTWRleuEJ uX6Qu3l10es3nEuPve/R1+GyE4+tXaZq8mK8fd3Z13ZG8wAY693rSHLAN8YJoyucGU52WGOk PgKQaYcy+BljJ2HzONHabU/rJq2JzQ/Y4wM8RhbfjON9dWXu/GYPR/uQzTdGe90bxd1+4wkG 6C5k3GHkxmrNncS80egObxfYHxCi2UPRwSEa37TruLwMeN64zZTPI3/q0I96KIc6F6QXDkC0 nkubo0WAwFfNpIkcRjyg7CjLEw5FFOqwH9zoh34mH5RUaSolrsEV7cjA2AIojAlvIuYBI/QF 0rJnicOGyepGg6xuXkDxSllMZcIYD8lzwknkP+AGGiFnHKIvUg34vVQD5ZHcGRhkQNGMuCfx xiNuFhWPZx2LDlWqh8Tg7Vv1GGHBm/pZ5WzS/EmhbRywB+rH+3YwLnftSG3P75/ff1d//vz6 OTC9c4GXYcmPVIWZ8KqxqLIaucD9EZf0SMnDpRzCUlJr+PlPdC4o56zv/60nA282an/Gom5P GBydDJ5fPSk4Gc68yZ1PLL7qUPb5pt5DN/TqtNMnwvVYdpgI8a3+t8+Cn//63H7+hi/XiXs9 zKswQ34vz5//8uXyPFZQdSfV0kltr3oFLX3YdiUUWyLehBRxlKpfbcxXzZD5yrUOZwyjVt2R 2mtH3h/p9TQh6LDoisMoo5c2dhK2ELUvM0ZfeiwjZ+WNp0T7EJoCPEskJszuq1irrsauezZO SJ7E0nRkeNORCMPqMRvnEPutVVx3mNk1Y+0klpyBN5Jj8HbGavkBcWdiMgIAVK7byLp78v05 GLei8JLhTx0OUx7sSS3r9MLxgtZrr6XSaFLvKMo/8PJEEpjwrsTuMcJNl5ZyCEsp6cPnov0a QVF1utW+tamkeyw4iqGaHSvnrO+iZzAoguYFUMGVV63YgWlFkKzHjInDxRwePHpZcyJh3nyU MOUKhOCIgkWFOla+WddFopwMa97kzicWX3Uo++wluuUXJyAwv4UVJiBUQUdO41aP7aVY3tS+ eqQ9RNMOV554jDWR5Kay1trvwAVHHgzQYcaJByXqWZy1sYhEweDJkuQFiMkfzOuEo9KkR3W4 TLDwRGTNsUTbBhYlUcVQUZQ7WM4RBY8S9Sy+2lBYspIsa97k1iceX3UYXpyIUErt1ToRfXf3 AeagL5+fZQhKKeU22aj7wYd6GVB3AzUxxVBSfxlfvDx97XPS043BB0sMC6f+zHt7JrHgnJOV GpGjavYjJt3rJ8zdOY14qO32pIRkjJ3SnNWI1g+ikVmaW5ryIYU3wWCOBscSiw2V6xAN8lEh CwjzcLMUy5uOR/jfaKE3j9nD1iV+L0dHVu1+KZJq9HbEVQpzgrA5eLR4Ft9taHK0cLKdfasO Jx1815Lk5WadhlK3mur0x1p6urSWN9XHPa+uUs9WqtdPWK98/XC30FgSXNUK5vLoN8iw0Yk0 XCTjXu88R799nRrau0vfrGqr5/JbKfkt1Vvtm1QHvT28cNfZSot2T3hst6825qvmPjg3roWk OLS93LtMBCB14MIEu6+0skM59q25kzjREI++jJzEcQ/akwF3ijm2TfWs4rBhdleVweppzU+t CFmAueVs6TJzQBeTZaG1PYMvx2FcRYx9riIvLCWpI2PsJIbkoSeSWfB5xmrpAWmPjmKlAMyO +7jaHmoL52DfysOLh791SKH7sqhjdCq3T0I0f/Q55GTVHziXMGPJkiImvAVOxYBbq0eOxSOu pQrhwEGFCoHuAiT8BVsgyX3CSu6O+oaI+B0r321oIv5uT7RP5gVXLXjNuN4RLMP6oHCPCRON SzuEpZXi4XNRPCWKRD8wJRqE5JH2AkXumJ31sYjIOZPOvk29Fwa+7FAS4UW+9YPm5j0JzKew wH2RVALMmU5hheuq4k1ttkcqATTucOH7ojUnot5ULpuoB86PUkOeEhypJGLP4qsNRUTc7Z2s UjTPiL5QYicclUo9qnNqgrCUEi18zqLlRJA3YYExhkqMEwfLeaTYUaSeyVkfC4tWMuXs29T6 xOPLDmXjJxMYSSk1i3UE4/vNz3ZfMh36tVMig2ZoVMoiScSoz4wJJ31mTPhQh4bHond9QNaU tXZKG1dzXj/RRs7qCBlZDXbmmZoUHR+OJQwbJVdis8aHCZom2GQ2WsqU9Ywx45f76Z5UIA7t wYIfy8GSVetfypSjjmXCapoa6NDHjmcKxMcpxw5n35o3YvFKwlcdSrpu1u9ethbbf+QKxYUZ a9dszbJH3AozhzbjucKMO9w0dgi3fbQx1w3aIgynCe6vgjAJ9zUGzNCyyLbDnTfEmjuJ03bg F9sOB9nIiZfzhDfWWLtyb86Wla82lO6O2KvO5AAnGZoX3Ii3MGFaL6/GHcbVOruOkTRXjLWT mPOQV7hhUJryqreWkELNi80asPLNut7XVxbOwb7NpM8zvuxQ9tlLlErhxtmB5o+I+qTeQlo9 8O6C6pjxv6gvvx85bhuOv/uv2MfdAruZ0WhmNA8FGuQHGjS1kzhoH4LgcLEviZPbu8vZjpH/ PqRIij9mzocWaNo+eHc/Z4kixa8oiq5rzvgGz0nVG3HkssReBGzSsbZUyX1HLacod4PHSWPZ 4qL6WFFnlR7YBOLjVKVXe2dRUzOvOFrlbzDFwvLZ4M5IP2BnzvQa2RVzEmi6noSWRfJmi5M9 KpabguQ3bxSfk8DGcx+YnpOWdmvcKCSKh4cF1Pxs3BSyTx2GoZ3ey+8u6lXx290VXxb51OW5 Nm/4v9f4nz9f0PUAiqsG006WnHFvwNoRL5aBTP79MMGVchz2nx2Ofbd/Bp/7j55++De5YaRY Sc899Hh5Tmnh3iLVi3yy6Vx3yH2mwuYQx2Pip2GohfD5wxsx5s7swqva7R66fW17LzjqB67E ET7htn3kSszQbnF89agyX1u2x2vNw0nLyJpM5VxharvljLWyghFMxhfkRcwP0wZP9p5Z4Uid lRSOwOq7C6yVETbX6oiaF27XEh1Ny7zeVDxnczwj+vockdfWysHzW+XQRJE7W5z1WK6QN4Nq Q0Djqw+llYqWyWDcZz7qok0NmDYv1LHDLoHvU5DHtr6XWs5by8esaV56KCSek94+KzT6db/Z qtEyzVQxg7fFinnFM361DEeUzpS069F46YNQ8ZK51u6peeHJNUlbbMQZyBzTDeSVjFTrdFVq SwEdJE0RCSCixMrSDKy+OcdVmS0vwbpPpMsxj0xrJzZUqWX3fW1ez903J7/X5p6x2O49cmo9 UF0youwnLRWw1+7bWfsvlV3jvI/t/7XwarLYnQ02DwZPbTNYzJEfLbyaTG/e5T4Kg2cG1DwE iee54OKPVd7/4c4CIxjNgUO2rs2LXAoUb8AirxuqvpEfaSTYXKvFal6YneHy2HxVHkwpijif JuOqJ15Ixcuz/5CuwXjqA/nPdg0P99YJx2B3DY+Cwh327aHsX1zgx/klvzSO8NSY+4xPje0+ /P7XS/y+lvH9KafCT5MwFH5c4Y/7+9v77bdKylhg5K2SaMGPP4fXSb+/gKfKsn/2EX0//fRw nPbPLr76pP7nl/7homUJXiy1isvDhat6HuxT1FYSfqzAGyybtwshjq+SgJcJvTSfPxk6TMjc 5dMEtzfm9JjmjKKlwgAZgJEYJnylropwWtwLb//V1S9vr16/OXz90xMwM+ww+h63sG5Dz2Ne 393evL764JO6fTAUtnoZYcPTqY7VEU9v78+X1zgEN6TH+3jBcGE7oB+BfRimzKeqq4d2gN2n 7dHBdTPSAAybU0+sAZxZcEeGqZxoH3550pddX/+Y4I+gqYQphU345+6m6pBFiKN2oL6R5Id/ wrpZdwI+3j3Z/8Suf+mnwRxYej3TvmsvX/xcH7Tf1c+/HI5p/w53lsQpJRpT1VVJw8wRZ4re 3oGN3S59kD6Abd89f3uz++L+MO5v7/Dz1dVh2b+5hA+k30jC4XQNHeoljSJkPjQfw6H6vB6C z57BTxDxU/j6FP4hVhUXEDEdkgrPAb6oS8x0ouAkgvGRTta/ZO9D+P6rHJB2naWST7idHe4G Xmdlwl/hOhs6KDy7Hs5L7RtxEgTGfA08YHuBXAXUOFfIda77TRVVzAYcWmNpDJ0F0tBTN40M 9W6Lh6kttsIaKWKtAZHFTxNA9cOYgnKQajVvpg1XT4bUk+3IvBaOzyuEIrUkdTSgrIXFyZk7 C7ftR3uT2Uiyv8K6NLZDxYPxy7tdl5a555VZn72YW5kYMHRXHGw/YWV8sLsi2wnLMesC817x 2uJkcrnmJpu65BprxypLBexOTRnWmooUIsiFzcFRQV6MCteYrCgjd0bi1neVJhk8S+7bApJD 8cdwMtqLOGJ5VfcCsm0jRZrfpNg2W07GBpvlAsmpYTlGVl99KE2eLRnBvMtdTGybWTZc2e6h 5KBCSFjkv6EGp7Y7dxe/UpPz+lC0AbrZao/spBsDP6qBPx+O8z49ULt7+CxyVobpgdINl/Gc TOkmvra8SOmG07Pm5WQEuIG2vgeE3rc+GNmVgOKJKfc0vx2mvnC7Ic5scDKHbYuNfleYT8Vc AAE1Fh9qO3psrh29Zt5wq7ulbHJn9LxmcohL5oq64kJxyJ7oQaXpemVwHsWViEWKCEWywbaa R+SdkjsmcovEhamXjKggWveyiaKiwQFN0h48x/08rt5CL17d1e6p7F/Cv8s3l9/A17d8it0h bgO/h3/Xlz/IYR0GDKuH27ujWGY8o/1QxQT+wUsCFTpINHY4bUaPT7R+THzXW8TxHU7s6UGD HbhWh6WAqaX+P1aHqcoqVgconrABmS4PnAIvAsLrhiOd9aW+VVdEp2OhZn+FNUpyJdBEZZgc cJDb9eLsnAUhpuYf+LD01iOg5kKJ1HHRg+ccHIyA1j/jPLshtiAFQz3AYloxMZFpOikNTXoj Gf9WjqMCrI2zoGw1L79G8QZfS56y+JbNbjONosts94BRHXdRVR919jladln1CaeBDnTbg5TH mt960fW4/KaU84SONikTtpwCjpNDFaSHmWs4GfXENlWfdWqT51h0EXB4jbIM/DnQwvcaxeup zm5iDWic17iacsnSWQQghgXJB1bOikaj4zUOKuRA82lS5x0sWtysnSZtyRL7J/tNdldk9OlJ /XHONrVy1oJZl+KQfhrpqW11UOtcn2+PqHWatbAJNTUALqawRSx0w3HK16jS8bBINz2taXYl T+w0ZZdOFwEHi7yoqmFPs7xeSZ8BjUfqbJNrkXct5V8MsyQL9++UVXGJUdah5KwoWf/SynVV JM1sipTdZRcizrJM9Vcxmz1lKq3UdZHUWxdK0yvnIph1iQtJpZGe2l4HvYKsUxG9wq9NvVK8 rboO0mdTHgFno8FBj7z7yXcIGfTE9lR/w6TqAw9nU0gjmpOTTSEv9Di0QWrV9DhE55siyUqr FWxT6pOpl+43OyhVbYXJ1DFP5ppaUdHXrrXUtCpZYO9kT8lyILn4O/9bvXGuNiXWrASDLoEh uTzJ0+7BdlwPKHbj1IlfHo7QV+PHDxd396/wx5n78CM0ywu4WRtxO+4Nfv14//bu7RszdO7z eujNz1f447eK9edNnfL1n+xz4PL6h40uH8esnXwNI9joN923MvDhMf238mQIB7OHy/PRDl56 040mmNB2thGTVvUNKprQQNqiOkvtwKLn1o0tthfeFi9apTfQXCkBbRgmxnai2Vi7ZJpxw5M5 zFuczT20xeZBFwga3mICCTibwm+MtRMu+dOXR8Cs7XSgJBdXtrvCqF66ENqxl/QGy04MQSgy z5OkIiod3gh4q7z/Dur7hKOa1pmvLRt1r3DQO2sDNQb/m9dQrfNMFXsatV1HNzbYrhQxSxCk 5sjWTxuGCjqNeg0ks4Dhzgp0xVm67mwdEhxOJRt3A47mhrDmmmYlDeLNGu1qEQe9tboNVl99 KE25LVnRvs+uSzyPzGs3NpQLd3T/uHLBTKuPWN+IVSLApi1GnAIXI5mI48nEECi78qfGVL4Q wWTlusHF1mbLOH2mLtPth8o3svXdhqZyJoMtY2JfyrMsLwLa4Mmqe4N7o7CIo15EG8i+GLXT fFW7pE7clQSw/TUWq+fI6p13XvUt2Yz2ffqDNnisJ01E1DjIf35c412Po7Q6E6uOumTSHsBd gxGHkz4UIvEaRt40W+XdWy33WR8FzNldAh4HiaCVYsfWURuHapkMammWBQwnK87ewSD9fLbe KHbZ+eowN1e8OVVql4wjHnq3UsQkMWQXtanR9rnTWDUrKYr2fU5DwnmsJ93wB58TeIahQYEH xXveEqsGPb4gVgP+kHfDe544Gy+II1xm+MZBUZVxpjkv6/iLGuvtC/198/3tRcVKdxcUZkIV zfjigNQN5d8xcUzQEKaCxeLUL934uA2/72gDimxNI75Aocfd5WnCqwdyeMTNTbR5nxygHu0/ OxyX/dN/fPj56hU1dtC57/qpPmKxepV6VYTqNSYsg/DonFCSNIfoutJQsfbliOBTwNJRjR27 an6NmcdOAXq6TMgBT+IAOGrtnIUwqNH4sMXzpBGsuTQ//O+ZOp2xowIZuAVgY6tOiiVwMdcW kc0KyPqZisQGk//AQ9rkJPamNU7UxokbAWUnna2zcEsYuRKwJZsCsWh0Ukk2JyebeCLjsQ+o +tVmn6Nhp4aglN8pL5edPGIYCr8KS6gEmvtl0bfoDrFArVSq/oMQgkp9+yaxc3zsTIu6qPg/ NbGd+Dj2yDZPyEisgnnIc0np4XPecVoF/Zif3AMqKWiiGVfpMVVkxiLKEWIBQ5RLi5v0vuo6 oLi2AtDtVgLzqCfXaE54JBVEHGROq0KPbMH6s5jYxeABCcOBcc8Sbrh6FA1H7O2ROUH1RaqW /VC1JUPDQbK0IhvWXIv7QKjGwSdW2WL3R4O4LXnevMt1FIIuDWh5OGn59XpWbfnaZrlL/SpN 8zX3T+lGN5227tJtpzl5Ky2LNz0TPJmBzze363X/t8aTGuHwYcV1s6pIVSt8YaZuknHjkkt/ Onq+W6SLDTSbhJwxq7B0APSGHMsJsyhbHKigAlqk/iBWX2LO6gvmjRd+UU/YHsZInQyO1VVA 9UTVVbZbcSENGkvNkqE+BSNfdcV6dK2eyAjVncNqCWkK1n1eQ9KxdWsD+Wct9XkAy6VkU+3X Hy861X5L/x7fHu/Tnzqx6vQnJYSFPP52d2lKHa9uddZMK7VM/29IdBXX5++BHT1umfPjFitu TPd7NRS9HWVLul7BC3DVR7ovmRxWvUxgpQhF1Zfqw5O6yBkkO0elfXAh7KOFkMaYFkdzG6g+ w3358gpI8dpRNKZqKcWUOk26OhgGShAJh82hOJIDAMXv3HFQQrWIxaknamps6aiIbEgUyJVi zWRxFEjPPnecRiEL0EVfosDeI5p1mQ1Zl5WecN1Bs0snLaIc4VSwE6l1Ip1MtTFKQltc7WoD 7bk91rADeaXKRkh1qQ+lhNAieYlkSuTfHBmFDV2KlaMmGUaBEymvQdw//Zx3HYyKfU/VvEkw bYT+cMXivEG4cD9rwJ1Li6CGZdFCdbh4MmgJCpmTNZ7sIoPi0hfDNlXRze1cUkwvO16o9OgU sjTv2vwkZWun7Qs4WtoD9TpzFh+e1IdJT7ZCettgbvJXW4MjvZItzvZMBurraYoeA3L8dDjI U2zh2YRp4EhqbbEjtUasgZR0NtQPfAImDcEkLHuhYuRLYqy5lTJhwNlFqwEtIhcuBIz0OcOc 6qAC2eXJrjuIOIl9+OjhzF9A9nQWuhBNpOEWqS8F2rRdiA9P6x2pA4ag4RT1wi94g7t7pD0N 9Gg64vgoeGhULOEJhWHgQrJr0ZqYh83e/oZ2C4HsQIK4bgmhRXMTYScJBkR4HDn0iFQ4s5y2 kFFZ58mu9mxglvrr8kyQBub7a55ZX17LKHvYl2aXpmAZk/1sm/48vb6/vMuU++VTWPD8s3x/ /i5Yfj7XhTSiP16+n0zeeY3+P1nMX7Jq9L57OPfKa/qHOn//EWAAyQiN9AplbmRzdHJlYW0K ZW5kb2JqCjMgMCBvYmoKPDwKL1Byb2NTZXQgWy9QREYgL1RleHQgXQovRm9udCA8PAovRjIg NCAwIFIKPj4KL0V4dEdTdGF0ZSA8PAovR1MxIDUgMCBSCj4+Cj4+CmVuZG9iago4IDAgb2Jq Cjw8Ci9MZW5ndGggMTAyNzEKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtDQpIidyX S4/cNhLH7/0pdOxeoBWJoijquPEjcDaJg525GUbD73Xc8gw6gzX22y+L9WAVpbZPayCLIBn+ 0mSxivVnqfjj7e6Hp67pm9v3O+ebLv2T/riub4fQTG5qZ9fcLrsu/3L5sPvhp5u++fDn7ti1 XZdWvdnR4Mtu//iX07Pnj07Pfnv6/HTz5LZpfr/c3d/9+ep8uP1j9+R21zcfm93oGz8MYHbs fHP07eCay7vd+7+ln+GftEc/x2aY+jb9npw49skZF2mSOOL60Lgww6QlryA4J5jaCX9My5BH t8npD7KLm5zMnMV4galrPVjKPhhgHyAcY2ZhhsDEjfQDsFheAQUU28mtWPlkHc6bi7Fl5wbf pqSKcWCylz1RnO0nrqGDNLFbBnrZ05pZmOVcgX3cZHEDtKe4HCgDBTCYgIm1zyYgdK6kwlrW ebMZhWkG9AkbtfbJARTr2PZuU6spSQPrAOSHfNask1ljbLvI+qtB5GfGZF9JEdcVKSavvbox wLNktganlGhR+WOdZSGiqSJENs15Yj8UO5aeGTvMBRo1QBaVDPPCokI6TxbdGofAWbZAsbDm Klb+addZcXzi1q5Oj8kbLYn15iu9+W+obXCh7Vltk/BZM2c+iU8jBJUwVBzlrC3QOeY99Zj2 K+qjdUV9XhKIPmzwJAKsQGnRQPHNOs5KRDML513MKva6JK6ZtahGUtBjDWRdqdIrTcox8+6U Bb4ma+xZliuIWpM1e1eHQPqUnFibOoEms3mWHpfj/TF93ztQK6gQR/kMBgd2cguRmoUX+9en N4dj3N99fji8vP05/d9j34YQpub2cW4lIk07xP0p/fs2/fvq4dV9mexdDHky27vcP4DFS56S 2wx1U/A6fb0yR9fOujIjl8ocudphdVtjkGJswbfyTTFjsq8qM65bSgWteplZepNYA5YLVYo1 Fn+ss3QfyJRUZjEtVXM2X9RvFqqNzxNZLHcAF5bKHEsF9XGNdFh5GwsUC1fmmot/2nWuzHzi 1q5Oj8lbnqXH5SCvKN/FCHaS8kGkX5KOtU67JrbDOPWV7BfQ/fkTif9hJenUjiTDMzUbk4Ok 1ZJONzK5Dr+k6LDZIj5rxrqGzdYW4xnAfhYmOuW8jQHaA07YmFmEk+deu5E48LYg8w3GRoHC NkBj1JBF8bgKB1wrppbd0GWfxbTi7Aewj5ZpuxyHYlpejSMda97UwCwNizGyCHJK2IcNdp43 qiFHO3SjiZ65eKidR09Kzoxdk2CTepxmoBy21W3o4FNwvRaDWZ8fhIsoALlkMrHWqR+pPoH9 GvBtEGMNZFLpFFcWnSZPg9bpBkdJZQ19UEqsWXysAmBlorFFsszGFQetvA3uA2fCwMh3D7Yx wHsoLeLKIkY6dr4ffITZsAWKh7VWs3LJ+Mvq40wYyyZtJqG8xgAf6EZhzvHOPbTepSV5e4a6 fPqY+5I3Zfz5/d1JuhTXhn6Gai3z83/f5YkfcmX/l+po5m6sSjt0MudDp7uaF+nPy2/U+HRV 0p+tqzIN+KBkUSKfNesauMYoCrBAVyzvoceDPCitke9f38XfKpi/Un2nfLALaxxE0zVEfaNq /np154wZuya9Ou84S4/LQefuPkQoP9C35qfOEBw/HPN1G1J+BmylZfK5KPmfP6XBz2nwR9NB Xr6kW9r82rx42TVvd3OX3hdjPjIZnstG9DW8UXIBHbp8yyUVgXq/sa2VxS+gDs59UWO1qkfn b8rdHNOs0IM48t3s8muvupywFOvrAvOl1o5D1vxIWks4ui3Muhmxh62QqyWYrca5gk/Ojnnr 5J6xsBBCLGp3QN5uCmt0sqMd9xQgPjAtFj9tCOCS2IEU5zcum2UkBxL6aJA2ys4XzPuCWsw4 /0yu6rETT4yFhVAOH3ffQN7deY3l2GfVEMTZhI+o/DQhZJdKsrRNnVad7rlbjTk6o96s8as9 WK4Y+cm5sCIRRaAxllzaMV45kKMZ43VTpWhSRoswcY0Is6NCQ9Lr6OOfI9BjvLGkPEPFC+sg Ca+jRpOUxkYpEbx9wUBq0kOHx53N6XFpMPV60RXlCL88ekyHAukzY/KcJGSxOKQ9JQHxoWqL +vhVVubODstpGfkMrvVflc+Q2v+R5AOZQ+QsDOml0HmDjo/NjKmCqQHZEcXQfFHMwBnAUlbj AO6icTOGeERAlsQh6ysJCK0sXJvYKKPjq8iGK+TqpIdcjH01JtNFT3kR60nOlKTMJwUmzZj8 J91UKH5oB1FGctLaos6JJCpPiNV+2504BOYcrC+N+OtT7r7vPj+oTjqEqeqkX6e2WXXR92Wy dzHkyWzvcp8788uqyZbSdLXHnlxpsan2cYtUUH3YVoT1y1djbKaycTUsrbVZv0jxKj2E7qup pVjj7Er51WOnyqOhqdTbqqEuVrhYitGCQZexWX+WxfXZdPJXys3G12M20udOb5aHQCg7r2jg ElqNc2zRvh2iPGHcylsqp5wUbVGnT2V17uywnOf2XXDTTI9SkO2XpGyt3K6J7TBOfXURFrgJ 5090HR5WIu9dUCqfHGSrUnmfcuP6/CpKXXxqskFeyGfNOSPAcEYbPOcuH/azMKEWcBsDtAd8 YIyZRZiVw26wsIC17BVnsXDYBmiss8pYPLbhZNdCUX8fnJG/YvQjsb4AwPoGKKbl1TjysYYa yjUwRhZBSQn5sMHO80Y1YPSht8kkLh5WN0KtXiq7OsE29XmaAXXYRrfYwF5tNbJZT00cK8BT 58WZ9HgDBUf8UGb7NeSGCK0aKM2cNVN0mjwNWqcbHCWVNfRBKbHm4qMNgJWJxhbJshgvHLTy Nhj3CzWMcvdCBbyH0iK/U5j52Ol+SFaCq4HjI63VrFwy/rL6JBPGsk6bTSitMdBdr8sYfAeW SpPy9vUpV+b/3L+j2uzbbuo91Gb49Qw/fjphNU66zeZc0wffgiszfAiStSPU8QlN/no6HKf9 s8Ox9/vnh+O4f/T3R/+Qai4FLMXZR/BLVb7cJrp5XBc77t1Surh161pelEKfHZg6726uRd7P ueqUyD/m5uzQ7XOXdqJIN786/Zzv7Fc/O31q/1A2sABBijmwukMbGNTF2GK5ZRXQlxg9MOCK uIwZuekQVVBerDF6FUJhgIDHrU5H7vmKi8c2HLr3ZAx1XyzjdSvbYhHYYnWnNrDn+1mBb0s0 ekz7l4JA66QglPSQCxvsVQQFlS6CPhesDytW/ppgqF6UfBrLOvlWFnmaAZUIo/cQoEp89WvV j7kvlTaL+Kx5Ul+vNffctqzGM9e6CmgHJem8sCg60BOOfUisXQpT+UhWEPhTi5KtWRw0vrOA 0ZR8uIpp5tCqDqp4WVg+IhXwlQzVeCpfTmOkiFSOm3zYYG6paqBYSYUVKv+M8yxKzogxq7Nn 80pLDLC5KyX4uiSn/OiRBor4rHl0Sg8bPKnWecWx4wToEZ0rOGCAd1dyTcu+f/klX1UIf9XC K/kgFzZ4VE+ZLXYicwN4RiRxA98qwphSY1An38oCFxjY0voQwV3Sevrz/9FuDNF+IBLLB2EK mzxLxavBeaXwmr/VcKCxRXQhxgubWqmZ9vPRsrxAK5i4woQaYnkpGjPfv89QHptw/hd9xrXu POYp0J2nh0SkDv3uEPdvTvCf5S29To7peeKTOtLzZLuPv/z7Ffw98/y+9S4GfM5UU9PgHQwu l7vL6n0DlzOriV83Drd7/Et62PT7U3rfzPvnj/Dvb08Px7B/frqBP0/yhFv94imVzHX5Ednx 0zUpNsnAdYNqRlSJSQ8dlwqjxzcPDmGWzw8ntrL56HFTWjBl8+q5l185pxx/Pqf71YPHzwM8 qPr0KLv+4hl7+Ez3PoADS1qDykE+a86JBwbFrniC8namPVcY8YvAm1VIeyW3rLmFGWKIyp0V uw4tAkMWNjj3NnQkFeYTQppXqGKxoWZ3lbllN7q5TfdHzCvGeBOHuGaOBnh2mxwI1xQ5EcMG sifJV2NsYZREki9bHLxst0I6i9FFczbE2lcVCHnjSp5r61YYtWxocoUlLdUN6Of/cl8uO3bc Rhje5ym0HGUx6Gaz2d1rxwEEOHHi0U4wBpYlX6LTI0WawMjbh3VjXcijycLIxQvN8NM0i0XW X8XiBG361Y6TzW8ZpHSKZpg17tt6GzBnlXjEQvVUbAdk00bxNL8pfj5mqiOSgANOGugOJ3qY iIYjq7d+M03TbPBsOpAFDE9Goz3zihydgHvL32WAspaqluerbDkY4k090EPMY9PGB8zLReTd imojW2ftXlS3Eq1o34c3Bp8/DqhHP6j9Iqcd8thfqm+ew+U43cC1CuPvHr97VX99++Q9MGxF yVe4vlZKgkVR62jlragKezaiikCtiBju8ND86JC90HTh+f8TF4TZi9/qb+KCSC2ryx6Bgy+7 sGyiJsinIldD5CeuihbxaN1LJAqozQ2Ynkq5NOF98mTDhY1pmvNCjSl/hT+xJX348RF+/cQf Q+OfDvct9bLf6/jhh/f3j30W151B/hY4LUzjCctkyOMMwXmWSobPTpy1NL5Ypsutcr0RR4wn luuZVL30zJchOtXTgciuBGRPIDussVMQtmh9GTH7gnvpeTPeRCpyDgmEH1l34raJvqq16mrt NmoGNuOC4gn1Jo4XuxNlWr3OLz2uXFx56YBbc81ZO4U1ZuSNxpi8HbFZvkM+GWoQAxrX/c7Q PZ1/dsa9KKJk2tSAKTZ6dCbTDg3j1UaPrdcyLakBUkdU5R/8gBMJDHg1Yo8odwEvFZCWMtI/ 5L5l7U9SnlnL08GPD95gwJ1faazmwMY577vqmQyqoNsCrODKs1Vsx7IiSTbixoHjxQIe+pBy 5lTC7fBZwhIrEkIg2SwrNLDxzbuuEm3B8OZd7GJg+dOAes5RossGd+UTEl3SjHewqACxhXFJ iZsX5cWca0Q5QzYdcJZ71ltTSS4malB+O96pn5UNBtz4pcESjazO+r2oRMng2STZFhAWfziu A85GkxHN5TJAXstIlOY3icIB7kaihimjJHa0XCDZPEs0svrqt9Ikq8Hy5l1sY+D504B67qOe iKUEn9aW6NXNa2qE3j88ahdUStmgszFvlNf1QWLeJ6Zlymkv+DF/eP/xAzZKH680PpxiV98v nJebiIX7nM2okTmbYt9jsbV+wK06lx4Pc9yRjJCcsVOLs2nR8CLquUlzKUM+NPEGmNzVEFj3 4rfa8pANtqtCF1Buzc20e17sfpT/jRJ69Zo9fF7yfL06NlPup11Dzd72OGtiDpAOh6+WyOq7 35peLS3Ywb5XR5AOf+tJ43I1T9Nej1ry9Jeaeja1pmfVx3WbQ6aekKqXd5yv7fkhz8TMabVj X5fozXbBJ28NclrkRrKf0/7X+ug5Dqg1pwX4Ft+eYAtjeQcbqscGu0kQkgVkXfYN1C/vsJtv 3v79H28/PT5/+TesI/PtscNO6v9/+vD+4dNb+AP4PUOtkOamNh4btMz5dsWjnAqW022Wjrp9 jD6npfKyks8GYOZOzy9Sz90oBnOBM1hrBzxzsfwTnOqLr+vPL15CIL7qatxcd1H2g7y89rRL 1d6641ZOnMFwqYCDFZUBUHMlAikSOO1DzvR9Bwd227S4HfPa0OM4I6dwOSb1oIKuWCeOuC3q gc4FAcQW2bjr94KeNWMYSViyGTeMngDn3fOWeBsCsjjIooMpG58VZEGSljFzCrd4sA8DVocb aBAIdtnX4s6B2XrrtsKKb+Hzlm2snQjwKzs2x+50vczQkF1tecFomQukztnUQnyxjO2JSGvA TS9uvNG9QYs44BVUvzRR5VuL2myyC7i0mHqYqeUUhUZuHjlnRZ9kSvUppiVO4kdjto6qi5CM Tyk6awRI85oA24FKEgx4Etl1YFUW0PjnnGfRyZl7szY+PnL4mQM9Sye6MsFynxddznQpS4CJ NU55pSvBcJEjdWM5KzTqILfb2hlRkRUJE+m8x9xU5mGi7kokFrn551wXyZGps0VXTBu2QYvI a2EAAqh/0W0jPpqj4pODlgyQY0ObHmgfoq6AxhfnqIiNz96btXHyEcTPHLRTHN37sNkVu+f/ 9BPJlLqrDyTcWu0rii2zxBfLhy2rHbeS2MECWcYn6IHXMDlAMzUJZtPHYLUXnyArA0jXsEfg qtBqbuDJVHO7FUkJMqZVeDY3MLGthrtR2bz5e/XJYtXfUnt7L3gjmiKTqaN5H3JpWeKhXR+L 23gr2Oqhc1+SRoLlLdvI+pjjZw70pK+lTW3Qy/7rvFhaUqRtQsPce2wJzjgmRU3WdeGHbapP AOiVF35cNl6p+lWEtqpHOoS6XICCjwRaw4718eqNnMJl4gchuzDizAhZ0SM1EHgCDjZWzTaR qjyr/2Fz4KoaO3+31BcUJIUYNyyY9xHiLoC3NGD2pnIPyewhxc1V/7yRU1jCxT50yLGAVQLw 1pdpdUfBbP1zzpMrGlJn2cXfCoO+smM9eafoddvhhrne2IDR+gKltysJglnDWnlNnqmCwQIe Mr8h0KwDtmlUTDObitft4GudE2nAbSE33vk2ZlVGbh4G91mlZOtsMWbTBhejup55NQxDhNk6 OUfvVYY8UWUoRy5eFCy+LXPaceJCEbZihBjZuOj8FyFKVJxlF0IXXPrMgR7voH5jHtKw1m/u Vd5coHzf/4wN0Pc6fvjh/X1rh+BRUjdQq3qbgD/f4pc/4g3w02d7J+iZLs8n2z+9qr++feou GDdIcArHfLtj5kBUCVpdXY/ETxLW74AXm1YDXptkPLAayAcH7INJM5r5378s2g7C9v6PLwuJ mGTpgCebtYY1VAR8GJKlgZ+8PiTIzrJThNMKfeZAo4Et1r6B5qHnTrjgPsvjFnO37DttxXyM k496cnmhY9LxRe3QhXo3qg3LNNExwTdaHf5Qc/QrzPcXX9fhF/cv/lx//bH+A7z/5kv421+x sZtuEO4q/EWy2ogTkgBD0AJduIldtBGzZ4sNacKonhbMzInO4U6rx1o/2yeQHxaP2vbWYw7V gysvdsorFsx2QyhmojUNqDBySQ+IFb364cd062zJD/fWbrv5JyNsRZfuSVaG4tCjLB7HtFM6 Qo/qst8N+NfsgLawKIhZRXSgYt5HyLSlAaEToFk7rieVM/ttx7IwSt5YOBlbUGjtAebmdk+y fBzjkvvhDobQOO32QynZYmpt2ugbURyTH+r5O4VTIlxtLLFIYRU8RSyEF4NbanHukBIUVvJD TE1bAjdjuomYJjUNJ65uLMzEnQzuw445uVmYAcUP5yDLMnELzToUoxwMWV4R7YK+3Hi+LeKJ GWrn7OaL0trJscp7zDsv4MbkP6vHkzrl/OXyzkdrDdoY2NgcUzdOAy2tMyjos1pa8B6UYsnY glFxKwbl7MyADgVN2THbUeEsRWWzchRIsB3topswno1oAi7RS9bPyo0kB00MKhr5dDTL8box F+rsh9ofuvlNTHKSLGQ5H7DoxkXVYsfqgXOOlbMU1o2YsiGwoaGP7fjZtQcF7CrjFVs7hlc3 r+/x/fD+gd4NL39vHwCvsUlozf8HfS3Ul0XB1wJ/eP/xAz4oPnZvAy1DV58G2BpyQOHilRaM b3bpAvky73HRMjPAlTXtx0s7RD/WeDsLTeKTdh32MaBo7vOOisjKjTnftWoaVKf9fjgN5A3A 6pv00jQvAClskez1F/Hz9SjeMtr5u/ntbj+0Ptq2X3HSnsSghuMwL7rdP4e0IB+p3wBf5xJE a9OG28rgmMJYj3ycTuuBvT2n0y81V2wuTM/222XdZswWTa0TcuvyjhPssUubORWbN1uCswl5 M9fual3y7YEPj7RiPhBflOkBAQjH1iM28rBcgIJyoTXsmO3DneSMnMJNTOzCiDOjSRODJbMX ATioFVyQmY3/fnPoatFsmUty6WJY0CSMQdpFSS5lGos3ZQDJ7CHFzYF/zsgp3MJFPnQosS4p ghxFmX2cia1/MV/M/DNYtvF3wsCv7NicvFU0d8dXOxc0Wrg3ZEEwa1gL1lrHWxHZesjUYpFZ B9okejNNxdDTLzaRBtwWcmN+AIgqI6uH3n1WKdk6W4zFtOJiVNezrFYGMFsn5+i9yrA9f4Tb kVOmaEhK6oF2x8KLbFxy/orwWhScZRsyH0z8zIEe56BckywOsFTrNXcvb17fY8H+54e3XLLz 7bTNGUo2/PUCf3x3r0Vai1F1dMbqr1VsBumXOQ8KF7d16YDTPM2Y5tU8rF7CTu6u+j5jzcC7 5mds3J5PN9jB3bObw5tknRe46D57k+Q8cdDrBIZWlXPmzpTVP+LFqHrEojc/5l6APHAwqTKs Ec3Qesb/4r5ceuO4kTh+96eYo+YgpZtNsrtv6/UDEWDYThxgD0YwUPyKk2nJq2hj7LdfFqvI epBjr7MLI8hBmvlJ3cUq1r/Iqkn40ONQRa5hwnu/1KRh9lA5XyoUTXGJVtPMkzhcWy6rxQ6M TjgpwNcuRJupRcpbjm70eBRFLFnkO4rwsWYNSo9VOFTCNUvKrMyoznV+TAHvttSwXwKU7Sfv Dp9K23EzhHiU6MVN0uPSUBiYBypnWEQBLSFUii9WmYLfTpbSEviKamCQMrTMLml/SZhkrDY4 bJwUUj1hrie3gVpK0ULg60qZYSHWbSXhdbgm23ynpmTU6SAWHir3i/DqxkvDIkc6efCQArmb Jw7P08JbF+5YAI4CIBcsQskQVWK/aJ7rbmso+5gXVGAOSnzta5yU7KD2/k9+VtZNJ0nWpFDl tFxUIr9R6CRPw589J2uS2GzNpU5xfkCB2GV1RoYI3TlJNX38Fe55H2Z1irdYD3GIoMN1qjUQ aZQhLVv+ZBdApviwraaZ1anXYSH0BiNPcw04L0IQMPPUpcx89R5BeqzC+f/3CKca5QGegB7/ Ykh/yn3++5v9cvbqAL+219Tpn6dW3yeZpFafpgHTVt/+fgWfx/L8eOHdEnE0MI+mL2/gy+3t zW1+PG1d9srt8ok1BFBA8ukcnHK43MMn+/NxPDtc7s/Xs2cP8PPp4/15PHt2eAEfj/IDP8jp g88zN+SBbKzDad7okKaJuTnDafzAS2wT3+E5DztX7XQHEDcPu7AGEJkYnvLYccg7kHfqA4ae hBXmcXee/Kn79TBt/ZP8+OWz9DUF+zR9PE4/gIfvH8H/vtvjVt5Pn98204xfp1QnYVpADCfn mTDCNOVjPgO39BI1JgGjE1ww9yUGIwR9pCU7CFRW0kTrJJe0sa0wBBCEKz1eCCGfFlfsfMpu GFywUgDXlkUkOtDsrjC43Qsu3zp1gcroDmBcephjAV5dl7M/wB2cKQtTh9ba8WljW+GSRPKl g1OsazVIGxHcojaGWHkqAyF3HKfZ2te6MKKpry6tK/KGp00YVjhiT7ajZDw96z0Jbo2FOemo BcH+YuSd7uAiBG6RbAu94/us9+T0LPVteRw4y/I7BstaNiz81GGwlpO1rUozmy5QfCjp7LDj lDRYK3bqYF5IyBRfZpmW7aeKqltI6bdIURVhGlauSc9ZmCUf1r5OoE1vfdfgePryLYKZoVL1 /ft6D/focAY3MHy/urt6mT5+/Pwh321m0dng8gVQhU58lOwXobQOz6IQeryKxFqcsOcqzhgk X7gw6P0/xUUgYtGh/hUugprH4kyH/SJC6bDjEuzgKivQ8meviioEa18rx+qKHjbIeTtZkcHl G+WzjVtucd3oJ2zZ6Kn8Oze31+/u4ONnethdRO9W9Sx2xa/4+/Xbm8NdW+QjdOhxScMV1viQ RwdT5B6ytQsLhLDld6aCR4l4t415XBGcpwngvF2C1/J+bNGnmABpaYO4NJSIsrYVhoikNz0m b7L3PRbudBApj0MGRSg60uyuMJe89TNMBWyeGb3xC+hF8SSjYS6vx4Y8HbW0tMHAvkljW0HO I/rWY16tJYor6pwjs6cqjOyLeH+zxpUIrELoUYOcAqt9N0PCT15waD6mZ0MRP+gH+SgZWynS X8NCPfo7nf60jEFahYUeR6lyR+cxOeFmnDGLiBteaOagyA3ONJiRqi2PzsbDgkZTLOhqunDx hDLdYVwLNWsxUjJpMYMLT07KXNUwZ4PqqWYP7VuUElXAfmm3q2IpPcasTqZNdXnRYF+tIULf d3LuINt+wGu4JB6Zs5d4kThSx4JrtDgLTVoc6p2qzbFKg8gXyLTDi9Bhi6OUoWX2VgfD4kSD W81+XYA5SjF2eBSpsShulg7SWkKc+D6LM+1gkOJkxtqpycP1LJbwSaKW2VsdDEu2psvY1/m1 2a/vGix73+uBSE8RLKYe6OXZT9j53FzfcdsTY5yhlREzy09pQBHziuiRvFtifpgePNx+yJ3R 7alOB8vsM6d9WItiqLdZhSQzzxejOP17PInTv8exasTSIje8QaEnaYyrbeS2LF9MDXp5JXQ4 iuKzWA6peiUo5khUmFyJaI6viWqeufY1w6J5EsEU/MwRSjNo9V0hLS0KcyxTIXU+qzjth0Vk mTqfhuu5MstUFaS9oVZIo/Bdh8a9UEmzMa5lYUVDDxusaTldpMMEsVGRfkx1J+tq2CWDYR5N mW5Qp8dfqVjrsEGjj3dUU0vMw1nyJfu05pEwBLqS5MMYfhjSdsxwzmwS4Nl8NoGlnMoXKbfQ PJfLf8jPxcXReTpE7DLX0rrVh/NCbkocJlxIALyZ58kInud1/nlvXHZj/qOLWRwOdi0dJv/Y XeddpS2Fp3ZpUAwOh74BD6N8XKVfH++d/bL/4RfYpe/0a+mddJK2b57Dfk+431evft2niS0d j/D7b/tzd/bxzW/lKC0nGtxlQ8lUgDezDQervzzb7dw37pth3O1e/Ot69/x2H85uPsDv92/2 69ndVfoF9O9s1GplGlzaELcMeqh9mLL/JE+el8/S1weHy6fp43H6ATx8/wj+912WyXCW4UWC 53mJ5WKEUfY8lRbdA19o7376/LY5/R2MPVlk4HY6/Zd8x5vTH+PxMSsyaQCnBuKj5FybDkfa lunUwkUb9HgBl8UM0lqgR2VuKwxBTMId4BDJfjJVKolD1jQg5u7RsvBVh5LdYXvbvckNsIHF /OTGYi/7JnjA/w9yeUK/YvGRaYNkOi2trW2F68aDuWXp8uzqcg1SbNO4qliJhbM6luyQsLA1 9nUqbaLpYYO80Va0UwB3TrYsaD6A01kkvhJLIuHihWQsuyoDlFiHSXLZukXq+sgTg+QKC5pe Z0GnAFcp6IapZS3RdJjry1LAu6EI3rITxSQjZcFneyx4Mi9wFJJrmWIpIupwFBVh0ZeTxfcQ XREFgq/XAql5LN5YLjoo0Ujm1BGVrSn1YZl916HVeqmJN+aVTqyI6E2DnKVOC1OOkwEavtzC 5CnjPfz6cPj9TW5kftvTX27SpXHdThT6pWsBPxcDJy6Z5CgVa9qF/g2zLDgCkoKJj5JFeUqE gyphNLzwGd+g3FdLSx0GtTEuzsFDl8PF2OFZnAUNilqTILzUQXDhoamtirWYFuyl3lpmwajv qcMUV6JFXy9gYYjvnLL55Y6h3FAFdVCe8i3mmMsNZLh4LcLhu6fkyVrWiTVZp2c18cafrCW/ wm/o8Ljbf/X+A3X6r9PP1d3Vy/TxIxVSqqMAxfdQPfgW5oOrd03hhAEmEp/6yLT1J7uz4NKB spvSgJDbe3gp7QvxUTJiHoka9HgC4pIdzEQraZrqpKCNbYUhgFW44pcxjziAU+xhvvJL9AYH 1GEY8vBmWXiuA8vuCYPJOz/Bqc4LMGeDPr8pkdbDWJhxfT8p9wpKz9ugwCtlaCtcE4RudBCz m9dpsGwCzouWhZc6iOyOsLA19nXOjSDqq0vrimzgaBNinjtPNnBoPE22cF1tVR7IR8l+EeLq MAkExdjhKs7Y4oiXdXHGIPnC2qf3WfspyCC13+FFRGNxxvu97JjBiNd9qQXLHIsOlWsBDXIt 1AUKkztUCy0GWQsdnkQxWAwXYqMNkR+iPvDtWh+cR3Kmw34RoXTYefamRSpzp1VCzLHoUGsF sRCMfa0cqyt62CDnrXMV0ZbN+d6Fu4juoa/S29HiqQOPn2jvKMoxXamikjOyOtPl7gwHx5Iw 6PQGGiTToi7xda7LKfvJddjhSQjSosdup1SaZXZWx8KVhwa3Kte6APMoBdZwWRElYnHCcaos ZjDwla3McXXVZFC1lFyheUMlWKoNy+ybdp1rpSZHm1e5tInGRw2KfT5dKcnQ8KVdG1dIt2s7 T06O3q27eLEO6ZhJj1IZPkxPPYHiOVw+S18fHC6fpo/H6Qfw8ODyOXw8gifgH/fTz99TdeZX HmXbIWUizLvzZLr2jv+71dFdrHNM2biYwjyi1dfHbBKPiFf8/frtzUGeDHcYs3MXbvTTDiT+ hyyAgTiuu/MpDQ/p8w86Ma5DgI2vOfpiL8JunP/LfbiCL/n/7+j9dtR1k8tN2qmOHaeP0dMg R9Mt8VFyPvZppGw5XLhdnRQ76P2uzG8NRjkINcjDnja3/Yf3csltI4bB8FV8AQ9G0jyXbZoA Boq2NzCKFu3G0256f0QS35ScAkGQhWF/8AxFij8lkriEOCtnerxJLBrL/qRhnBRG6BGzzssh 51n5bkOr7imDNPWxfRw3aXWcTTs4w+AXaHnPIz0/tRiGRYViKQ0ynmpjBzFlDX1pEFOOgfR4 jexLg7hPKR+Let+QVSQ20OqusnA09q1MvIjwYYeSNF8uufiW7X73AObjGMpNxOWCzKqI+akU Lc8qFx4TXHBk3CHalgLA96UAyomvBd9hJXFHK/R3JHjPoyomHYoIvto7WMJoXmHQCm4Y10MJ e1zonJl6CEspScPrLGlOBFUfbSaa90jVhqL0LM5Z31mknBlv36bSJxofdigb3ekfcAfKtVX7 h/dqsvHMzndJfKHJxipcx4HO6XJJVLxpXPRR2nCANhOXbFHtZYPjYGsGX+eaKRFM6lYorKTV wTipa8CzOGd95ypBgwcfl7QAHb7kj+Io0mxwhuaSjDtceEiz5uSsp82m073DajlHGByd5Z7F VxuKnOWUDGfe5M4nlt/cOq68VCRhrY3PmzbZ7dwZ5r14drfRgplhjSXLPHcC3hSuavBrcYe5 rw5yHtMgY5UHmLjAA0vogEyn9VWeTcMywgGKPvR4XiQCw4CbEGwRj6ae2WsdEI+pYIsGM7YM Q6GsCzNrj5OaWRtmV8B6i2kxcaQ2SBlh8XUeYTlV4EyLkGYIpcVJBlxHuGEw3FpSIdgIedSl TFvDRhZOMvSapeg6KdwVaLLuXhFgOsR8ZEg9ALIEQhqhC1ScRNgNBmhx0bRDNC1Kx9dF69nj XWu7YewiKUCH2FSSuD2LszYWETgYPFizvAAx+YMZ7fCkROtxktOkg7iWEjE13MScDCw5yhWY t8TBohg9i2/WdRanJMeaN7n0icY3Hco+d64Iquix3g94N7xLJ4Urh//WyL6hLECG+yayA5xE VQ1tcvw60ltoYRMpaDtSKGGCC52qNlgtNaiqQIM4ZLyVggBDByuODaNq2A9mSbv5jZ01mXWI VpX0g9I9bSkUYUNBVN8QRAeSd0j+KcdZ+7zd1qpJjs0bPGhANvO+6sfSHr1pX3QOObp5DXml /BTW06f8wOdSBdfL1/zz4Xr5kr+e8qfg9eHyrXw9lifKHx/y52Mus/rKI5jN0Yz7fEo56Cnu 4MHPW7UJZflDfv/59ff6vfyo//++/gPvh5i2cDrn06GaeoWJPGyFvAGncy68ZVlfYyIfAM8C DAB6zjSNCmVuZHN0cmVhbQplbmRvYmoKOSAwIG9iago8PAovUHJvY1NldCBbL1BERiAvVGV4 dCBdCi9Gb250IDw8Ci9GMiA0IDAgUgo+PgovRXh0R1N0YXRlIDw8Ci9HUzEgNSAwIFIKPj4K Pj4KZW5kb2JqCjEwIDAgb2JqCjw8Ci9UeXBlIC9IYWxmdG9uZQovSGFsZnRvbmVUeXBlIDEK L0hhbGZ0b25lTmFtZSAoRGVmYXVsdCkKL0ZyZXF1ZW5jeSA2MAovQW5nbGUgNDUKL1Nwb3RG dW5jdGlvbiAvUm91bmQKPj4KZW5kb2JqCjUgMCBvYmoKPDwKL1R5cGUgL0V4dEdTdGF0ZQov U0EgZmFsc2UKL09QIGZhbHNlCi9IVCAvRGVmYXVsdAo+PgplbmRvYmoKNCAwIG9iago8PAov VHlwZSAvRm9udAovU3VidHlwZSAvVHlwZTEKL05hbWUgL0YyCi9FbmNvZGluZyAxMSAwIFIK L0Jhc2VGb250IC9UaW1lcy1Sb21hbgo+PgplbmRvYmoKMTEgMCBvYmoKPDwKL1R5cGUgL0Vu Y29kaW5nCi9EaWZmZXJlbmNlcyBbIDQ1L21pbnVzIDEyOC9ldXJvIDEzMC9xdW90ZXNpbmds YmFzZS9mbG9yaW4vcXVvdGVkYmxiYXNlL2VsbGlwc2lzL2RhZ2dlci9kYWdnZXJkYmwKL2Np cmN1bWZsZXgvcGVydGhvdXNhbmQvU2Nhcm9uL2d1aWxzaW5nbGxlZnQvT0UgMTQyL3pjYXJv biAxNDQvZG90bGVzc2kvcXVvdGVsZWZ0Ci9xdW90ZXJpZ2h0L3F1b3RlZGJsbGVmdC9xdW90 ZWRibHJpZ2h0L2J1bGxldC9lbmRhc2gvZW1kYXNoL3RpbGRlL3RyYWRlbWFyawovc2Nhcm9u L2d1aWxzaW5nbHJpZ2h0L29lL2h1bmdhcnVtbGF1dC96Y2Fyb24vWWRpZXJlc2lzL3NwYWNl IDE2NC9jdXJyZW5jeQogMTY2L2Jyb2tlbmJhciAxNjgvZGllcmVzaXMvY29weXJpZ2h0L29y ZGZlbWluaW5lIDE3Mi9sb2dpY2Fsbm90L2h5cGhlbi9yZWdpc3RlcmVkL21hY3JvbgovZGVn cmVlL3BsdXNtaW51cy90d29zdXBlcmlvci90aHJlZXN1cGVyaW9yL2FjdXRlL211IDE4My9w ZXJpb2RjZW50ZXJlZC9jZWRpbGxhCi9vbmVzdXBlcmlvci9vcmRtYXNjdWxpbmUgMTg4L29u ZXF1YXJ0ZXIvb25laGFsZi90aHJlZXF1YXJ0ZXJzIDE5Mi9BZ3JhdmUvQWFjdXRlL0FjaXJj dW1mbGV4Ci9BdGlsZGUvQWRpZXJlc2lzL0FyaW5nL0FFL0NjZWRpbGxhL0VncmF2ZS9FYWN1 dGUvRWNpcmN1bWZsZXgKL0VkaWVyZXNpcy9JZ3JhdmUvSWFjdXRlL0ljaXJjdW1mbGV4L0lk aWVyZXNpcy9FdGgvTnRpbGRlL09ncmF2ZQovT2FjdXRlL09jaXJjdW1mbGV4L090aWxkZS9P ZGllcmVzaXMvbXVsdGlwbHkvT3NsYXNoL1VncmF2ZS9VYWN1dGUKL1VjaXJjdW1mbGV4L1Vk aWVyZXNpcy9ZYWN1dGUvVGhvcm4vZ2VybWFuZGJscy9hZ3JhdmUvYWFjdXRlL2FjaXJjdW1m bGV4Ci9hdGlsZGUvYWRpZXJlc2lzL2FyaW5nL2FlL2NjZWRpbGxhL2VncmF2ZS9lYWN1dGUv ZWNpcmN1bWZsZXgKL2VkaWVyZXNpcy9pZ3JhdmUvaWFjdXRlL2ljaXJjdW1mbGV4L2lkaWVy ZXNpcy9ldGgvbnRpbGRlL29ncmF2ZQovb2FjdXRlL29jaXJjdW1mbGV4L290aWxkZS9vZGll cmVzaXMvZGl2aWRlL29zbGFzaC91Z3JhdmUvdWFjdXRlCi91Y2lyY3VtZmxleC91ZGllcmVz aXMveWFjdXRlL3Rob3JuL3lkaWVyZXNpcwpdCj4+CmVuZG9iagoxIDAgb2JqCjw8Ci9UeXBl IC9QYWdlCi9QYXJlbnQgNiAwIFIKL1Jlc291cmNlcyAzIDAgUgovQ29udGVudHMgMiAwIFIK Pj4KZW5kb2JqCjcgMCBvYmoKPDwKL1R5cGUgL1BhZ2UKL1BhcmVudCA2IDAgUgovUmVzb3Vy Y2VzIDkgMCBSCi9Db250ZW50cyA4IDAgUgo+PgplbmRvYmoKNiAwIG9iago8PAovVHlwZSAv UGFnZXMKL0tpZHMgWzEgMCBSIDcgMCBSXQovQ291bnQgMgovTWVkaWFCb3ggWzAgMCA2MTIg NzkyXQo+PgplbmRvYmoKMTIgMCBvYmoKPDwKL1R5cGUgL0NhdGFsb2cKL1BhZ2VzIDYgMCBS Cj4+CmVuZG9iagoxMyAwIG9iago8PAovQ3JlYXRpb25EYXRlIChEOjE5MTAxMDIwMjEzMzM1 MykKL1Byb2R1Y2VyIChBY3JvYmF0IERpc3RpbGxlciBDb21tYW5kIDMuMDEgZm9yIFNvbGFy aXMgMi4zIGFuZCBsYXRlciBcKFNQQVJDXCkpCi9DcmVhdG9yIChXaW5kL1UgWHByaW50ZXIg VmVyc2lvbiAzLjEuMCBcKHNvbDJcKSBcKENvbXBpbGUgRGF0ZTogTWFyICA5IDIwMDAgMTI6 MDE6MjNcKSBcKGphY2tiXCkpCi9UaXRsZSAoKQo+PgplbmRvYmoKeHJlZgowIDE0CjAwMDAw MDAwMDAgNjU1MzUgZiAKMDAwMDAyMjQ0OCAwMDAwMCBuIAowMDAwMDAwMDE2IDAwMDAwIG4g CjAwMDAwMTA0ODMgMDAwMDAgbiAKMDAwMDAyMTIxMCAwMDAwMCBuIAowMDAwMDIxMTM5IDAw MDAwIG4gCjAwMDAwMjI2MDggMDAwMDAgbiAKMDAwMDAyMjUyOCAwMDAwMCBuIAowMDAwMDEw NTc3IDAwMDAwIG4gCjAwMDAwMjA5MjIgMDAwMDAgbiAKMDAwMDAyMTAxNiAwMDAwMCBuIAow MDAwMDIxMzA5IDAwMDAwIG4gCjAwMDAwMjI2OTUgMDAwMDAgbiAKMDAwMDAyMjc0NSAwMDAw MCBuIAp0cmFpbGVyCjw8Ci9TaXplIDE0Ci9Sb290IDEyIDAgUgovSW5mbyAxMyAwIFIKL0lE IFs8ODU2OWI5ZTNiZjNkZDE1MDlmOWNkOWQxZjc1ZTJmZDg+PDg1NjliOWUzYmYzZGQxNTA5 ZjljZDlkMWY3NWUyZmQ4Pl0KPj4Kc3RhcnR4cmVmCjIyOTg5CiUlRU9GCg== --------------34E70F3386422AB4EC150615-- From sac-list-owner Tue Feb 13 11:25:34 2001 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 13 Feb 2001 13:40:07 -0500 (EST) From: James Carlson To: jack.bochsler@sun.com cc: psarc@sac.eng.sun.com Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) Content-Length: 2856 Status: O X-Status: $$$$ X-UID: 0000000002 > ENOTSUP if the driver does support DL_IOC_INFO_REQ in general, > but doesn't support the requested primitive. What does that mean? Looking at the diagram, it seems that there's no additional information for M_IOCTL/DL_IOC_INFO_REQ. Does ENOTSUP in this context mean that the implementation supports DL_IOC_SET_REQ but not DL_IOC_INFO_REQ? If you get ENOTSUP on DL_IOC_INFO_SET, how do you tell which one of the dl_cap entries sent was not supported? > The primitive DL_IOC_INFO_REQ_NONE is allocated for any drivers that > elect to implement and support the DL_IOC_INFO_REQ interface but have > no hardware specific capabilities to report. They would set dl_cap > to DL_IOC_INFO_REQ_NONE, dl_length to 0 and have no further data. Why not just send back M_IOCACK/DL_IOC_INFO_REQ with dl_length set to 0? What's the distinction here? > On receipt of an DL_IOC_INFO_SET command, the leaf driver should > set the state of all capabilities listed in the dl_cap elements of the > attached mblk chain. The driver should then send an M_IOCACK back up > the stream. Is the implication here that the sets are atomic across all of the dl_cap elements? If (using the DL_IOC_INFO_SET example in the .pdf file) DL_IOC_INFO_REQ_ESP succeeded, but DL_IOC_INFO_REQ_AH failed, would the ESP change be rolled back? Or is this semantic delegated to the contract for the private interface? After failure, how does the requestor tell which linked primitive failed (and in what way it failed)? Is this interface usable from within STREAMS modules only? Or will the stream head be modified to produce the proper message fragmenting (on dl_cap element boundaries)? Since DL_IOC_INFO_REQ_ESP and DL_IOC_INFO_REQ_AH are subtypes within both DL_IOC_INFO_REQ and DL_IOC_INFO_SET, could the names have a common stem, such as DL_IOC_INFO_ESP and DL_IOC_INFO_AH? (I think the use of DL_IOC_INFO_REQ_ESP within a DL_IOC_INFO_SET message is a little misleading.) Or, perhaps, supply duplicate defines, as in: #define DL_IOC_INFO_SET_ESP DL_IOC_INFO_REQ_ESP #define DL_IOC_INFO_SET_AH DL_IOC_INFO_REQ_AH A related question -- will there always be an inter-related request and set primitive? Is is possible that some capabilities are request only or set only? Can these things be dispatched by multiple modules? Is it possible for a STREAMS module pushed over a driver to intercept and handle some primitives while the underlying driver handles the rest? (I'd guess the answer is yes, though this probably complicates the locking and error handling issues.) -- James Carlson, Internet Engineering SUN Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 Second Edition now available - http://people.ne.mediaone.net/carlson/ppp From sac-list-owner Wed Feb 14 15:14:07 2001 From: Peter Memishian MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Wed, 14 Feb 2001 18:15:06 -0500 (EST) To: jack.bochsler@sun.com, psarc@sac.eng.sun.com Cc: carlsonj@east.sun.com Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) Content-Length: 2372 Status: O X-Status: $$$$ X-UID: 0000000003 i'm not a psarc member, but this case was brought to my attention and upon reading it over, i had a number of concerns that i felt needed to be voiced. first, i want to reiterate a few points that jim made: 1. unless there's a technical need which is not stated in the case, it seems like a poor idea to allow multiple capabilities to be set at once. specifically, it leads to confusing error semantics and more complicated payloads. i think things would be a lot simpler if only a single capability could be set in each request. 2. DL_IOC_INFO_REQ_NONE seems overkill. it seems like it would be simpler to just pass a M_IOCACK upstream with a NULL b_cont. furthermore, i'm unclear why a driver would be written to understand the DL_IOC_INFO_REQ primitive only to report that it doesn't have any capabilities. secondly, i'm confused why this is being implemented using the ioctl mechanism. as it currently stands, this proposal seems to have nothing to do with dlpi aside from some naming conventions -- it is merely a low-level ioctl-based sideband protocol with an identity complex. it seems to me that these new primitives should either be changed to follow the conventions of the dlpi protocol (e.g., use M_PROTO messages, provide a set of new dl_primitive values for the new message types, ...) or the proposal should be rewritten to explicitly be an ioctl-based (or perhaps M_CTL-based) protocol that has nothing to do with dlpi. finally, some additional questions and comments on the proposal: 1. the namespace chosen (dl_ioc_info_*) seems a little generic; wouldn't something like dl_ioc_hwinfo_* be more appropriate? 2. in the dl_ioc_info_t, why is dl_data[1] a t_uscalar_t rather than a uchar_t? 3. i'm a little confused about DL_IOC_INFO_SET -- the proposal mentions that it allows capabilities to be enabled *or* disabled, but i don't see how one indicates that a given capability should be disabled. can you clarify this mechanism? 4. the document does not specify what payload is sent downstream with the DL_IOC_INFO_REQ. i'm guessing there is no payload; is this correct? furthermore, the document is not explicit about what the fields other than the dl_cap in the dl_ioc_info_t can be set to in a DL_IOC_INFO_SET. could you clarify this? -- meem From sac-list-owner Wed Feb 14 17:21:12 2001 Date: Wed, 14 Feb 2001 17:19:10 -0800 (PST) From: Jack Bochsler Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) To: james.d.carlson@east.sun.com Cc: jack.bochsler@sun.com, psarc@sac.eng.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: /94h9RHNJ8xwDcp+ukpCyQ== Content-Length: 5394 Status: O X-Status: $$$$ X-UID: 0000000004 > Date: Tue, 13 Feb 2001 13:40:07 -0500 (EST) > From: James Carlson > To: jack.bochsler@sun.com > cc: psarc@sac.eng.sun.com > Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) > > > ENOTSUP if the driver does support DL_IOC_INFO_REQ in general, > > but doesn't support the requested primitive. > > What does that mean? Looking at the diagram, it seems that there's no > additional information for M_IOCTL/DL_IOC_INFO_REQ. Does ENOTSUP in > this context mean that the implementation supports DL_IOC_SET_REQ but > not DL_IOC_INFO_REQ? > Sorry, this is incorrect and should be deleted (doc artifact from earlier design version). > If you get ENOTSUP on DL_IOC_INFO_SET, how do you tell which one of > the dl_cap entries sent was not supported? > The upper level software can send the entries down individually and check the status on each reply. The more likely case is that the upper level software will send down a DL_IOC_INFO_REQ and convert the response to a DL_IOC_INFO_SET, potentially pruning unwanted features. Although this doesn't guarantee correctly formed requests, it potentially simplifies the implementation. > > The primitive DL_IOC_INFO_REQ_NONE is allocated for any drivers that > > elect to implement and support the DL_IOC_INFO_REQ interface but have > > no hardware specific capabilities to report. They would set dl_cap > > to DL_IOC_INFO_REQ_NONE, dl_length to 0 and have no further data. > > Why not just send back M_IOCACK/DL_IOC_INFO_REQ with dl_length set to > 0? What's the distinction here? > This seemed to provide more explicit notification, but I defer to your opinion. > > On receipt of an DL_IOC_INFO_SET command, the leaf driver should > > set the state of all capabilities listed in the dl_cap elements of the > > attached mblk chain. The driver should then send an M_IOCACK back up > > the stream. > > Is the implication here that the sets are atomic across all of the > dl_cap elements? If (using the DL_IOC_INFO_SET example in the .pdf > file) DL_IOC_INFO_REQ_ESP succeeded, but DL_IOC_INFO_REQ_AH failed, > would the ESP change be rolled back? Or is this semantic delegated to > the contract for the private interface? After failure, how does the > requestor tell which linked primitive failed (and in what way it > failed)? > My intent was atomic, but I don't see any compelling reason why it must be that way, so the actual implementation is delegated to the private interface contract. The proposal will be rev'd to explicitly state this. DL_INFO_REQ provides a mechanism to capture failure. But as you point out, there is no mechanism in the DL_INFO_REQ proposal to address failure at a finer granularity. I will address this failure granularity and the atomicity in the DL_IOC_INFO_REQ_ESP proposal and re-submit. > Is this interface usable from within STREAMS modules only? Or will > the stream head be modified to produce the proper message fragmenting > (on dl_cap element boundaries)? > The intent is from within STREAMS modules only. > Since DL_IOC_INFO_REQ_ESP and DL_IOC_INFO_REQ_AH are subtypes within > both DL_IOC_INFO_REQ and DL_IOC_INFO_SET, could the names have a > common stem, such as DL_IOC_INFO_ESP and DL_IOC_INFO_AH? (I think the > use of DL_IOC_INFO_REQ_ESP within a DL_IOC_INFO_SET message is a > little misleading.) > > Or, perhaps, supply duplicate defines, as in: > > #define DL_IOC_INFO_SET_ESP DL_IOC_INFO_REQ_ESP > #define DL_IOC_INFO_SET_AH DL_IOC_INFO_REQ_AH > Agreed. The duplicate defines will clarify, will do. > A related question -- will there always be an inter-related request > and set primitive? Is is possible that some capabilities are request > only or set only? > At this time, I only envision inter-related request and set. It is possible to request only - to decide that the capabilities offered are not desirable (e.g. our US-V chip can implement the algorithm faster than the co-processor). A set implies that the sender fully understands the capabilities existing below on the stream prior to crafting the REQuest. Although this is possible, it seems improbable, unless a feature became so uniform in nature and so ubiquitous in implementation that it could be considered an optimization to just send the SET down. > Can these things be dispatched by multiple modules? Is it possible > for a STREAMS module pushed over a driver to intercept and handle some > primitives while the underlying driver handles the rest? (I'd guess > the answer is yes, though this probably complicates the locking and > error handling issues.) > Yes, multiple modules can issue REQ/SETs. There is potential to get conflicting states - not unlike the existing PSARC/1996/173/ implementation that this replaces (which doesn't make it right). This could be mitigated by a private interface contract flagging specifically what is turned on, so modules could poll for the current status of what is "on". Yes, it is possible for intermediate modules to intercept and handle some of the primitives. jack > -- > James Carlson, Internet Engineering > SUN Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 > MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 > Second Edition now available - http://people.ne.mediaone.net/carlson/ppp From sac-list-owner Wed Feb 14 17:52:43 2001 Date: Wed, 14 Feb 2001 17:50:42 -0800 (PST) From: Jack Bochsler Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) To: jack.bochsler@sun.com, psarc@sac.eng.sun.com, meem@east.sun.com Cc: carlsonj@east.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: r5i25z3+mU5nJP9quuw1cg== Content-Length: 3915 Status: O X-Status: $$$$ X-UID: 0000000005 > From: Peter Memishian > Date: Wed, 14 Feb 2001 18:15:06 -0500 (EST) > To: jack.bochsler@sun.com, psarc@sac.eng.sun.com > Cc: carlsonj@east.sun.com > Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) > > > i'm not a psarc member, but this case was brought to my attention and > upon reading it over, i had a number of concerns that i felt needed to > be voiced. > > first, i want to reiterate a few points that jim made: > > 1. unless there's a technical need which is not stated in the > case, it seems like a poor idea to allow multiple > capabilities to be set at once. specifically, it leads to > confusing error semantics and more complicated payloads. i > think things would be a lot simpler if only a single > capability could be set in each request. > I am reworking the DL_IOC_INFO_REQ_ESP the private interface contract to address the error issues raised. Jim's comments were on the mark. > 2. DL_IOC_INFO_REQ_NONE seems overkill. it seems like it would > be simpler to just pass a M_IOCACK upstream with a NULL > b_cont. furthermore, i'm unclear why a driver would be > written to understand the DL_IOC_INFO_REQ primitive only > to report that it doesn't have any capabilities. > > secondly, i'm confused why this is being implemented using the ioctl > mechanism. as it currently stands, this proposal seems to have > nothing to do with dlpi aside from some naming conventions -- it is > merely a low-level ioctl-based sideband protocol with an identity > complex. it seems to me that these new primitives should either be > changed to follow the conventions of the dlpi protocol (e.g., use > M_PROTO messages, provide a set of new dl_primitive values for the new > message types, ...) or the proposal should be rewritten to explicitly > be an ioctl-based (or perhaps M_CTL-based) protocol that has nothing > to do with dlpi. > Agreed. In part, this was a reflection of the initial orientation of the design. I already have the respin in the new proposed format (thanks to the help of cpj@west). > finally, some additional questions and comments on the proposal: > > 1. the namespace chosen (dl_ioc_info_*) seems a little generic; > wouldn't something like dl_ioc_hwinfo_* be more appropriate? > _hwinfo_ makes the assumption that these features will be performed in hardware, but I think the only valid assumption that can be made is that the features will be addressed by "something downstream". Potentially, as Jim pointed out, the request could be answered (at least in part) by intermediate modules between the sender and the driver. dl_ioc_info_ is painfully generic, but dl_ioc_hwinfo_ may be proven wrong. > 2. in the dl_ioc_info_t, why is dl_data[1] a t_uscalar_t rather > than a uchar_t? no compelling reason. It gets cast to the appropriate payload prior to use (most likely, implementation dependant). > 3. i'm a little confused about DL_IOC_INFO_SET -- the proposal > mentions that it allows capabilities to be enabled *or* > disabled, but i don't see how one indicates that a given > capability should be disabled. can you clarify this > mechanism? > The high level is that there are two entry points, one to retrieve capability and one to set the state. The actual implementation of this is subject to the private interface contract. Specific fields can be allocated to indicate SETting on or off. > 4. the document does not specify what payload is sent downstream > with the DL_IOC_INFO_REQ. i'm guessing there is no payload; > is this correct? furthermore, the document is not explicit > about what the fields other than the dl_cap in the > dl_ioc_info_t can be set to in a DL_IOC_INFO_SET. could > you clarify this? > There is no payload, I will clarify. This will be addressed in the respin. > -- > meem From sac-list-owner Thu Feb 15 09:09:40 2001 From: Peter Memishian MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 15 Feb 2001 12:10:42 -0500 (EST) To: Jack Bochsler Cc: psarc@sac.eng.sun.com, Peter.Memishian@east.sun.com, James.D.Carlson@east.sun.com Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) Content-Length: 2299 Status: O X-Status: $$$$ X-UID: 0000000006 > _hwinfo_ makes the assumption that these features will be performed > in hardware, but I think the only valid assumption that can be made > is that the features will be addressed by "something downstream". > Potentially, as Jim pointed out, the request could be answered (at least > in part) by intermediate modules between the sender and the driver. > dl_ioc_info_ is painfully generic, but dl_ioc_hwinfo_ may be proven > wrong. well, the project's mission statement was stated: This project is a proposal to define an extensible request/response framework for upper layer software to determine device specific characteristics provided by specific hardware adapters and their drivers. it doesn't seem to me that this mission statement is as generic as "dl_ioc_info_*" would imply. > > 2. in the dl_ioc_info_t, why is dl_data[1] a t_uscalar_t rather > > than a uchar_t? > no compelling reason. It gets cast to the appropriate payload prior > to use (most likely, implementation dependant). okay, though uchar_t or uint8_t seem more appropriate for representing opaque data of arbitrary length. > > 3. i'm a little confused about DL_IOC_INFO_SET -- the proposal > > mentions that it allows capabilities to be enabled *or* > > disabled, but i don't see how one indicates that a given > > capability should be disabled. can you clarify this > > mechanism? > > > The high level is that there are two entry points, one to retrieve > capability and one to set the state. The actual implementation of > this is subject to the private interface contract. Specific fields > can be allocated to indicate SETting on or off. i see. i guess i would find a name like DL_IOC_INFO_CTL less confusing, especially given the comment in the document: #define DL_IOC_INFO_SET (DLIOC|12) /* device specific control */ furthermore, the document states: The format of the DL_IOC_INFO_SET command is a M_IOCTL mblk with ioc_cmd set to DL_IOC_INFO_SET and one or more mblks chained from the b_cont of the M_IOCTL mblk, each containing dl_ioc_info_t data, with the dl_cap field set to the capability that is to be enabled. ^^^^^^^ which seems to further confuse the issue. -- meem From sac-list-owner Thu Feb 15 09:29:46 2001 Date: Thu, 15 Feb 2001 09:27:45 -0800 (PST) From: Jack Bochsler Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) To: Jack.Bochsler@west.sun.com, meem@east.sun.com Cc: psarc@sac.eng.sun.com, Peter.Memishian@east.sun.com, James.D.Carlson@east.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: KJPiwB2mC4NoU9vRSvfK1Q== Content-Length: 2923 Status: O X-Status: $$$$ X-UID: 0000000007 > From: Peter Memishian > Date: Thu, 15 Feb 2001 12:10:42 -0500 (EST) > To: Jack Bochsler > Cc: psarc@sac.eng.sun.com, Peter.Memishian@east.sun.com, James.D.Carlson@east.sun.com > Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) > > > > _hwinfo_ makes the assumption that these features will be performed > > in hardware, but I think the only valid assumption that can be made > > is that the features will be addressed by "something downstream". > > Potentially, as Jim pointed out, the request could be answered (at least > > in part) by intermediate modules between the sender and the driver. > > dl_ioc_info_ is painfully generic, but dl_ioc_hwinfo_ may be proven > > wrong. > > well, the project's mission statement was stated: > > This project is a proposal to define an extensible > request/response framework for upper layer software to > determine device specific characteristics provided by specific > hardware adapters and their drivers. > > it doesn't seem to me that this mission statement is as generic as > "dl_ioc_info_*" would imply. > Agreed. The statement is skewed towards my specific need, and will be genericized in the respin. > > > 2. in the dl_ioc_info_t, why is dl_data[1] a t_uscalar_t rather > > > than a uchar_t? > > no compelling reason. It gets cast to the appropriate payload prior > > to use (most likely, implementation dependant). > > okay, though uchar_t or uint8_t seem more appropriate for representing > opaque data of arbitrary length. > Agreed. Will change. > > > 3. i'm a little confused about DL_IOC_INFO_SET -- the proposal > > > mentions that it allows capabilities to be enabled *or* > > > disabled, but i don't see how one indicates that a given > > > capability should be disabled. can you clarify this > > > mechanism? > > > > > The high level is that there are two entry points, one to retrieve > > capability and one to set the state. The actual implementation of > > this is subject to the private interface contract. Specific fields > > can be allocated to indicate SETting on or off. > > i see. i guess i would find a name like DL_IOC_INFO_CTL less > confusing, especially given the comment in the document: > > #define DL_IOC_INFO_SET (DLIOC|12) /* device specific control */ > > furthermore, the document states: > > The format of the DL_IOC_INFO_SET command is a M_IOCTL mblk > with ioc_cmd set to DL_IOC_INFO_SET and one or more mblks > chained from the b_cont of the M_IOCTL mblk, each containing > dl_ioc_info_t data, with the dl_cap field set to the > capability that is to be enabled. > ^^^^^^^ > which seems to further confuse the issue. > Agreed, this is a common point of confusion. Proposal respin replaces _SET with _CTL. jack From sac-list-owner Thu Feb 15 09:46:08 2001 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Thu, 15 Feb 2001 12:47:39 -0500 (EST) From: James Carlson To: Jack Bochsler Cc: psarc@sac.eng.sun.com Subject: Re: DL_IOC_INFO_REQ Request and Set Primitive (PSARC/2001/070) Content-Length: 4429 Status: O X-Status: $$$$ X-UID: 0000000008 Jack Bochsler writes: > > If you get ENOTSUP on DL_IOC_INFO_SET, how do you tell which one of > > the dl_cap entries sent was not supported? > > > The upper level software can send the entries down individually and > check the status on each reply. Then why doesn't it always do that? All I see here is added semantic complexity, since the error cases become some much harder to handle. > The more likely case is that the upper level software will send down > a DL_IOC_INFO_REQ and convert the response to a DL_IOC_INFO_SET, potentially > pruning unwanted features. Although this doesn't guarantee correctly > formed requests, it potentially simplifies the implementation. If you can't strictly guarantee correctly-formed requests, then why do this? I must be missing something, because it doesn't look all that hard to me to break up the DL_IOC_INFO_REQ response and send each element individually, since you need to iterate over it anyway to determine what things you want to enable or configure. > This seemed to provide more explicit notification, but I defer to your > opinion. I think fewer response types is better. > My intent was atomic, but I don't see any compelling reason why it > must be that way, so the actual implementation is delegated to > the private interface contract. The proposal will be rev'd to > explicitly state this. Atomicity across multiple unrelated requests concatentated together might be hard to handle. > DL_INFO_REQ provides a mechanism to capture failure. But as you point > out, there is no mechanism in the DL_INFO_REQ proposal to address > failure at a finer granularity. > I will address this failure granularity and the atomicity in the > DL_IOC_INFO_REQ_ESP proposal and re-submit. OK. > > A related question -- will there always be an inter-related request > > and set primitive? Is is possible that some capabilities are request > > only or set only? > > > At this time, I only envision inter-related request and set. > It is possible to request only - to decide that the capabilities > offered are not desirable (e.g. our US-V chip can implement the > algorithm faster than the co-processor). I meant that it might be the case that the request and set options are a little disjoint. There may be things where you can see that the capability exists, but there's no way to modify anything about it. (Think hardware configuration -- for example, "fiber is one element of a 4-fiber BLSR ring." There's nothing about this that software could "set," though it's conceivable that software may need to *know* this.) There may also be things that can be set but not queried (though I can't think of a good example off-hand). In this case, it makes sense to define DL_IOC_INFO_SET_xxx and DL_IOC_INFO_REQ_xxx for each capability, omitting one or the other if the corresponding operation is impossible. Otherwise, if there were a compete match-up, then it'd probably be better to define them from a common pool: #define DL_IOC_INFO_OPT_ESP 1 #define DL_IOC_INFO_OPT_AH 2 .... > A set implies that the sender fully understands the capabilities > existing below on the stream prior to crafting the REQuest. > Although this is possible, it seems improbable, unless a feature > became so uniform in nature and so ubiquitous in implementation > that it could be considered an optimization to just send the SET down. Perhaps it would happen if the corresponding feature represented merely a "hint" to the lower driver and didn't imply any necessary feature. > Yes, multiple modules can issue REQ/SETs. There is potential to get > conflicting states - not unlike the existing PSARC/1996/173/ implementation > that this replaces (which doesn't make it right). This could be mitigated > by a private interface contract flagging specifically what is turned on, > so modules could poll for the current status of what is "on". > > Yes, it is possible for intermediate modules to intercept and handle > some of the primitives. The latter is what I was actually referring to. If that is possible, then you probably have some interesting issues with the atomicity goal. -- James Carlson, Internet Engineering SUN Microsystems / 1 Network Drive 71.234W Vox +1 781 442 2084 MS UBUR02-212 / Burlington MA 01803-2757 42.497N Fax +1 781 442 1677 Second Edition now available - http://people.ne.mediaone.net/carlson/ppp From sac-list-owner Wed Feb 21 11:58:33 2001 Date: Wed, 21 Feb 2001 12:58:52 -0700 (MST) From: Andy Rudoff Subject: 2001/070 (DL_IOC_INFO_REQ Request and Set Primitive) To: psarc@sac.eng.sun.com Cc: Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: yh+sXyq9YiGJfkuc9vQ3Ag== Content-Length: 162 Status: O X-Status: $$$$ X-UID: 0000000009 I've marked this case as "waiting need spec". Once the off-line discussion has converged, please post the revised proposal and we can restart the timer. -andy From sac-list-owner Wed Feb 21 13:23:33 2001 Date: Wed, 21 Feb 2001 13:21:31 -0800 (PST) From: Jack Bochsler Subject: PSARC/2001/070/ DL_CAPABILITY_REQ/DL_CONTROL_REQ To: psarc@sac.eng.sun.com Cc: Jack.Bochsler@west.sun.com, David.Robinson@ebay.sun.com, sac@eng.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Span_of_Mules_165_000 Content-Length: 37143 Status: RO X-Status: $$$$ X-UID: 0000000010 --Span_of_Mules_165_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: swTjCS5NNIix1QrHE8Fm0g== Attached is a resubmit of my significantly revised fast-track case. I would also petition to rename the case (if possible) to more accurately reflect the revised proposal. New title: DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interfaces The revised proposal maintains the same charter - but provides a simpler, cleaner and more well-defined interface than the original proposal. Thanks to Erik Nordmark (EN), James Carlson (JC) and Peter Memishian (PM) who provided significant out-of-band feedback, input and review on the docs. Changes from original proposal (and requester for change): 1) Format from M_IOCTL to M_PROTO msg. (JC) 2) Creation of _CONTROL device control component (EN) 3) Increased error case definitions. (JC, PM) 4) DL_IOC_INFO_REQ_NONE removed (JC,PM) 5) Removed pure HW implication (PM) 6) changed data types to match convention (JC,PM) 8) Naming change from DL_INFO_REQ_ to DL_CAPABILITY - less generic (EN) 9) DL_INFO_REQ_ESP & DL_INFO_REQ_AH to DL_CAPABILITY_REQ (JC) 10) Capability to chain a command across multiple capabilities was removed. (JC, PM) 11) Added more complete control mechanism DL_CONTROL_REQ. (EN) 12) Removed chained mblk format (EN) jack --Span_of_Mules_165_000 Content-Type: TEXT/plain; name="dl_capability_req_ft.txt"; charset=us-ascii; x-unix-mode=0444 Content-Description: dl_capability_req_ft.txt Content-MD5: RE3pnas/3AqZZrvdMeGymg== --- DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interface for detecting, enabling and controlling DLS provider capabilities. jack.bochsler@west.sun.com 19 February 2001 COMMITMENT LEVEL DL_CAPABILITY_REQ extensible interface for detecting and enabling DLS provider capabilities. Consolidation-Private DL_CONTROL_REQ extensible interface for controlling DLS provider capabilities. Consolidation-Private RELEASE Minor. 1. Introduction 1.1. Project/Component Working Name: DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interfaces 1.2. Name of Document Author/Supplier: Jack Bochsler, jack.bochsler@sun.com 1.3. Date of This Document: 02/19/01 1.4. Name of Major Document Customer(s)/Consumer(s): IE, PSARC 2. Description There is an expanding array of hardware optimization capabilities being implemented in Network Interface Cards (NICs) that need to be discovered by IP so that IP knows when and how to exploit them. There is an existing interface for IP to discover hardware checksumming capability but this interface is not extensible to cover the various requirements of other capabilities such as encryption and authentication. This project is a proposal to define an extensible request/response framework that supports arbitrary capabilities and lengths of data to allow upper layer software to determine device specific characteristics provided by lower level software, hardware adapters and drivers. This project proposes to introduce a generic Consolidation-Private DLS provider interface which is a simple but extensible framework for communications between upper layer software to assess and control capabilities provided by lower layers of software and hardware. Separate Contract-Private Interfaces between the producers (e.g. NIC device drivers) and consumers (IP) of the data would be necessary prior to the actual use of this interface. Note: PSARC/1996/173/ defined a non-extensible Contract Private interface between IP and the SMCC SAHI3 155/622 ATM network interface device driver (ba) for the negotiation of inetcksuming. The hardware checksum and M_DATA fastpath negotiations must evolve to use this new framework, but there are compatibility constraints which makes this a separate task. 3. External Requester N/A 4. Opportunity Window/Exposure Enable a uniform interface for communicating device specific info between IP and lower level software. 5. Business Opportunity/Exposure There are an increasing number of NICs that have hardware acceleration for multiple reasons. A mechanism needs to be in place for IP to detect and exploit these capabilities. The Venus project is slated for the S9U1 time period, so these changes need to be integrated in S9. 6. Technical Description This proposal creates the following interfaces: |-----------------------|-----------------------|---------------| |Interface | Classification | Comments | |-----------------------|-----------------------|---------------| |DL_CAPABILITY_REQ | Consolidation Private | | |DL_CAPABILITY_ACK | Consolidation Private | | |DL_CONTROL_REQ | Consolidation Private | | |DL_CONTROL_ACK | Consolidation Private | | |-----------------------|-----------------------|---------------| The following changes are recommended to the existing header file in the "Sun additions" area: #define DL_CAPABILITY_REQ (0x100) /* Sun device specific capability request */ #define DL_CAPABILITY_ACK (0x101) /* Sun capability ack */ #define DL_CONTROL_REQ (0x102) /* Sun device specific control request */ #define DL_CONTROL_ACK (0x103) /* Sun control ack */ The following #defines with proposed initial primitives and the generic response data structures format would be added to the bottom of the file: /* * DL_CAPABILITY_REQ, DL_CAPABILITY_ACK, M_PROTO type * * This is used both to determine the set of capabilities in the * DLS provider and also to turn on and off specific capabilities. * The response is a DL_CAPABILITY_ACK or DL_ERROR_ACK. * * DL_CAPABILITY_REQ can either be empty (i.e. dl_sub_length is zero) * which is a request for the driver to return all capabilities. * Otherwise, the DL_CAPABILITY_REQ contains the capabilities the * DLS user wants to use and their settings. * * DL_CAPABILITY_ACK contains the capabilities that the DLS provider can * support modified based on what was listed in the request. If a capability * was included in the request then the information returned in the ack might * be modified based on the information in the request. */ typedef struct { t_uscalar_t dl_primitive; /* DL_CAPABILITY_REQ/ACK */ t_uscalar_t dl_sub_offset; /* options length */ t_uscalar_t dl_sub_length; /* options offset */ /* Followed by a list of zero or more dl_capability_sub_t */ } dl_capability_t; typedef struct { t_uscalar_t dl_cap; /* capability type */ t_uscalar_t dl_length; /* length of data following */ /* Followed by zero or more bytes of dl_data */ } dl_capability_sub_t; /* * Capability types */ #define DL_CAPAB_FASTPATH 0x00 /* DLPI M_DATA fastpath capable */ /* dl_data is uchar_t[1] for\ enable/disable */ #define DL_CAPAB_INETCKSUM 0x01 /* inetcksum capable */ /* dl_data is inetcksum_t */ #define DL_CAPAB_IPSEC_AH 0x02 /* IPsec AH acceleration */ /* dl_data is dl_capab_ipsec_t */ #define DL_CAPAB_IPSEC_ESP 0x03 /* IPsec ESP acceleration */ /* dl_data is dl_capab_ipsec_t */ /* * DL_CONTROL_REQ, DL_CONTROL_ACK, M_PROTO type * * Extensible message to send down control information to the DLS provider. * The response is a DL_CONTROL_ACK or DL_ERROR_ACK. * * Different types of control operations will define different format * for the key and data fields. * ADD requires key and data fields; if the matches * an already existing entry a DL_ERROR_ACK will be returned. * DELETE requires a key field; if the does not exist, * a DL_ERROR_ACK will be returned. * FLUSH requires neither a key nor data; it unconditionally removes * all entries for the specified type. * GET requires a key field; the get operation returns the data for the * . If doesn't exist a DL_ERROR_ACK is returned. * UPDATE requires key and data fields; if doesn't * exist a DL_ERROR_ACK is returned. */ typedef struct { t_uscalar_t dl_primitive; /* DL_CONTROL_REQ/DL_CONTROL_ACK */ t_uscalar_t dl_operation; /* add/delete/purge */ t_uscalar_t dl_type; /* e.g. AH/ESP/ ... */ t_uscalar_t dl_key_offset; /* offset of key */ t_uscalar_t dl_key_length; /* length of key */ t_uscalar_t dl_data_offset; /* offset of data */ t_uscalar_t dl_data_length; /* length of data */ } dl_control_t; /* * Control operations */ #define DL_CO_ADD 0x01 /* Add new entry matching for */ #define DL_CO_DELETE 0x02 /* Delete the entry matching */ #define DL_CO_FLUSH 0x03 /* Purge all entries of */ #define DL_CO_GET 0x04 /* Get the data for the */ #define DL_CO_UPDATE 0x05 /* Update the data for */ /* * Control types */ #define DL_CT_IPSEC_AH 0x01 /* AH; key = spi; data = keying material */ #define DL_CT_IPSEC_ESP 0x02 /* ESP; key = spi; data = keying material */ #define DL_CT_FASTPATH_PROBE 0x03 /* DL_IOC_HDR_INFO replacement; key = dl_unitdata_req_t; /* data (in ack) = fastpath header */ #define DL_CT_INETCKSUM 0x04 /* ick M_CTL replacement; key = */ #define DL_CT_CLASSIFY 0x05 /* packet classifier; key = packet matching rule */ /* structure; data = actions on packet */ It is expected that additional capability types may be added to over time, e.g. for zero copy support. In order for these primitives to be implemented, separate ARC cases would be necessary to establish the specific semantics, data content, format and error handling. There are two components to this proposal, DL_CAPABILITY_REQ, and DL_CONTROL_REQ. If a command is not implemented, it will be readily discernible by the error acknowledgement as defined below. There is no expected or implied ordering in the use of these messages. Applicability of primitives to specific capabilities is subject to the definition from the Contract-Private agreement and definition of the use of this interface. The first primitive, DL_CAPABILITY_REQ, allows a DLS user to determine and control device specific capabilities provided by the DLS provider. The second primitive, DL_CONTROL_REQ, is used by the upper layer software to pass configuration data down to the DLS provider, or to retrieve configuration data from the DLS provider. 6.1 DL_CAPABILITY_REQ There are two forms of DL_CAPABILITY_REQ. If an empty message (dl_sub_length is 0) is issued to the DLS provider a DL_CAPABILITY_ACK will return all the capabilities provided by the device. The second form of the command will contain enumeration of the specific capabilities the DLS user wants the DLS provider to configure. If the lower level driver supports the DL_CAPABILITY_REQ primitive, if the dl_sub_length is 0, the DLS provider will reply with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t structures following in the response mblk which will enumerate the capabilities of the device. A request exchange to determine all capabilities will appear as follows: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - 0 dl_sub_length - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does not support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t with the following values: dl_error_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CAPABILITY_REQ dl_errno - DL_UNSUPPORTED dl_unix_errno - 0 If the driver has no DL_CAPABILITY_REQ capabilities, a dl_error_ack_t will be returned with the following values: dl_error_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CAPABILITY_REQ dl_errno - DL_NOTSUPPORTED dl_unix_errno - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does support DL_CAPABILITY_REQ and has capabilities to report, a successful response will consist of a dl_capability_t followed by one or more dl_capability_sub_t structures. Each dl_capability_sub_t will provide a capability type, and 0 or more bytes of capability data to enumerate the Contract Private details of the device capability available. Each dl_capability_sub_t can then encapsulate as much data as necessary to describe the feature being enumerated in the dl_data portion of the dl_capability_sub_t. The Contract Private format of the data should include specific information or status on the capability being reported so that the DLS user can determine how the capability is configured - for example whether it is enabled or disabled. A successful response will result in a reply with the structure elements set as following: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - total length of dl_capability_sub_ts structures following dl_capability_sub_t: dl_cap - first capability type dl_length - length of data following dl_data - Contract Private data The second form of the DL_CAPABILITY_REQ command is used by the DLS user to set capability settings on the DLS provider. The upper level software will issue a DL_CAPABILITY_REQ with a single dl_capability_sub_t containing the settings for the DLS provider to use. Note that unlike the DL_CAPABILITY_ACK response above, this command may contain only one dl_capability_sub_t. The dl_capability_sub_t will contain the Contract Private details of what capabilities the DLS provider should enable or disable. This command will be of the form: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - capability type dl_length - length of data following dl_data - Contract Private data In response to a DL_CAPABILITY_REQ of this form, the DLS provider will reply with dl_error_ack_t structures as above if the primitive is not recognized or supported. If the DL_CAPABILITY_REQ is supported, the reply will be of the form: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - capability type dl_length - length of data following dl_data - Contract Private data The information returned in the dl_data portion of the acknowledgement might be modified based on the information in the request. If an error occurs in enabling a configuring a capability, a dl_error_ack_t will be returned of the following form: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_BADPRIM dl_unix_errno - 0 6.2 DL_CONTROL_REQ The second primitive, DL_CONTROL_REQ, allows upper layer software to pass control data down to the DLS provider. This is implemented as a different interface to separate control and data information. The format of the DL_CONTROL_REQ command is a M_PROTO dl_control_t structure. This structure is followed by optional dl_key data and optional dl_data. Note that the actual content, format and resultant processing of the M_PROTO dl_key and dl_data fields are Contract-Private. This interface provides a mechanism for a DLS user to create and maintain one or more keyed data sets with the DLS provider. The commands consist of operation, meta-data as well as the data: - command of specifically what operations to perform with the data set (dl_operation) - type information so that the DLS provider can maintain more than one data set (dl_type) - key data so that the data set can be addressed with keyed record granularity (dl_key) - data, in a Contract Private format (dl_data) The contents of the dl_operation element will define the requirements for the dl_key and dl_data fields, the DLS provider behavior associated with the request, and the contents of the DL_CONTROL_ACK response. If the dl_operation is a DL_CO_ADD, the M_PROTO requires the dl_key and dl_data fields. If the matches an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will add the data, keyed on the . Successful operation will return a DL_CONTROL_ACK with the original dl_key and dl_data. If the dl_operation is a DL_CO_DELETE, the M_PROTO requires a dl_key field. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will delete the entry based on the . Successful operation will return a DL_CONTROL_ACK with the original dl_key. If the dl_operation is a DL_CO_FLUSH, the M_PROTO requires no further data. On receipt of this control message, the DLS provider will flush all entries for the specified dl_type. Successful operation will return a DL_CONTROL_ACK without dl_key or dl_data. If the dl_operation is a DL_CO_GET, the M_PROTO requires a dl_key field. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will return the entry based on the . Successful operation will return a DL_CONTROL_ACK with original dl_key and the corresponding dl_data. If the dl_operation is a DL_CO_UPDATE, the M_PROTO must also contain dl_key and dl_data fields. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will replace the existing data associated with the with the the data from the dl_data field. Successful operation return a DL_CONTROL_ACK with the original dl_key and dl_data. The format of a DL_CONTROL_REQ command is a dl_control_t with the following values: dl_control_t: dl_primitive - DL_CONTROL_REQ dl_operation - control operation desired dl_type - DLS provider capability to be controlled by this request dl_key_offset - sizeof (dl_control_t) dl_key_length - length of key following dl_data_offset - sizeof (dl_control_t) + dl_key_length dl_data_length - length of data following The dl_control_t is followed by optional dl_key and dl_data elements. The expected responses to a DL_CONTROL_REQ follow. In response to a DL_CONTROL_REQ, if the driver does not understand/support DL_CONTROL_REQ the lower level software will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_UNSUPPORTED dl_unix_errno - 0 If the driver has no capabilities, the lower level software will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_NOTSUPPORTED dl_unix_errno - 0 If the DLS provider does not recognize the capability type specified in the dl_type data field, it will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_BADPRIM dl_unix_errno - 0 If the driver does support DL_CONTROL_REQ, the DLS provider will perform the requested operation and if an error occurs as described per operation above, will return a DL_ERROR_ACK. If successful, the DLS provider reply with a dl_control_t with the following values: dl_control_t: dl_primitive - DL_CONTROL_ACK dl_operation - control operation requested dl_type - DLS provider capability to be controlled by this request dl_key_offset - sizeof (dl_control_t) dl_key_length - length of key following dl_data_offset - sizeof (dl_control_t) + dl_key_length dl_data_length - length of data following 7. Reference Documents PSARC/1996/173/ "TCP/IP support for SAHI3 hardware checksuming", 03/27/96, Bruce Curtis, brutus@Eng 8. Projected Availability Solaris 9 9. Cost of Effort 1.0 man month. 10. Prototype Availability 03/30/01 11. Prototype Cost 0.5 man month. 12. Cost of Capital Resources N/A --Span_of_Mules_165_000 Content-Type: TEXT/plain; name="dl_capability_ipsec_ft.txt"; charset=us-ascii; x-unix-mode=0444 Content-Description: dl_capability_ipsec_ft.txt Content-MD5: 7ZRcITDvs1X5BiBMQIWmWA== --- DL_CAPAB_IPSEC_ESP /DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ jack.bochsler@west.sun.com 15 February 2001 COMMITMENT LEVEL DL_CAPAB_IPSEC_ESP Encryption Sub-capability Primitive Contract-Private DL_CAPAB_IPSEC_AH Authentication Sub-capability Primitive Contract-Private DL_CT_IPSEC_ESP Encryption Sub-control Primitive Contract-Private DL_CT_IPSEC_AH Authentication Sub-control Primitive Contract-Private RELEASE Minor. BACKGROUND The Venus project is developing a NIC which provides hardware acceleration of IPsec algorithms. In order for IPsec to exploit this capability, it needs to identify, control and configure this functionality. PSARC/2001/070/ defined a generic request/control mechanism to identify and control DLS provider capabilities. This proposal describes a Contract-Private implementation for IPsec to identify and control DLS provider authentication and encryption capabilities via the messages and payload areas of PSARC/2001/070/. PROBLEM In order for IPsec to take advantage of hardware acceleration in the Venus card, a method must be employed to identify specifically which algorithms may be accelerated in hardware. PSARC/2001/070/ defines a method for polling DLS providers to determine their specific capabilities. IPsec will also need a mechanism to control (enable/disable) functionality, and a mechanism to push SA information down to the Venus driver/card. This proposal defines interfaces and implementation to be layered on PSARC/2001/070/ to provide IPsec acceleration capability identification and control. This describes primitives and data structures used to pass information between IP and the Venus driver. PROPOSAL This proposal creates the following interfaces, layered on the Consolidation Private interface as defined in PSARC/2001/070: |-----------------------|-----------------------|---------------| |Interface | Classification | Comments | |-----------------------|-----------------------|---------------| |DL_CAPAB_IPSEC_AH | Contract Private | | |DL_CAPAB_IPSEC_ESP | Contract Private | | |DL_CT_IPSEC_AH | Contract Private | | |DL_CT_IPSEC_ESP | Contract Private | | |-----------------------|-----------------------|---------------| The following changes are recommended to the existing DLPI interfaces and are to be incorporated into : /* * Note that numbers correspond to the types in the IPsec DOI document. */ #define DL_CAPAB_IPSEC_AH 0x02 /* IPsec AH acceleration */ #define DL_CAPAB_IPSEC_ESP 0x03 /* IPsec ESP acceleration */ typedef struct { t_uscalar_t cip_version; /* interface version */ t_uscalar_t cip_nciphers; /* number ciphers supported */ dl_capab_ipsec_alg_t cip_data[1]; /* data */ } dl_capab_ipsec_t; typedef struct { t_uscalar_t alg_prim; /* algorithm primitive */ t_uscalar_t alg_thruput; /* approx throughput metric in Mb/s */ t_uscalar_t alg_nkeylen; /* num supported key lengths */ t_uscalar_t alg_flag; /* flags */ t_uscalar_t alg_skey[1]; /* supported key length */ } dl_capab_ipsec_alg_t; /* alg_prim ciphers */ #define DL_CAPAB_IPSEC_ESP_NONE 0x00 /* no cipher */ #define DL_CAPAB_IPSEC_ESP_DES 0x02 #define DL_CAPAB_IPSEC_ESP_3DES 0x03 #define DL_CAPAB_IPSEC_ESP_BLOWFISH 0x07 /* alg_prim authentications */ #define DL_CAPAB_IPSEC_AH_NONE 0x00 /* no authentication */ #define DL_CAPAB_IPSEC_AH_MD5HMAC 0x02 #define DL_CAPAB_IPSEC_AH_SHA1HMAC 0x03 /* alg_flag values */ #define DL_CAPAB_ALG_ENABLE 0x01 /* enable this algorithm */ /* * Control types */ #define DL_CT_IPSEC_AH 0x01 /* AH; key=spi; data=keying material */ #define DL_CT_IPSEC_ESP 0x02 /* ESP; key=spi; data=keying material */ /* * For DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP, the optional dl_key data * that follows the dl_control_t will be the IPsec SPI (Security * Parameters Index) value. This is defined as being unique per protocol. */ uint32_t dl_key_spi; /* Security Parameters Index value */ #define DL_MAX_KEY_LEN 512 /* max key length in bytes */ /* * minimal SADB entry content * fields are defined as per RFC 2367 and * This defines the content and format of the dl_data portion of * the dl_control_t */ typedef struct capability_cfg_sadb { uint8_t sadb_sa_auth; /* Authentication algorithm */ uint8_t sadb_sa_encrypt; /* Encryption algorithm */ uint32_t sadb_sa_flags; /* SA flags. */ uint16_t sadb_address_exttype_s; /* SA address - SRC */ struct sockaddr_storage sockaddr_s; uint16_t sadb_address_exttype_d; /* SA address - DST */ struct sockaddr_storage sockaddr_d; uint16_t sadb_address_exttype_p; /* SA address - PROXY */ struct sockaddr_storage sockaddr_p; uint16_t sadb_key_len_a; /* AUTH key length */ uint16_t sadb_key_exttype_a; /* AUTH */ uint16_t sadb_key_bits_a; /* Actual key length in bits */ uint16_t sadb_key_data_a[MAX_KEY_LEN]; /* key data */ uint16_t sadb_key_len_e; /* ENCRYPT key length */ uint16_t sadb_key_exttype_e; /* ENCRYPT */ uint16_t sadb_key_bits_e; /* Actual key length in bits */ uint16_t sadb_key_data_e[MAX_KEY_LEN]; /* key data */ } dl_capability_cfg_sa_t; The above structures define the payload (dl_data) area of the input and response to DL_CAPABILITY_REQ and DL_CONTROL_REQ commands as defined in PSARC/2001/070. The capability mechanism is initiated by the DLS user (IPsec) issuing a M_PROTO mblk containing a dl_capability_t with dl_primitive set to DL_CAPABILITY_REQ to the DLS provider. Zero or more dl_capab_ipsec_t structures will follow in the M_PROTO. The control mechanism is initiated by the DLS user as well, issuing a M_PROTO message containing a dl_control_t with the dl_primitive set to DL_CONTROL_REQ, a dl_operation specified and a dl_type to specify the DLS provider dataset on which the dl_operation is to take place. The dl_control_t followed by optional dl_key and dl_data, based on the dl_operation primitive. The dl_data encapsulates a dl_capability_cfg_sa_t if present. Based on the contents of the M_PROTO message, the DLS provider (Venus driver) will perform the actions and respond with a DL_CAPABILITY_ACK or DL_CONTROL_ACK as described in more detail below. DL_CAPABILITY_REQ: On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is 0, this initiates a query for capabilities. The DLS provider will reply with a DL_CAPABILITY_ACK (as per PSARC/2001/070/) with dl_capability_sub_t structures following, one per capability supported by the DLS provider. For DLS providers that provide IPsec acceleration capabilities there would be one dl_capability_sub_t for ESP (encryption) and a separate one for AH (authentication), with potentially additional dl_capability_sub_t structs for other capabilities. The contents of the dl_capability_t and the dl_capability_sub_t are defined as: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have zero or more dl_capab_ipsec_t structures following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms supported by the device. The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - 0 alg_skey - variable number of elements reflecting each supported key length for this algorithm On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is not 0, this signifies that the DLS user is passing additional data to the DLS provider to configure a capability or its' subfeatures on or off. Note that all capabilities supported by the DLS provider are disabled until they are explicitly enabled. The contents of the dl_capability_t and the dl_capability_sub_t are: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers being enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms to be enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - DL_CAPAB_ALG_ENABLE to enable this algorithm or 0 to disable alg_skey - variable number of elements reflecting each supported key length to be enabled In reply to the DL_CAPABILITY_REQ with attached data, the DLS provider will return a DL_CAPABILITY_ACK of the following form: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - In the DL_CAPABILITY_ACK response to a configuration request (DL_CAPABILITY_REQ command with no data), DL_CAPAB_ALG_ENABLE will be set if this algorithm has been previously enabled, 0 otherwise. In a DL_CAPABILITY_REQ with configuration data, DL_CAPAB_ALG_ENABLE will be set to enable this algorithm. The response DL_CAPABILITY_ACK will also to have this flag set to confirm that the capability was enabled, or cleared to confirm that it was disabled. alg_skey - Variable number of elements reflecting each enabled algorithm key length DL_CONTROL_REQ: As described in PSARC/2001/070/, a DL_CONTROL_REQ command is used by upper layer software to configure the capabilities returned by the response to DL_CAPABILITY_REQ. Capability configuration as defined by this proposal is the process of downloading, updating and retrieving IPsec SADB information to the underlying Venus driver. The Venus driver will use the SADB information to cache and maintain keying material which is necessary to apply hardware acceleration to IPsec packets. In constructing a DL_CONTROL_REQ command, upper layer software will send down to the DLS provider an M_PROTO mblk containing a dl_control_t structure with dl_primitive set to DL_CONTROL_REQ. For this proposal, the dl_type field will be set to DL_CT_IPSEC_AH to control IPsec authentication data at the DLS provider or DL_CT_IPSEC_ESP to control IPsec encryption data at the DLS provider. Based on the dl_operation element of the dl_control_t structure, optional dl_key data and dl_data will follow. The dl_key for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is dl_key_spi, which is the IPsec SPI value of the SA. PSARC/2001/070/ specifies which data fields are mandatory per dl_operation control operation. The dl_data for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is the dl_ct_ipsec_t structure, which contains a minimal SADB entry to provide the required information to enable hardware accelerated encryption and authentication. The dl_data element following dl_control_t contains a dl_ct_ipsec_t which will be filled in with the the elements defined as per and RFC 2367. dl_control_t: dl_primitive - DL_CONTROL_REQ or DL_CONTROL_ACK dl_operation - control operation dl_type - DL_CT_IPSEC_AH or DL_CT_IPSEC_ESP dl_key_offset - sizeof(dl_control_t) dl_key_length - length of key data dl_data_offset - sizeof(dl_control_t) + dl_key_length dl_data_length - sizeof(dl_ct_ipsec_t) The dl_operation element of the dl_control_t has the following semantics (these semantics are aligned with base message types defined in RFC 2367): DL_CO_ADD - Requires dl_type, dl_key and dl_data. Add this entry to the DLS provider dataset, keyed at . If an entry already exists at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. If there is insufficient room to add the entry, response is a DL_ERROR_ACK with return value DL_TOOMANY. Otherwise a DL_CONTROL_ACK will be returned, containing the same data as the original DL_CONTROL_REQ. DL_CO_DELETE - Requires dl_type, dl_key. Delete the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_FLUSH - Requires dl_type. Delete all entries of type dl_type from the DLS provider dataset. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_GET - Requires dl_type, dl_key. Return the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned with a dl_ct_ipsec_t structure populated with the data associated with this . DL_CO_UPDATE - Requires dl_type, dl_key and dl_data. Update the entry in the DLS provider dataset, keyed at . If an entry doesn't already exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. REFERENCE DOCUMENTS PSARC/2001/070/ DL_CAPABILITY_REQ/CTL extensible interface for detecting and/or enabling lower level driver capabilities. pf_key(7P) man page /usr/include/net/pfkeyv2.h RFC 2367: PF_KEY Key Management API, Version 2 --Span_of_Mules_165_000-- From sac-list-owner Wed Feb 21 14:08:26 2001 Date: Wed, 21 Feb 2001 14:06:16 -0800 (PST) From: Jack Bochsler Subject: Re: PSARC/2001/070/ DL_CAPABILITY_REQ/DL_CONTROL_REQ To: David.Robinson@ebay.sun.com Cc: Jack.Bochsler@west.sun.com, psarc@sac.eng.sun.com, sac@eng.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Pladge_of_Wasps_202_000 Content-Length: 19265 Status: RO X-Status: $$$$ X-UID: 0000000011 --Pladge_of_Wasps_202_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: 5I/uJUghspR2YHRCONnf5A== David: Can you please submit the following doc as a separate fast-track, per James Carlson. Although they need to go through together for clarity and continuity, the IPsec implementation is dependant on the interface design (and not the opposite). Note that the attached is an updated doc of the one I sent out earlier - It has revised text that notes the dependancy on PSARC/2001/070/ (as well a typo fix and structure member rename for clarity). jack > Date: Wed, 21 Feb 2001 13:21:31 -0800 (PST) > From: Jack Bochsler > Subject: PSARC/2001/070/ DL_CAPABILITY_REQ/DL_CONTROL_REQ > To: psarc@sac.eng.sun.com > Cc: Jack.Bochsler@west.sun.com, David.Robinson@ebay.sun.com, sac@eng.sun.com > > Attached is a resubmit of my significantly revised > fast-track case. > > I would also petition to rename the case (if possible) to > more accurately reflect the revised proposal. New title: > > DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interfaces > > The revised proposal maintains the same charter - but provides > a simpler, cleaner and more well-defined interface than the > original proposal. Thanks to Erik Nordmark (EN), James Carlson (JC) > and Peter Memishian (PM) who provided significant out-of-band feedback, > input and review on the docs. > > Changes from original proposal (and requester for change): > 1) Format from M_IOCTL to M_PROTO msg. (JC) > 2) Creation of _CONTROL device control component (EN) > 3) Increased error case definitions. (JC, PM) > 4) DL_IOC_INFO_REQ_NONE removed (JC,PM) > 5) Removed pure HW implication (PM) > 6) changed data types to match convention (JC,PM) > 8) Naming change from DL_INFO_REQ_ to DL_CAPABILITY - less generic (EN) > 9) DL_INFO_REQ_ESP & DL_INFO_REQ_AH to DL_CAPABILITY_REQ (JC) > 10) Capability to chain a command across multiple capabilities > was removed. (JC, PM) > 11) Added more complete control mechanism DL_CONTROL_REQ. (EN) > 12) Removed chained mblk format (EN) > > jack --Pladge_of_Wasps_202_000 Content-Type: TEXT/plain; name="dl_capability_ipsec_ft.txt"; charset=us-ascii; x-unix-mode=0444 Content-Description: dl_capability_ipsec_ft.txt Content-MD5: LkvL7p+B1HhfAX6zlB09Ag== --- DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ jack.bochsler@west.sun.com 21 February 2001 COMMITMENT LEVEL DL_CAPAB_IPSEC_ESP Encryption Sub-capability Primitive Contract-Private DL_CAPAB_IPSEC_AH Authentication Sub-capability Primitive Contract-Private DL_CT_IPSEC_ESP Encryption Sub-control Primitive Contract-Private DL_CT_IPSEC_AH Authentication Sub-control Primitive Contract-Private RELEASE Minor. BACKGROUND The Venus project is developing a NIC which provides hardware acceleration of IPsec algorithms. In order for IPsec to exploit this capability, it needs to identify, control and configure this functionality. PSARC/2001/070/ defined a generic request/control mechanism to identify and control DLS provider capabilities. This proposal describes a Contract-Private implementation for IPsec to identify and control DLS provider authentication and encryption capabilities via the messages and payload areas of PSARC/2001/070/. This proposal is being submitted simultaneously with PSARC/2001/070/ and is dependant on it. PROBLEM In order for IPsec to take advantage of hardware acceleration in the Venus card, a method must be employed to identify specifically which algorithms may be accelerated in hardware. PSARC/2001/070/ defines a method for polling DLS providers to determine their specific capabilities. IPsec will also need a mechanism to control (enable/disable) functionality, and a mechanism to push SA information down to the Venus driver/card. This proposal defines interfaces and implementation to be layered on PSARC/2001/070/ to provide IPsec acceleration capability identification and control. This describes primitives and data structures used to pass information between IP and the Venus driver. PROPOSAL This proposal creates the following interfaces, layered on the Consolidation Private interface as defined in PSARC/2001/070: |-----------------------|-----------------------|---------------| |Interface | Classification | Comments | |-----------------------|-----------------------|---------------| |DL_CAPAB_IPSEC_AH | Contract Private | | |DL_CAPAB_IPSEC_ESP | Contract Private | | |DL_CT_IPSEC_AH | Contract Private | | |DL_CT_IPSEC_ESP | Contract Private | | |-----------------------|-----------------------|---------------| The following changes are recommended to the existing DLPI interfaces and are to be incorporated into : /* * Note that numbers correspond to the types in the IPsec DOI document. */ #define DL_CAPAB_IPSEC_AH 0x02 /* IPsec AH acceleration */ #define DL_CAPAB_IPSEC_ESP 0x03 /* IPsec ESP acceleration */ typedef struct { t_uscalar_t cip_version; /* interface version */ t_uscalar_t cip_nciphers; /* number ciphers supported */ dl_capab_ipsec_alg_t cip_data[1]; /* data */ } dl_capab_ipsec_t; typedef struct { t_uscalar_t alg_prim; /* algorithm primitive */ t_uscalar_t alg_thruput; /* approx throughput metric in Mb/s */ t_uscalar_t alg_flag; /* flags */ t_uscalar_t alg_nkeylen; /* num supported key lengths */ t_uscalar_t alg_keylen[1]; /* supported key length list */ } dl_capab_ipsec_alg_t; /* alg_prim ciphers */ #define DL_CAPAB_IPSEC_ESP_NONE 0x00 /* no cipher */ #define DL_CAPAB_IPSEC_ESP_DES 0x02 #define DL_CAPAB_IPSEC_ESP_3DES 0x03 #define DL_CAPAB_IPSEC_ESP_BLOWFISH 0x07 /* alg_prim authentications */ #define DL_CAPAB_IPSEC_AH_NONE 0x00 /* no authentication */ #define DL_CAPAB_IPSEC_AH_MD5HMAC 0x02 #define DL_CAPAB_IPSEC_AH_SHA1HMAC 0x03 /* alg_flag values */ #define DL_CAPAB_ALG_ENABLE 0x01 /* enable this algorithm */ /* * Control types */ #define DL_CT_IPSEC_AH 0x01 /* AH; key=spi; data=keying material */ #define DL_CT_IPSEC_ESP 0x02 /* ESP; key=spi; data=keying material */ /* * For DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP, the optional dl_key data * that follows the dl_control_t will be the IPsec SPI (Security * Parameters Index) value. This is defined as being unique per protocol. */ typedef uint32_t dl_key_spi; /* Security Parameters Index value */ #define DL_MAX_KEY_LEN 512 /* max key length in bytes */ /* * minimal SADB entry content * fields are defined as per RFC 2367 and * This defines the content and format of the dl_data portion of * the dl_control_t */ typedef struct capability_cfg_sadb { uint8_t sadb_sa_auth; /* Authentication algorithm */ uint8_t sadb_sa_encrypt; /* Encryption algorithm */ uint32_t sadb_sa_flags; /* SA flags. */ uint16_t sadb_address_exttype_s; /* SA address - SRC */ struct sockaddr_storage sockaddr_s; uint16_t sadb_address_exttype_d; /* SA address - DST */ struct sockaddr_storage sockaddr_d; uint16_t sadb_address_exttype_p; /* SA address - PROXY */ struct sockaddr_storage sockaddr_p; uint16_t sadb_key_len_a; /* AUTH key length */ uint16_t sadb_key_exttype_a; /* AUTH */ uint16_t sadb_key_bits_a; /* Actual key length in bits */ uint16_t sadb_key_data_a[MAX_KEY_LEN]; /* key data */ uint16_t sadb_key_len_e; /* ENCRYPT key length */ uint16_t sadb_key_exttype_e; /* ENCRYPT */ uint16_t sadb_key_bits_e; /* Actual key length in bits */ uint16_t sadb_key_data_e[MAX_KEY_LEN]; /* key data */ } dl_capability_cfg_sa_t; The above structures define the payload (dl_data) area of the input and response to DL_CAPABILITY_REQ and DL_CONTROL_REQ commands as defined in PSARC/2001/070. The capability mechanism is initiated by the DLS user (IPsec) issuing a M_PROTO mblk containing a dl_capability_t with dl_primitive set to DL_CAPABILITY_REQ to the DLS provider. Zero or more dl_capab_ipsec_t structures will follow in the M_PROTO. The control mechanism is initiated by the DLS user as well, issuing a M_PROTO message containing a dl_control_t with the dl_primitive set to DL_CONTROL_REQ, a dl_operation specified and a dl_type to specify the DLS provider dataset on which the dl_operation is to take place. The dl_control_t followed by optional dl_key and dl_data, based on the dl_operation primitive. The dl_data encapsulates a dl_capability_cfg_sa_t if present. Based on the contents of the M_PROTO message, the DLS provider (Venus driver) will perform the actions and respond with a DL_CAPABILITY_ACK or DL_CONTROL_ACK as described in more detail below. DL_CAPABILITY_REQ: On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is 0, this initiates a query for capabilities. The DLS provider will reply with a DL_CAPABILITY_ACK (as per PSARC/2001/070/) with dl_capability_sub_t structures following, one per capability supported by the DLS provider. For DLS providers that provide IPsec acceleration capabilities there would be one dl_capability_sub_t for ESP (encryption) and a separate one for AH (authentication), with potentially additional dl_capability_sub_t structs for other capabilities. The contents of the dl_capability_t and the dl_capability_sub_t are defined as: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have zero or more dl_capab_ipsec_t structures following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms supported by the device. The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - 0 alg_keylen - variable number of elements reflecting each supported key length for this algorithm On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is not 0, this signifies that the DLS user is passing additional data to the DLS provider to configure a capability or its' subfeatures on or off. Note that all capabilities supported by the DLS provider are disabled until they are explicitly enabled. The contents of the dl_capability_t and the dl_capability_sub_t are: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers being enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms to be enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - DL_CAPAB_ALG_ENABLE to enable this algorithm or 0 to disable alg_keylen - variable number of elements reflecting each supported key length to be enabled In reply to the DL_CAPABILITY_REQ with attached data, the DLS provider will return a DL_CAPABILITY_ACK of the following form: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - In the DL_CAPABILITY_ACK response to a configuration request (DL_CAPABILITY_REQ command with no data), DL_CAPAB_ALG_ENABLE will be set if this algorithm has been previously enabled, 0 otherwise. In a DL_CAPABILITY_REQ with configuration data, DL_CAPAB_ALG_ENABLE will be set to enable this algorithm. The response DL_CAPABILITY_ACK will also to have this flag set to confirm that the capability was enabled, or cleared to confirm that it was disabled. alg_keylen - Variable number of elements reflecting each enabled algorithm key length DL_CONTROL_REQ: As described in PSARC/2001/070/, a DL_CONTROL_REQ command is used by upper layer software to configure the capabilities returned by the response to DL_CAPABILITY_REQ. Capability configuration as defined by this proposal is the process of downloading, updating and retrieving IPsec SADB information to the underlying Venus driver. The Venus driver will use the SADB information to cache and maintain keying material which is necessary to apply hardware acceleration to IPsec packets. In constructing a DL_CONTROL_REQ command, upper layer software will send down to the DLS provider an M_PROTO mblk containing a dl_control_t structure with dl_primitive set to DL_CONTROL_REQ. For this proposal, the dl_type field will be set to DL_CT_IPSEC_AH to control IPsec authentication data at the DLS provider or DL_CT_IPSEC_ESP to control IPsec encryption data at the DLS provider. Based on the dl_operation element of the dl_control_t structure, optional dl_key data and dl_data will follow. The dl_key for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is dl_key_spi, which is the IPsec SPI value of the SA. PSARC/2001/070/ specifies which data fields are mandatory per dl_operation control operation. The dl_data for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is the dl_ct_ipsec_t structure, which contains a minimal SADB entry to provide the required information to enable hardware accelerated encryption and authentication. The dl_data element following dl_control_t contains a dl_ct_ipsec_t which will be filled in with the the elements defined as per and RFC 2367. dl_control_t: dl_primitive - DL_CONTROL_REQ or DL_CONTROL_ACK dl_operation - control operation dl_type - DL_CT_IPSEC_AH or DL_CT_IPSEC_ESP dl_key_offset - sizeof(dl_control_t) dl_key_length - length of key data dl_data_offset - sizeof(dl_control_t) + dl_key_length dl_data_length - sizeof(dl_ct_ipsec_t) The dl_operation element of the dl_control_t has the following semantics (these semantics are aligned with base message types defined in RFC 2367): DL_CO_ADD - Requires dl_type, dl_key and dl_data. Add this entry to the DLS provider dataset, keyed at . If an entry already exists at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. If there is insufficient room to add the entry, response is a DL_ERROR_ACK with return value DL_TOOMANY. Otherwise a DL_CONTROL_ACK will be returned, containing the same data as the original DL_CONTROL_REQ. DL_CO_DELETE - Requires dl_type, dl_key. Delete the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_FLUSH - Requires dl_type. Delete all entries of type dl_type from the DLS provider dataset. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_GET - Requires dl_type, dl_key. Return the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned with a dl_ct_ipsec_t structure populated with the data associated with this . DL_CO_UPDATE - Requires dl_type, dl_key and dl_data. Update the entry in the DLS provider dataset, keyed at . If an entry doesn't already exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. REFERENCE DOCUMENTS PSARC/2001/070/ DL_CAPABILITY_REQ/CTL extensible interface for detecting and/or enabling lower level driver capabilities. pf_key(7P) man page /usr/include/net/pfkeyv2.h RFC 2367: PF_KEY Key Management API, Version 2 --Pladge_of_Wasps_202_000-- From sac-list-owner Wed Feb 21 16:30:24 2001 Date: Wed, 21 Feb 2001 16:32:09 -0800 From: David Robinson X-Accept-Language: en MIME-Version: 1.0 To: psarc@sac.eng.sun.com CC: Jack Bochsler Subject: Re: PSARC/2001/070/ DL_CAPABILITY_REQ/DL_CONTROL_REQ Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Length: 415 Status: O X-Status: $$$$ X-UID: 0000000012 I has reset the timer on this case to be 2/28/01 so everyone has time to review it. I am going to leave this as a large fast-track of which the first part is the infrastructure and the second is the first use of the infrastructure by IPSEC. Although they can be considered separately, given the approval of the first the second is trivially obvious. If anyone objects speak up and I will seperate them. -David From sac-list-owner Tue Feb 27 19:52:48 2001 Date: Tue, 27 Feb 2001 19:53:52 -0800 (PST) From: Glenn Skinner Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: 8hfclq3BiQq6A9ypXi0qxw== Content-Length: 2101 Status: O X-Status: $$$$ X-UID: 0000000013 In dl_capability_req_ft.txt, we have: ... 6.1 DL_CAPABILITY_REQ There are two forms of DL_CAPABILITY_REQ. If an empty message (dl_sub_length is 0) is issued to the DLS provider a DL_CAPABILITY_ACK will return all the capabilities provided by the device. The second form of the command will contain enumeration of the specific capabilities the DLS user wants the DLS provider to configure. If the lower level driver supports the DL_CAPABILITY_REQ primitive, if the dl_sub_length is 0, the DLS provider will reply with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t structures following in the response mblk which will enumerate the capabilities of the device. A request exchange to determine all capabilities will appear as follows: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - 0 dl_sub_length - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does not support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t with the following values: dl_error_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CAPABILITY_REQ dl_errno - DL_UNSUPPORTED dl_unix_errno - 0 If the driver has no DL_CAPABILITY_REQ capabilities, a dl_error_ack_t will be returned with the following values: dl_error_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CAPABILITY_REQ dl_errno - DL_NOTSUPPORTED dl_unix_errno - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does support DL_CAPABILITY_REQ and has capabilities to report, a successful response will consist of a dl_capability_t followed by one or more dl_capability_sub_t structures. It seems to me that the second case above (driver has no capabilities), is really just a subcase of the third, and should report success, but with zero dl_capability_sub_t structures. Also, the interface table lists the four new DLPI primitives as Consolidation Private. This classification doesn't seem feasible; isn't the intent to make these primitives available to drivers that don't deliver into the ON consolidation? -- Glenn From sac-list-owner Tue Feb 27 20:04:10 2001 Date: Tue, 27 Feb 2001 20:05:14 -0800 (PST) From: Glenn Skinner Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: EMBYGnp/pNv7WtSKwa22TQ== Content-Length: 283 Status: O X-Status: $$$$ X-UID: 0000000014 The DL_CAPAB_IPSEC_ESP, DL_CAPAB_IPSEC_AH, DL_CT_IPSEC_ESP, and DL_CT_IPSEC_AH primitives that this proposal introduces are all proposed to be Contract Private. However, the case directory contains no contracts. We'll need to see them before the case can be approved. -- Glenn From sac-list-owner Wed Feb 28 15:57:02 2001 Date: Wed, 28 Feb 2001 15:54:57 -0800 (PST) From: Jack Bochsler Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, glenn@ivrel.eng.sun.com Cc: Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: nDSMN6J+K7XksW1vSzFw/w== Content-Length: 2724 Status: O X-Status: $$$$ X-UID: 0000000015 > Date: Tue, 27 Feb 2001 19:53:52 -0800 (PST) > From: Glenn Skinner > Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ > To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com > > In dl_capability_req_ft.txt, we have: > > ... > 6.1 DL_CAPABILITY_REQ > > There are two forms of DL_CAPABILITY_REQ. If an empty message > (dl_sub_length is 0) is issued to the DLS provider a > DL_CAPABILITY_ACK will return all the capabilities provided by the > device. The second form of the command will contain enumeration > of the specific capabilities the DLS user wants the DLS provider > to configure. > > If the lower level driver supports the DL_CAPABILITY_REQ > primitive, if the dl_sub_length is 0, the DLS provider will reply > with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t > structures following in the response mblk which will enumerate the > capabilities of the device. > > A request exchange to determine all capabilities will appear as > follows: > > dl_capability_t: > dl_primitive - DL_CAPABILITY_REQ > dl_sub_offset - 0 > dl_sub_length - 0 > > In response to a DL_CAPABILITY_REQ, if the DLS provider does not > support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t > with the following values: > > dl_error_ack_t: > dl_primitive - DL_ERROR_ACK > dl_error_primitive - DL_CAPABILITY_REQ > dl_errno - DL_UNSUPPORTED > dl_unix_errno - 0 > > If the driver has no DL_CAPABILITY_REQ capabilities, a > dl_error_ack_t will be returned with the following values: > > dl_error_ack_t: > dl_primitive - DL_ERROR_ACK > dl_error_primitive - DL_CAPABILITY_REQ > dl_errno - DL_NOTSUPPORTED > dl_unix_errno - 0 > > In response to a DL_CAPABILITY_REQ, if the DLS provider does > support DL_CAPABILITY_REQ and has capabilities to report, a > successful response will consist of a dl_capability_t followed by > one or more dl_capability_sub_t structures. > > It seems to me that the second case above (driver has no capabilities), > is really just a subcase of the third, and should report success, but > with zero dl_capability_sub_t structures. > Agreed. This is a logical simplification and makes for easier implementation and documentation. > Also, the interface table lists the four new DLPI primitives as > Consolidation Private. This classification doesn't seem feasible; > isn't the intent to make these primitives available to drivers that > don't deliver into the ON consolidation? > As you noted in the subsequent email, this should be Contract Private, between producer and consumer. The contract is in-process and is out for review and signatures. I will post when signed off. Thanks. jack From sac-list-owner Thu Mar 8 11:57:49 2001 Date: Thu, 8 Mar 2001 11:55:41 -0800 (PST) From: Jack Bochsler Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com, glenn@ivrel.eng.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=String_of_Ponies_224_000 Content-Length: 6139 Status: O X-Status: $$$$ X-UID: 0000000016 --String_of_Ponies_224_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: Ly4nhhdwtJdea42YTxYZQQ== > Date: Tue, 27 Feb 2001 20:05:14 -0800 (PST) > From: Glenn Skinner > Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ > To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com > > The DL_CAPAB_IPSEC_ESP, DL_CAPAB_IPSEC_AH, DL_CT_IPSEC_ESP, and > DL_CT_IPSEC_AH primitives that this proposal introduces are all > proposed to be Contract Private. However, the case directory contains > no contracts. We'll need to see them before the case can be approved. > > -- Glenn > Please find the signed contract for IPsec implementation of 2001/070 attached. Contract is signed by CPG: Mehdi Bonyadi IPsec: Bruce Grove jack --String_of_Ponies_224_000 Content-Type: TEXT/plain; name="contract_2001_070.txt"; charset=us-ascii Content-Description: contract_2001_070.txt Content-MD5: KuADkIuI2gSKuTSvB05gwA== @(#)/shared/sac/doc/templates/contract [1.5 00/04/06] CONTRACT FOR CONTRACT PRIVATE INTERFACES 0. Number: PSARC/2001/070-001 1. This contract is between a SUPPLIER of INTERFACES and a CONSUMER of those INTERFACES, both of whom are entities within Sun Microsystems, Incorporated. 2. The SUPPLIER (definer and/or implementor) is identified by the following: Product or Bundle: Solaris Consolidation: OS/Networking Department or Group: N&S/CPG Bugtraq Category/SubCategory: venus Responsible Manager: Mehdi Bonyadi 3. The CONSUMER is identified by the following: Product or Bundle: Solaris Consolidation: OS/Networking Department or Group: Internet Engineering Bugtraq Category/SubCategory: network/ipsec Responsible Manager: Bruce Grove 4. The INTERFACES are: DL_CAPAB_IPSEC_ESP Encryption Sub-capability Primitive DL_CAPAB_IPSEC_AH Authentication Sub-capability Primitive DL_CT_IPSEC_ESP Encryption Sub-control Primitive DL_CT_IPSEC_AH Authentication Sub-control Primitive As described in the "DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ" document which supplemented PSARC/2001/070. 5. The ARC controlling these INTERFACES is: PSARC 6. The CASE describing these INTERFACES is: PSARC/2001/070 7. Changes to INTERFACES requires ARC approval. If SUPPLIER decides to change (including replace or remove) any portion of the INTERFACES, SUPPLIER will notify CONSUMER of the proposed new version, no later than the application for ARC approval of the new version. If SUPPLIER and CONSUMER are contained in the same bundle, they have the option of arranging for simultaneous conversion to the new interfaces. If this is not possible, or if they are not in the same bundle, then SUPPLIER will either make best effort to work with CONSUMER so that CONSUMER can detect which version of INTERFACES is being supplied, or else SUPPLIER will make best effort to supply both old and new versions of INTERFACES. If SUPPLIER cannot make both versions of INTERFACES available, and SUPPLIER and CONSUMER cannot devise a method whereby CONSUMER can detect which version of INTERFACES is being supplied, and the old version of CONSUMER will not run with the new version of SUPPLIER, then either the EOL process must be followed by SUPPLIER, or else a major release of SUPPLIER will be required. 8. If CONSUMER requires changes in INTERFACES, SUPPLIER will make best effort to accommodate such changes, which shall then be treated in accordance with paragraph 7 above. 9. Notwithstanding paragraphs 7 and 8, a change to any portion of the INTERFACES shall be regarded as a completely new set of INTERFACES, and requires execution of a new contract. 10. SUPPLIER and CONSUMER agree that evolution of INTERFACES shall be handled as follows: The SUPPLIER will provide the CONSUMER with specifications for the DL_CAPABILITY_REQ and DL_CONTROL_REQ API. The SUPPLIER will gain agreement from the CONSUMER that the interfaces being proposed are sufficient to support the CONSUMERS needs prior to submitting the interfaces for ARC approval. 11. SUPPLIER and CONSUMER agree that INTERFACES will be supported as follows: The SUPPLIER agrees to develop, test and support the PSARC/2001/070 API within the Venus device driver "ve" according to the specification provided with "DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ" which accompanied PSARC/2001/070. The CONSUMER agrees to develop, test and support the PSARC/2001/070 API within IPsec according to the specification provided with "DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ" which accompanied PSARC/2001/070. 12. SUPPLIER and CONSUMER agree that INTERFACES will be documented as follows: The PSARC/2001/070 API between IPsec and Venus are documented as a supplement document as part of the case materials. 13. SUPPLIER and CONSUMER agree that changes to the INTERFACES will be tested as follows: Changes to the interface will be tested by the SUPPLIER using the existing test suites in addition to the exposure provided by ON. 14. SUPPLIER and CONSUMER agree that this contract can be terminated as follows: This contract will terminate after mutual signed agreement between SUPPLIER and CONSUMER. 15. This contract is not valid until "signed" via agreement from the SUPPLIER and CONSUMER, and approved by the ARC CASE referenced by this contract. E-mail agreement to the contract should be archived in the mail archive of CASE; verbal agreement to the contract should be noted in the meeting minutes. This contract remains valid until superseded or invalidated. For SUPPLIER: Mehdi Bonyadi Date: 3/5/2001 For CONSUMER: Bruce Grove Date: 8th Mar 2001 For ARC: Date: A copy of this contract shall be deposited in the CASE directory as "contract-" or in a "contracts" subdirectory. An e-mail alias "contract-yyyy-nnn-ss@sun.com" shall be created via netadmin for notification of any desired changes. The SUPPLIER shall be the alias owner. 16. (Not to be filled in until superseded or invalidated.) This contract was superseded or invalidated by CASE: For ARC: Date: --String_of_Ponies_224_000-- From sac-list-owner Wed Mar 14 17:56:21 2001 Date: Wed, 14 Mar 2001 17:54:12 -0800 (PST) From: Jack Bochsler Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, glenn@ivrel.eng.sun.com Cc: Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Host_of_Sparrows_909_000 Content-Length: 21769 Status: O X-Status: $$$$ X-UID: 0000000017 --Host_of_Sparrows_909_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: d++FF4yBnCeBLnd7ZghHbA== > Date: Tue, 27 Feb 2001 19:53:52 -0800 (PST) > From: Glenn Skinner > Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ > To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com > > In dl_capability_req_ft.txt, we have: > > ... > 6.1 DL_CAPABILITY_REQ > > There are two forms of DL_CAPABILITY_REQ. If an empty message > (dl_sub_length is 0) is issued to the DLS provider a > DL_CAPABILITY_ACK will return all the capabilities provided by the > device. The second form of the command will contain enumeration > of the specific capabilities the DLS user wants the DLS provider > to configure. > > If the lower level driver supports the DL_CAPABILITY_REQ > primitive, if the dl_sub_length is 0, the DLS provider will reply > with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t > structures following in the response mblk which will enumerate the > capabilities of the device. > > A request exchange to determine all capabilities will appear as > follows: > > dl_capability_t: > dl_primitive - DL_CAPABILITY_REQ > dl_sub_offset - 0 > dl_sub_length - 0 > > In response to a DL_CAPABILITY_REQ, if the DLS provider does not > support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t > with the following values: > > dl_error_ack_t: > dl_primitive - DL_ERROR_ACK > dl_error_primitive - DL_CAPABILITY_REQ > dl_errno - DL_UNSUPPORTED > dl_unix_errno - 0 > > If the driver has no DL_CAPABILITY_REQ capabilities, a > dl_error_ack_t will be returned with the following values: > > dl_error_ack_t: > dl_primitive - DL_ERROR_ACK > dl_error_primitive - DL_CAPABILITY_REQ > dl_errno - DL_NOTSUPPORTED > dl_unix_errno - 0 > > In response to a DL_CAPABILITY_REQ, if the DLS provider does > support DL_CAPABILITY_REQ and has capabilities to report, a > successful response will consist of a dl_capability_t followed by > one or more dl_capability_sub_t structures. > > It seems to me that the second case above (driver has no capabilities), > is really just a subcase of the third, and should report success, but > with zero dl_capability_sub_t structures. > > Also, the interface table lists the four new DLPI primitives as > Consolidation Private. This classification doesn't seem feasible; > isn't the intent to make these primitives available to drivers that > don't deliver into the ON consolidation? > > -- Glenn > I had replied earlier but failed to submit a revised document. Following is a revised proposal with the following changes: - Re-classification from Consolidation Private to Contract Private - Re-write of "error case" above to returning zero capability structures as suggested. I owe a contract between my org and the consolidation for this interface. jack --Host_of_Sparrows_909_000 Content-Type: TEXT/plain; name="dl_capability_req_ft.txt"; charset=us-ascii; x-unix-mode=0444 Content-Description: dl_capability_req_ft.txt Content-MD5: VohNiuYRsraNApd58lqA8A== --- DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interface for detecting, enabling and controlling DLS provider capabilities. jack.bochsler@west.sun.com 14 March 2001 COMMITMENT LEVEL DL_CAPABILITY_REQ extensible interface for detecting and enabling DLS provider capabilities. Contract-Private DL_CONTROL_REQ extensible interface for controlling DLS provider capabilities. Contract-Private RELEASE Minor. 1. Introduction 1.1. Project/Component Working Name: DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interfaces 1.2. Name of Document Author/Supplier: Jack Bochsler, jack.bochsler@sun.com 1.3. Date of This Document: 03/14/01 1.4. Name of Major Document Customer(s)/Consumer(s): IE, PSARC 2. Description There is an expanding array of hardware optimization capabilities being implemented in Network Interface Cards (NICs) that need to be discovered by IP so that IP knows when and how to exploit them. There is an existing interface for IP to discover hardware checksumming capability but this interface is not extensible to cover the various requirements of other capabilities such as encryption and authentication. This project is a proposal to define an extensible request/response framework that supports arbitrary capabilities and lengths of data to allow upper layer software to determine device specific characteristics provided by lower level software, hardware adapters and drivers. This project proposes to introduce a generic Contract-Private DLS provider interface which is a simple but extensible framework for communications between upper layer software to assess and control capabilities provided by lower layers of software and hardware. Separate Contract-Private Interfaces between the producers (e.g. NIC device drivers) and consumers (IP) of the data would be necessary prior to the actual use of this interface. Note: PSARC/1996/173/ defined a non-extensible Contract Private interface between IP and the SMCC SAHI3 155/622 ATM network interface device driver (ba) for the negotiation of inetcksuming. The hardware checksum and M_DATA fastpath negotiations must evolve to use this new framework, but there are compatibility constraints which makes this a separate task. 3. External Requester N/A 4. Opportunity Window/Exposure Enable a uniform interface for communicating device specific info between IP and lower level software. 5. Business Opportunity/Exposure There are an increasing number of NICs that have hardware acceleration for multiple reasons. A mechanism needs to be in place for IP to detect and exploit these capabilities. The Venus project is slated for the S9U1 time period, so these changes need to be integrated in S9. 6. Technical Description This proposal creates the following interfaces: |-----------------------|-----------------------|---------------| |Interface | Classification | Comments | |-----------------------|-----------------------|---------------| |DL_CAPABILITY_REQ | Contract Private | | |DL_CAPABILITY_ACK | Contract Private | | |DL_CONTROL_REQ | Contract Private | | |DL_CONTROL_ACK | Contract Private | | |-----------------------|-----------------------|---------------| The following changes are recommended to the existing header file in the "Sun additions" area: #define DL_CAPABILITY_REQ (0x100) /* Sun device specific capability request */ #define DL_CAPABILITY_ACK (0x101) /* Sun capability ack */ #define DL_CONTROL_REQ (0x102) /* Sun device specific control request */ #define DL_CONTROL_ACK (0x103) /* Sun control ack */ The following #defines with proposed initial primitives and the generic response data structures format would be added to the bottom of the file: /* * DL_CAPABILITY_REQ, DL_CAPABILITY_ACK, M_PROTO type * * This is used both to determine the set of capabilities in the * DLS provider and also to turn on and off specific capabilities. * The response is a DL_CAPABILITY_ACK or DL_ERROR_ACK. * * DL_CAPABILITY_REQ can either be empty (i.e. dl_sub_length is zero) * which is a request for the driver to return all capabilities. * Otherwise, the DL_CAPABILITY_REQ contains the capabilities the * DLS user wants to use and their settings. * * DL_CAPABILITY_ACK contains the capabilities that the DLS provider can * support modified based on what was listed in the request. If a capability * was included in the request then the information returned in the ack might * be modified based on the information in the request. */ typedef struct { t_uscalar_t dl_primitive; /* DL_CAPABILITY_REQ/ACK */ t_uscalar_t dl_sub_offset; /* options length */ t_uscalar_t dl_sub_length; /* options offset */ /* Followed by a list of zero or more dl_capability_sub_t */ } dl_capability_t; typedef struct { t_uscalar_t dl_cap; /* capability type */ t_uscalar_t dl_length; /* length of data following */ /* Followed by zero or more bytes of dl_data */ } dl_capability_sub_t; /* * Capability types */ #define DL_CAPAB_FASTPATH 0x00 /* DLPI M_DATA fastpath capable */ /* dl_data is uchar_t[1] for\ enable/disable */ #define DL_CAPAB_INETCKSUM 0x01 /* inetcksum capable */ /* dl_data is inetcksum_t */ #define DL_CAPAB_IPSEC_AH 0x02 /* IPsec AH acceleration */ /* dl_data is dl_capab_ipsec_t */ #define DL_CAPAB_IPSEC_ESP 0x03 /* IPsec ESP acceleration */ /* dl_data is dl_capab_ipsec_t */ /* * DL_CONTROL_REQ, DL_CONTROL_ACK, M_PROTO type * * Extensible message to send down control information to the DLS provider. * The response is a DL_CONTROL_ACK or DL_ERROR_ACK. * * Different types of control operations will define different format * for the key and data fields. * ADD requires key and data fields; if the matches * an already existing entry a DL_ERROR_ACK will be returned. * DELETE requires a key field; if the does not exist, * a DL_ERROR_ACK will be returned. * FLUSH requires neither a key nor data; it unconditionally removes * all entries for the specified type. * GET requires a key field; the get operation returns the data for the * . If doesn't exist a DL_ERROR_ACK is returned. * UPDATE requires key and data fields; if doesn't * exist a DL_ERROR_ACK is returned. */ typedef struct { t_uscalar_t dl_primitive; /* DL_CONTROL_REQ/DL_CONTROL_ACK */ t_uscalar_t dl_operation; /* add/delete/purge */ t_uscalar_t dl_type; /* e.g. AH/ESP/ ... */ t_uscalar_t dl_key_offset; /* offset of key */ t_uscalar_t dl_key_length; /* length of key */ t_uscalar_t dl_data_offset; /* offset of data */ t_uscalar_t dl_data_length; /* length of data */ } dl_control_t; /* * Control operations */ #define DL_CO_ADD 0x01 /* Add new entry matching for */ #define DL_CO_DELETE 0x02 /* Delete the entry matching */ #define DL_CO_FLUSH 0x03 /* Purge all entries of */ #define DL_CO_GET 0x04 /* Get the data for the */ #define DL_CO_UPDATE 0x05 /* Update the data for */ /* * Control types */ #define DL_CT_IPSEC_AH 0x01 /* AH; key = spi; data = keying material */ #define DL_CT_IPSEC_ESP 0x02 /* ESP; key = spi; data = keying material */ #define DL_CT_FASTPATH_PROBE 0x03 /* DL_IOC_HDR_INFO replacement; key = dl_unitdata_req_t; /* data (in ack) = fastpath header */ #define DL_CT_INETCKSUM 0x04 /* ick M_CTL replacement; key = */ #define DL_CT_CLASSIFY 0x05 /* packet classifier; key = packet matching rule */ /* structure; data = actions on packet */ It is expected that additional capability types may be added to over time, e.g. for zero copy support. In order for these primitives to be implemented, separate ARC cases would be necessary to establish the specific semantics, data content, format and error handling. There are two components to this proposal, DL_CAPABILITY_REQ, and DL_CONTROL_REQ. If a command is not implemented, it will be readily discernible by the error acknowledgement as defined below. There is no expected or implied ordering in the use of these messages. Applicability of primitives to specific capabilities is subject to the definition from the Contract-Private agreement and definition of the use of this interface. The first primitive, DL_CAPABILITY_REQ, allows a DLS user to determine and control device specific capabilities provided by the DLS provider. The second primitive, DL_CONTROL_REQ, is used by the upper layer software to pass configuration data down to the DLS provider, or to retrieve configuration data from the DLS provider. 6.1 DL_CAPABILITY_REQ There are two forms of DL_CAPABILITY_REQ. If an empty message (dl_sub_length is 0) is issued to the DLS provider a DL_CAPABILITY_ACK will return all the capabilities provided by the device. The second form of the command will contain enumeration of the specific capabilities the DLS user wants the DLS provider to configure. If the lower level driver supports the DL_CAPABILITY_REQ primitive, if the dl_sub_length is 0, the DLS provider will reply with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t structures following in the response mblk which will enumerate the capabilities of the device. A request exchange to determine all capabilities will appear as follows: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - 0 dl_sub_length - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does not support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t with the following values: dl_error_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CAPABILITY_REQ dl_errno - DL_UNSUPPORTED dl_unix_errno - 0 If the driver has no DL_CAPABILITY_REQ capabilities, a dl_capability_t will be returned with a dl_sub_length of zero and no dl_capability_sub_t structures following to represent that there are no capabilities to report: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - 0 In response to a DL_CAPABILITY_REQ, if the DLS provider does support DL_CAPABILITY_REQ and has capabilities to report, a successful response will consist of a dl_capability_t followed by one or more dl_capability_sub_t structures. Each dl_capability_sub_t will provide a capability type, and 0 or more bytes of capability data to enumerate the Contract Private details of the device capability available. Each dl_capability_sub_t can then encapsulate as much data as necessary to describe the feature being enumerated in the dl_data portion of the dl_capability_sub_t. The Contract Private format of the data should include specific information or status on the capability being reported so that the DLS user can determine how the capability is configured - for example whether it is enabled or disabled. A successful response will result in a reply with the structure elements set as following: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - total length of dl_capability_sub_ts structures following dl_capability_sub_t: dl_cap - first capability type dl_length - length of data following dl_data - Contract Private data The second form of the DL_CAPABILITY_REQ command is used by the DLS user to set capability settings on the DLS provider. The upper level software will issue a DL_CAPABILITY_REQ with a single dl_capability_sub_t containing the settings for the DLS provider to use. Note that unlike the DL_CAPABILITY_ACK response above, this command may contain only one dl_capability_sub_t. The dl_capability_sub_t will contain the Contract Private details of what capabilities the DLS provider should enable or disable. This command will be of the form: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - capability type dl_length - length of data following dl_data - Contract Private data In response to a DL_CAPABILITY_REQ of this form, the DLS provider will reply with dl_error_ack_t structures as above if the primitive is not recognized or supported. If the DL_CAPABILITY_REQ is supported, the reply will be of the form: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - capability type dl_length - length of data following dl_data - Contract Private data The information returned in the dl_data portion of the acknowledgement might be modified based on the information in the request. If an error occurs in enabling a configuring a capability, a dl_error_ack_t will be returned of the following form: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_BADPRIM dl_unix_errno - 0 6.2 DL_CONTROL_REQ The second primitive, DL_CONTROL_REQ, allows upper layer software to pass control data down to the DLS provider. This is implemented as a different interface to separate control and data information. The format of the DL_CONTROL_REQ command is a M_PROTO dl_control_t structure. This structure is followed by optional dl_key data and optional dl_data. Note that the actual content, format and resultant processing of the M_PROTO dl_key and dl_data fields are Contract-Private. This interface provides a mechanism for a DLS user to create and maintain one or more keyed data sets with the DLS provider. The commands consist of operation, meta-data as well as the data: - command of specifically what operations to perform with the data set (dl_operation) - type information so that the DLS provider can maintain more than one data set (dl_type) - key data so that the data set can be addressed with keyed record granularity (dl_key) - data, in a Contract Private format (dl_data) The contents of the dl_operation element will define the requirements for the dl_key and dl_data fields, the DLS provider behavior associated with the request, and the contents of the DL_CONTROL_ACK response. If the dl_operation is a DL_CO_ADD, the M_PROTO requires the dl_key and dl_data fields. If the matches an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will add the data, keyed on the . Successful operation will return a DL_CONTROL_ACK with the original dl_key and dl_data. If the dl_operation is a DL_CO_DELETE, the M_PROTO requires a dl_key field. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will delete the entry based on the . Successful operation will return a DL_CONTROL_ACK with the original dl_key. If the dl_operation is a DL_CO_FLUSH, the M_PROTO requires no further data. On receipt of this control message, the DLS provider will flush all entries for the specified dl_type. Successful operation will return a DL_CONTROL_ACK without dl_key or dl_data. If the dl_operation is a DL_CO_GET, the M_PROTO requires a dl_key field. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will return the entry based on the . Successful operation will return a DL_CONTROL_ACK with original dl_key and the corresponding dl_data. If the dl_operation is a DL_CO_UPDATE, the M_PROTO must also contain dl_key and dl_data fields. If the doesn't match an existing entry, a DL_ERROR_ACK will be returned. Otherwise the DLS provider will replace the existing data associated with the with the the data from the dl_data field. Successful operation return a DL_CONTROL_ACK with the original dl_key and dl_data. The format of a DL_CONTROL_REQ command is a dl_control_t with the following values: dl_control_t: dl_primitive - DL_CONTROL_REQ dl_operation - control operation desired dl_type - DLS provider capability to be controlled by this request dl_key_offset - sizeof (dl_control_t) dl_key_length - length of key following dl_data_offset - sizeof (dl_control_t) + dl_key_length dl_data_length - length of data following The dl_control_t is followed by optional dl_key and dl_data elements. The expected responses to a DL_CONTROL_REQ follow. In response to a DL_CONTROL_REQ, if the driver does not understand/support DL_CONTROL_REQ the lower level software will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_UNSUPPORTED dl_unix_errno - 0 If the driver has no capabilities, the lower level software will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_NOTSUPPORTED dl_unix_errno - 0 If the DLS provider does not recognize the capability type specified in the dl_type data field, it will reply with a dl_err_ack_t with the following values: dl_err_ack_t: dl_primitive - DL_ERROR_ACK dl_error_primitive - DL_CONTROL_REQ dl_errno - DL_BADPRIM dl_unix_errno - 0 If the driver does support DL_CONTROL_REQ, the DLS provider will perform the requested operation and if an error occurs as described per operation above, will return a DL_ERROR_ACK. If successful, the DLS provider reply with a dl_control_t with the following values: dl_control_t: dl_primitive - DL_CONTROL_ACK dl_operation - control operation requested dl_type - DLS provider capability to be controlled by this request dl_key_offset - sizeof (dl_control_t) dl_key_length - length of key following dl_data_offset - sizeof (dl_control_t) + dl_key_length dl_data_length - length of data following 7. Reference Documents PSARC/1996/173/ "TCP/IP support for SAHI3 hardware checksuming", 03/27/96, Bruce Curtis, brutus@Eng 8. Projected Availability Solaris 9 9. Cost of Effort 1.0 man month. 10. Prototype Availability 03/30/01 11. Prototype Cost 0.5 man month. 12. Cost of Capital Resources N/A --Host_of_Sparrows_909_000-- From sac-list-owner Tue Mar 20 15:59:57 2001 Date: Tue, 20 Mar 2001 16:01:02 -0800 (PST) From: Jack Bochsler Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: psarc@sac.eng.sun.com, glenn@ivrel.eng.sun.com Cc: Jack.Bochsler@west.sun.com MIME-Version: 1.0 Content-Type: MULTIPART/mixed; BOUNDARY=Cast_of_Hawks_380_000 Content-Length: 27336 --Cast_of_Hawks_380_000 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: kseoh3PAPJ3E50navAnhpA== Signed contract for the changed (Consolidation Private to Contract Private) DL_CAPABILITY_REQ and DL_CONTROL_REQ interfaces below. Also corrected one-pager for 2001/070 DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ Changes: 1) SA entry uniqueness is guaranteed across the tuple . The proposed implementation did not include the final parameter. The dl_key was renamed to dl_ct_ipsec_key_t and expanded to include the destination IP address as well as the SPI. 2) Rename of DL_MAX_KEY_LEN to DL_CT_IPSEC_MAX_KEY_LEN for namespace clarity. 3) Rename of structure definition capability_cfg_sadb to dl_ct_ipsec for namespace clarity (and to match rest of document). Changebars in left column to reflect these changes. Thanks. jack > Date: Wed, 14 Mar 2001 17:54:12 -0800 (PST) > From: Jack Bochsler > Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ > To: psarc@sac.eng.sun.com, glenn@ivrel.eng.sun.com > Cc: Jack.Bochsler@west.sun.com > > > > Date: Tue, 27 Feb 2001 19:53:52 -0800 (PST) > > From: Glenn Skinner > > Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ > > To: psarc@sac.eng.sun.com, Jack.Bochsler@west.sun.com > > > > In dl_capability_req_ft.txt, we have: > > > > ... > > 6.1 DL_CAPABILITY_REQ > > > > There are two forms of DL_CAPABILITY_REQ. If an empty message > > (dl_sub_length is 0) is issued to the DLS provider a > > DL_CAPABILITY_ACK will return all the capabilities provided by the > > device. The second form of the command will contain enumeration > > of the specific capabilities the DLS user wants the DLS provider > > to configure. > > > > If the lower level driver supports the DL_CAPABILITY_REQ > > primitive, if the dl_sub_length is 0, the DLS provider will reply > > with a DL_CAPABILITY_ACK with one or more dl_capability_sub_t > > structures following in the response mblk which will enumerate the > > capabilities of the device. > > > > A request exchange to determine all capabilities will appear as > > follows: > > > > dl_capability_t: > > dl_primitive - DL_CAPABILITY_REQ > > dl_sub_offset - 0 > > dl_sub_length - 0 > > > > In response to a DL_CAPABILITY_REQ, if the DLS provider does not > > support DL_CAPABILITY_REQ the it will reply with a dl_error_ack_t > > with the following values: > > > > dl_error_ack_t: > > dl_primitive - DL_ERROR_ACK > > dl_error_primitive - DL_CAPABILITY_REQ > > dl_errno - DL_UNSUPPORTED > > dl_unix_errno - 0 > > > > If the driver has no DL_CAPABILITY_REQ capabilities, a > > dl_error_ack_t will be returned with the following values: > > > > dl_error_ack_t: > > dl_primitive - DL_ERROR_ACK > > dl_error_primitive - DL_CAPABILITY_REQ > > dl_errno - DL_NOTSUPPORTED > > dl_unix_errno - 0 > > > > In response to a DL_CAPABILITY_REQ, if the DLS provider does > > support DL_CAPABILITY_REQ and has capabilities to report, a > > successful response will consist of a dl_capability_t followed by > > one or more dl_capability_sub_t structures. > > > > It seems to me that the second case above (driver has no capabilities), > > is really just a subcase of the third, and should report success, but > > with zero dl_capability_sub_t structures. > > > > Also, the interface table lists the four new DLPI primitives as > > Consolidation Private. This classification doesn't seem feasible; > > isn't the intent to make these primitives available to drivers that > > don't deliver into the ON consolidation? > > > > -- Glenn > > > I had replied earlier but failed to submit a revised document. > Following is a revised proposal with the following changes: > > - Re-classification from Consolidation Private to Contract Private > - Re-write of "error case" above to returning zero capability > structures as suggested. > > I owe a contract between my org and the consolidation for this > interface. > > jack --Cast_of_Hawks_380_000 Content-Type: TEXT/plain; name="contract_2001_070b.txt"; charset=us-ascii; x-unix-mode=0600 Content-Description: contract_2001_070b.txt Content-MD5: nNuNtwYxbHJtVDwrQGosLw== @(#)/shared/sac/doc/templates/contract [1.5 00/04/06] CONTRACT FOR CONTRACT PRIVATE INTERFACES 0. Number: PSARC/2001/070-001 1. This contract is between a SUPPLIER of INTERFACES and a CONSUMER of those INTERFACES, both of whom are entities within Sun Microsystems, Incorporated. 2. The SUPPLIER (definer and/or implementor) is identified by the following: Product or Bundle: Solaris Consolidation: OS/Networking Department or Group: Internet Engineering Bugtraq Category/SubCategory: network/ipsec Responsible Manager: Bruce Grove 3. The CONSUMER is identified by the following: Product or Bundle: Solaris Consolidation: OS/Networking Department or Group: N&S/CPG Bugtraq Category/SubCategory: venus Responsible Manager: Mehdi Bonyadi 4. The INTERFACES are: DL_CAPABILITY_REQ interface for detecting and enabling DLS provider capability. DL_CONTROL_REQ interface for controllling DLS providers As described in the "DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interface for detecting and controlling DLS provider capabilities." as described in PSARC/2001/070. 5. The ARC controlling these INTERFACES is: PSARC 6. The CASE describing these INTERFACES is: PSARC/2001/070 7. Changes to INTERFACES requires ARC approval. If SUPPLIER decides to change (including replace or remove) any portion of the INTERFACES, SUPPLIER will notify CONSUMER of the proposed new version, no later than the application for ARC approval of the new version. If SUPPLIER and CONSUMER are contained in the same bundle, they have the option of arranging for simultaneous conversion to the new interfaces. If this is not possible, or if they are not in the same bundle, then SUPPLIER will either make best effort to work with CONSUMER so that CONSUMER can detect which version of INTERFACES is being supplied, or else SUPPLIER will make best effort to supply both old and new versions of INTERFACES. If SUPPLIER cannot make both versions of INTERFACES available, and SUPPLIER and CONSUMER cannot devise a method whereby CONSUMER can detect which version of INTERFACES is being supplied, and the old version of CONSUMER will not run with the new version of SUPPLIER, then either the EOL process must be followed by SUPPLIER, or else a major release of SUPPLIER will be required. 8. If CONSUMER requires changes in INTERFACES, SUPPLIER will make best effort to accommodate such changes, which shall then be treated in accordance with paragraph 7 above. 9. Notwithstanding paragraphs 7 and 8, a change to any portion of the INTERFACES shall be regarded as a completely new set of INTERFACES, and requires execution of a new contract. 10. SUPPLIER and CONSUMER agree that evolution of INTERFACES shall be handled as follows: The SUPPLIER will provide the CONSUMER with specifications for the DL_CAPABILITY_REQ and DL_CONTROL_REQ API. The SUPPLIER will gain agreement from the CONSUMER that the interfaces being proposed are sufficient to support the CONSUMERS needs prior to submitting the interfaces for ARC approval. 11. SUPPLIER and CONSUMER agree that INTERFACES will be supported as follows: The SUPPLIER agrees to develop, test and support the PSARC/2001/070 API within IPsec according to the specification "DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interface for detecting and controlling DLS provider capabilities", PSARC/2001/070. The CONSUMER agrees to develop, test and support the PSARC/2001/070 API within Venus according to the specification "DL_CAPABILITY_REQ/DL_CONTROL_REQ extensible interface for detecting and controlling DLS provider capabilities", PSARC/2001/070. 12. SUPPLIER and CONSUMER agree that INTERFACES will be documented as follows: The PSARC/2001/070 API between IPsec and Venus are documented as a supplement document as part of the case materials. 13. SUPPLIER and CONSUMER agree that changes to the INTERFACES will be tested as follows: Changes to the interface will be tested by the SUPPLIER using the existing test suites in addition to the exposure provided by ON. 14. SUPPLIER and CONSUMER agree that this contract can be terminated as follows: This contract will terminate after mutual signed agreement between SUPPLIER and CONSUMER. 15. This contract is not valid until "signed" via agreement from the SUPPLIER and CONSUMER, and approved by the ARC CASE referenced by this contract. E-mail agreement to the contract should be archived in the mail archive of CASE; verbal agreement to the contract should be noted in the meeting minutes. This contract remains valid until superseded or invalidated. For SUPPLIER: Bruce Grove Date: 3/20/2001 For CONSUMER: Mehdi Bonyadi Date: 3/15/2001 For ARC: Date: A copy of this contract shall be deposited in the CASE directory as "contract-" or in a "contracts" subdirectory. An e-mail alias "contract-yyyy-nnn-ss@sun.com" shall be created via netadmin for notification of any desired changes. The SUPPLIER shall be the alias owner. 16. (Not to be filled in until superseded or invalidated.) This contract was superseded or invalidated by CASE: For ARC: Date: --Cast_of_Hawks_380_000 Content-Type: TEXT/plain; name="dl_capability_ipsec_ft.txt"; charset=us-ascii; x-unix-mode=0444 Content-Description: dl_capability_ipsec_ft.txt Content-MD5: gME4CPaSOU1li2UCSwlqhQ== --- DL_CAPAB_IPSEC_ESP/DL_CAPAB_IPSEC_AH implementation of DL_CONTROL_REQ jack.bochsler@west.sun.com 21 February 2001 COMMITMENT LEVEL DL_CAPAB_IPSEC_ESP Encryption Sub-capability Primitive Contract-Private DL_CAPAB_IPSEC_AH Authentication Sub-capability Primitive Contract-Private DL_CT_IPSEC_ESP Encryption Sub-control Primitive Contract-Private DL_CT_IPSEC_AH Authentication Sub-control Primitive Contract-Private RELEASE Minor. BACKGROUND The Venus project is developing a NIC which provides hardware acceleration of IPsec algorithms. In order for IPsec to exploit this capability, it needs to identify, control and configure this functionality. PSARC/2001/070/ defined a generic request/control mechanism to identify and control DLS provider capabilities. This proposal describes a Contract-Private implementation for IPsec to identify and control DLS provider authentication and encryption capabilities via the messages and payload areas of PSARC/2001/070/. This proposal is being submitted simultaneously with PSARC/2001/070/ and is dependent on it. PROBLEM In order for IPsec to take advantage of hardware acceleration in the Venus card, a method must be employed to identify specifically which algorithms may be accelerated in hardware. PSARC/2001/070/ defines a method for polling DLS providers to determine their specific capabilities. IPsec will also need a mechanism to control (enable/disable) functionality, and a mechanism to push SA information down to the Venus driver/card. This proposal defines interfaces and implementation to be layered on PSARC/2001/070/ to provide IPsec acceleration capability identification and control. This describes primitives and data structures used to pass information between IP and the Venus driver. PROPOSAL This proposal creates the following interfaces, layered on the Consolidation Private interface as defined in PSARC/2001/070: |-----------------------|-----------------------|---------------| |Interface | Classification | Comments | |-----------------------|-----------------------|---------------| |DL_CAPAB_IPSEC_AH | Contract Private | | |DL_CAPAB_IPSEC_ESP | Contract Private | | |DL_CT_IPSEC_AH | Contract Private | | |DL_CT_IPSEC_ESP | Contract Private | | |-----------------------|-----------------------|---------------| The following changes are recommended to the existing DLPI interfaces and are to be incorporated into : /* * Note that numbers correspond to the types in the IPsec DOI document. */ #define DL_CAPAB_IPSEC_AH 0x02 /* IPsec AH acceleration */ #define DL_CAPAB_IPSEC_ESP 0x03 /* IPsec ESP acceleration */ typedef struct { t_uscalar_t cip_version; /* interface version */ t_uscalar_t cip_nciphers; /* number ciphers supported */ dl_capab_ipsec_alg_t cip_data[1]; /* data */ } dl_capab_ipsec_t; typedef struct { t_uscalar_t alg_prim; /* algorithm primitive */ t_uscalar_t alg_thruput; /* approx throughput metric in Mb/s */ t_uscalar_t alg_flag; /* flags */ t_uscalar_t alg_nkeylen; /* num supported key lengths */ t_uscalar_t alg_keylen[1]; /* supported key length list */ } dl_capab_ipsec_alg_t; /* alg_prim ciphers */ #define DL_CAPAB_IPSEC_ESP_NONE 0x00 /* no cipher */ #define DL_CAPAB_IPSEC_ESP_DES 0x02 #define DL_CAPAB_IPSEC_ESP_3DES 0x03 #define DL_CAPAB_IPSEC_ESP_BLOWFISH 0x07 /* alg_prim authentications */ #define DL_CAPAB_IPSEC_AH_NONE 0x00 /* no authentication */ #define DL_CAPAB_IPSEC_AH_MD5HMAC 0x02 #define DL_CAPAB_IPSEC_AH_SHA1HMAC 0x03 /* alg_flag values */ #define DL_CAPAB_ALG_ENABLE 0x01 /* enable this algorithm */ /* * Control types */ | #define DL_CT_IPSEC_AH 0x01 /* AH; key=spi,dest_addr; | data=keying material */ | #define DL_CT_IPSEC_ESP 0x02 /* ESP; key=spi,des_taddr; | data=keying material */ /* * For DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP, the optional dl_key data * that follows the dl_control_t will be the IPsec SPI (Security | * Parameters Index) value and the destination address. This is defined | * as being unique per protocol. */ | #define DL_CTL_IPSEC_ADDR_LEN 16 /* IP addr length in bytes */ | typedef struct dl_ct_ipsec_key { | uint32_t dl_key_spi; /* Security Parameters Index value */ | u_char dl_key_dest_addr[DL_CTL_IPSEC_ADDR_LEN]; /* dest IP address */ | } dl_ct_ipsec_key_t; | #define DL_CT_IPSEC_MAX_KEY_LEN 512 /* max key length in bytes */ /* * minimal SADB entry content * fields are defined as per RFC 2367 and * This defines the content and format of the dl_data portion of * the dl_control_t */ | typedef struct dl_ct_ipsec { uint8_t sadb_sa_auth; /* Authentication algorithm */ uint8_t sadb_sa_encrypt; /* Encryption algorithm */ uint32_t sadb_sa_flags; /* SA flags. */ uint16_t sadb_address_exttype_s; /* SA address - SRC */ struct sockaddr_storage sockaddr_s; uint16_t sadb_address_exttype_d; /* SA address - DST */ struct sockaddr_storage sockaddr_d; uint16_t sadb_address_exttype_p; /* SA address - PROXY */ struct sockaddr_storage sockaddr_p; uint16_t sadb_key_len_a; /* AUTH key length */ | uint16_t sadb_key_exttype_a; /* SADB_EXT_KEY_AUTH */ uint16_t sadb_key_bits_a; /* Actual key length in bits */ | uint16_t sadb_key_data_a[DL_CT_IPSEC_MAX_KEY_LEN]; /* key data */ uint16_t sadb_key_len_e; /* ENCRYPT key length */ | uint16_t sadb_key_exttype_e; /* SADB_EXT_KEY_ENCRYPT */ uint16_t sadb_key_bits_e; /* Actual key length in bits */ | uint16_t sadb_key_data_e[DL_CT_IPSEC_MAX_KEY_LEN]; /* key data */ | } dl_ct_ipsec_t; The above structures define the payload (dl_data) area of the input and response to DL_CAPABILITY_REQ and DL_CONTROL_REQ commands as defined in PSARC/2001/070. The capability mechanism is initiated by the DLS user (IPsec) issuing a M_PROTO mblk containing a dl_capability_t with dl_primitive set to DL_CAPABILITY_REQ to the DLS provider. Zero or more dl_capab_ipsec_t structures will follow in the M_PROTO. The control mechanism is initiated by the DLS user as well, issuing a M_PROTO message containing a dl_control_t with the dl_primitive set to DL_CONTROL_REQ, a dl_operation specified and a dl_type to specify the DLS provider dataset on which the dl_operation is to take place. The | dl_control_t is followed by optional dl_key and dl_data, based on the | dl_operation primitive. The dl_key data encapsulates a | dl_ct_ipsec_key structure. If present, the dl_data encapsulates a | dl_ct_ipsec_t structure. Based on the contents of the M_PROTO message, the DLS provider (Venus driver) will perform the actions and respond with a DL_CAPABILITY_ACK or DL_CONTROL_ACK as described in more detail below. DL_CAPABILITY_REQ: On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is 0, this initiates a query for capabilities. The DLS provider will reply with a DL_CAPABILITY_ACK (as per PSARC/2001/070/) with dl_capability_sub_t structures following, one per capability supported by the DLS provider. For DLS providers that provide IPsec acceleration capabilities there would be one dl_capability_sub_t for ESP (encryption) and a separate one for AH (authentication), with potentially additional dl_capability_sub_t structs for other capabilities. The contents of the dl_capability_t and the dl_capability_sub_t are defined as: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have zero or more dl_capab_ipsec_t structures following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms supported by the device. The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - 0 alg_keylen - variable number of elements reflecting each supported key length for this algorithm On receipt of a DL_CAPABILITY_REQ, if the dl_sub_length is not 0, this signifies that the DLS user is passing additional data to the DLS provider to configure a capability or its' subfeatures on or off. Note that all capabilities supported by the DLS provider are disabled until they are explicitly enabled. The contents of the dl_capability_t and the dl_capability_sub_t are: dl_capability_t: dl_primitive - DL_CAPABILITY_REQ dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of dl_capability_sub_t dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers being enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms to be enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure | alg_thruput - (ignored) alg_nkeylen - number of supported key lengths alg_flag - DL_CAPAB_ALG_ENABLE to enable this algorithm or 0 to disable alg_keylen - variable number of elements reflecting each supported key length to be enabled In reply to the DL_CAPABILITY_REQ with attached data, the DLS provider will return a DL_CAPABILITY_ACK of the following form: dl_capability_t: dl_primitive - DL_CAPABILITY_ACK dl_sub_offset - sizeof (dl_capability_t) dl_sub_length - length of data following dl_capability_sub_t: dl_cap - DL_CAPAB_IPSEC_AH or DL_CAPAB_IPSEC_ESP dl_length - length of data following The dl_capability_sub_t will have one dl_capab_ipsec_t structure following it. The dl_capab_ipsec_t elements are defined as: dl_capab_ipsec_t: cip_version - set to 1 to reflect the current version of this interface cip_nciphers - number of ESP or AH ciphers enabled. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_nciphers - number of ESP or AH ciphers supported. This number corresponds to the number of dl_capab_ipsec_alg_t structures following the dl_capab_ipsec_t cip_data - contains cip_nciphers of dl_capab_ipsec_alg_t structures which define the actual hardware algorithms enabled The elements of the dl_capab_ipsec_alg_t encapsulated in the cip_data have the following meaning: dl_capab_ipsec_alg_t: alg_prim - the specific ESP or AH algorithm supported and described by this structure alg_thruput - approx. throughput in Mbit/sec alg_nkeylen - number of supported key lengths alg_flag - In the DL_CAPABILITY_ACK response to a configuration request (DL_CAPABILITY_REQ command with no data), DL_CAPAB_ALG_ENABLE will be set if this algorithm has been previously enabled, 0 otherwise. In a DL_CAPABILITY_REQ with configuration data, DL_CAPAB_ALG_ENABLE will be set to enable this algorithm. The response DL_CAPABILITY_ACK will also to have this flag set to confirm that the capability was enabled, or cleared to confirm that it was disabled. alg_keylen - Variable number of elements reflecting each enabled algorithm key length DL_CONTROL_REQ: As described in PSARC/2001/070/, a DL_CONTROL_REQ command is used by upper layer software to configure the capabilities returned by the response to DL_CAPABILITY_REQ. Capability configuration as defined by this proposal is the process of downloading, updating and retrieving IPsec SADB information to the underlying Venus driver. The Venus driver will use the SADB information to cache and maintain keying material which is necessary to apply hardware acceleration to IPsec packets. In constructing a DL_CONTROL_REQ command, upper layer software will send down to the DLS provider an M_PROTO mblk containing a dl_control_t structure with dl_primitive set to DL_CONTROL_REQ. For this proposal, the dl_type field will be set to DL_CT_IPSEC_AH to control IPsec authentication data at the DLS provider or DL_CT_IPSEC_ESP to control IPsec encryption data at the DLS provider. Based on the dl_operation element of the dl_control_t structure, optional dl_key data and dl_data will follow. The dl_key for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is | dl_ct_ipsec_key_t, which is the IPsec SPI value of the SA, and the | destination address. PSARC/2001/070/ specifies which data fields are mandatory per dl_operation control operation. The dl_data for this implementation (both DL_CT_IPSEC_AH and DL_CT_IPSEC_ESP) is the dl_ct_ipsec_t structure, which contains a minimal SADB entry to provide the required information to enable hardware accelerated encryption and authentication. The dl_data element following dl_control_t contains a dl_ct_ipsec_t which will be filled in with the the elements defined as per and RFC 2367. dl_control_t: dl_primitive - DL_CONTROL_REQ or DL_CONTROL_ACK dl_operation - control operation dl_type - DL_CT_IPSEC_AH or DL_CT_IPSEC_ESP dl_key_offset - sizeof(dl_control_t) | dl_key_length - sizeof(dl_ct_ipsec_key_t) | dl_data_offset - sizeof(dl_control_t) + | sizeof(dl_ct_ipsec_key_t) | dl_data_length - sizeof(dl_ct_ipsec_t) The dl_operation element of the dl_control_t has the following semantics (these semantics are aligned with base message types | defined in RFC 2367). A SA is uniquely identified by the tuple | where dl_type is from the | dl_control_t dl_type field, and the SPI and destination address | are from the dl_ct_ipsec_key_t dl_key_spi and dl_key_dest_addr | elements following the dl_control_t. This tuple is necessary | to correctly and uniquely identify a Security Association. DL_CO_ADD - Requires dl_type, dl_key and dl_data. Add this entry to the DLS provider dataset, keyed at . If an entry already exists at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. If there is insufficient room to add the entry, response is a DL_ERROR_ACK with return value DL_TOOMANY. Otherwise a DL_CONTROL_ACK will be returned, containing the same data as the original DL_CONTROL_REQ. DL_CO_DELETE - Requires dl_type, dl_key. Delete the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_FLUSH - Requires dl_type. Delete all entries of type dl_type from the DLS provider dataset. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. DL_CO_GET - Requires dl_type, dl_key. Return the entry from the DLS provider dataset, keyed at . If an entry doesn't exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned with a dl_ct_ipsec_t structure populated with the data associated with this . DL_CO_UPDATE - Requires dl_type, dl_key and dl_data. Update the entry in the DLS provider dataset, keyed at . If an entry doesn't already exist at this , the DLS provider will return a DL_ERROR_ACK with return value DL_OUTSTATE. Otherwise a DL_CONTROL_ACK will be returned containing the same data as the original DL_CONTROL_REQ. REFERENCE DOCUMENTS PSARC/2001/070/ DL_CAPABILITY_REQ/CTL extensible interface for detecting and/or enabling lower level driver capabilities. pf_key(7P) man page /usr/include/net/pfkeyv2.h RFC 2367: PF_KEY Key Management API, Version 2 --Cast_of_Hawks_380_000-- From sac-list-owner Wed Mar 21 13:29:02 2001 Date: Wed, 21 Mar 2001 13:30:05 -0800 (PST) From: Glenn Skinner Subject: Re: 2001/070 DL_{CAPABILITY,CONTROL}_REQ To: Jack.Bochsler@west.sun.com Cc: psarc@sac.eng.sun.com MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Content-MD5: HFepKtgrmKBKJd9QsXJqcQ== Content-Length: 82 This case was approved during ARC business of today's PSARC meeting. -- Glenn From sac-owner Fri Feb 15 16:13:11 2002 From: Bill Sommerfeld To: psarc@sac.eng.sun.com cc: ipsec-core@eng.sun.com, bruce.grove@sun.com, michael.mcpherson@sun.com, mehdi.bonyadi@sun.com, jack.bochsler@sun.com Subject: PSARC 2001/070: contract administrivia. Date: Fri, 15 Feb 2002 19:12:45 -0500 Content-Length: 764 I will very shortly be proposing a fasttrack making minor changes to parts of the contract-private interfaces in 2001/070. The one existing contract, dated 3/8/2001, says (as part of unedited boilerplate): A copy of this contract shall be deposited in the CASE directory as "contract-" or in a "contracts" subdirectory. This did not happen. I extracted the contract from the "mail" file and put it in the case directory. An e-mail alias "contract-yyyy-nnn-ss@sun.com" shall be created via netadmin for notification of any desired changes. The SUPPLIER shall be the alias owner. This also apparently didn't happen. I just created contract-2001-070-1@sun.com with the folks listed on the CC: line above as initial members. - Bill From sacadmin Thu Aug 14 14:25:38 2008 Received: from sac.sfbay.sun.com (localhost [127.0.0.1]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7ELPcbr002745; Thu, 14 Aug 2008 14:25:38 -0700 (PDT) Received: (from sommerfe@localhost) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id m7ELPcLO002741; Thu, 14 Aug 2008 14:25:38 -0700 (PDT) Date: Thu, 14 Aug 2008 14:25:38 -0700 (PDT) From: William Sommerfeld Message-Id: <200808142125.m7ELPcLO002741@sac.sfbay.sun.com> To: PSARC-record@sac.sfbay.sun.com Cc: ipsec-core@sun.com Subject: EOF of 2001/070 IPsec HW Acceleration support [PSARC/2008/522 FastTrack timeout 08/21/2008] Status: RO Content-Length: 580 Template Version: @(#)sac_nextcase 1.66 04/17/08 SMI This information is Copyright 2008 Sun Microsystems 1. Introduction 1.1. Project/Component Working Name: EOF of 2001/070 IPsec HW Acceleration support 1.2. Name of Document Author/Supplier: Author: Dan McDonald 1.3 Date of This Document: 14 August, 2008 4. Technical Description See the case directory for more detail 6. Resources and Schedule 6.4. Steering Committee requested information 6.4.1. Consolidation C-team Name: ON 6.5. ARC review type: FastTrack 6.6. ARC Exposure: open From sommerfeld@sun.com Thu Aug 14 14:29:11 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7ELTAgS003017 for ; Thu, 14 Aug 2008 14:29:11 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m7ELSsk6020812; Fri, 15 Aug 2008 05:29:07 +0800 (SGT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K5M00F0710HXH00@brm-avmta-1.central.sun.com>; Thu, 14 Aug 2008 15:29:05 -0600 (MDT) Received: from thunk.east.sun.com ([129.148.174.66]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K5M00LPW10GJ1C0@brm-avmta-1.central.sun.com>; Thu, 14 Aug 2008 15:29:05 -0600 (MDT) Received: from [IPv6:::1] (localhost [IPv6:::1]) by thunk.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m7ELT1qc003236; Thu, 14 Aug 2008 17:29:01 -0400 (EDT) Date: Thu, 14 Aug 2008 17:28:59 -0400 From: Bill Sommerfeld Subject: 2008/522 EOF of 2001/070 IPsec HW Acceleration support To: PSARC-ext Cc: Dan McDonald Message-id: <1218749340.2286.25.camel@thunk> MIME-version: 1.0 X-Mailer: Evolution 2.22.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 Status: RO Content-Length: 3286 I'm sponsoring this case for Dan McDonald. Timeout expires 8/21/2008. Release binding is Minor - this interface will only be removed from post-Solaris-10 releases. This is an Open case about a Contract Private interface implemented by a closed-source driver for EOL'ed hardware which is used by the open-source Solaris IP stack. ---- We propose to remove the use of the DL_CAPAB_IPSEC_* interface capabilities from the solaris IP stack in a future Minor release. This is a Contracted Consolidation Private interface between ON and CPG. The hardware product which exported this interface (SCA 4000 aka. "Venus") has been EOLed due to RoHS. The SCA4000 combined a gigabit ethernet port with a cryptographic coprocessor; the DL_CAPAP_IPSEC_* interfaces allowed the crypto-aware NIC on board the SCA4000 to encrypt and then transmit, or receive and decrypt, an IPsec-encrypted packet without sending the packet data over the I/O bus three times (in and out of a crypto unit and then out the ethernet). The follow-on RoHS-compliant SCA-6000 does not include the on-board NIC; no other devices are known to export the same acceleration interface. If we want to support similar devices in the future, we do not recommend reusing this interface; instead, we recommend extending GLDv3 to control the functionality and share the IPsec SADB with the card via a synchronous function-call interface. Description: ------------ Part of PSARC 2001/070 describes STREAMS interfaces provided by a hardware driver to enable Network Interface Card (NIC)-level acceleration of IPsec encryption or data integrity. The only provider of this interface was the Sun Crypto Accelerator 4000, aka. "Venus". Venus has been EOLed due to non-compliance with EU RoHS laws. According to the contract in 2001/070: 7. Changes to INTERFACES requires ARC approval. If SUPPLIER decides to change (including replace or remove) any portion of the INTERFACES, SUPPLIER will notify CONSUMER of the proposed new version, no later than the application for ARC approval of the new version. If SUPPLIER and CONSUMER are contained in the same bundle, they have the option of arranging for simultaneous conversion to the new interfaces. If this is not possible, or if they are not in the same bundle, then SUPPLIER will either make best effort to work with CONSUMER so that CONSUMER can detect which version of INTERFACES is being supplied, or else SUPPLIER will make best effort to supply both old and new versions of INTERFACES. If SUPPLIER cannot make both versions of INTERFACES available, and SUPPLIER and CONSUMER cannot devise a method whereby CONSUMER can detect which version of INTERFACES is being supplied, and the old version of CONSUMER will not run with the new version of SUPPLIER, then either the EOL process must be followed by SUPPLIER, or else a major release of SUPPLIER will be required. The SUPPLIER was CPG - the Venus folks. We propose to stop using this interface within the Solaris IPsec stack. The Venus card can continue to be used to provide general cryptographic acceleration and keystore functionality even without the use of this interface. From gdamore@sun.com Fri Aug 15 00:45:23 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7F7jNI2026252 for ; Fri, 15 Aug 2008 00:45:23 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m7F7jHUB022133; Fri, 15 Aug 2008 08:45:22 +0100 (BST) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K5M00G0DTJJLU00@brm-avmta-1.central.sun.com>; Fri, 15 Aug 2008 01:45:19 -0600 (MDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K5M00D3UTJIV610@brm-avmta-1.central.sun.com>; Fri, 15 Aug 2008 01:45:19 -0600 (MDT) Received: from fe-sfbay-09.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m7F7jIZo009441; Fri, 15 Aug 2008 00:45:18 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-09.sun.com by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K5M00801THQE700@fe-sfbay-09.sun.com> (original mail from gdamore@sun.com) ; Fri, 15 Aug 2008 00:45:18 -0700 (PDT) Received: from [10.7.251.172] by fe-sfbay-09.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K5M00KCQTJILEF0@fe-sfbay-09.sun.com>; Fri, 15 Aug 2008 00:45:18 -0700 (PDT) Date: Fri, 15 Aug 2008 00:39:27 -0700 From: "Garrett D'Amore" Subject: Re: 2008/522 EOF of 2001/070 IPsec HW Acceleration support In-reply-to: <1218749340.2286.25.camel@thunk> Sender: Garrett.Damore@sun.com To: Bill Sommerfeld Cc: PSARC-ext , Dan McDonald Message-id: <48A532AF.2010201@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1218749340.2286.25.camel@thunk> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 4321 I'm wholly in favor of this case. However, to be fully clear, the SCA4000 should not suffer any loss in functionality, as it can use what we used to call "slow path IPsec acceleration", that is, it can use kcf to perform crypto operations for it. HOWEVER, there may be potential ramifications for * performance -- the "slow path" causes packets to traverse PCI busses 3 times, as noted, and certainly involves more complexity in the paths (notably kcf scheduling of crypto gets involved). * FIPS 140-2 compliance for SCA 4000? (Not that anyone would be certifying SCA 4000 on post S10 releases)... * Potential (uncertain!) management considerations for secure key management, although its unclear to me that SCA 4000 ever properly secured IPsec session keys. (IKE should be able to use secure storage for public keys via kcf, though.) -- Garrett Bill Sommerfeld wrote: > I'm sponsoring this case for Dan McDonald. Timeout expires 8/21/2008. > Release binding is Minor - this interface will only be removed from > post-Solaris-10 releases. > > This is an Open case about a Contract Private interface implemented by a > closed-source driver for EOL'ed hardware which is used by the > open-source Solaris IP stack. > > ---- > > We propose to remove the use of the DL_CAPAB_IPSEC_* interface > capabilities from the solaris IP stack in a future Minor release. > This is a Contracted Consolidation Private interface between ON and > CPG. > > The hardware product which exported this interface (SCA 4000 > aka. "Venus") has been EOLed due to RoHS. > > The SCA4000 combined a gigabit ethernet port with a cryptographic > coprocessor; the DL_CAPAP_IPSEC_* interfaces allowed the crypto-aware > NIC on board the SCA4000 to encrypt and then transmit, or receive and > decrypt, an IPsec-encrypted packet without sending the packet data > over the I/O bus three times (in and out of a crypto unit and then out > the ethernet). > > The follow-on RoHS-compliant SCA-6000 does not include the on-board > NIC; no other devices are known to export the same acceleration > interface. > > If we want to support similar devices in the future, we do not > recommend reusing this interface; instead, we recommend extending > GLDv3 to control the functionality and share the IPsec SADB with the > card via a synchronous function-call interface. > > Description: > ------------ > > Part of PSARC 2001/070 describes STREAMS interfaces provided by a > hardware driver to enable Network Interface Card (NIC)-level > acceleration of IPsec encryption or data integrity. > > The only provider of this interface was the Sun Crypto Accelerator 4000, > aka. "Venus". Venus has been EOLed due to non-compliance with EU RoHS laws. > According to the contract in 2001/070: > > 7. Changes to INTERFACES requires ARC approval. If SUPPLIER decides > to change (including replace or remove) any portion of the > INTERFACES, SUPPLIER will notify CONSUMER of the proposed new > version, no later than the application for ARC approval of the new > version. If SUPPLIER and CONSUMER are contained in the same bundle, > they have the option of arranging for simultaneous conversion to the > new interfaces. If this is not possible, or if they are not in the > same bundle, then SUPPLIER will either make best effort to work with > CONSUMER so that CONSUMER can detect which version of INTERFACES is > being supplied, or else SUPPLIER will make best effort to supply both > old and new versions of INTERFACES. If SUPPLIER cannot make both > versions of INTERFACES available, and SUPPLIER and CONSUMER cannot > devise a method whereby CONSUMER can detect which version of > INTERFACES is being supplied, and the old version of CONSUMER will > not run with the new version of SUPPLIER, then either the EOL process > must be followed by SUPPLIER, or else a major release of SUPPLIER > will be required. > > The SUPPLIER was CPG - the Venus folks. We propose to stop using this > interface within the Solaris IPsec stack. The Venus card can continue > to be used to provide general cryptographic acceleration and keystore > functionality even without the use of this interface. > > > > From sommerfeld@sun.com Fri Aug 15 07:51:21 2008 Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7FEpKYZ005770 for ; Fri, 15 Aug 2008 07:51:21 -0700 (PDT) Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11]) by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id m7FEpJO2031414; Fri, 15 Aug 2008 08:51:19 -0600 (MDT) Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K5N00201D9I8V00@brm-avmta-1.central.sun.com>; Fri, 15 Aug 2008 08:51:18 -0600 (MDT) Received: from dm-east-02.east.sun.com ([129.148.13.5]) by brm-avmta-1.central.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K5N00LJFD9H3O30@brm-avmta-1.central.sun.com>; Fri, 15 Aug 2008 08:51:18 -0600 (MDT) Received: from localhost.east.sun.com (vroom.East.Sun.COM [129.148.19.3]) by dm-east-02.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m7FEpHvP001304; Fri, 15 Aug 2008 10:51:17 -0400 (EDT) Received: from localhost.east.sun.com (localhost [127.0.0.1]) by localhost.east.sun.com (8.14.3+Sun/8.14.3) with ESMTP id m7FEpHDv027666; Fri, 15 Aug 2008 10:51:17 -0400 (EDT) Received: (from sommerfeld@localhost) by localhost.east.sun.com (8.14.3+Sun/8.14.3/Submit) id m7FEpGFc027665; Fri, 15 Aug 2008 10:51:16 -0400 (EDT) Date: Fri, 15 Aug 2008 10:51:16 -0400 From: Bill Sommerfeld Subject: Re: 2008/522 EOF of 2001/070 IPsec HW Acceleration support In-reply-to: <48A532AF.2010201@sun.com> To: "Garrett D'Amore" Cc: PSARC-ext , Dan McDonald Message-id: <1218811876.6977.23.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.22.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1218749340.2286.25.camel@thunk> <48A532AF.2010201@sun.com> X-Authentication-warning: localhost.east.sun.com: sommerfeld set sender to sommerfeld@sun.com using -f x_sac_archived: PSARC/2008/522 Status: RO Content-Length: 1946 On Fri, 2008-08-15 at 00:39 -0700, Garrett D'Amore wrote: > * performance -- the "slow path" causes packets to traverse PCI > busses 3 times, as noted, and certainly involves more complexity in the > paths (notably kcf scheduling of crypto gets involved). The SCA4000 IPsec offload interface only accelerates 3DES; all the cool kids are now using AES. AES in software (not to mention the niagara 2 crypto functional units) on current systems will be faster than 3DES offloaded through an SCA4000 using this interface and the data will also make only one traversal of an I/O bus. IMHO the primary motivation for building IPsec offload in the 2001/070 style would be for assurance, not performance reasons: 1) to allow sensitive keying material to stay entirely within the boundary of the cryptographic device 2) to allow for clear separation between cleartext and ciphertext. 2001/070 was built purely as an acceleration interface with no possibility of keeping session keys material out of the hands of the host processor. > * FIPS 140-2 compliance for SCA 4000? (Not that anyone would be > certifying SCA 4000 on post S10 releases)... can you be more specific about why this might matter? the 2001/070 interface involves the host IPsec passing raw (unwrapped) keying material to the driver. > * Potential (uncertain!) management considerations for secure key > management, although its unclear to me that SCA 4000 ever properly > secured IPsec session keys. it's quite clear that 2001/070 interface used with IKE and IPsec ESP/AH required keying material to leave and then reenter the FIPS 140 evaluation boundary of the card. > (IKE should be able to use secure storage > for public keys via kcf, though.) More importantly, IKE will continue to be able to use secure storage for the long-term private authentication/signature keys via kcf/PKCS#11 and either the SCA4000 or SCA6000 keystore. - Bill From gdamore@Sun.COM Fri Aug 15 08:05:38 2008 Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7FF5bf3006238 for ; Fri, 15 Aug 2008 08:05:38 -0700 (PDT) Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74]) by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id m7FF5RUB001969; Fri, 15 Aug 2008 23:05:36 +0800 (SGT) Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K5N00G01DX94U00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 15 Aug 2008 08:05:33 -0700 (PDT) Received: from sca-es-mail-1.sun.com ([192.18.43.132]) by nwk-avmta-1.sfbay.Sun.COM (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K5N00GEBDX90N00@nwk-avmta-1.sfbay.Sun.COM>; Fri, 15 Aug 2008 08:05:33 -0700 (PDT) Received: from fe-sfbay-10.sun.com ([192.18.43.129]) by sca-es-mail-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id m7FF5X2Q023499; Fri, 15 Aug 2008 08:05:33 -0700 (PDT) Received: from conversion-daemon.fe-sfbay-10.sun.com by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) id <0K5N00L01DJ95N00@fe-sfbay-10.sun.com> (original mail from gdamore@sun.com) ; Fri, 15 Aug 2008 08:05:33 -0700 (PDT) Received: from [10.7.251.172] by fe-sfbay-10.sun.com (Sun Java System Messaging Server 6.2-8.04 (built Feb 28 2007)) with ESMTPSA id <0K5N004UDDX8HDF0@fe-sfbay-10.sun.com>; Fri, 15 Aug 2008 08:05:33 -0700 (PDT) Date: Fri, 15 Aug 2008 07:59:40 -0700 From: "Garrett D'Amore" Subject: Re: 2008/522 EOF of 2001/070 IPsec HW Acceleration support In-reply-to: <1218811876.6977.23.camel@localhost> Sender: Garrett.Damore@Sun.COM To: Bill Sommerfeld Cc: PSARC-ext , Dan McDonald Message-id: <48A599DC.7040201@sun.com> MIME-version: 1.0 Content-type: text/plain; format=flowed; charset=ISO-8859-1 Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1218749340.2286.25.camel@thunk> <48A532AF.2010201@sun.com> <1218811876.6977.23.camel@localhost> User-Agent: Thunderbird 2.0.0.14 (X11/20080616) Status: RO Content-Length: 3603 Bill Sommerfeld wrote: > On Fri, 2008-08-15 at 00:39 -0700, Garrett D'Amore wrote: > >> * performance -- the "slow path" causes packets to traverse PCI >> busses 3 times, as noted, and certainly involves more complexity in the >> paths (notably kcf scheduling of crypto gets involved). >> > > The SCA4000 IPsec offload interface only accelerates 3DES; all the cool > kids are now using AES. AES in software (not to mention the niagara 2 > crypto functional units) on current systems will be faster than 3DES > offloaded through an SCA4000 using this interface and the data will also > make only one traversal of an I/O bus. > Agreed. Recall, I'm in favor of this case. If Venus is EOL, then simplifying our stack would be a good thing. There may be folks who still have to use 3DES. (Though its been enough years now that hopefully that number is approaching vanishingly small. But then I suspect the number of customers who purchased SCA 4000 boards was pretty small as well.) > IMHO the primary motivation for building IPsec offload in the 2001/070 > style would be for assurance, not performance reasons: > 1) to allow sensitive keying material to stay entirely within the > boundary of the cryptographic device > 2) to allow for clear separation between cleartext and ciphertext. > > 2001/070 was built purely as an acceleration interface with no > possibility of keeping session keys material out of the hands of the > host processor. > Yes. Session keys make it to the host driver. At the time we did the work, the IPsec offload "inline" using these primitives was substantially faster than the out-of-band approach using kcf. > >> * FIPS 140-2 compliance for SCA 4000? (Not that anyone would be >> certifying SCA 4000 on post S10 releases)... >> > > can you be more specific about why this might matter? the 2001/070 > interface involves the host IPsec passing raw (unwrapped) keying > material to the driver. > The driver wraps the key material before passing it to the hardware. Its goofy. Read more below. > >> * Potential (uncertain!) management considerations for secure key >> management, although its unclear to me that SCA 4000 ever properly >> secured IPsec session keys. >> > > it's quite clear that 2001/070 interface used with IKE and IPsec ESP/AH > required keying material to leave and then reenter the FIPS 140 > evaluation boundary of the card. > Ah, but there are some goofy things in the language of FIPS 140-2, where if the keying material is "encrypted" before leaving the boundary, (and no specification is given for the mode of encryption) then it is considered OK. (Basically even very lame encryption with a fixed key qualifies as seperation of key material and data into separate logical channels.) So the driver negotiates a wrapping key with the board, and the board exports session keys wrapped, where the driver unwraps them. From a security standpoint, it offers absolutely *no* improvement. But it allowed for a FIPS evaluation. I'm not sure this is the way the final product shipped, but it was the way we were doing at the time I left the team. (I wrote that goofy code, with grave misgivings about it at the time.) > >> (IKE should be able to use secure storage >> for public keys via kcf, though.) >> > > More importantly, IKE will continue to be able to use secure storage for > the long-term private authentication/signature keys via kcf/PKCS#11 and > either the SCA4000 or SCA6000 keystore. > Right. -- Garrett > - Bill > > > > From sommerfeld@sun.com Tue Aug 26 15:44:39 2008 Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165]) by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id m7QMic1M014892 for ; Tue, 26 Aug 2008 15:44:38 -0700 (PDT) Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6]) by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m7QMiaw3016577; Tue, 26 Aug 2008 23:44:36 +0100 (BST) Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) id <0K6800B01CIB4L00@nwk-avmta-2.sfbay.sun.com>; Tue, 26 Aug 2008 15:44:35 -0700 (PDT) Received: from dm-east-01.east.sun.com ([129.148.9.192]) by nwk-avmta-2.sfbay.sun.com (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005)) with ESMTP id <0K68004FACIA4W90@nwk-avmta-2.sfbay.sun.com>; Tue, 26 Aug 2008 15:44:35 -0700 (PDT) Received: from localhost.SFBay.Sun.COM (dhcp-umpk17-109-36.SFBay.Sun.COM [129.146.109.36]) by dm-east-01.east.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id m7QMiY3c057216; Tue, 26 Aug 2008 18:44:34 -0400 (EDT) Received: from localhost.SFBay.Sun.COM (localhost [127.0.0.1]) by localhost.SFBay.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id m7QMiWqE009647; Tue, 26 Aug 2008 15:44:33 -0700 (PDT) Received: (from sommerfeld@localhost) by localhost.SFBay.Sun.COM (8.14.3+Sun/8.14.3/Submit) id m7QMiW0P009646; Tue, 26 Aug 2008 15:44:32 -0700 (PDT) Date: Tue, 26 Aug 2008 15:44:32 -0700 From: Bill Sommerfeld Subject: Re: 2008/522 EOF of 2001/070 IPsec HW Acceleration support In-reply-to: <1218749340.2286.25.camel@thunk> To: PSARC-ext Cc: Dan McDonald Message-id: <1219790672.1822.17.camel@localhost> MIME-version: 1.0 X-Mailer: Evolution 2.22.2 Content-type: text/plain Content-transfer-encoding: 7BIT X-PMX-Version: 5.4.1.325704 References: <1218749340.2286.25.camel@thunk> X-Authentication-warning: localhost.SFBay.Sun.COM: sommerfeld set sender to sommerfeld@sun.com using -f Status: RO Content-Length: 110 According to the minutes of the PSARC meeting of 8/20/2008, this case was approved. I've marked it as such.