From darrenm@sac.sfbay.sun.com Tue Oct 13 02:27:15 2009
Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9D9REIL022844
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 13 Oct 2009 02:27:14 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9D9RE1n026725;
	Tue, 13 Oct 2009 02:27:14 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRG008034XDLO00@nwk-avmta-1.sfbay.Sun.COM>; Tue,
 13 Oct 2009 02:27:13 -0700 (PDT)
Received: from dm-sfbay-02.sfbay.sun.com ([129.146.11.31])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRG000UD4XD9P30@nwk-avmta-1.sfbay.Sun.COM>; Tue,
 13 Oct 2009 02:27:13 -0700 (PDT)
Received: from sac.sfbay.sun.com (sac.SFBay.Sun.COM [129.146.226.132])
 by dm-sfbay-02.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2)
 with ESMTP id n9D9RBm2046235; Tue, 13 Oct 2009 02:27:11 -0700 (PDT)
Received: from sac.sfbay.sun.com (localhost [127.0.0.1])
 by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9D9RAoA022839; Tue,
 13 Oct 2009 02:27:10 -0700 (PDT)
Received: (from darrenm@localhost)
 by sac.sfbay.sun.com (8.13.8+Sun/8.13.8/Submit) id n9D9RA4e022835; Tue,
 13 Oct 2009 02:27:10 -0700 (PDT)
Date: Tue, 13 Oct 2009 02:27:10 -0700 (PDT)
From: Darren J Moffat <darrenm@sac.sfbay.sun.com>
Subject: OpenSSL RSA keys by reference in PKCS#11 keystores through the PKCS11
 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
To: PSARC-ext@sun.com
Cc: Jan.Pechanec@sun.com, crypto-discuss@opensolaris.org,
        openssl-interst@sun.com
Message-id: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
Status: RO
Content-Length: 17208


Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
    1.1. Project/Component Working Name:
	 OpenSSL RSA keys by reference in PKCS#11 keystores through the PKCS11 engine
    1.2. Name of Document Author/Supplier:
	 Author:  Jan Pechanec
    1.3  Date of This Document:
	13 October, 2009
    1.5. Email Aliases:
    	1.5.1. Responsible Manager: Anup.Sekhar@Sun.COM
    	1.5.2. Responsible Engineer: Jan.Pechanec@Sun.COM
    	1.5.3. Marketing Manager: Mark.Thacker@Sun.COM
	1.5.4. Interest List: openssl-interest@Sun.COM

2. Project Summary
   2.1. Project Description:

	OpenSSL applications can not currently take advantage of the added
	security provided by hardware keystores/tokens. Changes are needed to
	allow OpenSSL applications to use PKCS#11 keystores provided via the
	Solaris Crypto Framework (softtoken, Sun Crypto Accelerator 6000, etc.)
	so that it's possible to access existing RSA private and public keys
	in the keystores through the OpenSSL API. Certificates cannot be used
	due to the limitations of the ENGINE API.

	The change needed is entirely within the PKCS#11 engine that has been
	internally developed at Sun. Presently the engine can load only RSA keys
	from files in PEM format stored on disk so no support for DSA keys will
	be added as part of this project. DSA keys in PKCS#11 keystores seem not
	be of the interest to our PKCS#11 engine users for now.

	Given the limitation of the existing OpenSSL ENGINE API we can only work
	with existing keys in the keystore. We will not be able to generate
	keys. For such operations, the PKCS#11 API or different tools such as
	pktool(1) must be used.

	While DSA keys support seems to be possible to add in the future, we
	would not be able to add support for referencing symmetric keys without
	extending the external ENGINE API.

	A potential use case might be mod_ssl in Apache, for example, or using
	OpenSSL scripts that implement basic certification authority together
	with SCA-6000 hardware keystore. Note that Apache's mod_ssl would need
	nontrivial changes to make use of RSA keys by reference and such changes
	are not part of this project.

   2.2. Risks and Assumptions:

	None.


3. Business Summary
   3.1. Problem Area:

	Users are more familiar with the OpenSSL API than the PKCS#11 API or NSS
	API, and more applications exist using the OpenSSL API than those using
	the latter ones. It's usually easier to modify an existing application
	that uses OpenSSL than to develop a new one, possibly using a different
	API. From those reasons, users are asking for a way to access RSA keys
	stored in PKCS#11 keystores through the OpenSSL API.

   3.3. Business Justification:

	People who buy SCA-6000 crypto cards very often want to use its HW
	keystore for RSA keys through OpenSSL API.

4. Technical Description:
    4.1. Details:

	4.1.1 Overview

	The OpenSSL ENGINE interface provides 2 functions for accessing private
	and public keys through an engine from any OpenSSL application:

		ENGINE_load_private_key(), ENGINE_load_public_key()

	the identification of the key is in the 1st parameter, "char *key_id",
	and it is up to the engine how to interpret it. Currently, the PKCS#11
	engine supports only RSA keys and "key_id" is interpreted as a filename.

	The present semantics of the "key_id" parameter will be extended 
	to accept a PKCS#11 URI. Such URI should uniquely identify the key
	in the PKCS#11 keystore. We will also start accepting "file://" URI to
	provide a workaround to solve possible clashes with the existing
	filenames starting with "pkcs11:" prefix. Note that "file://" URI will
	be only accepted when the engine is used.

	4.1.2 PKCS#11 URI

	The format of PKCS#11 URI was designed in Sun with possible future
	integration into the PKCS#11 standard in mind, and was discussed openly
	in the community mailing list. Its format follows. The ordering of
	attributes is NOT significant but every attribute can can be used at
	most once.

	pkcs11:[token=<label>][;manuf=<label>][;serial=<label>][;model=<label>]
    	       [;object=<label>][;objecttype=(public|private|cert)]
	       [;passphrasedialog=(builtin|exec:<file>)]

	where:
		"token" is the token label, eg. "Sun Metaslot"

		"manuf" is the manufacturer ID, eg. "Sun Microsystems, Inc."

		"serial" is the serial number

		"model" is the model, eg. "sca6000"

		"object" is the object (key) label, eg. "mykey"

		"objecttype" is either "public", "private", or "cert"

		"passphrasedialog" was modeled after Apache's
			SSLPassPhraseDialog directive. With "builtin", the
			PKCS#11 engine uses getpassphrase(3c) to read the PIN
			from the terminal. With "exec:<command>", the external
			command is run and the PIN is read from its standard
			output. Maximum PIN length supported is 256 bytes which
			is the maximum that getpassphrase(3C) supports.

	Currently, the PKCS#11 engine ignores the "objecttype" keyword since by
	using the specific load function it is determined what key type will be
	searched for. "objecttype" is specified here for the sake of
	completeness of the PKCS#11 URI specification. Also note that we cannot
	work with certificates that way, only with public and private keys.

	The keyword "pin" is intentionally NOT provided due to inherent security
	problems of using a piece of secret information in the process arguments
	that are visible in the process list to all users of the system.

	If "passphrasedialog" is not provided but ENGINE_load_private_key() is
	used and the token requires PIN the ENGINE_load_private_key() function
	will fail. The same is true for cases where token is not initialized or
	PIN is not set (but required). However, if PIN is not required by the
	token the engine will continue. That's the case when public keys are
	referenced.

	The only mandatory attribute is the "object" attribute that contains the
	key label. As noted below, the engine's use of the slots automatically
	determines the keystore used which might seem to obsolesce the use of
	token attributes "token", "manuf", "serial", and "model". However, users
	might want to provide such information to make sure they pick the right
	keystore. If the provided information does NOT match with the keystore
	used by the slot chosen in the engine, the load operation will fail.

	4.1.3 Technical Details

	The PKCS#11 engine never reads private components of the keys stored in
	the keystores to the memory, no matter whether those keys are labeled
	sensitive or not. Both ENGINE_load_private_key() and
	ENGINE_load_public_key() return a native OpenSSL structure called
	EVP_PKEY. Even for the private key, the structure will contain only
	public components of the key and those components will be used to look
	up the PKCS#11 object handle for the private key for every operation
	performed on that key.

	If an application is using the key by reference approach it must NOT
	expect to be able to use private key components internally. If the
	application uses the OpenSSL API properly, private key components in the
	process memory would be needed only for exporting the private key from
	the keystore but never for signing.

	4.1.3 Sun's PKCS#11 engine specifics

	Currently, the PKCS#11 engine uses the same slot for all RSA/DH/DSA
	operations and the slot is chosen during the initialization of the
	engine. That means that we can use only the keystore that the chosen slot
	provides. Usually that will be "Sun Metaslot" because the PKCS#11 engine
	will use that slot unless manually disabled. Note that by default, the
	softtoken keystore is used as the metaslot's token. Users can use
	metaslot's METASLOT_OBJECTSTORE_TOKEN environment variable to switch to
	a different, possibly hardware, keystore. More specifically, if the URI
	specifies a different token name in its "token" attribute
	than the one used by the metaslot, the engine will fail when trying to
	load the key. That way the user can use those attributes to make sure
	the correct key is used. The same stands for "manuf", "serial", and
	"model" attributes. For more information on the metaslot's keystore, see
	libpkcs11(3LIB).

	If more than one key with a given specification is found the engine will
	fail in its load functions.

	Given the known issues with the PKCS#11 fork safety requirements the
	engine tries to re-initialize itself in the child when the fork is
	detected. When the parent is logged into the token, the child must log
	in again in order to continue to access private keys loaded in the
	parent. A built-in passphrase dialog will not be probably suitable for
	re-reading the PIN from the child. See the "4.1.4 PIN Caching Policy"
	for how to cache the PIN in memory. Alternatively, use
	"passphrasedialog" with the "exec" option. Failure to provide a PIN in
	the child is a failure to continue to use the engine. It's up to the
	user to adequately secure the external command if used.

	4.1.4 PIN Caching Policy

	The PKCS#11 engine does NOT cache the PIN in memory by default. We
	assume that if the application does not fork then caching the PIN by
	default would not be only useless but also potentially dangerous (PIN
	could end up in swap or system and process core dumps). However, we
	provide 2 options for PIN caching so that the PIN can be used in the
	child to relogin to the token. The engine honors a special environment
	variable:

	OPENSSL_PKCS11_PIN_CACHING_POLICY

	which can have "none", "mlocked-memory", and "memory" values.

	The "none" value is the default where the PIN is forgotten after the
	login. "memory" will keep the PIN in the memory, and "mlocked-memory"
	will keep the PIN in a locked page via mlock(3C). Note that mlock()
	requires the PRIV_PROC_LOCK_MEMORY privilege which is not in the default
	user set. An admin must grant the application or the user the relevant
	privilege if the "mlocked-memory" option is to be used. With that option
	and without the privilege the load function will fail on loading the 1st
	private key if the token requires the PIN - it will NOT resort to use
	the "memory" option. Also, if the PIN is required then any incorrect
	value of OPENSSL_PKCS11_PIN_CACHING_POLICY will result in a failure in
	the load function. If the PIN is not required (eg. providing a PIN in
	URIs for public keys only) the PIN will not be used and will be erased
	from memory no matter how OPENSSL_PKCS11_PIN_CACHING_POLICY is set.

	While we could use the ENGINE control API to provide such information to
	the engine the use of an enviroment variable seems to be more apropriate
	here since it will provide more options to applications using RSA keys
	by reference without any need for code modifications.

	4.1.5 Accessing DSA keys

	Currently, the engine does not provide for loading DSA keys from the
	disk through the engine. While this could be done and DSA keys could be
	used from the HW key stores it is not part of the project. What's more,
	we haven't received any user requests to provide such functionality yet.


    4.2. Bug/RFE Number(s):

	the project implements this RFE:

		6479874 OpenSSL should support RSA key by reference/hardware
			keystores

	I'll fix following PKCS#11 engine CRs as part of this project:

		6732677 make check to trigger Solaris specific code automatic in
			the PKCS#11 engine
		6872415 pkcs11 engine's check_new_rsa_key_priv() should use a
			public key component

    4.5. Interfaces:

	- we do NOT add any new command line option or OpenSSL API functions

	- we overload the filename options so that PKCS#11 URI can be provided,
	  see above.

	
    4.6. Doc Impact:

	Manual page for openssl(5) will be modified. Draft follows.


--- openssl.5   Wed Aug 19 16:04:12 2009
+++ openssl.5.new       Wed Sep 30 12:39:57 2009
@@ -36,40 +36,116 @@
      to the Cryptographic Framework through the PKCS#11 engine on
      a given machine, run the following command:
 
        /usr/sfw/bin/openssl engine -vvv -t -c
 
      Due  to  requirements  of  the  PKCS#11  standard  regarding
      fork(2) behavior, some applications that use the OpenSSL EVP
      interfaces and fork()  with  active  crypto  contexts  might
      experience unexpected behavior.
 
+  Building an OpenSSL Application
+
      To build an OpenSSL application, use the following  cc  com-
      mand line options:
 
        cc [ flag... ] file... -lcrypto -lssl [ library... ]
 
+  Accessing RSA Keys in PKCS#11 Keystores
+
+     OpenSSL can access RSA keys in PKCS#11 keystores using the
+     following functions of the ENGINE API:
+
+       EVP_PKEY *ENGINE_load_private_key(ENGINE *e,
+                       const char *key_id, UI_METHOD *ui_method,
+               void *callback_data)
+
+       EVP_PKEY *ENGINE_load_public_key(ENGINE *e,
+                       const char *key_id, UI_METHOD *ui_method,
+               void *callback_data)
+
+     key_id, formerly for filenames only, can be now also set to
+     a PKCS#11 URI. To avoid clashes with existing filenames,
+     "file://" prefix for filenames is now also accepted but
+     only when the PKCS#11 engine is in use. The PKCS#11 URI
+     specification follows:
+
+       pkcs11:[token=<label>][;manuf=<label>][;serial=<label>]
+              [;model=<label>][;object=<label>]
+              [;objecttype=(public|private|cert)]
+              [;passphrasedialog=(builtin|exec:<file>)]
+
+     The ordering of keywords is not significant. The PKCS#11
+     engine uses the keystore for the slot chosen for public key
+     operations whic is metaslot on a standardly configured
+     machine. Currently, the PKCS#11 engine ignores "objecttype"
+     keyword. The only mandatory keyword is "object" which is
+     the key object label. For information on how to use a
+     different, possibly hardware, keystore with metaslot see
+     libpkcs11(3LIB).
+
+     The token PIN is provided via "passphrasedialog" keyword and
+     is either read from the terminal ("builtin") or from the
+     output of an external command ("exec:<file>"). The PIN is
+     used to log into the token and by default is deleted from
+     the memory then. The keyword "pin" is intentionally not
+     provided due to inherent security problems of possible use
+     of a password in the process arguments.
+
+     Due to fork safety issues the application must re-login if
+     the child continues to use the PKCS#11 engine. It is done
+     inside of the engine automatically if fork is detected and
+     in that case, "exec:<file>" option of the "passphrasedialog"
+     keyword can be used. Alternatively, an enviroment variable
+     OPENSSL_PKCS11_PIN_CACHING_POLICY can be used to allow the
+     PIN to be cached in memory and reused in the child. It can
+     be set to "none" which is the default, "memory" to store
+     the PIN in memory, and "mlocked-memory" keep the PIN in a
+     locked page via mlock(3C). Note that PRIV_PROC_LOCK_MEMORY
+     privilege is required in that case.
+
+     Sensitive parts of private keys are never read from the
+     token to the process memory no matter whether the key is
+     tagged with sensitive flag or not. The PKCS#11 engine uses
+     the public compoments as a search key to get a PKCS#11
+     object handle to the private key.
+
+  Additional Documentation
+
      Extensive additional documentation for  OpenSSL  modules  is
      available       in      the      /usr/share/man/man1openssl,
      /usr/share/man/man3openssl, /usr/share/man/man5openssl,  and
      /usr/share/man/man7openssl directories.
 
      To view the license terms, attribution,  and  copyright  for
      OpenSSL, see /var/sadm/pkg/SUNWopensslr/install/copyright.
 
+EXAMPLES
+
+     Example 1: generate and print a public key stored in an
+               already initilized PKCS#11 keystore. Note the
+               use of "-engine pkcs11" and "-inform e".
+
+       $ pktool gencert keystore=pkcs11 label=mykey \
+           subject="CN=test" keytype=rsa keylen=1024 serial=01
+       $ openssl rsa -in "pkcs11:object=mykey;passphrasedialog=builtin" \
+           -pubout -text -engine pkcs11 -inform e
+
 ATTRIBUTES
      See attributes(5) for a description of the following  attri-
      butes:
 
      ____________________________________________________________
     |       ATTRIBUTE TYPE        |       ATTRIBUTE VALUE       |
     |_____________________________|_____________________________|
     | Availability                | SUNWopensslr, SUNWopenssl   |
     |_____________________________|_____________________________|
     | Interface Stability         | External                    |
     |_____________________________|_____________________________|
 
 SEE ALSO
-     cryptoadm(1M), libpkcs11(3LIB), attributes(5)
+     cryptoadm(1M), libpkcs11(3LIB), attributes(5),
+     privileges(5), mlock(3C)
 
      /usr/share/man/man1openssl/openssl.1openssl,
-     /usr/sfw/man/man3/engine.3, /usr/sfw/man/man3/evp.3
+     /usr/share/man/man3openssl/engine.3,
+     /usr/share/man/man3openssl/evp.3


6. Resources and Schedule
    6.4. Steering Committee requested information
   	6.4.1. Consolidation C-team Name:
		SFW
    6.5. ARC review type: FastTrack
    6.6. ARC Exposure: open


From Kais.Belgaied@sun.com Fri Oct 16 17:05:05 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9H054av017955
	for <psarc-ext@sac.sfbay.sun.com>; Fri, 16 Oct 2009 17:05:04 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n9H04mgi003069;
	Sat, 17 Oct 2009 08:05:01 +0800 (SGT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRM0070TTKCL300@brm-avmta-1.central.sun.com>; Fri,
 16 Oct 2009 18:05:00 -0600 (MDT)
Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRM000DMTJREB30@brm-avmta-1.central.sun.com>; Fri,
 16 Oct 2009 18:04:39 -0600 (MDT)
Received: from [129.146.11.144]
 (sr1-jurassic-01.SFBay.Sun.COM [129.146.11.144])	by
 jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n9H04cJ2672452
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri,
 16 Oct 2009 17:04:38 -0700 (PDT)
Date: Fri, 16 Oct 2009 17:04:37 -0700
From: Kais Belgaied <Kais.Belgaied@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
To: Darren J Moffat <darrenm@sac.sfbay.sun.com>
Cc: PSARC-ext@sun.com, Jan.Pechanec@sun.com, crypto-discuss@opensolaris.org,
        openssl-interst@sun.com
Message-id: <4AD90A15.9080305@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
User-Agent: Thunderbird 2.0.0.21 (X11/20090311)
Status: RO
Content-Length: 1593

+0.75


a couple of questions below

>
> +     OpenSSL can access RSA keys in PKCS#11 keystores using the
> +     following functions of the ENGINE API:
> +
> +       EVP_PKEY *ENGINE_load_private_key(ENGINE *e,
> +                       const char *key_id, UI_METHOD *ui_method,
> +               void *callback_data)
> +
> +       EVP_PKEY *ENGINE_load_public_key(ENGINE *e,
> +                       const char *key_id, UI_METHOD *ui_method,
> +               void *callback_data)
>   

given the semantics described in the case, these functions will fail for 
multiple reasons: bad argument, key not found,
bad internal state (engine hasn't initialized or hasn't authenticated to 
the token). Yet the return value
can be either NULL: failure or Not NULL: a matching key was retrieved.
It will be more helpful to give the app developers some info as to the 
reason of failure, so that they
know what to do when the load function returns NULL.

Possibly Missing:
----------------------
1. Need to mention somewhere that the caller of the load functions is 
responsible for calling EVP_PKEY_free().

2. since the private parts of the on-token keys are never read by the 
engine, there is an implication on all OpenSSL
  access routines, like  EVP_PKEY_copy_parameters(), 
EVP_PKEY_get1_RSA(), etc. The'll all gonna fail when the
pkey arg comes from a token.
Rather than chasing the dozens of functions that use RSA private keys in 
openssl, maybe it suffices to
document that EVP_Decrypt() and EVP_PKEY_free() are the only routines 
that can use an  RSA private  key by reference.

    Kais.

From Jan.Pechanec@Sun.COM Sun Oct 18 23:29:25 2009
Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9J6TP55013666
	for <psarc-ext@sac.sfbay.sun.com>; Sun, 18 Oct 2009 23:29:25 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9J6TOCP028656;
	Sun, 18 Oct 2009 23:29:25 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRR003050P1ZM00@nwk-avmta-1.sfbay.Sun.COM>; Sun,
 18 Oct 2009 23:29:25 -0700 (PDT)
Received: from gmp-eb-inf-1.sun.com ([192.18.6.21])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRR00MDZ0OZYK10@nwk-avmta-1.sfbay.Sun.COM>; Sun,
 18 Oct 2009 23:29:24 -0700 (PDT)
Received: from fe-emea-09.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
 by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9J6TN0s023518; Mon,
 19 Oct 2009 06:29:23 +0000 (GMT)
Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRR00H000ADUB00@fe-emea-09.sun.com>; Mon, 19 Oct 2009 07:29:17 +0100 (BST)
Received: from rejewski ([unknown] [10.18.138.121])
 by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KRR00LPW0OSAX00@fe-emea-09.sun.com>;
 Mon, 19 Oct 2009 07:29:17 +0100 (BST)
Date: Mon, 19 Oct 2009 08:29:12 +0200 (CEST)
From: Jan Pechanec <Jan.Pechanec@Sun.COM>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4AD90A15.9080305@Sun.COM>
Sender: Jan.Pechanec@Sun.COM
X-X-Sender: jp161948@rejewski
To: Kais Belgaied <Kais.Belgaied@Sun.COM>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@Sun.COM,
        crypto-discuss@opensolaris.org, openssl-interst@Sun.COM
Message-id: <Pine.SOC.4.64.0910190758140.25074@rejewski>
MIME-version: 1.0
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM>
Status: RO
Content-Length: 5453

On Fri, 16 Oct 2009, Kais Belgaied wrote:

> a couple of questions below
>
>>
>> +     OpenSSL can access RSA keys in PKCS#11 keystores using the
>> +     following functions of the ENGINE API:
>> +
>> +       EVP_PKEY *ENGINE_load_private_key(ENGINE *e,
>> +                       const char *key_id, UI_METHOD *ui_method,
>> +               void *callback_data)
>> +
>> +       EVP_PKEY *ENGINE_load_public_key(ENGINE *e,
>> +                       const char *key_id, UI_METHOD *ui_method,
>> +               void *callback_data)
>>  
>
> given the semantics described in the case, these functions will fail for
> multiple reasons: bad argument, key not found,
> bad internal state (engine hasn't initialized or hasn't authenticated to the
> token). Yet the return value
> can be either NULL: failure or Not NULL: a matching key was retrieved.
> It will be more helpful to give the app developers some info as to the reason
> of failure, so that they
> know what to do when the load function returns NULL.

	hi Kais, there is a standard way of providing error output in 
OpenSSL. There is a stack of error messages that is used and a user can 
then print it out (for example, by calling 
"ERR_print_errors_fp(stderr);"). This is how it looks then in standard 
openssl(1) command which can work with RSA keys by reference without any 
modifications since it already uses ENGINE_load* functions:

# This is an invalid URI ("model" has no value)
#
$ openssl rsa -in
"pkcs11:object=mycert;passphrasedialog=exec:/export/tmp/askpass.sh;model" 
-pubout -text -engine pkcs11 -inform e
engine "pkcs11" set.
unable to load Private Key
4875:error:800780A9:lib(128):PK11_LOAD_PRIV_KEY:pkcs11 URI provided is 
invalid:hw_pk11_uri.c:343:
4875:error:26096080:engine routines:ENGINE_load_private_key:failed 
loading private key:eng_pkey.c:126:

# This is a non-existent key label.
#
$ openssl rsa -in 
"pkcs11:object=nonexistent;passphrasedialog=exec:/export/tmp/askpass.sh" 
-pubout -text -engine pkcs11 -inform e
engine "pkcs11" set.
unable to load Private Key
4885:error:800A00B5:lib(128):PK11_FIND_ONE_OBJECT:specified object not 
found:hw_pk11_pub.c:3233:
4885:error:26096080:engine routines:ENGINE_load_private_key:failed 
loading private key:eng_pkey.c:126:

# This is a bad password printed out of askpass.sh script.
#
$ openssl rsa -in 
"pkcs11:object=nonexistent;passphrasedialog=exec:/export/tmp/bad-passwd.sh" 
-pubout -text -engine pkcs11 -inform e
engine "pkcs11" set.
unable to load Private Key
4929:error:800A40A7:lib(128):PK11_TOKEN_LOGIN:C_Login() failed on 
token:hw_pk11_err.c:285:PK11 CK_RV=0Xa0
4929:error:26096080:engine routines:ENGINE_load_private_key:failed 
loading private key:eng_pkey.c:126:

...
...
(there are more error branches)

	those 2 functions above are part of original OpenSSL ENGINE API. 
We just can't change that easily. Also, this is a standard way of how 
OpenSSL does things in other functions.

> Possibly Missing:
> ----------------------
> 1. Need to mention somewhere that the caller of the load functions is
> responsible for calling EVP_PKEY_free().

	this should be documented in OpenSSL's engine(3) manual page 
which is unfortunately not the case. That's the problem with OpenSSL 
documentation as a whole, there are many things missing. I can add a 
note to our openssl(5) man page that EVP_PKEY_free() should be called 
then.

> 2. since the private parts of the on-token keys are never read by the engine,
> there is an implication on all OpenSSL
> access routines, like  EVP_PKEY_copy_parameters(), EVP_PKEY_get1_RSA(), etc.

	it's hard to say what EVP_PKEY_copy_parameters() does without 
reading the code since this is an undocumented fuction (sigh). From the 
code I can see that it does nothing for RSA keys, only for DSA and EC.

	EVP_PKEY_get1_RSA() just returns a pointer to RSA structure that 
is already there.

	in general, what I saw from the code and what I can see when 
writing new test cases, RSA related functions always check for NULL 
before processing a specific part of an RSA key. If there is NULL, they 
ignore the parameter. That stems from the fact that thay do not have 
separate structures for an RSA public key and an RSA private key. They 
must always assume that a public key only is there.

	of course, a user can use low level RSA functions that are below 
the engine. However, that's not how OpenSSL is to be used. OpenSSL team 
always say that programmers should always use high level APIs.

 > The'll all gonna fail when the
> pkey arg comes from a token.
> Rather than chasing the dozens of functions that use RSA private keys in
> openssl, maybe it suffices to
> document that EVP_Decrypt() and EVP_PKEY_free() are the only routines that can
> use an  RSA private  key by reference.

	EVP_Decrypt() is for symmetric crypto only so you probably mean 
RSA_private_encrypt(), for example. I'm also working on a new 
openssl-engine test code, please see for example how encrypting with RSA 
private key is done (one could also used EVP_SignInit() and stuff that 
is takes care of digesting the data):

http://rejewski.czech/workspace/rfe-6877504/webrev/usr/closed/suites/security/openssl-engine/tests/stress/pkcs11/kbr_multi_sign_with_multi_keys.c.html

	I may add a note to our openssl(5) draft change that high level 
API must be used for that, and can add an example of few such functions 
so that a user can get the picture. Is that OK?

	thanks, Jan.
 
-- 
Jan Pechanec

From Darren.Moffat@sun.com Mon Oct 19 01:22:05 2009
Received: from sunmail3mpk.sfbay.sun.com (sunmail3mpk.SFBay.Sun.COM [129.146.11.52])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9J8M5Uv028177
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 19 Oct 2009 01:22:05 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail3mpk.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9J8M4oq005828;
	Mon, 19 Oct 2009 01:22:04 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRR0090H5WRGO00@nwk-avmta-2.sfbay.sun.com>; Mon,
 19 Oct 2009 01:22:03 -0700 (PDT)
Received: from gmp-eb-inf-2.sun.com ([192.18.6.24])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRR004D75WPP130@nwk-avmta-2.sfbay.sun.com>; Mon,
 19 Oct 2009 01:22:02 -0700 (PDT)
Received: from fe-emea-10.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
 by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9J8Lxci004071; Mon,
 19 Oct 2009 08:22:01 +0000 (GMT)
Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRR009005TV3I00@fe-emea-10.sun.com>; Mon, 19 Oct 2009 09:21:52 +0100 (BST)
Received: from [192.168.1.105]
 (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150])
 by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KRR0065X5WCDS80@fe-emea-10.sun.com>; Mon,
 19 Oct 2009 09:21:49 +0100 (BST)
Date: Mon, 19 Oct 2009 09:21:48 +0100
From: Darren J Moffat <Darren.Moffat@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4AD90A15.9080305@Sun.COM>
Sender: Darren.Moffat@sun.com
To: Kais Belgaied <Kais.Belgaied@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        Jan.Pechanec@sun.com, crypto-discuss@opensolaris.org,
        openssl-interst@sun.com
Message-id: <4ADC219C.4070108@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM>
User-Agent: Thunderbird 2.0.0.22 (X11/20090818)
Status: RO
Content-Length: 2044

Kais Belgaied wrote:
> +0.75
> 
> 
> a couple of questions below
> 
>>
>> +     OpenSSL can access RSA keys in PKCS#11 keystores using the
>> +     following functions of the ENGINE API:
>> +
>> +       EVP_PKEY *ENGINE_load_private_key(ENGINE *e,
>> +                       const char *key_id, UI_METHOD *ui_method,
>> +               void *callback_data)
>> +
>> +       EVP_PKEY *ENGINE_load_public_key(ENGINE *e,
>> +                       const char *key_id, UI_METHOD *ui_method,
>> +               void *callback_data)
>>   
> 
> given the semantics described in the case, these functions will fail for 
> multiple reasons: bad argument, key not found,
> bad internal state (engine hasn't initialized or hasn't authenticated to 
> the token). Yet the return value
> can be either NULL: failure or Not NULL: a matching key was retrieved.
> It will be more helpful to give the app developers some info as to the 
> reason of failure, so that they
> know what to do when the load function returns NULL.
> 
> Possibly Missing:
> ----------------------
> 1. Need to mention somewhere that the caller of the load functions is 
> responsible for calling EVP_PKEY_free().
> 
> 2. since the private parts of the on-token keys are never read by the 
> engine, there is an implication on all OpenSSL
>  access routines, like  EVP_PKEY_copy_parameters(), EVP_PKEY_get1_RSA(), 
> etc. The'll all gonna fail when the
> pkey arg comes from a token.
> Rather than chasing the dozens of functions that use RSA private keys in 
> openssl, maybe it suffices to
> document that EVP_Decrypt() and EVP_PKEY_free() are the only routines 
> that can use an  RSA private  key by reference.

While these are all good points we (Solaris) don't own the documentation 
for these APIs and we didn't design them.  These are OpenSSL APIs that 
are documented in OpenSSL documentation and we don't modify those docs.

What you have said is true regardless of which OpenSSL ENGINE is in use 
and isn't unique to the Solaris provided "pkcs11" engine.

-- 
Darren J Moffat

From Nicolas.Williams@sun.com Mon Oct 19 08:17:32 2009
Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9JFHWva003983
	for <psarc-ext@sac.sfbay.sun.com>; Mon, 19 Oct 2009 08:17:32 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9JFHRoG009465;
	Mon, 19 Oct 2009 08:17:30 -0700 (PDT)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRR00D1BP55L700@nwk-avmta-2.sfbay.sun.com>; Mon,
 19 Oct 2009 08:17:29 -0700 (PDT)
Received: from binky.Central.Sun.COM ([129.153.128.104])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRR009GUP54FZ50@nwk-avmta-2.sfbay.sun.com>; Mon,
 19 Oct 2009 08:17:28 -0700 (PDT)
Received: from binky.Central.Sun.COM (localhost [127.0.0.1])
 by binky.Central.Sun.COM (8.14.3+Sun/8.14.3) with ESMTP id n9JF64PO003071;
 Mon, 19 Oct 2009 10:06:04 -0500 (CDT)
Received: (from nw141292@localhost)
 by binky.Central.Sun.COM (8.14.3+Sun/8.14.3/Submit) id n9JF63mK003070; Mon,
 19 Oct 2009 10:06:04 -0500 (CDT)
Date: Mon, 19 Oct 2009 10:06:03 -0500
From: Nicolas Williams <Nicolas.Williams@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
To: Darren J Moffat <darrenm@sac.sfbay.sun.com>
Cc: PSARC-ext@sun.com, Jan.Pechanec@sun.com, crypto-discuss@opensolaris.org,
        openssl-interst@sun.com
Message-id: <20091019150603.GA892@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT
Content-disposition: inline
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
X-Authentication-warning: binky.Central.Sun.COM: nw141292 set sender to
 Nicolas.Williams@sun.com using -f
User-Agent: Mutt/1.5.7i
Status: RO
Content-Length: 1364

On Tue, Oct 13, 2009 at 02:27:10AM -0700, Darren J Moffat wrote:
> 	4.1.2 PKCS#11 URI
> 
> 	The format of PKCS#11 URI was designed in Sun with possible future
> 	integration into the PKCS#11 standard in mind, and was discussed openly
> 	in the community mailing list. Its format follows. The ordering of
> 	attributes is NOT significant but every attribute can can be used at
> 	most once.
> 
> 	pkcs11:[token=<label>][;manuf=<label>][;serial=<label>][;model=<label>]
>     	       [;object=<label>][;objecttype=(public|private|cert)]
> 	       [;passphrasedialog=(builtin|exec:<file>)]

This could be extremely useful in general (for example, in the
configuration for software such as pam_pkcs11(5), and pam_krb5(5), with
PKINIT).

I think you should follow the procedures described in RFC4395 and
register this URI scheme, particularly since you call it a URI.  The
process is rather simple: fill out a form, send it to a specified
mailing list, answer questions from an expert reviewer, and you're done.

Aside: IMO we should eventually register the SMF/FMA FMRI (particularly
when we add remote SMF support), and even IPS pkg URI schemes, once they
are sufficiently stable.  URI scheme registration costs little, buys
us a modicum of protection from potential future conflicts, and helps
others understand our URI schemes, should they run into them.

Nico
-- 

From Kais.Belgaied@sun.com Tue Oct 20 11:09:38 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9KI9bF2022946
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 20 Oct 2009 11:09:37 -0700 (PDT)
Received: from nwk-avmta-2.sfbay.sun.com (nwk-avmta-2.SFBay.Sun.COM [129.145.155.6])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n9KI9Xbu015742;
	Tue, 20 Oct 2009 19:09:36 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-2.sfbay.sun.com by
 nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRT0070RRRZH100@nwk-avmta-2.sfbay.sun.com>; Tue,
 20 Oct 2009 11:09:35 -0700 (PDT)
Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63])
 by nwk-avmta-2.sfbay.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRT0031MRRWFQ70@nwk-avmta-2.sfbay.sun.com>; Tue,
 20 Oct 2009 11:09:32 -0700 (PDT)
Received: from [129.146.11.144]
 (sr1-jurassic-01.SFBay.Sun.COM [129.146.11.144])	by
 jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n9KI9VU5507478
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue,
 20 Oct 2009 11:09:32 -0700 (PDT)
Date: Tue, 20 Oct 2009 11:09:31 -0700
From: Kais Belgaied <Kais.Belgaied@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4ADC219C.4070108@Sun.COM>
To: Darren J Moffat <Darren.Moffat@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        Jan.Pechanec@sun.com, crypto-discuss@opensolaris.org,
        openssl-interst@sun.com
Message-id: <4ADDFCDB.8070101@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM> <4ADC219C.4070108@Sun.COM>
User-Agent: Thunderbird 2.0.0.21 (X11/20090311)
Status: RO
Content-Length: 534

On 10/19/09 01:21, Darren J Moffat wrote:
>
> While these are all good points we (Solaris) don't own the 
> documentation for these APIs and we didn't design them.  These are 
> OpenSSL APIs that are documented in OpenSSL documentation and we don't 
> modify those docs.

is there any doc that comes from Sun where we could capture these 
gotchas that a developer will encounter?

    Kais.

>
> What you have said is true regardless of which OpenSSL ENGINE is in 
> use and isn't unique to the Solaris provided "pkcs11" engine.
>




From Kais.Belgaied@sun.com Tue Oct 20 11:10:46 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9KIAiCY023076
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 20 Oct 2009 11:10:45 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n9KIAYJs016739;
	Tue, 20 Oct 2009 19:10:43 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRT00903RTUVX00@nwk-avmta-1.sfbay.Sun.COM>; Tue,
 20 Oct 2009 11:10:42 -0700 (PDT)
Received: from jurassic-x4600.sfbay.sun.com ([129.146.17.63])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRT004JDRTTK030@nwk-avmta-1.sfbay.Sun.COM>; Tue,
 20 Oct 2009 11:10:41 -0700 (PDT)
Received: from [129.146.11.144]
 (sr1-jurassic-01.SFBay.Sun.COM [129.146.11.144])	by
 jurassic-x4600.sfbay.sun.com (8.14.3+Sun/8.14.3) with ESMTP id n9KIAeX7507699
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue,
 20 Oct 2009 11:10:41 -0700 (PDT)
Date: Tue, 20 Oct 2009 11:10:40 -0700
From: Kais Belgaied <Kais.Belgaied@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <Pine.SOC.4.64.0910190758140.25074@rejewski>
To: Jan Pechanec <Jan.Pechanec@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        crypto-discuss@opensolaris.org, openssl-interst@sun.com
Message-id: <4ADDFD20.6070202@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM> <Pine.SOC.4.64.0910190758140.25074@rejewski>
User-Agent: Thunderbird 2.0.0.21 (X11/20090311)
Status: RO
Content-Length: 254


> 	I may add a note to our openssl(5) draft change that high level 
> API must be used for that, and can add an example of few such functions 
> so that a user can get the picture. Is that OK?
>   

sounds good.

+1.

    Kais
> 	thanks, Jan.
>  
>   


From Jan.Pechanec@sun.com Tue Oct 20 11:12:33 2009
Received: from sunmail2sca.sfbay.sun.com (sunmail2sca.SFBay.Sun.COM [129.145.155.234])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9KICW6e023211
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 20 Oct 2009 11:12:32 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail2sca.sfbay.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.4) with ESMTP id n9KICRcX022235;
	Tue, 20 Oct 2009 11:12:32 -0700 (PDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRT00D0DRWUFA00@brm-avmta-1.central.sun.com>; Tue,
 20 Oct 2009 12:12:30 -0600 (MDT)
Received: from gmp-eb-inf-2.sun.com ([192.18.6.24])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRT00CMFRWTDC30@brm-avmta-1.central.sun.com>; Tue,
 20 Oct 2009 12:12:30 -0600 (MDT)
Received: from fe-emea-09.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
 by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9KICSd8016138; Tue,
 20 Oct 2009 18:12:29 +0000 (GMT)
Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRT00800RJP9N00@fe-emea-09.sun.com>; Tue, 20 Oct 2009 19:12:08 +0100 (BST)
Received: from rejewski ([unknown] [10.18.138.121])
 by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KRT006GKRW7Z0C0@fe-emea-09.sun.com>;
 Tue, 20 Oct 2009 19:12:07 +0100 (BST)
Date: Tue, 20 Oct 2009 20:12:01 +0200 (CEST)
From: Jan Pechanec <Jan.Pechanec@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4ADDFCDB.8070101@Sun.COM>
Sender: Jan.Pechanec@sun.com
X-X-Sender: jp161948@rejewski
To: Kais Belgaied <Kais.Belgaied@sun.com>
Cc: Darren J Moffat <Darren.Moffat@sun.com>,
        Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        crypto-discuss@opensolaris.org, openssl-interst@sun.com
Message-id: <Pine.SOC.4.64.0910202011230.1171@rejewski>
MIME-version: 1.0
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM> <4ADC219C.4070108@Sun.COM>
 <4ADDFCDB.8070101@Sun.COM>
Status: RO
Content-Length: 554

On Tue, 20 Oct 2009, Kais Belgaied wrote:

> On 10/19/09 01:21, Darren J Moffat wrote:
>>
>> While these are all good points we (Solaris) don't own the documentation for
>> these APIs and we didn't design them.  These are OpenSSL APIs that are
>> documented in OpenSSL documentation and we don't modify those docs.
>
> is there any doc that comes from Sun where we could capture these gotchas that
> a developer will encounter?

	the only thing we have is openssl(5). Not to be confused with 
openssl(1) which is from the community. J.

-- 
Jan Pechanec

From Jan.Pechanec@sun.com Tue Oct 20 11:13:03 2009
Received: from sunmail4.singapore.sun.com (sunmail4.Singapore.Sun.COM [129.158.71.19])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9KID252023243
	for <psarc-ext@sac.sfbay.sun.com>; Tue, 20 Oct 2009 11:13:03 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by sunmail4.singapore.sun.com (8.13.4+Sun/8.13.3/ENSMAIL,v2.2) with ESMTP id n9KICuhJ000250;
	Wed, 21 Oct 2009 02:13:01 +0800 (SGT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRT00D0PRXOFW00@brm-avmta-1.central.sun.com>; Tue,
 20 Oct 2009 12:13:00 -0600 (MDT)
Received: from gmp-eb-inf-2.sun.com ([192.18.6.24])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRT00CRERXNDA30@brm-avmta-1.central.sun.com>; Tue,
 20 Oct 2009 12:13:00 -0600 (MDT)
Received: from fe-emea-10.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
 by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9KICxW4016175; Tue,
 20 Oct 2009 18:12:59 +0000 (GMT)
Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRT00D00RSQ0H00@fe-emea-10.sun.com>; Tue, 20 Oct 2009 19:12:35 +0100 (BST)
Received: from rejewski ([unknown] [10.18.138.121])
 by fe-emea-10.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KRT00KLJRWZE460@fe-emea-10.sun.com>;
 Tue, 20 Oct 2009 19:12:35 +0100 (BST)
Date: Tue, 20 Oct 2009 20:12:29 +0200 (CEST)
From: Jan Pechanec <Jan.Pechanec@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4ADDFD20.6070202@Sun.COM>
Sender: Jan.Pechanec@sun.com
X-X-Sender: jp161948@rejewski
To: Kais Belgaied <Kais.Belgaied@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        crypto-discuss@opensolaris.org, openssl-interst@sun.com
Message-id: <Pine.SOC.4.64.0910202012150.1171@rejewski>
MIME-version: 1.0
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM> <Pine.SOC.4.64.0910190758140.25074@rejewski>
 <4ADDFD20.6070202@Sun.COM>
Status: RO
Content-Length: 325

On Tue, 20 Oct 2009, Kais Belgaied wrote:

>
>> 	I may add a note to our openssl(5) draft change that high level API
>> must be used for that, and can add an example of few such functions so that a
>> user can get the picture. Is that OK?
>>  
>
> sounds good.

	I'll update the man page draft tomorrow. J.

-- 
Jan Pechanec

From Jan.Pechanec@sun.com Wed Oct 21 02:32:06 2009
Received: from sunmail5.uk.sun.com (sunmail5.UK.Sun.COM [129.156.85.165])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9L9W5k8023063
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 21 Oct 2009 02:32:05 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by sunmail5.uk.sun.com (8.13.8+Sun/8.13.8/ENSMAIL,v2.2) with ESMTP id n9L9Vq6X019876;
	Wed, 21 Oct 2009 10:32:04 +0100 (BST)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRU00K0JYHDA700@nwk-avmta-1.sfbay.Sun.COM>; Wed,
 21 Oct 2009 02:32:01 -0700 (PDT)
Received: from gmp-eb-inf-1.sun.com ([192.18.6.21])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRU005M0YHB3Y90@nwk-avmta-1.sfbay.Sun.COM>; Wed,
 21 Oct 2009 02:32:00 -0700 (PDT)
Received: from fe-emea-09.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
	by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9L9VvuL027470; Wed,
 21 Oct 2009 09:31:58 +0000 (GMT)
Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRU00D00WBHV700@fe-emea-09.sun.com>; Wed, 21 Oct 2009 10:31:51 +0100 (BST)
Received: from rejewski ([unknown] [10.18.138.121])
 by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KRU004ESYH2F050@fe-emea-09.sun.com>;
 Wed, 21 Oct 2009 10:31:50 +0100 (BST)
Date: Wed, 21 Oct 2009 11:31:45 +0200 (CEST)
From: Jan Pechanec <Jan.Pechanec@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
Sender: Jan.Pechanec@sun.com
X-X-Sender: jp161948@rejewski
To: Darren J Moffat <darrenm@sac.sfbay.sun.com>
Cc: PSARC-ext@sun.com, crypto-discuss@opensolaris.org,
        openssl-interest@sun.com
Message-id: <Pine.SOC.4.64.0910211128180.11798@rejewski>
MIME-version: 1.0
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
Status: RO
Content-Length: 2229

On Tue, 13 Oct 2009, Darren J Moffat wrote:

	there is one nit. I did not understand properly how mlock(3C) 
worked. It does not prevent the page to end up in core dumps. So, 
"mlocked-memory" for OPENSSL_PKCS11_PIN_CACHING_POLICY will only prevent 
it to end up in a swap.

	the paragraph below explicitly mentions that mlock(3C) will help 
in those core dump situations as well. Should I fix the fasttrack?

	J.

>	4.1.4 PIN Caching Policy
>
>	The PKCS#11 engine does NOT cache the PIN in memory by default. We
>	assume that if the application does not fork then caching the PIN by
>	default would not be only useless but also potentially dangerous (PIN
>	could end up in swap or system and process core dumps). However, we
>	provide 2 options for PIN caching so that the PIN can be used in the
>	child to relogin to the token. The engine honors a special environment
>	variable:
>
>	OPENSSL_PKCS11_PIN_CACHING_POLICY
>
>	which can have "none", "mlocked-memory", and "memory" values.
>
>	The "none" value is the default where the PIN is forgotten after the
>	login. "memory" will keep the PIN in the memory, and "mlocked-memory"
>	will keep the PIN in a locked page via mlock(3C). Note that mlock()
>	requires the PRIV_PROC_LOCK_MEMORY privilege which is not in the default
>	user set. An admin must grant the application or the user the relevant
>	privilege if the "mlocked-memory" option is to be used. With that option
>	and without the privilege the load function will fail on loading the 1st
>	private key if the token requires the PIN - it will NOT resort to use
>	the "memory" option. Also, if the PIN is required then any incorrect
>	value of OPENSSL_PKCS11_PIN_CACHING_POLICY will result in a failure in
>	the load function. If the PIN is not required (eg. providing a PIN in
>	URIs for public keys only) the PIN will not be used and will be erased
>	from memory no matter how OPENSSL_PKCS11_PIN_CACHING_POLICY is set.
>
>	While we could use the ENGINE control API to provide such information to
>	the engine the use of an enviroment variable seems to be more apropriate
>	here since it will provide more options to applications using RSA keys
>	by reference without any need for code modifications.

-- 
Jan Pechanec

From Darren.Moffat@sun.com Wed Oct 21 02:55:17 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9L9tHDq023545
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 21 Oct 2009 02:55:17 -0700 (PDT)
Received: from brm-avmta-1.central.sun.com (brm-avmta-1.Central.Sun.COM [129.147.4.11])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n9L9tEnk058217;
	Wed, 21 Oct 2009 03:55:16 -0600 (MDT)
Received: from pmxchannel-daemon.brm-avmta-1.central.sun.com by
 brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRU00M05ZK34L00@brm-avmta-1.central.sun.com>; Wed,
 21 Oct 2009 03:55:15 -0600 (MDT)
Received: from gmp-eb-inf-1.sun.com ([192.18.6.21])
 by brm-avmta-1.central.sun.com
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRU008UPZK2QJ60@brm-avmta-1.central.sun.com>; Wed,
 21 Oct 2009 03:55:15 -0600 (MDT)
Received: from fe-emea-10.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
	by gmp-eb-inf-1.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9L9tDKp001852; Wed,
 21 Oct 2009 09:55:14 +0000 (GMT)
Received: from conversion-daemon.fe-emea-10.sun.com by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRU00600ZINVC00@fe-emea-10.sun.com>; Wed, 21 Oct 2009 10:55:07 +0100 (BST)
Received: from [192.168.1.105]
 (cpc2-rdng20-2-0-cust917.15-3.cable.virginmedia.com [86.28.167.150])
 by fe-emea-10.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 with ESMTPSA id <0KRU0045IZJR9V00@fe-emea-10.sun.com>; Wed,
 21 Oct 2009 10:55:04 +0100 (BST)
Date: Wed, 21 Oct 2009 10:55:03 +0100
From: Darren J Moffat <Darren.Moffat@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <Pine.SOC.4.64.0910211128180.11798@rejewski>
Sender: Darren.Moffat@sun.com
To: Jan Pechanec <Jan.Pechanec@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        crypto-discuss@opensolaris.org, openssl-interest@sun.com
Message-id: <4ADEDA77.3010007@Sun.COM>
MIME-version: 1.0
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <Pine.SOC.4.64.0910211128180.11798@rejewski>
User-Agent: Thunderbird 2.0.0.23 (X11/20090929)
Status: RO
Content-Length: 601

Jan Pechanec wrote:
> On Tue, 13 Oct 2009, Darren J Moffat wrote:
> 
> 	there is one nit. I did not understand properly how mlock(3C) 
> worked. It does not prevent the page to end up in core dumps. So, 
> "mlocked-memory" for OPENSSL_PKCS11_PIN_CACHING_POLICY will only prevent 
> it to end up in a swap.
> 
> 	the paragraph below explicitly mentions that mlock(3C) will help 
> in those core dump situations as well. Should I fix the fasttrack?

Sending the email like you did is sufficient in my opinion but if you 
give me an updated spec I will put it in the case directory.

-- 
Darren J Moffat

From Jan.Pechanec@sun.com Wed Oct 21 06:39:00 2009
Received: from newsunmail1brm.central.sun.com (newsunmail1brm.Central.Sun.COM [129.147.62.245])
	by sac.sfbay.sun.com (8.13.8+Sun/8.13.8) with ESMTP id n9LDd0rv026437
	for <psarc-ext@sac.sfbay.sun.com>; Wed, 21 Oct 2009 06:39:00 -0700 (PDT)
Received: from nwk-avmta-1.SFBay.Sun.COM (nwk-avmta-1.SFBay.Sun.COM [129.146.11.74])
	by newsunmail1brm.central.sun.com (8.13.7+Sun/8.13.7/ENSMAIL,v2.2) with ESMTP id n9LDckV3043189;
	Wed, 21 Oct 2009 07:38:59 -0600 (MDT)
Received: from pmxchannel-daemon.nwk-avmta-1.sfbay.Sun.COM by
 nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 id <0KRV005959WY2D00@nwk-avmta-1.sfbay.Sun.COM>; Wed,
 21 Oct 2009 06:38:58 -0700 (PDT)
Received: from gmp-eb-inf-2.sun.com ([192.18.6.24])
 by nwk-avmta-1.sfbay.Sun.COM
 (Sun Java System Messaging Server 6.2-3.04 (built Jul 15 2005))
 with ESMTP id <0KRV004FR9WOKQ10@nwk-avmta-1.sfbay.Sun.COM>; Wed,
 21 Oct 2009 06:38:49 -0700 (PDT)
Received: from fe-emea-09.sun.com
 (gmp-eb-lb-1-fe1.eu.sun.com [192.18.6.7] (may be forged))
	by gmp-eb-inf-2.sun.com (8.13.7+Sun/8.12.9) with ESMTP id n9LDclXM019744; Wed,
 21 Oct 2009 13:38:48 +0000 (GMT)
Received: from conversion-daemon.fe-emea-09.sun.com by fe-emea-09.sun.com
 (Sun Java(tm) System Messaging Server 7u2-7.04 64bit (built Jul  2 2009))
 id <0KRV009008L4NZ00@fe-emea-09.sun.com>; Wed, 21 Oct 2009 14:38:26 +0100 (BST)
Received: from rejewski ([unknown] [10.18.138.121])
 by fe-emea-09.sun.com (Sun Java(tm) System Messaging Server 7u2-7.04 64bit
 (built Jul  2 2009)) with ESMTPSA id <0KRV00FEO9VTHR90@fe-emea-09.sun.com>;
 Wed, 21 Oct 2009 14:38:17 +0100 (BST)
Date: Wed, 21 Oct 2009 15:38:10 +0200 (CEST)
From: Jan Pechanec <Jan.Pechanec@sun.com>
Subject: Re: OpenSSL RSA keys by reference in PKCS#11 keystores through the
 PKCS11 engine [PSARC/2009/555 FastTrack timeout 10/20/2009]
In-reply-to: <4ADDFD20.6070202@Sun.COM>
Sender: Jan.Pechanec@sun.com
X-X-Sender: jp161948@rejewski
To: Kais Belgaied <Kais.Belgaied@sun.com>
Cc: Darren J Moffat <darrenm@sac.sfbay.sun.com>, PSARC-ext@sun.com,
        crypto-discuss@opensolaris.org, openssl-interest@sun.com
Message-id: <Pine.SOC.4.64.0910211534470.11798@rejewski>
MIME-version: 1.0
Content-type: TEXT/PLAIN; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT
X-PMX-Version: 5.4.1.325704
References: <200910130927.n9D9RA4e022835@sac.sfbay.sun.com>
 <4AD90A15.9080305@Sun.COM> <Pine.SOC.4.64.0910190758140.25074@rejewski>
 <4ADDFD20.6070202@Sun.COM>
Status: RO
Content-Length: 720

On Tue, 20 Oct 2009, Kais Belgaied wrote:

>
>> 	I may add a note to our openssl(5) draft change that high level API
>> must be used for that, and can add an example of few such functions so that a
>> user can get the picture. Is that OK?
>>  
>
> sounds good.

	hi Kais, I'll add this at the end of the section. Obviously, 
precise wording might be changed by the doc writer:

     Note that in order to use the RSA keys by reference, high
     level API functions must be used, like RSA_public_decrypt(),
     EVP_PKEY_set1_RSA(), or EVP_SignInit(). Low level functions
     might go around the engine and thus fail to make use of the
     feature.

	J.
>
> +1.
>
>   Kais
>> 	thanks, Jan.
>>   
>
>

-- 
Jan Pechanec