Oct 6 ’14

Using Linux on System z Hardware Cryptography With the PKCS#11 Cryoptography Stack

by Reinhard Buendgen in Enterprise Tech Journal

Recent revelations in the areas of security and privacy have caught the attention of a wide audience and certainly have changed the attitude of many companies. But these revelations come with one piece of reassurance: Cryptography works. So whenever secrets are to be shared securely or privacy must be preserved, the best thing to do is encrypt all sensitive information. And as Google and others have experienced, this advice isn’t restricted to communication over the Internet; it extends to communication within intranets. Cryptography methods are highly complex. Therefore, using standard application programming interfaces (APIs), referring to well-known and trusted implementations of those methods, is a best practice to avoid security flaws when writing software that needs cryptography.

The Public Key Cryptographic Standard #11 (PKCS#11) is a popular cryptographic standard meant to support cryptographic hardware using a standard API. PKCS#11 can support so-called hardware “tokens” that may be cryptographic accelerators or hardware security modules (HSMs). The goal of accelerators is to improve the performance of cryptographic operations either by employing special-purpose circuits or by offloading cryptographic operations to an additional processor or adapter card. Accelerators don’t enhance the security of a system, as they use clear text keys (aka clear key) that can be observed by anybody who has access to the system memory. On the other hand, HSMs never reveal keys in clear text outside the tamper-proof HSM hardware. Therefore, any key that leaves an HSM must be wrapped (i.e., encrypted) and is therefore called a secure key. IBM System z provides:

Cryptographic accelerators such as the CP Assist for Cryptographic Functions (CPACF) instructions on the CPU and the CryptoExpress Adapter in accelerator mode
HSMs such as the CryptoExpress adapters in either Common Cryptographic Architecture (CCA) or Enterprise PKCS#11 coprocessor mode.
The PKCS#11 standard is largely independent of specific cryptographic hardware yet it allows programs to deal with many hardware-specific implementations. It supports the use of multiple, possibly different tokens.

Given the popularity of PKCS#11, many software products that support encryption provide plug-in mechanisms that if configured will redirect cryptographic functions to a PKCS#11 library. For example, IBM middleware such as the IBM WebSphere Application Server (WAS) and the IBM HTTP Server, including the IBM internal crypto library GSKIT, can be configured to use a PKCS#11 library. Open source software such as the mod_nss plugin for Apache can also call to a PKCS#11 library. In addition, some libraries support plug-ins for PKCS#11. Foremost, the Java Cryptographic Architecture supports PKCS#11 providers (IBMPKCS11Impl for IBM Java and SunPKCS11 for Oracle Java).

PKCS#11 isn’t used only to implement applications requiring specific cryptographic support but also to enable existing software to use cryptographic hardware.

The PKCS#11 standard was published by RSA and has been transferred to the Organization for the Advancement of Structured Information Standards (OASIS). It defines a C/C++ API that’s called cryptoki. The current version of the standard is 2.2 (which includes three amendments). While version 2.3 remains in draft status, OASIS is reviewing version 2.4, which greatly extends the set of cryptographic mechanisms that will be supported.

Here we provide an overview of the concepts of the PKCS#11 standard as well as a description of openCryptoki, an open source implementation of PKCS#11 that’s available for major Linux distributions such as Red Hat Enterprise Server Linux (RHEL) and SUSE Linux Enterprise Server (SLES). In addition, we’ll present two openCryptoki tokens that exploit System z cryptographic hardware in Linux on System z.

PKCS#11 Overview
Since this section focuses on the PKCS#11 standard, it’s general and isn’t dependent on System z or Linux. The PKCS#11 concepts mostly come in pairs: slots and tokens, roles and sessions, functions and mechanisms, and objects and keys.

Slots and Tokens
It seems the use of smart cards has provided the model behind the hardware plug-in mechanism of PKCS#11. In the same way a smart card is inserted into a smart card reader, a PKCS#11 token is inserted into a PKCS#11 slot where a slot is identified by its ID (a small number) and a token is a piece of library code that knows how to interface with the cryptographic hardware. Clearly, there’s no requirement for the token to use any hardware, and accordingly, there are so-called soft tokens that represent a pure software library accessible via the PKCS#11 API.

The PKCS#11 API provides a set of slot and token management functions:

C_GetSlotList(), C_GetSlotInfo() obtains a list of available slots and information on each slot (e.g., whether a token is present or the token represents a removable device).
C_WaitFor Slot Event() waits for a token to be inserted or removed.
C_InitToken(), C_GetTokenInfo() initializes an inserted token and provides information on the token (e.g., whether a login is required to use the token, a count of failed logins, information on whether the token has a random number generator, etc.).
C_InitPIN(), C_SetPIN() manages the PIN used to protect a token from unauthorized access.
Figure 1 shows a cryptoki with different tokens plugged into its slots. The first token uses a cryptographic adapter, the second token uses a smart card, the third token is a self-contained cryptographic library and the forth token interfaces to another cryptographic library. The application uses the second token for signing and the fourth token for decryption.


Roles and Sessions
PKCS#11 knows two different roles per token. Each role can authenticate itself by a PIN specific to that role and a token. The first role is that of the security officer (SO). The SO initializes the token and can set the PIN of the user. The second role is that of the (normal) user. The user can log-in to sessions, create and access private objects and perform cryptographic operations (which will be explained later).

A session is a token-specific context for one or more cryptographic operations. It maintains the intermediate state of multipart functions such as encryption of a message that’s worked on one network packet at a time. Roughly speaking, within one session, only one cryptographic operation can be processed at a time, but a program may open multiple sessions concurrently.

There are different types of sessions: Each session is either a read-only session or a readwrite session and each session is either a public session or user session. Read-only sessions may not create or modify objects. A public session can only access public objects whereas a user session can access the user’s private objects. All sessions of a token become user sessions after a log-in to one of the sessions of that token.

The PKCS#11 API provides session functions such as:

C_OpenSession(), C_CloseSession() to open and close a session
C_GetSessionInfo() to provide information on the type of a session
C_GetOperationState() and C_SetOperationState() to allow checkpoint and restart of a multipart cryptographic operation
C_Login(), Logout() to toggle sessions from being public to being user sessions and vice versa. The user PIN is required to log-in.
Generic Cryptographic Functions and Mechanisms
The PKCS#11 API defines a small set of generic cryptographic functions to do the following tasks:

Encrypt and decrypt messages
Computing digests (aka hashes) of messages
Sign messages and verify signatures
Generate symmetric keys or asymmetric key pairs and deriving keys
Wrap and unwrap keys
Generate random numbers.
With the exception of generating random numbers, these generic cryptographic functions accept a mechanism parameter that defines the specific instance of that function. Figure 2 shows an example of a specific call to a generic cryptographic function where the generic encryption function takes an AES_CBC mechanism as argument to encrypt a message with AES encryption in the cipher block chaining (CBC) mode of operation using a key (and an initialization vector not shown in the figure). Each cryptographic function is executed in the context of a session.


Most cryptographic functions must be initialized by calling an initialization function that takes a session parameter, a mechanism parameter and function-specific parameters such as keys. For cryptographic function xyz, the initialization function is called C_XyzInit(). Once a function is initialized, the actual function invocation can take place either as a single-part function of the form C_Xyz() or as a multipart function where one or more calls to a function of the form C_XyzUpdate() are finalized by a call to C_XyzFinal(). Figure 3 lists the set of generic cryptographic functions together with their variants to initialize, update and finalize the operation. Here the occurrence of x in the line of Verify and the column headed by C_FktUpdate denotes the function C_VerifyUpdate() as belonging to the PKCS#11 API.


It’s token-specific which functions are supported by a token. The PKCS#11 API provides the function C_GetFunctionList() to obtain the function available with a specific token.

A mechanism describes a specific set of cryptographic operations. For example, CKM_AES_CBC refers to AES encryption with the CBC mode of operation. A mechanism may apply to one or more cryptographic functions. For example, the CKM_AES_CBC mechanism may be used to define encryption, decryption, wrapping and unwrapping functions whereas CKM_ECDSA_KEY_PAIR_GEN, the mechanism to generate keys for the elliptic curve DSA signatures, only supports the key (pair) generation function.

The list of mechanisms supported depends on the tokens and can be queried using the C_GetMechanismList() function. Each mechanism has token-specific attributes, such as the set of supported functions, minimal and maximal supported key sizes or a hardware support flag. These attributes are token-specific and can be queried with C_GetMechanismInfo(). Some mechanisms have parameters such as CKM_AES_CBC, which define the initialization vector (IV) required by the CBC mode of operation.

Figure 4 shows a list of mechanisms and their supported functions. An x in the column headed with E/D means the mechanism to the left of the x supports encryption and decryption. Similarly, S/V stands for signing and verification; SR/VR stands for message signing and verification together with message recovery; Dig stands for message digests; Gen stands for key or key pair generation; U/W stands for key wrapping and unwrapping; and Der stands for key derivation. This is an excerpt of a similar list in the PKCS#11 standard, which for version 2.2 lists about 300 possible mechanisms.


Objects and Keys
PKCS#11 objects belong to different orthogonal classes, depending on their life span, access restrictions and modifiability:

Session objects exist for the duration of a session, whereas token objects are associated with a token and not with any running code.
Private objects can be accessed only by the PKCS#11 user if he has logged into the token, whereas public objects can always be accessed by both user and SO.
Read-only objects can’t be modified, whereas read-write objects can.
In addition, each object has a set of attributes. Most notably the attribute class (CKA_CLASS), which determines which further attributes are associated with an object. Other typical attributes contain the value of an object or determine whether an object is a token object.

The PKCS#11 API provides a set of functions to manage objects such as C_CreateObjects(), C_CopyObject(), C_DestroyObject(), C_GetObjectSize(), C_GetAttributeValue(), C_SetAttributeValue() and C_FindObjects().

Note, token objects must be stored in a token-specific object store.

The most important object classes are those that implement keys: private keys (CKO_ PRIVATE_KEY), public keys (CKO_PUBLIC_ KEY) and secret keys (CKO_SECRET_KEY). Private and public keys are the members of an asymmetric key pair whereas secret keys are symmetric keys or MAC keys.

There are many key-specific attributes. For example, the Boolean attribute CKA_WRAP denotes whether a key may be used to wrap another key. The Boolean attribute CKA_SENSITIVE is only applicable for private and secret keys and if the value of CKA_SENSITIVE is set to true, the value of the key may never be revealed in clear text. CKA_MODULUS is an attribute specific to RSA keys. Not all tokens support all key types with all possible attributes.

Object management functions to create keys are C_GenerateKey(), C_GenerateKeyPair() and C_DeriveKey(). To import a key not generated by one of these three functions, C_CreateObject() is used with all key-specific attributes defined. Alternatively, a wrapped key can be imported with C_Unwrap() where a wrapped key is a standard representation of a key specific to the key type (e.g., a byte array for secret keys or a BER encoding for other key types) that’s encrypted by a wrapping key.

PKCS#11 also defines objects for certificates. However, no functions to operate on certificates are part of the PKCS#11 API.

Additional PKCS#11 Concepts
Each program using PKCS#11 must call C_Initialize() once before calling any other PKCS#11 function. C_Initialize() takes as arguments information on the threading capabilities required and optionally call-back functions (i.e., MUTEX functions) to be used for the synchronization of multiple threads. The end of all PKCS#11 processing should be indicated with C_Finalize().

For many hardware tokens, a PIN entry is required at the physical token (e.g., at a smart card reader in an ATM). If the CKF_PROTECTED_AUTHENICATION_ PATH flag of the PKCS#11 token is set by the program, the C_Login() function takes a NULL_PTR as the PIN argument and blocks until the PIN has been entered at the hardware token.

Putting It All Together
Figure 5 shows a skeleton of a typical program using PKCS#11 where within a session one or more cryptographic operations may be performed. For example, encrypting a message spread over multiple buffers could look like Figure 6.



OpenCryptoki is an open source implementation of PKCS#11 version 2.2 in the C programming language meant to run on the Linux operating system. The project is maintained by the IBM Linux Technology Center and hosted on SourceForge (http://sourceforge.net/projects/opencryptoki). The latest versions are version available with RHEL 6 and SLES 11 Linux distributions, and version 3.1, which hasn’t yet been picked up by any distribution.

OpenCryptoki supports 32 slots with up to 32 tokens. Multithreading is supported with locking from the native operating system. All processes calling openCryptoki must be members of the UNIX group pkcs11.

OpenCryptoki comes with a set of tokens; some are platform-independent and some are specific to Linux on System z:

Soft token is a platform-independent token providing clear key cryptographic functions using a pure software implementation from openssl.
ICA token is a Linux on System z-specific token for clear key cryptography exploiting symmetric crypto algorithms and hashes provided by CPACF and possibly RSA algorithms provided by either CryptoExpress accelerators or CCA coprocessors.
CCA token is a Linux on System z-specific token for secure key cryptography calling the CCA library, which exploits CCA coprocessors.
ICFS token is a platform-independent token that calls services from a remote cryptography server hosted on z/OS.
TPM token is a token for platforms that support a Trusted Platform Module (TPM).
Let’s examine the ICA and CCA tokens.

OpenCryptoki Components
The openCryptoki package consists of one library for the generic API (libopencyptoki.so) and one (so-called stdll) for each token. In addition, there’s a configuration tool called pkcsconf and a slot manager daemon called pkcsslotd that maintain a shared memory region that’s used to coordinate multiple tokens. Running this daemon is a prerequisite to running a program linking to openCryptoki. For openCryptoki 2.4.x, the pkcs11-startup script generates a configuration file in /var/libopencryptoki/pk_config_data. For openCryptoki 3.x, the configuration file is customizable and located in /etc/opencryptoki/opencryptoki.conf.

The directory /var/lib/opencryptoki contains a subdirectory for each token containing:

NVTOK.DAT, which is configuration data and state
MK_SO, which is an encrypted master key to encrypt SO’s private objects
MK_USER, which is an encrypted master key to encrypt the user’s private objects
TOK_OBJ, which is a directory for token objects (token key store). Each private object is represented by an encrypted file.
The openCryptoki package comes with UNIX manual pages that describe the use of the tools and the format of the configuration files.

Configuring OpenCryptoki
Configuring openCryptoki starts with installing the openCryptoki RPM using the installation tool specific to the Linux distribution (e.g., anaconda/yum for RHEL or yast/zypper for SLES). Depending on the tokens to be used, further libraries will need to be installed and crypto adapters must be enabled.

Next, the opencryptoki configuration file must be generated with the pkcs11_startup script. This step is needed only for openCryptoki version 2.4.x. The pkcs11_startup script also generates the UNIX group pkcs11. Then the pkcsslotd daemon must be started. Finally, each token to be used must be configured, which means a token label must be set and the SO PIN must be changed from its default (“87654321”). Then the SO must set an initial PKCS#11 user PIN and the PKCS#11 user must change that initial user PIN.

Following is an example of how to install openCryptoki and configure the ICA token. All outputs shown are approximate and may differ from system to system.

1. Check whether libica is installed and install it if needed:

Check: rpm -qa | grep -i libica
Install for RHEL: yum install libica
Install for SLES: zypper install libica

2. Load the device driver for the CryptoExpress adapters if such an adapter is available: # modprobe z90crypt

3. Install openCryptoki with yum or zypper, respectively. Once openCryptoki has been installed, it must be initialized:

# pkcs11_startup
# pkcsslotd

Note that pkcsslotd must be started after every boot. Therefore, it may be useful to have an init script that starts that daemon.


To check which tokens are available, we call pkcsconf with the -t option (see Figure 7). The ICA token has the Model value IBM ICA. Thus, in our example, the ICA token is in slot 0 (Token #0). First, we set the label (option -I) of the ICA token to a value of our choice (here icatoken):

# pkcsconf -I -c0.
Enter the SO PIN: ********
Enter a unique token label: icatoken

Now we can change the SO PIN (option -P) of the token in slot 0 (option -c0):

# pkcsconf -P -c0
Enter the SO PIN: ********
Enter the new SO PIN: ********
Re-enter the new SO PIN: ********

Let the SO set PKCS#11 User PIN (option -u) of ICA token:

# pkcsconf -u -c0
Enter the SO PIN: ********
Enter the new user PIN: ********
Re-enter the new user PIN: ********

And eventually let the PKCS#11 User change its PIN (option -p) of ICA token:

# pkcsconf -p -c0
Enter the user PIN: ********
Enter the new user PIN: ********
Re-enter the new user PIN: ********

The command in Figure 8 is used to verify the configuration of the ICA token. If the Flags attribute shows USER_PIN_INITIALIZED and TOKEN_INITIALIZED, then the token is ready for use.


The ICA Token
The ICA token provides access to clear key cryptographic algorithms implemented by the CPACF feature of the CPU and the CryptoExpress accelerator or CCA coprocessor. To use the ICA token, the CPACF feature (feature code LIC 3863) must be installed on the system and the libica library must be installed in Linux. If functions from the CryptoExpress adapters are to be used, the z90crypt device driver must be loaded into the Linux kernel.

In the list of tokens displayed by pkcsconf –t, the ICA token is the one with the attribute Model IBM ICA. The token directory for the ICA token is /var/lib/opencryptoki/lite; hence, the ICA token objects are stored in /var/lib/opencryptoki/lite/TOK_OBJ.

The ICA token exploits System z cryptographic hardware as follows:

SHA-1, SHA-256/386/512 via CPACF
DES, 3DES, AES128/192/256 using ECB, CBC, and CTR modes of operation via CPACF starting with version 3.0: 3DES, AES128/192/256 using OFB, CFB, CBCMAC modes of operation via CPACF
RSA with 1024-4096 bit keys using CryptoExpress adapters or a software fall back (openssl)
(Pseudo) random numbers using CPACF and CryptoExpress CCA coprocessor if available.
Figure 9 shows the full list of mechanisms supported by the ICA token. This information can also be retrieved using the -m option of pkcsconf.


The CCA Token
The CCA token provides secure key cryptography using the CryptoExpress CCA coprocessor. It requires the libcsulcca library to be installed and the z90crypt device driver to be loaded into the kernel. The CCA RPM containing the CCA library libcsulcca can be downloaded for free from http://www-03.ibm.com/security/ cryptocards/pciecc/ordersoftware.shtml.

In the list of tokens displayed by pkcsconf –t, the CCA token is the one with the value “Model: IBM CCA TOKEN.” The token directory for the CCA token is /var/lib/opencryptoki/ccatok; therefore, the CCA token objects are stored in /var/lib/ opencryptoki/ccatok/TOK_OBJ.

The CCA token exploits System z cryptographic hardware as follows:

DES, 3DES, AES128/192/256 using ECB and CBC via a CryptoExpress CCA coprocessor
RSA with 1024-4096 bit keys via a CryptoExpress CCA coprocessor
ECDSA via a CryptoExpress 3 (or later) CCA coprocessor.
The full list of mechanisms supported by the CCA token is shown in Figure 4. This information can also be retrieved using the -m option of pkcsconf.

All keys generated by the CCA token are CCA secure keys. To work with keys not generated by the CCA token, such keys must be imported. RSA key pairs can be imported with calls to C_CreateObject() for each key component. The imported private RSA key will then be wrapped with the CCA master key. Note, only private RSA keys in CRT format can be imported.

PKCS#11 and Other Software
PKCS#11 and thus openCryptoki can be exploited with the Java Cryptography Architecture (JCA) using a PKCS#11 provider such as the IBMPKCS11Impl for IBM Java. Using openCryptoki via Java was described in detail in the article “Using Crypto Hardware With Java in Linux on System z,” which is available at http://esmpubs.com/nsnre. For IBM Java and the IBMPKCS11Impl, the set of supported cryptographic methods is described as a table mapping each Java method to PKCS#11 mechanisms (e.g., on the IBM infocenter under Security Reference for IBM SDK, Java Technology Edition, Version 7 > Security Reference for IBM SDK, Java Technology Edition, Version 7 > IBMPKCS11Impl Provider > IBM PKCS 11 Implementation Provider for Java available at http://ibm.biz/ BdRexT). To find out which Java method is supported by cryptographic hardware in a specific configuration, this table must be related to the mechanism list supported by the openCryptoki token to be used (e.g., Figure 9 for the ICA token).

Some standard software supports being configured to call a PKCS#11 library. Examples of such software are WAS, the IBM HTTP Server or Apache with the mod_nss security module. These configurations typically ask for a library path, a slot or token id and sometimes a user PIN. The paper “IBM WebSphere Application Server Version 8 for Linux on IBM System z SSL Setup and Performance Study” (available at http://pic.dhe.ibm.com/ infocenter/lnxinfo/v3r0m0/topic/liaag/ l0wascry00_2013.htm) describes the configuration needed to configure WAS to use System z cryptographic hardware via openCryptoki and shows that considerable performance improvements can be observed due to the usage of cryptographic hardware.

Note, to ensure cryptographic hardware is used in applications using SSL/TLS, cipher suites that contain components supported by cryptographic hardware should be selected and given priority.

PKCS#11 is a widely used standard that allows access to functions from cryptographic hardware devices. OpenCryptoki is an open source implementation of PKCS#11 available with SLES and RHEL distributions. Many System z hardware crypto functions from CryptoExpress adapters and CPACF can be accessed via openCryptoki: The openCryptoki ICA token provides access to clear key cryptography and the openCryptoki CCA token provides access to secure key cryptography. Java and many software products provide plug-in options for a PKCS#11 library.


“Using Crypto Hardware With Java in Linux on System z,” Reinhard Buendgen and Peter Spera, Enterprise Tech Journal, March/April 2013: http://esmpubs.com/nsnre
“IBM WebSphere Application Server Version 8 for Linux on IBM System z SSL Setup and Performance”: http://pic.dhe.ibm. com/infocenter/lnxinfo/v3r0m0/topic/liaag/ l0wascry00_2013.htm.