libfcgi2

TLS with Schannel


INTRODUCTION

  • 32 bit windows DLL SSPI Schannel examples are available for download along with an example SMTPS application.

Secure network communication has become increasingly important. One of the most widely used protocols is Secure Sockets Layer (SSL) which is gradually being replaced by its successor, Transport Layer Security (TLS). These protocols ensure:

  • confidentiality (the data is kept secret from the third party that is not involved in the data exchange)
  • message integrity (the message received is the message sent)
  • endpoint authentication (one of the communication endpoints or both are verifiably known)
For Windows developers, Microsoft has provided a version of Generic Security Service Application Program Interface (GSS-API) , called Security Support Provider Interface (SSPI). SSPI allows developers to implement SSL/TLS without needing to know the details of SSL. The C libraries and header files are freely available from the the platform SDK (any one) and are included with my Visual Studio project available for download.



This article provides everything you need to implement a secure connection. You will not need to buy and install any certificates, or ship a 3rd party dll. I have written procedural code in C and BASIC so that the flow can easily be followed, although an OOP implementation would probably be more suitable for production code.


OVERVIEW

Microsoft operating systems include Schannel, the Microsoft implementation of SSL. SChannel is an integral part of the operating system. SSPI is a set of generic functions that can be used to access a specific security provider like Schannel, to obtain an authenticated connection. These calls do not require extensive knowledge of the security protocol's details. SChannel calls follow GSS API standards, unlike OpenSLL for example.

Implementing this API is not trivial however as you can see from the example. Although these 2000 lines of code hide the gory details, creating a secure connection is not simple to implement.

SSPI allows an application to call one of several security providers to obtain an authenticated connection. In this case,  we will call SSPI routines directly as a socket-based application. The routines use request/response messages to carry data over the secure connection.

When a dynamic-link library (DLL) loads into the address space of an .exe file, the trust level of the .exe file determines the final access level of the DLL. Also, the DLL runs in the address space of the .exe which is not desirable for a secure connection. To get around this,  SECUR32.DLL creates a secure virtual address space (outside of the .exe address space) when InitSecurityInterface() is called. 

Accessing the data and functions in this space is done via a SecurityFunctionTable structure. This is a dispatch table that contains pointers to the functions defined in SSPI.h. It contains the following functions:


  • dwVersion                  
  • EnumerateSecurityPackagesA 
  • QueryCredentialsAttributesA
  • AcquireCredentialsHandleA  
  • FreeCredentialHandle  
  • InitializeSecurityContextA 
  • AcceptSecurityContext      
  • CompleteAuthToken          
  • DeleteSecurityContext      
  • ApplyControlToken          
  • QueryContextAttributesA    
  • ImpersonateSecurityContext 
  • RevertSecurityContext      
  • MakeSignature              
  • VerifySignature            
  • FreeContextBuffer          
  • QuerySecurityPackageInfoA  
  • ExportSecurityContext      
  • ImportSecurityContextA     
  • AddCredentialsA            
  • QuerySecurityContextToken  
  • EncryptMessage             
  • DecryptMessage             
  • SetContextAttributesA      
  • SetCredentialsAttributesA  

Each member of this structure is a DWORD (unsigned 32 bit Integer), and is the entry point for the function in the secure address space. To call the function, you must use the function definition provided in SSPI.h, then use a technique like:

    g_pSSPI->DeleteSecurityContext( &hContext );

This de-references the SSPI pointer and invokes the structure member DeleteSecurityContext() using the function prototype that follows. To do this in another language like BASIC:

    CALL DWORD @gpSSPI.DeleteSecurityContext USING DeleteSecurityContext( phContext )


  


STEPS TO A SECURE CONNECTION
  • Initialize Security Interface
  • Initialize Windows Sockets
  • Create an SSPI credential
  • Connect to the server
  • Perform handshaking
  • Authenticate the server's credentials
  • Get the server's certificate.
  • Verify the server's certificate
After these steps, a secure connection is in place and we may begin our request/response session.

Keep in mind that the example code uses Winsock to make a socket connection. When writing any code using Windows Sockets, be sure to read The Windows Sockets Lame List, a great list of what to avoid when programming with Windows Sockets ranging from the "Nauseatingly lame" to "Stooping to unspeakable depths of lameness".


CERTIFICATES

A Client certificate is NOT required for a secure connection. Remember, it IS possible to have a one sided certificate exchange during a secure connection handshake!

In the example code, finding a certificate in the system store will work if you already have a certificate and it was successfully imported into the "MY" certificate store. Since you probably don't, you will see none when you use Internet Explorer to view your certificate store:



In this case the code creates a NULL credential which is stored in the paCred member of the SCHANNEL_CRED structure. This is sent to the Server which will accept it and return its certificate allowing us to create a connection. We will validate the Server Certificate, but obviously the Server cannot validate the Client certificate as there isn't one.

As specified in rfc3760, a request using a NULL will prompt the server to return all credentials stored under the current user account. The client recovers this certificate and then validates it by calling QueryContextAttributes().

If you want to use a Client certificate, ultimately it must be purchased. An easy place to get a certificate with a recognized authority is GoDaddy. You can get an SSL cert valid for 1 year for $30. There are other places that might be cheaper if you hunt around a bit. It's not too difficult to do. Most of the sites are very user friendly and make it easy to buy a cert.

Remember, since you are doing mutual authentication, you'll need a certificate with a valid authority, and not just a self-signed one which you can create yourself because the Gmail server would have no way to verify it.

INSTALLING A CERTIFICATE (if you decide to go this route)

Once you have bought a certificate it needs to be installed:
  1. Install the certificate into the machine store. Go to Start | Run and enter in MMC to get a blank MMC console. Go to File | Add/Remove Snap-in. Click the Add button to select the snap-in. Choose Certificates from the list and click Add. Choose the Computer account option, then Next, Finish, Close, and finally OK.
  2. Importing the certificate into the Machine Store. Expand out the tree and choose the Personal folder. Right click on Personal and choose All Tasks, Import. Follow the wizard to complete the process. The certificate should now be in the machine store.
  3. Adjusting the private key permissions. On Windows 2008, this is added to the Certificate Manager GUI. In Windows 2003, you'll need to download the Windows Server 2003 Resource Kit Tools. Use the following command's format to set permissions:
C:\Program Files\Windows Resource Kits\Tools\winhttpcertcfg.exe -g -c LOCAL_MACHINE\My -s "mycert" -a "USER_ACCOUNT"

Where 'LOCAL_MACHINE\My' is the Personal Certificate store in the local machine, 'mycert' is the name of your certificate, and USER_ACCOUNT is the account which will be given read permissions for the private key. You can try the IUSR account, but you might need to use the 'NETWORK SERVICE' depending on the exact context your CGI app is running under. You will need to restart IIS for changes to take effect. The documentation for WinHttpCertCfg covers what it's doing.

You should now have permissions to access the private key. You will still use CertOpenSystemStore with the MY flag.




REQUEST/RESPONSE SESSION
At this point a secure connection is established and we may begin the task we connected for. This takes the form of a request and a response, just like an HTTP session for example. You should expect the server to immediatly send a response once the connection is esatablished even though a request was not issued. This is usually the welcome message. If you do not read it prior to issuing your first request, it will appear at the beginning of the response to your first request as all date is buffered.

Data is read on the secure connection by calling Windows Sockets recv(). This is a blocking function, so you need to check the data to know when the response is complete. In the case of a SMPTS session, we are looking for a CRLF preceeded by a space. Once this is parsed, the response has been read. If you call recv() again, it will block until the server sends more data.

The send() and recv() operations are sending/reading encrypted data. In the case of sending the data is encrypted prior to sending and in the case of reading it is decrypted after being read by calling the SChannel functions EncryptMessage() and DecryptMessage(0 in SECUR32.DLL.

SMTP session is now possible. Now keep in mind the "Mind bogglingly lame" assuming that stream sockets maintain message frame boundaries is wrong. Stream sockets (TCP) are called stream sockets, because they provide data streams (duh). As such, the largest message size an application can ever depend on is one-byte in length. No more, no less.

The Server may respond so fast that the welcome message is recovered by the handshake and reported as extra bytes. This can be decrypted by calling DecryptMessage(). If not, it will be found as the first part of the response to the first EHLO request. The example code simply detects the first occurrence of a CRLF, so the Read Decrypt function will need to be called to recover the welcome message before the actual response can be recovered.

Requests and responses are Sent or Received as Encrypted Envelopes. Each envelope contains a complete request or response. Each time you call recv() it blocks until data is available. The buffer returned contains a complete envelope. This envelope must be decrypted and checked to see it more data is expected.






STARTTLS

SMTP ports:
  1. Port 25 (SMTP) is plain text. The conversation between server and client is held in plain text. This is where the majority of internet MTAs are and have been since SMTP was introduced. The big problem with plain text being that user credentials and passwords are easy to intercept by sniffing the transmission media.
  2. Port 465 (SMTPS) is TLS encrypted. The conversation between server and client is held through an encrypted connection.
  3. Port 587 (STARTTLS) is a transitional hybrid. Think of it as a special interim measure between 1 and 2. Although 'interim' on the internet likely means the next decade or so. STARTTLS is an enhanced SMTP command; an optional extension to the SMTP protocol. STARTTLS allows the MTA and client to switch (one way) from plain text to TLS encrypted transmission. STARTTLS is often provisioned on Port 25 for compatibility with either plain text or encrypted transmission. When provisioned on port 587 it is implicitly a requirement that transmission must become encrypted prior to authentication.

A Plain Text session can be upgraded to a fully encrypted session with some servers, particularly google on port 25 or port 587 after the connection is innitiated. So STARTTLS is really an extension to plain text communication protocols. It offers a way to upgrade a plain text connection to an encrypted (TLS or SSL) connection instead of using a separate port for encrypted communication as described in RFC 5246 which states:
One advantage of TLS is that it is application protocol independent. Higher-level protocols can layer on top of the TLS protocol transparently. The TLS standard, however, does not specify how protocols add security with TLS; the decisions on how to initiate TLS handshaking and how to interpret the authentication certificates exchanged are left to the judgment of the designers and implementors of protocols that run on top of TLS.[1]
Several library implementations of TLS like RFC 3207 SMTP extension use this style for connecting client and server with a plain text session that can start a secure session.

The example code can support this mode. If you fail to implement a secure connection, you will get an error message that looks something like:
"530 5.7.0 Must issue a STARTTLS command first"

An example SMTP session with google:

    C:                <establish a connection>
    S: 44Bytes:  220 mx.google.com ESMTP 6sm661677yxg.48

    C:  7 Bytes:   EHLO
    S: 125Bytes: 250-mx.google.com at your service, [46.311.221.11]
                        250-SIZE 35651584
                        250-8BITMIME
                        250-STARTTLS
                        250 ENHANCEDSTATUSCODES
   

    C:  initiate TLS session and begin the SMTP dialog



IIS

Using SSPI Schannel in a Windows Server setting raises a problem. Accessing user certificates requires permissions of a user, and accessing the users private key requires either that the user has logged on locally with the password, or that the server is enabled for delegation. Machine certificates are readable by everyone, but accessing the private key requires admin rights (or local user) unless you adjust the ACL.

If you want a CGi/FastCGI process to send an email (to confirm a purchase and supply download information for example) the CGI application will need this access if you intend to use a Client Certificate for the secure session. Since IIS runs a CGI application under IUSR (which has very limited priviledges to maintain security) this poses a problem. 

The handshake process begins by checking for a Client certificate in the "MY" certificate store.Without permission, this function returns zero and GetLastError() also returns zero. A solution I did not try is to buy a Client Certificate and then grant permission to the IUSR account to read the private key of the certificate as covered above.

Also in IIS 6, you may receive an error message when you try to start a CGI program. This is by design! You will need to set CreateProcessAsUser to false, and specify the application pool's identity to local system.

Since the only reason we want to open the "MY" certificate store is to look for a potential certificate that can be used for client authentication as part of the initial handshake, it makes more sense to just use a NULL certificate if we don't care about the Server validating our certificate, and for sending email via gmail, you probably don't, because all we really care about is making sure our client is talking to the server it thinks it is.

This makes life a lot easier, and we just need to adjust the code to skip the call to CertOpenSystemStore(0, "MY"). With that done the handshake process will complete but with a wrinkle. Some servers (like gmail) return a little welcome message (63 bytes of encrypted data to be precise) that decoded looks like:

  • 220 mx.google.com ESMTP 4sm2476575ywg.43

This raises a problem because it is returned in the buffer as  SECBUFFER_EXTRA. This data must be processed as it contains a check sum. If it is incomplete, the next recv() will complete it and the decryption will succeed. If you do not process it, the next read is likely to return SEC_E_MESSAGE_ALTERED or perhaps SEC_E_INCOMPLETE_MESSAGE. The example code was not updated to do this, but you have the road map here.