One common and potentially time consuming administrative task I hear my customers discuss is maintenance of SSL certificates on large Web farms (initial set up, renewals, etc.). These Web farms can be for Outlook Web Access, Ecommerce, intranet pages, etc. One of the new features we added in IIS8 in Windows Server 2012 is Central Certificate Store (CCS). The gist of this feature is to allow IIS8 to go get certificates for SSL sites it hosts on demand from a central location instead of its local certificate store.

What is CCS?

I originally planned on breaking down how it works, etc. however a colleague named Kaushal Kumar Panday has already done a fantastic job covering CCS in the blog post found here:

http://blogs.msdn.com/b/kaushal/archive/2012/10/11/central-certificate-store-ccs-with-iis-8-windows-server-2012.aspx

So instead of repeating all this info, I thought it would be cool to peek into a couple areas of the feature to get to know it a little better, as well as provide guidance for a robust implementation. So if you want to geek out on CCS, read on!

Let's Geek Out on CCS!

One thing I thought would be neat to see in a packet capture via Network Monitor is the interaction between the Web server and the file server that hosts the certificates for CCS. Here is my setup which will be used for the duration of this post.

1. Server2012-2 (10.10.14.152) - Web server hosting an SSL site "centcertdemo" configured to use CCS.
2. Server2012-3 (10.10.14.153) – File server hosting the share with the certificate. Note this server does not have to be a Windows Server 2012 server but there is a security enhancement you will gain by using 2012. I'll cover this a bit later in the "Security" portion of this article.
3. Win8ent (10.10.14.154) – Windows 8 client accessing the SSL site.

The first thing I did was an IISReset on the Web server just to get the trace as clean as possible. It's also the #1 thing I use to troubleshoot IIS issues but that's just because I am a newb when it comes to IIS, but I digress J. Let's see how this works:

1. Win8ent makes a TCP connection (3-way handshake) and initial SSL hello to Server2012-2 (Web server). We can see that here in the trace.
   
   
   
   
   In that Hello should be the Server Name as part of the ClientHelloExtension for the site the client is requesting. We can see that here in the Hello packet.
   
   
   
2. Next, Server2012-2 (Web server) should go look on the share hosted on Server2012-3 (File server) for the certificate file name matching the same of the site/servername the client is requesting. This is occurring via SMB2 as we see in the trace snip below. The key point here is that the file name must match the site the client is requesting.
   
   
   
   
3. Now here's something interesting, Server2012-2 (Web server) doesn't ACK back the SSL Hello until after it has the certificate, then finishes the SSL Hello.
   
   

So perhaps one thing to learn here is to use at least Windows Server 2008 so you can reap the benefits of the updated TCP/IP stack as well as SMB2.

High Availability

Another thing to plan for is failure conditions. It would be most unfortunate if the file server that hosts the central certificate store was unavailable when an SSL request came in and the cert was not cached. At first I wondered if we kept a local copy so I decided to test that theory out.

To set up the scenario where the file share hosting the certs for CCS dies, I stopped the server service on Server2012-3 (File server) then did an IISReset on Server2012-2 (Web server)

1. Same as before, Win8ent makes a TCP connection and initial SSL hello.
   
   
   
   
2. Now Server2012-2 calmly calls over to Server2012-3 to get the cert but notice packet 77, this Reset is expected since the Server service is stopped on Server2012-3.
   
   
   
   
3. Server2012-2 freaks out and frantically starts to reattempt to set up SMB2 and pull the cert, but his/her attempts are futile.
   
   
   
   
4. Now here's the interesting part, Server2012-2 finishes the SSL Hello, but check out what cert is used.
   
   
   
   
   
   That cert is bound to another 443 SSL Web site on the server (Powershell Web Access) and is not retrieved via the CCS functionality. The client also got a nasty cert error since the names don't match.
   
   

    

So as you can see it's critical to make sure your certs are hosted on a highly available file share. Failover clustering would be a nice choice J.

Security

I also want to point out a couple things from a security perspective pertaining to this screenshot. To navigate to this screen, in the IIS Manager, click the Server name in the Connections pane on the left, then double-click on the Centralized Certificates icon under Management in the center pane. As a reminder, the centralized Certificates icon will only show up if you have installed the feature via Server Manager.

1. Use least privilege practices for the account you configure IIS to use to access the share for CCS. Lock down your share and NTFS permissions as well as remove the account from Domain Users and set workstation logon restrictions (don't forget to add the computer objects for the servers hosting the share!). Also restrict others from accessing the share since it will have full certificates stored there.
   1. At this time I am not aware of the ability to use Group Managed Service Accounts for this feature. I will update the blog if I find out that you can.
2. Bitlocker protect the volume that hosts the share for the certs. We introduced the ability to Bitlocker protect Cluster Shared Volumes in Windows Server 2012. The volume hosting the share holding your SSL certs might be a good candidate perhaps? J
3. Also note in the config dialog how the Private Key password is optional. This assumes that you did not use a password to protect the private key when you exported the certificate to place on the share. I can't think of a reason as to why you would NOT want to use a private key password. So set a private key password and configure this option.

Powershell Integration

Finally, there are Powershell commands which can be used to configure CCS. This is a good way to further automate Web server build processes that host SSL sites.

That's all for now on CCS. I hope you enjoyed this blog and please post in the comments if you have any questions, comments or concerns.

Jake Mowrer