First, it’s much easier to lose physical keys than it is to forget critical system passwords (assuming a robust password management system is in place). Loss of physical keys renders some or all of the HSM contents unusable to applications. Therefore, when setting up a HSM to prepare it for use by applications, it’s critical to make backups of your physical keys. The approach I recommend to clients is to identify at least 2 system owners that will act as key custodians for the physical keys. In addition to the keys that are assigned to system owners, a third and possible fourth set of keys are created for disaster recovery or for unlikely scenarios such as the two system owners simultaneously winning the lottery! Passwords or PINs used to access the material stored on the physical keys need to be protected as any other high value system password needs to be. For backup material, I always recommend that a slip of paper with the PIN/password used by the physical keys be stored directly with the keys in a tamper evident bag that is locked away in a safe (along with backup media that stores copies of the key material stored on the HSM itself of course!)

Another concern with physical key management is ensuring that all copies of keys are carefully inventoried and controlled. Unfortunately, I have found that this is much easier said than done and that you are better off designing your system under the assumption that there will be duplicate physical keys made that you will lose track of. The best defense against uncontrolled key material is to ensure that additional credentials/keys are necessary in order to use a duplicated key. The method that I recommend to clients using HSMs that support this feature is to require MofN authentication, otherwise known as witness keys. Witness keys are basically additional physical keys that all need to be presented together with a master key in order to gain access to HSM contents. The intent is that there would need to be collusion between multiple parties to actually make use of any single set of duplicated key material.

The last concern that I’ll discuss in this post are the responsibilities associated with being a physical key custodian. Obviously, anyone who is assigned a physical key needs to take all possible precautions to protect against the loss of their key and report it’s loss as soon as possible to system owners. Assuming that good backups are in place, it is relatively simple to replace a lost key. Perhaps a more important role, however, is to always question why their physical keys are needed for accessing the system. Applications which use HSMs for key storage can generally be configured to require physical keys to be used to unlock the HSM for each key access or to permit the HSM to be unlocked for a period of time while key access is required. The method and frequency of access to the HSM must be understood by all key holders (including witness key holders) so that out of the ordinary requests for presenting physical keys to the HSM can be detected.

As with any other security control, HSMs using physical keys can greatly reduce an organization’s risk, but only if solid operational processes are in place.

If you are not familiar with the concept of a HTTP reverse proxy, think of it as a specialized web server that inspects incoming HTTP requests, forwards them to another (usually internal) web server after any local processing is completed, and makes sure that HTTP responses sent back to the client contain proper host names. The following diagram illustrates the typical processing flow when using a SSL enabled reverse proxy server that is acting as a front end to an internal web server.

In this diagram, the reverse proxy server is performing 3 major tasks:

• Performing an additional level of authentication for client connections.
• Proxying HTTP requests and rewriting hostnames in HTTP responses.
• Terminating SSL connection requests from clients.

In addition to the tasks above, it is possible to use a reverse proxy server  as a poor mans web application firewall by filtering HTTP requests that contain invalid characters that could be used for basic SQLi attacks. Reverse proxy functionality is provided by external modules such as mod_proxy for Apache and URL Rewrite/Application Request Routing for IIS.

To get clients to connect to the reverse proxy server instead of the internal application server, the reverse proxy server is usually configured with a public DNS hostname and publicly routable IP address. Thus, any links that are made available to users will resolve to the public IP address/DNS hostname of the reverse proxy server. The reverse proxy server will then perform the job of mapping the external HTTP requests to the proper internal web server.

With that introduction out of the way, let’s discuss how a HTTP reverse proxy can be used to add an additional layer of authentication to a web application. As pointed out earlier, a HTTP reverse proxy is really nothing more than a dedicated web server with additional modules that decide when and how to forward traffic to another web server. Thus, when a user connects to a HTTP reverse proxy server, any authentication method that is supported by the reverse proxy server’s web server platform can be used to authenticate users.

For my engagement, I used IIS 7.5 with URL Rewrite 2.0 as my reverse proxy server. To provide an additional layer of authentication, I used the RSA Web Authentication Agent 7.0 along with Authentication Manager Express. After installing the web authentication agent, I configured the agent to protect all resources on the reverse proxy server. This has the effect of forcing all HTTP requests that are accepted by the reverse proxy server to be intercepted and processed by the web authentication agent. Once the web authentication agent completed the authentication sequence with the client, a record of the authentication is stored in the client’s browser as a cookie. Immediately after the authentication sequence completes, the agent redirects the user back to their original destination which again points to the reverse proxy server. Since authentication is now complete, the web authentication agent will no longer intercept the request and will instead forward the request over to the URL rewrite module. The URL rewrite module will then simply forward requests to the internal web server per the rewrite rules that were configured.

As for the rewrite rules themselves, when using the RSA web authentication agent on the reverse proxy server I found it necessary to write 2 rules for incoming HTTP requests to ensure that the reverse proxy server did not forward requests for the web authentication agent to the internal web server. To handle requests that should be processed by the web authentication agent, I created an inbound URL rewrite rule that matched the URL path for requests to the web authentication agent. The action for this rule was to simply not process it, which permitted the request to drop out of the URL rewrite module and be handled by the web authentication agent. Any other request that did not match the agent’s URL path was forwarded to the internal web server. The following snippet from my web.config file illustrates the rules that were used:

  <rules>
     <rule name="ReverseProxyInboundRule1" enabled="true" stopProcessing="true">
      <match url="(.*)" />
      <conditions>
       <add input="{CACHE_URL}" pattern="^(https?)://" />
       <add input="{REQUEST_URI}" pattern="(.*IISWebAgentIF.dll.*)" />
      </conditions>
      <action type="None" />
     </rule>
     <rule name="ReverseProxyInboundRule2" enabled="true" stopProcessing="true">
      <match url="(.*)" />
      <conditions>
       <add input="{CACHE_URL}" pattern="^(https?)://" />
      </conditions>
       <action type="Rewrite" url="http://www.example-internal.com:8080/{R:1}" logRewrittenUrl="true" />
      </rule>
   </rules>

For fans of open source solutions, the same functionality could easily be provided with Apache’s mod_proxy module using the ProxyPass and ProxyPassReverse directives. The virtual servers that contain these directives would then need to be configured to require authentication at which point any of the standard Apache user authentication methods such as mod_auth or mod_auth_ldap. Apache has the additional advantage of supporting the mod_security web application firewall, so the mod_proxy directives could be easily combined with mod_security to provide both additional client authentication and a transparent WAF.

Some key takeaways for me:

• Cloud service providers pretty much cover the entire stack, from infrastructure all the way to software. However, you as a smart consumer still need some in-house expertise on the entire stack so you can adequately manage your providers.
• Transparency is key for a cloud provider, but transparency means more than just sales sheets and sanitized ISO/ITIL compliant security policies. Think open Bugzilla style issue trackers that customers can follow to see issues affecting the service offered by their cloud providers.
• Good lawyers are needed by both cloud providers and cloud consumers to manage liability (yes, even cloud consumers are exposed to some new liabilities when using cloud services.)
• Vendor lock in to a cloud provider is scary to consumers; again good in house expertise is needed to design your cloud strategy to migrate easily between providers.

Overall, it was a useful, thought provoking discussion that provided insight into areas of cloud computing I hadn’t thought of before. For any Chicago locals interested in the Chicago CSA, their website can be found here.

As a little background, I broke into the infosec biz in the opposite way that many of my peers have. My first formal role in infosec was as a security systems engineer for Motorola. My job was to basically look at security standards, product management requirements, and security threats and design security features. Problem is, when you take that direction you miss out on the real world, hands on perspective of infosec and have difficulty understanding exactly how organizations infosec functions (if it even exists!) It’s way too easy to fall into the ivory tower mentality. Add a couple of years of grad school in infosec, and you have the makings of a great infosec philosopher, but not a practitioner. Consulting has really opened my eyes and is helping me understand what I never fully grasped before.

With that out of the way, during my last couple of engagements I’ve learned a ton about enterprise identity management and have corrected a major misconception I had held in the past. Earlier in my career, I was faced with solving the problem of figuring out how to integrate enterprise identity management systems with wireless equipment; so that system administrators could use their existing credentials to log into the management interfaces of the wireless infrastructure. At the time, the solution that struck me was to use a AAA server with RADIUS; after all RADIUS is easily extensible, simple to develop and support, and is flexible. What I completely failed to realize at the time is that a AAA server is in fact not intended to act as an identity management system. Sure, AAA stands for Authentication, Authorization, and Accounting, but the AAA server is generally just a relay and is not the actual identity management system.

As I have since learned, identity management is actually centered around one of the oldest protocols in the book, LDAP or Lightweight Directory Access Protocol. LDAP is a directory service that is designed to provide extremely fast lookups and extensive search capabilities. LDAP gets it speed from the hierarchical design of objects that are stored within the directory. All objects descend from a single root node, with each child node having one or more child nodes and one or more sibling nodes. Here’s a figure of a basic tree structure of an LDAP directory.

Each node in the directory is uniquely identifiable by use of a distinguished name. A distinguished name is made up by simply concatenating the identifiers for each ascendent node to the node being named. In this figure, the DN for me would be CN=Walter,OU=Person,OU=Users,DC=jdt,DC=com. Each node also has a collection of attributes associated with it that contain information such as date of creation, address, member groups, object identifier and any other information that is pertinent to the object itself. Lookups are done by connecting to the directory server and issuing queries. LDAP queries are interesting in that they are written in a reverse polish notation style; e.g. if you wanted to search for my name in the query above quickly, you could search for a person named Walter with the following query: (&(OU=Users)(CN=Walter)).

To make LDAP directories generally useful, they usually provide some information to LDAP clients without requiring the client to authenticate. Examples would be a web based corporate address book that is accessible from a organizations intranet. However, in order to gain access more sensitive information, LDAP clients usually authenticate to a directory via a bind operation. In fact, this is how many network operating systems such as Active Directory, Novell etc. authenticate users; a bind against a corporate LDAP directory is performed using the user’s credentials. Once the bind is successful, the user is granted access to the local system and further authenticated LDAP queries can be performed to obtain additional information needed to grant the user access to resources.

LDAP itself does not provide any integrity or encryption mechanisms to protect information that is transmitted between the directory and LDAP clients. However, bind operations usually use authentication protocols which protect credentials that are transmitted such as NTLM or GSSAPI. To provide integrity and confidentiality for LDAP data, LDAP may also be transmitted over a TLS secured port (LDAPS). Yet another use case for an enterprise PKI that I had absolutely no idea about.

So, to complete the original story, what I’ve since discovered when it comes to centralized user management, enterprise identity management, don’t think of AAA services. Rather, think of LDAP directory services. As a matter of fact, AAA servers usually have plug-ins (such as FreeRADIUS’s rlm_ldap plugin) that let them use LDAP directories as a data source when performing authentication and authorization services.

As an example, in my current engagement I’m deploying a part of a tokenization solution to satisfy PCI requirements that my customer is required to satisfy. The scope of the engagement from my perspective is rather limited, but I’ve taken a deliberate effort to go above and beyond the minimum requirements outlined in the SOW (without increasing my scope, a fine line to walk for sure!) Specifically, while I was in the customer’s data center performing some configuration tasks, I looked around and noticed that there didn’t appear to be any cameras in position to observe the equipment I was configuring. As the customer explained that the equipment was considered ‘in scope’ of the PCI DSS requirements, I pointed out that requirement 9.1.1 of the DSS requires the use of a video camera to monitor equipment (and yes, I know that the requirement is an and/or requirement). The customer contact I was working with was not part of the physical security team and couldn’t confirm whether or not the area was in fact monitored, but he took it as an action item to follow up on.

While in all likelihood this will turn out to be a non-issue, the customer expressed appreciation at my observation. This in turn leads to a stronger sense of trust between us and, in my opinion, enhances the overall value of the engagement. So next time you find yourself in a position where you can offer a little extra advice to your customers, consider going the extra mile. You won’t regret it!

In general, the design philosophy behind HSMs is that it should fail closed; meaning that if an attacker or unauthorized user attempts to repeatedly gain access to the HSM key material, the HSM will zeroize all stored keys. This design makes sense because if an attacker were to gain physical access to the HSM, it is preferable that the secret keys be destroyed rather than possibly be exposed. However, since the HSM protects high value keys, it is imperative that organizations which make use of HSMs have a robust key backup scheme in place such as sharing the key across multiple HSMs that are physically separated.

HSMs typically have a strong role based authentication mechanism built in that is designed to differentiate between key owners and HSM administrators. This separation of duties between key owners and administrators is crucial as it prevents HSM administrators from gaining access to key material. Authentication can be provided via the use of passphrases, or in some HSMs, via the use of individual hardware keys that are physically presented to the HSM.

Since access to key material and to the HSM itself needs to be carefully audited and tracked, a principle that is implemented in some HSMs is the concept of witness keys. A witness key is a separate set of keys that are distributed to other people (usually from different departments/organizations than HSM administrators or key owners) that must be presented to the HSM before access will be granted to the HSM and/or key material stored within the HSM. Witness key systems are also known as MofN systems, where ‘M’ witness keys out of a total of ‘N’ existing witness keys must be presented before access can be granted.

In summary, proper protection of high value keys is an important role that any security organization should take very seriously. HSMs can be a viable solution to help ensure that key material is stored in a safe, controlled fashion.

First, a little background. Previous visitors that have read my post on certificate validation will recall that a standard PKI hierarchy has a single CA (root CA) that is issuing certificates to subscribers. In this model, subscribers are assumed to have a copy of the root CA certificate in their devices. That certificate is then used to validate that certificates presented to the subscriber have been issued by the CA. In an enterprise network environment, a simple flat hierarchy where the root CA directly issues all subscriber certificates, all devices which will need to access resources protected by subscriber certificates would then need to be provisioned with the root CA certificate. In some simple environments, this is easy. For example, in a standard Microsoft Active Directory network, a domain controller could be configured to both act as a CA and could add the root CA certificate to group policy. Computers which are domain members could then fetch the root CA certificate as part of the domain login process.

However, as the set of devices which are used in enterprise networks continue to grow (iPads, iPhones) and as other web browsers (Mozilla, Opera) gain even more acceptance, the problem of provisioning root CA certificates gets more complicated. These devices/software applications obviously cannot be joined to the domain to obtain the root CA certificate. Therefore, the enterprise network administrator will have to provision the root CA certificate on these other devices using some other centralized certificate management solution.

Several commercial certificate authorities offer a service where they will issue a subordinate CA certificate to enterprise customers (full disclosure: my company offers this service as well.) In this case, an enterprise PKI CA certificate would be signed by a root CA certificate that is already embedded in most browsers, operating systems and devices. This permits the enterprise root CA to be trusted by those devices/applications without a need to provision an additional root CA certificate in them.

Such an approach has pros and cons to it. Let’s analyze this approach in the context of 4 common enterprise PKI use cases as outlined in my SANS gold paper using a simple pros/cons approach.

802.1x WLAN

Pros:

• The WLAN authenticator (typically a AAA server) is automatically trusted by wireless clients which makes it easy to seamlessly support both enterprise and guest WiFi use cases in the same WLAN infrastructure.

Cons:

• Opens up WiFi clients to rogue AP attacks (if an attacker can obtain a valid certificate, he can easily setup a rogue AP that would be automatically trusted by enterprise WiFi clients which would enable him to exfiltrate credentials from those clients.)

HTTPS

Pros:

• Enables enterprises to secure Intranet websites with HTTPS without requiring users to ‘click through’ standard browser warnings.
• Allows any device/web browser software to access the Intranet website without requiring additional user actions to provision the root CA certificate.

Cons:

• Enterprise sub CAs could potentially issue valid certificates for Internet websites. A rogue enterprise CA administrator could easily issue certificates that do not belong to his enterprise domain. These certificates could be resold to attackers who could use them to setup MITM attacks for other HTTPS protected Internet sites.

S/MIME

Pros:

• Digitally signed emails could be validated by email recipients that are outside of the enterprise.

Cons:

• If an attacker is able to obtain a valid S/MIME certificate, he/she could send very convincing spear-phishing emails to enterprise users. It is unlikely that emails with valid digital signatures would be marked as spam by spam filtering software.

Infrastructure (routers, VPN gateways etc)

Pros:

• Could possibly simplify the provisioning of new network infrastructure components.
• Makes it easier to support additional VPN client types.

Cons:

• Makes it much easier for an attacker who has physical access to the enterprise network to setup rogue devices; especially since many network devices which support certificate based authentication use the certificate as the sole means of identifying valid peers.

In my opinion, the risks of using certificates rooted to a public CA for enterprise WLAN, S/MIME and infrastructure authentication outweigh the benefits. For securing Intranet websites with HTTPS, the benefits outweigh the risks because HTTPS protected websites could easily require the user to present credentials that are valid for the enterprise network. That being said, there are certainly additional controls that could be implemented in S/MIME and WLAN deployments to mitigate the risks pointed out above.

What do you think? Are there other risks or use cases for publicly trusted enterprise PKIs?

The schema of the XML files is simple enough; each file contains 3 fields of note: an issuer field a shortname field, and a certificate-data field. For my purposes, the field that I needed to analyze was the ‘Issuer’ field. Since there are 255 certificates listed in the latest directory with the Opera site, I decided to automate my search using a combination of python, wget, and openssl commands. As this is my first foray into python, I ran into a few speedbumps along the way (especially since my previous scripting language of choice was Perl). In the end, I used the following script to automate my search. Hopefully this will save someone else time in the future.

#!/usr/bin/python

import os
import sys
import subprocess
import re

url = sys.argv[1]
cmd = 'wget --quiet -O - %s' % (url)

print "Fetching the list of certs from ", url

# get the list of filenames
page = subprocess.Popen(cmd,shell=True, \
 stdout=subprocess.PIPE).communicate()[0]

p = re.compile(r".*\"\>(.*xml)<")
matches = p.findall(page)

for m in matches:
 certsrc = url + '/' + m
 cmd = 'wget --quiet -O - %s' % (certsrc)
 cert = subprocess.Popen(cmd,shell=True,\
 stdout = subprocess.PIPE).communicate()[0]
 p = re.compile(r"<issuer>\n(.*)\n</issuer>",re.MULTILINE)

 issuer = p.search(cert)
 print "%s issuer: %s" % (m,issuer.group(1))

I’ve been a Dropbox user for about 2 years now and regularly store sensitive personal information in my Dropbox folder. However, I don’t just trust that Dropbox is going to implement data security best practices to protect my data. Rather, I encrypt my sensitive files before storing them in my Dropbox folder. Solutions I’ve used are Truecrypt to create small data partitions that I can mount on my devices and 7zip with it’s built in AES encryption function. With these solutions, I alone am responsible for controlling when my encryption keys are disclosed. Even if Dropbox is required to share my files with law enforcement or if there is a major data breach in their service, my files are still safe because I control my private data encryption keys.

While it is perfectly reasonable to trust service providers with your information, it is not reasonable to always assume that you will have full control of your information. Users need to take responsibility for their own data protection.

In the Comodo incident, it’s quite possible the attacker was able to obtain credentials to the reseller’s vettor interface (I’m not with Comodo so this is speculation on my part). This would enable them to submit certificate enrollment requests for domains they didn’t own and immediately approve them. Luckily, the fraudulent certificates were discovered 15 minutes after issuance according to Comodo. Comodo isn’t saying how the fraudulent certificates were discovered, but most likely there are audit tools that scan issued certificates against a whitelist of known high value domains.

The implication of this attack and it’s effect on the industry has served to cause site operators to seriously question the value of CAs and look at alternative trust models. As explained in my other post, the current CA trust model requires users to trust their browser, operating system, and device vendors to include a list of ‘trusted’ CA root certificates in their products. Site operators which wish to use SSL certificates to secure their websites must purchase certificates from one (or more) of these CAs, during which they prove domain ownership and, in the case of extended validation certificates, additional authentication information. In this model, note that end users (who have the most to lose from accessing fraudulent websites), have no power to choose who they trust (the decisions are made for them by site operators and browser vendors). Nominally, this is actually a good thing since end users generally can’t be expected to have enough information to make sensible trust decisions.

The biggest weakness that this trust model has exposed as a result of the Comodo incident is the difficulty in providing an effective way for fraudulently issued certificates to be revoked. While both CRLs and OCSP were used to revoke the certificates, the fact is that many end user devices do not regularly fetch certificate revocation information due to complexities with maintaining effective access to CRL/OCSP responders and continuing debate on how CRL/OCSP failures should be handled in browsers.

With that background, a couple of alternative trust schemes are being proposed within the IETF. Perhaps the most promising scheme is DANE (DNS based Authentication of Named Entities). The cool thing about DANE is that it leverages another major security protocol overhaul that’s been happening for the last 8 years or so (DNSSEC). Essentially, DANE provides a mechanism for a domain registrar to add a certificate/public key for a given website to that site’s DNS record. Since DNS lookups themselves will be secured using the underlying DNSSEC mechanisms (and the contents of the DNS records will be authentic and integrity protected via DNSSEC services), the DNS records are as safe a place as any to store key material used to secure a site (or any client resource that can use keys to secure connections). The controversy that I’m seeing with the DANE proposals is that the trust model doesn’t require a CA to issue certificates used to secure a site. In fact, DANE fully supports the notion of self signed certificates. The idea is that if you trust that the owner of the domain has already passed necessary validation to get a DNSSEC secured DNS record, why can’t you simply trust the owner to provision a key for his/her site especially since that key is protected by DNSSEC resources?

There are still lots of open questions and discussions around DANE, and the CA vendors are making strong arguments that their services are still required even in a DANE enabled ecosystem (the anti-fraud checks performed by CAs before issuing certificates are unlikely to be performed by DNS registrars and obviously a site operator would not need to perform such checks).

I know that I’ll be following the development of DANE and discussions on the IETF DANE lists.