Insufficient Transport Layer Security (HTTPS, TLS and SSL)

Communication between parties over the internet is fraught with risk. When you are sending payment instructions to a store using their online facility, the very last thing you ever want to occur is for an attacker to be capable of intercepting, reading, manipulating or replaying the HTTP request to the online application. You can imagine the consequences of an attacker being able to read your session cookie, or to manipulate the payee, product or billing address, or to simply to inject new HTML or Javascript into the markup sent in response to a user request to the store.

Protecting sensitive or private data is serious business. Application and browser users have an extremely high expectation in this regard placing a high value on the integrity of their credit card transactions, their privacy and their identity information. The answer to these concerns when it comes to defending the transfer of data from between any two parties is to use Transport Layer Security, typically involving HTTPS, TLS and SSL.

The broad goals of these security measures is as follows:

  • To securely encrypt data being exchanged
  • To guarantee the identity of one or both parties
  • To prevent data tampering
  • To prevent replay attacks

The most important point to notice in the above is that all four goals must be met in order for Transport Layer Security to be successful. If any one of the above are compromised, we have a real problem.

A common misconception, for example, is that encryption is the core goal and the others are non-essential. This is, in fact, completely untrue. Encryption of the data being transmitted requires that the other party be capable of decrypting the data. This is possible because the client and the server will agree on an encryption key (among other details) during the negotiation phase when the client attempts a secure connection. However, an attacker may be able to place themselves between the client and the server using a number of simple methods to trick a client machine into believing that the attacker is the server being contacted, i.e. a Man-In-The-Middle (MitM) Attack. This encryption key will be negotiated with the MitM and not the target server. This would allow the attacker to decrypt all the data sent by the client. Obviously, we therefore need the second goal - the ability to verify the identity of the server that the client is communicating with. Without that verification check, we have no way of telling the difference between a genuine target server and an MitM attacker.

So, all four of the above security goals MUST be met before a secure communication can take place. They each work to perfectly complement the other three goals and it is the presence of all four that provides reliable and robust Transport Layer Security.

Aside from the technical aspects of how Transport Layer Security works, the other facet of securely exchanging data lies in how well we apply that security. For example, if we allow a user to submit an application’s login form data over HTTP we must then accept that an MitM is completely capable of intercepting that data and recording the user’s login data for future use. If we allow pages loaded over HTTPS to, in turn, load non-HTTPS resources then we must accept that a MitM has a vehicle with which to inject Cross-Site Scripting attacks to turn the user’s browser into a pre-programmed weapon that will operate over the browser’s HTTPS connection tranparently.

In judging the security quality of any implementation, we therefore have some very obvious measures drawn from the four goals I earlier mentioned:

  • Encryption: Does the implementation use a strong security standard and cipher suite?
  • Identity: Does the implementation verify the server’s identity correctly and completely?
  • Data Tampering: Does the implementation fully protect user data for the duration of the user’s session?
  • Replay Attacks: Does the implementation contain a method of preventing an attacker from recording requests and repetitively send them to the server to repeat a known action or effect?

These questions are your core knowledge for this entire chapter. I will go into far more detail over the course of the chapter, but everything boils down to asking those questions and identifying the vulnerabilities where they fail to hold true.

A second core understanding is what user data must be secured. Credit card details, personally identifiable information and passwords are obviously in need of securing. However, what about the user’s session ID? If we protect passwords but fail to protect the session ID, an attacker is still fully capable of stealing the session cookie while in transit and performing a Session Hijacking attack to impersonate the user on their own PC. Protecting login forms alone is NEVER sufficient to protect a user’s account or personal information. The best security is obtained by restricting the user session to HTTPS from the time they submit a login form to the time they end their session.

You should now understanding why this chapter uses the phrase “insufficient”. The problem in implementing SSL/TLS lies not in failing to use it, but failing to use it to a sufficient degree that user security is maximised.

This chapter covers the issue of Insufficient Transport Layer Security from three angles.

  • Between a server-side application and a third-party server.
  • Between a client and the server-side application.
  • Between a client and the server-side application using custom defenses.

The first addresses the task of ensuring that our web applications securely connect to other parties. Transport Layer Security is commonly used for web service APIs and many other sources of input required by the application.

The second addresses the interactions of a user with our web applications using a browser or some other client application. In this instance, we are the ones exposing a secured URL and we need to ensure that that security is implemented correctly and is not at risk from being bypassed.

The third is a curious oddity. Since SSL/TLS has a reputation for not being correctly implemented by programmers, there have been numerous approaches developed to secure a connection without the use of the established SSL/TLS standards. An example is OAuth’s reliance on signed requests which do not require SSL/TLS but offer some of the defences of those standards (notably, encryption of the request data is omitted so it’s not perfect but a better option than a misconfigured SSL/TLS library).

Before we get down to those specific categories, let’s first take a look at Transport Layer Security in general as there is some important basic knowledge to be aware of before we get down into nuts and bolts with PHP.

Definitions & Basic Vulnerabilities

Transport Layer Security is a generic title for securing the connection between two parties using encryption, identity verification and so on. Most readers will be familiar with the three common abbreviations used in this topic: HTTPS, SSL, TLS. Let’s briefly define each so everyone understands how they are related.

SSL/TLS From PHP (Server to Server)

As much as I love PHP as a programming language, the briefest survey of popular open source libraries makes it very clear that Transport Layer Security related vulnerabilities are extremely common and, by extension, are tolerated by the PHP community for absolutely no good reason other than it’s easier to subject users to privacy-invading security violations than fix the underlying problem. This is backed up by PHP itself suffering from a very poor implementation of SSL/TLS in PHP Streams which are used by everything from socket based HTTP clients to the file_get_contents() and other filesystem functions. This shortcoming is then exacerbated by the fact that the PHP library makes no effort to discuss the security implications of SSL/TLS failures.

If you take nothing else from this section, my advice is to make sure that all HTTPS requests are performed using the CURL extension for PHP. This extension is configured to be secure by default and is backed up, in terms of expert peer review, by its large user base outside of PHP. Take this one simple step towards greater security and you will not regret it. A more ideal solution would be for PHP’s internal developers to wake up and apply the Secure By Default principle to its built-in SSL/TLS support.

My introduction to SSL/TLS in PHP is obviously very harsh. Transport Layer Security vulnerabilities are far more basic than most security issues and we are all familiar with the emphasis it receives in browsers. Our server-side applications are no less important in the chain of securing user data.Let’s examine SSL/TLS in PHP in more detail by looking in turn at PHP Streams and the superior CURL extension.

PHP Streams

For those who are not familiar with PHP’s Streams feature, it was introduced to generalise file, network and other operations which shared common functionality and uses. In order to tell a stream how to handle a specific protocol, there are “wrappers” allowing a Stream to represent a file, a HTTP request, a PHAR archive, a Data URI (RFC 2397) and so on. Opening a stream is simply a matter of calling a supporting file function with a relevant URL which indicates the wrapper and target resource to use.

file_get_contents('file:///tmp/file.ext');

Streams default to using a File Wrapper, so you don’t ordinarily need to use a file:// URL and can even use relative paths. This should be obvious since most filesystem functions such as file(), include(), require_once and file_get_contents() all accept stream references. So we can rewrite the above example as:

file_get_contents('/tmp/file.ext');

Besides files, and of relevance to our current topic of discussion, we can also do the following:

file_get_contents('http://www.example.com');

Since filesystem functions such as file_get_contents() support HTTP wrapped streams, they bake into PHP a very simple to access HTTP client if you don’t feel the need to expand into using a dedicated HTTP client library like Guzzle, Buzz or Zend Framework’s \Zend\Http\Client classes. In order for this to work, you’ll need to enable the php.ini file’s allow_url_fopen configuration option. This option is enabled by default.

Of course, the allow_url_fopen setting also carries a separate risk of enabling Remote File Execution, Access Control Bypass or Information Disclosure attacks. If an attacker can inject a remote URI of their choosing into a file function they could manipulate an application into executing, storing or displaying the fetched file including those from any untrusted remote source. It’s also worth bearing in mind that such file fetches would originate from localhost and thus be capable of bypassing access controls based on local server restrictions. As such, while allow_url_fopen is enabled by default, you should disable it without hesitation to maximise security.

Back to using PHP Streams as a simple HTTP client (which you now know is NOT recommended), things get interesting when you try the following:

$url = 'https://api.twitter.com/1/statuses/public_timeline.json';
$result = file_get_contents($url);

The above is a simple unauthenticated request to the (former) Twitter API 1.0 over HTTPS. It also has a serious flaw. PHP uses an SSL Context for requests made using the HTTPS (https://) and FTPS (ftps://) wrappers. The SSL Context offers a lot of settings for SSL/TLS and their default values are wholly insecure. The above example can be rewritten as follows to show how a default set of SSL Context options can be plugged into file_get_contents() as a parameter:

$url = 'https://api.twitter.com/1/statuses/public_timeline.json';
$contextOptions = array(
    'ssl' => array()
);
$sslContext = stream_context_create($contextOptions);
$result = file_get_contents($url, NULL, $sslContext);

As described earlier in this chapter, failing to securely configure SSL/TLS leaves the application open to a Man-In-The-Middle (MitM) attacks. PHP Streams are entirely insecure over SSL/TLS by default. So, let’s correct the above example to make it completely secure!

$url = 'https://api.twitter.com/1/statuses/public_timeline.json';
$contextOptions = array(
    'ssl' => array(
        'verify_peer'   => true,
        'cafile'        => '/etc/ssl/certs/ca-certificates.crt',
        'verify_depth'  => 5,
        'CN_match'      => 'api.twitter.com',
        'disable_compression' => true,
        'SNI_enabled'         => true,
        'ciphers'             => 'ALL!EXPORT!EXPORT40!EXPORT56!aNULL!LOW!RC4'
    )
);
$sslContext = stream_context_create($contextOptions);
$result = file_get_contents($url, NULL, $sslContext);

Now we have a secure example! If you contrast this with the earlier example, you’ll note that we had to set four options which were, by default, unset or disabled by PHP. Let’s examine each in turn to demystify their purpose.

  • verify_peer

Peer Verification is the act of verifying that the SSL Certificate presented by the Host we sent the HTTPS request to is valid. In order to be valid, the public certificate from the server must be signed by the private key of a trusted Certificate Authority (CA). This can be verified using the CA’s public key which will be included in the file set as the cafile option to the SSL Context we’re using. The certificate must also not have expired.

  • cafile

The cafile setting must point to a valid file containing the public keys of trusted CAs. This is not provided automatically by PHP so you need to have the keys in a concatenated certificate formatted file (usually a PEM or CRT file). If you’re having any difficulty locating a copy, you can download a copy which is parsed from Mozilla’s VCS from http://curl.haxx.se/ca/cacert.pem . Without this file, it is impossible to perform Peer Verification and the request will fail.

  • verify_depth

This setting sets the maximum allowed number of intermediate certificate issuers, i.e. the number of CA certificates which are allowed to be followed while verifying the initial client certificate.

  • CN_match

The previous three options focused on verifying the certificate presented by the server. They do not, however, tell us if the verified certificate is valid for the domain name or IP address we are requesting, i.e. the host part of the URL. To ensure that the certificate is tied to the current domain/IP, we need to perform Host Verification. In PHP, this requires setting CN_match in the SSL Context to the HTTP host value (including subdomain part if present!). PHP performs the matching internally so long as this option is set. Not performing this check would allow an MitM to present a valid certificate (which they can easily apply for on a domain under their control) and reuse it during an attack to ensure they are presenting a certificate signed by a trusted CA. However, such a certificate would only be valid for their domain - and not the one you are seeking to connect to. Setting the CN_match option will detect such certificate mismatches and cause the HTTPS request to fail.

While such a valid certificate used by an attacker would contain identity information specific to the attacker (a precondition of getting one!), please bear in mind that there are undoubtedly any number of valid CA-signed certificates, complete with matching private keys, available to a knowledgeable attacker. These may have been stolen from another company or slipped passed a trusted CA’s radar as happened in 2011 when DigiNotor notoriously (sorry, couldn’t resist) issued a certificate for google.com to an unknown party who went on to employ it in MitM attacks predominantly against Iranian users.

  • disable_compression

This option was introduced in PHP 5.4.13 and it serves as a defence against CRIME attacks and other padded oracle derived attacks such as BEAST. At the time of writing, it had been available for 10 months and locating a single example of its use in open source PHP was practically a quest in extreme patience.

  • SNI_enabled

Enables support for Server Name Indication where any single IP address may be configured to present multiple SSL certificates rather than be restricted to a single certificate for all websites or non-HTTP services hosted at that IP.

  • ciphers

This setting allows programmers to indicate which ciphers should or should not be used when establishing SSL/TLS connections. The default list of ciphers supplied by the openssl extension contain a number of unsafe ciphers which should be disabled unless absolutely necessary. The above cipher list, in a syntax accepted by openssl, was implemented by cURL during January 2014. An alternative cipher list has been suggested by Mozilla which may be better since it emphasises Perfect Forward Secrecy which is an emerging best practice approach. The Mozilla list is a bit longer:

ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:AES128:AES256:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK

Limitations

As described above, verifying that the certificate presented by a server is valid for the host in the URL that you’re using ensures that a MitM cannot simply present any valid certificate they can purchase or illegally obtain. This is an essential step, one of four, to ensuring your connection is absolutely secure.

The CN_match parameter exposed by the SSL Context in PHP’s HTTPS wrapper tells PHP to perform this matching exercise but it has a downside. At the time of writing, the matching used will only check the Common Name (CN) of the SSL certificate but ignore the equally valid Subject Alternative Names (SANs) field if defined by the certificate. An SAN lets you protect multiple domain names with a single SSL certificate so it’s extremely useful and supported by all modern browsers. Since PHP does not currently support SAN matching, connections over SSL/TLS to a domain secured using such a certificate will fail. SAN support for PHP will be introduced in PHP 5.6.

The CURL extension, on the other hand, supports SANs out of the box so it is far more reliable and should be used in preference to PHP’s built in HTTPS/FTPS wrappers. Using PHP Streams with this issue introduces a greater risk of erroneous behaviour which in turn would tempt impatient programmers to disable host verification altogether which is the very last thing we want to see.

SSL Context in PHP Sockets

Many HTTP clients in PHP will offer both a CURL adapter and a default PHP Socket based adapter. The default choice for using sockets reflects the fact that CURL is an optional extension and may be disabled on any given server in the wild.

PHP Sockets use the same SSL Context resource as PHP Streams so it inherits all of the problems and limitations described earlier. This has the side-effect that many major HTTP clients are themselves, by default, likely to be unreliable and less safe than they should be. Such client libraries should, where possible, be configured to use their CURL adapter if available. You should also review such clients to ensure they are not disabling (or forgetting to enable) the correct approach to secure SSL/TLS.

Additional Risks?

CURL Extension

Unlike PHP Streams, the CURL extension is all about performing data transfers including its most commonly known capability for HTTP requests. Also unlike PHP Streams’ SSL context, CURL is configured by default to make requests securely over SSL/TLS. You don’t need to do anything special unless it was compiled without the location of a Certificate Authority cert bundle (e.g. a cacert.pem or ca-bundle.crt file containing the certs for trusted CAs).

Since it requires no special treatment, you can perform a similar Twitter API call to what we used earlier for SSL/TLS over a PHP Stream with a minimum of fuss and without worrying about missing options that will make it vulnerable to MitM attacks.

$url = 'https://api.twitter.com/1/statuses/public_timeline.json';
$req = curl_init($url);
curl_setopt($req, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($req);

This is why my recommendation to you is to prefer CURL for HTTPS requests. It’s secure by default whereas PHP Streams is most definitely not. If you feel comfortable setting up SSL context options, then feel free to use PHP Streams. Otherwise, just use CURL and avoid the headache. At the end of the day, CURL is safer, requires less code, and is less likely to suffer a human-error related failure in its SSL/TLS security.

At the time of writing, PHP 5.6 has reached an alpha1 release. The final release of PHP 5.6 will introduce more secure defaults for PHP streams and socket connections over SSL/TLS. These changes will not be backported to PHP 5.3, 5.4 or 5.5. As such, all programmers will need to implement secure default settings as a concious choice until such time as PHP 5.6 is a minimum requirement for their code.

Of course, if the CURL extension was enabled without the location of trusted certificate bundle being configured, the above example would still fail. For libraries intending to be publicly distributed, the programmer will need to follow a sane pattern which enforces secure behaviour:

$url = 'https://api.twitter.com/1/statuses/public_timeline.json';
$req = curl_init($url);
curl_setopt($req, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($req);

/**
 * Check if an error is an SSL failure and retry with bundled CA certs on
 * the assumption that local server has none configured for ext/curl.
 * Error 77 refers to CURLE_SSL_CACERT_BADFILE which is not defined as
 * as a constant in PHP's manual for some reason.
 */
$error = curl_errno($req);
if ($error == CURLE_SSL_PEER_CERTIFICATE || $error == CURLE_SSL_CACERT
|| $error == 77) {
    curl_setopt($req, CURLOPT_CAINFO, __DIR__ . '/cert-bundle.crt');
    $result = curl_exec($req);
}

/**
 * Any subsequent errors cannot be recovered from while remaining
 * secure. So do NOT be tempted to disable SSL and try again ;).
 */

The tricky part is obviously distributing the cert-bundle.crt or cafile.pem certificate bundle file (filename varies with source!). Given that any Certificate Authority’s certificate could be revoked at any time by most browsers should they suffer a breach in their security or peer review processes, it’s not really a great idea to allow a certificate file to remain stale for any lengthy period. Nonetheless, the most obvious solution is to distribute a copy of this file with the library or application requiring it.

If you cannot assure tight control over updating a distribute certificate bundle, or you just need a tool that can periodically run this check for you, you should consider using the PHP Sslurp tool: https://github.com/EvanDotPro/Sslurp.

SSL/TLS From Client (Client/Browser to Server)

So far, most of what we’ve discussed has been related to SSL/TLS connections established from a PHP web application to another server. Of course, there are also quite a few security concerns when our web application is the party exposing SSL/TLS support to client browsers and other applications. At this end of the process we run the risk of suffering security attacks arising from Insufficient Transport Layer Protection vulnerabilities.

This is actually quite basic if you think about it. Let’s say I create an online application which a secure login to protect the user’s password. The login form is served over HTTPS and the form is submitted over HTTPS. Mission accomplished. The user is then redirected to a HTTP URL to start using their account. Spot the problem?

When a Man-In-The-Middle (MitM) Attack is a concern, we should not simply protect the login form for users and then call it quits. Over HTTP, the user’s session cookie and all other data that they submit, and all other HTML markup that they receive, will not be secure. An MitM could steal the session cookie and impersonate the user, inject XSS into the received web pages to perform tasks as the user or manipulate their actions, and the MitM need never know the password to accomplish all of this.

Merely securing authentication with HTTPS will prevent direct password theft but does not prevent session hijacking, other forms of data theft and Cross-Site Scripting (XSS) attack injection. By limiting the protection offered by HTTPS to the user, we are performing insufficient transport layer protection. Our application’s users are STILL vulnerable to MitM attacks.