The content of the article
Very often, the study of the internal operation of application programs can be reduced to the study of their traffic, and most often it is transmitted over the HTTP family of protocols. But each application in its own way can be protected from wiretapping. Today we will try to find out what all approaches to protection have in common and how this common part can be used to create a universal method for intercepting HTTPS.
A bit about HTTP and what does OpenSSL have to do with it
It's no secret that HTTP traffic prevails on the Internet. The popularity and popularity of the HTTP protocol has led to its widespread use even where at first it might seem ridiculous and redundant – for example, in mobile and desktop applications.
During its existence, HTTP has grown into a bunch of different features and bells and whistles that eliminate the drawbacks of its original "web" version: Cookies to eliminate Stateless, Keep-Alive and Long polling to simulate continuity of connection, the concept of REST, binary HTTP / 2 with universal compression and much more . Looking ahead, I note that HTTP / 2 was generally designed for use with TLS 1.2+.
At a certain stage in the development of the protocol, it became clear that HTTP is becoming a universal way for exchanging data in a client-server model. The developers secretly accepted this and began to use it when creating new applications.
In parallel with HTTP, SSL was developing, and their synthesis – HTTPS – gradually grabbed the share of traffic on the global Internet, until at one fine moment at the end of January 2017 did not exceed half. Now its share is striving for 80%. Many saw the point in encrypting data, but those who did not have time, began to rush including browser developers. As a result, everyone again accepted this (which is good!) And began to apply it everywhere in their applications, many of which already worked on the usual HTTP.
The development of SSL, and subsequently TLS (SSL 3.0+), was largely determined by the development of the openSSL open source project. He slowly but surely absorbed more and more RFC specifications. As you know, nobody likes to invent bicycles, so the OpenSSL library has become the de facto standard when implementing secure transport for HTTP, and now its traces can be found in countless software projects.
How classic HTTPS works
Using HTTPS helped protect against MITM attacks on the user, but not against his own mistakes. Just look at the classic SSL protocol scheme, and it becomes clear that the confidentiality here rests solely on the server certificate.
Receiving a certificate from the server at the third step of the handshake, the client decides whether to trust the server or not (is it the one he claims to be?). The client does this not without outside help, which, as a result, leads to a significant simplification of traffic analysis.
For verification, certificate authorities are used, which are root and intermediate. There are few public root centers, and every SSL / TLS client knows about all of them. And there can be a lot of intermediate centers – their only difference from root centers is that the certificates they issue are signed by the private key of the superior CA. For example, if you look at the google.com certification path in the figure below, the root certificate will be issued by GlobalSign, and the intermediate one will be Google Internet Authority.
When receiving a certificate from the server, the client can always find out who signed it and verify this with the help of the attached public keys. And so on down to the root center. If there is not a single trusted certificate encountered on the way to the root certificate, the client completes the handshake procedure without transferring any useful data to the server (after all, it is probably not the one who claims to be). This process is called certificate chain validation.
For developers, difficulties in hiding the traffic of their application begin at the moment when the client has the opportunity to add their own trusted certificates. By default, the source of such certificates is a folder or group of folders (depending on the operating system) in the local file system, into which OS developers placed certificates of trusted centers at the assembly stage. Let's try to make sure of this in practice.
Explore the OpenSSL principles of operation in an experimental application
The OpenSSL library “in a vacuum” does not trust anyone by default, and in order to initiate the process of this trust, the library needs to be informed about who, in fact, we intend to trust. Looking at the classic TLS client example in wiki.openssl, then you can notice the loading of trusted certificates there just before making an HTTP request:
...ctx = SSL_CTX_new(method);if(!(ctx != NULL)) handleFailure();SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);SSL_CTX_set_verify_depth(ctx, 4);const long flags = SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION;SSL_CTX_set_options(ctx, flags);// загрузка доверенных сертификатовres = SSL_CTX_load_verify_locations(ctx, "random-org-chain.pem", NULL);if(!(1 == res)) handleFailure();...
In addition to the function
SSL_CTX_load_verify_locations still exists several similar ways to download verified certificates inside OpenSSL. Surely the developers who wrote their TLS clients did not hesitate to spy on such publicly available examples.
Let's try to track the behavior of the presented classic TLS client on some real application. To get started, let's take something simple that could be related to TLS — for example, a library Okhttp. It provides HTTP / S communications in countless modern Android apps.
I note in advance that OkHttp is written in Java and runs on the JVM, so it is not interesting in itself, as it is a kind of wrapper for a more interesting component – the low-level implementation of OpenSSL for Android. So we’ll do it.
We are preparing a test application
To start the study, you need to get an application that uses OkHttp logic for secure requests. The easiest way to create it from scratch, taking as a basis reference examples of code from the Internet. Just like hundreds and thousands of other developers do it. 🙂
We will include two buttons in the test application interface, by clicking on which the following will happen:
// Кнопка "Do just HTTPS!"val okHttpClient = OkHttpClient.Builder().build()val request = Request.Builder().url("https://google.com").get().build();okHttpClient.newCall(request).execute()...// Кнопка "Do Https with pin!"val certificatePinner = CertificatePinner.Builder() .add("google.com","sha256/f8NnEFZxQ4ExFOhSN7EiFWtiudZQVD2oY60uauV/n78=") .build()val okHttpClient = OkHttpClient.Builder() .certificatePinner(certificatePinner) .build()val request = Request.Builder().url("https://google.com").get().build();okHttpClient.newCall(request).execute()
A completely standard code, wandering around Ctrl-C and Ctrl-V from one project to another, while undergoing minor changes. So, we launch the newly made application and immediately look at the markup of its memory.
Continuation is available only to participants
Materials from the latest issues become available separately only two months after publication. To continue reading, you must become a member of the Xakep.ru community.
Join the Xakep.ru Community!
Membership in the community during the specified period will open you access to ALL Hacker materials, increase your personal cumulative discount and allow you to accumulate a professional Xakep Score!
I am already a member of Xakep.ru