Linux 4.13 introduces support for nothing less than... TLS! [The 1600 LoC patch]() allows userspace to pass the kernel the encryption keys for an established connection, making encryption happen transparently inside the kernel. The only ciphersuite supported is AES-128-GCM as per [RFC 5288](), meaning it only supports TLS version 1.2. Most modern TLS connections on the Internet use that. The kernel only handles the record layer, that is, it only takes care of encrypting packets. Handshake, key exchange, certificate handling, alerts and renegotiation are left out of kernelspace. The userspace application, like OpenSSL, will do all that and then delegate to the kernel once the keys are established. Moreover, only encryption is supported, not decryption. This wasn't clear to me until I failed to find the `TLS_RX` constant. These limitations are very good to contain complexity and attack surface, but they mean that kTLS won't replace any userspace complexity as you still need a TLS library to do the handshake, for all other cipher suites, and for the receiving side of the connection. That makes kTLS purely a performance feature. [Keys are passed to the kernel with a `setsockopt(2)` call]() on the TCP socket. Once that call is made everything written to that socket is transparently encrypted. A TLS record is made for each `send(2)` call unless a flag is used to request buffering. Alerts and any other messages which are not application data are sent via `CMSG`. The main motivation seems to be to allow use of [`sendfile(2)`]() on TLS connections. `sendfile(2)` allows data to be transferred from a file descriptor (like a file) to another (like a TCP connection) without paying the price of a copy round-trip through user space. If the kernel is handling TLS, `sendfile(2)` can be used also for encrypted connections. [The original paper by Facebook]() claims a significant improvement in 99th percentile performance. I also saw a mention of using BPF filtering rules on plaintext data, which is clever. It seems all very sensibly executed, but I can't help being terrified nonetheless. First because even just the record layer of TLS 1.2 with AEAD has significant legacy baggage and attack surface, and secondly because any compatibility issue introduced by this code will add a dimension to the compatibility matrix. Moreover, the end goal seems to be to do TLS offloading on dedicated hardware managed by kernel drivers, and poorly-implemented hard-to-update hardware is exactly [why TLS 1.3 hasn't been deployed yet](). > That said hold my mate, I'm adding support to crypto/tls. > > -- Filippo Valsorda (@FiloSottile) [4 September 2017]() So yeah, [not a huge fan](). But of course, it wouldn't be me if I didn't fork the Go `crypto/tls` package to work with it. To test kTLS I'll need a Linux VM running Linux 4.13. My favorite Linux distribution (Alpine) is not that cutting-edge with kernel versions, but one can of course trust Arch to have an up-to-date _linux-mainline_ package. _3 painful hours of learning Arch and compiling Linux from AUR follow..._ To enable TLS support make sure to compile with [`CONFIG_TLS`](). [root@localhost ~]# uname -a Linux localhost 4.13.0-mainline #1 SMP PREEMPT Wed Sep 6 01:04:02 BST 2017 x86_64 GNU/Linux I tried toying with `mkerrors.sh` to get the TLS constants into the `syscall` package, but eventually gave up and redefined them in `crypto/tls`. There is [a CL for `golang.org/x/sys/unix`]() to update to Linux 4.13, but it's compiled without kTLS. The easiest place to hook kTLS into `crypto/tls` seemed to be `(*halfConn).changeCipherSpec()`. That's where encryption is enabled for that half (receiving or sending) of the connection. However that happened before sending the Finished handshake message, which we would have to send with a `CMSG`. Instead, I added a hook at the end of both client and server handshake. The new function `(*Conn).enableApplicationDataEncryption()` checks that the cipher is a `*fixedNonceAEAD` (i.e. AES-GCM) with the right key length, and that the connection is a `*net.TCPConn`, and then invokes `kTLSEnable()` with all the key material. [kTLSEnable]() constructs the `tls_crypto_info` structure [as per the docs](), hoping Go won't add padding for alignment purposes. It then uses [`syscall.RawConn.Control`]() and `syscall.SetsockoptString` to make the `setsockopt` syscalls to pass it to the kernel. There's a lot of `unsafe` involved, but no need for cgo (I just redefine the types manually). One of the quirks of AES-GCM in TLS 1.2 is that [it has 4 bytes of implicit IV and 8 bytes of explicit IV (counter)](). Linux calls the explicit part `iv` and the implicit part... `salt`. Ok. I guess. For a moment I thought it would be enough to then switch the cipher internally to unencrypted, but while it's not documented, for `sendfile(2)` to work the data must be sent with no framing at all. [Even unencrypted TLS packets are framed](). So as a last hack I added a dummy `kTLSCipher` that strips the record header instead of encrypting the record before sending it on the wire. I ran a simple HTTPS web server with `net/http`, loaded a page on Chrome, and instead of causing a kernel panic... [root@localhost ~]# go run https_server.go 2017/09/06 16:21:39 kTLS: enabled 2017/09/06 16:21:39 kTLS: sent 33 bytes of plaintext to the kernel 2017/09/06 16:21:39 kTLS: enabled 2017/09/06 16:21:39 kTLS: dropped alert on the floor 2017/09/06 16:21:39 kTLS: sent 0 bytes of plaintext to the kernel 2017/09/06 16:21:39 kTLS: enabled 2017/09/06 16:21:39 kTLS: sent 33 bytes of plaintext to the kernel 2017/09/06 16:21:39 kTLS: sent 107 bytes of plaintext to the kernel 2017/09/06 16:21:39 kTLS: sent 37 bytes of plaintext to the kernel ![Browser loading kTLS](https://words.filippo.io/content/images/2017/09/Untitled.png) 💥 Future work includes getting the [`io.WriterTo`]()-based `sendfile(2)` interface upgrade to work across the `*tls.Conn`, and running the various TLS test suites and fuzzers against kTLS. You can see the code [here](). Please don't use it. But maybe [follow me on Twitter]()! > Wrote about kernel TLS support in Linux 4.13 and gave in to the temptation of making it work with Go's crypto/tls 🔥 [pic.twitter.com/3IVe4lzoDl]() > > -- Filippo Valsorda (@FiloSottile) [6 September 2017]()

Playing with kernel TLS in Linux 4.13 and Go