Exploring Programmable Proxy Pipy through TLS Handshake

5 min readSep 21, 2023

Transport Layer Security (TLS) is an encryption protocol designed to provide communication security for computer networks. This protocol is widely used in applications such as email, instant messaging, and IP voice, but its most common application is HTTPS. TLS operates on layer 6, the presentation layer, of the OSI model and can be divided into two layers: the TLS record and the TLS handshake protocol.

The encrypted transmission of TLS is used in many application scenarios of Pipy, such as HTTPS/TLS proxy, service mesh with mTLS, encrypted tunnels, etc.

Today, we will introduce how to use programming to complete the TLS handshake and briefly understand the working mechanism of Pipy.

What is a TLS Handshake

A TLS handshake is a process to initiate a TLS communication session. During the TLS handshake, both parties exchange messages to mutually acknowledge, verify, agree on the encryption algorithms they will use, and agree on session keys.

When does TLS occur

The TLS handshake takes place after a TCP connection is opened through a TCP handshake.

Handling Process of TLS

The handshake process of TLS (Transport Layer Security) is a complex process that includes several key steps to ensure secure data transmission. Here is a detailed explanation of the TLS handshake process:

1. Communication between the client and the server starts

ClientHello: The client sends a “ClientHello” message to the server, containing information such as the client’s TLS version, supported encryption algorithms, and supported compression methods.

2. Server’s Response

ServerHello: The server responds with a “ServerHello” message, which contains information such as the protocol version, encryption algorithm, and compression method decided by the server.
Server Certificate: The server sends its certificate to the client. This certificate contains the server’s public key and some identity information.
Server Key Exchange Message: (Optional) If necessary, the server can also send a key exchange message to the client.

3. Client Verification and Key Generation

Certificate Verification: The client verifies whether the server’s certificate was issued by a trusted certificate authority and whether the certificate is valid.
Client Key Exchange Message: The client generates a string of random data (pre-master key) and encrypts this data using the server’s public key, then sends it to the server.

4. Key Calculation and Confirmation

Decryption of the Pre-Master Key: The server uses its private key to decrypt the received pre-master key.
Session Key Generation: Both the client and the server use the pre-master key to generate a session key, which will be used for subsequent data encryption.

5. Handshake Completion

Change Cipher Spec Message: Both the client and the server send a change cipher spec message, indicating that subsequent communication will use the previously negotiated encryption parameters and keys.
Handshake Completion Message: Both the client and the server send a handshake completion message. At this point, the handshake process is complete, and subsequent communication will be encrypted using the negotiated encryption algorithm and keys.

6. Encrypted Data Transmission

After the handshake process is completed, the client and server can begin encrypted data transmission.

This handshake process ensures that the client and server can securely exchange keys and encryption parameters and establish a secure encryption channel to protect the privacy and integrity of the data. The process also verifies the server’s identity to prevent man-in-the-middle attacks.

Implementing TLS with Pipy

With a simple PipyJS script, we can implement encrypted transmission via TLS.

((
  certs = {
    "example.com": {
      cert: new crypto.Certificate("-----BEGIN CERTIFICATE-----\nMIICsjCCAZoCFEOlbMsCFG+UuQtmQX1HJIVcNc3RMA0GCSqGSIb3DQEBCwUAMBUx\nEzARBgNVBAMMCmZsb21lc2guaW8wHhcNMjMwOTE4MjA0MTIzWhcNMjQwOTE3MjA0\nMTIzWjAWMRQwEgYDVQQDDAtleGFtcGxlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD\nggEPADCCAQoCggEBAK9FMoKKep5iwXJAYMAs9TC9MyZJO6eeDGiwuRY0NBiu10yn\nXzP6vqrdkW+OeQlwdn3J6g2mFyhwcgJ30Yd+npRK4L+IYSZsW96Cp78TwkzI+MS9\nJJWfdvUbO61cTjCvHe12VvoKxZXf/b2dR44NLXRiSqC8L+fZbM6G9eIgae9CCn0G\ng9rQdgs886bGqcmptCyiA9OBtvGabLwlyYWE77Z5E9aioFsmAaleplOlbbsNB5yk\npidldLKqPDUxnQe5WkWkrUv3pAtcHILUSgWFE3Uz1ISC3b+2FkDNQ7bALSG8bxf0\n0DTapaAEKxk/0+D2s7yLIEtO6065Kg6m6V9OpLkCAwEAATANBgkqhkiG9w0BAQsF\nAAOCAQEAPWIEDl1JW7/1hZCCYZJWDhaNf2edqM+wNm8Jpm+UqU88ug0h/x3ZINVW\niqEbvImqKsskXIp5JHnmVp/5JbbRx4iILgeHkuIpdbw2iJyD33cG89VPw3g9Itqf\nO01Kg1UAxWCZyweB1Iq8vADf0N7fXEJ3z19Hq5ZBFn14dUjXc7/30UuQ5k5I80gX\nka+cZTsHbzqCbYapZvUJI1UFmWSYjjC461aZWbSKNdpBiyrjK6P3h1FDQmswXWlG\nL8JaV5UIfGx/6sDkWgwadp9K6m1k+E0UJd7yEZW6JNfk8SNOTNlBW/rApyl9AQ4b\nUtiOGw191GWDt5HhTBxmyM9F/NTFEw==\n-----END CERTIFICATE-----\n"),
      key: new crypto.PrivateKey("-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCvRTKCinqeYsFy\nQGDALPUwvTMmSTunngxosLkWNDQYrtdMp18z+r6q3ZFvjnkJcHZ9yeoNphcocHIC\nd9GHfp6USuC/iGEmbFvegqe/E8JMyPjEvSSVn3b1GzutXE4wrx3tdlb6CsWV3/29\nnUeODS10YkqgvC/n2WzOhvXiIGnvQgp9BoPa0HYLPPOmxqnJqbQsogPTgbbxmmy8\nJcmFhO+2eRPWoqBbJgGpXqZTpW27DQecpKYnZXSyqjw1MZ0HuVpFpK1L96QLXByC\n1EoFhRN1M9SEgt2/thZAzUO2wC0hvG8X9NA02qWgBCsZP9Pg9rO8iyBLTutOuSoO\npulfTqS5AgMBAAECggEAEJDiG56Kb7/2eB2yR1e4e7krms8fLqeXP/88Ged96RCq\n1C7u+6hpLxiEGjQwJ9PpDy3vdlzKzDgHPp55P8JDANHW+PjN8ztuBNOlQT/VLzwW\nRJolWWhaGQP67wF4zncuxUwDH90MHISwT09h/QDX2VaU+9OYVOSjDnOJIqyyS+Uc\nw4fB+K55gR11wd1TnH8Hq20Rc3uNMapUjfluJ76hnkVBWBDBfeEYeJCpPU9rbyCM\nE+3Eu6PWrtljBC379IxF5YCFOEuZcBqKz6kWhqnjjhDnIMeEDyeHGW+k71cNyttZ\n9wjTXjNiDvNsueUEQJWdcGT+RfLkM8Niu9Qq+RIM1QKBgQDtuzibw8Q6/QYfTzO/\nKochbYiULA6NAqxSnuVRqgFabvkH7bTa/K90WWVnNHAdGVAck1b9kpdYzD21FhMT\nT2QVRQpnYWobL/MH58VJcqqCpfmeaBaibnh/wPD2XJX8vlgrD0RLWytokHhi9BUE\n/S1Pcz8hfkQYPNZ65765w14AuwKBgQC8vTWJZyod8y/kQPbF5uxJ72GkM192srwV\n/xU70HXuCtX0vTacADl7bm1Nzu+FusUQpx044dEq7cqOBwnsZ0Bvgr3JElhz6n78\n5w0mBvRtkn9b6WzDCeXWfZkdaYoQkb34B4b3szRfh0Donam8BPN5/zgTO7LGj806\nZ6+aK7EjGwKBgBbt1NLpOdb6qomAiRwqDOiDeQXZjfm5xUcevj13KTRAudIy8hug\n/Yc6TBZ9L0cPNvPanixM5D4TTOxXWbWmFaAbjZpVAffnUwhzKUshPSLmwUGvBmfa\nQdMIl/UbDvhWn4hfq5WdEH3AWWwp4JEfniwokHE5jXXNAF7QVUvzdPCbAoGACYMj\nupwXdFD4XfGkPk8oI4XMDwGD0zCo6BbmFBCqkOe85svOf8hHHWBwY9aFHeFO40r4\n3xAKgbZgWg4iwuZlSfl15Tdme9kas1ZVxE/fa9JRVumJ0L6j9c54tBHIopMl1uVS\nOROwFQx1CgRANLivKLvjMJz2oqlGt6XYJNYE5HcCgYAVYBQW9MSI2cjxwJ1MM2wp\nDyKW5QC5e8/6rJs4pmkQjYbBsa1CHXkS8RisVidaePahbh3bIV61c9zt/bXMi5hf\naheVtJxwiXjPnEF+XWS1FiXRCP6vh9zY1w1FnYVGBHgc7vEjZCmSQyUYzrtNUNcT\nnDYMtA8zAB7Mx4jigl7M/A==\n-----END PRIVATE KEY-----\n"),
    }
  }
) =>
  pipy()
    //tls proxy
    .listen(8443)
    .acceptTLS({
      certificate: sni => certs[sni] || null,
      verify: () => true,
    }).to($ => $.connect('127.0.0.1:8081'))
    //http upstream
    .listen(8081)
    .serveHTTP(new Message('Hello, Pipy!'))
)()

In this script, we use a self-invoking function, calling the Pipy API pipy() to create a JS object Configuration. This object is understood as the configuration that Pipy will use. Configuration provides a rich set of APIs to create pipelines and filters.

In the script, two port pipelines are created to listen to different ports using the listen API:

8081: Returns an HTTP response using the filter serveHTTP with the content "Hello, Pipy!"
8443: A TLS proxy that proxies requests to the upstream listening on 8081. Here the server-side process of completing the TLS handshake is needed: receive the
client’s “ClientHello” request, return the “ServerHello” containing the server certificate, encryption algorithm, etc., using the filter acceptTLS. After completing the handshake, the request is decrypted before being forwarded to the upstream.

Besides port pipelines, there is also an anonymous sub-pipeline, which is the part that forwards the request to the upstream, using the connect filter.

Additionally, there are certificates needed for the proxy, which are configured in the script.

Pipy’s Implementation

The mentioned pipelines should be precisely referred to as pipeline layouts in the configuration, which can be understood as the definition of the pipeline. It specifies which filters are used in the pipeline and the order of the filters. The actual “pipeline” is an object that processes data in operation (similar to the instantiation of the pipeline layout).

When processing multiple concurrent requests, multiple “pipelines” will be instantiated using the same “pipeline layout” as a template. Each pipeline has its independent state, and each request can be processed in an independent pipeline.

We use JS programming to orchestrate (configure) filters and pipelines and save them in script files. When Pipy starts, it loads the defined script, uses its built-in JS engine to parse it into C++ objects, obtaining a series of pipeline layout objects.

After loading, when it starts running, pipelines (including the filters in them) will be created through pipeline layouts. For example, the port pipelines in this example will start listening to ports, waiting for network data.

Upon receiving network data, Pipy will package it as an event Event and hand it over to the pipeline for processing. After a series of pipeline and filter processing, the Event output from the pipeline will be unpacked and written back to the network.

Both pipelines and filters receive and return events, meaning both inputs and outputs are events.

Conclusion

In this article, we implemented a TLS proxy with less than 20 lines of code, achieving TLS encrypted data transmission. With a simple configuration, support for multiple domains can be added — such is the magic of programmable proxies.

Supported by Pipy’s rich API, a wide range of general and customized scenarios can be flexibly supported, realizing more functions and supporting more protocols.