How can I provide a custom TLS CACert bundle with Arduino on ESP32?

How to use CA Cert Bundle in Arduino ESP32?

The process for using the ESP32 setCACertBundle() was described in Arduino Library README.md of WiFiClientSecure Library for Arduino ESP32 framework release 2.x. For some reason it has been reduced to a simple vague paragraph in release 3.x README.md when the WiFiCleintSecure library is re-factored to NetworkClientSecure library. Espressif lately like to do that and sometime break the backward compatibility or remove something useful from its Github without explanation.

Create a CA Cert Bundle

In order to use a cert bundle, first you would need to generate a cert bundle. You could generate it yourself, or get CA certificates extracted from Mozilla from here. The download Mozilla CA certificate store cacert.pem is in PEM format and is around 200KB uncompressed.

Esp-idf has a python utility for generating a binary version of CA bundle which is only around 64kB. Download the gen_crt_bundle.py (you might need to install python package cryptography with pip3 install cryptography before you can run the python script).

Run the python script with the following command line to generate the binary file:

python3 gen_crt_bundle.py -i cacert.pem

This should generated a file called x509_crt_bundle, make a directory in your project directory and move the created bundle into the directory.

mkdir data data/cert
mv x509_crt_bundle data/cert/

Setup PlatformIO

The code that I'm going to shown work for PlatformIO, but does not work for Arduino IDE (I don't know why and in fact I don't really use Arduino IDE these days so I never spent the time to find out why).

Assuming you had PlatfromIO installed and have created an Arduino project. Add build flag in platformio.ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
board_build.embed_files = data/cert/x509_crt_bundle
monitor_speed = 115200

Call setCACertBundle() before making an secure connection

The code for using the setCACertBundle() is actually quite simple and similar to the usage of WiFiClientSecure example, with only two differences:

1. add extern const uint8_t rootca_crt_bundle_start[] asm("_binary_data_cert_x509_crt_bundle_bin_start"); to your sketch;
2. call client.setCACertBundle(rootca_crt_bundle_start); before making a client connection.

Here is the complete example sketch that I used for testing, it access the httpbin.org test site (you could try other sites) and httpbin server would return a response in json format.

#include <WiFiClientSecure.h>

const char* ssid     = "wifi ssid";
const char* password = "wifi password";

const char*  server = "httpbin.org";

extern const uint8_t rootca_crt_bundle_start[] asm("_binary_data_cert_x509_crt_bundle_bin_start");

WiFiClientSecure client;

void setup() {
  Serial.begin(115200);
  delay(100);

  Serial.print("Connecting to SSID: ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(1000);
  }
  Serial.println("Connected");

  // client.setCACert(ISRG_Root_X1_CA);
  client.setCACertBundle(rootca_crt_bundle_start);
  // client.setCertificate(test_client_cert); // for client verification
  // client.setPrivateKey(test_client_key);  // for client verification

  Serial.println("\nStarting connection to server...");
  if (!client.connect(server, 443))
    Serial.println("Connection failed!");
  else {
    Serial.println("Connected to server!");
    // Make a HTTP request:
    client.printf("GET https://%s/get HTTP/1.1\n", server);
    client.printf("Host: %s\n", server);
    client.println("Connection: close");
    client.println();

    while (client.connected()) {
      String line = client.readStringUntil('\n');
      if (line == "\r") {
        Serial.println("headers received");
        break;
      }
    }
    // read the response body
    while (client.available()) {
      char c = client.read();
      Serial.write(c);
    }

    client.stop();
  }
}

void loop() {
  // do nothing
}

The screen capture shown the overall directory structure of platformIO setup as well as the response received from the test server (httpbin.org).

For reference, here is the system setting of my PlatformIO show during the compilation.

PLATFORM: Espressif 32 (6.9.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.20017.0 (2.0.17) 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - tool-mkfatfs @ 2.0.1 
 - tool-mklittlefs @ 1.203.210628 (2.3) 
 - tool-mkspiffs @ 2.230.0 (2.30) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5