MQTT-TLS with Google IoT Core


#1

now I checked Photon could connect to the Google IoT Core with MQTT-TLS.
I hope this library is helpful for IoT Photon developers.

here is sample source code.

#include "MQTT-TLS.h"

void callback(char* topic, byte* payload, unsigned int length);

// use GlobalSign Root CA - R2
#define GOOGLE_IOT_CORE_CA_PEM                                              \
"-----BEGIN CERTIFICATE----- \r\n"                                      \
"MIIDujCCAqKgAwIBAgILBAAAAAABD4Ym5g0wDQYJKoZIhvcNAQEFBQAwTDEgMB4G\r\n"  \
"A1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjIxEzARBgNVBAoTCkdsb2JhbFNp\r\n"  \
"Z24xEzARBgNVBAMTCkdsb2JhbFNpZ24wHhcNMDYxMjE1MDgwMDAwWhcNMjExMjE1\r\n"  \
"MDgwMDAwWjBMMSAwHgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEG\r\n"  \
"A1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjCCASIwDQYJKoZI\r\n"  \
"hvcNAQEBBQADggEPADCCAQoCggEBAKbPJA6+Lm8omUVCxKs+IVSbC9N/hHD6ErPL\r\n"  \
"v4dfxn+G07IwXNb9rfF73OX4YJYJkhD10FPe+3t+c4isUoh7SqbKSaZeqKeMWhG8\r\n"  \
"eoLrvozps6yWJQeXSpkqBy+0Hne/ig+1AnwblrjFuTosvNYSuetZfeLQBoZfXklq\r\n"  \
"tTleiDTsvHgMCJiEbKjNS7SgfQx5TfC4LcshytVsW33hoCmEofnTlEnLJGKRILzd\r\n"  \
"C9XZzPnqJworc5HGnRusyMvo4KD0L5CLTfuwNhv2GXqF4G3yYROIXJ/gkwpRl4pa\r\n"  \
"zq+r1feqCapgvdzZX99yqWATXgAByUr6P6TqBwMhAo6CygPCm48CAwEAAaOBnDCB\r\n"  \
"mTAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUm+IH\r\n"  \
"V2ccHsBqBt5ZtJot39wZhi4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDovL2NybC5n\r\n"  \
"bG9iYWxzaWduLm5ldC9yb290LXIyLmNybDAfBgNVHSMEGDAWgBSb4gdXZxwewGoG\r\n"  \
"3lm0mi3f3BmGLjANBgkqhkiG9w0BAQUFAAOCAQEAmYFThxxol4aR7OBKuEQLq4Gs\r\n"  \
"J0/WwbgcQ3izDJr86iw8bmEbTUsp9Z8FHSbBuOmDAGJFtqkIk7mpM0sYmsL4h4hO\r\n"  \
"291xNBrBVNpGP+DTKqttVCL1OmLNIG+6KYnX3ZHu01yiPqFbQfXf5WRDLenVOavS\r\n"  \
"ot+3i9DAgBkcRcAtjOj4LaR0VknFBbVPFd5uRHg5h6h+u/N5GJG79G+dwfCMNYxd\r\n"  \
"AfvDbbnvRG15RjF+Cv6pgsH/76tuIMRQyV+dTZsXjAzlAcmgQWpzU/qlULRuJQ/7\r\n"  \
"TBj0/VLZjmmx6BEP3ojY+x1J96relc8geMJgEtslQIxq/H5COEBkEveegeGTLg==\r\n"  \
"-----END CERTIFICATE----- "

const char googleIotCoreCaPem[] = GOOGLE_IOT_CORE_CA_PEM;
MQTT client("mqtt.googleapis.com", 8883, callback, 768);  // set max packet size to 768 for JWT password.

// recieve message
void callback(char* topic, byte* payload, unsigned int length) {
    char p[length + 1];
    memcpy(p, payload, length);
    p[length] = NULL;
    String message(p);

    if (message.equals("RED"))
        RGB.color(255, 0, 0);
    else if (message.equals("GREEN"))
        RGB.color(0, 255, 0);
    else if (message.equals("BLUE"))
        RGB.color(0, 0, 255);
    else
        RGB.color(255, 255, 255);
    delay(1000);
}

#define ONE_DAY_MILLIS (24 * 60 * 60 * 1000)
unsigned long lastSync = millis();
void setup() {
    if (millis() - lastSync > ONE_DAY_MILLIS) {
        Particle.syncTime();
        lastSync = millis();
    }
    RGB.control(true);

    // connect to Google IoT Core
    client.enableTls(googleIotCoreCaPem, sizeof(googleIotCoreCaPem));
    client.connect("projects/{your project-id}/locations/{cloud-region}/registries/{registry-id}/devices/{device-id}",
                   "unused",
                   "your JWT text data");
                   
    // publish/subscribe to Google IoT Core
    if (client.isConnected()) {
        Serial.println("client connected");
        client.publish("/devices/device1/events/function", "from particle phton publish message");
        client.subscribe("/devices/device1/config");
    }
}

void loop() {
    if (client.isConnected())
        client.loop();
    delay(1000);
}

Regards,
Hirotaka


#2

Hi Hirotaka,

Thank you for your wonderful guide. Could I ask you to help with the JWT data section, please? So far I understand that JWT data is encrypted by the private key and used to handshake with the Iot Cloud, but I don’t understand how the string is generated.

Kind Regards
Finn


#3

Hi, if you could use golang, check this site(makejwt command).


#4

Hello,
Thank you for your work, I really appreciate. I am planning to test your library, but I cannot generate JWT token for Google IoT Core. I cannot find any suitable library which will be using mbedtls. I want to generate JWT one per 24 hours on device, any suggestion? What is important I am trying to use this lib on Argon and Xenon devices.


#5

I also had the same problem and what I ended up doing was creating an App Engine endpoint and calling that to generate the JWT to send back to the device.

I should note I did this with an Adafruit huzzah esp32 not any particle devices


#6

hi, @Toyokomi @tyczj

here is sample golang make JWT token source code.

package main

import (
  "flag"
  "io/ioutil"
  "log"
  "time"

  jwt "github.com/dgrijalva/jwt-go"
)

var (
  deviceID = flag.String("device", "", "Cloud IoT Core Device ID")
  projectID  = flag.String("project", "", "GCP Project ID")
  registryID = flag.String("registry", "", "Cloud IoT Registry ID (short form)")
  region     = flag.String("region", "", "GCP Region")
  certsCA    = flag.String("ca_certs", "", "Download https://pki.google.com/roots.pem")
  privateKey = flag.String("private_key", "", "Path to private key file")
)

func main() {
  flag.Parse()

  token := jwt.New(jwt.SigningMethodRS256)
  token.Claims = jwt.StandardClaims{
    Audience:  *projectID,
    IssuedAt:  time.Now().Unix(),
    ExpiresAt: time.Now().Add(24 * time.Hour).Unix(),
  }

  keyBytes, _ := ioutil.ReadFile(*privateKey)
  key, _ := jwt.ParseRSAPrivateKeyFromPEM(keyBytes)
  tokenString, _ := token.SignedString(key)

  log.Printf("[main] jwt token: %v\n", tokenString)
}

And here is sample result.

openssl req -x509 -newkey rsa:2048 -keyout rsa_private.pem -nodes  -out rsa_cert.pem -subj "/CN=unused"
go run makejwt.go  --device=[your device id] --project=[your project id] --registry=[your registory] --region=[your region] --ca_certs=./roots.pem --private_key=rsa_private.pem 
2019/01/07 13:20:35 [main] jwt token: eyJhbGciOiJS....

Photon sample source code is same on this topic my source code.


#7

@Toyokomi @hirotakaster I’ve attempted to modify the mbedtls library to create a JWT token locally on an electron based on a process I found here:
https://www.esp32.com/viewtopic.php?t=6828&p=35458

A reference code sample was supplied in that article and I’ll link it here: https://github.com/nkolban/esp32-snippets/blob/master/cloud/GCP/JWT/main.cpp

It is possible to port ‘createGCPJWT()’ for the electron and to modify the mbedtls library by removing calls to fopen and printf etc. so that it to compiles, flashes, and begins execution but presently I’m running out of memory prior to delivering the JWT. (This may be due to the sample using MBEDTLS_MD_SHA256 in addition to x509 now… could x509 be used to sign the token Hirotaka??? This is all very new to me.)

My next step would be to instantiate the JWT creator with mbedtls support, build and save the token, then release the JWT creator and try to recover enough memory resources to allow myself to continue on to instantiate an MQTT object (and run my app code) after I generate the relatively small token. @hirotakaster Hirotaka, considering you have more experience modifying the library, I wonder if you might have more luck than me. If you’re interested in digging in I can share with you the very rough mbedtls modifications I’ve made so far. I’ll likely put this aside myself as it’s not a high priority at the moment.

Without token generation at the device I’m planning to do something similar to @tyczj and generate the token daily in the cloud, then share it via particle.publish/subscribe prior to invoking MQTT. That’s a big workaround to coordinate though.


#8

Hello,
Thank you for your reply, I have tried to import code from
https://github.com/nkolban/esp32-snippets/blob/master/cloud/GCP/JWT/main.cpp.
I have no problems to compiles files from MQTT-TLS lib, but there is a problem with using mbedtls_entropy_init(&entropy); which is causing error. Commenting this function allow me to compile user part. There is a linker problem, compile output:

...
/usr/local/gcc-arm-embedded/bin/../lib/gcc/arm-none-eabi/5.3.1/../../../../arm-none-eabi/lib/armv7e-m/fpu/libg_nano.a(lib_a-lseekr.o): In function `_lseek_r':
lseekr.c:(.text._lseek_r+0x10): undefined reference to `_lseek'
/usr/local/gcc-arm-embedded/bin/../lib/gcc/arm-none-eabi/5.3.1/../../../../arm-none-eabi/lib/armv7e-m/fpu/libg_nano.a(lib_a-readr.o): In function `_read_r':
readr.c:(.text._read_r+0x10): undefined reference to `_read'
collect2: error: ld returned 1 exit status 

I think this could problem with porting mbedTLS to Argon,Xenon, especially entropy.


#9

@ian.c
Thank you for your help. If you okay, could you please send me a pull request or issue about your modifications to my MQTT-TLS/TlsTcpClient github.

How to get the JWT token from server or create on the devices(with mbedTLS) have security and mbedTLS library size problem.

Now particle firmware include mbedTLS for DTLS that could not use for TLS(tcp).
Then I port mbedTLS(tcp) with MQTT simple implementation, unfortunately if application use MQTT-TLS it have two mbedTLS on the device as a result(this is waste of size). MQTT-TLS library have a JWT token create method like a makeJWT(…) is one of good idea, but MQTT library including JWT token create function is a little bit strange. I think it would be better to implement MQTT and JWT as separate library.(but…become waste of size…)

Download JWT token from server with trusted secure pub/sub or something method is good idea. As you write, generate JWT token everyday on the server takes a server cost.

anyway I will implements generate JWT token function on MQTT-TLS library with mbedtls, then I check the library total size.


#10

I think we can do something like separate JWT library.


#11

The _lseek and _read etc. errors are due to a few (many) of the mbedtls library files referencing functions like printf and fopen, etc. I believe. Commenting out mbedtls_entropy_init must have taken most of them out of your compile. I had many more, and ended up modifying a bunch of library files to get rid of the linker errors.

If you want to leave the entropy function in, look for references to MBEDTLS_FS_IO. It’s is not defined in the config.h but the code within the #ifdef tags causes the linker problem. I deleted a bunch of instances of it to get the code to compile. I’ll see if I can get the modifications to Hirotaka’s git. (Edit: With separate JWT library files based on the reference)


#12

I actually just found this github example from Google that shows how to connect to IoT core on and arduino and it might be useful here?

It appears they have a library that we can integrate, I have yet to try it but it looks promising

Also here is a blog post about it


#13

I asked on forum about using internal mbedTLS lib, this will maybe solve out problems related to proper configuration for that lib and porting properly for each platform.


#14

@hirotakaster Pull request is sent now. I’m still running out of memory before the JWT is built though, so I sent over as much of the background as I can too. I have to move on to some other work for the next little while, but I’m really interested to know if this works out down the line.