Saving IoT Hub Connection String into EEPROM

argon
Tags: #<Tag:0x00007f1ca24cf728>

#1

I have been using Particle hardware for quite a while now and can generally get around the problems I run into. However I am looking at saving an IoT Hub connection string into the EEPROM on several Argons and have no idea where to start.

The problem stems from updating the deviceOS. When I set up a device at a plant so that it MQTT publishes directly into IoT Hub I can justify the manual set up process. However, for something as simple as updating to the newest OS I have to send down a new binary to each individual MCU, which will be quite frustrating.

I am unable to find any details on accessing a config file from an MCU so I am beginning to think this is not an option, which leads me to saving the connection string at installation with the EEPROM.put command and then in my AzureIotHubClient code having it EEPROM.get said connection string. This way I can monitor all devices with the benefit of fleetwide OTAs that Particle products offer.

I have no idea where to begin, has anyone stored connection strings in EEPROM that can provide some advice?


#2

However, for something as simple as updating to the newest OS I have to send down a new binary to each individual MCU, which will be quite frustrating.

This should not normally be the case. If you want to forcibly upgrade the deviceOS manually, yes you do have to flash the system deviceOS binary explicitly. However, usually the way this is done is to compile a new application firmware version (your own code), but with the newer deviceOS as the target OS version. You would either flash that directly / manually OR if you are using a Particle Product, simply release that new firmware to a group or individual devices to have both their application and system firmware updated to the latest versions.

When you say:

I am unable to find any details on accessing a config file from an MCU

What config file are you talking about / expecting? What is a “connection string” in your use case? Is it your MQTT private key or client id?

I use AWS IoT over MQTT. I store a backup certificate for restoring keys when needed, but otherwise store my certificates in an SD card. Storing on the EEPROM with the extra space for Gen 3 seems to make sense. I can definitely help share a few general approaches for this kind of thing if I’m understanding what you are trying to do correctly.

Putting information like that into the EEPROM should be fine and it is documented here as I’m sure you’ve seen. What exactly is your question? It sounds like you are doing the correct stuff, but without a specific question or code to evaluate not sure how best to help ya.


#3

Agree with the assessment on what is usually done. I dream of doing things simply and sending data through the Particle cloud and therefore using webhooks or rules engine after the fact, but Corporate IT and enterprise grade policies make life fun. The problem there is that my firmware saved in the Product today are specific to a connection string and a particular MCU (i.e. the deviceOS is part of the binary). So whenever I then go to update the devices to a new deviceOS OTA I have to essentially go back through and copy&paste all over again for every device and then upload that new binary. Great if I am talking 5 devices or so, but based on about 100 in use in the coming months I am going to end up with firmware Rev 200+ by the time Gen 3 OS 1.1 comes out of beta.

The connection string is the primary key for IoT Hub in the format;

"HostName=<Host Name>;<DeviceId=<Device Name>;<SharedAccessKey=<Device Key>"

So getting that into EEPROM and then out is something that I am just struggling to wrap my mind around. I am a BioMed/Mechanical Engineer that has taught myself coding through the Particle IDE and google so in some things I would consider myself almost an expert, but EEPROM storage is something I had never thought about until this morning, and there isn’t a lot of support available online.

My goal would ultimately be for each product to have an installation firmware where I assign the connection string to the EEPROM and then EEPROM.get (must be constructed like above). So I imagine I will start with the EEPROM.clear() command then;

addr = 0;
struct MyConnectionString {
     uint16_t HostName;
     uint16_t DeviceId;
     uint16_t SharedAccessKey;
};
MyConnectionString myCS = { companyname-iothub-dev.azure-devices.net, XXXXXXXX, XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX };
EEPROM.put(addr, myCS);

Then in the actual code where today I have;

#define CONNECTION_STRING "HostName=companyname-iothub-dev.azure-devices.net;DeviceId=XXXXXXXX;SharedAccessKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
IotHub hub(CONNECTION_STRING);

I would replace with;

IotHub hub(EEPROM.get(int 0, myCS))

I don’t really understand the Integer data types of C (i.e. the difference between uint8_t, uint16_t, etc.) as most all of my work is floating points and JSONs.


#4

Nope, you don’t want to clear the EEPROM if there is no good reason for it.
You can just write to it as it is preinitialised by default.
I usually use magic numbers - and when integrity is important a checksum - to tell whether a retreived data can be considered valid or not.

I’m not sure if your connection string is correct. It seems to open tags with < but fails to close them with the corresponding >.

However, your issue doesn’t seem to be putting the string into EEPROM but picking the correct datatypes for your individual fields and how to build the actual connection string.
uint16_t can only hold 16 bits of data representing an unsigned integer (0…65535).
You actually want for each individual field a character array big enough to take the longest thinkable entry.

e.g.

const uint32_t magicNumber = 0xbeefba11;
const uint32_t baseAddress = 0x00000000;

struct ConnectionData {
  uint32_t MagicNumber;
  char     HostName[128];
  char     DeviceId[32];
  char     SharedAccessKey[64];
};
...
  ConnectionData cd = { magicNumber, "companyname-iothub-dev.azure-devices.net", "XXXXX", "YYYYYYYYY" };
  EEPROM.put(baseAddress, cd);  

When you retreive the data and want to build the actual connection string from that data you’d do something like this

const char *connStringFormat = "HostName=%s;DeviceId=%s;SharedAccessKey=%s";
IotHub *hub;
 
...
  ConnectionData cd;
  EEPROM.get(baseAddress, cd);
  if (magicNumber == cd.MagicNumber) {
    char connString[256];
    snprintf(connString, sizeof(connString)
            , connStringFromat, 
            , cd.HostName
            , cd.DeviceID 
            , cd.SharedAccessKey
            );
    hub = new IotHub(connString);     
  }
  else {
    // take appropriate steps
  }

Subsequently you’d access the hub methods and fields this way

  hub->someField = value;
  hub->someMethod();

#5

@ScruffR thank you very much for the help with the start. I proceeded forward without a magicNumber, but wondered if this is necessary or optional?

For my configuration code (i.e. writing to EEPROM) I have;

const uint32_t baseAddress = 0x00000000;

struct ConnectionData {
    char    Hostname[128];
    char    DeviceId[32];
    char    SharedAccessKey[64];
};

void setup() {
    ConnectionData cd = { "companyname-iothub-dev.azure-devices.net", "XXXXXXXX", "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" };
    EEPROM.put(baseAddress, cd);
    
    pinMode(D7, OUTPUT);
    digitalWrite(D7, HIGH);
}

void loop() {

}

The Argon on board LED is in fact on so I believe this to have been a successful operation. I was now going to Particle.publish the EEPROM data and verify that it worked. I went ahead and put it into the off the shelf AzureIotHubCleintV2 library, but have run into some errors. For your reference this is the setup portion of the github library;

// This #include statement was automatically added by the Particle IDE.
#include "ArduinoJson.h"
#include "AzureIotHubClient.h"

#define CONNECTION_STRING "<Your Azure IoT Hub or Azure IoT Central Connection String>"

int count = 0;
int msgId = 0;
char telemetryBuffer[256];

IotHub hub(CONNECTION_STRING);

void setup()
{
  pinMode(LED_BUILTIN, OUTPUT);
  RGB.control(true);
  Time.zone(0);
}

I then created this;

// This #include statement was automatically added by the Particle IDE.
#include <AzureIotHubClientV2.h>
#include <ArduinoJson.h>

const uint32_t baseAddress = 0x00000000;
const char *connStringFormat = "HostName=%s;DeviceId=%s,SharedAccessKey=%s";

struct ConnectionData {
  char     HostName[128];
  char     DeviceId[32];
  char     SharedAccessKey[64];
};

ConnectionData cd
EEPROM.get(baseAddress, cd);
char connString[256];
snprintf(connString, sizeof(connString)
    , connStringFormat,
    , cd.HostName
    , cd.DeviceID
    , cd.SharedAccessKey
    );
IotHub hub(connString);
    
int count = 0;
int msgId = 0;
char telemetryBuffer[256];

void setup()
{
    Particle.publish("ConnectionString", String(connString), PRIVATE);
    
    pinMode(LED_BUILTIN, OUTPUT);
    RGB.control(true);
    Time.zone(0);
}

Right now I am receiving 2 errors’

Any further help would be greatly appreciated.


#6

The active code (lines 15-22) cannont exist in the declaration area but need to be inside a function (hence the indentation) and due to this you cannot use IotHub hub(connString); the way you do as the connection string will only be created during run-time.


#7

Thank You very much for the help here. Being able to get this up an running has essentially changed anyway we will go about deploying sensors into the future. Being able to have a user firmware that simply calls the DeviceId and SharedAccessKey allows one firmware to be used across an entire fleet of sensors. In case anyone has a similar question in the future I am putting my firmware here;

For the configuration step (i.e. saving Connection String information to EEPROM)

const uint32_t baseAddress = 0x00000000;

struct ConnectionData {
    char    Hostname[128];
    char    DeviceId[32];
    char    SharedAccessKey[64];
};

void setup() {
    ConnectionData cd = { "HostName from IoT Hub", "Device ID from IoT Hub", "Shared Access Key from IoT Hub" };
    EEPROM.put(baseAddress, cd);
    
    pinMode(D7, OUTPUT);
    digitalWrite(D7, HIGH);
}

void loop() {

}

The test code I created then takes the AzureIotHubClient publishes the connection string (will move away from this in the future, but worked well for debug), and then publishes my reading every minute.

// This #include statement was automatically added by the Particle IDE.
#include "ArduinoJson.h"
#include "AzureIotHubClientV2.h"

const uint32_t baseAddress = 0x00000000;
const char *connStringFormat = "HostName=%s;DeviceId=%s;SharedAccessKey=%s";
IotHub *hub;

struct ConnectionData {
  char     HostName[128];
  char     DeviceId[32];
  char     SharedAccessKey[64];
};

// This #include statement was automatically added by the Particle IDE.
#include "Adafruit_MAX31856.h"

Adafruit_MAX31856 sensor = Adafruit_MAX31856(A0, A1, A2, A3);

PRODUCT_ID(XXXX)
PRODUCT_VERSION(X)

int count = 0;
int msgId = 0;
char telemetryBuffer[256];

void setup()
{
  ConnectionData cd;
    EEPROM.get(baseAddress, cd);
    char connString[256];
    snprintf(connString, sizeof(connString)
        , connStringFormat
        , cd.HostName
        , cd.DeviceId
        , cd.SharedAccessKey
        );
    
    hub = new IotHub(connString);
    Particle.publish("ConnectionString", String(connString), PRIVATE);
  
  Serial.begin(115200);
  Serial.println("MAX31856 thermocouple test");

  sensor.begin();
 

  sensor.setThermocoupleType(MAX31856_TCTYPE_J);
  
  pinMode(LED_BUILTIN, OUTPUT);
  RGB.control(true);
  Time.zone(0);
}

void loop()
{
  digitalWrite(LED_BUILTIN, HIGH);

  //loop returns true if connected to Azure IoT Hub
  if (hub->loop())
  {
    if (count++ % 30 == 0)
    {
      Serial.printf("msg id: %d\n", msgId);
      hub->publish(telemetryToJson());
    }
  }

  delay(20); // allow for a short blink before turning led off
  digitalWrite(LED_BUILTIN, LOW);
  delay(1980);
}

char *telemetryToJson()
{
    WiFiSignal sig = WiFi.RSSI();
  
  /*  https://arduinojson.org/
    use to calculate JSON_OBJECT_SIZE https://arduinojson.org/v5/assistant/
    Have allowed for a few extra json fields that actually being used at the moment
*/
  DynamicJsonBuffer jsonBuffer(JSON_OBJECT_SIZE(3) + JSON_OBJECT_SIZE(5) + 0);
  
  JsonObject &root = jsonBuffer.createObject();
  root["deviceId"] = hub->getDeviceId();

Thanks again for the help.


#8

connString already is a string so that “conversion” is not only not needed but counterproductive.

Also instead of flashing some initialisation firmware and then flash your actual code, I’d reather create a Particle.function() or use Particle.subscribe() to receive the data remotely - only when the received data is not valid falling back to some defaults.


#9

Today most of our work is on internal products that may reflect 100-200 or so devices globally per product. The initialization would occur at the hardware assembly step and then once deployed in the field the Azure firmware would be sent down.

I am curious about the Particle.function() and Particle.subscribe() idea though. We have used the Particle.function() for cloud-to-device messages to activate things like calibration mode or others. The Particle.subscribe() however is something we haven’t really dealt with.

Big picture we are trying to create a similar environment that is seen with config files on our companies Linux Gateways. Would the Particle.subscribe(function) have the ability to reach out to a cloud based config file where lets say the MCU ID was used to corespondent with the Azure DeviceID and SharedAccessKey?


#10

That is doable, although the Particle.subscribe() wouldn’t reach out to your server but your server can publish an event your device(s) would subscribe to.
In order to “tell” your server to send the out the event the device can send a “request” event to which the server has subscribed to.

With a clever choice of event names and subscription filters you can either send the data to a specific device or to a collection of devices at once.