Ubidots MQTT Timestamp

So I’m having a hard time figuring out how to get a timestamp into a Ubidots mqtt publish.
formatted call:
client.add(“variableName”, int/float value, context as JSON char*, POSIX timestamp);
I’m using an Electron at 0.6.4.

As I understand it from the docs it takes a POSIX millisecond timestamp, essentially a UNIX timestamp with the milliseconds added to it. However a uint32_t (highest numeral value for an Electron) goes up to 4,294,967,295. The current UNIX stamp is 1,517,548,385 (added commas for easy reading) and the POSIX stamp that I will need to put in would be 1,517,548,385,000. Well beyond what a uint32_t can hold. So, how does one go about filling this value?
Here is the Ubidots example with some added in code as I was trying to make sense of what would and would not work. Thanks in advance.

/****************************************
 * Include Libraries
 ****************************************/

#include <UbidotsMQTT.h>

/****************************************
 * Define Constants
 ****************************************/

#ifndef TOKEN
#define TOKEN "TOKEN HERE"  // Add here your Ubidots TOKEN
#endif

bool haveDeviceName = false;
String deviceName;
char deviceNameArray[25]; // device name must be 25 characters or less
float value = 1;

/****************************************
 * Auxiliar Functions
 ****************************************/

void callback(char* topic, byte* payload, unsigned int length) {
    Serial.print("Message arrived [");
    Serial.print(topic);
    Serial.print("] ");
    Serial.println("payload obtained from server:");
    for (int i=0;i<length;i++) {
        Serial.print((char)payload[i]); // prints the answer of the broker for debug purpose
    }
    // Some stuff to make with the payload obtained
        //
   //
    Serial.println();
}


/****************************************
 * Instances
 ****************************************/

Ubidots client(TOKEN, callback);

/****************************************
 * Sub Routines
 ****************************************/

void nameHandler(const char *topic, const char *data) {
    deviceName = String(data);
    deviceName.toCharArray(deviceNameArray, deviceName.length()+1);
    haveDeviceName = true;
    
}

/****************************************
 * Main Functions
 ****************************************/

void setup() {
    Serial.begin(115200);
    Particle.subscribe("particle/device/name", nameHandler);
    Particle.publish("particle/device/name");
    while(!haveDeviceName) {
        Serial.println(F("waiting for device name"));
        delay(2000);
    }
    Serial.print(F("We have a Name! "));
    Serial.println(deviceName);
    while(!Time.isValid()) {
        Serial.println("Validating Time");
    }
    Time.zone(-5);
    Serial.println(Time.format(Time.now(), TIME_FORMAT_ISO8601_FULL));
    Serial.println(Time.now());
    Serial.println(Time.format(Time.local(), TIME_FORMAT_ISO8601_FULL));
    Serial.println(Time.local());
    client.initialize();
    pinMode(D7, OUTPUT);

    // Uncomment this line if you have a business Ubidots account
    //client.ubidotsSetBroker("business.api.ubidots.com");
}

void loop() {
    if(!client.isConnected()){
        client.reconnect();
    }
    char buf[12]; // "-2147483648\0"

    // Publish routine, if the device and variables are not created they will be created
    for(byte i = 0; i < 6; i++){
        int eventTime = Time.now();
        String strVal = "string value";
        char character = 'A';
        String eventTimeStr(eventTime);
        Serial.println("Sending value");
        client.add("val", value); // Insert as first parameter your variable label
        client.add("valcontext", value, "\"lat\":10.302, \"lng\":2.9384"); //Adds value with context
        client.add("valString", character);
        // client.add("valtime", value, NULL, (eventTime * 1000)); // Adds value with custom timestamp.  timestamp is not formatted properly and reads constant January 1, 1970 
        // client.add("valcontexttime", value, "\"lat\":10.302, \"lng\":2.9384", eventTime); // will not send any data to ubidots with this format in publish
        // client.add("valarrayContext", value, itoa(eventTime, buf, 10)); // will not send any data to ubidots with this format in publish
        // client.add("valStringContext", value, eventTimeStr); // will not send any data to ubidots with this format in publish
        // client.add("valarrayContexttime", value, itoa(eventTime, buf, 10), Time.now()); // will not send any data to ubidots with this format in publish
        client.ubidotsPublish(deviceNameArray); // Insert your device label where the values will be stored in Ubidots
        Serial.println(Time.format(Time.now(), TIME_FORMAT_ISO8601_FULL));
        Serial.println(Time.now());
        Serial.println(eventTime);
        digitalWrite(D7, HIGH);
        delay(5000);
        digitalWrite(D7, LOW);
        value++;
    }
    client.loop(); // Client loop for publishing and to maintain the connection
}

Where does the notion about milliseconds come from? I thought POSIX timestamp is equivalent to UNIX epoch :confused:
There may be extended timestamps but they most likely won’t be stored in 32bit integers.

According to Ubidots docs

But if you look at their publish example from the UbidotsMQTT library line 63 shows that they are throwing in a regular UNIX timestamp.

client.add("test-var-3", value, NULL, 1492445109); // Adds value with custom timestamp

Running the code and checking on Ubidots website shows all the values sent with that custom timestamp are given a timestamp of EPOCH.

Even the examples for Node.js that they give in their docs does not give a proper timestamp demonstration

Yup, this seconds vs. milliseconds confusion does exist but I’d rather tick that off as urban legend

But if you actually would need milliseconds in the JSON you can always add the 000 to the string as JSON is character based and not based on the binary types.

e.g.

int posixSec = Time.now();
char szJSON[128];

snprintf(szJSON, sizeof(szJSON), "{ ... \"posixMilliSec\":%d000 ... }", posixSec);

BTW, you are making things a bit too complicated for yourself with this

    Serial.println(Time.format(Time.now(), TIME_FORMAT_ISO8601_FULL));
    Serial.println(Time.now());
    Serial.println(Time.format(Time.local(), TIME_FORMAT_ISO8601_FULL));

For one Time.format() features an overload that only takes the format and “assumes” you want the time now to be converted.
And also don’t use Time.format(Time.local(), TIME_FORMAT_ISO8601_FULL) as this will introduce the time zone offset twice and hence give you unexpected results.

1 Like

Thanks for the feed back regarding handling particles Time class. I was doing all sorts of weird stuff earlier as I was confused about the difference between .local() and .now(). I think I have that cleared up and this code is more artifact of me working that out. Same goes for setting the device name (setting it as a string and then a char array and then sometimes back, stupid I know).

What you are suggesting, putting the time in JSON is a great idea (it’s the workaround that I am going to go with). The actual timestamp will then be part of the variable’s context, I’m fine with this. I may also try using their HTTP based library for sending data or else via a particle webhook.

However, there would be a separate timestamp that Ubidots will use for all of their dashboard graphs which is the timestamp of when the data is processed in their API. This would give an interesting view into the latency of the pipeline.

Hello @daviddefilippis,

After read the thread and made some test from our side, we found out that the Ubidots Particle MQTT library has a bug referring to the timestamp assignment. Normally, all the Ubidots Libraries are created to send the timestamp in seconds, but then the library is in charge of take the timestamp received in seconds and multiply the value by 1000, giving as result the right timestamp accepted by Ubidots(milliseconds).

Sorry for the issue presented, and thanks so much for reported it! I already assign a task to @jotathebest which is the maintainer of the library.

All the best,
Maria C.

2 Likes

Thanks @mariahernandez, that was a quick response. I assumed it was a bug but wanted to check on the forum to see what was up.

I haven’t heard anything about a patch going out to clear up the MQTT library. IS there something in the works?