Google Firebase with webhooks tutorial


#21

works! Thanks again

Rick


#22

Thanks Rick for going above and beyond here. I’m seeing what you did but don’t quite understand why for some parts. I think I copied all of the device name relevant code here without pasting in my whole program. I did not redesign it to use the case switching as in your example (which maybe I could/should). Should I see the Particle.publish(“spark/device/name”) show up in my logs after the sync time publish? n is still coming out blank in my webhook.

//device name variables
const uint32_t RETAINED_DATA_MAGIC = 0xa2c7206a;
const size_t DEVICE_NAME_MAX_LEN = 31;

typedef struct {
    uint32_t magic;
    char deviceName[DEVICE_NAME_MAX_LEN + 1];
} RetainedData;

retained RetainedData retainedData = {0};


//forward declaration -- example did this, not sure exactly why even after looking it up.
void deviceNameHandler(const char *topic, const char *data);
void publishToFirebase(int count, float batteryPercentFloat);

void setup()   



if(firstDeployment){//retained variable initialized as false, should only run once after power on
        connect();// turns on cell modem and connects to cloud. need to connect to sync clock
        if(waitFor(Particle.connected,60000)){//wait up to 10 minutes to connect to cell tower for first time
            Particle.syncTime();
            Particle.publish("Device initialized: syncing time", Time.timeStr(),60,PRIVATE);
            delay(1010);
            if (retainedData.magic != RETAINED_DATA_MAGIC || retainedData.deviceName[0] == 0) {
                memset(&retainedData, 0, sizeof(retainedData));
                retainedData.magic = RETAINED_DATA_MAGIC;

                Particle.subscribe("spark/", deviceNameHandler);
                Particle.publish("spark/device/name");
            }
            Particle.disconnect();
            Cellular.disconnect();
            Cellular.off();
    
            firstDeployment = FALSE;
        }
     
    timeNextPublish = Time.now() + (60 * (59 - Time.minute())) + (59 - Time.second()); // on first deploytment wakeup after number of seconds until next hour
        goToSleep();//runs sleep code
}

void deviceNameHandler(const char *topic, const char *data) {
    if (strlen(data) <= DEVICE_NAME_MAX_LEN) {
        strcpy(retainedData.deviceName, data);
    }
    else {
        // Truncate name
        strncpy(retainedData.deviceName, data, DEVICE_NAME_MAX_LEN);
        retainedData.deviceName[DEVICE_NAME_MAX_LEN] = 0;
    }
}

void publishToFirebase(int count, float batteryPercentFloat){

    char buf[256];
    snprintf(buf, sizeof(buf), "{\"c\":%d,\"b\":%.1f,\"n\":\"%s\"}",count,batteryPercentFloat,retainedData.deviceName);
    Particle.publish(publishEventName, buf, 60, PRIVATE);

#23

I’m pretty sure you need a delay, maybe 8000 ms., after Particle.publish(“spark/device/name”). This will only happen once, so it shouldn’t be a big deal.

I think what’s happening is that you’re disconnecting from the cloud probably before the publish goes out (which is why you don’t see it in the event log) and definitely before it would have time to come back.


#24

Hi rickkas7, awesome tutorial, very well done. I have a question about the webhook only storing strings. Is it possible to store numbers. I can translate fine on the Particle side, but I have a large web app that accesses Firebase for all the data i’m writing and having to go through that code to do all the string conversion is daunting. Any hope here for the webhooks to improve? I’m assuming its a webhook limitation?


#25

Correct, it is a webhook issue. You currently can’t set the desired representation of a mustache template variable; it is always a string. There is an intention to add this feature to webhooks, but I don’t have an estimated target date for that, unfortunately.


#26

Is there some other method to get the data into Firebase as their expressed variable type that I could explore? I’m digging into my web app and I really would like to avoid rewriting it!


#27

Yes, there is now a workaround that can be used to output true boolean, integer, and floating point numbers. You need to use the body parameter instead of json in the webhook. And you can only do it from a JSON file and the Particle CLI, not currently from the web UI.

https://docs.particle.io/reference/webhooks/#body

I’ll eventually update the Intermediate Webhooks and Firebase tutorials with this information, but hopefully that will be enough to get you started.


#28

yahoo, great to hear this @rickkas7 … I made the change to my JSON file, uploaded it via CLI and am writing NUMBERS to Firebase! Very slick! thanks for the help!


#29

@rickkas7 the WRITE to Firebase works perfect, but when I try to read it back in and do the jsonBuffer.parseObject my int fields appear to be parsed and come into the new variables, but the bool and double types are not being parsed into their variables.


//New Parameter Variables
bool    newaug = false;
int     newCycle = 20.0;
bool    newfan = false;
bool    newign = false;
double  newLReadPgm;
double  newLReadWeb;
double  newLWritten;
char    newmode[9] = "Off"; 
int     newPB = 60.0;
int     newPMode = 2.0;
bool    newpgm = false;  
double  newPToggle;
double  newtarget = 80;  
int     newTd = 45.0;
int     newTi = 180;
double  newu = 0.15;

void getDataHandler(const char *topic, const char *data)
{
    StaticJsonBuffer<768> jsonBuffer;
    char *mutableCopy = strdup(data);
    JsonObject& root = jsonBuffer.parseObject(mutableCopy);
    Serial.printf("data: %s\r\n", data);
    free(mutableCopy);
 
    if(!root.success())
    {
        Serial.println("parse failed");
    }
    else
    {
        newCycle =      root["Cycle"];
        newLReadPgm =   root["LReadPgm"];
        newLReadWeb =   root["LReadWeb"];
        newLWritten =   root["LWritten"];
        newPB =         root["PB"];
        newPMode =      root["PMode"];
        newPToggle =    root["PToggle"];
        newTd =         root["Td"];
        newTi =         root["Ti"];
        newaug =        root["aug"];
        newfan =        root["fan"];
        newign =        root["ign"];
        strcpy(newmode, (const char*)root["mode"]);
        newpgm =        root["pgm"];
        newtarget =     root["target"];
        newu =          root["u"];

        Serial.printf("Read Parameters from Firebase: %d %.3f %.3f %.3f %d %d %.3f %d %d %d %d %d %s %d %.0f %.2f\r\n", newCycle, newLReadPgm, newLReadWeb, newLWritten, newPB, newPMode, newPToggle, newTd, newTi, newaug, newfan, newign, newmode, newpgm, newtarget, newu);
    }

here’s the value in data field from my webhook read

data: {"Cycle":60,"LReadPgm":1488858654,"LReadWeb":1488858710,"LWritten":1488858654,"PB":60,"PMode":2,"PToggle":0,"Td":45,"Ti":180,"aug":1,"fan":1,"ign":1,"mode":"Start","pgm":0,"target":90,"u":0.25}

and here’s the output from the Serial.printf at the end of my code snip

Read Parameters from Firebase: 60 0.000 0.000 0.000 60 2 0.000 45 180 0 0 0 Start 0 0 0.25

do I need something in my else statement syntax to assign the values from root


#30

This appears to be a bug/non-feature of SparkJson. I originally got 0 values for LReadPgm and LReadWeb.

Then I modified the source data to make them actually decimal values by adding a “.0” to the end, then it worked.

It seems kind of silly to me that it won’t coerse an integer to a double, but it won’t.

#include "Particle.h"
#include "SparkJson/SparkJson.h"

const char *rawJsonData = "{\"Cycle\":60,\"LReadPgm\":1488858654.0,\"LReadWeb\":1488858710.0,\"LWritten\":1488858654,\"PB\":60,\"PMode\":2,\"PToggle\":0,\"Td\":45,\"Ti\":180,\"aug\":1,\"fan\":1,\"ign\":1,\"mode\":\"Start\",\"pgm\":0,\"target\":90,\"u\":0.25}";

void parseData(char *data); // forward declaration

void setup() {
	Serial.begin(9600);

	delay(5000);
	char *dataCopy = strdup(rawJsonData);
	parseData(dataCopy);
	free(dataCopy);

}

void loop() {

}

void parseData(char *mutableData) {
	StaticJsonBuffer<1000> jsonBuffer;
	JsonObject& root = jsonBuffer.parseObject(mutableData);

	if (!root.success()) {
		Serial.println("parseObject() failed");
	    return;
	}

	int cycle = root["Cycle"];
	double  newLReadPgm = root["LReadPgm"];
	double  newLReadWeb = root["LReadWeb"];

	Serial.printlnf("cycle=%d newLReadPgm=%lf newLReadWeb=%lf", cycle, newLReadPgm, newLReadWeb);
}

Output:

cycle=60 newLReadPgm=1488858654.000000 newLReadWeb=1488858710.000000

#31

@rickkas7 I was going through your tutorial on Github just today. Thanks for the write up, which is pretty straightforward.

Looks like Firebase console changed the location of the Database Secrets section since you last made edits to your GitHub tutorial. If anyone like me is reading it, they might find it useful that Database Secrets is now located in Settings > Service Accounts > Database Secrets.

I haven’t completely gone through the tutorial yet, will let you know if I see any further problems using the deprecated Database Secerts.

EDIT: I now have Firebase working with values as Strings in less than an hour. Now time to work the JSON body tag to get values to integers/float (which I’m now using your other guide for). Thanks again!

Cheers


#32

@rickkas7 Hi, referring back to this thread, is there a way to actually write BOOL out to firebase as true / false and read it back in as true / false without treating it as a string. I’ve done this before in RPi/Python/Firebase, but can’t seem to figure it out for my current project with webhooks in Photon/C++/Firebase.


#33

There’s a new section at the end of this tutorial on using the webhook body feature to do true booleans and numbers:

It’s explained in more detail here:

Rick


#34

I’ve looked at that but can’t find a snprintf % ref for bool, I’m using %d and getting a 1 or 0 in Firebase but not a True or False. Do I have to code this in the JSON body format?


#35

bool is only supported as numeric value via %d which would render 1 or 0 for true or false
But if you want to have the literals true or false in your string, you can do this

  snprintf(buf, sizeof(buf), "This bool is %s", yourBool ? "true" : "false");

#37

I generated a library based on the firebase-webhooks tutorial:
It needs some work but maybe it can be of value as a starting point. Note I’m new both with Photon and C++ so be gentle :wink:

Usage:

firebase = new Firebase("test3rdata");
firebase->subscribe();
firebase->setCallback(readCallback);

And:

firebase->readData();
firebase->publishData("test1data", "{\"a\":10}");

Where the callback receives a JsonObject:

void readCallback(JsonObject& root) {
	r = root["r"];
	Serial.printlnf("Yes! r=%d", r);
   }

#38

@rickkas7
Hello Rickkas7.
I have an issue that i am facing in the webhook to get the data from firebase.
So, I have two photons and I have stored the data in my firebase DB with the particledevice id.json .
Now im using a get webhook to fetch data from firebase based on the device that is calling,ie: I should get the data of that particular photon only. But when i substitute the value {{{coreid}}} or {{{PARTICLE_DEVICE_ID}}} to get the data it sends back null. when i hardcode my particle id in the url, it sends the correct data. Somehow i think there is an issue with the substitution part in url.
Could you please help me with this.
Thank you!:slight_smile:


#39

The data sent to the webhook (including device ID) aren’t automatically forwarded to your destination server. But even if you explicitly send them to the server, it would be up to the server to actively embed it in the response.
3rd party servers usually don’t do that.

A webhook call is not an atomic transaction. You have the iniating call - which can be pushed from any device - as one transaction and a sencond trasaction where the server reports back to the cloud which in turn - now possibly without knowledge of the initiator, as outlined above - pushes an event to all subscribers.

If you want to have the device ID reliably embedded in the response “event” message you may want to consider appending the device ID to the event name.


#40

@rickkas7 Thanks for taking the time to make the tutorial. It seemed like it would be straight forward. Unfortunately I think Google’s changed a couple things that add a couple hurdles.

First, I couldn’t even create a project. I found out after looking around that console was having issues and wouldn’t let anyone make any new projects for about the one hour I happened to try. Bad luck on my timing I guess. https://status.firebase.google.com/ <- Check here if you’re having problems. If you see red, go to the bar for a couple hours and come back.

I waited a day and it worked, but then I had 10 blank projects. Deleting them was a little hidden. In the project settings in the project overview, after scrolling to the bottom, in light gray, there is a “Delete Project” link.

More importantly, the database wouldn’t work. Any time I tried to get into it I was given an error. I found that if I created an app in the Project Overview section and filled it with junk, then the database tab worked for me.

The Database view didn’t look right either. At the top I noticed I had the option to change the Database from “Cloud Firestore BETA” to “Realtime Database.” Then it was looking like the tutorial.

From there I grabbed your hook.json example and followed along. I got the Particle CLI publish command to put data into my Firebase database just like you showed. I think I’m on my way to having my AssetTracker put data somewhere it is stored at all times now thanks to you.

I’m still surprised that there isn’t some sort of easier/prepackaged way to store published events without being logged into the console and waiting for them to come along. I feel like I’m missing a tutorial on something.


#41

I was looking through this tutorial and seems that the part where you get the secret has changed drastically.

Cheers,