MQTT "Test Client"

I’ve been experimenting with Home Assistant for the past few weeks, and I wanted to use MQTT to publish data from some of my Particle devices. Having never used MQTT before, and being new to Home Assistant, I decided that I needed to configure Home Assistant with test data. That led to the development of the code listed below.

I compiled and flashed the code to my Photon, and I now have an easy way to publish and subscribe to any MQTT broker via the Particle Console as seen in the following screen-shot:

  • The SetBrokerIP function allows me to enter the MQTT broker’s IP address or DNS name.
  • The SetPubTopic function allows me to enter an MQTT topic that will be used when I publish a message.
  • The Publish-Msg function allows me to publish an MQTT message.
  • The Subscribe function allows me to subscribe to a MQTT topic.
  • The getMessage variable allows me to display the last MQTT subscription message received.

I’ve found this to be very useful tool and wanted to share. Perhaps someone with more MQTT experience can take this and make it into something even more useful. In the meantime, you are more than welcome to use my code. To date, I’ve only tested it on a Photon.

#include "Particle.h"
#include "MQTT.h"

	MQTT* mqtt;
	char clientName[64] = "";       // obtained from particle/device/name
	byte brokerIP[] = {0,0,0,0};    // obtained via Particle.function when an IP address is entered
	char brokerDNS[256];            // obtained via the same Particle.function as brokerIP, when a DNS name is entered
	char pubTopic[256] = "";        // obtained via Particle.function
	char buffer[512] = "";          // obtained via Particle.function
    char subTopic[256] = "";        // obtained via Particle.function
    char subMessage[512] = "";      // obtained via MQTT subscribe, exposed via Particle.variable
    Timer connectTimer(15000, displayConnectionState); // a simple + or - is displayed in the Serial console
    bool mqttConnectViaDNS = false; // true: connect via brokerDNS, false: connect via brokerIP
    bool mqttIsInitialized = false; // true: mqtt points to an initialized MQTT instance
    bool mqttIsActive = false;      // true: mqttIsInitialized and mqtt->isConnected are true (maintained in loop())
 
void setup() {
    Serial.begin();
    Particle.function("SetBrokerIP", SetBrokerIP);
	Particle.function("SetPubTopic", SetPubTopic);
	Particle.function("Publish-Msg", Publish);
	Particle.function("Subscribe", Subscribe);
	Particle.variable("GetMessage", subMessage);
	Particle.subscribe("particle/device/name", handler, MY_DEVICES);
    Particle.publish("particle/device/name", NULL, 60, PRIVATE);
}

void loop() {
    if (mqttIsInitialized && mqtt->isConnected()) {
        mqttIsActive = true;
        mqtt->loop();
    } else {
        mqttIsActive = false;
    }
}

// --------------------------------------------- Connect to the MQTT Broker via DNS name or IP address
// triggered by Particle.function
int SetBrokerIP(String s) {
    // determine if IP address or DNS name was entered ...
    mqttConnectViaDNS = false;
    s.toCharArray(buffer, sizeof(s));
    for (int i = 0; i < strlen(buffer); i++) {
        if (( !isDigit(buffer[i])) && (buffer[i] != '.') && (buffer[i] != '\0')) {
            mqttConnectViaDNS = true;
            break;
        }
    }
    if (mqttConnectViaDNS) {
		// connect via DNS Name -----------------------------
        strncpy(brokerDNS, buffer, sizeof(brokerDNS));
        Serial.printf("brokerDNS: %s\n\r", brokerDNS);
	    mqtt = new MQTT(brokerDNS, 1883, 15, callback);
		mqttIsInitialized = true;
    } else {
	    // connect via IP Address ---------------------------
        for (int i = 0; i < 4; i++) brokerIP[i] = 0;
        sscanf( buffer, "%u.%u.%u.%u", &brokerIP[0], &brokerIP[1], &brokerIP[2], &brokerIP[3] );
        Serial.printf("brokerIP: %u.%u.%u.%u\r\n", brokerIP[0], brokerIP[1], brokerIP[2], brokerIP[3] );
	    mqtt = new MQTT(brokerIP, 1883, 15, callback);
		mqttIsInitialized = true;
    }
    mqtt->connect(clientName);
    connectTimer.start();
    return 0;
}

// --------------------------------------------- Set the Publish Topic
// triggered by Particle.function
int SetPubTopic(String s) {
	s.toCharArray(pubTopic, sizeof(buffer));
	return 0;
}

// --------------------------------------------- Publish an MQTT Message
// triggered by Particle.function
int Publish(String s) {
    if (mqtt->isConnected()) {
		s.toCharArray(buffer,sizeof(buffer));
		mqtt->publish(pubTopic, buffer);
		return 0;
	} else return 1;
}

// --------------------------------------------- Subscribe to a Topic
// triggered by Particle.function
int Subscribe(String s) {
    if (mqtt->isConnected()) {
        s.toCharArray(subTopic,sizeof(s));
	    mqtt->subscribe(subTopic);
    	return 0;
	} else return 1;
}

// --------------------------------------------- Receive an MQTT Message
// triggered when a message is received from the MQTT Broker
void callback(char* topic, byte* payload, unsigned int length) {
    strncpy(subTopic, topic, sizeof(subTopic));
	memcpy(subMessage, payload, length);
	subMessage[length] = '\0';
}

// --------------------------------------------- Display MQTT connection State
// triggered by connectTimer
void displayConnectionState(void) {  
    if (mqttIsActive) Serial.print("+"); else {
        Serial.print("-");
    }
    return;
}

// -------------------------------------------- Get ClientName
// triggered by the Particle.subscribe and Particle.publish run in setup()
void handler(const char *topic, const char *data) {
    strncpy(clientName, data, sizeof(clientName));
}
    

7 Likes

Hi Bear,
How is the integration with Home Assistant going? I read your recent post about HA and have been reading about it. The code you share here was one of my obstacles. Thanks!
Did you use the MQTT library from the WebIDE? I am wondering how much processing power and memory it takes up. I want to control a touch display and read a a few DS18 sensors.

Yes, I used the MQTT library from the web IDE. Here is its GitHub link: https://github.com/hirotakaster/MQTT

I’ve been testing it with my largest code set and I have not noticed any memory issues or degradation. It’s coexisting nicely with SparkJSON and my TzCfg libraries. Updates post to Home Assistant very quickly. I haven’t triggered any Particle Functions from HA yet, but those triggered from the code I shared occur almost instantaneously.

I have a DS18B20 sitting in front of me … My next circuit will use it to monitor outdoor temperature, three YHDC SCT013 current transformers to monitor the amperage used by our boiler, and will monitor the presence of power to our boiler via a small relay which opens when the power drops. It’s possible that I might also add another relay to prevent the boiler’s controller from powering on until I know the power is stable.

1 Like

As long as you aren’t using TLS, MQTT doesn’t take up too much flash or RAM. It’s mostly just a TCP client + a buffer of configurable length in the .h file that defaults pretty small.

Thank you @Bear. Sounds like you are running even more code than I would.
I forget if you mentioned it, is HA running locally on… RPi, PC…?

Thanks @justicefreed_amper for the insight. I’m planning to keep HA running on local network, so I can avoid encryption.

1 Like

First, I’m glad I decided to take a look at HA. It is exceeding my expectations. It is not a panacea, but it has a lot of power and I’ve found it relatively easy to work with once I got a grasp of the basics. It can run locally, which is important to me.

If I had a Windows, Linux, or Apple system that didn’t travel outside the house, I may have chosen to load Oracle’s (free) "VirtualBox’ software https://www.virtualbox.org/ to host a virtual machine running HA on Ubuntu, but I chose to host on a Raspberry Pi 3B+ which has also exceeded my expectations.

I tried Hassbian and HASS.IO, but they left me feeling like I had taken over someone else’s computer after they’d installed, deleted, modified, and configured the OS and applications … not a good place to start. I finally decided to load Raspbian, and HA myself, and that has worked well for me.

I took a risk when I chose to load Raspbian Stretch, but I really wanted a GUI interface that would:

  • Allow me to use a text editor similar to Windows Notepad.
  • Allow me to run HA’s front-end in a browser on the same desktop.
  • Allow me to copy/paste information into from the browser into the text editor.
    Stretch does all that and it performs very well.

I found an inexpensive smokin’ SD card (170MB/s reads, 90MB/s writes) at Best Buy … https://www.bestbuy.com/site/sandisk-extreme-plus-64gb-microsdxc-uhs-i-memory-card/6282920.p?skuId=6282920

Oops, I forgot to answer the encryption question. I’m starting with my MQTT broker running on the RPi, and all traffic will be local … so I don’t plan to encript MQTT traffic at this time. Later on, I may bring up another broker for encrypted traffic … we’ll see. I did activate SSL on my RPi, and I set up an internet pathway to it via DuckDNS. That lets me securely access my HA front-end via the internet.

If we have any more discussion on HA, let’s take it offline.

Thanks for the detailed reply. I’m thinking along the same lines.

Thank you for posting this, I am a novice to arduino/cpp (and that is being generous lol). I saw you had an 1820 temp sensor in a later post. how did you collect and publish that with this script? Did you replace ‘pubTopic, buffer’ with the final topic name and buffer for the temp data?

thank you in advance.

I’m not actively coding these days. We’re settling into a new home, and there are many other demands for my time. I posted some sample code on GitHub last year in an effort to help someone else with their project. Unfortunately, it looks like I deleted it when we were done. I think the code posted here is a working copy of the Particle portion of the code that was on GitHub.

I wish I could provide a more streamlined example, but this is the best thing I have available. To get your head around it, I’d suggest the following:

  • Note that the loop() function calls four commands:
    1. mqttSessionManager()
      • establishes and maintains a connection with the MQTT server.
      • periodically sends an MQTT message related to “host status”
      • processes incoming MQTT messages via a callback function named mqttCallBack().
    2. temperatureMonitor() reads the DS18R20, at a regular interval, formats and sends an MQTT message with the data. If the message is not sent the first loop, it will try again and again in subsequent loop() cycles.
    3. relayMonitor() reads the status of the relay and sends an MQTT message when the status changes. It also contains the a stand-alone function relayToggle() which is called by mqttSessionManager() based on incoming MQTT messages.
    4. hostMonitor() periodially updates the host status information which mqttSessionManager() will publish via MQTT.

You will also note that this code formats MQTT messages based on a time interval, and that the code that sends MQTT messages is based is executed in each loop() cycle. If a send fails, this logic keep trying until the message is sent or overwritten.

The MQTT and Temperature code blocks are obviously the ones that relate to your project.

I will point out one thing that is VERY important. MPU code must never hog the processor. In this example, mqttSessionManager() function is designed as a state-engine. It detects the MQTT connection state, and takes a single action if and when it is appropriate. The function then exits, allowing other functions and processes to have the processor. In subsequent loop() cycles, mqttSessionManager() again takes action base on the connection state it finds then.

The relay logic also toggles the relay and exits without knowing if the relay was toggled successfully. An MQTT message is not sent until a loop() cycle detects that he relay’s state has changed. This process consumes very little MPU time and makes the code more reliable.

I hope you find this useful.

2 Likes

Thank you! It will take me some time to absorb this as I’m not a cpp developer. I barely know python but trying to learn new things. :slight_smile:

Hope the move goes well and you find everything you need unbroken!

Thanks for sharing, I am planning on playing around with MQTT on an Argon myself.

There is nothing more important than learning new things. A lot can change in a lifetime.

My grandfather was born in the 1880s. At that time people traveled by horse, horse-drawn wagons, and steam-powered railroads / boats. There were few ways to communicate beyond the range of the human voice. These included smoke signals, using signal flags, or using Morse code via the telegraph or flashing lights.

Late in his life, I had the pleasure of watching TV with my grandfather as Neil Armstrong stepped onto the lunar surface. I didn’t think much about it at the time. Some years later, I came across a “Book of Knowledge” that was published in the 1880s. The tome was written in a “question and answer” format, and one of the questions it tried to answer was: “Will man ever travel to the moon?”. The answer started with the words: “Even if we could build a cannon big enough…”. That made realize the magnitude of change that my grandfather had witnessed. He saw the invention of the radio, automobiles, airplanes, rockets, television, telephone, nuclear fission, and a host of other technologies. In a way, he witnessed a “big bang” in many technologies.

I was born the same year the transistor was invented. A couple of years later, Popular Mechanics published the statement" Computers in the future may weigh no more than 1.5 tons.". I think we can agree that this was a true statement. Core memory was invented when I was 4-years old. After college, I was hired to program an IBM mainframe that had 16K of memory. That was in the 1970s, so a lot has happened in the past 50 years, and the pace of change is accelerating.

Today, we are witnessing big-bangs in Deep Learning and Quantum Computing. These technologies will fuel even faster acceleration in the future. Those who are eager to learn will never run out of opportunities.

5 Likes

Agreed! 1.5 tons…dang. lol.

I have been using your script to help me learn the MQTT operations. It’s slow going but it is helping a lot, thank you!

Hi Bear,
Really enjoyed your story of your grandfather , and the history lesson. its amazing how far we have come in a short time.
Anyway just a question . As you are using particle functions there are going to the cloud, so online and back to HA with is local. Is that right or is the Particle running MQTT local.That is what i would like to do keep everything offline.
Thanks

I use “unsecure” MQTT ONLY to communicate between devices and Home Assistant via the local network.

I have two paths for communicating with my devices via the internet.

  1. The primary path is via the Particle Cloud.

  2. The secondary path is via my Home Assistant server (which runs on an RPi).

a.) My Google WiFi router exposes the RPi ports used by the Home Assistant Console.
b.) The RPI runs SSL to make HA communications secure.
c.) The RPi uses DuckDNS so authorized devices can find the HA server via the internet.

FYI: My MPU coding processes triggers, checks status, detects changes, and reports status

  • Triggers are initiated by either an MQTT subscription message or via a Particle Function. They simply trigger the change and exit.

  • Status checking is performed at regular intervals … say every 2 seconds or whatever makes sense for the application.

  • Changes are detected by comparing current status to the last reported status.

  • Reporting involves publishing an MQTT status message with the retain-flag turned on. The MQTT server then sends the status message to all devices that have subscribed to the status message.

  • The retain-flag tells the MQTT server to resend the most current status message to any device that reconnects to the MQTT server. This allows devices that reboot or reconnect after an outage to immediately know the current status.

I am bumping this up the list. I am going to try Bears code for my Mqtt stuff. But thank you for sharing this.

1 Like