Looks like I have some catching up to do. This collaboration is great! Hope to eventually catch up on things and provide a bit more detailed response but in the meantime wanted to at least comment on this:
As long as the MCU is awake and you call manager.recvfromAck() it'll route messages. I effectively call it from loop(). The rest is handled by the library. For your initial test, I'd just keep the MCU always on until you get the repeater functionality going. Here's the description from the library: https://www.airspayce.com/mikem/arduino/RadioHead/classRHMesh.html
Starts the receiver if it is not running already, processes and possibly routes any received messages addressed to other nodes and delivers any messages addressed to this node. If there is a valid application layer message available for this node (or RH_BROADCAST_ADDRESS), send an acknowledgement to the last hop address (blocking until this is complete), then copy the application message payload data to buf and return true else return false. If a message is copied, *len is set to the length.. If from is not NULL, the originator SOURCE address is placed in *source. If to is not NULL, the DEST address is placed in *dest. This might be this nodes address or RH_BROADCAST_ADDRESS. This is the preferred function for getting messages addressed to this node. If the message is not a broadcast, acknowledge to the sender before returning.
I'm still in the thick of things for another 4-6 weeks. Hoping to then catch up more on this thread and report my observations as well.
@chipmc - Two quick things… you probably are wondering how do you test this easily and how do you know if you are talking through the repeater or direct.
First, the Radiohead library has provisions to force it based on node address. Might be good to do a bench test forcing it before deploying. This is in RHRouter.cpp:
If you comment out RH_TEST_NETWORK=1 in RHRouter.H and your node address is 1, 2, 3, 4 then it’ll force the linear network. There are a few variations and I personally added a few based on my unique node address I was testing. It works for good bench testing before deploying in the field. Just make sure to comment out the line in the library when you want to do some final/actual tests.
Secondly… you likely want to know if it talked direct OR if it used the repeater. Easiest way is to capture “hops” just like you do RSSI and SNR. To capture hops just add it to your rcvfromAck().
This way… you can can then plot RSSI, SNR and hops. I personally capture the SNR and RSSI at the node and let the node send that data to the hub as part of the next message. This then truly represents RSSI between the node and repeater. Here’s the RSSI at the Hob, RSSI at the node and number of hops for one of my devices that keeps flip flopping between using the repeater and talking direct.
@jgskarda I know this is crunch time for you (we all need our Maple Syrup after all) so I really appreciate your taking the time to lay this out.
When you get a chance, or if anyone else wants to chime in, your note brings up a couple questions.
I am capturing and reporting the RSSI and SNR already but I capture these numbers from the gateway’s perspective. When I am on-site and connected via serial to a node and via cellular to a distant gateway, I can see small differences between these numbers. Given both have the same radio and antenna, is one better than the other?
I am also currently capturing the hops using the same approach you laid out but I don’t report them via the webhook, my rationale was that this was not something that would change so why report it. However, in reading your posts, it is clear that when a node fails to make a connection, I will “rediscover” a route to the gateway if possible. So, do you see the number of hops change? Said another way, is your LoRA network static or dynamic in its topology?
The ability to force hops using the routing table is simply fantastic. I plan to play with this before heading out to Jordan Lake again.
One thing I am relieve to see is that one of the nodes does miss connections which is not surprising given the signal strength (-141/-17 !!!) but it will recover and reconnect on a later window. This gives me some confidence in the network and is the silver lining in a bad signal environment.
As soon as you add a repeater, you'll want to capture RSSI and SNR from both the gateway's perspective AND the node's perspective. When the #hops = 0 then the numbers should be close relatively speaking (see the trend below) especially when there is only 1 node and 1 gateway. In that case, you can use either. However as soon as you have a repeater in between, then the RSSI from the gateway's perspective is just the RSSI between the gateway and the repeater it is NOT the RSSI from the node to the repeater. Therefore, the only way to really tell the signal strength between the repeater and node is for the node to capture that when it receives a message from the repeater and then send that value with it's next transmission so it can tell the gateway. Not sure if I'm making sense... but I capture both sides. You can see the dynamic in this plot:
Related to this, you'll also want to make sure you "capture" the RSSI and SNR as soon a you receive the message intended for you. When you have neighboring nodes, airing out it's messages, it's my understanding RSSI and SNR is whatever the radio last heard. So you'll need to capture/hold the RSSI/SNR value when it processes a message for it self vs simply heard a neighboring node and ignore it. Make sense?
Unless you plan on statically setting the route in the routing table, the route will also be dynamic even with fixed device locations. As soon as it misses a single message, it will drop that route from the table and attempt to find a new route. I'd say it flip/flops between using the repeater and talking direct maybe 3-4 times a day in my use case. I've had a node 20' away form the gateway first talk 1,000' out to a repeater and then back before. It'll just find a route and keep using it until it misses a message and then find a new route. Unless you are doing every install yourself I wouldn't recommend statically setting the routing table. It seems easier for the end user that any node can be located anywhere than to try and tell someone, this specific node must talk to this node, etc.
Yeah, it's handy for bench testing for sure. One word of caution is testing with a forced route in the routing table may provide slightly different results then it having to find it's own route. It's great for testing and to learn but doesn't 100% transfer to a final install. One main difference is a route discovery message only attempts to find a route once. Whereas a forced route will retry x number of times (I set mine to 3) and thus using a forced route will produce more consistent results. Early on I was hopeful in having a stable 3-4 even 5 hops away when I forced the routes. This was successful. However, when I disabled that and used route discovery it just wasn't as consistent when trying to go 3+ hops away. And besides, it was VERY hard to have it self discover a route to 4 hops, it almost always found a 2-3 hop route or didn't talk at all no matter how far I went.
Impressive!!! I'll have to query the database, but I don't think I ever saw a -17 SNR. Maybe -8. To me, this is what I suspect the benefit of SF11/SF12 is. You can still talk with very low RSSI and with a signal well below the noise floor.
Thanks for all of this. I have been able to rewrite my code to take advantage of mesh and I have further refined the radio settings. Here is how things work now:
All the nodes wake together and go into listening mode for 5 minutes. Unconfigured nodes talk first with their Join requests. As all the nodes are listening, new nodes are free to find a route to the gateway.
Then the configured nodes talk with each off-set by 10 seconds * their node number. This leaves a fair bit of “empty air” but, I think that is OK given my low node density.
Once a node transmits, it comes back to the listening state until the window is over
Finally, the nodes go to sleep and the gateway connects to Particle and sends it cached web hooks.
RSSI and SNR are reported by the node now and the number of hops is reported as well.
A few changes based on more testing:
Using the larger spreading factor for LoRA means the transmissions take longer. This was causing issues with different devices stepping on one-another and the gateway seemed to miss messages, likely due to timeouts. Here is my new configuration that strikes a balance between he defaults and long range:
driver.setModemConfig(RH_RF95::Bw125Cr45Sf2048);
driver.setLowDatarate(); // https://www.airspayce.com/mikem/arduino/RadioHead/classRH__RF95.html#a8e2df6a6d2cb192b13bd572a7005da67
manager.setTimeout(2000); // 200mSec is the default - may need to extend once we play with other settings on the modem - https://www.airspayce.com/mikem/arduino/RadioHead/classRHReliableDatagram.html
I also simplified and sped up retries to get them handled by simply adding a straight random offset to the delay.
My next step is go (back!) to Jordan Lake and try these items:
The new software - including the new radio settings
A super long SeeeeeeedStudio 8dBi omnidirectional antenna on the gateway raised another 6’
Testing putting a repeater node on the far side of the lake from the gateway to connect to the gates.
I will share more as I try these next steps in building out this solution.
Great update @chipmc. Thanks for sharing! Good stuff!
What do you mean by this? Did you change this in the library? Are you doing this within your application code or within the library itself? Just curious to learn more as it's my understanding the library handles the retries already but without any kind of random offset.
Yes, the “exponential” backoff meant that the more tries, the more time between tries. This meant that it took a while for a device to “timeout”.
So, made it simpler too. I added a state specifically to introduce delay. So, in the transmit state, I track the result
if (result) {
retryCount = 0;
state = LoRA_LISTENING_STATE;
}
else if (retryCount >= 3) {
Log.info("Too many retries - giving up for this period");
retryCount = 0;
state = LoRA_LISTENING_STATE;
}
else {
Log.info("Transmission failed - retry number %d",retryCount++);
state = LoRA_RETRY_WAIT_STATE;
}
Then, there is a state to randomize the delay.
case LoRA_RETRY_WAIT_STATE: { // In this state we introduce a random delay and then retransmit
static unsigned long variableDelay = 0;
static unsigned long startDelay = 0;
if (state != oldState) {
publishStateTransition(); // Publish state transition
variableDelay = random(20000); // a random delay up to 20 seconds
startDelay = millis();
Log.info("Going to retry in %lu seconds", variableDelay/1000UL);
}
if (millis() >= startDelay + variableDelay) state = LoRA_TRANSMISSION_STATE;
} break;
Also, I am not making use of the library for retries. Are you?
Fantastic posts and info! Thank you. Takes me back to my 900MHz modem design days with range of over several miles.
The Adafruit LoRa RFM95W are based on the older Semtech chipsets.
The newer Semtech SX126x transceivers offer lower power & improved power amplifier that can transmit up to +22dBm, with a total link budget up to 170dB for extended range.
Rakwireless offer SX1262, though not in the convenient Featherwing format. I plan to use these soon. It’ll be interesting to compare real-life performance of SX1262 vs. RFM95 (SX127x) chipsets.
I went back (again!) to Lake Jordan State Park, to see if I could improve the reliability of the reporting for the two gates. I believe I have a solution that, while not 100% perfect, may well be close enough. The only variable will be the effect of foliage has on the connections.
For this round, I implemented a number of improvements / changes:
I installed a HUGE antenna with 7dBI gain and another 3’ of elevation for the gateway. I had to modify an existing mount and 3D print the mounting hardware (more on this if anyone is interested). Here is a picture of the new installation with the monster antenna. This seemed to make a big difference in the SNR values I read across the lake.
I decided to change the radio settings as the 4096 chips setting was taking way too long to transmit and the Radiohead documentation warned of heating in the LoRA module causing frequency drift. I with with @thrmttnw’s recommendation of 2048 chips that did not seem to have a big impact on the signal but seemed to keep better timing. Going forward, these will be my settings:
const double RF95_FREQ = 926.84; // Center frequency for the omni-directional antenna I am using (works for both short and HUGE antennae)
driver.setFrequency(RF95_FREQ); // Frequency is typically 868.0 or 915.0 in the Americas, or 433.0 in the EU - Are there more settings possible here?
driver.setTxPower(23, false); // If you are using RFM95/96/97/98 modules which uses the PA_BOOST transmitter pin, then you can set transmitter powers from 5 to 23 dBm (13dBm default).
driver.setModemConfig(RH_RF95::Bw125Cr45Sf2048);
driver.setLowDatarate(); // https://www.airspayce.com/mikem/arduino/RadioHead/classRH__RF95.html#a8e2df6a6d2cb192b13bd572a7005da67
manager.setTimeout(2000); // 200mSec is the default - may need to extend once we play with other settings on the modem - https://www.airspayce.com/mikem/arduino/RadioHead/classRHReliableDatagram.html
I changed the firmware to enable all the nodes to be in listen mode for 5 minutes in order to repeat any messages that are not getting to the gateway. So, all my nodes are also repeaters. Following @Rftop’s advice, I installed a dedicated repeater near the beach for the weakest node - Ebenezer - thing is, as of this writing, the Ebenezer node has not yet failed to reach the Buffy new gateway so it has yet to start relaying. I suspect this will change in the coming days and I will see as I am reporting “hops”
I updated the two nodes with the new firmware that implemented the software settings above and the new reporting as suggested by @jgskarda
Next week is my biggest challenge as I will be heading into NC’s mountainous West to install this solution at Grandfather Mountain. Given how smoothly this installation went, I am cautiously optimistic.
As you can see form this post, I could not have made this progress without your collective advice and support - thank you all.
First an update on installations of LoRA and their long term reliability.
The installation at Lake Jordan is has now been running for 6 weeks and during that time we have gone from winter to full foliage. I was worried that the greenery would degrade the signal but it has not had much of an impact. So, 6-weeks of not one hourly data packet lost. This is a good thing as the gateway is 15’ up a pole and would be a pain to reset.
We also completed an installation in the mountains that required some impressive tree climbing by the park staff. Unlike the Lake Jordan install, this one was in the mountains of NC so this was not long distances but it was around a granite mountain. The biggest challenge here was to place the gateway high enough to connect to the two nodes and to get the high-gain cellular antenna high enough to connect to the weak cellular signals in this part of the park. While these runs are fairly short, the terrain and dense woods made me nervous about this install. Especially since the gateway is essentially out of reach. This one has been up for a month now and is also proving to be very reliable.
So, we have covered two use cases for LoRA + Particle:
Installations where nodes cannot get cellular signal but are within LoRA range of a point that can.
Installations where there are a large number of nodes close together - like @jgskarda has done with the Sap Spy.
Installations where you cannot install a solar panel and where long battery life is important.
It is the third use case that I wanted to show here. I have been working with a local town to install soil moisture sensors to help open and close the town’s soccer fields based on soil conditions. With cellular, I needed to place an enclosure on the side of the field with a solar panel and a 3m wire to the buried sensor. This did not allow measuring soil moisture on the field itself and the buried sensor and wire was hard to maintain. With LoRA, I can install the node below ground in a valve box and place it on the pitch itself. Turns out LoRA signal still reaches a gateway about 500 yards (457m) away.
The issue here is that it is possible for this box to fill with water so, I think I am going to need to pot this sensor. It has been very reliable in reporting however.
I hope these successful installs inspire more activity around LoRA. It solves some key use cases and - coupled with the improved cellular performance (and new carriers coming!) could make Particle solutions possible in just about any location.
Very Impressive Update Sir !
Especially a Solar Powered Boron on the top of a Mountain with the Panel half way up a tree
You have ReDefined my opinion of what a “Remote” Installation is now.
I assume you get some nasty weather up there in the winter also ?
Again, great update and thanks for taking the time to share.
Great update @chipmc! The amount of work a customer/end user will go through to make the device connect to cellular is a testimony to the value your device adds to their lives. Like you, I've had some "creative" installs in low signal strength areas but I've yet to mount a device quite that high up a tree!
This is great! Overall seems like you achieved stability.
I love the soccer field use case for IoT and LoRa in general. It's making the judgement call a coach has to make if they can play or not tangible with data and I suspect not having to drive to the field! Not sure of the business case for a sensor per field, but I'd think different areas of a soccer/sports complex have different drainage. Maybe each soccer field on a complex needs a LoRa node. I'm sure the guy who mows the lawn is appreciative of not having the wire or post in the way. Good stuff!
I've been meaning to provide an update here as well the last few weeks. Overall, I had a very successful Maple Syrup harvest season using Particle + LoRa. The statement "Particle + Lora Better together" continues to prove itself to me! We had 100+ Particle+LoRa node systems in operation all talking to several hundred individual LoRa nodes "in the wild". In general, this was split across devices in a few different setups/node counts.
50% - had 1-2 LoRa Nodes. I have 1-2 other locations that I want to monitor but I don't have power, I don't have cellular where I need it or I don't want to have to deal with another cellular subscription.
30% - had a moderate 3-10 nodes within LoRa range. I have a smaller woods but I want sensors everywhere.
20% - used between 10-30 nodes in a system within 1 sometimes 2 LoRa hops of the gateway. Our largest was 30 nodes all talking to 1 Particle + LoRa gateway.
As an example, here is a "typical" system: Particle + LoRa is upper left. All the red dots with a number by them are LoRa nodes. In this scenario, they all normally talked directly to the hub. Occasionally ML300 (bottom right) would use a repeater to talk that far due to the terrain. This system has been going since early January and I just checked on it today and still 100% operational not skipping a beat.
Overall, I was very pleased with the performance and capabilities. The Particle + LoRa continues to provide significant advantages over other connectivity options outlined earlier in this thread. You could make an argument for LoRaWAN in larger systems but I think the competing factors is 1) Cost of an order LoRaWAN cellular gateway is significant compared to Particle. 2) Requires full LTE cellular connectivity and higher power requirements. 3) Does not allow repeating a signal over terrain/longer distances. 4) Super simple for the end user to connect a device (just turn it on).
My biggest takeaways and next steps from the last few months of learnings include:
As I add more and more nodes (i.e. more than say 10+). I need a larger power budget for the repeaters since repeaters need to stay awake longer to service the system. I will either revisit the channel activity detection (CAD) and see if I can get that working to lower battery consumption when listening, add more battery capacity and/or add a larger solar panel for those larger systems.
Having more information on the LoRa routing table would be useful. On the largest installs out there, a device may drop off for a little bit. At times it would take 2 even 3 LoRa hops. I knew the last hop a message took but didn't know signal strength between earlier hops. I likely will have repeaters publish their routing table or try and add each node address a message visited along it's journey. Just to provide more insight into the data. Would be useful to see not only number of hops but who it visited along the way when commissioning a system.
There are some edge case out there that locked up the radio on the Particle + LoRa gateway. Only happened a handful of times across all devices but did require a device reset to recover from.
Will likely add a self reset to nodes if it hasn't heard from the hub in x hours with an exponential backoff.
Hope to keep the collaboration going here in this thread with @chipmc as well as others. I'd love to hear other use cases for Particle + LoRa as well. Happy to share my learnings/observations if you have any specific questions.
@jgskarda Great work on this and its been super helpful in my LoRA + Boron work. I will admit I have not read every single post in this thread, but i'm curious... When you show the units on the map, how did you pinpoint their locations. Was this done offline knowing the landscape? GPS? or did you somehow use LoRA to triangulate its position.
The reason I ask was I wanted to know if there was a way to use LoRA to gauge a position of another node over a significant distance. I have a feeling that radio waves are too fast to discern with any real accuracy.
@krixen1 That’s a really good question… LoRaWAN does support geolocation in the range of 10-50 meters accuracy. It uses triangulation as long as multiple LoRaWAN gateways all heard the same message and provide the exact time stamp when it heard the message to the network server. That’s LoRaWAN though.
For my use cases with Particle + LoRa, I only have 1 Particle device per location, everything else is LoRa nodes and therefor no ability for triangulation. From my perspective, it would be a lot easier to just add a GPS to each node then try and figure out the triangulation using multiple gateways. Would be good for both precise time synronization as well as determining location. It’s crossed my mind to add GPS but generally speaking for my use case once a node is deployed the location of the node itself doesn’t change. It didn’t seem worth the added hardware cost and complexity to add GPS. I may revisit adding GPS in the future for other use cases though.
As for displaying sensor data on the map... in the specific industry I am in, many users already use onX hunt to map out their woods. Rather than rebuilding the entire mapping process in my own web app, and having customers remap their locations in a different application, I simply have them use onX to add waypoints for each device as well as lines for each line the device is monitoring. They then name each waypoint and line within OnX. OnX allows them to export their entire configuration as a KML file. I then upload that KML file and convert it to geojson. I then store the GeoJSON for that sugarbush in my database to use in my web app. I use the name assigned in OnX to then map it to the same name assigned to each sensor node. From there, I can then animate the lines/waypoints based on the vacuum level that was sensed using standard features from Map Box. Here's my little infographic on it:
And here was a preliminary tutorial video walking someone through the process:
@summitsurf - Helped get me started with the initial framework using MapBox. If you are looking for further guidance on mapping he might be able to get you started with some building blocks as well.
I plan on adding a few new features in the web app this upcoming season to make this a little easier. Hope that helps!
Hi all, I have an off-grid forested property that is going to be getting bat boxes and other wildlife-friendly improvements that I want to monitor. There is one spot where I have cell service, but most of the property has no signal, so a mesh network is an ideal solution. Has someone consolidated all of your collective wisdom onto a web page somewhere with the latest info on hardware & setup? Thanks!
I am currently rewriting my code to work on the following:
a LoRA gateway with a Particle Boron
a low-cost LoRA node that is optimized for outdoor use
I would be happy to share in the next week or two when I have it all working. I know Jeff is very open to sharing as well. Perhaps we could do something like what @gusgonnet did with the "Lessons Learned" Wiki.
Good luck with the project and I hope our work is helpful to you.
Great, looking forward to it. Just knowing that you have gotten something similar working already is a huge time saver, since I know where to start. I will also be happy to share whatever customizations I make.
Yeah, we certainly have something similar working well and deployed. The setup in my personal woods reference in a few of the screen shots throughout this post as been up and active since Feb. It's been 100% hands off and all nodes are still talking today. Battery life on the nodes depleted a bit over the summer with full canopy but I expect in the next few weeks the leaves will start falling and batteries should fully charge again before winter settles in.
I've thought about doing a hackster post or full/well documented github project but as always, there are pieces to clean up yet. Here as a WiKi might work as well.
A few quick questions...
How many acres is the property that you are trying to cover?
How hilly is the terrain?
How many nodes are you thinking throughout the property?
Depending on that, you might not even require a Mesh or repeater capability. You might be surprised with the range capability of LoRa and would simplify a few things not requiring repeaters.
Acres: ~60
Hilly: Fairly hilly. The property is up in the mountains and extends over a gully and a hill, but it's also forested with pretty good sized trees.
Nodes: Unclear. I expect to start with 5-10 but depending on how crazy I go and what else I can do with them (call detection, proximity?), it will likely go up from there. I realize the bandwidth is really low, so some things (photos) are not realistic.
You may be right about the mesh part since the Boron will go on the top of a hill, but the back side is fairly steep and I may not get full coverage without some forwarding/repeaters due to that and the trees.
In terms of other questions, I purchased a Boron and I have a solar panel I'm planning to repurpose, but what do you suggest for LoRa modules? Thanks!
This collaboration is great! Hope to eventually catch up on things and provide a bit more detailed response but in the meantime wanted to at least comment on this:
