Handshakes / frequent disconnects

If anyone has time I could really use a hand…

I have a few questions to hopefully help me understand my incredibly frustrating situation a bit better. I am having the most awful time of it trying to figure out where my network instability lies.

Firstly, I’d like to say that my problems, frustratingly, seem to be random. I can’t marry up disconnects or packet loss with events on my network or with my “broadband” connection.

I’d also like to say I’m now running the Photons off a Ubiquiti UniFi AP…

Question: Does a recent handshake timestamp indicate a Photon or a Core has disconnected and reconnected? Or simply that it’s “checked in”? Anecdotally I would say it indicates a broken cloud connection that gets re-established but I’d like to have this confirmed.

Question: would putting two Particle devices within two feet of each other affect their network stability?

Question: Why might a device go for a day with rock solid uptime (no recent handshakes) and then disconnect/reconnect every half hour until it’s physically reset with the reset button?

Question: How could I check whether my network has constant access to the Particle cloud? With a ping? I’d like to correlate failed pings to the Cloud from a computer with handshakes if poss…

Question: Anyone doing MQTT with a Photon - have you needed to change the default
#define MQTT_DEFAULT_KEEPALIVE from 15 to another number? I’m worried that in so doing I might give the impression of a more solid connection but in reality I’m just allowing for a more lossy connection and masking the problem?

EDIT: It’s super bizarre. So this morning, in my house, I’ve physically disconnected from power a Photon. THAT Photon has had a stable connection now for three hours. The other three, which were NOT disconnected from power, all have the exact same recent handshake of 10:59. I don’t get it… Same WiFi access point, all about equidistant from it.

Thanks…!

@rickkas7 cloud wifi app might help...but I'd be looking more for something that would KEEP pinging both my router and the cloud and tell me which is failing and WHY I'm getting these disconnects/frequent handshakes

OK, the plot sort of thickens. There was nobody in the house all morning, and all devices were stable when viewed from the console. Within 10 minutes of my wife arriving home, suddenly I’ve got handshakes and disconnects coming out of my ears.

Could it be that the Particle devices actually have an intolerance to sharing a WiFi network with other traffic? Even if it’s simply a phone checking in with the access point? Which is a (low-end) enterprise grade AP, let’s not forget!!

I could try putting all the Particle devices on their own SSID and seeing if that helps?

I would start with the UniFi hardware. I also run UniFi APs in all 4 of my warehouse locations. They are pretty solid but have their quirks depending on your settings. Do you run a dedicated controller? I run the UniFi CloudKey but previously ran the controller on a shared server. Both have their issues and the CloudKey needs reset periodically which is a bit frustrating (it locks up and won't allow new connections until reset). I also have one AP in one location that always exhibits weird behavior (dropping clients, not allowing reconnects, etc.) and has to be reset via the UniFi controller. The controller could be your culprit. Do you have more than one AP? If so, there are some controller-driven features that you should not use such as the seamless roaming feature (I forget the actual name but the controller pushes devices off of one AP to favor another based on signal strength.)

Also, what Device OS are you running? Reading posts on here says that v0.7.0 has some issues. You are also running MQTT which also has many post of persons complaining about connection/device stability.

Thanks @ninjatill it's nice to have some input from someone with a similar setup.

I'm actually mostly not running the AP with a controller, it's just doing its own thing. Of course if I need to change any settings I fire up the java controller on my Mac purely for the 10 minute access window I need to get the settings modified then I close it down again.

I don't.

I'm targeting the default firmware and I will DEFINITELY be targeting the beta firmware when I next flash the devices.

It's true that the Core has massive problems with MQTT but I firmly believe that the Photon can do MQTT without issues. Indeed, I have personally had my Photon maintain an MQTT connection for an entire DAY (for me that is HUGE). But then the next day after changing precisely nothing about my setup I get constant handshakes blah blah blah.

I have an additional theory that I'm just trialling now. I have had a Raspberry Pi doing a bit of bluetooth presence detection location next to ONE OF my Particle devices. I'm wondering if that has been causing interference. However...that is only ONE of the devices and when they go offline, they tend to go offline and reconnect in a group. I've seen that referenced a few times before here on the forum.

Wow. Sorry for wall of text.

When you say they all go off and come back online with the same timestamp, I start looking at the networking stack/equipment instead of at the Particle devices themselves. Perhaps if you can recreate the issue when your wife walked into the house you could sniff the network traffic with Wireshark and see what is happening. Is your cable/DSL router stable? Do you have enough DHCP addresses? Are you running all switches and not hubs (should be all switches this day and age but I have to ask)? Is there any wi-fi device doing “rogue AP detection”? Is there any other wi-fi device that creates a duplicate SSID in the house? Do you have a separate SSIDs for 5G and 2.5G (do not have identical SSID for a mixed environment)? Is there any other SSIDs in range using the same channel? Try moving your SSID to a different, less-congested channel. Download a WiFi scanner to your phone and see what else is out there. If you have the UniFi controller running, there is a “Insights” area with a drop down selector for “Neighboring Access Points” that will show you the same thing. I like the scanner on my phone as it is more of a “live” look at the wireless environment.

Perhaps someone else can jump in with better ideas. I hope the networking questions get you a bit closer to finding the cause.

1 Like

If it is bluetooth interference, would wireshark help?

Broadband reasonably stable as far as I know, but I'd like to gather more info on that front.

The Unifi is doing rogue AP detection and hasn't found anything. Definitely no duplicate SSID. There used to be, weeks ago, but I turned it off in an effort to help the Particle devices.

I don't have separate SSIDs for 5G and 2.4G - should I? I figured that since it's a separate hardware radio doing the work, there was no point? VERY happy to try....

I did the Unifi site survey and I'm on the least congested channel already. As I understand it, the Unifi device is pretty darn good at moving channel and adapting to interference.

Really appreciate your help, @ninjatill .

Well, where to start with all that… definitely some areas to test! Perhaps do each one in sequence and give it some time to flush out before trying the next. In order of my gut instinct…

Start by putting the SSID onto a static channel. It could be that when another WiFi device enters the house, the congestion changes dynamically, and the AP jumps channels. As the devices pickup the SSID on another channel, it causes the behavior you are seeing. Try putting into static channel mode and see if that helps.

Next, try separate SSIDs for 5G and 2.5G. Same as above, when another WiFi device enters the scene it could cause the strongest signal to not be the strongest any more and cause devices to jump channels/bands.

Next, turn off rogue AP detection. I wanted to add onto this thought… I believe the UniFi APs (at least the cheaper models) do not have a separate radio to do the rogue AP detection. So they have to stop receiving WiFi traffic momentarily, do a quick scan of the environment, and then resume accepting traffic again. Normally, you don’t notice. But the Particle devices could be more sensitive to momentary delays or disruptions.

1 Like

Thank you!! I will report back. Am really hoping either one of these things will help or they will all help in combination. Feel so thwarted right now!!

I also want to know the answer for this question. And I also have Ubiquiti Unify AP here.
So for my Photon, I am using extra wifi antenna.
I still see some disconnects but fewer. And since handshakes are established fast enough, I just accept it :slight_smile:

Gah, I’ll be so annoyed if the Ubiquity Unifi AP is actually hindering rather than helping here! The ONLY reason I bought it was based on others suggesting it was more solid network gear and would help my IoT devices on my network…sounds more and more like I’ve just wasted £100! :slight_smile:

@netpex Some answers to some of your questions from my experience:

Question: would putting two Particle devices within two feet of each other affect their network stability?

Not seen this happen and have tested 10 photon on a tray next to one another. What it does highlight is the photon where the antenna has a slightly better gain! I now use an external antenna in all cases and switch the WiFi module to use the external antenna only. This generally gives a gain of 4dB, which doesn't sound a lot but is just over twice as strong.

Have you put a signal strength and quality monitor on your devices? I did this recently for some Photons deployed in a classroom - the impact of people in a room is amazing and as soon as school is finished the signal on each device is really stable. I am not sure what physical environment you are using your Ubiquiti UniFi AP.

Lastly, my recent experience with trying to get 11 photons online and stable suggests that the operation of the application has a significant impact on connectivity (loop cadence, interrupts and free memory). Once those issues were quelled the connectivity was near to 100%. I would definitely try using the diagnostics history with 0.8.0-RC.10 onwards - it gives you a starting point to identify when and why connection is being lost.

1 Like

Thanks - might look into this - am I correct that Photons come equipped for either onboard antenna OR external whereas with a Core you're stuck with what you bought?

For what it’s worth. I have a photon sitting on my desk connected to an RS232 barcode scanner and a UniFi AP (we have 6 APs in the building and one outside.) My photon never drops. I have minimal code running on it. You haven’t shown any code so we have no idea how robustly your code handles the connection. I see many posts on here that all leads back to user code… but some of your early diagnostics pointed to a WiFi environment issue.

I don’t think your money was wasted but the whole point of the UniFi is to run the central controller. In a residential application, I don’t think it makes sense. For that, you might as well buy an off-the-shelf router/AP like a Netgear or Linksys and use a repeater to get better range. But I do love the Ubiquity product line for their price point and enterprise features such as central management. Mind you, I have hundreds of persons walking into my building and the “Neighboring AP” shows hundreds of SSIDs because apparently every Chevy that drives by my building has a built-in AP now days (picked up by my outdoor AP).

Correct Photons have an onboard antenna and a u.Fl connector for an external WiFi antenna. I thought the Core has a u.Fl connector as well but I could be wrong, it is a while since I had the 1 Core I bought out.

Advice I got was to use a DrayTek Vigor 902 AP - slightly over £130 but has the ability to run a RADIUS server for WPA Enterprise. I’m sure if you have stuck with default settings you will be fine.

Like armor and also said in my previous post, I am using the same external uFL connector wifi antenna in all cases. I can buy it around USD2 here.

I am not blaming Ubiquiti Unifi here since all other devices are working just fine.

The Core came in two versions - chip antenna OR uFL.
There is no switching of antenna since there only ever is one

So last night I spent hours making various changes to the Unifi AP. Nothing made any difference whatsoever. Isolating 2.4Ghz IoT devices for example on their own SSID, static channels etc etc etc.

This morning, in desperation, I turned off bluetooth beacon scanning on a raspberry pi I have feeding presence info into Home Assistant, and so far…all quiet (no frequent shaking of hands). I’m going to turn ON bluetooth scanning again at lunchtime and if that suddenly causes a mass disconnect I think I’ve found my culprit.
If it IS Bluetooth, it seems to be only Cores that are susceptible to this interference…(not my Photons).
I’m a bit hesitant in posting this because it’s not a definite finger pointing yet!

1 Like

Yup. Seems to be a Raspberry Pi doing occasional bluetooth presence detection which is knocking the Cores (but not the Photons) offline.
Stable connection all morning and then within 5 mins of turning on the Pi, the Cores all get knocked offline.
Does that make sense to anyone?

1 Like

Hi @netpex

This sounds like a Raspberry Pi problem then. I have a few cores running still and they are in an office environment with several computers all of which use Bluetooth peripherals (mouse, keyboard, speakers, etc.) plus the odd fitness tracker and phone that come and go.

My core that runs a clock in that office stays up all the time and other than a small period of downtime when the router the cores are connected to gives them a new DHCP address, my clock just keep going. I do not have a Raspberry Pi that runs all the time in that room.