No, never used a 3rd party SIM, but thank you very much for your suggestion, as I didn’t know this, but this is good to know for the future!!!
Interference can be source from a wide range of things, and can persist. it's not unreasonable to believe that some new environmental factor was introduced somewhere between your location and the tower and causes continuous interference.
Particle devices are intended to be able to function indoors. However, sake of diagnostic purposes, we may ask users to try and troubleshoot by attempting to connect outside as a means of eliminating possible variables that may cause interference.
Based on the diagnostics and evidence thus far, I can confirm beyond reasonable doubt that the device OS is initializing as intended, and is initializing the ublox module as intended as well. We can see that the ublox module is performing expected behavior of trying to register a tower, but is unable to find an operator tower willing to accept it.
Without being able to fully reproduce the environment, I cannot say for absolute certainty where the fault lies here, but evidence suggests that the issue lies in talking out to the tower, and not the device OS initialization.
The included antenna for the 3G Electron that we provide is the PC104. For the Boron LTE, we provide the FXUB63. There are some overlapping bands, but the FXUB63 supports ranges outside the PC104. It may be worth trying for troubleshooting purposes--and if it gets online, it definitely suggests a problem with your LTE antenna. if it doesn't get online though, that doesn't provide anything conclusive.
Feel free to provide the Electron's device ID to me and I can look into its behavior on Particle's cloud to get some perspective on why this device may be falling short. It may not hurt to run electron-clouddebug on this device as well.
It should be noted that the Electron/E Series 2G/3G services are handled by an entirely different carrier and operator than the Boron LTE. As such, each are relying on completely different infrastructures.
To reinforce this point further, the +CEREG: 2,2 AT commands clearly are stating that the modem is in a state that is searching for operators and unable to find an operator able to even talk its frequency to negotiate with. If it were able to find a tower that were denying it, it should be reporting back +CEREG: 2,3 -- which is an explicit statement that a tower was found, but rejected the SIM.
Interference can still be a cause here, or simply general coverage. Any anecdotal cellular evidence from the Electron is not applicable to the Boron as they rely on different cellular infrastructures.Since the v1.1.x release, though, it should be noted that the Boron and Electron operate under the same Device OS, which seems to suggest against device OS being the culprit, here.
We can see about potentially having the unit returned to diagnose further if we are unable to conclude cause.
We have no issue with replacing determined faulty devices. However, we like to have reasonable cause before replacing units. In this particular situation, I am hesitant to replace the unit yet as issues to this point point to cellular coverage or interference. I would be fearful of a replacement unit exhibiting identical symptoms--and this would put us back in the same situation we are in now.
Evidence in our cloud side logs supports this. We saw OTA flashing occurring today.
Assuming you flashed all new parts, the outdated versions should not have had an impact.
I understand that there are some bugs with Tinker on the mobile app and Gen 3 devices. Our mobile teams are working on redesigns to accommodate this. I wouldn't read too much into it not working, here.
Can you elaborate on this behavior? What do you mean by "it will occasionally connect, but very quickly loses connection? Please describe the LED status on this, and any behavior you notice from the console.
Are you flashing your own application after flashing Tinker?
As a general response in regards to ordering new units and replacing the existing one--if you are to order replacement units and acknowledge that they behave better and more reliably than this unit in a side-by-side test (or determine the antenna to be faulty, we can absolutely get a replacement out your way, or refund you the cost of one of the new units you ordered after the fact.
Replacement units are handled by our fulfillment team--as well as newly purchased units. As today is a holiday, none of Particle's staff is scheduled to be working, and no orders will be processed until tomorrow at the earliest. As such, I do encourage you to place an order now for timeliness. We can always place retro-active refunds or replacements as mentioned above.
Ok, so I finally have some more helpful information. I am testing both my own software and tinker on the Boron, but I am NOT reporting ANY of these issues, based on my experience running my own code, only with Tinker, so we have a consistent baseline.
So here is where I am at:
The Boron, with Tinker OR my own code, with a known working 3G antenna or its current LTE Antenna is still frequently disconnecting in the same location and others as well. The 3g antenna does seem to be less prone to dropping, but it still is consistently dropping more than before on previous firmware builds. The LTE Antenna is pretty much unusable with Tinker or my own code.
So either there is some absolutely insane interference, as moving around my house or outside has not made a difference here, a cell tower has been down for two weeks, there is a firmware/bootloader/system part issue within the newest build 1.21.-rc.1, or this device has some bad components in it. Pick your poison.
My money is still on the firmware/bootloader/system part or possibly bad parts in the boron. I don’t believe the antenna to be the issue anymore, as the 3g one, should not be having the same issues still, while running tinker.
If it ends up being your software, I spent 10+ hours troubleshooting your guys buggy software, which I didn’t have the spare time to do this weekend, as I need to be working every hour possible, on prepping for a demo with your hardware, which does not feel good at all, as a customer, and even worse as a product developer. Hope I’m wrong though🤕.
I just ordered some new hardware a little while ago, with your quickest shipping, so I will know in very short order, if it is the device, or firmware.
It would be really nice, if you guys could just send me an extra one, even if it means putting a hold on my card for the amount until this one is sent back. We are now at over 10+ hours troubleshooting here and I am officially done, as we have not gotten anywhere with this, and it is taking time away from other work now, which is not worth it by a long shot. This has cut way too far into my life for a $60 device, that as I stated originally, began behaving this way, after upgrading to the newest firmware, not sure what else to say. I would really appreciate you guys taking some responsibility here.
I still really do very much appreciate your fast responses and continued diligence on this, and if I wouldn’t have had the two emails I sent to the support staff ignored, I would be far less frustrated here, but enough is enough. This is what I mean when I say you guys are strangling your products potential. Could you imagine if in this same scenario, this product was deployed to customers? They are going to expect a replacement before they give you 10+ hours of their life or they will just abandon the product entirely, and if I am a product creator would be expected to swap their units out within a week to 2 max, because that is what I feel is reasonable here, we would have already failed royally, and more than likely, lost a customer. I need you guys to do better than this in the future, paid program or not. Thank you again!!!
-Spencer
Update: The new hardware arrived. Thank you for shipping it out so quickly!!! I am still having a ton of issues with latency with the cloud and my boron devices ONLY getting stuck trying to connect. My Electrons are working on 1.2.0, which was the last stable firmware for me, but borons are still having the same issues on this firmware too, which makes me think this is something on the backend that is causing this.
All the borons I have tried running both as products and individual devices, that are running any of the 1.2.x(includes1.2.1-rc.2), have been an absolute mess trying to keep connected to the cloud!!!
Only way I have been able to maintain a consistent connection to the cloud has been on 0.9.0, which was rock solid in the same location, further proving this is all software related!!!
As I ate a bunch of money rushing shipping new hardware, only to find out, it has been 100% software or backend related this whole time, is really a huge slap in the face, as I have now easily spent 15-20 hrs of my own time troubleshooting your newest rollout, which if I could describe in one word, GARBAGE!!!
I am really sorry to be so blunt, but there is absolutely no excuses for how bad this has been at this point. There are absolutely zero down cell towers on ATT’s network in my entire city, as I checked that too.
My final diagnosis, is that this issue is be related to the new Intelligent OTA and something with 1.2.x. Is there any way, you guys can take Intelligent OTA back to the drawing board and pull this feature, until it is in way better shape than it is now, or at least temporarily turn it off on my account??
Thank you!!!
Hi All,
Just wanted to drop a note that I’ve reached out privately via email to help.
Thanks,
David
2 Likes
It blows my mind that Particle is spending a second of their time working on ANYTHING BUT this ridiculous issue. I am facing similar issues which have required me to drive 2.5 hours both ways to my remote wilderness station which has imperfect though ABSOLUTELY EXTANT LTE signal.
On 1.1.0, after installing a high gain industrial $120 cellular antenna 8ft above ground which costs more than 2 Borons itself, my unit is FAILING to reconnect to cloud. It died after a few hours. I have it on automatic mode.
This was after a similar FAILURE on 1.2.something firmware to maintain the connection, on a physically distinct Boron unit which had semiautomatic and reconnect code built into the loop. This previous one had lasted a day or two, then permanently lost the connection and never returned.
At EVERY event, physically resetting i.e. power cycling the device has caused immediate successful cloud reconnects. That is ridiculous, absurd, and frustrating. That basically proves it is a software problem.
There was before all of this an earlier run at the same station. This was using a shitty antenna, and the Boron successfully uploaded every minute for about a solid month from Feb to March. I’m shooting myself for not knowing the exact firmware BUT it was probably older i.e 0.9 which would make sense.
Incredibly frustrating that my first deployment resulted in connection kept for a few weeks. Then my second manual intervention (new antenna, new Boron) lasted a few days before connection permanently lost. Then my third and most recent (new Boron, automatic mode, industrial grade 10dbz Proxicast cell antenna) last for a few HOURS.
Can Particle PLEASE address these insane issues and help me? I would love to keep using this product if it will work. I need to be able to deploy an lte board and not drive 5 hours all the time to debug/reset it in person.
Before you ask, the power supply has been impeccable for my Borons. Current board has a LiPo attached.
Any help?
1 Like
Hi folks - Jumping in here to provide an update and to give you confidence that our team is aware of and working to resolve the issue. Apologies for any frustration this has caused.
The issue is complex and our product & engineering teams are still working to diagnose the issue such that we can provide a fix as quickly as possible. That effort includes working with our partners (carriers and hardware suppliers) as well as digging into recent Device OS releases.
I imagine many of you are working directly with support engineers, please continue to do so. We will also be updating this thread as we make progress on diagnosing and plans to address it.
Thanks for your patience and we will keep you posted.
Dan
3 Likes
Hey @Paul_M (and other folks who may be having issues as well)
To add onto Dan’s statement, we are aware of the issue and are making perpetual efforts to try and get to the bottom of the issues at hand.
Just under 2 hours ago we release a new release candidate, v1.2.1-rc.3 which includes a number of fixes for LTE based devices. Should you still be experiencing issues, I highly consider upgrading to the latest device OS version and relaying findings here or in a support ticket at support.particle.io
1 Like
@mstanley @danjamieson Dan and mstanley - thank you for the reassuring response and know that your efforts to fix this are extremely profitable for retaining me. I REALLY like the Particle infrastructure and do not want to have to abandon it because of this.
What I want to ask you is as follows. Can you specifically state what the issue is which I only anecdotally described above with a couple clues.
Is it such that all post 0.9.0 OS Borons will not retain rock solid connection in varying signal environments? Is it limited to certain carriers/towers?
I am about to drive out 2.5 hours each way to replace my Boron with a reverted 0.9.0 Boron. Should I do this as a temporary solution or wait?
Can you give more detail and a timeline?
This information alone will be very helpful. Thank you!
@Paul_M we unfortunately cannot yet definitively describe the issue you are experiencing. We have several hypotheses which we are working to rule in or out.
On 0.9.0, I can’t confirm this will solve the issue. If you’ve tested and seen that it’s stable in your environment, that’s certainly a possible workaround. As Matt mentions above, worth testing the most recent release as well.
Concerning timeline, until we’ve properly diagnosed the cause we can’t provide a meaningful timeline.
I can commit that the problem is on our radar and that reliability of the platform is always a top priority for Particle.
1 Like
Does that include the new ublox firmware?
The ublox firmware is not something that we package with our device OS updates. Updating the ublox firmware can be risky for multiple reasons. We are still pursuing avenues to best understand how to roll out ublox module updates as well as the implications to what these updates will be addressing. Active communications with ublox are ongoing between our engineering team with theirs.
2 Likes
@danjamieson
Dan, thanks for the response. tl;dr: I can’t even currently get a Boron to run v0.9.0 to test it out at my location, which would be a very useful test.
Unfortunately you updated your Particle mobile app to include the very unwise aspect of FORCING the user to update the device to the newest device OS as a part of the setup process (there is no “skip and update later” option).
This has incapacitated the following very-helpful test I was able to do for you and for me: I was going to drive out and place two Borons, a 0.9.0 and a new V1.3.0-rc.1, at the site, and take a look if both, one, or neither exhibit proper reconnection behavior or if both, none, or either permanently lose the LTE connection after some time.
Of course I tried manually flashing V0.9.0 device OS to the second test Boron using USB + Particle CLI, but then it would flash red invariably upon reset (SOS crash error). Only by running Particle Update and restoring 1.1.0 will it not crash. I haven’t tested trying to flash other old device OS releases but we sort of need to flash 0.9.0 because of the OP’s anecdotal experience and because of mine.
Any thoughts how I can obtain a Boron running 0.9.0 or how to flash mine without it blinking red afterwards? I did the proper particle flash --usb “system-part1-0.9.0-boron.bin” method and, while the flash is “successful”, it is failing i.e. LED blinking red upon any and every boot after said successful flash.
I don’t want to drive 5 hours only to test the 1.3.0-rc.1 - it will be worth my time if I can test that AND one of the old ones that WAS working for a solid month*.
(*= by the way, in my post above, about the old (I believe 0.9.0) Boron that WAS successfully connected for a solid month Feb to March: when the connection died, it was probably for a legitimate different reason. The antenna was crappy and was bent and rusted when recovered in May. There was also some sort of power event which caused the SD card in my other MCU to stop writing after that date. So clearly that Boron LTE connection death/permanent failure in March with the 0.9.0 Boron was quite likely for totally different reasons. It’s my wishful thinking that 0.9.0 Device OS maintains PERFECT LTE connection (i.e. RECONNECTING after inevitable here-and-there signal loss), and then there was a subsequent software error breaking in somewhere post-0.9.0, and you’re currently working on finding and fixing it. This view DOES have evidentiary support in the anecdotal experience of me and OP. It’s not necessarily true - yes, it’s possible this error always has been there - but it seems likely. Too bad I can’t do a side-by-side test of 0.9.0 - 1.3.0-rc.1 in situ, because I can’t downgrade a Boron to 0.9.0 without it flashing red and wholly failing…)
Hey there @Paul_M and @sdevo619,
Your frustration is completely understandable. I’m sincerely sorry this has been such a difficult process for you, as that is never our intention with our products or customer support experience. I am acutely aware and appreciative of the good will and patience you’ve shown in the pursuit of resolving this issue.
I want to start by emphasizing Dan’s prior comment, which is that we believe this to be a complex issue, or multiple issues, that is leading us to further investigate
- (1) differences between various Device OS releases,
- (2) differences in the AT command interface between the Boron (Gen 3) and Electron (Gen 2),
- (3) firmware running on the u-blox modem, and
- (4) characteristics of AT&T’s very new LTE M1 network itself (same expected coverage, potentially very different network behavior)
as potential root causes.
As an example – you have had better luck with LTE reliability on v0.9.0, while I previously observed symptoms on my personal Particle device similar to the one characterized by @sdevo619 (flashing green with +CREG 2,2 response in a previously working area) occur on v0.9.0 and seem to be resolved with an update to v1.2.0.
Next steps
-
@Paul_M, as a first step, let’s confirm whether or not the issue you’re seeing is the same or different from the issue from the OP (flashing green with
+CREG 2,2network response). I will help organize the team internally to provide very clear instructions for how to collect these logs and provide them to us so that we do not waste more of your time. -
In the future, to better understand how variables may be contributing, we’d like to make sure we have a complete picture of the device displaying the symptoms which includes:
- Device ID
- Hardware platform (Electron, E Series, Boron, B Series)
- Device OS version
- Application code (Tinker preferred as a universal control).
Other notes
-
To @paul_m and @sdevo619 – we are appreciative of the time you’ve already spent troubleshooting this issue and don’t want to waste any more of it. If you do not already have a “control” device at your primary workstation for reliability testing, please PM me and I’ll work with our support team to get you one.
-
@Paul_M, rather than testing with
v1.3.0which includes no LTE specific fixes and might introduce new risk variables with the introduction of BLE, please comparev0.9.0against1.2.1-rc.3(released yesterday) which includes various LTE-related bug fixes. -
@Paul_M this post includes instructions for upgrading and downgrading Device OS versions, but has not been updated to include the Boron yet. I will ask @rickkas7 to do so, so that you can get your existing device back down to
v0.9.0for comparison testing. -
@Paul_M for context, there is method to the decision to force update new devices to particular Device OS versions during setup. There are various software dependencies between the on-board hardware (nRF52840 and SARA-R410-02B) and Device OS that must be managed in order to ensure continued compatibility. More specifically, updating the Nordic SDK or u-blox modem firmware (which pulls in bug fixes for those components) occasionally requires compatibility modifications to Device OS that requires devices to run a particular Device OS version or later to function properly.
3 Likes
@will I am very thankful for the detailed response and please know this attention to detail is going to keep me an enthusiastic Particle customer; I am 100% on board in helping you fix this, and I want to have it fixed as much as you do. The Boron will be THE unbeatable LTE/Cellular IoT solution when we can fix this. I abandoned probably around $500 of Pycom GPy/FiPY boards when I found far-superior (i.e., less power consumptive and Arduino-like-programmable) Particle boards.
Unsurprisingly, unstable LTE was the biggest reason I trashed Pycom and went to Particle - BUT Particle is MUCH closer to having stable LTE, and I am confident our transaction here will achieve it.
Let me respond in turn and get you the info you requested:
1. I will benefit from the official Boron-OS-downgrade instructions. I’ve had further massive failures even on the older Boron to downgrade. I tried both --usb and --serial flashing the 0.9.0 bootloader as well. I either get red blinking crashes or I successfully install 0.9.0 only for a forced OTA update to 1.1.0 to inescapably occur. There’s no point in me giving further detail until you publish your official downgrade instructions for the Boron.
2. Once I can get one of my Borons as a permanent stable v0.9.0 test, I will happily drive out to my site and do the test. I have both installed as v1.2.1-rc.3 in the meantime. I will downgrade the older one instead of the newer one, because I think you have a good point about modem firmware differences. Said differences might explain why I got red blink crash post-0.9.0.-CLI-flash on the newer Boron, and a successful 0.9.0 boot but then immediately a forced OTA upgrade to 1.1.0 on the older one.
3. Here is the exact detail you requested on the physical Borons at play:
3.1 There are 3 Borons at play. Here is the history.
Boron A (e00fce68b429b007a5867b85) was deployed at my remote site on 2/16/19 and kept the connection until 3/23/19 using a nothing-special outdoor antenna. In my post above, I explicate the likelihood that OTHER and valid reasons (not the Particle software issue here) caused the 3/23 signal loss on this Boron. I THINK (but I’m not sure) Boron A had 0.9.0 when it was working.
Boron B (e00fce68a08527c53f132e5b) replaced Boron A on 5/18/19 along with a new, restored, functional antenna of identical make. Boron B was decisively post-0.9.0 when deployed. I think it was a 1.2.something beta. Whatever was newest around 5/17.
Boron B’s LTE connection lasted almost 4 full days in the same exact spot as Boron A, then the connection died shortly before 5/23/19.
The 6/10/19 Servicing Mission
On 6/10/19 I went back out, installed an even better outdoor cellular antenna, and rebooted Boron B. Boron B instantly reconnected and permanently lost signal a few hours later. Interestingly, it lost signal after 2 hours for about 30 mins, then came back ONCE for another hour or two, then permanently lost signal.
At this time, I wishfully thought it was my code which was causing the issue. I thought perhaps the SEMI_AUTOMATIC + THREAD_ENABLED foreclosed reconnection upon signal loss, even though I absolutely did have a if(!Particle.connected()) { //Make reconnect calls } in my loop. I also had a 60 second watchdog and wishfully thought maybe some sort of inescapable race condition was permanently killing my connection. I’ve subsequently learned these thoughts were false and indeed it’s the Particle internal software issue. My wishful thinking also didn’t explain why it worked for a solid month before.
The 6/13/19 Servicing Mission
Still aspirationally fantasizing for myself that I myself needed to do something better and it wasn’t a Particle firmware issue, I bought a yet-even-better 10db high gain super duper industrial outdoor cellular antenna and installed it higher on the tower. Cell signal was even better.
I went back out and replaced Boron B back with Boron A. This time, Boron A’s application code had no watchdog, no system thread, and was on fully automatic mode. Also updated Boron A to the newest OS, this time the newest stable release, 1.1.0.
NOTABLY, when I arrived, before I removed Boron B and reinstalled Boron A, a power cycle to Boron B + reset resulted in Boron B’s instant reconnection to the LTE network. At that point I knew it was probably a Particle software issue but I went ahead with my system_automatic + Boron A 1.1.0 install anyways.
Boron A permanently lost signal about 2 hours later while I was still driving back home.
Subsequently, the other day, I busted-out my new Boron C (e00fce68a7beb303ce9fe615) from the box and tried to install 0.9.0 as a test. See issues above. Only Borons A and B were actually deployed and exhibited the issue. I plan to deploy Boron B and C side-by-side 0.9.0 vs 1.2.1-rc.3 once you get me working downgrade instructions.
3.2 Application code at play.
If you want I can email you the application code. But the relevant facts are as follows.
All it does is take Serial1 input and upload to MQTT. It receives 132-byte packets from my Teensy 3.6 every 60 seconds and uploads.
Pre 6/13/19, I had system_thread and semi_automatic with connection code in setup() and reconnection code in loop() based off if(!particle.connected). Also had 60 second watchdog.
Post 6/13/19, I’ve commented out the relevant lines to make it 1. fully automatic mode (connection is fully on Particle firmware) and 2. no watchdog. So that pretty much guarantees the causality of the reconnection failure is in the Particle firmware.
4. I acknowledge the good cause you provided for the force-newest-update-upon-device-setup decision. It’s your decision. I also develop and publish commercial Windows software with its own forum and userbase so I am sensitive to the tradeoffs between increased user control vs. making sure people aren’t complaining about already-subsequently-solved issues.
5. Further steps
I am awaiting working downgrade instructions. Once I can get my Boron B into solid repeatable 0.9.0 state that won’t automatically OTA update, I’ll deploy it along with Boron C 1.2.1.rc-3. I’ll connect Boron C to my currently-in-use nice antenna and will use regular junk antenna for Boron B 0.9.0, as long as however I confirm it does successfully connect before I leave. Thus we will be testing the reconnect behavior, not the existence of working signal in the first place.
Another thing to note is that the LTE signal weirdly varies at this location. I made it upload RSSI values every 20 mins and noticed that the quality inexplicably would drop from 50% to 10% on 6/13 right before connection loss. This makes it a great site to test this issue because even with my nice antenna the signal will occasional drop out for a short time (which is completely acceptable, as long as it reconnects properly and automatically).
For reference, I do not believe the cell tower is AT&T. I could be wrong but I believe my site’s signal is from a Verizon tower in Eastern Massachusetts owned by Nextel: https://bit.ly/31BShxc So the AT&T variable is probably able to be isolated. When I was using Pycom GPy in the same site, I had a Verizon specific SIM to make LTE connection, however there could also be AT&T.
2 Likes
Due to the number of variables that exist in a device, downgrade instructions can be generically written, but small caveats may exist. I suspect you may be running into one of these cases.
The red flash SOS is curious if you are downgrading everything--including the bootloader. If you wouldn't mind, please provide me with the exact device OS version and user application you are making use of as well. I will setup the same environment on my Boron and attempt a downgrade to v0.9.0. That way I can catch any caveats personally and address them in downgrade instructions for your specific scenario.
In regards to be mandated to upgrade your device, would you mind providing me a bit more context on where you are finding yourself mandated to upgrade?
I know that our mobile application can mandate a forced update upon claiming. This is specific to the mobile application, the commandline interface will not mandate any sort of updates for the claiming process. Consider using the CLI, if possible, to go through your setup process in order to avoid updates via the mobile applications.
It also is possible to downgrade post-claim. Even if the device were updated, there should be no need to unclaim in order to downgrade.
You mention OTA, so my suspicion is the more likely scenario is flashing code to the device remotely. For this, ensure your target firmware version is set to the desired version. It's possible your may be targeting a later version, which may inadvertently be causing your user application updates to also update the device OS. I've included a screenshot below of the web IDE of where this option is.
I would like to make this investigation as exhaustive as possible. More information is better. Feel free to direct message this code to myself, or at @will and we will relay it for our engineering team.
if the tower is Verizon, it's unlikely our device is able to successfully establish connection to it. Our CAT-M1 LTE carrier in the United States is AT&T.
1 Like
@Paul_M I am having disconnection issues as well. I went through a Telco contact and said that there was an issue with the Qualcomm chip on the uBlox modem that caused the modem just to drop off in thin air for random periods of time. They said they they have seen it drop for over 5 hours and then just come back like nothing had happened… They further said that it was related to cell signal and in areas where the quality was quite high devices were fine, but as soon as the signal was not really good is when issues start to develop. My issue is very random 5 - 6 minute drop outs. I can almost count to the second when they come back on line. Initially though the tower was banning me for 5 minutes but the telco confirmed that was not the case.
They believe this issue with the chip set has been fixed in uBlox latest firmware so that’s what I am desperately hanging for. This is the release from ublox - https://www.u-blox.com/sites/default/files/SARA-R410M-02B-01_PCN_(UBX-19024506).pdf
For your info when the modem is off in noddy land on my devices for those 5 minutes it just flashes green also.
I have 40 Boron’s which are sitting under my desk that I can not release until this issue is sorted out. Maybe it might fix some of your problems as well?
Maybe @mstanley may have an indication of when this firmware update might take place. I know you said you were pursuing avenues how to update… but is there a time frame?
1 Like
A part of Particle's device OS attempts state management. If it's in a state that it's unable to connect for 5 minutes, the device will make efforts to recover the modem by restarting for whatever the cause may be. This is consistent with your statement and likely the observed behavior you're seeing (and why it's so predictable)
I am personally not apart of the diagnostics and evaluation of the u-blox module at this level. This is a part of our hardware engineering team, specifically. I understand talks have been ongoing between our hardware folks and u-blox to understand the issue better and that our hardware team has access to the later revisions.
Further testing is ongoing to understand if it will resolve the issue present in our devices. Once we have confidence in the newer ublox firmwares, discussions into how to roll it out will then commence.
Because of the inherit risks around updating, I understand it's wise to try to avoid flashing the ublox module more times than necessary. This is why engineering is being extra thorough in these discussions with u-blox.
In short: We don't have a timeframe yet, but it's an active priority to get one set.
1 Like
@mstanley THANK YOU for the reply and continued attention to this matter:
-
I have subsequently secured a reliable downgraded v0.9.0 Boron thanks to your instructions. Yes, it appears the Device OS Target being higher was forcing an OTA update. With that set to v0.9.0 and having flashed just system1 over usb/CLI, I now have my old Boron B (see above) at 0.9.0. This still doesn’t explain why I was getting the red flash while doing the downgrade with my newer Boron C, but that’s an irrelevant problem at this point. I have what I need to do the test.
-
I am going to do the following test. I will drive out and install Boron B (downgraded to 0.9.0) and Boron C (1.2.1-rc.3) in situ. Boron B will have “nice” cell antenna signal. Boron C will have at least a working connection before I leave with a new antenna. They will run the same application code. Then, we will see.
-
It was in the mobile application only that I was forced to update upon claiming. I haven’t used CLI for setup, only for manual flashing OS versions thereafter.
-
You’re probably right about the carrier so ignore what I said; it could be AT&T sharing the same tower or whatever.
I am excited to share with you the results in a few days.
In the meantime, is it know if electronically activating the Reset pin by a different MCU would reset an LTE-connection-dead Boron as a full power cycle would? I don’t have any anecdotal experience addressing this question. Does the Reset pin also reset the U-Blox modem? (Probably, these two questions are synonymous). If the Reset pin does do a total reset like powercycling, then I can live with this problem until it’s fixed by implementing a hardware watchdog using my other MCU. E.g. Boron would ping my MCU for every successful connected LTE publish, and if the MCU didn’t get a ping in 5 minutes it would reset the Boron by pulling Reset high.
@ric_hard Thank you for sharing your experiences which seem to relate the same problem I and OP are experiencing. I am happy to know you have 40 Borons sitting around and waiting for a resolution because that means I am certainly not the only one who needs this issue to be fixed, and to be able to have reliable LTE re-connection when signal goes and comes back.
I hope others will share their experience if they too are seeing such problems.
1 Like