We are aware of the issue described here. I’m uncertain of its relevance to the issue you are experiencing, but is something engineering will certainly investigate. Particle already has existing communications open with ublox in regards to known bugs and discussions on means of handling firmware update for our in-use ublox modules.
I’ve got three devices with this problem. They blink green trying to connect to the cloud forever. They were running 1.1.0-rc.2 when encountering the issue. Updating them to 1.2.1 does not fix the issue.
clouddebug: press letter corresponding to the command a - enter APN for 3rd-party SIM card k - set keep-alive value c - show carriers at this location p - select MNO profile t - run normal tests (occurs automatically after 10 seconds) or tap the MODE button once to show carriers starting tests... turning cellular on... 0000015009 [system.nm] INFO: State changed: DISABLED -> IFACE_DOWN 0000036158 [hal] ERROR: No response from NCP deviceID=e00fce68f378b52473295f7d manufacturer= model= firmware version= ordering code= IMEI= IMSI= ICCID= 0000117160 [app] INFO: enabling trace logging attempting to connect to the cellular network... 0000117164 [system.nm] INFO: State changed: IFACE_DOWN -> IFACE_REQUEST_UP 0000117164 [system.nm] INFO: State changed: IFACE_DOWN -> IFACE_REQUEST_UP 0000117168 [hal] TRACE: PPP netif -> 8 0000117169 [net.ifapi] INFO: Netif pp3 state UP 0000117169 [net.ifapi] INFO: Netif pp3 state UP 0000117171 [hal] TRACE: Powering modem on 0000117172 [system.nm] INFO: State changed: IFACE_REQUEST_UP -> IFACE_UP 0000117172 [system.nm] INFO: State changed: IFACE_REQUEST_UP -> IFACE_UP 0000117322 [hal] TRACE: Modem powered on 0000117323 [hal] TRACE: Setting UART voltage translator state 1 0000118325 [ncp.at] TRACE: > AT 0000119326 [ncp.at] TRACE: > AT 0000120327 [ncp.at] TRACE: > AT 0000121327 [ncp.at] TRACE: > AT 0000122328 [ncp.at] TRACE: > AT 0000123329 [ncp.at] TRACE: > AT 0000124330 [ncp.at] TRACE: > AT 0000125330 [ncp.at] TRACE: > AT 0000126330 [ncp.at] TRACE: > AT 0000127331 [ncp.at] TRACE: > AT 0000128332 [ncp.at] TRACE: > AT 0000129333 [ncp.at] TRACE: > AT 0000130334 [ncp.at] TRACE: > AT 0000131335 [ncp.at] TRACE: > AT 0000132335 [ncp.at] TRACE: > AT 0000133336 [ncp.at] TRACE: > AT 0000134337 [ncp.at] TRACE: > AT 0000135338 [ncp.at] TRACE: > AT 0000136338 [ncp.at] TRACE: > AT 0000137339 [ncp.at] TRACE: > AT 0000138340 [hal] ERROR: No response from NCP 0000138340 [hal] ERROR: No response from NCP 0000138342 [hal] TRACE: Setting UART voltage translator state 0 0000138344 [hal] TRACE: Hard resetting the modem 0000149345 [net.pppncp] TRACE: Failed to initialize ublox NCP client: -210 0000149347 [hal] TRACE: Setting UART voltage translator state 0 0000149349 [hal] TRACE: Modem already off 0000149450 [hal] TRACE: Powering modem on 0000149600 [hal] TRACE: Modem powered on 0000149601 [hal] TRACE: Setting UART voltage translator state 1 0000150603 [ncp.at] TRACE: > AT 0000151604 [ncp.at] TRACE: > AT 0000152605 [ncp.at] TRACE: > AT 0000153606 [ncp.at] TRACE: > AT 0000154607 [ncp.at] TRACE: > AT 0000155608 [ncp.at] TRACE: > AT 0000156609 [ncp.at] TRACE: > AT 0000157610 [ncp.at] TRACE: > AT
Serial Inspect output
Platform: 13 - Boron Modules Bootloader module #0 - version 311, main location, 49152 bytes max size Integrity: PASS Address Range: PASS Platform: PASS Dependencies: PASS System module #1 - version 1213, main location, 671744 bytes max size Integrity: PASS Address Range: PASS Platform: PASS Dependencies: PASS Bootloader module #0 - version 311 User module #1 - version 5, main location, 131072 bytes max size UUID: 362808E0DB24B4ED309227C9121675856A32E73D5D16D2C7922C49C9AEF353C2 Integrity: PASS Address Range: PASS Platform: PASS Dependencies: PASS System module #1 - version 326
These three devices are in the field with 17 others for a R&D project we are running with Borons. All three experienced this issue after running completely out of battery. They were not powered by a LiPo battery, but were powered by a 12V SLA battery run through a regulator.
All three of these devices had this problem after running the 12V SLA completely out of power (down to ~6V). I think this may have something to do with that? Maybe a modem brownout?
Those logs are fantastic, thanks so much for providing those! I have ensured that engineering will be taking a look at these.
We are still investigating this issue on our end. We have a device OS release coming sometime next week to mitigate some of the Boron LTE fail cases. As that comes out, we will be posting a new community topic for the release and instructions on how to capture additional debug logging so that we can continue to investigate connectivity concerns.
Boron LTE Stuck Flashing Green
Not sure if this helps, but our carrier board can sit either a Boron or Electron and we have about 20 of each in the field. None of the Electrons have failed this way (or any other way).
Certainly insightful. At the moment, our upcoming release candidate for v1.3.1 addresses a Boron specific issue. If you continue to notice this issue post v1.3.1-rc, it is helpful to know that this issue is still exclusively impacting Borons.
Hi folks, we have released device OS v1.3.1-rc.1 to tackle some of the issues causing difficulty to all the users here. Please refer to Status Update for cellular connectivity issues | 8/21/19 for instructions on how to provide feedback and debug logs. Thanks!
@hwestbrook - Did OSv1.3.1-rc.1 fix your issue? I had a device on my test bench that has very similar looking cloud debug logs “Trace->AT” that started up flashing green. The RC didn’t fix it.
I discovered this week that I had 6 devices in the field also permanently flashing green. I’ve only taken one out of the weatherproof case (I’ve had issues with case screws stripping out, so I try to avoid it), and it also had a similar looking clouddebug log. THis is an old thread I understand. I’m delayed, because my devices are mostly only used in summer, I assumed the devices were offline because the operator had shut off the main power, but I was surprised to see they were all flashing green even after power cycles. Out of 15 devices, 6 are experiencing this behavior.
It did recover a few of them, and didn’t others. I didn’t have time to dive much deeper into the issue, and Particle replaced a few of them for me.
The main takeaway for these LTE boron’s without the updated ublox firmware is to avoid using them in situations where the power is not guaranteed.
Did the 1.3.1-rc1 and 1.4.4 update that ublox modem firmware to your knowledge? I’m trying to get to a build where I can set and forget. I had one device fail on solar and 6 of a different type of setup on a power source that’s either on or off for months at a time. The cell reception is spotty.
To my knowledge, Particle does not have the ability to update the ublox firmware through device-os. If you search elsewhere in the forum you’ll find people have made some custom PCBs for performing this.
Thanks for pointing that out. I found Ublox modem firmware update with very specific directions. Fortunately, I’m in an off season until March, so I can sit on this a while longer and see if Particle finds a solution. I’ve been 0 for 7 in bringing devices that started flashing green to life with an upgrade to 1.4.4 using “particle update” in the CLI. Unfortunately, i’m not sure the root cause is addressed and it’s just a matter of time before my other deices fall into the same state.