Boron LTE Fails to Connect

boron
Tags: #<Tag:0x00007fe21df8e5b0>

#21

We are aware of the issue described here. I’m uncertain of its relevance to the issue you are experiencing, but is something engineering will certainly investigate. Particle already has existing communications open with ublox in regards to known bugs and discussions on means of handling firmware update for our in-use ublox modules.


#22

Hi @mstanley,

I’ve got three devices with this problem. They blink green trying to connect to the cloud forever. They were running 1.1.0-rc.2 when encountering the issue. Updating them to 1.2.1 does not fix the issue.

clouddebug: press letter corresponding to the command
a - enter APN for 3rd-party SIM card
k - set keep-alive value
c - show carriers at this location
p - select MNO profile
t - run normal tests (occurs automatically after 10 seconds)
or tap the MODE button once to show carriers
starting tests...
turning cellular on...
0000015009 [system.nm] INFO: State changed: DISABLED -> IFACE_DOWN
0000036158 [hal] ERROR: No response from NCP
deviceID=e00fce68f378b52473295f7d
manufacturer=
model=
firmware version=
ordering code=
IMEI=
IMSI=
ICCID=
0000117160 [app] INFO: enabling trace logging
attempting to connect to the cellular network...
0000117164 [system.nm] INFO: State changed: IFACE_DOWN -> IFACE_REQUEST_UP
0000117164 [system.nm] INFO: State changed: IFACE_DOWN -> IFACE_REQUEST_UP
0000117168 [hal] TRACE: PPP netif -> 8
0000117169 [net.ifapi] INFO: Netif pp3 state UP
0000117169 [net.ifapi] INFO: Netif pp3 state UP
0000117171 [hal] TRACE: Powering modem on
0000117172 [system.nm] INFO: State changed: IFACE_REQUEST_UP -> IFACE_UP
0000117172 [system.nm] INFO: State changed: IFACE_REQUEST_UP -> IFACE_UP
0000117322 [hal] TRACE: Modem powered on
0000117323 [hal] TRACE: Setting UART voltage translator state 1
0000118325 [ncp.at] TRACE: > AT
0000119326 [ncp.at] TRACE: > AT
0000120327 [ncp.at] TRACE: > AT
0000121327 [ncp.at] TRACE: > AT
0000122328 [ncp.at] TRACE: > AT
0000123329 [ncp.at] TRACE: > AT
0000124330 [ncp.at] TRACE: > AT
0000125330 [ncp.at] TRACE: > AT
0000126330 [ncp.at] TRACE: > AT
0000127331 [ncp.at] TRACE: > AT
0000128332 [ncp.at] TRACE: > AT
0000129333 [ncp.at] TRACE: > AT
0000130334 [ncp.at] TRACE: > AT
0000131335 [ncp.at] TRACE: > AT
0000132335 [ncp.at] TRACE: > AT
0000133336 [ncp.at] TRACE: > AT
0000134337 [ncp.at] TRACE: > AT
0000135338 [ncp.at] TRACE: > AT
0000136338 [ncp.at] TRACE: > AT
0000137339 [ncp.at] TRACE: > AT
0000138340 [hal] ERROR: No response from NCP
0000138340 [hal] ERROR: No response from NCP
0000138342 [hal] TRACE: Setting UART voltage translator state 0
0000138344 [hal] TRACE: Hard resetting the modem
0000149345 [net.pppncp] TRACE: Failed to initialize ublox NCP client: -210
0000149347 [hal] TRACE: Setting UART voltage translator state 0
0000149349 [hal] TRACE: Modem already off
0000149450 [hal] TRACE: Powering modem on
0000149600 [hal] TRACE: Modem powered on
0000149601 [hal] TRACE: Setting UART voltage translator state 1
0000150603 [ncp.at] TRACE: > AT
0000151604 [ncp.at] TRACE: > AT
0000152605 [ncp.at] TRACE: > AT
0000153606 [ncp.at] TRACE: > AT
0000154607 [ncp.at] TRACE: > AT
0000155608 [ncp.at] TRACE: > AT
0000156609 [ncp.at] TRACE: > AT
0000157610 [ncp.at] TRACE: > AT

Serial Inspect output

Platform: 13 - Boron
Modules
  Bootloader module #0 - version 311, main location, 49152 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 1213, main location, 671744 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      Bootloader module #0 - version 311
  User module #1 - version 5, main location, 131072 bytes max size
    UUID: 362808E0DB24B4ED309227C9121675856A32E73D5D16D2C7922C49C9AEF353C2
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 326

These three devices are in the field with 17 others for a R&D project we are running with Borons. All three experienced this issue after running completely out of battery. They were not powered by a LiPo battery, but were powered by a 12V SLA battery run through a regulator.

All three of these devices had this problem after running the 12V SLA completely out of power (down to ~6V). I think this may have something to do with that? Maybe a modem brownout?


#23

Hi Heath,

Those logs are fantastic, thanks so much for providing those! I have ensured that engineering will be taking a look at these.

We are still investigating this issue on our end. We have a device OS release coming sometime next week to mitigate some of the Boron LTE fail cases. As that comes out, we will be posting a new community topic for the release and instructions on how to capture additional debug logging so that we can continue to investigate connectivity concerns.


Boron LTE Stuck Flashing Green
#24

Not sure if this helps, but our carrier board can sit either a Boron or Electron and we have about 20 of each in the field. None of the Electrons have failed this way (or any other way).


#25

Hi Heath.

Certainly insightful. At the moment, our upcoming release candidate for v1.3.1 addresses a Boron specific issue. If you continue to notice this issue post v1.3.1-rc, it is helpful to know that this issue is still exclusively impacting Borons.


#26

Hi folks, we have released device OS v1.3.1-rc.1 to tackle some of the issues causing difficulty to all the users here. Please refer to Status Update for cellular connectivity issues | 8/21/19 for instructions on how to provide feedback and debug logs. Thanks!

@hwestbrook @trpropst


#27

@hwestbrook - Did OSv1.3.1-rc.1 fix your issue? I had a device on my test bench that has very similar looking cloud debug logs “Trace->AT” that started up flashing green. The RC didn’t fix it.

I discovered this week that I had 6 devices in the field also permanently flashing green. I’ve only taken one out of the weatherproof case (I’ve had issues with case screws stripping out, so I try to avoid it), and it also had a similar looking clouddebug log. THis is an old thread I understand. I’m delayed, because my devices are mostly only used in summer, I assumed the devices were offline because the operator had shut off the main power, but I was surprised to see they were all flashing green even after power cycles. Out of 15 devices, 6 are experiencing this behavior.


#28

@jonpasc,

It did recover a few of them, and didn’t others. I didn’t have time to dive much deeper into the issue, and Particle replaced a few of them for me.

The main takeaway for these LTE boron’s without the updated ublox firmware is to avoid using them in situations where the power is not guaranteed.


#29

Did the 1.3.1-rc1 and 1.4.4 update that ublox modem firmware to your knowledge? I’m trying to get to a build where I can set and forget. I had one device fail on solar and 6 of a different type of setup on a power source that’s either on or off for months at a time. The cell reception is spotty.


#30

To my knowledge, Particle does not have the ability to update the ublox firmware through device-os. If you search elsewhere in the forum you’ll find people have made some custom PCBs for performing this.


#31

Thanks for pointing that out. I found Ublox modem firmware update with very specific directions. Fortunately, I’m in an off season until March, so I can sit on this a while longer and see if Particle finds a solution. I’ve been 0 for 7 in bringing devices that started flashing green to life with an upgrade to 1.4.4 using “particle update” in the CLI. Unfortunately, i’m not sure the root cause is addressed and it’s just a matter of time before my other deices fall into the same state.