Updating from DeviceOS 1.4.4 to 1.5.0-rc.1 -> SOS Panic 1

Have just PM’s the my two binaries to @avtolstoy for research.

@peekay123 I don’t know how complicated “AssistNow” is (nor what it does), but it strikes me that we may have an opportunity with its source code to work out the cause of the issue…

Updated bootloader from version 400 to 501:

Platform: 8 - P1
Modules
  Bootloader module #0 - version 501, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 1500, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 207
  System module #2 - version 1500, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 1500
      Bootloader module #0 - version 400
  User module #1 - version 6, main location, 131072 bytes max size
    UUID: 4962D89....
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 1500

Issue remains…

I’ve moved all my code out of STARTUP and am still encountering the same issue, tried an OTA update and that didn’t make a difference vs DFU.
I see mentions about SPI and should inform you I’m using both SPI buses on my P1 for I/O (SPI has an ePaper display ans SPI1 an SD card)

Attempting to flash the bootloader off github got me the following:

particle flash --usb p1-bootloader@1.5.0-rc.1+lto.bin
Error writing firmware: unknown module function 2, use --force to override

I ran an OTA flash of Tinker targeting 1.5.0-rc1 and got the following from an inspect

particle flash *** tinker --target 1.5.0-rc1  
? Which type of device? P1
attempting to flash firmware to your device ***
Flash device OK:  Update started

Flash success!

particle serial inspect
Platform: 8 - P1
Modules
  Bootloader module #0 - version 400, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 1500, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 207
  System module #2 - version 1500, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #1 - version 1500
      Bootloader module #0 - version 400
  User module #1 - version 3, main location, 131072 bytes max size
    UUID: ***
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 6

@TrikkStar, my experiments revealed that the bootloader version was not the issue.

I think we are circling the same SPI issue brought up by @avtolstoy because my kit also uses the same SPI bus to interface to both the

  • OLED display, and,
  • SDCARD

We should now chase this theory. Unfortunately, for me to “unhook” the display and SDCARD interfaces from my very complicated app is a nightmare.

Wondering if you are able to unhook these two interfaces from your application without too much pain? My guess (hope) is that it will work after this…

@peekay123, are you using any SPI interfaces in your errant app by any chance?

@UMD I don’t believe the @rickkas7’s GPS library or the asset tracker v2 uses SPI.

1 Like

It is the SPI mutex issue causing the immediate SOS+1 fault on 1.5.0-rc.1.

It’s the LIS3DH accelerometer that’s causing it.

panic_@0x0803b9a2 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/services/src/panic.c:68)
prvGetRegistersFromStack@0x08026baa (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/hal/src/stm32f2xx/core_hal_stm32f2xx.c:104)
<signal handler called>@0xfffffff9 (Unknown Source:0)
uxListRemove@0x08045a6c (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/third_party/freertos/freertos/FreeRTOS/Source/list.c:183)
xTaskRemoveFromEventList@0x08046980 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/third_party/freertos/freertos/FreeRTOS/Source/tasks.c:3114)
xQueueSemaphoreTake@0x08045f72 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/third_party/freertos/freertos/FreeRTOS/Source/queue.c:1479)
os_mutex_lock@0x0802be30 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/hal/src/stm32f2xx/concurrent_hal.cpp:387)
Mutex::lock@0x08025662 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/wiring/inc/spark_wiring_thread.h:246)
SPIClass::lock@0x080256a4 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/wiring/inc/spark_wiring_spi.h:246)
SPIClass::begin@0x080256a4 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/wiring/src/spark_wiring_spi.cpp:110)
LIS3DHSPI::LIS3DHSPI@0x080226cc (/Test/lib/LIS3DH/src/LIS3DH.cpp:297)
__static_initialization_and_destruction_0@0x08020c44 (/Test/lib/AssetTrackerRK/src/AssetTrackerRK.cpp:23)
_GLOBAL__sub_I_emptyResponse@0x08020c44 (/Test/lib/AssetTrackerRK/src/AssetTrackerRK.cpp:289)
call_constructors@0x0802b7a0 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/hal/src/electron/newlib_stubs.cpp:48)
LoopFillZerobss@0x080201f6 (/.particle/toolchains/deviceOS/1.5.0-rc.1/firmware-1.5.0-rc.1/build/arm/startup/spark_init.S:6)

@rickkas7, for some reason I thought the LIS3DH was using I2C. I guess this means a library update. The more reason to create the SPI_HAS_TRANSACTION #define in Particle.h and/or Arduino.h IMO.

2 Likes

It’s actually because I called SPI.begin() from the constructor, which shouldn’t be done. I’ll release new versions of the libraries to fix this, and also enable SPI transactions.

4 Likes

Good to hear that for your library, but it would be time to address this long standing issue about the Arduino SPI_HAS_TRANSACTION switch mentioned here

Over a year should have been plenty of time to add that single line of code to the device OS repo.

Excuse the sentiment, but I have lost count of issues I reported that never got a single official response.
Indifference is creeping in on my side for quite some time now.

4 Likes

I released version 0.3.2 of AssetTrackerRK that fixes the SOS+1 at startup with Device OS 1.5.0.

It contains LIS3DH 0.2.5 which actually has the fix; the problem was that SPI.begin() should not be called from a global object constructor. It also uses SPI transactions for compatibility with the Ethernet FeatherWing.

1 Like

@rickkas7, I can confirm that it is now working.

2 Likes

This is excellent news then - we have at least one root cause to the panic, ie SPI access

To close this case, it looks like I may need to update from SDFat 0.0.7 to 1.0.16 as per @ScruffR’s comments: DeviceOS 1.5.0-rc.1 - SoftAP using SDCARD - problem (assumption being that 0.0.7 may not be using SPI TRANSACTION or has SPI.begin() in the constructor???).

I note that I had tried quite some time ago to use the updated lib but came to grief for some reason.

As I had already modified Adafruit’s SSD1306 lib for SPI TRANSACTION usage, this should be sweet as it is.

Will report back with results…

Results are in:

First - removed SD CARD routines from my app - problem remains
Secondly - removed DISPLAY routines from my app - problem remains

Nothing else uses the SPI interface…

Conclusion

In my case, the issue is not related to SPI usage (as suspected by @avtolstoy here: Updating from DeviceOS 1.4.4 to 1.5.0-rc.1 -> SOS Panic 1)

Hmmmm… investigations to continue…

A small update: I have submitted a PR that resolves the SPI issues https://github.com/particle-iot/device-os/pull/2023.

@UMD Unfortunately, I was not able to figure out what causes the problem from the binaries you’ve provided. If you can find a way to provide a minimal application showcasing the problem, we can continue debugging.

1 Like

@avtolstoy, happy to report that the SOS #1 PANIC with DeviceOS 1.5.0-rc.1 with compiling to the same target 1.5.0-rc.1 issue has been resolved in my app.

The issue was traced down (after a lot of work and #ifdefs !) to my NFC code (which out of interest uses I2C).

I had a call to pinMode(D7,input) in the constructor. I moved this to the begin() method which is called in setup().

I will report back on any other issues found under separate tickets.

Thanks for the assistance, as per usual, greatly appreciated!

Case closed!

@avtolstoy here is the minimal source code for you to use to reproduce the issue…


#include <Particle.h>


#define CAUSE_PANIC

class Test
{
	public:

		Test();
	    void begin(void);
};

Test::Test()
{
#ifdef CAUSE_PANIC
    pinMode(D7,INPUT);
#endif
}

void Test::begin()
{
   
#ifndef CAUSE_PANIC
    pinMode(D7,INPUT);
#endif 
}


Test test = Test();

void setup() {
    
    test.begin();

}

@UMD Confirmed that this issue is also resolved by https://github.com/particle-iot/device-os/pull/2023. This is the same one that I’ve outlined in Updating from DeviceOS 1.4.4 to 1.5.0-rc.1 -> SOS Panic 1, except in the context of STARTUP() macro.

2 Likes

Excellent!

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.