Argon crashing if BLE client deepsleeps

I have a couple of Xenons acting as BLE clients to an Argon. Both of them wake up, read some sensor values, start BLE and sit waiting for a connection to be made by the central device. Once connected they do a setValue a few times before going into deep sleep.

The argon as the central device, crashes with a flashing red light after a few minutes (10 blinks). I previously thought it was because I was doing a BLE disconnect on a peer after receiving data, however I commented that out and set my peripheral devices to deep sleep after a certain time.

If I change the peripheral to use a delay rather than deep sleep and maintain the BLE connection, it is fine. I was able to run this for half an hour, but as soon as I told one of the xenons to deepsleep, the Argon fell over.

Using debugging statements, I have found that it seems to crash at different parts of the code, sometimes scanning, sometimes just in the general loop. It does allow both devices to connect once and it is usually a bit after a device has disconnected it fails. Could it be that the connection is not being cleaned up or perhaps that I’m not setting the BlePeerDevices to be global?

The code itself receives data in the call back and does some editing of the JSON to add in an API key. This was to prevent the need to hard code the API keys in each peripheral. As you can’t call Particle.publish in a call back, it puts messages on a stack and then services that stack in the main loop.

Any idea what could be causing the crash, or is this something that a Particle debugger would be useful for?

// Testing receiving BLE messages with an Argon.
// Eventually to publish them and use the Argon as a gateway, replacing a mesh connection


#define SCANRATE 5      // How many seconds 'rest' between scans

// Log over Serial USB
SerialLogHandler logHandler(LOG_LEVEL_WARN, { {"app", LOG_LEVEL_ALL} });

const size_t SCAN_RESULT_MAX = 30;
// Define UUIDs in use
const BleUuid testBLEservice("0ee45138-e516-4ed8-9668-a7848b797fa8");
const BleUuid jsonOutUuid("d692e9d8-1c92-4b3d-b387-801247613d66");

// Define BLE characteristic objects for call back
BleCharacteristic jsonOutChar;

//BleScanResult scanResults[SCAN_RESULT_MAX];
//BlePeerDevice peer;

// Store device name against API key
struct devAPI {
    String name;
    String apiKey;
    boolean disconnect;     // Disconnect client after data is received
};

// Get API key from Thingsboard->Devices->Device->Copy access token
devAPI apiKeys[] = {
    {"BLE-Xen1", "xxxxxxxxxxxxxxxxxxxxxxxx", true},
    {"BLE-Xen6", "yyyyyyyyyyyyyyyyyyyyyyy", false}
};

// Define a message stack. It is simpler to implement than a queue and incoming data should not be too heavy that
// out of order messages matter
#define STACK_LIMIT 10
#define MSG_LIMIT 255   // Define max size of message
struct stackObj {
    char jsonMsg[MSG_LIMIT];
    BlePeerDevice stackPeer;
};
stackObj msgStack[STACK_LIMIT];
int stackPointer=0;         // Empty stack
bool lockStack=false;       // Lock the stack - prevent a write while we are reading

// Define call back function prototypes
void onDataReceived(const uint8_t* data, size_t len, const BlePeerDevice& ble_peer, void* context);

void setup() {
    waitFor(Serial.isConnected, 3000);
    Log.info("Starting BLE Argon gateway");
    
    // Start BLE
    BLE.on();
    
    // Set up call back functions
    jsonOutChar.onDataReceived(onDataReceived, (void*)"json");
}

unsigned long nextScan=0;   // Forces an immediate scan
void loop() {
    // Check if there is data on the stack to process.
    if(stackPointer>0) {
        processStack();
    }
    if(millis()>nextScan) {
        // Scan for BLE devices
        Log.info("Scanning for new devices");
        BleScanResult scanResults[SCAN_RESULT_MAX];
        int count = BLE.scan(scanResults, SCAN_RESULT_MAX);
        for (int ii = 0; ii < count; ii++) {
            size_t len;
            uint8_t buf[BLE_MAX_ADV_DATA_LEN];
    
            // Scan for 128-bit service UUIDs
            len = scanResults[ii].advertisingData.get(BleAdvertisingDataType::SERVICE_UUID_128BIT_COMPLETE, buf, BLE_MAX_ADV_DATA_LEN);
            if(len>0) {
                // Loop in blocks of 16 bytes (128 bits)
                for(size_t jj=0; jj < len; jj+=16) {
                    
                    if((BleUuid)&buf[jj] == testBLEservice) {
                        // Found a new service
                        Log.info("  Found new device to peer with, name="+scanResults[ii].advertisingData.deviceName());
                        // Connect to access characteristics
                        BlePeerDevice peer = BLE.connect(scanResults[ii].address);
                        if(peer.connected()) {
                            //Log.info("  Connected to device");
                            // Call for random data
                            peer.getCharacteristicByUUID(jsonOutChar, jsonOutUuid);
                        }
                    }
                }
            }
        }
        // Set time for next scan
        nextScan=millis()+SCANRATE*1000;
        Log.info("Scan complete");
    }
}

void onDataReceived(const uint8_t* data, size_t len, const BlePeerDevice& ble_peer, void* context) {
    // Log.info("Got data back of length %d", len);
    if(context=="json") {
        // Copy the buffer to a character array on the stack. If we do this directly with a cast, as the string
        // is not null terminated, we can pick up extra data, especially if the string is shorter
        // than last time
        // We must use a stack as you can't Particle.publish or BLE.disconnect from within a call back
        // Particle.publish will cause a comm error and a disconnect causes a complete crash
        if(stackPointer<STACK_LIMIT) {
            // Check data size
            if(len>=MSG_LIMIT) {
                Log.info("Incoming message too large, ignoring: %d bytes", len);
            } else {
                // We may be processing a stack object, hold. This should be very brief and not a long wait
                while(lockStack) {};
                
                for(int i=0; i<len; i++) {
                    msgStack[stackPointer].jsonMsg[i]=data[i];
                }
                msgStack[stackPointer].jsonMsg[len]=0;    // Null terminate
                msgStack[stackPointer].stackPeer=ble_peer;
                Log.info("Incoming data on stack: %s", msgStack[stackPointer].jsonMsg);
                // Increase stack pointer
                stackPointer++;
            }
        } else {
            Log.info("Stack full, dropping data");
        }
    } else {
        if(len==2) {
            uint16_t value=data[0] | (data[1] << 8);
            
            Log.info("Data=%02X%02X, %d Type=%s", data[1], data[0], value, context);
        }
    }
}

void processStack() {
    // Process the message stack. Take the top object and publish encoded JSON.
    // Lock the stack so we don't get a write above this object from the call back function
    // Processing should not be too heavy in this function, we don't want to lock the stack for long
    lockStack=true;
    
    // Lower stack pointer as this points to the next free object
    stackPointer--;
    
    //Log.info("Processing stack object %d", stackPointer);
    //Log.info("  Stack data=%s", msgStack[stackPointer].jsonMsg);

    // Convert into a JSON object
    JSONValue jsonObj = JSONValue::parseCopy(msgStack[stackPointer].jsonMsg);
    // Iterate through object, should have 3 sections, sensorName, apiKey and payload
    JSONObjectIterator iter(jsonObj);
    //Log.info("Parsing JSON");
    String j_name = "";
    String j_api = "";
    String j_pay = "";
    boolean discon=false;       // Assume we will not disconnect it.
    while (iter.next()) {
        //Log.info("key=%s value=%s",  (const char *) iter.name(), (const char *) iter.value().toString());
        if(iter.name() == "sensorName") {
            j_name = iter.value().toString().data();
        }
        else if (iter.name() == "apiKey") {
            j_api = iter.value().toString().data();    // Don't actually need to do this as it should be unset
        }
        else if (iter.name() == "payload") {
            j_pay = iter.value().toString().data();
        }
        // Find API key
        int l=sizeof(apiKeys)/sizeof(devAPI);
        for(int j=0; j<l; j++) {
            if(apiKeys[j].name==j_name) {
                j_api=apiKeys[j].apiKey;
                // Set disconnect status based on lookup table
                discon=apiKeys[j].disconnect;
            }
        }
        // Note, if not found, the api key will still be an empty string.
    }
    // Disconnect sensor if flagged
    if(j_api=="") {
        Particle.publish("GW_ERROR", String::format("Unable to find API key for device %s",j_name));
    } else {
        String jout = String::format("{\"sensorName\": \"%s\", \"apiKey\": \"%s\", \"payload\":\"%s\"}",
            j_name.c_str(), j_api.c_str(), j_pay.c_str());
        Log.info("Outgoing from stack %s", jout.c_str());
        Particle.publish("BLEgwData", jout, PRIVATE);
        if(discon) {
            Log.info("Disconnecting client");
            //BLE.disconnect(msgStack[stackPointer].stackPeer);
        }
    }


    
    // Unlock the stack
    lockStack=false;
}

I can’t delete this follow up post (so editing) but hit this bug Ble.scan returning -270 using OS 3.1.0. The above example is OS 2.1.0.

Confirming that with the edits required to upgrade to v3.1.0, I still get the same error.

Sorry, it feels like I’m spamming my own topic and I know a lot of the regulars on here are in a different time zone.

I think I have found where the problem lies. Putting in a few more debugging lines into my code to make the connect:

            if(len>0) {
                // Loop in blocks of 16 bytes (128 bits)
                for(size_t jj=0; jj < len; jj+=16) {
                    
                    if((BleUuid)&buf[jj] == testBLEservice) {
                        // Found a new service
                        Log.info("  Found new device to peer with, name="+scanResults[ii].advertisingData().deviceName());
                        // Connect to access characteristics
                        //BlePeerDevice peer = BLE.connect(scanResults[ii].address);
                        Log.info("Trying to peer with device");
                        peer = BLE.connect(scanResults[ii].address());
                        Log.info("Peer attempt complete");
                        if(peer.connected()) {
                            Log.info("  Connected to device");
                            // Call for random data
                            peer.getCharacteristicByUUID(jsonOutChar, jsonOutUuid);
                        } else {
                            Log.info("Could not connect");
                        }
                    }
                }

I get the errors

0000095453 [app] INFO: Outgoing from stack {"sensorName": "BLE-Xen1", "apiKey": "HbEEQuhbzmjyUINYaemE", "payload":"{'randomNumber': 25997, 'temperature': 22.20, 'humidity': 74.60}"}
0000095455 [app] INFO: Disconnecting client
0000095583 [wiring.ble] TRACE: Disconnected by remote device.
0000095584 [app] INFO: Scanning for new devices
0000100537 [app] INFO: Found 8 devices
0000100538 [app] INFO:   Found new device to peer with, name=BLE-Xen1
0000100538 [app] INFO: Trying to peer with device
0000100944 [wiring.ble] TRACE: New peripheral is connected.
0000100945 [wiring.ble] TRACE: Start discovering services.

The scan result finds the Xenon even though it has gone into a deep sleep. It tries to connect to it and about 10 seconds later it fails with the assertion failure. The BLE.connect never returns. The sleep mode I was using on the Xenon was:

    SystemSleepConfiguration config;
    config.mode(SystemSleepMode::STOP)
        .gpio(WKP, RISING)
        .duration(SLEEPTIME);
    SystemSleepResult result = System.sleep(config);

I think the Argon crashing is a firmware bug. It it detecting the BLE advertisements running in sleep mode when scanning, but when it tries to connect and can’t, it crashes rather than return an error. I suspect the same thing might happen if it scanned a device that was turned off just before it managed to call BLE.connect().

But, I don’t really want to have BLE on when sleeping anyway. On the Xenon, I have done BLE.off() before it sleeps and that seems to have fixed the issue. I can see that ble() (SystemSleepConfiguration) can be used to keep BLE on.

What is the best way to sleep for a period of time and shut everything down, including BLE, and only leave the clock running to wake it up again?

Hi @DaveH,

Sorry for the delayed response. I’ll try to reproduce the issue first and then post my findings asap.

Cheers!

Thanks. If you struggle, I can probably produce a cut down version of both Argon and Xenon, that have only the bare minimum to cause this.

Hi @DaveH, could you try building the Argon firmware against this branch: GitHub - particle-iot/device-os at fix/ble_timeout/ch83983 and see if the issue goes away?

I will do, though I’m away at the moment so I’ll not get a chance for a couple of weeks

Sorry, is there a guide somewhere that details how to build firmware against a dev deviceOS?

I can see how to pull the chain and make the deviceOS locally. I use the webIDE to compile locally. What I can’t see is how to tell the web ide to use my local dev OS.

Hey @DaveH ,

The webIDE doesn’t support building your application against a custom Device OS. To build the Device OS locally basing on a particular branch, this doc may be more or less helpful: device-os/gettingstarted.md at develop · particle-iot/device-os · GitHub.

To summaries, after you install all the necessary tools (including Particle CLI) locally, you can follow this steps to build the Device OS locally:

  1. Pull and check the branch I mentioned above: git pull && git checkout origin/fix/ble_timeout/ch83983
  2. Navigate to the modules folder and run make PLATFORM=argon. It will generate the system-part1.bin for Argon under build/target/system-part1/platform-12-m/.
  3. Make your Argon enter the DFU mode and run particle flash --usb <path_to>/system_part1.bin to update Device OS on your Argon.