Device Failure - red flashing light

I am setting up my spark core to talk to a Bluetooth 4.0 device, sometimes it works but sometimes the system fails.
When I get a failure the device gives be about 3 quick red flashes, then 3 slightly slower flashes before it returns to its green then cyan startup sequence.

Does anyone know what this red flash sequence means?

Thanks

ArthurGuy, that sounds like the core is in PANIC mode, typically because you code exceeds available RAM. Are you compiling locally or on the web IDE?

1 Like

The Web IDE.

The problem seems to be related to the serial data coming back from the module and my processing of it, sometimes it works and sometimes it fails

ArthurGuy, if you are using Serial1 then it is presently lacking “hardiness” as I believe it is not using hardware interrupts yet. The Spark Team has this on it todo list. Can you post your Serial handling code?

I am running a few more tests to determine where in the program it fails but bizarrely so far it seems if I echo out the raw data the problem doesn't occur.

This is a snipit of the code

for (int i = 0; i < data_len; i++) {
buf[i] = Serial1.read();
Serial.println(buf[i]); //<- this seems to help
}

Hopefully a bit more testing will confirm this.

This is the chunk of code

if (Serial1.available())
    {
        //Serial.println(Serial1.read());
        type = Serial1.read();
        delay(35);
        event_code = Serial1.read();
        data_len = Serial1.read();
        
        int timeout = 0;
        while(!Serial1.available() && (timeout < 100)) {
            timeout++;
            delay(1);
        }
        if (timeout > 100) {
            Serial.println("Timeout");
        }
        for (int i = 0; i < data_len; i++) {
            buf[i] = Serial1.read();
            //Serial.println(buf[i]);
        }
    
        event = BUILD_UINT16(buf[0], buf[1]);
        status1 = buf[2];
  
        if (event == 1537)
        {
            //Scan complete
            Serial.print("Devices Found: ");
            Serial.println(buf[3]);
            Serial.print(buf[6], HEX);
            Serial.print(buf[7], HEX);
            Serial.print(buf[8], HEX);
            Serial.print(buf[9], HEX);
            Serial.print(buf[10], HEX);
            Serial.print(buf[11], HEX);
            
            Serial.println();
        }
        
    }

The failure point doesn’t seem to be a specific thing.
Can you see anything that might consume a lot of RAM?

ArthurGuy, I think your problem is related to how you Serial1.read() after the Serial1.available(). The latter returns the number of bytes actually available which you are not checking. At the top, you do 3 reads but don’t check if the data is actually available! If none is available, read() will return -1. So you may want to change your IF statement to:

if (Serial1.available() >= 3) {

That way you know those 3 consecutive reads will be valid. The same thing applies in the while() statement. You look for received data but not a specific number. So the code will continue even if one byte is available. If your data_len is -1 because of the previous issue, then all hell breaks loose! So with the IF fixed, the while should read:

while((Serial1.available() < data_len) && (timeout < 100)) {

Try those fixes and let me know how it goes. :smile:

Thanks for the suggestions but I am still having problems.
I refactored a bit of the other code but what I am currently using is below.
Can you see any other problems?

void detectDevice() {
    uint8_t data_len;
    uint16_t event;
    uint8_t buf[64];
    if (Serial1.available() >= 1)
    {
        Serial1.read(); //not interested
        delay(35);
        if (Serial1.available() >= 2) {
            Serial1.read(); //not interested
            data_len = Serial1.read();
    
            int timeout = 0;
            while((Serial1.available() < data_len) && (timeout < 100)) {
                timeout++;
                delay(1);
            }
            if (timeout > 100) {
                Serial.println("Timeout");
            }
        
            for (int i = 0; i < data_len; i++) {
                buf[i] = Serial1.read();
                //Serial.println(buf[i]);
            }
        
            event = BUILD_UINT16(buf[0], buf[1]);
      
            if (event == 1537)
            {
                //Scan complete
                Serial.print("Devices Found: ");
                Serial.println(buf[3]);
                
                Serial.print(buf[6], HEX);
                Serial.print(buf[7], HEX);
                Serial.print(buf[8], HEX);
                Serial.print(buf[9], HEX);
                Serial.print(buf[10], HEX);
                Serial.print(buf[11], HEX);
                
                Serial.println();
            }
        }
        
    }
}

ArthurGuy, can you please tell me which BLE device you are interfacing with? That will probably help a lot.

If I understand correctly, the first two bytes of the BLE response are garbage then comes the data_len byte and then the message of length data_len. If the detectDevice() function is continuously called, then the IF() statements make sense. However, if you call this function once to “detect” the BLE then the code needs be rewritten somewhat. Can you please tell me how detectDevice() is used?

Thanks for your help with this.
The device I am using is the BLE Mini (http://redbearlab.com/blemini/) configured into HCI mode to act as a controller.
They provide sample arduino code to interface with the device.
https://github.com/RedBearLab/BLE_HCI/blob/master/arduino/libraries/BLEHCI/examples/BLE_HCI_BLEShieldCentral/BLE_HCI_BLEShieldCentral.ino

I am initiating the device scan every 30 seconds and the continually calling detectDevice to look for the response.
It needs to be capable of handling random failures and continuing on after. My function looks like it should do this but the system is still randomly crashing.

I think I may have found the cause, i have added a bit of code which detects a pending buffer overrun and outputs a message, this then got printed out before every failure.

In my code I wait for all the bites in data_len to be ready but if this is above 64 then it seems like the system crashes rather than overriding the data or ignoring new data.

Have you heard of this bug before?

64 sounds like the buffer size given for the serial right now…

Does that imply you aren’t reading fast enough?

Its a problem I think I have solved as the data I am interested in comes through in a small chunk, less than 64 bites. If I detect that the bluetooth device is going to send me more I ignore it.

I think my previous assumption about the cause of the failure is wrong, still no idea what triggers the device to crash.

It’s best if you can replicate it with some code and the community can chip in to ‘catch’ it :smiley:

I have tried a couple of things which I thought were causing the problem but I couldn’t directly replicate it. The code is now working reliably which is good.

My only worry is that after crashing it didn’t seem to reboot correctly, the usb serial connection didn’t work, it might have been just this or it might have been other things as well.

Usually pressing the Reset would get everything back to normal.

Keep us updated and we shall see what the problem might be :smiley:

ArthurGuy, I will have to look at the BLEmini to get a better understanding but now that I’ve got a little more info, I would tell you that you need to clear the Spark receive buffer while you are waiting for the right data by doing “empty” reads so the buffer doesn’t overflow. However, it is really hard to help you if we don’t have a full picture of the code, especially if you are leveraging an existing arduino library. Can you post your code for us to look at?

This is the current function that runs every 100 milliseconds, I then trigger a search option every 30 seconds

void detectDevice() {
    uint8_t data_len;
    uint16_t event;
    int timeout = 0;
    uint8_t buffer[64];
    
    
    if (Serial1.available() >= 1)
    {
        Serial1.read(); //not interested
        delay(35);      //BLE device delay
        if (Serial1.available() >= 2) {
            Serial1.read(); //not interested
            data_len = Serial1.read();
            
            if (data_len > 60) {
                Serial.println("Probable Buffer Overrun");
                while (Serial1.available()) {
                    Serial1.read();
                }
                return;
            }
    
            while((Serial1.available() < data_len) && (timeout < 100)) {
                timeout++;
                delay(1);
            }
            if (timeout > 100) {
                Serial.println("Timeout");
                while (Serial1.available()) {
                    Serial1.read();
                }
            }
        
            for (int i = 0; i < data_len; i++) {
                buffer[i] = Serial1.read();
            }
        
            event = BUILD_UINT16(buffer[0], buffer[1]);
      
            if (event == 1537)  //Scan Complete
            {
                Serial.print("Devices Found: ");
                Serial.println(buffer[3]);
                
                if (buffer[3] > 0) {
                
                    Serial.print(buffer[6], HEX);
                    Serial.print(buffer[7], HEX);
                    Serial.print(buffer[8], HEX);
                    Serial.print(buffer[9], HEX);
                    Serial.print(buffer[10], HEX);
                    Serial.print(buffer[11], HEX);
                    
                    Serial.println();
                    
                    pendingScan = false;
                    
                    previousScanResult = scanResult;
                    scanResult = true;
                    
                    //We cant get a false positive so update imediatly
                    atHome = true;
                    
                } else {
                    pendingScan = false;
                    
                    previousScanResult = scanResult;
                    scanResult = false;
                    
                    //If the last two scans were false incase of a false negative
                    if (!previousScanResult) {
                        atHome = false;
                    }
                }
            }
        }
        
    }
}

I have added some buffer clearing loops when it detects a problem which has seemed to help, I am getting the odd problem but I am not sure what can be improved.

Does anyone know if there is a watchdog type timer which can recover the system incase it locks up?

HI @ArthurGuy

How do you know how the serial data is “framed” that is, where the start and end of the data packets are? It looks like you just assume the framing is always in phase between the BLEmini and the core. Is there a way you could look at those two discarded bytes and know that you have framed the data correctly? It looks very likely to me that if you miss data or wake-up in the middle of the data stream, the code cannot recover.

Another minor point, the current USART handling code in the core has a 64 byte circular buffer but that could change in the future. I think it is a slightly bad idea to do this:

            while((Serial1.available() < data_len) && (timeout < 100)) {
                timeout++;
                delay(1);
            }

since you are assuming the entire data_len of serial bytes can fit in the core firmware circular buffer.

Thanks for the comments, I am checking this data on a regular basis so the odd lost packet isn’t important but saying that I know what the first byte should be so I will add an additional check for this.

In the line before the snippet you mentioned I check the length of the incoming data packet, this tells me how many bytes are incoming.