Continuous losing connection

Over the last few days I have had big problems keeping my core online.

I got a AP which is only used for one spark core, and two electric imps. I made a page with just a javascript running and pulling data from the api on all of them.

Both of the electric imps are at 0.01% failures of all calls, where the spark core is currently at 9.32% and still counting up, so it is not just my connection dropping.

The spark core can do its breathing, then suddenly go into green flashing, rapid white flashing, then breathing again. Today it went even more crazy with going from breathing to green flashing to breathing, and just a few minutes after back go green flashing… Now it has then just sat there flashing blue for over 3 hours, and if the blue flashing state it is in right now (Flashing blue: Smart Config, waiting for Wi-Fi credentials), it looks like it would never be able to recover from that on its own.

The code running on it is nothing crazy

float temperature;
char tempStr[16];

void setup()
{
    Spark.variable("temperature", &tempStr, STRING);
}

void loop()
{
    readTMP36();
    
    float targetTemp = 23.8;
    float targetHyst = 0.25;
    
    pinMode(D0, OUTPUT);
    if (temperature < targetTemp-targetHyst)
    {
        digitalWrite(D0, HIGH);
    }
    else if (temperature > targetTemp+targetHyst)
    {
        digitalWrite(D0, LOW);
    }
    
    sprintf(tempStr, "%f", temperature);
}

int readTMP36value = -1;
unsigned long readTMP36last = 0;
void readTMP36()
{
    if (millis() - readTMP36last >= 500)
    {
        readTMP36last = millis();
        
        if (readTMP36value == -1) analogRead(A0);
        readTMP36value += analogRead(A0);
        readTMP36value /= 2;
        
        float voltage = (readTMP36value * 3.3)/4095.0;
        temperature = (voltage - 0.5) * 100.0;
    }
}

And I don’t see anything in this that should be able to mess with the connection, or wifi credentials if that is the case.

What can I do? I planned to use this in my garage to control the garage door, but with all the problems I have had so far with it, with it daily losing connection a few times and often not being able to recover after having failed flashing itself with new firmware, I don’t feel like it is worth spending the time getting it to work, when the brain behind it all seems to be the biggest problem in the project.

Hey @Mikey, it sounds like your problem is related to the Cyan Flash of Death. Have you read up on this LOOOONG thread? https://community.spark.io/t/bug-bounty-kill-the-cyan-flash-of-death/1322

There’s a bug in the CC3000 wifi chip that we’re waiting on TI to release a fix for. Thanks to the hard work from a bunch of community members and the spark team here, they’ve come up with some workarounds in the spark core firmware to get the wifi connection more stable, although it’ll still disconnect temporarily until we get the CC3000 firmware fix from TI. After that happens, you should see stability increase greatly.

I myself have also run into the other problem you’re describing where when the CC3000 pukes, and the sparkcore firmware reboots, sometimes it doesn’t fully reboot and goes into a flashing blue state, even though I know the wifi credentials are good, since the core was just connected. I see this happen very often on my work wifi network, but never at home. Since I can replicate this, I’m in the process of trying to get some debug information about where it’s happening that I can provide to the community and get some help with it.

I’ll be sure to post anything here if I can find anything.

Just had a look at it, and didn’t understand much of it, and a lot of it is not something I am planning on learning any time soon either.

So I guess bottom line is that the core isn’t ready to be put into remotely use yet?

I guess it depends on what you’re doing with it. I know some people are using it. I’ve also added on a hardware watchdog using an adafruit trinket that watches for activity from one of the spark pins, when it stops, it pulls the RST low to physically reset the core.

We were chatting about this on IRC as well, just for fun, here’s another tweak of that code:

int size = 5;
double temps[5];
double temperature =0;
int idx = 0;

int readTMP36value = -1;
unsigned long readTMP36last = 0;
 
float targetTemp = 23.8;
float targetHyst = 0.25;
 

void setup()
{
    pinMode(D0, OUTPUT);
    Spark.variable("temperature", &temperature, DOUBLE);
}
 

 
void loop()
{
    readTMP36();

    if (temperature < (targetTemp-targetHyst))
    {
        digitalWrite(D0, HIGH);
    }
    else if (temperature > targetTemp+targetHyst)
    {
        digitalWrite(D0, LOW);
    }
   
    // sprintf(tempStr, "%f", temperature);
}
 

void readTMP36()
{
    unsigned int now = millis();
    
    if (now - readTMP36last < 500)
    {
        return;
    }

    readTMP36last = now;
   
    temps[idx] = ((((analogRead(A0)*3.3)/4095.0)-0.5)*100.0);
    idx++;
    if (idx >= size) {
        idx = 0;
    
        double total = 0;
        for(int i=0;i<size;i++) {
            total += temps[i];
        }
        temperature = total / size;
    }
}

For other people finding this thread, there was a stability update of the firmware this last week that included a lot of improvements from the Community. So please try modifying some whitespace on your code file (so it re-compiles), and re-flashing firmware to update.

The newer firmware may still be knocked offline sometimes, but it should do a better job recovering.

Thanks!
David

The new firmware is much better about keeping your core connected to the internet.

I’m constantly testing it and only 1 time did I have the spark core flash blue and not reset. So while its much more stable the only real 100% sure way to make sure it automatically resets is the use a Hardware or Software based watch dog timer that will automatically rest the core if your main loop freezes up.

You can learn how to do both methods in this thread. https://community.spark.io/t/can-the-spark-core-trigger-the-reset-pin/2693

Or you could always give up for now and come back in 30 days to run the latest code. The Spark will be rock solid shortly, to many smart people working on it for that not to happen.

I was a little frustrated that the core was not 100% solid from day one but kinda fell in love with all the possibilities it offers, I’m sticking around.

1 Like

I’ve got a debug log for when my core goes into blue-flashing mode after reconnecting a few times. @david_s5, @BDub does this mean anything to you guys?

0001698344:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001713571:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001713574:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001738678:<DEBUG> int Spark_Connect() (599):sparkSocket Now =0
0001738678:<DEBUG> int Spark_Disconnect() (637):
0001738682:<DEBUG> int Spark_Disconnect() (646):Close
0001742288:<DEBUG> set_socket_active_status (810):Sd=0, Status SOCKET_STATUS_IACTIVE
0001742290:<DEBUG> int Spark_Disconnect() (648):Closed retVal=0
0001742296:<DEBUG> set_socket_active_status (810):Sd=0, Status SOCKET_STATUS_ACTIVE
0001742303:<DEBUG> int Spark_Connect() (606):socketed sparkSocket=0
0001742309:<DEBUG> int Spark_Connect() (627):connect
0001742375:<DEBUG> int Spark_Connect() (629):connected connect=0
0001742439:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 40
0001742633:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 384
0001742940:<DEBUG> set_socket_active_status (810):Sd=1, Status SOCKET_STATUS_ACTIVE
0001742944:<DEBUG> set_socket_active_status (810):Sd=1, Status SOCKET_STATUS_IACTIVE
0001743454:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001743457:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001758524:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001758526:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001773605:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001773607:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001788675:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001788678:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001804065:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001804068:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001819155:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0001819157:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0001844265:<DEBUG> int Spark_Connect() (599):sparkSocket Now =0
0001844265:<DEBUG> int Spark_Disconnect() (637):
0001844269:<DEBUG> int Spark_Disconnect() (646):Close
0001847874:<DEBUG> set_socket_active_status (810):Sd=0, Status SOCKET_STATUS_IACTIVE
0001847876:<DEBUG> int Spark_Disconnect() (648):Closed retVal=0
0001847882:<DEBUG> set_socket_active_status (810):Sd=0, Status SOCKET_STATUS_ACTIVE
0001847889:<DEBUG> int Spark_Connect() (606):socketed sparkSocket=0
0001847895:<DEBUG> int Spark_Connect() (627):connect
0001855900:<ERROR> hci_event_handler (248):Timeout now 1855900 start 1847900 elapsed 8000 cc3000__event_timeout_ms 8000
0001855904:<ERROR> hci_event_handler (249):Timeout waiting on tSLInformation.usRxEventOpcode 0x1007
0001855913:<DEBUG> int Spark_Connect() (629):connected connect=-1
0001855919:<DEBUG> int Internet_Test() (553):socket
0001855924:<DEBUG> set_socket_active_status (810):Sd=1, Status SOCKET_STATUS_ACTIVE
0001855931:<DEBUG> int Internet_Test() (555):socketed testSocket=1
0001855937:<DEBUG> int Internet_Test() (577):connect
0001863942:<ERROR> hci_event_handler (248):Timeout now 1863942 start 1855942 elapsed 8000 cc3000__event_timeout_ms 8000
0001863946:<ERROR> hci_event_handler (249):Timeout waiting on tSLInformation.usRxEventOpcode 0x1007
0001863955:<DEBUG> int Internet_Test() (579):connected testResult=-1
0001863961:<DEBUG> int Internet_Test() (589):Close
0001863966:<DEBUG> set_socket_active_status (810):Sd=1, Status SOCKET_STATUS_IACTIVE
0001863973:<DEBUG> int Internet_Test() (591):Closed rv=0
0001863978:<ERROR> void SPARK_WLAN_Loop() (627):Resetting CC3000 due to 2 failed connect attempts
0001864498:<DEBUG> void SPARK_WLAN_Loop() (480):Resetting CC3000!
0001866187:<DEBUG> void SPARK_WLAN_Loop() (480):Resetting CC3000!

At this point, the core is flashing blue and no more output is produced until I reset it.

@david_s5 I have 2 cores running the latest firmware and both cores will start Blue Flashing (Frozen) at the same time when my phones data connection is lost sometimes. Whatever happens it causes both cores to react the exact same way even though they are running 2 different sketches. One sketch does no web based stuff the other does.

This has only happend 2 times now so not often but they have to be manually rest to get them working again.

I never really saw the flashing blue led until the most recent update so something changed that made it more prone to flashing blue and locking up.

Other than that the new code stays connected to the net just fine. I’m assuming that if I was not using my cellphone as my hot spot then I would probalby never see the flashing BLUE freeze up.

@Hypnopompia, @RWB

What the log is saying is connect failed to the Cloud (Spark_Connect) then the internet test (Internet_Test)
So the code the Resetting CC3000 due to 2 failed connect attempts but never got connected again. Either at the wifi level or IP.

Which blue are we talking about? Color and Frequency?

hmm have not pulled the latest code to test with. Are you are on master? There has been some changes to thing that could cause a hangs.

@david_s5 When my phones hotspot connection drops out but WiFi stays on the Spark Core will flash Dark Blue and the cores will remain that way until manually reset.

I have 2 Spark Cores and they both do this at the same time. One Spark is sending data to the cloud every min, and the other Spark Core code is doing nothing with the internet.

When I manually reset the Spark Cores there is no RED led flash as an indication of which error caused the problem. This has happened to me about 4 times now which is a lot more often than before, and it has happened with the last 2 online IDE updates.

The Spark Core stays connected most of the time when my WiFi connection goes in and out but if my WiFi stays on and my Data connection is lost it triggers the Blue LED flash and freezes up the core on both Spark Cores.

@RWB Sound like something got broke. Have look at the history and what went in it the last couple commits.

It sound like it is time to learn how to check out a commit from git hub :smile:

1 Like

@david_s5 I built this binary on Saturday using the spark master branch. Last commit was https://github.com/spark/core-firmware/commit/dc4734e216c01d1d85e96372d7ccda51048ce99c

The flashing blue is identical to what the smart config “listening” mode flashing blue looks like.

@david_s5 @Hypnopompia Same here.

And just to be clear. It only flashes blue if the data connection drops out while the core stays connected to my WiFi network. If the WiFi completely drops out the Sparks will flash Green until the WiFi is available again and then it will successfully reconnect to the network, even if it has been disconnected for hours.

There is something that triggers the Blue Flashing once there is no data feed but the WiFi is available.On my smart phone Android I can go into Mobile Networks settings and turn off Data Enabled which will block the phone from allowing data to transfer to the internet. Turning off the Data Enabled will trigger the Blue LED Flash and that will require a manual reset to get it going again. If I manually reset and the data is still not available then it will go right back to the Blue Flashing LED instantly.

I have a set up that duplicates maintaining wifi while breaking IP

good [WIFI]------[hub]----------[router]-------[Cable model]
bad [WIFI]------[hub]---- ------[router]-------[Cable model]

The bins from the:
core-firmware/master of a5ae9f420b4eeddf14e4aec09d004752bf4e4973
core-common-lib/master of 66b36736b23167842ec558e764ca25e1a6175c48

work.

What are you saying here?

Is this the same firmware that is being used currently on the web IDE?

@RWB no. Follow me here https://github.com/spark/core-firmware/network

use you mouse to slide the graph around and hover over dots. See the tag

Click on it 1077009c04682b2a9a7d7535da2a2f5980dd75ec is what is on the WebIde for core-firmware

Then you can do the same for

and

You should be able to locate my 2 SHAL1 that I said I know work still.

Ok I was able to find out that you can click the dots to find the code that was used for each update.

I don't see or understand what you mean when you say "2 SHAL1" I'm assuming its your code but where do I find that.

Sorry I try to follow along but I can't without asking all these questions.

@RWB I just pulled and built master on all the repos. Found and fixed 2 bugs. One in the bootloader and and a second in core-firmware that kept DEBUG_BUILD=y builds from running.

Tested the IP disconnect and no BLUE light. In the AM I will build and deploy a windows release build and see if I can replicate the Blue Light issue.

1 Like

I just tried to shoot a video showing the Blue light flashing after I cutoff the data connection to my phone but it did not happen. It reset successfully both times.

I then started downloading a movie from the torrent and it started pulling about 1.1 MB per second through my wifi connection. Then Verizon cut my data or something caused it to stop transmitting data and this caused the 2 spark's to show the Blue Flashing LED. This happened about 3 times earlier today while I was pulling large amounts of data through my WiFi connection. Verizon cut my data and I had to restart my WiFi Hotspot app to get it going again. So what ever is triggering the Blue Flashing LED and the core to lock up happens when Verizon or my data connection locks up and stops. I'm using a app on my Android phone called FoxFi to turn my phones data into free HotSpot WiFi data. Maybe FoxFi is causing this issue with the Spark's because to get the data working again all I usually have to do is turn FoxFi Off and then back ON, I don't have to turn my phones data OFF and then On again?

I guess either way we want the Spark to automatically recover from these events what ever they are.