Latest firmware released to Spark Build

This topic is now pinned. It will appear at the top of its category until it is either unpinned by a moderator, or the Clear Pin button is pressed.

So I just got tripped up by something I though I would share…

Executive Summary: *** Change one line in your code then verify and flash ***

Long story:

Heard the Great news! (Yeah go team!)
Jumped on https://www.spark.io/build/

Verified and then Flashed my code (added in the webIDE last week) that tests if the build has the new feature PANIC added.

I call it aptly die.cpp

void setup() {

}

typedef void(*pv)(void);
void loop() {
  pv p = 0; // pointer to bad location 
  p();        // Should cause a PANIC and send red LED SOS message (but in current release the watchdog will cut it short)

}

Core took the update rebooted and NO PANIC…

hmmm made a change to code…

in foo = 0; // Note missing t 

Verify - got waning…

In file included from ../inc/spark_wiring.h:30:0,
from ../inc/application.h:31,
from die.cpp:2:
../../core-common-lib/SPARK_Firmware_Driver/inc/config.h:12:2: warning: #warning "Defaulting to Release Build" [-Wcpp]
die.cpp: In function 'void loop()':
die.cpp:8:3: error: 'in' was not declared in this scope
die.cpp:8:6: error: expected ';' before 'k'
make: *** [die.o] Error 1

with new ``` warning: warning Defaulting to Release Build `` that is good…

Fixed the int foo = 0; // Not missing t

Verify and Flashed.

Boom! PANIC (albeit cut short)

Moral of the story:

Not sure if the first re-flash failed with or if the build system did not re-make my app.

@dave Does the webIde force a clean build on repo changes?

1 Like

@david_s5 Hey Dave are all your important upgrades included with this release? I’m running it now so I will be able to report back on wifi performance now.

If you don’t change anything, the previously existing binary (without the new debug features) is used. As soon as you change something, then the code has to be recompiled with all the new goodies.

2 Likes

Hey @RWB—yup, David submitted lots of pull requests all of which are included in the web IDE builds now.

1 Like

@RWB

@RWB

Yes! This adds

  1. Reliability, Reliability, Reliability: The core will re-connection in ~20 seconds or less, on CFOD, wifi resets with no address within 30 seconds or no wifi reconnect after being connected. (try it pull the WAN plug on your AP or router, or if you have a wifi switch (or the ability (openwrt) to do a “wifi down”, “wifi” commands on your router)

  2. A complete re-write of TI/Sparks CC3000 driver

  3. A refactoring of TI event handle to add reliability timeouts on network actions like connect etc.

  4. Numerous fixes for memory over-wites in TI driver and Spark Protocol

  5. Moved time base to the core-common-lib

  6. PANIC - any Faults will blink the RGB led in a red blink pattern of …—… (SOS) N Blinks …—… repeat.
    (for release build this is cut short but the WDT @zachary - we need a ticket for this)

Where N is one of the panic codes below:

1	(Faults,RGB_COLOR_RED,HardFault) 
2	(Faults,RGB_COLOR_RED,NMIFault)
3	(Faults,RGB_COLOR_RED,MemManage)
4	(Faults,RGB_COLOR_RED,BusFault)
5	(Faults,RGB_COLOR_RED,UsageFault)
6	(Cloud,RGB_COLOR_RED,InvalidLenth)
7	(System,RGB_COLOR_RED,Exit)
8	(System,RGB_COLOR_RED,OutOfHeap)
9	(System,RGB_COLOR_RED,SPIOverRun)
10	(Softare,RGB_COLOR_RED,AssertionFailure)
11	(Softare,RGB_COLOR_RED,InvalidCase)
  1. Added SPARK_ASSERT a macro to PANIC on assertion failure

  2. Full runtime and compile time controlled logging (like log4j) For now only avaiable under a local build (There is a ticket in to get a DEBUG setting in the WebIDE)

To use it

define DEBUG_BUILD=y on make command line

Add the following to your Application.cpp

    void debug_output_(const char *p)
    {
      static boolean once = false;
     if (!once)
       {
         once = true;
         Serial1.begin(115200); // You can choose where the debug goes 
       }
    
     Serial1.print(p);
    }

The API looks like and supports printf format specifies.


LOG("Want %d more cores",count);
WARN("Running %s on cores only %d more left",,"Low",count);
DEBUG("connection closed %d",retValue);
ERROR("Flash write Failed @0x%0x",badd_address);
PANIC(HardFault,"Hit Hardfault");

Output looks like:

0003187040:<DEBUG> int Spark_Connect() (639):connet
0003187179:<DEBUG> int Spark_Connect() (641):connected connect=0
0003233369:<DEBUG> void loop1() (164):0 total Bytes Read
0003233374:<DEBUG> virtual void TCPClient::stop() (175):closesocket=134277261
0003233381:<DEBUG> void loop1() (166):connection closed state 4
0003235908:<DEBUG> void loop1() (84):connecting
0003235916:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (61):socket=1
0003236023:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (78):connected=1
0003236037:<DEBUG> void loop1() (97): Send
0003236042:<DEBUG> void loop1() (99): Sent
0003260515:<DEBUG> int Spark_Disconnect() (648):
0003260519:<DEBUG> int Spark_Disconnect() (653):send
0003260524:<DEBUG> int Spark_Disconnect() (656):Close
0003260529:<DEBUG> int Spark_Disconnect() (659):Closed retVal=-57
0003260535:<DEBUG> int Spark_Connect() (617):socket
0003260540:<DEBUG> int Spark_Connect() (619):socketed sparkSocket=0
0003260546:<DEBUG> int Spark_Connect() (639):connet
0003261230:<DEBUG> int Spark_Connect() (641):connected connect=0
0003311025:<DEBUG> void loop1() (164):0 total Bytes Read
0003311030:<DEBUG> virtual void TCPClient::stop() (175):closesocket=134277261
0003311037:<DEBUG> void loop1() (166):connection closed state 4
0003313554:<DEBUG> void loop1() (84):connecting
0003313561:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (61):socket=1
0003313668:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (78):connected=1
0003313682:<DEBUG> void loop1() (97): Send
0003313687:<DEBUG> void loop1() (99): Sent
0003334431:<DEBUG> int Spark_Disconnect() (648):
0003334435:<DEBUG> int Spark_Disconnect() (653):send
0003334440:<DEBUG> int Spark_Disconnect() (656):Close
0003334445:<DEBUG> int Spark_Disconnect() (659):Closed retVal=-57
0003334451:<DEBUG> int Spark_Connect() (617):socket
0003334456:<DEBUG> int Spark_Connect() (619):socketed sparkSocket=0

My test app, does http get, 1 UDP receive , and 1 UDP sendto but only on a CodeSoucery tool stream build. This is because CS builds the app using less memory.

But now if you app uses too much memory not you will get an OutOfHeap PANIC (SOS 8 blinks SOS)


/* Includes ------------------------------------------------------------------*/  
#include "application.h"


#define RENEW_INTERVAL      5*1000      // 30 secs
#define RETRY_INTERVAL      5*1000      // 10 secs
#define RESPONSE_INTERVAL   1*1000       // 1 sec
#define LET_IT_FILL_INTERVAL   3*1000       // 1 sec

TCPClient client;
char server[] = "nscdg.com";

// IO
int led = D2;

// Globals
volatile int state = 0;
volatile int wait = 0;
volatile int loopwait = 10;

volatile int tries = 0;
volatile int store = 0;
volatile int hash = 0;

volatile char command[32];
volatile int command_i=0;


#include "application.h"

uint8_t buffer[TCPCLIENT_BUF_MAX_SIZE+1]; // for EOT
int loops = 0;
int total = 0;
int wcl=2;


#define SERVER_IP 10,10,0,1
//------------------------------
#define SERVER_PORT 9999
#define WAIT_TIME 3000

UDP udpClient;
IPAddress serverAddress(SERVER_IP);

void setup2() {
  pinMode(D2,OUTPUT);
}

system_tick_t lastMillis = (system_tick_t) -WAIT_TIME;

char buf[50];


void loop2() {
    if(millis() > lastMillis + WAIT_TIME){
        udpClient.begin(SERVER_PORT);
        udpClient.beginPacket(serverAddress,SERVER_PORT);
        snprintf(buf, sizeof(buf),"tick %ld", lastMillis);
        udpClient.write(buf);
        udpClient.endPacket();
        lastMillis = millis();
        DEBUG("tick ************ %ld",lastMillis);
        digitalWrite(D2,HIGH);
        delay(20);
        digitalWrite(D2,LOW);
        udpClient.stop();

  }
}


UDP Udp2;
void setup3()
{

    DEBUG("UDB RX");

    pinMode(led, OUTPUT);
    // Button resistorless
    state = 0;
    wait = RETRY_INTERVAL;
    Udp2.begin(8738);
    // Connecting
}

void loop3()
{
      delay(1);
      if (++loops == 100) {
          loops = 0;
        int packetSize = Udp2.parsePacket();
        DEBUG("parsePacket %d", packetSize);
        if(packetSize > 0)
        {
          int count = Udp2.read(buffer,sizeof(buffer));
          buffer[count] = '\0';
          DEBUG("read %d %s", count, buffer);
          total += count;
        }
      }
}




int bad_mod = 0;
int bad_every = 0;
void setup1()
{

    LOG("Test TCP BAD Every %d Usage!",bad_every);

    pinMode(led, OUTPUT);
    // Button resistorless
    state = 0;
    wait = RETRY_INTERVAL;

    // Connecting
}

void loop1()
{
    delay(1);
    switch(state){
        case 0:
            // Waiting for next time
            wait-=10;
            if(wait<0){
                wait = 0;
                state = 1;
            }
            break;
        case 1:
            // Connecting
            bad_mod++;
            DEBUG("connecting");
            total = 0;
            if (client.connect(server, 80)){
                state = 2;
            }else{
                DEBUG("connection failed state 1");
                wait = RETRY_INTERVAL;
                state = 0;
            }
            break;
        case 2:
            // Requesting
            if(client.connected()){
                DEBUG (" Send");
                client.println("GET /t.php HTTP/1.0\r\n\r\n");
                if (bad_every  && ((bad_mod % bad_every) == 0))
                  {
                    DEBUG (" Sent but not Reading it!");
                    wait = 1000 * 18; // longer then spark com time
                    state = -2;
                  } else {
                      DEBUG (" Sent Doing Read");
                      wait = RETRY_INTERVAL;
                      state = -3;

                  }
            }else{
                DEBUG("connection lost state 2");
                wait = RETRY_INTERVAL;
                state = 0;
            }
            break;

        case -2:
          if ((wait % 500) ==0) {
              DEBUG("Waiting client.status()=%d",client.status()) ;
          }
#if defined(GOOD)
          if (!client.status()) {
              state = 4;
              DEBUG("Waiting Aborted - Closing") ;
          }
#endif
          wait--;

          if(wait<0){
              wait = 0;
              state = 4;
          }
          break;

        case -3:
          if (client.available()) {
              DEBUG ("Ready");
              state = 3;
              loops = 0;
          }
          wait--;
          if(wait<0){
              wait = 0;
              state = 4;
          }
          break;

        case 3: {
          if(!client.connected()){
              wait = 1;
              state = 4;
          } else {
              // Receiving
            int count = client.available();
            if (count) {
                  DEBUG("client.available() %d", count);
                  // Print response to serial
                  DEBUG("client.peek %d", client.peek());

                  count = client.read(buffer, arraySize(buffer));
                  buffer[count] ='\0';
                  char *p = strstr((const char *)buffer,"0.1.2.3");
                  if (p)
                    {
                     total = -(p-(char*)buffer);
                    }
                  total += count;
                  DEBUG("client.read() %d", count);
                  debug_output_((const char*)buffer);
                  debug_output_("\r\n");
             }
          }
        }
            break;
        case 4:
          DEBUG("\r\n\r\n");
          DEBUG("%d total Bytes Read\r\n\r\n",total);
            // Disconnecting
            if(client.connected()){
                client.stop();
                DEBUG("connection closed state 4");
                wait = RENEW_INTERVAL;
                state = 0;
            }else{
                DEBUG("connection closed by server state 4");
                wait = RENEW_INTERVAL;
                state = 0;
            }
            break;
      }
}



/* Function prototypes -------------------------------------------------------*/
int tinkerDigitalRead(String pin);
int tinkerDigitalWrite(String command);
int tinkerAnalogRead(String pin);
int tinkerAnalogWrite(String command);

/* This function is called once at start up ----------------------------------*/
void setup()
{
	//Setup the Tinker application here

	//Register all the Tinker functions

        setup1();
        setup2();
        setup3();
	Spark.function("digitalread", tinkerDigitalRead);
	Spark.function("digitalwrite", tinkerDigitalWrite);

	Spark.function("analogread", tinkerAnalogRead);
	Spark.function("analogwrite", tinkerAnalogWrite);

}

/* This function loops forever --------------------------------------------*/
void loop()
{
	loop1();
	loop2();
	loop3();
	//This will run in a loop
}

void debug_output_(const char *p)
{
  static boolean once = false;
 if (!once)
   {
     once = true;
     Serial1.begin(115200);
   }

 Serial1.print(p);
}


/*********************************************************************
 * Function Name  : tinkerDigitalRead
 * Description    : Reads the digital value of a given pin
 * Input          : Pin 
 * Output         : None.
 * Return         : Value of the pin (0 or 1) in INT type
                    Returns a negative number on failure
 *******************************************************************************/
int tinkerDigitalRead(String pin)
{
	//convert ascii to integer
	int pinNumber = pin.charAt(1) - '0';
	//Sanity check to see if the pin numbers are within limits
	if (pinNumber< 0 || pinNumber >7) return -1;

	if(pin.startsWith("D"))
	{
		pinMode(pinNumber, INPUT_PULLDOWN);
		return digitalRead(pinNumber);
	}
	else if (pin.startsWith("A"))
	{
		pinMode(pinNumber+10, INPUT_PULLDOWN);
		return digitalRead(pinNumber+10);
	}
	return -2;
}

/*******************************************************************************
 * Function Name  : tinkerDigitalWrite
 * Description    : Sets the specified pin HIGH or LOW
 * Input          : Pin and value
 * Output         : None.
 * Return         : 1 on success and a negative number on failure
 *******************************************************************************/
int tinkerDigitalWrite(String command)
{
	bool value = 0;
	//convert ascii to integer
	int pinNumber = command.charAt(1) - '0';
	//Sanity check to see if the pin numbers are within limits
	if (pinNumber< 0 || pinNumber >7) return -1;

	if(command.substring(3,7) == "HIGH") value = 1;
	else if(command.substring(3,6) == "LOW") value = 0;
	else return -2;

	if(command.startsWith("D"))
	{
		pinMode(pinNumber, OUTPUT);
		digitalWrite(pinNumber, value);
		return 1;
	}
	else if(command.startsWith("A"))
	{
		pinMode(pinNumber+10, OUTPUT);
		digitalWrite(pinNumber+10, value);
		return 1;
	}
	else return -3;
}

/*******************************************************************************
 * Function Name  : tinkerAnalogRead
 * Description    : Reads the analog value of a pin
 * Input          : Pin 
 * Output         : None.
 * Return         : Returns the analog value in INT type (0 to 4095)
                    Returns a negative number on failure
 *******************************************************************************/
int tinkerAnalogRead(String pin)
{
	//convert ascii to integer
	int pinNumber = pin.charAt(1) - '0';
	//Sanity check to see if the pin numbers are within limits
	if (pinNumber< 0 || pinNumber >7) return -1;

	if(pin.startsWith("D"))
	{
		pinMode(pinNumber, INPUT);
		return analogRead(pinNumber);
	}
	else if (pin.startsWith("A"))
	{
		pinMode(pinNumber+10, INPUT);
		return analogRead(pinNumber+10);
	}
	return -2;
}

/*******************************************************************************
 * Function Name  : tinkerAnalogWrite
 * Description    : Writes an analog value (PWM) to the specified pin
 * Input          : Pin and Value (0 to 255)
 * Output         : None.
 * Return         : 1 on success and a negative number on failure
 *******************************************************************************/
int tinkerAnalogWrite(String command)
{
	//convert ascii to integer
	int pinNumber = command.charAt(1) - '0';
	//Sanity check to see if the pin numbers are within limits
	if (pinNumber< 0 || pinNumber >7) return -1;

	String value = command.substring(3);

	if(command.startsWith("D"))
	{
		pinMode(pinNumber, OUTPUT);
		analogWrite(pinNumber, value.toInt());
		return 1;
	}
	else if(command.startsWith("A"))
	{
		pinMode(pinNumber+10, OUTPUT);
		analogWrite(pinNumber+10, value.toInt());
		return 1;
	}
	else return -2;
}

4 Likes

Awesome, ill put it through the most common things that caused a manual reset before and see if the new code fixes it and report back. Thanks so much for your work on this!

@zachary What are analogRead changes in this batch of updates? I’ve got some problems correctly reading my analog inputs now. Seems second reading is affected by first one. What is minimum delay between readings required now?

1 Like

@zachary

Does this mean that if we have sketches we have already uploaded to the :spark: core and those skectches are saved in the Web IDE that we have to make a change to the sketch and then recompile before the new improved firmware will be added in and flashed to the :spark: core?

If I try to just flash an old saved sketch it will not load the new firmware?

@david_s5 I got the new firmware loaded and did a quick test.

While I had the old firmware loaded I could force the Core to freeze up if I cut the internet feed to my router, and then cut off the actual Wifi. Then when I turned wifi and internet access back on the Spark core would stay flashing green but would never leave that state and the main loop would not run, it would remain frozen in that state. My Wifi would show it as reconnected to the network though.

I then ran the same test with the new firmware and when the wifi turned back on the Spark core reset, flashed the Red LED 2 times to indicate the error code and then it reconnected to the network started running successfully again with no intervention from me. So its working for me good.

I’ll keep running test under different conditions and report back to you.

1 Like

@RWB Great! @zach Can I get the bounty now? :smile:

2 Likes

@zachary hmmm… given the delta in reliability, would it be prudent to do a “clean” on all the bins on the webBuild server so a request to flash builds the new code and fixes things.

Then we need to resolve the replacement of the factory image in serial Flash.

@david_s5, great I was just looking for such a debug feature.
I have added it immediately in my code and I saw the debug print I expected:
0000148169:<DEBUG> void loop() (126):SDS interrupt

But it was surrounded by a lot of other debug printouts, I wonder what the reason is?
Is it possible to filter those other debug printouts?

0000122720:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0000122730:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16
0000137860:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0000137869:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16

0000148169:<DEBUG> void loop() (126):SDS interrupt

0000148211:<DEBUG> set_socket_active_status (810):Sd=2, Status SOCKET_STATUS_ACTIVE
0000148220:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (72):socket=2
0000148229:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (90):_sock 2 connect
0000148277:<DEBUG> virtual int TCPClient::connect(IPAddress, uint16_t) (92):_sock 2 connected=1
0000150287:<DEBUG> virtual int TCPClient::available() (153):recv(=83)
0000152717:<DEBUG> virtual int TCPClient::available() (153):recv(=56)
0000155011:<DEBUG> virtual int TCPClient::available() (153):recv(=39)
0000157221:<DEBUG> virtual int TCPClient::available() (153):recv(=36)
0000159416:<DEBUG> virtual int TCPClient::available() (153):recv(=46)
0000163098:<DEBUG> virtual int TCPClient::available() (153):recv(=96)
0000163668:<DEBUG> set_socket_active_status (810):Sd=2, Status SOCKET_STATUS_IACTIVE
0000165592:<DEBUG> virtual int TCPClient::available() (153):recv(=78)
0000166126:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 2
0000166135:<DEBUG> int Spark_Receive(unsigned char*, int) (350):bytes_received 16

EDIT: how can I use the other debug levels such as LOG and WARN?

Thanks,
Henk

I just flashed the latest firmware onto a core and took it to work. This means a chatty environment, the former firmware crashed after a minute or so and then I had to reset the core manually. Now, with the new firmware it already seems to be more stable and the core seems to reset itself automatically - still I get maybe ~3-4 requests to my function through before it will get stuck.

If I’ve read it correctly, there was a new TI firmware for the CC3000 in this release. Can we still expect more improvements for the CC3000 or is that work done for now (well, in this case I need to say that more work needs to be done).

One the plus side - I was running the same code+firmware at home yesterday. It ran several hours without any issues. I was hitting the spark cloud with a function call every second. It still worked when I unplugged the device :slight_smile: If I could have the same here at work/conferences, that would mean a big deal.

1 Like

@nika8991

The short answer is no, not in this build. You can use tail -f | grep loop

The long answer is: I wanted to monitor the network until we got confirmation that the code was an improvement. We will need to add a compile time module level of debugging control .

The API to use LOG and WARN is listed in the above post. have a read on common-lib/debug.h

@hansamann

Let me clarify a few things:

“TI firmware for the CC3000” refers to the code running on the cc3000 module. This is updated using the Patch Programmer.

The TI CC3000/Spark driver refers to the code running on the STM32F103CB of the sparkcore to interface with the cc30000. This is what I re-engineered.

We are still waiting on TI for the firmware for the CC3000 to fix the ARP buffer bug causing the CFOD.

3 Likes

Might be a silly question, but how easy vor hard will it be for non-experts to upgrade the ti cc3000 Firmware. Do we need extra Hardware or tools? If yes, can you post links so we can order the required ahead of time to be ready?

It’ll be as simple as using the WebIDE to flash your Core.

Basically the team will either release a special sketch or have a button somewhere on the WebIDE that will flash the Core with a patch programmer app, which in turn will update the CC3000 over the existing SPI interface!

2 Likes

This new firmware completely kills my spark cores :frowning: Endless cycle of panic, ECOP™ :frowning:

I have an app (MessageTorch, https://community.spark.io/t/messagetorch-torch-fire-animation-with-ws2812-leds-message-display/2551) which was running fine on several spark cores for over a month.

Today, I re-flashed the app to get the new firmware (hoping for less CFODs, or auto recovery thereof), and now both of the two cores I tried it with seemingly start normally including connecting to the WiFi (green flashing), but then immediately show a red-flashing panic code.

I can’t tell which panic code it is, because it does not stay in panic mode long enough to even get the first full SOS, let alone the “N Blinks” that would reveal the type of error. Instead, the watchdog seems to kick in and resets the core.

So what I get is a spark lost in an endless cycle of cyan startup flashing, then green network connection flashing, then showing the beginning of a red SOS Panic, repeating forever.

I tried this with 2 different cores (my kickstarter award, and one of the 5 cores I ordered later), both show identical behaviour.

The cores are not bricked, I can recover them by doing a full reset (holding mode during reset until LED flashes white rapidly). Then I can connect them again to the WiFi, Tinker works ok. But as soon as I re-flash the app, it’s back in endless panic loop.

Any idea what could be wrong?

[Update: I thought it might be that my app blocks IRQs during critical WS2812 timing generation, but that’s not that causing the issue. I removed the __disable_irq() / __enable_irq() statements for test - no difference. Other than that, I have not the slightest idea what my app would be doing out of the ordinary]

I know your app is kind of intense, so perhaps you are running into an issue where it’s running out of memory (Flash or RAM). I’m guessing because you are seeing the SOS that the debugging is turned on by default. This will add about 20KB of Flash memory. In the local build environment you can disable the debugging and see if that helps. If you go this route, grab the compile-server2 branch for a proper comparison.

Maybe @david_s5 can shed some light on how to debug the code via Serial1 or Serial. I think you have to add a debug routine to your app that will enable it… then you should be able to tell what’s happening.

1 Like