@mohit I’ve got a CC3000 sample module from TI, do you want me to sacrifice it to find out?
@timb, thats very nice of you. I have some dead cores here, I’ll try to do it tonight. no worries
Good news, guys! We have a (partial) fix.
We’re still working on figuring out why CFOD happens, which will likely require feedback from Texas Instruments, as presented in my post yesterday. However, we have a firmware update that will ensure that the Core recovers from CFOD relatively pretty much instantly by causing the Wi-Fi module to reset whenever it happens.
If you would like to test this fix right away, please pull the feature/debug-cfod branch of the core-firmware repository (the other two respositories should be at the latest master).
cd core-firmware
git checkout feature/debug-cfod
cd ../core-communication-lib
git checkout master
cd ../core-common-lib
git checkout master
cd ../core-firmware/build
make
dfu-util -d 1d50:607f -a 0 -s 0x08005000:leave -D core-firmware.bin
Note that there is still some period of time where your Core will be out of touch, although it is brief; although CFOD no longer occurs, CFOD is and has always been preceded by 5-15 seconds of an uncommunicative Core (that’s the real error). So in other words, watch the status codes of your responses and re-send requests that get a 404 response.
We will release this via the web IDE ASAP (probably tomorrow) once we’ve tested it; please try it and let us know if it works for you!
Thanks @mohit for making this happen.
4 Likes
So I did try opening up the metal shell, but unfortunately pins 2 and 4 are not exposed in any way:
I’ll try modding and hooking up traces on the dev board tommorow.
1 Like
For the next version of the Core, it would be nice to have some tiny exposed pads that break out the CC3000 UART, plus the CC3000 I2C and CC3000 EEPROM I2C lines (which could be combined). This way advanced users could debug and even restore a bricked CC3000 module without desoldering it (which effectively destroys the module if you don’t have the equipment to reball the BGA package).
I’d also really like to see the VBAT trace go out to a 0-Ohm resistor which connects to normal 3V3 rails (so it would be functionally identical to the way it works now). This way you could unsolder the resistor and tap a wire onto VBAT which could hook to a coin cell to keep the RTC running if so desired!
1 Like
Fully agreed on both points. We’re kind of out of space on the board, but we’ll see what we can do…
1 Like
@zach Yeah, it is tight. I don’t think the CC3000 debug access will be critical once all the big bugs get ironed out.
However, I feel like not bringing VBAT out was a big oversight in this (original) version of the Core. (And honestly, that’s really the only mistake I see with the design! Otherwise this a really amazing piece of work, really. You guys crammed a lot into a very small space and the routing is pretty slick!)
Perhaps you guys can fix that and just do a “silent update” on a future production run?
1 Like
I just compiled this version and flashed my core. It flashed fine and rebooted and I get a breathing cyan. I can even use tinker to turn the blue LED on and off. After a while, I stopped being able to send commands to the core (on this network, the core disconnects about every 2 minutes, so it’s easy to replicate the problem here). Shortly after, the core started flashing green, then quickly went back to breathing cyan, and I could send api commands again. Yay!
However, the next time it happened, it went green, but instead of going to breathing cyan, it went to slow flashing blue and won’t reconnect. If I reset it, sometimes it resets to breathing cyan, sometimes to flashing blue.
Update:
I’ve been watching it for awhile now. It will go between the breathing cyan/flashing green/breathing cyan cycle every 2 minutes or so. Sometimes flashing green takes longer than others. After doing that 2-6 times, it goes slow flashing blue. A reset starts the cycle over.
I compiled this version too (some hours ago). And about 3 hours later of running nice (flashing cyan about every 2 minutes seems to be “normal”) it turned to CFoD again and sometimes flash 2 times red.
Did you run git pull to make sure you had the latest on all firmware repositories before compiling?
I did, yes. but I’ll do it again to be extra sure.
I just went through and did a pull on all three repos, and the only commit it looks like I was missing was this one: https://github.com/spark/core-common-lib/commit/f754204a09788ca1e21e4863a00482bb590fb3b4 from today. I compiled again and flashed. It’s reconnecting every 2 minutes, but no slow blue flashing yet. I’ll keep you updated.
Edit: Posted too soon. Slow blue flashing again.
I did it too. I’m sure that I cloned all of it…
If you want to be really really sure… use git reset --hard origin/master especially if it’s complaining about merging crap.
@Dragonsshout, this fix won’t stop the Core from going cfod on you, but should be able to recover after a max time of 40secs. Does it not recover after 2 red flashes?
If you need a CC3000 breakout board to get at the debug pins, I have one in the bag: http://oshpark.com/shared_projects/fV1HOc8j
OSHPark link includes schematic and assembled photo. I only have one assembled (and need it for the time being). Reset/EN works a bit differently (by a reset generator triggered from power applied to the board), but if you use what would have been the EN signal to turn on power, this works the same +/- 100msec reset generator timeout.
Currently I have this connected to the same medium density 128k stm32 (except its the 36pin vqfn). I’m super busy with work right now but if I have time this weekend I can port the spark s/w to my breakout board and download the log with the debug header – if this is still needed when I can get to it let me know.
TI is not going to be much help unless you can definitively prove that this is their problem or you have them on support contract.
No, it doesn’t recover after the 2 red flashes.
On power up or reset I have CFoD every 2 minutes for about 3 seconds (without red flashes) and then about 1-2 hours of working time, it’s turning definitely into CFoD non-stop and don’t recover after the red flashes.
I will try to get all firmwares again.
Thanks for the inputs @Dragonsshout
I’m playing with the aucInactivity timeout (presently set to 20, earlier 60) in the spark_wlan.cpp file. My Core currently goes into non-responsive breathing cyan state after about 60mins but recovers from successive CFOD inbetween.
You welcome @mohit
I saw those changes in the code.
Ok, actually I made those commands:
git clone -b feature/debug-cfod https://github.com/spark/core-firmware.git
git clone https://github.com/spark/core-common-lib.git
git clone https://github.com/spark/core-communication-lib.git
cd core-firmware/build/
make
dfu-util -d 1d50:607f -a 0 -s 0x08005000:leave -D core-firmware.bin
The core is running nice now for about 10 minutes.
Actually It changes a bit. After each 40 seconds it flashes first green for about 8 secondes and then cyan for 2 secondes. Inbetween the core is breathing cyan.
I will wait and see what the core will do in the next few hours.
1 Like
After 9 hours the core is still working.