I suddenly have an odd behavior on a device that have been working well for a long time. Here’'s how it looks: https://youtu.be/GyLscO02UPE
It starts, blinks briefly white, then a solid red blink and then it restarts.
I can place it in setup mode and in DFU mode, so the chip is sort of working, but it’ll never run a program (not even Tinker). I had a similar issue before and then it was a matter of replacing the device keys. “particle keys server” executes successfully, but does not restart the device as it should. When I try to set the device keys with "particle keys doctor " it ends with the message “permission denied”.
I’ve replaced the firmware with Tinker and done a “particle update”, so it’s not a software issue. Any pointers as to what to try next?
Nobody has seen this before? How about regulars like @bko, @peekay123, @Moors7 or @ScruffR? I’m stumped with 3 dead PCB’s now and really need some help since the device does not send out any SOS pattern or anything. It just reboots instantly.
@jenschr, make sure you have the latest Particle CLI and run Particle device doctor. Go through all the steps to reset your Photon. The blink suggest a keys problems so doctor should help resolve that.
As a side note, low power conditions on a Photon can cause it to wipe its keys. System firmware version 0.7.0 has a self healer for a condition like that.
Thanks for suggestions! I’ll test the boards tomorrow morning when I get to the office.
I had CLI 1.21 installed, so I’ve now upgraded to 1.25. The device was updated using “particle update” from CLI. Won’t that update the bootloader as well?
Using "particle device doctor" did the trick! Thanks a bunch! The device still outputs an error upon updating the Server keys. This is the output:
Opening DFU capable USB device...
ID 2b04:d008
Run-time device DFU version 011a
Claiming USB DFU Interface...
Setting Alternate Setting #1 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 011a
Device returned transfer size 4096
DfuSe interface name: "DCT Flash "
Downloading to address = 0x00000022, size = 608
Download [=========================] 100% 608 bytes
Download done.
File downloaded successfully
Saved!
attempting to add a new public key for device 330031000351353432383931
Error sending public key to server: Permission Denied
Make sure your device is in DFU mode (blinking yellow), and that your computer is online.
Error - undefined
I had done all the steps in the device doctor manually with CLI 1.21, but I presume 1.25 improved something?
I also learned from the client that they had problems with static electricity. Is that a known problem?
Static is always a problem. The STM32/Photon are designed for certain ESD (Electrostatic Discharge) protection but your product needs to be designed for its operating environment's worst case. You can find lots of ESD resources on the web to guide your hardware design.
I am seeing exactly this behavior on more and more devices now. I just got it with a device that I know for certain have not been exposed to any ESD. The entire case is grounded and the floor in here does not produce static. The device worked well 15 minutes ago. After restarting it (by cutting power) it now does the white blinkey and then red single blink before restarting.
Right now this is not a problem 8since most devices are within a few hours drive from my office) but we cannot have this happening to production models. Is there such a thing as an Applications Engineer in the Particle system that could explain why this happens and how I can prevent it from ever happening at a client site? I have two products almost ready for worldwide distribution now…
@jenschr, was power cut instantly or did supply voltage slowly decay to zero? It may now be worth discussing the hardware design of your device. Can you share the schematics?
I can certainly share the MCU and power parts of it without the customer becoming paranoid:
“CONN_01X02” at the bottom is the input from the Power supply. It is protected by a blade fuse (for current) and a MBR745 power diode (for polarity). After that there’s two 100uF capacitors. One is located next to the input and the other near some fans where I expect an uneven current draw.
The 24V the goes into a small switching supply (OKI-78SR-3.3 from Murata) that creates the 3.3V domain on the board. It is filtered with dual 47uf and a single 0.1uf cap before feeding the P1 and other parts. The 5V regulator is completely isolated and only powers an input on the Stepper drivers (not included in the above).
The P1 module controls fans, brushless motors, steppers, a solenoid and temperature sensors. All motors are 24V and driven straight from that, but controlled via different IC’s and sub-circuits (i.e. a FET for the solenoid). The design has been rock solid for close to a year, but this last month I’ve gotten this problem. Restoring with “particle device doctor” makes the device behave well so it seems like something being triggered?
Well we think that most of the time random key problems are due to power issues, so I spent some time looking at the power portion of your schematic and I have some observations and questions:
How is your 24V DC created? A commercial power supply or something you built? Can you scope the 24V supply and look at the ripple? Are there large value electrolytic caps that would hold the 24V supply up when you turn off power?
The OKI switching 3.3V regulator datasheet calls for low ESR caps at input and output. Are the caps you are using quality low ESR types?
The 5V supply is not really “isolated” in the way we usually mean–it is derived from the same 24V supply as the 3.3V which is OK but noise on the 24V can appear on the 5V or 3.3V lines. Do you have bypass caps on the 5V regulator? It probably needs some too at the input and output.
It wasn’t clear in your schematic how the USB 5V was hooked up (if at all).
The power supply is a “decent” quality commercial chinese PSU that is certified and we can get for a decent price at low volumes. I’ve scoped it and there certainly is ripple on the 24V, but not very bad. When scoping the output of the OKI 3.3V, the signal looks quite smooth and after that there is the recommended 0.1uF + 10uF caps on all inputs to the P1 module itself, so I’m pretty sure it should be fine. Then again - I’ll measure it again tomorrow morning to see what noise gets past the filtering.
It’s quite some time since I read the datasheet for the OKI and low ESR caps is something I completely missed. I’ve learned a bit more about capacitors since then also, so that’s something I’ll research. I’ll also add filter caps on the 5V and resistors to the USB data lines on the next revision.
The new revision will also incorporate TVS diodes for any signal reaching off the card via a connector, so that should solve the ESD problem mentioned. I’ll also have an external expert look it over with regards to analog noise and voltage spikes. PWR_FLAG is what you use to indicate your power input pins in Kicad, so that it can alert you to errors in current flow (inputs connected to outputs and such).
