Using Electron / Photon in high volume commercial product

But I hope that this does not apply for particle.publish() functions, right - since they seem to be encrypted (hopefully) - and are based on CoAP !?

And you can do webhooks with TLS, right? And Embed Auth-Info in user-defined headers, right?

But however - this topic is about reliability / long-term / large-scale of especially Electron as a base for industrial controller units.

Hey there!

My name is Will, and I’m the GM of Developer Tools at Particle. Was holding back on replying because you mentioned that you wanted to hear from customers, but am happy to share our perspective as well. Here are some of the most important things I think you should consider, given your comments and concerns in the thread:

1. Embedded devices are complicated, and “poor reliability” can come from lots of places. We have customers reach out to our customer success team (an internal group whose mission is to help our customers solve scaling problems with their products) with new concerns and issues every day. After receiving such reports from more than 50 individual enterprise customers, I can tell you that 9/10 of problems fall into these categories:

  • RF or power issues caused by module integration onto host PCBs
  • Disconnects or firmware reliability issues caused by problematic user application firmware

In other words, 9 out of 10 times, the issue is resolved by reworking problematic firmware or fixing electrical issues with the host PCB. That doesn’t mean that there aren’t areas where we can improve our development tools, our documentation of best practices, our examples, and application notes to help product creators, because there certainly are. All I mean is that far more often than not in enterprise customer applications, the underlying source of instability is not Particle’s core infrastructure.

Remember that we’ve created a highly integrated solution that becomes more difficult to use as you go off the beaten path.

  • Want to use a third party SIM? You’ll have to provide your own networking timeouts, consider and update the keep_alive network cadence for your specific carrier, and manage the APN independently across firmware updates.
  • Want to operate your device in MANUAL mode? No problem! But you’ll have to manage the connection yourself, otherwise reconnections, OTA updates, and device handshakes will be negatively affected.

In my experience, I’ve never seen a Photon or Electron development kit fail to receive an OTA flash 25% of the time when connected to the Internet. If you’re not seeing the reliable development experience extend to your integrated product, I highly recommend you reach our to our customer success team to chat through the sources of the issue. We’ve never found an issue we haven’t been able to resolve for our customers.

2. A bunch of questions

  • Are the devices that are listed as “online” Photons or Electrons? Remember that Photons connect by default over TCP, whereas Electrons leverage UDP, which is a connection-less protocol. When you’ve had issues flashing, are you using the physical device or the API to determine whether or not the devices are online?
  • Shoot us an email and we can let you know if your publishes are getting rate limited, you’ve hit the messages limit on your pricing plan, or messages are falling through the cracks.
  • Because the Electron communicates over UDP, your Telstra dashboard tool won’t likely work for your cellular devices unless you’ve configured them to connect over TCP.

You’re right – I’m a Particle employee, so I have trust in our tools and infrastructure. I’ve also seen enough customer issues, though, to know that building integrated connected products is really hard, even with good tools. We’d be happy to engage more deeply to help you find the source of instability, and am sure we can restore your confidence in the platform.

8 Likes

@Moors7 @simmikolon
Yes I agree with you about webhook. But here becomes another link in the system that makes it theoretically and sometimes practically unstable.
For example if you want to enable direct connection without the mediation of Particle.
@will Do you have a plan to enable SSL for TCP.client ?

Hi Will!

Nice to meet you!

Well yes, it was a bit misleading but I am happy that you share your perspective as well. To be true I wanted to sound a bit provocative just to make sure that we are on the right side. As I said it's about a large number of devices (at least for us :D).

So we really love the Electron and we cannot find any different hardware / platform that offers quite as much power and utilities like fuel gauge, etc. and that is of such high quality. So it's a bit like a dream come true but we wanted to make sure that things fit.

Yes we want to. We are using Soracom.io SIM-Cards as their pricing is lower - especially when using 3k+ SIM-Cards. I like the Particle.io Sim-Cards but we are having quite a lot of data so price per mb counts.

That's good to hear and I experienced myself that:[quote="will, post:22, topic:31602"]
Embedded devices are complicated, and "poor reliability" can come from lots of places.

  • Disconnects or firmware reliability issues caused by problematic user application firmware
    [/quote]

... firmware design was often causing the problems that a device was unable to be flashed OTA since the firmware caused a dead-lock, etc. So that's why you should separate Production from Development ...

But anyway,

I guess you agree that we all would love to hear a Use-Case / Story from one of your "enterprise" customer - We all would really love to see a customer using 10k or even 100k+ Particle.io Devices in a product and live in the field. This would put an even bigger trust in particle!

Thanks,

Simon

Hi!

We are small company, with only around 100 photons deployed (we make digital clocks for consumers). We like the Particle’s platform and would choose it again.

Our experience is practically identical to Garadget:

  • the biggest problems we had so far were with copper pours around the antenna - they somehow interfere with antenna resulting in DHCP failing to start during initial connection process (softap). It works well, since we figured out the PCB layout.

  • OTA updates are reliable, no problems at all (use interrupts/timers instead of delays to keep the Photon in permanent connection with cloud).

We use instance of “spark server” (local cloud) in production, no issues here at all.

3 Likes

I dont have a lot of units deployed yet (50) but I’m using the photon in an offline mode only with softap feature to program my device over a webpage (and firmware upgrade via softap).

I also had the issue of the PCB layout. That is a bit annoying because with a photon with headers, it work well then you switch to the surface mount version of the photon and the issue appear…

Right now my problem is the memory space (128k is too small for me) so I need to work out on something and I dont want to do that.

I had initially designed me product for the P1 and when I wanted to go in production the P1 was out of stock for severals months so I had to redesign and use the photon instead.

@Suprazz How much of your memory is being eaten up by LCD image data or images?

I’m testing 4D Systems LCD displays and it allows you to offload most of the video stuff to the memory on the LCD or built-in SD card.

TLS/SSL is not a simple thing, but this library exists now that you might be able to use:

Last year I was working for a while on my own TLS library that saves memory and code space and allows binary data by working directly with our HAL instead of through TCPClient, but I haven’t had time to work on it for several months. It’s definitely an incomplete work-in-progress right now. That’s the beauty of open source though — PRs welcome!

1 Like

Yes I tried the 4d systems. It work very well but I can’t use it because of the physical dimension of the boards… And the price is high too…

Hey there -- I want to confirm that you got a volume quote for data from our Sales team...our enterprise rates are governed by a different model than our developer rates.

3 Likes

Hey @will - i haven't received any quote or info about a different model from your sales team yet ... just to let you know... or did I got you wrong?

Did you request a quote for volume data with Particle SIM?
If not you should ask for one with some indication what kind of volume you are thinking of.

I can recommend displays that use the RA8875 chip if it’s mostly graphical UI you need. If you check out for instance buydisplay.com they offer a line of displays based on this chip at good prices compared to what 4D systems or Nextion offers.

@dfarny @sparkly Regarding copper pours: I presume this means that you used the P1 with the builtin PCB antenna? Not an External U.fl antenna? Or did you use Photon’s directly?

I know it is not easy when it comes to MCU. But I see that you have great progress in that direction. :slight_smile:
Are you planning an official library for TLS during this year?

1 Like

@dfarny Could you expand on what you mean by copper pours - are you referring to circuit traces or ground plate? I have about 50 photon devices in the field with one product so far and we are adding at the rate of about 5 - 10 /month. I have followed the Particle advice of not have any circuits or ground plate in the area around/under the built in antenna. We have noticed that some devices find it harder to find and keep a cloud connection but have not been able to do a definitive RCA on this and this does not appear to be circuit board related. We are looking to launch a another product in May with shipping volumes nearer to 100/month - if you have any examples of definitely what to avoid that would help everyone.

@armor By copper pours I mean ground planes. We are in same situation, around 100 photons deployed, adding 5-10 every month. We had no issues with Photons connecting to cloud/wifi before, just the DHCP failures.

Our PCB in detail:

@simmikolon,

We started using the Photon / P1 about 7 months ago to determine if it was really worthy for the consumer market.
The short answer is YES!!! I would not even consider any product if i had one instance where it failed or acted hanky to do something. If it failed once i can guarantee you it will do it again even if it does not seem to do it again for 30 days. Problems just don’t go away. With all the months of hard testing we have done with it I have extreme confidence in the Photon / P1 product.

If I could nit pick anything about Particle is that they don’t have a GUI tool to upload the user .bin file via USB. For us this is the biggest problem, as some of our customers may need to update the firmware without using Wi-Fi, and it would be sure nice to simply send them the .bin file in an e-mail, and they just plug it into the USB port and click a few GUI buttons and flash it. Not sure why Particle is so slow to develop such a thing.

The longer version is:

There was a little learning curve in the beginning as I also had some of the same issues you were talking about in you OP. However, all of the issues with connectivity and updating were because of the way I wrote my code, and not because of the device itself. Many people just jump right in and start coding without talking the time to read about how things work and the correct way to do things. People do far to much copy and pasting code examples just to get it to work, rather than to take the time to understand what its really doing and how it may impact the system.

From what I have seen when some people have an issue with updating is that it’s their code is what is causing the problem. One needs to keep in mind that not only is your code running here but there is allot of other code that needs to run as well, like all the networking code. So if the user code (loop function) does not release CPU time to allow the networking code to run because your stuck in some while loop, its not the devices fault it does not work properly.

One of the things that helped us was:

//https://docs.particle.io/reference/firmware/photon/#system-thread
SYSTEM_THREAD(ENABLED);

Another was to avoid things like:

while(digitalRead(A0) == TRUE);

One needs to keep in mind this is not an Arduino, It much more than that. the above code is fine for Arduino, because it does not need to do anything else but sit there and wait. However, for us we need to periodically release CPU time to run the network code to check for updates and other things. Being stuck in some loop like the above we cannot release CPU time to do that. Once people understand this, and code with this in mind things run smoothly.

To bore you even more, but really bring this concept home. Say you added a 4.3" LCD like we did using SPI to update the screen. You can imagine just how much data is needed to send to the LCD to set every pixel color.
480 x 272 = 130,560 pixels. Each pixel has an RGB value, so now your talking 130,560 * 3 = 391,680.
At this point you need to think at a bare minimum without commands codes I need to shoot out 391,680 SPI writes to the LCD. I’ll just stick that in a for loop and do that. Ok, no problem. But while your doing that time intensive process are you releasing time to the CPU to run the network stack?
Moreover, how often do you want to update the screen? If you do it to often all you will do is tie up the CPU sending data to the LCD and never have enough CPU time to handle all the background processes it needs to run.

Sorry all if I got long in the tooth here. These are really great devices, and I just wanted to hopefully make the point of that clear.

6 Likes

There was some discussion on getting the Electron as Industrial Grade. Any one aware on this development? What is the status on this development?
We are looking for industrial grade Electron with Industrial Grade SIM Card that can be packaged in IP66 Enclosure.

Anyone has used Electron in IP66 requirement?

Thanks
Mahesh

My WIP lib would essentially be the official one. I started working on it when we were building the “verified libraries” feature, thus the decision to put it in my own account rather than the :particle: account. I am the CTO though, so ¯_(ツ)_/¯ :wink:

3 Likes