Error 500 trying to get device info from Particle Cloud

It has been a few days and now it happens more and more often. We occasionally receive error 500 from the cloud API when trying to get device info:

https://api.particle.io/v1/devices/PARTICLE_ID?access_token=SECRET_STUFF

We use Photons.
To double check that it’s not us, I’ve also pointed the browser to the url to get the device info and set it to refresh every 10 seconds and every now and then I get 500. It is being a major deal and it happens across multiple Photons.

To give an idea, the error appeared hundred of times in the last hour across 20 particles or so.

And I noticed the same problem when reading the variable.
Here the theme previously.

So far there is no confirmation of Particle team.
Probably a problem with Cloud.

I also wrote to support few days ago and nothing. :confused: sigh

1 Like

Hey All,

Thanks for the ping! We’re looking into it.

Thanks,
David

1 Like

If it helps I’m using Electron with firmware 0.5.2.

Hey All!

I think I’ve reproduced the issue you’re seeing, we’re working on a fix.

Thanks!
David

2 Likes

Hey Gang!

It sounds like a recent spike in requests was causing a misconfigured load balancer to occasionally run out of file handles. We estimate that it was impacting about 3% of API request since Monday, and we’ve fixed the issue for now, and added alerts to help us catch this if it crops up again in the future.

Thanks again!
David

3 Likes

It seems it is working with no issue since 1h ago indeed.
Thanks Dave for the prompt reaction.

Quick question, are you guys planning to Open source the Cloud again? It looks like the open repo it hasn’t been updated for a while.

Hi @Alain1405,

Glad it’s working for you again! We try to follow the principle of open-sourcing basic technology, and licensing advanced technology. We’ve invested countless hours and expertise making our main production cloud robust and scalable. We haven’t ruled out releasing more cloud-software as open-source, but I’m not aware of any major releases of cloudy pieces in the coming months.

Thanks!
David

Thanks Dave,
Even though it’s a bad news considering that Photons are not supported in the opensource version.

Hi @Alain1405,

Photons are definitely supported in the open source version!

Thanks,
David

@Alain1405 @developer_bt are you both back up and working?

Hello @KyleG @Dave ,
Now worked great :smile:
Thanks for the quick intervention.

1 Like

@developer_bt we are happy to help! (Even though @Dave did all the heavy lifting here! :wink:)

Haha, I did the ping`ing, our cloud team did the heavy lifting fixing this one :slight_smile:

1 Like