Spark Cloud Outtage

1

Is anyone experiencing spark cloud outrage tonight? (CST)

I can’t connect. And I can’t monitor the status. The site appears to be down too!

Maybe it’s on my end?

2

I have the same problem, I cant connect to my spark cores either by web app, or using android app.

3

same here both the build and spark.io are down with app error :frowning:

4

Hope it get’s fixed soon…

A good indication that the open source cloud app is needed :smile:

5

Looks like it’s coming back… At least the IDE.

6

@dave and @jgoggins,

something you guys need to check the logs for. :smile:

Thanks for everyone who posted!

7

Thanks for bringing it back so quick!

Any progress report on the open source cloud code?

BY THE WAY: I am truly impressed with the Spark Core… Wish you the best!

8

@frlobo,

As of now, the open source cloud is on the list but just on the initial phase. But i believe it will be worked on in the next few Sprints ahead!

We hear you guys loud and clear and it’s always nice to know you guys are anxious about it :smiley:

9

Just for anyone else having trouble:

I had to clear the cache on my browser to get the build button to come back.

After that, don’t forget that you want Log In, not create an account.

10

Hey Guys!

Thanks for posting! We’re still reviewing the logs and collecting all the details, but it looks like like our website failed to recover after our database provider had a brief outage last night. PagerDuty woke up Joe, and he quickly brought the website back online. We’ll post more details on the status page when we know more!

We’re continuing to add redundancies, but the local cloud will also help to spread out the risk for stuff like that. :slight_smile:

Thanks,
David

1 Like
11

Outstanding response guys! @jgoggins @Dave You rock! :blue_heart: :spark:

3 Likes
12

I was wondering what kind of alarm did he set or maybe a Spark Core was flashing his house lights on and off for emergency alerts!

X D

3 Likes
13

Dang, this community is incredible. The Pager Duty alert came in at 1:37am CST and @frlobo posted at 1:41am–with folks as vocal and helpful as y’all, I think we’ll just drop PagerDuty and use the community site for alerts ( :smile: just kidding), but really–thanks for creating this thread and helping us and others understand the scope + impact.

@Dave 's summary is accurate, I’ve reached out to our database provider as well to see if they could provide more details I could share with you guys on the cause. Will post here when I here back.

Thanks!

14

The alert came via a boring text message followed by an automated phone call...I need a ridiculous Spark Core PagerDuty integration that lights my whole house up and wakes up my whole family + neighborhood. Anybody have a design + firmware I can use for this? :smiley:

1 Like
15

The reply:

"random reboot - the worst type of thing to track down and ever so frustrating...we recommend bla bla"

where "bla bla" is "you should upgrade to a bigger better plan with more fault tolerance built in".

Indeed! We'll be investigating a migration to bigger, better db infrastructure over the next couple of weeks.

Thanks to @Gugunner , @frlobo , @tidwelltimj, and @kennethlimcp for surfacing the impact of the issue so quickly and sorry for the inconvenience!

16

Hey guys, we’ve noticed the build IDE isn’t working right at the moment, looking into it.

17

Fixed it! Database was cranky again. :frowning: