Hi all,
I’ve researched the current situation and while Webhooks & the GlowFish library seems to work for many cases it won’t let you connect to Amazon’s IoT services. I have a client that has existing infrastructure based on AWS and they have no intention of moving anywhere else. Given the capabilities of the chip used in the Photon, I mistakenly thought this would be supported right out of the box, but I guess I was wrong.
I could of course flash my Particle with Adafruit’s firmware that supports AWS but then I’d loose what I think are Killer features such as OTA updates, simple wifi config and the Particle Cloud backend for management. What I really want is a Photon with full TLS support so that I can connect to basically any web service. I would love to hear from the Particle Team if/when this is planned.
This is something I’m also really interested in.
I’ve been developing product on the Particle system for over a year and I’ve been hoping for an ‘out of the box’ HTTPS solution for a while, as I have a customer who requires secure websockets.
I have been looking at https://realtimelogic.com/products/sharkssl/
However I can say that the license is not super cheap and I’m hesitant to pay up front without knowing exactly if I’ll run into problems or not.
Any info on the roadmap ahead would be much appreciated!
I and others just had this conversation about wanting to send data to AWS using secure MQTT to eliminate the need for hacks to get the data over with other methods.
Seems like sending data via webhooks is still the way Particle prefers to go. I didn’t hear of any plans on adding the secure TLS MQTT to the Photon.
I ended up sending data to a Microsoft Azure Table Database using Webhooks.
I have Azure EventHub’s receiving the messages and Azure Stream Analytics to drop that webhook data into a Table Database. This has worked flawlessly for the last five days, and 15,000 messages received at the cost of about $0.77 cents per day for 2880 webhooks collected and databased.
@RWB that sounds cool, but it wouldn’t work for me. When I suggested using Azure I just got a big “LOL” back from the client. Pushing data via Particle Cloud works great, but it can take 3-4 seconds to receive data at the other end (if unlucky). For most applications this isn’t a problem, but we’re building a realtime transaction system controlling hardware.
I’ve built a quick prototype of my product using the Adafruit WICED Feather and the response times I’m getting are crazy good! It takes on average 55ms to store data via AWS IoT & MQTT. It’s almost so fast I don’t believe in it
When I see stuff like that I understand why my client is sceptical towards other solutions (even if they could move to something else) and I really want the same with the Photon.
Sounds good. I have not tested the response times for the webhooks and how quickly they go into the database because it’s not that necessary for my application.
Microsoft Power BI has added a data connector for Amazon RedShift Data Warehouse service if that’s something that may work for you. They are always working on adding new data connectors to bring in data from other sources, and they are not against having Power BI work with Amazon. I have not seen a better dashboard solution than Power BI
I’ve asked for and up voted this idea several times. I think the basic problem is that the Particle folks really don’t want to provide the option to “go around” their servers, at least not easily. The whole concept of how Photons etc. are managed, and the services provided by the Particle servers are a differentiator for their products.
Using a completely local development environment and then interacting directly with your own web services cuts Particle out of the loop, and makes them compete solely on hardware capabilities and cost. That’s a volume / low margins game with lots of players, and probably not a business I’d choose to get into either. Don’t blame them a bit.
Still, they have to understand that this make the Photon a non-starter for lots of applications. Consumers are largely OK with cloud services that they’ve never heard of, but business are not. Most of my work is in industries where regulation makes cloud a no-go, even if the company is OK with it.
So the Photon is a play thing for me. Monitors my water well and keeps the humidity right in my office. I love the platform personally, but it’s not even a consideration for any of the real IoT projects we have going on.
Nope, that's definetly not the reason! There were/are just more pressing topics to be addressed.
There are several real business companies which have opted for that platform and are actually paying good money for the service - and I'm quite positive they would contradict (if they were active on this forum or wanted to go public ;-))
@ScruffR Adafruit has used the WICED WiFi chip and now has the secure MQTT working with Amazon AWS services.
Since Adafruit provides all their code as open source code does that help Particle out at all when it comes to implementing the secure MQTT messaging or is it way more complicated than that?
That would be best answered by @mdma or @jvanier, but even if it was a relatively easy task, it would still require time to plan, implement, test, release it.
And as I said, more pressing (for the business) stuff was and is currently being implemented.
If you look through all the threads (like we do), you’ll see loads of people demanding that their presonal “dealbreaker” has to be addressed first (HTTPS/TLS vs. Enterprise WiFi vs. Defered Update vs. Data Storage vs. Ethernet support vs. SoftAP vs. better tool/librariy support vs. …) - some individuals even want several things done but each individual one first
For businesses, they want a direct connection from their connected products to the primary database services like Amazon AWS, Microsoft Azure, etc… all of which require the secure TLS or HTTPS communication. I’ve worked around it using webhooks which work just fine but it does cost me more money to do it that way plus there is one more layer in the equation “Particle Clould” that could fail or cause down times.
I also see Particle’s side of the equation where they have limited programmers and resources so improvements needed to be prioritized in a way that makes sense to them and their customers.
I think Particle is doing a great job at improving everything they are doing. As far as I have heard, adding TLS or HTTS for communicating directly with AWS or the likes is not on the list of things to work on anytime soon. Or is it?
Regardless I love the Particle platform & the community that has grown around it. It makes things much easier for nonexperts like me who have ideas but not enough experience make it happen with the advice of others.
Folks, one of the primary issues with HTTPS/TLS is the amount of flash and RAM required to implement it. The Particle platform integrates amazing cloud functionality not implemented in other platforms. This comes at a cost of both code size and RAM usage. If you forgo all of that and go “bare metal” with WICED then of course you can implement it like Adafruit.
The possibility of implementing HTTPS was demonstrated by Glowfish, though only as a user library. So it is possible.
The politics of not implementing a fully supported HTTPS/TLS library is another matter. There is no question that Particle’s limited resources are focused on serving existing and new enterprise (aka paying) customers who clearly are not demanding this functionality or otherwise it would have been implemented already. Simply said, Particle needs to make money to survive and it will do so with their primary target - large volume users. Particle devices may NOT be a panacea for all things IoT and it doen’t need to be. If Adafruit’s device is more suitable to an application, use it! If the Onion 2 is more suitable, use it! Just please don’t dump on Particle for not “having it all”!
I get it but are you saying Particle can’t have this HTTPS/TLS feature without removing other features due to flash and RAM limitations?
What ideally some want need is Particle + HTTPS/TLS.
I may be wrong, but if I remember correctly others have tried using the Glowfish library to do secure messaging with Amazon, but they couldn’t get it running properly, or they would be using it.
I would think having the “Non - Paying / AKA the people who just purchased the Photon/Electron kits” customers who are constantly using webhooks to push out to a cloud database would benefit from us sending data out directly to Amazon AWS or Azure vs. using Particles resources to send a few thousand webhooks per day per device.
Then again maybe it’s not a big deal or cost for Particle. I have zero clue what sending 90,000 webhooks per month per device cost Particle monthly. 90k webhooks x 100 devices get’s up to 9 million web hooks per month.
One thing I have realized is that IOT is certainly not free, and cost can add up quickly.
Again, I love Particle, and it’s community. The other options do not provide all that Particle does are not viable alternative options because they lack the cloud functionality that Particle provides.
@RWB, what I am saying is adding HTTPS/TLS functionality AND keeping all other functionality requires a substantial effort. This has been a persistent request and the Particle team is aware of it. The Glowfish solution was always considered experimental. Personally, I would love to see a user library (vs system firmware) that would support TLS at whatever flash/RAM cost associated with it. That way, if a developer doesn't need it, they don't include it!
@TheMadTexan Using a completely local development environment and then interacting directly with your own web services cuts Particle out of the loop
By no means? If I want to use AWS for data storage, it's actually a really good deal for Particle. I want to use Particle in my devices for the great API, OTA updates, short path from prototype to product and simple wifi reconfiguration. I may or may not want to use their cloud services, but those other things are (to me) the real killer features.
Most of their costumers will use the Particle Cloud service quite extensively. If I push all my data through other channels, it is actually VERY beneficial for them since I'm not costing them extra bandwidth. I fully see the point @ScruffR is making here. They already have HTTPS with Webhooks working and that covers +90% of the use cases. What I'm asking for is to add support for yet another layer on top of that that can only be used with some services.
AWS is still a requirement from my customer, so it just might be that I can't solve this case using Particle just now? That's ok. I made a prototype using the WICED Feather that logs data from a device. I'm getting crazy fast response times for storing data on AWS ( <60ms !!! ) so this will do the job for now. I just won't get those extra things that Particle offers (for now)
@RWB For now we’re only testing and sure - I know AWS can be expensive
Quite sure this won’t kill us though as each log entry = income. I’ve ordered a few more to stress-test, but I’m expecting that to work without a problem. Not sure about what the traffic will eventually be, but maybe 10 updates per minute at the most? Typically less though and it would only be for 1/3rd of the day. I could ping you once I have an idea of the cost?
Hi,
I’m running on aws about 450 customers and peak is 70 concurrent, over the last year. Cost is negligible, about $15/mth for lambda & gateway.
Having particle on the path for web hooks does not make sense in any way. It costs them money and it upsets my customers because Particle struggles to keep the service working. Yesterday’s acknowledged issues (http://status.particle.io/incidents/77dsssj2cght) didn’t hit me hard because not a lot of folk use my product on a Monday, but I’ve seen plenty of other issues over the past 6 months that haven’t even made it to status.particle.
Anyone reading this: up for grabs is a bounty of $500 USD for the person who sends me working direct SSL code (no webhook) that connects to AWS http gateway. If it needs to be via custom domain then that’s ok, just include a screencast as well as I’d want to see that before going into the depths of cloudfront custom ssl. Payment via PayPal. Offer limited to first person and end of March.
Edit: if its a particle employee that’s fine, but if its publicly released within a fortnight then it’s up to you to take colleagues out for lunch/dinner/drinks etc with the proceeds.
If you take a look at the TlsTcpClient library, a2-example, it looks hirotakaster has been working on getting it to work with AWS. That example doesn’t look complete to me because it’s not sending any data, but it might be worth asking him if he’s got anywhere with it since.
