Libraries, gcc build environment and documentation

Firmware
1

Hi, I’ve just got my SparkCore and am not really interested in using the cloud stuff, I want to compile locally.

I’ve managed to edit application.cpp and compile using gcc-arm-none-eabi and flash with dfu-util, and would like to move onto using libraries.

There doesn’t seem to be any documentation regarding where to put libraries. I found the uber-library-example but that doesn’t actually say where to put the libraries. i assume you put it in the core-firmware/inc directory?

Also are people really working within the core-firmware directory and cleaning it out/copying it for every project - there must be a better way than that? I haven’t had a good look yet, but could we not have something similar to Arduino in that you have a libraries directory that’s automagically searched from the Makefile? Or can we do the reverse and put our libraries/sourcecode wherever we want and have it pull in the firmware/comms/common libs?

How do people distribute their projects, they can’t seriously be tarring up the entire core-firmware directory?

On the cloud IDE there appears to be no way to upload libraries although you seem to refer to that ability in various posts - without actually saying how to do it, but hey nice screenshots!

What’s Sparkulator, is that another name for the cloud IDE that’s accessed via the build button? Its certainly not called Sparkulator anywhere on the IDE page itself, maybe the build button should be renamed?

I hate to say it but the documentation is a bit all over the place, some of the links don’t work and the forum/github has some useful information that isn’t included in the docs, like spark-cli, gcc and dfu-util are all relogated to github like they’re second-rate to the cloud stuff. Also its probably worth mentioning somewhere that you must be root or use sudo with dfu-util, like you must use 0.7 or later.

That said I prefer the github layout to the spark website with all those scrolling pages and broken anchor tags that take an age to render. Maybe we should move all of the docs to github then users could edit them and move important stuff hidden in forum posts into somewhere obvious.

From the cloud IDE the docs link doesn’t take me to any documents regarding the IDE, it takes me to getting started plugging in my core - well if I’m using the IDE i’ve pretty much got that far lol!

There really should be a place to go to find out if there is a #define SPARK for example, which seems to be dotted all over various forum posts, but with no solid info. Seems like a pretty basic thing to implement and is essential if you’re going to get any traction porting libraries.

I’d like to know if/how we can do OTA firmware uploads from binaries we’ve compiled using gcc but have no idea where to look for that, spark-cli’s flash facility seems to use USB only.

Oh and that’s another thing, the pretty major point of modes is very well hidden, took me a while to realise you couldn’t just turn on your core and flash it or use the serial monitor, you have to press various buttons in a specific order for a specific time. Hell splash that info all over the homepage please! Funnily enough I can’t even find the link to where that info is anymore…

2

Hi,

I’d like to try to answer a few of the issues raised here.
Compiling locally: there are several tutorials to be found on the forums on how one would go about building firmware locally. They’re not perfect for every user scenario, but they give you a good idea in which direction you’d have to look.

Libraries: being a heavily debated, and highly anticipated, topic for the last couple of weeks/months, the :spark: team is working on finding/creating a most user friendly implementation so that even beginners can use it without much know-how. I believe the idea is to make it work with GitHub so that you can easily manage, create, edit and share your work. With sharing being a highly requested feature, they have to make sure they implement it really well, otherwise it will cost them only more work to try and fix it after the fact.
Currently you can add libraries using the Spark IDE (which has unofficially been dubbed the “Sparkulator” on the forums). Seeing as I’ve already answered that question once, I’d like to redirect you here: https://community.spark.io/t/how-to-use-libraries/3050 Since I haven’t got much experience with the local environment, I’m not sure how you’d go about adding them there. Maybe one of the :spark: elites can chime in on this one.
In the end sharing, editing, distributing, and in general, working with libraries should become a lot more pleasant once the library functions are being published by :spark:.

Documentation: I’d have to agree with you on the fact that the documentation isn’t yet where it needs to be. There are a lot of improvements to be made, and I’m sure those will be made accordingly in due time. However, with the current situation, it seems as if :spark: is spitting out more upgrades than they are able to document. To be honest: THAT’S AWESOME! I’d much rather have them working on actual problems and providing amazing consumer/community support, than I’d have them writing up dull documentation.
Of course it has to be done, but seeing as we’ve got an awesome community, which helps out wherever it can, most people can get their support on the forum if needed. I’ve learnt more from these guys, reading up on the forum, than I’d ever been able to by reading the docs. Because truthfully; if you’ve bought a new computer, you’re going to power it up and mess around with it. You’re not going to sit down for a week just to read the manual, trying to remember where the power button is. That’s what makes the Spark great: you can tinker around with it, learning as you go.
To address your suggestion about moving the docs to github; they’re already there. The docs on the website are being pulled from the github files, which you can edit, and make a pull request from. That being said, feel free to make improvements and submit them. That’s the great thing about being open source: everybody can help out. If you feel you can make a meaningful improvement, wherever it may be, feel free to do so!
The Cloud IDE (AKA Sparkulator) probably takes you to the docs homepage because there’s not much to tell about the IDE. You simply write your code, hit ‘verify’, edit if needed, and then hit ‘flash’ to get it on your Spark. That’s how easy it is. All other buttons have a nice label to them which makes them pretty much self-explanatory as far as I’m concerned.

OTA binary flash using CLI: https://github.com/spark/spark-cli

Assuming you’ve figured out how to use the CLI, and thus have found the github page explaining how to install/use it; I could be mistaken, but I believe the sixth point to be “Spark cloud flash”. As far as I can understand the functions described below it, this will take a ,user specified, file and upload this to your Core.

Modes
Having typed in just “mode” in the forum search bar, a nice selection of topics came up, this being one of them: https://community.spark.io/t/what-does-the-mode-button-do/534 which is pinned by Zach, the CEO. Also, that information kinda IS on the homepage. Be it the documentation home page, but a homepage nonetheless. The ‘getting started’ page explained the modes under ‘buttons’. Check it out if you’d like to: https://github.com/spark/docs/blob/master/docs/start.md (the github from which the docs are pulled), and on http://docs.spark.io (that same github file, now on a webpage)

Rest assured that they are actively working on it, but there are several other things that have to be dealt with/upgraded/fixed as well. They’re a relatively small team, but they’ve been doing some awesome work the last couple of months, with new and awesome features coming out very couple of weeks. It’s just a lot of work to make it all work nicely for everyone, and to do this, they need time. (Fortunately) the docs aren’t their first priority, and they take the time to make sure their new features work well, for everybody.

I hope some things I’ve said here are of use to you, and if not, feel free to comment/ask away. As I’ve said before; this is a really AWESOME community, and there are a lot of people here, more than willing to help out if and where they can!

Have a good day,

Jordy

2 Likes
3

Ah thanks Jordy, a lot of helpful suggestions there - including the github link to the docs which I shall use from now on in preference to the actual docs page. Markdown is inifintely superior to this mishmash of scrolling anchors, and just to think I was about to wget mirror the site so I’d have a local copy, now I can git clone it!

One thing I’ve noticed throughout your post though is that all the good stuff is actually in the forums, well to me that’s far from ideal. As I work with the SparkCore I may actually make a pull request with some of my notes now I know the docs are on github, or perhaps we need some moderators to make an FAQ summarising the useful stuff hidden in the forums.

As far as libraries go, again I’m not really interested in how they will eventually work in Sparkulator, I just wonder how everyone is using them now as it seems local builds are the only way to do it. I’ll have a look around github and see if the build farm setup is on there and see how that works, oh but isn’t that the bit that’s not been opensourced yet?

I found the “spark cloud flash” from the spark-cli help “spark --help” is actually more helpful than the docs pages! I haven’t tried it yet but it sounds like it should work.

1 Like
4

Hi @Moors7 and @sej7278

Thanks for the great explanation Jordy! You are right on all counts. There is a new feature coming that will allow users to pull libraries into the web IDE in a simple and easy to use way! That’s great news, but there is a small catch: the library owners must “format” the library in a way that the automated tool can work with them, so it is taking some time to get the feature tested and into everybody’s hands. It is very close now! Today the “Sparkulator” webIDE has a tabbed editor for libraries but you have to cut and paste–the new feature will make those tabs pullable from other sources.

Libraries in the local build environment can be handled in many different ways depending what environment you are using. For instance, I am happy to use emacs and the makefile to build locally, so I just add what ever libraries I am using to the local makefile and away it goes. I have certainly thought about adding a user library directory and a set of make rules for it, but I think the folks that use Netbeans or Eclipse or other IDEs have their own solution. This hasn’t really been a problem that I have seen since it seems that most of the folks that use a local IDE are more experienced and can set it up however they like it.

The documentation, like all other aspects of Spark, is a work in progress. Several of the elites, who are the moderators that you are asking for, have started contributing to the documentation, including the relatively new troubleshooting page.

Sometimes the doc lags behind the code for a reason, like we have WiFi and Cloud control at startup via include files, but the Spark team is spec’ing out a better way, so they don’t want to doc the existing way and then throw that doc away and doc it again when there is a new way. In a perfect world you would not have to make these kind of tough choices, but personally I think that is the right call, based on the team resources. Those that need the existing way can find help here in the forum.

My experience is that the online doc works better in some browsers than others. Another good thing to do is to make a local repro of the doc and web surf it locally–that’s much faster.

There is a lot of other stuff I could comment on, like I don’t need to be root or use sudo to use dfu-util, but if you do it means your user is probably just not in the group that the device belongs to. This is more of a general Unix problem.

So @sej7278, just keep asking questions and we’ll try to help! Searching the forum is always a good idea too!

1 Like
5

Glad that I could help.
I’m not really fond of the docs page either, for it seems to gain pleasure from making my iOS Safari crash any time a decide to use it… The github is a nice alternative to this :slight_smile:

I’ve been following the forums for a good while, trying to read anything that seems remotely interesting to me. Doing so, I’ve made a ‘brain backup’ of the most useful topics. Most of the time I know I’ve seen it somewhere on the forum, I then just have to find it. What makes this a LOT easier is the ability to ‘star’ topics, which makes then SO much more convenient. Anytime you find something interesting, just star it, you’ll thank me later :stuck_out_tongue:

Seriously, DO make a request I’ve you feel you’ve got useful material you could contribute. The whole idea behind open sourcing this is that anybody can help out. It’d be great if you could help setup some documentation, for I’m sure a lot of people would be eager to use it.
On some categories there are :spark: elites trying to summarize the huge amount of info. But since there is a lot of info to be dealt with, any and all help would be greatly appreciated!

Current implementation of libraries on the Sparkulator can be done by pressing the little ‘+’ sign on the top tabs bar. It’ll create two new files, .h and .ccp. There you can paste your code accordingly, after which you might have to add #include “application.h” to both. (I believe that’s because they’re not being pre-processed, but don’t hold me down on that one, I’m not sure).

Personally I like to code in the Sparkulator, or notepad++. That’s probably because I kinda ‘suck’ at coding, so it wouldn’t matter where I’d code, it’d probably fail anyway, so I thought I’d keep it basic :stuck_out_tongue:. What I do think is nice, is the fact that you can paste your code in the Sparkulator, and then download the compiled binary. It’s a lot more convenient than setting up a local IDE, since my code is so basic, I wouldn’t need all of those fancy local IDE stuff. But being able to flash that downloaded binary via USB is a gift from the heavens! It’s just SO much faster!

Like @bko said, feel free to ask questions, we’ll help out where we can :smiley:!

6

thanks guys, i was hoping not to come across too negative, but i unpacked my core, got a flashing led via my phone then thought “well how do you do something more advanced?” and really struggled to find the info.

anyway i’m going to plod along now and see if i can setup a makefile that does something clever like copy my source/libs into the core-firmware directory (or symlink) its all good fun!

2 Likes
7

Definitely not negative! It’s constructive criticism!

Docs are something that eats at everyone, but we’re not sure of a good fit for how to best consolidate the trove of knowledge into a single source. A wiki has been suggested many times, but as I once heard, “wikis are where information goes to die”. I think that basically means that things can get a little too loose and out of control and sometimes pages can get lost in the clutter.

And, like @bko said, the docs are a moving target. It takes a sprint cycle or two (2-4 weeks) for new features to settle down. I personally work at a place where we like to “release early, release often”. It keeps things moving, users engaged, and the product evolving at a much faster pace. However, documentation always seems to lag behind. I know a lot of the elites can sometimes not see the documentation lagging as we are so active in documenting and helping out in the forums (which is a terrible way for people to learn the product!).

I am still on the hunt for a cozy fit for documentation that’s easy to maintain without bogging everything down, and would love any suggestions we could aggregate, research, and present to the Spark Team!

To your other question about local build environment setups, I’ve also been struggling with trying to figure out a best approach. I can hack user firmware reasonably well, but going much deeper into the core firmware, makefiles, and the like is beyond the scope of my current abilities! Maybe some of the smarter elites can share their methods and possibly propose some sort of template that the noobs like me can follow. I’ve been developing 99.9% in the web IDE due to this. I use a real IDE in my day job as a web developer, but fortunately HTML/JavaScript/CSS doesn’t need to be compiled! I’d love to use a real IDE locally to develop, but don’t want to incur the burden of cloning/copying repos for every possible project, figuring out what to do with makefiles, and all that other stuff. At the moment, it probably takes me just as long to follow that entire process as it does to just do it in the web IDE!

8

I've never heard that said about wikis before and I would rather assert wikis are where information goes to thrive. Whatever, the documentation is dead where it is, or as good as.

What needs to be released early and often is the documentation as well as the code. The docs would be best pre-released - before the code - in my view. Are the Spark team tasked by the Spark management (I know there is considerable overlap) to really be developing blue sky, do what you want, do what you think is cool, tell us about it afterwards? No, surely not. I can't believe it would not be helpful to the Spark Team to have to work against user documentation as a spec. I am not saying this might not be modified during development, just that it ought to exist ideally before devt starts or be among the first things written, first version, before any variables are declared.

How it all works is only in the heads of a few. There will always be a few inaccuracies and even perhaps inadequacies in the docs, but the Spark Core currently, and as excellent as it is, is unnecessarily difficult to develop for because the language and library docs are inadequate. Too inadequate.

So true! If all the advice so helpfully given out here, much of it patiently repeated again and again, but through over-eagerness and misunderstanding this help is occasionally inaccurate or sometimes not as full as it might be, if it were all in a wiki, then a pointer to the appropriate wiki page could be given rather than having to trot out the same old advice much of which is necessary advice for new and old here. Necessary because the documentation needs improving.

Imagine if the advice give here could just be pasted into an existing or new wiki page, or that better advice here could be used to update a wiki page. And that anyone (with a login) could do so.