Local compile, let's put an end to the confusion

Firmware
1

I hope the title does not upset anyone, but I have spend way too much time trying to be able to do this. There are posts all over the place on issues regarding this, and steps to do them all vary. It would be much better if either Particle gave a sticky post on the proper way to do these step by step, providing all the links and even an example for say to compile a hello world example and flash a photon with it. - Sorry if I sound a bit pissy about it all, Its just that things should be made allot clearer, and why there is no single source install like Particle Dev to include all this I just don’t know.

.
When I try to follow the directions here:
https://github.com/spark/firmware/blob/latest/docs/dependencies.md#1-download-and-install-dependencies

There are few issues that are confusing (at least to me) on that page.

#1) GCC for ARM Cortex processors.
It says “The build requires version 4.9.3 20150529 or newer of ARM GCC…” When it tried the latest version I get a whole new set of errors. Which one is the right one to use ?

#2) Make
Go to the link provided and what exactly do you want me to download there? I count 7 things to download.

#3) Device Firmware Upgrade Utilities
What Am I downloading here, I got the DFU from searching in the forums here, but the doc should be updated with proper link.

#4) Zadig
clear direction here

#5) Git
clear direction here

#6 Command line tools
This one is really messed up. It says: “On Windows, you’ll need to install MinGW and have it in your path to make available some of the typical *nix command line tools.”

So what about a download link for it ? Ok, I’ll let Google be my guide, that takes me here:
http://mingw.org/

Now what on this page should we get under the downloads section?
This further down on the guide, it says “The tool crc32 is also needed:” OK? so how and where do I get it?
If you say “•available in MinGW on Windows” then why mention it at all if its included in MinGW ?

==============================================================================

So, there are some good posts about doing it different ways like this one here:
Toolchain for Windows Installer
I tried this years ago and it worked great, Thank you buy the way for offering such a nice thing to all of us. But the problem today is that it does not work with the latest version of ARM GCC, at least on my PC.

So, what I would like to do is make a video showing all the steps on a clean install of Win7 or 10 with all its updates,
But until I am able to get it all working I of course cannot do it. I want to make this as painless as possible for anyone who wants to do this.

P.S. to the writer/s of the instructions, please don’t take me as saying anything bad about it, lets just clear it up and help make it all better for people.

2 Likes
2

@mikemoy I totally feel your pain on this, especially from back in the day.

The good news is that is as easy as pie to get all this up and running today without needing all those individual install processes and following long instructions from different people and places on the forum.

Today I just use the following tools that have windows installers for both that make the install just as easy as any other program that you are used to installing on Windows.

I use Particle DEV to code offline, compile & flash online easily. You can find the applications on Windows, Mac, or Lunix here : https://www.particle.io/products/development-tools/particle-local-ide

If you want to flash, compile, and do tons of other programming stuff then you just need to install the Particle CLI. You can find the Windows or Mac installer here: https://www.particle.io/products/development-tools/particle-command-line-interface

Are you aware of or have you tried these applications yet?

So if I have misunderstood something and this is not what your referring to :smiley:

3

I understand the frustrations, but wanted to chime it with some reasons why it could be hard to make specific tutorials.

Most people are generally fine with the current online compile system in which only the user code gets changed. I don’t think the majority of folks using the platform have the need/desire to do everything locally.
It’s mostly advanced users that have the incentive to try and do things locally to either make changes to the system firmware or keep everything in-house. While that’s perfectly okay, it’s necessary to understand the system is quite complex which is why they abstracted it all behind the online compilers. Should you shoose to do it all yourself, you’ll need to equip yourself with the required tools. That, as you may have noticed, can be a bit of a pain.

Having multiple operating systems, with different versions, floating around only further complicates the matter. Making a guide on Windows 7 doesn’t guarantee it works on Windows 10, let alone OSX or the endless Linux varieties. You’d then have to make guides for all of them should you choose not to exclude anyone.

The combination of local compiling being for ‘advanced’ users and the variance in operating systems is probably why they opted to go for generalized instructions that should be valid on most/all platforms. It means you’ll have to figure out some things on your own, but that’s not unreasonable to expect if your planning on dabbling in the system firmware, in my opinion.

All that said, of course it’d be nice it there were some more thorough guides/steps to follow. Unfortunately, those are susceptible to version changes, breaking them, as you’ve noticed.

If you do figure out a way to remedy these concerns, then please do share, improvement are more than welcome!

@RWB neither the CLI or Dev will allow you to compile locally, and as such will not allow you to dabble with the system firmware. If that’s okay with you, then yeah, those are probably the easiest to install.

To address some of the points mentioned:
2) Make does offer 7 downloads, but it explains what they’re for in the paragraph above them. It’s up to you to decide which one you prefer. Different operating systems/users can have different needs. If you’re like:“I don’t care, just make it work.”, why not opt for the “everything-included setup”?

  1. DFU Util. The docs offer a link with “everyone can get it here”, which is very much true. Click the link and you’ll be taken to the creator’s page. There you’ll find a part mentioning ‘releases’ with a link to the, well, releases. There you can pick the version you need according to your OS and preferences.

  2. it’s indeed ‘missing’ a link, but googling it wasn’t the end of the world, right? Regardless, that link could be added if it makes things easier.
    In regards to mentioning crc32 at all is because of:“linux users, please check with your package manager.” apparently it’s not included everywhere on every OS. On Windows it’s in MinGW, on OSX it’s in the OS, and on Linux it might not be in either. That’s why it’s mentioned.
    That’s also what makes it hard to write instructions, as I’ve alluded to above.

Hopefully this was somewhat helpful.

4

Sorry for the difficulties you had; it is indeed confusing. Also, for the unfortunate timing as a new official Particle FAQ on setting up a gcc-arm local build on Windows, Mac and Linux should be out in the next few weeks.

3 Likes
5

@Moors7,

I agree with with everything you said.
For me personally, I have been using Particle Dev. I like it a allot, and prefer it far over the Web IDE as it has many more features. However, the one thing Particle Dev lacks which the Web IDE has is to show how much ram and flash my app is consuming. Some may find this a trivial thing, but for me it’s imperative to know how close you are to the wall. It’s a nice way to gauge where you are at now, and how much to need to add to know if you should even continue on this platform, or move to another larger one.

So what I tend to do is every once in a while create and copy all my source files from PD and put them into Web IDE and compile it to know where i stand on code space. The problem there is that PD is ok with certain things and Web IDE is not. For instance I can use uint8_t & uint16_t in PD, but Web IDE complains about it. That I find strange if they are both using the same online compiler.

Though I am quite content with just using PD, and have no plans to modify other sources by compiling locally it gets me the ram & flash size results I want & moreover never need to be concerned that I need an internet connection to write code or should this great company go by-by I still will be able to compile, and support my devices until a replacement is found.

I know I am probably a rare case, but if so many other companies are able to put out a simple install.exe file to support their product I just don’t see why Particle has not done this as well.

6

Just install the CLI and compile with that tool... you get error and memory report without having to go through all of those machinations, compiling the exact same directory structure.

7

Fair points, they make sense.

If this is mainly about being able to see the sizes after compiling, you can compile with the CLI as well, which will also allow you to pick a specific version to compile against. (@BulldogLowell was quicker :’( )

Regarding the install.exe; it’s hard to do, honestly. For the reasons mentioned in my previous post (different OS, versions, etc). It’s not impossible, but it’d mean quite a bit of work, potential licensing issues due to dependencies, and yet another thing to maintain. It’s done for Dev and the CLI, but that hasn’t be/isn’t hassle free either.

The local setup is quite doable if you can follow the instructions and are willing to maybe Google a thing or too. It’s mostly for ‘advanced’ users who like to mess with lower level/system firmware stuff. For those, it shouldn’t really be an obstruction to install the required tools.

As for the others who’d just like to have the same functionalities in either IDE solution, they should really be getting those. Being forced into local development because Dev doesn’t show something as trivial as the compile size, that’s the real issue, in my opinion. I’m in the fortunate position that I can pester the team with my opinions every now and then, and this is something they’re definitely panning on reworking. Once the new library system is out, the next things are hopefully the developer tools. Fingers crossed.

8

I’d settle for compiling using the CLI, but that is the whole point of this post, without being able to have the clear instructions to install all the dependency’s I cannot use the CLI to compile it. Unless I totally missed something here.

9

yes, you did. The CLI is a cloud compilation tool.

from the command line...

$ particle compile photon myDeviceFolderName

all happens in the cloud, and you simply target the same folder as you are working with Dev.

10

Just throwing in my 2 cents: I have been working on a script for quite some time that makes it very easy to build locally on macOS and Linux distributions.

Website: https://nrobinson2000.github.io/po-util/

GitHub: https://github.com/nrobinson2000/po-util

Install:

$ curl -fsSLO https://raw.githubusercontent.com/nrobinson2000/po-util/master/po-util.sh
$ chmod +x po-util.sh ; ./po-util.sh install ; . ~/.bashrc
11

So does the CLI compile locally or only if you have an internet connection?

12

CLI uses cloud compile, just as Dev does (normally).

3 Likes
13

Thank you all for the help. CLI get around all my issues.

2 Likes
14

Perhaps to compliment instructions on how to set up a local build environment, a virtual machine image configured with all the necessary tools could be provided. That way someone who needs a local build solution, but doesn’t want to spend hours figuring out how to make one at least has a quick starting point.

15

But which OS, and which editor, and… There are quite a few personal preferences involved in a local toolchain that contribute to wanting one in the first place.
That said, I’d guess a community contribution to improve would be more than welcome :wink:

16

Choose the most popular free solution for each question. For example, which OS? Ubuntu 16.04 64bit LTS. Which editor? Atom.

An opinionated reference machine with everything configured correctly is better than nothing at all. Perfect is the enemy of good.

While a community contribution could work, an official reference would be better for troubleshooting and reporting bugs. It would likely also be easier and faster for someone who knows the system very well to build it.

17

Agreed, it’d be cool if there was a reference. But then you run into: I’ve never used Ubuntu, and never have used/really dislike Atom.
At that point you start running into:“how can I change this to [insert different editor]??”.

Yes, doing everything yourself might be (relatively) harder, but it ‘forces’ you into understanding it (albeit a little bit). That, in turn, will allow you to customize it to your liking, and more importantly, helps you understand were errors may come from should you face any in the future.

The underlying ‘issue’ is much more interesting though: what causes folks to want to compile locally? What benefits does it offer over the current solutions (with the online compiler)?
Personally I think it’s mostly for folks who want to make adjustments to the system firmware. If for that reason, I feel like you should be able to install some programs if you’re planning on dabbling with the the more advanced system firmware. Just an opinion, feel free to convince me otherwise :smile:

18

For myself, i have a few.

For hobbyists using the Particle product, I'd say the cloud compiler is the best idea.

For companies using the Particle product to depend on an outside source for compilation is not a good idea at all.
Having the tools in-house ensures that you have full control over everything. There will never be a surprise. Whereas should anything in the cloud compile change, it may cause an issue.

For myself I live out in the fringe areas, Internet connection is not always the best. So to be at the mercy of the internet to code my project gets frustrating real fast.

I said it before, and don't mean to sound like an old lady here, but if Arduino, and so many other even smaller companies have produced an install package to compile locally I cannot see why Particle has done the same. Everything they use to compile is freeware so whats the big deal.

19

The most useful feature of a reference image would be have a common platform as a starting point. Especially important to figure out if something that’s not working as expected is because of how you configured your system, or because of a bug in the system firmware.

For the most part, Ubuntu and Atom are user friendly enough for basic use, especially if any sticking points are well documented. Of course it’s a good idea to understand what’s happening under the covers. Starting from a working system and going down as deep as you need to is more approachable than going the other way though. It’s the reasons why systems like Arduino and Particle are helping bring in more people into electronics. Instead of figuring out the details of how microcontrollers work, you can start blinking an LED in a couple minutes. Then, when you need more, you can dive deeper until you graduate with a degree in electrical engineering.

I agree, I think the main reason to compile locally is to modify the system firmware, either to add a new feature, or to work out some bug. Making that process easier is going to help bring in more contributors.

20

Hey, I understand your reasons for needing a local compiler, and I share them too.
Have you looked at this by any chance?
https://docs.particle.io/faq/particle-tools/local-build/

It was added just few days ago.
Hope it’s what you are looking for!
Gustavo

1 Like