Tweak request to core-firmware/src/build.mk

Today I’ve been experimenting getting firmware for my :spark: Core compiling locally.

I’ve now un-Clouded my software from the Web IDE so that I can compile and flash it locally (used the great Windows tutorial https://community.spark.io/t/how-to-video-for-compiling-locally-in-windows/2457 by @seulater, tweaked to use Netbeans on Linux).

As I was using the ‘Multiple Files’ feature of the Web IDE, to compile locally I had to create separate .h and .cpp files in the inc and src directories respectively, and make sure my main code went in src/application.cpp.

I was struggling to tweak the relevant makefile so that my additional libraries would be built too.

In the end I had to add a few lines to the core-firmware/src/build.mk script, to that it referenced my extra files:

...
# C++ source files included in this build.
CPPSRC += $(TARGET_SRC_PATH)/application.cpp
CPPSRC += $(TARGET_SRC_PATH)/SimpleTimer.cpp
CPPSRC += $(TARGET_SRC_PATH)/LiquidCrystal.cpp
CPPSRC += $(TARGET_SRC_PATH)/DHT11.cpp
CPPSRC += $(TARGET_SRC_PATH)/main.cpp
CPPSRC += $(TARGET_SRC_PATH)/newlib_stubs.cpp
...

Is there some clever Make-fu that can been deployed that would auto-magically pick up all the .cpp files in src, so that you didn’t have to add each one manually?

Thanks :smile:

2 Likes

Actually, @jgoggins is working on some scripts that will make this process easier locally, as well as the command line tool which will make it easy to edit locally but compile via the cloud. I think the hope is to release both of these tools by the end of next week. :smile:

Thanks,
David

1 Like

i’ve found the exact same issue today, it does seem a bit weird that we have a massive list of files that we have to add our files to rather than just using a wildcard *.cpp in src/ like we do with *.h files in inc/

can’t we just have something like:

CPPSRC += $(wildcard $(TARGET_SRC_PATH)/*.cpp)

i tried the wildcard and it didn’t work.

another idea, is that as we recursively include all build.mk files, can we not remove application.cpp (and its line from src/build.mk) and have a directory structure like:

src/myapp/
   myapp.cpp
   mylib.cpp
   mylib.h
   build.mk

with myapp/build.mk saying:

INCLUDE_DIRS += src/myapp
CPPSRC += src/myapp/myapp.cpp
CPPSRC += src/myapp/mylib.cpp

this works, but i still can’t get the wildcard to work, which is a shame as it would mean we could have subdirectories with libraries in too.

what’s handy is that the myapp directory isn’t included in git, so you can safely pull new firmware without it deleting your program.

@sej​7278, what I have done to use local compile is to create a “lib” directory under the core-firmware folder. The makefile will automatically include whatever is in there as part of the build.

Then, under the lib folder I create a project folder, for example “test” and under that create “src” and “inc” folders to separate the .cpp and .h files. So the directory tree looks like this:

core-firmware
	- lib
	    - test
		- inc
		- src

Then, I create a build.mk file in the “src” folder which the makefile will find during the build. The build.mk file looks something like this:

# This file is a makefile included from the top level makefile which
# defines the sources built for the target.

# Define the prefix to this directory. 
# Note: The name must be unique within this build and should be
#       based on the root of the project
TARGET_PATH = lib/test
TARGET_SRC_PATH = $(TARGET_PATH)/src

# Add include paths.
INCLUDE_DIRS += $(TARGET_PATH)/inc

# C source files included in this build.
CSRC +=

# C++ source files included in this build.
CPPSRC += $(TARGET_SRC_PATH)/test.cpp
CPPSRC += $(TARGET_SRC_PATH)/otherfile1.cpp
CPPSRC += $(TARGET_SRC_PATH)/otherfile2.cpp

# ASM source files included in this build.
ASRC +=

The final step is to rename “application.cpp” in the core-firmware\src directory to “application.bak” and create an empty application.cpp file. I do this since the file containing the setup() and loop() code is in the files I am trying to compile.

I can only compile one “project” at a time using this approach since the makefile will find any and all .cpp and .h files in the lib directory.

I hope that all makes sense. If you have any questions, don’t hesitate to ask! :smile:

2 Likes

that’s essentially what i’ve done, i notice that you’ve not managed to get a wildcard working either for the c++ files.

maybe we need one of the spark team to RFC this sort of setup - i.e. removing application.cpp and having a user “sketchbook” directory like arduino…?

directory structure:

core-firmware/sketchbook/
   myapp/
      myapp.cpp
      mylib.cpp
      mylib.h
      build.mk

build.mk looks like:

INCLUDE_DIRS += sketchbook/myapp
CPPSRC += sketchbook/myapp/myapp.cpp
CPPSRC += sketchbook/myapp/mylib.cpp

delete core-firmware/src/application.cpp and remove it from core-firmware/build/build.mk

git ends up like:

    modified:   build/core-firmware.bin
    modified:   build/core-firmware.elf
    modified:   build/core-firmware.hex
    deleted:    src/application.cpp
    modified:   src/build.mk

    untracked:  sketchbook/

we just could do with a way to build one project at a time from inside the sketchbook directory (save for renaming all of the build.mk files) maybe pass in a project name to make?

@sej​7278. what I was trying to point out is that a folder named “lib” or “libraries” is automatically included by the makefile, wildcards and all. I don’t have to modify anything!

nah, that’s handled by the recursive search from the toplevel core-firmware directory for all build.mk files, it works if you rename lib to sketchbook

SRC_MAKEFILES := $(call rwildcard,$(SRC_PATH)/,build.mk)

you’re also manually setting your INCLUDE_DIRS and CPPSRC in your build.mk

that’s why you couldn’t have multiple projects inside lib (or sketchbook) as it would build them all.

@sej​7278 you sound a lot smarter than me at makefiles. All I know is that it works! If I understand correctly, you want to NOT have to put a build.mk file in the path like I do. If so, I understand more clearly now. Is this correct?

no, i want our way of doing things to be considered as the norm and get merged into upstream, so that we don't have to delete application.cpp and edit build.mk @dave - could you consider this?

i actually think its a lot less confusing than people editing application.cpp and having all their libraries in one monolithic file and editing build.mk that will get reset whenever they sync with github

if you do it our way you get a single directory of your own with all your files in, you own minimal makefile (especially if we can get wildcards working) and it doesn't get deleted. the only thing to solve is how to not build all the directories under sketchbook (or lib in your case) just build the one project you want.

I know the makefile got a make-over ( :slight_smile: ) a while back to be considerably faster, but our firmware guru would be @zachary. He’d be able to speak more intelligently than I on pros and cons of makefile changes. I think a wildcard sounds innocent enough, but there might be trade-offs of which I’m not aware.

Thanks!
David

@sej​7278, I agree but they put the application.cpp file there as a default program containing the setup() and loop() code. I just put a blank application.cpp file and all the problems go away. Then building “out” way is a done deal! :smile:

i’ve solved the problem of building a single project in the core-firmware/sketchbook/ directory

first you can either have a core-firmware/sketchbook/build.mk containing PROJECT=blink (and whatever other global variables you want to set in all your programs, e.g. PATH) which you change based on what program you want to compile, or you can pass it as a variable to make like:

make clean all PROJECT=blink

then in each program directory (e.g. core-firmware/sketchbook/blink/) the build.mk looks like this - with the paths all enclosed in an ifeq-endif string match:

ifeq ($(PROJECT),blink)

# look for includes in cwd
INCLUDE_DIRS += sketchbook/blink

# list c++ files here
CPPSRC += sketchbook/blink/blink.cpp
CPPSRC += sketchbook/blink/mylib.cpp

endif

1 Like

@sej​7278, nice work! I will definitely try this one. This needs to go under he tips and tricks topic for sure. :slight_smile:

Thanks for all the thought, effort, and time you’ve put into this @sej7278. It’s appreciated, and I like where this is going. Please submit a pull request to the core-firmware repo. That would help ground our discussion in working code, specific issues, and changes to existing behaviors. Here are some requirements for how to structure this.

  • Tinker (the current application.cpp) must remain in the repo (somewhere) and be the default app built by simply calling make with no further arguments on the command line. The file could be called “tinker.cpp”.
  • Multiple apps should be able to be stored (and ignored by git), while only one is built with a given make command (your main point).
  • Rather than “sketchbook”, let’s call the directory containing the apps “applications”.
  • Similarly the environment variable passed to make should be APP=tinker or APPLICATION=tinker.
  • Ideally, no Makefile or build.mk changes would be required to build a new app. If some change is required, it should be inside the application-specific directory and easily, clearly documented.

Let me know your thoughts, and I look forward to a pull request.

Cheers!

1 Like

@zachary I think i’ve covered everything you want in this pull request:

I don’t know how this will effect your cloud build setup, and I guess part of the getting started docs may need to be changed where it refers to src/application.cpp

I actually think (hope!) this will be less confusing for end users than editing application.cpp, certainly took me a while to realise that application.cpp was Tinker!

1 Like

Another thing that occurs to me if we take up the “applications” directory idea is that we can bundle a lot more example applications than just Tinker, as we have an unlimited directory rather than a single application.cpp file

I can’t believe we don’t have Blink or Hello World! (via Serial) examples :wink:

1 Like

On the web IDE, if you create a new sketch there is an option to load one of several canned sketches, including the blink sketch.

ah i guess they're not on github then, maybe in docs instead of core-firmware. still would be nice if we could bundle those into the local build environment.

Hi @sej7278

They are here:

There is a lot of development going on right now for a library feature in the webIDE too. That should be available very soon now.