Neopo: A lightweight solution for local Particle development

Hello everyone.

I am proud to announce the release of neopo, a utility for locally building and flashing Particle projects to Particle devices that integrates with Particle Workbench.

TL;DR

I’ve made a replacement for po and po-util written in Python that is fully compatible with Particle Workbench and particle-cli. Here is the GitHub repository:

You can install neopo with the following command:

$ bash <( curl -sL https://git.io/JfwhJ )

Story

Four years ago I created po-util, a large, messy shell script for building, flashing, and interacting with Particle devices. After maintaining it for two years, it was in desperate need of a rewrite because the script was over 2,000 lines long.

I created a new repository called po and continued development there, breaking up the script into multiple files and taking on dependencies in a vain attempt to improve the maintainability of the project. Over the past year, I only made a few updates to po, letting the project grow stale. It grew stale because I had not fully integrated the project with newly developed tools like Workbench and the improvements made to particle-cli.

Since the initial release of Workbench, I dreamed of integrating its features and dependencies into a slick, useful tool that can be used on Linux and macOS independent from the GUI of Visual Studio Code.

In the past week, I’ve realized that dream. Neopo natively uses the Workbench project format and relies on the same dependencies and Makefile. It can even manage dependencies as well. Overall, not only is neopo an excellent companion to Workbench, but it offers unparalleled utility in build environments where Workbench is not an option as well.

To all users of po, I highly recommend switching over to neopo for your local Particle needs, as it is an immense improvement over the previous project. To everyone else, I would greatly appreciate it if you could give neopo a try.

I await your feedback and bug reports. Feel free to make an issue on the GitHub repository.

8 Likes

Update: Raspberry Pi support

Last night I compiled gcc-arm 5.3.1 for the Raspberry Pi. I have uploaded the compiled toolchain to the neopo repository on GitHub. Now when you install neopo on a Raspberry Pi it will download my compiled toolchain instead of the x86 toolchain hosted on binaries.particle.io

I’ve also fixed the install script to download the required dependencies for Particle CLI.

Neopo now works fully on Raspberry Pi.


I timed the compilation of an empty cleaned project on a Raspberry Pi4 with:

time neopo run compile-all

It took 3m53.841s to compile, compared to my Linux desktop which took 0m45.049s.

Building again without cleaning first took 0m7.179s on the Pi4, versus 0m1.622s on my Linux desktop.

3 Likes

If anyone wants to try to get neopo working on Windows I’d greatly appreciate it if you could share your results.

great work Nathan.

Why would one use neopo vs just using the Workbench?
I looked at the examples and couldn’t figure it out.
Thanks!

Thanks Gustavo.

Since the early days of Particle Dev and Particle Workbench, there have been many scenarios where other developers and I have needed a tool that could be used to quickly compile a project or flash firmware to a device without having the time or interest to open up a large IDE.

Back when the Web IDE was king and before Workbench was released, I attempted to build a wrapper around the makefiles in deviceOS, called po-util, or later po, that provided that experience, but it was large and clumsy because the neatly organized and now easily available dependencies of Workbench had not been released yet.

Neopo is most useful for developers who are confident with the command line, as it provides a powerful CLI experience much like Particle CLI but it offers the build features of Workbench.

It’s also incredibly portable and extensible, meaning that it can be easily used in automated, restricted, or remote environments like Travis CI, Raspberry Pi, or any macOS or Linux (Ubuntu/Debian) host.

Since the core functionality of neopo is kept in a single Python script, savvy users can integrate the module into existing Python solutions, providing possibilities for sophisticated Particle development workflows. I’ve done something similar in the past, integrating po into a script that could be used to flash firmware and import devices into a product in series, greatly accelerating the provisioning process.

While neopo is complementary to Particle Workbench, it is by no means a replacement. I have no idea how to implement Intellisense or the Debugger on the command line or in an editor like Vim. Overall, I hope that with the redesigned tool I can provide an improved experience for developers who had used po or po-util in the past, and reach new users who may be interested in improving their Particle development workflow.

alright, thanks Nathan for the extra explanation!
Gustavo.

Update: Windows support

I have added Windows (Cygwin) support for neopo. Here are the installation instructions: https://github.com/nrobinson2000/neopo/blob/master/docs/windows.md

Update:

I have made tons of improvements to neopo over the last month. I’ve also written some documentation: https://nrobinson2000.github.io/neopo/full-docs.html

Update: New domain

I purchased neopo.xyz today to host the documentation and install script.

Neopo can now also be installed using the following:

bash <(curl -sL neopo.xyz/install)
2 Likes

Nice work and thank you, Nathan.

I’ve been using neopo for a few days now in my projects and it’s working great. The reason why I’ve always wanted something like this is because I want to be able to do command-line local compiles and deployment. This is really the first practical easy to use way to do this.

I will also say that the exact same project which works well with neopo, will also open and build great with Particle Workbench, and can compile using the Particle CLI.

Also note that the compiles that come out of neopo are exactly the same (byte size) as those that come from Particle Workbench. That gives me some reassurance that neopo is compatible.

1 Like

Update: Support for deviceOS 2.0.0 and gcc-arm 9.2.1

I have made improvements to neopo to select the appropriate gcc-arm version when compiling a project and download gcc-arm versions on the fly if needed.

I compiled gcc-arm 9.2.1 natively on Raspberry Pi 4 so that projects using deviceOS 2.0.0 can be built successfully.

To upgrade neopo to get these new features simply run:

$ neopo upgrade
1 Like

Update: Support for building on aarch64

After many hours of compilation and needing to get a larger micro SD card I have produced gcc-arm 5.3.1 and 9.2.1 toolchains for aarch64.

I used Manjaro ARM XFCE on Raspberry Pi 4 to produce the toolchain and test neopo.

I have not updated the universal install script to detect aarch64 yet, but for anyone who is using Manjaro or Arch on Raspberry Pi or other aarch64 systems the following commands are the required steps to install neopo. Afterwards you can use neopo like on any other Linux environment.

# Install essential dependencies if you haven't already
sudo pacman -S base-devel

# Clone neopo and prepare to create package
git clone https://github.com/nrobinson2000/neopo
cd aur/

# Edit PKGBUILD to comment out x86_64 depends and uncomment aarch64 depends
vim PKGBUILD

# Create and install the neopo-git package
makepkg -si

# Install neopo dependencies
neopo install

# Fix and install particle-cli
export GODE_DEBUG=info  # This is optional
neopo particle # This will fail

# Replace node executable
mv ~/.particle/node-v12*/bin/node ~/.particle/node-v12*/bin/node.old
ln -s $(which node) ~/.particle/node-v12*/bin/node

# Complete particle-cli install and setup non-root USB access
neopo particle configure usb

UPDATE:

I have started my own package repository on GitHub for x86_64 and aarch64 packages. You can install neopo with the following on Manjaro/Arch:

# Install essential dependencies if you haven't already
sudo pacman -S base-devel

# Clone the package repository and install neopo
git clone https://github.com/nrobinson2000/packages
cd packages
./install-pkg neopo

All of the commands I listed above to fix the installation are run automatically after building and installing the package.

UPDATE:

I have modified the universal installer to use nrobinson2000/packages to install neopo. Now on any x86_64 or aarch64 system running Manjaro/Arch you can use the universal installer to properly install neopo as a package.

bash <(curl -sL neopo.xyz/install)
2 Likes

Update: neopo-git on the AUR

I have submitted neopo-git to the Arch User Repository.

yay -G neopo-git
cd neopo-git && makepkg -sif

Due to a bug in yay, the commands above are required to install neopo on aarch64.

On x86_64 the following will suffice:

yay -S neopo-git

Until the bug in yay is resolved, I recommend using the universal installer as it will work on either architecture and apply post-installation tweaks automatically.

1 Like

Update: Manpage for neopo, better Python support

I have finally written a manpage for neopo. It can be accessed with:

$ man neopo

I’ve made an online version available: https://neopo.xyz/docs/neopo.html

Additionally, when installing neopo it is now properly built as a Python module using setuptools/pip, providing better support for using neopo within scripts and on the command line. In future releases, I plan to break the neopo source into separate files and follow Python conventions better.

I’m in the process of migrating from Travis CI to GitHub Actions. You can see the GitHub actions builds for neopo here.

2 Likes