Startup weirdness p1: seems to skip startup code and goes straight to wifi.listen

hello all,

we have a board with a p1 which we program via usb in the factory. we flash device os 2.0.1 (2 parts) and our application. then we power cycle and test if the device operates correctly

the application starts by logging some stuff, does a modbus request and, if no credentials are set, does Wifi.listen() where it will wait for the user to enter credentials etc. see code below.

now what we see is that after factory programming the device will not log anything, it does not do the modbus request but does go to listen mode. restart the device and same thing

i can add/join/claim the device and the application starts logging and works fine

a function is added to clear the credentials. it will do:
WiFi.clearCredentials();
EEPROM.clear();
System.reset();

after this is done, when i restart the device, it does log and it does do the modbus request

anyone any idea why it does not log after the first time programming?

thanks so much
frank

#include "Particle.h"
#include "MeasureModule.h"
#include "MMFirmware.h"
#include "ModBusClient.h"
#include "Site.h"
#include "Socket.h"
#include "SocketFirmware.h"
#include "Version.h"
#include "ParticleAPI.h"
#include "AVRProgrammer.h"
#include "ChargingAlgorithm.h"
#include "edge-site.h"

PRODUCT_ID(12543);
PRODUCT_VERSION(1);

// does not startup connectivity only after Particle.connect() is called
SYSTEM_MODE(SEMI_AUTOMATIC); 

// allow loop to continue without wifi
SYSTEM_THREAD(ENABLED);

// WiFi Antenne 
STARTUP(WiFi.selectAntenna(ANT_EXTERNAL)); // selects the u.FL antenna

SerialLogHandler logHandler(LOG_LEVEL_ALL);

LEDStatus blinkOrange(RGB_COLOR_ORANGE, LED_PATTERN_BLINK, LED_SPEED_NORMAL, LED_PRIORITY_IMPORTANT);
//  blinkOrange.setActive(true);
    
unsigned char debug_out = 0;
        
#define LOOP_CYCLE_PERIOD      1000
#define LOOP_ID_PERIOD          100
#define DEBUG_COUNTER            10

unsigned long next_id_nr_time;
unsigned long next_cycle_time;
unsigned char debug_counter;

unsigned int current_id_nr;
unsigned int read_or_write;

unsigned int interrupt_loop;

unsigned long pwh_last_publish = 0;

void setup() {

    // just so i can hit 'connect' in time to capture serial logging
    delay(2000);

    WiFi.setHostname("edge-ev-hub");

    Log.info("System version: %s", System.version().c_str());
    Log.info("DeviceID: %s",System.deviceID().c_str());
    Log.info("Hostname: %s",WiFi.hostname().c_str());

    int errorCode = ERROR_OKAY;

    // do not do this in the factory
    if(WiFi.hasCredentials()) {
    
        Log.info("got credentials");
        
        // tell the server/particle cloud how we are doing, every 5 minutes
        Particle.publishVitals(300); 
    }

    // test

    // fuse default:    0b.0110.0010 --> internal rc 8 mhz / 8
    // fuse programmed: 0b.1xxx.1111 --> external crystal 16 mhz / 1

    // mm_set_fuses used bit-bang mode to read and possible set the fuses
    // after this the spi mode van be used (e.g. to program the atmega)

    mmf_set_fuses();
    
    // initialize measure module (spi)
    mm_init();

    // initialize modbus client (serial)
    mb_init(); 

   
    // checking mm version and upgrade if needed
    unsigned short version = mm_version();
    
    Log.info("mm version: v%d.%d.%d",(int) (version>>12),(int) (version>>8&0xf),(int) (version&0xff));
        
    int need_upgrade = mmf_needs_upgrade(version);
    
    if(need_upgrade) {

        Log.info("upgrading mm");

        errorCode = mmf_firmware_update();

        Log.info("upgrading finished with %d", errorCode);
    }

    // internal boot test 
    if(!WiFi.hasCredentials()) {

       if(errorCode == ERROR_OKAY) {
        
            int mm_read_status;
            int tries = 3;
        
            do {
                
                mm_read_status = mm_read();

                Log.info("got return code %d", mm_read_status);

            } while( (mm_read_status == MM_READ_CRC_ERROR) && ((tries--)>0));

            if(mm_read_status != MM_READ_OK) {

                errorCode = ERROR_SPI;

            } else {

                Log.info("got values %f, %f, %f",site_current.current_current[0],site_current.current_current[1],site_current.current_current[2]);

                if((site_current.current_current[0] > 1.0) || 
                    (site_current.current_current[1] > 1.0) ||
                    (site_current.current_current[2] > 1.0)) {
                        
                    errorCode = ERROR_MEASURE;
                }
            }
        }

        Log.info("sending error code %d", errorCode);

        mb_send_id_and_error_factory(errorCode);

        Log.info("entering listen mode");

        WiFi.listen();

        Log.info("done with listen mode");

        // delay(2000);
    }

--- end so on ---

this problem only occurs when the device is never claimed/added/joined. reprogramming does not help and it will still not log

claiming/adding/joining once and it will keep on working normal, also after “clearing” and reprogramming

You also need to flash the bootloader. It’s going into listening mode because the bootloader dependency is not met, so the device needs to go into safe mode. But since there are no credentials, it can’t connect to the cloud to get the bootloader OTA, so it goes into listening mode. Your user firmware won’t run with a missing system dependency.

that explains the short magenta flash after connecting the p1! it is downloading the bootloader!

i was under the impression that the bootloader was installed automatically, which is correct but not till connected

Prior to 0.7.0, the bootloader was embedded in one of the system parts on the P1. Since then, there hasn’t been enough room to fit the bootloader, so it needs to be separately flashed, either by USB or OTA.

If you have a normally functioning device flashing OTA happens automatically, but for devices that are not set up yet it’s best to flash the bootloader by USB in --serial mode (listening mode, blinking dark blue).

okay, thanks, makes sense

the confusing part was that i do have the 2.0.1 bootloader (p1-bootloader@2.0.1+lto.bin) in my factory project but i do not flash it

one more question about this: what is the right order?

thanks for your help!

right now we do this (our wrappers around the particle cli code, but you get the gist):


let tests = p1adaptor()
  .then((adaptor) => {
    p1port = adaptor
    return (identifyp1(p1port))
  }).then(() => {
    return (osversionp1(p1port))
  }).then(() => {
    return (dfumode(p1port))
  }).then(()=> {
    return(flashdfup1(config.systempart1))
  }).then(()=> {
    return(flashdfup1(config.systempart2))
  }).then(()=> {
    return(flashdfup1(config.application))
  })

When flashing by USB the order is not critical. Normally it would happen after system part 2, but it can sometimes be easier to do it before system part 1, as the device may already be in listening mode.

hi rickkas7: works! thanks, frank

ps: i am willing to share our factory programming code if you or anyone thinks this is helpful. it took us a while to get this all up and running

@boddeke, I believe that would be very useful for the community!

so this all worked fine for the first batch which had device is 0.5.3 from the particle factory… second batch had 0.5.5 and it fails. unfortunately we did not figure this out as flashing when fine (at least, it reported successful)

it seems both parts 2.0.1 are flashed fine as well as the application only the bootloader is not correct. see the FAIL below

anyone any idea why this is different when starting with 0.5.5 vs 0.5.3?

MackMyra:restore-to-0.5.5 frank$ particle serial inspect
Platform: 8 - P1
Modules
  Bootloader module #0 - version 7, main location, 16384 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
  System module #1 - version 2011, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 207
  System module #2 - version 2011, main location, 262144 bytes max size
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: FAIL
      System module #1 - version 2011
      Bootloader module #0 - version 1003
  User module #1 - version 6, main location, 131072 bytes max size
    UUID: B97CE4407450DA2EAF3616EE61CC1973231EC5855003F9C6F425CAF3C008EDBA
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 2011
  User module #1 - version 6, factory location, 131072 bytes max size
    UUID: 012219464FF44050FFF7C0FFA842F4D10023022219464FF44050FFF7B7FF031B
    Integrity: PASS
    Address Range: PASS
    Platform: PASS
    Dependencies: PASS
      System module #2 - version 1406

It should not matter between 0.5.3 and 0.5.5. Both should fail.

The reason is that the bootloader needs to be upgraded, and in Device OS 0.7.0 and later, there isn’t enough room to include it in the system parts. For a device that has Wi-Fi configured, the bootloader will be loaded from the cloud OTA.

Basically, on all initial firmware flash by USB or JTAG you should flash Device OS (parts 1 and 2), user firmware, and the bootloader to prevent going into listening mode to get the bootloader OTA.

hi rickkas7

sorry, the code above is incomplete i see. we do flash all… (see below, first the bootloader, then system part 1, system part 2 and our application). starting with a device with 0.5.3 is works and with 0.5.5 it seems to work but the bootloader will not be correct…

any idea?

thanks
frank

let tests = p1adaptor()
  .then((adaptor) => {
    p1port = adaptor
    return (identifyp1(p1port))
  }).then(() => {
    return (osversionp1(p1port))
  }).then(()=> {
    return(flashserialp1(p1port, config.bootloader))
  }).then(() => {
    return (dfumode(p1port))
  }).then(()=> {
    return(flashdfup1(config.systempart1))
  }).then(()=> {
    return(flashdfup1(config.systempart2))
  }).then(()=> {
    return(flashdfup1(config.application))
  })

Can you stop the process after flashing the bootloader then do a particle serial inspect? Presumably either it’s uploading the wrong one, or the change didn’t stick and that might help determine which.

Usually when flashing over USB we do the bootloader last, not first, that might also make a difference. (When doing in using JTAG, it doesn’t matter.)

ok, will stop and inspect

as for order of things: first system part 1&2, then bootloader and then application? or bootloader after application?

any idee why 0.5.5 is different compared to 0.5.3? it is very reproducible: restore to 0.5.3 → all is fine after flashing, restore to 0.5.5 and it always has wrong bootloader.

thanks
frank

Application before bootloader. I can’t think of a reason why 0.5.3 would behave differently than 0.5.5. There were very few changes, and they have the same bootloader version.

we now flash deviceos and application first and bootloader as last step. before flashing the boatloader we have to put a long delay (4 seconds). first i tried usb reset but that also fails right after flashing the application. now i wait and do not need to do the usb reset.

is there a way to do this faster?

one difference for sure is that 0.5.5 advertises the (hex) id in caps in listen mode (where 0.5.3 did lower case)

The fastest way is to flash a device is using SWD/JTAG and a hex file. You can flash a single hex file that contains the bootloader, system parts, and user part in one shot. The hex file generator can simplify the creation of the file.

For the P1 you can use a ST-LINK/v2 or Particle debugger, but the absolute fastest is the Segger J-Link. It’s expensive, but if you’re programming many devices it’s by far the fastest.

(It’s even faster on Gen 3 devices because the configuration is not stored on the main MCU flash so you can bulk erase the entire chip instead of doing a sector erase. That’s not possible on Gen 2 STM32 devices so the JTAG programmer needs to erase sectors individually.)

okay, thanks, we had trouble getting jtag to work on the p1 that is why we switched to usb. we will check again for the next batch