SPI ethernet + peripheral deadlocks on P2

We are developing a POE device that has a SPI peripheral connected to the shared SPI bus as the ethernet driver. Ethernet connects fine to the particle cloud and I am able to publish/subscribe as expected. However, as soon as I try to write a byte to my SPI peripheral (using SPI.beginTransaction) my P2 deadlocks. The deadlock is due to the while(isBusy); never being cleared in the SPI drivers. Any suggestions on how to resolve this issue?

We are using device os 5.3.2

We are thinking of moving the SPI peripheral to SPI1 instead so that it is not shared with ethernet. However, this seems strange to me that we can't have 2 devices (ethernet + 1) on the same SPI bus...

Using SPI.beginTransaction() and SPI.endTransaction() is definitely necessary in your code whenever you access SPI along with Ethernet. This is also required by any libraries that you use that access SPI.

The most common cause for deadlock is calling SPI.beginTransaction() from any guarded blocks, such as SINGLE_THREADED_BLOCK, ATOMIC_BLOCK, calling it with noInterrupts(), or calling it from a worker thread with a higher priority than the system thread. If anything those conditions is true and the system thread is already using SPI (for Ethernet), your thread will deadlock because SPI is in use by the system thread, but any of those reasons will prevent the system thread from being swapped in, so everything comes to a halt.

So it is possible to use SPI with ethernet and another peripheral on SPI?

Here is an example I'm using for testing that deadlock issue. Sometimes it deadlocks after one loop sometimes it makes it through two loops:

// SAMPLE APPLICATION
#include "Particle.h"

SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(SEMI_AUTOMATIC);
// Serial1LogHandler log1Handler(115200, LOG_LEVEL_ALL);
SerialLogHandler log1Handler(LOG_LEVEL_TRACE, {{"app", LOG_LEVEL_TRACE}});

retained uint8_t resetRetry = 0;
    static uint8_t tx_buffer[5] = {0x00, 0x01, 0x02, 0x03, 0x04};
    static uint8_t rx_buffer[5];

void setup() {  
    Serial.begin(9600);
    waitFor(Serial.isConnected, 15000);
    Serial.println("STARTING");
    // os_mutex_create(&TestMutex);
    WiFi.clearCredentials();
    WiFi.off();
#if HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY
    // To force Ethernet only, clear Wi-Fi credentials
    Log.info("Clear Wi-Fi credentionals...");
    WiFi.clearCredentials();
#endif // HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY

    // Disable Listening Mode if not required
    if (System.featureEnabled(FEATURE_DISABLE_LISTENING_MODE)) {
        Log.info("FEATURE_DISABLE_LISTENING_MODE enabled");
    } else {
        Log.info("Disabling Listening Mode...");
        System.enableFeature(FEATURE_DISABLE_LISTENING_MODE);
    }

    Log.info("Checking if Ethernet is on...");
    if (Ethernet.isOn()) {
        Log.info("Ethernet is on");
        uint8_t macAddrBuf[8] = {};
        uint8_t* macAddr = Ethernet.macAddress(macAddrBuf);
        if (macAddr != nullptr) {
            Log.info("Ethernet MAC: %02x %02x %02x %02x %02x %02x",
                    macAddr[0], macAddr[1], macAddr[2], macAddr[3], macAddr[4], macAddr[5]);
        }
        Ethernet.connect();
        waitFor(Ethernet.ready, 30000);
        Log.info("Ethernet.ready: %d", Ethernet.ready());
        resetRetry = 0;
    } else if (++resetRetry <= 3) {
        Log.info("Ethernet is off or not detected, attmpting to remap pins: %d/3", resetRetry);

        if_wiznet_pin_remap remap = {};
        remap.base.type = IF_WIZNET_DRIVER_SPECIFIC_PIN_REMAP;

        remap.cs_pin = D5;
        remap.reset_pin = D3;
        remap.int_pin = D4;

        auto ret = if_request(nullptr, IF_REQ_DRIVER_SPECIFIC, &remap, sizeof(remap), nullptr);
        if (ret != SYSTEM_ERROR_NONE) {
            Log.error("Ethernet GPIO config error: %d", ret);
        } else {
            if (System.featureEnabled(FEATURE_ETHERNET_DETECTION)) {
                Log.info("FEATURE_ETHERNET_DETECTION enabled");
            } else {
                Log.info("Enabling Ethernet...");
                System.enableFeature(FEATURE_ETHERNET_DETECTION);
            }
            delay(500);
            System.reset();
        }
    }

    Particle.connect();

    waitFor(Particle.connected, 100000);
}

void loop() {
    static system_tick_t lastPublish = millis();
    static int count = 0;
    static bool reconnect = false;

    if (Particle.connected()) {
        if (millis() - lastPublish >= 10000UL) {
            count++;
            Particle.publish("mytest", String(count), PRIVATE, WITH_ACK);
            lastPublish = millis();
            Log.info("Sent mytest %d", count);
            SPI.beginTransaction();
            SPI.transfer(0x00);
            SPI.transfer(0x01);
            SPI.transfer(0x02);
            SPI.transfer(0x03);
            SPI.endTransaction();
        }
    }

}

Welcome to the Community!

Yes it is possible, you just need to use a different CS signal for the peripheral device. using the same CS signal will indeed create bus contention as both the ethernet and the peripheral chips will be thinking the Host, P2, is communicating with that device. Another alternative is you can just use SPI1 for the peripheral device, up to you. Cheers!

Yes, we are using a separate CS for the peripheral. We are trying to avoid a board revision and moving peripheral to SPI1. For the above test application (which deadlocks) I am monitoring SPI bus with logic analyzer (no peripheral).

Hey, happy to take a look at those logic analyzer captures if you want to share them.

1 Like

Yes, it should work. I see two issues with your code:

  • You should pass SPISettings to your SPI.beginTransaction() because the system may reset the SPI settings to a different value.
  • You're not managing the CS line for your other peripheral, so it won't work. You should set it to OUTPUT and HIGH in setup. Just after SPI.beginTransction() set it LOW. And just before SPI.endTransaction() set it HIGH.
1 Like

I updated the code to pass SPISettings and controlling of the CS line, but I know this is not the issue because sometimes it never makes it past the SPI.beginTransaction call without deadlocking. There is something happening on the ethernet driver that I can't quite figure out....

Here is updated code:

// SAMPLE APPLICATION
#include "Particle.h"

SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(SEMI_AUTOMATIC);
// Serial1LogHandler log1Handler(115200, LOG_LEVEL_ALL);
SerialLogHandler log1Handler(LOG_LEVEL_TRACE, {{"app", LOG_LEVEL_TRACE}});

retained uint8_t resetRetry = 0;
    static uint8_t tx_buffer[5] = {0x00, 0x01, 0x02, 0x03, 0x04};
    static uint8_t rx_buffer[5];

void setup() {  
    Serial.begin(9600);
    waitFor(Serial.isConnected, 15000);
    Serial.println("STARTING");
    pinMode(D18, OUTPUT); 
    digitalWrite(D18, HIGH);

    WiFi.clearCredentials();
    WiFi.off();
#if HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY
    // To force Ethernet only, clear Wi-Fi credentials
    Log.info("Clear Wi-Fi credentionals...");
    WiFi.clearCredentials();
#endif // HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY

    // Disable Listening Mode if not required
    if (System.featureEnabled(FEATURE_DISABLE_LISTENING_MODE)) {
        Log.info("FEATURE_DISABLE_LISTENING_MODE enabled");
    } else {
        Log.info("Disabling Listening Mode...");
        System.enableFeature(FEATURE_DISABLE_LISTENING_MODE);
    }

    Log.info("Checking if Ethernet is on...");
    if (Ethernet.isOn()) {
        Log.info("Ethernet is on");
        uint8_t macAddrBuf[8] = {};
        uint8_t* macAddr = Ethernet.macAddress(macAddrBuf);
        if (macAddr != nullptr) {
            Log.info("Ethernet MAC: %02x %02x %02x %02x %02x %02x",
                    macAddr[0], macAddr[1], macAddr[2], macAddr[3], macAddr[4], macAddr[5]);
        }
        Ethernet.connect();
        waitFor(Ethernet.ready, 30000);
        Log.info("Ethernet.ready: %d", Ethernet.ready());
        resetRetry = 0;
    } else if (++resetRetry <= 3) {
        Log.info("Ethernet is off or not detected, attmpting to remap pins: %d/3", resetRetry);

        if_wiznet_pin_remap remap = {};
        remap.base.type = IF_WIZNET_DRIVER_SPECIFIC_PIN_REMAP;

        remap.cs_pin = D5;
        remap.reset_pin = D3;
        remap.int_pin = D4;

        auto ret = if_request(nullptr, IF_REQ_DRIVER_SPECIFIC, &remap, sizeof(remap), nullptr);
        if (ret != SYSTEM_ERROR_NONE) {
            Log.error("Ethernet GPIO config error: %d", ret);
        } else {
            if (System.featureEnabled(FEATURE_ETHERNET_DETECTION)) {
                Log.info("FEATURE_ETHERNET_DETECTION enabled");
            } else {
                Log.info("Enabling Ethernet...");
                System.enableFeature(FEATURE_ETHERNET_DETECTION);
            }
            delay(500);
            System.reset();
        }
    }

    Particle.connect();

    waitFor(Particle.connected, 100000);
}

void loop() {
    static system_tick_t lastPublish = millis();
    static int count = 0;
    static bool reconnect = false;

    if (Particle.connected()) {
        if (millis() - lastPublish >= 10000UL) {
            count++;
            Particle.publish("mytest", String(count), PRIVATE, WITH_ACK);
            lastPublish = millis();
            Log.info("Sent mytest %d", count);
            SPI.beginTransaction(SPISettings(4*MHZ, MSBFIRST, SPI_MODE0));
            digitalWrite(D18, LOW);
            SPI.transfer(0x00);
            SPI.transfer(0x01);
            SPI.transfer(0x02);
            SPI.transfer(0x03);
            digitalWrite(D18, HIGH);
            SPI.endTransaction();
        }
    }

}

Also, @erik.fasnacht here is the logic analyzer capture:

When the deadlock happens MOSI stays high and CLK stay high and MISO stays low. The deadlock happens before I set my CS (D18) low

Sometimes it makes it through 1-3 loops before deadlocking....

Here is the log trace messages (before deadlocking):

0000105397 [system] INFO: All handshake messages have been processed
0000105527 [comm.coap] TRACE: Received CoAP message
0000105547 [comm.coap] TRACE: CON POST /E/particle/device/updates/pending size=47 token=02 id=35785
0000105589 [comm.coap] TRACE: Sending CoAP message
0000105609 [comm.coap] TRACE: ACK 0.00  size=4 token= id=35785
0000105742 [system] INFO: Cloud connected
0000105810 [comm.coap] TRACE: Sending CoAP message
0000105823 [comm.coap] TRACE: NON POST /E/mytest size=15 token= id=179
0000105853 [app] INFO: Sent mytest 1
0000115853 [comm.coap] TRACE: Sending CoAP message
0000115866 [comm.coap] TRACE: NON POST /E/mytest size=15 token= id=180
0000115896 [app] INFO: Sent mytest 2
0000125906 [comm.coap] TRACE: Sending CoAP message
0000126088 [comm.coap] TRACE: NON POST /E/mytest size=15 token= id=181
0000126354 [app] INFO: Sent mytest 3

@rickkas7 please let me know if you have any other suggestions for me to try.

This is still an issue for us. If there are any other suggestions on things we can do to help troubleshoot that would be much appreciated.

Have you been following this thread? Anyone got wired ethernet working? - #77 by marekparticle

1 Like

We are experiencing a similar issue to this post so wanted to jump in to see if there was any further learnings or troubleshooting paths. (If this should be posted in a new thread please let me know).

We are trying to use a POE featherwing with a P2 device alongside an RFID peripheral that also requires SPI. We are using an Argon-RFID-MFRC522 library but it does seem to work fine.

We have successfully set them up separately (though with the ethernet we were having issues connecting it to cloud but for our use case we can have the wifi on we just wanted the realiability of ethernet/poe for our local network). We have tested them using a shared CS pin as well as moving them to separate SPIs altogether.

However, once we try to use both at the same time the device becomes unresponsive, is unreachable via the ethernet or cloud connection, and the RFID stops registering taps.

I don't understand why this would be an issue if they are both running on separate SPIs and I'm hoping someone can give me some pointers on how to resolve this issue.

Thank you.

@chonigman Welcome to the Particle Community.

Could you be a bit specific about the schematic and device OS you are using. Are you sure that each SPI device has a CS pin defined. It is now possible to specify a different CS pin for ethernet than the standard.

If all is correct with the wiring, a possible reason for the lock up is that the Argon-RFID-MFRC522 library over SPI is not implementing SPI transactions beginTransaction() and endTransaction() to ensure that each device on the SPI bus has the correct mode and speed.

I will doing an integration of TFT and SD on SPI with ethernet using P2 and Device OS 5.6.0 in a few weeks - just need to try and optimise driving a TFT display using the P2 first.

1 Like

Thanks for the response @armor. While this did not end up being the issue, it did help me realize what was going on. Looking more closely at the Argon-RFID-MFRC522 library I linked, they are in fact calling begin/endTransaction whenever they are reading/writing to the peripheral devices registry. However, it looks like they hardcode the builtin SPI class to a member variable called _spiClass and then within the class functions (i.e. PCD_WriteRegister, etc) they are calling the begin/endTransaction methods on the _spiClass.

After digging into this and then how the Featherwing itself was connecting to Photon2 I realized that the Ethernet needed (as far as I can tell) to use the default SPI interface (D15-17) and the SDA/SCL of D0 and D1, but the CS/IRQ/RST pins that are connected by jumpers can be configured to different pins. With this, I just needed to move the RFID to the SPI1 pins, and update the library to use SPI1 instead of SPI and everything started working!

I know it's possible to share the SPI interface but for our use where we will be sending messages back forth to the Photon and constantly polling the RFID chip, it seemed best to separate them completely.

1 Like