Unreliable Ethernet Connectivity with P2

[AAw]

Hello,

We are using the P2 with the Wiznet W5500 Ethernet controller via SPI for internet connectivity. Using particle's reference Ethernet application we are able to connect to ethernet and publish messages. However, after about 1-2 minutes the ethernet always disconnects because of COAP message transmission failure and the P2 is unreachable. The P2 then never reconnects to the particle cloud unless we power cycle the device. Any ideas why these messages always fail after 1-2 minutes?

Here is the code:

// SAMPLE APPLICATION
#include "Particle.h"
SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(SEMI_AUTOMATIC);
SerialLogHandler logHandler(115200, LOG_LEVEL_ALL);

uint8_t resetRetry = 0;

void setup() {
#if HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY
    // To force Ethernet only, clear Wi-Fi credentials
    Log.info("Clear Wi-Fi credentionals...");
    WiFi.clearCredentials();
#endif // HAL_PLATFORM_WIFI && !HAL_PLATFORM_WIFI_SCAN_ONLY

    // Disable Listening Mode if not required
    if (System.featureEnabled(FEATURE_DISABLE_LISTENING_MODE)) {
        Log.info("FEATURE_DISABLE_LISTENING_MODE enabled");
    } else {
        Log.info("Disabling Listening Mode...");
        System.enableFeature(FEATURE_DISABLE_LISTENING_MODE);
    }

    Log.info("Checking if Ethernet is on...");
    if (Ethernet.isOn()) {
        Log.info("Ethernet is on");
        uint8_t macAddrBuf[8] = {};
        uint8_t* macAddr = Ethernet.macAddress(macAddrBuf);
        if (macAddr != nullptr) {
            Log.info("Ethernet MAC: %02x %02x %02x %02x %02x %02x",
                    macAddr[0], macAddr[1], macAddr[2], macAddr[3], macAddr[4], macAddr[5]);
        }
        Ethernet.connect();
        waitFor(Ethernet.ready, 30000);
        Log.info("Ethernet.ready: %d", Ethernet.ready());
        resetRetry = 0;
    } else if (++resetRetry <= 3) {
        Log.info("Ethernet is off or not detected, attmpting to remap pins: %d/3", resetRetry);

        if_wiznet_pin_remap remap = {};
        remap.base.type = IF_WIZNET_DRIVER_SPECIFIC_PIN_REMAP;

        remap.cs_pin = D5;
        remap.reset_pin = D3;
        remap.int_pin = D4;

        auto ret = if_request(nullptr, IF_REQ_DRIVER_SPECIFIC, &remap, sizeof(remap), nullptr);
        if (ret != SYSTEM_ERROR_NONE) {
            Log.error("Ethernet GPIO config error: %d", ret);
        } else {
            if (System.featureEnabled(FEATURE_ETHERNET_DETECTION)) {
                Log.info("FEATURE_ETHERNET_DETECTION enabled");
            } else {
                Log.info("Enabling Ethernet...");
                System.enableFeature(FEATURE_ETHERNET_DETECTION);
            }
            delay(500);
            System.reset();
        }
    }

    Particle.connect();
}

void loop() {
    static system_tick_t lastPublish = millis();
    static int count = 0;
    static bool reconnect = false;

    if (Particle.connected()) {
        reconnect = false;
        if (millis() - lastPublish >= 10000UL) {
            Particle.publish("mytest", String(++count), PRIVATE, WITH_ACK);
            lastPublish = millis();
        }
    }

    // Detect a network dropout and reconnect quickly
    if (!reconnect && !Ethernet.ready()) {
        Log.info("Particle disconnect...");
        Particle.disconnect();
        waitFor(Particle.disconnected, 5000);
        Log.info("Network disconnect...");
        Network.disconnect();
        Particle.connect();
        reconnect = true;
    }
}

These are the log messages:

003482418 [comm.coap] TRACE: Retransmitting CoAP message; ID: 218; attempt 3 of 3
0003482587 [comm.coap] TRACE: Sending CoAP message
0003482751 [comm.coap] TRACE: CON 0.00  size=4 token= id=218
0003483142 [comm.coap] TRACE: Retransmitting CoAP message; ID: 221; attempt 2 of 3
0003483341 [comm.coap] TRACE: Sending CoAP message
0003483447 [comm.coap] TRACE: CON 0.00  size=4 token= id=221
0003499456 [comm.protocol] ERROR: Event loop error 1
0003499721 [system] WARN: Communication loop error, closing cloud socket
0003499889 [system] INFO: Cloud: disconnecting
0003500016 [system] INFO: Cloud: disconnected
0003500121 [system] INFO: Cloud: connecting

Thanks,
Aaron

[rickkas7]

Part of it sounds like the router on the Ethernet LAN is removing the temporary UDP port forwarding mapping back to the device. The P2 is sending packets ("Sending CoAP message") but never receives messages back from the cloud.

However, the mysterious part is that the code publishes every 10 seconds, which should be enough to keep the port forwarding alive regardless of the keep alive. The keep alive is 25 seconds on the P2 and Argon.

And even if the port forwarding was removed, once the cloud disconnects and reconnects, the router should establish a new temporary port forwarding, which I'd expect to work for a few minutes again.

It may be necessary to get an Ethernet packet capture such as by using Ethereal, to see what's actually happening on the wire.