Cloud flash OTA should work, but flashing from Workbench by DFU (Flash application only (local) or Flash application and Device OS (local)) will exhibit that behavior if your application is less than 128K bytes.
The problem is that Device OS 3.1.0 added support for 256K user firmware binaries. Older versions of Device OS only supported 128K. This was done by moving the start 128K earlier in flash, so the end of a 256K binary is in the same place as a 128K binary used to be.
When you target 3.1.0 or later, a 256K binary is always created, and it’s flashed to the 256K binary slot. However, two special things occur:
In a normal firmware OTA update, the user firmware binary is flashed first, then the cloud updates any dependencies.
However, when you cross the 256K binary boundary (3.1.0) on Gen 3 devices, the cloud intercepts the user firmware flash and flashes the bootloader first. Otherwise, the pre 3.1.0 bootloader does not know how to load from the new 256K binary slot. After the bootloader is flashed, the user binary is flashed, then the other dependencies are updated.
Problem 1 with Workbench DFU is that this doesn’t happen, and by default Workbench doesn’t flash the bootloader. It’s normally updated OTA, however you can run into a situation if your user binary is greater than 128K where the bootloader on the device does not know how to load a 256K binary, and there is no 128K binary, so the device can’t boot into safe mode to get the bootloader OTA because there is no user firmware binary. The device goes into DFU mode instead.
Problem 2 is when your new user binary is less than 128K. When you DFU a 256K binary, you’re in a situation where you have an old 128K binary in the original slot, and a new 256K binary in the 256K binary slot. Though it initially seems counter-intuitive, the 128K binary, the old one, has precedence. This has to be done, otherwise flashing Tinker with DFU from the CLI or from the mobile apps would not actually replace the 256K binary.
If you Flash application and Device OS by DFU this is how you end up with 4.0.0 and the old binary. Flashing Device OS 4.0.0 from Workbench upgraded Device OS, but did not update the bootloader and SoftDevice. It did flash the new binary, but because there is still the old 128K 2.x binary, that’s the one that’s run because it was not overwritten.
Device Restore USB, Device Restore JTAG, and Hex Generator all invalidate the 128K binary slot when flashing a 256K binary so this does not happen.
It would require some digging to figure out exactly why your OTA update didn’t work, but it should work.