I assume it might be advisable to disable updates while writing to the SD card?
I didn't know a firmware update could occur while a user program is running. Maybe someone can answer this question.
The only danger of SD corruption or data loss is if a file that is open for write is not properly closed.
You can reduce the chance of data loss by calling sync() or flush() like this:
file.sync();
flush() is just another name for sync().
This is equivalent to closing and immediately reopening the file. No data will be lost if your program ends after this call but before you call write() or print().
1 Like
Yes, OTA flashing can occure at any time while the device is connected to the cloud.
Which is usually no problem if the user and the one flashing new code are the same person (mental diseases negelcted here ;-)).
But since this might become a problem with products sold to customers, Particle is working on solutions like protection against OTA flashing or an event to subscribe to which signals a pending flash/update.
Have a look at this
https://docs.particle.io/reference/firmware/photon/#ota-updates
So @tjp, yes it might be advisable, but this should be the final developers responsibility and not necessary up to the library contributor.
The final dev knows best which way of dealing with the situation is the best for his product.
2 Likes
Are you telling me that a photon I own and have setup can do an OTA firmware update spontaneously without my action while a program is executing?
Would this program suddenly stop printing a series of numbers for a firmware update?
void setup() {
Serial.begin(9600);
}
int n = 0;
void loop() {
Serial.println(n++);
delay(500);
}Yes, absolutely! I did not mean to imply that I was expecting SdFat to do that. I was simply asking if that is something I should do 
Thanks, @ScruffR!
1 Like
If youāre the end developer, no. Youād have to trigger the OTA update yourself. But, you wouldnāt want to do that in the middle of a write and risk rebooting before close is called. Thatās why I asking about using the APIs that @ScruffR linked to disable OTA updates during writes.
Disabling updates only during writes wonāt help. SD cards are written in 512 byte blocks and there are cache blocks in any SD library.
In addition to the cache blocks, directory info is also cached and not updated on the SD until a file is closed or sync/flush is called. I like the name sync better than flush since these calls do more than flush buffers. They read directory info, update it, and write it to the SD. You must read an entire directory block, update it, and write the block back to the SD.
SdFat on Particle uses two cache blocks, one for file data and one for system structures. These are not synced after every write since the overhead would be huge. Updating file directories after each write would add even more overhead.
You can call sync/flush for every open file after any write and enable updates until the next write. This is OK if it does not cause too large of a performance hit.
Understood. My SD card shares the SPI bus with my OLED so I keep my writes atomic (I open, write then close every time). My writes only happen in response to a cloud function call, so the file does not need to stay open.
You can leave files open when sharing the SPI bus. There can be a very large overhead in opening files and seeking to EOF. The FAT filesystem requires a sequential search of directories so an open with 1,000 file in a directory can take a long time. Seeking to the end of a very large file also takes time since the FAT must be read to get the cluster chain.
I have had users that create many small files in a directory so the open close sequence finally took a long time.
1 Like
Iām glad weāre having this conversation then. Perhaps, I will leave the file open then. However, the way Iām currently using it is with O_TRUNC so EOF is pretty easy to find. 
If I leave the file open, I could just call truncate() manually, correct? Iām writing a list of alarms (described by cron expressions), so I just write the whole list every time since the list could change drastically (especially in development). Is this what youād recommend, @whg?
The fastest way would be to rewind the file, write the new version, truncate to the current position, and finally call sync().
A good and simpler way is to truncate the file to zero length, write the new version, and call sync().
The first way saves freeing and allocating a cluster, usually 32 KiB on a properly formatted SD.
1 Like
Like this:
// ...
file.rewind();
file.print(alarms); // alarms is Printable
file.truncate(file.curPosition());
file.sync();
// ...
Well, after writing it out next to what you wrote, I'm no longer questioning it. lol
Thanks, @whg!
@whg Iām toiling away trying to dual wield SdFat and the Particle Webduino library:
I have a web server listing files, and Iām able to click them to make BMP graphics appear on an LED display controlled by the Photon. Itās great, but I also want to:
- Read HTML file from SD
- Write files to SD
All examples of this Iāve found on the web use very old versions of SdFat, and more often than not appear to be less than optimized (e.g. reading/writing one byte at a time). I realize an SdFat-powered web server is outside the scope of your example files, due to dependency on another library, but if you find this interesting I would be grateful for a basic sketch that shows an example of each of these techniques using your latest code.
The multiple byte API for read and write is used in the bench example. See lines 152 and 187.
You should look at the Doxygen html documentation for the complete SdFat API.
SdFat has several file classes. The most basic class is FatFile. This class has no Arduino style features. The āFileā class tries to be compatible with the Arduino SD.h library. The Arduino SD.h library is a wrapper for a version of SdFat that I wrote in 2009. Sadly the result of history is more file classes than I would like.
I would be happy to help if you have specific questions about your needs in Webduino.
Has anyone had any luck using this library with a 32GB uSD card? Iāve been able to get things working very well with up to 16GB but 32GB cards just throw an error 0x2,0xDF (presumably card not supported) fault.
The cards Iām trying this with are Samsung 32GB class 10, FAT32 formatted.
SdFat supports any size card.
You card fails in initialization before the size is determined. The response code 0xDF to the CMD8 initialization request is invalid.
What is the hardware setup? Some cards are more inclined to SPI failure with marginal SPI signals. Try a slower SPI speed to check for an SPI problem.
It is also possible to enable software CRC to check for SPI errors.
32GB cards are commonly used with SdFat. I have tested with 128GB SDXC cards formatted FAT32.
I have used the core library for 500GB USB hard drives in the UsbFat version of the library.
2 Likes
Interestingāglad to hear others have been able to use 32GB cards at least though Iām still confused as to why Iām unable to see similar results.
Iām using a custom P1 module but the SD is on the primary SPI lines (SPI_CONFIGURATION 0). The only thing that might be slightly different is we use the āDACā pin for chip select instead of the standard chip select pin, but I havenāt come across any problems at all with lower capacity cards so Iām doubtful itās related to a chip select problem. In the past Iāve seen the same behavior with a standard Photon board and an off-the-shelf SD card shield (which is why I suspected this may have been more systemic) but I havenāt tried the normal Photon with this newer library.
Currently Iām running SPI_HALF_SPEED, I can try 1/4 speed though to see if there are any improvements or changes.
Any other suggestions on what to try?
Is there anything special about how the card has to be formatted perhaps?
The failure is simple, it has nothing to do with the amount of flash in the card or the card format.
It almost always is an SPI noise problem or flaky chip select.
Early in initialization I use SD CMD8 to make sure the card is a version 2 card and the SPI bus is clean. CMD8 has a field for a one byte check pattern that is traditionally filled with 0XAA. If that is not returned, you know there is a problem.
This loop never returns the check pattern. The last value in your case was 0XDF.
while (1) {
if (cardCommand(CMD8, 0x1AA) == (R1_ILLEGAL_COMMAND | R1_IDLE_STATE)) {
type(SD_CARD_TYPE_SD1);
break;
}
for (uint8_t i = 0; i < 4; i++) {
m_status = spiReceive();
}
if (m_status == 0XAA) {
type(SD_CARD_TYPE_SD2);
break;
}
if (((unsigned)millis() - t0) > SD_INIT_TIMEOUT) {
error(SD_CARD_ERROR_CMD8);
goto fail;
}
}
I have been using simple microSD sockets from ebay that only have 10k pull-ups and caps on 3v3. I have had no failures.
Give another pin a try for this. I've filed an issue in connection with the DAC pins (introduced with 0.4.9), where some actions on a different pin have cross effects on the digitalWrite() behaviour of the DAC pins.
This might not be the cause for your problem, but worth considering.
I agree with your assessments though Iām still not quite sure whatās going on here.
I tried running the SPI line at SPI_SIXTEENTH_SPEED (32 divider) which is the slowest I could configure it to operate at and Iām getting the same results (SD errorCode: 0X2,0XDF). Again this only happens with 32GB cards (I havenāt tried larger, though I have pretty extensively tried cards smaller than 32GB, at least 4 different types and they all worked flawlessly). The signals show clean on the oscilliscope though I will check this again to verify.
We do have pull-ups and caps on the signal lines.
The P1 module is running V0.4.9 (though I did try it also with V0.4.7), system threads are enabled (I did try it with them disabled as well though).
To recap the exact card Iām using is a Samsung 32GB EVO UHS-I Class 10 micro-SD cardāmy understanding is it should be fully compatible with other micro-SD cards. Iām able to successfully access (read and write reliably) the SD card using a different micro-SD board (non-particle though, so not this library and not this architecture) so the card itself doesnāt seem to be the problem.
@ScruffR Unfortunately I canāt easily try another chip select as the board has already been fabricatedāthis could be switched when the board goes to production if it really is the chip select pin though.
I guess my next step will be to try this code again on a standard Photon board with the off-the-shelf SD shield at a lower SPI bit-rate (have tried this in the past but am willing to give it another go) unless anyone has any other ideas?