First off I just wanted to give a massive thank you to the community, the elites of the forum, and those working at the Spark company especially those active on the forum! For the past couple months I’ve been posting frequently and receiving great support and direction, so I wanted to say thank you for all of that
Onto a question ! Really short, I have two cores accessing one SD card, can one Core have one file open and read from it and the other Core have a different file open and write to it, all at the same time? Something tells me no … because I remember reading it here ha
This will be much easier when the Photon comes out, which will eliminate user code blocking during WiFi Reconnect.
But for now I’m using two Cores, one for remote logging, which saves the data onto two txt file while WiFi is up and running and one third text file while WiFi is down. First text file for data storing, continuously, the first text file is for when WiFi is up and running (I know I could parse the first, but this made my life tons easier) and the third file is for when WiFi is down.
The second Core will read the appropriate file (WiFi up , or Wifi (was) down txt file) then send the data off, and erase the contents of the file so things don’t get too big.
Biggest concern is if two files are open at the same time, will things crash? or is it even possible.
@UST, if I understand correctly, you have one non-connected Core doing data logging so you don’t get any code blocking when the cloud connection drops. You have have another Core which can suffer the block but when it reconnects, you want it to send the logged data that was not sent during the outage. Correct?
Sharing an SD between two master SPI buses is not only tricky but really asking for trouble. The SPI protocol was not designed to support multi-master configurations (two Core, one SD).
It seems to me that you should have a simple Serial or I2C connection between the data logging Core and the Data Sending Core with simple commands to support your requirements. The Data Sending core would query the Data Logging Core at regular intervals to see if data is available. The Data Logging Core would respond with a stream of bytes that you would then send out as required. The Data Logging Core could keep a file pointer to the first byte unsent and update it whenever it sends data to the Sending Core. You could even have a keep-alive command so that Data Logging Core can know when the Sending Core is waiting on a reconnect. Hopefully, you get the idea.
I sort of do, I already have a “heart beat” signal, showing the logging core when it needs to save into two different files.
If things run smoothly it might work, but when the “wifidown” file gets two large, and both are opening files at once it might get sticky as you and @AndyW stated! So it seems I’ll need to look into and research Serial or I2C connection to send over my data as intended, cause at the moment I’m not sure how I would tackle this!
But thank you for the direction, I’ll post back here if I find a solution!
@UST, here is a little Arduino SerialCommand library that allows you to really easily setup a serial command structure for the Logging Core. I ported this already if that interests you.
What if I did it so that when WiFi was on, only one core talked to the SD card, and when WiFi is off the other talks. But they’ll never talk at the same time? Is this also a fix, before I go pouring time into the alternate avenue
@UST, the SPI hardware in the Core is supposed to tri-state the SCK/MISO/MOSI lines when a byte has been transferred. However, the chip select line will not as it must be held high or low to control the slave device. So my hunch is you would need to add a pull-up to the SD CS line so that it never “floats” and the Core that is NOT using the SD would have to set the pin that it uses for CS to an INPUT so it is tri-stated. The Core that IS using the SD would set its CS pin as an output so it can interact with the SD. You would have to make sure you don’t have a collision on the CS pin by using some delays for example. The CS on the slave (SD) ensures that ITS pins are tri-stated when not selected.
As I’m starting to realize just by trying to do simple work around.
Again, I’ll look into another avenue that @peekay123 suggested and post here when I find an answer.
EDIT: Going to be looking into I2C interface as @peekay123 described, the serial commands are neat but I want the communication between the two cores to be happening automatically, with no input commands at Serial required if possible
just to add more momentum to the current direction of “no don’t do that!” - FAT is not a concurrent access filesystem - even without the shared SPI bus issues, the file system would become corrupted very quickly. However, if you can interleave accesses to the filesystem, then it’s ok.
But really using 2 cores is a much harder solution that it needs to be. You can do this with one core. In fact you probably don’t need an SD card, but a large circular buffer. You write to one end of the buffer (wifi on or off) and read from the other end - both of these operations can happen in the main loop. My flashee library supports a circular buffer in the 1.5Mb external flash - depending upon your anticipated wifi downtimes and how much data is buffered, that may work for you.
If I can do my operation, essentially what you described, with one core that is my ideal situation ! Does your flashee library somehow handle this, because if it does then with your circular buffer and library I eliminate a lot of hardware at the current moment
Otherwise it’s as @peekay123 said, I’m only using the second Core to avoid the user code blocking “feature” the Core current had But still appreciate the suggestions none the less !
User code surely doesn’t block for that long? You can take control over when the cloud connects, something like this: (In semi auto mode)
if (!Spark.connected()) {
Spark.disconnect(); // dont reconnect if not connected
// attempt one connect every 60 seconds
if ((millis()-lastConnect)>60000) {
Spark.connect(); // ok we can connect now. This will block waiting for wifi to connect
lastConnect = millis();
while (!WiFi.ready() && (millis()-lastConnect)<5000) { // wait max 5 s for wifi to connect
}
if (WiFi.ready()) {
SPARK_WLAN_Loop();
}
if (!Spark.connected()) // if we didn't connect, then don't retry
Spark.disconnect();
}
}
else {
lastConnect = millis();
}
This is just off the top of my head, so it’s untested. The basic idea is to control exactly when the cloud reconnects when disconnected. User code is only blocked for a few seconds while reconnecting to the cloud. At least in theory! If it doesn’t work for you, let me know and I’ll help troubleshoot/refine the code.
@mdma, if problems exist with the cloud or wifi, it could block for longer. I had originally proposed using timer interrupts to do sampling in an ISR/background so that part would not stop when the cloud drops. That’s the theory at least. I am not sure anyone has tested this yet. Maybe I should!!!
I’d like to know where it blocks, so we can work on getting this fixed. Lately, I’ve read through all the code relating to the wlan and cloud connectivity and feel confident that it should at least be possible to allow the user to decide when wifi/cloud reconnection and other blocking operations happen. Sure reconnection blocks for a few seconds, but not endlessly - at least it shouldn’t!
@UST - How long can your code be blocked for? A few seconds? if your app can’t tolerate a block of even a few seconds, then let’s discuss what kind of data you are logging. Perhaps this can be sampled in an interrupt and stored in a buffer while the cloud is connecting, as @peekay123 suggested.
A comment on the “user code blocks” issue. An unofficial fix for this issue has been provided by @mummblepins. The issue and fix are discussed is in this thread:
This would work great, and my understanding is that you are correct. If you control when and how the core reconnects to WiFi (or the cloud) then you can control how long user code is blocked, so Semi-Auto wouldn’t do “auto-reconnect” and thus you can control when it reconnects.
This actually might work. What I could do also is if WiFi drops, and my timer (of 1 minute) is triggered, if the Core then can’t send the data off (via WiFi) it can save it on the SD and the file can just grow larger and larger, until it does connect with WiFi then it can send all the data that was on the .txt file. Then just wipe the contents of the file (?*) and rinse and repeat. Also that way in-between the timer trigger I can attempt a WiFi connect for lets say, 10-15 sec, then if it doesn’t connect break the loop and go back to user code, am I on the right track here?
Also just saw that you posted while I was typing my lengthy ass reply, I am storing counting data on an interrupt, every time interrupt is triggered count, which is why I don’t want it to get blocked off at all; however, if blocking for a couple seconds means eliminating a second spark Core, I can adjust until the Photon comes out.
?* - to eliminate the contents of a txt file on a SD, do I just open the txt file and write to it, then I write nothing and then close the file? I saw this on another forum on how to delete contents, without deleting the file, and was wondering if it’s correct.
! Really short, I have two cores accessing one SD card, can one Core have one file open and read from it and the other Core have a different file open and write to it, all at the same time? Something tells me no … because I remember reading it here ha
