Suggestions for using external flash SPI

Right now our product is using the pixels library so that we can use anti-aliased fonts and compressed bitmaps (at the time all images have to be stored on our P1). To simplify this images would not need to be compressed if they were on a SPI flash. We are using the ILI9341 driver and hardware SPI.

I realized that this Pixels library is not used by many people so support is probably limited. But we found it extremely valuable to utilize the compressed bitmaps and anti-aliased fonts.

This draw functions are below:
Compressed BMP line 751
raw BMP line 673

@peekay123 For now, I only need the SharpMem to work with the SPI flash, but I can see using this extra memory for images to be displayed on the RGB panels.

I can also see this additional memory holding Audio files that can be downloaded and played via some sort of Audio chip.

I can see uploading jpg image files to be used with color LED displays.

With 1 or half a Gig of memory, you could easily upload full video files if you have an LCD Display that can decode video files. I think that would be possible the Photon can download the video file to the SPI flash memory.

I imagine a marketing display using a large LCD screen or RGB panel connected to a Photon where you can track customer interaction live, gather potential customer info like phone numbers or email address, plus have the ability to update the images, or marketing videos at any time. I can see this being used by businesses for more effective marketing.

Being able to easily and remotely update the Photon + up to 1Gig of external flash memory opens up a lot more opportunities than only having the 128K of Photon memory to work with :smile:

To sum it up I see the other file formats being added as time goes on as each one is needed or requested by the people who are needing the support for their particular project like what @wesner0019 is asking for right now.

The reason I choose the larger 1Gig memory card is just to prove it can be used and that it works reliably. I will not ever need that much room for the Sharp display, but I know I can use the extra memory for other things as different needs or ideas come up.

@wesner0019 I originally picked the P1 because I also wanted to store data on its extra 1MB of memory but what pissed me off what that you couldn’t update the 1MB of extra flash on the P1 along with the Firmware update :smile:

Now we have a way to do that. Plus now you can use the more available Photon instead of the unavailable P1 module and add more SPI flash memory for the extra $2 price difference between the P1 and Photon modules.

I’m very happy we now have a solution to update the external SPI flash at any time. I think it opens up many new opportunities like what your already talking about.

@RWB, having the external storage, whether its a flash chip or a microSD, can be very useful. With device sizes now being so large and so cheap, file compression is no longer critical. Having a file system to manage these files is absolutely a must and both SPIFFS and SDFat do a great job for each storage device. Streaming audio from a file is definitely possible though I can’t speak to streaming video due to the bandwidth requirements.

The SDFat library is time tested and robust. More testing of SPIFFS may be necessary but it seems like an excellent library so far. :wink:

@peekay123 I see the SDFat library as a great solution with higher data transfer speeds.

The main revolutionary feature for me is the ability to easily load this external memory by simply providing the web link to the files your wanting to store on the card. Up until now, as far as I know there has been no easy way to go about doing this :smiley: I remember asking for a way to update the extra 1MB of flash on the P1 and finding no way to do it.

So are the main differences from your point of view on the SPIFFS vs. SDFat libraries?

I like how easy it is to change out SD Cards to increase storage capacity + the ability to load the SD card from a laptop which would be beneficial for Electron users.

It sounds like you can run SPIFFS on the SD card also, so I’m struggling between thinking an SD card is better vs. 1Gig flash chip since the SD Card can just be switched out.

I wonder which solution is more efficient power wise?

@RWB, I’ll be testing some Particle.function() code to command the photon to fetch a specified file into flash. Other commands could be used for display sequences, listing files, etc. I’m just bantering ideas about.

The main difference between the SDFat and SPIFFS is that SDFat is optimized for SD and has been around a long time. The author has also created a version for Particle which uses SPI DMA for maximized speed. I don’t know enough about SPIFFS but it seems to depend on the user to optimize operation via callback functions. The biggest difference with an SD card is the mechanical aspect as it is designed to be removable whereas chip flash is not. The environmental conditions (shock, dust, heat, accessibility,etc.) should dictate which is best for the application. As for power, both provide low power operation though again, the application will dictate the best solution. There is no “one shoe fits all” solution here.

@peekay123 Yes was thinking about the SD card + vibration / shock and how that could cause possible issues instantly or over time with contact loss potentially just like we have heard happen with the Electron SIM cards.

For the product I’m building Heat can get high so if that is not good for SD cards, then the chip solution is better.

I guess the best way to do it for my Electron-based solutions would be to just add a Photon to the mix also and update the Flash Chip over Wifi Only when we’re talking about downloading MB’s of data.

I just talked myself out of SD cards again :smile:

@RWB, adding a Photon to an Electron seems odd. Why use an Electron if you have WiFi in the first place!! You could, instead, update the files on the flash using a (read-only) microSD for fast transfer, which would only be used for transfer and removed for normal operation. If you recall, another member had developed a way to upgrade firmware using an SD so you could exploit that as well.

@RWB, I just read this from the SPIFFS readme:

Poor scalability. Spiffs is intended for small memory devices - the normal sizes for SPI flashes. Going beyond ~128Mbyte is probably a bad idea. This is a side effect of the design goal to use as little ram as possible.

So it seems the 1Gbit flash you picked is the max that he recommends. Phew!!

@peekay123 I was just thinking of just using the Wifi as a way to update the firmware & SPI flash knowing it would have to be moved to a location where there was WiFi or a Wifi Hotspot (Like a cellphone with Hotspot) would need to be brought close to the device before updating the firmware & flash memory.

But yes you are right about using the SD card slot for firmware & flash memory updates only. I like that idea a lot actually for Electron situations.

This has me leaning back towards just using the SD card again :blush:

Also this whole time I though the 1Gbit flash memory was actually 1 Gigabyte of memory :smiley: Now I see it’s only 128MB of storage :smile:

Considering all the work you have done already getting the Sharp Memory Display working with the SPIFFS file format how much of that time would have been wasted if we just decide to do the same thing but with the SD card using the SDFat library?

It’s looking like the largest flash chips I can find that are under $10 are 128MB. For that price, the SD card solution starts to become cheaper and more flexible when it comes to offline firmware & Flash storage updates.

The only reason we’re using SPIFFS is because the HTTP file download program that @Suprazz provided us was dependent on the SPIFFS library he was using. If I knew that the SD card could also be used, I would have given more thought to which I thought was better.

Now that I know were limited to 128Mb for Flash Chips vs. GigaBytes of memory for SD cards I’m back leaning towards wanting to use SD cards :smile:

So how much trouble is it to display Bitmap images stored on an SD card on the Sharp Mem display?

How hard to push the downloaded files from the HTTP links to the SD card instead of the Flash chip?

It looks like you already have the RGB Matrix panels working with images stored on the SD card.

Regardless I’ll cover you for the time you have invested.

Let me know what you think. Hope you don’t think I’m crazy :ghost:

Nearly no trouble :stuck_out_tongue_winking_eye:

Incredibly hard and will take over at least 10 minutes to do. :smirk:

I'll rewrite the code to use SD and retest. One learns a lot by doing!

@peekay123 All great news!

The SD Card provides the most flexibility plus it’s already well supported as you have already mentioned.

I’m assuming that maybe the draw function may be a little bit quicker if the data from the SD card transfers quicker, maybe not though if it’s based on Photon’s processing speed.

I just ordered up 2 SD card breakout boards for testing. I have plenty of 16 Gig micro SD cards laying around the house, so that’s a big plus!

I did learn a few things about SPI Flash along with the different NOR and NAND Flash technologies, and power consumption levels between the different sized chips. :smile: So that was not time wasted for me either.

1 Like

That will be easy enough to test when I do the changeover :wink:

1 Like

@mdma do you think the flashee eeprom library will work with the external flash that is on the core but use it on Electron?

I was able to track down the flash chip used on the P1: MX25L8006E, so I should be able to just use this chip in conjunction with the Electron.

http://www.digikey.com/product-detail/en/macronix/MX25L8006EM1I-12G/1092-1117-ND/2744800

@wesner0019, the guy to talk to about flashee eeprom library is @mdma. However, as you know, the SPIFFS library works very well with low cost external flash.

Ok thanks I’ll ping him. The reason I’m looking at this is because we’ve already developed firmware for the P1 but we are developing the same product with the electron and need external flash with the electron. I was looking at maintain the firmware as much as possible between the two. As we are at 128K program limit.

1 Like

@RWB, I got the SDFat version of the code to work. It was easy to switch between SPIFFS and SDFat but when it came to drawing pixels to the display, nothing worked. It turns out that every SDFat SPI operation resets the SPI clock, making it too fast for the SharpMem display. Resetting the SPI clock before drawing a bmp row of pixels fixed the problem. With SDFat, the display time for a full screen is 158ms. Another thing I noticed is that compiled code is smaller by about 7KB with SDFat vs SPIFFS.

I also received your care package. Do you still want me to look at using these larger flash chips with SPIFFS? :wink:

1 Like

@peekay123 Excellent!

I was thinking the SDfat would hog more memory than Spiffs but it’s really nice it’s the other way around.

The SD card solution should serve all my needs now and into the future so I have no need for the SPI flash chips or Spiffs library.

The SD card breakout arrived but I am out of town for a few days so can’t text anything till I’m back.

If I want to also use the lower resolution 4 inch sharp mem display does the buffer need any modifications?

You also have the http downloading to SD card working? How does it handle a download error or WiFi loss during download? Will it show an error or retry automatically if the data transfer does not complete? I’ve never used it before.

The quicker write to screen times are great. What do you use to calculate these times?

THANKS A TRILLION FOR ALL YOUR HELP!

1 Like

@RWB, the lower res display will work fine with the BMP draw code and SharpMem. There is no fancy timeout, wifi loss or other detection/recovery in the http code right now. I would need to spend more time to do add robustness to the code.

If you build any boards, you should consider a “captive” SD socket instead of a push-in/push-out type. The timing is measured from the start to end of the bmpDraw() function using a millis() counter. I will be doing sub-timing to see where the time is being spent though I have a pretty good idea already. :smile:

1 Like

@peekay123 No worries about the Http timeout or incomplete data transfer warnings, I was just curious and have no idea if something like that was standard or not.

I guess I could code something that would verify the file size after download and then trigger the download again if it’s not correct.

I will look into Capacitive SD sockets for sure. I’m sure make military grade or robust versions for rugged use scenarios.

I thought you may be using a scope to read the time between the start and end of the write to LCD. Using the milli timer makes more sense.

1 Like