I’m downloading a large (2MB) file from S3 on my P1 and have a hard fault happening.
To do the download, I’m using this library as a starting point: https://github.com/rsteckler/http-download. However I updated the API to be non-blocking and I call a HttpDownload::process() function from my main loop() to do each call to TCPClient::read(byte *buffer, int length).
The download appears to work as I’d expect for the most part - I get each 128-byte chunk and store it in my custom board’s external SPI flash. After anywhere from 15-30 seconds, I’ll get a hard fault SOS panic. What’s weird is that my application appears to keep running. If the download is far enough along, I’ll even see the download complete successfully (using usb serial logging) before the SOS code finishes flashing and the P1 resets.
Does this behavior indicate that the fault is happening in the system thread? I am using:
SYSTEM_THREAD(ENABLED);
SYSTEM_MODE(MANUAL);
Any idea what might be causing this hard fault? I’m instrumenting the System.freeMemory() during the download too and I never see it dip below 45K.
Is it safe to conclude that since my application code keeps running after the hard fault SOS code starts flashing, that the hard fault is not coming from my application code? Or is it possible that my application code is doing the bad instruction that hard faults, but somehow keeps running due to system thread being enabled?
I was able to fix the issue I was having with the downloads anyways. I was allocating a byte buffer on the stack that was not as big as my download chunk processing routine expected so I was overrunning that buffer. That’s why it would hard fault after some indeterminate time depending on how the stack memory was situated.
I’m still unclear as to why the app code would continue running during the SOS led flashes. Maybe the buffer overrun stomped on some system thread stuff causing the panic, but allowed the user thread to continue running? I know very little about the FreeRTOS architecture…
If you’re doing a bigger project I definitely recommend the offline compiler with Eclipse, it takes a while to set up but is much nicer to work with than the online IDE or Particle Dev.
If you’re still having issues feel free to create a topic about it and link from here.
I’m looking over the HPPT.cpp file in your Github link and I can see you made small changes to the original functions and added more detailed LOG serial print data which is nice.
I’m just curious what the main reasons were for you making changes some of the client calls.
Was the original code just not working since it was written for the Spark Core a few years back?
The biggest change I made from rsteckler’s library was to make it asynchronous. Once the download begins, you need to call http.process(...) repeatedly from your main loop() function until the download completes. I wanted to do it this way to ensure that I wasn’t blocking the loop() function for the entire download, thinking it might cause a problem with the particle system firmware. Admittedly this shouldn’t be much of a concern if you have SYSTEM_THREAD(ENABLED).
The other reason I went in this direction is that I want to be able to run other processes while the download is in progress, so I’m servicing other state machines from the loop() function besides just the download state machine.
So I think the original library from rsteckler would have worked fine for me if I had fixed that buffer overrun issue before refactoring to asynchronous downloading.
That makes sense. I would be using System_Thread so that’s nice to know.
I can’t figure out why his library will not compile in DEV or the CLI compiler. It gives tons of errors and I’m guessing because it’s outdated. I’ve asked for some help with figuring this out in another thread.
I can see there are just a few differences in your code where you not pushing the data to an SD card and he is so pushing to an SD card from your library should be an easy fix.
What is the biggest file you ever tried to download using this library? I assume not too big since you were not pushing the data to a SD card.
I am downloading a 2.2 MB file from S3 and storing it in an external SPI flash. Usually takes I think 30-60 seconds to download. Feel free to dm me a link to your thread or post here, I will take a look at your compiler errors.
I’ve been looking for an asynchronous HTTP download class and found this thread. I’ve taken a look at https://github.com/andrewaarestad/particle-http-downloader mentioned above. The readme says that it’s asynchronous, but the source code in the repo doesn’t seem to match the example e.g. there’s no http.process(&responseState, &code, downloadBuffer, &length) function.
Hey you are correct, I uploaded the wrong version of that code to github. I’ve updated the github repo now with my latest async download code. Let me know if you have any issues with it!
