I’m now trying to use the “iot_demo_https_s3_upload_sync.c” to upload a file from the SPIFFS File System on ESP32 flash, to Amazon S3.
I’m using the wrapper “esp_vfs.h” to use SPIFFS. So I’m able to operate on the files and folders contained in the SPIFFS partition on flash using the C standard functions fopen, fread, fwrite…
At first, I succesfully tried to fopen the file, fread (from the stream represented by the FILE *) on a “uint8_t buffer[file size]” and after fclosed the file, to pass the uint8_t pointer of the buffer to “reqSyncInfo.pBody” at this line of the demo: iot_demo_https_s3_upload_sync.c#L297
and to pass the file size to “reqSyncInfo.bodyLen” at this line of the demo: iot_demo_https_s3_upload_sync.c#L298
But the problem comes when the file size is larger than the stack size: the demo crashes due to the excess of the stack size by the buffer allocated. I can try to increase the stack size, but the RAM size is limited, I would like to take advantage of all 2 MB of SPIFFS and to upload on Amazon S3 log files of my application which can reach hundreds of KB, or almost 1 MB.
And my question is: how can I retrieve the uint8_t pointer to an already established file elsewhere in memory? What exactly does it means?
I saw that the FILE struct for my system is the _sFILE Struct implementation for Linux, so here “https://lattice.github.io/quda-ref/v0.9.0/doc/struct____s_f_i_l_e.html” I found which are the fields of the structure and I tried, without success, to retrieve here the uint8_t pointer to assign to the “pBody” data field of “reqSyncInfo”.
My hope was that in the FILE structure there was a pointer to a buffer that it contained the file contents.
At first, I succesfully tried to fopen the file, fread (from the stream represented by the FILE *) on a “uint8_t buffer[file size]” and after fclosed the file, to pass the uint8_t pointer of the buffer to “reqSyncInfo.pBody” at this line of the demo: iot_demo_https_s3_upload_sync.c#L297
and to pass the file size to “reqSyncInfo.bodyLen” at this line of the demo: iot_demo_https_s3_upload_sync.c#L298
But the problem comes when the file size is larger than the stack size: the demo crashes due to the excess of the stack size by the buffer allocated. I can try to increase the stack size, but the RAM size is limited, I would like to take advantage of all 2 MB of SPIFFS and to upload on Amazon S3 log files of my application which can reach hundreds of KB, or almost 1 MB.
For the above comments, it sounds like you are saving the file in a buffer allocated on the stack in the local function space. Could you declare the buffer in the static global space and try again?
And my question is: how can I retrieve the uint8_t pointer to an already established file elsewhere in memory? What exactly does it means?
This is referring to a file that may be stored for example at an address in the memory mapped flash space. This file must be stored in a contiguous memory location in order for the library to send it.
For the above comments, it sounds like you are saving the file in a buffer allocated on the stack in the local function space. Could you declare the buffer in the static global space and try again?
Yes, I was saving the file in a buffer allocated on the stack in the local function space.
Yes, I can declare the buffer in the static global space and thank you for the advice. But the RAM size is still limited, I would like to be able to send larger files.
This is referring to a file that may be stored for example at an address in the memory mapped flash space. This file must be stored in a contiguous memory location in order for the library to send it.
Ok, so if I memory map the flash spiffs partition where my file is stored in a contiguous memory location, then I just need to pass the file name to pBody?
It might be a good solution dedicate a spiffs partition only for the file to be sure that it is saved contiguously?
From HTTP library’s perspective, it can send any data stored in a contiguous memory location. Then you need to pass the location of that memory (i.e. pointer) to pBody.
I am not very familiar with the SPIFFS library and related partitions, so it will probably be better to ask on an Espressif forums.