Using FreeRTOS+coreHTTP as a example to make chunk upload to aws s3

Partners & Sponsors AWS
1

Hi,
I have a problem where I am firstly connect to IOT to get the temporary credentials then connect to aws and send HTTP request. I converted the code from the link above: FreeRTOS/FreeRTOS-Plus/Demo/coreHTTP_Windows_Simulator/HTTP_S3_Download/DemoTasks/S3DownloadHTTPExample.c at main · FreeRTOS/FreeRTOS · GitHub
To enable sending chunks and I send them as described here:
Signature Calculations for the Authorization Header: Transferring Payload in Multiple Chunks (Chunked Upload) (AWS Signature Version 4) - Amazon Simple Storage Service
At the moment, I can send maximum of 96,4KB in a single file that has 11 chunks each one of 8976B and the last empty as described in the previous link, everything works. But when I have a larger file than 9640 KB on my sd card it always stops uploading at 101 file. Upload works for up to 100 files in the s3 directory. What could be the problem since this always happens, even when I upload 3 chunks first 8KB second 1 KB and the third 0KB total uploaded file has 9KB and It always stops uploading on file 101 and i can’t received any response from the server. Can it be problem with the settings of aws?
Second question is that the aws server is not responding and when i am uploading much more chunks for example 25 even any value greater than 11 i get stuck because tcp buffer is full and the server not doing the ACK response to make tcp buffer free as the protocol should work. And it is all works like this:
Firstly I send the HTTP request without body:
httpStatus = HTTPClient_Send(ptransportInterface,
&requestHeaders,
NULL,
0,
&(response),
(HTTP_SEND_DISABLE_CONTENT_LENGTH_FLAG | HTTP_SEND_DO_NOT_RECEIVE_FLAG));
And then in the loop i send the chunks with the signature calculated
uint32_t sendFlags = (HTTP_SEND_DISABLE_CONTENT_LENGTH_FLAG |
HTTP_SEND_DO_NOT_SEND_HEADERS_FLAG |
HTTP_SEND_DO_NOT_RECEIVE_FLAG);
httpStatus = HTTPClient_Send(ptransportInterface,
&requestHeaders,
&(pBufferToSend[0]),
chunkLength,
&(response),
sendFlags);
And the last chunk required by Transfer-Encoding.
httpStatus = HTTPClient_Send(ptransportInterface,
&requestHeaders,
“0\r\n\r\n”,
sizeof(“0\r\n\r\n”) - 1,
&(response),
(HTTP_SEND_DISABLE_CONTENT_LENGTH_FLAG | HTTP_SEND_DO_NOT_SEND_HEADERS_FLAG));

2

Can you break the program in the debugger and see what it is doing? Also, please capture network trace and share.

How did you determine that?

3

What do you mean “break the program in the debugger and see what it is doing”?
When i send 101 file i get stuck in program in line with cyw43_arch_poll(); here:

int32_t recvUsingLwIP(NetworkContext_t *pNetContext, void *buf, size_t buflen)
{
    int32_t ret = ERR_VAL;
    if(buf == NULL)
    {
        printf("Error: recvUsingLwIP - buf pointer is NULL\n");
        return ret;
    }
    if(pNetContext == NULL)
    {
        printf("Error: recvUsingLwIP - Network Context pointer is NULL\n");
        return ret; 
    }
    if(buflen < 1)
    {
        printf("Error: recvUsingLwIP - buf less then 1\n");
        return ret;
    }

    while(1)
    {
        cyw43_arch_poll();
        if(pNetContext->readFlag == 1)
        {
            pNetContext->readFlag = 0;
            memcpy(buf, lwipRecToCorehttpRec, pNetContext->bytesToRecv);
            memset(lwipRecToCorehttpRec, 0, sizeof(lwipRecToCorehttpRec));
            return pNetContext->bytesToRecv;
        }
    }
    
}

I know my tcp buffer is full because I used this function: altcp_sndbuf(pNetContext->pcb);

4

That is what I wanted to know - i.e. the function call where the code is stuck. Where does this function coms from? Is it possible for figure out why this function is stuck?