Client/Server TCPIP communication - intermessage delay <22ms ERROR

Hi there,
I have a board with 2 ethernet peripherals.
I already enstablish the communication between these two peripherals embedding a client/server communication using FreeRTOS+TCP IP.

Below you can find the code snippet, just to give context:

//SERVER]
..
..

        	for( ;; )
        	{
	            xReceivedBytes = FreeRTOS_recv(xConnectedSocket, (void*)(&rx_eth_msg), sizeof(rx_eth_msg), 0 );
	            if( xReceivedBytes > 0 )
	            {
	            	tx_eth_msg = compute_client_request(rx_eth_msg);
	                encode_and_send_ETH_payload( tx_eth_msg, local_client_socket);
	                }
	            }
	           vTaskDelay(pdMS_TO_TICKS(6)); // small delay
	    	}

//CLIENT
..
..

    for (;;)
    {
    	encode_and_send_ETH_payload( msg_list[msg_id],  local_server_socket);
    	msg_id++;
    	if(msg_id>7) {msg_id = 0;}
    	vTaskDelay(pdMS_TO_TICKS(22)); // 1msg every 22 ms 
    }

In this example I send one ETH message every 22ms.

My communication works well, but, if I decide to lower the Client delay time,
I.E.:

( vTaskDelay (pdMS_TO_TICKS (21)))

something happens and my nodes do not talk anymore.

Is there a freeRTOS variable or setting you suggest to tweak in order to solve my issue?
I really wish to send/receive messages at a faster rate :smiley:

what does pdMS_TO_TICKS(22) evaluate to on your platform? I suspect 2 which means that if you make it lower than 20, you will get into the range where the task may typically not delay at all and thus starve your other tasks.

Can you explain or show the function encode_and_send_ETH_payload()?

Do you messages pas from 1 peripheral to the other and back? Are they connected by a LAN/switch?

What about the socket RCV/SND maximum waiting times?

I recommend checking every return value. Show them and take the proper action.

if ( ( iReturn < 0 ) &&
     ( iReturn != -pdFREERTOS_ERRNO_EAGAIN ) &&
     ( iReturn != -pdFREERTOS_ERRNO_EINTR ) )
{
    /* A fatal error occurred, like a disconnection. */
    return -1;
}

Actually, pdMS TO TICKS is 22:

For context:

This funciton just wraps the FreeRTOS send:

    void encode_and_send_ETH_payload(RCP_msg_t message, Socket_t xSocket){
    uint8_t xSentBytes;
    // Serialize the msg
     size_t messageSize = sizeof(message.msg_header) + sizeof(message.opt_header) + 
     sizeof(message.payload)/sizeof(message.payload[0]) + sizeof(message.footer_Checksum);
     uint8_t *buffer = (uint8_t *)malloc(messageSize);
     memcpy(buffer, &message, sizeof(message));
     FreeRTOS_send(xSocket, buffer, messageSize, 0);
     free(buffer);
    }

There are no switches in the system. Just 2 eth phy thransceivers enstablishing a communication.

Regarding rcv/snd timeouts, I set sockopts to 5seconds each when configuring socket options(both for client and server):

	static const TickType_t acpt_tout =  pdMS_TO_TICKS(5000); // 5 sec
	static const TickType_t rcv_tout =  pdMS_TO_TICKS(5000); // 5 sec
..

          FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_RCVTIMEO, &rcv_tout, sizeof(rcv_tout));
          FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_SNDTIMEO, &snd_tout, sizeof(snd_tout));

..

          FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_RCVTIMEO, &rcv_tout, sizeof(rcv_tout));
          FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_SNDTIMEO, &acpt_tout, sizeof(acpt_tout));

Would it be possible Simone, that you show a lot more from the testing program? You can take away any secrets or private code, I’d like to see all code related to +TCP.
Also I’d like to know which task is doing what? Or better: which tasks owns which sockets?
What platform are you using? Where did you get the driver?
Are you using a recent copy of the +TCP library?

But please start checking all return values. When you don’t, you might miss important information.

About +TCP and delays: normally a call to recv() might block if no packet is yet available. The function send() will normally return immediately, unless there is a queue of outgoing data.

Sure,
I didn’t attach all the code because I thought it was’t necessary.
I can achieve communication and it’s robust, I repeat.
But Server isn’t able to receive consecutive Client sends at a 21ms rate. (22ms works fine)

//MAIN

    void vApplicationIdleHook(void) {}

    int main(void)
    {
        uint8_t error;
	BOARD_init();
	error = startTCP_looped_communication();
	if (!error)
	{	vTaskStartScheduler();	}
	else	{	}
	for( ;; )	{	}
	return 0;
    }

//high level task

    uint32_t startTCP_looped_communication(void)
    {
      int32_t ret = pdPASS;
      if (ret == pdPASS)
      {	  /* Create server main task */
    	  ret = xTaskCreate(server_conn_Task, "server_conn_Task", REMOTE_SERVER_MAIN_TASK_STACK_SIZE, NULL, REMOTE_SERVER_MAIN_TASK_PRIORITY, &server_TaskHandler);
      }
      if (ret == pdPASS)
      {	  /* Create clientmain task */
	  ret = xTaskCreate(client_conn_Task, "client_conn_Task", REMOTE_SERVER_MAIN_TASK_STACK_SIZE, NULL, REMOTE_SERVER_MAIN_TASK_PRIORITY, &client_TaskHandler);
    	}
      return (ret == pdPASS) ? 0 : 1;
    }

//Client TASK

     portTASK_FUNCTION(client_conn_Task, pvParam)
     {    (void)pvParam;
     uint8_t ret;
     static const TickType_t tout =  pdMS_TO_TICKS(5000);
    Socket_t local_server_socket;
    const char *pcInterfaceName = "eth1"; 
    struct freertos_sockaddr xServerAddress, xClientAddress;
    BaseType_t xSentBytes;
    RCP_msg_t tx_msg;
    local_server_socket = FreeRTOS_socket(FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP);
    configASSERT(local_server_socket != FREERTOS_INVALID_SOCKET);
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
    xClientAddress.sin_family = FREERTOS_AF_INET;
    xClientAddress.sin_port = FreeRTOS_htons(ETH_PORT);
    xClientAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.2");
    xServerAddress.sin_family = FREERTOS_AF_INET;
    xServerAddress.sin_port = FreeRTOS_htons(ETH_PORT);
    xServerAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.1"); 

    vTaskDelay(pdMS_TO_TICKS(1000)); 
    do {
        ret = FreeRTOS_connect(local_server_socket, &xServerAddress, sizeof(xServerAddress));
        vTaskDelay(pdMS_TO_TICKS(750)); 
    } while (ret < 0);
    for (;;)
    {
    	encode_and_send_ETH_payload( msg_list[msg_id],  local_server_socket);
    	msg_id++;
    	if(msg_id>7) {msg_id = 0;}
    	vTaskDelay(pdMS_TO_TICKS(22)); 
      }
     }

//SERVER task

	portTASK_FUNCTION(server_conn_Task, pvParam ) {
	(void)pvParam;
	Socket_t xListeningSocket, xConnectedSocket, local_client_socket;
	struct freertos_sockaddr xBindAddress, connected_client_sockadd;
    socklen_t connected_client_size = sizeof(connected_client_sockadd);
	static const TickType_t tout =  pdMS_TO_TICKS(5000); 
	const char *pcInterfaceName = "eth0"; //test8650 // eth0=8670
    RCP_msg_t rx_eth_msg, tx_eth_msg;
    BaseType_t xReceivedBytes, xSentBytes;
	xListeningSocket = FreeRTOS_socket(FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP);
	if (xListeningSocket != FREERTOS_INVALID_SOCKET)
	{
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
		xBindAddress.sin_family = FREERTOS_AF_INET;
		xBindAddress.sin_port = FreeRTOS_htons( ETH_PORT );
		xBindAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.1");
		FreeRTOS_bind( xListeningSocket, &xBindAddress, sizeof( xBindAddress ) ); //bind to 5555
		FreeRTOS_listen( xListeningSocket, xBacklog );
        xConnectedSocket = FreeRTOS_accept( xListeningSocket, &connected_client_sockadd, &connected_client_size );
        if (xConnectedSocket != 0x0)
        {
      	local_client_socket = FreeRTOS_socket(FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP);
         if(local_client_socket != 0x0){
           FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
           FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
           FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
        	for( ;; )
        	{
	            xReceivedBytes = FreeRTOS_recv(xConnectedSocket, (void*)(&rx_eth_msg), sizeof(rx_eth_msg), 0 );
	            if( xReceivedBytes > 0 )
	            {
	            	tx_eth_msg = compute_client_request(rx_eth_msg);
	                //send response
	                encode_and_send_ETH_payload( tx_eth_msg, local_client_socket);
	                }
	            }
	           vTaskDelay(pdMS_TO_TICKS(6)); // small delay
    }}}}

I’m quite sure there is some freeRTOS configuration parameter which is “bottlenecking” my application. :smiling_face_with_tear:

The following code looks wrong to me:

    xConnectedSocket = FreeRTOS_accept( xListeningSocket, &connected_client_sockadd, &connected_client_size );
    if( xConnectedSocket != 0x0 )
    {
  	    local_client_socket = FreeRTOS_socket( FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP );
        if(local_client_socket != 0x0)
        {
            FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
            FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
            FreeRTOS_setsockopt(local_client_socket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
            for( ;; )
            {
                xReceivedBytes = FreeRTOS_recv(xConnectedSocket, (void*)(&rx_eth_msg), sizeof(rx_eth_msg), 0 );
                if( xReceivedBytes > 0 )
                {
                    tx_eth_msg = compute_client_request(rx_eth_msg);
                    //send response
                    encode_and_send_ETH_payload( tx_eth_msg, local_client_socket);
                }
            }
            vTaskDelay(pdMS_TO_TICKS(6)); // small delay
        }
    }

Why do you need local_client_socket? Should it not be just the following:

    xConnectedSocket = FreeRTOS_accept( xListeningSocket, &connected_client_sockadd, &connected_client_size );
    if( xConnectedSocket != 0x0 )
    {
        for( ;; )
        {
            xReceivedBytes = FreeRTOS_recv(xConnectedSocket, (void*)(&rx_eth_msg), sizeof(rx_eth_msg), 0 );
            if( xReceivedBytes > 0 )
            {
                tx_eth_msg = compute_client_request(rx_eth_msg);
                //send response
                encode_and_send_ETH_payload( tx_eth_msg, xConnectedSocket);
            }
        }
        vTaskDelay(pdMS_TO_TICKS(6)); // small delay
    }

@aggarg wrote:

Why do you need local_client_socket? Should it not be just the following

I understand that neither.

You are trying to communicate through local_client_socket, but that socket is not yet connected, either through accept() or through connect().

Where did you find the socket option FREERTOS_SO_BINDTODEVICE? Google can not help me with it.

It looks like you have implemented a loop: a socket is listening on one interface, and it receives a connection from the other interface.

You are using malloc() without testing the result. Two questions: are you sure it is safe to call malloc(): normally I use pvPortMalloc(size_t size), after having chosen one out of:

    portable\MemMang\heap_1.c
    ...
    portable\MemMang\heap_5.c

Normally I use heap_5.c.

-    if( xConnectedSocket != 0x0 )
+    if( xConnectedSocket != NULL )

because in FreeRTOS+TCP a socket is a pointer, not a number.

Hi there, sorry I’ve been on vacancy.
I decided to use 2 sockets in order to differentiate “incoming message socket” and “output message socket”. (building the basis for an half duplex communication)
Both Server and Client nodes should be able to send ETH messages, sometime even at the same time. I thought that doing so concurrency issues were avoidable.

If you assure me that freertos manages concurrency, I will be glad to cut 1 socket :slight_smile:

@htibosch

-    if( xConnectedSocket != 0x0 )
+    if( xConnectedSocket != NULL )

because in FreeRTOS+TCP a socket is a pointer, not a number.

By debugging my variables “xConnectedSocket” assumed “0x0” when not valid. I think it may depend by the compiler.

FREERTOS_SO_BINDTODEVICE

It’s just a custom opt defining a default uLocalAddress, here follows the core operations:

NetworkInterface_t * pxNetIf = FreeRTOS_GetIfByName( pxIfName );
...
pxSocket->ulLocalAddress = pxNetIf->ulDefaultIPAddress;
...

Anyway, my communication works well, but I had to add a delay of at least 22ms between 2 sent messages in order to make it work.
I tried without delay and it doesn’t work.
Do you think tweaking a freeRTOS setting may solve my issue?

TCP is a connected protocol that is generally bi-directional. If what you have there is working with no connect call on the local client socket, I will be very surprised, and may indicate a bug in the stack socket functions. Binding a local TCP socket to a specific interface (SO_BINDTODEVICE) only specifies the interface through which a “connect()” will initiate the connection over or a “listen()”, and you should not attempt it on the socket you get from the “accept()” call, since that socket is for the established connection which is already effectively bound to the interface (device) on which the connection from the client has been established.

I recommend that you look up a TCP echo server example, for example [TCP echo client-server in C · GitHub](this one).
I found that one at random with a Google web search, and it has some questionable coding practices (see the comments below), the use of the TCP/IP sockets is correct.

Yes, it is required to send and receive data on the connected socket returned by FreeRTOS_accept. As @danielglasser also mentioned, using another local unconnected socket is wrong.

1 Like

Hi there,
I’ve tried using only one socket for the server, as you suggested.

Unfortunately, then nothing works anymore. So my hypothesis was correct. The second socket is needed and safer.

To make you understand better the code here follows a scheme:

22ms seems to be the lower limit achievable.
I’ve tried increasing the granularity of the clock but nothing.
I’ve also tested both task assigning different priorities but nothing eighter.

HELP

As several people have pointed out before, it is a fundamental networking principle - you cannot transmit data on an unconnected socket.

I recommend the following approach:

  1. Start by implementing a basic TCP server using a single PHY, referring to our demonstration examples for guidance.
  2. Test connectivity by attempting to reach the server from another device on the network.
  3. Once you’ve confirmed the single PHY setup is working correctly, proceed to implementing the dual PHY configuration.

Ok.
To prove that’s not limiting my speed, let me show another testing environment which displayed the same behavior, even with only one connected socket. (Which is the former question of this thread)

In this code I have:

  • 1 socket for the client (connect)
  • 1 socket for the server (listen and accept)
  • the client sends an ETH msg each 22 ms to the connected socket.
  • Each time a message is received an SPI signal is triggered, based on the payload received.
  • The action is monitored via logic analizer and the SPI signal is correct.

The 22ms communication works perfectly.

Now,
If I reduce the sending delay from 22 to 21ms the ETH message is no more transmitted (sent only 4 times and then the communication shuts)

Based on your experience, do you think there is some FreeRTOS configuration parameter which may be bottlonecking my application?

Thanks,
Simone

Please show us the full code for the revised (single socket only) application.

Also, do you see the same behavior if you run the client and server on different DUTs (ie not on two interfaces on the same DUT)?

do you see the same behavior if you run the client and server on different DUTs

Not yet tested, I’m waiting for another board.

The code is similar to the one I already shared, without the second server socket, and without response sent, just server processing messages:

//MAIN

    void vApplicationIdleHook(void) {}

    int main(void)
    {
        uint8_t error;
	BOARD_init();
	error = startTCP_looped_communication();
	if (!error)
	{	vTaskStartScheduler();	}
	else	{	}
	for( ;; )	{	}
	return 0;
    }

//high level task

    uint32_t startTCP_looped_communication(void)
    {
      int32_t ret = pdPASS;
      if (ret == pdPASS)
      {	  /* Create server main task */
    	  ret = xTaskCreate(server_conn_Task, "server_conn_Task", REMOTE_SERVER_MAIN_TASK_STACK_SIZE, NULL, REMOTE_SERVER_MAIN_TASK_PRIORITY, &server_TaskHandler);
      }
      if (ret == pdPASS)
      {	  /* Create clientmain task */
	  ret = xTaskCreate(client_conn_Task, "client_conn_Task", REMOTE_SERVER_MAIN_TASK_STACK_SIZE, NULL, REMOTE_SERVER_MAIN_TASK_PRIORITY, &client_TaskHandler);
    	}
      return (ret == pdPASS) ? 0 : 1;
    }

//Client TASK

     portTASK_FUNCTION(client_conn_Task, pvParam)
     {    (void)pvParam;
     uint8_t ret;
     static const TickType_t tout =  pdMS_TO_TICKS(5000);
    Socket_t local_server_socket;
    const char *pcInterfaceName = "eth1"; 
    struct freertos_sockaddr xServerAddress, xClientAddress;
    BaseType_t xSentBytes;
    RCP_msg_t tx_msg;
    local_server_socket = FreeRTOS_socket(FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP);
    configASSERT(local_server_socket != FREERTOS_INVALID_SOCKET);
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
    FreeRTOS_setsockopt(local_server_socket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
    xClientAddress.sin_family = FREERTOS_AF_INET;
    xClientAddress.sin_port = FreeRTOS_htons(ETH_PORT);
    xClientAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.2");
    xServerAddress.sin_family = FREERTOS_AF_INET;
    xServerAddress.sin_port = FreeRTOS_htons(ETH_PORT);
    xServerAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.1"); 

    vTaskDelay(pdMS_TO_TICKS(1000)); 
    do {
        ret = FreeRTOS_connect(local_server_socket, &xServerAddress, sizeof(xServerAddress));
        vTaskDelay(pdMS_TO_TICKS(750)); 
    } while (ret < 0);
    for (;;)
    {
    	encode_and_send_ETH_payload( msg_list[msg_id],  local_server_socket);
    	msg_id++;
    	if(msg_id>7) {msg_id = 0;}
    	vTaskDelay(pdMS_TO_TICKS(22)); 
      }
     }

//SERVER task

	portTASK_FUNCTION(server_conn_Task, pvParam ) {
	(void)pvParam;
	Socket_t xListeningSocket, xConnectedSocket, local_client_socket;
	struct freertos_sockaddr xBindAddress, connected_client_sockadd;
    socklen_t connected_client_size = sizeof(connected_client_sockadd);
	static const TickType_t tout =  pdMS_TO_TICKS(5000); 
	const char *pcInterfaceName = "eth0"; //test8650 // eth0=8670
    RCP_msg_t rx_eth_msg, tx_eth_msg;
    BaseType_t xReceivedBytes, xSentBytes;
	xListeningSocket = FreeRTOS_socket(FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP);
	if (xListeningSocket != FREERTOS_INVALID_SOCKET)
	{
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_RCVTIMEO, &tout , sizeof(tout ));
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_SNDTIMEO, &tout , sizeof(tout ));
		FreeRTOS_setsockopt( xListeningSocket, 0, FREERTOS_SO_BINDTODEVICE, pcInterfaceName, strlen(pcInterfaceName));
		xBindAddress.sin_family = FREERTOS_AF_INET;
		xBindAddress.sin_port = FreeRTOS_htons( ETH_PORT );
		xBindAddress.sin_addr = FreeRTOS_inet_addr("192.168.1.1");
		FreeRTOS_bind( xListeningSocket, &xBindAddress, sizeof( xBindAddress ) ); //bind to 5555
		FreeRTOS_listen( xListeningSocket, xBacklog );
        xConnectedSocket = FreeRTOS_accept( xListeningSocket, &connected_client_sockadd, &connected_client_size );
        if (xConnectedSocket != 0x0)
        {
        	for( ;; )
        	{
	            xReceivedBytes = FreeRTOS_recv(xConnectedSocket, (void*)(&rx_eth_msg), sizeof(rx_eth_msg), 0 );
	            if( xReceivedBytes > 0 )
	            {
	            	tx_eth_msg = compute_client_request(rx_eth_msg);
	                }
	            }
    }}}

encode_and_send_ETH_payload just encode and sends ETH:

defines variables..
...
    memcpy(buffer, &message, sizeof(message));
    FreeRTOS_send(xSocket, buffer, messageSize, 0); 
    free(buffer);
...

I’ve tested the speed of functions using ticks to verify any “blocking” point.
It turns out my procedures gets some dozens of microseconds each.

Somehow, somewhere, some FreeRTOS process cannot go under 22ms.

Another thought:

Which HW are you using? In another thread you mention AEK-COM-10BASET which is based on the SPC58EC80E5 MCU. According to the data sheet, that MCU can be clocked at most at 180MHz. At 1ms FreeRTOS system clock, that may be too slow because the timer ISR may be eating up too many cycles for the rest of the system not to get starved. The 21/22ms threshold may just be coincidentally the limit that separates a barely functioning system from a starved one.

Yes, this is the micro.
I’m actually clocking at 8MHz, so 0,125 μs is the clock period.
To reach 1 ms we need 8000 clock cycles.
I think we have plenty of resources for FreeRTOS to run but, maybe, I have to change the ones reserved somehow.

Do you know how do I set the FreeRTOS working rate from the config files?

Anyway, I can assure that the micro is capable to drive peripherals (SPI, CAN, GPIO, PWM etc.) at a faster rate than 1 ms (depending on the peripheral it requires from 250 ns to 25 ÎĽs )

8000 instructions is not a whole lot and depending on the optimization level can easily be reached during procession of a timer isr. You may want to experiment with optimization levels (eg -Ot) and see if that makes a difference, but I would put a fair amount of money on the table to bet on your system being too slow.

Look up configTICK_RATE_HZ in Customization - FreeRTOS™ for details.