TCP Communication

Kernel
1

xterra wrote on Monday, February 11, 2019:

Hi all,

we have a linux system (10.222.6.51) and an embedded system (10.222.187.99), the embedded system (Renesas RX65N) works with freeRTOS+TCP.
We connect from the linux system to the embedded system an send 4xMSS data.

The TCP socket option settings on the embedded system are:

  • winsize: 4*MSS
  • buffersize: 4*MSS

On the recording (see file pcapng) there a two things we don´t understand:

a) winsize notification from freeRTOS+TCP, see in recording no. 9
the notified winsize is 1463? Why is it not 1460 like in no. 12?

b) bigger problem:
why is there a RST in no. 3? this one is ignored from the linux system?
the RST in no. 11 is not ignored, the linux quits this in no. 13!
But we don´t want any RST? the connection should be on furthermore.
If we try this several times, it isn´t always the same! Sometimes the RST in no. 11 is not there???
But the RST in no. 3 is always there which is fortunately ignored form the linux system…

Thanks a lot!
Best regards
Wolfgang

2

heinbali01 wrote on Monday, February 11, 2019:

Hi Wolfgang, would it be possible that you send (attach) two things?
First I’d like to see the TCP code that is handling this connection on the FreeRTOS+TCP side.
From FreeRTOS_socket(), any FreeRTOS_setsockopt() calls and also the calls to FreeRTOS_recv()?
Also, I’d be interested in seeing your FreeRTOSIPConfig.h

3

heinbali01 wrote on Monday, February 11, 2019:

b) bigger problem:

Sorry, I forgot to answer about your bigger problem. Yes indeed, the RST packets are weird and unexpected. I don’t see any reason why reason why FreeRTOS+TCP would send them.

Normally, a RST packet is a result of some protocol error, like : socket not connected, or a sequence number out-of-range.

Well, the more source code that you show me, the more likely it is that I can pinpoint where things go wrong.

4

xterra wrote on Monday, February 11, 2019:

Hi Hein,

Attached is the source file for tcp communication as well as FreeRTOSIPConfig file

The following actions are triggered by serial interface (code not included):
• TCPConncetion_CreateListenSocket
• TCPConncetion_AcceptConnection
• TCPConncetion_Receive
The listening socket is reused for connection.
The socket options are configurable (winsize, buffersize) by serial interface.

Then linux system is sending data.

Best regards

Wolfgang Schur

5

xterra wrote on Tuesday, February 12, 2019:

Hi Hein,

we found our big problem: we had two embedded systems with the same ip-address in the network!
Thanks for the hint at protocol error in case of RST!!!

So there remains the topic with the reported winsize.

I have the following socket options in embedded system:

  • buffersize 4*MSS
  • winsize 4*MSS
    (MSS = 1460)

see my first pcapng-file above
a) linux system sends data of size 4*MSS to embedded system
b) embedded system acknowledges the first 3 packets and reports winsize = 1463
why 1463 instead of 1460?
c) second tcp stack acknowledges the last packet with winsize = 0
d) afterwards embedded system reads step-by-step 1000 bytes of data out of the socket.
We would expect that tcp stack informs about new win-size as soon as 2000 byte are read out
e) but tcp stack reports not until almost all bytes are read out

questions:

  • when will tcp stack report the new winsize? is it possible to configure this?
  • why is winsize 1463 instead of 1460?

Thanks a lot

Best regards
Wolfgang

6

heinbali01 wrote on Wednesday, February 13, 2019:

Wolfgang, thanks for reporting this.

I had to look this up: why is de WIN size reported as 1463 and not 1460?
It has to do with the TCP/IP stream buffers: The length of the buffer can be used minus on byte.
That has to do with the fact that a read and write index are being used.
So in stead of making a bufer N x 1460 bytes, an extra 4 bytes are reserved.
For example: 4 x 1460 + 4 = 5844 bytes, of which 5843 bytes can be used. There are the extra 3 bytes!

It does not hurt, the reported buffer space is really available!

I just tried to make a patch for this but the resulting code would be too ugly.
Regards, Hein

7

heinbali01 wrote on Wednesday, February 13, 2019:

I just wrote:

So in stead of making a buffer N x 1460 bytes, an extra 4 bytes are reserved.

A next question could be: why not reserve N x 1460 + 1 bytes?

That has to do with efficient copying. Calling memcpy() for aligned buffers is much more efficient than for unaligned buffers.

When the stream buffer has a length which is a multiple of 4 ( or sizeof( size_t ) ), it is more likely that access in that buffer is always aligned.

8

xterra wrote on Wednesday, February 13, 2019:

Hi Hein,

thanks a lot for this information, so I think this offset of 3 is okay for us!

Another thing is more problematic:

see winsize pcapng file

a) embedded system configures socket with 4MSS buffersize and 4MSS winsize
b) linux system send 4*MSS = 5840 byte to embedded system
c) embedded system acknowledges 5840 and reports winsize 0, okay!
d) on embedded system I read out 1xMSS (no.1) and would expect that a new winsize would be reported
e) but nothing happens, even if I read out another 1xMSS (no.2)!
f) still noting happens when I read out 1xMSS (no.3)
g) until I read out the last 1xMSS (no.4) there is the report of new winsize, but only 1460(1xMSS) not 4xMSS!?

Could it be that reporting new winsize is not working correctly when data are read out of socket?

We want to use the tcp buffer as buffer of pixeldata to be printed out.
This buffer should be fill with additional data as soon as there is space, not when it is empty!
Because if buffer is empty printing would have to be stopped! But printing must not be stopped!!!

Best regards
Wolfgang

9

heinbali01 wrote on Wednesday, February 13, 2019:

this post was a copy of the next message…

10

heinbali01 wrote on Wednesday, February 13, 2019:

Could it be that reporting new winsize is not working correct when data are read out of socket?

I am not aware of any problem with that. But let me explain how it works:
When receiving data, the driver has a definition of “low-water”. “low-water” means that the reception buffer has little space.
The algorithm inFreeRTOS_recv() is simple: only in case of “low-water”, a signal will be sent to the IP-task, that will result in sending an update of the WIN size.

Could you try a simple thing: put the TCP window size (for reception) to 1460 (of 2920) bytes?

So like this:

	windowsProperties.lRxBufSize = 4 * ipconfigTCP_MSS;
	windowsProperties.lRxWinSize = 1; /* or 2 */

I think that the TCP server will send WIN updates more often because the TCP window is smaller.

11

xterra wrote on Thursday, February 14, 2019:

Hi Hein,

could you please explain how the low-water mark is calculated considering rxbuf size and win size?

if rxbuf is under low-water mark then an update of winsize is reported.
for us it would be fine to be able to set the low-water mark higher!
then we could get more pixeldata earlyer!

here are some tests:
i tested rxbufsize 4xMSS (5840 bytes) and winsize 2xMSS:
winsize is updated if i read out of rxbuf 4680bytes (3xMSS + 300bytes) (low-water=4680 of 5840)
but the reported winsize is only 1460 (1xMSS) although there are now 3xMSS free in rxbuf?!? ->winsize should be 2xMSS

i tested rxbufsize 4xMSS (5840 bytes) and winsize 4xMSS:
winsize is updated if i read out of rxbuf 4680byte (3xMSS + 300byte)
-> here reported winsize should be 3xMSS but is also only 1xMSS

i tested rxbufsize 8xMSS (11680 bytes) and winsize 2xMSS:
winsize is update if i read out of rxbuf 9360 (6xMSS + 600byte), here too the reported winsize is only 1460 (1xMSS) althougth there are now 6xMSS free in rxbuf -> winsize sould be 2xMSS?

result:
low-water mark is very low, i.e. almost all data are read out of rxbuf!
win-update doesn´t report the correct win-size

Best regards
Wolfgang

12

heinbali01 wrote on Friday, March 01, 2019:

Hi Wolfgang, thank you for reporting these observations.

I could replicate you problem, and I made a change in FreeRTOS_TCP_IP.c, see this pull request. The problem was that the WIN size reported was running a bit behind.