FreeRTOS+TCP with BufferAllocation_1.c: does it (still) work?

system · July 4, 2016, 11:40am

sbrauss wrote on Monday, July 04, 2016:

Hi!

I currently try to port FreeRTOS+TCP to Renesas RX62N using the code snippets from interactive.freertos.org. The code uses statically allocated buffers with BufferAllocation_1.c. I downloaded the current release of FreeRTOS+TCP (160112). However, it does not compile on UNIX/Linux machine,as there is a typo in BufferAllocation_1.c, as it includes “FreeRTOS_sockets.h” instead of “FreeRTOS_Sockets.h”. This is maybe not an issue on a win32 platform, I don’t know.
What I wanted to ask for is if a build using statically allocated buffers with BufferAllocation_1.c is still working.

Thank you,
Stephan

rtel · July 4, 2016, 11:43am

rtel wrote on Monday, July 04, 2016:

Thanks for reporting the problem with building under Linux. I think
that issue has been resolved in the SVN repository already.

system · July 4, 2016, 12:09pm

sbrauss wrote on Monday, July 04, 2016:

Hi!
Thanks for your quick reply! So it still should working?
Where is the SVN repository?
Best regards,
Stephan

Am 04.07.2016 um 13:43 schrieb Real Time Engineers ltd.:

Thanks for reporting the problem with building under Linux. I think
that issue has been resolved in the SVN repository already.

FreeRTOS+TCP with BufferAllocation_1.c: does it (still) work?
https://sourceforge.net/p/freertos/discussion/382005/thread/7f1e9a92/?limit=25#dff8/0991

Sent from sourceforge.net because you indicated interest in
SourceForge.net: Log In to SourceForge.net

To unsubscribe from further messages, please visit
SourceForge.net: Log In to SourceForge.net

rtel · July 4, 2016, 12:17pm

rtel wrote on Monday, July 04, 2016:

The SVN repository is not [yet] public.

htibosch · July 4, 2016, 12:51pm

heinbali01 wrote on Monday, July 04, 2016:

Hi Stephan,

So it still should working?
Yes both BufferAllocation_1 and 2 still supported. The former indeed uses statically allocated buffer space, the second one allocates the buffer space dynamically.

BufferAllocation_1.c : by default it will use critical sections to protect the List_t xFreeBuffersList.

#define ipconfigBUFFER_ALLOC_LOCK()					taskENTER_CRITICAL()
#define ipconfigBUFFER_ALLOC_UNLOCK()				taskEXIT_CRITICAL()

Some users found it more efficient to use a different method, especially if NetworkBuffers are never accessed from within an interrupt (also my personal preference). So those users may use their own version:

#define ipconfigBUFFER_ALLOC_LOCK()					myAllocLoc()
#define ipconfigBUFFER_ALLOC_UNLOCK()				myAllocUnloc()

How is the porting to Renesas RX62N going? Please don’t hesitate to ask more questions, it might save you time.

Regards.

system · July 4, 2016, 3:01pm

sbrauss wrote on Monday, July 04, 2016:

Hi Hein!

How is the porting to Renesas RX62N going? Please don’t hesitate to
I had a “small” success approx. an hour ago: pinging the device works.

I then included SimpleTCPEchoServer.c into the project to test the
socket interface. My system crashes as soon as I open the echo port by
my linux machine (“telnet 192.168.200.169 7”). I thought that I maybe
get out of memory, but I have >5K left, at least before the crash…
Stack size is 512bytes, so the same as in the demos.

As a ping works, my first thought was that the MAC and PHY layer works
fine.

Thank you & best regards,
Stephan

Am 04.07.2016 um 14:51 schrieb Hein Tibosch:

Hi Stephan,
So it still should working?
Yes both BufferAllocation_1 and 2 still supported. The former indeed
uses statically allocated buffer space, the second one allocates the
buffer space dynamically.
BufferAllocation_1.c : by default it will use critical sections to
protect the |List_t| |xFreeBuffersList|.

#define ipconfigBUFFER_ALLOC_LOCK() taskENTER_CRITICAL()
#define ipconfigBUFFER_ALLOC_UNLOCK() taskEXIT_CRITICAL()

Some users found it more efficient to use a different method, especially
if NetworkBuffers are never accessed from within an interrupt (also my
personal preference). So those users may use their own version:

#define ipconfigBUFFER_ALLOC_LOCK() myAllocLoc()
#define ipconfigBUFFER_ALLOC_UNLOCK() myAllocUnloc()

How is the porting to Renesas RX62N going? Please don’t hesitate to ask
more questions, it might save you time.

Regards.

FreeRTOS+TCP with BufferAllocation_1.c: does it (still) work?
https://sourceforge.net/p/freertos/discussion/382005/thread/7f1e9a92/?limit=25#82f9

Sent from sourceforge.net because you indicated interest in
SourceForge.net: Log In to SourceForge.net

To unsubscribe from further messages, please visit
SourceForge.net: Log In to SourceForge.net

system · July 4, 2016, 4:29pm

sbrauss wrote on Monday, July 04, 2016:

The problem turns up as soon as FreeRTOS_listen() is called, and a
connect packet arrives for this port. I commented our the endless loop
in prvConnectionListeningTask(), so no accept is executed. It seems that
just the arrival of the TCP connect request from my linux machine let’s
it crash. The interrupt routine is still working when it is crashed, so
the interrupt system does not hang.

Am 04.07.2016 um 17:00 schrieb Stephan Brauss:

Hi Hein!

How is the porting to Renesas RX62N going? Please don’t hesitate to
I had a “small” success approx. an hour ago: pinging the device works.

I then included SimpleTCPEchoServer.c into the project to test the
socket interface. My system crashes as soon as I open the echo port by
my linux machine (“telnet 192.168.200.169 7”). I thought that I maybe
get out of memory, but I have >5K left, at least before the crash…
Stack size is 512bytes, so the same as in the demos.

As a ping works, my first thought was that the MAC and PHY layer works
fine.

Thank you & best regards,
Stephan

Am 04.07.2016 um 14:51 schrieb Hein Tibosch:
Hi Stephan,
So it still should working?
Yes both BufferAllocation_1 and 2 still supported. The former indeed
uses statically allocated buffer space, the second one allocates the
buffer space dynamically.
BufferAllocation_1.c : by default it will use critical sections to
protect the |List_t| |xFreeBuffersList|.

#define ipconfigBUFFER_ALLOC_LOCK() taskENTER_CRITICAL()
#define ipconfigBUFFER_ALLOC_UNLOCK() taskEXIT_CRITICAL()

Some users found it more efficient to use a different method, especially
if NetworkBuffers are never accessed from within an interrupt (also my
personal preference). So those users may use their own version:

#define ipconfigBUFFER_ALLOC_LOCK() myAllocLoc()
#define ipconfigBUFFER_ALLOC_UNLOCK() myAllocUnloc()

How is the porting to Renesas RX62N going? Please don’t hesitate to ask
more questions, it might save you time.

Regards.

FreeRTOS+TCP with BufferAllocation_1.c: does it (still) work?
https://sourceforge.net/p/freertos/discussion/382005/thread/7f1e9a92/?limit=25#82f9

Sent from sourceforge.net because you indicated interest in
SourceForge.net: Log In to SourceForge.net

To unsubscribe from further messages, please visit
SourceForge.net: Log In to SourceForge.net

htibosch · July 4, 2016, 5:39pm

heinbali01 wrote on Monday, July 04, 2016:

[ please do not include the entire text when replying to a message ]

I then included SimpleTCPEchoServer.c into the project to test the
socket interface. My system crashes as soon as I open the echo port by
my linux machine (“telnet 192.168.200.169 7”). I thought that I maybe
get out of memory, but I have >5K left, at least before the crash…

5K doesn’t sound like a lot of memory. The cost of creating a TCP socket depends partly on the config settings. Especially the buffer sizes are important:

    /* Define the size of Rx buffer for TCP sockets */
    #define ipconfigTCP_RX_BUF_LEN                ( 5840 )

    /* Define the size of Tx buffer for TCP sockets */
    #define ipconfigTCP_TX_BUF_LEN                ( 5840 )

In the above case, when communication starts, more than 5KB of buffer space will be allocated in either direction. Give them a smaller size, e.g. 2880.

You might consider using the cheaper option:

    /* USE_TCP_WIN: Let TCP use windowing mechanism. */
    #define ipconfigUSE_TCP_WIN            ( 0 )

And if you do use sliding TCP windows, you can decrease the value of:

    #define ipconfigTCP_WIN_SEG_COUNT                 256

E.g. make it 16 to allow for 16 simultaneous TCP connections.

But even if you run out of dynamic memory, the IP-task will not crash (at least I haven’t seen it happening).

Stack size is 512bytes, so the same as in the demos.

Do you mean 512 words in stead of bytes?
Are you testing from a debugger? Do you have configASSERT() defined and also:

    #define configCHECK_FOR_STACK_OVERFLOW            2

?

Stack overflows, of course, will lead to a crash. Even when CHECK_FOR_STACK_OVERFLOW is defined, a crash may still occur.

Have you tried running the application with e.g. :

    #define ipconfigIP_TASK_STACK_SIZE_WORDS	( 1024 )

The IP-task uses a lot less stack but it can be useful as a quick test.

How much RAM do you have available, 64 or 96 KB?
Maybe you can post your config files (as an attachment please): both FreeRTOSConfig.h and FreeRTOSIPConfig.h?

As a ping works, my first thought was that the MAC and PHY layer works
fine.
That sounds logical. The higher levels will also work pretty soon

system · July 5, 2016, 7:23am

sbrauss wrote on Tuesday, July 05, 2016:

First of all, thank you very much for your detailed answer!!

[ please do not include the entire text when replying to a message ]
Sorry. Standard setting of my email tool…

ipconfigTCP_RX_BUF_LEN and ipconfigTCP_TX_BUF_LEN are set to 1000, each.

When I turn off windowing, it no more crashes, and the “telnet
192.168.200.169 7” connects.

I have 96KB RAM, but my application uses “much ;-)” RAM. I increased the
heap to 24KB, so I have approx. 14KB free before the connection to port
7, still too less for ipconfigTCP_WIN_SEG_COUNT set to 240.

I have seen that in SimpleTCPEchoServer.c the buffers and sliding
windows are set “manually” by FREERTOS_SO_WIN_PROPERTIES. Why that?

But even if you run out of dynamic memory, the IP-task will not crash
(at least I haven’t seen it happening).
It does crash on my system. When I for example open several echo telnet
sessions, free heap memory is decreased, and when it goes under water,
my system crashes.
What should happen instead? - I mean, for example in
SimpleTCPEchoServer.c, there is no error handling if for example
FreeRTOS_socket() has no more memory to create a socket.

Do you mean 512 words in stead of bytes?
Yes…

Are you testing from a debugger?
No.

I tried
tcpspray -v -e 192.168.200.169 -n 2 -b 128
to test the echo service, but it hangs, no data seems to be sent back.
Maybe there is still something wrong in my mac layer…

I have appended my current FreeRTOSIPConfig.h file.

Thanks again & best regards,
Stephan

system · July 5, 2016, 7:46am

sbrauss wrote on Tuesday, July 05, 2016:

It is somewhat strange:
The connection is established, but FreeRTOS_accept() does not return,
and therefore the prvServerConnectionInstance() task is also not created.
Any idea? Maybe I have a fault in the config file? - I have reduced the
number of network buffers to 4 each.

htibosch · July 5, 2016, 9:16am

heinbali01 wrote on Tuesday, July 05, 2016:

Hi Stephan,

I see that you defined:

#define ipconfigZERO_COPY_TX_DRIVER			1
#define ipconfigZERO_COPY_RX_DRIVER			1

That is the difficult way of writing a NetworkInterface.c
If you want you can contact FreeRTOS directly ( see contact ) and send your driver code.

I suppose that you use a recent version of the kernel, at least V8.1.2 ?

Your default TCP buffer sizes are very small:

/* Each TCP socket has a circular buffers for
 * Rx and Tx, which have a fixed maximum size.
 * Define the size of Rx buffer for TCP sockets.
 */
#define ipconfigTCP_RX_BUF_LEN			( 1000 )

/* Define the size of Tx buffer for TCP sockets. */
#define ipconfigTCP_TX_BUF_LEN			( 1000 )

unless you use a smaller MTU, for instance:

#define ipconfigNETWORK_MTU  ( 1000 + ipSIZE_OF_IP_HEADER + ipSIZE_OF_TCP_HEADER )

which sums up to 1040.

FreeRTOS_accept() does not return

which means that the socket hasn’t reached the status eESTABLISHED for whatever reason.

Regards.

system · July 5, 2016, 9:35am

sbrauss wrote on Tuesday, July 05, 2016:

Hi Hein!

I suppose that you use a recent version of the kernel, at least
I currently use 8.2.3.

Your default TCP buffer sizes are very small:
Yes, I know. I increased it also to your 5840, but the behaviour stays
the same. Yes, 1000 is probably no good idea with an MTU of 1500…

Best regards,
Stephna

which means that the socket hasn’t reached the status |eESTABLISHED| for
whatever reason.
Which is strange as telnet of the linux machine brings up the connected
message.

Best regards,
Stephan

htibosch · July 5, 2016, 11:02am

heinbali01 wrote on Tuesday, July 05, 2016:

Somemore text in response to your earlier post:

I have 96KB RAM, but my application uses “much ;-)” RAM. I increased the
heap to 24KB, so I have approx. 14KB free before the connection to port 7

We always have to be very careful when malloc starts to fail
A failure to allocate a certain pointer may have many consequences.

… still too little for ipconfigTCP_WIN_SEG_COUNT set to 240.

As you are using small TCP windows, there will be few outstanding segments. And so ipconfigTCP_WIN_SEG_COUNT doesn’t have to be so high.

I have seen that in SimpleTCPEchoServer.c the buffers and sliding
windows are set “manually” by FREERTOS_SO_WIN_PROPERTIES. Why that?

It is just demo code. It shows that the defaults can be easily overridden by a call to FreeRTOS_setsockopt().

But even if you run out of dynamic memory, the IP-task will not crash
(at least I haven’t seen it happening).
It does crash on my system. When I for example open several echo telnet
sessions, free heap memory is decreased, and when it goes under water,
my system crashes.

As a test, isn’t it possible to give a lot of RAM to the TCP stack? Maybe you can let your application sleep or exclude the RAM-hungry parts?

What should happen instead? - I mean, for example in
SimpleTCPEchoServer.c, there is no error handling if for example
FreeRTOS_socket() has no more memory to create a socket.

FreeRTOS_socket() will return FREERTOS_INVALID_SOCKET if pvPortMalloc() fails.
When a TCP stream buffer can not be created, all subsequent API’s will return an error code like pdFREERTOS_ERRNO_ENOMEM.

Are you testing from a debugger?
No.

But I hope that you have some way of sending logging? Serial? UDP?

I tried
tcpspray -v -e 192.168.200.169 -n 2 -b 128
to test the echo service, but it hangs, no data seems to be sent back.
Maybe there is still something wrong in my mac layer…

A PCAP of the conversation might be handy here.

Regards.

system · July 5, 2016, 1:49pm

sbrauss wrote on Tuesday, July 05, 2016:

I was not able so far to get any further.

As a test, isn’t it possible to give a lot of RAM to the TCP stack?
I have now approx. 40KB free on the heap.
ipconfigNUM_NETWORK_BUFFER_RECEIVERS is 4, and
ipconfigNUM_NETWORK_BUFFER_SENDERS 8, ipconfigTCP_RX_BUF_LEN and
ipconfigTCP_TX_BUF_LEN are 2920
but this did not have any impact.

But I hope that you have some way of sending logging? Serial? UDP?
Yes, over USB.

A PCAP of the conversation might be handy here.
I have captured the packets of a ping and a echo telnet session where I
typed “abc” into the window. Have you a place/email address where I can
send it to?

For me, the captured packets looks fine, but you have for sure much more
knowledge.
Just the echo answers are missing. The contents of the echo reply
packets I think also look fine, nor gargabe. The stack also correctly
replies with ACK packets.
Can it be a problem of the network driver? Maybe it is better to start
looking for the source of the problem at routine FreeRTOS_accept() and
follow it down?

Best regards,
Stephan