An application (server) that makes an active close on a socket, causes that socket enter the “TIME_WAIT” state (as far as I understand) and this state can last a long time. I have read that this is to avoid a late arriving TCP segments from disrupting any new socket that happens to use the some port number (or something like that). That makes sense on the Internet where routing may perhaps delay some part of the data stream.
My application (small HTTP server with WebSocket support) is used on a LAN where “late TCP segments” will not happen.
To limit resource usage on my small system, I would like to drastically shorten the time spent in TCP state “TIME_WAIT”.
On a LAN, the time (“maximum-segment-lifetime (MSL)”) could well be <100 ms instead of being in the order of minutes. I have not determined the time, but I have noticed that new sockets are rejected in lack of resources (server socket backlog limit).
In a dream world, there would a config item: ipconfigMSL_MS(t), but I am not expecting that!!!
How can I make this time much shorter in FreeRTOS-Plus-TCP??
Of course it is possible to shorten the time spent in the TIME_WAIT state. Therefore, I would like to see some conversations that end in TIME_WAIT. If you want you can post a PCAP file. You can filter the interesting packets with e.g. ip.addr==192.168.1.xxx.
I think that you must make sure that the socket will be closed on both sides by closing it gracefully.
I just wrote a very complete example function that makes sure that the peer will be notified and both ends will shut down properly and quickly. See attachment below.
I tested this mechanism by sending thousands of small files to and from a device equipped with a +TCP FTP server. On a LAN it should take a few ms.
Normally, a TCP server will not shutdown a connection, unless a client is miss-behaving or taking too long.
But if you like, you can let the server end take the initiative to close the connection.
So normally, a web-server will never initiate a shutdown (send a TCP/FIN packet).
You are of course right in that a server normally would not close a TCP socket. I may well be that I trying to solve the situation in the wrong way. I will look at your example and digest it.
A clarifying explanation of what I am a bit stuck with is this:
I use Chrome to access the server (/). The root page refers to style sheets, javascript and an icon.
Everything works fine. It all loads quickly and the javascript, which fetches yet another resource (json) using the “Fetch API” (succesfully) and then opens a Websocket with the server. Total time 28 ms.
Everything is happy. The websocket works fine. However, there are still two open sockets opened by Chrome during the load (in excess of the ws). I could say “connection: close” when serving a resource, but the server does not really know whether there are more resource requests coming ( on a particular socket). Three socket are open, but only one (ws) will be used from here on. (Maybe, it is better to ignore “connection: keep-alive” from client and reply “close”, but it feels inefficient). I try to shutdown the unused (http) sockets to free resources (memory and on maximum child socket limit)
AND (here is one complication). if the user (by mistake or intentionally) hit a reload icon or navigates away, I receive a close event on the ws from Chrome as expected ( RFC 6455 ).
As I understand the standard, the server should return the close event (as an ACK, I guess), and then “immediately” close the socket. From what I see in Wireshark, my message with the returned close event is not sent, a RST is sent on the socket but no response is seen from Chrome. Maybe my call to FreeRTOS_closesocket eats up the response message?! I am probably just messing things up.
If I try to connect again (soon after), the server socket will not accept the attempt (RST) because the maximum number of child sockets is reached. This made me start thinking about TIME_WAIT.
The board uses a TM4C129ENC processor so resources are not abundant.
Well, I will now look at your example to see how that enlightens me. Again, thanks for taking the time to answer!
Thanks, Your example helped me to correctly shutdown the dangling socket.
You are welcome.
Maybe my call to FreeRTOS_closesocket eats up the response message?! I am probably just messing things up
Many developers have got mislead by the word FreeRTOS_closesocket(). The function shall only be called after the connection is closed. After the socket is closed, do not refer to its handle any more.
And yes when you close a socket, you also discard the stored incoming bytes. The function that I attached here yesterday should handle it correctly. It will keep on reading from the socket until an error ( eg. -ENOTCONN) is returned.
Some philosophical blabla: TCP (ISO/OSI level 4) ensures that while a connection is open, all data exchanged over the connection is delivered in order; furthermore, it can be ascertained that before true closing of a connection, all pending data (regardless of direction and high level interpretation) is being flushed to that connection’s data storage on the peer side (see below).
Higher level protocols such as HTTP may assume some kind of ordering of data. In HTTP, typically a transaction consists of an upstream (client->server) GET request followed by a downstream response, so after establishing a connection, the server end (timeout based) waits for a syntactically valid GET request, then prepares and sends the response (while the client is timeouting waiting for the response). There is no further acknowledgement of the reception of the response. Thus, as far as the server is concerned, the transaction is complete once its TCP stack has indicated that all data have been flushed to the network (typically under control of the lower level TCP sequencing architecture) and can thus terminate the connection if it wants to.
Yet on the higher (http) level, the transaction may not have been completed successfully - the client may have fallen into a timeout while waiting for the response or too busy doing something else (not having flushed all bytes from the connection storage to the application storage), or the response could not be evaluated, for example due to an incorrect length indication - with the underlying TCP nevertheless having deposited all response data to the connections data buffer, so on the TCP Level, everything looks good. TCP is satisfied, HTTP is not (more likely one side of a HTTP exchange).
There is nothing one can really do about that except attempting to gracefully recover from that - extending the protocol to stipulate an upstream ACK does not help because then the problem will reappear - that ACK may get lost, requiring another ACK for the ACK. Repeat infinitely.
A number of more or less subtle problems stem from this clash between protocols on different levels having different understandings of when a transaction is completed. For example, the return of a send() call in many TCP implementations does not necessarily indicate that the peer application has received all the data, only the peer’s socket. If the server sends a response packet of size 2k AND the peers receive window has > 2k space, the send() will return success and 2k as the number of bytes sent. If the peers receive window is down to, say, 1k free space, the send may (depending on setup) return success but indicate only 1k data delivered (in which case the remaining 1k must be resent on application level - amazingly few people know that and do NOT evaluate the bytes sent return value) OR failure. If the connection has been closed in the meantime, this may or may not result in an error return from send(). So if a HTTP server is coded to be satisifed when the response send() in a single chunk returned success, it may result in a faulty communication, with the client still waiting for a partial response which the server will not send.
Hi! Thanks for your thoughts. BTW: I do check the return value from the sends. This little server only acts inside a LAN and is fast. The total load time for the html, css, icon, additional json fetched and the establishment of a WebSocket for interaction takes less than 30ms. No browser is likely to timeout. Thanks for your interest!