Does FreeRTOS have a fair-mutex object or something else that is easy to configure as such? And, do you think that would help with the issue below?
I’ve got a protocol stack of SocketIO over a WebSocket over TLS over TCP(LwIP), with two important threads: a high priority thread reads messages from the websocket and the low priority thread writes to the websocket. The reader repeats: [read a whole frame from the TCP connection, locks the mutex, processes a websocket frame, unlocks the mutex]. But writer starvation occurs when enough data is available in the TCP buffers that the reader thread doesn’t need to block - for a period every TLS read results in enough data to finish the current websocket frame.
I tried swapping the priorities, but when the writer thread has a lot to upload it can block the reader thread.
I can’t post links yet but the upper-layer issue we’re trying to fix in the ESP32’s websocket transport implementation is in the Espressif group’s esp-protocols repo, issue number 625.
No, FreeRTOS doesn’t have a fair mutex. Nothing comes to mind that could easily be configured as such.
It sounds like the websocket is natively not threadsafe – meaning that you cannot mix reading from it and writing to it from different threads. How about a single task that does all the websocket I/O? The task itself can use all kinds of logic to prioritize reading and writing depending on conditions.
The one way.I know to get a “fair” mutex, is to have all the tasks be on the same priority level, then, by using either the round-robin scheduling, or using an explicit yield after giving the mutex, you can avoid the starvation.
I’m not really familiar with you application, but in general it should not be required to have a mutex at all for bidirectional socket communication.
I am using the FreeRTOS TCP module and not lwip. I have an RxTask which reads from the socket and puts the read bytes onto a lockless SPSC. Also there is a TxTask which receives bytes from a second lockless SPSC and writes the bytes to the socket. A higher level task is then in charge of reading / writing to the 2 SPSCs.
I would be really surprised if it was not possible to also do the same thing with the lwip sockets.
no mistake on your side. Semantically, you are perfectly right, of course.
Which causes me to think (uh-oh…): In the use case of the OP, there is something strange going on. Distinct reader/writer tasks on the same web socket object make sense only if the application level protocol is truly full duplex and asynchronous, ie the (single) client can send multiple messages without waiting for an acknowledgement. That is fairly rare, though, except possibly in applications where a connection serves as a multiplexer for multiple clients that are distinguished on the protocol level (possibly this is the way Web sockets work, I do not know).
I’m not really familiar with you application, but in general it should not be required to have a mutex at all for bidirectional socket communication.
mike919192 - agreed. In this case, the lock is used to sequence accesses to the WebSocket client state, since either side can send a WebSocket-level close, as well as sequencing multiple writers - I didn’t consider it until now, but the reader-thread does respond to WebSocket ‘ping’ messages with ‘pong’ messages immediately on receipt.
RAc - that’s exactly what WebSockets do - provide asynchronous, full-duplex framing over TCP or TLS. Either side can send as long as there is buffering room. We’re playing a turn-based game over the WebSocket, as well as exchanging various data(who is online, game status, things like that).
How about a single task that does all the websocket I/O? The task itself can use all kinds of logic to prioritize reading and writing depending on conditions.
That may be the right solution. I’m hoping not to replace or rewrite the Espressif WebSocket client, which is where the reader-task comes from. It doesn’t currently have anything like a command-queue where other threads could send it write-requests.
