Stream Buffers vs Message Buffers vs Queues

Can someone please explain conceptually the difference between Stream Buffers, Message Buffers and Queues? Specifically, why would one choose one versus the other for inter-task communication or IRQ-task communication?

I am aware of this post and the information it links to but I’m looking for a more conceptual explanation before digging into the API/low-level details. Thanks!

1 Like

In case you are not aware, there are pages on the site that describe each in some detail.

Short reply as using my cellphone.

Stream buffers carry messages that don’t necessary have a beginning and end, hence stream. Thing of a series of bytes arriving on a serial port.

Message buffers hold discrete messages that have a fixed size, but each message does not have to be the same size (variable sized messages).

The above two are light weight implementations that are fast but have usage restrictions documented on the website.

Queues are much more sophisticated objects with multiple senders and receivers that are held in priority lists. Each message sent to a queue must be the same size, the size being specified when the queue is created. The website also documents techniques
to use queues for variable size messages to, but only indirectly.


Thanks Richard. Based on your response and my initial reading, I guess the idea is if there is a queue application with only one sender and one receiver (one task to another or an ISR to a taks), then it is probably better to use a stream buffer or a message buffer since they are both lighter weight, correct?

I am also guessing that the main difference btw stream and message buffer is whether the reader task need to be triggered based on fixed data size (trigger level) or variable data size (message length).

Let me expand a bit on Richard’s reply, based on being a user (as opposed to the designer). One big difference between the Queue and the Buffers (Stream/Message) is, as was mentioned, that Queues support innately multiple senders or receivers while the buffers are somewhat lightweighted by having a restriction that at any given time only 1 task can be sending to it and 1 task can be receiving. You can get around this problem a bit by wrapping the access (on a given side) in a mutex, which ensures that you can’t get concurrent access, but also breaks somewhat the priority model as if first a low priority task tries to read/write a message but gets blocked, a higher priority task can’t get in front of that in line, due to the mutex. With a Queue, the fact that the queue itself handles the multiple access says that it can order the access based on priority.

Besides that, if we look at what sort of thing each can send:

  • A Queue deals with a fixed sized message, generally a struct but it could also be a base type including a pointer.
  • A Stream Buffer deals with messages as streams of bytes. A Message will be put in the queue as a unit, and doesn’t have any fixed size (just a maximum as it should fit the buffer), and when pulling out, it doesn’t need to be retrieved chunked the same way.
  • A Message buffer is internally built on a Stream Buffer, but is designed to handle a variable-sized message that is held together so it comes out in the same chunks as it went in.

Personally, if the message is always a fixed-sized, i.e. always the same structure, I will use a queue and not a message buffer, largely because I use a C++ type-safe wrapper which doesn’t work for the potentially variable-sized message of a message buffer.

The difference between a stream buffer and a message buffer is that with stream buffers, the message is mostly amorphous, the messages really are just a stream of characters or bytes, and there isn’t a strong need to keep messages distinct. A Messgage buffer deals with messages with definite structure, but perhaps after a common header to the message, there is a variable-length data block based on some field or fields in the header section.