But I need the smallest representation of the data. I can’t use the max length all the time. If there are only three bytes, obviously I only want to pack that and retrieve them on the other side.
I currently do this by calculating where the end of the array is and loading just up to that point.
It seems to only work because my array is at the end of the object. I can stop after object+three bytes. This doesn’t seem portable or “safe”.
What if I wanted to store two smallest representations of arrays in a single object as a single messagebuffer entry?
I think one option would be to have the stream load my object, then separately as another load my data as just a standard array. When then when pulling I always attempt to pull two things as a pair from the messagebuffer. I can’t then mess up and load an object without it’s data pair being next, but I could probably detect that in a couple different ways, and there is an overhead cost of two entries in the buffer now.
I should know the answer to this, but I’m a little burnt out so any best practices would be appreciated.
Sorry, its not fully clear what you are wanting to do. If your data[] array is only using 3 bytes, does the len member of the structure hold 3? If so then you could always check len matches the number of bytes expected when reading from the message buffer.
There are potential portability issues here as I think you suggest. One is to do with the size of the structure - which is not necessarily going to be 1 byte for the id, plus two bytes for the len plus however many bytes are in your data array as there could be padding bytes too - and the number of padding bytes can depend on the compiler options even. You could tell the compiler to ‘pack’ the structure so it is always the same size, or alternatively use sizeof() to obtain its full size and just subtract how many bytes fewer you are actually posting.
I have messages coming from another system, ultimately I need to serialize them and send to another system. This is fine, except that I need to process and store all the incoming components before I can encode a large outgoing message.
So I need to store objects with data lengths of 3, 5, 200, 1000, etc. This gives me a problem with how large the messagebuffer will need to be, but I can handle that.
The issue is I don’t think it’s entirely correct to just put a data structure at the end of my object and just tell the messagebuffer_send to truncate it after as many bytes as I know will exist.
Even if I’m packing the structure, it seems questionable because it only works with the array at the end. If I had two arrays in my structure and wanted the smallest representation of both, that’s clearly not going to work.
So what is the right thing to do? Include pointers and lengths to some more-static bytes somewhere? Send the actual data as a separate entry following the object entry? I must be missing something because this seems like a common issue with variable length data in C and FreeRTOS.
Assuming the two buffers are of the same C type, I would just make one array that had both messages with the header part saying where the break was, and have the array at the end of the structure.
So. It turns out my hacky idea of putting the array at the end and truncating it is a couple things. It’s not “safe”, but it’s no worse than anything else C let’s you do! And it is a real thing, flexible length arrays. https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
I’m still not sure if there is an issue or best practice with message buffers. I confirmed the base idea does work. I can load three bytes of data in, but when I get the data out I need to use the largest actual buffer to store it because I won’t know how big the data is before I pull the message (this is a case for storing a structure first and the data separately).
And yes, if you needed two variable length arrays I guess you would put the data in one and include where the “split” is.
Note, what you are doing is a bit different than the Zero-Length Array technique. By having the full array in your structure, you allow yourself to create a variable of that type on the stack or statically, abet it will alway take the maximum size. If you declare the structure with a zero length array at the end, variables of that type can only really be created on the heap, and the call to malloc needs to add the amount of space needed for the flexible array allocation.
Note, that by allocating more array space than you need, you do NOT create the undefined behavior that the non-standard variants of the flexible array create, so it is ‘safe’ (as long as you don’t try to access elements in the array that were not set).
In the example above I do allocate. In consideration this will work so long as I can allocate one buffer to receive into and keep reusing it. If I’m clever about the work I won’t need “true” flexible length and malloc. The part I am using is the putting the array at the end, and “cutting it short” on the buffer send().