Storing and retreiving data from a structure

dibosco wrote on Wednesday, September 25, 2013:


I have a large structure of IDs that I need to store values to and retrieve values from. In FreeRTOS terms, what is the best way to do this? As far as I can see it I could do a couple of things:

Store them in a task then have a queue that I can both read from and write to in the following fashion:

void FooTask

struct BigStructOfUIDs UIDs;

while (1)
    // Read from or write to the UIDs structure.   
    // If reading from, write the read data into another queue



PortMapData is a structure of the type below.

struct PortMapDataStruct

uint8_t uid[TOKEN_UID_LENGTH];
int16_t CallType;
int16_t CallingRoutine
uint8_t Address;
uint16_t Port;


So I use the above structure to get data in and out of the task. If CallType = STORE_UID I write what’s in uid[] in PortMapDataStruct into the task’s UIDs struct. If it is CallType = READ_UID then I get the data out of ask’s UIDs struct and put it in a PortMapDataStruct which is then sent to a different queue, depending on what the value of CallingRoutine is. The reason for the CallingRoutine is that if three different tasks could all want a particular ID sent to them, this task I’m describing needs to know the correct queue to place it in.

This seems rather convoluted and I’m wondering whether I’m better just having a normal function with my large array of IDs as a static variable, then simply passing a pointer to struct PortMapDataStruct in and out with a call type that either stores data from the passed PortMapDataStruct or puts data in to it. I think I would need to protect it by having a semaphore that any task that needs to write to or read from it would have to have before accessing the function and therefor the data.

So it would be like this:

void BarFunc(struct PortMapDataStruct *Data)

static struct BigStructOfUIDs UIDs;

if (Data->CallType = STORE_UID)
    // Write data here from data->uid[] into static struct
else (Data->CallType = READ_UID)
    // Read data out of static struct and up it in Data->uid[]



…is there a better way again that has not occurred to me?

Many thanks!


edwards3 wrote on Wednesday, September 25, 2013:

Although not following all your logic I would concur it seems convoluted. I would go for the second method. If the operation is quick then you could just protect the structure with a critical section. If the operation is slow (maybe you have to search for the ID?) then consider either a semaphore or suspending the scheduler depending on the rest of your design.

dibosco wrote on Thursday, September 26, 2013:

Thanks. The operation will be very quick, it’s just a look-up table - either storing or retrieving - with indexes and IDs passed in and out. Method two, or something similar, is how I’d do it in run-to-complete software, but I was wondering really whether there was a better way of doing it with FreeRTOS.

The whole issue of storing and retrieving globally available values without using global variables in C seems fraught, with many conflicting opinions. Especially so in embedded C. I quite like the method of simply having a function that returns a pointer to a static variable quite neat; however, I have seen some people have vehement objections to that varying from “it’s a pain in the **** passing pointers all over the place” to “it’s dangerous in re-entrant code”. Personally, I dislike file-wide static variables that any function within a .c file can access, it seems very nearly as nasty as global variables to me.

Up until now in FreeRTOS, I have managed to pass everything around in queues and I like that method, but this project is just a little too complicated for that I think. I was just trying to get a feel for whether other people have come up with better methods of doing what I am trying achieve.

I think I will probably go for something like method two with a semaphore, so only one thread at a time can access it.

Thanks again.

richard_damon wrote on Thursday, September 26, 2013:

Having a function that returns a pointer to a local static really isn’t any different than just having a global variable, unless the function does something like initializes the local on first use.

As far as multi-thread access, passing around pointers doesn’t help if multiple tasks have the same pointer values.

Normally, non-atomic global accesses in multi-threaded code need some form of interlock. Note that atomic accesses generally do not, unless some other thread might be modifying that value non-atomically.

What method is used depends very much on how slow the accesses might be. Using a semaphore/mutex is a good base, not interfering with interrupts or other tasks, but can be a bit “heavy” at times. A critical section can be better if the access will be quick, as it is very light weight, and if the time in the critical section is brief, its impacts on interrupt responses in minimal. The one limitation is that if the read side uses critical sections, the write side can’t use a semaphore to block it.

Disabling the schedule is a middle ground, in actual operation it itself can be heavier than a semaphore/mutex, as it blocks ALL other tasks, but it has the advantage that it does work well with critical sections, making it good for a few long writes/many quick reads situation.

Creating a task to serialize access is even WAY heavier. Unless there is a reason you might want the request to be “asynchronous”, I wouldn’t do it.

If your accesses might be somewhat complicated, and here it seems that it conceptually might be, funneling accesses through an API layer can make sense. In my own practice, I rarely make a single function for universal access with an opcode field, but multiple functions accessing a file static structure, if only because the single function requires work at every call to set the operation, and work in the function to figure out what is to be done. It also says that the reader and writer functions can have different signatures if needed. (For the same reason I hate the style of “put all the parameters in a struct and just pass it”.

dibosco wrote on Thursday, September 26, 2013:

Thanks for your thoughts, Richard.

A couple of points. I’m not sure a function returning a pointer to a local static is the same as a global as you have to access that function to get at the data. However, I suppose the point you’re making is that any thread at any time can get hold of that data which is why it is dangerous. If you locked access out to other threads with a semaphore I’m not all sure it’s so bad.

I have real issues with the file-wide statics, because if you have a number of functions within a module that access the file wide static, it seems just as hard to read (for a person reading it back, not a computer) as having a global as a number of different functions can still access that variable with gay abandon!

I have tried in the past, with other similar variables, having a file that holds a structure as a static variable and had various functions that access that variable to get data in and out [of the static structure]. This method seems like C++ style accessor functions to me. In many ways this produces the neatest, simplest code, but something just gnaws away at me seeing lots of functions access what is a “file-wide global”. I know that’s a contradiction in terms, but it just seems to me that a file-wide static is one small step away from a global and is just a “file-wide global”. Yet I can’t really see a way around it other than a function having a local static variable and multiple call types to ferry data in and out. I have tried this latter version with APIs that call the function that holds the local static (using a structure to pass the data in and out of the function with the local static!) and in many ways I really like this; however, it is the least easy to follow when you read it back. (I have no issue at all with passing lots of variables in a structure, yet have come across many like you who dislike it. Funny how different things seem better to different people!)

In my current case in point, accesses are fast, both reading and storing and I could happily disable access or have a semaphore. (The latter seems best to me as interrupts cannot access the data directly.)

I’m not sure whether what I am trying to do is that complicated. I have to read from and write to from different threads and in different chunk sizes. Maybe I should just go with the file-wide static, have semaphores and suck-it-up when it comes to not liking multiple functions all directly accessing the same file-wide static variable. I am going round in circles with this one and have been for quite some time. (Whether with an OS or just run-to-complete code.)

It’s good to mull it over though, so your thoughts really are much appreciated.

richard_damon wrote on Friday, September 27, 2013:

The function returning the pointer to a local static is just like a global because once the pointer gets out, you have the same issue of locating what can access that variable as if it was a global, you can do something before giving out the pointer, but you can’t know when they are done with it.

The problem I have with using the “structure of parms” to pass in and out of a function, is you get zero support from the compiler to detect the missing of filling in a member of that structure on a call (and I have seen may bugs caused by this from people who like to do this). It also adds a lot of code to your program. The only case where it saves space is if this block is passed through several levels of calls, which should be unusual.

As to file statics, a source file should represent a single concept, That concept may require some shared state to implement, which naturally will at time be best represented with a file static object. It shouldn’t be hard to define and document what that variable does and this can be verified by checking that single file, and every time you touch the code of that file, you can look to make sure you have maintained that definition. Generally in the file there normally aren’t that many points in the file that will access the variable. If there are a lot, it may be worth looking if many of these represent some common code, and extract that into a function.

Compare this to a program wide global, to verify that its rules are obeyed, require a inspection of the full program. This is a major task as you might not even know what files that will contain. Hiding the file static as a file static with an access function doesn’t really simplify the analysis if it is a thin function.

dibosco wrote on Tuesday, October 01, 2013:

Sorry for the delay, long weekend…

Is the issue here that although any part of the program can call any of the functions in a particular .c file that access the data, the fact that only these functions are able to actually modify the data means you are able to completely control how the data is changed?

In other words a function such as WriteUID(uint8 *uid) allows you to write a new UID from absolutely anywhere, but at least if you are forced to call this function and not just pass a pointer back, then you have absolute control of how the UID is stored? For example, it stops a rogue piece of code writing way over the end of the UIDs’ segment of memory?

Or is there another reason as well as this?

I’m still not quite clear about function reuse if a function just accesses a file-wide static rather than being passed a variable. With the latter, it means the function is tied to only that data. This might become a lot clearer as I use your way of working though. :~)