v5.3.1 Starved Tasks

jwestmoreland wrote on Sunday, June 28, 2009:

I’ve tried using the search function - but I guess that either doesn’t work so good anymore or I don’t know where the one is just to search this forum.

I am using Preemption - and it seems that tasks that are just 1 level difference in priority are getting somewhat starved - has anyone else noticed this?

Thanks,
John

richard_damon wrote on Sunday, June 28, 2009:

For a task to get time assigned to it, all task of higher priority need to block, priority is strict. Preemption affects task with exactly the same priority, causing them to run in a round robin manner, it will not force a task to yield to a lower priority task.

rtel wrote on Sunday, June 28, 2009:

There is a menu system going from left to right at the top of the page, starts "Summary - Tracker - Forums - Code - etc.".  If you hover over Forums there is a search link.  These forums are a bit crappy but SourceForge are replacing them with a completely new system soon.

Regards.

rtel wrote on Sunday, June 28, 2009:

V5.3.1 introduced two new ports, but didn’t change any of the scheduler code, so I don’t think the issue (if there is one) is related specifically to the V5.3.1 version.

John - can you provide a more details of the scenario you are seeing?  How many tasks are running, what are their relative priorities?

Regards.

jwestmoreland wrote on Tuesday, June 30, 2009:

Richard,

27 tasks running on an MSP430 - maybe that’s pushing it…:wink:

I had priority levels set to 7 - cut that back to 4.  I saw tasks with only a different priority 3 vs. 2 getting starved - or at least I think that is what I saw.  Kernel reported 27 tasks running.

I guess I should say this as well - the processor I am using has some significant errata - and some is when you are running the debugger with the processor in LPM (low power mode) - so it is not outside of the realm of possibility that the tasks are running but in the debugger it appears they aren’t.

jwestmoreland wrote on Tuesday, June 30, 2009:

Sorry - somehow the last msg posted before I was finished.

Obviously I can try disabling LPM mode and giving it another try - and/or reduce the # of tasks and see if that affects what is going on.

Thanks,
John

enginayd wrote on Friday, July 03, 2009:

Actually this is quite the problem I’m having. FreeRTOS works as described, lower priority does not get any slice of the time unless all higher level tasks block.

I thought it would get a proportional slice of time according to its priority level, but it does not.

So I have a task (which is the main purpose of whole application btw) that needs to run as fast as possible. So I think I have two choices;

1) Delay the task;

Delaying task is in the resolution of SYSTICK interval, which is 1ms at the moment, and even that is said to be faster than necessary. 1ms delay for that task is inacceptable. So this option is not viable.

2) Yield

This looks like the perfect solution. It would request a context switch process others. But this doesn’t work as expected. Because I believe that in the context switch, FreeRTOS again choses highest priority task to run.

So what is the best way to build a task that consumes ALL available CPU time while don’t let other tasks to starve. Maybe suspend the high priority task, and then resume it from other tasks ?

Besides, I moved my watchdog petting in IDLE task, so that I’m sure that the system runs stable. If a starvation occur (idle never runs) that I reset the device, with the highest priority task turned off. So use can fix the problem. Is this a good idea ?

richard_damon wrote on Friday, July 03, 2009:

A task that want to consume all available cycles and not starve other tasks must be put at the idle priority.  Priority doesn’t mean how much of the CPU the task will get (at least as far as FreeRTOS), but how urgent the task needs to be run.

Yield only is useful if you have multiple tasks at a given priority.

If you have a task that need to run before some other task, but also wants to consume all available CPU resources, then you have an error in specification, you need to split the task into two pieces, one that needs to be done urgently, but has a finite resource need, and a second that can be placed at idle priority that consumes all available CPU time.

enginayd wrote on Friday, July 03, 2009:

Hello Richard,

Thanks again for your prompt and informative reply.

Yes, FreeRTOS scheduler concept is a bit non-traditional with respect to modern OSes, hence the confusion I believe.

Imagine a situation like this;

1) Ethernet task
2) Serial communication task
3) User program

Now, user program needs to run as fast as possible with all available horse power and is very important (the main purpose of the whole application).

If I put this task at a higher priority level, it will consume all CPU and no other tasks can run (SYSTICK rate delay, 1ms, is not acceptable here).

If I put this User Task in the same priority level of other tasks (ethernet, serial) they all will work… but if there’s excessive use of ethernet or serial communication they will make User Prog run slower (I don’t want this).

If I put User Task in Idle priority level, the above problem will occur again.

What would be the best solution ?

Kind regards,

Engin

aturowski wrote on Friday, July 03, 2009:

The answer is that all depends on project requirement.

I would (almost) always assume, that communication tasks are more important, because usually one is after data throughput. So the user program can run on lower priority, and that way it will get all awailable CPU power not affecting throughput of data processed by Ethernet and serial.

If it is your use case please add the estimate of Ethernet and serial cpu power needs and extract it from total available cpu power. This will give you and estimate, how much cpu power user task will get. If it is not enough, you should either consider using faster CPU or changing requirement for projects.

Hope that help,
Adam

richard_damon wrote on Friday, July 03, 2009:

This is a spec issue. The communications tasks will need to have a certain amount of CPU time to do their job. PERIOD. If a message comes in, it must be processed or dropped. It also must be processed within a certain amount of time, or you will over fill your buffers and lose info. If they can not take this much time and still leave enough time for your main task, than you don’t have enough CPU to meet your requirements.

One possible solution would be to have the communication tasks (or ISR) see if they are starting to get too many messages in a short period of time, and if so throw away the extra messages and send a "shut up" reply back.

enginayd wrote on Friday, July 03, 2009:

In this application the most important thing is the User Programmed task. It checks inputs and performs login on them and update outputs (this are electrical outputs).

If communication has higher priority, they will make the user program slower during heavy communication… hence yield a instable outcome.

enginayd wrote on Friday, July 03, 2009:

And Richard,

I want to leave the throttling to the protocol… i.e. TCP will understand that it didn’t receive ACK and other end will end up slowing down the connection. Same applies for serial transmission.

Is that non-sense ?

aturowski wrote on Friday, July 03, 2009:

I see. So your "user application" is really real time measurement/output stuff. In that case the solution is a follows:
-  the actual measurement/outputtind data  is done on timer driven interrupt
- processing of these data is done in high priority task
- Ethernet/serial protocols runs as lower priority task and these protocols have to have throttling of data build in.

That way your "user application" task will get as much processing power as it needs and Ethernet/serial task will get the rest and that amount . But Ethernet/serial interface throughput will adapt without loosing the data.

richard_damon wrote on Friday, July 03, 2009:

You have stated that the user program is to run "as often as possible", this is incompatible with it being a high priority task, this implies that if the comm is lower priority then this, that it should be run and it is right that it starves.

I think the issue is the "as often as possible" is incorrect, but should be defined as needing to be run at least at a certain rate. (I am assuming that it is some form of loop that has distinct cycles). If this can be defined, then you can record the time the loop starts, and when a loop finishes, you can see how long you can delay till you need to run again, and delay that long, letting the comm task use that slack. You can also base the time to start the next off of the time you scheduled the last to start + your interval allowed.

If you want it to run faster than this rate if the comms aren’t using time, then rather than just delaying, wait on a semaphore for with a may as computed above, and have a small task at idle priority signal the semaphore when it gets time. (The main computation task will want to first take the semaphore with 0 wait, then again with the desired wait to make sure there is time available NOW).

aturowski wrote on Friday, July 03, 2009:

It makes sense.

enginayd wrote on Friday, July 03, 2009:

Adam exactly!

My previous design was to run the user app via a timer interrupt. While it is a rock-solid solution it has its draw backs. In our application user can download both very small and very big applications. And unless you do some complex design you’ll end up configuring your timer for the biggest application.

i.e. smallest app could take 30us and you can run it via 20KHz timer. In that case it will consume 60% CPU and everything is fine.

But a bigger program can take 200us… so you have to run it in around 4KHz timer.

So you are bound to worst case with that design and you’ll end up running user prog like at 4KHz and a small program could (which could be run much more faster) will be bound to that speed.

To overcome this drawback, I thought I could run the user app in a task so it will use as much as possible. But I think the scheduler is not very suitable for this kind of design.

Kind regards,

Engin

richard_damon wrote on Friday, July 03, 2009:

I can see a couple of possible solutions to your problem.

1)  Put comm task above user in priority, and add code there to limit CPU that can be taken with forced waits.

2) Make the user define a repeat period they want from their program, and if it starves the I/O accept it or back off the rate some.

3) Have a wrapper around the user task and measure the time it takes and adjust the rate it runs at based on this (something like wait for 10% of the time it took to  run that loop, that give 90% to the user and 10% to comm)

rtel wrote on Friday, July 03, 2009:

> Yes, FreeRTOS scheduler concept is a bit non-traditional with
> respect to modern
> OSes, hence the confusion I believe.

It’s a fixed priority preemptive scheduler - about as traditional as you can get for this class of RTOS.  [fixed priority meaning the scheduler does not change the task priorities itself, not that application writers cannot change the priority].

Regards.

aturowski wrote on Friday, July 03, 2009:

I would go for third solution because:
- it needs almost no additional user involvement than providing task code
- it allows you to easily adapt to (almost) any user aplication giving the best performance
- your program still controls everything and can preserve say 10% of CPU for transmission tasks. It can also output the user message saying: "Your task is running with speed x kHz". If it is not acceptable for user, he can refactor his code.

This is my personal opinion. The final solution of course depends on specification. And last thing: scheduler is really applicable to this kind of solution.

Regards,
Adam