V5.0.2 Trace Hooks Question

jhenshaw wrote on Tuesday, July 08, 2008:

I’ve implemented a logging scheme that logs errors to a serial terminal. Some of these come from trace hook macros like Ques Send Failed and such - mostly I’ve just implemented macros for failure conditions. Since then, I’ve had endless trouble with my app crashing, typically by ending up trying to run code located at 0x0. It finally occurred to me that requesting service from the kernel like QueSend() to place characters into the serial transmit queue might be a bad idea. (Duh.) And, forcing my Log() function to return before doing anything does indeed cause my app to go back to running OK.

So, can I call kernel services from within these macros? Or is my problem (possibly) that some of these macros are called within IRQ (or, since it’s an Ateml AVR32 UC3A0512) or the Supervisor exception context?

rtel wrote on Wednesday, July 09, 2008:

How are characters sent to the UART?  When you say "place characters into the serial transmit queue" are you referring to a FreeRTOS.org queue?  If so then you definitely cannot call the xQueueSend() function from within an ISR context, you must instead use the xQueueSendFromISR() version - but I think you have been using FreeRTOS.org long enough to know that already.

It was my intention when adding the macros that they be very short, returning quickly.  I would have to inspect their placement to know the effect of calling another queue function from within a macro, especially with regard to whether the scheduler was suspended or a critical section entered when the macro gets executed.

A couple of easy things to check are 1) You are not simply running out of stack space by calling the extra code from within the macro, 2) You are not getting into some form of recursion whereby a queue API fails, so you call the error log function which calls a queue function, which fails and calls the error log, etc.


jhenshaw wrote on Wednesday, July 09, 2008:

Hi Richard,

Yup, chars go into a FreeRTOS.org queue. Inside the ISR, the queue ISR versions are used for the xmit & receive queues. In the ISR, I receive from the TX queue, but traceQUEUE_RECEIVE_FROM_ISR_FAILED remains unimplemented. When I receive a character, that’s pushed into the RX queue and I’ve implemented traceQUEUE_SEND_FROM_ISR_FAILED to call my Log() function, which then dumps chars into the TX queue. So I COULD be getting recursive errors, not sure if I am though. Unfortunately, the darned trace facility in the debugger always loses data when the jump to 0x0 happens, so I can’t tell exactly what’s going on. I assume somebody’s TCB is getting overwritten, but I can’t tell whose or when.

I think I’ve solved my problem, or at least a major one. I added detection of being within an ISR to the Log() code that pushes chars onto the queue, so I’m now calling xQueueSendFromISR() when Log() is called from an ISR, that seems to get things running again. I had actually implemented this earlier, but other problems masked this issue and so I backed those changes out since they didn’t appear to keep the app from being killed. Your post spurred me to re-examine & re-implement this, thanks !

However, I take your point about short & quick within ISRs, which calling printf() definitely isn’t. I think I’m going to rework this code so I can avoid text formatting within the ISR and do it later within a task. That will also eliminate any recursion problems since I can check the queue free space before placing chars into it.

Thanks again!

edwards3 wrote on Wednesday, July 09, 2008:

Depending on your library, printf can be massive, use lots of stack, even attempt dymamic allocation, and be non reentrant.There are some cut down versions of sprintf in the FreeRTOS zip file that might help.

jhenshaw wrote on Wednesday, July 09, 2008:

I’m using Newlib, and I’ve seen a stack depth of about 9 calls deep from printf() to my low level write() routine that actually puts bytes into the queue. And I found the sprintf in the FreeRTOS.org disti - mainly because I’d forgotten I’d included it as part of the demo code months ago. Imagine my surprise when I traced a sprintf() call back into my own code rather than out to the runtime library! That was embarrassing. :slight_smile:

The library mallocs are protected by a locking mechanism that ends up calling vTaskSuspendAll(), but I note that the FreeRTOS.org docs declare this as illegal from an IRQ, so I’m liable to still be in trouble there. I’m going to move all use of printf()/scanf() and their cousins out to the task level, and then further guard them with a Mutex so only one task at a time can use them. In addition to keeping me re-entrancy safe, that should keep printf()'s from multiple tasks from getting their output all woven together.

Sigh. A new OS, a new processor, a runtime library that I can’t rebuild and can’t tell what options were used during the build, all to build an app in an industry I’ve never worked in. Lots of fun, but it’s hard to discover & remember all the rules!

Now if I could just figure out why the UART is issuing receiver full interrupts when the durn cable isn’t even connected to the board…

Thanks !