glenenglish wrote on Wednesday, August 17, 2016:
HI
I wonder if there is a method, or preferred method for dealing with access to the network device.
I am using a SPI device, and there needs to be mutex style behaviour between the HandlerTask prvEMACHandlerTask and whoever calls xNetworkInterfaceOutput. ANy light on this would be useful. Just using a mutex around the access sections would be dangerous- priority inversion I am sure would reign. it would get stuck.
regards
glen english
rtel wrote on Wednesday, August 17, 2016:
The best method might be to ensure only one task accesses the SPI bus,
so no mutual exclusion is required, effectively handling sending and
receiving data across the SPI bus from a single task. Alternatively, if
you have only one sender and one receiver a simple mutex might be the
simplest way, as with only one sender and one receiver it would be easy
to manage (you can prioritise the sender over the receiver to prevent
too many packets being queued for Tx at any one time).
Can you describe how you have allocated sending and receiving data to
tasks and interrupts?
For example…
Sending:
xNetworkInterfaceOutput() should only ever be called from the IP task
itself (correct me if I’m wrong!), and not application tasks, so you
don’t need to worry about that part.
xNetworkInterfaceOutput() will initiate an SPI communication - and then
presumably the actual data transfer will be handled by an interrupt or DMA.
Does anything else in your application send data to the network? For
example, could an interrupt generated when one packet has been sent
initiate the sending of the next packet?
Receiving:
Presumably there is only one place in your code that is receiving
packets over the SPI, then sending the received packets to the IP task
for processing. Where is this being done? For example, do you create a
task that is triggered by an interrupt when new data arrives, and have
the task manage the reception of the data?
richard_damon wrote on Wednesday, August 17, 2016:
When I have a controller that is used by multiple tasks, because it is used to talk to multiple ‘devices’ (like a multi-channel SPI or I2C bus), then I find the best option is normally to just use a mutex in the code for the controller so which ever task is using it to talk to their device will have exclusive access to that device for the period of the transaction.
Yes, you do get an inversion issue if a low priority task is doing a transaction when a high priority device wants to do an operation, but this is fundamental to your archicecture. The mutex will assure that when the transaction finishes, the highest priority operation will be the one that starts.
A shared resource mrrding atomic operations (the controller) used by multiple users inherentlyy has the inversion problem, since, at least in general, we need to let the current operation complete before you can start the next. If this causes unacceptable delays, then you some measure to mitigate. Some options:
glenenglish wrote on Wednesday, August 17, 2016:
Hi All
Yeah it’s a problem (multiple access of peripherals) I am well familar with. Just wondered what was suggested in FreeRTOS+TCP land.
Given that prvEMACHandlerTask could be responding to interrupts (and working the SPI) ( and remember that prvEMACHandlerTask offloads it to IPTask when it can got it) , the SPI must be guarded.
I did try a mutex when I considered the problem right in the beginning, but it got stuck, so I will need to understand the scheduler a bit better. There are a few other syncronization primatives I can use also that work a little different. Other flavors of semaphores etc used as resource counters. But they are usually 'backward to what I want" (they unblock >0). (but I could just use them backwards)
Yeah there will be an HTTP server for one client only (TCP) , and a bunch of fast UDP bidirectional traffic (approx 500 pps full to the brim) . Gratuitous broadcasts are really the scurge of networks…
with thanks
glen
rtel wrote on Wednesday, August 17, 2016:
A mutex should not get ‘stuck’ - are you sure you didn’t get stuck in a
configASSERT() failing? Which would happen if, as an example, you used
the mutex from inside an interrupt.
glenenglish wrote on Wednesday, August 17, 2016:
Hi
Working now with original MUTEX. was getting itself tied up in knots elsehwre and had both tasks unable to run…
No, not in a ASSERT loop, somewhere in the scheduler though, can’t remember exactly where.
Which I had written that down…It was in xQueueGenericReceive and it was there every time I stopped the processor. There is only right now the idle, the IP task and the prvEMACHandlerTask.
The FreeRTOS really is sensational. I am very impressed with its engine room. Also, the +TCP is well written easy to understand , also. (unlike LwIP) .
Well, if anyone is interested, now I have a working driver / ntegration using the ASIX 88796C chip. ip4, ip6, 40 Mbps SPI…quite a good chip used for a couple of years, very solid
Gimme a month and I will do some doco and post it publicly.
Need to spend more time reading/ studying the scheduler and interrupt priority variations.
cheers
PS
ah ha
here is where it fails
configASSERT( pxTCB == pxCurrentTCB );
line 3529 tasks.c
see below next postglenenglish wrote on Wednesday, August 17, 2016:
here is where it fails if it is worked a bit more (not really hard, though- I will drive some pins and see how busy it is in IDLE. putLEDs on it. Nope not very busy at all.
this is more sinister.
configASSERT( pxTCB == pxCurrentTCB );
line 3529 tasks.c in "xTaskPriorityDisinherit () "
Somethign for me to look for on the weekend when I am back.
pxTCB (the mutex holder)
Task : IP-task (currentTCB is EMAC)
MutexsHeld : 1
priority 1
base priority : 1
eNotifystate : eNotWaitingNotification
I’ll have a look at this later. I need to make sure , double sure and triple sure that the interrupt cannot occur while it is being (deferred serviced)
It is possible that the interrupt could occur that (would usually) unblock the thread (prvEMACHandlerTask) and also generates a Yield from ISR while that segment is holding onto the mutex.
Now, I removed YieldFrom ISR and it still happened.
So , in prvEMACHandlerTask , I moved where the interrupts get unmasked outside where the mutex is given up.
Nup stil happens.
ahhh this time , config-assert at the same location, but
pxTCB (the mutex holder) == EMAC
Task :EMAC (currentTCB is IP-task)
MutexsHeld : 1
priority 6
base priority : 6
eNotifystate : eNotWaitingNotification
anyway, it’s after midnight. more news in a couple of days
cheers
rtel wrote on Wednesday, August 17, 2016:
configASSERT( pxTCB == pxCurrentTCB );
line 3529 tasks.c in "xTaskPriorityDisinherit () "
Asserts are placed to stop mutexes being used in interrupts, because,
due to the priority inheritance, mutexes (unlike other types of
semaphores) are associated with tasks.
If this is what you are doing then there is a work around I keep meaning
to document…which I could describe here.
glenenglish wrote on Wednesday, August 17, 2016:
Although, the mutex is not being used in an interrupt. Although it is ASSOCIATED…
Here is the code segment, (simplified)
void EXTI3_IRQHandler(void) /* handle packet or link change interrupt */
{
//LED2_ON;
EXTI->PR|=(1<<3);/* clear the pending interrupt */
xSemaphoreGiveFromISR(AsixTaskSemaphore,NULL);
//LED2_OFF;
/* option for faster attention */
#if 1
portYIELD_FROM_ISR(pdTRUE);
#endif
}
then …```
void prvEMACHandlerTask( void *pvParameters ) /* own task */
{
AsixTaskSemaphore = xSemaphoreCreateBinary();
AsixSPIMutex = xSemaphoreCreateMutex();
EXTI->IMR |= 1 << IOA_STIRQ_BIT; /*unmask ext interrupts */
for (;;) {
if (xSemaphoreTake(AsixTaskSemaphore,0)==pdPASS ) {
/* proceed through the SPI guard */
xSemaphoreTake(AsixSPIMutex,0);
ASIXWriteReg(ASIXREG_IMR, IMR_MASK_MASKED);
prvNetworkInterfaceInput();/* get packets */
xSemaphoreGive(AsixSPIMutex);
ASIXWriteReg(ASIXREG_IMR, IMR_MASK_NORMAL);/* permit interrupts*/
}
}
}
and also :
xNetworkInterfaceOutput( () {/* belongs to IP TASK it seems */
xSemaphoreTake(AsixSPIMutex,0);
result=SendPkt((uint8_t *) pxDescriptor->pucEthernetBuffer, ulTransmitSize);
xSemaphoreGive(AsixSPIMutex);
}
etc etcglenenglish wrote on Wednesday, August 17, 2016:
It actually blows up here now, - ALWAYS !!!
in IP-Task…NetworkInterfaceOut
xSemaphoreTake(AsixSPIMutex,0);
result=SendPkt((uint8_t *) pxDescriptor->pucEthernetBuffer, ulTransmitSize);
xSemaphoreGive(AsixSPIMutex);
call Stack
IPTask()
HandleEthernetPacket
ProcessEthernetPacket
ReturnEthernetFrame
NetworkInterfaceOut {
xSemaphoreTake(AsixSPIMutex,0);
result=SendPkt((uint8_t *) pxDescriptor->pucEthernetBuffer, ulTransmitSize);
xSemaphoreGive(AsixSPIMutex);
which calls .....xQueueGenericSend
prvCopyDataToQueue
xTaskPriorityDisinherit
in <configASSERT( pxTCB == pxCurrentTCB );>** bang ***
These tests are only ICMP pings at 20ms rate. More scope probes required, more understanding required.
It seems there are plenty of bullets left in my revolver to solve this one- I am by no means stuck. Still nice to have some people to comment. discuss with. comments are appreciated.
What I am not familiar with (after only 2 days with freertos) is the scheduler, the internals etc which, in time I will get to know.
I’ll attach some LEDS to interrupt handler, and another LED around the mutex in EMAChandler and another LED around the mutex in the networkout and see what the concurrency is .
cheers
richard_damon wrote on Thursday, August 18, 2016:
You ca Take on the mutex with a delay of 0, so the take will always immediately return, and indicate success or failure via its return value that you are ignoring. If the take fails you do the operation anyway and then try to give the mutex (which you don’t have) back, causing the assert.
You could make the wait be ‘forever’ so it will always succeed, or test the return value of the take and decide what to do if the device is busy right now.
heinbali01 wrote on Thursday, August 18, 2016:
Hi Glen,
Very glad to hear that you are developing a +TCP driver for the ASIX 88796C chip.
You mention “ip4, ip6”? Is that IPv6?
+TCP will be extended soon with IPv6. If anyone is interested in testing it, please let it know.
Sorry to drop in so late in this discussion.
When you write :
xSemaphoreTake(AsixSPIMutex,0);
result=SendPkt((uint8_t *) pxDescriptor->pucEthernetBuffer, ulTransmitSize);
xSemaphoreGive(AsixSPIMutex);
Is that symbolically? I hope you always test the outcome of xSemaphoreTake()?
One thing is clear: xNetworkInterfaceOutput() will only be called from the IP-task.
Do you want xNetworkInterfaceOutput() to wait for the SPI-transfer to be finished?
Beside sending there is a second thing to do: use the same SPI-bus to receive messages. It is custom in +TCP to create an extra task called prvEMACHandlerTask(). It is a “Deferred Interrupt Handler”, it handles events that were detected from within some ISR. The ISR may be triggered by the EMAC, the PHY or the DMA or a SPI controller. If any of the devices need attention, it will vTaskNotifyGiveFromISR() the prvEMACHandlerTask().
If I were you, I would let prvEMACHandlerTask() send all packets to the EMAC! You stop using the couple:
vTaskNotifyGiveFromISR();
ulTaskNotifyTake( pdFALSE, ulMaxBlockTime );
and you start using :
xQueueSendFromISR ();
xQueueReceive();
Now xNetworkInterfaceOutput() will just forward the NetworkBuffer to the prvEMACHandlerTask() by calling xQueueSend().
Make sure FreeRTOSIPConfig.h contains :
#define ipconfigZERO_COPY_TX_DRIVER 1
Now when the IP-task sends a packet, the parameter bReleaseAfterSend will always be true:
BaseType_t xNetworkInterfaceOutput( NetworkBufferDescriptor_t * const pxDescriptor,
BaseType_t bReleaseAfterSend )
bReleaseAfterSend == pdTRUE means that the Network Buffer must be released by you, failing to release it would cause a memory leak.
bReleaseAfterSend == pdFALSE means that the Network Buffer may not be released and that the driver can only temporarily inspect it. After returning from xNetworkInterfaceOutput(), the contents of the Network Buffer may change.
So when ipconfigZERO_COPY_TX_DRIVER == 1, your driver will own the NetworkBuffer and it can forward them to the prvEMACHandlerTask(), who will send them.
Within prvEMACHandlerTask() you will check many things:
● Reception of packets
● Sending of packets (from the queue)
● Check the Link Status of the PHY
● Check and report any errors
If you want to implement this suggestion, please have a look at FreeRTOS_IP_Private.h for the definition of eIPEvent_t and IPStackEvent_t. You can make some equivalent enum and struct for your prvEMACHandlerTask().
Advantages: no more problems with priority inversion. You will never hold-up the IP-task while waiting for the SPI bus.
Within prvEMACHandlerTask() you can easily decide which direction gets priority: TX or RX. You can let this depend on the actual working load.
If you have more questions, don’t hesitate to shoot!
Good luck
glenenglish wrote on Thursday, August 18, 2016:
Hi Richard
My code segment was simplified - IE not showing the processing of the result = …
I always check the result.
Umm doesnt Take with 0 specified wait forever (ie no timeout) ?
OH I just read it
" If tick wait is zero, the semaphore will return immediately"
ooops ! My bad.
thanks for the pointer.
actually in NetworkOutput I was NOT checking the result- using timeout of ZERO I had assumed wait forever.
AHHHH so I am running right through that stop sign ! and probably then doing a GIVE when I never really TOOK . ’
OK I see there is <portMAX_DELAY>
right… RTFM… read the f**ing manual I say to myself.
cheers
heinbali01 wrote on Thursday, August 18, 2016:
Hi Richard
It was Hein pointing that out, not Richard 
I think that when usingportMAX_DELAY you also want to define:
#define INCLUDE_vTaskSuspend 1
Here is some more text to keep you busy:
If you are sharing the same SPI-bus with other devices apart from the EMAC, it would be nicer to share it properly using a Mutex.
If the SPI-bus only serves the EMAC, I would give prvEMACHandlerTask() exclusive access to it.
For +TCP projects, it is beneficial to assign the following task priorities:
/* highest */
prvEMACHandlerTask();
prvIPTask();
/* tasks that use +TCP */
prvIdleTask();
/* lowest */
prvEMACHandlerTask() normally uses very little CPU-time: it starts and stops DMA or SPI transactions and it’ll poll a PHY. In order to avoid an overflow of the reception buffers, it wants a high priority.
prvIPTask() typically has more work to do: it handles incoming packets and it prepares all outgoing traffic: TCP ACK’s and data packets. If your EMAC can handle all checksums, that is great. Calculating CRC’s in software, although heavily optimised, takes more CPU time.
User tasks that use +TCP: if given a lower priority, it’ll mean that sendto() and send() will normally work immediately. There won’t be much queuing of packets, because the IP-task will have a higher priority and deliver each packet upon request.
Regards.
glenenglish wrote on Thursday, August 18, 2016:
Hi Hein
Thanks very much for the information and your time to write.
Given that there is only one SPI device on this bus (it’s not like I am sharing the one SPI between say a temperature sensor, and other sensors with different chip-selects) it really makes no sense for me to do what I am doing. (apart from that it is convenient). IE what I am doing is dumb but it grew that way because of the ‘template’.
To change what NetworkOutput (that runs in IPTask) does I think that where I currently send a packet , I can instead, signal the EMACS thread that there is something to send , and post it the txdescriptor soemhow . Then, it can wait. Then, when EMACS has finished with the buffer, EMACS can signal IPtask (that is blocked) that it is done with the buffer and it can continue. Or I can free the buffer from the EMACS thread etc. a few options I will investigate. INetworkOutput does not wait (IE it just signals EMACS with the buffer descriptor # or ptr) then it can go about what it was doing, something else.
The only requirement for that to work is to have EMACS thread waiting on both the IRQnotify or the NetworkOutput notify. Like WaitForMultipleObjects in win32 land. I did read somewhere there is a facility to wait on multiple objects.
This way , only one thread needs to deal with the SPI. much better ! { which was of course the original suggestion :- ) }
cheers
heinbali01 wrote on Thursday, August 18, 2016:
A short response now:
When xNetworkInterfaceOutput() returns, the packet has been queued to be sent by the EMAC task. You don’t have to wait for that to happen, you can just return:
BaseType_t xNetworkInterfaceOutput( NetworkBufferDescriptor_t * const pxDescriptor,
BaseType_t bReleaseAfterSend )
{
EMACEvent_t xEvent;
/* if ipconfigZERO_COPY_TX_DRIVER == 1 then
bReleaseAfterSend will be true. */
configASSERT( bReleaseAfterSend != pdFALSE );
configASSERT( xEMACEventQueue != NULL );
xEvent.pxDescriptor = pxDescriptor;
xEvent.eEventType = eEMACTxEvent;
xQueueSendToBack( xEMACEventQueue, &xEvent );
return 0; /* Return value is actually ignored. */
}
glenenglish wrote on Thursday, August 18, 2016:
Hi Hein
Yeah. I see what you mean (looking at the freertos+TCP code) I don’t really have to wait, I can just dispatch the descriptor. Freeing the descriptor buffer (once i have used the data) can be done in emacs.
nice to have imbuilt queues. Something else taken care of for me.
Off for 1.5 days interstate.
cheers
joehinkle wrote on Thursday, August 18, 2016:
Glen:
May I offer my 2 cents as someone who has just gone thru the process of implementing a driver.
Use a binary semaphore to tell who has the spi bus: Either Xmit opperations or Rcv operations.
Use a task to process received messages. Make the priority of this task higher than the IP task priority.
When you receive a RCV interrupt, use xTaskNotifyFromISR to wake up the RCV task (may not be able to actually run if the Xmit operation has the SPI semaphore.
Make sure you set ipconfigZERO_COPY_TX_DRIVER to one in your FreeRTOSIPConfig file. This will allow you to return from xNetworkInterfaceOutput() prior to actually completing the xmit operation.
In xNetworkInterfaceOutput() get the SPI semaphore. This will lock out the RCV task. If you are using interrupts to manage you spi transfer - start the transfer and return. This will allow the stack to continue operating – but NOT to receive any new messages.
In the xmit complete interrupt, vNetworkBufferReleaseFromISR() to release the buffer and xSemaphoreGiveFromISR() to give back the spi semaphore.
In your RCV task … use:
xSemaphoreTake(SPI_GateSemaphore, portMAX_DELAY);
You will now wait until Xmit operations are completed. Once you have the semaphore and are processing received messages, the IP task can not run because its of lower priority so there is no concern about xNetworkInterfaceOutput() being called.
When done with all the processing of received messages in the RCV task, give the semaphore back and wait for the RCV interrupt to wake you again. The IP task will most likely start running and now has access to the SPI semaphore via xNetworkInterfaceOutput().
The most important thing (to me) is the use of the ipconfigZERO_COPY_TX_DRIVER set to 1.
This allows you to return from xNetworkInterfaceOutput() while your spi transmits in the background.
if ipconfigZERO_COPY_TX_DRIVER is set to 0 AND xReleaseAfterSend is FALSE in the call to xNetworkInterfaceOutput() then you MUST NOT return until xmit is complete (unless you copy the message to another buffer and xmit out of it – but that’s just a waste of time and memory).
Hope that helps.
Joe
heinbali01 wrote on Thursday, August 18, 2016:
In xNetworkInterfaceOutput() get the SPI semaphore. This will lock out the
RCV task. If you are using interrupts to manage you spi transfer - start the
transfer and return. This will allow the stack to continue operating – but
NOT to receive any new messages.
Joe, thanks for your input.
I wouldn’t advice to use an SPI semaphore for this reason: if the SPI bus is busy, it would hold-up the IP-task. I would rather pass the Network Buffer to the EMAC task and let that task work on it.
And therefore, I would replace the task-notify mechanism with the a working queue to wake up the EMAC task.
joehinkle wrote on Thursday, August 18, 2016:
From what I understand of who has access to Glen’s spi, there are only two (2) users. IP task for xmit and RCV task for receiving.
if ipconfigZERO_COPY_TX_DRIVER == 0 and xReleaseAfterSend == FALSE, you are going to stall the IP task no matter what. if ipconfigZERO_COPY_TX_DRIVER == 1, the IP task can keep running until it attempts to xmit again and then stall if the transmit operation is still active.
I personally would keep the xmit and rcv processes seperate (not have them in the RCV task). To me – it keeps responsibilities clear a seperate. The SPI semaphore manages who runs based on hightest priority at the time SPI access is needed.