Software architecture of a control system with many communication interfaces

I’m developing a system similar to the one in this tutorial. The system has several inputs, several control functions, several communication interfaces. The input, control function require strict timing, and most of the communication interfaces require deadline-only timing. (One communication interface requires strict timing though)

The tutorial is very enlightening, but it doesn’t elaborate on interactions and data sharing among tasks. That’s why I’m posting this question.

Say the data of the system consists mainly of these:

  • Input, such as GPIO / ADC results
  • Control state, the state of the business logic
  • Command, set by various human interfaces
  • Parameter, control the behavior of the system in many aspects

For clarity, I use Solution #2 to describe the system, so that logically separate tasks run in separate FreeRTOS tasks. Note that the use of GlobalData and these tasks is a design choice and can be improved.

typedef struct {
	Input input;
	ControlState ctrlState;
	Command cmd;
	Parameter param;

	// other data, like communication interface status
	bool webServerOnline;
	bool rs232Online;
} GlobalData;
GlobalData globalData;
// shorter names
#define gInput globalData.input
#define gCtrlState globalData.ctrlState
#define gCmd globalData.cmd
#define gParam globalData.param

void PlantControlTask(void *pvParameters) {
	TickType_t xLastWakeTime = xTaskGetTickCount();
	for (;;) {
		vTaskDelayUntil(&xLastWakeTime, CYCLE_RATE_MS);
		gInput = ReadInputFromSensors(gParam);
		// notation: in function arguments, use variable without & for input only args, use &variable for other
		// read input, ctrlState, cmd, param, write ctrlState and output
		Output output = PerformControlAlgorithm(gInput, gCmd, gParam, &gCtrlState);
		WriteOutputToActuators(output, gParam);
	}
}
void WebServerTask(void *pvParameters) {
	for (;;) {
		HttpRequest request;
		HttpResponse response;
		// low-level HTTP+TCP/IP stack sends http requests to this queue
		if (xQueueReceive(xHttpRequestQueue, &request, HTTP_TIMEOUT)) {
			if (IsRequestReadMonitorPageData(request)) {
				// the monitor webpage may need to display anything in GlobalData
				response = ReadMonitorPageData(globalData);
			} else if (IsRequestWriteCmd(request)) {
				response = WriteCmd(request, &gCmd);
			} else if (IsRequestWriteParam(request)) {
				response = WriteParam(request, &gParam);
			} else {
				response = HttpErrorResponse();
			}
			SendHttpResponse(response);
			SetWebServerStatusAsOnline();
		} else {
			SetWebServerStatusAsOffline();
		}
	}
}
void RS232Task(void *pvParameters) {
	for (;;) {
		// say this is a modbus server
		ModbusRequest request;
		ModbusResponse response;
		// low-level modbus stack sends modbus requests to this queue
		if (xQueueReceive(xModbusRequestQueue, &request, RS232_TIMEOUT)) {
			if (IsRequestReadRegisters(request)) {
				// read regs may map to anything in GlobalData
				response = ReadRegs(request, globalData);
			} else if (IsRequestWriteRegisters(request)) {
				// write regs map to cmd and param
				response = WriteRegs(request, &gCmd, &gParam);
			} else {
				response = ModbusErrorResponse();
			}
			SendModbusResponse(response);
			SetRs232StatusAsOnline();
		} else {
			SetRs232StatusAsOffline();
		}
	}
}
void LocalOperatorInterfaceTask(void *pvParmeters) {
	// say the keyboard is used to select UI and write cmd and param
	// and lcd is used for display all information in GlobalData
	TickType_t xLastWakeTime = xTaskGetTickCount();
	for (;;) {
		vTaskDelayUntil(&xLastWakeTime, DELAY_PERIOD);
		UpdateDisplay(globalData);		   // may read anything in GlobalData
		InterpretKeyInput(&gCmd, &gParam); // may write cmd, param
	}
}

In the system, every task needs to access most of the data, so my current implementation uses this global variable, with a global mutex to protect it. A task must hold this mutex wherever it accesses any member of globalData.

This is not a very good design, mutex can affect the timing of the control task, programmers may forget the mutex somewhere and results in a data race. For some reasons the low-level TCP/IP stack requires the scheduler to be preemptive. How can this design be improved with these requirements and limitations? Any ideas are welcome.

Using mutex is not necessarily bad if the application requires. You may try to make the locks more granular. Another thing is to consider to use taskENTER_CRITICAL/taskEXIT_CRITICAL. For example -

void Task( void * param )
{
    Parameter localParam;

    for( ;; )
    {
        /* Make a local copy of gParam. */
        taskENTER_CRITICAL();
        localParam = gParam;
        taskEXIT_CRITICAL();

        /* Use localParam to do all the work. */

        /* If needed, update gParam. */
        taskENTER_CRITICAL();
        gParam = localParam;
        taskEXIT_CRITICAL();

    }
}