If I can interpret what it is you are trying to do.
You have a peripheral that uses a semaphore, presumably for synchronisation. The semaphore is created externally from the peripheral driver and a reference to it is passed into the peripheral driver.
I think this should work, but you can pass semaphore handles by copy. You don’t need to pass them by pointer.
This is because the semaphore handle is only a reference to the semaphore. It is not the entire semaphore structure.
Take a look at the files serial.c and serialISR.c in the FreeRTOS\Demo\ARM7_STR75x_GCC\serial\serial.c file for an example that uses queue handles (same thing really). The function vConfigureQueues() is used to pass the handles to queues created in the serial.c file into the serialISR.c file. The latter file contains the ISR function in which the queues are used.