FreeRTOS+ TCP & FAT FTP On Zynq-7000 With Xilinx SDK/XSCT

system · December 29, 2018, 9:59pm

otzen42 wrote on Saturday, December 29, 2018:

Hi,

I admit to being a bit new to the SW relm (I’m an FPGA person by trade), so I appologize if I’m missing something obvious here. I have a Zybo-Z7 Xilinx Zynq-7000 board that I have been running FreeRTOS on. I started out using the FreeRTOS packaged into Xilinx SDK, but have recently moved to a bare metal project, and am pulling in the FreeRTOS source myself to allow more control over FreeRTOSConfig.h (which Xilinx insists on autogenerating if you use a FreeRTOS SDK project).

I have successfully integrated CLI into my project using the latest FreeRTOS-Plus demo code (from the v10.1.1 tag), and the UART command console is working great. However, I am running into issues trying to incorperate TCP & FAT into my project (end goal being to run the demo FTP server).

One question I have is what the best place to start from for this is. With TCP being part of the main release, but FAT only existing in the old 160919 labs distribution, I have been struggling to figure out where the best starting point is. I have tried following the steps outilined on https://www.freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_TCP/TCPIP_FAT_Examples_Xilinx_Zynq.html, but I have been unable to get the projects to build for either the MicroZed or ZC702 (running Vivado 2018.3). Since I don’t have either of those boards, I am eventually going to need to do it in my own environment as well. Without a buildable example that I understand, I have just been piecing this together by trial and error, so I may be missing something here.

To simplify things a bit, I started by trying to integrate just TCP into my project (from the v10.1.1 tag), but that is also not going well. First of all, I have had to modify all of the files in https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/NetworkInterface/Zynq to remove the “Zynq/” from the #include directives. I assume there is a way I can work around this, but I have not figured out how to get it to work with XSCT yet (I would much prefer to use a scripted build flow rather than using the SDK GUI if at all possible). I’m not sure why, but these seem to be the only files in FreeRTOS I have run into which have a directory as part of the #include.

Second, I had to remove the #include for Demo_Logging.h from https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/NetworkInterface/Zynq/uncached_memory.c, and replace the call to vLoggingPrintf with a simple xil_printf(). I was unable to locate Demo_Logging.h anywhere in the FreeRTOS SVN repo, the 160919 labs download, or my Xilinx install.

Finally, it appears that the x_emacpsif code in https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/NetworkInterface/Zynq is not compatible with the xNETWORK_BUFFER structure defined in https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/include/FreeRTOS_IP.h. I am getting a lot of errors about attempts to access a member called pxNextBuffer which does not exist. I have not tried reverting to the TCP code from the 160919 labs download yet, but I think that might be my next step. If possible, I would like to get this working with what is availible in the official FreeRTOS SVN repo so that I can manage it as an external to my repo.

if it is at all helpful, I am currently importing the following files into my project (XSCT seems to be automatically pulling the xemacps library from my Xilinx install for me, and if I manually include the version in the FreeRTOS repo I see the same results so I decided to stick with the latest from Xilinx unless it becomes a problem):

I am incorperating the TCP into my application with a file based on 160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/FreeRTOS_Plus_TCP_and_FAT_Zynq_SDK/RTOSDemo/src/main.c, and have my #defines configured for mainCREATE_UDP_CLI_TASKS. It isn’t throwing any errors right now, so until I can get things to compile I am assuming it is OK (I commented out the parts that interact with the FAT library until I can get TCP working).

Any help or guidance would be greately appreciated.

Thanks!

htibosch · December 31, 2018, 9:45am

heinbali01 wrote on Monday, December 31, 2018:

I have a Zybo-Z7 Xilinx Zynq-7000 board that I have been running FreeRTOS on.

The demo’s were developed both on Zybo as well as a MicroZed board and I keep on testing them.

One question I have is what the best place to start from for this is.
With TCP being part of the main release, but FAT only existing in the
old 160919 labs distribution, I have been struggling to figure out where
the best starting point is.

We’re lacking time: time to review the old demo projects and time to pick-up FreeRTOS+FAT and add it to the official AWS/FreeRTOS release on Github
Due to this lack of time, the two libraries were split-up: +TCP moved to the official release, and +FAT stayed in the FreeRTOS/Labs distribution.

You can find an unofficial release of +FAT here:

https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/?limit=250#5c65

which (I think) is pretty similar to the release with MIT license on this link:

https://freertos.org/FreeRTOS-Labs/downloads/FreeRTOS-Plus-FAT-160919a-MIT.zip?dl=0

I have had to modify all of the files in …/NetworkInterface/Zynq
to remove the “Zynq/” from the #include directives.

You can also leave the #include statements as they are and add a search path like this
“/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/NetworkInterface”
so that “Zynq/x_topology.h” will be found

I would much prefer to use a scripted build flow rather than using the SDK GUI if
at all possible

I also prefer to control the projects my self in stead of being guided by preprogrammed intelligence. I always use Makefile.
All Eclipse IDE’s have a possibility to import a Makefile project. You will also be able to do in-circuit debugging when you write your own Makefile.

Upgrading will be very easy: just change the value of makefile variables like e.g. FreeRTOS_Kernel, FreeRTOS_TCP:

	FreeRTOS_PATH = $(ROOT_PATH)/framework/FreeRTOS_v1.1.0

I’m not sure why, but these seem to be the only files in FreeRTOS I have
run into which have a directory as part of the #include.

That is true. In the original FreeRTOS kernel releases there was only #include “fname.h”
not even #include “include/fname.h”

The following:

	#include "Zynq/xemacpsif.h"
	#include "Zynq/xemacpsif_hw.h"

was chosen to avoid including the original “xemacpsif” files located in BSP directories.
Later on, it was decided to insert an underscore: “x_emacpsif.h” and “x_emacpsif_hw.h”.

and replace the call to vLoggingPrintf with a simple xil_printf().

I’m sorry again, the demo projects need a long-awaited revision. We spent most our time on the libraries them selves. The drivers and the demo projects received much less attention.
Most of the +TCP/+FAT questions on this forum are of the type: “How do I get the demo running???”.

One reoccurring difficulty is always drivers, where we are dependent on third-party software. The BSP files need changes for the drivers and at the same time the factories keep-on releasing new versions of BSP ( or HAL ) libraries… which may not be downward compatible.
( I wish that the chip manufacturers would maintain the FreeRTOS +TCP and +FAT libraries )

I am getting a lot of errors about attempts to access a member called pxNextBuffer which does not exist.

Maybe you miss a define in your FreeRTOSIPConfig.h:

	#define ipconfigUSE_LINKED_RX_MESSAGES   1

XSCT seems to be automatically pulling the xemacps library from my Xilinx install for me

We had a good reason to adapt the xemacps.c ( and other files ) from the DSK and put them in the “NetworkInterface/Zynq” directory. I don’t think that the EMAC will work properly with the original library files:

Zynq\NetworkInterface.c
Zynq\x_emacpsif.h
Zynq\x_emacpsif_dma.c
Zynq\x_emacpsif_hw.c
Zynq\x_emacpsif_hw.h
Zynq\x_emacpsif_physpeed.c
Zynq\x_topology.h

Here below I will attach the latest version of the Zynq driver: Zynq_2018_12_31.7z

Note that this is an informal and private release. Hopefully there will be more time available in 2019 to work on the drivers

system · January 1, 2019, 12:47am

otzen42 wrote on Tuesday, January 01, 2019:

Thanks for the info!

I had a feeling the demo was probably just a bit out of date with all of the changes going on around TCP and FAT.

I pulled in the code you attached and added the #define, and that seems to have cleared up the error I was seeing. However, I am now missing the define for ipconfigNIC_N_RX_DESC. Do you have a recomendation for a FreeRTOSIPConfig.h to reference? I can’t recall for sure, but I think the one I have is from the Zynq TCP+FAT demo in the labs, so I bet it is just a bit out of date.

For some reason I have been having a hard time getting the XSCT tool to use a search path (I expect I am just doing something stupid), but since the files are now x_emac I just temporarily removed the “Zynq/” again till I can get the tool sorted out (XSCT is supposed to be Xilinx’s way to do scripted builds with SDK, but custom Makefiles may be the easier way to go). It is jsut the other xemac files (in https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS/Demo/CORTEX_A9_Zynq_ZC702/RTOSDemo_bsp/ps7_cortexa9_0/libsrc/emacps_v3_7/src) that I have been letting the tool pull from my Xilinx install.

htibosch · January 1, 2019, 6:05am

heinbali01 wrote on Tuesday, January 01, 2019:

Hi Marty,
ipconfigNIC_N_RX_DESC and ipconfigNIC_N_TX_DESC define the number of DMA descriptors for reception and transmission. These are used in the NetworkInterface.
ipconfigNUM_NETWORK_BUFFER_DESCRIPTORS defines to total number of network buffers. Note that with a zero-copy reception, ipconfigNIC_N_RX_DESC network buffers are being reserved for reception.
ipconfigNIC_N_TX_DESC is often much smaller: it doesn’t take long before queued packets are being sentso it needs a short queue ( +/- 4 should be enough ).

See the attached picture below that shows how to start a new Makefile project.

Also I attached the FreeRTOSIPConfig.h that I used for the Xilinx project.

system · January 2, 2019, 1:51am

otzen42 wrote on Wednesday, January 02, 2019:

Thanks, that got me a lot farther! It looks like the only thing I am missing now is a definition for xCheckLoopback. It is in your new NetworkInterfaces.c, and I’m not seeing a definition for it anywhere. (If I comment that out everything is compiling for me, but I assume it is there for a reason, so I haven’t tried running the output yet without it)

Also, in case anyone else finds this, I discovered that in https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/CLICommands/CLI-commands.c, the CLI Command Definition for xIPDebugStats is wrapped in a #if for configINCLUDE_DEMO_DEBUG_STATS, but the command registration in vRegisterCLICommands is not, so I had to add a check there as well. I also chose to wrap the xRunTimeStats logic in a check for configGENERATE_RUN_TIME_STATS similar to the standard CLI example commands (since I don’t currently have it defined, and that caused the funciton to break).

Finally, I had to add a #include for FreeRTOS_IP.h to https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/CLICommands/UDPCommandServer.c to get the backwards compatible data types defined.

htibosch · January 2, 2019, 2:41am

heinbali01 wrote on Wednesday, January 02, 2019:

xCheckLoopback(): as many users asked for a loop-back mechanism, I added one.
This is the original post about it.

The function xCheckLoopback() will be called from xNetworkInterfaceOutput(). If the packet is targeted to the device it self ( matching with MAC-address ), the packet will be returned to the IP-task for processing. This simple mechanism works wonderfully well.
So for instance, you can create two TCP sockets and connect them to eachother by using the board’s IP address.

I’m attaching xCheckLoopback.c which you can use for modifying FreeRTOS_ARP.c

I hope that in the coming year, the loop-back mechanisme will make it to the main release. It would make testing easier.

Regarding the CLI : thanks for taking the effort to write this down. It may help other users.
I never used the CLI examples my self.

system · January 2, 2019, 9:04pm

otzen42 wrote on Wednesday, January 02, 2019:

Hein, thank you so much for all the help. I have successfully got my Zynq booting and can access the UDP CLI interface. Next step is FAT (not sure if I will get to that right away or not - have to see how smoothly it integrates). I will definitley check out the updated FAT you linked to above.

system · January 14, 2019, 5:22am

otzen42 wrote on Monday, January 14, 2019:

I just wanted to follow up as reference for future generations. I was able to get the demo FTP server running on my Zynq!

As reference, here are the files I have to include in my project:

https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS/Source/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS/Source/include/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS/Source/portable/MemMang/heap_4.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-CLI/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-CLI/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/include/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/Compiler/GCC/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/BufferManagement/BufferAllocation_1.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/portable/NetworkInterface/include/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/Demo_IP_Protocols/Common/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/Demo_IP_Protocols/FTP/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/Demo_IP_Protocols/include/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/CLICommands/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/CLICommands/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_CLI_Demos/Sample-CLI-commands.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/TraceMacros/Example1/*.h
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/TraceMacros/Example1/*.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_CLI_Demos/UARTCommandConsole.c
https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_CLI_Demos/UDP-Related-CLI-commands.c
<XILINX_INSTALL>/SDK/2018.3/data/embeddedsw/ThirdParty/bsp/freertos10_xilinx_v1_2/src/Source/portable/GCC/ARM_CA9/*.h
<XILINX_INSTALL>/SDK/2018.3/data/embeddedsw/ThirdParty/bsp/freertos10_xilinx_v1_2/src/Source/portable/GCC/ARM_CA9/*.c
<XILINX_INSTALL>/SDK/2018.3/data/embeddedsw/ThirdParty/bsp/freertos10_xilinx_v1_2/src/Source/portable/GCC/ARM_CA9/*.S
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/*.c
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/include/*.h
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/portable/common/*.h
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/portable/common/*.c
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/portable/Zynq/*.h
https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z - FreeRTOS-Plus-FAT/portable/Zynq/*.c
https://sourceforge.net/p/freertos/discussion/382005/thread/3656c42f33/2651/attachment/Zynq_2018_12_31.7z - Zynq/*.h
https://sourceforge.net/p/freertos/discussion/382005/thread/3656c42f33/2651/attachment/Zynq_2018_12_31.7z - Zynq/*.c
160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/FreeRTOS_Plus_TCP_and_FAT_Zynq_SDK/RTOSDemo/src/hr_gettime.h
160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/FreeRTOS_Plus_TCP_and_FAT_Zynq_SDK/RTOSDemo/src/hr_gettime.c
160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/CreateAndVerifyExampleFiles.c
160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/test/ff_stdio_tests_with_cwd.c

In addition, I had to make the following modifications in order to get things to compile:

https://sourceforge.net/p/freertos/discussion/382005/thread/d9a716db/5c65/attachment/FreeRTOS-Plus-FAT_april_2018.7z
Modified ‘FreeRTOS-Plus-FAT/portable/Zynq/xsdps.c’ and ‘FreeRTOS-Plus-FAT/portable/Zynq/xsdps_options.c’ to change from ‘#include "<FreeRTOS.h>’ to '#include “FreeRTOS.h” (Probably not requried if I could get my build scripts setup correctly).

https://sourceforge.net/p/freertos/discussion/382005/thread/3656c42f33/2651/attachment/Zynq_2018_12_31.7z
Modified ‘Zynq/NetworkInterface.c’, ‘Zynq/uncached_memory.c’, ‘Zynq/x_emacpsif_dma.c’, ‘Zynq/x_emacpsif_hw.c’, ‘Zynq/x_emacpsif_hw.h’, and ‘Zynq/x_emacpsif_physpeed.c’ to remove the ‘Zynq/’ from the #include statements (as described before, I need to rework my build scripts to support this). I also chose to modify ‘Zynq/x_emacpsif_physpeed.c’ to replace the ‘xil_printf’ calls with ‘lUDPLoggingPrintf’ calls since I have implemented ‘lUDPLoggingPrintf’ with a mutex to avoid multiple threads printing at once.

https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/include/FreeRTOS_ARP.h
Modifed per https://sourceforge.net/p/freertos/discussion/382005/thread/3656c42f33/72fb/attachment/xCheckLoopback.c

https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Source/FreeRTOS-Plus-TCP/FreeRTOS_ARP.c
Modifed per https://sourceforge.net/p/freertos/discussion/382005/thread/3656c42f33/72fb/attachment/xCheckLoopback.c

https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_UDP_Demos/CLICommands
Modified ‘CLI-commands.c’ to #include FreeRTOS_IP.h and wrap all code for the RunTimeStats command in a #if checking that ‘configGENERATE_RUN_TIME_STATS’ is 1, as well as wrapped all code for the DisplayIPDebugStats command in a #if checking that ‘configINCLUDE_DEMO_DEBUG_STATS’ is non-zero. I also added a #incldue of ‘FreeRTOS_IP.h’ to ‘UDPCommandServer.c’.

160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/CreateAndVerifyExampleFiles.c
Added #if check for ‘ipconfigUSE_HTTP == 1’ around the #include of ‘DefaultWebPages.h’.

160919_FreeRTOS_Labs/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/test/ff_stdio_tests_with_cwd.c
Commented out assertion for ‘ff_filelength( pxFile ) == 0’ on line 960 because it was failing.

I did not include https://svn.code.sf.net/p/freertos/code/tags/V10.1.1/FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_CLI_Demos/File-Related-CLI-commands.c in my project yet (it doesn’t appear to be including the FreeRTOS-Plus-FAT library, so I haven’t even tried to see if it will work with the FreeRTOS-Plus-FAT APIs).

Finally, I have attached a my cofig.h files and the top-level files I am using to implement the TCP and FTP code (the CLI is simply calls to vStartUDPCommandInterpreterTask(), vRegisterSampleCLICommands(), and vRegisterUDPCLICommands() after the network is up). I simply have an initialization sequence that calls tcpInit() and ftpInit(). Note that the files I attached use some custom functions I have defined elsewhere like a custom debug print function and a vTaskDelay wrapper, but there shouldn’t be anything too difficult to replace in there (I included examples of what my functions do for reference).

rtel · January 14, 2019, 5:13pm

rtel wrote on Monday, January 14, 2019:

Thanks for sharing.