WiFi Porting for PIC32MZ EF

freeRToes wrote on February 14, 2018:

Hi,

I am working on porting pic32mz curiosity dev board demo to work with the on board WiFi module but it made me raise multiple questions.

Firstly, in more general way, I noticed that there is .mhc file which is MPLAB harmony configuration file, but AWS demo project structure is not MPLAB harmony compatible. Is there is a purpose for it? Besides that you can open it and understand some configurations knowing that you can’t generate an new one an update it, because it would break AWS demo.

Should MPLAB harmony and Amazon FreeRTOS be working together?

Secondly, what should be a porting procedure for a Wi-Fi interface?
Because MPLAB harmony way is that while configuring TCP/IP stack you select Network interface which can be MRF24WN0MA Wi-Fi Module and then configure SPI and Wi-Fi driver. But I see that there is /lib/wifi/portable/microchip/curiosity_pic32mzef/aws_wifi.c file on the github, but not included on software libraries from Amazon FreeRTOS console.

So, should one still use MPLAB harmony for configuring peripherals?
What porting approach should be taken for adapting AWS demo to use onboard wifi module?
I am not sure if I understand how aws_wifi.c should be edited for working with MRF24WN0MA Wi-Fi Module.
Besides TCP/IP configurations, where else should be changed to use Wifi instead of Ethernet?

Thanks

SarenaAtAws wrote on February 15, 2018:

Hello,

Amazon FreeRTOS uses the same directory structure for each of our supported platforms in the interest of portability and consistency in experience. Partly as a result of that, the MPLAB Harmony Configurator and Amazon FreeRTOS are not directly compatible. However, you can find the Harmony library files applicable to the current Amazon FreeRTOS configuration under lib/third_party/mcu_vendor/microchip/harmony/v2.04.

Regarding a Wi-Fi interface for the Microchip Curiosity PIC32MZ EF, we will be adding support for the ATWINC1500 in the future. We do not have plans to support the MRF24WN0MA Wi-Fi Module. In the meantime, the code in lib/wifi/microchip/curiosity_pic32mzef/aws_wifi.c is available, but not supported at this time and is written for the ATWINC1500.

For more information on the ATWINC1500 part included with the Amazon FreeRTOS Curiosity PIC32MZ EF Bundle, please see https://www.microchip.com/DevelopmentTools/ProductDetails.aspx?PartNO=dm320104-bndl.

If you require the MRF24WN0MA Wi-Fi Module I suggest the following steps:

  • . Create a separate Harmony MPLAB project and, using the Harmony Configurator, select the drivers and other libraries needed for the MRF24WN0MA Wi-Fi Module.
  • . Copy over the Harmony library files needed into the AWS Demos project
  • . The Harmony Configurator will generate system configuration and initialization files. Copy the lines needed to use the MRF24WNOMA from the system_xxx.* files to the corresponding system_xxx.* files in Treadstone/demos/microchip/curiosity_pic32mzef/common/application_code/microchip_code/.
  • . Compile lib/FreeRTOS-Plus-TCP/source/portable/NetworkInterface/pic32mzef/NetworkInterface_wifi.c instead of Treadstone/lib/FreeRTOS-Plus-TCP/source/portable/NetworkInterface/pic32mzef/NetworkInterface_eth.c in the AWS Demos project. NetworkInterface_wifi.c is available but not supported at this time and it is written for the ATWINC1500. You may need to update it for the MRF24WN0MA driver API and other compilation errors.
  • . Compile Treadstone/lib/FreeRTOS-Plus-TCP/source/portable/BufferManagement/BufferAllocation_2.c instead of Treadstone/demos/microchip/curiosity_pic32mzef/common/application_code/microchip_code/BufferAllocation_2.c
  • . lib/wifi/microchip/curiosity_pic32mzef/aws_wifi.c uses the Harmony Wi-Fi driver supporting the ATWINC1500. Per above, this file is not currently supported and you may need to update it for the MRF24WN0MA driver API and other compilation errors.

Regards,
Sarena

Edited by: SarenaAws on Feb 16, 2018 9:54 AM

Edited by: SarenaAtAws on Feb 23, 2018 5:07 PM

andylong0206 wrote on February 21, 2018:

Hi,

I’m at the same point as you and somewhat pulling my hair out. I also have the PIC32 curiosity board, and using AWS RTOS version 1.1.0. I have also got the on-board MRF24 module working running a HTTP server within the AWS RTOS, similar to the curiosity RGB demo from microchip having morphed the driver code (very poor for the MRF24 module btw). My AWS MQTT echo demo works perfectly on the Ethernet wired connection, and my MRF24 module works perfectly with a HTTP server. Now for the issue, how do I get the required MQTT to use the WiFi socket? The AWS RTOS is using Freertos+IP TCIP stack and Berkley sockets whereby the MRF24 doesn’t, it uses the Harmony TCIPIP Stack. I realise that I need to somehow ‘port the WiFi’ however not being an expert in this area, I’m struggling to understand what is needed. I downloaded the later AWS RTOS master demo version 1.2.0 from GitHub. However, WiFi appears to be not fully implemented/supported as previously stated, it looks to be work in progress. There are some missing header files also (Port_WiFi.h, Port_Sockets.h & pic32_NetworkConfig.h). I feel that I am so close to having a working WiFi AWS demo, I just need some guidance on the porting.

Any help would be most appreciated.

Many thanks in advance.

freeRToes wrote on February 21, 2018:

That’s great! thanks!

So I actually have Wifi 7 Click from the amazon pic32mzef bundle which includes WINC1500. Decided to try on it first as I have TCPIP client demo working on the pic32mzef curiosity board. It’s WiFi 7 click mounted on mikroBus2 and UART USB click on mikroBus1. I followed your recommendations and was able to successfully build a project but it freezes or gets stuck so I have to debug that.

While porting I noticed that aws_wifi.c has drivers libraries for WILC1000 and not WINC1500. So just to clarify. Will AWS be adding support for WILC1000 or WINC1500?

For debugging purposes, would it be possible to see what serial output should look like for a successful demo?
As I can see mine stops after random number generation.

Thanks again,

freeRToes wrote on February 21, 2018:

Hi,

So I actually don’t even remember where I found this Amazon_FreeRTOS_Demo1.7 but it contains demo folder named AWS_IoT_MCU_Demo_Wi-Fi which has those missing Port files, but I learned that Port_Sockets.h is the same as aws_secure_sockets.h and port_wifi.h is just a header file for aws_wifi.c I am attaching the one I have.

For pic32_NetworkConfig.h it’s just regular NetworkConfig.h

I think the wifi file was created back in the day where the project structure and naming were different.

So if you follow the steps that Sarena suggest you should be able to compile it. In my case I think I messed up porting system_xxx.* lines because SPI uses interrupts/priorities and other configuration bits and maybe something here makes my board freeze.

Also, note that aws_wifi.c contains wifi driver functions like WDRV_IsDisconnectRequestedSet which is only for wilc1000 and not winc1500. Not sure how critical some of those are.

Cheers,

SarenaAtAws wrote on February 24, 2018:

Hello again,

Amazon FreeRTOS will have support for the WILC1000 driver code, but the hardware will use the ATWINC1500 Wi-Fi Module. The WILC1000 driver is better suited for applications that work with an external network stack. Amazon FreeRTOS has the FreeRTOS Plus TCP TCP/IP network stack utilized by Microchip Curiosity PIC32MZ EF, so the WILC1000 driver was added first.
The WINC1500 driver code is more suitable to applications where the host MCU doesn’t want to deal with the network stack and wants to offload that to the WINC1500 device.

In regards to using the MRF24WN0MA Wi-Fi Module, which we will not support:

. If you have properly updated your MCU’s system configuration and initialization files (system_xxx.c/.h), then the code should move past the random number generation. Not moving past the random number generation indicates issues with the configuration and initialization of the hardware peripherals.
. The NetworkInterface_wifi.c file, in the FreeRTOS Plus TCP library, calls the public API WIFI_On() and WIFI_ConnectAP(). If you have left the console printing in these functions you should see “Start Wi-Fi Connection…” and “Wi-Fi Connected” in your terminal after the random number generation.
. The demo will start running from the vApplicationIPNetworkEventHook() function in main.c, if the network connection was successful.
A successful MQTT Hello World demo will send 11 round trip messages and the final print out will be: “[MQTTEcho] MQTT echo demo finished.”

  • Sarena

Edited by: SarenaAtAws on Feb 23, 2018 5:02 PM

andylong0206 wrote on February 24, 2018:

Hi,

Thank you for that, much appreciated. I’ll take a look into your suggestions. You have also clarified some missing links. I’ll let you know how I progress. I think I may play with the WINC1xxx driver too if that’s the better supported. I went down the MWRF24 route for convenience of it being on board.

Many thanks.

Andy

andylong0206 wrote on March 01, 2018:

Hi,

I just wanted to provide some feedback on this post. I have got the Wi-Fi working with a little help from that latest release 1.2.2 today :slight_smile:

However, when I switch back to Ethernet i.e. #Define PIC32_Use_ETHERNET it didn’t compile, it’s missing a bunch of PIC32 specific functions in BufferAllocation_2.c. No problem I thought I’ll just add them. This is what I added:

//
// PIC32 specific stuff
//
#ifdef PIC32_USE_ETHERNET

// MAC packet acknowledgment, once MAC is done with it
static bool PIC32_MacPacketAcknowledge(TCPIP_MAC_PACKET* pPkt, const void* param);

// allocates a MAC packet that holds a data buffer that can be used by both:
// - the FreeRTOSIP (NetworkBufferDescriptor_t->pucEthernetBuffer)
// - the Harmony MAC driver: TCPIP_MAC_PACKET->pDSeg->segLoad
// from the beginning of the buffer:
// - 4 bytes pointer to the network descriptor (FreeRTOS)
// - 4 bytes pointer to the MAC packet (pic32_NetworkInterface.c)
// - 2 bytes offset from the MAC packet (Harmony MAC driver: segLoadOffset)
//
// NOTE: segLoadLen should NOT include:
// - the TCPIP_MAC_FRAME_OFFSET (== ipBUFFER_PADDING which should be == 10!)
// - the sizeof(TCPIP_MAC_ETHERNET_HEADER)
// These are added by the MAC packet allocation!
//
static uint8_t* PIC32_PktAlloc(uint16_t pktLen, uint16_t segLoadLen, TCPIP_MAC_PACKET_ACK_FUNC ackF, TCPIP_MAC_PACKET** pPtrPkt)
{
uint8_t* pBuff = 0;

// allocate standard packet
TCPIP_MAC_PACKET* pPkt = TCPIP_PKT_PacketAlloc(pktLen, segLoadLen, 0);

// set the MAC packet pointer in the packet 
if(pPkt != 0)
{
    pBuff = pPkt->pDSeg->segLoad;
    TCPIP_MAC_PACKET** ppkt = (TCPIP_MAC_PACKET**)(pBuff - PIC32_BUFFER_PKT_PTR_OFFSET);
	configASSERT(((uint32_t)ppkt & (sizeof(uint32_t) - 1)) == 0 );
    *ppkt = pPkt;     // store the packet it comes from
    pPkt->ackFunc = ackF;
    pPkt->ackParam = 0;
}

if(pPtrPkt != 0)
{
    *pPtrPkt = pPkt;
}

return pBuff;    

}

// standard PIC32 MAC allocation function for a MAC packet
// this packet saves room for the FreeRTOS network descriptor
// at the beginning of the data buffer
// see NetworkBufferAllocate
// Note: flags parameter is ignored since that’s used in the Harmony stack only
TCPIP_MAC_PACKET* PIC32_MacPacketAllocate(uint16_t pktLen, uint16_t segLoadLen, TCPIP_MAC_PACKET_FLAGS flags)
{
TCPIP_MAC_PACKET* pPkt;

PIC32_PktAlloc(pktLen, segLoadLen, 0, &pPkt);

return pPkt;

}

// standard PIC32 MAC packet acknowledgment
// function called once MAC is done with it
static bool PIC32_MacPacketAcknowledge(TCPIP_MAC_PACKET* pPkt, const void* param)
{
configASSERT((pPkt != 0));

TCPIP_PKT_PacketFree(pPkt);
return false;

}

// associates the current MAC packet with a network descriptor
// mainly for RX packet
void PIC32_MacAssociate(TCPIP_MAC_PACKET* pRxPkt, NetworkBufferDescriptor_t* pxBufferDescriptor, size_t pktLength )
{
uint8_t* pPktBuff = pRxPkt->pDSeg->segLoad;

pxBufferDescriptor->pucEthernetBuffer = pPktBuff;
pxBufferDescriptor->xDataLength = pktLength;

// make sure this is a properly allocated packet
TCPIP_MAC_PACKET** ppkt = (TCPIP_MAC_PACKET**)(pPktBuff - PIC32_BUFFER_PKT_PTR_OFFSET);
configASSERT(((uint32_t)ppkt & (sizeof(uint32_t) - 1)) == 0 );
if(*ppkt != pRxPkt)
{
    configASSERT(false);
}

// set the proper descriptor info
NetworkBufferDescriptor_t** ppDcpt = (NetworkBufferDescriptor_t**)(pPktBuff - ipBUFFER_PADDING);
configASSERT(((uint32_t)ppDcpt & (sizeof(uint32_t) - 1)) == 0 );
*ppDcpt = pxBufferDescriptor;

}

// debug functionality
void PIC32_MacPacketOrphan(TCPIP_MAC_PACKET* pPkt)
{
TCPIP_PKT_PacketFree(pPkt);
configASSERT(false);

}

// FreeRTOS allocation functions

// allocates a buffer that can be used by both:
// - the FreeRTOSIP (NetworkBufferDescriptor_t->pucEthernetBuffer)
// - the Harmony MAC driver: TCPIP_MAC_PACKET
// See PIC32_PktAlloc for details
//
// NOTE: reqLength should NOT include the ipBUFFER_PADDING (which should be == 10!)
// or the sizeof(TCPIP_MAC_ETHERNET_HEADER)
// These are added by the MAC packet allocation!
//
uint8_t* NetworkBufferAllocate(size_t reqLength)
{
return PIC32_PktAlloc(sizeof(TCPIP_MAC_PACKET), reqLength, PIC32_MacPacketAcknowledge, 0);
}

// deallocates a network buffer previously allocated
// with NetworkBufferAllocate
void NetworkBufferFree(uint8_t* pNetworkBuffer)
{
if(pNetworkBuffer != 0)
{
TCPIP_MAC_PACKET** ppkt = (TCPIP_MAC_PACKET**)(pNetworkBuffer - PIC32_BUFFER_PKT_PTR_OFFSET);
configASSERT(((uint32_t)ppkt & (sizeof(uint32_t) - 1)) == 0 );
TCPIP_MAC_PACKET* pPkt = *ppkt;
configASSERT((pPkt != 0));

    if(pPkt->ackFunc != 0)
    {
        (*pPkt->ackFunc)(pPkt, pPkt->ackParam);
    }
    else
    {   // ???
        PIC32_MacPacketOrphan(pPkt);
    }
}

}
#endif

This obviously resolved the compilation issue. However when it runs it falls over in NetworkInterface_eth.c line# 225 configASSERT( pTxPkt != 0 ); because pTxPkt is zero.

I’ve checked the value of PIC32_BUFFER_PKT_PTR_OFFSET in NetworkConfig.h (which was missing from 1.2.2 build btw), it’s 6 which as far as I can see is correct within the comments.

Am I missing something?

Andy

SarenaAtAws wrote on March 02, 2018:

Hello Andy,

Are you using the source code in the project downloaded from Github (https://github.com/aws/amazon-freertos)?
I strongly recommend that you start your development from that project because there have been many changes to switch between the Ethernet and Wi-Fi in the network layer code. We have also given the WINC1500 module extensive testing.
In regards to your errors in BufferAllocation_2.c, the Microchip Curiosity PIC32MZEF project in Amazon FreeRTOS version 1.2.2 uses the BufferAllocation_2.c located at lib\FreeRTOS-Plus-TCP\source\portable\NetworkInterface\pic32mzef\BufferAllocation_2.c, not the one under lib\FreeRTOS-Plus-TCP\source\portable\BufferManagement.

The macro “PIC32_USE_ETHERNET” needs to be defined at the project level.
You can do so under the Project Properties under Conf: [pic32mz_ef_curiosity] > XC32 (Global Options) > xc32-gcc.
Under Option categories choose Preprocessing and messages. Define PIC32_USE_ETHERNET in the Preprocessor macros section. Click the Apply button.

With Ethernet only enabled you will not see the WILC1000 initialization message in your serial terminal.

Here is an example success output on Ethernet:

AWS Validate: no valid signature in descr: 0xbd000000
AWS Validate: no valid signature in descr: 0xbd100000

AWS Launch: No Map performed. Running directly from address: 0x9d000020?
AWS Launch: wait for app at: 0x9d000020
0 0 [None] Seed

for randomizer: 849859286
1 0 [None] Random numbers: 00001437 000042DA 00003C5C 00002CFD
2 7526 [IP-task] vDHCPProcess: offer a3cd46fip
3 7535 [IP-task] vDHCPProcess: offer a3cd46fip
4 7535 [IP-task] Creating MQTT Echo Task…
5 7536 [IP-task]

IP Address: 10.60.212.111
6 7536 [IP-task] Subnet Mask: 255.255.252.0
7 7536 [IP-task] Gateway Address: 10.60.212.1
8 7536 [IP-task] DNS Server Address: 10.43.23.72

9 7537 [MQTTEcho] MQTT echo attempting to connect to xxx.iot.us-west-2.amazonaws.com.
10 7537 [MQTTEcho] Sending command to MQTT task.
11 7537 [MQTT] Received message 10000 from queue.
12 8745 [IP-task] Socket sending wakeup to MQTT task.
13 10358 [MQTT] Received message 0 from queue.
14 10408 [IP-task] Socket sending wakeup to MQTT task.
15 10408 [MQTT] Received message 0 from queue.
16 10903 [IP-task] Socket sending wakeup to MQTT task.
17 10903 [MQTT] Received message 0 from queue.
18 10903 [MQTT] MQTT Connect was accepted. Connection established.

Regards,
Sarena

Edited by: SarenaAtAws on Mar 1, 2018 4:52 PM

Edited by: SarenaAtAws on Mar 1, 2018 4:54 PM

Edited by: SarenaAtAws on Mar 1, 2018 4:54 PM

Edited by: SarenaAtAws on Mar 1, 2018 4:55 PM

Edited by: SarenaAtAws on Mar 1, 2018 4:56 PM

andylong0206 wrote on March 02, 2018:

Hi Sarena,

That’s fantastic. Thank you for your support. Indeed I pulled the latest project from Git. I can see that there has been a lot of changes. I’m all up and running now :slight_smile:

Andy

andylong0206 wrote on April 02, 2018:

Hi,
I have it working with harmony 2.04. It’s a pain as you have to put the project into the harmony project structure. I do understand why this demo isn’t harmony structured from AWS out of the box because it’s for multiple targets. It’d offer to share my base project, but unfortunately there’s some application specific stuff in there which I can’t share. To be honest, it didn’t take to long to do, I had to do it twice because when I started out, there was no WiFi support, I then tried to implement my own driver for the MWRF24 which is onboard the curiosity board only to find it really didn’t work too well, and the actual driver from microchip is not fully implemented. Fast forward to a later project from AWS with WiFi implemented albeit a direct WiFi module, restructured into harmony again and all is good.

I how this helps.

mrpackethead wrote on April 02, 2018:

What would i need to do to get FreeRTOS working with Harmony? Seriously its a massively disappointing that this isn’t a harmony based setup, as we have tons and tons of work already done in harmony, * using free-rtos already.

andylong0206 wrote on April 03, 2018:

Hi,

I started with a baseline harmony configuration with all the core elements needed. I found that the configuration packed from AWS is out dated to where it’s evolved to. Once I had a configuration, I then ported over the FreeRTOS stuff, third party libs, etc. The biggest pain was actually adding the paths into the GCC compiler. Watch out because a couple of files have been duplicated and modified. I fell into that trap, so check the path of origin. Read through this thread as it explains a couple of issues I had. I have to say AWS support is good.

Oh, one more thing start with the AWS demo master in their GIT, I think it’s version 1.2.2, or may have been up issued since as I pointed out a couple of bugs.

I hope this helps.

mrpackethead wrote on April 02, 2018:

Well, thats great to hear that it as at least possible. I also understand why they did what they did. Harmoney does have its challenges of course, however its got a lot better and you can build apps very quickly now once you’ve got past the initial learning curve.

The real challenge is that a Curosity board is yet another dev board, and if you dont’ have one its just annoying when you have plenty of other ones! Also, ‘real’ projects are not build on dev projects, so you have to build a new set of configurations etc.

I started to have a look at the code base and i saw that they did at least start with some harmony configurations.

When you created your harmony based project did you start with the harmony project and then move the FreeRTOS things into it, or did you do it the other way around? I am guessign it will be better to bring the FreeRTOS into a harmony project…

mrpackethead wrote on April 03, 2018:

Did you use the FreeRTOS ip stack or the microchip one? Im thinking that it woudl be great to get a ‘stock’ harmony project on a PIC32MZ Ethernet Starter Kit, which pretty much everyone has.

Edited by: mrpackethead on Apr 3, 2018 1:18 AM