+TCP IPv4 multicasts and IGMP

Libraries
1

Hi all,
I found a topic here that said that Multicast and IGMP support is not planned, and I need it fairly quick, so I took it upon myself to add support for it to FreeRTOS+TCP. Please consider this a preliminary discussion.

I need implementation for sending/receiving UDP/IPv4 multicasts and IGMP snooping, pruning, and a querier. Currently, FreeRTOS+TCP only supports sending UDP datagrams to multicast IP addresses, so for the past week or so, I’ve been working on adding proper support for multicasts and IGMP.

My general question is: Are the maintainers and the community interested in bringing these features into FreeeRTOS+TCP “main”?

Below are some details on my vision of what needs changed or implemented to achieve this goal:

  • Add at least the following socket options for UDP sockets: SO_IP_ADD_MULTICAST, SO_IP_DROP_MULTICAST
  • Keep track (per socket) of which multicast the socket is subscribed for and be able to add/remove subscriptions
  • Add proper filtering for sockets that are subscribed for receiving a multicast group → Done
  • Add a network driver function for adding and removing multicast MACs to the EMAC hardware ( the failsafe approach here is to receive all multicasts if multicast support is enabled )
  • Support for multiple groups per socket as well as multiple interfaces ( aim is to make this code portable to “multi” branch )
  • IGMP <–> Socket interface so that IGMP knows what the local host is registered for.
  • IGMP v1/v2/v3 frame parsing
  • IGMP timers for sending “report” frames
  • optional IGMP querier
  • network driver hooks for multicast pruning ( only applicable when using a managed switch that is under the MCU control )

There is a lot more details and decisions behind every one of these bullet points, but let’s keep it simple for now.

I have a working first draft of the socket-related stuff and I haven’t yet started on the IGMP part. I will be able to share sample code in a few days as I’m still cleaning up and testing. I will also probably need a little help on some performance-related decisions as well as guidance on the feature-set what would be best for the general FreeRTOS+TCP user.

Is there general interest in this? Again, I’m already doing it because I need it, but do you think a PR along these lines will be met with any interest or is it totally out of the question?

3 Likes
2

Maybe a big change like that can live on a branch or in the labs while it’s tested properly and the implementation polished and ironed out.

3

quick update:
The socket options are done, adding/dropping subscriptions to multiple groups works.
EMAC multicast filter functions work for my hardware ( same70 )

I’m starting to work on IGMP v1/v2/v3 which should be fairly straight-forward, but will surely take me a few days. I’ve started working on responding to multicast queries, but later, I’ll be implementing extra functionality related to IGMP snooping. Leavinf IGMP snooping for last because using it depends on using a managed switch instead of a straight-forward PHY chip.

4

Wow, that’s a whole lot. I can’t speak for all developers involved, but I think it would be a welcome extension of the stack. And preferably, we should add it to the /multi branch.

We are still working on both branches, main and multi, to increase the quality and safety of the code.
And so I hope that the core source files won’t need many changes. I propose to add a new module called FreeRTOS_IGMP or so, which will handle all multicast traffic and the IGMP memberships. Of course we will add hooks and callbacks in the main source, depending on some new macro like e.g. ipconfigUSE_IGMP and/or ipconfigUSE_MULTICAST.

Your observation was correct, there is some basic support for outgoing multicast traffic, the message eNetworkTxEvent can be sent to the IP-task. But received packets will mostly be dropped.

These 4 functions can be of help:

BaseType_t xIsIPv4Multicast( uint32_t ulIPAddress )
BaseType_t xIsIPv6Multicast( const IPv6_Address_t * pxIPAddress )
void vSetMultiCastIPv4MacAddress( uint32_t ulIPAddress,
								  MACAddress_t * pxMACAddress )
void vSetMultiCastIPv6MacAddress( IPv6_Address_t * pxAddress,
								  MACAddress_t * pxMACAddress )

I guess that you would need 2 call-backs, one for IGMP packets, and one tp forward received UDP/multicast packets?

5

I am working on a project that also needs Multicasting and would like to help with testing, which often leads to fixing code :slight_smile:

I also need more than one iface (Ethernet and USB)

Where can I get a copy/clone?

You need it “fairly quick” when do we need to have this finished?

Tom/W2VY

1 Like
6

Hi Thomas,
I got side-tracked from this for about a month, but I’m slowly getting back to this multicast stuff.
I have created a “first draft” type of a branch based on the main branch. You are welcome to take a look at it here but just keep in mind I might have to drop it an re-create it.
Lately, I’ve been trying to figure out how some cisco SG250 switches handle IGMP snooping, IGMP queriers and so on and I think I’m starting to figure out how all that multicast stuff tries together. The internet is full of basic multicast info, but I have a few issues with understanding the nitty-gritty details of a proper implementation. I just created an IGMP querier as well and I have the code to respond to IGMP queries, but not all of it is pushed to that branch.
Bare with me for a few days and I should be able to post more info and details here.

1 Like
7

@w2vy and others that may want to use multicasts in their implementations… I have a question. Can you elaborate a little on what you are doing with multicasts?
I’m curious, but I also really need to know if you envision ever needing the ability to subscribe one socket to multiple group addresses?

My current test implementation allows for a socket to subscribe to multiple multicast addresses. I did this because normal TCP stacks like on Linux and Windows allow this, however this causes a much more CPU-intensive implementation because every UDP socket needs a list of subscriptions and parsing a list is always going to be slower that just allowing a single subscription per socket.

Let me know what you think guys. I can’t think of a normal use case where a single socket would need to receive many multicast addresses.

8

I am sorry for the late reply!
We are mostly using it for local discovery of our devices on a network.
Also some configuration of the devices as well.

I am pretty confident that I agree that one multicast address per socket is sufficient.
We also use different ports so we can keep the traffic isolated (easier to keep the traffic straight)

Tom

9

Thanks for getting back with me @w2vy

That’s pretty cool that you’ve decided to use multicasts for discovery. I usually use broadcasts for that. If you are interested, you can take a look at my second draft here
I remember you mentioned that you are using the ipv6_multi, so my branch might be af limited use to you.
I will be updating it to drop the multiple groups per socket and replace it with single group per socket. This loosens the requirements on the code that filters incoming frames.

I’m curious how you are using multicasts with the regular version of +TCP. Did you just hack around to let the multicasts in and hardcode the by-address filtering?

In the link above, you might find it interesting that once you register a socket for a multicast group, there will be an unsolicited IGMP report spit out and also if you have an IGMP querier on your network, the stack will respond to these as it should. This is very important because if your devices are placed on a network that does IGMP snooping, your multicasts will get filtered by the switches. This new IGMP code ensures that your device informs the switches that it is interested in multicasts.

If you decide to try my code, add the following for your udp receiving socket after you bind it:

struct freertos_ip_mreq mreq;
mreq.imr_multiaddr.sin_addr = FreeRTOS_htonl(0xEFEFEF01);
mreq.imr_multiaddr.sin_len = sizeof( struct freertos_sockaddr);
mreq.imr_interface.sin_addr = FreeRTOS_htonl(0x00000000);
mreq.imr_interface.sin_len = sizeof( struct freertos_sockaddr);
FreeRTOS_setsockopt( MCastRecvSock, 0, FREERTOS_SO_IP_ADD_MEMBERSHIP, ( void * ) &mreq, sizeof( mreq ) );

Let me know if you have any questions on how to try out my code.

1 Like
10

We need 2 interfaces and do not have an IPv6 requirement at this time.
We aren’t using multicast just yet (since it doesn’t work) but it is a requirement for our final product.
(We are using on other platforms)

Yes the IGMP support is important!

We will give it a try!

Tom

11

Hello @epopov , I work with @w2vy and we use Multicasting for local discovery of our devices on a network as discussed above.

I tried your code, “second draft” linked in this post for our purpose and I want to give you the status update that it worked with no change at the core of the code. I only needed to implement these two interface APIs in order to perform the test:
BaseType_t xEMAC_AddMulticastAddress( uint8_t * pMulticastMacAddrBytes )
BaseType_t xEMAC_RemoveMulticastAddress( uint8_t * pMulticastMacAddrBytes )

I also used the regular
BaseType_t xNetworkInterfaceOutput( NetworkBufferDescriptor_t * const pxNetworkBuffer, BaseType_t xReleaseAfterSend ) in stead of xNetworkInterfaceOutputToPortX(…)

The other thing I noticed is that if
eFrameProcessingResult_t eConsiderFrameForProcessing( const uint8_t * const pucEthernetBuffer ), implemented in “freeRTOS_IP.c” is used, I believe MULTICAST mac addresses has to be included in the filter.

This is just preliminary test and I didn’t go through the details of the implementation yet, I just wanted to give you the status update in this response and thank you for all your hard work.

Endale

12

Hi @selamta .
I really appreciate the feedback. I apologize for the lack of documentation on this, but you are absolutely correct about xEMAC_AddMulticastAddress and xEMAC_RemoveMulticastAddress. These are functions of the network driver and need to be implemented.

In regards to xNetworkInterfaceOutputToPortX… This is specific to my implementation and I would have gotten rid of it if this was a more “official” release. In my case, my ethernet hardware is a 3 port switch, so I have reworked my network driver to allow me to force packets like LLDP to go out certain ports. Yes, please use the regular xNetworkInterfaceOutput function.

As for eConsiderFrameForProcessing…
If ipconfigETHERNET_DRIVER_FILTERS_FRAME_TYPES is set, then eConsiderFrameForProcessing is an empty macro. In my case, I cannot setup my hardware to do perfect filtering because it’s based on a multicast hash match. That means I’d have to have a malloc-ed list of registered multicasts in my driver code and filter there, but I didn’t feel like making the code that complicated.
if ipconfigETHERNET_DRIVER_FILTERS_FRAME_TYPES is zero, then the stack has to do the filtering, but in this function I don’t have enough information to filter based on multicast address. This information is in the UDP socket, so I left this filter for later, once the UDP socket has been located.
Please let me know if the above makes sense to you or if I maybe misunderstood your comment on this function. I’d like to filter packets as early as possible, but multicasts are a bit hard in that regard and I’m not convinced that digging through linked lists is a great idea for the network driver.

Once again, thanks for the feedback. I really appreciate it.

13

Hi @epopov

I have started working and testing with your code from here https://github.com/evpopov/FreeRTOS-Plus-TCP/tree/Multicasts_Draft_2 and so far appears to work great.

14

Thanks @DBSS
I appreciate you keeping me in the loop. I haven’t touched this fork in a while due to more pressing things at work, but I will be getting back to it here in the next 2-3 weeks. My plan is to bring in the latest changes from upstream and also to probably remove the ability for one socket to subscribe to multiple multicast addresses. Having a socket only receive a single multicast group will make the packet filtering code much faster and more stream-lined. I’m doing an unofficial survey about this feature, so please let me know if registering multiple multicast groups to a single socket is a feature that is important for you.
I think this feature is kind of useless, but I need feedback.

15

In our application we use different ports under the same multicast address.

So each socket only has a single multicast group that it belongs to.

Tom

16

Hello @epopov! I’m very glad to have just come across this thread.

I’m also developing an application that uses UDP multicast. We only need a single multicast address per socket. Especially if it simplifies and speeds up the driver code I think removing the multi-subscription is a good idea.

In our current implementation we’ve been getting away with hard coding the mac address filtering & not participating in IGMP because the network we’re operating in is controlled and doesn’t do IGMP snooping.

17

We only need one multicast address per socket.

18

Is there any work happening on this behind the scenes? I might be able to help out with refactoring out the multiple multicast group registration code if you need a hand.

19

@restep, thanks for your interest on this topic.
To answer your question, I haven’t done anything lately purely because I have more pressing issues at work. I may have been real quiet on the subject, but in reality, I will have this updated within one month because I’m running up against some due dates at work and I need to have this multicast code functional by mid March, meaning I have to rebase it or merge it with the latest upstream code and remove the multiple address stuff. To be honest, I don’t need help with refactoring. I can knock that out real quick.

What I do need help with is for someone to suggest how I should go about maintaining this code. My problem is that as of right now, there is no plan for this fork to be pulled into the main repository. I’m fine with that, but I don’t know how to handle that properly. Do I rebase the for on the latest TCP stack commit and then push the changes to my for? Wouldn’t that mess up everyone else that has checked out my fork?
The alternative is for me to periodically merge my fork into main and push my fork to my repository… This way, people that have checked out my for will be able to stay on it as long as I regularly merge it with the changes on the main branch.
My third option is to maintain a set of patches and whenever someone is interested in using multicast, they could checkout the main repository and apply the patches.
Please guys, help me out with this decision as it has bothered me from the very beginning. I’m quite familiar with embedded coding, I’m just not experienced enough with the proper ways to collaborate on open-source projects.

20

I’d prefer the changeset being pulled back into FreeRTOS + TCP (makes it easier from my company’s policy standpoint). But I’m also inexperienced in open-source collaboration so I’m not sure how hard a lift that is.

If its not going to go back into the main +TCP repo: I’ll probably be most interested in a set of patches I can apply on company’s own checkout of the FreeRTOS + TCP repo. But whatever works for you honestly! I can make a patch-set from any of your other options :slight_smile: