The code in avahi-core/socket.c attempts three methods of multicast source routing. The first, IP_PKTINFO, is Linux specific. The second, IP_SENDSRCADDR, is subtly wrong, at least on my FreeBSD 6.1 setup.
To reproduce the issue, I configured two interfaces via autoipd and tore down all default routes, leaving only an IPv4LL network with two interfaces on each VM. I then kicked off avahi-daemon. avahi-daemon complains about "No route to host" immediately, specifically for the 224/4 addresses. I tried to hack around this using "route," but there's really no way to enter duplicate routes (I suppose I could've hacked around with mrouted, but chose not to go through that pain). If I entered a manual route for 224/4 for one interface, I could see mDNS traffic on that interface, but not on the other. So, I started digging in code.
It looks like for my setup, IP_MULTICAST_IF is the correct way to make this happen. There is some rumblings on the mailing list about an IP_SENDIF, but this won't work for 6.1 (I'm not even sure it's implemented yet).
Even with IP_MULTICAST_IF, I had to "route add -net 224.0.0.0 -iface lo0", otherwise I'd get "Network is unreachable" errors from sendmsg.
I have a rudimentary patch that just reverses the elif order, but this is probably the wrong direction for portability. I'll try to figure out if there's a better way and submit a full patch (though any assistance would be appreciated). (Also, I wish the file descriptors were split up by iface, that would make the setsockopt more palatable).
This might explain ticket #114 or ticket #66 as well.
