I am fairly sure that you will not be able to bock this without disrupting other traffic, as it can still go over http, so you would have to block all traffic. I believe that the only way to effectively block it is with a proxy. Microsoft publish this article
http://support.microsoft.com/kb/925120 which explains how to do it in ISA server. I have found the following entry on a blog page at
http://blogs.techrepublic.com.com/networking/?p=308 which explains how to do it with Squid.
I’ve been looking for a simple and reliable way of blocking MSN Messenger without stopping Windows Update from doing its job. After trying to refine blocking based on both IP address and hostname, I decided that the only way to reliably block it is using the mime type tag and HTTP gateway using Squid. I found reference to the tag application/x-msn-messenger while reading a Microsoft KB article that describes blocking MSN Messenger with ISA Server 2000. Squid gives the ability to block based on MIME type and request content using access control lists (acls).
If you’re already running all of your Web traffic through a Squid proxy server, then updating your configuration to block Messenger is as simple as adding four lines to the Squid.conf file:
acl msnmime req_mime_type ^application/x-msn-messenger
acl msngw url_regex -i gateway.dll
http_access deny msnmime
http_access deny msngw
hope this helps.
Mark.