There are two different firewall options included with Windows, and with your Windows Dedicated Server, the one you are most likely aware of is the standard Windows Firewall, and then there is the basic firewall that is located in Routing and Remote Access. Since most of you won't have a need to enable RRAS, I will cover getting the Windows firewall setup to cover your needs. I will cover RRAS at a later time, since it can also be sued for setting up an easy VPN, or even make a Windows machine act as a full blown router.
If you have a control panel, such as Plesk, the Windows firewall is automatically managed for you, and it has a set of policies by default that would be ok for most. But if you really want to lock things down, as much as you can with the firewall anyhow, you are going to have to login to the server via remote desktop. You can find the firewall settings in the control panel, just like you would on your home machine, assuming it is running Windows Vista or XP.
When you are in the properties box for Windows Firewall, the Exceptions tab is where you want to be, this is where you decide which ports and programs get to have data transmitted through the firewall. Windows firewall operates with a default deny all setting, so setting the exceptions is very important is you want ports to be accessible on your server, otherwise you wouldn't be able to do much of anything with it. Now, the most important port to have opened on a windows server is 3389. This is the port that your computer uses to connect to the remote desktop service on your server. If this is not open, then you will probably be putting in a ticket with us to find out why you can't connect to it. If you are running things like a web site, DNS, mail, or ftp, you will need to open their ports as well. For website serving, you want to have 80 and 443 open; DNS operates on 53, SMTP (sending mail) on 25, pop on 110, 143 for IMAP, and 21 for ftp. If you are interested in an exhaustive list of ports, and what programs use them by default, check out this link.
You can also make exceptions for entire programs, so if you have a game server that operates on multiple ports all of the time, you can add an exception to allow any port that program is listening on. This can be done by clicking the Add Program button in the exceptions tab instead of Add Port.
This should only be a firewall you use if you aren't too concerned with security, there are many more feature filled firewalls available to you. As far as software firewalls go, this is the bottom of the barrel, followed by RRAS, then the many subscription software firewalls out there. Though, if you are going to pay for a software firewall, might as well get a hardware firewall, since they are more robust, and take the strain of filtering traffic off of your server. If you think you would be interested in one, let your account executive know, and they will be glad to get you a quote.