Connect your server from Windows using mRemoteNG
mRemoteNG is the best remote connections manager I can found under Windows. It is open source and currently using PuTTY for SSH connection.
Connect to your server using password
Login the ubuntu server using password is quite straightforward. Just add a connection and select SSHv2 fill up the hostname/IP, Username, Password.
Connect to your server using SSH keys
generate new SSH key
Here is a good tutorial
mRemoteNG Menu => Tools => Options