when you try to direct connect are you including the port number? because if its not the default port then you need to put the ip and the port.
example: 23.66.234.65:30121