One piece of advice would be to get there as early as possible. The boat to Liberty Island and Ellis island goes from Castle Clinton at Battery Park (it's a circular hop on/hop off ride). We were there before it opened and sorted our tickets. It is quite a lengthy security procedure - think airport security. We were on the first boat of the day, went to Liberty first and then Ellis island. When we got back at around 1.30 pm the queue for the boat was easily 2 hours+ long!!
Your trip probably won't take as long if there is no access to the statue but I should imagine it will still be very busy!