Yup, take the ferry to St Georges. The Bermuda Aquarium, Museum & Zoo is pretty nice.
And yes too cold to go in the water unless you have a wetsuit or are a crazy teen
The dockyard will get old pretty fast and you'll get bored of it. I would come up with a plan. Looks like it will be 70 next week with a low if 65 so not too bad of temps, however on or near the water will be cooler