Within walking distance of the port: the Kura Hulanda museum (history of slavery), which is on the same side of the bay (Otrobanda), and the historic district (Punda) including the Mikvé Israel-Emanuel Synagogue, the oldest synagogue in the western hemisphere that is still in use, with a small museum, and the Maritime Museum, across the Queen Emma swinging footbridge. (When the bridge is open for ships, there is a ferry service, but you'll want to see the bridge operate.) The Curacao Museum, on the Otrabanda side, is a long, hot, boring walk from the port, but if you're interested the history of aviation, it has the cockpit of the Snip, the Fokker F-XVII that flew the first airmail from the Netherlands to Curacao in 1934.
For lunch, try the Governeur de Rouville restaurant, very near the Kura Hulanda museum and facing the bay; there is some balcony seating. Have the banana soup if it's on the menu.