We have done this trip multiple times with our 2 small children (now 4 & 7 but we have been doing it since they were 2 & 5) Its a nice & easy drive, we choose to head down the day before the cruise and stay a hotel in the "Parsippany" area as its about 45 minutes to Cape Liberty and hotels are reasonably priced. We park at the cruise terminal because by the time you factor in Ubers to & from the port converted into Canadian, it ends up not being worth it. We leave our house about an hour north of Toronto about 5am to put us at the border about 6:30a-7a where we never typically wait longer than 15 mins crossing over.
The drive home is just as simple, we are usually off the ship around 8ish and home by dinner time. We just made this trek in May for the NCL Joy and will be heading out again in Sept for the Symphony of the seas
We are sept 29-Oct 6th where we could have snow or it could be +25 lol