Maybe, maybe not. It varies from port to port and even cruise to cruise. The people who are checking the tickets for arrival times don't know how you got there.
I certainly wouldn't count on being able to get into the terminal early, especially if you would not handle waiting outside very well (i.e., heat, standing, impatience, etc.)