It's not always easy to distinguish a blazer from a suit jacket. Certainly sometimes it is easy. But a dark blue blazer with subdued buttons worn with dark grey pants will look very close to a suit. And I doubt anyone would notice much or care.
On the other hand, here in South Florida you get to see old geezers in their green, orange, pink, yellow, red, or purple blazers with shiny metallic or pearl buttons. It's pretty obvious it's not a suit coat.