Don't know how it works in the US but wearing a UK military mess kit outside the military environment would require special permission from the relevant military authority. I'm sure it will be fine - you guys are much less stuffy - but I'd hope no RN, Army or RAF retirees would do the same - I think it would just strike us as wrong, somehow. (I'm a retired RAF Officer, BTW.)