Seems to me we are dealing with a unique situation here.
First of all, I think we all know that for serving personnel, it is inappropriate to wear another ship's ball cap onboard your current ship. By extension, I would consider inappropriate to wear another ship's ball cap ashore when I know I have even a slight chance of running into current shipmates (I might make an exception here for an "historical" ball cap from a ship retired from service of special significance to the member).
However, for civilians, visitors, family members and any other person, not currently serving in the Navy, wearing any ship's ball cap, properly and respectfully, is always acceptable.
In the present case, if I understand well, Pegcity was an NOAB and attending at the pub with an officer. Two things: Some NCMs may have had a little too much and decided to look for trouble or, seeing him with an officer they recognize, they may have decided to "test" that officer a bit, for some reasons totally unrelated to Pegcity.
In either case, Pegcity, I would tell you to continue to wear your ball cap with pride and just ignore such interventions, especially in pubs.