Swab came back negative but the sore looked exactly like herpes

It can, and timing is the usual reason. A swab looks for virus present in the lesion, so the amount you can detect drops as the sore heals. Swabbed early in a fresh lesion it is a strong test. Swabbed once it has crusted over it is much less reliable.

How many days in were you when they took it?

That would do it. Two options from here: if it recurs, go in as early as you possibly can, ideally in the first day or two while it is still fluid filled, and ask them to swab it then. That is the test that actually answers the question.

The other option is a type specific blood test, but discuss the timing with your clinic, because antibodies take time to appear and a test taken too soon after a first episode can be negative while the swab would have been positive.