Perhaps you could avoid it in the Navy. In the Army, it wasn't an option. Actually, that's not entirely true. It technically was an option...that refusal of would result in a GOMOR and the end of your career. In other words, you just did it. No one I knew personally had any sort of serious reaction to it outside of a sore arm which many vaccines cause anyway. However, there's no way to look deeper into this since there are currently no peer-reviewed studies of long-lasting effects of the anthrax vaccine that I'm aware of.

