It's a bit more credible than that. It's moved less than 20 times since the invention of the atomic bomb. And its most recent move was something like 5 minutes BACK. As well, nuclear war & climate change are definitely valid concerns for a possible extinction of the species.
No, it's not as though the world is suddenly going to end if they ever decide to put it at midnight (which I'm not even sure they can). It is symbolic, for obvious reasons, the time left until the extinction of humanity is impossible to actually quantize. But that doesn't mean it's not worth putting into some kind of perspective, & it's disingenuous to call it "fear mongering."