Physical Danger
Unless you are in the movie Deep Impact and there's an asteroid heading toward the earth that will cause total annihilation, you are pretty safe. And, even then, we would all be in the same boat. You spend most of your time looking at the sky, making observations, doing math, and talking physics...your biggest threats are a paper cut or pink eye from that telescope you share with Phil (who looks like he hasn't showered since his first postdoctoral).
If there's anything threatening your life as an astronomer, it's probably doomsday for all of us anyway, so don't worry about it.