Physical Danger
You're hanging around a lot of sick people, so depending on the contagion factor, you're always dealing with some risk of getting sick. But since anesthesiologists don't physically rummage around the innards of patients like surgeons do, there's less risk of infection.
Your office will also likely contain all sorts of sharp and otherwise pointy gizmos, gadgets, and tools laid out beside shelves of drugs designed to either make people dopey or knock them out. It's not exactly the safest-sounding recipe in the world, but hospitals know the dangers and do their best to protect practitioners. Be careful, and you'll be just fine.