Stress
People who work with math face unique stresses which might seem to be a mild annoyance to the lay person. Every day you will go to work and solve problems to questions that haven't even been asked yet. You'll be expected to explain complex algorithms and equations to people who have to do the actual work of building the darn thing.
Sometimes, you will work for days or weeks on end analyzing the same seemingly unsolvable problem, only to find the solution was a simple reworking of programming that you could have gotten done in five minutes. Or maybe you won't be able to figure it out at all, and have to scrap a perfectly good artificial nasal cavity.
You'll also have to deal with everyone around you being terrible at math by comparison. Sure it may make you feel superior at first...but it fades. Fast.