Stress
The worst stresses about this job may be the physical stressors on your body, specifically your eyes, hands, and back. If you do this even semi-professionally, you can expect to be hunched in a chair over a worktable or desk, looking very closely at tiny bits of metal and leather, and positioning your fingers ever-so-delicately in order to build or repair whatever it is you're building or repairing.
Forty hours a week of this can get to a person; even office workers, after all, get their lunches away from their claustrophobic cubicles. Make sure to step away from the time bits every once in a while.
The other big stress is on your wallet. If you make watches, can you sell enough of them to pay your bills? If you repair them, can you get enough repeat business to keep the lights on in your workshop? This career is already in a very small and specific market, and even when everything is working fine, you might not be making much cash.
Working for someone else means you won't have to worry about the business responsibilities, but then you're beholden to their needs, which comes with its own level of stress. Whatever the case, avoid struggling through these leaner periods by saving something when work is more plentiful.