Stress
As a capable traveler and experienced outdoors person, most of your stresses will not come from the adventure part or travel part. After all, you know seventeen different kinds of knots, thirty-six species of bugs you can eat to survive, and flying has become second nature. Learning it all was stressful; going through the motions is less so. For the seasoned adventurer, even sudden changes in the weather will be no match for your quick thinking and expertise.
Then there's the leader part, and that's where everything goes to poop. Your clients, or customers, or pains in the butt (depending on how new you are to this) will be the best and worst part of every trip, and that's true even if every last one of them is a wonderful person with really great jokes.
Coordinating a group of people to travel for sometimes hundreds or even thousands of miles into the middle of nowhere can be an absolute mind-numbing migraine headache on wheels. There are airlines to book, insurance paperwork to fill out, permits to permit, medical clearance to receive, and you just have to hope your customers can get to a dang airport on time.
Then, there's the journey itself. They'll want to touch things they aren't supposed to touch. They'll want to eat things that should never go into their mouths. Or they'll just sit around all day complaining, moaning, and dreaming of their Xbox.
And these are the grown-ups we're talking about. That's why patience is such an important skill to have; it's not going to be good for your career if you "forget" about the obnoxious New Yorker and "accidentally" leave him in the jungles of Madagascar.