Terence Young was as close to actually being James Bond as a director could get.
He served as a tank commander during World War II (so maybe that was him piloting Dr. No's dragon around Crab Key). He almost got hit by the car flying off the cliff during the big car chase scene, yet was pulled away at the very last second. And he came up with the one-liner "make sure he doesn't get away" that Bond utters after dropping off a car with a dead body in it. (Source)
We'd totally watch a movie about this guy's real-life adventures.
Young also brought in much of the talent made iconic by Dr. No. He discovered Ursula Andress and cast her as Honey. He had worked with both Sean Connery and Lois Maxwell (Moneypenny). And although the film initially had "Underneath the Mango Tree" as its theme, Young thought that was so stupid, he brought in John Barry to compose the Bond theme we can all still hum from memory.
Young continued honing Bond in the sequel, From Russia with Love, before taking a break. Although Bond's reins were passed to another director for Goldfinger, Young would return to direct Bond's biggest adventure, Thunderball, in 1965.
He later directed other films, but none were as successful as 007. Just as we can't imagine Bond being anything but a spy (or maybe a model for Tom Ford), it's hard to imagine Young directing anything but Bond.