The Israelites had an idea about Heaven, but it was nothing like how we think about it today. Back in the day, Heaven was a purely divine domain, not somewhere you went when you died. Basically, it was the spot where Satan and God hung out, chatting about humans.
There had to be some conception of the afterlife, right? Yep. And that's where we get Sheol. Sheol is the Biblical underworld—think Hades in Greek mythology. When you die, you go to Sheol, a grim underworld where God forgets about you and your voice is silenced. We never really get a precise idea of what Sheol looks like, but it's definitely not a destination resort.
Sheol is especially important in Job because, as you may remember, his family is dead, his property is burned by fire from the sky, and he is himself diseased. Sounds like he has one stop left on the train, right? The ancients sure would have thought so.
This begs the grim question, "What's the point in waiting around?" In fact, Job's wife asks him flat out: "Do you still persist in your integrity? Curse God, and die" (2:9). But rather than die, Job chooses to stick around and ask God about life on behalf of mankind.
[Want to see Sheol for yourself? Check out 10:21-22, 11:8, 14:13, and 17:12-16.]