"To Althea, from Prison" really says it all as far as this poem is concerned. The title tells us that it is written from the perspective of a prisoner and that it is addressed to a woman named Althea. This name ultimately derives from the Greek name Althaea, which means "healer." This shows that the speaker sees his lover as a comforting presence. But it goes a bit deeper than that. The two words of the title—prison and Althea, or healer—illustrate the poem's major theme: how to heal oneself in prison, or rather how to find solace or comfort even within the confines of a cell.
The way one goes from prison to "Althea"—from the pain of confinement to the pleasures of freedom that the speaker celebrates throughout—is by using one's imagination. The speaker constantly dreams up imaginary scenarios (Althea being delivered by Love, he and his buddies drinking, etc.) and suggests that turning a prison into a cozy, comfy "hermitage" is only a matter of having the mental will-power to do so.