Loneliness is a feeling, just like anything else. But it's not the only feeling mentioned in this poem. We've got everything from "joy" (7) and "sorrow" (6), to more complicated things like love. There's also a brief discussion of "passions" (4), which more than anything makes us think of powerful feelings. Despite the negative emotions that are foregrounded, then, there are a few positive things to think about.
Lines 3-4: The speaker clearly has lots of "passions," or feelings. The idea is that there is more to him than just loneliness. These feelings just come from a different source ("spring" is a metaphor for source here).
Lines 5-6: In a neat little chiasmus, the speaker says he hasn't taken his sorrows from the same source either. The balance of passion and sorrow here is just like the poem's balancing of "good and ill" (11), torrent and fountain (13), and death and life.
Line 8: Despite everything, the speaker has strong feelings of love for… well, something, but he does it all alone. Note the ambiguity here. The speaker is either loving by himself, or he's the only who loves, as in "I alone am the one who loves." The poem leaves both interpretations open.