We know the idea of carrying a person's heart is spoken in a figurative sense in Cumming's poem. You really can't go around carrying actual hearts. Folks tend to freak out at you. But the idea of carrying a lover's heart at all times seems to be the speaker's main point. Visually and contextually, Cummings furthers this idea with his use of parentheses and squishing of words.
Lines 1-2: The poem opens with the title and the idea of carrying a lover's heart. The squishing of the parenthesis with the adjacent word ("me") creates a sense of unity between the lovers.
Lines 2-3: We start to see some ambiguity in words like "anywhere," which creates a universal kind of vibe. Specifics don't matter because the only thing that's important is the speaker's love.
Line 15: In a pretty conventional fashion, Cummings ends his poem with the title (though it's not word for word exact). But by beginning and ending with the idea of carrying hearts, we know it's the most important idea of the poem.