Don't get caught up in the rainbow part of "My Heart Leaps Up When I Behold." This poem is about the speaker's identity as much as it is about nature, even if nature is an important part of this identity. The speaker shows us a moment: he sees a rainbow, and his heart leaps up. Then he uses that moment to establish his entire identity, from when he was born to what he wants to be like when he grows old. He says he wants his days to be glued together by what he calls "natural piety," an almost religious respect and love for nature.
I am Nature Boy! The speaker hopes that rejoicing in nature will remain a constant part of his identity even as he ages.
He's not the only one either. The speaker feels that childhood plays an important role of shaping the identity of adults.