The speaker is telling us about the feeling he gets, has always gotten, and will always get when he sees a rainbow in the sky: his heart rejoices. He says that if he were ever to stop feeling this joy, he'd want to die.
He presents the paradox (contradictory statement) that the child is the father of the man. In other words, our adult selves still contain the kernel of our childhood selves. He wants his days to be, perhaps, like the days of a child, filled with—and tied together by—a reverence for nature.