We definitely need a little historical Irish background to really understand what a "salley garden" is in the poem. But once we understand that "salley" is just another word for a willow tree, we understand that the speaker is honing his super-romantic poetry chops in order to give us a lesson in love. Everything about the poem's landscape then becomes willowy and perfect for young lovers.
Lines 1-2: The poem opens with the speaker meeting his love down by the salley gardens. So immediately we can imagine a romantic landscape with weeping willows dangling overhead.
Lines 3-4: As the girl passes the salley gardens, we notice she has little snow-white feet. So we get a sense of innocence and youth to accompany the speaker's description of the girl and their young love.
Lines 5-6: The girl uses the simile of leaves growing on trees here, so the gardens help to suggest a life lesson the speaker is supposed to learn. The ease of the leaves growing is how the speaker should take love, but of course, he doesn't get it, the poor sap.