Wilt thou forgive that sin where I begun, Which was my sin, though it were done before? (1-2)
How can a baby already have committed a sin? The idea of Original Sin isn't observed by every Christian tradition, but apparently the speaker believed he was born guilty. He even claims the sin as his own, though it was done long before he was born.
When thou hast done, thou hast not done, For I have more. (5-6)
There are more sins where that came from. The speaker already knows he will sin again, because it is his nature. Is he saying that he might as well not try too hard to avoid sin, since it'll happen anyway? If so, why bother trying to behave, if sinning is inevitable? Maybe he's simply recognizing his own inherent flaws here.
I have a sin of fear (13)
The speaker believes that his fear is a sin. Is that because it causes him to doubt God, even though that's a pretty understandable and universal condition? Ultimately, it seems like the sins the speaker confesses to are all sins of human nature, like fear.