Carter Burwell, who composed the music for many other Coen Bros. films, wrote the score for Fargo. In particular, his soundtrack features a recurring motif from a Norwegian folk song, "The Lost Sheep " ("Den Bortkomne Sauen").
That's kind of Burwell's jam: he likes to incorporate pieces of old folk songs into his music. For instance, in the Coen Bros.' movie, Miller's Crossing (1990), he used the old Irish tune, "Lament for Limerick." And Burwell hasn't just scored Coen films: he also did the soundtrack for Sidney Lumet's Before the Devil Knows You're Dead (2007) and Spike Jonze's Where the Wild Things Are (2009).
"The Lost Sheep" fits well with Fargo for multiple reasons: like most of the people in the movie, it's of Scandinavian origin. And since it's about a lost sheep, it's like the lost souls in the movie, the characters from Jerry to Carl and Gaear who go wandering through an empty world while making a mess of things. We hear the music in the very first scene, as Jerry's car moves through the snowy void, headed towards his meeting with Carl and Gaear. The music highlights the fact that he has no idea what he's really doing, or the consequences of the chain of events he'll kick into action. He's a "lost sheep" in more than one sense.
Songs by Merle Haggard ("Big City") and Chuck Mangione ("Feels So Good") are audible in the background, but aren't part of the official score, because they're actually being played on radios within the scene and heard by the characters. Still, there's a reason why they're there—they add a definitely Midwestern vibe.