Silas is an artisan. Rather than a farmer like the villagers or a hunched-over factory worker, he works with his own loom in his own house. He's what English historian E.P. Thompson calls a "customer-weaver." By the end of the 18th century, customer-weavers were practically irrelevant, being replaced first by weavers working all together in a type of factory, as Silas used to; and then, just a few decades later, by real factories full of power-looms.
So the loom partly signals how precise Eliot was being in setting her novel at the beginning of the 19th century. Sure, the sound of the loom might have been strange to Raveloe—but it's a sound that almost no one in 1861 would have recognized, either.
The loom also represents the difference between monotony and rhythm. Eliot spends a lot of time talking about the rhythmic nature of Raveloe life, focusing in particular on the ritual of Christmas and New Year's. The yearly ball at the Red House, for example, "renew[s] the charter of Raveloe" through the ritual of allowing the villagers to sit in the doorway and watch the parishioners dance (1.11.61). These are ceremonies that happen every year. They follow, at least to some degree, the rhythm of the agricultural seasons. Eliot refers to this rhythm as the natural cheerful trotting of the winnowing machine, or the simpler rhythm of the flail" (1.1.2) and contrasts it to the "mysterious action of the loom" (1.1.2).
What's so mysterious about Silas's loom? It's not cyclical; it's just repetitive. He works at it "unremittingly" (1.2.3); "his ear filled with its monotony" (1.2.10); "he wrought in it without ceasing" (1.5.3). The ceaseless noise that you can hear at the beginning of this BBC version represents the unchanging rhythm of his life. Back and forth, back and forth in the same path over and over: like Silas's circuit from loom to gold to bed, the sound of the loom is inhuman and frightening because it never changes.