Symbolism, Imagery, Allegory
So many characters discuss horseback riding that it just has to be important. Dorothea gives it up at the beginning of the novel, even though she enjoys it, because she wants to concentrate on more spiritual things. Sir James tries to persuade her to take it up again because she is "such a perfect horsewoman" and because every woman ought to be in good practice so that "she may accompany her husband" (1.2.50). Sir James probably represents the conventional view: horseback riding is a "genteel" pastime for upper-class women and will make them better companions for their husbands. But Dorothea considers it to be a distraction from the real business of her life. Horseback riding, then, isn't wrong in itself – it depends on the purpose for which you use it.
Take Rosamond, for instance – she uses her love of horseback riding as an excuse to get what she wants. She wants to pump Mary Garth for information about Mr. Lydgate, so she manages to convince her brother Fred to ride with her to Stone Court to visit Mary, while pretending that it "is indifferent to [her] where [they] go" (1.11.76). Later, after her marriage to Mr. Lydgate, she lies to her husband and goes horseback riding with his cousin just to impress the townspeople by "riding on a fine horse, with Captain Lydgate, Sir Godwin's son, on another fine horse by her side" (6.58.25).
Horseback riding is good for exercise, but not for showing off or for trying to fit yourself into the perfect "pattern of a lady" like Sir James wants Dorothea to do (1.2.51). So, in Middlemarch, maybe horseback riding can stand in for all the things, like money or fine clothes, that aren't bad in themselves, but only if they're used for superficial reasons.