No one likes a know-it-all—except Tevye. One of the reasons that Tevye is obsessed with his knowledge of and ability to recall the many samples of midrash that he treats everyone he knows to is that he wants always to look like the smartest, wisest guy in the room. Because he's not the most anything else—not the most successful, not the richest, not the most well-connected—he settles for the superlative "most knowledgeable." The only problem is that Tevye the Dairyman hints that maybe, just maybe, being fixated with appearing wise isn't really the same thing as internalizing and learning from all the midrash that you can quote. Plus, it's really annoying to watch a movie with someone who's just quoting the whole thing.
Tevye willfully misreads situations in which he should have known better (like, say, with Menachem-Mendl, or with Ahronchik's family) because he'd rather be lucky than wise.
Tevye is insecure about his level of knowledge or understanding, and so feels most comfortable in the company of those who cannot talk back to him—God, Sholem Aleichem, or his horse.