At first it's a little hard to see how the view that only ideas exist would follow from an empiricist's standpoint. Empiricism is the view that all knowledge is based on sense experience, on observation. And you would think that what we directly observe is the material world, in all its physical, non-mental glory. But, of course, Berkeley's genius is to show that a truly rigorous empiricism leads to the opposite conclusion.
Most philosophers are not willing to follow Berkeley this far. However clever his arguments are, in the end his particular view of immaterialism just seems a little cuckoo to a lot of people. Nevertheless, quite a number of folks have accepted the view that experience is the only proper source of knowledge. Accepting that view is enough to get you into this club.
Locke is the originator of the modern tradition of empiricism. In his Essay Concerning Human Understanding, he asks where the mind derives all its materials of reason and knowledge. No fancy French terminology for Locke: "To this I answer, in one word, from EXPERIENCE" (source).
Far be it from us to point out to the Master that "from experience" is actually two words. Regardless, the basic principle is clear, and Locke goes on to build his entire philosophical system on the basis of it.
When you think of Locke and Berkeley, what other name immediately comes to mind? That's right: David Hume. Locke, Berkeley, and Hume are forever linked as the three main British Empiricists.
Hume, as you would expect, accepts the premise that all knowledge comes from experience. But whereas our good friend Bishop Berkeley claims that this premise implies immaterialism, Hume thinks that it leads to… skepticism. That wouldn't make Berkeley very happy (he hates skepticism, as you know), but, hey, you gotta take that empiricist love where you can find it.
John Stuart Mill insists so strongly that experience is the basis of knowledge that he even tries to explain mathematics in purely empirical terms. Neither Berkeley nor Hume ever went that far. Of course, Frege, the creator of modern mathematical logic, schooled Mill on his argument and made him look sort of silly (source), but luckily, Mill wasn't around to find out about it.
By the time you get to the 20th century, everyone and their brother starts claiming to be an empiricist. Carnap, the best-known member of the Vienna Circle (a.k.a. the dreaded logical positivists), is no exception.
Carnap makes a point of connecting his empiricism with the physical sciences in particular. For ol' Rudolph, the empirical sciences are the sole source of knowledge.
Unfortunately for Berkeley, Carnap's commitment to a scientifically based philosophical approach means he views Berkeley's subjective idealism as an example of "metaphysics." After all, Berkeley's claims are incapable of empirical verification, and, according to the logical positivists, this is a sign that the claims are metaphysical. And what that implies is that Carnap views Berkeley's philosophical system as not simply false but utterly meaningless.