But 'mermaid' does refer to something. We all know what the word means; it makes sense. In the actual world mermaids do not exist, but there are (logically) possible worlds in which they do.
Likewise, 'matter' has an established definition. (Let us say, for present purposes, that it means 'that which is independent of perception'.) Now, unless the idea of matter contains a logical contradiction (and I can't see one), to dismiss it as nonsensical looks to me like rationalism rather than empiricism. To say that every apparent or conceivable instance of matter isn't really matter is to prejudge the issue.
Yes, "mermaid" has a meaning. It is in the dictionary. But that doesn't mean it refers to anything. The meaning of a term is not its referent. "The Fountain of Youth" certainly has a meaning, but no one ever found the Fountain of Youth, and we now know there is no Fountain of Youth. So, the term, "Fountain of Youth" fails to refer to anything. To say that there are logically possible worlds in which mermaids exist, is only to say that it is logically possible for there to be a mermaid, which, so far as I know, is true, because the idea of mermaid does not imply a contradiction. But that does not mean that there are mermaids. And, even if you want to talk in what I think is a confusing way about possible worlds as if they existed, we were talking about the actual world, and there are no mermaids in the actual world, and, so, the term, "mermaid" does not refer to anything in the actual world, or, in other words, "mermaid" does not refer to anything. If you need me to add the phrase, "in the actual world", fine, if you are more comfortable with that.
The same thing is true of any noun or noun-phrase. And, according to Berkeley, it is as true of the term, "matter" as it is of "mermaid". In fact, he thinks that matter is a kind of evil delusion, and that delusion is the cause of atheism, because it masks the working of God in the universe.