Ah I see now. See, the term Dwarf is a word mostly used to describe a short or smaller version of a creature. It's used in relation to many species of things. The Dwarf Caiman for instance, is a small species of Caiman. Usually 4 1/2 feet in length. A Dwarf human is an adult who's height is no more than 4 feet and 10 inches. See? You could replace the word with short or small. But for the sake of categorization, people will choose the term dwarf.
Another thing to consider is that many ancient civilizations, who had no contact with each other, have stories and artwork of these beings. When an entire planet of people in ancient times are saying they saw something, they saw something. You might enjoy studying their folklore.