They are the faithful servants of Beatrice, and their primary weapon of choice is a sword. Their physical strength is enormous but they lack the intelligence to do anything but faithfully follow orders. Of all the beings summoned by Beatrice, their rank is the lowest, and they can be killed by being shot in the head. (from: Wikipedia)

Spoiler

During EVA-Beatrice's attack on Rokkenjima, she summons hordes of Goats that are the personifications of random opinions and theories in the future, who start erasing the island because all of them want the tale to be a tragedy. According to Erika, these Goats keep getting born from the sea of the internet, numbering in the hundreds of thousands.