Lousy Labelling

Three boxes filled with lots of balls are on the table. One box is full of red balls, one is full of blue balls, and one is filled with both red and blue. Three labels are made for the boxes, but they are stuck to the wrong ones so that no box ends up with the right label. You can’t see what colour the balls in each box are unless you pull some balls out to look. How many balls do you need to pull out of the boxes to know which box is which?