Neither test audiences nor managers are ideal judges of creative ideas. They’re too prone to false negatives; they focus too much on reasons to reject an idea and stick too closely to existing prototypes. And we’ve seen that creators struggle as well, because they’re too positive about their own ideas. But there is one group of forecasters that does come close to attaining mastery: fellow creators evaluating one another’s ideas. In Berg’s study of circus acts, the most accurate predictors of whether a video would get liked, shared, and funded were peers evaluating one another.
When artists assessed one another’s performances, they were about twice as accurate as managers and test audiences in predicting how often the videos would be shared. Compared to creators, managers and test audiences were 56% and 55% more prone to major false negatives, undervaluing a strong, novel performance by five ranks or more in the set of ten they viewed.
– from “Originals” by Adam Grant
