It's fair to believe that the more varied experiences a belief survived, the less likely it would be false. But still, how could we protect ourselves from obsolete beliefs?

First, Paul Graham advice to have an explicit belief in change. Don't even try to predict the future. Stay aggressively open-minded and super sensitive to the winds of change. Admit you have no idea what the right direction is. He thinks this mindset works for evaluating new ideas, but also for having them. If you are expert enough in a field (a year of focused work plus caring a lot is most of the time sufficient), plus if you are reasonably open-minded, any weird idea or apparently irrelevant question will seems worth exploring.

Need an incentive to correct obsolete beliefs? Turn your comments into bets. For example, if you have to share your thoughts on a topic through a blog post, you'll worry much more about getting things right than most people would in a casual conversation.

Second, focus on people rather than ideas. It seems easier to predict what sort of people will make future discoveries, than to predict future discoveries themselves. To Paul Graham, good new ideas come from earnest, energetic, and independent-minded people. Simply surround yourself with these people, the ones who will precisely make your beliefs obsolete, and you will be safe from concluding the world is static. I feel it's no coincidence if highly successful people are very attentive to youngsters.

Source: http://paulgraham.com/ecw.html