Thank you for this article! In my thinking, one of the most important traits for scientists is to be humble in the sense that the world is right no matter what. I came to this thinking through early educational experiences, including reading the autobiographical work of psychologist B.F. Skinner (who was profoundly humble in some respects, but not others). Skinner notes in his early years becoming angry when the animals in his experiments did not do what they were "supposed" to do. He overcame this by realizing the animal could never be wrong, and latter he was lead to many important discoveries by leaving his initial hypotheses behind and refocusing on the behavior he did not expect.
Turning to your more specific discussion questions:
1. ...Might not vanity for professional recognition and rewards drive scientific research as effectively as humility?
I think the current "crises" in psychology shows that the drive for professional recognition also puts tremendous pressure on scientists to engage in questionable practices. Though few resort to outright fraud, even the encouragement of a culture of overstatement and flexible analysis and reporting standards can be dangerous for a field. Further, the drive for professional reward often comes at the expense of casual curiosity - seeing new methods and techniques, looking at other's data, and seeking crucial experiments that could fail. This too can lead fields in bad directions. (I discuss these problems and solutions further <a href = http://fixingpsychology.blogspot.com/2011/12/christmas-special-year-of-scandals-in.html>here</a>.)
4. Psychological studies show that most persons asked to assess their merits and abilities rate themselves as above average—the Lake Woebegon Effect—and this contributes to our overall mental well being. Does humility, then, work against our mental health?
I have wondered about this as well. I cannot, however, help but thinking that even "above average" people should still be humble, and so the two feelings are not in as much conflict as they initially appear. As a personal confession, I think I am quite good at a great many things. However, there is nothing I do where I don't know there are people better than I am, and in most cases far better than I am.
For example, I am currently a student of Aikido, and am quite good given the amount of time I have been training. That said, the humility studying Aikido can generate is still huge. Obviously one is less skilled than their teahers, but you can find interviews with top Aikido sensei, people practicing for 50 years, who will say that they could still improve techniques they learned in their first years. To have the best in the world demonstrating extreme humility has a very positive impact on students, and I wish there was more of that in science. When I advise my college students, I suggest that they try to determine what they are both good at and find enjoyable, then never forget that they have a lifetime of improvement ahead.