---------------------------- S16a. On progress in science ---------------------------- The frontier in science is the frontier because there is no clear understanding of what is beyond. All that is there is a set of questions bothering those close to the frontier, and a set of experiences of more or less failed attempts to push the frontier forward. Real improvements in difficult matters never come by starting from scratch - they come from patiently building upon the best of what already exists, being open-minded but critical about new possibilities, and trying to integrate what looks most promising. Those who had the questions and found real answers published it and andvanced the state of the art. The others can only share their experience and their chart of the uncharted territory. As one can see from the conflicting opinions, these charts are not reliable. If you (the reader) want to proceed further, you need to learn to see with your own eyes, take your own risks, and find out for yourself what can be trusted. There are no guides beyond a certain point. And don't count on recognition before you actually succeed! As long as ideas are tentative and not validated by experiment, they are always hard to defend. Success comes late - either with a triumphal experimental verification, or if people realize that a new way is significantly simpler than the tradition. If neither happens, people will stick to the tradition, except for a minority who lives from exploring the consequences of the idea. Innovative research is always a risky business - one must be prepared to continue one's work no matter how much it is criticied, but one must also learn as much as possible from one's critics. Then - if it is indeed the right track - success will come sooner or later. But who knows beforehand what will turn out to be the right track? So people have a right to be critical... Critics usually just present a statement, or point to an incoherence in an opponent's statement. To learn from it is a nontrivial task, since it means that one has to find out a) how to make the criticism strongest, in a constructive sense, and b) how best to defend the original statement. Finding this out is learning from it. Everyone starts their journey from where they are, in the direction they find most promising. The others observe what they do and have to make up their own mind. If people knew what is the right start and the right direction, all important unsolved problems were solved by now. The journey is a journey to collect understanding of the ill-understood. To find bugs in a computer code one doesn't go around speculating, but one carefully compares evidence available and stays as close as possible to the code. Physics needs to find the bug in its foundations, and as with computer programs, it will be very subtle and will be found only by a careful investigator, not by a dreamer. Of course, a certain amount of creativity is needed. But it must be guided closely by general knowledge of similar problems already solved and on the structure of the system, together with the information turned up by a detailed analysis of the code. Thus imaginative speculation works only if checked and confirmed by detailed code analysis. And most of the wild ideas are useless. Not a procedure I'd recommend for research, though, unfortunately, it has become fashionable in some quarters of theoretical physics... Rather, learn as much as you can about how and why the good theories work, and if you have the calibre to be an innovator, you might be able to spot what went wrong. But not by searching in the mist; your search should always be well-directed, or you'll go in circles... Judging from my own experience, understanding is not something that springs into one's head without preparation, but is the result of walking attentively and openminded along many blind alleys, until one sees one which smells like being the real thing. Then one starts grinding away in this direction, and in this process discovers what should have been the guiding principle that would have avoided all the dead ends, bringing one directly to the goal. Then, and only then, the right understanding governs the remainder of the search. This is not only my personal experience but seems to be the general pattern: See G. Polya, Mathematical Discovery, John Wiley and Sons, New York, 1962.