The computer science publication system revisited

Sometimes I get to hear that computer science is not a real science, and sometimes I believe it myself. Science has many definitions and often refers to the scientific method that is used to systematically acquire and disseminate new facts about nature. I think in this sense, computer science has to be split in an applied part (engineering) and a theoretical part (mathematics). I worked on the engineering side for several years and reached the point where I am disappointed about its state. It seems to be less systematical than it needs to be, often focusing on mere implementation details in system xyz while forgetting about generality and nature. I think the main problems are missing reproducibility, the importance of conferences, the current review system, and the self-imposed pressure on the number of publications. This often leads to rushed publications only containing the “least publishable increment” or even wrong results. I also see many conference presentations that don’t present anything new but merely implemented something that is clear to work in theory. Rather than “uh, that’s a nice idea”, I feel more like “oh yes, they’ve implemented xyz with technology zyx”. I am also following the mathematical side of computer science and I have to say that there are different issues. It is good that the general state of the publication system in computer science is being discussed. I agree to most of the points from Lance Fortnow and Moshe Vardi. Both reports are worth a read! The usenix collected wisdom on the review system are also interesting! The last very relevant link deals with the evaluation of individuals in computer science. I’ll close by citing a note on double-blind reviews.

[Update] An excellent paper: “Stop the Numbers Game” by David Lorge Parnas