Review by many eyes does not always prevent buggy code
'There is a view that because open source software is subject to review by many eyes, all the bugs will be ironed out of it. This is a myth.'
-- source: https://opensource.com/article/17/10/many-eyes
Even if a project has many contributors, you will get a lot of contributors in areas that are easy to contribute in (eg spelling, expanding documentation, simple additions) and hardly anyone in the difficult areas. The less contributors in an area, the less likely they will overlap and therefore the less code review will happen.
Cheers, Peter
On Mon, 9 Oct 2017 13:16:29 +1300, Peter Reutemann wrote:
'There is a view that because open source software is subject to review by many eyes, all the bugs will be ironed out of it. This is a myth.'
-- source: https://opensource.com/article/17/10/many-eyes
... The less contributors in an area, the less likely they will overlap and therefore the less code review will happen.
That last statement is just a rephrasing of the very “myth” that the article purports to debunk.
'There is a view that because open source software is subject to review by many eyes, all the bugs will be ironed out of it. This is a myth.'
-- source: https://opensource.com/article/17/10/many-eyes
... The less contributors in an area, the less likely they will overlap and therefore the less code review will happen.
That last statement is just a rephrasing of the very “myth” that the article purports to debunk.
Based on my own experience.
Cheers, Peter
On Mon, 9 Oct 2017 13:58:12 +1300, Peter Reutemann wrote:
'There is a view that because open source software is subject to review by many eyes, all the bugs will be ironed out of it. This is a myth.'
-- source: https://opensource.com/article/17/10/many-eyes
... The less contributors in an area, the less likely they will overlap and therefore the less code review will happen.
That last statement is just a rephrasing of the very “myth” that the article purports to debunk.
Based on my own experience.
Which statement would you say reflects your own experience?
* Fewer eyes make fewer bugs shallow (the flipside of many eyes make more bugs shallow) or * Fewer or more eyes make no difference
?
Based on my own experience.
Which statement would you say reflects your own experience?
- Fewer eyes make fewer bugs shallow (the flipside of many eyes make more bugs shallow)
or
- Fewer or more eyes make no difference
?
Easy domain: many eyes will make bugs shallow (it is easy enough to spot and fix for a novice). Hard domain: even for "good" eyes it is much harder to spot bugs (and you have far fewer of these contributors).
Cheers, Peter
On Mon, 9 Oct 2017 14:17:35 +1300, Peter Reutemann wrote:
Easy domain: many eyes will make bugs shallow (it is easy enough to spot and fix for a novice). Hard domain: even for "good" eyes it is much harder to spot bugs (and you have far fewer of these contributors).
Does the article talk about certain bugs being harder to spot by the same number of eyes, or only about fewer eyes being able to spot certain bugs?
Easy domain: many eyes will make bugs shallow (it is easy enough to spot and fix for a novice). Hard domain: even for "good" eyes it is much harder to spot bugs (and you have far fewer of these contributors).
Does the article talk about certain bugs being harder to spot by the same number of eyes, or only about fewer eyes being able to spot certain bugs?
Security related domain: "for many areas of security functionality—crypto primitives implementation is a good example—the number of suitably qualified eyes is low."
Cheers, Peter
On Mon, 9 Oct 2017 14:39:42 +1300, Peter Reutemann wrote:
Security related domain: "for many areas of security functionality—crypto primitives implementation is a good example—the number of suitably qualified eyes is low."
So what exactly is the “myth”? The common form of the maxim seems to be “Many eyes make all bugs shallow”. But this seems a bit vague to me.
How about this to be more specific: “Sufficiently many eyes make all bugs shallow”. Expressed that way, is it a myth?
Consider also: “There will always be sufficiently many eyes”. Put that way, it seems pretty clear there will be cases (many cases!) where it is not true. But is it implied by the original form of the maxim?
Because that is in fact what the article is addressing, is it not? But if it is not a reasonable conclusion from the maxim, then the whole argument becomes what is called a “strawman”: the article writer has set up a false claim only to shoot it down.
Security related domain: "for many areas of security functionality—crypto primitives implementation is a good example—the number of suitably qualified eyes is low."
So what exactly is the “myth”? The common form of the maxim seems to be “Many eyes make all bugs shallow”. But this seems a bit vague to me.
But that's how it is perceived.
How about this to be more specific: “Sufficiently many eyes make all bugs shallow”. Expressed that way, is it a myth?
Even *if* you have many eyes, there is *no* guarantee that you will catch all bugs (or even a specific one). Only if *sufficient* is translated to *infinite* will it be guaranteed.
Cheers, Peter
participants (2)
-
Lawrence D'Oliveiro -
Peter Reutemann