Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Code Quality: Open Source vs. Proprietary

Soulskill posted about 5 months ago | from the put-your-money-where-your-code-is dept.

Programming 139

just_another_sean sends this followup to yesterday's discussion about the quality of open source code compared to proprietary code. Every year, Coverity scans large quantities of code and evaluates it for defects. They've just released their latest report, and the findings were good news for open source. From the article: "The report details the analysis of 750 million lines of open source software code through the Coverity Scan service and commercial usage of the Coverity Development Testing Platform, the largest sample size that the report has studied to date. A few key points: Open source code quality surpasses proprietary code quality in C/C++ projects. Linux continues to be a benchmark for open source quality. C/C++ developers fixed more high-impact defects. Analysis found that developers contributing to open source Java projects are not fixing as many high-impact defects as developers contributing to open source C/C++ projects."

cancel ×

139 comments

Sorry! There are no comments related to the filter you selected.

Not a surprise (4, Insightful)

Tontoman (737489) | about 5 months ago | (#46774379)

Sunlight is the best bleach.

Re:Not a surprise (0)

Anonymous Coward | about 5 months ago | (#46774409)

Sunlight is the best bleach.

Link?

Re:Not a surprise (1)

Anonymous Coward | about 5 months ago | (#46774419)

You can't fix a problem that you can't see!

Re:Not a surprise (0)

Anonymous Coward | about 5 months ago | (#46774423)

You can't fix a problem that you can't see!

Link?

Re:Not a surprise (0)

Desler (1608317) | about 5 months ago | (#46774435)

Sure you can. It's called binary patching. It's how people patch bugs in closed-source games.

Re:Not a surprise (3, Insightful)

viperidaenz (2515578) | about 5 months ago | (#46774745)

bugs, like DRM?

Re:Not a surprise (1)

Anonymous Coward | about 5 months ago | (#46777749)

DRM is not a code bug. It's a human bug.

Re:Not a surprise (2)

aliquis (678370) | about 5 months ago | (#46775963)

Without seeing it?

They patch at random and let evolution take care of the rest? ..

Re:Not a surprise (1, Informative)

Anonymous Coward | about 5 months ago | (#46776947)

Sure you can. It's called binary patching. It's how people patch bugs in closed-source games.
Yes its possible, you just need to ignore the license, use a decompiler, and single step through the code, fixing and patching. The only problem is that you are fixing *your copy only*. Now you can continue to ignore the license and spread your copy around, or you can make a binary patch and spread that around (ignore the first instance of license violation by using a decompiler). The only other issue is that you may have to patch the next version of the software if the company doesn't fix the original (they don't necessarily recognise you as a developer or important), and if they catch you spreading patches or fixes, they will prosecute (read the license for more info.). So yes, like cutting a slice out of a baked cake, you can do a 'binary patch', unlike those 'open source' patches which is more like altering the ingredients list. Note also that your fix might be duplicated by dozens or hundreds of others.

Re:Not a surprise (-1, Redundant)

invictusvoyd (3546069) | about 5 months ago | (#46774463)

Given enough eyeballs, all bugs are shallow .

--- Linux Torvalds

Re:Not a surprise (1, Interesting)

Anonymous Coward | about 5 months ago | (#46774503)

Only if they're actually is sufficient enough people looking at the code. OpenSSL proves that there isn't.

Re:Not a surprise (0)

budgenator (254554) | about 5 months ago | (#46774989)

Your assuming that heartbleed was a bug and not an undocumented feature requested by a Governmental sponsor, even unwashed libertarian programing hippies have to eat.

Re:Not a surprise (1)

Anonymous Coward | about 5 months ago | (#46775187)

Wait, wut?..

So you're claiming that there _were_ enough people looking at the code, but they were all bought out by The Government (which, apparently, reached all developers around the world easily, but missed Google and Codenomicon)

Re:Not a surprise (0)

Anonymous Coward | about 5 months ago | (#46776991)

Only if they're actually is sufficient enough people looking at the code. OpenSSL proves that there isn't.
Not really. OpenSSL represents an extreme. It takes a lot to audit the 370,000+ lines of OpenSSL. There are a large number of cryptographic functions, tied tightly to advanced networking. Of a total number of maybe 1000-2000 people in the world who can properly audit OpenSSL, over its life, a few dozen have been working on OpenSSL (there are about 4-5 part timers working on it now, not including the team of Google staffers who did analysis of the bug). So the story 'given enough eyeballs, all bugs are shallow' is still true. It depends on the complexity of the software and the number of people able to audit it. Whenever open source software comes up, some nutter comes up with this gem 'I don't want some kid writing my operating system'. So show me a kid (any kid) that has a firm understanding (the non-drugstore version) but an actual textbook understanding of what an operating system is, and what it does. Next, show me a kid that can write one. Its a 20:1 shot to find a kid who can write 'hello world'. Complex application programs from this kid is 1 in 200. System level bit banging driver code is a 20000:1 shot. Operating system? But I've seen lots saying they don't want the kid or his software. That a kid could even write a simple OS is a million : 1 shot. So go ahead and look at the source to OpenSSL (there are only about 3500 files, 70% written in C). Go ahead and audit.

Re:Not a surprise (0, Redundant)

exomondo (1725132) | about 5 months ago | (#46774627)

Given enough eyeballs, all bugs are shallow . --- Linux Torvalds

Actually that was Eric Raymond, and it is evident that in fact there never are enough eyeballs (at least ones that can comprehend what they are looking at). The theory is sound but in practice it is not.

Re:Not a surprise (2)

dkf (304284) | about 5 months ago | (#46774849)

Actually that was Eric Raymond, and it is evident that in fact there never are enough eyeballs (at least ones that can comprehend what they are looking at). The theory is sound but in practice it is not.

It's a fundamental truth that, the more of the system you have to comprehend to truly understand it, the harder it is to debug. Syntax problems? Trivial. Global liveness checking? Much harder. (There's just so many ways to screw up.)

Re:Not a surprise (1)

exomondo (1725132) | about 5 months ago | (#46775131)

Right so just having lots of people looking over it won't necessarily accomplish anything, in fact you probably need an unrealistically large amount of people with the ability to understand the system looking over it for that to be of benefit.

Re:Not a surprise (4, Interesting)

K. S. Kyosuke (729550) | about 5 months ago | (#46774959)

It's called stochastic sampling. There are "never enough" stochastic samples if you want to get to zero error, but given an arbitrary acceptable error, there are usually acceptable sample numbers and sampling strategies.

Re:Not a surprise (1)

exomondo (1725132) | about 5 months ago | (#46775171)

but given an arbitrary acceptable error, there are usually acceptable sample numbers and sampling strategies.

Well you need people that can fully understand a particular complex system to find the tough bugs, and you need a lot of them dedicated to it. I would say there is rarely ever enough, except maybe on the Linux kernel where the critical error rate is pretty low (though they do happen). Demonstrated by the key advantage of free/open source software being that it is easier/quicker to fix bugs in it, not that is necessarily more bug-free than proprietary software in general.

Re:Not a surprise (5, Funny)

Anonymous Coward | about 5 months ago | (#46774935)

You can't even attribute a quote correctly.
Linus was the guy that said "Look what you did to my code! You @#$%&! I'm gonna @#)+-*&$! You. You &$(#*%+.

"My name is Linux Torvalds..." (1)

tlambert (566799) | about 5 months ago | (#46775095)

"My name is Linux Torvalds... and I pronounce him 'Linus'...".

that's ESR, not Linus (1)

raymorris (2726007) | about 5 months ago | (#46775575)

That would be:
Given enough eyeballs, all bugs are shallow ... the fix will be obvious to someone.

      Eric S Raymond

Although ESR called it " Linus' Law", it's ESR's writing, from CATB. Linus has a completely different concept that he calls "Linus' Law". Linus talks about motivations for what we do.

Re:Not a surprise (0)

Anonymous Coward | about 5 months ago | (#46775953)

Yes, heartbleed is a great example there. It took 2 years for the (good guys') sunlight to shine on it? Super!!! Meanwhile the bad guys were having a field day.

Re:Not a surprise (2, Insightful)

Anonymous Coward | about 5 months ago | (#46776779)

Well - at least it was found...

With closed source at the other hand..
Well - let's say the bad guys still have great day's for a loooong time...

Re:Not a surprise (1)

mrbluejello (189775) | about 5 months ago | (#46776353)

Yeah, tell that to the OpenSSL team, it will cheer them up.

Re:Not a surprise (0)

Anonymous Coward | about 5 months ago | (#46776719)

Study: Here's some data!
You: B..b..but.. what about my anecdote?!?

Managed langauges (5, Insightful)

Anonymous Coward | about 5 months ago | (#46774427)

Java project developers participating in the Scan service only fixed 13 percent of the identified resource leaks, whereas participating C/C++ developers fixed 46 percent. This could be caused in part by a false sense of security within the Java programming community, due to protections built into the language, such as garbage collection. However, garbage collection can be unpredictable and cannot address system resources so these projects are at risk.

This is especially amusing in light of all the self-righteous bashing that C was getting over OpenSSL's problems. Seems it's true that using a "safe "language just makes the programmer lazy.

Re:Managed langauges (1)

Anonymous Coward | about 5 months ago | (#46774457)

Resource leaks are hardly as critical as "undefined behavior" (read: buffer overflows and all kinds of other nastiness).

At best a resource leak gets you a DoS.

Re:Managed langauges (1)

Anonymous Coward | about 5 months ago | (#46774549)

You're right. Exploiting the swiss cheese of the JVM is a much better target.

Re:Managed langauges (1)

Anonymous Coward | about 5 months ago | (#46774563)

Swiss cheese security, that is.

The JVM (1)

Anonymous Coward | about 5 months ago | (#46777345)

Guess which language the JVM is mostly written in? Dumbass.

Re:Managed langauges (2, Insightful)

viperidaenz (2515578) | about 5 months ago | (#46774807)

Resource leak in Java = DoS, as mentioned already
Resource leak in C = Heartbleed.

Personally, I'd rather my application crash than expose my private keys and other data that was supposed to be encrypted.

Re:Managed langauges (4, Informative)

Anonymous Coward | about 5 months ago | (#46774869)

Apparently you missed the cyrpto flaws in Android 's Java crypto library from last year that exposed private keys. Apparently writing things in Java guarantees jack and shit.

Re:Managed langauges (5, Funny)

sexconker (1179573) | about 5 months ago | (#46775035)

Apparently you missed the cyrpto flaws in Android 's Java crypto library from last year that exposed private keys. Apparently writing things in Java guarantees jack and shit.

No, writing things in Java guarantees your shit will be jacked.

Re:Managed langauges (0, Troll)

Anonymous Coward | about 5 months ago | (#46775777)

As someone forced to use programs written in Java, I feel like my shit was packed.

Re:Managed langauges (0)

Anonymous Coward | about 5 months ago | (#46777373)

That's what free garbage collection does. It's jacks your shit. I prefer to toss things out and not have to deal with my shit, but there's always macho developers who want to mess around with their own shit.

Beta Sucks (1)

Anonymous Coward | about 5 months ago | (#46775057)

Java calls C for anything performance-critical, anyway.

Re:Managed langauges (5, Interesting)

Anonymous Coward | about 5 months ago | (#46775275)

You mean this one [securitytracker.com] , lol?

Solution: The vendor has issued a fix for the Android OpenSSL implementation and has distributed patches to Android Open Handset Alliance (OHA) partners.

Oh, that notorious piece of Java code, OpenSSL!

Re:Managed langauges (3, Informative)

Anonymous Coward | about 5 months ago | (#46775997)

The underlying bug was in the Android PRNG handling, not a flaw in OpenSSL.

@viperidaenz (0)

Anonymous Coward | about 5 months ago | (#46777249)

Remember that Linux itself is written in C.

Re:Managed langauges (5, Interesting)

Darinbob (1142669) | about 5 months ago | (#46775397)

I also think that with a low level language that more developers are aware of potential problems than developers using high level languages. In some sense I think this is also due to the types of programs being developed. C/C++ today is common used for embedded systems, operating systems, runtime libraries, compilers, security facilities, and so forth. So systems programmers versus application programmers versus apps programmers. The system programmers are forced to take a close look at the code and must be mindful of how the code affects the system. I think that if you had such a comparison done back in the 80s that the numbers would be different because many more application programmers were using C/C++.

Ie, interview for a systems programmer: do you know about priority inversion, do you understand how the hardware works, do you know the proper byte order to use, what does the stack look like, etc.
Interview for the modern applications programmer: have you memorized the framework and library facilities.

Re:Managed langauges (2)

reanjr (588767) | about 5 months ago | (#46776429)

I've never been in an interview where they asked for memorized framework and library facilities. As a web developer, I get questions about data normalization, graph theory, and complicated SQL JOINs.

If you have to make any obligations (1)

Sla$hPot (1189603) | about 5 months ago | (#46774453)

Always proprietary ( your own code )

Re:If you have to make any obligations (0)

Anonymous Coward | about 5 months ago | (#46774507)

BSD

Re:If you have to make any obligations (1)

Anonymous Coward | about 5 months ago | (#46774555)

Crap code remains crap code.

People have to be able to provide updates... and feel appreciated by doing so. BSD licenses allow companies to "appropriate" code, and then sue the original author for copyright violations...

Re:If you have to make any obligations (1)

Immerman (2627577) | about 5 months ago | (#46774953)

> People have to be able to provide updates... and feel appreciated by doing so. BSD licenses allow companies to "appropriate" code,
Very true
>and then sue the original author for copyright violations...
Say what!?!?! Source code typically must be published before being appropriated, making it trivial to prove who had the prior claim.

Of course that doesn't stop a lawsuit from being *filed*, but then nothing stops me from filing a lawsuit against you for stealing my pink unicorn either.

Patent violation are of course another thing entirely - but even there the "publish early, publish often" nature of open source will very likely work very strongly in their favor if they are the original inventors.

Not a surprise, but no reflection of O/S vs Prop. (4, Insightful)

RonBurk (543988) | about 5 months ago | (#46774511)

First, we shouldn't confuse Coverity's numerical measurements with actual code quality, which is a much more nuanced property.

Second, this report can't compare open source to proprietary code, even on the narrow measure of Coverity defect counts. In the open source group, the cost of the tool is zero (skewing the sample versus the commercial world) and Coverity reserved the rights to reveal data. Would commercial customers behave differently if they were told Coverity might reveal to the world their Coverity-alleged-defect data?

Again, having good Coverity numbers can't be presumed to be causally related to quality. For example, Coverity failed to detect the "heartbleed" bug, demonstrating that the effect of bugs on quality is very nonlinear. 10 bugs is not always worse than 1 bug; it depends on what that one bug is.

Re:Not a surprise, but no reflection of O/S vs Pro (1)

dkf (304284) | about 5 months ago | (#46774635)

First, we shouldn't confuse Coverity's numerical measurements with actual code quality, which is a much more nuanced property.

Yeah, but good quality might well correspond to some sort of measurable anyway. Provided you've got the right measure. Maybe some sort of measure of the degree of interconnectedness of the code? The more things are isolated from each other, across lots of levels (in a fractal dimension sense, perhaps) the better things are likely to be.

Maybe that would only apply to a larger project, and I'm not sure what effect system libraries (and other externals) would have. Yet the fact that it might be a scale-invariant approach makes me a bit more hopeful, as it wouldn't be so susceptible to the "ravioli code" problem, where the code's nicely packaged up into little pieces, but the pieces interconnect in a horrible mess of higher-level spaghetti code. Worked on a large project? You'll have probably seen it in the wild. (Yeah, I've had people argue to me that their code didn't use goto and so it had no spaghetti code problems, despite the fact that everything was so nastily interconnected that nobody else could understand it. If that's not indicative of a problem, what is?)

Re:Not a surprise, but no reflection of O/S vs Pro (1)

viperidaenz (2515578) | about 5 months ago | (#46774877)

The more things are isolated from each other, across lots of levels (in a fractal dimension sense, perhaps) the better things are likely to be.

Language has a lot to do with that.
If your project is written in a managed language, allocated memory is always initialised first, there is no pointers arithmetic and array bounds are always checked, so it's impossible to read random data from memory.
If your project is written in C, all code has access to all memory.

Beta Sucks (1)

Anonymous Coward | about 5 months ago | (#46775107)

"If your project is written in a managed language, allocated memory is always initialised first, there is no pointers arithmetic and array bounds are always checked, so it's impossible to read random data from memory."

Except when you forgot to remove some reference to an object, so it's still stitting around in a list somewhere because it can't be garbage-collected, and some code then uses whatever objects happen to be in that list.

No language is safe for an unthinking programmer to use.

Re:Not a surprise, but no reflection of O/S vs Pro (4, Insightful)

gbjbaanb (229885) | about 5 months ago | (#46777193)

are you sure about that?

unsafe
{
// srcPtr and destPtr are IntPtr's pointing to valid memory locations
// size is the number of long (normally 4 bytes) to copy
    long* src = (long*)srcPtr;
    long* dest = (long*)destPtr;
    for (int i = 0; i < size / sizeof(long); i++)
    {
        dest[i] = src[i];
    }
}

that's valid C#, all you need to do is inject something like that into the codebase and let the JIT compile it (using all the lovely features they added to support dynamic code) and you're good to get all the memory you like.

Now I know the CLR will not let you do this so easily, but there's always a security vulnerability lying around waiting to be discovered that will, or an unpatched system that already has such a bug found in any of the .NET framework, for example this one [cisecurity.org] that exploits... a "buffer allocation vulnerability", and is present in Silverlight. [mitre.org]

The moral is ... don't think C programs are somehow insecure and managed languages are perfectly safe.

Re:Not a surprise, but no reflection of O/S vs Pro (1)

Anonymous Coward | about 5 months ago | (#46777327)

A managed language would not have protected against Heartbleed, because the program maintained it's own freelist to prevent memory from being unallocated. If it did not do this then being written in a managed language would have prevented Heartbleed - but then again, if it did not do this then the C code wouldn't have been vulnerable either.

Code Quality Sucks on Either (1, Troll)

hackus (159037) | about 5 months ago | (#46774553)

Yeah, I have seen the source code to the Windows 7 OS, CISCO's iOS and LINUX of course.

They all suck equally.

However, that being said, I am currenrlty running a version of the LINUX OS I built and modified for my customers use in a PostGRES server which is quite large.

Open Source wins again because I can correct the suck. :-)

Re:Code Quality Sucks on Either (0)

Anonymous Coward | about 5 months ago | (#46774607)

Was Mainstream Linux too mainstream for you?

Re:Code Quality Sucks on Either (0)

Anonymous Coward | about 5 months ago | (#46774657)

But can you correct your CASE?

Re:Code Quality Sucks on Either (0)

Anonymous Coward | about 5 months ago | (#46774671)

Open Source wins again because I can correct the suck. :-)

And which patches did you contribute back?

Re:Code Quality Sucks on Either (2)

viperidaenz (2515578) | about 5 months ago | (#46774897)

"none, because someone might find out I've made the code worse, not better"

Re:Code Quality Sucks on Either (1)

Anonymous Coward | about 5 months ago | (#46775197)

I am currenrlty running a version of the LINUX OS I built and modified for my customers use in a PostGRES server which is quite large.

Assuming you mean Linux, that's a kernel and not an OS. Are you saying you run a custom kernel, or are you actually insane enough to run LFS on a server? If the later, WHY?

Did you write your improvements in C+ ? (4, Funny)

raymorris (2726007) | about 5 months ago | (#46775691)

Your four-sentence comment has five glaring errors that make it obvious that you have absolutely no idea what you're talking about. You very much remind me of the job applicant who told me he has experience in C, C+, and C++.

Re:Did you write your improvements in C+ ? (1)

the phantom (107624) | about 5 months ago | (#46775761)

Maybe they meant C#? The [+] and [#] keys might be very close together on the custom keyboard layout your applicant used to type their resume...

good thought. This was spoken face-to-face (1)

raymorris (2726007) | about 5 months ago | (#46776531)

That's an interesting thought. Had it been typed, it might be a typo. I was thinking of a guy who said that, out loud, face-to-face. That's not the only comment that made it clear he was claiming four times as much as he in fact knew.

Of course, in a interview I give someone leeway - my mind went blank once in an interview when I was asked "what are the four pillars of object oriented programming?". At the time, I could have implemented objects in C using the preprocessor*, but interview stress caused a brainfart. This guy was obviously clueless and trying to BS his way through it, though. Perhaps hoping he'd only be interviewed by a manager who wasn't a programmer.

* thanks to Perl for teaching me objects from the inside out. Understanding Perl's implementation of objects, I could see that language support for objects in 98% syntactic sugar, object.method() is the same thing as function(* object), where "object" is an associative array aka namespace aka lookup table, plus a list of class names it has.

Re:good thought. This was spoken face-to-face (1)

the phantom (107624) | about 5 months ago | (#46776693)

Yeah, but "plus" and "sharp" (or should that be "hashtag"?) are almost the same word. I mean, anyone could make that mistake!

Re:Code Quality Sucks on Either (1)

hackus (159037) | about 5 months ago | (#46776197)

First of all, built means compiled and modified means I switched off u32 support as well as targeting a Xeon processor class.

I did not need to write code, the kernel had to be rebuilt and the binary replaced/modified for the target processor and memory architecture.

You use the .config for that and rebuild the kernel tree. You don't need to write code.

SO! The included Redhat kernel was way too generic for the application performance required.

Among other things. But the point is, you can't do that with an OS binary, so LINUX as an OS Kernel, is far more flexible than a proprietary one.

Python (0)

Anonymous Coward | about 5 months ago | (#46774571)

From another article/puff-piece on the site:

"Since 2006, Python has achieved a defect density of .005 (or .005 defects per 1,000 lines of code) and has eliminated all high-risk defects [that they recognized] in its codebase."

Re:Python (2)

Desler (1608317) | about 5 months ago | (#46774651)

But that's written in C and C is the worst programming language ever!! How dare they be so dumb to not write Python in a "memory safe" langauge!

@Desler (0)

Anonymous Coward | about 5 months ago | (#46777257)

So you can't use Linux any more, since it is just C.

heartbleed (0)

Anonymous Coward | about 5 months ago | (#46774575)

So why didn't this scan catch it? Scans are not good enough for everything. It was a weird memcpy call.

The problem often with open source is culpability. I'm not going to yell about the openSSL guys. They are good guys...at least from a coding perspective and from what i hear from a human perspective as well.

If it was MS you bet everyone will be yelling and there is someone you could actually sue.

This sounds like a MS report claiming there stuff is great. I think we need for more humility here and less deflection.

Also, fixing is great an all, but how bout not producing in the first place. Shouldn't these scans be part of the builds before stuff even gets into git.

Re:heartbleed (1)

nanolith (58246) | about 5 months ago | (#46774649)

Most static analysis tools look for bugs and potentially buggy behavior. They must rely on limited pattern matching and data flow analysis. They can't find all bugs. See: The Halting Problem.

Re:heartbleed (5, Interesting)

Anonymous Coward | about 5 months ago | (#46774809)

I'm not going to yell about the openSSL guys.

I'm going to be honest here, they deserve yelling at, and I'm an open source fan. The error they made is exactly the same mistake that everyone else has made in years past when dealing with SSL: x509 and the SSL protocol demands [lengthofstring][string], "pascal" style. This is how everyone (open and closed source) got hit with that domain validation bug where the certificate said "(26)bank.com\0.blahblahblah.com". Certificate signers looked at the domain on the end of the string "blahblahblah.com" and validated it. Client programs treated it like a C string and thought it was a certificate for "bank.com". Not a single person anywhere said "whoa there, null bytes are not part of a valid hostname!"

The attack asks server to respond with "(65535)Hello" and the server replies with 65535 bytes of data. Falling for this attack is exactly like the guy who points and laughs at the person who just fell off their bike, seconds before falling off their own bike. They should have known better, especially with how high-profile these attacks were in the past.

The bit about writing their own malloc implementation, poorly, was just icing on the cake.

Re:heartbleed (5, Informative)

loom_weaver (527816) | about 5 months ago | (#46775089)

Disclaimer, I work for Coverity. There's a write-up on why Coverity didn't find it out of the box here:

http://security.coverity.com/b... [coverity.com]

Polishing old code or writing good code (4, Interesting)

mtippett (110279) | about 5 months ago | (#46774589)

The report doesn't really go into an important measure.

What is the defect density of the new code that is being added to these projects?

Large projects and old projects in particular will demonstrate good scores in polishing - cleaning out old defects that are present. The new code that is being injected into the project is really where we should be looking... Coverity has the capability to do this, but it doesn't seem to be reported.

Next year it would be very interesting to see the "New code defect density" as a separate metric - currently it is "all code defect density" which may not reflect if Open Source is *producing* better code. The report shows that the collection of *existing* code is getting better each year.

Re:Polishing old code or writing good code (1)

organgtool (966989) | about 5 months ago | (#46774695)

Next year it would be very interesting to see the "New code defect density" as a separate metric - currently it is "all code defect density" which may not reflect if Open Source is *producing* better code. The report shows that the collection of *existing* code is getting better each year.

This is exactly what I would expect. Odds are that open source and closed source software start out with similar defect densities. The difference is that open source software, over time, is available for more people to inspect and find bugs that weren't found by the original cast of developers.

Re:Polishing old code or writing good code (4, Interesting)

Anonymous Coward | about 5 months ago | (#46777731)

Actually it's the reverse. Fengguang Wu does automatic defect reporting so new bugs get found and reported within a week. We've had great success with this.

But if we delay then here is the timeline:
- original dev moves to a new project and stops responding to bug reports (2 months).
- hardware company stops selling the device and doesn't respond.
- original dev gets a new job and his email is dead (2 years).
- driver is still present in the kernel and everyone can see the bug but no one knows the correct fix.
- driver is still in the kernel but everyone assumes that all the hardware died a long time ago. (Everything in drivers/scsi/ falls under this catagory. Otherwise it takes 8 years).

Each step of the way decreases your chance of getting a bug fix. I am posting anonymously because I have too much information about this and don't want to be official. :)

troolkore (-1)

Anonymous Coward | about 5 months ago | (#46774593)

Biased data set perhaps? (0)

Anonymous Coward | about 5 months ago | (#46774653)

Stuff that matters?
I'm not convinced that they can ever truly account for bias in the data in this case. With open source, a far greater quantity of code can be analysed, and as such you are likely to eventually approximate the mean "code quality" of all software.

With proprietary code, the ratio being analysed is in all likelihood a far smaller subset of the whole, and so is far more susceptible to outliers dragging the quality metric (regardless of whatever that is) one way or the other.

I would expect both "open source" code to be of approximately equal quality to proprietary code. In each ideology you will get people who care (about quality), and people who don't, in approximately equal proportions, the same with skill, ingenuity and passion for the work.

There are such a massive range in quality across all code, that drawing a generic "x is better" conclusion is ridiculous at best, and purely the realm of the uptight illogical zealots on either side of the debate.

Re:Biased data set perhaps? (3, Interesting)

Squiggle (8721) | about 5 months ago | (#46774993)

I would expect both "open source" code to be of approximately equal quality to proprietary code. In each ideology you will get people who care (about quality), and people who don't, in approximately equal proportions, the same with skill, ingenuity and passion for the work.

The difference is that proprietary software is constrained by the number of developers able to view and work on the code. An open source project may have a similar number, or smaller set of core developers, but a much larger pool of developers that can spot problems, suggest alternatives, fix the one bug that is affecting them, etc. Having a more diverse set of developers will increase the chances that the software improves.

You could also make an argument about the motivations of the developers. Open source projects are often a community of people passionate about what they are building and have a strong incentive to make their code readable by others. By the nature of open source a developers reputation is on the line with every bit of code they make public. I've met far more developers scared to make their horrible code public than those worried about getting fired for equivalently horrible code.

Ah, Coverity (5, Insightful)

ceoyoyo (59147) | about 5 months ago | (#46774709)

Coverity: Hey you, proprietary software developer with the deep pockets. Yeah, you. We've got this great tool for finding software defects. You should buy it.

Proprietary software developer: get lost.

Coverity: Hey, open source dudes, we've got this great defect scanner. Want to use it? Free of course!

Open source dudes: Meh, why not?

Coverity: Hey proprietary software developer, did we mention those dirty hippie neck beards are beating the stuffing out of you in defect (that we detect)-free code?

PSD: Fine, how much?

Taking out the garbage... (0)

Anonymous Coward | about 5 months ago | (#46774753)

That's because C/C++ developers don't count on the garbage man to take their trash out.

Re:Taking out the garbage... (0)

Anonymous Coward | about 5 months ago | (#46774905)

OK, I'll play.

In C/C++, the trash piles up in the house but nobody seems to notice until the house catches on fire. They were too busy taking out trash that they already took out.

Useless analogy (4, Interesting)

Virtucon (127420) | about 5 months ago | (#46774777)

This is a useless analogy. Code Quality is a function of both skill and the stewardship of the team supporting the code. Tools help as well but you can write some elegant, high quality code regardless of the language chosen. You can also write some real shit too but ultimately how many defects a piece of software has comes down to the design and testing that goes along with it. Some bodies of work get rigorous testing and it's not like OpenSSL's recent problem wasn't about deficient design it was about a faulty implementation. Faulty implementations in logic happen all the time and there are some bugs that just take awhile to become known. I mean even with test driven development and tools for code analysis probably couldn't have found this particular issue but considering how long it was in the code base without somebody questioning it goes back to not only stewardship by the team but the rest of the world who are using the code. If anything this situation points out that FOSS can have vulnerabilities just like proprietary software however the advantage is that with FOSS you can get it fixed much more quickly and because other people can see the implementation it can become scrutinized by folks outside the team that develops and maintains it.

In the case of Heartbleed the system works. A problem was found, it was fixed it's now just a matter of rolling out the fix and regressions are put into place to help insure that it doesn't happen again. The repercussions of what it means is that another gaping hole in our privacy was closed and that "bad guys" may have stolen data, rollout the fix ASAP. Your guess is as good as mine as to what was stolen is a matter of research and conjecture at this point. I doubt that the bad guys will tell us what they gained by exploiting it. Let's also be sure that until the systems with the bug are patched, they're vulnerable so cleanup on aisle 5.

To be honest it's a bit naive if we all assume that FOSS software that handles security doesn't have potential vulnerabilities. Likewise it's also naive to assume that proprietary code has it licked as well given the revelations of NSA spying for the past year. Given that there are numerous nefarious companies that sell vulnerabilities to anybody who can pay for it, that means unless you're buying them you probably will never know what is exposed until somebody trips over it. What this means for everybody that you can depend on is when those vulnerability-selling companies are out of business can assume that your software is free of the easier to exploit vulnerabilities; governments will always use all their tools to get intelligence including subverting standards and paying off companies who can give them access to what they want.

What's up with Dice Developers (1)

yxyband (1362863) | about 5 months ago | (#46774833)

Now all we have to do is get Dice to Open Source their stuff so we can FIX IT!

Re:What's up with Dice Developers (1)

lister king of smeg (2481612) | about 5 months ago | (#46775305)

its all ready is open sourced and that is what the soylent news guys did but the community didn't fallow.

Re:What's up with Dice Developers (1)

jones_supa (887896) | about 5 months ago | (#46776727)

One big reason is that they never turned on the D2 discussion system. So right now Soylent News is even more clunky to use than the Slashdot Beta. You get directed to another page every time you want to reply or moderate.

Re:What's up with Dice Developers (1)

stderr_dk (902007) | about 5 months ago | (#46776767)

its all ready is open sourced and that is what the soylent news guys did but the community didn't fallow.

Yes, SlashCode is open source, but the latest public release is 5 years old and not at all what's running on slashdot now.

It would be very nice, if Dice would release a newer version of the code, not only for SoylentNews [soylentnews.org] , but also for the Japanese slashdot.jp [slashdot.jp] and the Spanish barrapunto.com [barrapunto.com] , both of them are still using the old version.

FreeBSD looks just as good as Linux (2)

Lawrence_Bird (67278) | about 5 months ago | (#46774879)

with nearly 2x the LOC.

Re:FreeBSD looks just as good as Linux (1)

rev0lt (1950662) | about 5 months ago | (#46775373)

I'd say the kernel actually looks quite better, but I might be biased,

What if.... (1)

koan (80826) | about 5 months ago | (#46775071)

Code repositories were compromised by the NSA (or other capable group)

Good and Bad news (0)

Anonymous Coward | about 5 months ago | (#46775535)

Good news: Open source has good quality

Bad news: Sometimes we get a bug that that affects most of the internet (Heartbleed)......

proprietary: VBscipt BEST EVER (0)

Billly Gates (198444) | about 5 months ago | (#46775543)

Just kidding

Did they test OpenSSL? (2)

jonwil (467024) | about 5 months ago | (#46775631)

With all the noise about OpenSSL lately, running this Coverity test on it (and other security software like GNUTLS) and sharing the results seems like it would be a good thing...

Re:Did they test OpenSSL? (3, Informative)

ljw1004 (764174) | about 5 months ago | (#46775951)

They did examine heartbleed.

http://ericlippert.com/2014/04... [ericlippert.com]

Quality of people process (2)

hessian (467078) | about 5 months ago | (#46775965)

If you have good quality people, especially a good leader, your code will be good.

Even if the people are relatively inexperienced.

At this point, just about everything in IT/CS is a research project, not innovation.

So it's a matter of diligently doing the work based on past archetypes.

Proprietary code - Poster Child (0)

Anonymous Coward | about 5 months ago | (#46776015)

Of course for the poster child of all code proprietary and closed, even secret, look at Microsoft. Their record of vulnerabilities upon vulnerabilities goes back well over a decade and there seems no end in sight for this circus.

The fiasco with OpenSSL is due to everyone giving zero dollars, thanks for the freeware.

Re:Proprietary code - Poster Child (1)

techno-vampire (666512) | about 5 months ago | (#46776579)

Well over a decade? It's more like well over thirty years. There were virus infections written for MS-DOS 3.X back in the 1980s.

In My experience ... (1)

BrianPRabbit (2020846) | about 5 months ago | (#46776029)

Coverity is no the best "yardstick". Too many false negatives and too expensive.

There is no comparision (2)

Murdoch5 (1563847) | about 5 months ago | (#46776241)

Some open source projects will have better code then closed source projects and vice vesa, you can't just make a clean line.

Re:There is no comparision (1)

jones_supa (887896) | about 5 months ago | (#46776747)

Hey, you're trying to find a reasonable and truthful middle ground. That prevents all the juicy flame wars. Someone call the guards!

Re:There is no comparision (0)

Anonymous Coward | about 5 months ago | (#46777329)

Indeed, I have seen very nicely coded closed source things, but also code that utterly sucked.

It depends what it is used for. If you need to deliver some product in a week and you know it is not intended for any application requiring real reliability the customer will be more happy with a program with some defects, but delivered on time than a perfect one delivered late (unless the defects make the customer lose so much time that they don't gain... :P)

Same for opensource.

Re:There is no comparision (1)

nyctopterus (717502) | about 5 months ago | (#46777403)

Yes how could ever compare to groups that might have a significant amount of overlap? It can't be done! There's no branch of mathematics that would allow us to do such a thing! It's impossible.

The only problem (1)

bucket_brigade (1079247) | about 5 months ago | (#46777573)

The only problem with this is, of course, that what they claim to be doing (automatically examining code for defects) is literally impossible.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>