×

Announcing: Slashdot Deals - Explore geek apps, games, gadgets and more. (what is this?)

Thank you!

We are sorry to see you leave - Beta is different and we value the time you took to try it out. Before you decide to go, please take a look at some value-adds for Beta and learn more about it. Thank you for reading Slashdot, and for making the site better!

When Bugs Aren't Allowed

CowboyNeal posted more than 8 years ago | from the absolutely-positively-perfect dept.

Programming 489

Coryoth writes "When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs. Praxis High Integrity Systems, who were the feature of a recent IEEE article, write exactly that kind of software. In "Correctness by Construction: A Manifesto for High-Integrity Software" developers from Praxis discuss their development method, explaining how they manage such a low defect rate, and how they can still maintain very high developer productivity rates using a more agile development method than the rigid processes usually associated with high-integrity software development."

Sorry! There are no comments related to the filter you selected.

nearly unlimited funding (5, Funny)

demonbug (309515) | more than 8 years ago | (#14405792)

probably helps too :P

Re:nearly unlimited funding (5, Interesting)

GileadGreene (539584) | more than 8 years ago | (#14405886)

And yet the reports I've seen on Praxis claim costs and schedules the same or less than the development of software of similar complexity...

the old axiom applies (2, Insightful)

User 956 (568564) | more than 8 years ago | (#14405900)

nearly unlimited funding probably helps too

The old technology axiom applies:

High Speed, Low Cost, High Quality.

Pick 2 out of 3.

Re:the old axiom applies (1)

zobier (585066) | more than 8 years ago | (#14406152)

The old technology axiom applies:

High Speed, Low Cost, High Quality.

Pick 2 out of 3.

I've thought about this though, and you really can't have a Low Cost, High Quality product. The reason for this is that you can't rush Quality (as you stated, 2 out of 3) and Time is Money.

So here's hoping more people come to appreciate Quality [levity.com] and are willing to spend the Time/Money necessary to achieve it!

Re:nearly unlimited funding (1, Interesting)

Anonymous Coward | more than 8 years ago | (#14405903)

Doesn't seem to help Microsoft, despite their bottomless coffers....

productivity around 30 LOC per day (0)

Anonymous Coward | more than 8 years ago | (#14405938)

30 LOC - That's 30 lines of code a day. Unlimited budget combined with enough time and resources to do the job right.

Re:productivity around 30 LOC per day (3, Insightful)

blair1q (305137) | more than 8 years ago | (#14406056)

30 LOC is net. You spend the first 45% of a high-reliability project doing the design work, and the last 45% doing the verification. The 10% in the middle is code generation.

These guys seem to be claiming they can reduce redundancy in the design work, and rework in the verification work. They're doing it by using a design-description method that prevents unambiguity (and therefore using a team that is TRAINED to write unambiguous requirements, so their magic language may not be the key), a coding method that avoids unprovable structure (and probably eliminates a lot of other sorts of flexibility), and a verification method that first validates the design and then verifies the code as it's produced (no new value there as everything has to be touched at least once anyway, and if a big bug turns up that causes a lot of code to be redone you have to redo formal verification on those units again; something that's less likely if formal verification is delayed until full-alpha code is demonstrated, having been informally verified along the way).

Their claims of massive error reduction are, at best, anecdotal. Let's see them do this after taking over a half-coded project with minimal design requirements, a hard deadline, and a budget that can be cut by governmental forces at will.

Re:nearly unlimited funding (4, Insightful)

Coryoth (254751) | more than 8 years ago | (#14405953)

In fact this is the whole point - Praxis manages to deliver software with several orders of magnitude less bugs than is standard in the software industry, but does so in standard industry time frames with developer productivity (over the lifecycle of the project) on par with most non-critical software houses. Praxis does charge more - about 50% above standard software daily rates - but then when you are getting the results in the same time frame with massively less bugs a paying little extra is worth it... you'll likely save money in the support and maintenance cycle!

Jedidiah.

So... (3, Insightful)

Coppit (2441) | more than 8 years ago | (#14405979)

nearly unlimited funding probably helps too :P
I guess that's why Microsoft's software is so good.

Re:nearly unlimited funding (1)

Skuld-Chan (302449) | more than 8 years ago | (#14406000)

Its true - I've been in software quite some time (currently work for a company every man woman and child has heard of). Most bugs are classified on time it requires to fix, risk of fixing the bug (most of the time you want to avoid fixing something only to create two more issues), how critical the issue is and mainly > budget availble to fix bugs.

At any rate - there's no such thing as bug free software. Never will be. There is such a thing as a product that appears to work without bugs and I think thats what most software companies aim for.

Nearly unlimited risk helps too (4, Interesting)

goombah99 (560566) | more than 8 years ago | (#14406005)

unlimited risk can be an incentive too.

Professor Middlebrook at caltech was an innovator in an unusual field. Sattelite electronics. Since no repairman was coming they wanted robust electronics. He desigined circuits in which any component could fail as an open or a short and it would remain in spec. You know that's a remarkable achievement if you've ever desinged a circuit before. Notably you can't really do this using SPICE. Speice will tell you what comething does but not how to design it. To do that you need a really good sense of approximations of the mathematical formula a circuit represents to see which components are coupled in which terms. And you need one more trick. The ability to put in a new element bridging any two points and quickly see how it affects the cicuit in the presence of feedback. To do that he invented the "extra element theorem" which allows you to compute this in analytic form from just a couple simple calculations. They still don't teach this in stardard courses yet. You can find it in Vorperians text book, but that's it. If you want to learn it you gotta either go to the original research articles from the 70s.

Re:nearly unlimited funding (1)

GileadGreene (539584) | more than 8 years ago | (#14406017)

See also this [slashdot.org] comment, from further down the page.

Re:nearly unlimited funding (0)

Anonymous Coward | more than 8 years ago | (#14406106)

As a Government Contract Administrator, I can tell you, our funding is anything but unlimited.

Speaking of bugs... (2, Funny)

zegebbers (751020) | more than 8 years ago | (#14405794)

You slashdotted stsc.hill.af.mil!

err (0)

Anonymous Coward | more than 8 years ago | (#14405798)

Get a decent connection, the site is loading fine for me.

Nice FP attempt, I guess.

Wake me up.. (-1, Flamebait)

Anonymous Coward | more than 8 years ago | (#14405796)

When open source developers figure out this method.

Do you think it would help? (-1, Flamebait)

BCW2 (168187) | more than 8 years ago | (#14405799)

If someone sent a copy of this to Micro$oft? Would any of them read or comprehend it? It could make a difference in the version after Vista.

Re:Do you think it would help? (0)

Anonymous Coward | more than 8 years ago | (#14405869)

It would be stupid for M$ to try to make error proof code, it is only done in cases like this because the result of a bug may mean death

Re:Do you think it would help? (1)

BondGamer (724662) | more than 8 years ago | (#14406036)

So Windows is never used in any life or death situations? That is hard to believe.

Re:Do you think it would help? (0)

Anonymous Coward | more than 8 years ago | (#14406084)

Give an example.

Re:Do you think it would help? (1)

BCW2 (168187) | more than 8 years ago | (#14406139)

Not by anyone intelligent. Anyone that uses a M$ product in a mission critical area is courting catastrophe.

Flamebait this! (-1, Flamebait)

Hosiah (849792) | more than 8 years ago | (#14406057)

If someone sent a copy of this to Micro$oft? Would any of them read or comprehend it? It could make a difference in the version after Vista.

Please mod me down flamebait, too. It is unfair of me to not recieve the same consequence for holding the same opinion.

Re:Do you think it would help? (0)

Anonymous Coward | more than 8 years ago | (#14406085)

Only the part about 50% price increase.

Re:Do you think it would help? (2, Interesting)

Coryoth (254751) | more than 8 years ago | (#14406090)

If someone sent a copy of this to Micro$oft? Would any of them read or comprehend it? It could make a difference in the version after Vista.

Oddly enough it would make perfect sense to some people at MS. The Singularity OS project from MS research uses a lot of the same ideas in development methodology and formalism. Whether Singularity will ever make it out of MS research, or simply remain a curious little side project, is of course an interesting and quite open question. Only time will tell.

For other OSs developed in a similar mold, try Coyotos [coyotos.org] which, while still getting seriously underway, looks quite promising indeed when it comes to secure and robust OSs.

Jedidiah.

Microsoft error rates (0)

Anonymous Coward | more than 8 years ago | (#14406099)

Considering the millions of lines of code in Windows XP, I suspect that Microsoft's error rates are not a whole lot higher (and may even be significantly lower) than their target of 0.1 errors/ KLOC. The situtations aren't entirely comparable, though. Windows is written primarily in C and C++, languages that are not well suited to extensive static modelling and analysis. Also, Windows XP is a third generation product, and has been extensively field tested through previous generations and customer betas, options that are not available to a product that must be almost bug free for the very first release.

No Bugs for NSA? (5, Funny)

ChePibe (882378) | more than 8 years ago | (#14405809)

Uh... it's going to be kind of hard for the NSA to do its job without bugs, isn't it?

*rimshot*

Re:No Bugs for NSA? (1)

Wilson_6500 (896824) | more than 8 years ago | (#14406030)

Yeah, the kind that get them around legal restraints. I don't want to hear my spy agencies going "oops."

Whatever (4, Insightful)

HungWeiLo (250320) | more than 8 years ago | (#14405811)

When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs

I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.

Here, here... (4, Interesting)

crimson30 (172250) | more than 8 years ago | (#14405898)

When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs

I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.


That was my first thought, particularly with military avionics. A few years ago they put out a hardware/software update for the ENS system (Enhanced Navigation System) which led to frequent crashing... and it took over a year for them to come out with a message saying that it was a bug and not to waste countless man hours trying to repair it.

It's sort of a new concept, though, as I'd never really seen such problems with traditional avionics systems (non glass-cockpit stuff). I've always attributed it to people being used to the behavior of MS Windows. And I'm not saying that to start a flamewar. I'm serious. Unreliable avionics systems should be unacceptable, but these days, that doesn't seem to be the case.

Re:Here, here... (4, Funny)

Raul654 (453029) | more than 8 years ago | (#14405916)

Could you clarify here. When talking about bad guidance software for planes, "crashing" is an ambigious term ;)

Re:Here, here... (1)

crimson30 (172250) | more than 8 years ago | (#14405993)

Er... obviously, I meant: "Hear, hear...".

Re:Here, here... (4, Insightful)

drew (2081) | more than 8 years ago | (#14406002)

I've always attributed it to people being used to the behavior of MS Windows. And I'm not saying that to start a flamewar. I'm serious. Unreliable avionics systems should be unacceptable, but these days, that doesn't seem to be the case.

Many years ago, I remember reading a quote from an employee at a major aircraft subcontractor along the lines of "If my company paid as much attention to the quality of our work as Microsoft, airplanes would be falling out of the sky on a weekly basis, and people would accept this as normal." I've heard many people, even programmers, claim that bugfree programs are impossible to write. They are not- they just cost far more in time and money than most companies can afford in this commercial climate. When success depends largely on being first to market and bugs and crashes are accepted as a normal fact of life, then they always will be a normal fact of life.

Unfortunately, I think the blame lies at least in large part with the consumer. As long as people put up with programming errors in a $500 software suite that they would never accept in an $80 DVD player, we will continue to have these problems. Unfortunately, too many people still consider computers to be too much black magic that is out side of their (or anyone else's) grasp. Most people have little to know knowledge of how their car works under the hood, but they still believe that the engineer who designed it has enough knowledge to do it without making mistakes and expect the manufacturer to pay for those mistakes when they happen. Why should they believe any differently about the people who write the software they use?

Re:Here, here... (1)

GlassHeart (579618) | more than 8 years ago | (#14406053)

I've heard many people, even programmers, claim that bugfree programs are impossible to write. They are not- they just cost far more in time and money than most companies can afford in this commercial climate.

In most cases, even if the additional money isn't the deal-breaker, the additional time would very much be. The war could be over by the time you fix the last bug in your jet fighter's avionics. Now, even though what you say has an element of truth in it, can you actually cite a program over one million lines long that was actually made bug-free? If not, how do you know they can be made bug-free?

Re:Here, here... (0)

Anonymous Coward | more than 8 years ago | (#14406082)

grammar nazi :) I believe you mean "Hear! Hear!" as one might hear in a Westminster parliament as opposed to identifying the location of an object...

Re:Whatever (3, Informative)

Coryoth (254751) | more than 8 years ago | (#14405925)

I've been in this industry for quite some time and let me be the first to say that I wish I could repeat this sentence with a straight face.

I was pitching for "how people would like to think things are" rather than how things actually work. In practice Praxis, at least, does deliver such software, and does so with extremely low defect rates. They are proof that it can be done, even if it isn't always how things work now.

Jedidiah.

Re:Whatever (1)

recharged95 (782975) | more than 8 years ago | (#14405945)

same here. having some exposure in those areas, I can say they do production software as cheaply as possible.

Re:Whatever (1)

slickwillie (34689) | more than 8 years ago | (#14405980)

When you're writing software for an air traffic control system, military avionics software, or an authentication system for the NSA, the delivered code can't afford to have bugs

So a few bugs in commercial avionics is acceptable?

Re:Whatever (0)

Anonymous Coward | more than 8 years ago | (#14405995)

So a few bugs in commercial avionics is acceptable?

Sure, haven't you ever flown on an Airbus?

Re:Whatever (0)

Anonymous Coward | more than 8 years ago | (#14406065)

Here is a link describing the software design of the "Safeguard" anti-ballistic missile system, which was around in the 60's.

http://www.paineless.id.au/missiles/Computers.html [paineless.id.au]

Bugs and Beta testing. (4, Insightful)

iluvcapra (782887) | more than 8 years ago | (#14406087)

TFA cites a particular NSA biometric identification program which has "0.00" errors per KSLOC.

Now, this got me thinking. It is completely possible for a biometric identification program to identify two different individuals as the same person (like identical twins), or for it give a false negative identification (dirt on a lense, etc). Is this a bug? The code is perfect: no memory leaks, the thing never halts or crashes or segfaults, all the functions return what they should given what they are.

I think the popular definition of "bug" tends to catch too many fish, in that it seems to include all the behaviors a computer has when the user "didn't expect that output," what a more technical person might call a "misfeature." TFA outlines a working pattern to avoid coding errors, not user interface burps -- like for example, giving a yes/no result for a biometric scan, when in fact it's a question of probabilities and the operator might need to know the probabilities. Such omissions (the end user would call this a 'bug'), are solved thru good QA and beta-testing, but TFA makes no mention of either of these things, and seems to think that good coding is the art of making sure you never dereference a pointer after free()'ing it. It does mention formal specification, but that is only half the job, and alot of problems only become clear when you have the running app infront of you.

Discussion of TFA has its place, but it promises zero-defect programming, which is impossible without working with the users.

Re:Bugs and Beta testing. (2, Informative)

Helios1182 (629010) | more than 8 years ago | (#14406155)

Even Identical twins have different finger prints. They aren't exactly identical, despite sharing the same DNA.

It's not hard to write bugless code... (0)

Krach42 (227798) | more than 8 years ago | (#14405824)

You just have to be l33t enough.

HAHAHA... yeah right...

You mean, it's not hard when... (2, Insightful)

No Such Agency (136681) | more than 8 years ago | (#14405920)

It's not hard to produce nearly-bugless code when you have both the budget to do proper quality control, and the incentive to do so.

The reason why Windows is not bugless is that they have the budget to properly debug it... but little incentive to do so before launch. The customers will purchase it anyway and gratefully accept bug fixes after the fact. Airports or the military who bought faulty mission-critical software would not be so forgiving.

Still can have bugs (3, Interesting)

Billly Gates (198444) | more than 8 years ago | (#14405829)

The only method I have seen with almost perfect reliability is where the inputs and outputs are overloaded to handle any datatype and can be proven mathamatically not to crash. I guess a CS degree is still usefull.

The problem is to obtain it you need to write your own libraries and not use ansi or microsoft or any other products as you can not see or trust the source code.

If you can prove through solid design and input and output types that the program wont lose control then your set. Its buffer overflows and flawed design that has not been tested with every concievable input/output that causes most serious bugs in medical and aerospace applications.

However in practice this challenge is a little unpractical when deadlines and interopability with closed source software get in the way.

Re:Still can have bugs (0)

Anonymous Coward | more than 8 years ago | (#14405850)

An English degree would have helped you too.

Re:Still can have bugs (1)

heavy snowfall (847023) | more than 8 years ago | (#14405983)

Did you have trouble parsing his output?

Re:Still can have bugs (1)

sphealey (2855) | more than 8 years ago | (#14405874)

=== If you can prove through solid design and input and output types that the program wont lose control then your set. ===
Well, if that were the case your program would never crash on input, but it could still take that input data and make an incorrect calculation on it. Add the difference between the airport's height and sea level, for example, rather than subtracting it.

sPh

Still can have leaky abstractions. (0)

Anonymous Coward | more than 8 years ago | (#14405907)

"If you can prove through solid design and input and output types that the program wont lose control then your set. Its buffer overflows and flawed design that has not been tested with every concievable input/output that causes most serious bugs in medical and aerospace applications."

I've always felt that resiliancy comes from the bottom up, and the reason our code's so brittle, is because the hardware is as well. A leaky abstraction can only be wallpapered over so much before failures start popping up all over the place.

Re:Still can have bugs (4, Interesting)

GileadGreene (539584) | more than 8 years ago | (#14405923)

Did you actually bother to RTFA? Oh yeah, this is /. - stupid question. Allow me to suggest that you make an exception, and actually take the time to read TFA in this case. Praxis does not claim no bugs, they claim significantly lower defect rates than development using other methods. Which in turn means much greater confidence that the system will work as desired.

No one can ever make something that is completely "bug-free" - even in traditional, non-software disciplines. All you can do is make the probability that the system will work as high as possible. Praxis has some techniques that can help developers create software with a much higher probability of working correctly than it would otherwise have. That's a good thing, even if it doesn't result in perfection.

Its buffer overflows and flawed design that has not been tested with every concievable input/output that causes most serious bugs in medical and aerospace applications.

It's the fact that Praxis relies on static checking far beyond anything you've ever seen (using a carefully designed subset of Ada that can be statically checked in all sorts of interesting ways) that helps to ameliorate this problem, since the static check is effectively equivalent to an exhaustive input/output test.

Re:Still can have bugs (4, Informative)

Coryoth (254751) | more than 8 years ago | (#14406029)

If you can prove through solid design and input and output types that the program wont lose control then your set. Its buffer overflows and flawed design that has not been tested with every concievable input/output that causes most serious bugs in medical and aerospace applications.

Praxis uses a subset of Ada together with annotations in a language called SPARK to write most of their software. They also have tools which work with such code to do considerable static checking - much as type checking catches errors, checking the annotations catches many more just as efficiently - and generate proof obligations, which they can then formally prove. That means, for many of their projects, the actaully have formal proofs that buffer overflows cannot and will not occur.

However in practice this challenge is a little unpractical when deadlines and interopability with closed source software get in the way.

Again, this is where the tools and methodology matter. Praxis delivers code as fast as traditional development techniques, so deadlines aren't the problem. They can do this by using SPARKAda and the SPARK tools to do exceptionally robust testing on a regular basis for each incremental deliverable. This allows catching bugs much earlier, when they are cheaper and faster to fix.

The only method I have seen with almost perfect reliability is where the inputs and outputs are overloaded to handle any datatype and can be proven mathamatically not to crash. I guess a CS degree is still usefull.

It is pretty much this sort of mathematical rigor, injected into the development process as early as possible, that allows Praxis to produce the sort of defect rates that they do. And yes, that does mean that developers at Praxis are probably required to have stronger math and CS backgrounds that elsewhere. Given that, due to their ability to deliver almost bug free software in very reasonable time frames, Praxis charges 50% more than the industry daily rate, yes having a math or CS degree really does count for something - more money for starters.

Jedidiah.

Non turing complete programming languages (1)

autopr0n (534291) | more than 8 years ago | (#14405831)

Can be proven safe. I wonder what subset of modern OS design could be done in such programming languages.

Re:Non turing complete programming languages (1)

Krach42 (227798) | more than 8 years ago | (#14405943)

Well, it couldn't be able to construct the NAND of two elements, as NAND is turing-complete (for arithmetic).

After that you need control structures. If you were to not allow control structures, I think it would be very hard to make an OS. (Unless it's one of them "Hello World" OSes, that then crash and reboot.)

I'd say it's impossible to write an OS in a non-turing complete language.

Uh (3, Funny)

autopr0n (534291) | more than 8 years ago | (#14405998)

control structures != Turing complete. You can have loops as long as they have constant maximum bounds. Whatever it happens to be that you mean when you say "Nand is Turing complete" it makes no sense when you actually typed it. "turing-complete (for arithmetic)." makes no sense at all. WTF? Someone failed CS 315.

Re:Uh (1)

Krach42 (227798) | more than 8 years ago | (#14406089)

Whatever it happens to be that you mean when you say "Nand is Turing complete" it makes no sense when you actually typed it.

I was told that NAND is Turing-complete by someone smarter than I. I realized that NAND itself has no program flow, thus it can't be turing complete all by itself.

"turing-complete (for arithmetic)." makes no sense at all. WTF?

This means that it satisfies everything you need for arithmetic. Using this, one can build any mathematical system out of NAND.

Someone failed CS 315.

Not all universities have the same CS classes. As such, I never took a CS 315 (my University never offered one) and in any case, I've never failed a CS course. I took one class that I got a D in, which I had to repeat simply for failure to do the homework. The next semester the class had programming assignments, and I aced the course.

You can have loops as long as they have constant maximum bounds.

TECHNICALLY, this applies to every computer ever built. But we're not getting that picky. We're assuming that there are no artificial bounds which when allowed to expand to infinity they would then allow for Turing completeness.

So, just by saying that you're writing a programming language where no individual loop will repeat more than 10 times does not make it Turing-incomplete. Because this artificial boundary could be raised to infinity, and the system would become Turing-complete.

So either you accept the fact that all modern OSes are written in Turing incomplete languages, or you accept that an artificial boundary cannot be established as the deciding factor of the Turing-incompleteness.

economics (4, Insightful)

bcrowell (177657) | more than 8 years ago | (#14405843)

The authors contend that there are two kinds of barriers to the adoption of best practices... First, there is often a cultural mindset or awareness barrier... Second, where the need for improvement is acknowledged and considered achievable, there are usually practical barriers to overcome such as how to acquire the necessary capability or expertise, and how to introduce the changes necessary to make the improvements.
No, the reason so much software is buggy is economics. Proprietary software vendors have to compete against other proprietary software vendors. The winners in this Darwinian struggle are the ones who release buggy software, and keep their customers on the upgrade treadmill. Users don't typically make their decisions about what software to buy based on how buggy it is, and often they can't tell how buggy it is, because they can't try it out without buying it. Some small fraction of users may go out of their way to buy less buggy software, but it's more profitable to ignore those customers.

Re:economics (1)

riprjak (158717) | more than 8 years ago | (#14405901)

I contend that the upgrade treadmill with users who accept getting shit upon by their suppliers and smile is a dead end path.
Most other Industries start in this state (You can have any colour as long as its black), but once something becomes enough of a commodity, inevetably the voice of the customer starts to win out.
Either they sue you into the stoneage for destroying their billion dollar enterprise with your crap thats not fit for purpose or they begin to understand that defects COST them and purchase from the folks without them.

Two hundred and fifty years ago, no-one cared if seventeen workers got shredded a day in the cotton `gin, so ole' Eli didnt bother with safety features and, if he had, probably wouldnt have been successful. I imagine even 100 years ago you bought the cheap machine before the one that had extra features to avoid grinding workers into your dogfood (cheap raw materials!). Now days, a manufacturer can demand a massive premium by being significantly and demonstrably safer than the competitor...

Whilst Human-At-Risk tends to be a bigger motivator than money on an individual scale, I guarantee corporations consider Caiptal-At-Risk a much more critical issue; therefore, once they realise that bugs cost them more than paying for well developed software, they will begin to demand it.

*THATS* Economics IMHO.

err!
jak

Re:economics (1)

alphafoo (319930) | more than 8 years ago | (#14406024)

Yep, good to keep the economics aspect in mind.

Reminds me of one of Murphy's Laws of Combat:

"Remember, your weapon was made by the lowest bidder."

Re:economics (2, Insightful)

ChrisA90278 (905188) | more than 8 years ago | (#14406124)

"No, the reason so much software is buggy is economics. Proprietary software vendors have to compete against other proprietary software vendors."

No, that's not it either. Bugs happen because the people who buy the software do not demand bug free code. I do write software for a living. When the customer demends bug-free software he gets it.

I've been around the building bussines too. when I see por work there, say badly set tile, I don't blame the tile setter so much as the full ideot who paid the tilesetter after looking at his poor work.

Re:economics (1)

Coryoth (254751) | more than 8 years ago | (#14406126)

Proprietary software vendors have to compete against other proprietary software vendors. The winners in this Darwinian struggle are the ones who release buggy software, and keep their customers on the upgrade treadmill. Users don't typically make their decisions about what software to buy based on how buggy it is, and often they can't tell how buggy it is, because they can't try it out without buying it.

Unfortunately this is very true. It is, however, slowly beginning to change. In todays highly networked world the tolerance for bugs that cause security breaches is dropping very fast - just witness the current .WMF issues and MS's early patch release in response. As people begin to demand more in terms of security, there will (I hope) be an increasing demand for assurance rather than vague promises. If you have to provide real guarantees on the quality of your software instead of denying all responsibility in EULAs then the sorts of design methods used by Praxis will inevitably become far more widespread.

Jedidiah.

IV&V (1)

Artie_Effim (700781) | more than 8 years ago | (#14405859)

This is where Independant Verification and Validation testing comes into play. I happen to do this for a USGov client and the amount of man hours that go into testing ONE SINGLE PRODUCT that has been fielded for the past 10 years, which is still in production, would astound even you. We test the crap outa this thing and you know we still manage .7 bugs a week.

Bugs are fine... (5, Insightful)

Paladin144 (676391) | more than 8 years ago | (#14405880)

Luckily, bugs are just fine [washingtontimes.com] if you happen to run a company that builds voting machines, such as Diebold. And if you think that elections aren't in the same category as air traffic control, I suggest you take a tour of Iraq. Elections are very important for your continued existance upon the earth.

Re:Bugs are fine... (1)

GoodbyeBlueSky1 (176887) | more than 8 years ago | (#14405896)

They're not bugs if they're designed to sway an election...

Re:Bugs are fine... (1)

B3ryllium (571199) | more than 8 years ago | (#14405927)

why? forty-two

I tried that, doesn't work.

Re:Bugs are fine... (1)

Coryoth (254751) | more than 8 years ago | (#14406063)

Luckily, bugs are just fine if you happen to run a company that builds voting machines, such as Diebold. And if you think that elections aren't in the same category as air traffic control, I suggest you take a tour of Iraq. Elections are very important for your continued existance upon the earth.

Indeed! This is one of the more baffling points to me with regard to electronic voting. We shouldn't just be demanding open source voting software, we should be demanding formally specified voting software with published proofs of various key properties (like, for instance, a proof that the name printed on the paper trail receipt is guaranteed to be the same as name against which the electronic vote is counted). Given the importance of elections, and the fact that, as Praxis demonstrates, such code can be developed relatively efficiently and cheaply, this would seem the least we could ask...

Jedidiah.

Paper doesn't mention open source model (0, Offtopic)

gasjews (941147) | more than 8 years ago | (#14405890)

I'm not sure how much credibility can be lent to any kind of study on the software development process that does not include the open source (OSS) model. By its nature, having more eyes look over you work rather than depending on a fixed and closed system of code assurance finds and fixes bugs faster and implement new features. This is why Windows and UNIX are constantly playing catch up to the Linux platform. I remember reading a study [blogspot.com] on Google's weblog that essentialy endorsed this as a philosophical concept (that applies to much more than just code writing). I don't know who works for the NSA these days, but I would venture to say that the people who work are Google are probably collectively the brightest and wisest folks on the planet.

Linux, Firefox, and OpenOffice are some of the best software on the planet. I think is a good practical testament to the OSS philosophy.

This is the reason why I use Linux and demand that everything I purchase, consume, use, or buy uses or depends on a open source model.

Re:Paper doesn't mention open source model (1)

voxel (70407) | more than 8 years ago | (#14405926)

Open source model doesn't work for most (99%?) of military and government applications.

So, its wise to spend alot of time discussing methods when you can't just "LET THE WORLD SEE".

Sure the author could throw in a one-liner "Use open source model if you can.", and make you happy, but why bother.

Re:Paper doesn't mention open source model (1)

voxel (70407) | more than 8 years ago | (#14405941)

Oh, even better argument... What if you open source your code and no one looks at it... because no one cares.

Then what?... Yes you do have to learn to write solid code without relying on the rest of the world, it's true.

Re:Paper doesn't mention open source model (0)

Anonymous Coward | more than 8 years ago | (#14405973)

"...but I would venture to say that the people who work are Google are probably collectively the brightest and wisest folks on the planet."

Sheesh, Google has some neat tools and all but statements like this don't even fall under the definition of fanboy.

Re:Paper doesn't mention open source model (4, Insightful)

GileadGreene (539584) | more than 8 years ago | (#14405994)

Except that the articles in question aren't about finding and removing bugs in already-implemented code, they are about a method that allows one to construct code that doesn't have bugs in the first place.

Linux, Firefox, and OpenOffice are some of the best software on the planet. I think is a good practical testament to the OSS philosophy.

And yet they all still suffer from a metric crapload of bugs. Praxis produces software with so few bugs that they are willing to provide a warranty that says they'll fix any bug found within the first 10 years, for free. If their software had the defect rate of Firefox or OpenOffice they'd be bankrupt in short order.

Re:Paper doesn't mention open source model (0)

Anonymous Coward | more than 8 years ago | (#14406119)

Too bad Linux, Firefox, and OpenOffice don't offer to fix bugs you find in *their* software for free, huh?

AC

Re:Paper doesn't mention open source model (1)

GileadGreene (539584) | more than 8 years ago | (#14406153)

Yes, yes, open source projects fix bugs for free. The point is that they can afford to do that, because they have so many volunteers to do the bug-fixes. Praxis doesn't have volunteers, so the only way they can afford to do bug-fixes for free is by having a very low number of bugs.

Re:Paper doesn't mention open source model (-1, Troll)

Anonymous Coward | more than 8 years ago | (#14406104)

Hello? Gasjews.com? Slashdot ID >900K? "OpenOffice is some of the best software on the planet"? "Windows and Unix constantly playing catchup to Linux"?

You people have been trolled. Nicely trolled, at that.

Commercial avionics (0)

Anonymous Coward | more than 8 years ago | (#14405892)

Military avionics is more risk tolerant than commercial avionics.

When bugs aren't allowed? (4, Funny)

vertinox (846076) | more than 8 years ago | (#14405899)

Ususually when the software and the phrases "life support" or "nuclear weapons" are together in the same sentence.

Not unlimited funding (5, Interesting)

david.emery (127135) | more than 8 years ago | (#14405905)

The Master Money server done by Praxis was done Fixed Price, and with a warranty that says Praxis would fix any bug discovered over the net 10 years -for free-.

How many of you would be willing to place that kind of warranty on YOUR CODE?

dave (who's tried SPARK and liked it a lot, although proofs are much harder than they should be...)

Re:Not unlimited funding (2, Insightful)

Fnord666 (889225) | more than 8 years ago | (#14406111)

I do. Not intentionally, that's just how it usually works out.

Re:Not unlimited funding (1)

MLopat (848735) | more than 8 years ago | (#14406122)

We do. 100% guarantee on every application that we ship.

Re:Not unlimited funding (0)

Anonymous Coward | more than 8 years ago | (#14406134)

All you really need is the following:

10k...
A tribe of Indians...
Jack Abramoff as a friend... PRICELESS

Oh... Wrong Indians.

Automatic Verification Systems? (1)

mochan_s (536939) | more than 8 years ago | (#14405915)

Why can't automatic verifications systems be used for this? You start with an input set and define the output set. Run a program verification system to make sure the outputs are in the output set and don't go out of it?

The inputs or outputs could be infinite but in that case use logical constructs to verify it.

I'm not a researcher or student of this theory. So, maybe someone can illustrate to me why this wouldn't work or be applied to industry?

Re:Automatic Verification Systems? (1)

cnettel (836611) | more than 8 years ago | (#14406010)

That's what's sometimes done, and it's mentioned in TFA. But, what kind of logical constructs would you use to define the expected output for any input, in a logical manner, instead of just lining them up? Hey, that's the program itself. If you can define exactly what you want in a concise manner, with a way to verify it, then the only remaining problem in the resulting code is performance. (Of course, in practice many of these systems have a realtime requirement to a varying degree.)

You might be able to simulate the system as a whole and instead define "forbidden" states. You might also be able to test individual parts, where you think the set of inputs/outputs is more limited (unit testing, more or less). You might organize your code in a logical manner, suiting the problem at hand, limiting the number of things that you need to keep in mind while writing code, instead making it the task of the compiler or other tools to check them for you. But what very many bugs boil down to, when not typos (which is surprisingly common) is: "Hey, I didn't think about that ever happening, and that it would have those consequences.".

"Just" testing might cause you to think, but to think about That Thing That Is The Big Bug is the problem. If it eluded you while writing the code, it might elude you while writing the test.

Re:Automatic Verification Systems? (1)

GileadGreene (539584) | more than 8 years ago | (#14406130)

But, what kind of logical constructs would you use to define the expected output for any input, in a logical manner, instead of just lining them up? Hey, that's the program itself.

No, that's the specification. The program is the method for performing that actual mapping from input to output. The two are not the same. I can tell you how to recognize a square root when you see one (e.g. sqrt(x) is y such that y*y = x), without actually telling you how to construct a square root (there are, of course, several possible methods for doing so). The former is a specification. The latter is a program.

But what very many bugs boil down to, when not typos (which is surprisingly common) is: "Hey, I didn't think about that ever happening, and that it would have those consequences."

Which is why the Praxis approach includes some extremely good requirements elicitation, to understand the problem domain as completely as possible, and rigorous specification using notations that are amenable to formal analysis, so that those "Hey I didn't think of that" situations can be found and fixed, often before a line of code has been written.

Re:Automatic Verification Systems? (1)

FriedTurkey (761642) | more than 8 years ago | (#14406032)

That's great if you have a 1 parameter function. Multiple infinity by 100 diffrent variables. I am going to try running the test on my Pentium III. I should have it fully tested when they wake my body up from carbon freezing in 5995.

Re:Automatic Verification Systems? (1)

Coryoth (254751) | more than 8 years ago | (#14406159)

That's great if you have a 1 parameter function. Multiple infinity by 100 diffrent variables. I am going to try running the test on my Pentium III. I should have it fully tested when they wake my body up from carbon freezing in 5995.

The key, then, is to actually use some computer science and mathematics and do proofs instead of relying on trying to test all the possible cases. We know that for a right angled triangle the square of the hypotenuse is equal to the sums of the squares of the other two sides. We have not drawn, measured and calculated the results for every possible triangle. Instead we used formal mathematics to prove that this was the case. You can do the same for software - it involves a little more sophistication when writing to code, and proofs are still very hard, but it can be done. This is, in fact, part of what Praxis does - they use a language, SPARKAda, and tools designed around it, to do signficant formal static checking and even proofs of properties of the software.

Jedidiah.

Re:Automatic Verification Systems? (1)

GileadGreene (539584) | more than 8 years ago | (#14406039)

Uh, did you RTFA(s)? They are using automatic verification - take a look at their SPARK Ada [praxis-his.com] toolkit.

Re:Automatic Verification Systems? (0)

Anonymous Coward | more than 8 years ago | (#14406040)

When you write a subroutine asking for someone's name, you don't really expect them to enter "", do you? It's not realistic to write a program to test _all_ possible inputs, because of special cases like that.

2 defacto models of software development (4, Informative)

MLopat (848735) | more than 8 years ago | (#14405918)

In the world of software development, there have come to be two defacto models.

1. Get the software out the door ASAP - quite simply, bang out code as fast as possible that meets a loosely defined specification. Then once the product is adopted, parachute help in like no tomorrow to steadily improve the product.

2. Engineer the software - not as a simple as it sounds. This requires that a specification be drawn. A plan be prepared. A team of solid engineers formed and lead by a competent manager. Then, throughout the entire development cycle, test and debug code.

My company does the latter and to do date we have retained 100% of our customers. I'm shocked by the number of developers that approach our company for jobs that don't have the first clue about how to even write a test harness, let alone do any real debugging. Then again, they don't teach much of that stuff in school and it seems that unless your role was specifically in testing at a previous job, that you're not going to have too much experience in that area. Its economics and marketing that put the bugs in software, not computer science.

Secure code (2, Interesting)

Muhammar (659468) | more than 8 years ago | (#14405922)

The main obstacle in writing a decent code is usualy the management - their frequent changes of mind (about what they want - which is usualy different from what is helpful to the users) and their "good enough" and "legacy first" attitude. Overreaching ambition is another problem - one needs to limit himself to fewer things to do them well - and the management pressures usualy run in oposite direction. (Salesmanship bullshit never helps, especialy if it starts to influence the direction of your project.)

Re:Secure code (1)

iamlucky13 (795185) | more than 8 years ago | (#14405968)

Right, but the article addresses this in point 4:
4.) Saying things only once. For example, by producing a software specification that says what the software will do and a design that says how it will be structured. The design does not repeat any information in the specification, and the two can be produced in parallel.
The key to this point, obviously, is to know exactly what you want when you start and have a detailed outline of the components that will enable it.

JOVIAL (1)

IEBEYEBALL (827052) | more than 8 years ago | (#14405956)

That's why systems and platforms like these are written in a tried and true language like JOVIAL [af.mil] .

seems kinda small (2, Interesting)

khallow (566160) | more than 8 years ago | (#14405977)

I count less than 400k source code lines among their examples ("SLOC"). Collectively, this is at least an order of magnitude (maybe two or more actually, I don't know) shorter than the really big projects out there. So I guess I have two questions. First, is this rate really good given the size of the projects described? And second, for the huge projects, what sort of bug rates are theoretically achievable?

Microsoft should... (1)

themysteryman73 (771100) | more than 8 years ago | (#14405999)

I think Microsoft should hire some Praxis employees...

X windows in Ada for ATC (4, Funny)

dlleigh (313922) | more than 8 years ago | (#14406013)

I was at an X windows technical conference many years ago when someone gave a presentation on X with Ada. When the speaker mentioned that it was for an air traffic control application, there was a sharp intake of breath all around the audience, most of whom had flown in for the meeting.

Defects/SLOC (1)

recharged95 (782975) | more than 8 years ago | (#14406014)

maybe not the best metric. Knowing MIL-spec coding standards and federal critical systems standards, having 0 Dfects/SLOC shows that the test plan was completed to 100% satisfaction and that the use cases were "validated" for correctness.

It still doesn't show anything about the quality of the code. I've been on teams that built great systems (like the ones that supply those great maps in google maps) under mil-spec/SEI standards, but the performance and extensibility of that system really was lacking (and those requirements were in the use cases).

Still, knowing some of those guys, they do some quality work.

PRODUCTIVITY? (0)

Anonymous Coward | more than 8 years ago | (#14406042)

I stopped reading when I saw that they equate productivity with lines of code per day...

What's this I hear? (1)

wetfeetl33t (935949) | more than 8 years ago | (#14406052)

Code without bugs. Well I never...

I can get 0 defects per line of code: assert(1); (1)

strook (634807) | more than 8 years ago | (#14406068)

Not that anyone else rtfa, but defects per line of code seems like a bad measurement of how high quality your code is. More lines != more productive.

Then again I've been suspicious of hyped press releases claiming that the government has super efficient ways to write superior code, ever since that mars orbiter crashed due to mistaken units conversion....

Re:I can get 0 defects per line of code: assert(1) (1)

geekoid (135745) | more than 8 years ago | (#14406146)

in practicallity, it does.
You ned to keep the lines of code measurment in contexts with other issues. Like design, and all the other usual suspects. But that applies to ANY metric you use.

Also, this may help Perl programmers to write script that only does ONE thing per line!

also, just:

  assert(1);

won't compile.
Of course you, like many ignorant /.ers, break it down to an unrealistic and non-practical point. Making your arguement little more then shaky ramblings.

They write the right stuff (2, Interesting)

data64 (300466) | more than 8 years ago | (#14406080)

In school, this article [fastcompany.com] was required reading for our class. It addresses the same topic as TFA.

The essential point being there is a trade-off between quality and quantity and each organization/project needs to decide how far they want to lean in either direction.

Complexity of testing (1)

tedgyz (515156) | more than 8 years ago | (#14406135)

Who was it that said a test system can never be perfect because it must be at least as complex, or more complex than the system being tested?

Yeah, right... (0)

Anonymous Coward | more than 8 years ago | (#14406150)

Praxis manages to deliver software with several orders of magnitude less bugs than is standard in the software industry

Or so they say. I've spent many years in this industry too. Even the best teams I worked with had lots of bugs - lots! Every project has defects - every one. Government contractors sell billable hours - not software. Praxis has marketing good enough to sell their billable hours for more than the competition. Good job!

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?