UK's National Health Service Moves To NoSQL Running On an Open-Source Stack 198
An anonymous reader sends this news from El Reg:
The U.K.'s National Health Service has ripped the Oracle backbone from a national patient database system and inserted NoSQL running on an open-source stack. Spine2 has gone live following successful redevelopment including redeployment on new, x86 hardware. The project to replace Spine1 had been running for three years with Spine2 now undergoing a 45-day monitoring period. Spine is the NHS’s main secure patient database and messaging platform, spanning a vast estate of blades and SANs. It logs the non-clinical information on 80 million people in Britain – holding data on everything from prescriptions and payments to allergies. Spine is also a messaging hub, serving electronic communications between 20,000 applications that include the Electronic Prescription Service and Summary Care Record. It processes more than 500 complex messages a second.
Holy shit! (Score:5, Insightful)
Is this a big IT project that actually worked? Where's my fainting couch???
Re: (Score:2, Funny)
Maybe because they were removing an Oracle database instead of implementing one.
Re: (Score:3)
Whilst I'm all for open source in government, I can;t help thinking every time they come out with press releases saying "we used " describes a process where being different with the technology stack is an end in itself.
You could write an open source application in C++ rather than the much less mainstream R language and you'd have lots of people ready skilled to maintain it. Using R just seems like it was the choice of the devs who persuaded the agency to adopt their tools rather than an agency who thought a
Re: (Score:3)
You may be right in general. But R is not a general-purpose language. It's a programmable tool for statistical computing; you'd have to spend a lot of time to reimplement a set of high-quality statistical libraries to do the same. Doing that correctly is very non-trivial and not quick. Very similar to saying you can replace Matlab with your own C++ cod
Re: (Score:2)
Whilst I'm all for open source in government, I can;t help thinking every time they come out with press releases saying "we used " describes a process where being different with the technology stack is an end in itself.
You could write an open source application in C++ rather than the much less mainstream R language and you'd have lots of people ready skilled to maintain it. Using R just seems like it was the choice of the devs who persuaded the agency to adopt their tools rather than an agency who thought about what they needed up front.
I wonder in 5 years if we see headlines "Immigration Agency dumps open source for Oracle. A spokeperson said,'we used a bunch of obscure languages and tools for the old system that served us well we had difficult finding people skilled in them, so we successfully outsourced the system to our new partners who will deliver increased performance and efficiency savings over.blah blah blah". If they'd done it "maturely" in the first place, this kind of nightmare scenario wouldn't happen.
(and I speak of experience - currently discussing details with a company that has a system "built with a mix of Erlang, Scala and Ruby on Rails". You know its been cobbled together by a bunch of hacks more interested in whatever language seemed coolest at the time.
R isn't obscure, and for statistical analysis and data analytics is more likely the Right Tool for the Job. I was helping my stats friends in college setup Linux desktops and servers for R 14 years ago, so it isn't really a trendy language du jour either.
Your Erlang, Scala, and Ruby on Rails system sounds like a nightmare though.
Re: (Score:2)
How did you find them mentioning using 'R'?
R is a query language for statistical models, I doubt anyone implements a software system in such a language, is that even possible?
Re: (Score:2)
Is this a big IT project that actually worked? Where's my fainting couch???
Here's another one. Wonder what they have in common?
What Immigration did with just $1m and open source software
The Department of Immigration has showed what a cash-strapped government agency can do with just $1 million, some open source software, and a bit of free thinking.
Speaking at the Technology in Government forum in Canberra yesterday, the Department's chief risk officer Gavin McCairns explained how his team rolled an application based on the 'R' language into production to filter through millions of incoming visitors to Australia every year.
Despite working for one of the largest bodies in Canberra - and one of the most security conscious - McCairns put his endorsement firmly behind the use of open source.
http://www.itnews.com.au/News/... [itnews.com.au]
I'm going to wager that the open source project successes are more due to the people involved than open source itself. Usually when somebody does that they have an open source cheerleader in a position of influence and a staff already familiar with open source. So you end up with a project led by somebody who is invested in the project's success at a level beyond "this will make the bosses happy". From my limited experience, the proprietary implementations result from those influential people going "we don'
Re: (Score:2)
Six_phases_of_a_big_project [wikipedia.org]
1. Enthusiasm,
2. Disillusionment,
3. Panic and hysteria,
4. Hunt for the guilty,
5. Punishment of the innocent, and
6. Reward for the uninvolved.
Complex? (Score:2)
What is a "complex" message, exactly? And why is 500/sec substantial for a full cluster?
Re:Complex? (Score:5, Interesting)
Re:Complex? (Score:5, Informative)
Obviously you have never worked with HL7. One message will have hundreds, if not thousands of pieces of data.
Yeah - at least in the US and Canada, even parsing HL7 transactions can be a pain. Different rules and practices in different hospitals, inconsistent rules and practices within the same hospital, apparently contradictory transactions, out of order transactions... I predict a royal mess with NoSQL. With Relational they had at least some assurance that what was read out of the DB was an accurate representation of what was put into it.
Re:Complex? (Score:5, Interesting)
Re: (Score:2)
Your post shows you don't know what you are talking about. Tables are often implemented with cross linked b-trees, with one b-tree per column. Other alternative is to use a hash array, or an array alone.
Re: (Score:2, Funny)
Hmm.. I think you just coined a new word.
"Oponion: an opinion so irritating it brings tears to your eyes." :)
(the WTF-observations seem valid btw, but I couldn't resist adding to the Devil's Dictionary :))
Re: (Score:2)
It's more like the system was likely so badly put together that the data wasn't relational or optimized in almost any way. Not to say that NoSQL isn't a mess for this situation, but I can hardly imagine it was much less of a mess before.
Re: (Score:3)
HL7? I thought Valve couldn't even count to HL3.
Oh, you meant that HL7 [wikipedia.org].
Re: (Score:3)
Re: (Score:2)
Re: (Score:3)
Given the application, I imagine most of the data stored is of the schema:
Patient NHS ID number
Patient data.
where the ID is the standard ID we all have, and the
"data" s a huge lump of XML. This is probably why it was easy to dump Oracle for a NoSQL DB - if you only store 2 columns in each table a migration is trivial.
How quickly will they run back to Oracle? (Score:4, Insightful)
I can't help but get the feeling that within a few months they'll be running back to Oracle or some other real database system.
At this point, anyone who works with databases in industry knows that "NoSQL" has come to mean inconsistent data, corrupted data, and silently lost data.
One just can't throw away atomicity, consistency, isolation and durability without running into some serious problems.
And that's totally ignoring how it becomes damn near impossible effectively query NoSQL databases. Sorry, writing complex queries in some imperative subset of JavaScript is totally the wrong way of doing things. Intentionally not learning SQL takes more effort than learning how to use it!
Re: (Score:2)
And that's totally ignoring how it becomes damn near impossible effectively query NoSQL databases. Sorry, writing complex queries in some imperative subset of JavaScript is totally the wrong way of doing things. Intentionally not learning SQL takes more effort than learning how to use it!
Maybe you missed the key phrase in the summary: "non-clinical information". In other words, the structured data that needs to be queried is still in Oracle, or at least that's how I read it.
Re: (Score:2, Insightful)
I've heard managers and analysts say, "Oh, those data won't need to be queried!" in the past. And guess what happens? The data do need to be queried, and usually on a very tight deadline.
Re: (Score:3, Informative)
NoSQL stands for Not Only SQL. Running NoSQL doesn't preclude SQL. In fact if they did a rip-n-replace they are more than likely using an implementation that still supports SQL queries.
Quit it with the revisionist history. (Score:5, Interesting)
The "NoSQL means Not Only SQL" crap you're shitting out is nothing more than the NoSQL community frantically backtracking after their "NoSQL means No SQL" ideas were shown to be disastrous bunk.
Instead of owning up to the fact that they were horribly, horribly wrong, and made some really fucking stupid suggestions, the NoSQLers have just decided to change history. They pretend that they weren't saying what they very clearly said in the past. And they obviously need to admit that SQL and relational databases are the only viable option, but can't do this without looking like the fools that they are, so they admit that it's okay to use "sometimes". And this "sometimes" ends up being "all the time", but again, they can't openly say that without looking like the incompetents that they are.
Face it, "NoSQL" does mean "No SQL". It always has, and it always will. No amount of backtracking will change the fact that the NoSQL crowd was full of shit, and still is.
Re: (Score:3)
Everyone using NoSQL is doing it horribly, horribly wrong, and made some really fucking stupid suggestions ...
So why are they doing it?
Did you ever work with a NoSQL DB?
No? So why do you feel qualified to comment?
Sorry, you are just a ranting moron who has no clue at all about the topic. Pretty clear from your last three posts, get at least a nick, so we can avoid you more easy, coward.
Face it, "NoSQL" does mean "No SQL". It always has, and it always will. No amount of backtracking will change the fact that
Re: (Score:2)
NoSQL stands for Not Only SQL.
Bull. Fucking. Shit. It was always "No SQL".
Re: (Score:2)
Really, Wikipedia, Google, and the NoSQL site itself disagree with you. It was originally called "Not Only SQL" and later many started calling it "No SQL".
The intent of the initial title was to indicate that it was an alternative to SQL, but later in life SQL-like query functionality was grafted into many implementations. It's likely the NHS implementation has sql querying capability if they did a rip-n-replace of the underlying database, otherwise the project becomes immensely larger as you have to re-wr
Re: (Score:3)
Re: (Score:2)
It always was 'not only SQL', then suddenly morons on /. called it 'no SQL' then suddenly some advocated adopted that idea ... but since over a decade it is settled to: 'not only SQL' ...
Even wikipedia has it right
Most NoSQL databases support natively SQL or OQL as query language anyway, not as DDL for obvious reasons.
Re: (Score:3)
They are expecting cost savings due to not paying Oracle licenses, so that strongly implies that there will no longer have any data to be queried in Oracle.
Maybe. Or maybe they are reducing their Oracle licensing costs to the minimum actually needed. Or maybe they will move the more traditionally-structured kinds of data to PostgreSQL in a later update.
The NoSQL solution they are moving to, Riak, is a key-value store that can to full text searches. It is very unlikely to scale when performing full text searches of millions of very long text documents.
Agreed. Where I do not completely agree with you is that the devs responsible for the NHS system are clueless enough to rush into an ill-advised transition that will not scale to their needs. I suspect that they're well aware of both the advantages and disadvantages of the software they just adopted, and will in
Re: (Score:2, Funny)
If you like your ys^d%f7, you can keep your jf# -^',{ ~
Re: (Score:2)
SQL sucks.
Not only does it look ugly, it has ugly commands, that make GPU shaders look intuitive.
How many complex queries does a medical system need? You only deal with one patient at a time, you arent doing *ALL* records data mining for patterns.
Re: (Score:2)
Clearly, you have never heard of database normalization.
Re: (Score:2)
Re: (Score:2)
I only have a little experience with Cassandra, but I can tell you, that consistency is very easily tuneable in it and it is also provides durability. Atomicity is restricted (AFAIK you can get atomicity if all your data goes onto a single data partition). Isolation does not exist.
I believe that it is very easy to say that something need ACID, while actually most data does not require ACID. They can, and as I read the article they do use relational database for those data which actually require ACID. NoSQ
Re: (Score:3)
What we would indeed need, is the multi-datacenter capability. Which you get for free with Cassandra... We also sorely needed performance a few years ago (15k SAS drives was slow after an internet hiccup for exampl
Re: (Score:2)
Again we could get infinite scalability with Cassandra for free.
Jesus Christ......
Re: (Score:2)
Again we could get infinite scalability with Cassandra for free.
Jesus Christ......
No, no, it is the Apacha Foundation.
Re: (Score:2)
Consistency is easy when there is a single non-distributed database. That's not always possible and even if when it is possible its not always desirable because it is an inherent bottleneck. I agree many many companies pretend that they're facebook and end up with NoSQL for stupid reasons (hey, if my website ends up with a 100,000,000 active users then a single db won't cut it...) but there are situations where availability is more important than consistency. Funnily enough, one of them is banking.
Re: (Score:2)
Its a confusing point, but ACID is only one way of ensuring the things you want. Yuo can, for example, use a form of check-it-worked-and-compensate-afterwards to achieve the same level of reliability without actually having an ACID system.
Most banking transaction, I'm told, use this instead of traditional ACID transactions. I suppose you could say its a coarser-grained version of ACID and therefore still ACID, but I think that would confuse most people who think ACID = relational DB with integrity checking.
Making it easy to write queries isn't the priority (Score:3, Insightful)
> Sorry, writing complex queries in some imperative subset of JavaScript is totally the wrong way of doing things. Intentionally not learning SQL takes more effort than learning how to use it!
With 80 million records and heavy load, the number one priority is not "make it easy for any teenager to write queries ".
I system that requires the programmer to think things through, and therefore write an efficient query, is better in some cases.
Just as manually chosen mutexes are sometimes better than automati
Re:How quickly will they run back to Oracle? (Score:4, Interesting)
Sure you can. You can do that and get useful gains. Your imagination may be too limited to understand it, but it's perfectly possible out here in the real world.
It's a distributed database. They don't need the data to be consistent across every node on every write; "eventually consistent" is fine provided that they know the write has reached at least n+1 nodes, and modern NoSQL databases can do just that.
Maybe for you, but they were probably able to hire real developers who knew how to write queries and map/reduce algorithms to perform effective queries.
I also love the implicit suggestion that performing an effective query on any large normalised relational dataset is easy, by the way.
Look, we get it: you've invested a lot of time and effort into becoming an Oracle developer, and yes your skills are now under attack by something that's totally foreign to you, but perhaps you could try learning about those new things and extending your knowledge base instead of dismissing them an hoping they'll go away. It'll make you more employable and you'll look like less of a petulant child, all at the same time.
Re: (Score:2)
I love the wailing that RDBMS fanboys make when anyone mentions NoSQL. Most of it is downright wrong, but the sound is nice.
I also love the uncomfortable bum shuffling that NoSQL fanboys do when they have to tell people they've lost data and all those corner cases they said were unlikely to happen.....well, they kind of have happened. It's also amusing to see people labelled as 'RDBMS fanboys' considering that relational database systems have worked very well for decades.
Sure you can. You can do that and get useful gains. Your imagination may be too limited to understand it, but it's perfectly possible out here in the real world.
Yes, you can get gains. Just don't expect your data to be there in the event of a failure, which is exactly what a system to handle data is supposed to prevent.
Re: (Score:2)
tell people they've lost data and all those corner cases
If an IT department loses data, they did something very very terribly wrong. And the choice which DB vendor, technology or paradigm to use was certainly not amoung those mistakes.
Care to point out how a DB like cassandra, orient db, noql4j etc. should 'lose' or 'corrupt' data?
Sorry. you are very illusioned or simply make up that nonsense.
Yes, you can get gains. Just don't expect your data to be there in the event of a failure, which is exactly what a s
Re: (Score:2)
I can't help but get the feeling that within a few months they'll be running back to Oracle or some other real database system.
At this point, anyone who works with databases in industry knows that "NoSQL" has come to mean inconsistent data, corrupted data, and silently lost data.
One just can't throw away atomicity, consistency, isolation and durability without running into some serious problems.
And that's totally ignoring how it becomes damn near impossible effectively query NoSQL databases. Sorry, writing complex queries in some imperative subset of JavaScript is totally the wrong way of doing things. Intentionally not learning SQL takes more effort than learning how to use it!
I was hoping to learn why/how they might be doing this somewhere in this thread, because their implementation seems to be the exact thing that you want a relational database for, and not where NoSQL shines. I also don't see what implementation they used since NoSQL is pretty non-descriptive, and at this point seems to basically mean that they aren't using Oracle, DB2, MySQL, Postgres, or MSSQL.
Re: (Score:2)
Just because a DB system is not based around SQL and relations it by far does it not make 'a real database system'.
Before SQL was invented we had roughly half a dozen competing database models and 'standards' and ways to program them.
Now as SQL and relations hit their limits those approaches are reemerging. Nothing wrong with that.
But keep your academic standpoint ... that is how progress is made. Leaning back in your armchair and neglecting the values of alternatives.
We see the same attitude in the green e
Re: (Score:2)
Congratulations!
Judging from the comments below, your troll has been wildly successful :)
Re: (Score:2)
acid isn't so important when the unit is a patient's records. there is also no need for a rigid data model.
Is today opposite day? Of all the bits of my life stored in databases, I think my medical records and financial data are the two where I absolutely want those things.
Re:Are you fucking serious? Tell me you aren't! (Score:5, Informative)
I know I shouldn't feed the trolls, but I'm bored and can't help myself.
acid isn't so important when the unit is a patient's records. there is also no need for a rigid data model.
This is unbelievable. Holy fuck, I sure hope that you don't work with databases professionally. I hope you don't work with them as a hobby! Nobody with an ounce of intelligence and even a minute of working with data would ever consider saying something as utterly stupid as what you just said.
As someone who actually has worked with patient data in hospitals, he is pretty on the money regarding the non-structured nature of some patient records. Full ACID compliance is not that important in many cases, often a proper audit trail will suffice. It is similar to banking transactions, which are almost never ACID (despite being used in so many textbook examples of ACID compliance).
One difference between an amateur and professional is knowing how to balance a system's requirements and create a design that actually fit the system's needs. Strict adherence to some guidelines is just plain stupid.
Re:Are you fucking serious? Tell me you aren't! (Score:5, Informative)
I have worked for a health insurer in UK that treated ACID compliance as a bonus, not a requirement. At the time I left them, they had a whole "data correction team" - 12 people working full-time to do live SQL queries to fix database inconsistencies. I wish I made this up, but it's real. If this is considered acceptable practice, I don't want to work in this industry ever again.
Re:Are you fucking serious? Tell me you aren't! (Score:5, Interesting)
There are certain ways ACID compliance is important and certain ways it is not, in fact sometimes it's a hinderance. In particular the following:
One patient's records must be consistent only with itself, you don't need the whole patient table to be consistent. It's a problem because you do need to have cross-table consistency (patients, episodes, diagnosises, treatments, medications and so on) which can lead to locking issues while they're really millions of records living in parallel. Really I'd like to treat them as millions of microtables that happen to share the same structure but never cross lock.
Perhaps in a hospital you can do synchronization at a database level but for an exchange or common journal you have to assume records come in asynchroneously, your general physician might finish some paperwork while you're in emergency care at the same time as a lab result you've waited a week for comes in. The actual ordering they're applied in doesn't matter, there must be rules so (A,B,C), (C,A,B) and (B,C,A) all end up the same result. This means you can relax the hard synchronization of for example a bank account where it is essential that the transactions are applied in order and rejected if you're overdrawing your account, but that's hard in SQL.
That doesn't just apply to the ordering of writes but also querying. If two people at different hospitals tries to pull up your medical records it is important they're not corrupt but it's not essential that an update being distributed is presented to both or none. In fact, for essential robustness they should be able to continue working independently if the connection is broken and when the connection is restored the records are reconciled. That kind of shard and merge is generally a problem relational databases don't handle while the distributed synchronization is rather essential and implicit in NoSQL solutions.
If you run a hospital, you are killing people. (Score:2)
Medical mistakes kill nearly 100k people a year in the US, and you think removing ACID from your data store is beneficial? Where do you work - I want to know what to avoid. mistaken fatalities [npr.org]
If you run a hospital, you are killing people. (Score:2)
Re: (Score:2)
Why the fuck are you storing the data if you don't give a damn about keeping it consistent?
There are thousands (and thousands) of cases where it is simply not reasonable to expect homogeneity in your data. Of those thousands of cases, a very large number of them not only have extremely heterogeneous data, they still need to be stored and queried. NoSQL is a useful tool in such cases.
Is it 'safe' — i.e. does it do all of the data integrity stuff we've come to associate with RDBMSes? No. Emphatically no. If you didn't code it into the right logic in the right places, you are probably worse th
Re:Are you fucking serious? Tell me you aren't! (Score:5, Informative)
If you're storing data, you need to use a system that provides atomicity, consistency, isolation and durability. Using anything less is pure idiocy. [etc, etc]
They are using Riak [basho.com] which is currently being used by 25% of the Fortune 50 (fifty, not five-hundred).
The CAP theorem states there is a trade off between: Consistency, Availability, and Partitioning tolerance. Riak sacrifices consistency (although it does have eventual consistency) in favor of availability and partitioning. The people who wrote Riak (in Erlang) actually seem to be very smart. They say they are firmly in the "right-technology-for-the-right-job" camp. They are not crusading to replace all RDBMS with NoSQL.
The availability and partitioning tolerance of Riak are amazing. For certain applications these strengths greatly outweigh sacrifices in atomicity and consistency. Due to the CAP theorem, there is no one single database architecture that will be optimal for all applications. Granted, a completely different mindset is needed to use Riak if your previous database experience is all RDBMS.
From a cursory look, Riak seems to have some excellent documentation. I suggest you look at their page that explains the trade offs between using Riak and a traditional RDBMS [basho.com]. It also contains links to similar documentation.
Re:Are you fucking serious? Tell me you aren't! (Score:4, Informative)
I also suggest you read CAP Twelve Years Later: How the "Rules" Have Changed [infoq.com] by Eric Brewer. He concludes with:
In general, because of communication delays, the banking system depends not on consistency for correctness, but rather on auditing and compensation. Another example of this is "check kiting," in which a customer withdraws money from multiple branches before they can communicate and then flees. The overdraft will be caught later, perhaps leading to compensation in the form of legal action.
You can claim Eric Brewer is a fucking idiot as much as you want. Eventually all you will do is destroy your own credibility.
Re: (Score:2)
In general, because of communication delays, the banking system depends not on consistency for correctness, but rather on auditing and compensation. Another example of this is "check kiting," in which a customer withdraws money from multiple branches before they can communicate and then flees. The overdraft will be caught later, perhaps leading to compensation in the form of legal action.
You can claim Eric Brewer is a fucking idiot as much as you want. Eventually all you will do is destroy your own credibility.
You're misunderstanding what's been written in that article. This is exactly the scenario that banks *have* to prevent before and as it happens. Chasing around for compensation later cannot be an option in many cases because it is going to be abused. It could happen in the case that a bank branch loses connectivity and they have to work manually, but this is a very rare exception and not the rule, and in that scenario many bank branches simply won't process certain types of transactions. Whatever system you
Re: (Score:3)
You're misunderstanding what's been written in that article. This is exactly the scenario that banks *have* to prevent before and as it happens.
These excerpts from one of Brewer's talks seem to substantiate my "misunderstanding": Eric Brewer on Why Banks are BASE Not ACID - Availability Is Revenue [highscalability.com]
segedunum:
Chasing around for compensation later cannot be an option in many cases because it is going to be abused.
When the system is functioning normally, the difference between strong consistency and eventual consistency is on the order of a few milliseconds. I don't think that leaves much of a window for abuse. The fundamental question is what do you do when there is partitioning? Or as you call it, system degradation. If you take an ACID approach
Re: (Score:2)
I never thought I'd ever see the ACID properties referred to as "buzzwords", especially here at Slashdot of all places.
When you keep repeating four common terms that have a standard acronym, that fits the meaning of buzzword pretty well. The only reason to spell them out instead of just typing ACID is to try and impress people by showing you actually know what the acronym stands for. Then you go on to list them out three times; in back to back sentences even. All you had to do next was spell out what CAP stands for a couple times and you would sound like a genius.
Re: Are you fucking serious? Tell me you aren't! (Score:2, Informative)
Sorry, I'm not the AC using the expanded ACID acronym names. Regardless of how they're referred to, they aren't buzzwords. They are the essential properties of any modern and safe database system. Anyone who insists on them all being present is totally justified, and totally correct.
Re: (Score:2, Insightful)
Sorry, I'm not the AC using the expanded ACID acronym names. Regardless of how they're referred to, they aren't buzzwords. They are the essential properties of any modern and safe database system. Anyone who insists on them all being present is totally justified, and totally correct.
Anyone who insists on all four aspects of ACID being present without knowing anything about a system's requirements is not justified at all. There are plenty of use cases where ACID compliance is ridiculous, such as most banking transactions.
Re: Are you fucking serious? Tell me you aren't! (Score:5, Insightful)
Do you even know what ACID means?
Atomicity - either everything is committed or nothing is. I find it crucial to avoid inconsistent states.
Consistency - data is valid in all states.
Isolation - ensure that concurrent transactions work correctly.
Durability - no data is lost due to software crashes or power failures
How could these not be important for banking is beyond me.
Re: (Score:2)
> How could these not be important for banking is beyond me.
It's not that they're not important, its just that they are not the *most* important thing. Banks care about making money for themselves more than they care about anything else:
http://highscalability.com/blo... [highscalability.com]
Re: Are you fucking serious? Tell me you aren't! (Score:5, Informative)
If a bank doesn't care about ACID, which means it doesn't care about losing completed transactions, which means losing track *OUR* money so they can get more profit.
Perhaps this is where you have gone astray. The opposite of ACID is BASE where the "E" stands for eventual consistency. The beauty of this is that it DOES NOT lose completed transactions and at the same time it allows for high availability.
Strict consistency (the "C" in ACID) is a much more stringent requirement than eventual consistency. In particular it conflicts with high availability. This is the essence of the CAP theorem. In many industries, including banking, eventual consistency plus high availability (NoSQL) is preferable to strict consistency plus lower availability (RDBMS). Of course there are many other factors involved in selecting a database architecture.
One way to see this is by noting the three typical things you can do at an ATM: deposit, withdrawal, and show balance, commute (in a sense) when you are only worried about eventual consistency but they don't commute when you require strict consistency. This is why relaxing the requirement to eventual consistency gives you higher availability (when the database is partitioned). Transactions can be logged and later merged when the partition has healed. It is true that "show balance" does not strictly commute with deposits and withdrawals but: a) this does not cause the system to lose track of your money, and b) no one expects it to strictly commute. There is usually a warning that it may take X hours or days before a transaction shows up on your balance. IOW the balance will eventually be correct after you stop making transactions.
The strict consistency alternative you think is better will mean that all ATMs have to stop working whenever the database is partitioned. For most customers this is totally unacceptable especially since the only value it adds is ensuring that the "show balance" function always includes all of the latest transactions. Even the average person on the street would tell you this approach is really "stupid". No one wants the ATMs to be broken most of the time just to be sure "show balance" is always perfectly up to date.
Re: (Score:2)
The opposite of ACIDis BASE where the "E" stands for eventual consistency. The beauty of this is that it DOES NOT lose completed transactions and at the same time it allows for high availability.
I'm awaiting the rest of the comment with a mixture of trepidation and mild amusement......
In many industries, including banking, eventual consistency plus high availability (NoSQL) is preferable to strict consistency plus lower availability (RDBMS). Of course there are many other factors involved in selecting a database architecture.
No. Creating corner cases like this is what all the NoSQL nutters do, and certainly what they've had to do in recent years when it's become painfully apparent that if you start mucking about with data consistency in any way and telling people things don't matter you WILL get burned.
Acquainting a traditional RDBMS with a phrase like 'lower availability' just highlights to kind of twilight zone you start getting into
Re: (Score:3)
There clearly seems to be a failure of communication here. Since you did not like my dumbed down explanation, perhaps you would prefer to hear what Eric Brewer [wikipedia.org] has to say. He seems to have gotten a whole lot of awards [wikipedia.org] for someone who is a "NoSQL nutter".
Eric Brewer on Why Banks are BASE Not ACID - Availability Is Revenue [highscalability.com]:
Myth: Money is important, so banks must use transactions to keep money safe and consistent, right?
Reality: Banking transactions are inconsistent, particularly for ATMs. ATMs are designed to have a normal case behaviour and a partition mode behaviour. In partition mode Availability is chosen over Consistency.
There are more details here [infoq.com] and in many other places.
Acquainting a traditional RDBMS with a phrase like 'lower availability' just highlights to kind of twilight zone you start getting into when talking to any of the NoSQL crowd.
Are you saying you think the CAP theorem is false? I'm assuming large distributed data sets so partitioning is inevitable. Ac
Re: (Score:2)
Which part of your banking data do you not want to be consistent?
Which transations do you not want atomic?
Personally, I like the idea of my banking data being durable, and I sure dont wat to bank with anyone who sees it otherwise (might feel different if the buggers would give me an overdraft, but I cant see the shareholders or tax man agreeing to it).
and not isolating the transactions? How will that not end in tears
I can understand that maybe you had too much LSD, bu
Re: (Score:2)
Banking transactions are generally not ACID. I'm sure the multi-trillion dollar banking industry are all complete idiots compared to the AC on /.
http://highscalability.com/blo... [highscalability.com]
Re: (Score:2)
Re: (Score:2)
You mean the banking industry that had to be bailed out not that long ago, bankrupting several nations in the process? The one that bundled up bad debt with good debt and pretended it was all good debt? That banking industry?
Re: (Score:2)
Re: (Score:2)
So, when you have an operation and they wind up performing a sex reassignment surgery instead of an appendectomy due to the lack of atomicity, consistency, isolation and durability in their database, you would be OK with it?
Demagogy. Your example has nothing to do with ACID. Such a case would inditate wrong data entry or a client software bug.
By the way, such errors do occur. Even in systems where the database is ACID.
I have not read the article, but I guess they store either very frequent data (measure
Who knew? (Score:5, Funny)
Re: (Score:2)
Re: (Score:3)
had the impression that the NHS database was a large Excel workbook
Technically, that is NoSQL......
Re: (Score:2)
Except Excel is less prone to errors and data loss.
Re: (Score:2)
Yeah but (Score:2)
80 million people? (Score:2)
Fuck me, that's impressive! It even manages to track 17 million ghosts!
https://www.google.com/search?... [google.com]
Re: (Score:3)
Re: (Score:2)
http://en.wikipedia.org/wiki/E... [wikipedia.org]
Re: (Score:2)
I expect they don't usually keep their appointments.
Impressive (Score:4, Informative)
Surprise! Summary has wrong information (Score:4, Interesting)
Summary says: "It logs the non-clinical information on 80 million people in Britain"
Well, yes it does hold clinical information. That is a big deal.
From the UK's HSCIC web site there's more (and authoritative) information on SPINE
http://systems.hscic.gov.uk/sp... [hscic.gov.uk]
"The Summary Care Record:
SCRs provide emergency and out-of-hours healthcare professionals with faster access to key clinical information, including details of allergies, current prescriptions and bad reactions to medicines. The Summary Care Record helps to ensure continuity of care across a variety of care settings, and is provided by the Spine."
Having or losing corrupt information in a clinical record is a good way to kill some random person. However, it is a summary, so if a physician suspects a problem in the summary, they can go to the patient's main record. Getting prescriptions crossed can also be problematic for the patient.
Ignoring the NOSQL issue, I wish we had something like SPINE here in the USA.
Re: (Score:2)
Bullshit Numbers (Score:4, Interesting)
Re: (Score:2)
Tourists and dead people.
Re: (Score:2)
Re: (Score:2)
Other countries may offer some form of free health care only after ensuring a person is eligible.
Payments then follow to the dr, hospital and health care can be fully tracked by a national gov. From the patient to the costly over use, billing of medical services by staff.
Re: (Score:2, Informative)
There are also an estimated 13 million British nationals living outside the UK (source: Foreign & Commonwealth Office). They'll all have NHS records too. That brings us pretty close to the 80 million figure.
Maybe the rest are, at a wild guess, foreigners who've been treated in NHS hospitals for some reason at some time. That sounds vaguely plausible...
Re: (Score:2)
Re: (Score:2)
It includes people who died, people visiting the UK who then left, people who emigrated, even some people who never came to the UK but are medically related to people who live there (holiday STDs etc.)
Re: (Score:3)
The article and summary is wrong, and practically every comment in this thread is misinformed.
The system is used by NHS England. It contains patient data for NHS England. Not UK. Not Britain. England.
So the population of the UK has nothing to do with it.
Proof (Score:5, Funny)
That dropping ACID is not hazardous to your health.
Re: (Score:2)
Re: Proof? (Score:2)
You hear that? That was the giant whoosh sound that just went over your head.
âSuccessfulâ redevelopment (Score:5, Informative)
I use the Electronic Prescription Service (EPS) component of the spine and take issue with the successful claim. The upgrade has been appalling.
It was rolled out over the UK's August bank holiday, with no advance notification. After the holiday, prescriptions pulled down from the spine (they haven't implemented push messaging ... ) had invalid digital signatures, rendering them illegal. Prescriptions that had been completed and payment claimed for in Jan 14 were redownloaded from the spine. Post dated prescriptions for October also began appearing. These are only supposed to be downloadable on after the valid date for obvious reasons.
Not only was this a logistical nightmare, some issues are still broken after two weeks.
I am amazed that so many issues got through testing.
Utter shambles.
Great, they've invented "MedBook"... (Score:2)
Great, they've invented "MedBook"... what you see when you look at it is a fraction of the available data at any one time because it has "arrived" at the node where you are viewing it from yet.
What do I have to do so that my drug allergies and blood type are "sponsored postings" so that when my doctor looks at them, he doesn't kill me due to all of the auto-play video advertisements for Cialis being there instead of the information I want to be there?
Re: (Score:2)
Er.... what? Neither TFA nor the summary make any mention of the GUI, nor advertisements. I suppose you started with the premise that the NHS did something IT-related, automatically assumed it must be bad, and then just started randomly making stuff up.
Selling it short (Score:2)
What do they call the opposite of stocks?
I've always heard them being called short positions [wikipedia.org], which is the origin of the phrase "selling something short". A put option [wikipedia.org] is something different: the right to sell a security for a given price between two given dates. Both can be used to make money in a falling market, though a short is a blunter instrument.