Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
Programming Software Wikipedia

Wikipedia Chooses Lua As Its New Template Language 145

bonch writes "In an attempt to tackle the inefficient complexity of its current template system, Wikipedia will be adopting the Lua scripting language. Known most for its use in videogame scripting, particularly World of Warcraft, Lua is lightweight and designed for easy integration into existing applications. The transition is expected to begin after the release of MediaWiki 1.19, possibly in May." Basically, the template system started turning into an ugly programming language. There was debate over using Javascript or Lua; Lua ultimately won due to implementation concerns. The mailing list threads announcing the decision and discussing the change have further details.
This discussion has been archived. No new comments can be posted.

Wikipedia Chooses Lua As Its New Template Language

Comments Filter:
  • by MadKeithV ( 102058 ) on Wednesday February 01, 2012 @11:39AM (#38891677)
    "the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp.
    Stop delaying the inevitable!
    • by dkf ( 304284 ) <donal.k.fellows@manchester.ac.uk> on Wednesday February 01, 2012 @11:43AM (#38891723) Homepage

      "the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp.

      This includes Common Lisp, which contains itself as a proper subset.

      • by jbolden ( 176878 )

        No... Norvig's variant: Any sufficiently complicated Common Lisp program contains an ad-hoc, informally-specified bug-ridden slow implementation of Prolog.

        • by JamesP ( 688957 )

          And any sufficiently complicated Prolog program contains an ad-hoc, informally-specified bug-ridden slow implementation of Minesweeper.

    • by goldaryn ( 834427 ) on Wednesday February 01, 2012 @11:51AM (#38891795) Homepage

      "the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp. Stop delaying the inevitable!

      LEEEEEEEROOOY JEEEEEENKIIIIIIIIINNSSS

    • by David Gerard ( 12369 ) <slashdot.davidgerard@co@uk> on Wednesday February 01, 2012 @02:41PM (#38894081) Homepage

      The trouble with domain-specific languages is that they are Turing complete. This is a fatal trap: your hammer may be a great hammer, but if it's Turing-complete you will (this is a law of the universe) one day be forced to use it as a screwdriver, spanner, soda siphon, and nail. You will end up having to build a working full-scale replica of the Titanic from toothpicks and spit, complete with iceberg.

      Your rule is more like - any domain-specific language will eventually evolve into brainfuck [wikipedia.org]. ParserFunctions certainly did.

      • by jbolden ( 176878 )

        That's why DSL's in LISP are nice. They just admit the problem and include LISP underneath for when you want to do something different.

        • What Lisp-type DSLs are there that people actually use much? The only one I can think of is Guile in GIMP, but it doesn't have a community. (And Emacs Lisp, of course, but Emacs is for goddamn geeks.) Most DSLs that even become popular enough in their area for many people to use them are not far above batch files, with Turing-completeness pretty much a bolt-on. This results in crawling horrors and a need to replace it with a real language, but the beginner user wants something very like batch files.

          • by obsess5 ( 719497 )
            Guile is Scheme, not Lisp and not a DSL. According to the GIMP web site, their extension language was SIOD (Scheme In One Defun), but is now TinyScheme [sourceforge.net]. TinyScheme is not itself a DSL, although GIMP has probably added some image-processing-specific extensions.
            • Scheme is a Lisp. It was specifically a reaction to Common Lisp's baroque excesses, but it's accepted by all as a Lisp. Assume I'm asking in the sense that Scheme is a Lisp.

          • by jbolden ( 176878 )

            Autolisp (Autocad), ELisp (Emacs), Franz (became Mathematica), Nyquist (sound processing), MELT (internal programming language for writing compilers inside of GCC).

      • The trouble with domain-specific languages is that they are Turing complete. This is a fatal trap: your hammer may be a great hammer, but if it's Turing-complete you will (this is a law of the universe) one day be forced to use it as a screwdriver, spanner, soda siphon, and nail.

        Just this weekend I was thinking of the following reformulation of that classic:
        "If all you have is a hammer, you might be really good at turning problems into nails."

    • "the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp. Stop delaying the inevitable!

      Only in this case, it is a case of a system od evolving into a limited and broken version of Scheme. ;-)

  • Sounds exciting (Score:5, Interesting)

    by YutakaFrog ( 1074731 ) on Wednesday February 01, 2012 @11:41AM (#38891703)
    Lua has some notable differences from more prominent languages like Java, but as a World of Warcraft addon developer, I find it a surprisingly robust and fun language to program in. I look forward to this change to Wikipedia and hope it works well for all of their contributors.
    • by Talderas ( 1212466 ) on Wednesday February 01, 2012 @12:12PM (#38892003)

      I want a DPS meter addon for wikipedia.

      • by Dotren ( 1449427 )

        I want a DPS meter addon for wikipedia.

        I think you mean an EPS meter.. or "Edits Per Second". I think that would take all of those cross-editing arguments on there to a whole new level.

        • by Anonymous Coward

          DPS == Delete Per Second

          Of couse, only useful for admins.

      • by Anonymous Coward

        And a port of Omen to measure editor threat, too.

    • Lua is a language with neat syntax and clear semantics which tend towards minimalism without sacrificing usefulness, that seems to be designed by sane people for a change. Compared to some other *cough* JS *cough* it's practically a godsend.

  • by Phrogz ( 43803 ) <!@phrogz.net> on Wednesday February 01, 2012 @11:55AM (#38891829) Homepage

    I'll be interested to see if they go for WoW-style "raw", imperative Lua (gobs of functions) or a more OOP-style Lua [phrogz.net] (NB: my site).

    In designing the Lua interface for an old Game UI authoring product [gamasutra.com] I originally went with OOP-style Lua. It was (IMHO) a rather elegant wrapper on our DOM. However, we soon found that the memory thrash of using Lua's lightweight userdata to go back and forth between C++ and Lua resulted in poor performance on consoles, and I ultimately had to redesign the interface to be more WoW-like for our next release.

    It was a shame, putting more onus on the scripter to manage objects (tables of properties in Lua) based on a 'pointer' passed around to uniquely identify each element in the DOM, and passing that pointer to all relevant functions. But the performance increase was dramatic.

    • by Phrogz ( 43803 ) <!@phrogz.net> on Wednesday February 01, 2012 @12:03PM (#38891911) Homepage

      BTW, my personal opinion on Lua:

      It's a fun language to learn, because at the core it is *so* simple. In less than a week a good scripter can fully wrap their head around everything that Lua has to offer from the scripting side (not the C++ side; that might be another week). It's rather elegant, really, with convenient syntax for integer-based for-loops that automatically create a new copy of the loop variable on each pass for simple closure creation.

      However, when you get down to actually typing in itwell, it's not as verbose as Java, but there's some real RSI danger there. With it's simple core come decisions like "not only will we not give you foo++, we won't even give you foo+=1". Try typing things like "frameCounter = frameCounter + 1" many times and you'll start to scream. Every day I scripted in Lua at work I would long for the times when I could use Ruby to actually get something done.

      For those who know JavaScript and want to get a glimpse of what Lua is like, I have a page on my site: Learning Lua from JavaScript [phrogz.net].

      • by speps ( 1108625 )
        After integrating Ruby into the SciTE editor, I can say that Ruby's biggest drawback is it's complete lack (at least when I tried it, maybe not now with another VM) of simple C embedding like Lua does. It was pain to embed but it worked somehow, it was not as clean as SciTE's Lua integration though.
        • Re: (Score:1, Interesting)

          by Anonymous Coward

          And the previous two posts, just for starters, is exactly why Python continues to be so populuar. Well that and the fact that Ruby works hard to be the anti-culture programmer's language. Which means its unlikely to ever be more than an obscure language.

      • by lahvak ( 69490 )

        If that is really a problem, why don't you make an editor shortcut that will automatically expand frameCounter++ to frameCounter = frameCounter + 1? Who still types out every little piece of repetitive code these days?

      • by mwvdlee ( 775178 )

        Lua is supposed to be embedded and extended with domain-specific functions.
        If you're developing RSI while using Lua, you're probably trying to do too much inside the Lua script that should be done in the application code.

      • Look at how it's implemented though. Lua is small and efficient. Ruby looks like a committee has been working on it, and Python is even bulkier. That's why you see Lua in embedded systems so much more than other interpreted languages. It doesn't get in the way of the application like say Tcl does. Yes it comes with very little in the way of "standard library" which is also a benefit in many ways. I've seen some extremely elegant OOP systems for Lua that are short and do what is needed and no more, whi

      • One thing about Lua is that its source code is also small and clean and easy to work with. So it's trivial to change things that you don't like, or add small things that seem to be missing.

      • by pnot ( 96038 ) on Wednesday February 01, 2012 @04:13PM (#38895305)

        However, when you get down to actually typing in itwell, it's not as verbose as Java, but there's some real RSI danger there. With it's simple core come decisions like "not only will we not give you foo++, we won't even give you foo+=1". Try typing things like "frameCounter = frameCounter + 1" many times and you'll start to scream.

        Which, for me, immediately raises the question "Are there any good Lua IDEs?". I mainly code in Java, and it's true that it can often read like the Book of Deuteronomy -- but fortunately I don't have to type all that shit out, because NetBeans autocompletes a lot of it for me. Is there anything similar for Lua?

        • As for IDE I was somewhat successful with IntelliJ Idea [jetbrains.com] and its Lua plugin which seems to be mature enough. You can supply your own autocompletion scripts and docs as well.
          There's also Lua plugin for Eclipse but that was not versatile enough for my purposes.
    • I hope they're going with a PHP extension, and not implementing LUA in PHP. I'd rather have the execution happen in C code than PHP, for execution speed reasons, and I imagine their server operators feel similarly. (Though there's the obvious counterpoint that you're adding a new chunk of less-tested code that has fewer barriers to cross to exploit some vulnerability...)

      In any case, I can't say I'm upset at the change--I'm actually a bit giddy. MW templates are a royal PITA.

      • by Anrego ( 830717 ) *

        To be honest, while that would be great for wikipedia itself, for users of wikimedia, having to install a PHP add-on would be a nightmare (especialyl those with shared hosting).

        • To an extent. Shared hosting providers are a dime a dozen, though, so finding a different one is pretty easy. Then there's Wikia and a number of other wiki farms ranging from free to an equivalent to managed hosting. And migrating a MW site isn't all that difficult, and not necessarily even time-consuming; the software is very good about that kind of thing, including cases where you need to upgrade.

          Server administration to add that kind of thing is easy, and where a shared hosting provider falls behind, mig

          • by Anrego ( 830717 ) *

            True enough for people who run a website that mainly revolves around their wiki.

            However, lots of people just throw a mediawiki install to supplement the rest of their site, usually precisely because it's dead simple to get running and works on just about any host. Moving to another host just to preserve their little 10 page wiki is probably not sensible, and the content is probably in-appropriate for external wiki hosts (or isn't desirable for other reasons).

            Obviously for people with their own server (or in

            • Those users will be able to stay on their pre-LUA version of MW for a major release or two (upstream doe security and bugfix releases for a while), which should give their vendor time to upgrade, if necessary.

              And, yes, I'm sure someone will come up with a MediaWiki extension to implement Lua in pure PHP, and patches will probably be accepted to allow selection of Lua providers in LocalSettings.php. Of course, the reverse is also plausible; upstream might choose to use a pure-PHP solution, and the existing P

              • Roll on Apache 2.3 it has Lua scripting built in ....forget implementing Lua in PHP, if you really want to you could implement PHP in Lua ...

            • True enough for people who run a website that mainly revolves around their wiki.

              However, lots of people just throw a mediawiki install to supplement the rest of their site, usually precisely because it's dead simple to get running and works on just about any host. Moving to another host just to preserve their little 10 page wiki is probably not sensible, and the content is probably in-appropriate for external wiki hosts (or isn't desirable for other reasons).

              Obviously for people with their own server (or in my case, a VPS) this is a non-issue .. but I figure there are probably enough people for which this would be an issue that I can't see them not at least providing a PHP only implementation as an option.

              In your simple case, is there any reason to upgrade to a version of MW that implements LUA for templates, especially if it means an incompatibility with your current provider?

              • I think that bug fixes and security patches might be one reason, but maybe they will be backported.
      • by nahdude812 ( 88157 ) * on Wednesday February 01, 2012 @01:44PM (#38893283) Homepage

        There is a LUA PHP PECL extension: http://pecl.php.net/package/lua [php.net]

        It's relatively new, but this kind of attention could really skyrocket the extension forward. It's a great idea at large, there are a variety of situations where you want to defer decisions to your customer. Historically that meant creating a kind of pseudo DSL with a bunch of forms to fill out for the customer, with hopefully most major options covered, but usually failing to satisfy a variety of corner cases.

        Another alternative is the V8JS extension (JavaScript). The advantage of JS is that more people know it already, and in may ways, JS is surprisingly elegant (not that Lua isn't). It won't perform as well as LUA though, and requires more resources to maintain the VM.

    • by bonch ( 38532 ) *

      I thought a light userdata was a pointer. I know one technique is to use a full userdata to represent a pointer and use the metatable capability of full userdata to register C functions as methods that interact with the properties of the real object.

    • They'll probably end up writing brainfuck in it, like they did with the existing ParserFunctions language.

  • The author specifically modeled it on MVP to maintaining a strict separation of concerns. Presentation and model don't co-mingle. Might seem a bit unusual at first since the language will seem limited at first. But, it'll keep you from running into the "bloating script" problem.

  • by zarlino ( 985890 ) on Wednesday February 01, 2012 @12:23PM (#38892145) Homepage

    Wikipedia could stick to PHP or switch to any other language. But that's not their problem. Their problem is the messy markup language they slowly created. I know cause once I tried to render their markup inside another app. Basically, they have all sorts of tags that reference obscure server-side behaviour and everything is so entangled that creating a new renderer is basically impossible. This is sad because they are wasting the work of volunteers.

    • This actually is a serious problem that has been discussed on the Wikipedia development and foundation mailing lists. Because Wikicode is not rigorously defined like real HTML/XML, the only definition of correct output is "whatever the current parser generates." This not only makes it nearly impossible to independently implement Wikicode in other products besides MediaWiki, but it also makes it far more difficult to create a WYSIWYG editor that doesn't break things. And doing the latter has been a goal of m
    • by David Gerard ( 12369 ) <slashdot.davidgerard@co@uk> on Wednesday February 01, 2012 @02:50PM (#38894195) Homepage

      That's the precise problem. 1. the language was never designed, it accreted, and is mathematlcally impossible to describe fully in most sensible formats. 2. we can't throw it away because there's billions of words of text in it accumulated over ten years. 3. we can't throw it away because the existing editor base demand it stay because they're used to it.

      So WMF is (a) throwing money as well as brilliance at the problem, and (b) has put Brion Vibber onto sorting out what is to be removed from wikitext, because he's one of two people (Tim Starling the other) that people will accept the opinion of on this matter. All proceeds well :-)

      So now the problems are with seriously complicated things like doing bidirectional text [mediawiki.org] properly - a hard requirement for an international project, and one that is not done quite properly by anyone else. Something where mere dev brilliance has half a chance :-)

      • by doom ( 14564 )

        That's the precise problem. 1. the language was never designed, it accreted, and is mathematlcally impossible to describe fully in most sensible formats.

        Ah. So they should re-implement it in perl.

        Perl hackers live for problems like that.

        So now the problems are with seriously complicated things like doing bidirectional text properly

        I would've thought that was more of a problem for the browser developers. Getting the right UTF-8 output shouldn't be that difficult.

        • Perl, or at least PCRE, is the basic problem: the parser is a series of processing steps, many of which are regular expressions. You could translate that directly into Perl without much work, but it's entirely unclear it would actually buy you anything.

          The bidi problem is not display - per the link I helpfully provided, it's editing. Wherever LTR (usually Latin) and RTL (usually Hebrew or Arabic) are mixed, which is everywhere Hebrew or Arabic are.

          • by dkf ( 304284 )

            The bidi problem is not display - per the link I helpfully provided, it's editing.

            But that's still a browser problem, not really a wiki problem (unless they're writing their own editor in Javascript or something equally boneheaded). You've got to delegate some problems to others, even if you know they're doing some stuff wrong, because there's just never enough time to do everything yourself.

            • Per the page, they considered delegating it to the browsers, but the browser versions suck too badly. Read the link, it's informative.

      • by Haeleth ( 414428 )

        That's the precise problem. 1. the language was never designed, it accreted, and is mathematlcally impossible to describe fully in most sensible formats. 2. we can't throw it away because there's billions of words of text in it accumulated over ten years. 3. we can't throw it away because the existing editor base demand it stay because they're used to it.

        Wait, are you talking about MediaWiki templating or PHP?

    • by Aluvus ( 691449 )

      They are sticking to PHP. What they are changing is the language used by templates (the system that allows reuse of text, but has grown to provide all programmatic features available to editors). The current templating system is at least part of what you accurately describe as "obscure server-side behaviour".

      As it stands right now, "plain" templates that just insert some text or whatever are not too bad. But templates that do anything more complex, especially if they require the use of ParserFunctions, q

  • by alexbirk ( 2565251 ) <alex&petrasoft,net> on Wednesday February 01, 2012 @12:43PM (#38892385)
    Embedding Lua for configuration or building templates is it's real strength. I've used it many times in programs that require pretty extensive configuration and it's a joy in that environment. I think it's a great choice for this.
  • Wiki syntax did always strike me as stunningly stupid. It has essentially become just another markup language. Except that it's much more unclear, shitty and limited than original XML. Which is a textbook example of the inner-platform anti-pattern.

    Another horrible example is TypoScript. Which is a template language, written in another template language (PHP)!! Again becoming a shitty clone of PHP. Which itself already is a shitty clone of a proper scripting language. (I've had to use it in my day job, every

    • by Lennie ( 16154 )

      As I understand it:

      MediaWiki is written in PHP and they wanted to create a sandbox for the templates scripts, so they choose to use Lua as a PHP extension.

      Because Lua is very suitable for embedding, as that is what it's general purpose in life is.

      There is no PHP in a sandboxing inside PHP as far as I'm aware of.

    • by dkf ( 304284 )

      Except that [wiki syntax is] much more unclear, shitty and limited than original XML.

      Yes, but it's far simpler to write in wiki syntax. I know you don't necessarily agree, but my experience is that hardly anyone is able to write correct XML; they just don't bother to create well-formed documents (never mind validity). With wiki syntax, you can do a pretty good job of recovering something sensible out of virtually any character sequence.

      Well, unless you have a complicated templating system like mediawiki's one. Writing good templates for anything other than the very simplest of substitutions

  • by PortHaven ( 242123 ) on Wednesday February 01, 2012 @01:17PM (#38892863) Homepage

    Seriously, Wikipedia's #1 fault and the reason I ceased actively contributing is that it requires humans to use a mark-up language for what is essentially a simple text based document.

    And all such edits would be handled much easier via a WYSWIG editor. Yes, elitist monkeys with far too much time on their hands love that feel of doing something complicated for the sake of it.

    Those more intelligent and or beings who have furthered the race through reproduction tend not to want to waste time.

    Implement a simple editor that facilitates editing. And let computers do what they do best, process. And humans do what they do best collate ideas and knowledge.

    First rule of computers. Don't waste time doing what a computer can do better than you.

    • Implementing a "Simple editor" which allows the user to use all of wikipedias features is not a simple task. It is a very very very tough task, for the same reason that making a general html editor is very difficult. There is a reason that so much html/css is still written "by hand" and it is not because we like it.

    • by lahvak ( 69490 )

      And all such edits would be handled much easier via a WYSWIG editor.

      YMMV, but I find that editing structured documents is much faster using plain text, than a WYSIWYG editor. That is if the mark up language is not idiotically verbose, like XML.

      Don't waste time doing what a computer can do better than you.

      Exactly! Why should I waste time formatting the document, when the computer will easily do it for me. I just type what I want to say, and let the computer place it on the screen.

      • " I find that editing structured documents is much faster using plain text, than a WYSIWYG editor"

        I assume you mean something like HTML or such. But I also suspect this does not apply to something like a word processor document, where you likely use Word or something similar?

        Wikipedia documents are supposed to be word processor documents. Unfortunately, they've been implemented in code. *THAT* is the problem here.

        And as one of the Wiki's more prolific authors, I state this from more than a little experience

        • by Hatta ( 162192 )

          I assume you mean something like HTML or such. But I also suspect this does not apply to something like a word processor document, where you likely use Word or something similar?

          He's talking about LaTeX. It's both a word processor document AND code. And it's better than Word in every way except the steepness of the learning curve.

        • by epine ( 68316 )

          Wikipedia documents are supposed to be word processor documents.

          Huh, I never got that memo. I thought it was a system for collaborative editing and mass distribution. In my own experience, I've always found WYSIWYG turns far too quickly into WYGIWYSW (what you get is what you're stuck with). And besides, the collaboration system depends on a diff tool with a human-accessible interpretation. It's not as if authoring is the only mission-critical task.

          I started writing some WordPress posts recently in the

        • by lahvak ( 69490 )

          But I also suspect this does not apply to something like a word processor document, where you likely use Word or something similar?

          Actually, I use Vim editor to enter LaTeX or ConTeXt code for all my documents. I find Word clunky and hard to use, OpenOffice and LibreOffice are even worse.

          A good references and media manager that would work with a variety of mark up languages would be a nice thing to have, though. I personally would prefer something keyboard driven over dragging things with mouse, but so f

    • Huh? You're comparing apples to oranges here. Editing the content in a wiki entry has nothing to do with this Slashdot article.
    • by UpnAtom ( 551727 )

      Would have thought /. to be the last place people would complain about having to embed their own codes.

      However, I agree totally -- especially when it's using nonstandard tags.

      Not sure if this is the biggest problem with Wikipedia. Any subjects where the zealotry is one-sided end up incredibly biased. This is because Wikipedia policy assigns too much scope to consensus, which in turn is decided by how unfriendly the zealots can be to new editors.

      I also cheered the other week when I discovered /. had starte

  • by Osgeld ( 1900440 ) on Wednesday February 01, 2012 @01:31PM (#38893081)

    the only language that would use more words to describe the article than what's in the article

  • The nmap guys seem to have considered a few scripting languages too for a while, and stuck to Lua because of a couple of reasons addressed in this [nmap.org] conference (and probably in some other place in the NSE docs). While I know nothing of the people behind the scenes of Wikipedia, I do kind of trust the decisions made by the nmap team, so my guess is it's not a clueless decision.
  • I don't have the bookmark here, but I followed the discussion ( I am on that mail list, and I am a huge fan of javascript ) is that with Lua, is possible to have "quotas". You can limit what LUA do in cpu and ram useage, while a javascript vm maybe will end stressing the server. This was the ultimate motive. This and that some features we easy to implement (where already implemented in the discussion). I think this mean that Javascript must add these things, and make easy for "language embeders" to co

    • Javascript is a well known scripting language and has been made into a reasonable one because it is used a lot ...but it is not a good language

      Lua was designed as an embedded scripting language, and is very good at it, but is not a general purpose scripting language

      Now look at the the scripting languages people use outside Web sites and JavaScript is almost unheard of ...there is a reason for this

  • For example, the lua table object is used by the standard library (table functions [lua.org]) to represent C arrays with integer indexes or more accurately C++ vectors. However, in general you should not attempt to use the resulting table object in any other way than provided for by the table functions. It would have been more user-friendly to have an own vector type for this.

    Although it isn't a problem most of the time, sometimes you want to preserve the order in an associative array with non-numerical indexes like

  • User-generated anything (code, data, content, etc) is best supported when you allow many modes of expression, and freedom to change without a standards committee getting in the way.

    On the other hand, machines require a fixed standard, or something that changes relatively infrequently.

    For this reason, I think the choice of any scripting language here is as ill-conceived as the web itself being standardized on HTML/JavaScript.
  • Gebus, we're spending developer cycles on THIS?

    How about the ability to drag an URL into the body of an article to automatically create a reference?

    Which is more important?

    • The problem you are seeing is with the editor you are using ....not with MediaWiki..

      Try dragging a URL into a text editor and it will fail ...what a surprise ?

The use of money is all the advantage there is to having money. -- B. Franklin

Working...