Cucumber and Natural Language...WHY?!?

Date May 11, 2011

For the record, this is another post I expect to get flamed for...throw another cuke on the barbie, I suppose...

If you're unfamiliar with Cucumber, the description on the main page is "behavior driven development with elegance and joy". If that wasn't clear enough, typically, you use Cucumber to develop tests and perform actions based on those tests. In that, it's no different than a bunch of other functional test suites. The feather in Cucumber's hat is that it is supposedly "natural language".

What? AI? Not quite. What you actually end up doing is writing code that looks like this:

Scenario Outline: Add two numbers
Given I have entered <input_1> into the calculator
And I have entered <input_2> into the calculator
When I press <button>
Then the result should be <output> on the screen

I could write a long diatribe about why that sucks, but it would be much easier for me to cut and paste the long diatribe that I already wrote on the Reddit thread about Cucumber-Nagios here:

I've looked into Cucumber, and to be really, really honest, it seems like it's a little heavy handed.

Great, natural language is like English, so when we read it, we understand immediately, but I think I would hate to write it. It seems stilted and counterintuitive. Plus, it's not really natural language.

Feature: www.google.com
It should be up

Who talks like that?

It's all syntax, but while it's being syntax, it might as well be a little more tightly designed. There are enough synonyms and valid phrasologies in the english language that:

It should be up

reads the same to me as

It should be running

or

Make sure it is running

or

Ensure it is up

Don't even get me started with

Scenario: Clicking on the Videos link
When I go to http://www.google.com
And I follow "Videos"
Then I should see "Google Videos"

Reading something that's supposed to look like english tells me what it does, it doesn't make my brain parse code. I look at "Ensure it is up" and think, "yeah, I see that this service needs to be up", but I don't think, "the syntax for that statement is wrong". There are few, if any, synonyms in "normal" computer programming.

if (! isRunning(servicename) ) {

is pretty specific. Sure, it doesn't do all of the nifty implied actions of "It should be up", but that's what the rest of the statement is for.

I just find it counter-intuitive, which is exactly opposite of its goal.

(minor phrase changes edited in for blogification)

That's pretty much how I feel. I can appreciate some of the very cool things about the language (for example, if Spanish is your first language, you can write in Spanish...or Mandarin, for that matter...), but what I perceive to be a bad form outweighs a decent function.

I don't want to feel like I'm beating up on Cucumber here. It's really natural language programming that I'm against. Here's an example from the wiki page I just linked to:

If U_ is 'smc01-control', then do the following. Define surface weights Alpha as "[0.5, 0.5]". Initialise matrix Phi as a 'unit matrix'. Define J as the 'inertia matrix' of Spc01. Compute matrix J2 as the inverse of J . Compute position velocity error Ve and angular velocity error Oe from dynamical state X, guidance reference Xnow . Define the joint sliding surface G2 from the position velocity error Ve and angular velocity error Oe using the surface weights Alpha. Compute the smoothed sign function SG2 from the joint sliding surface G2 with sign threshold 0.01. Compute special dynamical force F from dynamical state X and surface weights Alpha. Compute control torque T and control force U from matrix J2, surface weights Alpha, special dynamical force F , smoothed sign function SG2. Finish conditional actions.

If you really, honestly, in your heart of hearts, think that's better than the equivalent C, perl, python, or ruby, then that's fine, but I'm definitely not in that group.

Please comment to tell me how wrong I am.

  • http://paperairoplane.net Oliver Hookins

    I don't think you are wrong at all. That being said, I'm still using cucumber-puppet. Why? It's a good tool that allows you to express some higher-level goals and verify in some way that your code does what you expect it to.

    Yes, you do end up duplicating your code (in this case, Puppet code) in a ratio of about 1:1 but it still helps you refactor and test in a useful way. It *is* extremely verbose, and very time consuming but I feel that if there was another tool that did the job better, we'd all be using that instead.

    I eagerly await the delivery of such a tool ;)

  • http://danieru.com/ danieru

    Natural language programming really is an easy thing to hate. Programming is programming, and programming is "hard". Adding "familiar" syntax dosn't make it any easier it just lies to the non-programmer folling them about the a programming language mustrealities of programming. With that said one interesting take on natural language was for setting files. If I remember right there were a few protected keywords and excess words were permited anywhere in the line. So you got something like: 'Init $length to half the room height "50"'. Since settings files have very limited functionality all your parse needs to do is find the variable ($length) and the value ("50"). Thus you could also get something like 'Turn $usehardwood "off"', which could just as easily be 'Do "not" $useHardwood'.

    Out side of config files I don't think natural language is appropriate. Config files get a pass because they are used by non-programmers. Once you start programming things need to get serious and so should the language.

  • http://www.richard-slater.co.uk/ Richard Slater

    From my point of view as a Developer BDD and Cucumber/Natural Language specification is not for the benefit of the developer. It is for the benefit of the Business Analysts and product owners (that is the person who is responsible for specifying it's function).

    These people don't understand why if (!isRunning(serviceName)) means what it means to a programmer, they do however understand a statement such as When I go to http://www.google.com And I follow “Videos”...

    Yes it seems forced and stilted, however we need to find ways of bridging the gap between the programmers and the product owners minds in a structured way that doesn't require explaining the specific syntax of a programming language.

  • http://blog.sysdroid.com Giovanni

    Natural language is not very efficient so I understand why a programmer would hate it. When you know something and use it extensively, you start thinking of ways to not waste too much time. The way humans speak is not very efficient, we waste words because they are not precise, we have to explain things in dozen different ways to capture all the details. All this "wastage" is considered bad in programming languages.

    I can understand why people think natural languages are good, perhaps to bring people without computer experience to do some work but as soon as these people learn just a little bit of what programming is.. they will wish the programming language could be better.

  • http://www.straw-dogs.co.uk Doug

    I'm glad I'm not the only one who thinks this. I remember when I first saaw Cucumber I thoguht it looked pretty cool. But then when I considered the practical use of it I got chills. Programming is programming - natural language is far too variable for programming to be made like it with any great success.

    To think a programming language can cater to all natural language just because it happens to cover Spanish and even Mandarin is crazy. I could probably give a good 50 variations of British English dialects in just one sentence alone. Not to mention the multitude of American dialects then other languages.

    I can only imagine get incredibly frustrated that the so-called natural language doesn't understand MY natural language use. There's a reason programming languages are the way they are. Symbols and keywords enable a strong and clearly defined syntax and grammar that can't be confused with natural language.