The nonsense of risk management

April 2, 2012 at 8:36 pm | Posted in Programming | 3 Comments

Let me start this blogpost with a little fictitious story about two project managers:

Paul and Stephen were sitting outside the conference room, waiting for the steering committee to call them in. Paul looked at Stephen with a smug expression on his face. “I’m so glad that I spent that extra money on hiring two extra developers right from the start. That was quite expensive, but since one of our developers has quit during the project and another one got ill for almost a whole month, this was money well spent. We made the deadline! So how about your project?” Stephen sighed. His project hasn’t been that successful. Their test server had failed a couple of months ago and ordering a new one took quite some time. This has caused a severe delay in his project. If only he had invested in buying a backup server. At the project start the team had created a list of risks. They hadn’t forgotten about this one, but the chance of this happening was considered very low. How wrong they were!

Continue Reading The nonsense of risk management…

Advertisements

Bug reporting: 8 ways to annoy your software development team

March 14, 2012 at 12:04 pm | Posted in Programming, Ramblings | 18 Comments

As usual this blog post should be read with a large grain of salt. It is a collection of bad practices I have seen during many software development projects. There are positive exceptions. For example when the testers are part of the development team and the whole team is committed to delivering valuable software instead of two opposite parties trying to fight each other. Having said that, the ugly situation mostly happens in fixed-price contracts where the bug versus feature discussion often takes place.

Disclaimer: all the examples are made up. Any resemblance with bug reports from projects I did is pure coincidence 😉

So here is some practical advice for acceptance testers on how to maximally help a software development team:

1. Since you have spent a lot of time finding this bug, prioritize it at least as Major. Of course Critical or Blocking is even better so that the development team will realize it is urgent and pick it up immediately. Luckily most bug tracking systems (for example JIRA) have priority Major as the default setting so you don’t have to spent too much time thinking about this. Example: a missing help text on what the field ‘zipcode’ means should be marked as major because it is going to leave the end user clueless.

2. Make sure that the description of the bug is short. A single word is better than a long descriptive sentence. The advantage is that it forces the developer to open this issue every time to understand what it means. This will help him to not ignore this bug just by reading the description. For example ‘Login’ is a much better description than ‘Login failure when I fill in a non existing username’.

3. Don’t let yourself be fooled and use improvement, change or new feature when filling in the issue type. Everything you find is a bug. Otherwise your organization (the customer) will have to pay for it, especially in fixed-price contracts. There are two situations to be handled here: if it is in the specification, then it obviously is a bug if a feature is missing or not accordingly to the spec. If it is not (clear) in the specification then you can always defend it by saying ‘it is obviously that this is not acceptable for the end user, so it is a bug’.

4. Never provide details in your bug report! This might put developers on the wrong track. It is much better to let them find out themselves what caused this problem and how to reproduce it. After all, they are the experts. That will also give them a strong incentive to deliver better software the next time. A good example is of course the good old ‘Doesn’t work’ or ‘Program crashed’.

5. You can (and should!) often re-open an existing bug-report. This is just a practical thing and saves you some time. Especially if you have a good generic bug description (Like for example ‘Login’ mentioned before) this is very convenient. But don’t be bothered if the new bug isn’t related to the old one. You are just helping the developers to limit the total amount of bugs and re-opening fixed issues gives them a nice historic perspective. Software developers like re-use and DRY (Don’t Repeat Yourself), so should you as a tester.

6. Another practical tip: combine multiple bugs in one single bug report. Again you are helping the developers here because many bug reports might give a bad impression. It also helps to give them focus: suppose the login functionality in your application doesn’t work, then it is also a good moment to combine that with fixing a spelling error on that same page and probably adjust the colors a bit so they are more conform the specification.

7. Bug-reports are also great ways to communicate, especially with introvert developers. So you can create a major bug with a question like “What does the specification say about this Cancel button?”. That will help them to think about the spec, enable single/double/triple loop learning, etc. It might take some time, but someday they will appreciate what you did for them.

8. Another great way to stimulate software developers in a positive way is to use bug reports to give them some good advice. For example “Feature xyz is not very easy to use.” This example can be easily marked as ‘Critical bug’ and it will sparkle the creativity of the development team to come up with a great solution. That is also the reason that you, as a tester, shouldn’t give away any information on how this feature should be improved because you want to empower the developers instead of extinguishing  their creativity.

I’m sure there are many more ways to improve the communication between testers and software developers. Don’t hesitate to share them!

Extracting data from JIRA using Clojure and REST

March 2, 2012 at 10:45 pm | Posted in Clojure, Programming | 2 Comments

I have previously written two posts about getting data from JIRA. In the first one I described how you can do that in Ruby. In the other one I used JavaScript to import JIRA data directly into Google Documents. Both methods worked and both heavily rely on the SOAP interface that JIRA offers. However, as of JIRA version 5 a REST interface is the preferred way. So I decided to try this using Clojure.

Initially this seemed to be very straightforward. I have been using a well-written Clojure lib that provides an Async Http Client in the past to extract for example Twitter stream data. So it would only take a couple of minutes to setup a new Clojure project, use basic authentication to connect to a JIRA instance at work and extract all the data I needed. Well, it didn’t.

My first goal was to list all projects. On the commandline I successfully used:

$ curl -u user:password https://intranet.mycompany.com/jira/rest/api/latest/project

In Clojure, using the aforementioned library that would become:

(ns jira.core
  (:require [http.async.client :as c])
  (:use [clojure.data.json :only (read-json)]))

(def jira-project-url "https://intranet.mycompany.com/jira/rest/api/latest/project")

(defn get-all-projects
  "Return all projects from JIRA"
  []
  (with-open [client (c/create-client) :auth {:user "user" :password "password"}]
    (let [response (c/GET client jira-project-url)]
      (c/await response)
      (read-json (c/string response)))))

However, this always returns an empty vector and I still haven’t found out why. So I started to looking for an alternative. I imagined a better way would be to setup a session, retrieve a session cookie and use that for subsequent calls. I’ll begin with showing the end result:

(ns jira.core
  (:require [http.async.client :as c])
  (:use [clojure.data.json :only (read-json json-str)]))

(def session-url "https://intranet.mycompany.com/jira/rest/auth/latest/session")

(defn login
  "Login into JIRA"
  [username password]
  (with-open [client (c/create-client)]
    (let [response (c/POST client session-url
                           :headers {:Content-Type "application/json"}
                           :body (json-str {:username username :password password}))]
      (c/await response)
      (c/cookies response))))

This is more or less the Clojure equivalent of:

$ curl -c cookie_jar -H “Content-Type: application/json” -d ‘{“username” : “user”, “password” : “password”}’ https://intranet.mycompany.com/jira/rest/auth/latest/session

With this implementation of login that returns a session cookie I rewrote get-all-projects to:

(defn get-all-projects
  "Return all projects from JIRA"
  [cookies]
  (with-open [client (c/create-client)]
    (let [response (c/GET client jira-project-url :cookies cookies)]
      (c/await response)
      (read-json (c/string response)))))

This version works perfectly and enables me to write compact code like:

(def session (login "user" "password"))
(map :name (get-all-projects session))

This example generates a map with the abbreviated names of all projects currently in JIRA.

Right now the code is mostly proof of concept. As soon as it is cleaned up a bit I will start a new project on GitHub to share it. Enjoy!

Help, my manager has a vision!

February 19, 2012 at 5:26 pm | Posted in Ramblings, Uncategorized | Leave a comment

I started my career about 20 years ago. The first week I got an email from my group leader in which she told us that she had scheduled a meeting with an important topic. This was back in ’91 or ’92 and there were quite a few layoffs due to the crisis and first Gulf war around that time. So a couple of days later when the meeting actually took place and the whole group (that was 3 people!) looked at her in anticipation and a bit of fear. And then she started the meeting with “It is important that we as a group have a mission statement.” I only remember that we looked at each other slightly uncomfortable and thought what the hell is she talking about.

In the following 20 years I have been in this situation many times, so I thought this would be a nice opportunity to write down a simple step-by-step plan to handle such a situation.

Step 1: the Steve Jobs test

This is a very simple test and only takes a couple of seconds. Is your manager a major pain in the butt but at the same time you get a lot of energy when he or she is around? In that case he might actually have a vision which is worth listening to. And even if you don’t listen, if he really is like Steve Jobs his vision will take off anyhow. On the other hand, if your manager is a dull bureaucratic type of person who never had an original idea in his life, it might very well be that he recently has read Steve Jobs biography and mistakenly thinks that he can do that as well.

Step 2: gather the facts

Maybe your manager did read the biography. If not, his silly ideas must have originated somewhere else. There are a couple of possibilities: if you noticed many consultants around the office during the last weeks or months, they might be the source. The other possibility is that there has been an announcement at the company level. You might have noticed an e-mail from the CEO in your mailbox, or maybe even a new company logo. Typically a vision trickles down to the workfloor, where each management layer will take anywhere between a week and a month. This also means that your manager probably doesn’t have a choice: he must have a vision, aligned to the company vision.

Step 3: try to understand the vision

This sounds like a useless step, especially if it is already so obvious to you that your manager isn’t Steve Jobs. However, this only takes a couple of minutes and it is important for the next step. The goal of this step is to find out if by mere coincidence, your boss still has laid out a brilliant vision. Just like the Infinite Improbability Drive this might give tremendous energy to your unit, your department of your organization.

Read the document or presentation. If it is larger than 1 page or 1 slide you can already stop. This is not a vision. Next scan the document for words like ‘better’, ‘cheaper’, ‘faster’, ‘most cost effective’, ‘best in class’, ‘thought leaders’, etc. etc. Real visions don’t include those words.

In the unlikely event that the vision is real (it is motivating and inspiring) you can skip step 4. You are a lucky person with a great manager! Congratulations!

Step 4: handling the situation

You are still reading this because apparently your manager’s vision is as was to be expected: meaningless. Now you have a couple of options to handle this. Let me explain them for you:

Wrong way to react

Telling your boss straight-out that his vision is stupid and a waste of your time is not going to help. Of course his vision is never going to happen but you don’t want to be the person he is going to point to as a scapegoat. It is also not going to help during your yearly performance appraisal.

OKish way

Usually you can quite safely ignore your manager’s vision. Main reason is that there will be a new manager and a new vision anyhow within 1 or 2 years. If you want to have a little bit more fun you can start a lively discussion on the difference between a vision, mission and a strategy, because most likely your manager will have them mixed up. You don’t need a degree in business administration to do this. However, don’t overdo this in large groups because it might make your manager feel stupid.

Right way

However the best way to react is to fully embrace your manager’s vision! First you express your enthusiasm: tell him his vision is very motivational to you. Tell him that he really should share his brilliant ideas with the world for example by writing a white-paper or a blogpost. Make sure that you do this in such a way that he will have to do all the work and won’t be able to delegate it to you. Also make sure that you don’t overdo it: telling him that he should get the peace nobel price for his ideas is probably a bit too much and might raise his suspicion.

After you have shared your enthusiasm you can now weave in your personal agenda. Here is where the results of step 3 come in: if one of the keywords for example is ‘thought leader’ than you can use this to explain that this cool conference in Hawaii on some state-of-the-art technology you want to go to is really helping to support his vision. If you feel self-confident you can also rephrase this as ‘with this practice I can help you to operationalize your vision so that as a group we can really meet our business goals.” Additional benefit: you can refer to all your support during your performance appraisal.

Final words: don’t take me, this blogpost, your manager or yourself too seriously 🙂

Have fun!

First ClojureScript experiences: using Raphaël

February 13, 2012 at 10:24 pm | Posted in Clojure, Programming | 4 Comments

I recently started to experiment with ClojureScript. In order to get a little bit beyond ‘hello world’ I decided to convert one of Raphaël‘s examples to ClojureScript. According to the projects homepage Raphaël is ‘a small JavaScript library that should simplify your work with vector graphics on the web’.

The sample I used draws a kind of clock. It can be found here. The relevant JavaScript code:

window.onload = function () {
    var r = Raphael("holder", 640, 480),
        angle = 0;
    while (angle < 360) {
        var color = Raphael.getColor();
        (function (t, c) {
            r.circle(320, 450, 20).attr({stroke: c, fill: c, transform: t, "fill-opacity": .4}).click(function () {
                s.animate({transform: t, stroke: c}, 2000, "bounce");
            }).mouseover(function () {
                this.animate({"fill-opacity": .75}, 500);
            }).mouseout(function () {
                this.animate({"fill-opacity": .4}, 500);
            });
        })("r" + angle + " 320 240", color);
        angle += 30;
    }
    Raphael.getColor.reset();
    var s = r.set();
    s.push(r.path("M320,240c-50,100,50,110,0,190").attr({fill: "none", "stroke-width": 2}));
    s.push(r.circle(320, 450, 20).attr({fill: "none", "stroke-width": 2}));
    s.push(r.circle(320, 240, 5).attr({fill: "none", "stroke-width": 10}));
    s.attr({stroke: Raphael.getColor()});
};

As you can see the JavaScript code is already quite compact. I especially like the use of the anonymous function in the inner loop. What I did next is a very straightforward conversion to ClojureScript just to see if I could get that to work. This turned out as:

 
(ns raph.hand) 
(defn clj->js
  "makes a javascript map from a clojure one"
  [cljmap]
  (let [out (js-obj)]
    (doall (map #(aset out (name (first %)) (second %)) cljmap))
    out))

(defn- attr [object attributes]
  (.attr object (clj->js attributes)))

(defn create-hand [paper]
  (.getColor.reset js/Raphael)
  (-> (.set paper)
      (.push (-> (.path paper "M320,240c-50,100,50,110,0,190")
                 (attr {:fill "none", :stroke-width 2})))
      (.push (-> (.circle paper 320 450 20)
                 (attr {:fill "none", :stroke-width 2})))
      (.push (-> (.circle paper 320 240 5)
                 (attr {:fill "none", :stroke-width 10})))
      (.attr "stroke" (.getColor js/Raphael))))

(defn create-circle [paper angle hand]
  (let [c (.getColor js/Raphael)
        t (str "r" angle " 320 240")
        circle (.circle paper 320 450 20)]
    (-> circle
        (attr {:stroke c, :fill c, :transform t, :fill-opacity .4})
        (.click #(.animate hand (clj->js {:stroke c, :transform t}) 2000 "bounce"))
        (.mouseover #(.animate circle (clj->js {:fill-opacity .75}) 500))
        (.mouseout #(.animate circle (clj->js {:fill-opacity .4}) 500)))))

(defn ^:export draw []
  (let [paper (js/Raphael 0 0 640 480)]
    (let [hand (create-hand paper)]
      (doseq [angle (range 0 360 30)]
        (create-circle paper angle hand)))))

The ClojureScript takes a couple of more lines than the JavaScript original: 36 versus 23. This has mostly to do with a couple of helper functions to convert Clojure maps to JavaScript maps.

A couple of things I learned/noticed during the conversion:

  • The threading macro (->) really is very convenient as an alternative to the fluent interface used in this example
  • It take me a couple of minutes to realize that “getColor.reset()” is a single function call instead of a reset() called on a property
  • Using macros in ClojureScript is possible, but not as straightforward as I thought. Macros are handled at compile time and you have to put them in a separate Clojure file. Next you reference them with require-macros in the namespace definition. How this is done exactly is shown here
  • I replaced the ‘clj->js’ function by a macro according to this excellent blogpost by keminglabs. However the ClojureScript compiler couldn’t find the macro on its classpath and for the moment I settled for the ‘clj->js’ function
  • There is lots of room for improvement in the code. My first idea was to make it more DSL like, using macros. But then again, SVG is already kind of a DSL, so it would probably make more sense to let the code output SVG directly instead of using Raphaël as an extra layer. David Edgar Liebke already started a project called Apogee, A ClojureScript SVG and charting library.

Finally the html code I used to test my code:

<html> 
  <head>
    <title>Hello, world</title>
    <script type="text/javascript" src="jslib/raphael-min.js"></script>
    <script type="text/javascript" src="raphtest.js"></script>
  </head>
  <body>
    <script>
      	raph.hand.draw();
    </script>
  </body>
</html>

And to compile

cljsc src '{:optimizations :advanced :output-to "raphtest.js" :externs ["lib/raphael-min.js"]}

For a really good explanation on how to set up a more robust ClojureScript development environment, have a look at Sean Corfields blog.

Improving the Scrum standup questions

January 19, 2012 at 8:39 pm | Posted in Agile | 1 Comment

Anyone practicing Scrum will probably know the three questions that are asked during the daily standup:

  1. What have you done since yesterday?
  2. What are you planning to do today?
  3. Any impediments/stumbling blocks?

There are several problems with the way these questions are phrased. Firstly team-members might get defensive because they feel like you have to explain you really did do something important since the last standup. So regularly I hear answers like “Well, I spent 8 hours on this really very difficult task. But I’m almost done. Will probably finish it today and there are no impediments. Next!”. Wow, this developer managed to burn 8 hours during the last 8 hour working day. Impressive! So there is no information at all in this answer for the Scrum Master or the other team-members.

To tackle this last problem some Scrum Masters rephrase the questions:

  1. What have you accomplished since yesterday?
  2. What are you planning to accomplish today?
  3. Any impediments/stumbling blocks to reach your goal?

At first glance these questions look ok. You really force every team-member to explain what progress he/she has made. Everyone is happy apart  from those poor developers that didn’t accomplish anything since yesterday. Maybe they needed more time to think about a difficult design, maybe they were tackling a nasty bug. In my opinion these questions can be very demotivating.

Of course we all learned that the standup is not meant to be a progress meeting. It is a coordination mechanism between team-members. So can we do better than the two previous approaches?

Yes we can. I recently read a book that has become quite popular “The Lean Startup” by Eric Ries. What I really liked is his message that a start-up company is all about learning, not about the perfect result. And that is exactly what can motivate Scrum teams: software developers are knowledge workers, not just people working at an assembly line accomplishing small predictable tasks in an endless way. So why not state the standup questions in terms of learning:

  1. What have you learned since yesterday?
  2. What are you planning to learn today?
  3. Any impediments/stumbling blocks to keep you from learning?

One of the benefits is that this will greatly improve knowledge exchange within the team. People might even feel comfortable with spending a day doing hammock-driven development (by Rich Hickey, Clojure inventor) as long as they can explain what they have learned. And remember, learning that an approach didn’t work is still learning. Of course at the end a team will need to produce software, but I’m convinced that concentrating on learning instead of accomplishing smaller tasks will get you there a lot faster.

Please give these questions a try and share the results here or in your favorite Scrum forum!

Why Scrum will never work

July 13, 2011 at 11:22 am | Posted in Agile, Programming | 76 Comments

With such a slightly provocative title I will probably have to start with the disclaimer first: what is written here is my own opinion and not necessarily that of my employer. That is, if I still have one after posting this blog. What’s more: I’m a big fan of Scrum and other Agile methods. It pays my bills. Uh wait, let me rephrase that a bit more accurate: I’m totally 100 % convinced that Scrum works for software development.

Now with these formalities behind me let’s get a bit more serious: I really like Scrum. I have been using it for the last 5 years, I haven given several presentations about (distributed) Scrum at several conferences, I’ve written an article about it together with Scrum guru Jeff Sutherland, etc. However now that the Agile Manifesto is 10 years old I thought it would be fun to put on Edward de Bono‘s black hat and explain why Scrum is never ever going to work.

Reason 1: the cornerstone of Scrum is about trusting people. Creating a safe environment so that we can be open to each other, learn from our mistakes. And all that other touchy-feely hippy back to the 60’s stuff. That is not going to work! Did you notice my disclaimer when I started this blog? I had to put it there because quite a few people read my blog, including customers and my boss. There are a lot of pointy haired bosses out there (oh no, another disclaimer: by boss isn’t one, he is a nice friendly guy, bla bla bla). This world is full of alpha males (and females?) who are not at the least interested in you, your process, the outcome, etc. Being open is only going to hurt your career. Room for mistakes, taking risks? Don’t be naive!

Reason 2: according to Scrum ‘people do the best they can’ if you give them enough freedom. What the hell is this based upon? They don’t. They will probably do the least they can because in general most software developers are underpaid, especially compared to their managers. That’s why they want to become managers or software architects as soon as possible, since they can then still be lazy without anyone noticing and the added bonus that they are getting better paid.

Reason 3: because of the previous reason we still have to put project management on top of Scrum teams, so at least it has some output. So this is probably going to be business as usual. Assigning tasks to team members, micromanaging developers, demanding progress reports, etc. All the usual actions to slow your team down as much as possible.

Reason 4: Scrum is just a process. I have seen many processes (like CMM which is nowadays called CMMI) and I have seen them all fail and leave a lot of frustrated people behind. So if you are stuck (like most companies) with a bunch of average people then nothing is going to change. Scrum doesn’t improve your software, good people do! And by definition, you don’t have those good people since you just have Joe Average (or worse) as a programmer because your company doesn’t want to pay a decent salary.

Reason 5: Scrum delivers ‘business value’. Well no, actually it doesn’t. For many reasons. The guys or girls that know about business are not going to be involved in your project. They like to lunch with customers, not work on this weird thing called backlog to explain a bunch of introvert nerdy software developers what to do. So your team ends up with a junior help desk employee as a product owner. And besides, your whole ICT department is a cost center anyhow. So don’t start about business value.

Reason 6: an Agile team is supposed to continuously improve. That is why Scrum has retrospectives to see what went well, what can be improved and to define actions. Now do you really think people want to improve? First they have to think of possible improvement actions. Next they may even have to execute them, which might well take them way out of their comfort zone. People resist change, and therefor improvements. Your old working habits may suck, but at least they kinda work and it gets you through the day.

Reason 7: the Product Owner focusses on the ‘what’ and ‘why’ questions, the development team decides ‘how’. Nicely separated so the team can go for quality and thus high velocity on the long term. However, this is not going to work. Your product owner wants this functionality right now and doesn’t care the least about software quality. Just deliver those features as fast as possible because there always is a deadline, promises made to this important customer, etc. And don’t think you can blow away this junior product owner, because behind him is this business manager ranked high in the companies hierarchy. You as a developer are just part of a cost center and probably going to be outsourced soon anyhow. Now how is that for motivation and trust?

Reason 8: my previous point was about quality. There is some evidence that pushing productivity lowers the quality of software. On the other hand, when you focus on quality, you will get higher productivity. However Joe Average programmer doesn’t care about software quality. If the quality is poor, developing some piece of software takes more time, but why care? He is getting paid between 9 and 5. The project manager (or Scrummaster!) will take the blame for missed schedules. Even worse, if this developer is hired from another company it is in his (and his company’s) interest to stay as long as possible. So this all means that your productivity in Scrum isn’t going to be the least bit higher than with any other method.

Reason 9: “yes, but if we only build the necessary features, then at least we will have a lower total cost, right?” It never stops to amaze me how naive people are when they say something like that. You don’t build necessary features. Most of the time you are on a fixed-price contract for a major banking or insurance company. Or even worse: a government contract. They have selected you because you offered the cheapest bid (which is pretty naive in itself) but they are going to make sure that you will deliver all the requirements they have stated up-front. Of course at least 50 % of these requirements have no business value at all, but hey, you aren’t going to fool the project manager that handles the project from the customer side. He is an alpha male and you are going to deliver that last bloody feature as well!

Wow, wearing that black hat was even more fun than I thought! Must…. take…. it…. off.. now…

Less code matters: don’t repeat yourself

June 27, 2011 at 9:24 am | Posted in C#, GIMP, Programming, Uncategorized | Leave a comment

I got quite a few reactions on my previous blog post both positive and negative. The message of the negative comments was mostly that short code shouldn’t be a goal since it tends to result in unmaintainable code. Next some people accused me of writing ‘clever code’ like using lambda functions which they reasoned are not going to be understood by those who will maintain my code. Despite of the tone of some of these comments, they are all very interesting and surely hold some value. At least they gave me a couple of new ideas for future blogs.

In this post I’ll show how I removed another 4 lines from the GIMP# AverageBlur plug-in. The following lines of code have been bothering me for a long time but I never took the time for fixing them:

using System;

namespace Gimp.AverageBlur
{
  class AverageBlur : Plugin
  {
    static void Main(string[] args)
    {
      new AverageBlur(args);
    }

    AverageBlur(string[] args) : base(args, "AverageBlur")
    {
    }

    // Remaining code omitted
}

There are 2 things I didn’t like about this code:

  1. The Main function creates an object which is apparently unused. The reason is that in the constructor all kind of magic happens, include a call to gimp_main which starts the actual GIMP plug-in. This is hard to understand: constructors should be straightforward. Just create the object and make sure it is in a defined state. Nothing more than that.
  2. Every GIMP# plug-in is derived from the Plugin class but still seems to need it’s own constructor. The only reason for this is that I have to parse a couple of parameters like the arguments (args) and the package name that is used for translation. However this package name is always the same as the plug-in name.

While it is easy to get rid of the first disadvantage by introducing an explicit function call with the object as parameter, that would still leave me with a constructor for every plug-in I write. However, that was my first attempt to clean up this code:

using System;

namespace Gimp.AverageBlur
{
  class AverageBlur : Plugin
  {
    static void Main(string[] args)
    {
      // Do all magic stuff in GimpMain instead of in constructor
      GimpMain(new AverageBlur(args));
    }

    AverageBlur(string[] args) : base(args, "AverageBlur")
    {
    }

    // Remaining code omitted
}

Now if I just remove the parameters from the constructor and pass them to GimpMain I can use the default constructor so I don’t have to define it anymore:

using System;

namespace Gimp.AverageBlur
{
  class AverageBlur : Plugin
  {
    static void Main(string[] args)
    {
      // Do all magic stuff in GimpMain instead of in the constructor
      GimpMain(new AverageBlur(), args, "AverageBlur");
    }

    // Remaining code omitted
}

We just removed 4 lines of code while at the same time improving readability. We are left with one single line that calls GimpMain where all the magic happens. Still this line looks a bit like boilerplate code since it will be exactly the same for every plug-in. The final code I used looks like:

using System;

namespace Gimp.AverageBlur
{
  class AverageBlur : Plugin
  {
    static void Main(string[] args)
    {
      // Do all magic stuff in GimpMain instead of in the constructor
      GimpMain<AverageBlur>(args);
    }

    // Remaining code omitted
}

The Plugin base class instantiates the object as follows:


abstract public class Plugin
{
   protected static void GimpMain(string[] args) where T : Plugin, new()
   {
      var plugin = new T();
      Catalog.Init(typeof(T).Name, Gimp.LocaleDirectory);

      // Remaining code omitted
   }
}

I introduced a generic method. The type parameter T has the constraints that it should be derived from the Plugin class and that it should have a default constructor. In line 5 an object of this type is created. Line 6 now gets the class name (typeof(T).Name) which I use as catalog name for the locale.

Over time the size history for the AverageBlur plug-in now looks like this:

Current code size is now 54 % of what it once was, down from 60 % in the previous revision.

Some people might wonder if this code isn’t ‘too clever’. Generic methods in C# is not something every programmer might be 100 % comfortable with. In a next blog post I am going to write about clever code and try to quantify this a bit more. Until then, enjoy reading this post and I’m looking forward to any comments and/or improvements!

Less code matters

June 16, 2011 at 9:31 pm | Posted in Agile, C#, GIMP, Programming | 11 Comments

One of Edsger Dijkstra‘s quotes I really like is: If we wish to count lines of code, we should not regard them as “lines produced” but as “lines spent”. This hasn’t changed since those days despite of almost infinite amounts of processing power, memory and powerful IDE support. In fact I would argue that thanks to this power we are now able to build huge systems which makes carefully spending these precious lines of code even more important. The main reason still is that once you start producing code, you have to maintain it. And maintaining code will take effort and therefore will cost money.

Over the years I have worked on a lot of code, both existing (as in maintenance projects) and while building new software. My personal statement was and still is that any non-trivial amount of code can easily be reduced to half it’s size while increasing readability at the same time. There are several ways to reduce code size. As an example I will take 18 revisions (from CVS) from a small GIMP# plug-in and show what I did to gradually reduce the size.

Let’s start with looking at the graph that depicts the code size first:

This is a small plug-in that calculates the average color of all pixels in an image. Next the plug-in sets the color of all pixels in the image to this average color. This functionality is provided as the “Blur Average” filter in Photoshop and I wrote it because it didn’t exist for the GIMP yet.

I took the size in lines of code from CVS. I included all white lines and comment lines, apart from the standard 20 lines of GPL header at the start of every file. Next I used the maximum size (at revision 3) as 100 % and scaled the rest of the revision sizes relative to this maximum. Now let’s see what happened at these 18 revisions:

  1. Revision 1.1. Checked in initial code. This was mainly the boiler plate code that comes with any plug-in written in the GIMP# framework. At that moment this was 50 LOC.
  2. Revision 1.2. Plug-in now fully functional. Wrote two straightforward loops, one to calculate the average, the other one to apply it to all pixels. Code has grown to 63 LOC.
  3. Revision 1.3. A GIMP# co-developer added code for i18n. Code now at it’s maximum size of 68 LOC. Still pretty small of course. Keep in mind that a similar basic plug-in written in C (the default GIMP programming language) takes about twice as much code.
  4. Revision 1.4. Accidentally  in revision 1.3 a key (from a key/value pair) was translated. Fixed that which saved 2 lines. Nice start on my way to a smaller plug-in. 66 LOC left.
  5. Revision 1.5. The i18n initialization that happened in all plug-ins, was moved to the constructor of the base class, removing another 3 lines. 63 LOC left.
  6. Revision 1.6. Until this revision I had to handle all pixels as arrays of 3 (or 4, including alpha channel) bytes which held the RGB(A) values. I abstracted this into a Pixel class and wrote an iterator that calls a delegate for every pixel in this particular image. I added some overloading magic for Pixel objects that allows me to add pixels and easily calculate the average. Of course this added to the main GIMP# library, but my plug-in code shrunk to 55 LOC.
  7. Revision 1.7. A bit of cleaning up, code size stayed at 55 LOC
  8. Revision 1.8. Instead of returning a set of all supported procedures (needed to register a plug-in within GIMP) this same function now returns the C# yield construct, saving another 3 lines. 52 LOC left.
  9. Revision 1.9. Minor clean-up, improving readability. Code still at 52 LOC.
  10. Revision 1.10. Improved algorithm to calculate the average. In previous revisions I was updating a counter (that had to be initialized) inside the first iterator so I could later divide the sum by this counter to calculate the average. Of course the number of pixels within an image (or selection) is already known and can be asked directly from the iterator class. This allowed me to remove another 2 lines, leaving the size now at 50 LOC.
  11. Revision 1.11. The delegate to calculate the average was since the previous revision a short one-liner. No need anymore to spread that over 3 lines, including the curly braces. Inlining this delegate into the iterator call removed 3 lines. Code size at 47 LOC.
  12. Revision 1.12. Oh no! Code has grown to 48 LOC. Reason is that I tried anonymous function support in Mono, concluded that it didn’t work (yet) but left the updated line as a comment in the code.
  13. Revision 1.13.  Still 48 LOC. Only minor textual changes to code.
  14. Revision 1.14. Finally I realized that have commented out code is bad practice. Removed it, reducing the size to 47 LOC again.
  15. Revision 1.15. Mono 1.2.6 supported lambda functions. Did a bit of cheating and removed 1 empty line that divided 3 lines that logically belonged together. Size now at 46 LOC.
  16. Revision 1.16. A new C# 3.0 feature (object initializers) allowed me to remove another 2 lines. Code size at 44 LOC.
  17. Revision 1.17. Another 2 lines moved to the Plugin base class. 42 LOC left.
  18. Revision 1.18. Simplified GIMP# framework API a bit, allowing me to remove another line from almost all plug-ins. This is the most recent version which was checked in on June 10th, 2010. No changes since that time. Code size is 41 LOC.
As you can see I went from a maximum of  68 LOC to the current size of 41 LOC. I didn’t manage to remove half of the code, but 40 % still isn’t bad for such a small amount of code. At least I don’t have to maintain those 27 removed lines anymore. At the same time the readability has improved a lot. In a next blog I will categorize all the methods that can be used to reduce your code size, based on my personal experience.
For completeness the final code:
using System;
using System.Collections.Generic;

namespace Gimp.AverageBlur
{
  class AverageBlur : Plugin
  {
    static void Main(string[] args)
    {
      new AverageBlur(args);
    }

AverageBlur(string[] args) : base(args, "AverageBlur")
    {
    }

    override protected IEnumerable<Procedure> ListProcedures()
    {
      yield return new Procedure("plug_in_average_blur",
                                 _("Average blur"),
                                 _("Average blur"),
                                 "Maurits Rijk",
                                 "(C) Maurits Rijk",
                                 "2006-2009",
                                 _("Average"),
                                 "RGB*, GRAY*")
        {MenuPath = "<Image>/Filters/Blur"};
    }

    override protected void Render(Drawable drawable)
    {
      var iter = new RgnIterator(drawable, _("Average"));

      var average = drawable.CreatePixel();
      iter.IterateSrc(pixel => average.Add(pixel));
      average /= iter.Count;

      iter.IterateDest(() => average);
    }
  }
}

A simulation to show the importance of backlog prioritization

June 8, 2011 at 6:36 pm | Posted in Agile, Uncategorized | 10 Comments

A successful agile software project depends on a prioritized backlog. You can have a perfectly functioning development team that efficiently finishes one feature after the other, but if these features hold no business value it still is waste.

To show just how important this prioritization is I did a Monte Carlo simulation, using Google Docs spreadsheet. Here are the assumptions I made to create the model:

  • the business value of a user story is randomly generated from a uniform distribution between 1 (very low value) and 100 (very high value).
  • the effort to implement this user story is generated by drawing from a standard set of Scrum poker cards with the values 0, 1/2, 1, 2, 3, 5, 8, 13, 20 and 40 (100 and ? are omitted). The drawing was done first selecting an index based on a normal distribution with average 5 and standard deviation 1. This (zero-based) index is used to select a card from the range of poker cards. For example index 5 corresponds with the fifth card which happens to be 5 as well. Index 6 corresponds with the value 8, etc.
  • the assumption was made that there is no correlation between the business value and the effort needed to implement the corresponding story.
  • the whole backlog can be delivered in 10 sprints.
  • during the first 4 sprints the velocity is increasing in a ration 1, 2, 3, 5. This means that the development team is 5 times as fast in the forth sprint compared to the first. After that the velocity stays constant at a value of 5.

With these assumptions I calculated the delivered business value (as a percentage of the business value of the whole backlog) using 4 different prioritization algorithms:

  • no prioritization. The development team picks up the stories in the (random) order from the top of the backlog.
  • prioritization using the business value of the stories. Stories with high business value are built first.
  • prioritization using the needed effort to build a story. Stories with low effort are built first.
  • prioritization build on the ratio between business value and effort. Stories with a high ratio (high business value, low cost) are built first.

The results can be seen in the following graph:

As you can see from this graph is that not prioritizing your backlog will deliver business value quite late. Since there is no correlation between business value this will be a straight line after the first 3 iterations in which we have a low velocity.

The other extreme is the prioritized backlog using the business value / effort ratio. This is the red line. Here you can see that after 5 iterations you already have delivered 80 % of the business value. For the backlog without prioritization this takes more than 8 iterations!

What looks a bit surprising initially is the difference between the prioritization on business value (the yellow line) and prioritization based on effort (the green line). This is introduced by the fact that in our simulation we have created a bias towards more expensive user stories: the number of user stories with effort lower than 5 is equal to the number of user stories with an effort higher than 5, but remember that the Scrum poker card values are not symmetric: 0, 1/2, 1, 2, 3 versus 8, 13, 20 and 40. That is the reason that in this simulation it is better to prioritize on effort instead of business value.

There is one more interesting aspect to investigate. What is the impact of the standard deviation that we use to draw from our poker cards? I repeated the simulation using the business value / effort ratio for the prioritization and different values for the standard deviation. The result:

The results are easy to explain: when using a higher value for the standard deviation you will get more extremes (either 0 or 40) for the effort to implement a user story. This means that there will be more user stories which require little effort and deliver good business value. These can be build first.

Bottom line: make sure your product owner works hard on a prioritized backlog. Let him quantify the business value of his user stories. As you can see from this simulation half of the effort (and cost) needed to complete the whole project might already be sufficient to fullfil his business needs. This is a huge money saver!

« Previous PageNext Page »

Blog at WordPress.com.
Entries and comments feeds.