Maurits thinks aloud

Extracting data from JIRA using Clojure and REST

I have previously written two posts about getting data from JIRA. In the first one I described how you can do that in Ruby. In the other one I used JavaScript to import JIRA data directly into Google Documents. Both methods worked and both heavily rely on the SOAP interface that JIRA offers. However, as of JIRA version 5 a REST interface is the preferred way. So I decided to try this using Clojure.

Initially this seemed to be very straightforward. I have been using a well-written Clojure lib that provides an Async Http Client in the past to extract for example Twitter stream data. So it would only take a couple of minutes to setup a new Clojure project, use basic authentication to connect to a JIRA instance at work and extract all the data I needed. Well, it didn’t.

My first goal was to list all projects. On the commandline I successfully used:

$ curl -u user:password https://intranet.mycompany.com/jira/rest/api/latest/project

In Clojure, using the aforementioned library that would become:

(ns jira.core
  (:require [http.async.client :as c])
  (:use [clojure.data.json :only (read-json)]))

(def jira-project-url "https://intranet.mycompany.com/jira/rest/api/latest/project")

(defn get-all-projects
  "Return all projects from JIRA"
  []
  (with-open [client (c/create-client) :auth {:user "user" :password "password"}]
    (let [response (c/GET client jira-project-url)]
      (c/await response)
      (read-json (c/string response)))))

However, this always returns an empty vector and I still haven’t found out why. So I started to looking for an alternative. I imagined a better way would be to setup a session, retrieve a session cookie and use that for subsequent calls. I’ll begin with showing the end result:

(ns jira.core
  (:require [http.async.client :as c])
  (:use [clojure.data.json :only (read-json json-str)]))

(def session-url "https://intranet.mycompany.com/jira/rest/auth/latest/session")

(defn login
  "Login into JIRA"
  [username password]
  (with-open [client (c/create-client)]
    (let [response (c/POST client session-url
                           :headers {:Content-Type "application/json"}
                           :body (json-str {:username username :password password}))]
      (c/await response)
      (c/cookies response))))

This is more or less the Clojure equivalent of:

$ curl -c cookie_jar -H “Content-Type: application/json” -d ‘{“username” : “user”, “password” : “password”}’ https://intranet.mycompany.com/jira/rest/auth/latest/session

With this implementation of login that returns a session cookie I rewrote get-all-projects to:

(defn get-all-projects
  "Return all projects from JIRA"
  [cookies]
  (with-open [client (c/create-client)]
    (let [response (c/GET client jira-project-url :cookies cookies)]
      (c/await response)
      (read-json (c/string response)))))

This version works perfectly and enables me to write compact code like:

(def session (login "user" "password"))
(map :name (get-all-projects session))

This example generates a map with the abbreviated names of all projects currently in JIRA.

Right now the code is mostly proof of concept. As soon as it is cleaned up a bit I will start a new project on GitHub to share it. Enjoy!

First ClojureScript experiences: using Raphaël

I recently started to experiment with ClojureScript. In order to get a little bit beyond ‘hello world’ I decided to convert one of Raphaël‘s examples to ClojureScript. According to the projects homepage Raphaël is ‘a small JavaScript library that should simplify your work with vector graphics on the web’.

The sample I used draws a kind of clock. It can be found here. The relevant JavaScript code:

window.onload = function () {
    var r = Raphael("holder", 640, 480),
        angle = 0;
    while (angle < 360) {
        var color = Raphael.getColor();
        (function (t, c) {
            r.circle(320, 450, 20).attr({stroke: c, fill: c, transform: t, "fill-opacity": .4}).click(function () {
                s.animate({transform: t, stroke: c}, 2000, "bounce");
            }).mouseover(function () {
                this.animate({"fill-opacity": .75}, 500);
            }).mouseout(function () {
                this.animate({"fill-opacity": .4}, 500);
            });
        })("r" + angle + " 320 240", color);
        angle += 30;
    }
    Raphael.getColor.reset();
    var s = r.set();
    s.push(r.path("M320,240c-50,100,50,110,0,190").attr({fill: "none", "stroke-width": 2}));
    s.push(r.circle(320, 450, 20).attr({fill: "none", "stroke-width": 2}));
    s.push(r.circle(320, 240, 5).attr({fill: "none", "stroke-width": 10}));
    s.attr({stroke: Raphael.getColor()});
};

As you can see the JavaScript code is already quite compact. I especially like the use of the anonymous function in the inner loop. What I did next is a very straightforward conversion to ClojureScript just to see if I could get that to work. This turned out as:

 
(ns raph.hand) 
(defn clj->js
  "makes a javascript map from a clojure one"
  [cljmap]
  (let [out (js-obj)]
    (doall (map #(aset out (name (first %)) (second %)) cljmap))
    out))

(defn- attr [object attributes]
  (.attr object (clj->js attributes)))

(defn create-hand [paper]
  (.getColor.reset js/Raphael)
  (-> (.set paper)
      (.push (-> (.path paper "M320,240c-50,100,50,110,0,190")
                 (attr {:fill "none", :stroke-width 2})))
      (.push (-> (.circle paper 320 450 20)
                 (attr {:fill "none", :stroke-width 2})))
      (.push (-> (.circle paper 320 240 5)
                 (attr {:fill "none", :stroke-width 10})))
      (.attr "stroke" (.getColor js/Raphael))))

(defn create-circle [paper angle hand]
  (let [c (.getColor js/Raphael)
        t (str "r" angle " 320 240")
        circle (.circle paper 320 450 20)]
    (-> circle
        (attr {:stroke c, :fill c, :transform t, :fill-opacity .4})
        (.click #(.animate hand (clj->js {:stroke c, :transform t}) 2000 "bounce"))
        (.mouseover #(.animate circle (clj->js {:fill-opacity .75}) 500))
        (.mouseout #(.animate circle (clj->js {:fill-opacity .4}) 500)))))

(defn ^:export draw []
  (let [paper (js/Raphael 0 0 640 480)]
    (let [hand (create-hand paper)]
      (doseq [angle (range 0 360 30)]
        (create-circle paper angle hand)))))

The ClojureScript takes a couple of more lines than the JavaScript original: 36 versus 23. This has mostly to do with a couple of helper functions to convert Clojure maps to JavaScript maps.

A couple of things I learned/noticed during the conversion:

• The threading macro (->) really is very convenient as an alternative to the fluent interface used in this example
• It take me a couple of minutes to realize that “getColor.reset()” is a single function call instead of a reset() called on a property
• Using macros in ClojureScript is possible, but not as straightforward as I thought. Macros are handled at compile time and you have to put them in a separate Clojure file. Next you reference them with require-macros in the namespace definition. How this is done exactly is shown here
• I replaced the ‘clj->js’ function by a macro according to this excellent blogpost by keminglabs. However the ClojureScript compiler couldn’t find the macro on its classpath and for the moment I settled for the ‘clj->js’ function
• There is lots of room for improvement in the code. My first idea was to make it more DSL like, using macros. But then again, SVG is already kind of a DSL, so it would probably make more sense to let the code output SVG directly instead of using Raphaël as an extra layer. David Edgar Liebke already started a project called Apogee, A ClojureScript SVG and charting library.

Finally the html code I used to test my code:

<html> 
  <head>
    <title>Hello, world</title>
    <script type="text/javascript" src="jslib/raphael-min.js"></script>
    <script type="text/javascript" src="raphtest.js"></script>
  </head>
  <body>
    <script>
      	raph.hand.draw();
    </script>
  </body>
</html>

And to compile

cljsc src '{:optimizations :advanced :output-to "raphtest.js" :externs ["lib/raphael-min.js"]}

For a really good explanation on how to set up a more robust ClojureScript development environment, have a look at Sean Corfields blog.