A group of blind men heard about a strange animal, which had been brought to the town. It was called an “Elephant“, but none of them knew its shape or form. They said: “Let us inspect and know it by touch, of which we are capable”. So, they went. The first, whose hand landed on the trunk, said “It is like a giant snake”. The second, whose hand reached the elephant’s ear, claimed it seemed like a kind of fan. The third, whose hand was upon its leg, said, the elephant is a pillar like a tree-trunk. The blind man who placed his hand upon the side said the elephant, “is a wall”. Another who felt its tail, described it as a rope. The last felt its tusk, stating the elephant is that which is hard, smooth and like a spear.
Indian fable
It seems we all have a tendency to claiming to know the truth, based on a usually very limited, subjective experience. Within our limited perception, with the importance of considering context, we forget that other limited, subjective experiences may be equally true.
This blog post is about tool integration and new approaches to visualization of information flow. All I intended to do was to apply the 3 ways of DevOps and take them a little further. However, it’s been a while since I came up with an unwieldy academic article, so here’s a made-up title for readers with matriculation backgrounds:
Towards an integrated framework for shorter feedback cycles
State of the Union
Tool Integration
There are several integration solutions on the market, from basic nightly CSV exports to internally developed API-based integrations, from ChatOps approaches such as Atomist to large-scale middleware-like solutions such as the Tasktop Integration Hub … not to mention ESBs and message brokers.
So it is quite fair to assume that integration of two or more software tools is more or less a solved problem. We can make data flow between tools.
How about the visualization of data?
Visualization
In IT operations, monitoring solutions like Nagios have long been able to visualize how the infrastructure is doing – almost in real time.
With Kibana (and Elasticsearch/Logstash under the hood) we can not only analyze the UP/DOWN state of a host, disk usage and CPU load, but also visualize and trend various structured data – usually the log files generated by our services.
And last but not least, tools such as eazyBI and Tasktop Viz can visualize whole process chains and business data.
While an integration optimizes the flow of data and a visualization gives feedback, we currently have no direct relation between the two, as it’s different tools.
Having the Cake and Eating it
So far so semi-good, as this breaks the “Experimentation Principle” from the 3 Ways of DevOps (remember Gene Kim’s book?).
First Way: Work always flows in one direction – downstream. Second Way: Create, shorten and amplify feedback loops. Third Way: Continued experimentation, in order to learn from mistakes, and achieve mastery.
We will see a change at some point in the future, without knowing where it came from. Just like the blind men in the fable, we lack context!
For example,
- Assume we double the WIP limits for feature development in a Jira Kanban, what impact would that have downstream, for example in quality assurance?
- Let’s improve the quality of our client-facing documents by hiring another technical writer, how does that change affect customer support?
- Does it affect the error rate if we came up with a policy to have all phones, Slack and Skype muted?
What I propose is to combine both dimensions, i.e. to allow changes directly from the visualization interface.
Objection handling
One of the constants in a consultant’s world is many people knowing beforehand what isn’t possible or viable. So, to make the most of everybody’s time, here are two objections that I could immediately think of (contact me if you have more, I’ll add them)
- Martin, headcount changes or internal policies are things that we cannot change from a tool interface! It involves people!!
- Such changes takes ages before we could actually see a change, so what’s the point?
My Approach: Annotations and Trigger Tickets
#1: Introducing annotations.
In a first iteration, we can work with annotations. So note in the appropriate place what has been changed where and how.
#2: Allow inline changes directly from the visualization.
Instead of just adding comments, the next iteration allows for inline manipulation (where feasible). This way, we make our visualization even more interactive, which would also allow for a simulation mode. What would happen, if more team members take days off during the summer? What’s the effect of more/less investment into education and training?
To accomplish this, we use sliders to change values directly inline, in this example the WIP limit for development.
#3: Using trigger tickets.
Why leave the interface we’re working in? Being a developer means you most probably work in Jira or commit your code to a repo. So, instead of manually fiddling with sliders, let’s use trigger tickets to control system behavior [1]. A similar mechanism already exists with the so-called “Smart Commits” in some repos. We apply this in a broader context now and again change the WIP limit:
If you’ve made it this far…
#4: From triggers to Liquid Democracy with delegated voting.
Why stop here? By now, we have a system which is optimized for the flow of data. We visualize the data in a meaningful way and allow for adhoc changes via trigger tickets. Therefore, it seems natural to use this mechanism to allow for team members to actively influence their working environment. We can implement a full-blown Liquid Democracy – see this example from the Pirate Wiki:
Here, distributed teams can vote on initiatives or, if they’re meh about it, delegate their vote. Rather than sitting in meetings for up/downvoting initiatives, we can now use this mechanism to
continuously improve our working environment.
[1] I am fully aware that processing data and instructions in the same system is basically a Von-Neumann-architecture.