What’s wrong with technical tutorials

I’ve been diving into a few new technologies recently, and I’ve been heading to Google to find tutorials and documentation to help me out.
I was describing my frustration with the way these tend to be written to my colleagues. A typical example is the “Do something with x in 20 minutes” tutorial. For Ruby on Rails, it’s writing a blog engine, and for Angular, it’s a to-do list.
These are great for showcasing techniques that overcome the itches that they were written to scratch, but they don’t explain what those original itches were.
It’s a bit like being taught to parallel park, and your instructor saying “This car has power steering. Power steering makes everything easier. See how quickly and easily I can turn the steering wheel, even though I’m not moving. Power steering is the best technology ever”. For people who know how to park, and know how hard it is to turn a non-powered steering wheel while stopped, this kind of information is great; but for people who are there to learn how to park, it doesn’t give them any of the information they need to know about how to park, why they’d want to parallel park or that there might be other parking alternatives.
These newbies might stumble across a method for parallel parking, but they attribute it to the power steering because that’s what their tutor was raving on about during their lessons. When faced with a new problem, they might then say “What we need now is power-gearing, or power-acceleration” and end up using the wrong tool for the wrong job because they never understood the motivation behind the right tool.

Soap Opera Personas

Teams often use personas to help them identify their users and the types of actions they’ll perform. This is usually a workshop exercise that will last many hours and identify 5-10 key characters with a few paragraphs of history and motivation.
I find that these personas tend to get forgotten and that user stories are still written “As a Customer” or “As a Customer Service Agent”.
I’ve been recommending that my teams use a method I call “Soap Opera Personas”. In TV soap operas, we don’t get a big introduction to the characters at the beginning of the episode; we get thrown in at the deep end and we learn about the characters as they go about their daily lives. For persona writing, this means that we can avoid the workshop exercise and start with very basic characters.
Take “Carl, the Customer Service Agent”. As we start to define stories, we write them “As Carl” and bit by bit, we get to learn a little bit about him. For the first few stories, we might not need any other characters. As the project progresses, we might find a few behaviours that seem outside of Carl’s usual routine. If someone suggests a feature that requires setting accelerator keys so that keyboard users are more effective, we can introduce “Kevin, the keyboarding CSA”. When we need to have reporting and administration functions, we can introduce “Sharon the Supervisor”.

This might seem a little cute, but it has immense power. Security features can now be tested by having Carl try to do something that only Sharon is permitted to do. The team can push back on features that seem at odds with the stories they’ve built so far; “This feature is labelled as a Sharon feature, but it seems more like something that Kevin and Carl would be doing day-to-day. Is there a reason for that?”
It gives the team something to identify with. Just as somebody can jump into a soap opera at any point, and infer the backstory; new team members absorb the characters as they listen to discussions during planning or story writing. It enriches the experience and gives another dimension for people to ask questions.
In soap operas, characters sometimes suddenly reveal a secret history; Phil is in witness protection from the Mafia. As long as the revelation doesn’t completely invalidate the character we’ve come to know, we’re happy to accept it. Our personas can also cope with significant changes when key aspects of the project change, allowing stories and automated acceptance criteria written in this style to adapt as the project progresses.

Like with all good characters, once you get to know them well, the stories write themselves.

Opscode Chef and the Single Responsibility Principle

My current project is managing the software and distribution of a hardware appliance. We send out physical machines to our customers and we really need to know that they are all identical.
I’ve set up a build system that uses PXE to boot the Ubuntu installer, which is preseeded with a very basic install of linux. We then use chef (actually chef-solo) to configure the remainder of the software. (I’ll write a bit more about the reasons behind this later)

While I’ve been writing new recipes, I’ve started using a pattern that ensures good separation of responsibilities while making the recipes themselves easy to read.

In my pattern, the default.rb file in the recipe consists only of includes. The default recipe is used to make sure that something useful happens.
For example, I created a recipe to install and configure the Hamachi VPN client. This consists of two recipes; install.rb (which downloads and installs the appropriate package and dependencies) and connect.rb (which authenticates with Hamachi and joins the appropriate network). The default.rb includes these two recipes.

I’ve found that this style results in much tidier recipes.

Maintainable Code

I’ve uploaded a presentation about Maintainable Code that I put together for a client almost a year ago.

It’s a lot more text than I’d prefer in a presentation, but this was for a small group of developers and was more of a face to face conversation with examples on the slides to back up the points

It’s mainly a series of thoughts that I should have shared on my blog or twitter; hopefully this will go some way to helping get the ideas out there

What can a consultancy do for me?

An external consultancy can help an organisation progress in a number of ways. When not outsourcing, these usually fall into three categories; training (someone shows you what to do, in a controlled environment), mentoring (someone shows you what to do, in your own environment) and coaching (someone encourages you to explore and develop your own environment)

Training is usually fairly low cost, but it usually only provides an overview of what something can do for your team. Training can be a useful morale boost, but it very rarely becomes immediately applicable to your business domain. Acquiring a new skill requires practice in a real environment. This loosely corresponds to moving someone onto the Dreyfus skills ladder or moving from Unconscious Incompetence to Conscious Incompetence in the four stage model.

Mentoring allows your team to try out new skills under the guidance of a skilled practitioner. When the team member does something inconsistent with current practices, the mentor will intervene, explaining why they are intervening, any likely problems from the original solution and demonstrate an alternative solution, explaining why they think this may be more appropriate. After the discussion, the pair may decide to continue with the original solution and see if it works, or go with the alternative. This loosely corresponds to moving the team member to Competent on the Dreyfus model or to Conscious Competence in the four stage model.

Coaching is a more hands-off approach. The team will continue to work in their normal style, while the coach observes. Using Socratic questioning, the coach will generally guide the team members to explore new avenues of thought, to discover problems that they are likely to encounter and to work in a way that minimises the likelihood of any unknown issues encountered becoming an insurmountable block. Coaching offers the highest value, as it gives the team a toolset for modifying their thinking on an ongoing basis. It corresponds loosely to moving into the advanced stages of the Dreyfus model (Proficient / Expert) or towards Unconscious Competence in the four stage model.

Agile is a means, not an end

At RiverGlide, we’re often asked to help with “Agile transformations”.

My definition1 of agile is being flexible to adapt an existing process.

That is, if your existing process looks like waterfall, you’re still agile if you introspect, retrospect and experiment to improve your process. If at the end, your process looks like waterfall, then as long as you have introspected, retrospected and experimented and found that this approach gives the best results, then your process is agile.

[1] Although this might not fit with other definitions. Your mileage may vary

Finally, a use for checked exceptions

Those of you who have followed my blog, been subjected to it on a client site or have talked to me about programming will know that I have a few issues with checked exceptions.
Well, this morning I had an idea of a potentially powerful use of checked exceptions (if only temporarily).

The same people who will have heard about my dislike of checked exceptions will probably also know about my dislike of returning null. I was mulling over a legacy code base that I may be taking over in the future and one of the first things that I would like to do is to replace all null returns with something meaningful.

Normally, I would remove nulls by searching for the offending return statement, fixing it to do something meaningful (and if I don’t know what meaningful means in this place, throw an exception), then find usages of that method in the workspace and remove any (now defunct) null checks.

Enter the checked exception as a refactoring tool!

public Result someMethod() {
Result result = someMethodIDontControl();
if(null == result) {return null;} // Actually, this is really crap because in reality, I'd just return result, but bear with me!
return result;


public Result someMethod() throws IUsedToReturnNullPleaseFixMe {
Result result = someMethodIDontControl();
if(null == result) {throw new IUsedToReturnNullPleaseFixMe();} // This is temporary. The right thing _might_ be an exception, but we'll work that out later
return result;

Now we can lean on the compiler to find all the places that call this method. We can strip out any null checks (that are now unnecessary), look to see if there is something meaningful that we should be doing, and then add throws IUsedToReturnNullPleaseFixMe to the method we’ve just fixed. When we’ve propagated our exception to the top level, we’re done. What I would do at this point is probably delete the exception class, fix all the resulting compile errors by removing the throws clauses, until finally the last compilation error is the original throw new IUsedToReturnNullPleaseFixMe(); which I can now replace with something meaningful (e.g. throw a meaningful exception, return a meaningful NullResult, convert to a collection and return emptySet() etc)

I’ll show you mine if you show me yours

Bob Marshall posted on Twitter:

I replied with:

and he replied:

In 1956, George A. Miller wrote a paper arguing that the conscious mind, the part that contains our inner voice and that does the reasoning processes, can handle 7 ± 2 items at a time. My view is that, in order to be able to function within these constraints, we learn to group similar things together so that they take up less processing space.

For example, if I want to remember a six digit number, it might be easier for me to remember it as 3 two digit numbers than as 6 individual numbers. Very familiar numbers, such as the phone number of my parents only take up one space.

It’s like I’m carrying a fixed number of boxes, all of which are the same size, but in which I can store things that (in my own mind) are grouped together.

One thing I’ve noticed though, is that I begin to speak of the ideas inside in terms of the labels that I’ve added to the boxes. In terms of hard facts, like numbers or addresses, this doesn’t really make much difference, however, when the ideas and opinions in the boxes are much more subjective, this can lead to confusion and disagreements (or sometimes, vigourous agreement :-) )

For example, imagine that I am moving house and packing my belongings into boxes. In one box, I pack the contents of the fridge and because the fridge is in the kitchen, I label the box “Kitchen Stuff”.

I say to the removals people, “The box labelled ‘Kitchen Stuff’ has to be moved quickly. It’s very important that this box is moved as fast as possible”.

The removals people reply “You’re insane. Boxes labelled ‘Kitchen Stuff’ have to be moved carefully, and that means that we have to take it slow”

“No,” I say, “this box has to be moved quickly. If you move it slowly the contents will be ruined”

“No,” they say, “if we move it quickly, the contents will break. We must move it slowly and carefully”

I open the box and show them the contents and they understand. “Oh, it’s fridge stuff. Why didn’t you say so? When you said ‘Kitchen Stuff’ we assumed that you meant plates and glasses”

When we put labels on things in the boxes, it becomes much quicker for us to talk in terms of the labels. However, although we may have boxes labelled in a similar way, that does not mean that the contents are the same.

The box in my mind labelled “Scrum” may match a great deal of the contents of the box that you have labelled “XP” or that someone else has labelled “Lean”. The good thing about these boxes in my mind is that I can have an object appear in more than one of them. I might even place these three boxes into another box labelled “Common sense” (although I strongly suspect that the box with that particular label is guaranteed to be vastly different between every person)

As with my conversation with the removals men, I can get into a heated discussion with someone if we’re talking in terms of the labels on our boxes.

“XP is this, this and this”.

“No, that’s Scrum!”

The only way we can truly reach an understanding is if we recognise that we’re arguing over a label, and open our boxes to compare contents.

Sometimes, I even forget what I have stored in my boxes. I might only be talking about my memory of the things that I think I stored in the box. Opening the box and reviewing the contents, especially when someone else is doing the same, can help me to find new ways of categorising the ideas within. We might even find a common ground and start a revolution.

So, next time we don’t see eye to eye about the memories of the contents of our boxes, remind me “I’ll show you mine, if you show me yours”

That’s just the way we do it here

There is a story about a cage of monkeys. In the middle of the cage is a ladder and at the top of the ladder is a banana.
One of the monkeys climbs the ladder to get the banana, but as he reaches for it, the entire cage becomes electrified and all the monkeys get a sharp shock.
Another monkey climbs the ladder, with the same result.
After a few attempts, the monkeys learn that climbing the ladder means everyone gets a shock, and they don’t try it again.

After a few days, one of the monkeys is replaced with a new monkey. This new monkey sees the banana and moves towards the ladder. The other monkeys, not wanting to be shocked, leap on him and beat him up. The new monkey attempts to go for the banana a few more times, before learning that going for the ladder means that the other monkeys will beat him up.

Over time, each of the original monkeys are replaced with new monkeys. Each new monkey learns the lesson that going for the ladder means getting beaten. Eventually, there are no monkeys who ever experienced the shocks. The electric shock equipment is then switched off. Still, no monkey ever climbs the ladder, for fear of being beaten up.

Unfortunately, monkeys are unable to ask “Why do you beat me up when I go for the banana?”, but if they could, the other monkeys would probably say “That’s just the way we do it here”.

Sometimes, our processes are based on a situation that no longer exists (like the inactive shock equipment). Sometimes, people we ask “Why do we do it this way?” won’t be able to provide an answer. Sometimes, we’ll try doing something different and we’ll get the banana and sometimes we’ll get a shock. Either way, we learn something.

I’m going to do something different today. Are you?

The monkey story is not mine, I think I came across it in an NLP book. There are plenty of examples on the web

What do you keep in your Shu Box?

Alistair Cockburn uses the term Shu-Ha-Ri regularly to explain different levels of learning.
The literal translation is approximately Learn/Follow(Shu), Detach(Ha), Transcend(Ri). A number of people I’ve discussed this with equate these levels to the Dreyfus model of skill acquisition. In fact, the Wikipedia article on Dreyfus links back to Shu-Ha-Ri.
In my opinion, this is doing an injustice to both models, and my slightly different interpretation follows.

Driving an example

Take, for example, learning to drive. As I learn to drive, I progress through the Dreyfus model, becoming a novice, then competent, then with a lot of practice, an expert. However, having only learned one way to drive, I would still consider myself to be in the Shu box.

Riding my way to the top

When I learn to ride a motorbike, I can take a number of similar practices from my car experience and apply them to the similar practices in the motorbike world, but I still need to be taught new practices (particularly around observation and hazard avoidance). After a suitable period of tuition, I am now competent in riding a motorbike. I would still consider myself to be a Shu level rider though.

Mixing it up

Now, while driving my car, I suddenly realise that the heightened observation and hazard avoidance techniques, essential while riding, are actually pretty useful while driving. I begin to blend the two skills together. I’m still doing it as taught, but I can pick the practices that are most useful from each and blend them to create something new. This is where I start to become Ha level, taking the most useful practices from a number of techniques and blending them to create something useful in a given context.

Taking it to new heights

Next, I learn to fly a plane, bringing with it new observational practices and new disciplines (e.g. pre-flight checks). Becoming competent here, I can blend some of the practices from flying back into my driving and riding and vice-versa.
Soon I begin to notice a strange feeling. I’m doing all the right things in my practices, but there’s something that seems not quite right, something that could be made better. After some thinking, I identify the missing piece of context and synthesise an appropriate set of techniques for that situation. I have gone beyond the known contexts and identified a new situation, picked or synthesised the appropriate practices for that situation and resolved it. This is Ri level. Having identified this new context, I can teach it to my Shu level students.

But mum, I don’t want to ride a bike

Each of the situations described above are very relevant to me, but that is not to say that you have to ride a bike to progress to Ha or Ri in a car.
Other practices and contexts can be learned on the race track, taking a 4×4 course, rally driving or on a skid pan. Indeed, the beauty of Ha and Ri is that I may bring skills from a completely unrelated area (First Person Shooter, Knitting) and use them to improve my driving techniques.

Softening it up a bit

So, how can we apply this to software? There are a number of practices, techniques and methodologies (e.g. Kanban, Scrum, XP, Lean, Real Options). By the definition explained above, becoming in all of these makes me a Shu level practitioner in each of them.
If I identify a practice from, say, Kanban, that is useful in a context in a Scrum environment, then blending two or more Shu practices takes me to Ha.
When I come across a situation or context where there is no guidance (all of my Shu boxes are empty), then I need to create a new practice or technique. I use my knowledge of similar techniques from a number of methodologies and come up with something that fits the situation.

In Shu-mmary (sorry, summary)

In my interpretation of Shu-Ha-Ri, we have:

  • Shu (Learn) – Practice driven. We learn the practices by rote and apply them without understanding of the context.
    Shu practices are our tools, along with an instruction manual on how to use them in the situation they were created for. We can progress through Dreyfus model with each of our tools
  • Ha (Blend) – Context driven. Having learned a number of practices, and a number of contexts in which they are appropriate, we can pick or blend a number of different practices for any known context.
    Ha practices are using our Shu tools in situations that weren’t described in their manuals
  • Ri (Transcend) – Dealing with unknown context / identifying new practices. Coming across a brand new situation, we are able to identify an appropriate way of handling the situation.
    Ri practices are about creating new tools, either to handle a unique situation, or to handle an existing situation in a better way

I find that separating the Dreyfus model from the Shu-Ha-Ri model allows me to become an expert with each of my Shu tools without feeling that I should be able to apply them to unusual situations.
This is a useful way of thinking (to me at least). Rather than map Shu-Ha-Ri to the Dreyfus model, I choose to think of Shu-Ha-Ri as a skill in identifying contexts and tools.
You might want to discuss this with me some more in the comments :-)