Sunday, June 14, 2015

Architectures of a different type

My father-in-law is an architect -- the kind who designs buildings. He has been an architect for many years and he has worked on a wide variety of interesting projects, from dormitories at Harvard University to pump houses in New York state reservoirs.

As a software architect, I have always been fascinated by the kind of architect whose end product is something you can touch and feel. He's the kind of architect whose work you live in rather than one whose work you live with. I am an architect of the latter -- you have to live with what I do and my goal is to make that process simple, reliable and efficient.

As an architect of the abstract, I've always been curious about how things get designed and built in the tangible world. Building projects can be exceedingly complex and require a collection of many different types of skills, yet we have become pretty good at it. When a company starts to build a building, more often than not, the building gets finished. While it may be a little late or over budget, it gets done and it works like it's supposed to.

Software projects are far less reliable. I've read some articles that claim that most software projects fail. I've seen software projects get abandoned -- something that rarely happens midway through the construction of a new home or office tower.

So the question that comes to mind is this: Why?

Why is writing well-designed software seemingly harder to do than building a well-designed building?

Lately, my father-in-law has been grumbling a lot about the bureaucracy of building a building. He spends more time meeting with planning committees, seeking the approval of inspectors and providing government agencies with various impact reports than he does in the process of dreaming up the design or executing the plan.

On a recent evening, I was mulling this over and I came up with a few ideas for why we seem to be better at building buildings than we are at writing software:
  1. A building's architecture is designed before the process of building begins.
  2. Building architects have to design buildings according to building codes -- clearly defined standards of quality that can be measured and these plans must be approved before construction begins.
  3. The building is inspected by experts as it is being built.
I could keep going, but you get the picture. You would never start a building before the designs were done. You'd never just trust an architect's design; you'd want to make sure the design adhered to quality standards. Likewise, you wouldn't just trust the construction crew to do the right thing and build the building according to the design.

You wouldn't do any of these things with a building, but companies do this with software all of the time. But it's always a bad idea.

Software architects also have a similar set of tools at their disposal, and these tools should always be used:
  • Design patterns: these are the equivalent of building codes and standards. The human race has been writing software for a long time. We've learned a few things in the meantime. Design patterns encapsulate this knowledge into practical solutions.
  • Code reviews: code reviews are the inspections of the software development world. They can be performed by peers, or by managers of projects, but they should always be performed. No software should ever be deployed into a production environment that has not been reviewed by at least one other person.
  • Automated tests: all software development projects should have automated tests that demonstrate that the software does what it's supposed to do. This is another form of inspection -- it makes sure that the software still works, even when changes are made to it later.

Monday, April 13, 2015

Managing for Software Quality

In my previous post, I said that to be an effective manager, you need to control variation and that the key to managing software development is to control variation in the process.

There are a lot of books about software development processes. Unfortunately, these books are written by software developers. The books are often quite good. Especially if you are a software developer. What they don't address is how to manage a software development project from the perspective of the business itself. They claim that following the process will help the business that uses it, but the perspective is too skewed and the focus is too narrow for someone who isn't a software developer herself to get the information they need to understand the process and to operate the "levers of control" that we learn about in business school.

Here are a few thoughts on the process of writing software. I plan to write about each of these in more detail in subsequent posts.
  1. Hire the very best programmers. The degree of variability in productivity between programmers is nothing short of miraculous. All programmers are not created equal. A small group of three really good programmers beats a dozen average programmers every time.
  2. The development team should be able to demonstrate that their code works by the execution of automated tests that generate a report that you can read and understand.
  3. The process should include code reviews, automated tests and continuous integration. 
  4. Words like "agile" and "waterfall" don't mean anything. Really. 
  5. The architecture should be well-defined and understood by the team.
  6. Design patterns matter. No matter what you are doing, chances are that someone has done it before, learned from their mistakes and then did it again. Listen to their advice.
  7. And the most important rule of all: The fewer the moving parts there are, the fewer the errors there will be. Do the simplest thing that works.

Sunday, April 05, 2015

How to Manage Software Development

I write software. I started my career working for a newspaper. Now I work for a book publisher. I even once tried to teach humanities graduate students at Georgetown University how to program in Python.

Like many of my colleagues, I have an English degree. I also have an MBA. I was never formally schooled in software development, aside from a few college courses here and there. Nevertheless, software development has been my trade for most of my adult life. I'm pretty good at it and there's always plenty of work to be had.

I've made a career out of being a software developer for companies that aren't software companies -- at least they haven't been software companies historically. The reality is that for these companies the times have changed and software is becoming increasingly important to their bottom line.

If publishers and media companies want to thrive in the future, then they must become better software developers.

I like working for publishers; I love words. I remember the first time I saw an HTML page load in the Mosaic web browser. It was a simple page, primitive by today's standards, but I remember it very clearly.

The reason I remember that moment is because that's when I realized that computers were more than calculators, more than a tool to be used to perform tedious mathematical chores. I think this was the very first time I vividly understood that computers represented a new means of communication. All at once, I realized that writing software was simply another way of writing. It was a wholly new way of writing and one that has inspired great changes in the way we create and communicate with one another.

When I was an undergrad, being a software developer meant getting a job at a bank. It meant writing reports and calculating sales figures for corporations. But in the mid-nineties, being a software developer suddenly could also mean being a communicator, an expresser of creativity, a writer, an artist.

In addition to writing software, I am also a manager and it is my job to manage software developers. Managing the development of software is more than simply managing developers. If it takes a village to raise a child, then it takes a cross-functional team of people to devise, develop and deploy a successful software application.

The problem is this: industries that have not historically been technology companies often do not have the institutional experience to understand and execute a software development project. And the reality is that there are a lot of people in a lot of companies that are tasked with working on software-related projects who have no interest in writing software. A lot of them don't even like to use a computer. But because software is essential to support their work, they have been forced by circumstances to become participants in the process.

It's kind of like eating Kale. It's good for you, but eating it sucks.

I think non-technical managers shouldn't be forced to learn how to write code, but they do need to learn how to participate in the software development process and they need to know enough to manage in a technology-dependent environment.

Here's the problem that occurs when a non-technical person is tasked with managing some aspect of a technical project: They manage what they understand and what they understand is how much money is being spent and how long the process is taking.

Of course, controlling costs and managing schedules are Good Things. The issue is that cost overruns and missed deadlines aren't the problem. They are symptoms of the problem. In order to be an effective participant in, or an effective manager of, a software development project, you need to care about more than how much it costs and how long it's taking.

I have a lot to say about this, but I want to focus now on something my statistics professor told me: Management is controlling variation.

What he meant by this is that one of the essential requirements of a successful product or service is that every time a customer buys your product or service, they get what they are expecting. In other words, when I order a latte in a Starbucks in Manhattan, it tastes exactly like the latte I order from Starbucks in Seattle.

Quality means consistency.

The idea applies to software development, too.

In the case of software, the goal isn't to write the same software over and over again. That's not the kind of variation I'm talking about. The kind of variation that managers of software development need to worry about is variation in the process of software development.

In the case of Starbucks, the reason the lattes taste the same wherever you are, is because they are made the same way in all of their stores. If I get a job at a Starbucks in Seattle, I'm taught to make a latte the same way that I would be taught to make a latte in New York.

Let's think about it this way: imagine you are the Starbucks management team many years ago and you are trying to train new employees in the art of latte making. The way you go about this is that you hire a few folks who are experienced baristas.  You start by letting them sample a latte that you've made, and then you explain to them that you expect them to be able to make one latte every three minutes. Then you point them in the direction of the espresso makers and tell them to get to work.

What's the likelihood that all of their lattes will taste the same and be of the same quality? Not very good, I'm afraid. The reason why this is true is that you can't expect people from different backgrounds, with different tastes and experiences, to automatically know how to make a latte that tastes exactly like the latte their barista colleagues are making. In fact, the only way to insure that the lattes always taste the same is to insure that each individual barista follows the exact same steps every time they make a latte.

And if you can't make the perfect latte without managing the process by which lattes are made, how can you possibly expect to manage a software development project without doing the same thing?

The key to making a great latte is the same as the key to making great software: you have to manage the process.