regression | petersironwood

Tags

HCI, productivity, programming, regression, software, UX

As I mentioned recently, when I first arrived at IBM Research in the early 1970’s I began to work on Query By Example and other schemes to make it easier for non-programmers to interact productively but flexibly with computers. Some of the lessons learned have to do, not with my own work, but with the work of others.

At that time, some researchers labelled their work as “The Psychology of Programming.” There were many debates — and some studies — about structures, syntax, which language was better than others, etc. There were also lengthy discussions about the process that one should use for software development. Doing any kind of “controlled” experiment on large scale code development is prohibitively expensive. It is rare that a company is willing to have two independent teams build the same piece of software in order to learn which of two methods is “better.”

Of course, one such comparison would likely not prove much. It may be that one of the two teams had a “super-programmer” or an extremely good manager. Perhaps, flu broke out in one of the two teams. You would really need to study many more than two teams to properly and empirically study the impact of language, or syntax, or process. Doing reasonable-sized experiments would be far too costly and impractical. Our lab and others did do laboratory tasks in order to test one syntax variant against another and so on. The problems were generally quite small in order for the study to be practical. So, the applicability to real-world development projects was questionable.

Photo by Christina Morillo on Pexels.com

Walston & Felix (See below), on the other hand, were able to find data on a fair number of real-world projects and rather than try to control for languages, processes, etc., they did a multiple regression analysis based on what languages, methods, etc. real projects used and what the important predictors were of actual productivity.

Personally, I learned two lessons from their study.

Lessons Learned: #1 — Sometimes, when it comes to what matters in the real world, controlled laboratory experiments have to give way to other methods such as studying “natural experiments.” Despite the many issues with trying to interpret such findings, no-one will pay for massive controlled experiments that parametrically vary programming methods, programming languages, etc. while controlling for quality of management, experience, complexity of task, etc. Multiple regression studies and in-depth case studies; ethnographic studies; interviews: all of these can provide useful input.

Lessons Learned: #2 — The impact of the variables that our community of people were looking at in terms of syntax, structure, etc. were dwarfed by the impact of organizational variables. For example, Walston & Felix found that the complexity of the interface between the developers and the customers was extremely important.

DeMarco & Lister (See below) claim that, based on their decades of experience as consultants to the software development process, projects almost never fail for technical reasons; when they do fail, it’s almost always for organizational and management reasons.

That conclusion dovetails with my experience. Many decades later, working for IBM in “knowledge management,” it was amazing how many companies wanted us to “solve” their knowledge management issues by building them a “system” for knowledge sharing.

But…

Management at the company would not provide:

Incentives to share knowledge

Space to share knowledge

Time to share knowledge

Or, commit any personnel to gathering, vetting, organizing, and promoting the knowledge repository.

So — knowledge sharing was something they were simply supposed to do on top of everything else they were doing.

They did not want a computer system, IMHO; they wanted a magic system.

These experiences were part of my motivation for attempting to catalog “best practices” in collaboration and teamwork in the form of a Pattern Language. Christopher Alexander and his colleagues looked at “what worked” in various parts of the world when it came to architecture and city planning. What they did for architecture and city planning, I want to do for collaboration.

Naturally, merely creating a catalog is not sufficient. I need to have people who will read it, understand it, modify and improve it, and then promulgate it via actual use. For now, it’s free. Comments and critiques are always welcome.