Query By Example

08 Tuesday Sep 2026

Posted by petersironwood in AI, design rationale, HCI, leadership, management, psychology, science, Uncategorized, user experience

≈ Leave a comment

Tags

AI, Artificial Intelligence, expertise, HCI, human factors, IBM, leadership, QBE, research, technology, usability, UX, writing

This is part of a series on experiences in my career in Human Computer Interaction and some lessons learned.

I joined IBM Research on the winter solstice of 1973. I had earned a Ph.D. in Experimental Psychology from the University of Michigan and for the previous few years, I had managed a research project at Harvard Medical School on the “Psychology of Aging.” At the time, I was married and had three small children. I mention this because I was funded by so-called “soft money” which basically meant that my salary depended on a research grant. I helped write a renewal of the grant but the decision was “deferred”; that is, it was neither funded nor unfunded. Then, it was deferred again. This meant that if the grant were not funded, I would only have a few weeks to find a new job. That seemed far too short so I began to look other places for a job.

Lessons Learned: #1 If you want continuity of personnel in your laboratory, make sure you have overlapping and multiple grants or other sources of income.

In this case, the grant actually was ultimately approved, but by that time, I had already agreed to join IBM Research. That turned out to be fine, by the way. It was a wonderful place to work.

One of the reasons that I got the job at IBM was that I already knew something about computers. I had taken several computer science courses in grad school along with the needed psych courses. More importantly, our “Psychology of Aging” study was run by a PDP-8 and I had programmed the computer to run our suite of experiments and to do data analyses on the results. I had taken a week-long course at DEC in Maynard, Massachusetts on the assembly language, another week-long course on the machine language, and another week-long course actually tracing the circuitry with a probe and oscilloscope. I felt I “understood” the PDP-8 at a fairly deep level.

At IBM, I did not have that familiar machine. Instead, I was connected to a mainframe via a dumb terminal. The first day at IBM, I got my userid and tried to log on to APL (A programming language I had not used before). I tried following the manual but I could not seem to get logged on. After hours of trying, I finally gave up and went down to the computer room and found someone willing to help. I showed him the logon instructions I was trying to follow and he immediately said, “Oh, yeah, that doesn’t work any more. We changed that months ago. Here’s how you need to do it now.” The manual I had may have looked new, but it was out of date.

Lessons Learned: #2 Manuals can be wrong. These days, most are online. But they can still be wrong.

Lessons Learned: #3 Someone who knows how to do something can save you hours with a few minutes of their time.

Of course, it’s more respectful, efficient, and a better learning experience if you can figure it out on your own. But sometimes you can’t. My stumbling block was not due to an error in logic, or a lack of in-depth knowledge. It was simply that the computer center administrators had changed something arbitrary so that the documentation I was given about how to log on for the first time was no longer accurate.

In order to teach myself APL, I wrote a very small program to “predict” how long I was going to live “based on” some behaviors that I was interested in controlling. My main goal was to learn APL. My secondary goal was to motivate myself, for instance, to exercise more, lose weight, and not drink too much alcohol. I had no intention or pretensions of making this prediction “accurate.” If I had been doing a consulting gig for an insurance company setting life insurance rates, for example, I would have given far more attention to see precisely what the real data were and incorporated many more variables into the regression model.

Here’s a link,

https://www.death-clock.org

by the way, to a more accurate model than the one I used, but it’s still simple to use. Note that my goal was to motivate myself and so I intentionally exaggerated the impact of those behaviors I was trying to change. I had programmed it. I knew how “bogus” the calculation was — nonetheless — here’s the interesting thing though:

Lessons Learned #4: Even an over-simple model that the user knows is over-simple can still motivate change.

At last we come to the actual project I worked on — the usability and learnability of Query By Example. One of my colleagues, Moshe Zloof, invented the language for relational data bases. He had designed the language but not yet implemented it. I did not immediately test the design; first, I sought to understand it. In seeking to understand it in depth, prior to testing it, the two of us had some sense-making discussions. Moshe improved the design; in particular, our discussions uncovered some ambiguities and inconsistencies that were not at all obvious when he simply gave talks about the design. This brings me to the next lesson learned which has proven true in nearly every study of early stage designs that I’ve been involved with over the course of six decades.

Lessons Learned #5: Don’t just accept a surface description of something; understand it as deeply as you can before designing a study.

In this particular case, it was possible for me to understand it in some depth. Relational data bases and second order logic are things I was capable of understanding. If it had been an interface to running a nuclear reactor or using the artificial heart that Moshe had designed earlier in his career, that would have been a much more difficult task for me.

I wanted to understand, not just the “logic” of Query By Example, but also possible contexts of use. For instance, my manager & I visited Burlington, Vermont to talk with IBMer’s who actually used query languages to understand what was happening in chip production lines. At one point, a particular production line that had been producing nearly 100% perfect chips starting having a much higher error rate. Using their query facility, they were quickly able to diagnose the cause of the change which was a supplier of one of the raw materials using a different source. In turn, this meant a slightly different profile of trace impurities in the substrate. Of course, this is only one example, but to me, understanding something in depth means not only understanding its internal logic but also understanding real users, their real tasks, and their context of use.

Photo by Chokniti Khongchum on Pexels.com

I won’t go into all the details of the pencil & paper study or the results. High School students and then college students were taught the basics of the language and then given a simple relational data base and a set of questions stated in English which they had to translate into Query By Example. Briefly, the bottom line was that Query By Example was easy to learn and easy to use. However, there were still questions that people had difficulty with. In analyzing the data and doing some further experiments, the difficulties that people tended to have, stemmed not so much from Query By Example per se, but from what I much later came to call “labelism” — that is, confusing a label with the thing that label refers to.

Here’s a simple example of the type of confusion we saw. In Query By Example (and other query languages) there is usually an OR operator and an AND operator. (These operators can be important for doing advanced queries with search engines as well). If you are interested in getting a list of pets you might adopt and you’re willing to adopt dogs or cats, you might ask for “cat OR dog.” If you only want long-haired cats, you might ask for “cat” AND “long hair.”

English, however, can be tricky.

If you and I (as opposed to you and a query language) are having a conversation, you might say, “I hear there are many pets that need to be adopted.”

I say, “Yes, there are all kinds of pets. There are snakes, dogs, turtles, rabbits, cats…”

You say, “Let me stop you right there. I’m only interested in adopting cats and dogs. Those are the only animals I’d want to adopt.”

See what you said there? You exact words included: “…cats and dog.” If you put “Cats AND dogs” into a query against the data base of available pets, however, you will get the null set (that is, nothing) back. There are no animals who are both cats and dogs! (Though my part Main Coon cats, Luna and Charles Wallace, can play fetch like dogs).

When people were presented with an English statement that included the English word “and” — regardless of the actual syntax and context, some of them had difficulty using the OR operator. If instead, the query in English had set up like this: “Oh, I don’t want reptiles. I’d be happy with adopting a cat or a dog, however” then, they’d have no problem translating it into the OR operator in the query language.

Lessons Learned: #6 Sometimes the difficulty people have in using a product, a service, or a prototype is not due to the interface details but with the structure of the task, their background, and their training.

By analogy, you will not allow me to beat Carlos Alcaraz or Jannik Sinner at tennis by giving me a better tennis racquet! (Although if you gave one of them a toothpick for a tennis racquet, I might have a shot).

That sounds obvious and even absurd, but I promise you, some companies get so greedy that they want you to design a system that allows people who do not understand the task and have minimal background and training to nonetheless be able to perform that task. And, to be fair, it isn’t just the companies who are greedy. They are steered into thinking that they can get away with this absurdity because some outsourcing companies (and more recently, AI companies) tell them they can do it.

One example you may have encountered is having “help desk” personnel who have no understanding of a product go through a script to help you “solve your problem.” Sometimes, it works. But many times it doesn’t. When it does not work, you might not be able to “fix” the system by making the interface to the scripts easier to use for the help desk folks. The problem is much deeper (in some cases). Yes, a really bad interface can make it difficult even for a really knowledgeable and capable person to do the job. But often, even a really great interface cannot always substitute for actual expertise.

——————————————————-

Essays on America: Labelism

“The Psychology of Design”

22 Wednesday Jul 2026

Posted by petersironwood in creativity, design rationale, HCI, management, politics, psychology, science, Uncategorized, user experience

≈ Leave a comment

Tags

AI, creativity, Design, education, HCI, human factors, IBM, leadership, research, technology, UX

“The Psychology of Design”

I worked at IBM, all told, about 28 years. During that time, management put increasing pressure on us to make our work “relevant” to the business. In fact, the pressure was always there, even from the beginning. Over the years, however, we were “encouraged” to shorten evermore the time gap between doing the research and having the results of that research impact the bottom line. This was not an IBM-only phenomenon.

I was a researcher, not a politician, but it seemed to me that at the same time researchers in industrial labs were put under pressure to produce results that could be seen in terms of share price (and therefore payouts to executives in terms of stock options), academia was also experiencing more and more pressure to publish more studies more quickly — and to make sure “intellectual property” was protected to make sure the university could monetize your work. This was about the same time that, at least in America, increasing productivity and the wealth that sprung from that increased productivity stopped being shared between the workers and the owners.

In the late 1970’s, the “Behavioral Sciences” group at IBM Research began to study the “psychology of design.” For the first few months, this was an extremely pleasurable & productive group, due mainly to my colleagues. Over the next few blogs, I’ll focus on some specific techniques and methods that you may find useful in your own work.

In this short recounting though, I want to focus instead on some broader issues relevant to “technology transfer”, “leadership” and “management.” Even if you are or aspire to be an expert in UX or HCI or design, I assure you that these broader issues will impact you, your work and your career. I wouldn’t suggest becoming obsessed with them, but being aware of their potential impact could help you in your own work and career.

It is telling that, almost invariably, whenever I told someone inside IBM (or, for that matter, outside IBM) that I was studying the “psychology of design,” people responded by asking, “the design of what?” So, I would explain that we were interested in the generic processes of design and how to improve them. I would explain that we were interested in understanding, predicting, and controlling these processes to enable them to be more effective. I would explain that we could apply these findings to any kind of design: software design, hardware design, organizational design, and (see last post about IBM) communication design. I would explain that design was a quintessentially human activity. I would also explain that design was an incredibly leveraged activity to improve.

Looking back on it, I still think all these things are true. I also see that I missed the “signal” people were giving me that, while I thought of design as something that could be studied as a process, that most people did not think of it that way. To them, it was never the “psychology of design,” but only the design of something.

Don’t get me wrong. I agree that somewhat different skills are involved in designing a great advertising campaign, a great building, and a great application. I agree that different communities of practice treat various common issues differently. I still think it’s worth studying commonalities across domains. For one thing, we may find an excellent way of generating ideas, say, that the advertising community of practice uses that neither architects nor applications developers had ever tried. Or, vice versa.

My own academic background was in “Experimental Psychology.” We were forever doing experiments that we believed were about psychological processes that were thought to be invariant regardless of the domain. It was an axiom of our whole enterprise that studying memory for any one thing also shed light on how we remember every other thing. Similar studies looked at decision making or problem solving or multi-tasking. We came to understand that there were some interesting exceptions to being able to separate content from process. For instance, it is much easier to multi-task a spatial task and a verbal task than it is to multi-task two independent spatial tasks or two independent verbal tasks.

In our Behavioral Sciences lab, we used a spectrum of techniques to study “design” ranging from laboratory studies of toy problems, to observing people doing real-world design problems while thinking aloud. After about 3-4 months of very productive work, we were told that we had to make our work relevant to software development. That one domain should be the focus of our work. We were told that this command came from higher-ups in IBM. That might have been true, or perhaps partly true.

It might also be relevant that someone in our management chain might have been the recipient of a grant from ONR which was specifically focused on software development. So far as I can tell, nothing had been done on that grant. So, our past, present, and future work could have been co-opted to be “results” done under the auspices of the ONR grant.

In any case, regardless of the “reasons,” the group began to focus specifically on software design. In one study, we used IBM software experts as subjects. Each person was given information that was geared toward a specific transformation that occurred in software development. One person was presented with the description of a “situation” that included a number of “issues” and they were asked to write a requirements document. In real life, I would hope that this would be done in a dialogue (and, indeed, in other studies, we recorded such dialogues). Absent such dialogues, what we found was that different software experts — all from IBM research — and all given the same documentation about a set of problems generated vastly different problem statements and overall approaches.

In other parts of the study, other experts were variously given requirements documents and asked to do an overall, high level system design, or given a high level design and asked to design an algorithm, or given an algorithm and asked to code a section. There was always diversity but the initial phases showed the greatest diversity of behavior. The initial stage is also the one that can cause the most expensive errors. If you begin with a faulty set of requirements — a misreading about how to even go about the problem — then, the overall project is almost certain to incur schedule slip, cost overruns, or outright failure.

While the vital importance of the initial stages of design is true in software development, I would argue that it is likely also true for advertising campaigns, building designs — and even true for the design of research programs. We designed our research agenda under the assumption that we had a long time; that we were studying design processes independently of specific communities of practice or the nature of the problems people were attempting to address. We assumed that there was no “hidden agenda.” Although we believed we would eventually need to show some relevance to IBM business, we had no idea, when we began, that only relevance to software design would be “counted.”

—————-

Some of our studies on the “Psychology of Design.”

Carroll, J. and Thomas, J.C. (1982). Metaphor and the cognitive representation of computer systems. IEEE Transactions on Man, Systems, and Cybernetics., SMC-12 (2), pp. 107-116.

Thomas, J.C. and Carroll, J. (1981). Human factors in communication. IBM Systems Journal, 20 (2), pp. 237-263.

Thomas, J.C. (1980). The computer as an active communication medium. Invited paper, Association for Computational Linguistics, Philadelphia, June 1980. Proceedings of the 18th Annual Meeting of the Association for Computational Linguistics., pp. 83-86.

Malhotra, A., Thomas, J.C. and Miller, L. (1980). Cognitive processes in design. International Journal of Man-Machine Studies, 12, pp. 119-140.

Carroll, J., Thomas, J.C. and Malhotra, A. (1980). Presentation and representation in design problem solving. British Journal of Psychology/,71 (1), pp. 143-155.

Carroll, J., Thomas, J.C. and Malhotra, A. (1979). A clinical-experimental analysis of design problem solving. Design Studies, 1 (2), pp. 84-92.

Thomas, J.C. (1978). A design-interpretation analysis of natural English. International Journal of Man-Machine Studies, 10, pp. 651-668.

Thomas, J.C. and Carroll, J. (1978). The psychological study of design. Design Studies, 1 (1), pp. 5-11.

Miller, L.A. and Thomas, J.C. (1977). Behavioral issues in the use of interactive systems: Part I. General issues. International Journal of Man-Machine Studies, 9 (5), pp. 509-536.

——————————

Blog posts about the importance of solving the “right” problem.

The Doorbell’s Ringing. Can you get it?

https://petersironwood.com/2021/01/13/reframing-the-problem-paperwork-working-paper/

Problem Framing. Good Point.

https://petersironwood.com/2021/01/16/i-say-hello-you-say-what-city-please/

Problem formulation: Who knows what.

How to frame your own hamster wheel.

The slow seeming snapping turtle.

Author Page on Amazon.

Design – Interpretation Model of Communication

16 Thursday Jul 2026

Posted by petersironwood in AI, creativity, design rationale, HCI, leadership, politics, psychology, Uncategorized, user experience

≈ 1 Comment

Tags

communication, deception, experiment, HCI, IBM, life, media, psychology, relationships, truth, UX, writing

In my early days at IBM Research (1970’s), we were focused on trying to develop, test, or at least conceive of ways that a larger proportion of people would be able to use computers. One of the major ways of thinking about this was to use natural language communication as a model. After all, it was reasoned, people were able to communicate with each other using natural language. This meant that it was possible, at least in principle. Moreover, most people had considerable practice communicating using natural language.

One popular way of looking at natural language (especially among engineers & computer scientists) was essentially an “Encoding – Decoding” model. I have something in my head that I wish to communicate to you. So, I “encode” my mental model, procedure, fact, etc. into language. I transmit that language to you. Then, you “decode” what I said into your internal language and — voila! — if all goes well, you construct something in your head that is much like what is in my head. Problem solved.

Of course, people who wrote about communication from this standpoint acknowledged that it didn’t always work. For instance, as speaker, I might do a bad job of “encoding” my knowledge. Or, I might do a good job of encoding, but the “transmission” was bad; e.g., static, gaps, noise, etc. might distort the signal. And, you might do a bad job of decoding. It’s an appealing model and helped engineers and computer scientists make advances in “communication theory” and helped make practical improvements in coding and so on.

As a general theory of how humans communicate, however, that notion is vastly over-simplified. I argued then that a better way of looking at human communication was as a design-interpretation process, not as an encoding-decoding process. One of the examples that pointed this out was a simple observation by Don Norman. Suppose someone comes up to you and asks, “Where is the Empire State Building?” You will normally give a quite different answer depending on whether the two of you are in Rome, Long Island, or Manhattan. In Rome, you might say, “It’s in America.” Or, you might say, “It’s in New York City.” If you are on Long Island, you might well say, “It’s in Manhattan.” If you are already in Manhattan, you might say, “Fifth Avenue, between 33rd and 34th.”

Building on Don Norman’s original example, but based on your own experience, you can easily see that it isn’t only the geographical relationships that influence your answer. If you were originally from Boston, now on your own in Rome, struggling with Italian and homesick and someone came up to you and asked that question in American English with a Boston Accent, your response might be: “Are you joking? But how did you know I was an American. My name’s … “

On the other hand, if you’re a 13-year old boy in Manhattan — one with a mean streak — and someone asks you this question in broken English and they’re looking around like they are totally lost, you might say, “Oh, no problem. Just follow 8th Avenue, all the way north up to 133rd. It’s right there. You can’t miss it.” (Note to potential foreign visitors, most kids in Manhattan would not intentionally mislead you. But they point is, someone could. They are not engaging some automatic encoding process that takes their knowledge and translates into English. Absurd!

You design every communication. I think that’s a much more useful way to conceive of communicating. Yes, of course, there are occasions when your “design” behavior is extremely rudimentary and seems almost automatic. It isn’t though. It just seems that way. Let’s go back to our question-asking example. Suppose you work at an information booth in New York City. People ask you this same question day after day, year after year. You’re seemingly giving the answer without any attention whatsoever. Suppose someone asks you the question, but with a preface. “Look here, chap! I’ve got a gun! And if you give me the same stupid answer you’ve given me every time before, I’ll shoot your bloody brains out!” You are going to modify your answer. It only seemed as though it was automatic.

When you design your answer you take into account at least these things: some knowledge that you communication about, the current context (which itself has hundreds of potentially important variables), a model of the person you’re creating this communication for, a set of goals that you are trying to achieve (e.g., get them safely to their goal, mislead them, entertain them, entertain yourself, entertain the people around you, demonstrate your expertise, practice your diction, etc.). The process is inherently creative. In many circumstances (writing, playing, exploring, discovering, partying), you can choose how creative you want to make it. In other cases, circumstances constrain you more (though likely not so much as you think they do).

Many readers think this is a classic example of a straw man argument. “No-one believes communication is a coding-decoding process.”

Well, I beg to differ. I worked for relatively well-managed companies. I’ve talked to many other people who have worked in different well-managed companies. We’ve all seen or heard requests like this: “I need a paragraph (or a slide or a foil) on speech recognition. Thanks.”

What??

Who’s the audience? Are they scientists, investors, customers, our management? How much do they already know? What are your goals? What other things are you going to talk about with them? The people who have left me such messages were all smart people. And, providing the necessary info would have only taken a minute or two. But it would have substantially improved the outcome. It’s not a straw man argument.

Sit-com plots often hinge on the characters doing poorly at designing and/or interpreting communications. A show based on encoding-decoding? No. What could be funny — indeed what often is shown in comedy — are people failing to do good design and in the extreme case, that can arise by having an actual robot as a character or someone who behaves like one.

People also interpret what was said in terms of their goals, the context, what they believe about your goals and capacity, what they already know, and so on. And, even though this may seem obvious, millions of people believe what advertisers or politicians say without questioning their motives, double-checking with other sources, or even looking for internal inconsistencies in what is being touted as true. In other cases though, the same people will not believe anything the “other side” says no matter what. Just as one can do faulty design, one can also do faulty interpretation.

In any case, I decided that it would be good to “show” in a controlled laboratory setting that the Encoding-Decoding model was woefully inadequate. So, I brought in “subjects” to work in pairs at a simple task about communicating Venn diagram relationships. The “designer” had a Venn diagram in front of them. “The “interpreter” was supposed to draw a Venn diagram. The “designer” was constrained to say something true and relevant. In addition to a “base” pay, the “interpreter” subjects would be given a bonus according to how many relationships matched those of the “designer.” The designer’s bonus depended on condition. In the “cooperation” condition, their payoff would also, like the interpreter’s, be determined by how much agreement was shown in the two diagrams. In the “competition” condition, the designer’s bonus depended on how different the two diagrams were.

Photo by August de Richelieu on Pexels.com

I ran about half the number of subjects I had planned to run when the experiment was ended by corporate lawyers.

What?

IBM had no unions at that time. And, they didn’t want any unions. One of their policies, which they believed would help them prevent the formation of unions was that they never paid their workers for piece-work. Apparently, somehow, IBM CHQ had gotten wind of my experiment. People were being paid different amounts, based (partly) on their performance. They couldn’t have this! People might think were paying people for piece-work!

It hardly needs to said, I suppose, that IBM definitely tried to pay for performance. This was true in sales, research, development, HR, management, and so on. No-one in IBM would argue that your pay shouldn’t be related to your performance. That was exactly — in one way of describing it — was going on here. By the way, these were not IBM employees and each subject only “worked” for about an hour.

Basically, regardless of how irrelevant this experimental set-up might have been to the genuine concern of unions not to pay people in an insanely aggressive and ever-changing piece-work scheme, the lawyers were concerned that it would be somehow misrepresented to workers or in the press and used as evidence that IBM should unionize. In a way, the lawyers were proving the point of the experiment in their own real-life behavior even as they insisted that the experiment needed to be shut down.

Lessons Learned: #1 Corporate lawyers are not only concerned about what you actually do or how you represent your work; they are also worried about how someone might misrepresent your work.

Lessons Learned: #2 Even when constrained to say something true and relevant, ordinary people are quite capable of misleading someone else when it’s to their benefit and considered okay to do.

It is this second aspect of the experiment that I myself felt to be “edgy” at the time. Sure, people can mislead, but I was providing a context in which they were being encouraged to mislead. Was that ethical? Obviously, I thought it was at the time. On reflection, I still think it’s okay, but I’m glad that there are now review boards to look at “studies” and give a less biased opinion than the person who designed the study would do.

I view the overall context of doing the study as positive. As adults, these people all already knew how to mislead. I was letting them, and many other people, know that we know you know how to mislead and we’ll be on the lookout for it.

What do other people think about studies wherein the experimenter encourages one person to deceive another?

2026 Update: I’ve spend a fair amount of time recently “chatting” with ChatGPT and with Claude. It’s clear that the implementers of these systems (or, at least some of them) were quite familiar with the idea of designing and interpreting text. Claude, in particular, seems like a “nice guy” and an honest one at that. It acknowledges some of its own limitations and often praises the questioner. This shouldn’t come as a complete shock. After all, good sales people and advertisers have relied for centuries on “Design and Interpretation.” Nonetheless, I sometimes wonder whether AI systems might be more ethically employed if they were operating on principles closer to “Coding and Decoding.” What do you think?

———————-

References published literature that describes some of the research that was done around that time.

Malhotra, A., Thomas, J.C. and Miller, L. (1980). Cognitive processes in design. International Journal of Man-Machine Studies, 12, pp. 119-140.

Carroll, J., Thomas, J.C. and Malhotra, A. (1980). Presentation and representation in design problem solving. British Journal of Psychology/,71 (1), pp. 143-155.

Carroll, J., Thomas, J.C. and Malhotra, A. (1979). A clinical-experimental analysis of design problem solving. Design Studies, 1 (2), pp. 84-92.

Thomas, J.C. (1978). A design-interpretation analysis of natural English. International Journal of Man-Machine Studies, 10, pp. 651-668.

Thomas, J.C. and Carroll, J. (1978). The psychological study of design. Design Studies, 1 (1), pp. 5-11.

———————

Other essays that touch on communication.

Freedom of Speech is not a License to Kill

Ohayogozaimasu

The Sound of One Hand Clasping

Fool Me

Claude the Radioman

Know What?

The Story of Story, Part 1

The Temperature Gauge

The Destruction of Natural Intelligence

A Little is not a Lot

Try the Truth

Stoned Soup

Turing’s Nightmares

“Wizard of Oz”

15 Wednesday Jul 2026

Posted by petersironwood in AI, apocalypse, design rationale, HCI, leadership, management, politics, psychology, The Singularity, Uncategorized, user experience

≈ Leave a comment

Tags

AI, HCI, IBM, research, technology, usability, UX, Wizard of Oz, writing

(Some Lessons Learned from studies in Human-Computer Interaction/User Experience conducted at IBM Research in the mid-70’s.)

Wizard of Oz

One of the studies I conducted at IBM Research in the mid 1970’s was part of an effort to do “Automatic Programming” — a department under Pat Goldberg. The first level manager I worked with was Irving Wladawsky (later Irving Wladawsky-Berger). His group wanted to develop a system that would allow the owner/operator of a small business to type requirements into a computer in English (or something English-like) and have the system itself produce RPG code to run the business so described.

The underlying motivation from an IBM business perspective was that many small businesses could well afford a computer to do inventory, fulfill orders, etc. but they couldn’t afford to hire programmers to create such a system from scratch. The small business owner in the mid-1970’s did not program! Yet, for the most part, they understood how their business worked. The notion was that a natural language understanding and generation program could dialogue with the user/owner and through that process, understand their “business rules.” No costly programmers needed!

An interesting side note: at that time, we were told that IBM corporate forbade us to use the terms “Artificial Intelligence” or “Robotics” to describe our work because some PR firm had determined that these terms were too scary for the general public. So, IBM had research in “mechanical assembly” but not “robotics.” We had work in character recognition, speech recognition, handwriting recognition, automatic program generation, and compiler optimization. But no work in “Artificial Intelligence.” (Wink, wink, nod, nod).

Labelism: Confusing a thing with the label for that thing.

Another interesting side note: I worked at IBM Research for a dozen years; started an AI lab at NYNEX where I worked another 13 years; came back to IBM Research and several years later found myself working on the same problem! We were still trying to make a system to allow small businesses to generate their code automatically. In my second iteration, rather than using natural language, we were trying to make the specification of business rules in a graph language that was intuitive enough for business owners. This was a different approach, but trying to address the same underlying desire: to bring computing to small business without incurring the heavy costs of programming and maintenance.

Let’s return to iteration one — the natural language approach @ 1975. Well, one issue was that no-one had a natural language program that even approximated being able to do the job. So…how to study people’s interaction with a system that doesn’t exist?

We used an approach that my colleague Jeff Kelly called the “Wizard of Oz” technique; viz., use a human being (in this case, me) to simulate how the system might work and record people’s behavior. In this way, we could discover many of the issues that such a natural language programming system would have to deal with. I had already had plenty of experience interacting with a computer; and I had acting experience. I could “play the part” of a computer fairly well as I typed in my questions and answers.

(Description of “The Wizard of Oz” technique).

IBM Research in Yorktown had roughly a thousand people including not only scientists, programmers, and engineers but also a number of business people (who did not know how to program). I knew some of them from playing tennis and table tennis and we used those folks as initial subjects. What did I find? Good news and bad news.

Dealing with natural language is tricky for many reasons. One of those reasons is that English, including the English that people normally use to describe their business, is filled with words that have multiple meanings; e.g., “file”, “run”, “program”, “object”, “table”, etc. But here is the good news: although it’s true that many English words have many meanings, when these business people described business procedures, almost all of the lexical ambiguity vanished! The program to understand business English would not have to distinguish between a business file and a nail file; it wouldn’t have to worry about distinguishing a run in baseball or a run in stockings from a run of the payroll program; it wouldn’t have to distinguish between the table in a relational data base and the table in your dining room. The domain would mainly constrain! That’s the good news.

The bad news was dialogue management. How can the machine recognize a misunderstanding and how can it correct it? To make matters worse, while business people were fairly consistent in the way they described how their business ran, they were not consistent in how they talked about the communication. If a human being senses that another one is misunderstanding, then, depending on context they might: raise their eyebrows, say “Huh?”, “Come again?”, “What?” “I think I lost you.” “WTF?” “Are you kidding?”, “We’re on different wavelengths,” “I don’t get it.” “But…wait.”

Sometimes, these are referred to as “meta-comments.” Here’s a simple example that took place in the study.

One of the business people told me about various discounts. I had assumed (playing the part of the computer) that he was talking about discounts for items that were being discounted due to inventory management. I recorded all the various percentages and so on. Then, he said, “Now, we also give discounts for various items.”

At that time, most natural language systems of that era simply ignored words like “now” and “also” in this context. Stepping out of my role as a “computer system” and thinking about from the perspective of a human conversational partner though, these words are crucial! What it signals is a change in topic. In the larger context of our conversation, it shows that everything that had just been said, which I thought had been about item discounts, was not about item discounts!

This is just one example, but there were many more. In my more recent experience interacting with various computer dialogue systems, being able to recognize the signals of miscommunication and being able to repair misunderstandings is still not very well-handled more than four decades later.

I’d be interested in any pointers you have to a system that you think deals with meta-communication in a natural and robust manner. I do not think that it is beyond the pale of possibility. The general categories of the ways that people misunderstand each other is not infinite. John Anderson developed excellent tutoring systems for LISP and geometry and those systems worked something like human tutors in that, the tutor inferred the mental model of an individual student and focused instruction on correcting any misconceptions. My intuition is that a generic system built with equal complexity could deal with most of the issues as well as the average human being deals with them; i.e., imperfectly.

—————————————

Lessons Learned: #1 You can test aspects of a system even before it’s built or even completely defined. One method that has been used many times: “Wizard of Oz.”

Lessons Learned #2: Language used by professionals to talk about their domain is much more constrained in terms of lexical ambiguity than is language when considered by all native speakers.

Lessons Learned #3: People in “our culture” (i.e., US business culture) do not have an agreed upon and consistent vocabulary for talking about communication nor a consistent process for dealing with them.

Lessons Learned #4: Speaking of communication errors, I don’t recall why, but it was about this time, that I realized that my notion about how research results would be transferred to other parts of IBM was a complete and utter fantasy. I hadn’t articulated it, but it was basically that I would do research, write the results up for publication in scientific journals for an academic audience and publish Research Reports which would be eagerly consumed by anyone who needed to know. I’m not proud of this. LOL. But that’s really kind of how I viewed it. And, then, after a few years, I realized that it really mainly came about through relationships. That was something that people had been showing me all my life, but which I don’t think anyone ever stated it explicitly enough.

Update for 2026: While the four “Lessons Learned” above still seem apropos, some of today’s chatbots such as ChatGPT and Claude are much better at dealing with connected conversations with humans that were the systems of the twentieth century. Not only that, people are now describing systems in “natural language” and the computer is writing code. That’s the “good news” I suppose.

The bad news? While my impression of most of the AI researchers of the twentieth century is that they were largely motivated by intellectual challenges and a desire to help make people more productive, I get the impression that most of the AI work of today is controlled by people who are already astoundingly wealthy and want to become even wealthier. And, as UN-laudable as we may view that goal, the wealth is only a means to an end and that end is the total enslavement of most of humanity. Even being a trillionaire isn’t enough.

That’s not to say that all the workers advancing AI have those nefarious aims just because those in charge do, but as the surveillance state becomes more pervasive, their private motivations could have less influence over the outcomes.

———————————-

Author Page on Amazon

The Myths of the Veritas (an exploration of leadership & ethics in free, no ads fiction)

Index to a Pattern Language for Collaboration and Teamwork

Experiences in Human-Computer Interaction

Post on “The Story of Story”

The After Times

After All

When Greed is the only Creed

Destroying Natural Intelligence

“Turing’s Nightmares” comprises 23 Sci-Fi short stories that explore the implications and ethics of AI

Here is an example chapter from “Turing’s Nightmares” that explores a possible dilemma of working in AI.

To Be or Not To Be

The Story of Story: Part 3

17 Saturday Jan 2026

Posted by petersironwood in creativity, essay, HCI, psychology, story, Uncategorized

≈ Leave a comment

Tags

books, education, fiction, HCI, knowledge, leadership, learning, life, management, sense_making, story, Storytelling, thinking, truth, UX, writing

The Story of Story: Part 3 – Good Story, Well Told.

Often in my English classes, (and yours?) we talked about the mechanisms of writing: spelling, grammar, word usage, punctuation, paragraph construction, metaphor, rhythm, and rhyme scheme, for instance. We talked very little about how to tell a story well. And we talked zero about what makes for a good story.

In the last article, I described some guidelines for soliciting stories from users and other stakeholders. From these, one may gain insight into potential problems that a product or service might solve, ameliorate, bypass, or avoid. Later, I will describe more about how stories may be used in the design and development process. Before getting into that, however, I want to describe more about what makes for a good story. In the following articles, I will also suggest ways to make the story well told.

What Makes for a Good Story?

You might find it helpful to write down a short list of 5-10 novels, short stories, movies, or TV shows that you really liked. It doesn’t have to be your all time ten best; just something good that springs to mind. Then put that list aside. Read through the criteria I propose and then check back after you’re done reading to see whether or not most of these criteria were met. I’m betting that they mostly were met.

fullsizeoutput_139c

The Story Cube.

Imagine a cube of some really nice material that you like; e.g., polished wood, lead ore, malachite, silver. This cube has three dimensions: height, width, and depth. It must have all three dimensions. In the case of a story, there are also three dimensions in this sense: Plot, Setting, and Character. If a story lacks any of these three, it will be “flat” (not so interesting). For example, if you spent time working in a large company or government agency, you were probably given training materials about how you’re not supposed to do unethical things like steal from your company. They may have provided you with scenarios and asked what you would do or what was the “right” response. These stories tend to have people in situations making decisions. The problem with these stories is that, in order for them to be “efficient”, they spend almost zero time on character development. “Joe wants to impress his boss and make his quota for the fourth quarter so he puts down as sold this-quarter things he is sure he will sell early in January. After all, he rationalizes, calendars are arbitrary.” Of course, the answer is no Joe should not be lying on his sales report. But we really don’t know much about Joe. We don’t know enough about him to really care much about him. Of course, he shouldn’t lie. If he does, it’s pretty hard to feel anything but contempt for Joe. It should have been obvious to him that he shouldn’t lie on a sales report and if he does lie, he should be fired. Good riddance. Let’s replace Joe with someone who follows the rules.

OLYMPUS DIGITAL CAMERA

This story is so flat that it seems to me that the story is constructed, not so much to really educate, but more to prove that you were shown that it’s wrong to lie on sales forms so that, should the court case arise, you will not be to argue effectively that it was a mere technicality that you didn’t know about. If you really wanted to change someone’s mind about what was right, knowing about Joe’s character could make you empathize much more. Maybe he came from a Mafia-type crime family and no-one would bat any eye about lying on a sales report. They would expect him to lie on the report. Maybe even now, he is looked down upon by everyone else in his family for being such a chump and working for “the man” instead of being “the man.”

OLYMPUS DIGITAL CAMERA

Or, perhaps Joe just found out that his wife has serious cancer and is understandably but severely depressed. He desperately wants to bring her some good news. If we reveal, not only what situation Joe is in but also, how he sees that situation, how he feels about it and what conflicts he faces, we will begin to have real empathy for Joe. His choices become real, rather than predetermined.

TV commercials, like corporate training videos, are typically pretty flat too. But in some cases, the ad agency has gone out of their way to introduce you to some character that is recognizable and re-appears in commercial after commercial. Each time, just a little bit of character is revealed and eventually you find yourself watching the commercial largely because you start to care about the character. In a similar way, one might be able to make the corporate training stories more intriguing & educational if there were a cast of characters that persisted over time.

fullsizeoutput_1308

Two Paths Diverged in a Yellow Wood…

Typically (but not invariably) the author knows how the story will turn out before he starts writing. But for the reader (or viewer), it is not at all obvious how the story will turn out. For compelling stories, the reader must be convinced to “play along with” the uncertainty of the outcome even if they are sure ‘the good guys will win.’ In good stories, bad things happen to the protagonist, but he or she is not a cork tossed on the ocean waves. The protagonist must want something; they must have a goal that is overwhelmingly important to them. They must react to changing circumstances, overcome the obstacles that are thrown at them. Characters are engaged in battles! Battles test them. If winning the battles is easy or inevitable, the character isn’t someone we can really relate to.

catman

Kryptonite

Superman is basically super-human and invulnerable! But watching someone who is invulnerable and has super-powers win battle after battle is boring. Superman has to have weaknesses. To make it more interesting and allow for more plot variation, he actually had three original weaknesses: kryptonite, friends, secret identity. In one episode, someone will have some kryptonite while in the next, someone will kidnap one of his friends. Recent movies have added a fourth weakness: other super-human and invulnerable beings.

Whatever the story, your character must have weaknesses. Otherwise, no-one will “believe” the character and you as the writer will be stymied when you try to develop an interesting plot. The weaknesses can be physical, moral, social, intellectual, situational, and so on. But they should not be merely irrelevant weaknesses. Imagine a story where Sue is the main character. She’s tone deaf. She’s also brilliant, hard-working, imaginative, driven to succeed. And, indeed, she becomes a very successful trial lawyer. Eventually, she is made partner. OK. Isn’t this exactly what we’d expect to happen? What does being tone deaf have to do with anything?

fullsizeoutput_129c

Imagine instead, that Sue was inspired at the age of four when she went to the opera. It was her life-long dream to become an opera singer. Indeed, she was blessed with a beautiful voice. She was also brilliant, hard-working, imaginative and driven to succeed. Unfortunately, she was tone deaf. Now, the weakness becomes interesting. Perhaps she will fail and kill herself. Perhaps she will fail but find another goal that is even more important to her and succeed at that. Perhaps she will fail time after time but eventually develop a career as an improvisational opera singer. She will ask people in the audience to name five things and then and there, she will create a beautiful aria that weaves a tale of some considerable interest about the five things. No-one knows that she is singing out of tune because she is composing on the spot.

The more improbable the odds and the more horrendous the journey, the more challenge you give yourself to make it work! Blind at birth but wants to be an artist? Surely, that’s just stupid. It’s impossible. But is it? What if feedback were provided in such a way that it influenced her to make unique and beautiful paintings? What if genetic engineering allows her to grow new neural pathways? What if she can be equipped with artificial eyes? If it’s fiction, a magic spell can do the trick. Even if your ultimate goal is a real product for the real world, imagining a magical solution may lead you to a new (and real) path, previously hidden by your own expectations.

It is easy for a writer to identify with their hero. And that is potentially quite a problem. After all, if you were superman, you sure as heck would not go out of your way to go near kryptonite. You’d quite sensibly stay away from the stuff! But if you are writing about superman, you need to get him near the deadly stuff every third or fourth episode! The “weaknesses” in the character generate interest. The failures, injuries, betrayals, and conflicts of your protagonist provide materiel that allows you to architect a more interesting plot.

A Garden of Delights, Flashy Sights, or Sword Fights?

Three dimensions of story is a weak metaphor only. The three dimensions of a cube can be manipulated independently. This is not generally true for the three dimensions of story. The character makes a decision, the decision determines the next step of the plot. That will influence the setting for the next scene. In addition, the actions of the protagonist may also change state of the underlying and cross-cutting conflicts.

Imagine:

two rival gangs fighting for urban turf and maybe sex,

two gardeners in a fierce competition for sex with the town’s most eligible “catch” as well as for the blue ribbon prize for best garden,

two rival secret agents vying for victory and maybe sex,

two life long friends now vying for #1 in their Harvard Law class, and maybe sex.

The structure of the underlying plot might look quite similar, but the specifics will depend a lot on how the character is developing. If they develop from ego-centric to altruistic, then they will tend to make different decisions near the beginning than near the end of the story. In addition, the setting will have to be consistently portrayed.

The four descriptions above would most naturally lead to a lot of the setting for the stories respectively in urban settings, garden settings, foreign settings & dangerous situations, mainly Law School and campus settings. Of course, you could violate expectations in a way that increases interest. Imagine that rather than have another garden scene–

The rival gardeners arrive at an urban parking lot dressed in expensive gowns, fully jeweled in their finest, both fully knowing that they will win first prize (but secretly fearing that they might not). These life-long friends now exchange icy greetings, make back-handed compliments about each other’s appearances. The verbal exchange escalates. Precisely because they know each other so well, they know exactly how the other person’s escalator functions. Soon, they are rolling around on the parking lot in their fancy gear; ruining each other’s clothes and hairdos. At this point, they hear in the distance, the loudspeaker and the chairman about to announce the Blue Ribbon Winner! In their trashed and ripped clothing, they sneak in together to hear the awards, hanging out together in the shadows so as not to be seen in their tattered clothes. “And the blue ribbon goes to” {drumroll}:

someone else entirely.

At this, the two life long friends look at each other, laugh uproariously, hug each other, and then become even more intimate friends than they were before their fight in the urban parking lot.

The fact that there are “expected” relations among various dimensions of story is wonderful. For every such expectation, you can decide to follow, bend, or break that expectation. The more expectations people develop, the greater the number of variations for creative exploration. One valid reason for the choice of setting is really where you want to spend your time. That goes for an author — but it also goes for any designer or business person or User Experience expert. What kind of setting do you want to be in? What kind of customers do you want to serve? Do you really want to make their life better or just get them to buy more product? What sorts of application areas are really cool to you? Of course, I understand people need to eat and often there is a conflict within us all about what to do. That’s what a good story is really about.

The reason that stories resonate is that, regardless of setting, people face the same kind of dilemmas. We all do. And, how we handle those dilemmas? In life, as in story,

character is revealed by choices under pressure…

——————————————

Author Page on Amazon

Dream Planet on Barnes and Noble

The Impossible

Donnie Boy gets a hamster

What could be better? A Horror Story

All that Glitters is not Gold

The Forgotten Field

Choosing the Script

The Story of Story, Part 2

16 Friday Jan 2026

Posted by petersironwood in creativity, HCI, psychology, story, Uncategorized

≈ Leave a comment

Tags

AI, books, communication, education, fiction, HCI, interview, knowledge, narrative, needs, psychology, story, truth, user experience, UX, wants, writing

Introduction:

This is the second in a series about using stories and storytelling in the design, development, and the deployment of products and services. In each post, I will weave in some advice about what makes for a good story as well as how to use stories. In this first case, the emphasis is on using stories to help uncover customer needs and wants. Needs and wants are not quite the same thing. For an extremely worthwhile discussion on the difference, check out this classic article by George Furnas.

We Human Beings are not just Information Processors; we are also Energy Processors.

I had just attended a conference on “knowledge management” co-sponsored by IBM consultants and IBM Research. On the plane ride back, after finishing the crossword, I turned the page to find a full page color ad by an IBM competitor that proclaimed: “Knowledge Management is simply [sic] providing the right information to the right person at the right time.” Color me skeptical, I thought. It isn’t simple to do those things. Beyond that, the formulation seemed simplistic even in its formulation.

The image of one of my undergraduate professors flashed into my brain. Professor MacCaw, (as we will call him), taught advanced German, a language which he had learned in a Russian prison camp, which might explain his approach to testing. At semester’s end, he asked, “Who in class wants A?!”

All two dozen of us raised our hands, of course. At this point, he proceeded to — there is no other word — attack one of the students in the class who had had four years of German in high school and had also lived in Germany for two years. The contents of his questions were not really that difficult, but the manner in which he demanded the answers was horrid. He would ask, for instance, “In first story, main character went where?” (He would always ask the questions in English).

And she would begin to answer (necessarily in German), “Er geht…” And after a couple words were out of her mouth, he would scream, “Please to conjugate!” This meant that she would have to think back to the last verb she uttered and then conjugate it. “Ich gehe, Sie gehen, …” Then, he would interrupt again and scream a completely different and unrelated question in English. She would begin to answer; he would interrupt after she uttered only a few words: “Please to decline!” This meant, that she would have to give the various forms of the last noun she spoke according to the case. But once again, she could not finish but only begin declining the noun when he would once again interrupt. After 40 minutes, she was in tears and he looked menacingly around the room and asked, “NOW! Who in class STILL wants A?”

I have zero desire to go hang gliding or sky diving. But when it comes to the danger of mere social humiliation, I say, “Screw it. Been there. Done that.” I was one of only two of the remaining students who raised their hand. This act won me the next turn on the chopping block. He proceeded the same whip-saw questioning fest with me. The two-period class was almost over when he finished with me and began questioning “Mr. Lepke.” The bell rang and everyone else in the class left. Later that evening, I chanced to see Professor MacCaw in the Student Union. He walked up to me, eyes blazing. “Ha! I had Mr. Lepke after class for two hours! Finally, he said to me, ‘No, No, Dr. MacCaw, no more, I beg you. No more!’”

This oral exam was difficult (even with my “screw it” attitude). It was much “harder” than my dissertation defense, for example. Again, it was not the information requested but the manner of questioning that made it difficult. People are not emotionless robots, as it turns out.

The next semester, not surprisingly, only about half the class returned. One day in class, as Dr. MacCaw began one of his lengthy digressions on Eastern European history, he stopped himself in mid-sentence to say, “What is THIS!? Someone is passing notes in my class! I will take note and read in front of entire class!” He snatched the note, unfolded it, and indeed read the note in his loud ringing voice: “Doctor MacCaw: your zipper is down.” And, indeed it was. He had meant to humiliate someone in front of the entire class — and he had succeeded. He had the necessary information delivered at the right time to the right person, but — thanks to his own actions — it had not been done in the best manner — at least not the best manner for him.

Human beings are not just information processors. We are living things and as such, the emotions, the vibes, the manner, the intensity of presentation — these are all vital to how we will react at the time and also how we will feel about the people involved and what we will recall years later. And this fact also means that the atmosphere you create when you interact with various stakeholders will vastly impact the quality of the insights and stories that you receive. If you really care about the people and are really committed to doing something to making people’s lives better; if you are truly open to hear and take in something unexpected or even disruptive to the project; and if you allow your informant to feel that truth about you, you will obtain the gold ring.

Stakeholder Stories Solicited at their Sites.

If you use a mechanical method and a mechanical tone and a mechanical manner to ask your users and other stakeholders about their needs and wants, what you will uncover are the most mundane, most rudimentary, most superficial and socially acceptable needs and wants. You can indeed use this information to design a product or service, and you may even have a product or service that succeeds in the marketplace. It will likely be, however, a rather short-lived “win.” Why? Because you are designing to fulfill wants that are subject to the wild winds of passing fashion rather than to catch the fire of an underlying passion.

What I found for myself was that it typically took about an hour of talking with a stakeholder, and most importantly, listening attentively, before they began to tell me their real stories. Your mileage may differ according to culture, context, power relations, your personality, and so on. I like to use a semi-structured interview. In this type of interview, there are known questions that I want to ask. But I also schedule plenty of time to let them elaborate, tell me what’s what behind the scenes. I know that in the corporate world, there is an ever-present push for being “efficient” and getting the job done as quickly as possible. So, it’s tempting to get the informant “back on track.”

I always prefer to interview an informant in their workplace. This seems like common courtesy; it puts them more at ease; and it sometimes reveals their use of other people, references, private notes, etc. as well as what they are dealing with in terms of atmosphere, noise levels, interruptions, desk space, etc. It also makes it much more likely that they can retrieve more of their own memories about work incidents more accurately because of all the contextual cues.

John Whiteside, who ran the Usability group for Digital Equipment Corporation for a time, recounts running various usability studies and gathering data in various ways about a product they were designing for manuscript centers (places where human beings, historically almost always women, transcribed the dictation of others into text on a computer so that it could be edited, re-written, stored, etc.). The first time that they visited their users in the field, they discovered that they spent about seven hours a day typing and about an hour every day counting up, by hand, the number of lines they had typed. So, in one instant, they realized a feature that would improve productivity significantly.

Guidelines for Soliciting Stakeholder Stories.

When I managed the storytelling project at IBM Research, I was fortunate enough to hire Deborah Lawrence to help with the project. She thought it would be a cool idea to interview experts in a number of fields whose job, in one way or another, involves soliciting stories. So, she went out and did just that. I believe that her interviewees included medical doctors, policeman, reporters, social workers, and psychotherapists. These various practitioners had very similar guidelines.

Story Elicitation Guidelines:

Provide a “warm-up” period.
Tell something personal and revealing about yourself; perhaps tell a story that is a model of the kind of story you’re looking for.
Observe an implicit contract of trust.
Provide a motivation for the story — why it’s important.
Accept the storyteller’s story and worldview. Don’t resist the story.
Reveal who you are, how the story will be used, potential audience and goals, answer questions.
Use questions to probe. Sometimes, a totally “off the wall” question can create space for story to emerge.
Empower the storyteller — they are the expert.
Avoid threat; don’t appear as an expert yourself.
Listen with avid interest.

These may seem fairly obvious such as does a lot of the advice in the book, How to Win Friends and Influence People. (Come to think of it, that might be the single best book you can read if you want a career in HCI/UX). However — back to the guidelines. I think they seem obvious once pointed out, in much the same way that once someone points out the “pig in the clouds” (or the face in the tree) you cannot not see it.

The above list is not, of course, meant to be the definitive such list. This was based on one study. If you have additional guidelines or disagreements, please let me know.

Author Page on Amazon

The Walkabout Diaries

The Myths of the Veritas

A Pattern Language for Collaboration

Imagine All the People

The Dance of Billions

Roar, Ocean, Roar

Wordless Perfection

11 Thursday Dec 2025

Posted by petersironwood in AI, creativity, HCI, psychology, sports, Uncategorized, user experience

≈ 1 Comment

Tags

AI, art, creativity, drawing, education, intuition, life, problem formulation, Representation, Right-brain, sports, thinking, writing

————————————

I like to write. In fact, I like to write so much that I wrote before I could even read. When my early crayon “writings” in my grandfather’s books were discovered, instead of praise, I was spanked. I’m not even sure they really tried hard to read my learned annotations. Their missing the point didn’t deter me though. I like words! I like writing poetry, essays, stories, plays, and even novels. Words help human beings communicate and collaborate. However…

In this essay, I’d like to mention some instances of wordless success.

In the neighborhood where I grew up, we spent most of the summer playing baseball, basketball, and football. I had never played golf nor paid much attention to it as a kid and when it came on TV I walked by with hardly a glance. At that point in my life, I deigned to consider something a sport only if there were a good chance to smash into one of the other players. I had never touched a golf club or a golf ball until one summer day when I was about ten, one of the kids brought one of his uncle’s golf clubs to our baseball field along with a tee and a golf ball. He demonstrated how to hit the ball and showed us how to put our hands on the club. Kids took turns hitting the ball and retrieving it for another go.

When it came to my turn, I mainly remember just loving the shiny wood of the club. I loved wooden baseball bats back then, but the driver!! Wow! That was in a whole different category of cool. You didn’t need to be an adult or a golfer to know that! It shone opalesquely. I teed up the golf ball, and swung the unfamiliar and impossibly long club.

The resulting sound – exquisite. An explosion. A rifle shot. A cousin of the crack of a home run shot into the upper deck. But more penetrating. More elegant. More poignant.

We all looked up in amazement. My golf shot started low and straight. Then it rose and rose and disappeared far beyond the dirt road that marked the outer limit of our makeshift baseball field. It rose over the hill beyond the road and disappeared into the field beyond. There was no hope of retrieving the golfball. None of us even suggested trying. My shot was wordless perfection.

Fast forward to graduate school. In the summer afternoons, I got into the habit of playing frisbee with the neighbors. One day, I parked my car and ran into the back yard. One of my neighbors spied me and threw me the frisbee, I noticed that they had placed an empty beer can atop a utility box about a hundred feet away. I caught the frisbee on the run and threw it with the next step. The frisbee sailed with a nice arc and smacked the beer can right off. My neighbors said that they had been trying to knock that beer can off for about a half hour. My throw was wordless perfection.

Meanwhile, at the University of Michigan, several of my friends and classmates liked puzzles as much as I did. One such puzzle consisted of a triangular “board” with a regular pattern of holes. There were pegs in every hole save one. The goal was to “jump” pegs much as one does in checkers and then remove that peg from the board. Eventually, one was supposed to end up with one and only one peg. I worked on it for awhile and thought about various strategies and moves. I couldn’t seem to solve it. My phone rang. I picked it up and conversed with my friend. Meanwhile, I toyed with the puzzle while my “mind” was on the conversation. I toyed with the puzzle and solved it. Wordless perfection.

A few months or weeks later, my officemates and I worked on another puzzle. This one consisted of four cubes (aka “instant insanity”). Each cube had a different arrangement of colors. The goal was to arrange the cubes so that every “row” of faces had four different colors. I fiddled with the puzzle trying out various strategies and noting various symmetries and asymmetries. Once again, someone called and interrupted my musings. Again, I idly fiddled around with the cubes while talking on the phone. And solved it. Wordless perfection strikes again!

https://en.wikipedia.org/wiki/Instant_Insanity

Fast forward four decades. For best results, borrow Hermione’s time-turner. Otherwise, you’ll have to rely on your imagination.

Betty Edwards (“Drawing on the Right Side of the Brain”) gave a plenary address at one of the Association of Computing Machinery’s premier conferences: CHI. Among other things, she showed example after example of how much people improved in their drawing skills based on her methods. A few months later, it so happened that my wife and I had an opportunity to go to one of her five day classes.

I would have to honestly say, that course was one of the best educational experiences of my life. It was an immensely pleasurable experience in and of itself. Beyond that, the results in terms of improved drawing skills were dramatic. And, as if that were not enough, I looked at the world differently. I noticed visual things about the environment that I had never seen before.

The essence of the method Betty Edwards uses is to get you to observe and draw — while “shutting up” or “turning off” the part of your brain (or mind) that talks and plans and categorizes. In one exercise, for instance, we took a line drawing and turned it upside down. Then, we copied that image onto our pad of paper by carefully observing and drawing what we saw. She also instructed us not to try to “guess” what they were drawing, but just to copy the lines. When every line had been copied, we turned the drawings right side up again. The result jolted me! I had created an excellent likeness of the original. So had everyone else in class. The quality stunned me. Wordless Perfection.

There’s a larger lesson here, too.

I had within me, the capacity to make a very decent copy of a drawing, but had never achieved that result for 60 years. All it took was five minutes of instruction to enable me to achieve that.

What else is like that? Imagine that we have, not just one, but a dozen or even a dozen dozen “hidden talents.” Some of them, like drawing, may depend more on Not-Doing than on Doing; on Being rather than Achieving.

There was a longer lasting side-effect of the drawing course. My day to day life, as is typical of most achievement-driven people had been very much “goal-driven” and there was always an ongoing plan and dialogue. After having learned to turn that off in order to draw, I can also turn it off in order to see, whether or not I draw. Seeing (or otherwise sensing or feeling) in the moment also makes me much less judgmental. If you decide to think about the physical appearance of people in terms of how interesting they would be to draw, you end up with an entirely different way of thinking about people’s appearance.

What are your hidden talents?

——————————————

The Invisibility Cloak of Habit

Big Zig-Zag Canyon

The Great Race to the Finish!

You Fool!

Horizons University

How the Nightingale Learned to Sing

Comes the Dawn

Dog Trainers

Where Does Your Loyalty Lie?

The Dance of Billions

Roar, Ocean, Roar

Imagine All the People

Your Cage is Unlocked

Author Page on Amazon

I Went in Seeking Clarity

10 Wednesday Dec 2025

Posted by petersironwood in AI, creativity, HCI, psychology, Uncategorized, user experience

≈ 1 Comment

Tags

AI, Artificial Intelligence, coding, parallel programming, problem formulation, problem framing, problem solving, programming, technology, thinking, tools, X10

“I stopped by the bar at 3 A.M.
To seek solace in a bottle or possibly a friend
And I woke up with a headache like my head against a board
Twice as cloudy as I’d been the night before
And I went in seeking clarity” — Lyrics from The Indigo Girls: Closer to Fine

If you think programming is cognitively difficult, try parallel programming. It is generally harder to design, to code, and to debug than its sequential cousin. One of the fun projects I worked on at IBM Research was on the X10 language which was designed to enable parallel programmers to be more productive. Among other things, I fostered community among X10 programmers and used analytic techniques to show that X10 “should be” more productive. Although these analytic techniques are very useful, we also wanted to get some empirical data that the language was, in actuality, more productive.

Photo by Dominika Greguu0161ovu00e1 on Pexels.com

One part of those empirical studies involved comparing people doing a few parallel programming tasks in X10 to those using a popular competitor. But, like many other “chicken and egg” problems, there were no X10 programmers (other than the inventors and their colleagues). I was part of a team who travelled to Rice University in Houston. The design called for one group to spend a chunk of time learning X10 (perhaps half a day) and another chunk of time coding some problems.

Besides the three behavioral scientists like me who were there to make observations, there were also three high-powered Ph.D. computer scientists present who would teach the language. Programmers tend to be very smart. Parallel programmers tend to be very very smart. People who can invent better languages to do parallel programming? You do the math.

Anyway, after the volunteers students had arrived, one of the main designers of the language began to “teach them” X10.

But — there was a problem.

The powerpoint presentation designed to teach the students X10 was far too blurry to read!

Immediately, the three computer scientists tried to issue commands to the projector to put the images in focus. Nothing worked. The three of them began a fascinating problem solving conversation. The conversation concerned what communication protocol(s) among the PC, the projector, and the controller was the likely source of the problem. I suppose it might not have been fascinating to everyone, but it was to me. First, it fascinated me because I was learning something about computer science and communication protocols. Second, it fascinated me because I loved to watch these people think. I suppose many of the advanced computer science students who were in this classroom to learn X10 also found it interesting. Third, I found it fascinating because my dissertation was about human problem solving and I’ve been interested in it ever since.

But the study itself had completely stalled.

After a few minutes of fascinating conversation that did nothing to focus the images, something possessed me to walk over to the projector and turn the lens by hand. The images were immediately clear and the rest of the experiment continued.

The three computer scientists had “framed” the problem as a computer science problem and I found the discussion that sprang from that framing to be fascinating. But one of the part-time jobs I had had as an undergraduate was as a “projectionist” at Case-Western, and it was that experience that allowed me to try framing the problem differently. All of us have huge reservoirs of experience outside of our professional “training” and those experiences can sometimes be important sources of alternative ways to frame a problem, issue, or situation.

———————————-

Essays on America: Wednesday

Essays on America: The Update Problem

Essays on America: The Stopping Rule

The Invisibility Cloak of Habit

Labelism

Tools of Thought

Where Does Your Loyalty Lie?

Stoned Soup

The First Ring of Empathy

Travels with Sadie: Teamwork

Author Page on Amazon

I Say: Hello! You Say: “What City Please?”

09 Tuesday Dec 2025

Posted by petersironwood in AI, creativity, design rationale, HCI, management, psychology, Uncategorized, user experience

≈ 1 Comment

Tags

art, communication, conversation, Design, efficiency, HCI, human factors, photography, primacy, problem framing, problem solving, sensemaking, technology, thinking, UX

In the not so distant past, people would often call directory assistance operators. These operators would find a number for you. For an additional charge, they would dial it for you. In fact, this was a very commonly used system. Phone companies would have large rooms filled with such operators who worked very hard and very politely, communicating with what was often a hostile and irrational public.

Customer: “I have to get the number of that bowling alley right near where the A&P used to be before they moved into that new shopping center.”

Operator: “Sir, you haven’t told me what town you’re in. Anyway…”

Customer: “What town?! Why I’m right here in Woburn where I’ve always been!”

There were so many operators that the phone companies wanted their processes to be efficient. Operators were trained to be friendly and genial but not chatty. The phone companies searched for better keyboards and better screen layouts to shave a second here or there off the average time it took to handle a call.

There are some interesting stories in that attempt but that we will save for another article, but here I want to tell you what made the largest single impact on the average time per call. Not a keyboard. Not a display. Not an AI system.

It was simply changing the greeting.

Photo by eberhard grossgasteiger on Pexels.com

Operators were saying something like: “New England Telephone. How can I help you?”

After our intervention, operators instead said, “What city please?” It’s shorter and it’s takes less time to say. But the big change was not in how long the operator took to ask the question. The biggest savings was how this change in greeting impacted the customer’s behavior.

When the operator begins with “How can I help you?” the customer, or at least some fraction of them, are put into a frame of mind of a conversation. They might respond thusly:

“Oh, well, you know my niece is getting married! Yeah! In just a month, and she still hasn’t shopped for a dress! Can you believe it? So, I need the number for that — if it were up to me, I would go traditional, but my niece? She’s — she’s going avant-garde so I need the number of that dress shop on Main Street here in Arlington.”

Photo by Tuu1ea5n Kiu1ec7t Jr. on Pexels.com

With the “What City Please?” greeting, the customer was apparently put into a more businesslike frame of mind and answers more succinctly. They now understand their role as proving information in a joint problem solving task with the operator. A typical answer would now be:

“Arlington.”

“In Arlington, what listing?”

“Dress shop on Main Street.”

The way in which a conversation begins signals what type of conversation it is to be. We know this intuitively. Suppose you walked up to an old friend and they begin with: “Name?” You would be taken aback. On the other hand, suppose you walk up to the line at the DMV and the clerk says, “Hey, have you seen that latest blog post by J. Charles Thomas on problem framing?” You would be equally perplexed!

Conversation can be thought of partly as a kind of mutual problem solving exercise. And, before that problem solving even begins, one party or the other will tend to “frame” the conversation. That framing can be incredibly important.

Even the very first words can cause someone to frame what kind of a conversation this is meant to be.

Words matter.

The Primacy Effect and The Destroyer’s Advantage

https://petersironwood.com/2018/02/13/context-setting-entrance/

Essays on America: Wednesday

After the Fall

The Crows and Me

Cancer Always Loses in the End

Come Back to the Light

Imagine All the People…

Roar, Ocean, Roar

The Dance of Billions

How the Nightingale Learned to Sing

Travels with Sadie

The First Ring of Empathy

Donnie Visits Granny!

You Must Remember This

The Walkabout Diaries: Bee Wise

Author Page on Amazon

Problem Framing: Good Point!

08 Monday Dec 2025

Posted by petersironwood in AI, America, design rationale, HCI, management, psychology, story, Uncategorized, user experience

≈ Leave a comment

Tags

AI, art, life, politics, problem finding, problem formulation, problem framing, problem solving, technology, thinking, tools, USA

You have probably heard variations on this old saw, “To a hammer, everything looks like a nail.” I’ve also heard, “If you have a hammer, everything looks like a nail.” There is also this popular anecdote:

One night, I took my dog out for a walk and I noticed one of my neighbors under a nearby street lamp crawling around on his hands and knees, apparently looking for something. I walked over and asked, “What are you looking for?”

“My car keys!” He replied.

I have pretty good vision, so I helped him. I didn’t see any car keys so after a minute or so I asked, “Where exactly did you lose your keys?”

He stood up, cracked his back, and pointed back to a nearby park. “Over there.”

“Over there?! Then, why are you looking under the street lamp? Why aren’t you looking over at the park entrance?”

“Oh, that’s obvious! The light is so much better here!”

For a time, I had to very interesting and challenging job in the mid 1980’s at IBM Headquarters to try to get the company to pay more attention to the usability of their products and services. As a part of this, I visited IBM locations throughout the world. At one fabrication plant, our tour guide took us by an inspection station. This was not an inspection statement for chips. It consisted of one person whose job was to look through a microscope and make sure that two silver needles were perfectly aligned.

After we left the station, our tour guide confided that they were strongly considering replacing the person with a machine vision system. The anticipated cost would be substantial, but they hypothesized that the system would be more accurate and faster. It was, our host, insisted, just the nature of humans to be slow and inaccurate.

Maybe.

When I looked at the inspection station however, with my background in human factors, I had a completely different impression of the situation. The inspector sat on a fixed height stool and had to bend his neck at an absurd angle to look into the microscope. He was trying to align these silver needles against a background that had almost the same hue, brightness and saturation.

Other than blindfolding the man, I’m not sure what they could have done to make the task more unnecessarily difficult. I suggested, and eventually, they implemented, a few inexpensive ergonomic changes and time and accuracy improved.

Like other companies in the technology segment, IBM often saw problems as ones that could be solved by technology. At that time, technology systems was their main business. Since then, they have expanded more fully into software and services. In fact, those services now include experience design.

If you find yourself enamored of technology in general, or some specific class of technology such as machine vision, speech recognition, or machine learning, you might overlook much simpler and cheaper ways to solve problems or ameliorate situations. Of course, you might lose some revenue doing that, but you can also win long term customer loyalty.

Even if you are a hammer, everything is not a nail.

That applies as well to User Experience. You might design the most wonderful UX imaginable for a particular product or service. But if it is shoddily made so that it is error prone; if it lacks important functionality; if the sales force is inept; or if service is horrible, those failures can completely overwhelm all the good work you have done on the UX. Because of the nature of UX, you might learn important knowledge or suggestions for other functions as well. It often requires finesse to have such suggestions taken seriously, but with some thought you can do it.

During my second stint at IBM, I worked for a time in a field known at that time as “Knowledge Management.” One of our potential clients was a major Pharma company who felt that their researchers should do a better job of sharing knowledge across products. They wanted us to design a “knowledge management system” (by which they meant hardware and software) to improve knowledge sharing.

Simply building a “Knowledge Management System” would be looking under the streetlamp. They knew how to specify a technology solution from IBM and have it installed.

However — they were unwilling to provide any additional space, time, or incentives for their employees to share knowledge with their colleagues!

They were convinced that technology would be the silver bullet, the solution, the answer, the Holy Grail, the magic pill. They viewed technology as less disruptive than it would have been to change employee incentives, or space layout, or give them time to actually learn and use the technology system.

This reaction to “knowledge management” was not unique. It was common.

To me, this seems very similar to the notion that health problems can all be solved with a magic pill. What do you think?

—————————————

Since originally writing, we have had the spectacle of DOGE: Destroying Our Government’s Effectiveness under the excuse of making it “more efficient.” It might be (as I strongly suspect) that the destruction was quite intentional. It might be (as some think) that it was accidental. In either case, the result was predictable because the method was guaranteed not to work to actually make things more efficient. If you really wanted to do that, you would take the time to understand a system before trying to redesign it. You would identify all relevant stakeholders and get their input. You would not redesign a system using a gang of young hackers but instead use an interdisciplinary team of experienced experts. You would check out your redesign both with those who were doing the work and with at least one group who were not familiar but had similar experience. Then, on the basis of feedback, you would redesign. When you were sure that you had the design right, you would not then institute it everywhere but in one small trial installation.