In the Brain of the Beholder. 

Most people in the related fields of “Human Factors”, “User Experience”, and “Human Computer Interaction” learn how to run experiments. Formal study often largely focuses on experimental design and statistics. Indeed, these are important subjects. In today’s post though, I want to relate three experiences with actually running experiments. Just for fun, let’s go in reverse chronological order. 

In graduate school at the University of Michigan Experimental Psychology department, one of my classmates told us about an experiment he had just conducted. Often, we designed experiments in which a strictly timed sequence of stimuli (e.g., printed words, spoken words, visual symbols) were presented and then we measured how long it took the “subject” to respond (e.g., press a lever, say a word). Typically, these stimuli were presented fairly quickly, perhaps 1 every second or at most every 4-5 seconds. This classmate, however, had felt this was too stressful and wanted to make the situation less so for the subjects. So, instead of having the stimuli presented, say, every 4 seconds, my classmate decided to be more humane and make the experiment “self-paced.” In other words, no matter how long the subject took to make a response, the next stimulus would be presented 1 second later. So, how did this “kindness” work out in practice? 

A few days later, I heard a scream in the lab down the hall and ran in to see whether everyone was okay. One of my classmate’s first subjects had just literally ran out of the experimental room screaming “I can’t take it any more! I quit!” My classmate was flabbergasted. But eventually, he got the subject to calm down and explain why they had been so upset. The subject had begun by responding carefully to the stimuli. So, perhaps they took ten seconds for the first item, and the new stimulus came up one second later. On the second go, they took perhaps 9.5 seconds and then the next stimulus came up one second later. As time went on, the subject responded more and more quickly so the next stimulus also came up more and more quickly. In the subject’s mind, the experiment was becoming more and more difficult as determined by the experimenter. They had no idea that had they slowed back down to responding once every 10 seconds, they’d only be presented with stimuli at that, much slower speed. 

So, here we have one way that these so-called subjects differ from each other. They may not interpret the experiment in the framework in which it is thought of by the experimenter. In this particular case, there was a difference in the attribution of causality, but there are many other possibilities. This is one of many reasons for doing a pilot experiment and talking with the subjects. 

The next earlier example took place at Case-Western Reserve. In my senior year, I was married and had a kid so I worked three part time jobs while going to school full-time. One of the jobs was teaching “Space Science” and “Aeronautics” to some sixth graders at the Cleveland Supplementary Educational Center. Another one of the jobs was as a Research Assistant to a Professor in the Psychology Department. We were doing an experiment with kids in an honest-to-God “Skinner Box.” The kids pulled a lever and won nickels. Meanwhile, on a screen in front of them, there appeared a large red circle and then we looked at how much the kid continued to press the lever (without winning any more nickels) when confronted with the same red circle, a smaller red circle, a red ellipse, etc. 

There was a small waiting room next to the Skinner Box and that had a greenboard on it. So, since there was another kid waiting there just twiddling his thumbs, I decided to give him a little mini-lecture on the solar system: sun at the center, planets in order, some of the major moons, etc. 

After each kid had finished the experiment, I always asked them what they thought was going on during the experiment. (This was despite the fact that the Professor I was working for was a “strict behaviorist”). When I asked this kid what he thought was going on, he referred back to my lecture about the solar system! 

Oops! Just because the lecture and the experiment were two completely unrelated things in my mind didn’t mean they were for the kid! Of course, they seemed related to him! Both involved circles and they both took place at the same rather unique and unusual place: a psychology laboratory. 

And this too is worth thinking about. We psychologists and Human Factors people typically report on the design of the experiment and hopefully relate the instructions. We, however, do not typically report on a host of other things that we think of as irrelevant but may impact the subject and influence their behavior. Was the receptionist nice to them or rude? What did their friends say about going to do a psychology experiment or a UX study? When the experimenter explained the experiment and asked whether there were any questions, was that a sincere question? Or, was it just a line delivered in a rather mechanical monotone that encouraged the subject not to say a word? 

Of course, the very fact that humans differ so much is why some psychologists prefer to use rats. And, the psychologists (as well as a variety of biologists and medical doctors) don’t just use any old rats. They use rats that are carefully bred to be “lab rats.” They are expected to act in a fairly uniform fashion. And, for the most part, they do.

I was helping my girlfriend with her intro psych project. We were replicating the Yerkes-Dodson Law. This states that as you increase stress, performance improves, but only to a point. After that, additional stress causes performance to deteriorate (something that software development managers would do well to note). One of the ways I helped was to get some of the rats out of their cages. I would open up the top of the cage, reach around the rat behind their next and pull them out. Not a big deal. All the rats were quite placid and easy to handle. They all acted the same. Then, it was time to get the day’s last rat who was to be placed in the “high stress” condition. I went to the cage and opened it just as I had done for the last dozen rats. But instead of sitting there placidly and twitching it’s nose, this rat raced to the bars of his cage and hung on with both of his little legs and both of his little arms with all his might! Which might was not equal to mine but was rather incredible for such a tiny fellow. Rats sometimes squeak rather like a mouse does. But not this one! This carefully bred clone barked! Loudly! Like a dog. Whether this rat had suffered some previous trauma or was subject to some kind of odd mutation, I cannot say. 

But this I can say. Your “users” or “subjects” are not identical to each other. And, while modeling is a very useful exercise, they will never “be” identical to your model. They are always acting and reacting to a reality as beheld by them. And their reality will always be somewhat different from yours. That does not mean, however, that generalizations about people — or rats — are always wrong or that they are never useful. 

It does not mean that gravity will not affect people just because they refuse to believe in it. There really is a reality out there. And, that reality can kill rats or people in an eye blink; especially those who actively refuse to see what is happening before their very eyes. 

Who knows? You might be about to be placed in the “High Stress” condition no matter how tightly you hang on to the bars of your cage – or, to your illusions.  

————————————-

Author Page on Amazon