What is Interaction?

The Art of Interactive Design

In Chris Crawford’s book, The Art of Interactive Design <https://www.amazon.com/Art-Interactive-Design-Euphonious-Illuminating/dp/1886411840/>`_, Crawford defines interactivity in terms of a conversation. Specifically, interactivity is a cyclical process of two actors taking turns listening, thinking, and speaking.

Unfortunately, the word "Interactive" is often explained or defined poorly, and as a result is poorly understood by our culture. The term gets added to products as a buzzword to make them sound better, but often the objects don’t really "interact" in a way as described by Crawford’s definition. There is no "conversation" or cyclical process.

A literal conversation between two people fits this definition literally, but when the interaction is between a human and an electronic device, the steps become input, process, and output. Key point though is that there are two actors, not one. Each actor is some kind of "purposeful creature," so a wall or a rug cannot be interactive.

It isn’t always clear if something is interactive or not. Rather than being a boolean thing, there are different degrees of interactivity. We can evaluate high or low levels of interactivity by evaluating the quality of the listening, thinking, and speaking steps. Excelling in one area does not compensate for failures in another. A common design error when building interactive products is to fail to appreciate this idea.

Books and movies are not interactive. They can and do elicit emotional reactions, but those are reactions. The authors or producers cannot hear or respond to the reader’s or viewer’s responses to their work.

Crawford recalls the writing of Socrates, who lived during a time shortly after the advent of writing and books. He saw a weakness in books because the words were fixed and therefore were different from the words coming from a teacher.

User interface design considers the form of the interaction, or the "listening" and "speaking" steps. It does not consider the "thinking" step. This would be the software or core behaviors of the object.

In the Internet world, graphic design of web pages has come to be viewed as important, but the page design by itself is only part of the overall task of interactivity design. Form and function of a website need to be designed together.

A Brief Rant on the Future of Interaction Design

In Bret Victor’s essay A Brief Rant on the Future of Interaction Design, Victor writes about our "Vision Of The Future" and argues that the traditional vision of the future involves ubiquitous fancy computer screens that we interact with using our hands. In his view, this vision isn’t actually visionary. He refers to this technology "pictures under glass." We are still using our hands, but we aren’t interacting in the same way that we do when we interact with real-world objects using our hands.

When we use electronics with touch screens, we are usually using a single finger to interact with pictures of buttons or sliders under the glass screen. But our hands are capable of much more complex positions and grips; shouldn’t our vision of the future leverage this?

As Victor asks, "why aim for anything less than a dynamic medium that we can see, feel, and manipulate?" And he is right! As we think about interaction, we should have loftier goals than fancy glass screens.

Class Activity: Note-taking Device Design

In class we designed a hypothetical note-taking device that could use speech recognition to record class discussions and allow a student to highlight what they felt was most important and worth being saved to their class notes. The prototype is depicted below.

The dialog would be transcribed on the right and allow the user to highlight with their finger the text that should be saved. The left side of the screen would allow the user to reorder the notes and also record video.

Considering our new understanding of interactivity, how well does this device measure up?

Although this would be a useful tool, the interactivity provided by the device is minimal. The primary interaction taking place (or should be taking place) during class-time is between the student, teacher, and other students in the class. This device allows the students to record what is being discussed, but the interaction is analogous to grabbing words off a conveyor belt moving at a speed of a few hundred words per minute. The quality of the device’s "thinking" as defined by Crawford is minimal. It is probably better this way because if it wasn’t the device might distract students from class-time. The device also only offers "pictures under glass" as a means for user interaction. Although I still believe this would be a useful tool, it isn’t a visionary example of interactivity.