Wednesday, April 11, 2018

Studying the nature of interactivity (HCI)

I have always been drawn to research that is trying to understand the 'nature' of some phenomenon. There are several reasons for this. I am attracted to the idea that as a researcher I am actually improving my own understanding of the phenomenon I am studying and increasing my ability to explain its nature, and that this knowledge grows over time and is in some sense is cumulative. As a consequence of this, I have had two major research projects during my career. One has been to understand the nature of designing as a human approach to change and the other has been to understand the nature of the interaction between humans and interactive artifacts/systems.

I have realized that this aim is commonly confused with a radically different purpose which is to develop support that can improve peoples ability to design new technology and systems. As any kind of basic research, the results it produces will never be able to tell anyone what to design or create. Basic research primarily provides a deeper understanding of some phenomenon.  Instead of leading to prescriptive knowledge that could make design easier, it leads to increased complexity and from a design perspective often increased difficulty.

After being involved in basic research for many years I am more than ever convinced that it is not just necessary, it is also possible, even when it comes to a field like HCI. It is possible to study the nature of interactivity. It is possible to develop a deeper understanding of interactivity as a real phenomenon, something we, over time, can actually become better at explaining with sound and well-grounded theories and models.

For this to happen HCI research has to change some of its fundamental assumptions. For instance, HCI research has as a primary goal to lead to some form of (immediate) improvements or usefulness. In Habermas terms it is possible to say that HCI research is driven by a "control knowledge interest" and less of a knowledge interest of "understanding" or "emancipation" (which are his other two categories.  Of course, I am not saying that research with a "control" knowledge interest is not needed or should not be done. HCI is as a research field primarily aimed at improving practice so that is fine. But the field would benefit from also engaging in these other knowledge interests with a strong passion.

I am therefore both happy and proud of the work I have done with my colleague Lars-Erik Janlert about the nature of interactivity, where we did set out to do what I discussed above. We examined an existing phenomenon (interaction/interactivity) with the purpose to describe and explain its nature, to the best of our ability. Of course, we only reached so far. But I believe that it is a start that hopefully will inspire others. The result is the book "Things That Keep Us Busy -- the elements of interaction" (MIT Press, 2017)

Tuesday, March 27, 2018

One of the best books ever on design

Again I returned to the writings of David Pye. His writings have been with me since the early 80s. The nature and aesthetics of design" is one of the absolute best books ever written about design.

Monday, March 26, 2018

Interaction design, complexity, and virtuosity

One of the most preached principles in design, and particularly in interaction design, is to strive for simplicity. It is yet difficult to find any examinations of what simple really means when it comes to design (there are some good exceptions, such as Maeda and Mollerup, see references below).

In many cases, being simple is of course good. Nevertheless, we also know that we live in a world that is complex and sometimes requires complex actions. We also know that people can do amazing things even with devices that are highly complex. Virtuosity can be achieved. So, the question becomes, can we design artifacts that require complex actions in a way that could support the efforts of reaching virtuosity?

Below is an excerpt from our book "Things That Keep Us Busy - the elements of interaction" (MIT Press, 2018). This is from Chapter 6 "Control".

"6.5 Virtuosity

Can we imagine artifacts that are highly complex while still being inviting to a user and providing incentives for continuous engagement, maybe even spurring a few users to aim for extreme levels of mastery? Let us consider the violin example again. As we noted in chapter 5, the violin combines low internal and external complexity with high interaction complexity, which apparently invites a range of user behaviors. Most people initially find the violin extraordinarily difficult to interact with even though the artifact itself is quite simple: just a few strings mounted on a soundboard and a bow. Beginners are not able to make any real music on the violin even though they of course can make (terrible) sounds.

We also know that the violin invites virtuosity, a display of expertly handled extreme interaction complexity. Virtuosic violin performances have a long tradition and many famous composers have written music specially tailored to let distinguished players show off their virtuosity. When it comes to musical virtuosi, it is not uncommon to hear comments that virtuosity is more a technical achievement or circus act than an expression of musical insight and depth of interpretation. It may be that listeners, rather than being moved by the music, are impressed and awed by the display of almost superhuman skills.

Why is it that some artifacts seem to invite virtuosic use while others don’t? Are there examples of digital artifacts or systems that have the same virtuosity-inviting quality as the violin? There seem to be few parallels to the violin example: low external complexity yet still inviting virtuosity. We have to remember that up until recently digital artifacts and systems have typically had relatively high external complexity. The development of human–computer interaction has been dominated by a constant effort to refine and keep controllability on par with ever-expanding functionality, rather than any ambition to lower external complexity, not surprisingly resulting in externally complex artifacts and systems. Of course, lately with the proliferation of small digital artifacts and apps with specialized functionalities, increasingly under the pressure of the interface bottleneck
problem, external simplicity has become a key issue and mark of good design. However, these artifacts do not seem to invite virtuosity, perhaps because the interaction complexity usually is low, too low.

Another explanation of why we have not seen convincing examples of virtuosity with digital artifacts could be the up till now typically strongly discretized input and output and strict turn taking of digital artifacts, quite far from the analog and continuous flow of violin playing. However, discrete input combinations related to desired output combinations in a complex manner do exist, for instance in some computer games, which open for the possibility of some sort of virtuoso performances; similarly, computer hackers can make dazzling performances of rapidly finding and fixing software problems, hammering away at breakneck speed on a command-based interface. Even with more mundane examples, such as highly complex office software with huge numbers of commands, layers of functionality, we may be impressed by the brilliant technique the professional user displays. Such examples demonstrate fast decision making under pressure in a situation where there is a lot going on to keep track of. This could potentially be understood, and savored, as a kind of virtuosic performance.
A difference is that the interaction is overtly digital and the external complexity considerable: it is a form of “combinatory” virtuosity rather than the “smooth” virtuosity of top-class violin playing. A violin seems to allow more room for users to express themselves in a way that goes beyond functional achievements, possibly that might have something to do with the smoothness and very fine nuances the instrument affords the player. But the distinction between analog and digital interaction, from the user point of view, has become very blurred by now; first with the advent of graphical user interfaces (GUI) and pointing devices, then with new tracking, sensing, and presentation techniques partly deriving from virtual- reality technology, and with tangible user interfaces (TUI) where physical objects are used in analog mode to interact with the digital artifact, and lately with the breakthrough of gestural interaction. The stage is now set for new applications and forms of interactions that could be much more like violin playing. In fact, we already have some artifacts and systems like these around, for instance, in the form of games that with quite simple interfaces invite and compel the user to develop sophisticated interaction skills, often virtual versions of existing “real” games and sports. Using a gesture-controlled golf simulator seems to be close enough to the “real” thing to let expert golfers show their virtuosity in the simulator.

Although we have focused on the violin as a possible model and inspiration for future artifacts and systems inviting virtuosity, we do not want to exclude designs with high external complexity from consideration in this respect. After all, the piano and other keyboard instruments, externally much more complex than the violin, also often figure in virtuosic compositions and performances. Still, the external simplicity of the violin by its very sharp contrast to older-style digital artifacts and systems perhaps makes it a more intriguing and challenging model for a designer. We believe that virtuosity as a manner of interaction could, and maybe even should, be reconsidered and revisited in interaction design. There may be situations and technology use that would benefit from such a perspective. The history of virtuosity is rich, ranges over many fields, and might provide us with new insights into interaction design possibilities with contemporary technology.

There are of course arguments against a move toward virtuosity in the field of interaction design. One obstacle to virtuosic interaction with many digital artifacts is their short market lives and rapid development: the violin has been around for hundreds of years with hardly noticeable changes, giving the community of users ample opportunities to develop a culture of virtuosic use. Many of the new digital artifacts, in contrast, are on the market less than five years before they disappear or are superseded by completely different designs, sometimes building on new and different technologies. Another argument against virtuosity is of course that it requires extensive training over a long time period. We admire virtuosity since most of us do not have the time and maybe talent to engage in training to the degree needed. So, to require virtuosity from users may restrict the group of potential users to almost none.

A more obvious factor working against virtuosity is that while it might be a sometimes-desired manner of interaction, such extreme interaction is not typically what designers and users look for. In much everyday artifact use, most users care for little more than very basic performance—and there often is a conflict between the requirements of high-level performers and low-level performers that prefer low interaction complexity. But, maybe a general change of attitude and the demands of professional users can change that; we have actually come a long way from the time when “user- friendliness” was the key criteria, and we still keep moving."

Maeda, J. 2006. The Laws of Simplicity. Cambridge, MA: MIT Press.
Mollerup, P. 2015. Simplicity—A Matter of Design. Amsterdam: BIS Publishers.

Wednesday, March 21, 2018

VR, authenticity and killer apps

The hype around VR has lately been growing and ads are making the case that it is time to take on this new form of interaction for all kinds of applications. However, in a great article (published by TechCrunc) the writer  Sibjeet Mahapatra argues that there is a major problem in the world of VR. The problem is the missing killer app.

Mahapatra discusses what he sees as the two values that VR can offer, primarily a sense of presence. But he also argues that VR still lacks when it comes to authenticity. The author makes a good case for what is needed for VR to really become something wanted by a larger audience. So, a good read about interaction and interactivity.

[And of course, I always like someone who  mentions Rorty and his "experience machine"!]

Monday, March 19, 2018

PhD course on "The elements of interaction"

I am just back from a trip to Europe where I among other things taught a two day Ph.D. course called "The Elements of Interaction". The course was organized at the department of computer science at Aalborg University by my colleague Peter Axel Nielsen.

It was an intense experience. Two full days completely focused on our new book "Things that keep us busy--the elements of interaction". We worked through almost all chapters in the book. It led to wonderful discussions. The doctoral students were great. They were curious, critical and inquisitive. And to me, it was a great way of exploring if the content of the book make sense and work for others than me and my co-author Lars-Erik.

Based on the experience, I learned two things. Our book seems to work fine with PhD students and they were able to relate the content to their own research in ways that might help them. Secondly, to teach a PhD course in this format, two full days, is excellent. It leads to complete focus. I will definitely argue for this format when I have the chance.

Thursday, March 08, 2018

When is a copy and the original the same

In an interesting article, Byung-Chul Han examines the notion of what is an original artifact versus a copy. He explains the different notions in the East and West in a way that is relevant to anyone thinking about design and creativity. The major argument that Han makes is that in China (and other Eastern societies) the notion of what is an original might appear as strange to us in the West. According to him, in these cultures, a perfect copy is the same as the original and has no greater value than the original. The article tells a number of fascinating stories of when this difference in thinking between East and West has led to serious misunderstandings and conflicts.

I was intrigued by this article. I have no idea how correct it is and how true it depicts the cultural differences, but even if it is not a true depiction, it does raise a lot of exciting questions about how to think about what is an original and if an original should have any particular status. Again, all relevant questions to any designer.

I have earlier commented on two other books by the philosopher Byung-Chul Han on this blog. See links below.
"In the swarm"

"The burnout society"

Btw, the online magazine Aeon, where this article published, is excellent!

Monday, March 05, 2018

Interaction and Complexity

One aspect of interaction that keeps emerging is related to complexity. A lot of people complain that interacting with systems and devices today is too complex. As a natural reaction to that, a lot of designers argue for simplicity as an important design principle. But what is complexity when it comes to interaction and why does it appear? In our recent book "Things that keep us busy -- the elements of interaction" we spend two chapters on interaction complexity and the related notion of control.

We do this by examining what interaction complexity is and what causes it. This leads to a theory (or model) of interaction complexity that consists of four different types of complexity. This is what we write (on p 85).

"We will identify and define four main loci of complexity of an artifact or system (see figure 5.1), all with respect to its designed purpose:
      1. internal complexity
      2. external complexity
      3. interaction complexity
      4. mediated complexity
These four loci should not be thought of as different measures or types
of complexity; they represent a rough division into the main (more or less abstract) locations where complexity is residing in varying degrees, and manifesting itself in various ways."

and figure 5.1 lays out how these different forms of complexity relate to each other.

After having worked with this model for quite some time, I find it quite useful and it helps to understand many aspects of interactivity and its relation to complexity. One of the major consequences of the model is that it indicates (strongly) that there is no easy "fix". To design for simplicity does not have any optimal solutions, every design decision about how to handle (or where to put complexity) leads to serious trade-offs that are inevitable.

This is why I believe that understanding this model can help and prepare every interaction designer to better approach the design of any interactive system and device.

Featured Post

Why Design Thinking is Not Enough

If you go to Youtube and look for "design thinking" you will find a large number of videos with TED talks and other talks all expl...