April 29, 2017

Understanding interactions in the smart home

Vic Callaghan, Victor Zamudio, and Jeannette S. Y. Chin
Mathematically based tools are proving helpful in coming to grips with the dynamics⎯both human and machine⎯of advanced networked systems.

In a smart home most electronic and electrical devices (including computer-based systems) are connected to each other and the outside world via computer networks (see Figure 1). The potential of the technology ranges from simple sharing of digital media to coordinating actions between collections of networked devices. Examples include energy-efficient homes, technology to support independent living and augmented lifestyle (entertainment and communications). Smart homes also enable the creation of ‘virtual appliances’,1 which work by deconstructing conventional appliances and applications into their elemental services and offering these back to the user for recombination in various ways. Most of these ‘intelligent’ homes are still at a pre-commercial deployment stage. The main obstacles to their uptake are interactions between systems and between people and systems.

Sociotechnical interaction

A key question is how nontechnical people might be empowered to combine and program collectives of networked services to deliver the required functionality. Various approaches are possible, such as the use of intelligent autonomous-agent techniques to monitor, model and pre-emptively control the environment. These methods often employ life-long learning, in which they continually monitor and record a user's habitual behaviour so as to adapt to the changing needs of the occupant and environment. However, autonomous agents do not appeal to everyone. If they are not appropriately applied, they can remove control, transparency and creativity from the system, undermining its appeal, as well as the user's trust. This has led people to consider end-user programming,2 which mostly attempts to disguise the process by harnessing natural metaphors (e.g., jigsaw construction) and modes of interaction (e.g., spoken dialogue). A variety of hybrid solutions have also been tried.

The Essex iSpace: an experimental smart home.

The 3C model: a socio-agent framework.

Surveys have shown that user acceptance of technology in personal spaces is linked to issues of privacy, control and creativity. In terms of smart homes, control can be seen as a balance of technological autonomy versus user influence. To expose these concerns, we created a conceptual socio-agent framework (the 3C model) that illustrates the issues graphically (see Figure 2). To capture the balance of automation, we included an ‘autonomy axis’ that shows the possibilities for configuration from manual (end user) to automatic (agent based). In terms of sociology, reactions to technology vary from love to fear. Accordingly, we included an ‘attitude’ axis that shows user reaction (philia versus phobia) to the different possibilities. The quadrants represent differing combinations of technology and attitude, identifying potentially significant positions within this space. More detailed discussion is available elsewhere.3

Cyclic interaction

Another important interaction occurs between devices (or services). Networks enable smart homes to coordinate actions, leading to interdependencies in the behaviour of the devices. Commonly, behaviours are based on sets of rules formed by either autonomous agents or end-user programming. There could potentially be large numbers of interacting devices (e.g., tens to hundreds), programmed by different people or agents and thus giving rise to complex system-wide interactions. This behaviour is further complicated by various sources of nondeterministic behaviour arising from users (sometimes behaving somewhat idiosyncratically), nomadic devices (including random malfunctions), temporal delays (varying according to loads) and so on. While much of the behaviour of these systems is what the users (or programmers) intended, the complex nature of their interaction makes them unpredictable. Mario Kolberg4 produced a taxonomy for such destructive interactions, namely: multiple trigger, which refers to two services controlling the same appliance, shared trigger, where an event is sent to two different services that perform conflicting actions, missed trigger, in which one service prevents a second from operating, and sequential action, which describes a service request to an appliance that, in turn, causes the appliance to send notifications to another service.

Researchers have found solutions for all cases apart from cyclic interaction (part of the sequential action-interaction category). A simple way of understanding cyclic interaction in smart homes is to consider two light-control agents (see Figure 3). Here it is clear that if the rules are set such that ‘light A’ being ‘on’ depends on ‘light B’ being ‘off’, but ‘light B’ being ‘off’ depends on ‘light A’ being ‘off’, we have a mutually exclusive condition with a resulting oscillation (flashing lights) whose period is determined by the temporal properties of the system. In addition, if more devices are involved in a loop, loops overlap (nodes are shared) or the network is perturbed (by users, nomadic devices and variable loading), the problem quickly becomes intractable. This is especially difficult to solve because complex systems theory5 shows that, in general, it is not possible to predict theoretically whether a given rule-based system will suffer from unwanted cyclic instabilities. However, we have been able to devise methods that identify the potential for such instabilities and to ‘inoculate’ the system against the occurrence of cyclic interactions by developing a number of mathematically based tools.

The interaction network (IN) is a mathematical framework based on directed graph and set theory that provides a means of representing and reasoning about rule-based systems. An instability-prevention system uses IN analysis to generate an optimal locking strategy. The multidimensional model for visualization (MDM) is a graphical model that makes it possible to picture time, devices and their binary state. It is especially useful for understanding system dynamics. Finally, the cyclic-density metric relates to the usability of the system. Figure 4 shows some cyclic interactions occurring in a small 4×4×4 benchmark that we developed.

Simplified illustration of cyclic instability.

(a) 4×4×4(64-node) interaction benchmark featuring random connections containing 81 loops. (b) Unlocked response of the random system showing system-wide oscillations. (c) Multidimensional model for visualization.

In summary, interaction networks, the instability-prevention system and the multidimensional model of visualization work in a unified way, providing a formal description of the problem, a graphical representation of the dependencies of the rules, a mechanism to prevent periodic behaviour and a microvisualization of the system dynamics.6 The tools described in this article can be used to overcome problems related to interaction between systems, and between people and systems, in the design of smart-home technology. For our future work we intend to explore applying these techniques to a wider set of applications, such as economic and social systems.


Vic Callaghan
School of Computer Science and Electronic Engineering, Essex University

Victor Zamudio
School of Computer Science and Electronic Engineering, Essex University

Jeannette S. Y. Chin
School of Computer Science and Electronic Engineering, Essex University

  1. J. Chin, V. Callaghan and G. Clarke, Soft-appliances: a vision for user created networked appliances in digital homes, J. Ambient Intell. Smart Environ. 1, pp. 69-75, 2009.

  2. J. Chin, V. Callaghan and G. Clarke, A programming-by-example approach to customising digital homes, IET Int'l Conf. Intell. Environ., 2008.

  3. V. Callaghan, G. S. Clarke and S. J. Y. Chin, Some socio-technical aspects of intelligent buildings and pervasive computing research, Intell. Build. Int'l J. 1:1, 2008.

  4. M. Kolberg, E. Magill, D. Marples and S. Tsang, Feature interactions in services for Internet personal appliances, IEEE Int'l Conf. Commun. 4, pp. 2613-2618, 2002.

  5. V. Zamudio and V. Callaghan, Facilitating the ambient intelligence vision: a theorem, representation and solution for instability in rule-based multi-agent systems, Int'l Trans. Syst. Sci. Appl. 4, pp. 1, 2008.

  6. V. Zamudio, Understanding and Preventing Periodic Behaviour in Ambient Intelligence, 2009. PhD thesis Essex University UK

DOI:  10.2417/2200904.1625