Three Degrees of “G”s: How an Airbag Deployment Sensor Transformed Video Games, Exercise, and Dance

David Gerhard

Abstract


Introduction

The accelerometer seems, at first, both advanced and dated, both too complex and not complex enough. It sits in our video game controllers and our smartphones allowing us to move beyond mere button presses into immersive experiences where the motion of the hand is directly translated into the motion on the screen, where our flesh is transformed into the flesh of a superhero. Or at least that was the promise in 2005. Since then, motion control has moved from a promised revitalization of the video game industry to a not-quite-good-enough gimmick that all games use but none use well.

Rogers describes the diffusion of innovation, as an invention or technology comes to market, in five phases: First, innovators will take risks with a new invention. Second, early adopters will establish a market and lead opinion. Third, the early majority shows that the product has wide appeal and application. Fourth, the late majority adopt the technology only after their skepticism has been allayed. Finally the laggards adopt the technology only when no other options are present (62). Not every technology makes it through the diffusion, however, and there are many who have never warmed to the accelerometer-controlled video game. Once an innovation has moved into the mainstream, additional waves of innovation may take place, when innovators or early adopters may find new uses for existing technology, and bring these uses into the majority. This is the case with the accelerometer that began as an airbag trigger and today is used for measuring and augmenting human motion, from dance to health (Walter 84). In many ways, gestural control of video games, an augmentation technology, was an interlude in the advancement of motion control.

History

In the early 1920s, bulky proofs-of-concept were produced that manipulated electrical voltage levels based on the movement of a probe, many related to early pressure or force sensors. The relationships between pressure, force, velocity and acceleration are well understood, but development of a tool that could measure one and infer the others was a many-fronted activity. Each of these individual sensors has its own specific application and many are still in use today, as pressure triggers, reaction devices, or other sensor-based interactivity, such as video games (Latulipe et al. 2995) and dance (Chu et al. 184). Over the years, the probes and devices became smaller and more accurate, and eventually migrated to the semiconductor, allowing the measurement of acceleration to take place within an almost inconsequential form-factor. Today, accelerometer chips are in many consumer devices and athletes wear battery-powered wireless accelerometer bracelets that report their every movement in real-time, a concept unimaginable only 20 years ago.

One of the significant initial uses for accelerometers was as a sensor for the deployment of airbags in automobiles (Varat and Husher 1). The sensor was placed in the front bumper, detecting quick changes in speed that would indicate a crash. The system was a significant advance in the safety of automobiles, and followed Rogers’ diffusion through to the point where all new cars have airbags as a standard component. Airbags, and the accelerometers which allow them to function fast enough to save lives, are a ubiquitous, commoditized technology that most people take for granted, and served as the primary motivating factor for the mass-production of silicon-based accelerometer chips.

On 14 September 2005, a device was introduced which would fundamentally alter the principal market for accelerometer microchips. The accelerometer was the ADXL335, a small, low-power, 3-Axis device capable of measuring up to 3g (1g is the acceleration due to gravity), and the device that used this accelerometer was the Wii remote, also called the Wiimote. Developed by Nintendo and its holding companies, the Wii remote was to be a defining feature of Nintendo’s 7th-generation video game console, in direct competition with the Xbox 360 and the Playstation 3. The Wii remote was so successful that both Microsoft and Sony added motion control to their platforms, in the form of the accelerometer-based “dual shock” controller for the Playstation, and later the Playstation Move controller; as well as an integrated accelerometer in the Xbox 360 controller and the later release of the Microsoft Kinect 3D motion sensing camera.

Simultaneously, computer manufacturing companies saw a different, more pedantic use of the accelerometer. The primary storage medium in most computers today is the Hard Disk Drive (HDD), a set of spinning platters of electro-magnetically stored information. Much like a record player, the HDD contains a “head” which sweeps back and forth across the platter, reading and writing data. As computers changed from desktops to laptops, people moved their computers more often, and a problem arose. If the HDD inside a laptop was active when the laptop was moved, the read head might touch the surface of the disk, damaging the HDD and destroying information. Two solutions were implemented: vibration dampening in the manufacturing process, and the use of an accelerometer to detect motion. When the laptop is bumped, or dropped, the hard disk will sense the motion and immediately park the head, saving the disk and the valuable data inside.

As a consequence of laptop computers and Wii remotes using accelerometers, the market for these devices began to swing from their use within car airbag systems toward their use in computer systems. And with an accelerometer in every computer, it wasn’t long before clever programmers began to make use of the information coming from the accelerometer for more than just protecting the hard drive. Programs began to appear that would use the accelerometer within a laptop to “lock” it when the user was away, invoking a loud noise like a car alarm to alert passers-by to any potential theft. Other programmers began to use the accelerometer as a gaming input, and this was the beginning of gesture control and the augmentation of human motion.

Like laptops, most smartphones and tablets today have accelerometers included among their sensor suite (Brezmes et al. 796). These accelerometers strictly a user-interface tool, allowing the phone to re-orient its interface based on how the user is holding it, and allowing the user to play games and track health information using the phone. Many other consumer electronic devices use accelerometers, such as digital cameras for image stabilization and landscape/portrait orientation. Allowing a device to know its relative orientation and motion provides a wide range of augmentation possibilities.

The Language of Measuring Motion

When studying accelerometers, their function, and applications, a critical first step is to examine the language used to describe these devices. As the name implies, the accelerometer is a device which measures acceleration, however, our everyday connotation of this term is problematic at best. In colloquial language, we say “accelerate” when we mean “speed up”, but this is, in fact, two connotations removed from the physical property being measured by the device, and we must unwrap these layers of meaning before we can understand what is being measured.

Physicists use the term “accelerate” to mean any change in velocity. It is worth reminding ourselves that velocity (to the physicists) is actually a pair of quantities: a speed coupled with a direction. Given this definition, when an object changes velocity (accelerates), it can be changing its speed, its direction, or both. So a car can be said to be accelerating when speeding up, slowing down, or even turning while maintaining a speed. This is why the accelerometer could be used as an airbag sensor in the first place. The airbags should deploy when a car suddenly changes velocity in any direction, including getting faster (due to being hit from behind), getting slower (from a front impact crash) or changing direction (being hit from the side). It is because of this ability to measure changes in velocity that accelerometers have come into common usage for laptop drop sensors and video game motion controllers.

But even this understanding of accelerometers is incomplete. Because of the way that accelerometers are constructed, they actually measure “proper acceleration” within the context of a relativistic frame of reference. Discussing general relativity is beyond the scope of this paper, but it is sufficient to describe a relativistic frame of reference as one in which no forces are felt. A familiar example is being in orbit around the planet, when astronauts (and their equipment) float freely in space. A state of “free-fall” is one in which no forces are felt, and this is the only situation in which an accelerometer reads 0 acceleration. Since most of us are not in free-fall most of the time, any accelerometers in devices in normal use do not experience 0 proper acceleration, even when apparently sitting still. This is, of course, because of the force due to gravity.

An accelerometer sitting on a table experiences 1g of force from the table, acting against the gravitational acceleration. This non-zero reading for a stationary object is the reason that accelerometers can serve a second (and, today, much more common) use: measuring orientation with respect to gravity.

Gravity and Tilt

Accelerometers typically measure forces with respect to three linear dimensions, labeled x, y, and z. These three directions orient along the axes of the accelerometer chip itself, with x and y normally orienting along the long faces of the device, and the z direction often pointing through the face of the device. Relative motion within a gravity field can easily be inferred assuming that the only force acting on the device is gravity. In this case, the single force is distributed among the three axes depending on the orientation of the device. This is how personal smartphones and video game controllers are able to use “tilt” control. When held in a natural position, the software extracts the relative value on all three axes and uses that as a reference point. When the user tilts the device, the new direction of the gravitational acceleration is then compared to the reference value and used to infer the tilt. This can be done hundreds of times a second and can be used to control and augment any aspect of the user experience.

If, however, gravity is not the only force present, it becomes more difficult to infer orientation. Another common use for accelerometers is to measure physical activity like walking steps. In this case, it is the forces on the accelerometer from each footfall that are interpreted to measure fitness features. Tilt is unreliable in this circumstance because both gravity and the forces from the footfall are measured by the accelerometer, and it is impossible to separate the two forces from a single measurement.

Velocity and Position

A second common assumption with accelerometers is that since they can measure acceleration (rate of change of velocity), it should be possible to infer the velocity. If the device begins at rest, then any measured acceleration can be interpreted as changes to the velocity in some direction, thus inferring the new velocity. Although this is theoretically possible, real-world factors come in to play which prevent this from being realized. First, the assumption of beginning from a state of rest is not always reasonable. Further, if we don’t know whether the device is moving or not, knowing its acceleration at any moment will not help us to determine it’s new speed or position. The most important real-world problem, however, is that accelerometers typically show small variations even when the object is at rest. This is because of inaccuracies in the way that the accelerometer itself is interpreted. In normal operation, these small changes are ignored, but when trying to infer velocity or position, these little errors will quickly add up to the point where any inferred velocity or position would be unreliable. A common solution to these problems is in the combination of devices. Many new smartphones combine an accelerometer and a gyroscopes (a device which measures changes in rotational inertia) to provide a sensing system known as an IMU (Inertial measurement unit), which makes the readings from each more reliable. In this case, the gyroscope can be used to directly measure tilt (instead of inferring it from gravity) and this tilt information can be subtracted from the accelerometer reading to separate out the motion of the device from the force of gravity.

Augmentation Applications in Health, Gaming, and Art

Accelerometer-based devices have been used extensively in healthcare (Ward et al. 582), either using the accelerometer within a smartphone worn in the pocket (Yoshioka et al. 502) or using a standalone accelerometer device such as a wristband or shoe tab (Paradiso and Hu 165). In many cases, these devices have been used to measure specific activity such as swimming, gait (Henriksen et al. 288), and muscular activity (Thompson and Bemben 897), as well as general activity for tracking health (Troiano et al. 181), both in children (Stone et al. 136) and the elderly (Davis and Fox 581). These simple measurements are the first step in allowing athletes to modify their performance based on past activity. In the past, athletes would pour over recorded video to analyze and improve their performance, but with accelerometer devices, they can receive feedback in real time and modify their own behaviour based on these measurements. This augmentation is a competitive advantage but could be seen as unfair considering the current non-equal access to computer and electronic technology, i.e. the digital divide (Buente and Robbin 1743).

When video games were augmented with motion controls, many assumed that this would have a positive impact on health. Physical activity in children is a common concern (Treuth et al. 1259), and there was a hope that if children had to move to play games, an activity that used to be considered a problem for health could be turned into an opportunity (Mellecker et al. 343). Unfortunately, the impact of children playing motion controlled video games has been less than successful. Although fitness games have been created, it is relatively easy to figure out how to activate controls with the least possible motion, thereby nullifying any potential benefit.

One of the most interesting applications of accelerometers, in the context of this paper, is the application to dance-based video games (Brezmes et al. 796). In these systems, participants wear devices originally intended for health tracking in order to increase the sensitivity and control options for dance. This has evolved both from the use of accelerometers for gestural control in video games and for measuring and augmenting sport. Researchers and artists have also recently used accelerometers to augment dance systems in many ways (Latulipe et al. 2995) including combining multiple sensors (Yang et al. 121), as discussed above.

Conclusions

Although more and more people are using accelerometers in their research and art practice, it is significant that there is a lack of widespread knowledge about how the devices actually work. This can be seen in the many art installations and sports research studies that do not take full advantage of the capabilities of the accelerometer, or infer information or data that is unreliable because of the way that accelerometers behave.

This lack of understanding of accelerometers also serves to limit the increased utilization of this powerful device, specifically in the context of augmentation tools. Being able to detect, analyze and interpret the motion of a body part has significant applications in augmentation that are only starting to be realized.

The history of accelerometers is interesting and varied, and it is worthwhile, when exploring new ideas for applications of accelerometers, to be fully aware of the previous uses, current trends and technical limitations.

It is clear that applications of accelerometers to the measurement of human motion are increasing, and that many new opportunities exist, especially in the application of combinations of sensors and new software techniques. The real novelty, however, will come from researchers and artists using accelerometers and sensors in novel and unusual ways.

References

Brezmes, Tomas, Juan-Luis Gorricho, and Josep Cotrina. “Activity Recognition from Accelerometer Data on a Mobile Phone.” In Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living. Springer, 2009.

Buente, Wayne, and Alice Robbin. “Trends in Internet Information Behavior, 2000-2004.” Journal of the American Society for Information Science and Technology 59.11 (2008).

Chu, Narisa N.Y., Chang-Ming Yang, and Chih-Chung Wu. “Game Interface Using Digital Textile Sensors, Accelerometer and Gyroscope.” IEEE Transactions on Consumer Electronics 58.2 (2012): 184-189.

Davis, Mark G., and Kenneth R. Fox. “Physical Activity Patterns Assessed by Accelerometry in Older People.” European Journal of Applied Physiology 100.5 (2007): 581-589.

Hagstromer, Maria, Pekka Oja, and Michael Sjostrom. “Physical Activity and Inactivity in an Adult Population Assessed by Accelerometry.” Medical Science and Sports Exercise. 39.9 (2007): 1502-08.

Henriksen, Marius, H. Lund, R. Moe-Nilssen, H. Bliddal, and B. Danneskiod-Samsøe. “Test–Retest Reliability of Trunk Accelerometric Gait Analysis.” Gait & Posture 19.3 (2004): 288-297.

Latulipe, Celine, David Wilson, Sybil Huskey, Melissa Word, Arthur Carroll, Erin Carroll, Berto Gonzalez, Vikash Singh, Mike Wirth, and Danielle Lottridge. “Exploring the Design Space in Technology-Augmented Dance.” In CHI’10 Extended Abstracts on Human Factors in Computing Systems. ACM, 2010.

Mellecker, Robin R., Lorraine Lanningham-Foster, James A. Levine, and Alison M. McManus. “Energy Intake during Activity Enhanced Video Game Play.” Appetite 55.2 (2010): 343-347.

Paradiso, Joseph A., and Eric Hu. “Expressive Footwear for Computer-Augmented Dance Performance.” In First International Symposium on Wearable Computers. IEEE, 1997.

Rogers, Everett M. Diffusion of Innovations. New York: Free Press of Glencoe, 1962.

Stone, Michelle R., Ann V. Rowlands, and Roger G. Eston. "Relationships between Accelerometer-Assessed Physical Activity and Health in Children: Impact of the Activity-Intensity Classification Method" The Free Library 1 Mar. 2009.

Thompson, Christian J., and Michael G. Bemben. “Reliability and Comparability of the Accelerometer as a Measure of Muscular Power.” Medicine and Science in Sports and Exercise. 31.6 (1999): 897-902.

Treuth, Margarita S., Kathryn Schmitz, Diane J. Catellier, Robert G. McMurray, David M. Murray, M. Joao Almeida, Scott Going, James E. Norman, and Russell Pate. “Defining Accelerometer Thresholds for Activity Intensities in Adolescent Girls.” Medicine and Science in Sports and Exercise 36.7 (2004):1259-1266

Troiano, Richard P., David Berrigan, Kevin W. Dodd, Louise C. Masse, Timothy Tilert, Margaret McDowell, et al. “Physical Activity in the United States Measured by Accelerometer.” Medicine and Science in Sports and Exercise, 40.1 (2008):181-88.

Varat, Michael S., and Stein E. Husher. “Vehicle Impact Response Analysis through the Use of Accelerometer Data.” In SAE World Congress, 2000.

Walter, Patrick L. “The History of the Accelerometer”. Sound and Vibration (Mar. 1997): 16-22.

Ward, Dianne S., Kelly R. Evenson, Amber Vaughn, Anne Brown Rodgers, Richard P. Troiano, et al. “Accelerometer Use in Physical Activity: Best Practices and Research Recommendations.” Medicine and Science in Sports and Exercise 37.11 (2005): S582-8.

Yang, Chang-Ming, Jwu-Sheng Hu, Ching-Wen Yang, Chih-Chung Wu, and Narisa Chu. “Dancing Game by Digital Textile Sensor, Accelerometer and Gyroscope.” In IEEE International Games Innovation Conference. IEEE, 2011.

Yoshioka, M., M. Ayabe, T. Yahiro, H. Higuchi, Y. Higaki, J. St-Amand, H. Miyazaki, Y. Yoshitake, M. Shindo, and H. Tanaka. “Long-Period Accelerometer Monitoring Shows the Role of Physical Activity in Overweight and Obesity.” International Journal of Obesity 29.5 (2005): 502-508.


Keywords


Sensors; Movement; dance; games; sport; history; review



Copyright (c) 2013 David Gerhard

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

  • M/C - Media and Culture
  • Supported by QUT - Creative Industries
  • Copyright © M/C, 1998-2016
  • ISSN 1441-2616