MET:Augmented Reality

From UBC Wiki

This page originally authored by Allan Cho (January, 2012). Edited 2013 by Garry Wong, 2014 by Elisabeth Tower, and 2018 by R. Surette.
Stop Motion Animation (January, 2015) by Noan Fesnoux New Stop Motion Animation (January, 2016) by Jamie Out

Businessman ponders URL


Augmented Reality Defined

Augmented reality (AR), first coined by Boeing researcher Tom Caudell in 1990, refers to a view of the world which has been changed, modified, or enhanced in some way through the use of computer generated visual overlays or audio cues [1]. In short, AR is when the real world is overlayed with the digital world, resulting in technology enhancing one's perception of reality. Jim Vallino of the Rochester Institute of Technology further clarifies this definition: "[AR] is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information.” [2]

This augmented view of the world is created by integrating multiple streams of data such as: live visual information captured through cameras, live audio information through microphones, geographical data through GPS, and information and data that is available on the Web.

The combined image is displayed to the user through a digital device such as computer, smartphone, Virtual Reality (VR) headset, projectors, or head-mounted heads-up display. Typically today, AR is used with mobile devices. New devices are being created specifically to facilitate AR, such as GoogleGlass, AR contact lenses and Sixth Sense Technologies [3].

For a video explanation of the concept, watch this.

Common Misconceptions

Although technologies are being created specifically in support of AR, rather than being a technology in and of itself, AR is simply a different view of the world provided by a combination of different technologies such as object recognition, visual computer procesing, GPS, cameras, microphones, head-tracking, and display technologies. Tim Cook, Apple CEO, stated in January, 2018, while visiting Canadian e-Commerce Shopify HQ, "I believe that AR is the most profound technology of the future. It amplifies human performance. It amplifies humans, not substitutes, and doesn't isolate. I'm a huge believer in it." [4]

Although AR is similar to Virtual Reality and uses similar technology and is often confused with virtual reality, it is not the same thing. AR exists on what Paul Milgram called the Reality-Virtuality Continuum. [5] 100% reality is at one end of the continuum with 100% virtuality at the other end. Between these two poles, lies mixed relality, which includes augmented virtuality (the virtual augmented with the real) and augmented reality (the real augmented with the virtual). Virtual reality lies close to the virtuality end of the continuum in which the real is simulated by the virtual.

The Evolution of AR Hardware Technologies

Smart Rooms and QR codes

Prior to developments such as smartphones, object recognition systems, and high resolution webcams, etc., AR could be displayed by setting up an area in which researchers can manipulate the environment so that the computer generating the view would be able to process the scene easily. For example, rooms would be outfitted with QR codes, and other forms of markers to provide visual anchors for computers so that they would not have to rely on object recognition technology which was rudimentary at the time. This can be compared to putting motion-capture points onto an actor so that the camera can recognize the joints of the actor.

Using cards or posters with special QR codes or other unique patterns are still commonly used today to create AR applications.

Another early, but nonetheless complex, approach to AR before the advent of smartphones involved the use of projectors being set up in a room to project onto certain surfaces. The projectors would display images at different viewpoints depending on the position of the viewer’s head which was tracked using a head-tracking system. This gave the viewer the illusion of digital three-dimensional objects existing in the real world; however, any other viewers would simply see distorted images being projected onto surfaces.

Desktop Applications

While most if not all AR applications are based on mobile technologies, early AR research took place on desktop computers which were the only devices with enough processing power to run AR applications.Today, there are a few desktop applications that incorporate elements of AR technologies are still useful.

For example, Google allows users on desktop computers to search the Web by uploading a photo. This is the same technology that drives Google Goggles, the mobile smartphone version of the searching tool.

The ARMedia plug-in for Google Sketch-up allows users to visualize 3D renderings in physical space through the use of a webcam.

Smartphones

Currently, due to the recent increase in mobile computing efficiency, large battery capacity, and decrease in size, smartphones are the predominant tool through which AR is displayed. There are many examples of different tools that incorporate AR technology.

Yelp has integrated an AR browser into it's iPhone application. By holding your smart phone’s camera so that it can see the street you’re facing, the Yelp.com AR browser can overlay business information onto the image of the street that is viewed on the phone.

Layar has been one of the early adopters using AR technology for Android and iPhone applications. Layar is a platform with which users can contribute their own AR overlays for each other to use. Overlays are pieces of data presented to the user through graphics which are laid ontop of a video being taken with the phone’s camera. Recently, Layar has moved on to enhancing print media with AR features. For example, one can look at a magazine through a print magazine, and a video will start playing ontop of an image on the print magazine.

Junaio is another mobile augmented reality browser with features that are much like Layar.

Ubiquitous Computing Hardware

The improvement of AR technology is closely tied to that of ubiquitous computing, computing technology that is wearable or pocketable and always on. For most of the last decade, ubiquitous computing was only partially realized through the development of smartphones. While many individuals have smartphones, they’re not quite ubiquitous and using them requires us to take them out of our pockets or hold them up to perform some action. Furthermore, batteries on a smartphone tend to require daily charging.

The idea of ubiquitous computing is that it is always at the ready; imagine the voice-activated computer from Star Trek, but worn on the body like a necklace, hat, or pair of glasses. This always-at-the-ready feature has yet to be successfully implemented in any consumer product. However, Google has made great strides in AR ubiquitous computing with projects such as Google Glass leading the push for truly wearable pieces of technology. Google Glass is worn on the users’ face like a pair of glasses. An image is projected to the user which is overlaid on top of the user’s realtime field of vision.

The device is voice activated and can be used to provide supplemental data on objects that the user sees such as weather, time, flight information, or descriptive data. In addition the device can take photos provide travel directions, share live video feeds, as well as other functions. For an example of what it might be like to use a device such as this watch this.

Educational Considerations for Augmented Reality

The most prevalent uses of augmented reality have been in the consumer sector for marketing, social engagement, amusement, or location-based information. New uses of this emerging technology is growing rapidly, as tools for creating new applications become ever easier to use. In its early inception, AR devices required unwieldy headsets that limited users largely immobile beyond their computer desktop stations. However, with the advent of the camera phone, AR has quickly evolved from the fringe of novelty gadgets to the forefront of business, technology, entertainment, and education[6]. Considered an evolution of virtual reality, the effects of augmented reality resemble virtual spaces as users can immerse themselves in an experience that extends beyond the boundaries of physical reality. As educators, we are primarily interested in the pedagogical and learning applications of AR technology.

Pedagogical and Learning Applications

While there has already been much experimentation with augmented reality, particularly by the American military and private commercial industries, augmented reality is still relatively early in its development as an educational technology. The effect of AR and ubiquitous computing on eLearning is not yet known. However, if we look at past technological paradigm shifts such as desktop computing, high-speed internet, laptops, and smartphones/tablets, we notice enormous changes in education along with each technological jump. In rural areas, the full effects of high-speed internet are still influencing educational policy developed today and we are just starting looking at mobile technology. It stands to reason then, that the effect of a wearable AR system like Google Glass will be just as large.

Within the Horizon Report, a research report published annually by the education think tank Educause that charts the landscape of emerging technologies for teaching, learning and creative inquiry, Johnson et al. (2011) has forecast in their 2010 and 2011 reports that augmented reality as an educational technology could play a prevalent role in education, with applications pertinent in subject areas such as geography, chemistry, and history[7]. Specifically, augmented reality provides a powerful constructivist experience for exploration and discovery of the connected nature of information in the physical world[8]. Bucholz and Brosda (2011) believe that most recent technical advances, especially in the areas of Tangible User Interfaces (TUI) and Augmented Reality (AR) have very high potential to improve constructivist learning environments[9]. It also aligns with situated learning in that it that permits experimentation and exploration to take place in the same context in which the activity occurs[10].

Augmented reality also has much potential for serious gaming applications, as it adds an element of game simulation in the application. Bicen & Bal (2016) found that using AR gamification methods in the classroom, AR content positively affected student opinions, perception of independence in the classroom, and overall enjoyment. [11]

AR technologies can also be used to enhance existing media such as textbooks. For example, by pointing a camera equipped smartphone at a printed textbook embedded with AR markers, the smartphone can load and play a video on top of the textbook. This brings together the best of both print and digital media as the uptake on digital textbooks is still low[12]. The use of AR in education is in its infancy. Kaufmann (2003) admits that much research has yet to be done in evaluating the efficacy, benefits, or advantages to using AR in education[13]. He proposes that the following aspects of AR must be evaluated in order to better understand the educational efficacy of such technologies:

  • The technical aspect including usability issues, interface, physical problems, and hardware/software considerations.
  • The orientation aspect focusing on the relationship between user and environment, spatial considerations, immersion, and feedback issues.
  • The affective parameters including emotion, engagement, likes and dislikes of the user.
  • The cognitive aspect of any improvement in learning as a result of AR use.
  • The pedagogical aspect of how to best teach using these technologies.
  • Two pedagogical concepts that AR technology is closely related to are constructivism and situated learning.

Constructivism and Augmented Reality

Augmented reality technologies build on the on the value of exploratory learning, as the unstructured nature of augmented reality allows for learners to construct knowledge by making connections between information and their own experiences within a social and collaborative framework[14]. Augmented reality facilitates this process by offering virtual spaces where there are opportunities to learn and use a variety of web 2.0 tools to facilitate their learning online. In short, AR gives the student total control over their own learning environment. They are free to manipulate virtual objects, problem sets, situations, and virtual environment as they please without the consequences of failure such as interfering with the learning environments of other students’ in the real world. Specifically, Wang (2012) expands on this idea with the example of learning high-consequence skills such as surgery, firefighting, and heavy equipment operation[15]. The student is free to explore, test, and make mistakes without the worry of injuring themselves, the environment, or others. Another possible benefit of AR in education is easily creating Constructivist Learning Environments which are customized to each student. Jonassen (1999) states that Constructivist Learning Environments are education environments that facilitate the individual students’ construction of knowledge.[16] By using AR, students are more free to progress, digress, and explore. Wang (2012) states that AR “is well aligned with constructivist notions of education where learners control their own learning, through the active interactions with the real and virtual environments.”[17]

Situated Learning and Augmented Reality

As learning takes place with the environment encompassing a significant role in the processes of knowing and learning, the environment simultaneously constrains activity, affords particular types of activity or performance, and supports performance. As learners using augmented reality devices are free to move throughout the physical world, novel opportunities exist for learners to interact with the physical environment, literally reading the landscape as they conduct environmental investigations or historical studies[18]. This allows students to build connections between their lives and their education through the addition of a contextual layer[19]. In short, it becomes possible to situate the student in any environment that is relevant to the learning objectives. For example, a student learning about procedures in a lab can be virtually placed in that lab and explore as they please. Dede (2009) provides and example scenario where AR technology is used to create such a situated learning[20]. Dede (2009) placed students equipped with GPS-enabled handheld computers in the field. The students could move around in the real world and the handheld computers would be able to calculate the students position in comparison to geotagged pieces of data such as field data and virtual characters which could be interviewed in order to collaboratively investigate a simulated learning scenario. Liestol (2011) eloquently states that the benefit of using AR technologies is that “it [becomes] possible to support and extend the “situatedness” of learning and education in new ways.”[21]

Educational Applications of AR

MIT Education Program

The MIT Teacher Education Program has used "Augmented Reality" simulations to engage people in simulation games that combine real world experiences with additional information supplied to them by handheld computers. One of these AR games is Environmental Detectives (ED), an outdoor game in which players using GPS guided handheld computers try to uncover the source of a toxic spill by interviewing virtual characters and conducting large scale simulated environmental measurements and analyzing data. Used at three sites, including MIT, a nearby nature center, and a local high school. Early research has shown that this mode of learning is successful in engaging university and secondary school students in large scale environmental engineering studies, and providing an authentic mode of scientific investigation.

Georgia Institute of Technology's Augmented Reality Lab

Although the Augmented Reality Lab has been working with AR technologies since 1998, its current work is focused on handheld AR experiences and games, mobile AR, the interaction between online virtual worlds and AR, tracking and sensing for mobile AR, as well as in the support of business collaboration. Augmented-reality scratch is the first augmented-reality authoring environment designed for children. By adding augmented-reality functionality to the Scratch programming platform, this environment allows pre-teens to create programs that mix real and virtual spaces.

Alternative Education and AR

Outside of public classroom education, alternative education sites such as libraries, museums, parks, and cemetaries have been exploring the use and implementation of augmented reality to meet the educational needs of their publics. While museums in particular have a difficult ethical relationship with AR, they are increasingly exploring its use for education within and outside of its exhibits. AR allows museums to reach new audiences and to show audiences artifacts that they would not otherwise be able to interact with due to preservation concerns. On the other hand, museums, whose power and authenticity derive from exhibiting the 'real' may find themselves rethinking their role in an increasingly digital age.

Museums and other alternative education sites have moved towards a dialogic paradigm in which the visitor is understood to have useful knowledge that they bring with them and that meaning must be created not in a top down transmission model, but rather in a more constructivist approach in which the museum constructs experiences and conversations that it can have with its audience rather than maintaining separate non-interactive spheres. As part of this new methodology and theory, museums are increasingly looking for ways in which it can meaningfully participate with its visitors. Given that education and preservation are the two key objectives of a museum the impact that AR can have on assisting museums with meeting both of these needs simultaneously while also engaging and participating with their audience is incredible.

Example Alternative Education AR

Streetmuseum

The Museum of London undertook a project to overlay the streets of London, England with historical images from their archival collection using GPS located AR. Users see historical images of streetscapes while situated in the modern city allowing them to explore historical concepts of continuity and change. Streetmuseum

WolfWalk

Similar to the Street Museum, WolfWalk, created by NCSU Libraries, facilitates a historical exploration of the NCSU campus highlighting more than 1000 images of important people, places and events geolocated on the campus map. See more here Wolfwalk

The Darwin Centre

The Darwin Centre at the Natural History Museum in London has created an AR interactive film that allows users to input their own information and images which in turn are layered into the film. 3D renderings of ancient man walk through the modern theatre visible through handheld devices.

Londinium

The Museum of London, after the success of their Streetmuseum project, launched Londinium which allows users to see a map of the ancient roman city over the modern capital. Users see hidden archealogical finds within the modern cityscape.

ROM Dinosaurs

The Royal Ontario Museum developed AR to go with its dinosaur exhibit Giants of Gondwana[1]. This application allows visitors to put the flesh back on the bone, to see what dinosaurs may have looked like, behaved and sounded in their natural environment.

Halifax Explosion in AR

A research group is using AR to tell stories of the Halifax Explosion for the 100th anniversary of the disaster. GEM Dal Projects News Article

Example Tools and Resources

ZooBurst - A tool to create 3D AR pop-up books.

Google Skymap - An augmented reality map of the sky.

Google Goggles - Using image searching, this app allows users to see the different kinds of objects and places by scanning over the object

Layar - Browse thousands of layers on the world's leading mobile augmented reality platform

junaio - junaio allows one to create, explore and share information in a completely new way using augmented reality.

Wikitude - Wikitude points you to Wikipedia entries near users using geo-location

Augmented Car Finder - Augmented Car Finder uses Augmented Reality to help trace steps back to where users parked.

Star Wars: Falcon Gunner - Turn on your the camera on your iPhone or iPod Touch, watch the screen and users will notice that spaceships are flying towards them. The only way to survive is to shoot them down.

Word Lens - Word Lens can translate words and manipulate images - so that means that road signs, menus and everything else that do not have to be interpreted in confusion again.

Aurasma - Open-source tool which allows users to engage in and create AR experiences of their own.

AR Concerns

Copyright Implications

Recently, there has been discussions among legal and copyright experts about augmented reality virtually "creating" limitless sets of digital art and animations on top of existing original art pieces. Although the copyrighted image in question would be unaltered and unnoticeable to the unaided eye, the very "act" of viewing that code through an AR device that renders (or reproducing, depending on the circumstances) the digital image or animation has potential copyright implications. As “contributory infringement” exists in legal terms in the United States when one intentionally cause others to infringe copyrights, this could prove to be seeds to future legal battles to come when augmented reality becomes more mainstream.[22]

Space Hacking

Various artists and activists have sought to use AR to subvert dominant narratives in our society by overlaying art, space, logos, buildings and labels with augmented content. This content is not sanctioned by the original property owner. Some property owners have embraced this unsanctioned introducation of virtual commentary into their public space. See MoMA NYC augmented reality exhibition. However there remains significant potential for unwanted augmentation of private property including augmented adbusting of logos, labels, advertising campaigns and buildings. Examples include book critiques augmented on book jackets, information related to the labour or environmental conditions of particular products overlaid onto their packaging or restaurant reviews directly on storefronts.

Data and Unanticipated Costs

When users encounter AR in public sites or at educational institutions, they may be unaware of or unprepared for the associated data costs leaving AR providers responsible for either providing wifi access to limit data costs or providing warnings to users about the type and size of content they are about to connet with thereby limiting some of the serendipidous nature of the learning and discovery that makes AR so powerful. This is particularly important when the AR application may unexpectedly open large image files, sound or video files or 3D rendered files which use significant quantities of data. While providing wifi access may be an easy solution in some cases, this is not possible in large installations such as the Streetmuseum.

Sacred Space

Ethical debates exist about the use of AR in sacred spaces. For example, using AR in cemeteries to overlay tombstones with additional information about the person burried there or a video or messages from loved ones like this one at Geary St. Cemetery in Halifax NS. While on one hand the use of AR in sacred space preserves the physical space without interuption from other physical types of information augmentation, on the other hand it does introduce a type of virtual interference in the experience of the site.

Accessibility

AR depends on the use of a device in order to read the layered content within the real space. This is usually a mobile device but could be other devices as previously mentioned. While many AR applications rely on a BYOD model, this works poorly for educational applications where consistency of learning is critical. Different devices have different capacities and speeds and students who do not have devices are excluded from participation.

Implications for Teachers Adopting AR

Dunleavy, Dede, & Mitchell (2009), caution that there are entirely new challenges, namely technological, managerial, and cognitive, placed upon teachers to harness and implement AR in their classrooms to the benefit of learners.[23]

Stop Motion Video Links

AR Stop Motion Video by R. Surette for ETEC510 65A, Jan 2018

References

  1. ↑ Lee, K. (2012). Augmented Reality in Education and Training. TechTrends, 56(2), 13–21. doi:10.1007/s11528-012-0559-3
  2. ↑ Vallino, Jim. (2002). Interactive Augmented Reality. Rochester, NY. Retrieved from www.se.rit.edu/~jrv/research/ar/introduction.html
  3. ↑ Pranav, Mistry. (2009). The Thrilling Potential of Sixthsense Technology. India: TED Talks. Retrieved from http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense_technology
  4. ↑ Special to the National Post (2018, January 23). Tim Cook’s visit to Shopify all about augmented reality, as Apple CEO praises ‘profound’ emerging technology. National Post. 23 Jan, 2018. Retrieved from http://www.nationalpost.com/cook+visit+shopify+about+augmented+reality+apple+praises+profound+emerging+technology/16845910/story.html
  5. ↑ Milgram, Paul and Kishino, Fumio. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems, Vol E77-D, No.12 Retrieved from http://etclab.mie.utoronto.ca/people/paul_dir/IEICE94/ieice.html
  6. ↑ Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium. Retrieved from http://net.educause.edu/ir/library/pdf/HR2011.pdf
  7. ↑ Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium. Retrieved from http://net.educause.edu/ir/library/pdf/HR2011.pdf
  8. ↑ Jonassen, D., Cernusca,D., Ionas,I. 2006. Constructivism and instructional design: The emergence of the learning sciences and design research. Trends and Issues in Instructional Design and Technology, Eds.R. Reiser and J. Dempsey. Columbus, OH: Merrill/Prentice-Hall.
  9. ↑ Bucholz, H., & Brosda, C. (2011). Melting Interfaces - Learning in Mixed Realities: A retrospective on transitional objects. In Augmented Reality in Education: Proceedings of the “Science Center to Go” Workshops (pp. 63–72). Presented at the Open Classroom Conference, Athens, Greece: Ellinogermaniki Agogi. Retrieved from http://www.sctg.eu/materials/sctgo_proceedings_low.pdf
  10. ↑ Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium. Retrieved from http://net.educause.edu/ir/library/pdf/HR2011.pdf
  11. ↑ Bicen, H., & Bal, E. (2016) Determination of Student Opinions in Augmented Reality. World Journal on Educational Technology: Current Issues, 8(3), 205-209. Retrieved from https://files.eric.ed.gov.fulltext/EJ1141852.pdf
  12. ↑ Boezi, M. (2013, February 22). Digital Textbooks: Publishers and the unrealized promise. Publishing Perspectives. Retrieved from http://publishingperspectives.com/2013/02/digital-textbooks-publishers-and-the-unrealized-promise/
  13. ↑ Kaufmann, H. 2003. Collaborative Augmented Reality in Education, Position paper for keynote speech at Imagina 2003 conference, Feb. 3rd, 2003. Imagina03. from http://www.ita.mx/files/avisos-desplegados/ingles-tecnico/guias-estudio-abril-2012/articulo-informatica-1.pdf
  14. ↑ Kaufmann, H. 2003. Collaborative Augmented Reality in Education, Position paper for keynote speech at Imagina 2003 conference, Feb. 3rd, 2003. Imagina03. from http://www.ita.mx/files/avisos-desplegados/ingles-tecnico/guias-estudio-abril-2012/articulo-informatica-1.pdf
  15. ↑ Wang, X. (2012, October). Augmented Reality: A new way of augmented learning. eLearn Magazine: Education and Technology in Perspective. Retrieved from http://elearnmag.acm.org/featured.cfm?aid=2380717
  16. ↑ Jonassen, D., 1999. Designing Constructivist Learning Environments, In C.M. Reigeluth (Ed.) Instructional-Design Theories and Models: A New Paradigm of Instructional Theory (Volume II) (pp. 215-239). New Jersey: Lawrence Erlbaum Associates.
  17. ↑ Wang, X. (2012, October). Augmented Reality: A new way of augmented learning. eLearn Magazine: Education and Technology in Perspective. Retrieved from http://elearnmag.acm.org/featured.cfm?aid=2380717
  18. ↑ Squire, K. & Klopfer, E. (2007). Augmented reality simulations on handheld computers. Journal of the Learning Sciences, 16, 3, 371–413.
  19. ↑ Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium. Retrieved from http://net.educause.edu/ir/library/pdf/HR2011.pdf
  20. ↑ Dede, C. (2009). Immersive Interfaces for Engagement and Learning. Science, 323, 66–69.
  21. ↑ Liestol, G. (2011, January). Learning through Situated Simulations: Exploring mobile augmented reality. Educause. Retrieved from http://net.educause.edu/ir/library/pdf/erb1101.pdf
  22. ↑ Hainich, H., 2009. The End of Hardware. Charleston: Booksurge Publishing.
  23. ↑ Dunleavy, M., Dede, C., and Mitchell, R. (2009). Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning. Journal of Science and Educational Technology, 18, 7-22. Retrieved from http://www.academia.edu/17258054/Affordances_and_Limitations_of_Immersive_Participatory_Augmented_Reality_Simulations_for_Teaching_and_Learning
Retrieved from "https://wiki.ubc.ca/index.php?title=MET:Augmented_Reality&oldid=680943"