Sebastian Büttner
Thomas Kosch
ABSTRACT
Research on how to take advantage of Augmented Reality and Virtual Reality applications and technologies in the domain of manufacturing has brought forward a great number of concepts, prototypes, and working systems. Although comprehensive surveys have taken account of the state of the art, the design space of industrial augmented and virtual reality keeps diversifying. We propose a visual approach towards assessing this space and present an interactive, community-driven tool which supports interested researchers and practitioners in gaining an overview of the aforementioned design space. Using such a framework we collected and classified relevant publications in terms of application areas and technology platforms. This tool shall facilitate initial research activities as well as the identification of research opportunities. Thus, we lay the groundwork, forthcoming workshops and discussions shall address the refinement.
- M. Aehnelt and S. Bader. From information assistance to cognitive automation: A smart assembly use case. In B. Duval, van den Herik, Jaap, S. Loiseau, and J. Filipe, editors, Agents and artificial intelligence, volume 9494 of Lecture Notes in Computer Science, pages 207--222. Springer, Cham and Heidelberg, 2015.Google Scholar
Cross Ref
- M. Aehnelt and K. Wegner. Learn but work! towards self-directed learning at mobile assembly workplaces. In S. Lindstaedt, T. Ley, and H. Sack, editors, Proceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business, pages 1--7. ACM Press, 2015. Google ScholarDigital Library
- R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre. Recent advances in augmented reality. IEEE computer graphics and applications, 21(6):34--47, 2001. Google ScholarDigital Library
- R. T. Azuma. A survey of augmented reality. Presence: Teleoperators and virtual environments, 6(4):355--385, 1997. Google ScholarDigital Library
- L. Baechler, A. Baechler, M. Funk, S. Autenrieth, G. Kruell, T. Hoerz, and T. Heidenreich. The Use and Impact of an Assistance System for Supporting Participation in Employment for Individuals with Cognitive Disabilities, pages 329--332. Springer International Publishing, Cham, 2016.Google Scholar
- A. Bannat, F. Wallhoff, G. Rigoll, F. Friesdorf, H. Bubb, S. Stork, H. Müller, A. Schubö, M. Wiesbeck, and M. F. Zäh. Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In Proceedings of the 1st international workshop on cognition for technical systems, Cotesys, 2008.Google Scholar
- F. Biocca, A. Tang, C. Owen, and F. Xiao. Attention funnel: omnidirectional 3d cursor for mobile augmented reality platforms. In Proceedings of the SIGCHI conference on Human Factors in computing systems, pages 1115--1122. ACM, 2006. Google ScholarDigital Library
- A. Boud, D. J. Haniff, C. Baber, and S. Steiner. Virtual reality and augmented reality as a training tool for assembly tasks. In Information Visualization, 1999. Proceedings. 1999 IEEE International Conference on, pages 32--36. IEEE, 1999. Google ScholarDigital Library
- A. C. Boud, D. J. Haniff, C. Baber, and S. J. Steiner. Virtual reality and augmented reality as a training tool for assembly tasks. In 1999 IEEE International Conference on Information Visualization (Cat. No. PR00210), pages 32--36, 1999. Google ScholarDigital Library
- F. P. Brooks. What's real about virtual reality? IEEE Computer graphics and applications, 19(6):16--27, 1999. Google ScholarDigital Library
- S. Büttner, M. Funk, O. Sand, and C. Röcker. Using head-mounted displays and in-situ projection for assistive systems: A comparison. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, page 44. ACM, 2016. Google ScholarDigital Library
- S. Büttner, O. Sand, and C. Röcker. Extending the design space in industrial manufacturing through mobile projection. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pages 1130--1133. ACM, 2015. Google ScholarDigital Library
- S. Büttner, O. Sand, and C. Röcker. Exploring design opportunities for intelligent worker assistance: A new approach using projection-based ar and a novel hand-tracking algorithm. In Proceedings of the 2017 European Conference on Ambient Intelligence. Springer International Publishing, 2017 (forthcoming).Google ScholarCross Ref
- T. P. Caudell and D. W. Mizell. Augmented reality: an application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, volume ii, pages 659--669 vol.2, Jan 1992.Google ScholarCross Ref
- C. D. Fehling, A. Mueller, and M. Aehnelt. Enhancing vocational training with augmented reality. In S. Lindstaedt, T. Ley, and H. Sack, editors, Proceedings of the 16th International Conference on Knowledge Technologies and Data-driven Business. ACM Press, 2016.Google Scholar
- S. Feiner, B. Macintyre, and D. Seligmann. Knowledge-based augmented reality. Communications of the ACM, 36(7):53--62, 1993. Google ScholarDigital Library
- P. Fite-Georgel. Is there a reality in industrial augmented reality? In Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on, pages 201--210. IEEE, 2011. Google ScholarDigital Library
- W. Friedrich. Arvika-augmented reality in Entwicklung, Produktion und Service. John Wiley & Sons, 2004.Google Scholar
- M. Funk, A. Bächler, L. Bächler, O. Korn, C. Krieger, T. Heidenreich, and A. Schmidt. Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '15, pages 1:1--1:8, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- M. Funk, T. Kosch, S. W. Greenwald, and A. Schmidt. A benchmark for interactive augmented reality instructions for assembly tasks. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, MUM '15, pages 253--257, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- M. Funk, T. Kosch, R. Kettner, O. Korn, and A. Schmidt. motioneap: An overview of 4 years of combining industrial assembly with augmented reality for industry 4.0.Google Scholar
- M. Funk, T. Kosch, and A. Schmidt. Interactive worker assistance: Comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp '16, pages 934--939, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- M. Funk, S. Mayer, M. Nistor, and A. Schmidt. Mobile in-situ pick-by-vision: Order picking support using a projector helmet. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, page 45. ACM, 2016. Google ScholarDigital Library
- M. Funk, S. Mayer, and A. Schmidt. Using in-situ projection to support cognitively impaired workers at the workplace. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, ASSETS '15, pages 185--192, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- M. Funk, A. S. Shirazi, S. Mayer, L. Lischke, and A. Schmidt. Pick from here!: an interactive mobile cart using in-situ projection for order picking. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 601--609. ACM, 2015. Google ScholarDigital Library
- N. Gavish, T. Gutiérrez, S. Webel, J. Rodríguez, M. Peveri, U. Bockholt, and F. Tecchia. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments, 23(6):778--798, 2015.Google ScholarCross Ref
- R. Geissbauer, J. Vedso, and S. Schrauf. Industry 4.0: Building the digital enterprise. PWC, 2016.Google Scholar
- W. A. Günthner. Pick-by-Vision: Augmented Reality unterstützte Kommissionierung. Lehrstuhl für Fördertechnik Materialfluss Logistik, 2009.Google Scholar
- A. Gupta, D. Fox, B. Curless, and M. Cohen. Duplotrack: A real-time system for authoring and guiding duplo block assembly. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST '12, pages 389--402, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- P. Gurevich, J. Lanir, B. Cohen, and R. Stone. Teleadvisor: a versatile augmented reality tool for remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 619--622. ACM, 2012. Google ScholarDigital Library
- M. Hakkarainen, C. Woodward, and M. Billinghurst. Augmented assembly using a mobile phone. In Mixed and Augmented Reality, 2008. ISMAR 2008. 7th IEEE/ACM International Symposium on, pages 167--168. IEEE, 2008. Google ScholarDigital Library
- O. Korn. Industrial playgrounds: How gamification helps to enrich work for elderly or impaired persons in production. In Proceedings of the 4th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS '12, pages 313--316, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- O. Korn, M. Funk, S. Abele, T. Hörz, and A. Schmidt. Context-aware assistive systems at the workplace: Analyzing the effects of projection and gamification. In Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA '14, pages 38:1--38:8, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- O. Korn, M. Funk, and A. Schmidt. Towards a gamification of industrial production: a comparative study in sheltered work environments. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pages 84--93. ACM, 2015. Google ScholarDigital Library
- T. Kosch, R. Kettner, M. Funk, and A. Schmidt. Comparing tactile, auditory, and visual assembly error-feedback for workers with cognitive impairments. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS '16, pages 53--60, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- M. Kritzler, M. Murr, and F. Michahelles. Remotebob: Support of on-site workers via a telepresence remote expert system. In Proceedings of the 6th International Conference on the Internet of Things, pages 7--14. ACM, 2016. Google ScholarDigital Library
- M. R. Marner, A. Irlitti, and B. H. Thomas. Improving procedural task performance with augmented reality annotations. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 39--48, Oct 2013.Google ScholarCross Ref
- S. Mattsson, Å. Fast-Berglund, and J. Stahre. Managing production complexity by supporting cognitive processes in final assembly. In Proceedings of the 6th Swedish Production Symposium (SPS) 2014. 2014.Google Scholar
- N. Navab. Developing killer apps for industrial augmented reality. IEEE Computer Graphics and applications, 24(3):16--20, 2004. Google ScholarDigital Library
- A. Nee, S. Ong, G. Chryssolouris, and D. Mourtzis. Augmented reality applications in design and manufacturing. CIRP Annals-manufacturing technology, 61(2):657--679, 2012.Google ScholarCross Ref
- U. Neumann and A. Majoros. Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998, pages 4--11. IEEE, 1998. Google ScholarDigital Library
- M. Otto, M. Prieur, and E. Rukzio. Using scalable, interactive floor projection for production planning scenario. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pages 363--368. ACM, 2014. Google ScholarDigital Library
- V. Paelke. Augmented reality in the smart factory: Supporting workers in an industry 4.0. environment. In Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA), pages 1--4, Sept 2014.Google ScholarCross Ref
- V. Paelke, C. Röcker, N. Koch, H. Flatt, and S. Büttner. User interfaces for cyber-physical systems. at-Automatisierungstechnik, 63(10):833--843, 2015.Google ScholarCross Ref
- G. Papagiannakis, G. Singh, and N. Magnenat-Thalmann. A survey of mobile and wireless technologies for augmented reality systems. Computer Animation and Virtual Worlds, 19(1):3--22, 2008. Google ScholarDigital Library
- K. Pentenrieder, C. Bade, F. Doil, and P. Meier. Augmented reality-based factory planning-an application tailored to industrial needs. In Mixed and Augmented Reality, 2007. ISMAR 2007. 6th IEEE and ACM International Symposium on, pages 31--42. IEEE, 2007. Google ScholarDigital Library
- J. Platonov, H. Heibel, P. Meier, and B. Grollmann. A mobile markerless ar system for maintenance and repair. In Mixed and Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on, pages 105--108. IEEE, 2006. Google ScholarDigital Library
- H. Regenbrecht, G. Baratoff, and W. Wilke. Augmented reality projects in the automotive and aerospace industries. IEEE Computer Graphics and Applications, 25(6):48--56, 2005. Google ScholarDigital Library
- R. Reif and D. Walch. Augmented & virtual reality applications in the field of logistics. The Visual Computer, 24(11):987--994, 2008. Google ScholarDigital Library
- T. Salonen and J. Sääski. Dynamic and visual assembly instruction for configurable products using augmented reality techniques. In Advanced Design and Manufacture to Gain a Competitive Edge, pages 23--32. Springer, 2008.Google ScholarCross Ref
- O. Sand, S. Büttner, V. Paelke, and C. Röcker. smart. assembly--projection-based augmented reality for supporting assembly workers. In International Conference on Virtual, Augmented and Mixed Reality, pages 643--652. Springer International Publishing, 2016.Google ScholarCross Ref
- B. Sarupuri, G. A. Lee, and M. Billinghurst. An augmented reality guide for assisting forklift operation. In Mixed and Augmented Reality (ISMAR-Adjunct), 2016 IEEE International Symposium on, pages 59--60. IEEE, 2016.Google ScholarCross Ref
- B. Schwald and B. De Laval. An augmented reality system for training and assistance to maintenance in the industrial context. 2003.Google Scholar
- B. Schwerdtfeger and G. Klinker. Supporting order picking with augmented reality. In Mixed and Augmented Reality, 2008. ISMAR 2008. 7th IEEE/ACM International Symposium on, pages 91--94. IEEE, 2008. Google ScholarDigital Library
- M. Speicher, K. Tenhaft, S. Heinen, and H. Handorf. Enabling industry 4.0 with holobuilder. In GI-Jahrestagung, pages 1561--1575, 2015.Google Scholar
- A. Tang, C. Owen, F. Biocca, and W. Mou. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '03, pages 73--80, New York, NY, USA, 2003. ACM. Google ScholarDigital Library
- D. Van Krevelen and R. Poelman. Augmented reality: Technologies, applications, and limitations. 2007.Google Scholar
- S. Webel, U. Bockholt, T. Engelke, N. Gavish, M. Olbrich, and C. Preusche. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems, 61(4):398--403, 2013. Google ScholarDigital Library
- J. Weidenhausen, C. Knoepfle, and D. Stricker. Lessons learned on the way to industrial augmented reality applications, a retrospective on arvika. Computers & Graphics, 27(6):887--891, 2003.Google ScholarCross Ref
- P. Wellner. Interacting with paper on the digitaldesk. Commun. ACM, 36(7):87--96, July 1993. Google ScholarDigital Library
- W. Wohlgemuth and G. Triebfürst. Arvika: augmented reality for development, production and service. In Proceedings of DARE 2000 on Designing augmented reality environments, pages 151--152. ACM, 2000. Google ScholarDigital Library
- X. Wu, M. Haynes, A. Guo, and T. Starner. A comparison of order picking methods augmented with weight checking error detection. In Proceedings of the 2016 ACM International Symposium on Wearable Computers, pages 144--147. ACM, 2016. Google ScholarDigital Library
- X. S. Zheng, C. Foucault, P. Matos da Silva, S. Dasari, T. Yang, and S. Goose. Eye-wearable technology for machine maintenance: Effects of display position and hands-free operation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pages 2125--2134, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
Index Terms
- The Design Space of Augmented and Virtual Reality Applications for Assistive Environments in Manufacturing: A Visual Approach
Recommendations
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User InteractionTwo major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Interactive Augmented Live Virtual Reality Streaming: A Health Care Application
ICMHI '18: Proceedings of the 2nd International Conference on Medical and Health InformaticsVirtual Reality (VR) technology has been around for decades; however, we have probably only begun to realize the practical applications until recent years. With increase of computing power, decrease of cost and physical dimension, a new class of ...
What is XR? Towards a Framework for Augmented and Virtual Reality
AbstractAugmented Reality (AR), Virtual Reality (VR), Mixed Reality, and Extended Reality (often – misleadingly – abbreviated as XR) are commonly used terms to describe how technologies generate or modify reality. However, academics and ...
Highlights- XR should not be used as an abbreviation for “Extended Reality”; X should be the placeholder for “all” new reality formats.
Comments