Tactile Perception of Textiles in a Virtual-Reality System (Cognitive Systems Monographs)

Cognitive Systems Monographs Volume 10 Editors: Rüdiger Dillmann · Yoshihiko Nakamura · Stefan Schaal · David Vernon D...

Author: Dennis Allerkamp

31 downloads 574 Views 2MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Cognitive Systems Monographs Volume 10 Editors: Rüdiger Dillmann · Yoshihiko Nakamura · Stefan Schaal · David Vernon

Dennis Allerkamp

Tactile Perception of Textiles in a Virtual-Reality System

ABC

Rüdiger Dillmann, University of Karlsruhe, Faculty of Informatics, Institute of Anthropomatics, Humanoids and Intelligence Systems Laboratories, Kaiserstr. 12, 76131 Karlsruhe, Germany Yoshihiko Nakamura, Tokyo University Fac. Engineering, Dept. Mechano-Informatics, 7-3-1 Hongo, Bukyo-ku Tokyo, 113-8656, Japan Stefan Schaal, University of Southern California, Department Computer Science, Computational Learning & Motor Control Lab., Los Angeles, CA 90089-2905, USA David Vernon, Khalifa University Department of Computer Engineering, PO Box 573, Sharjah, United Arab Emirates

Author Dipl.-Math. Dennis Allerkamp Gottfried Wilhelm Leibniz Universität Hannover Institut für Mensch-Maschine-Kommunikation Fachgebiet Graphische Datenverarbeitung Welfengarten 1 30167 Hannover Germany E-mail: [email protected]

ISBN 978-3-642-13973-4

e-ISBN 978-3-642-13974-1

DOI 10.1007/978-3-642-13974-1 Cognitive Systems Monographs

ISSN 1867-4925

Library of Congress Control Number: 2010929202 c

2010 Springer-Verlag Berlin Heidelberg

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable for prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 543210 springer.com

Preface

This monograph originates from my work on the HAPTEX project. In December 2004 Prof. Franz-Erich Wolter, the head of the Institute of Man– Machine Communication of the Leibniz Universit¨ at Hannover, oﬀered me the opportunity to participate in that EU funded project. Being a mathematician I had only very little experience in the ﬁeld of haptic simulation in those days, but Prof. Wolter trusted in my ability to become acquainted with new ﬁelds of research in a very short time. I am still thankful for the conﬁdence he has shown me. Since then I indeed learned and found out a lot. With this monograph I try to pass on the knowledge I gained. Having a reader in mind who—like me at the beginning of the project—has no background in psychophysics, neurophysiology or textile engineering I will provide the necessary basics. The skilled reader may safely skip these parts. Nevertheless I presume some basic knowledge in mathematics. I hope that this thesis might help a newcomer to discover the fascinating ﬁeld of tactile simulation. This work would not have been possible without the funding of the project “HAPtic sensing of virtual TEXtiles” (HAPTEX) under the Sixth Framework Programme (FP6) of the European Union (Contract No. IST-6549). The funding is provided by the Future and Emerging Technologies (FET) Programme, which is part of the Information Society Technologies (IST) programme and focuses on novel and emerging scientiﬁc ideas. For inventing this project and accordingly requesting the funding I thank Prof. Nadia Magnenat-Thalmann, Dr. Harriet Meinander, Prof. Franz-Erich Wolter, Dr. Ian Summers and P.Eng. Fabio Salsedo. Special thanks go to my thesis advisor Prof. Franz-Erich Wolter for his guidance and support throughout my work on my PhD thesis which has greatly beneﬁted from all his feedback. I hope I inherited some of his good sense for interesting and profound research. My research was inﬂuenced a lot by Dr. Ian Summers who originally worked on the subject of tactile simulation. He not only provided several

VI

Preface

tactile displays for the experiments but also shared a lot of his knowledge and experience. I really appreciate I was able to work with him. I also thank Prof. Nadia Magnenat-Thalmann for the coordination of the HAPTEX project. She always applied very high standards to our research work. This was sometimes very stressful but the project would not have been nearly as successful as without her leadership. I appreciate her willingness to be a member of my examination committee. I really enjoyed the interdisciplinary collaboration within the HAPTEX project. I thank all the team members for the interesting discussions and the positive working atmosphere. I would like to give a special mention to Guido B¨ottcher; we worked closely together on the haptic rendering software and complemented each other very well. It was a pleasure to work with him. I had the pleasure of working with some very talented students who contributed to my research: I thank (in alphabetical order) Steﬀen Blume, Daniel Gl¨ockner, Tjard K¨ obberling, Karin Matzke and Natalya Obydenna. I also thank Steﬀen Blume, Wiebke Frey and Iris Lieske for proofreading the manuscript and for the helpful discussions. The experiments conducted for my research were only possible because numerous persons voluntarily participated as probands. I thank all of them for their support. Finally, I really appreciate the moral support of my colleagues, friends and especially of my family. Thank you! Hannover, March 2010

Dennis Allerkamp

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 4

2

Human Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction to Psychophysics . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Just Noticeable Diﬀerences and Weber’s Law . . . . . . . 2.1.2 Weber-Fechner Law and Stevens’ Power Law . . . . . . . . 2.1.3 Measuring Psychometric Distances . . . . . . . . . . . . . . . . 2.1.4 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.5 Signal Detection Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.6 Multidimensional Scaling . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 The Human Nervous System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Neural Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Sensory Reception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Human Tactile Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Mechanoreceptive Aﬀerents . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Perception of Form and Texture . . . . . . . . . . . . . . . . . . . 2.3.3 Perception of Fine Textures . . . . . . . . . . . . . . . . . . . . . . . 2.3.4 Perceptional Model of Tactile Simulation . . . . . . . . . . . 2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 5 6 7 8 9 10 12 15 15 17 17 18 18 19 20 20 22 22

3

Devices for Tactile Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Survey of Existing Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Electromagnetic Displays . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Pneumatic Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Displays with Shape Memory Alloys . . . . . . . . . . . . . . . 3.1.4 Piezoelectric Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25 25 26 27 27 27

VIII

Contents

3.1.5 Other Actuator Mechanisms . . . . . . . . . . . . . . . . . . . . . . 3.2 The Tactile Displays Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 The Piezoelectric Eﬀect . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Mechanical Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Geometrical Conﬁgurations . . . . . . . . . . . . . . . . . . . . . . . 3.3 Drive Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 USB Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Data Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Variable Voltage Supply . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4 Digital–Analogue Conversion . . . . . . . . . . . . . . . . . . . . . 3.3.5 Ampliﬁcation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Force-Feedback Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 The Haptic Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Modeling the Contact Forces . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Diﬀerent Degrees of Freedom . . . . . . . . . . . . . . . . . . . . . 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

28 28 29 30 31 33 34 37 39 39 41 42 43 43 44 44 45

4

Generation of Virtual Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Sample Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 Fibres, Yarn and Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Selection of Sample Fabrics . . . . . . . . . . . . . . . . . . . . . . . 4.2 Kawabata Evaluation System for Fabrics . . . . . . . . . . . . . . . . . 4.2.1 KES-F Roughness Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 KES-F Friction Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Spatial Frequency Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Selection of Appropriate Sections . . . . . . . . . . . . . . . . . . 4.3.2 Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Two-Dimensional Composition . . . . . . . . . . . . . . . . . . . . 4.4 The Correlation–Restoration Algorithm . . . . . . . . . . . . . . . . . . 4.4.1 Data Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Two-Dimensional Composition . . . . . . . . . . . . . . . . . . . . 4.5 Surface Reconstruction from an Optical Surface Scan . . . . . . 4.5.1 Symmetry Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Shape from Shading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Artiﬁcial Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Brownian Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

49 49 50 52 53 54 55 56 57 58 60 61 62 63 64 65 67 68 69 72 73

5

Tactile Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Rendering Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Position Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Experiment with a Force-Feedback Device . . . . . . . . . . . . . . . . 5.2.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75 76 77 80 80

Contents

5.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Vibrotactile Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Computation of Resulting Vibrations . . . . . . . . . . . . . . 5.3.2 Decomposition of Vibrations into Base Frequencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 First Results with Brownian Surfaces . . . . . . . . . . . . . . 5.3.4 Further Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Integration with a Force-Feedback Device . . . . . . . . . . . . . . . . . 5.4.1 Physical Simulation and Haptic Rendering . . . . . . . . . . 5.4.2 Integrated Interface Hardware . . . . . . . . . . . . . . . . . . . . 5.4.3 Software Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.4 Evaluation of the Integrated System . . . . . . . . . . . . . . . 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

IX

81 85 85 86 88 89 91 93 94 95 96 98 99

6

Summary and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

A

Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Acronyms

ACM CRA DAC DFT DIP DOF FFT fps IC IPC jnd KES-F lpi MDS OPA PC RA rms SA1 SA2 SFM SFS USB VR

autocorrelation matrix correlation–restoration algorithm digital–analogue converter discrete Fourier transform dual in-line package degrees of freedom fast Fourier transform frames per second integrated circuit inter-process communication just noticeable diﬀerence Kawabata evaluation system for fabrics lines per inch multidimensional scaling operational ampliﬁer Pacinian mechanoreceptive aﬀerent rapidly adapting mechanoreceptive aﬀerent root mean square slowly adapting type 1 mechanoreceptive aﬀerent slowly adapting type 2 mechanoreceptive aﬀerent statistical feature matrix shape from shading Universal Serial Bus virtual reality

Chapter 1

Introduction

Virtual Reality (VR) has a lot of applications ranging from entertainment to mechanical design and medical training. VR systems are often used in training situations where training in a real environment would be inappropriate and possibly even dangerous. Pilots, for example, often practise on a flight simulator before flying with a new type of aeroplane. Interestingly, flight simulators are also sold as games for personal computers. Today, computer games are probably the most common VR systems. VR systems can be categorised by the modalities they support. In today’s systems the modalities of seeing and hearing are the most commonly employed as these are also the modalities in which we as human beings mostly exchange information. They require least effort in terms of energy transfer, the corresponding sensory receptors are concentrated in the retina and the cochlea and can be excited remotely with light and sound waves respectively. In contrast to seeing and hearing the creation of appropriate haptic stimuli demands very sophisticated hardware. Firstly, the skin with its size of 1.5–2 m2 is a very large organ. Therefore most haptic devices focus on a rather small part of the human body—usually the fingertip. Secondly, forces cannot be transmitted contactfree with current technology. Thus haptic devices always need direct contact to the parts of the skin where the forces are applied. Thirdly, the amount of energy is relatively high compared to other modalities, e.g. if one wants to simulate the lifting of an object with a mass of 500 g the haptic device has to create a force of approximately 5 N. All these properties make haptic simulation a complex task still presenting a lot of problems awaiting a good solution. Today’s VR systems provide haptic interaction with virtual objects only indirectly via tools like a thimble or a pen-like probe or more specialised tools like, for example, the yoke of a plane, the handle of a scalpel or the handles of a pair of scissors. These tools usually appear twice in a VR system: as end effector transmitting the forces from the haptic device and as a virtual representation of the tool in the simulated virtual environment. The VR system is responsible of matching the physical state of the end effector and its virtual representation. Therefore, these tools can be seen as link between the real and the virtual world. However, this rather simple solution has two drawbacks. Firstly, special end effectors have to be designed for D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 1–4. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

2

1 Introduction

different applications, e.g. an end effector imitating the handle of a scalpel is probably useless for flight training of aviators. Secondly, it is not possible to directly touch the virtual objects with the hand, which reduces the possible realism of a VR system. In order to simulate direct touch the parts of the skin that are in contact with virtual objects have to be appropriately stimulated with a tactile display. Although there exists a large variety of tactile displays only very little research has been done on the tactile simulation of real objects. The latter is the topic of the work at hand. In order to further narrow the research question, the virtual objects to be simulated have been restricted to textiles and only one of the many different types of tactile displays was investigated. This work was part of the EU funded HAPTEX project, which aimed at developing a VR system for the visual and haptic presentation of textiles (cf. [4, 1, 2]). The project was coordinated by the MIRALab at the University of Geneva which also contributed the physical based simulation system of the fabrics. The Biomedical Physics Group at the University of Exeter developed the tactile stimulator hardware and was responsible for the multimodal integration and validation. The PERCRO Laboratory at the Scuola Superiore di Studi Universitari e di Perfezionamento Sant’Anna in Pisa developed the force-feedback hardware which also carried the tactile stimulator hardware. The SmartWearLab at the Tampere University of Technology provided a selection of fabrics together with measurements appropriate for the project. And the Welfenlab at the Leibniz Universit¨at Hannover developed the haptic rendering software computing appropriate signals for the force feedback and the tactile devices. The target scenario of the project is depicted in Fig. 1.1. The virtual cloth is attached to a fixed stand. The user can touch, squeeze, rub and stretch the fabric with the thumb and index finger, feeling appropriate tactile stimulation at the fingertips. Reaching the target scenario is a very challenging task, because force-feedback and tactile simulation have to be integrated, posing more than only mechanical Fig. 1.1 HAPTEX final scenario from [1] (courtesy of PERCRO Laboratory)

1 Introduction

3

difficulties. Therefore, an intermediate scenario has been defined as a test bed for the tactile rendering. Here the fabric is completely fixed on a rigid support. The user can tactually explore the fabric’s surface with one fingertip. Force feedback is only available to define the plane of the surface. This research has been conducted at the Welfenlab of the Leibniz Universit¨at Hannover. The workgroup can draw upon many years of experience in the field of haptics in VR environments. Already in the Nineties a cooperation with the Virtual Reality Research Laboratory at the Volkswagen AG in Wolfsburg existed. From this cooperation the doctoral theses [3] and [5] emerged. Figure 1.2 shows the structure of this monograph. Chapter 1 provides an introduction to the problem tackled in this work and describes the context of each chapter. Chapters 2, 3 and 4 build the foundation for the following chapters. In Chap. 2 human perception is described from two different angles: in psychophysics perception is regarded as a black box and reactions to stimuli are

!"#$%&'( )*$&+,-.$/+*

!"#$%&': ;-<"*'=%&.%#$/+*

!"#$%&'C 1%D/.%2'@+& 7".$/4%'B/<-4"$/+*

!"#$%&'6 7".$/4%'8%*,%&/*9

!"#$%&'0 1/2.-22/+*'"*,'3-$4++5

Fig. 1.2 Structure of this monograph

!"#$%&'> ?%*%&"$/+*'+@ A/&$-"4'B-&@".%2

4

1 Introduction

investigated whereas in neurophysiology the functionality of single neurons is regarded. The former provides methods for the evaluation of the tactile renderers developed within this work and the latter provides more insight into human perception and leads to a theoretical model that builds the theoretical frame of the research described in this book. Finally, tactile perception is described in more detail. In Chap. 3 an overview of existing tactile displays is given and the display used within this work is presented. In order to generate appropriate electric signals for the tactile display an electronics module had to be developed, which is also described in the chapter. Since also force-feedback devices are used in Chap. 5 an introduction to this class of devices is given. The generation of virtual surfaces is the topic of Chap. 4. As the virtual surfaces are intended to be representations of real fabrics the production and measurement of fabrics is treated first. Then three different methods for the generation of virtual representations from real samples are presented. The chapter concludes with the generation of artificial virtual surfaces that do not have a real counterpart. After all these preparations the main topic of this monograph, the tactile rendering of virtual surfaces, is presented in Chap. 5. Several different approaches have been investigated within this work which share a common software framework. This framework manages the virtual fabrics positioned on a virtual workspace and controls the device tracking the position of the tactile display. After the description of the framework several experiments on tactile rendering and also the integration with a force-feedback device are described in the chapter. Chapter 6 summarises the results of this work and suggests directions for future research on tactile rendering.

References [1] Fontana, M., Marcheschi, S., Tarri, F., Salsedo, F., Bergamasco, M., Allerkamp, D., B¨ottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: Integrating force and tactile rendering into a single VR system. In: 2007 International Conference on Cyberworlds, pp. 277–284. IEEE Computer Society, Los Alamitos (2007) [2] Magnenat-Thalmann, N., Volino, P., Bonanni, U., Summers, I.R., Bergamasco, M., Salsedo, F., Wolter, F.E.: From physics-based simulation to the touching of textiles: The HAPTEX project. International Journal of Virtual Reality 6(3), 35–44 (2007) [3] Rab¨atje, R.: Virtuelle Montage- und Demontageuntersuchungen mit Hilfe von flexiblen Bauteilen unter Verwendung geeigneter Kraftr¨uckkopplungsger¨ate. PhD thesis, Welfenlab, Universit¨at Hannover (2002) [4] Salsedo, F., Fontana, M., Tarri, F., Ruffaldi, E., Bergamasco, M., Magnenat-Thalmann, N., Volino, P., Bonanni, U., Brady, A.C., Summers, I.R., Qu, J., Allerkamp, D., B¨ottcher, G., Wolter, F.E., M¨akinen, M., Meinander, H.: Architectural design of the HAPTEX system. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 1–7 (2005) [5] Schulze, M.: Von computergraphischen zu haptischen Texturen. Virtual Reality f¨ur den Entwicklungsbereich Design/Styling in der Automobilindustrie. PhD thesis, Welfenlab, Universit¨at Hannover (2005)

Chapter 2

Human Perception

Perception is the process of attaining awareness or understanding of sensory information. Together with our motor skills it is nothing less than our gate to the reality outside our psyche. As the scope of this monograph is the simulation of a certain facet of reality, namely the tactile properties of textiles, a good insight into human perception is a necessary prerequisite. The intention of this chapter is to cover the aspects of human perception that are important for this task. In Sect. 2.1 an introduction to psychophysics is given. It provides theories and methods for the measurement of a subjects internal response to physical stimuli. Section 2.2 describes the biological background of our perception. It covers neurons, the structure of our nervous system and different methods for the measurement of neural activity. After these rather broad treatments of perception Sect. 2.3 focuses on tactile perception. It justifies the way the virtual textiles are rendered in Chap. 5.

2.1

Introduction to Psychophysics

Psychophysics is a discipline of psychology that establishes a relationship between a physical stimulus and its sensation (percept). Like in physics attainment of knowledge is achieved by finding theoretical models describing experimentally gained data. The term “psychophysics” was coined by Gustav Theodor Fechner in his work “Elemente der Psychophysik” released in 1860. Fechner states that bodily and conscious facts are different sides of one reality and that a mathematical relation can be established between them: In order that the intensity of a sensation may increase in arithmetical progression, the stimulus must increase in geometrical progression. The latter is known as the Weber-Fechner law and, together with Stevens’ power law, will be described in more detail in Sect. 2.1.2.

D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 5–23. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

6

2 Human Perception

2.1.1

Just Noticeable Differences and Weber’s Law

A just noticeable difference (jnd) is the smallest difference in the intensity of two stimuli that is still detectable by subjects. Its measurement leads to a psychometric function as shown in Fig. 2.1. Such a function usually resembles a sigmoid function with the percentage of the correctly noticed differences displayed on the ordinate and the difference in the intensity of the stimuli on the abscissa. The jnd is by definition the point where the function crosses 50%.

psychometric curve theoretical curve correctly noticed differences in %

100

80

60

40

20

0 0

5

10 difference in intensity

15

20

Fig. 2.1 An example of a psychometric function

A special case of a jnd is the absolute threshold which is the smallest detectable level of a stimulus. In 1834 Ernst Heinrich Weber studied jnds between stimuli and found the following relation, today known as Weber’s law:

Δ R = cw R

(2.1)

where Δ R denotes the jnd to a given stimulus intensity R. The constant cw depends on the kind of stimulus.

2.1 Introduction to Psychophysics

2.1.2

7

Weber-Fechner Law and Stevens’ Power Law

Weber’s law only relates jnds to stimulus intensities. It is not a psychophysical scale E(R) relating the intensity of a stimulus R to a value E indicating the intensity of the corresponding percept. E(R) is assumed to be a continuous function. Fechner constructed such a psychophysical scale based on Weber’s law. Its construction is described below. We define the following sequence of stimuli, where R0 denotes the absolute threshold: R1 = R0 + Δ R0 = R0 + cw R0 = (1 + cw)R0 R2 = R1 + Δ R1 = (1 + cw)2 R0 .. . Rn = (1 + cw )n R0 Assuming that the difference in perceived intensity is the same for all jnds we have E(Ri+1 ) − E(Ri ) = E(R + Δ R) − E(R) =: c p .

(2.2)

We claim E(R0 ) = 0, i.e. the perceived intensity is zero at the absolute threshold, and we arbitrarily set c p = 1. Now we can replace Rn by R(E) and use (2.2) to construct the inverse function of the psychophysical scale: R(E) = (1 + cw)E R0

(2.3)

1 By inverting this function and setting c = ln(1+c we obtain the Weber-Fechner law: w)

E = c ln

R R0

(2.4)

The Weber-Fechner law is not only a theoretical construct. It is also part of our everyday life. A well-known example is the keyboard of a piano. An octave is always perceived as the same interval and therefore there are always 12 semitones in an octave. However, from the lower to the higher pitch the corresponding frequency is always doubled. In other words: As the keys represent the sensation increasing in arithmetical progression, the corresponding frequencies increase in geometrical progression. (Most of today’s keyboards are well temperamented, so the situation is a little more complicated, but this is definitely out of the scope of this work.) One hundred years after Fechner constructed his famous psychophysical scale the psychologist Stanley Smith Stevens designed a different method for the construction of such scales: Subjects were presented a reference stimulus which was assigned a number value reflecting the perceived intensity. The subjects then had to rate the intensity of successive stimuli with respect to the reference stimulus, i.e. if the reference had an intensity value of for instance 100 another stimulus with only

8

2 Human Perception

half the intensity of the reference should be rated with an intensity value of 50 by the subject. According to Stevens the results could be well approximated by E = kRb

(2.5)

where E is again the perceived intensity and R is the intensity of the stimulus. The exponent b depends on the type of stimulation and k is a proportionality constant depending on the units used. Equation (2.5) is known as Stevens’ Power Law. For some modalities, e.g. the intensity of light, the Power Law resembles the Weber-Fechner Law, while for others, e.g. electric shocks, Stevens’ Power Law turns out to be superior. Stevens’ law also proved of value because it is applicable to nearly every dimension of physical perception. Psychologists used it for a variety of scales, e.g. tone pitch, length, beauty or weight of crimes. Human perception is a very complicated process. Approximating it with simple formulas such as the Weber-Fechner Law or Stevens’ Power Law appears to be very naive. But their applicability to a lot of practical examples proves their usefulness. Perhaps they might be compared to Newton’s laws of motion which later turned out to be too simplistic to accurately reflect reality. However, with some limitations these laws are still applicable and are indeed also applied in practice.

2.1.3

Measuring Psychometric Distances

In Sect. 2.1.1 the concept of jnds is introduced. How can the ability of a subject to detect a difference be measured? The subject could of course simply be asked but this is prone to response bias: the systematic tendency of a subject to answer in a certain way that has nothing to do with sensory factors. This biased behaviour can be measured with so called catch trials and then used to offset the original measured data. See Sect. 2.1.5 for a detailed discussion. The result is the theoretical probability that an arbitrary subject is able to detect the given difference. As shown in Sect. 2.1.1 these probability values are used to produce the psychometric function. According to [4] we distinguish between six standard discrimination methods: 2-Alternative Forced-Choice method (2-AFC) Two different objects are presented to the subject, who has to select the object with the strongest (or weakest) sensory characteristic. 3-Alternative Forced-Choice method (3-AFC) Three objects are presented to the subject, with two of them being identical. The subject has to select the object with the strongest (or weakest) sensory characteristic. Duo-Trio method Three objects are presented to the subject, with two of them being identical. One of the objects is labeled as the “control” sample. The subject has to decide which of the remaining two objects matches the control object. In this configuration it is possible that the odd object is chosen as control sample. Triangular method Three objects are presented to the subject, with two of them being identical. The subject then has to tell, which one of the three objects is different from the two other ones.

2.1 Introduction to Psychophysics

9

A–Not A method The subject is familiarised with two different objects labeled as “A” and “Not A”. One of these objects is presented to the subject, who has to decide whether it is “A” or “Not A”. Same–Different method Two objects, which can either be identical or different, are presented to the subject, who has to decide whether they differ or not. For our experiments we mostly used the triangular method, also referred to as the odd-man-out forced-choice procedure. Here the subject either really detects the difference or just randomly draws an object. In the latter case the subject might accidentally draw the right object with a probability of 31 . In our experiments we observed that subjects thinking they were just randomly drawing an object still performed better than the theoretical probability suggests. This might indicate that rather than measuring the conscious ability to detect differences, the odd-man-out procedure also measures the sub-conscious ability. However, this assumes that subjects really tried to intuitively draw an object when not being able to consciously detect the different object. Some subjects tended to systematically draw for instance the first object, so the results were in danger of being biased by the way subjects “randomly” drew objects. Therefore we generally advised all subjects to intuitively draw the object when being unsure rather than systematically choose one.

2.1.4

Statistical Analysis

For a given pair of stimuli the discrimination experiment is usually repeated several times. Let us assume the experiment is repeated n times, then the subjects will detect a difference in k of n times. Such an experiment is called Bernoulli experiment and the probability of k is given by a binomial distribution n k p (1 − p)n−k . (2.6) Pp (k) = bn,p (k) := k The letter p denotes the probability that a subject detects the difference or chooses the right object, respectively. For given n and k we want to estimate the probability p, ˆ such that Lk (p) := Pp (k) becomes maximal: ˆ := sup{Lk (p) : 0 ≤ p ≤ 1} (2.7) Lk ( p) p(k) ˆ is called maximum likelihood estimator. For a Bernoulli experiment we find k p(k) ˆ = . n

(2.8)

Depending on the number of experiments n we can have more or less confidence in the estimated probability p. ˆ For large n the error |pˆ − p|is probably small. We

10

2 Human Perception

can define a confidence interval C(k) for which we can assume that p ∈C(k) with a certain probability, called confidence level 1 − α : Pp(p ∈C(k)) ≥ 1 − α .

(2.9)

For many domains α = 0.05 is considered to be sufficiently low. This definition is not unique as there are various intervals fulfilling (2.9), a trivial example is C(k) = [0, 1]. For Bernoulli experiments a table of confidence intervals for confidence levels 0.95 and 0.99 that is suitable for our needs is provided in [5]. Related to confidence intervals is the concept of statistical hypothesis testing. Here we want to decide if an estimated parameter belongs to a previously defined interval. For our experiments we mostly want to know whether the subjects just guessed, denoted by p = p0 . This is called the null hypothesis. If we can falsify the null hypothesis we can assume the alternative hypothesis p > p0 which means that the subjects sometimes were able to detect a difference. As it is not possible to measure the probability p directly we have to rely on a Bernoulli experiment again. Assuming that the null hypothesis holds, we can explain the outcome k of the experiment with the binomial distribution bn,p0 (k). We now define t to be the smallest value that fulfills n

Pp0 (x ≥ t) = ∑ bn,p0 (x) ≤ α .

(2.10)

x=t

The value t is called critical value to the significance level α . Like for the confidence intervals α = 0.05 is usually considered to be sufficiently low. If k ≥ t we may reject the null hypothesis and assume the alternative hypothesis. We are interested in the probability pd that a subject detects the difference of a given pair of stimuli. This is not equal to the probability pm that a subject chooses the right object in an odd-man-out experiment, because, as already stated in Sect. 2.1.3, when the subject does not detect the difference he or she still chooses the right object with a probability of 31 . Given an estimation pˆm of pm we can estimate pd with 1 pˆd = max{0, pˆ m − (1 − pˆ m )}. 2

2.1.5

(2.11)

Signal Detection Theory

In Sect. 2.1.3 experiments for the measurement of psychometric distances were introduced. The subsequent statistical discussion in Sect. 2.1.4 only dealt with the probability that a subject detects a difference between two stimuli. However, we are more directly interested in the perceived distance between two stimuli. The WeberFechner law described in Sect. 2.1.2 assigned a perceived distance value of c p to a discrimination probability of 50% in (2.2). Under some assumptions it is possible to assign a distance value to every discrimination probability. Signal-detection theory provides a well-established framework for this kind of problems. Only a short

2.1 Introduction to Psychophysics

11

introduction to this theory can be given here. The interested reader is referred to [19]. In a classical signal-detection experiment an observer listening to white noise has to decide whether a faint tone (the signal) is added to the noise or not. There are four cases: there is no signal and the observer does not report a signal (correct rejection), there is no signal and the observer reports a signal (false alarm), there is a signal and the observer does not report a signal (miss) or there is a signal and the observer reports a signal (hit). The hit rate h is defined as h=

number of hits number of signal trials

(2.12)

and the false alarm rate f is defined as f=

number of false alarms . number of noise trials

(2.13)

In the same way we can define the miss rate and the correct rejection rate, but these do not carry any additional information. As stated in Sect. 2.1.3 such an experiment is vulnerable to bias, so the hit rate alone does not suffice to describe the result of the experiment. But also together with the false alarm rate the situation does not become immediately evident. It would be much better to have a single value describing the detectability of the signal and another value describing the bias. Signal-detection theory makes three assumptions: • The internal response of the observer to a stimulus can be represented by a single number. • The internal response is subject to random variation. • The choice is based on a simple decision criterion applied to the magnitude of the internal response. Figure 2.2 shows an example of these assumptions. The internal response is represented as a value on the abscissa. It is a random variable drawn from one of two probability distributions, each belonging to one of the two stimuli. The decision criterion is described by λ . If the internal response is larger than λ the signal is detected otherwise not. A detection model has to specify the form of the probability distributions. In the equal-variance Gaussian model both distributions are assumed to be normal distributions with a variance of one. The mean of one distribution is set to zero, the mean of the other to d ′ . With these assumptions λ and d ′ can be estimated from f and h:

λˆ = −Z( f ) dˆ′ = Z(h) − Z( f ).

(2.14) (2.15)

Z denotes the inverse of the cumulative normal distribution. A derivation of (2.14) and (2.15) is given in [19]. λ and d ′ describe the result of the experiment much

12

2 Human Perception

No

Yes

Stimulus 1

Stimulus 2

0

λ

d

internal response

Fig. 2.2 Decision based on probability distributions for two different stimuli

clearer than f and h: d ′ represents the detectability of the signal whereas λ represents the bias. The detectability d ′ can not be influenced by the observer, but λ is freely chosen. One can assume that an unbiased observer chooses a criterion such that the likelihood of a correct answer is maximised (see Chap. 9 of [19]). For many of our discrimination experiments the triangular method was used, for which [8] provides a table assigning a d ′ to a given probability. In such an experiment the internal response to two of the objects is drawn from one probability distribution, because these objects create similar stimuli. The remaining object creates a different stimulus and so the corresponding internal response is drawn from another distribution. Craven assumes that the observer chooses the object maximising the likelihood that the remaining objects are drawn from a single distribution. He shows that this rule is equivalent to simply choosing the object which is farthest from the mean of all objects. Because of the difficulty of analytically expressing the relationship between probability and d ′ the table was generated by the Monte Carlo method. Figure 2.3 depicts Craven’s result.

2.1.6

Multidimensional Scaling

So far the considered scales related only one-dimensional stimuli and their subjective correlates, which were also expressed with only one dimension. The scale was thus expressed as a function E : IR → IR. Nevertheless, in many stimulus domains the dimensions are not known. Often even the number of relevant dimensions is not clear. Here, a versatile tool called multidimensional scaling (MDS) turns out to be very useful. Given measurements of similarity among pairs of objects, MDS represents these measurements as distances between points of an embedding multidimensional space. Depending on the dimension of this space the distances between the

2.1 Introduction to Psychophysics

13

7 6

distance

5 4 3 2 1 0 0

0.2

0.4

0.6

0.8

1

probability

Fig. 2.3 Distances d ′ for given probabilities according to [8]

points can be consistent—to a greater or lesser extend—with the measured similarities. The overall error is called stress and decreases with an increase of dimensions. It vanishes for a dimensionality m ≥ n − 1 where n is the number of objects. How the positions of these points are computed is not described here. As we only use the MDS, we do not bother with such details. The interested reader is referred to [6]. The following example might be enlightening: In [10] 14 colours differing only in their wavelength but not in their brightness or saturation were used. Each of all possible pairs of different colours was projected onto a screen and 31 subjects were asked to rate the similarity on a scale from 0 (no similarity) to 4 (identical). The averaged and normalised ratings are summarised in Table 2.1. The first row and column show the wavelengths of the colours in nanometers. The same data is depicted in Fig. 2.4, where MDS was used to express proximities as geometrical distances. The result reminds of the CIE chromaticity diagram shown in Fig. 2.5. This similarity is quite interesting, because the CIE diagram has been constructed with a very different, more complicated experiment (for details see Chap. 13.2.2 of [11]). This example not only shows how MDS works in principle but also illustrates a case where a one-dimensional physical entity is represented in a two-dimensional perceptual space. The colour circle becomes a one-dimensional manifold embedded in this perceptual space.

14

2 Human Perception

Table 2.1 Similarity matrix of colours from [10] nm

434 445 465 472 490 504 537 555 584 600 610 628 651 674

434 445 465 472 490 504 537 555 584 600 610 628 651 674

.86 .42 .42 .18 .06 .07 .04 .02 .07 .09 .12 .13 .16

.50 .44 .22 .09 .07 .07 .02 .04 .07 .11 .13 .14

.81 .47 .17 .10 .08 .02 .01 .02 .01 .05 .03

.54 .25 .10 .09 .02 .01 .00 .01 .02 .04

.61 .31 .26 .07 .02 .02 .01 .02 .00

.62 .45 .14 .08 .02 .02 .02 .01

.73 .22 .14 .05 .02 .02 .00

.33 .19 .04 .03 .02 .02

.58 .37 .27 .20 .23

.74 .50 .76 .41 .62 .85 .28 .55 .68 .76

Fig. 2.4 MDS representation for color proximities in Table 2.1 (from [6])

The dimensionality m of the embedding space has not been chosen arbitrarily. As stated above setting m >= n − 1 = 13 would be an uninteresting choice, because the colours then can be perfectly represented in these spaces. We are more interested in the minimal number of dimensions that still results in an acceptable error. In this example the stress of the 1D, 2D and 3D MDS representations is 0.272, 0.023, 0.018, respectively. So two dimensions are clearly preferable to only one dimension. But higher dimensions do not have much of an effect, because the stress of the 2D solution is so close to zero already.

2.2 The Human Nervous System

15

Fig. 2.5 CIE 1931 xy chromaticity diagram (Source: Wikimedia Foundation)

2.2

The Human Nervous System

In order to understand human perception it is necessary to gain some insight into the human nervous system as this is the system that processes all information acting on the sensory organs. It performs three tasks: sensory input, integration of information and motor output. The content of this section is mainly based on [7].

2.2.1

Neurons

The basic unit of the nervous system is the neuron. Although the functionality of a single neuron is relatively simple, the interaction of several million neurons—every cubic centimetre of the human brain contains more than 50 million—is able to fulfil very complex tasks. The complexity is caused by the large number of interconnections, every neuron communicates with thousands others. Together the neurons control all aspects of our perception and motor activity. They allow us to learn, to remember and to consciously perceive our environment and ourselves. Neurons are electrically excitable cells that are composed of a cell body (called soma), dendrites, and an axon (see Fig. 2.6). The dendrites collect signals from other cells and forward these information as an electrical signal to the soma. Information sent by the neuron is transmitted by its axon. These are usually much longer than the dendrites, some more than a metre. At its end the axon branches out into several terminals which release neurotransmitters to transfer information to other cells. The contact where this information is transmitted is called synapse. While a neuron is in a resting state it has an electrical potential of −70 mV compared to the outside of the cell. This so called membrane potential can be changed by

16

2 Human Perception

Fig. 2.6 Signal propagation in neurons (Source: U.S. National Institute on Aging)

outside influences, usually by the neurotransmitters from other cells (see Fig. 2.7). If the membrane potential reaches a threshold potential of usually −50 to −55 mV, an action potential of about +35 mV is triggered. After this spike the membrane potential returns to its resting potential of about −70 mV. Fig. 2.7 A triggered action potential

"!

!

#

!

$

%

2.2 The Human Nervous System

17

The amplitude of the action potential is independent of the intensity of the stimuli. The action potential can thus be seen as a binary information: it is either there or not. The intensity is rather coded by the frequency of the occurrences of the action potentials, typically denoted by impulses per second. The maximal frequency is given by the length of the refractory period, i.e. the time until the neuron recovered its resting potential after an action potential. The action potential is transmitted to other parts of the nervous system by the axon, which enables signal speeds of up to 150 m s−1 .

2.2.2

Neural Integration

A single neuron can receive information from other neurons via thousands of synapses. Each synapse influences the neuron either in an inhibitory way (decrease of the membrane potential) or in an excitatory way (increase of the membrane potential). Depending on the location of the synapse it has more or less effect on the membrane potential. The effect of a synapse also decays some time after an impulse. There are two kinds of summation: spatial summation means that the simultaneous effects of several synapses on different places on the neuron are combined, whereas temporal summation means that the effects of several impulses of a single synapse are combined. The latter only occurs if the pause between two successive impulses is sufficiently short, i.e. the effect of the impulse does not fully decay.

2.2.3

Sensory Reception

Neurons communicate with each other via neurotransmitters. But how is information coming from outside the nervous system translated into a code that can be understood there? There must be some transducers that react to physical stimuli and accordingly excite neurons. And indeed, for every perceivable kind of physical stimulus a sensory transducer exists. Such a transducer emits neurotransmitters when excited by a stimulus it is reactive to. This signal is then forwarded by neurons called afferent neurons into the central nervous system. There are also some transducers like for example the pain receptors that are themselves neurons that can directly send action potentials via their axons into the central nervous system. The information is processed in the central nervous system by intraneurons and often results in the excitation of efferent neurons, also called motor or effector neurons, which then excite the muscles they are connected to. A very simple example of this process is the patellar reflex, which is sometimes tested by physicians. When the patellar tendon is tapped just below the knee, the lower leg automatically, i.e. without intentional control, kicks forward. The tap excites elongation receptors inside the quadriceps muscle which is connected to the patellar tendon. Afferent neurons connected to the elongation receptors forward this information to intraneurons inside the spinal cord. These excite efferent neurons controlling the quadriceps muscle which then contracts and in this way lifts the lower leg. In the same time the efferent neurons controlling the opposite muscle are inhibited to make this movement possible.

18

2 Human Perception

The elongation receptors inside the muscles, also called muscle spindles, are examples of mechanoreceptors. This type of receptors respond to mechanical stress or mechanical strain. The function of the muscle spindles is to send proprioceptive information, i.e. information on the relative position of neighbouring parts of the body, to the central nervous system. Another kind of mechanoreceptors are hair cells, which can be found in the cochlea of the inner ear and in the vestibular system. The latter is also known as balance system and provides information on acceleration and the direction of gravity. The cutaneous mechanoreceptors are responsible for touch perception and are described in detail in Sect. 2.3. Nocioceptors are free dendrites that send signals to the central nervous system that cause pain in response to a possibly damaging stimulus. Different types of nocioceptors react to extreme heat, strong pressure or chemical substances like histamine and several acids. Pain is one of the most important perceptions, because it can result in a defensive reaction that prevents damage to the body. Another kind of receptors are the thermoreceptors which are part of the system that regulates the temperature of the body. These receptors are not fully understood yet. Chemoreceptors detect chemical stimuli in their environment. These can either be very specialised to a certain substance or can react to a broad class of stimuli. The olfactory and gustatory systems use chemoreceptors to detect smell and taste, respectively. There are also chemoreceptors in the human body, that control blood parameters like the concentration of oxygen and carbon dioxide. Photoreceptors respond to electromagnetic radiation with wavelengths of about 400–700 nm. This is the range of light visibile to the human eye. Colour is perceived by three types of photoreceptors that respond to different parts of the visual spectrum.

2.3

Human Tactile Perception

Tactile perception is the sensation of pressure, vibration and temperature via receptors of the skin. Together with proprioception, which delivers information on the position of the joints and the tension inside of the muscles and the tendons, it belongs to the haptic perception.

2.3.1

Mechanoreceptive Afferents

The glabrous skin of the human hand is innervated by four types of mechanoreceptive afferents, which are responsible for the perception of pressure and vibration. They are presented in Table 2.2. They can be categorised by receptor type, rate of adaptation, size of the receptive field and by their function. In this context adaptation is the change over time in responsiveness of a sensory system to a constant stimulus. The receptive field of an afferent is the area of the skin where stimuli have an effect on its firing rate. The functions of the afferents are summarised according to [15, 14].

2.3 Human Tactile Perception

19

Table 2.2 Characteristics of mechanoreceptive afferents Afferent Receptor type

Adaptation Receptive field

Function

SA1

Merkel discs

slow

small

form and texture

SA2

Ruffini’s corpuscles slow

large

hand conformation and forces acting on the hand

RA

Meissner corpuscles rapid

small

minute, low-frequency skin motion and feedback required for grip control

PC

Pacinian corpuscles rapid

large

detection of distant events by vibrations

The receptive fields of the slowly adapting type 1 (SA1) and rapidly adapting (RA) afferents are very small in the fingertip, about 3 mm2 , whereas the fields of the slowly adapting type 2 (SA2) afferents are large and even larger for Pacinian (PC) afferents. The latter spread over the whole finger or half of the palm. SA1 sensors measure the indentation depth of a mechanical stimulus to the skin and continually send impulses to the central nervous system during the whole duration of the stimulus. Having small receptive fields they can collect information on form and texture of surfaces. Similar to the SA1 sensors the SA2 sensors continually send impulses during the whole duration of the stimulus. But their receptive fields are much larger and they mainly react to shear forces inside the skin. RA sensors react to the change of a stimulus and thus belong to the differential receptors. Their impulse frequency depends on the velocity of the indentation. PC sensors measure the acceleration of a pressure stimulus and thus are especially appropriate for the detection of vibration. In [15, 14] it is therefore concluded that PC sensors are mainly responsible for the detection of remote events via vibrations. They are most sensitive to vibrations between 100 and 300 Hz.

2.3.2

Perception of Form and Texture

Independent of the functionality of single receptors the sense of touch can also be investigated with psychophysical experiments. MDS was used in [12] to find tactile properties characterising surfaces. The sample surfaces employed in the experiment could be represented as points in a three-dimensional metric space. The relation between mechanical deformation of the skin and the firing rate of the SA1 sensors was analysed in [9] by means of a finite element model of the fingertip. Measurements on neural activity recorded on monkeys were used to match the model. A model based on continuum mechanics for the prediction of the SA1 and RA sensors is presented in [18].

20

2.3.3

2 Human Perception

Perception of Fine Textures

SA1 receptors cannot be the only receptors responsible for the perception of roughness, because humans are able to discriminate very fine textures with structure sizes well below the resolution of the SA1 receptors. Experiments described in [13, 1, 3, 2] show that these fine textures are perceived by the PC receptors. While being rubbed over a surface the skin is oscillated in a way characteristic of the surface’s texture, which enables the discrimination of fine surface textures. The importance of the fingerprints for the perception of fine textures is investigated in [17]. Two sensors imitating the geometrical and mechanical properties of the human fingertip were developed. Only one sensor surface was patterned with ridges mimicking the fingerprints. Comparing the frequency responses of the sensors, the sensor with ridges shows a reduced damping in a frequency range that corresponds to the sensitivity range of the PC sensors at a typical exploration velocity of 12–15 cm s−1 . The authors conclude that the fingerprints play an important role in the perception of fine surface textures.

2.3.4

Perceptional Model of Tactile Simulation

With the information given in this chapter a theoretical framework for the following chapters can be defined. Figure 2.8 outlines such a model, which was inspired by a systems theoretical model of human perception as proposed in [16]. On the left side of the figure the steps of the information processing taking place when directly touching a surface are depicted. The right side equals the left side in most parts, but the surface is palpated indirectly via the combination of a tactile sensor, renderer and display. Because of the latter, which can be seen as a filter of the signals, relevant information is lost and the stimuli and thus also all following steps of perception differ on both sides. In this model direct and indirect touch can be compared on every level of human perception: • On the level of the stimuli an objective measurement with appropriate sensors is possible. These sensors should correspond to the human fingertip in their geometrical and mechanical properties. • Neural activity of single afferent nerves can also be measured. This, however, is only possible with invasive methods. • The sensations emerging from neural activity can be investigated with appropriate methods of psychophysics as explained earlier in this chapter. • On the level of symbolic perception psychophysical experiments are also possible. Because of redundancies on the levels of information processing there exist subsets of the stimuli whose elements cause the same percept. This implies a partition into equivalence classes of the set of stimuli.

2.3 Human Tactile Perception

21

#$%5".! /%-/!%0'!#

%!"* #$%5".!

#!)#-% !"#$%!& #'()"*

203 %!)&!%!%

%!)&!%!& #'()"*

&203 &'#/*"+

&'%!.060-$.7 #0' $*'

#&203

)!$%"* ".0'1'0+

"&203

#!)#-%

8

#!)#-%

/%-,!

8

/%-,!

#'203

"'203

! #!)#"0'-)#

.&203 "

#+ ,-*'. /!%.!/0'-)

4&

! 9:;

8

9:;

.'203 " 4'

Fig. 2.8 A model of the contact mechanical interaction of the fingertip and the further processing of the nervous system

22

2.4

2 Human Perception

Conclusion

In this chapter an introduction to psychophysics is given. It starts with the basic laws of psychophysics and continues with methods for measuring psychometric distances. With a statistical analysis of the collected data and a detection model from signal detection theory it is possible to gain a measure of subjective similarity. Given a set of stimuli and psychometric distances for all pairs within this set, multidimensional scaling represents stimuli as points of a multidimensional space. The latter might be referred to as perceptional space of the given class of stimuli. The second section of this chapter describes the human nervous system. This section is necessary to understand the third section, which enlarges on human tactile perception. In order to embed the research on tactile simulation into a theoretical framework a perceptional model was developed within this work which is described in Sect. 2.3.4. With the information given in this chapter the reader should be prepared to fully understand the simulation systems and the experiments described in Chap. 5.

References [1] Bensma¨ıa, S.J., Hollins, M.: The vibrations of texture. Somatosensory & Motor Research 20(1), 33–43 (2003) [2] Bensma¨ıa, S.J., Hollins, M.: Pacinian representations of fine surface texture. Perception & Psychophysics 67(5), 842–854 (2005) [3] Bensma¨ıa, S.J., Hollins, M., Yau, J.: Vibrotactile intensity and frequency information in the pacinian system: A psychophysical model. Perception & Psychophysics 67(5), 828–841 (2005) [4] Bi, J.: Sensory Discrimination Tests and Measurements: Statistical Principles, Procedures, and Tables. Blackwell, Malden (2006) [5] Blyth, C.R., Still, H.A.: Binomial confidence intervals. Journal of the American Statistical Association 78(381), 108–116 (1983) [6] Borg, I., Groenen, P.J.F.: Modern Multidimensional Scaling: Theory and Applications, 2nd edn. Springer, Heidelberg (2005) [7] Campbell, N.A., Reece, J.B.: Biology, 8th edn. Pearson/Cummings (2008) [8] Craven, B.J.: A table of d ′ for M-alternative odd-man-out forced-choice procedures. Perception & Psychophysics 51(4), 379–385 (1992) [9] Dandekar, K., Raju, B.I., Srinivasan, M.A.: 3-d finite-element models of human and monkey fingertips to investigate the mechanics of tactile sense. Journal of Biomechanical Engineering 125(5), 682–691 (2003) [10] Ekman, G.: Dimensions of color vision. Journal of Psychology 38, 467–474 (1954) [11] Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and Practice, 2nd edn. Addison-Wesley, Reading (1990) [12] Hollins, M., Faldowski, R., Rao, S., Young, F.: Perceptual dimensions of tactile surface texture: A multidimensional scaling analysis. Perception & Psychophysics 54(6), 697– 705 (1993) [13] Hollins, M., Bensma¨ıa, S.J., Washburn, S.: Vibrotactile adaption impairs discrimination of fine, but not coarse, textures. Somatosensory & Motor Research 18(4), 253–262 (2001)

References

23

[14] Johnson, K.O.: The roles and functions of cutaneous mechanoreceptors. Current Opinion in Neurobiology 11(4), 455–461 (2001) [15] Johnson, K.O., Yoshioka, T., Vega-Bermudez, F.: Tactile functions of mechanoreceptive afferents innvervating the hand. Journal of Clinical Neurophysiology 17(6), 539–558 (2000) [16] Kammermeier, P., Buss, M., Schmidt, G.: A systems theoretical model for human perception in multimodal presence systems. IEEE/AMSE Transactions on Mechatronics 6(3), 234–244 (2001) [17] Scheibert, J., Leurent, S., Prevost, A., Debr´egeas, G.: The role of fingerprints in the coding of tactile information probed with a biomimetric sensor. Science 323(5920), 1503–1506 (2009) [18] Sripati, A.P., Bensma¨ıa, S.J., Johnson, K.O.: A continuum mechanical model of mechanoreceptive afferent responses to indented spatial patterns. Journal of Neurophysiology 95, 3852–3864 (2006) [19] Wickens, T.D.: Elementary Signal Detection Theory, Oxford (2002)

Chapter 3

Devices for Tactile Simulation

Nearly every office or home computer is capable of graphics or sound output in a quality that, because of the limits of human perception, hardly needs to be improved. The haptic impression however is limited to the contours of the keyboard and the mouse. In some special fields force-feedback devices are used, but these only simulate mechanical contact with virtual objects via special tools (e.g. like a thimble) and are therefore not applicable for more general uses. It would be better if the contact could be simulated without such tools limiting the perceptional capabilities. For a convicing simulation of direct contact with the skin a single force is not sufficient. Instead the skin has to be deformed in a more realistic way by a so called tactile display. In Sect. 3.1 existing tactile displays are evaluated according to their usefulness for virtual reality applications. The overview of existing tactile displays is followed by Sect. 3.2, which describes the tactile displays used within this work in more detail. The tactile signals calculated on a computer need to be transmitted to the tactile display. An electronic circuit that receives tactile data from the computer and converts these to analogue signals for the tactile display is presented in Sect. 3.3. The combination of a tactile display with a force-feedback device is described in a later chapter. Therefore, Sect. 3.4 provides a short introduction to the haptic simulation with force-feedback devices.

3.1

Survey of Existing Devices

As there exist a lot of very different tactile displays it would not make sense to describe all of them in this book. The main interest of this work is the applicability of existing displays for virtual-reality simulations. This restriction already excludes many displays that were only designed for the transmission of information, e.g. for systems that help blind persons to navigate around obstacles (cf. [12]). D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 25–47. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

26

3 Devices for Tactile Simulation

If a user of a virtual-reality simulation wants to touch a virtual object he or she will probably use the hand to explore the object. A restriction of the survey to tactile displays for the hand is therefore only natural. As the fingertips are the most sensitive parts of the human hand most displays are designed for that part of the skin. There are two completely different approaches to establish the connection between the finger and the display: the finger is either permanently connected to the display or not. The latter allows for an active exploration of the simulated surface, i.e. the user can move the fingers over the display to asses the properties of the virtual surface. An active exploration with a display tied to a finger is only possible if the display follows the movements of the finger. This approach seems to be more difficult to realise, but it is also more flexible. If the display is attached to a force-feedback device as demonstrated, for example, in [13] it can simulate the surface of curved objects in a three-dimensional space while the force-feedback device delivers information on the elasto-mechanical properties of the object. Because of its greater flexibility this approach is preferable for virtual-reality systems and this survey is restricted to this kind of displays. The most common approach in the development of tactile displays is based on a dense array of stimulator pins. The fingertip is stimulated by the movement of individual pins which are usually moved normally to the skin’s surface. Different tactile stimuli are created by different patterns of pin movements. In the following the large variety of different actuator principles is shown by presenting a selection of tactile displays. However, this is not a comprehensive survey which would go beyond the scope of this chapter.

3.1.1

Electromagnetic Displays

In [31], [26] and [18] electromotors are used to control the heights of the contact pins of the tactile displays. In [31] each pin is lifted by a small rotating lever. The presented prototype contains 36 independently controlled pins with a maximal deflection of 2 mm. The pins are positioned with an interspacing of 2 mm and are covered by a sheet of rubber that functions as a spatial low-pass filter. The system is relatively large and reaches a bandwidth of about 25 Hz. In [26] 4096 pins are positioned on an area of 200 mm times 170 mm. The distance between adjacent pins is about 3 mm. The pins are lifted by elevating screws, which allow for deflections of a few mm. Because of the transmission ratio of the screws large forces can be generated. However, an update of the heights of the pins might need as much as 15 seconds. In [18] a linear actuator is used to adjust the heights of the display pins. In this way contact forces as high as several Newtons at a maximal deflection of 2.5 mm are reached. The prototype offers 400 pins on an area of 1 cm2 , but it is very large because of the large number of linear actuators.

3.1 Survey of Existing Devices

3.1.2

27

Pneumatic Displays

A pneumatic approach is pursued in [11]. The 16 pins can be separately controlled with forces up to 2 N at a maximum height of 3.5 mm. Vibrations in the range of 20– 300 Hz are also possible. The pins are positioned with an interspacing of 1.75 mm and can additionally be moved laterally to the skin with a pneumatic muscle. In [23] the pressure inside 25 chambers made of silicone is controlled allowing for a maximum extension of the chambers of 0.7 mm normal to the skin. With this approach a bandwidth of only 5 Hz is reached and the distance between adjacent contact points is about 2.5 mm. Because of the pneumatic actuation additional unwanted vibrations occur.

3.1.3

Displays with Shape Memory Alloys

Shape memory alloys change their shape depending on the temperature. In [19], [32] and [30] wires made of these alloys are used to lift the pins of the displays. The approaches used differ in the transfer of the movement to the pins. All presented solutions have problems with hysteresis and a high power dissipation. The latter is mainly transformed into heat which requires active cooling for some displays. In [19] 24 pins are lifted by levers. In this way high amplitudes and high contact forces can be created, but the bandwidth of about 10 Hz is relatively low. In [30] 64 pins are pushed upwards by springs while the actuator wires pull the pins downwards when contracting. The prototype only reached a bandwidth of about 1–3 Hz. In [32] 10 pins are positioned in a single row. Each pin is attached to the middle of an actuator wire. When the wire contracts it pushes the pin upwards. This display allows a bandwidth of up to 30 Hz.

3.1.4

Piezoelectric Displays

Piezoelectric materials deform proportional to an electric voltage. This principle is used in [15, 14] to control the vibration of 50 contact pins. The prototype only allows a single vibration frequency of 250 Hz, the amplitude can be adjusted from 5 to 57 µm. Amplitudes up to 0.7 mm can be generated by the display presented in [22]. The bandwidth of the 48 contact pins is, however, limited to 20 Hz. The spatial resolution is about 8 mm2 per pin. A relatively high resolution of 1 mm2 per contact pin is described in [27]. The presented display can independently control 100 pins from 20 to 400 Hz with an amplitude of up to 50 µm.

28

3 Devices for Tactile Simulation

In [24] a different approach is taken: the 64 contact pins are not moved normally but laterally to the skin’s surface.

3.1.5

Other Actuator Mechanisms

Electroactive polymers change their form under an electric voltage. In [20] an ionic electroactive polymer is used for a tactile display with 55 actuators. It has a bandwidth of 100 Hz but only works in water. A rather unconventional solution is described in [8]. The skin is stimulated with an air jet which creates a perception not comparable to the other tactile displays. The prototype only offers one contact point. As the skin is conductive, forces act on it in an electric field. This effect is used in [29] to create friction forces. In this approach the finger moves over the display, i.e. the area of the simulated surface is the same as the area of the display. The system is sensitive to changes of the skin’s moisture. In [16, 25, 21] the forces occuring in an electric field are used for miniature actuators. Capacitor plates are linked by an elastic isolator which is contracted when a voltage is applied. Because of their small size the actuators can be arranged in a grid. Electrocutaneous displays stimulate the receptors of the skin with electric impulses. These displays can be very simple and compact, e.g. they could be integrated in a glove. Also the electrodes can be arranged in a dense grid. However, the variable electric resistance of the skin requires an adjustment of the current which makes it sometimes impossible to avoid pain. In [17] a display with 49 electrodes is presented.

3.2

The Tactile Displays Used

The Biomedical Physics Group at the University of Exeter kindly provided several tactile displays for the experiments described in this work. All displays are developed further from the display described in [27] and also use piezoelectric bimorphs for the transformation of electrical signals into movements of the stimulator pins. The displays mainly differ in their geometrical configuration. In the colour model of computer graphics a colour consisting of a whole spectrum of light is substituted with the combination of only three colours creating the same sensory impression. This model works because the human eye only provides three channels for the perception of colour. Virtually all computer screens and televisions are based on this principle. The success of this model suggests that a similar model for tactile perception might also be successful. The displays developed by the Biomedical Physics Group follow such a suggestion by Bernstein et al. (cf. [9]). Like the colour model two different tactile channels of the human skin are stimulated with sinewaves at two different frequencies. For the stimulation

3.2 The Tactile Displays Used

29

of the two types of mechanoreceptors a superposition of sinewaves at 40 and 320 Hz was chosen.

3.2.1

The Piezoelectric Effect

If a piezoelectric crystal is deformed under mechanical pressure a voltage is induced. This effect is known as the piezoelectric effect. Figure 3.1 shows this effect on a schematic drawing of a quartz. The displacement of the negative and positive ions results in a displacement of the centres of the positive and negative charges and thus in a voltage. The inverse effect is also possible: If a voltage is applied to the piezoelectric crystal, forces act on the positive and negative ions in opposite directions, which result in a deformation of the crystal.

F

− + −

+ Q+ Q−

+

−

− + −

Q+ Q−

+

+ −

F Fig. 3.1 Schematic drawing of the piezoelectric effect of a quartz

The inverse piezoelectric effect can be used to move the stimulator pins of a tactile display. Piezoelectric crystals do not provide sufficient displacement at moderate voltages, so either the voltages used in the system have to be increased or some form of mechanical leverage is necessary. For the displays used in this work the latter solution has been chosen. The piezoelectric material is used in a bimorph configuration. Figure 3.2 shows a piezoelectric bimorph that is connected to a voltage source such that one layer contracts while the other layer expands. As both layers are bonded the whole bimorph bends when a voltage is applied. One side of the bimorph is clamped to make use of the deflection and to control the force at the other end of the bimorph. Readily available bimorphs were acquired from APC International Ltd., USA. With these bimorph cantilevers of thickness 0.6 mm and a free length of 32 mm displacement amplitudes of maximal 100 µm are reached.

30

3 Devices for Tactile Simulation

Fig. 3.2 Schematic drawing of a piezoelectric bimorph

3.2.2

Mechanical Behaviour

Electromechanical systems like the piezoelectric bimorphs usually do not transform electrical signals into mechanical movements independent of the signal frequencies. Therefore, special care has to be taken that signals in the intended frequency range are transformed in an unbiased way. In [28] and [7] a mathematical model of the piezoelectric drive system and the mechanical load presented by the skin is described. The model assumes a linear behaviour of the piezoelectric material and the skin load, which is reasonable for the relatively small amplitudes of less than 100 µm. With this assumption the system can be represented by a simple mass-ona-spring model as depicted in Fig. 3.3. This model predicts the resonance and the frequency responses for different lengths of piezoelectric drive elements as depicted in Fig. 3.4. Fig. 3.3 Mass-on-a-spring model of the piezoelectric material and the skin load

finger

piezoelectric drive system

In Fig. 3.5 the response of the system with and without skin load measured with a miniature accelerometer is shown. From this data the effect of the skin can be estimated: Significant stiffness of about 100 N m−1 and resistance of about 0.1 N s m−1

3.2 The Tactile Displays Used

31

Fig. 3.4 Predicted frequency response for different length of the piezoelectric drive element from [28]

is added to the system. The additional effective mass of about 10−5 kg is negligible. As can be seen in the figure the resonant frequency of the system is increased by the load of the finger while the narrowness of the resonance (the Q-factor) is reduced. Fig. 3.5 Measured frequency response of the piezoelectric drive element with and without the mechanical load presented by the skin from [28]

As the tactile stimuli only consist of superpositions of 40 and 320 Hz sinewaves, bimorphs have been chosen which have a principal resonance that lies between the two operating frequencies of 40 and 320 Hz.

3.2.3

Geometrical Configurations

For the experiments conducted within this work three tactile displays mainly differing in their geometrical configuration were used. Figure 3.6 shows these displays together with a depiction of the arrangement of their stimulator pins as seen from the direction of the finger. Each circle represents a contactor pin. Not all pins are

32

3 Devices for Tactile Simulation

2 mm

2 mm

2 mm

Fig. 3.6 The tactile displays used within this work

driven by an actuator. The moving pins are represented by filled circles. The other circles represent stationary pins. The first display has been developed for a previous study in which five of these displays were used to stimulate the digits of one hand. Compared to the other displays it is relatively large. The second display is quite similar to the first. But instead of a 5-by-5 array of contactor pins it provides only 24 pins in a 6-by-4 configuration. The size of the display is significantly reduced compared to the first display. To achieve this the size of the bimorphs also had to be reduced. The third display is intended for the integration with a force-feedback device as described in Chap. 5. The bimorphs are positioned such that they do not lie beneath the contactor surface. This allows thumb and fingers to be brought close together in a system with more than one tactile display per hand. This display provides a curved contactor surface matching the shape of the fingertip.

3.3 Drive Electronics

3.3

33

Drive Electronics

The tactile displays used within this work are operated with input signals at voltages of up to 40 V rms. As the tactile signals are calculated on a digital computer an electronic component converting the digital data to appropriate analogue signals had to be developed. The analogue parts of the electronics have been designed following suggestions by the Biomedical Physics Group at the University of Exeter, which also provided the tactile displays used. The development and testing of some components is presented in [10]. Nowadays the most common interface to peripheral hardware is the Universal Serial Bus (USB). Nearly every modern personal computer provides at least one USB connector. Therefore, it was decided that the drive electronics should be controlled via USB.

Fig. 3.7 Schematic of the drive electronics

OPA

OPA

Data bus

USB controller

DAC

DAC

Figure 3.7 shows a block diagram of the drive electronics. A USB controller receives tactile data for all stimulator pins from the host computer. After some processing of the data as described below it distributes the signals to the different channels via a data bus. The bus is designed for the unidirectional transmission of data from the USB controller to the digital–analogue converters. To avoid noise from ground loops the bus is galvanically isolated from the USB controller via optical

34

3 Devices for Tactile Simulation

couplers. In each channel a digital–analogue converter (DAC) receives the data for a single stimulator pin and outputs an analogue voltage ranging from −Vref to +Vref . The reference voltage Vref for the DAC is set by a variable voltage supply to a value from 0 to 2.5 V for all channels. In this way the global gain of the system can be set. An operational amplifier (OPA) is used to amplify the analogue signal to voltages appropriate for the tactile display. Figure 3.8 shows the developed drive electronics for a tactile display with 24 pins. The electronic board on the bottom of the figure hosts the USB controller, the optical couplers and the variable voltage supply. Each of the six boards above hosts a DAC for four channels, four amplifications circuits and a circuit for the address logic of the data bus. The board on the right just provides connectors for the other boards.

Fig. 3.8 The drive electronics for one finger

3.3.1

USB Controller

The USB controller communicates with the host computer. It receives, processes and forwards data to the DACs. An integrated circuit (IC) that provides all necessary features is the CY7C64713 produced by the Cypress Semiconductor Corporation. It includes a USB transceiver and an Intel 8051 microprocessor with 16 kBytes of onchip random access memory (RAM). The software running on the microprocessor can be uploaded via USB, which makes the IC especially attractive for development purposes. The design of the supporting circuit is taken from [6]. Only a few capacitors stabilising the voltage of the power supply and an oscillator with a resonance of 24 MHz are needed to run the IC. The IC can be directly connected to the USB. Although the IC can also be directly connected to the power supply of the USB it was decided to include a fixed-voltage regulator reducing the 5 V of the USB to 3.3 V preferred by the IC.

3.3 Drive Electronics

35

Fig. 3.9 The board hosting the USB controller and the variable voltage supply

Figure 3.9 shows the board hosting the USB controller. On the left third of the board the CY7C64713 IC, the oscillator, the USB connector and the fixed-voltage regulator can be seen. The black connector provides access to most ports of the IC for debugging purposes. The software running on the USB controller is called firmware. The firmware processes the data sent from the host computer. For the tactile simulation of fabrics only the relevant properties have to be modeled and displayed. Human performance in tactile perception and the reduced abilities of the tactile display build the frame in which tactile signals make sense. The display used within this work creates vibrotactile stimuli consisting of a superposition of only two sinewaves (see Sect. 3.2). Therefore, instead of transmitting the whole tactile signal via USB only the timevarying amplitudes of the two base signals have to be transmitted. This results not only in a reduced amount of data to be transferred via USB but also in a reduced amount of computation time on the host computer running the tactile renderer, because the actual waveform of the signal is computed by the firmware running on the USB controller. It should be noted that the restriction to only two frequencies is only implemented in the software part of the system. The designed hardware is still capable of producing more complex signals. The implementation of the firmware is able to serve at least 24 channels with a sample rate of at least 5 kHz. It stores a table defining the wave form of the signal. With some optimisations of the assembler code, the firmware reached a sample rate of 10 kHz for 24 channels. The output signal is created from a periodic function f with period length 2π . Let ωi be the angular frequencies of the signals with indices i = 1; 2 and let an,i (t) be the time dependent amplitudes. Then the output signal for channel n is computed as (3.1) gn (t) = an,1 (t) f (ω1t) + an,2 (t) f (ω2t) .

36

3 Devices for Tactile Simulation

The angular frequencies are defined with constants wi and a basic angular frequency ω0 : ωi = wi ω0 (3.2) From the periodicity of the function f follows f (τi ) = f (wi ω0t)

τi := wi ω0t − 2π k = wi (ω0t − 2π

(3.3) k ) wi

k ∈ Z,

(3.4)

such that 0 ≤ τi < 2π . For the digital processing the function f and the amplitudes an,i have to be discrete in time and value. Let Δ t denote the time between two samples. Then the function f can be represented as an array of length l. The value of f at time m is found at position si,m = 2lπ τi (0 ≤ sm < l) within the array. The output values are computed as follows: t (3.5) with m := gn,m = an,1 f [s1,m ] + an,2 f [s2,m ] Δt Let ω0 :=

2π lΔ t

be the basic angular frequency. Then si,m+1 = (si,m + wi )

(mod l)

(3.6)

follows from (3.4). The host computer transmits data to the USB controller with a well defined protocol. The first byte of a data packet determines the command to be performed. Necessary data follow dependant on the command. The protocol is summarised in Table 3.1. The first and second commands are used to define the array f holding the values of the waveform of the tactile signals. The command “create new table” destroys a possibly already existing table and allocates memory on the USB controller for a new table of the given length. As the table might be larger than the maximum size of a data packet, the command “transmit part of the table” allows for the transmission of only a part of the table. The next command sets the number of channels, i.e. the number of stimulator pins to be controlled by the system. This value has also an impact on the sample rate of the generated tactile signals. The command “transmit amplitudes” is frequently issued by the tactile renderer running on the host computer. With this command the amplitudes an,i are set for each channel and signal frequency. The two following commands are used to start and stop the output of the USB controller. With the command “start output” the increments wi for the two signal frequencies can be configured.

3.3 Drive Electronics

37

Table 3.1 Protocol for the communication with the USB controller Command

Byte

Value

create new table

0 1

0 length

transmit part of the table

0 1 2 3–

1 offset length data

set number of channels

0 1

2 number of channels

transmit amplitudes

0 1 2–

3 signal number (0 or 1) amplitude data for the channels (descending order)

start output

0 1 2 3 4

4 reserved reserved increment for signal 1 increment for signal 2

stop output

0

5

set global gain

0 1

6 gain factor

The command “set global gain” is used to set the gain factor of the whole system. The transmitted value is forwarded to the variable voltage supply that sets the reference voltages −Vre f and +Vre f for the DACs.

3.3.2

Data Bus

The signals created by the USB controller are unidirectionally transmitted to DACs via a bus system which makes the system scalable in the number of contactors supported. The data bus consists of 8 data lines, 8 address lines and 2 control lines, namely “write” and “load”. The availability of a valid date and address is signalled with a high state on the control line “write” as depicted in Fig. 3.10. The data transmitted in this way is expected to be buffered at the location specified by the address lines. After all buffers have been updated a low state on the control line “load” signals that the data should be used, which guarantees that all DACs update their output simultaneously. The timing requirements of the bus are determined by the DACs used.

38

3 Devices for Tactile Simulation data

D1

D2

D3

D4

address

A1

A2

A3

A4

write load

Fig. 3.10 Digital timing diagram of the data bus

As there are 8 lines for the data 256 different voltages are available to define the output signals of the DACs. The bus also provides 256 different addresses of which the 4 highest are reserved for the variable voltage supply, which means that the system could be extended to drive up to 252 stimulator pins. On the upper right of Fig. 3.11 the address logic circuit can be seen. It consists of a 6 Bit dual in-line package (DIP) switch used to configure the address range of the board and a comparator IC. The comparator IC compares the address range defined by the switch with the highest 6 address lines and the “write” signal with a high state to signal the transfer of data into the buffer. The two lowest address lines are directly connected to the address ports of the four-channel DAC.

Fig. 3.11 The board hosting the DAC and the OPAs

The data bus galvanically isolates the USB controller and thus also the host computer from the rest of the system to avoid problems with noise. It also amplifies the signals from the USB controller. Both aims are reached with the circuit depicted

3.3 Drive Electronics

39

in Fig. 3.12. An optical coupler galvanically isolates both parts of the system. The 6N137 IC described in [5] features the necessary characteristics. Since the optical coupler inverts the signal and since the output currency of the CY7C64713 is not sufficient to drive the LEDs inside the optical couplers, an inverter is inserted between the output ports of the CY7C64713 and the input of the optical couplers. The use of optical couplers has the additional advantage that a different voltage may be used as ground voltage as needed for the DACs. The 18 optical couplers can be seen in the middle of Fig. 3.9. Fig. 3.12 The circuit galvanically isolating the USB controller from the rest of the system

VCC Vin

Vout

GND1

3.3.3

GND2

Variable Voltage Supply

The variable voltage supply is used to adjust the global gain of the system by setting the reference voltages for the DACs −Vre f and +Vre f . Its circuit is placed on the same board as the USB controller (see the right third of Fig. 3.9). Like the DACs for the channels it is also connected to the data bus and thus can be configured by the USB controller. A comparator compares the address on the data bus with a fixed address range from 252 to 255. If data is available for one of these addresses it is transfered to a DAC. In this circuit the AD558 IC is used for the digital–analogue conversion (see [1]). Dependent on the digital input the DAC sets its output to a voltage from 0 V to 2.55 V. This output is inverted twice by the circuit depicted in Fig. 3.13, which is implemented by the L272 IC (see [2]).

3.3.4

Digital–Analogue Conversion

The DAC receives data from the bus, which is kept in a buffer until the “load” signal indicates the output of the value as the new voltage. For the digital analogue conversion the MAX5100 IC described in [3], which provides four independent channels, was used. For the selection of the channel the two lowest address lines of the data bus were directly connected to the IC. The channels of the MAX5100

40

3 Devices for Tactile Simulation Vin

10k

10k

10k

10k

−Vref

Vref

Fig. 3.13 Amplification stages of the variable voltage supply

are already buffered. It provides a low-active input that initiates the transfer of the buffered value to the analogue output. The MAX5100 can be seen in the middle of the right half of Fig. 3.11.

Fig. 3.14 Passive, first order low-pass RC filter

R υ in C

υ out

To reduce imaging effects caused by the discrete quantisation of the output values of the DAC a passive, first order low-pass RC filter as depicted in Fig. 3.14 was used. The cutoff frequency fc , i.e. by definition the frequency at which the signal is damped by 3 dB, can be computed with fc =

1 . 2π RC

(3.7)

The filter used consists of a 33 kΩ resistor and a 4.7 nF capacitor, which results in a cutoff frequency of roughly 1 kHz. The values of the resistor and capacitor were chosen such that the sampling rate of the signals is well above and their maximum frequency content is well below the cutoff frequency.

3.3 Drive Electronics

3.3.5

41

Amplification

High-voltage operational amplifiers (OPA) are used to amplify the output of the DACs to the levels required by the bimorphs. The OPA551 described in [4] was used as amplifier in this circuit. The implemented circuit is able to amplify the output of the DACs, which have a maximum output of 2.5 V, to 60 V. To avoid high voltages with respect to the ground voltage the 2.5 V signal is first amplified by a non-inverting amplifier as depicted in Fig. 3.15. Fig. 3.15 Non-inverting amplifier with an adjustable gain factor between 11 and 13.5 200k υ in

50k

υ out

20k

The exact amplification factor can be adjusted with a potentiometer. The output voltage from the first amplification stage is inverted with respect to the ground voltage by an inverting amplifier with a fixed amplification factor of −1 as depicted in Fig. 3.16. Between the two resulting voltages the signal can in theory reach voltages of up to 60 V. Fig. 3.16 Inverting amplifier with gain factor -1 20k υ in

υ out 20k

Figure 3.17 shows the output of a single channel recorded with an oscilloscope. The signal is a superposition of two sinewaves with frequencies 40 and 320 Hz.

42

3 Devices for Tactile Simulation

Fig. 3.17 The output of a channel

3.4

Force-Feedback Devices

Force-feedback devices are much more common than tactile displays. While the latter are still rather experimental, a number of force-feedback devices is already commercially available. Figure 3.18 shows such a device. Fig. 3.18 PHANToM Omni haptic device by SensAble Technologies

Force-feedback devices only allow for a haptic interaction with virtual objects via a special tool. Such a tool could be a pen-like probe, a thimble or more special tools like for example the scalpel of a surgeon. The simulation of direct touch is not possible. Instead the user of the system touches the real representation of the tool, often called end effector. The force-feedback device tracks the position of the

3.4 Force-Feedback Devices

43

end effector and exerts forces on it. The end effector also has a representation in the virtual scene that follows the position of the real end effector.

3.4.1

The Haptic Loop

In the real world bodies never overlap. When real bodies touch, forces act on these bodies that prevent them from penetrating each other. The magnitude of the forces depends on the stiffness of the bodies. The same behaviour is simulated in the virtual scene. If the virtual end effector collides with another object in the virtual scene, the occuring forces are computed and the force acting on the end effector is sent to the force-feedback device. This uses its motors to create a similar force on the real end effector. This haptic loop is depicted in Fig. 3.19.

Fig. 3.19 The haptic loop

2!4+-+!,.! .$,&%$ $#-!"

!"#$% $$&'(#) &$*+#$ #!,,$#-$&.-! /01(,.!2$"(-!"

2/34+#(5.4+105(-+!, ! .*+"-0(5.4#$,$

!"#$.(#-+,6.!,.$,&%$ $#-!"

As the physical simulation of the virtual scene is implemented as a computer program, the physical state of the scene can only be updated in discrete moments in time. This, however, poses a problem: since the user decides on the position of the end effector between two discrete moments, the user can move it to a position that would require extremely high forces. This is particularly a problem if the stiffnesses of the virtual objects are high. To prevent these high forces the system has to be updated relatively often. Usually haptic systems are updated at 1000 Hz. Compared to computer graphics, where frame rates of 25 frames per second (fps) are usually sufficient, the relatively higher frame rate of force-feedback systems poses higher requirements to the computation of the physical simulation.

3.4.2

Modeling the Contact Forces

As the force acting on the end effector has to be computed in a very short time usually a simple model of the contact situation is used. The probably most simple and therefore widely used contact model is the spring–damper model: if the position

44

3 Devices for Tactile Simulation

Fig. 3.20 The spring– damper contact model

position at time t i−1

position at the surface

surface of the virtual object

position at time t i

of the end effector lies inside a virtual object a simple spring is attached to the end effector and to a point on the surface of the object as depicted in Fig. 3.20. At time frame ti−1 the end effector is still outside the object, but because of its movement it is already inside the object at time frame ti . To counteract this physically impossible situation the spring forces the end effector out of the object. The spring has a resting length of zero and a stiffness that is determined by the stiffnesses of the virtual object and the end effector. To avoid oscillation of the system a damper is added to the spring.

3.4.3

Different Degrees of Freedom

A body in three-dimensional space has a three-dimensional position and a three-dimensional orientation. Altogether it has six degrees of freedom (DOF). However, only some force-feedback devices fully provide six DOF. Many devices are able to track only the position and not the orientation of their end effector. These devices can only exert a three-dimensional force but no torque forces. One of these devices is the PHANToM Premium 1.0 haptic device by SensAble Technologies that has been used for an experiment described in Chap. 5. There also exist some force-feedback devices that are able to track all six DOF, but which cannot exert torque forces. Such a device was kindly provided by the PERCRO laboratory at the Scuola Superiore Sant’Anna in Pisa for the integration of the tactile display described in Sect. 3.2 with a force-feedback device as described in Chap. 5.

3.5

Conclusion

A lot of different tactile displays have been created by various research groups, but not one of these devices can create all tactile impressions necessary for a convincing tactile simulation. However, some displays follow interesting approaches that are worth to be further investigated. The display used in this work is such a display,

References

45

because it is based on the interesting idea of substituting a whole spectrum of frequencies with a signal consisting of only very few frequencies intended to cause an equivalent sensation. Such a scheme is already successfully utilised for the rendering of colours and might also be promising for tactile rendering. For the use of the displays with a tactile renderer running on a computer an electronic component had to be created to translate the digital signals into voltages suitable for the tactile display. This component has been designed following the tactile two-channel model, but is flexible enough to create other waveforms as well. Four modules of this drive electronics have been created and are in use at the MIRALab at the University of Geneva, PERCRO at the Scuola Superiore Sant’Anna in Pisa, the Biomedical Physics Group at the University of Exeter and the Institute of Man– Machine Communication at the Leibniz University of Hannover. A tactile display alone does not suffice to create a satisfactory simulation of direct touch to the full extend, because it does not provide the forces necessary for the creation of kinaesthetic sensory impressions. Therefore, it is sensible to combine a tactile display with a force-feedback device as described in Chap. 5. The principle function of these devices is described in Sect. 3.4.

References [1] AD558. Analog Devices, Inc., One Technology Way, Norwood, MA, USA (1987) [2] L272. SGS-THOMSON Microelectronics (1995) [3] MAX5100. Maxim Integrated Products, 120 San Gabriel Drive, Sunnyvale, CA, USA (1999) [4] OPA551, OPA552. Burr-Brown Corporation, 6730 S. Tucson Blvd., Tucson, AZ 85706, USA (1999) [5] 6N137. Fairchild Semiconductor Corporation (2005) [6] CY7C64713/14. Cypress Semiconductor Corporation, 198 Champion Court, San Jose, CA, USA (2005) [7] Allerkamp, D., B¨ottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: A vibrotactile approach to tactile rendering. The Visual Computer 23(2), 97–108 (2007) [8] Asamura, N., Tomokori, N., Shinoda, H.: A tactile feeling display based on selective stimulation to skin receptors. In: Proceedings of the Virtual Reality Annual International Symposium, pp. 36–42. IEEE Computer Society, Los Alamitos (1998) [9] Bernstein, L.E., Eberhardt, S.P., Demorest, M.E.: Single-channel vibrotactile supplements to visual perception of intonation and stress. The Journal of the Acoustical Society of America 85(1), 397–405 (1989) [10] Blume, S.: Konstruktion eines Ger¨ates zur Simulation von Oberfl¨achenstrukturen f¨ur den Einsatz in VR-Umgebungen. Student research paper, Welfenlab, Universit¨at Hannover (2006) [11] Caldwell, D.G., Tsagarakis, N., Giesler, C.: An integrated tactile/shear feedback array for stimulation of finger mechanoreceptor. In: Proceedings of IEEE International Conference on Robotics and Automation, vol. 1, pp. 287–292 (1999) [12] Cardin, S., Thalmann, D., Vexo, F.: Wearable obstacle detection system for visually impaired people. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 50–55 (2005)

46

3 Devices for Tactile Simulation

[13] Fontana, M., Marcheschi, S., Tarri, F., Salsedo, F., Bergamasco, M., Allerkamp, D., B¨ottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: Integrating force and tactile rendering into a single VR system. In: International Conference on Cyberworlds, pp. 277–284. IEEE Computer Society, Los Alamitos (2007) [14] Ikei, Y., Shiratori, M.: Textureexplorer: a tactile and force display for virtual textures. In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 327–334 (2002) [15] Ikei, Y., Wakamatsu, K., Fukuda, S.: Vibratory tactile display of image-based textures. IEEE Computer Graphics and Applications 17(6), 53–61 (1997) [16] Jungmann, M., Schlaak, H.F. (2002) Miniaturised electrostatic tactile display with high structural compliance. In: Proceedings of Eurohaptics (2002) [17] Kaczmarek, K.A., Tyler, M.E., Bach-y-Rita, P.: Pattern identification on a fingertipscanned electrotactile display. In: Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 4, pp. 1694–1696 (1997) [18] Killebrew, J.H., Bensma¨ıa, S.J., Dammann, J.F., Denchev, P., Hsiao, S.S., Craig, J.C., Johnson, K.O.: A dense array stimulator to generate arbitrary spatio-temporal tactile stimuli. Journal of Neuroscience Methods 161(1), 62–74 (2007) [19] Kontarinis, D.A., Son, J.S., Peine, W., Howe, R.D.: A tactile shape sensing and display system for teleoperated manipulation. In: IEEE International Conference on Robotics and Automation, vol. 1, pp. 641–646 (1995) [20] Konyo, M., Tadokoro, S., Takamori, T.: Artificial tactile feel display using soft gel actuators. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 3416–3421 (2000) [21] Matysek, M., Lotz, P., Schlaak, H.F.: Braille display with dielectric polymer actuator. In: Conference proceedings: Actuator. In: 10th International Conference on New Actuators & 4th International Exhibition on Smart Actuators and Drive Systems, pp. 997–1000 (2006) [22] Maucher, T., Meier, K., Schemmel, J.: An interactive tactile graphics display. In: Sixth International Symposium on Signal Processing and its Applications, vol. 1, pp. 190–193 (2001) [23] Moy, G., Wagner, C., Fearing, R.S.: A compliant tactile display for teletaction. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 3409–3415 (2000) [24] Pasquero, J., Hayward, V.: STReSS: A practical tactile display system with one millimeter spatial resolution and 700 Hz refresh rate. In: Proceedings of Eurohaptics 2003, pp. 94–110 (2003) [25] Schlaak, H.F., Jungmann, M., Matysek, M., Lotz, P.: Novel multilayer electrostatic solid state actuators with elastic dielectric. In: Bar-Cohen, Y. (ed.) Proceedings of the SPIE., Presented at the Society of Photo-Optical Instrumentation Engineers (SPIE) Conference, vol. 5759, pp. 121–133 (2005) [26] Shinohara, M., Shimizu, Y., Mochizuki, A.: Three-dimensional tactile display for the blind. IEEE Transactions on Rehabilitation Engineering 6(3), 249–256 (1998) [27] Summers, I.R., Chanter, C.M.: A broadband tactile array on the fingertip. The Journal of the Acoustical Society of America 112(5), 2118–2126 (2002) [28] Summers, I.R., Brady, A.C., Syed, M.K., Chanter, C.M.: Design of array stimulators for synthetic tactile sensations. In: Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 586–587. IEEE Computer Society, Los Alamitos (2005)

References

47

[29] Tang, H., Beebe, D.J.: A microfabricated electrostatic haptic display for persons with visual impairments. IEEE Transactions on Rehabilitation Engineering 6(3), 241–248 (1998) [30] Taylor, P.M., Moser, A., Creed, A.: The design and control of a tactile display based on shape memory alloys. In: IEEE International Conference on Robotics and Automation, vol. 2, pp. 1318–1323 (1997) [31] Wagner, C.R., Lederman, S.J., Howe, R.D.: A tactile shape display using RC servomotors. In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 354–355. IEEE Computer Society, Los Alamitos (2002) [32] Wellman, P.S., Peine, W.J., Favalora, G., Howe, R.D.: Mechanical design and control of a high-bandwidth shape memory alloy tactile display. In: Experimental Robotics V, pp. 56–66. Springer, Heidelberg (1998)

Chapter 4

Generation of Virtual Surfaces

To achieve a realistic tactile simulation of fabrics a display, a tactile renderer and virtual representations of the fabrics’ surface properties are needed. The two former requisites are covered in other chapters, the latter is the scope of this chapter. As this research aims at a realistic simulation of real fabrics a set of fabric samples has been acquired. These fabrics were used to test the methods for the generation of virtual surfaces and to conduct experiments for the evaluation of the tactile renderers. In Sect. 4.1 these sample fabrics are described. The first step in the generation of virtual surfaces from real surfaces is always the measurement of the real fabrics’ properties. A standard for the measurement of fabrics has been proposed by Kawabata in [11]. Although this standard was orignially intended to assess the subjective hand properties of fabrics, it was used within this work to measure the properties relevant for the generation of virtual surfaces and is therefore described in Sect. 4.2. There exist many different ways to model the surface properties of fabrics. In this work only a few methods could be tested. These are described in Sects. 4.3, 4.4 and 4.5. All these methods have in common that a fabric’s surface is represented as a two-dimensional property map P(u, v). However, the description of these properties depend on the model of the fabric. The generation of virtual fabrics is not a real-time operation and may even be manually supported. Virtual fabrics generated from real fabrics are not always suitable to test the tactile renderers. Artificial surfaces having well-defined properties were used instead. These surfaces are described in Sect. 4.6.

4.1

Sample Fabrics

The systems for tactile simulation developed in this work are not only theoretical constructs but aim at practical applications in the future. Therefore their usefulness has to be properly evaluated. In Chap. 2 the basis for appropriate evaluation methods is presented. This section is about the selection of fabric samples useful for the evaluation of the whole tactile simulation system and for testing the different algorithms. D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 49–74. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

50

4 Generation of Virtual Surfaces

Using fabrics for the evaluation is handy, because there exists a large variety of fabrics with different surface properties. It is possible to test the simulation system with samples having rather extreme properties, e.g. extremely rough or extremely smooth. Nevertheless it is also possible to test the system with samples that are not very different in order to investigate the ability of the system to simulate only very little differences. However, fabrics are just a test case, the systems could be generalised to a larger class of surfaces later.

4.1.1

Fibres, Yarn and Fabrics

Fabrics are usually made of yarns that are themselves made of fibres. There are also some fabrics that are not made of yarns. These are called “non-woven” fabrics and will be treated later in this section. Fibres are much longer than thick, i.e. the ratio length/diameter is very high. For example the length of a cotton fibre is about 2000 times greater than its diameter. This property of fibres makes spinning of yarns possible, because a large number of long, fine fibers grip each other when twisted in a thread. Figure 4.1 shows how fibres can be categorised (cf. [19]). Natural fibres either stem from plants like cotton, flax or jute or from animals like wool or silk. The manufactured fibres can either be created from natural or synthetic material. While natural fibres are always staple fibres, i.e. fibres with a length, there are some synthetic fibres that are continuous filaments. The latter are of indefinite length and run continuously throughout the length of the yarn without a break. The choice of the fibre material already determines a large quantity of a fabric’s properties. These properties can also be altered by applying special finishing processes. Fibres differ for example in their fineness, absorption of moisture, static electricity, tensile strength, extensibility, elastic recovery, stiffness, friction and their visual appearance. Fabrics are often made of blends to profit from the properties of the individual fibre types. For the production of yarn the fibres are usually first carded and sometimes even combed to align the fibres in one direction. After that the fibres are spun to form a yarn, which is called “single yarn”. If two or more single yarns are twisted together the result is called “folded yarn” and if two or more folded yarns are twisted together they form a “cabled yarn”. Fibres and yarns have a rather one-dimensional shape. Several mechanisms like for example weaving, knitting or felting exist to obtain a two-dimensional fabric from fibres or yarn, respectively. Fabrics are woven by interlacing two sets of threads to each other. These two sets are called warp and weft. The warp threads are already fixed to the loom while the weft threads run orthogonally to the warp threads. The interlacing pattern, known as the weave, is obtained by lifting some of the warp threads such that the weft thread currently interlaced runs under these warp threads and over the others. Figure 4.2a shows such a pattern where every second warp thread was lifted alternately. This pattern is called plain weave. It is the simplest and most frequently used weave (cf. [19]).

4.1 Sample Fabrics

51

!"#$!%&'()$*+

0!/#1!2"#$*3&'()$*+

,*%%#%-+* !""!# $%&' ()"*

!"#$!%&.-%45*$ ?-43!4* @!;&% A.-&3*"&"* =3*"&"* BC!3*%%

.$-"*(/ +!!% ,&-./0*121/3&456*.*7 8-%9/03)%"-:&"*;/!./<-%;7

64/"7*"(2&.-%45*$ D!%C&6-;* D!%C*4"*. E%&4"&#* 111

0(/*$!% =4>*4"!4

8"7*$+ &.>!#/F->.* G%&44/F->.*

Fig. 4.1 Categorisation of different fibre types

Weaves can be described on squared paper as shown in Fig. 4.2b. Columns correspond to warp threads while rows correspond to weft threads. Thus, each square represents a junction of a warp and a weft thread. It is filled when, looking at the fabric’s face side, the warp thread lies over the weft thread. Knitted fabrics consist of a series of interlinked loops of yarn. Two main categories of machine knitting exist: weft knitting and warp knitting. Figure 4.3 shows examples of these categories. A weft knitted fabric could be made from a single yarn. Loops are formed in succession by needles with hooks at their tops. The hook holds the loop. When the yarn arrives again it pulls the yarn through the existing loop forming a new loop in this way. Warp knitted fabrics are produced from a set of warp yarns. Loops are formed simultaneously by needles with hooks. To interlink the parallel threads, the threads are moved sideways at intervals. Unlike weft knitted fabrics warp knitted fabrics do not ladder, i.e. they cannot be unravelled. There exist several different definitions of “non-woven” fabrics (cf. [22]). In this work fabrics that are not produced by weaving or knitting are called “non-woven” fabrics. Examples for non-woven fabrics are leather, which is made from animal skin, and felt, which is directly made from fibres, not yarns.

52

4 Generation of Virtual Surfaces

(a) Plain weave fabric

(b) Weave diagram of plain weave

Fig. 4.2 Plain weave fabric with its weave diagram

(a) Face of plain weft knitting

(b) Back of warp-knitted half-tricot

Fig. 4.3 Examples of knitted fabrics

4.1.2

Selection of Sample Fabrics

A set of altogether 54 different fabric samples has been kindly provided by the Smart Wear Lab at the Tampere University of Technology. These fabrics are presented in [21] and the specification of all 54 samples is also tabulated in Annex A. The samples were acquired in three subsets. The first set of 32 samples was chosen such that the fabrics largely differed in many properties. It was expected to result in a very wide range of different virtual fabrics. The samples were differently fabricated (woven, knitted and non-woven), and they largely differed in their weight, thickness and in their fibre contents (cf. [14]).

4.2 Kawabata Evaluation System for Fabrics

53

The second set consists of 10 samples. These were taken from a narrower collection of men’s woollen or wool blend woven suit fabrics, weft and warp knitted blended or synthetic fabrics containing elasthane (cf. [21]). The third set of samples consists of 12 samples of which 6 are non-laminated synthetic warp knits used for car seats. The other samples consist of the originally non-laminated fabrics laminated to foam.

4.2

Kawabata Evaluation System for Fabrics

The subjective hand properties of fabrics are called “fabric hand”. These properties are assessed when a real fabric is touched and manipulated with the hands. Several more precise but slightly differing definitions exist (cf. [1]), but are not of greater interest for this work. The objective measurement of fabric hand preoccupied textile researchers for decades (cf. [4]). Today several measurement standards exist. The probably most well-known are the KES-F system (cf. [11]) developed by Sueo Kawabata and the FAST system (cf. [15]) developed by the research organisation CSIRO. The KES-F system consists of standardised testers for the bending, compression, tensile, shear, surface roughness and surface friction properties of fabrics. These testers measure the properties in weft and warp direction and for the surface properties on the face and back sides of the fabrics. From the data recorded by each tester characteristic values are calculated. Table 4.1 summarises these characteristic values together with their physical units. Tensile, bending, shearing and compression are elasto-mechanical properties while surface properties, weight and thickness are other physical properties related to the elasto-mechanical properties. The KES-F system is described in more detail in [11]. The KES-F standard uses the unusual unit gram-force (gf) which is defined as the magnitude of the force exerted on one gram of mass by standard gravity (9.80665 m s−2 ). Thus, 1 gf is equal to 0.00980665 N. Although the KES-F system is only intended to produce the aforementioned characteristic values, the testers could be used to obtain the whole measurements. To be able to further process the recorded data, a system for the conversion of the analogue measurements to digital data files was used as described in [14]. As the KES-F system is not intended for such a modification special care had to be taken to avoid problems with noise. As the surface measurements are the most important for this work they are described in more detail in this section. The surface properties were measured on both sides (right and back side) of most of the 54 fabric samples. For each side there are four measurement directions: weft and warp each in forward and backward direction. As most of the measurements were repeated there were altogether 611 measurement files available for the experiments. However, some measurements had to be discarded, because some fabrics were too difficult to measure, e.g. for some very stretchable fabrics material accumulated in front of the sensor.

54

4 Generation of Virtual Surfaces Table 4.1 The characteristic values of the KES-F properties (cf. [11]) Parameter

4.2.1

Definition

Unit

Tensile

LT WT RT EM

Linearity Tensile energy Resilience Max. extension

gf·cm/cm2 % %

Bending

B 2HB

Bending rigidity Hysteresis

gf·cm2 /cm gf·cm/cm

Shearing

G 2HG 2HG5

Shear stiffness Hysteresis at 0.5◦ Hysteresis at 5◦

gf/cm·◦ gf/cm gf/cm

Compression

LC WC RC

Linearity Compressional energy Resilience

gf·cm/cm2 %

Surface

MIU MMD SMD

Coefficient of friction Mean deviation of MIU Geometrical roughness

µm

Weight

W

Weight/unit area

mg/cm2

Thickness

T

Thickness at 0.5 gf/cm2

mm

KES-F Roughness Test

The KES-F roughness tester measures a one-dimensional height profile along a segment of length 2 cm. Therefore, the fabric is moved under the sensor at 1 mm s−1 for 20 s as depicted in Fig. 4.4. The tension of the fabric is kept at 20 gf cm−1 along the measurement direction.

Fig. 4.4 Schematic drawing of the KES-F roughness tester

4.2 Kawabata Evaluation System for Fabrics

55

10 gf 20 gf/cm 1 mm/s

(a) Photograph of the contactor

(b) Front and side view of the contactor

Fig. 4.5 The contactor of the KES-F roughness tester

The contactor of the sensor is a steel piano wire of diameter 0.5 mm. In the standard version it is bent as shown in Fig. 4.5. The sensor is pressed onto the fabric with a contact force of 10 gf by a spring with stiffness 25 gf mm−1 and the fabric is supported by a smooth steel plate. The data recorded by the sensor is digitised and saved in a measurement file for further processing. For the computation of the characteristic SMD value by the tester (see Table 4.1) the analogue signal is first passed through a high-pass filter with transfer function G( jω ) =

( jω )2 ( jω )2 + 2( jω )ξ ω

2 n + ωn

,

(4.1)

where ωn = 2π rad/s, ξ = 0.6 and ω is the angular frequency. This filter allows signals of wave length smaller than 1 mm to pass. The SMD value is the mean deviation of the surface thickness in µm and is calculated as 1 SMD = X

X

|T (x) − T¯ | dx

(4.2)

0

where X = 2 cm, x is the position of the contactor on the fabric, T (x) is the thickness of the fabric at position x and T¯ is the mean value of T . The calculation of the SMD value as done by the KES-F roughness tester has also been implemented for the digital data to test the consistency of the digital data and the result from the tester.

4.2.2

KES-F Friction Test

The surface friction is measured with the same device as the surface roughness, but with a different sensor. It consists of ten piano wires resulting in a larger contact area

56

4 Generation of Virtual Surfaces

50 gf 20 gf/cm 1 mm/s Fig. 4.6 Front and side view of the KES-F friction contactor

as shown in Fig. 4.6. The contactor is pressed onto the fabric with the compressional force of 50 gf by a dead weight. The speed at which the fabric is moved under the sensor is the same as with the roughness measurement. The MIU and MMD (see Table 4.1) values are calculated by the tester. As with the SMD value the measured signal first passes the high-pass filter. MIU is the mean value of the coefficient of friction and MMD is its mean deviation. The definitions of both values are 1 MIU = X MMD =

1 X

X 0 X

µ (x) dx

(4.3)

|µ (x) − µ¯ | dx

(4.4)

0

where X = 2 cm, x is the position of the contactor on the fabric, µ (x) is the frictional force at position x divided by the compressional force and µ¯ is the mean value of µ .

4.3

Spatial Frequency Analysis

The raw KES-F measurements are not adequate for use with the tactile renderer. They have to be converted to a two-dimensional property map P(u, v), where the map P depends on the requirements of the tactile rendering algorithm. One class of these algorithms translates the spatial frequency content of the surfaces into appropriate temporal frequencies. The preparation of the surface measurements for these algorithms is described in this section. The surface model is computed in two steps as depicted in Fig. 4.7. At first the fast Fourier transform (FFT) is used to obtain the spatial frequency content of the

Kawabata Surface Profile Measurements

Fast Fourier Transform

Two−Dimensional Composition

Small Scale Surface Model

Fig. 4.7 The sequence of the spatial frequency analysis

4.3 Spatial Frequency Analysis

57

measurements. The resulting spectra are combined to a two-dimensional property map.

4.3.1

Selection of Appropriate Sections

For the KES-F surface measurement the sensor is moved about 3 cm in forward direction and then about the same length in backward direction. During this procedure the height in µm for the roughness measurement and the friction force in gf divided by 50 gf for the friction measurement, respectively, are recorded at a sample rate of about 1000 Hz. For both measurements also the position on the fabric measured in cm is recorded. Before any further processing can take place two sections of length 2 cm are selected. The first section is taken from the part of the measurement with positive sensor speed (from 0.5 to 2.5 cm) and the second section is taken from the part with negative sensor speed (from 2.5 to 0.5 cm). The selected sections are sufficiently far away from the starting point at 0 cm, the turning point at 3 cm and the end point at 0 cm to avoid unwanted side effects that might occur in the proximity of these points, for example due to the change in velocity. Figure 4.8 shows an example of a selection of appropriate sections. 15 10 5 0 −5 −10 −15 −20

0

10000

20000

30000

40000

50000

60000

Fig. 4.8 Selection of appropriate sections

A linear regression analysis of the position measurements showed that a length of 2 cm corresponds to about 18,756 samples for all measurements. This result conflicts with the sensor speed of 1 mm s−1 specified in the KES-F and the sample rate of about 1000 Hz. However, as the KES-F tester is regularly calibrated it was assumed that it correctly measured the position and the sample rate was actually only

58

4 Generation of Virtual Surfaces

about 938 Hz. Therefore, a common length of 18,756 samples, corresponding to 2 cm, was used for all sections.

4.3.2

Discrete Fourier Transform

In Chap. 5 the surface models developed in this chapter are used to compute vibrations that occur when the finger moves over the fabric. In the approach that is prepared in this section the spatial frequency spectrum of the surface is directly transformed into the temporal frequency spectrum of the vibrations. As the measurements are discrete signals their frequency content is computed with the discrete Fourier transform (DFT). Here, only a very short introduction to the DFT is given. In [6] and [3] the topic is treated more comprehensively. The DFT detects harmonic components of a discrete signal fi with i = 0, . . . , n − 1 by comparing the signal with unit-waves of given frequency: n−1

Fk =

∑

i=0

fi cos

n−1 2π ki 2π ki − j ∑ fi sin . n n i=0

(4.5)

Here j is the imaginary number and therefore the Fk with k = 0, . . . , n − 1 are complex numbers. The absolute magnitude |Fk | = ℜ(Fk )2 + ℑ(Fk )2 describes the magnitude of the k-th frequency component while the angular measure ∠Fk = ℑ(Fk ) describes its phase. arctan ℜ(F k) If the DFT was directly implemented as shown in (4.5), each computation of the n different frequency components would involve a summation of 2n different multiplications. So the complexity would be O(n2 ). However, this complexity can be reduced: a FFT is an implementation of the DFT that has a complexity of O(n log n). The DFT assumes the signal to be periodic with a period corresponding to the length of the signal. However, this is rarely the case and discontinuities are introduced at the start and the end of the signal adding erroneous frequency content to its spectrum. To avoid this problem “windows” are used that are multiplied with the original signal and cause it to continuously vanish at the start and the end. Let zi be the discrete signal and wi be a window of the same length, i.e. i = 0, . . . , n − 1. The resulting signal is then defined as fi = zi wi . Windows commonly in use are the trivial rectangular window wi = 1, (4.6) the Blackman window wi = 0.42 − 0.5 cos 2π

i i + 0.08 cos 4π n−1 n−1

(4.7)

and the Hamming window wi = 0.54 − 0.46 cos

2π i . n−1

(4.8)

4.3 Spatial Frequency Analysis

59

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0 0

100

200

300

400

500

600

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

0

100

200

300

400

500

600

0

100

200

300

400

500

600

-1 0

100

200

300

400

500

250

600 160 140

200 120 100

150

80 100

60 40

50 20 0

0 1

10

100

1

10

100

Fig. 4.9 A rectangular (left column) and a Hamming window (right column) and their effect on a signal in time (second row) and frequency domain (third row)

Figure 4.9 shows the rectangular and the Hamming windows and their effect on a signal. In order to obtain a spatial frequency spectrum at different points on the fabric’s surface a Hamming window of length 4096, which corresponds to 4.37 mm, was used at different offsets of the measurement. The window was shifted by 938 samples or about 1 mm and the FFT was used to compute the spectra at the different offsets. In this way 16 different spectra were computed for a measurement with a length of 2 cm. Most fabrics are created such that there is a repetition in their surface pattern (cf. Sect. 4.1). Therefore, an ideal fabric would result in a periodic measurement which could be described with only one discrete frequency spectrum. However, due to irregularities in the yarns and in the weaves the measurements are usually not

60

4 Generation of Virtual Surfaces

perfectly periodic. As these irregularities affect the tactile properties of fabrics it is important to include them in the surface model. This is the reason why the frequency spectra are computed at different positions.

4.3.3

Two-Dimensional Composition

Up to now only the one-dimensional KES-F surface measurements were regarded in this section. However, these represent only a small part of the two-dimensional surface and are not sufficient to define the whole surface. But with some additional assumptions a reasonable surface model can be deduced. Most fabrics are created such that there is a repetition in their surface pattern (cf. Sect. 4.1). But due to irregularities in the yarns and in the weaves this repetition is not always clearly recognisable in the measurements. However, because of the randomness of these irregularities periodicities can often still be recognised in the measurements in weft and warp direction with lengths te and ta , respectively. It is assumed in the following that te ≤ 16 mm and ta ≤ 16 mm. As there are 16 different spectra per measurement, tˆe and tˆa are defined as tˆe := tˆa :=

max

kte ,

(4.9)

max

kta .

(4.10)

k∈IN, kte ≤16 mm k∈IN, kta ≤16 mm

If there is no recognisable periodicity observable at all, tˆe and tˆa are both set to 16 mm. The exact positions of the measurements on the fabric are not known. As the measurements are orthogonal to each other it is assumed that the measurement in weft direction lies on the u-axis of the fabric’s coordinate system whereas the warp measurement lies on the v-axis. For points on the u-axis or v-axis, i.e. if v = 0 or u = 0, the corresponding spectra can be found with ie = ⌊u

(mod tˆe )⌋

(4.11)

ia = ⌊v

(mod tˆa )⌋

(4.12)

for v = 0 and with for u = 0, respectively, where ie and ia denote the indices of the corresponding spectra. Here, the operator ⌊·⌋ is defined as ⌊x⌋ :=

max

x i∈Z, i≤ 0.001 m

i.

(4.13)

Figure 4.10 shows a grey-scale image of a fabric’s surface. As indicated by the green line the surface has a pattern with a certain angle. The surface structure along the weft and warp direction, respectively, has to be shifted according to that angle, as depicted by the red lines, to be consistent with each other.

4.4 The Correlation–Restoration Algorithm

61

Fig. 4.10 Example of a surface having a pattern with a certain angle

Let α be the angle between the u-axis and the direction of the surface pattern. Then a point (u, v) on the fabric’s surface can be projected to the axes with v , tan α vˆ =v − u tan α

uˆ =u −

(4.14) (4.15)

and after substitution of u, v with u, ˆ vˆ in (4.11) and (4.12) the indices of the spectra can be computed with

ie = u − v(tan α )−1 (mod tˆe ) , (4.16) ia = ⌊(v − u tan α ) (mod tˆa )⌋ . (4.17) These two equations are also depicted in Fig. 4.11.

4.4

The Correlation–Restoration Algorithm

In [9, 10] an algorithm is presented, which also uses the KES-F surface measurements to create a surface model. In contrast to the algorithm presented in the previous section, this algorithm constructs a two-dimensional height profile of arbitrary size from the one-dimensional measurements. In [9, 10] friction and sound is regarded as well, but for this research only the algorithm for the construction of the height profile was reimplemented (see [12]). As depicted in Fig. 4.12 the algorithm reduces the data from the KES-F surface tester to its essentials first and then constructs a two-dimensional surface from the remaining data.

62

4 Generation of Virtual Surfaces

Fig. 4.11 A point on the fabric’s surface is projected into the measurements

weft

1.

^te

1.

2.

α 2.

Kawabata Surface Profile Measurements

Data Reduction

Two−Dimensional Composition

warp

^ta

Small Scale Surface Model

Fig. 4.12 Construction of a two-dimensional height profile from the KES-F surface measurements

4.4.1

Data Reduction

The correlation–restoration algorithm (CRA) is used to decompose a signal into its strongest harmonic components and a rest, which is modeled as noise. It uses autocorrelation to emphasise the strongest harmonics and is therefore more stable in the presence of noise than the usual FFT alone. The algorithm is useful as a lossy compression technique as the components are computed in the order of their relevance. The latter holds paticularly for periodic functions. The remaining noise gives information about the error of the approximation, so maximum error values can be specified. Usually only a FFT is used to compute the frequency content of a given signal. The main idea of the CRA is to compute the autocorrelation of the signal before applying the FFT. According to [9] the advantage of this procedure is a greater stability in the presence of noise. For a discrete signal zi , with i = 0, . . . , n − 1, the autocorrelation is defined as Rzz (p) =

1 n−p−1 ∑ zi zi+p n − p i=0

(4.18)

4.4 The Correlation–Restoration Algorithm

63

for p = 0, . . . , n − 1, i.e. the autocorrelation itself is a discrete signal of the same length. The strongest harmonic component of the original signal is found by applying a FFT to its autocorrelation. In this way, however, only the frequency ω j of the harmonic component can be determined. To also obtain its amplitude and phase the correlation of the original signal with a unit wave si = cos(2πω j Δ xi) with i = 0, . . . , n − 1 is computed. Here, Δ x is the position difference between two consecutive sample points. This unit wave has the same frequency as the harmonic component found before, its amplitude is one and its phase is zero. The correlation of two discrete signals is defined as Rzs (p) =

1 n−1 ∑ zi si+p n i=0

(4.19)

for p = 0, . . . , n − 1. It is proven in [9] that the correlation Rzs (p) is a harmonic function with only one single frequency ω j and that the amplitude A j and the phase φ j can be obtained with A j = lim Rzs (p1peak ) − lim Rzs (p1valley ),

(4.20)

φ j = 2πω j Δ xp1peak = 2πω j Δ xp1valley + π .

(4.21)

n→∞

n→∞

It is of course impossible to have n → ∞ in the numerical implementation of the algorithm, but if N − p is large the error is negligible. After amplitude, frequency and phase of a harmonic component have been determined, the component is subtracted from the original signal and the procedure is repeated until there are not any clearly recognisable harmonic components left in the signal. The remaining signal is assumed to be noise.

4.4.2

Two-Dimensional Composition

Like before the one-dimensional KES-F surface measurements have to be composed to create a two-dimensional surface. Similarly to Sect. 4.3.3 the angle α between the fabric weaving pattern and the weft direction is used in [9, 10] to define a twodimensional height profile: ne

Z(x, y) = ∑ Aei cos(2πωie (x + y tan α ) + φie ) i=1 na

+∑

(4.22)

Aai cos(2πωia (y − x tan α ) + φia )

i=1

where −45◦ < α ≤ 45◦ . The height profile defined in this way is a superposition of cosine waves that linearly propagate in two perpendicular directions. These waves are defined by their amplitudes Aei and Aai , their projected frequencies ωie and ωia and their projected phases φie and φia .

64

4 Generation of Virtual Surfaces

A frequency component wei appears in the spectrum of the KES-F measurement in weft direction and multiplied with tan α also in the measurement in warp direction. Similarly wai appears in the warp measurement and multiplied with tan α also in the weft measurement. Therefore, these pairs of harmonic components in weft and warp direction have to be identified and modeled with only one corresponding harmonic component in (4.22). It is claimed in [10] that over 70% of all fabrics can be modeled with such a simple harmonic function. This is probably true for the large-scale structure of the fabrics but questionable for small structure sizes. However, for the indirect haptic interaction via a stylus as investigated in [9, 10] the large-scale structure is probably sufficient to model a fabric. This model is further investigated in Chap. 5.

4.5

Surface Reconstruction from an Optical Surface Scan

An optical surface scan of a fabric sample is easy to obtain and can be used to reconstruct its height profile. This method called “Shape from Shading” is tested in [18] and [12] for the application in haptic virtual-reality environments. Results are presented in [17].

Optical Surface Scan

Finding Tilings

Shape From Shading

Small Scale Surface Model

Fig. 4.13 Surface Reconstruction from an Optical Surface Scan

A method that reconstructs a height function from a grey-value image is described in this section (see Fig. 4.13). The first step in this reconstruction process is the detection of symmetry in the grey-value image. The term “symmetry” is defined with regard to a set P ⊆ E 2 (E 2 denoting the Euclidean plane) as a bijective map α : E 2 → E 2 with α (P) = P. Every symmetry of E 2 is a translation, rotation, reflection or glide reflection. The symmetries of most fabrics are combinations of two nonparallel translations. These translations define a basic element called “tile” or “primitive” that is repeated in the translational directions (see Fig. 4.14). An algorithm for finding such a tile was implemented based on [7] at the Institute of Man–Machine Communication at the Leibniz University of Hannover. As the symmetries of real surfaces are not perfect in general, stochastic methods are used to find an idealised tiling. Since the tile is repeated on the grey value image the average of these repetitions is used to create a new synthetic image of the surface, that way being cleaned from noise. This image in turn is used as input for a shape-from-shading algorithm (see [20]) that computes an appropriate height profile. See Fig. 4.15 for an example. The tile can be used to describe fabrics of arbitrary size with relatively few data.

4.5 Surface Reconstruction from an Optical Surface Scan

65

Fig. 4.14 Tile with translations t1 and t2

t2 t1

Fig. 4.15 Photograph of a fabric and the corresponding height profile

4.5.1

Symmetry Detection

In order to extrapolate the fabric sample beyond its borders the smallest tile has to be found, which is equivalent to the detection of the isometry group. An algorithm to obtain this tile using stochastic methods is described in [18] and in more detail in [16]. To detect symmetrical portions of a texture, its picture has to be segmented into repeating sections. The average of all these sections is called primitive and can provide synthetic textures of arbitrary size. Our implementation of the symmetry detection employs ideas originating from [7] and [13]. The first step of the identification of symmetries within textures is the creation of the autocorrelation matrix (ACM), which describes the translations of parts of the texture onto other parts of the same texture. A statistical feature matrix (SFM) of a discrete grey-value image t under the operation αα1 ...αn with an arbitrary feature f t is defined as

66

4 Generation of Virtual Surfaces ft

SFMαα1 ...αn (t) = ( f t(t, αα1 ...αn (t)))αα1 ...αn .

(4.23)

In this context an operation αα1 ...αn is a translation in the Euclidean plane E 2 . Let p, q, r ∈ E 2 . Then a translation τ from p to q is defined as

τ p,q (r) = r + (q − p).

(4.24)

The translation may not leave the region of the grey-value image. Furthermore, to achieve reasonable results each translation needs to affect at least one fourth of the original image. The correlation ρ was utilised as feature f t. It is defined as Cov(t,t(s)) ρ (t,t(s)) := Var(t)Var(t(s))

(4.25)

where Cov() denotes an estimator for the covariance and Var() for the variance. The ACM is an SFM with translation τ p,q as operation αα1 ...αn and correlation ρ as feature f t. From the ACM a grid of primitives can be computed with the following steps: 1. identification of the maxima of the ACM 2. computation of the main directions of the maxima 3. computation of the distances between the lines of the grid The correlation describes the amount of congruence of two parts of the texture being compared. Large elements of the ACM correspond to a large congruence (see Fig. 4.16). Therefore, the maxima of the ACM are of interest.

Fig. 4.16 The grey value image of a texture (left) and its ACM (right), bright pixels in the ACM correspond to large values

As not all maximal values are adequate for the computation of the grid, criteria for the selection of the maxima have to be defined. Because of the large amount of different textures an automatic determination of these criteria is not possible. Therefore, a minimum distance R between two adjacent maxima and a threshold S

4.5 Surface Reconstruction from an Optical Surface Scan

67

for the value of the maximum have to be set. Let ACM(i, j) denote the value of the autocorrelation matrix at position (i, j). It is a maximum if and only if S < ACM(i, j)

(4.26)

and for all ACM(p, q) with i − R2 ≤ p ≤ i + R2 and j − R2 ≤ q ≤ j + R2 and ¬(i = p ∧ j = q) ACM(p, q) < ACM(i, j).

(4.27)

Given the maxima of the ACM the main directions of the grid can be computed. Therefore the Hough transformation recognising straight lines and circles is used. Straight lines in the Euclidean plane can be uniquely described by d = x cos α + y sin α

(4.28)

where α is the angle between the normal vector and the x-axis and d is the distance to the origin. For the computation of the main directions the angle α runs in small steps from 0 to π . The size of the steps depends on the size of the image. Each point P(x, y) in the set of the maxima can be seen as point on a straight line with angle α . The distance d can be computed with (4.28). Points resulting in similar distances are considered to be on the same line. The two angles with most matching distances are selected as main directions. We denote them by q1 and q2 . To get the length of the vector with direction q1 , all maxima of the ACM are projected in direction q2 onto a straight line with direction q1 . The projected position of each maximum can be understood as a normal distribution with given variance σ . The maximum of the superposition of all these distributions is taken as optimal distance for the vector with direction q1 (see [16]). The length of the vector with direction q2 is computed in the same way, with q1 and q2 being interchanged. In Fig. 4.17 the intermediate steps are depicted: First the maxima of the ACM are extracted. Then the main directions are found (here represented by two lines). In the last step the lengths of the spanning vectors are computed. Here, the length of the vectors is represented by the shape of the primitive. The primitive is computed by averaging values from the original picture. In this way the primitive is cleaned from noise. The picture on the bottom right shows the synthetically generated image of the texture. It is created by repetition of the primitive.

4.5.2

Shape from Shading

Shape-from-Shading (SFS) algorithms compute a surface profile of an object from one or more two-dimensional images. Pioneer work in this field was done by Horn in [8].

68

4 Generation of Virtual Surfaces

Fig. 4.17 Maxima of the ACM, the directions represented by two lines, the primitive and the synthetically generated image (from top left to bottom right)

The algorithm used here was developed by Tsai and Shah in [20]. It extracts the information of the height from only one picture. In [23] the algorithm of Tsai and Shah is recommended for images with little noise and a source of light being close to the object. These two criteria are fulfilled in our case, since the symmetry detection cleans the image from noise and the textures were digitised with a flat bed scanner, so that the source of light was indeed very close to the texture.

4.6

Artificial Surfaces

For testing purposes artificial surfaces were generated. Figure 4.18 shows some examples, where the grey values represent different heights. These surfaces were used to validate the functioning of the tactile renderer. Synchrony of the visual and the tactile display was tested with bars of variable width. It is shown in [2] that there exists a strong correlation between the fractal dimension of a haptic texture and the impression of roughness perceived by tactually exploring it with a pen-like probe. Therefore, surfaces with different fractal dimensions were used for experiments.

4.6 Artificial Surfaces

69

Fig. 4.18 Artificially generated surfaces with bars (left and middle) and sinusoidal Surfaces (right)

4.6.1

Brownian Surfaces

Brownian surfaces are randomly generated surfaces with a defined fractal dimension. Before the construction of these surfaces is described, one of the various ways to define fractal dimension, the Hausdorff dimension, is presented at first. Let F ⊆ IRn , F = 0/ with d(x, y) denoting the Euclidean distance in IRn . Then diam(F) := sup{d(x, y)|x, y ∈ F} is called the diameter of F. Let δ ∈ IR+ and {Ui }i∈IN be a countable set of subsets of IRn with diam(Ui ) ≤ δ . Then {Ui }i∈IN is called a δ -cover of F if F ⊂ ∞ i=1 Ui . Let s ∈ IR+ . For every δ ∈ IR+ let Hδs (F) := inf

∞

∑ (diam(Ui ))s |{Ui}i∈IN δ -cover of F

.

(4.29)

i=1

Then the s-dimensional Hausdorff measure is defined as H s := lim Hδs (F). δ →0

Hausdorff proved the existence of a number DF fulfilling ∞ for s < DF s H (F) = 0 for s > DF .

(4.30)

(4.31)

Now the Hausdorff dimension dimH (F) can be defined as dimH (F) := inf{s|H s (F) = 0} = sup{s|H s (F) = ∞}. The Hausdorff dimension dimH (F) has some basic properties: • If F ⊂ IRn , then dimH (F) ≤ n. • If F ⊂ G, then dimH (F) ≤ dimH (G). • If F is countable, then dimH (F) = 0.

(4.32)

70

4 Generation of Virtual Surfaces

A Brownian surface XH : IR2 → IR with index H (0 ≤ H ≤ 1) is a stochastic process with: 1. XH (0, 0) = 0. 2. XH (x, y) is continuous. 3. Let σ , τ ≥ 0. Then the increments XH (x + σ , y + τ ) − XH (x, y) are normally disH tributed with expectation 0 and variance (σ 2 + τ 2 ) =: ρ , i.e. − 21

P(XH (x + σ , y + τ ) − XH (x, y) ≤ z) = (2π )

− 12

(ρ )

z

−∞

2 r dr. exp − 2ρ H

XH (x, y) itself is also normally distributed with expectation 0 and variance (x2 + y2 ) . Figure 4.19 shows some examples of Brownian surfaces with different indices. It is shown in [5] that a Brownian surface XH (x, y) with index H has Hausdorff and box dimension dimH XH (x, y) = dimB XH (x, y) = 3 − H. In the following these dimensions are shortly refered to as fractal dimension dimXH (x, y).

Fig. 4.19 Brownian surfaces with index 0.1 (red), 0.5 (green) and 0.9 (blue)

A Brownian surface with (x, y) ∈ [0, 1] × [0, 1] can be constructed with the following algorithm (see [18]): 1. Set XH (0, 0) = 0 and randomly draw XH (0, 1), XH (1, 0) and XH (1, 1) from a normal distribution with expectation 0 and variance 1. 2. Iterate over (i, j) and randomly draw XH (k2−i , l2− j ) from a normal distribution with expectation

4.6 Artificial Surfaces

71

1 XH (k − 1)2−i, (l − 1)2− j + XH (k − 1)2−i, (l + 1)2− j 4 +XH (k + 1)2−i , (l − 1)2− j + XH (k + 1)2−i, (l + 1)2− j

and variance

2(−i− j)H . The variables k, l, i and j have to satisfy 0 ≤ k ≤ 2i , i ≥ 1, k odd and 0 ≤ l ≤ 2 j , j ≥ 1, l odd. Because of the continuity of XH (x, y) the resulting surface is completely determined. The correlation between fractal dimension and impression of roughness was investigated in [2] using a single-point contact with the virtual surface. The PHANToM device was used to simulate the feel of surfaces with different indices H. The test subjects could use a pen-like probe to explore the surfaces. To permit comparability the smoothest and the roughest surface were presented and they were given fixed roughness values of 1 and 20. The test persons were asked to rate randomly chosen surfaces inside this range. Figure 4.20 shows some results of this experiment. As one can see there exists a strong correlation between the fractal dimension of a haptic texture XH (X,Y ), i.e. dim XH (x, y) = 3 − H and the impression of roughness received by tactually exploring it.

20 test person 1 test person 2 test person 3 test person 4 test person 5

18 16 14 12 10 8 6 4 2 0 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Fig. 4.20 Results of the experiment with the indices H of the Brownian surfaces on the abscissa and the impression of roughness on the ordinate (with 20 corresponding to the roughest and 1 to the smoothest surface)

72

4.7

4 Generation of Virtual Surfaces

Conclusion

Two different types of surfaces are investigated in this chapter: artificial surfaces with well-defined properties described in Sect. 4.6 and real surfaces of 54 different sample fabrics which are described in Sect. 4.1. The aim of this chapter is the generation of virtual surfaces that are useful for the tactile simulation systems described in Chap. 5. In the case of artificial surfaces only the computation of height profiles is regarded (see Sect. 4.6), but in the case of real surfaces the surface properties have to be measured and digitised before any further processing can take place. Three different sources of information are used in this chapter: information on the production process of fabrics described in Sect. 4.1, the one-dimensional KES-F surface measurements as described in Sect. 4.2 and a two-dimensional optical surface scan obtained with a standard flat bed scanner. From these sources of information either descriptions of the spatial frequency content at different positions on the surface or two-dimensional height profiles are computed with different methods. These methods are described in Sects. 4.3, 4.4 and 4.5 and are indicated with different types of arrows in Fig. 4.21.

()"% +('"-)

$,'/(2"#$/, /,&'"7($-"#$/, !(/-)

/,)01$2), $/,"% 34506 2)" +()2),#

!"#$"%& '()*+),-. -/,#),#

"(#$'$-$"% +('"-)

#8/01$2), $/,"% /!#$-"% +('"-)& -",

#8/01$2), $/,"% 9)$:9#&!(/'$%)

Fig. 4.21 Four different methods for the generation of surface models indicated by the different types of arrows

References

73

The method described in Sect. 4.3 is the only one described in this chapter that does not create a height profile. Instead it describes a surface in terms of its local spatial frequency content. In contrast to a height profile this is a description convenient for the tactile renderer, because it computes the tactile signals for the tactile display in the frequency domain anyway as described in Chap. 5. Both methods described in Sects. 4.3 and 4.4 analyse the frequency content of the KES-F surface measurements. While the spectra are adopted without any changes in Sect. 4.3, the method in Sect. 4.4 attempts to separate the true harmonic components from the noise. The latter is necessary because a two-dimensional height profile is constructed from the two perpendicular one-dimensional KES-F surface measurements under the assumption that the whole surface can be modeled with simple harmonic functions. As in Sect. 4.3 additional information on the production process of fabrics has to be integrated for the two-dimensional composition. Therefore, both methods can only be applied to regular fabrics or very similar surfaces. The methods fail with more complicated fabrics like Jacquard fabrics. The method described in Sect. 4.5 does not require specialised hardware like the KES-F testers but works with a standard flat bed scanner. Because the measurement is already two-dimensional it does not require the assumptions needed for the two-dimensional composition carried out by the other methods. However, it is assumed that a tiling of the regarded surface can be found in order to be able to clean it from noise and to extrapolate the height profile beyond the borders of the image taken by the flat bed scanner. Furthermore, the method does only work with monochrome surfaces, because otherwise the darker colours are recognised as shades by the shape-from-shading algorithm.

References [1] Behery, H.M. (ed.): Effect of mechanical and physical properties on fabric hand. Woodhead Publishing Limited (2005) [2] Bergmann, M., Herbst, I., von Wieding, R., Wolter, F.E.: Haptical rendering of rough surfaces using their fractal dimension. In: Proceedings of the First PHANToM Users Research Symposium, German Cancer Research Center, Heidelberg, Germany, pp. 9– 12 (1999) [3] Bloomfield, P.: Probability and Statistics, 2nd edn. Fourier Analysis of Time Series. John Wiley and Sons, Chichester (2000) [4] Boos, A.D.: Concepts and understanding of fabric hand, ch. 2, pp. 11–44. Woodhead Publishing Limited (2005) [5] Falconer, K.J.: Fractal geometry: mathematical foundations and applications. Wiley, Chichester (1990) [6] Grover, D., Deller, J.: Digital signal processing and the microcontroller. Prentice-Hall, Englewood Cliffs (1998) [7] Handley, C.: The analysis and reconstruction of repetitive textures. In: Computer Graphics International 1998, pp. 273–276. IEEE Computer Society, Los Alamitos (1998) [8] Horn, B.K.P.: Shape from shading: A method for obtaining the shape of a smooth opaque object from one view. PhD thesis, Department of Electrical Engineering, Massachusetts Institute of Technology (1970)

74

4 Generation of Virtual Surfaces

[9] Huang, G.: Feel the “fabric” via the phantom. PhD thesis, University of Pennsylvania (2002) [10] Huang, G., Metaxas, D., Govindaraj, M.: Feel the “fabric”: an audio-haptic interface. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer animation, Eurographics Association, pp. 52–61 (2003) [11] Kawabata, S.: The standardization and analysis of hand evaluation. Tech. rep., The Hand Evaluation and Standardization Commitee, The Textile Machinery Society of Japan. Osaka Science and Technology Center Bld., 8-4, Utsubo-1-chome, Nishi-ku, OSAKA 550 Japan (1980) [12] K¨obberling, T.: Verschiedene Methoden zur Generierung virtueller taktiler Oberfl¨achenstrukturen. Master’s thesis, Welfenlab, Universit¨at Hannover (2006) [13] Kruse, S.M.: Ein Beitrag zur Segmentierung von Bildszenen im Kontext der objektorientierten Bilddatencodierung. Wiss.- und Technik-Verlag, Berlin (1997) [14] M¨akinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of physical parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 8–16 (2005) [15] Minazio, P.G.: Fast – fabric assurance by simple testing. International Journal of Clothing Science and Technology 7(2/3), 43–48 (1995) [16] Peinecke, N.: Symmetriefindung in periodischen Texturen. Student research paper, Welfenlab, Universit¨at Hannover (1999) [17] Peinecke, N., Allerkamp, D., Wolter, F.E.: Generating tactile textures using periodicity analysis. In: 2007 International Conference on Cyberworlds, pp. 308–313. IEEE Computer Society, Los Alamitos (2007) [18] Schulze, M.: Von computergraphischen zu haptischen Texturen. Virtual Reality f¨ur den Entwicklungsbereich Design/Styling in der Automobilindustrie. PhD thesis, Welfenlab, Universit¨at Hannover (2005) [19] Taylor, M.A.: Technology of Textile Properties, 3rd edn. Forbes Publications, London (1990) [20] Tsai, P.S., Shah, M.: Shape from shading using linear approximation. Image and Vision Computing Journal 12(8), 487–498 (1994) [21] Varheenmaa, M., Meinander, H.: Mechanical properties as a base for haptic sensing of virtual fabrics. In: Proceedings of the Autex 2007 Conference, Tampere, Finland (2007) [22] Vaughn, E.A.: An introduction to textile manufactoring — principles, products and processes. Tech. rep., School of Textiles, Fiber and Polymer Science, Clemson University, Clemson, South Carolina, USA (1996) [23] Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Analysis of shape from shading techniques. In: IEEE Computer Vision and Pattern Recognition 1994 Conference, pp. 377–384. IEEE Computer Society, Los Alamitos (1994)

Chapter 5

Tactile Rendering

While moving a fingertip over a fine surface we experience a sensation that gives us an idea of its properties. A satisfactory simulation of this feeling is still an unsolved problem, although a lot of different tactile displays have been developed. In most attempts to display real surfaces only a few features, like e.g. ridges and grooves, are simulated. A system for the display of real surfaces is presented in [8], but the computation of the tactile signals is only based on a photograph of the surface texture and is not further motivated with respect to its physical correctness. In this chapter rendering strategies based on vibrations, which play an important role in the tactile exploration of fine surfaces, are described. To produce appropriate excitation patterns the tactile displays described in Sect. 3.2 are used. The tactile renderers described in this chapter are all based on the idea of using a superposition of vibrations at 40 and 320 Hz to stimulate the SA1 and PC channel, respectively. These frequencies have been chosen because the two tactile channels are especially sensitive in the respective frequency regions and the fact that one frequency is a multiple of the other is quite handy for the generation of the vibrations. This approach has the advantage that only amplitude data have to be transmitted to the electronic component driving the tactile display. But it is not immediately clear how the amplitudes have to be computed from a given surface model. Several reasonable approaches exist and only some of these could be investigated within this work. All renderers have in common that the position of the finger on the surface has to be tracked, that appropriate commands have to be sent to the tactile display via USB, that a possibility to select the fabric to be rendered has to be offered and that also a visual feedback has to be given to the user. Therefore a common framework was implemented which is described in Sect. 5.1. When the first experiment was conducted a tactile display was not available yet. In order to test the experimental methods presented in Sect. 2.1 with the methods for the generation of virtual surfaces described in Sects. 4.4 and 4.5 a force-feedback device was used instead. The experimental setup and the results are described in Sect. 5.2.

D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 75–100. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

76

5 Tactile Rendering

The tactile renderer presented in Sect. 5.3 extracts the two amplitudes for the 40 and 320 Hz vibrations from the frequency spectrum of the time-varying height along the trajectory of the finger on the rendered surface. For each tactile channel a band-pass filter is used to extract the relevant parts of the frequency spectrum. A single amplitude is obtained via integration. The tactile simulation system was integrated with a force-feedback system which was developed within the HAPTEX project. The involved software components and their integration as well as the integrated hardware are described in Sect. 5.4. The results of a subjective evaluation of the system is also described.

5.1

Rendering Framework

A common framework was developed for the different versions of the tactile rendering algorithms, which is described in this section. It consists of four threads executed in parallel as shown in Fig. 5.1. Two threads are responsible for acquiring the input data needed by the two other threads. The integrity of the data shared between the threads is guaranteed by using mutex objects.

#/"&)'10"9!%&' .,1-.!+'#"&)

-.!+'#"&)

*!"-10"9!%&'

!),*)! 78156

34156

,)(-.!+ ($!)"*

(!"&+%, ($!)"*

("&(%/) !),*)!%, ($!)"*

!"#$%&' ($!)"*

!),*)!

')('1#.'%(%.,1.0 ("&(%/)1*%'#/"2

#.'%(%., .01("&(%/) *%'#/"2

*!"-1("&(%/) *%'#/"2

Fig. 5.1 The four threads of the tactile rendering framework

The network thread listens on a network port for user requests and accordingly places fabrics on a virtual workspace. In this way an experiment supervisor is able to set up the workspace for the subject from a remote computer. The tracking thread continuously receives positional data from a device tracking the position of the tactile display. For this work a standard computer mouse, a

5.1 Rendering Framework

77

graphics tablet or a force-feedback device were used as tracking device. Figure 5.2 shows the tactile display connected to a computer mouse used as tracking device. The whole platform is supported by roller bearings that allow it to be moved on the table in any direction.

Fig. 5.2 The tactile display connected to a computer mouse used as tracking device

The function implementing the rendering algorithm is constantly called by the tactile rendering thread every 25 ms, which corresponds to 40 Hz. This function computes the new amplitudes for the tactile display based on the positional data from the tracking device and the configuration of the virtual workspace. The graphics thread shows the workspace and the position of the tactile display on the subject’s computer screen (see Fig. 5.3). The screen is constantly updated at 25 Hz.

5.1.1

Position Tracking

Different devices were used to track the position of the tactile display. The accuracy of the position measurements is mainly influenced by the spatial resolution of the device, usually specified in lines per inch (lpi), and by the sample rate at which the position is measured. Apart from the position the tactile rendering algorithms also need to know the velocity of the display. A Kalman filter not only reduces noise but also can compute the velocities corresponding to the measured positions (cf. [24]). It is based on the

78

5 Tactile Rendering

Fig. 5.3 Graphical presentation of the workspace and the position of the tactile display pins (depicted as green boxes)

idea of estimating the state x ∈ IRn of a discrete-time controlled process by predicting the state of the next time step and by correcting the predicted state using the corresponding measurement of the process as depicted in Fig. 5.4.

!"#$%&'()# *&+#'!,)!-./

0#(12+#"#.) %&'()# *,-++#,)!-./

Fig. 5.4 Kalman filter as a predictor–corrector method

The discrete-time controlled process mentioned above is described by xk = Axk−1 + Buk + wk−1 .

(5.1)

The matrix A ∈ IRn×n relates the state at time step k − 1 to the state at time step k. For the position tracking the state is chosen to be ⎛ ⎞ p x = ⎝v⎠ (5.2) a

5.1 Rendering Framework

79

where p, v and a denote the position, the velocity and the acceleration, respectively. The used model assumes the acceleration a to be approximately constant and thus the matrix A is chosen to be ⎞ ⎛ 1 Δ t 12 Δ t 2 (5.3) A := ⎝0 1 Δ t ⎠ 0 0 1 where Δ t denotes the time between two time steps. The term Buk describes a driving force, which is not existent in the model used for the position tracking. The random variable wk−1 represents the process noise, which is assumed to be white, with a normal probability distribution, a mean of zero and a covariance matrix Q. The latter is defined as ⎛ 1 4 1 3 1 2⎞ 4Δt 2Δt 2 Δt (5.4) Q := ⎝ 12 Δ t 3 Δ t 2 Δ t ⎠ var(a). 1 2 Δt Δ t 1 2

In this way the influence of the user on the position of the tactile display is modeled as noise of the acceleration. The variance of the acceleration has been experimentally estimated as 90 m s−2 . A measurement zk ∈ IRm is related to the state with zk = Hxk + vk .

(5.5)

As only the position is measured H has the form H := 1 0 0 .

(5.6)

R = (0.001 m).

(5.7)

The random variable vk represents the measured noise. Like the process noise it is assumed to be white, with a normal distribution, a mean of zero and covariance matrix R. For the position tracking R is a 1 × 1 matrix and has been estimated as

In our model (see Fig. 5.5) every contactor of the tactile array is assigned a twoxi 0u (t) dimensional device coordinate . Let denote the position of the origin 0v (t) yi of the device coordinate system on the fabrics surface (trajectory) and φ (t) the angle between the x-axis and the u-axis at the time t, then the position of the contactors on the fabric’s surface can be computed with ui (t) xi 0u (t) cos φ (t) − sin φ (t) = + (5.8) yi sin φ (t) cos φ (t) vi (t) 0v (t) Using the property map P(u, v) from Chap. 4 we can now define the time dependent property function Pi (t) := P(ui (t), vi (t)) along the trajectory.

80

5 Tactile Rendering

y

x

Fig. 5.5 Tactile contactors on the fabric’s surface

φ

v u 5.2

Experiment with a Force-Feedback Device

In Chap. 4 different methods for the generation of virtual surfaces are presented. In order to compare these methods using the evaluation methods described in Chap. 2 a first series of experiments was conducted. The main intention of these experiments was not to compare the virtual surface models but to test whether the evaluation methods are useful for such a comparison. The experiments presented in this section are also described in [9] and in [14]. When this first series of experiments was conducted a tactile display was not yet available then. However, this was not a real drawback as the intention was to try the evaluation methods first and not to already evaluate the tactile simulation systems. It was decided to simulate the surface textures with a force-feedback device as there already existed some experience in this area (cf. [16] and [13]). However, it should be noted that the pen-like probe used for the experiments is not a natural tool to assess the properties of fabrics. Therefore, this series of experiments is unsuitable for the evaluation of the usefulness of the methods for the generation of virtual surface models for tactile simulation systems.

5.2.1

Method

Two methods are compared in this section: The method described in Sect. 4.4, here shortly refered to as the “Kawabata method”, and the method described in Sect. 4.5, here simply called “optical method”. Both methods create a height profile which is then used to render the information using a PHANToM Premium 1.0 haptic device as described in Sect. 3.4. Since we wanted to find out more about discriminability, we chose a subset of relatively similar fabrics. Furthermore, for our experiments the haptic properties of the fabrics had to be visible in order to be reconstructable from an optical scan.

5.2 Experiment with a Force-Feedback Device

81

These criteria resulted in a test set of the cloth samples 8, 9, 18, 22, 23, 28, 29 (see Annex A). To compare both methods with each other, and to compare them with real fabrics, a setup similar to [18] was chosen. In a triangular method three objects were presented to the subject, with two of them being identical. The subject had to tell, which one of the three objects was different from the two other ones (cf. Sect. 2.1.3). After that, the subject had to judge, if it was “easy”, “normal” or “hard” to make a decision. Since each odd-man-out experiment incorporates two different textiles, each subject had to answer n(n − 1)/2 questions for n samples. In order to understand which properties of the fabrics are important for the discrimination task an MDS procedure was employed (see Sect. 2.1.6). To gather the perceptual dimensions subjective distances between the fabrics had to be found. In Sect. 2.1.5 the computation of subjective distances is explained. Here a little different approach was used: each answer was weighted with 1, 0.5 or 0.1 if the test was considered “easy”, “normal” or “hard”, respectively. If the answer was wrong, i.e. the subject did not identify the odd sample, the answer received weight 0. Note that the choice of weights is more or less arbitrary here. Obviously we had to choose a non-zero value for the “hard” value but this could have been a lesser value than 0.1 also. Aside from that choosing other values results in a slight transformation of the resulting spaces. But the overall qualitative distances should be kept as long as the “normal” value is located near the middle between “hard” and “easy”. The subjective distance is then defined as the sum of all weights. The distances between all samples build up a complete distance graph which is then embedded into a lowdimensional space. This projection is of course subject to an error which depends on the dimension of the embedding space. The dimension is chosen as maximal in a sense that higher dimensions do not reduce the error significantly (see Sect. 2.1.6). Twelve subjects (4 female, 8 male) participated in the experiments. They were aged between 20 and 64 years (on average 32 years). None of the subjects used a force-feedback device or participated in a similar experiment before. During the tests the subjects could neither see the fabric nor hear the noises generated by the pen or the device, so that visual and acoustic impressions could not interfere with the tactile sensation.

5.2.2

Results

In the first experiment the subjects could freely move a real pen (resembling the interface pen of the force-feedback device) over a sample of real fabric. Figure 5.6 shows an MDS plot of the subjective distances gathered from the first experiment. In contrast to Fig. 5.6 Fig. 5.7 shows the fabrics in an objective parameter space (with values taken from [9] and [12]). As one can see the relative positions of the fabrics (emphasised by triangles) are quite similar in both figures, with the exception of Sample 29. Thus the first dimension of Fig. 5.6 probably resembles the roughness of the fabric as also stated in [9]. The second dimension probably corresponds to the compressional resilience.

82

5 Tactile Rendering 6 18 4

dimension 2

2 8 22 23

0

9 -2 29 -4 28 -6 -6

-4

-2

0 2 dimension 1

4

6

8

Fig. 5.6 MDS plot of the first experiment

In the second experiment the subject could explore a virtual sample of fabric generated from the optical method (cf. Sect. 4.5) rendered by the force-feedback device. Just like in the first experiment the size of the virtual samples was 2 cm2 . Figure 5.8 shows an MDS plot of the subjective distances gathered from the second experiment. Note that the MDS method has reduced the data set to one dimension here. Similar to one of the dimensions in the first experiment the one dimension that is left after reducing gives a good approximation of the textures roughness. Just like in the real world set up the samples 8 and 28 are very close to each other, resembling the real impression. Since compressional resilience was not simulated by the system this dimension does not appear as in the first experiment. The third experiment was identical to the second, except that the haptic sensation was generated using the Kawabata measurements (cf. Sect. 4.4). Figure 5.9 shows an MDS plot of the subjective distances gathered from the third experiment. Just like the optical method the Kawabata method gives a good discrimination of the fabrics’ roughness. There is only one exception concerning the samples 8 and 28, that were very similar in the first two experiments but are placed at distant positions by the Kawabata measurements. This could possibly be caused by interpolation problems of the very fine grained surfaces of these fabrics. During these odd-man-out tests the subjects were asked if they found the discriminability of the samples “hard”, “normal” or “easy”. In Fig. 5.10 one can see the accumulated results of these questions. One observes that it seems to be much easier and more accurate to tell the different fabrics apart in the real setup. The two

5.2 Experiment with a Force-Feedback Device

83

100 90

18

compressional resilience

29 80 70 22 60

23 9

50

8

40 28 30 2

4

6

8 10 roughness

12

14

16

Fig. 5.7 Roughness and compressional resilience of the fabrics

29

23

-8

-6

9 22 -4

-2

18

0

8 28 2

4

6

Fig. 5.8 MDS plot of the second experiment

23 -8

-6

29

9 -4

-2

22 28 0

18

8

4

6

2

8

Fig. 5.9 MDS plot of the third experiment

virtual methods perform relatively similar with no significant drawbacks on both sides. In Fig. 5.11 the errors of the subjects are shown for each fabric. It can be seen that the optical method performs comparable to the Kawabata method with the only, already known, exception of Sample 8. One can see in the figure that the errors for the samples 18, 28 and 8 are very low for the experiment with real fabrics. This is probably due to their relatively high distance in the second perceptual dimension mentioned earlier in this section (see Fig. 5.6).

84

5 Tactile Rendering

real optical Kawabata

60

answers in %

50

40

30

20

10

0 wrong

hard

normal

easy

Fig. 5.10 Classification of discriminability

real optical Kawabata

30

25

error in %

20

15

10

5

0 29

23

9

22

18

Fig. 5.11 Errors of the subjects

28

8

5.3 Vibrotactile Rendering

5.3

85

Vibrotactile Rendering

Basically, a tactile renderer needs two ingredients to work properly: a computer representation of a fabric and the trajectory of the fingertip on the fabric’s surface. As described in Sect. 2.3 vibrotaction plays an important role in the perception of fine surface textures. Therefore, we compute the vibrations occurring in the fingertip while moving along the trajectory. These are decomposed into only two basic frequencies intended to directly stimulate the “Pacinian” (PC) and “non-Pacinian” (SA1) receptors. We describe this process later in this section. The used stimulator hardware is described in Sect. 3.2. This approach has already been presented in [1].

5.3.1

Computation of Resulting Vibrations

For each contactor we need to compute the vibrations occurring due to the movement of the fingertip over the surface. This computation depends on the model generated in Chap. 4. If we represent the surface as a height function as in Sects. 4.4 and 4.5, Pi (t) describes the height under the i-th contactor as a function of time. As a start we act on the simple assumption that the vibration at the contactor position on the finger directly corresponds to Pi (t). We want to express a vibration as a function F assigning to each frequency ω a corresponding amplitude F(ω ), which is done by Fourier transforming Pi (t). Since we need to render the device signals for the next 25 ms, this time segment becomes our window for the Fourier transform. We smoothed the boundaries of this segment with a Blackman window. If the surface is represented as spatial frequency spectra F˜e (k) and F˜a (k) in weft and warp direction respectively as computed in Sect. 4.3 a corresponding temporal frequency spectrum F(ω ) has to be computed. When the finger moves over a surface a spatial frequency k and a temporal frequency ω are related by

ω = vk

(5.9)

where v is the velocity of the finger on the surface. This results in the following conversion ω Fe (ω ) = F˜e ve (5.10) ω Fa (ω ) = F˜a va where ve and va denote the component of the velocity in weft and warp direction, respectively. Assuming that the vibrations in the two orthogonal directions can be simply combined by addition we obtain ω ω ˜ ˜ F(ω ) = Fe (ω ) + Fa(ω ) = Fe + Fa . (5.11) ve va

86

5.3.2

5 Tactile Rendering

Decomposition of Vibrations into Base Frequencies

The previous steps result in a frequency spectrum F(ω ), which now has to be decomposed into amplitudes at only two base frequencies that will then be reproduced on the tactile stimulator. We use 40 and 320 Hz as base frequencies. The function mapping the spectrum to these two amplitudes has to ensure that the resulting tactile sensation is very similar to the sensation which would result from reproducing the whole spectrum on the tactile stimulator. This approach is analogous to the approach used for multicolour video displays which produce almost arbitrary colours as mixture of only three fundamental colours. Similar to the colour model of computer graphics (e.g. see [4]) we assume the existence of two functions H40 and H320 such that the two amplitudes for 40 and 320 Hz can be computed as a40 = a320 =

H40 (ω )F(ω ) d ω (5.12) H320 (ω )F(ω ) d ω .

This hypothesis still has to be verified with appropriate experiments. Nevertheless, in the rest of this section we will show the reasonableness of this approach and we will even derive first approximations of the functions H40 and H320 . With I p (F) and Inp (F) we denote the stimulation intensity caused by a vibration with spectrum function F at the Pacinian and non-Pacinian channel, respectively. Note that these intensity functions are subjective and thus hard to define. Our suggestion for an approximation of these functions is 1 F(ω ) d ω Tp (ω ) 1 F(ω ) d ω Inp (F) = Tnp (ω ) I p (F) =

(5.13)

Since the channels react depending on the frequency (e.g. the Pacinian channel is most sensitive around 300 Hz while the non-Pacinian channel is most sensitive around 50 Hz) we use Tp (1ω ) and Tnp1(ω ) as frequency dependent damping factors. The assignment of a linear factor to each frequency seems to be a reasonable approximation suggested by some curves of equal sensation magnitude plotted as a function of stimulus frequency (see [20]). Furthermore, the approach of using an integral is supported by [19, 1], where mixtures of 40 Hz and 320 Hz sinewaves resulting in a constant subjective intensity have been measured. The results of these measurements can be explained with a linear model as taken as a basis for (5.13). In [6] thresholds of the tactile channels as functions of stimulus frequency were measured (see Fig. 5.12 for approximating curves). These thresholds refer to the smallest perceivable amplitude of a stimulus with only a single frequency. Along these measured curves the intensity is considered to be constant, making these

5.3 Vibrotactile Rendering

87

10000 T_p T_np 1000

Displacement

100

10

1

0.1

0.01 1

10

100 Frequency (Hz)

Fig. 5.12 The functions Tp and Tnp

functions suitable as frequency dependent damping functions Tp (ω ) and Tnp (ω ) that normalise the intensity to 1 at threshold level. Let α for ω = ω¯ (5.14) Sω¯ ,α (ω ) = 0 else denote a simple sinusoidal vibration with frequency ω¯ and amplitude α . Our goal is to replace a given vibration F by two vibrations S40,a40 and S320,a320 such that I p (F) = I p (S40,a40 ) + I p(S320,a320 ) Inp (F) = Inp (S40,a40 ) + Inp(S320,a320 )

(5.15)

These equations are equal to a two-dimensional linear equation system with (5.13): a320 a40 F(ω ) dω = + Tp (ω ) Tp (40) Tp (320) F(ω ) a40 a320 dω = + Tnp (ω ) Tnp (40) Tnp (320)

(5.16)

The solutions of this system have the form F(ω ) d ω + B12 Tp (ω ) F(ω ) a320 = B21 d ω + B22 Tp (ω ) a40 = B11

F(ω ) dω Tnp (ω ) F(ω ) dω Tnp (ω )

(5.17)

88

5 Tactile Rendering

Setting B12 B11 + Tp (ω ) Tnp (ω ) B22 B21 + H320 (ω ) := Tp (ω ) Tnp (ω ) H40 (ω ) :=

(5.18)

we obtain (5.12). H40 and H320 are depicted in Fig. 5.13. 1.2 H_40 H_320 1

0.8

0.6

0.4

0.2

0

-0.2 0

100

200

300 Frequency (Hz)

400

500

600

Fig. 5.13 The functions H40 and H320

5.3.3

First Results with Brownian Surfaces

We used Brownian surfaces (see Sect. 4.6) to test our vibrotactile rendering strategy. Since the system was not yet fully operable for active exploration of a surface in real time, we used a passive presentation, i.e. stimuli were presented to the stationary fingertip computed for the case of the virtual surface moving over the fingertip at a constant speed of 8 cm s−1 . As expected from the single-point-contact experiment, different Brownian surfaces could be distinguished on the basis of roughness corresponding to the fractal dimension. However, there was little or no sensation of movement over the fingertip, presumably because individual surface features were not sufficiently identifiable to allow them to be tracked as they moved over the array. In the case of active exploration envisaged for the fully operable system, the sensation of movement would be available from kinaesthetic and visual cues, and so the results would presumably be more convincing.

5.3 Vibrotactile Rendering

5.3.4

89

Further Evaluation

The vibrotactile simulation system was evaluated using the method already described and tested in Sect. 5.2. In order to allow for the comparability of the results the same set of fabric samples 8, 9, 18, 22, 23, 28, 29 (see Annex A) was used. Again the subjects had to tell which one of three presented fabrics was different from the two other ones (cf. Sect. 2.1.3) and had to judge whether it was “easy”, “normal” or “hard” to make a decision. Each subject performed all comparisons twice in random order. Two female and six male subjects aged between 22 and 32 years participated in the experiment. All subjects used the tactile display with the right hand although one subject stated to be left-handed. Two subjects used the tactile display before, the other subjects had no experience with such systems. The simulation system did not display the visual appearance of the fabrics and subjects wore ear protectors. So visual and acoustic impressions could not interfere with the tactile sensation. In the prior experiment (see Sect. 5.2) each answer was weighted with 1, 0.5 or 0.1 if the test was considered “easy”, “normal” or “hard”, respectively. If the answer was wrong, i.e. the subject did not identify the odd sample, the answer received weight 0. The subjective distance was then defined as the sum of all weights. This time subjects were asked to rate the weights for “normal” and “hard” where the weight for “easy” was predefined as 1 and a weight of 0 was considered to represent no distinguishablity. Interestingly almost all subjects chose weights very similar to the rather arbitrarily chosen weights of the prior experiments. The mean values of the answers are 0.51 for “normal” and 0.14 for “hard”. Therefore, for the sake of simplicity weights were chosen as in the prior experiments. The subjective distances between all samples build up a complete distance graph which is then embedded into a low-dimensional space (see Sect. 2.1.6). Figure 5.14 shows such an MDS plot of the subjective distances gathered from the experiment. As the underlying surface model of the tactile simulation system is mainly based on the roughness of the fabrics, the KES-F geometrical roughness values in weft and warp direction (cf. Sect. 4.2) have been chosen to position the fabrics in an objective parameter space. Figure 5.15 shows this space. In contrast to the prior experiments the relative positions of the fabrics are not very similar in Fig. 5.14 and Fig. 5.15. However, some similarities are still recognisable as depicted by the connecting lines in both figures. The visible deformation of the MDS plot might be caused by the design of the experiment. While the small differences between fabrics were rated by the subjects in a relatively detailed way this was not the case for the obvious differences which virtually always received a weight of 1. Therefore, the maximum subjective distance between fabrics was limited and the distance between very different fabrics was underestimated. During these odd-man-out tests the subjects were asked if they found the discriminability of the samples “hard”, “normal” or “easy”. In Fig. 5.16 one can see the results of these questions. As can be seen the answers of the subjects are

90

5 Tactile Rendering 8 6

22

dimension 2

4 2 9 18

23

0

28

-2 29 -4 -6 8 -8 -12

-10

-8

-6

-4 -2 dimension 1

0

2

4

6

Fig. 5.14 MDS plot of the experiment

18 16

28

8

roughness warp

14 12 18 10

22

8

9

6 4 29

23

2 0

2

4

6 8 roughness weft

10

12

Fig. 5.15 Roughness of the fabrics in weft and warp direction

14

5.4 Integration with a Force-Feedback Device

91

60

answers in %

50

40

30

20

10

0 wrong

hard

normal

easy

Fig. 5.16 Classification of discriminability

comparable to the results from the prior experiment with real fabrics and better than with the force-feedback simulation of the fabrics (cf. Fig. 5.10). In Fig. 5.17 the errors of the subjects are shown for each fabric. These are comparable to the errors of the prior experiments shown in Fig. 5.11.

5.4

Integration with a Force-Feedback Device

The simulation systems regarded until now only provide tactile feedback. With these systems only planar, not curved, surfaces can be simulated. Furthermore, mechanical properties like elasticity cannot be presented. In order to create a more comprehensive simulation system the combination of a tactile display with a force-feedback device providing the large-scale forces is necessary. Such an integrated system was developed within the EU-funded HAPTEX project (cf. [15, 5, 11]). It simulates fabrics that are square shaped with a side length of 20 cm. The user can select a fabric from the property database which is then simulated hanging from a stand. The user can touch the virtual fabric with thumb and index finger. The fabric can be squeezed, stretched, rubbed and lifted. The simulated scene is quite challenging, because the physical simulation of the fabric, the force-feedback renderer and the tactile renderer have to be integrated in a single simulation system. In order to reduce complexity the above mentioned subsystems were implemented and tested separately before they were integrated.

92

5 Tactile Rendering

30

25

error in %

20

15

10

5

0 29

23

9

22

18

28

8

Fig. 5.17 Errors of the subjects

global mesh adaption

bordering geometry

Local Geometry

refine

Textile Simulation Thread

force generation

Force Extrapolation contact area force distribution, velocity

deform

Local Simulation Thread

running at60Hz

Visual Display

Motion Estimation

movements

running at 1kHz

Fingertip Model contact formulation

Force Feedback Thread forces

running at 1kHz

Tactile Rendering Thread positions

Force Feedback Device

amplitudes

Tactile Array

Fig. 5.18 The structure of the integrated software system from [3]

Figure 5.18 shows the structure of the integrated system. In one thread the whole fabric is simulated by a simulation software module kindly provided by the MIRALab at the University of Geneva. As this simulation module is relatively accurate it does not provide an update rate sufficiently large for the integration with a forcefeedback device. Therefore, another simulation module has been introduced as intermediate layer between the simulation of the whole fabric and the force-feedback

5.4 Integration with a Force-Feedback Device

93

thread. This module reaches the necessary update rate but is less accurate and only simulates the part of the fabric that surrounds the contact point between the user’s virtual finger and the fabric. This part of the fabric is called “local geometry” in the following. The force renderer manages the whole communication between the physical simulation and the force-feedback devices. Incoming local geometries are processed and resulting forces are transferred to the force-feedback devices. Geometry updates are sent back to the physical simulation. The tactile renderer controls the tactile devices. The amplitudes generated from the fingertips’ movement over the surface are sent to the drive electronics at an update rate of 40 Hz. To enable the simulation system to work without the tactile renderer a loose coupling between both was chosen: the tactile renderer is implemented as a separate application, data is exchanged via inter-process communication (IPC).

5.4.1

Physical Simulation and Haptic Rendering

The simulation software used for the integrated system is described in [22, 21, 23]. Here only a very simplified model is presented in order to explain the fundamentals of the simulation of fabrics. Although the mass is quite continuously distributed on a fabric its simulation model consists of a finite set of discrete points called “particles” each having a position xi and a mass mi . These points are kept together by internal forces acting on adjacent masses that reflect the real tensile behaviour of the simulated fabric. Although these forces are usually non-linear they are simply depicted as springs in Fig. 5.19. In addition to the internal forces external forces, like for example gravity, act on the particles. Usually also damping forces are included in the model. The sum of all these forces acting on a single particle is denoted by

Fig. 5.19 A simple model for the simulation of fabrics

m1

m2

m3

m4

m5

m6

m7

m8

m9

m10

m11

m12

m13

m14

m15

m16

94

5 Tactile Rendering

Fi (x1 , x2 , . . . , x˙1 , x˙2 , . . .). With Newton’s second law of motion we obtain the system of differential equations mi x¨i = Fi (x1 , x2 , . . . , x˙1 , x˙2 , . . .).

(5.19)

This system of ordinary differential equations can be numerically solved with standard algorithms. The physical simulation used within the HAPTEX project considers stretch/ tensile, shear, bending and damping forces. Furthermore, the contact simulation considers friction forces. As for the tactile rendering the relevant physical data have been acquired with the KES-F system. In Fig. 5.20 the haptic loop is depicted. During the interaction with the fabric the position of the user’s fingers and the shape of the fabric change. Both is considered by the contact model that computes the forces affecting the finger and the fabric. Here a problem becomes evident: to create a convincing illusion and to avoid stability problems the system needs to react to the user’s motion within one millisecond which is not possible involving the slower simulation of the fabric. However, note that the single loop depicted in Fig. 5.20 can also be seen as two loops: one between the user and the contact model and the other between the simulation and the contact model. This view led to the solution described in [3]. A dual-layer approach is employed there to allow the two loops to run at different speeds.

io

fo rc e

sit

FF Device

po

Fig. 5.20 The haptic loop with the physical simulation

n

sh ap e

5.4.2

textile simulation

fo rc e

contact model

Integrated Interface Hardware

The GRAB device is a force-feedback device consisting of two identical and independent robotic manipulators (see Fig. 5.21), each having the base link fixed to the desktop and the end effector (contact part) attached to the palmar surface of the user’s thumb or index fingertip. Each manipulator measures the absolute position and orientation of the contact part. These are used by the haptic renderer to compute an appropriate force. The manipulator is able to generate this force on the contact part within a workspace of 400 mm in width, 300 mm in height and 200 mm

5.4 Integration with a Force-Feedback Device

95

Fig. 5.21 The final demonstrator of the HAPTEX Project

in depth. The workspaces of both manipulators overlap. Force errors are limited in a range of about 10 g (0.1 N). The device can exert forces up to 20 N. The GRAB device used for the HAPTEX project is described in [2]. It was kindly provided by the PERCRO laboratory at the Scuola Superiore Sant’Anna in Pisa. The end effectors of the GRAB device have been replaced with the tactile displays described in Sect. 3.2. These displays are designed such that the size of the mechanism between the finger and thumb is minimised. Still, the spacing between the digits caused by the tactile mechanisms is about 10 mm. As the maximum stiffness between the finger and thumb is limited because of instabilities in the force control loop the force-feedback renderer enforces an additional separation of 6 mm to avoid collisions between the mechanisms (cf. [3] and [17]).

5.4.3

Software Integration

The tactile renderer needs the following contact data to work properly: • contact area between the fingertip and the fabric (in local coordinates of the fabric) • velocity of the fingertip relative to the fabric (in local coordinates of the fabric) • distribution of normal forces inside the contact area

96

5 Tactile Rendering

For the computation of the force feedback the haptic renderer already requires among others the contact point, the velocity of the fingertip relative to the fabric and the normal force (cf. [22, 3]). Instead of a single contact point the tactile renderer requires a contact area to decide which contactor pins have to be activated. Also a single normal force is not sufficient for a convincing tactile simulation. Rather a distribution of the normal force over the whole contact area is needed by the tactile renderer. For each contactor pin the geometry of the tactile actuator determines the point on the fingertip where the stimulation caused by that pin occurs. These points have been included in the model of the fingertip. Employing the contact model the position (in local coordinates of the fabric) and the normal force at each point is computed. The velocity is computed by applying a discrete differential operator. The tactile renderer is called every 25 ms, i.e. it runs in a 40 Hz loop. However, the contact data is computed by the haptic renderer every millisecond and we need to resample that data. To avoid aliasing the data is passed through an antialiasing filter that suppresses frequency components above 20 Hz (cf. [7]). In the stand-alone version of the tactile renderer the fabric was assumed to be planar. Furthermore the finger was assumed to always being in contact with the plane of the fabric. However, in the integrated case the fabric may have folds and wrinkles resulting in a more complex contact geometry. Due to the contact pressure the fingertip deforms which also influences the shape of the contact area. As a consequence not all pins of the tactile actuator should be activated, only those inside the contact area. The shape of each finger’s contact area is provided by the forcefeedback renderer. Note that a contact area may not exist, i.e. in that case the finger does not touch the fabric. While moving a fingertip over a rough surface we experience a tactile sensation. Its intensity also depends on the force with which the finger is pressed against the fabric. For relatively small forces a linear dependency between the force and the sensation seems to be reasonable. Therefore the amplitudes computed by the tactile renderer are multiplied by kF F before they are transmitted to the tactile actuator. F denotes the force between the finger and the fabric and kF is a constant relating the force to the intensity. Previous to the integration there was no possibility to assess the contact pressure between the fingertip and the fabric. Therefore a constant force normal to the fabric was assumed implicitly by the tactile renderer.

5.4.4

Evaluation of the Integrated System

The goal of the HAPTEX project was the development of a system for the realistic haptic simulation of real fabrics. In order to evaluate the system four subjects were asked to rate the tensile, roughness, friction and bending properties of five fabrics. Two subjects (A, B) rated the properties of real fabrics whereas two other subjects (C, D) rated the same properties of the corresponding virtual fabrics simulated by the system. For each property a single manipulation for its assessment was defined (cf. [17] and [10]). The interaction with the fabrics was restricted to these manipulations for the experiments. As using the simulation system is very difficult without vision

5.4 Integration with a Force-Feedback Device

97

Fig. 5.22 Reference samples 1 (top row) and 5 (bottom row) for tensile (left) and bending (right)

a coarse wireframe model of the fabric was shown as a compromise. Consequently the tensile and bending tests were performed with vision on the real textiles as well. The fabric samples 33, 34, 36, 37 and 39 were used for the experiment (see Annex A). The subjects were asked to rate the fabrics’ properties in the range 1 to 5. For each property two other fabrics, called reference 1 and reference 5, were provided to define the scale. Figure 5.22 shows the reference samples for the tensile and bending properties. Samples 39 and 10 were used as reference 1 and reference 5 for tensile stiffness, 24 and 10 for roughness, 24 and 16 for friction, and 21 and 32 for bending. Each fabric was presented twice in a sequence of 10 samples to subjects C and D. The data obtained by the experiment were analysed by calculating the correlation of two successive runs of the same subject (“repeatability”), the correlation of the results of C and D (“consistency”) and the correlation of the mean ratings of A

98

5 Tactile Rendering

Table 5.1 The correlation results of the evaluation (cf. [17]) repeatability C D tensile roughness friction bending

0.99 0.91 0.66 0.30

0.97 0.95 0.67 0.07

consistency

realism

0.91 0.93 0.76 0.61

0.96 0.90 0.93 0.00

and D with the mean ratings of C and D (“realism”). Table 5.1 summarises the results of the evaluation. Tensile and roughness gave the best results with a correlation coefficient of at least 0.9 for repeatability, consistency and realism. Friction had a realism coefficient of 0.93 although its repeatability correlation coefficient was only average with about 0.66. This relatively high realism coefficient might be caused by the use of the mean ratings as averaging has the effect of reducing the amount of random variation. Unfortunately bending showed no real correlation, giving coefficients as low as 0.07 for successive runs with the same subject. The reason for the latter was that with the haptic feedback being unable to represent the small forces resulting from bending, the user had to rely on what one could see to rate the bending property. However, rating the bending property by visual assessment was most likely impaired by the coarseness of the mesh not giving enough hints to the user. For complete results see [17]. The results of the evaluation are also presented in [3].

5.5

Conclusion

Only few works exist on the tactile simulation of real surface textures. The only system intended for the rendering of arbitrary surface textures that is comparable to the systems presented in this chapter is described in [8]. However, this system is not aimed at a realistic simulation of the surfaces. In this chapter several experiments are described that were conducted to explore the problem from different points of views. Therefore, a common software framework was developed which manages a virtual workspace where the virtual fabrics are presented, which processes the date from the tracking device and which transmits the amplitude data to the tactile display. The experiment described in Sect. 5.2 was conducted as a pilot study. Its purpose was to test the experimental methods from Sect. 2.1. Instead of a tactile display a force-feedback device with a pen-like end effector was used to display the virtual surfaces since there already existed some experience in the simulation of surface textures with force-feedback devices (cf. [16] and [13]). The results of the experiment suggest that subjects mainly regarded compressional resilience and surface roughness when distinguishing real fabrics with a pen-like probe. The results further suggest that from these two properties only the roughness of the surfaces was

References

99

properly produced by the simulation system. Although the system accurately rendered the height profile of the two-dimensional surfaces, i.e. the space of all possible surfaces having a lot of dimensions, the perceptual space seems to consist of the one-dimensional roughness only. As the pen-like probe used for the experiments is not a natural tool to assess the properties of fabrics it is an interesting question whether the perceptual space has more dimensions when a tactile display is used. In Sect. 5.3 a tactile rendering system based on vibrations is presented. It is important to note that the rendering algorithm proposed in the section is only one of many possible rendering schemes. However, the renderer is based on current knowledge on human tactile perception and only a few reasonable assumptions had to be presumed. An evaluation of the system showed that subjects were able to distinguish relatively similar fabrics when rendered by the system. Statements of the subjects suggest that their decision was mainly based on the roughness of the fabrics in weft and warp direction. However, the latter assumption was not clearly supported by the results of the evaluation. In Sect. 5.4 the integration of the tactile simulation system with a force-feedback system is presented. In such an integrated system many different subsystems have to be integrated: a physical simulation of the fabric, a force-feedback renderer with an appropriate force-feedback device and a tactile renderer with a tactile display. As shown by a subjective evaluation the integration effort resulted in a system that is capable of not only simulating the surface roughness of fabrics but also their tensile and friction properties.

References [1] Allerkamp, D., B¨ottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: A vibrotactile approach to tactile rendering. The Visual Computer 23(2), 97–108 (2007) [2] Bergamasco, M., Salsedo, F., Fontana, M., Tarri, F., Avizzano, C.A., Frisoli, A., Ruffaldi, E., Marcheschi, S.: High performance haptic device for force rendering in textile exploration. The Visual Computer 23(4), 247–256 (2007) [3] B¨ottcher, G., Allerkamp, D., Gl¨ockner, D., Wolter, F.E.: Haptic two-finger contact with textiles. The Visual Computer 24(10), 911–922 (2008) [4] Foley, J.D., van Dam, A., Feiner, S.K., Hughes, J.F.: Computer Graphics: Principles and Practice, 2nd edn. Addison-Wesley, Reading (1990) [5] Fontana, M., Marcheschi, S., Tarri, F., Salsedo, F., Bergamasco, M., Allerkamp, D., B¨ottcher, G., Wolter, F.E., Brady, A.C., Qu, J., Summers, I.R.: Integrating force and tactile rendering into a single VR system. In: 2007 International Conference on Cyberworlds, pp. 277–284. IEEE Computer Society, Los Alamitos (2007) [6] Gescheider, G.A., Bolanowski, S.J., Pope, J.V., Verillo, R.T.: A four-channel analysis of the tactile sensitivity on the fingertip: frequency selectivity, spatial summation, and temporal summation. Somatosensory & Motor Research 19(2), 114–124 (2002) [7] Grover, D., Deller, J.: Digital signal processing and the microcontroller. Prentice-Hall, Englewood Cliffs (1998) [8] Ikei, Y., Wakamatsu, K., Fukuda, S.: Vibratory tactile display of image-based textures. IEEE Computer Graphics and Applications 17(6), 53–61 (1997)

100

5 Tactile Rendering

[9] K¨obberling, T.: Verschiedene Methoden zur Generierung virtueller taktiler Oberfl¨achenstrukturen. Master’s thesis, Welfenlab, Universit¨at Hannover (2006) [10] Luible, C., Varheenmaa, M., Magnenat-Thalmann, N., Meinander, H.: Subjective fabric evaluation. In: 2007 International Conference on Cyberworlds, pp. 285–291. IEEE Computer Society, Los Alamitos (2007) [11] Magnenat-Thalmann, N., Volino, P., Bonanni, U., Summers, I.R., Bergamasco, M., Salsedo, F., Wolter, F.E.: From physics-based simulation to the touching of textiles: The HAPTEX project. International Journal of Virtual Reality 6(3), 35–44 (2007) [12] M¨akinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of physical parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 8–16 (2005) [13] Meyer-Spradow, J.: Ein mathematisches Echtzeit-Modell taktiler Merkmale von Oberfl¨achen. Diploma thesis, Welfenlab, Universit¨at Hannover (2005) [14] Peinecke, N., Allerkamp, D., Wolter, F.E.: Generating tactile textures using periodicity analysis. In: 2007 International Conference on Cyberworlds, pp. 308–313. IEEE Computer Society, Los Alamitos (2007) [15] Salsedo, F., Fontana, M., Tarri, F., Ruffaldi, E., Bergamasco, M., Magnenat-Thalmann, N., Volino, P., Bonanni, U., Brady, A.C., Summers, I.R., Qu, J., Allerkamp, D., B¨ottcher, G., Wolter, F.E., M¨akinen, M., Meinander, H.: Architectural design of the HAPTEX system. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 1–7 (2005) [16] Schulze, M.: Von computergraphischen zu haptischen Texturen. Virtual Reality f¨ur den Entwicklungsbereich Design/Styling in der Automobilindustrie. PhD thesis, Welfenlab, Universit¨at Hannover (2005) [17] Summers, I.R.: Final demonstrator and final integration report. Tech. rep., HAPTEX – HAPtic sensing of virtual TEXtiles (2008), http://haptex.miralab.unige.ch [18] Summers, I.R., Irwin, R.J.: Tactile discrimination of paper. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 43–49 (2005) [19] Summers, I.R., Brady, A.C., Syed, M.K., Chanter, C.M.: Design of array stimulators for synthetic tactile sensations. In: Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 586–587. IEEE Computer Society, Los Alamitos (2005) [20] Verrillo, R.T., Fraioli, A.J., Smith, R.L.: Sensation magnitude of vibrotactile stimuli. Perception & Psychophysics 33, 379–387 (1969) [21] Volino, P., Magnenat-Thalmann, N.: Accurate anisotropic bending stiffness on particle grids. In: 2007 International Conference on Cyberworlds, pp. 300–307. IEEE Computer Society, Los Alamitos (2007) [22] Volino, P., Davy, P., Bonanni, U., Magnenat-Thalmann, N., B¨ottcher, G., Allerkamp, D., Wolter, F.E.: From measured physical parameters to the haptic feeling of fabric. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 17–29 (2005) [23] Volino, P., Davy, P., Bonanni, U., Luible, C., Magnenat-Thalmann, N., M¨akinen, M., Meinander, H.: From measured physical parameters to the haptic feeling of fabric. The Visual Computer 23(2), 133–142 (2007) [24] Welch, G., Bishop, G.: An introduction to the Kalman filter. Tech. rep., University of North Carolina, Chapel Hill (2001)

Chapter 6

Summary and Outlook

The research described in this monograph is the first attempt to develop a realistic tactile simulation system for real existing surfaces. Although there exist many different tactile displays developed by various research groups, most systems are designed to convey information to visually impaired persons (e.g. [3] or [2]). Other systems are limited to the simulation of simple geometrical surface features like ridges and grooves (e.g. [4]). However, unlike the work at hand none of these systems aims at a comprehensive and realistic tactile simulation of real existing surfaces. This work originated from the EU-funded HAPTEX project and thus inherited some of its prerequisites: the class of the surfaces to be simulated by the system is limited to fabrics and only one type of tactile displays was available for the experiments. The former is only a minor restriction as there exists a large variety of different fabrics—certainly enough to argue that the system is in principle able to simulate arbitrary surfaces. The latter prerequisite is a necessity since the resources of the research project were limited and the development of a tactile display is relatively cost-intensive. Therefore, it is reasonable to conduct an in-depth analysis of only a single display. Moreover, the display used within this work follows an interesting approach. Instead of simply reproducing the geometry of a surface it is intended for the direct stimulation of two different kinds of tactile receptors with superimposed vibrations of two different frequencies. Before a tactile rendering system could be implemented as presented in Chap. 5 a number of preparations had to be made which are described in Chap. 2, Chap. 3 and Chap. 4. In order to systematically tackle the problem it was necessary to gain insight into many different disciplines: psychophysics, neurophysiology, electronic circuit design, textile engineering and digital signal processing. For each discipline a short introduction is given in this monograph or at least a reference to an introductory book. In Chap. 2 three important preparations are made. Firstly, it presents the current knowledge on human tactile perception which is used later to motivate the tactile rendering algorithms presented in this book. Therefore, it is necessary to introduce psychophysics and neurophysiology first. Secondly, a perceptional model of tactile simulation is developed that builds a theoretical frame for this work. And thirdly, D. Allerkamp: Tactile Perception of Textiles in a Virtual-Reality System, COSMOS 10, pp. 101–103. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com

102

6 Summary and Outlook

psychophysical methods for the evaluation of the tactile simulation systems are presented. These are used later to analyse the developed simulation systems. The tactile displays used within this work are presented in Chap. 3. In order to use these displays the development of an electronic component was necessary. This component translates the digital amplitude data from the tactile rendering algorithm into appropriate electronic signals which are then transformed into mechanical vibrations by the display. The design of this electronic component is described in the chapter. As explained above the choice of the tactile display was inherited from the HAPTEX project. Nevertheless, the chapter still gives an overview on other existing displays. Finally, force-feedback systems are introduced as these are also used later. A tactile rendering system transforms measured surface data into appropriate tactile signals. For a realistic simulation of active touch the system needs to react to the movement of the user’s finger, i.e. the system needs to compute the amplitudes for the tactile display in real-time according to the movement of the finger. However, there is no need to use the measured data directly. The surface data can be prepared previously for the real-time part. The latter generation of virtual surfaces is the topic of Chap. 4. Three different methods for the generation of virtual surfaces are described. One of these uses an optical surface scan as input data whereas both other methods rely on the KES-F surface measurements. As all these methods more or less make use of surface properties specific to fabrics an introduction to the production of fabrics is also given. Furthermore, the selection and measurement of sample fabrics is described. A comprehensive list of all sample fabrics can be found in Annex A. After all these preparations a tactile simulation system can be developed which is described in Chap. 5. For different tactile rendering algorithms a common framework was created which tracks the position of the tactile display, i.e. the position of the user’s finger on the virtual fabrics. The tactile rendering algorithm presented in the chapter is based on vibrations occurring while moving a fingertip over a rough surface. The system attempts to reproduce these vibrations with a tactile display such that a similar sensation is created. An evaluation of the system showed that subjects were able to distinguish relatively similar fabrics when displayed by the tactile simulator. The tactile simulation system was combined with a force-feedback system. An evaluation showed that subjects recognised the tensile, roughness and friction properties of the sample fabrics. Psychophysical experiments described in [1] suggest that the tactile properties of surfaces build a three-dimensional perceptual space (cf. Sect. 2.3). The number of perceptual dimensions is further reduced when the surface is rendered by a tactile simulation system. The dimensionality of these perceptual spaces stands in contrast to the dimensionality of the space of all possible virtual surfaces which is much higher. This provokes the question whether the virtual surface models presented in Chap. 4 are still too sophisticated. However, it should be noted that the experiments conducted do not reveal the realism of the simulation. It might be possible that only a few perceptual dimensions are sufficient to distinguish the properties of a fabric but many more are needed for a realistic simulation. How the realism of the simulation could be evaluated is not answered in this work and could be the subject of further research. The experiments showed that subjects

References

103

were able to rate the roughness of the sample fabrics but this does not mean roughness was realistically rendered. It is possible that the stimuli causing a real sensation of roughness were substituted by other stimuli which gave enough information on the roughness but did not feel realistic. However, asking subjects to rate the realism of the simulation would probably rather reveal the subjects’ sympathy for the experimenter than anything else. So this is also not a solution. In this work the real situation and its simulation were only compared on the level of sensory impressions (cf. Sect. 2.3.4). A comparison on the level of neural activity or of physical stimuli could be more suitable for the evaluation of realism. Measuring the neural activity of single afferent neurons could probably deliver interesting insight into classes of stimuli resulting in equivalent sensations. However, the invasive methods needed in this case raise ethical problems. The displays used within this work follow an interesting approach. Recreating the vibrations occuring while moving a fingertip over a rough surface might result in an appropriate stimulation of the PC sensors, however, it is questionable whether the same holds for the SA1 sensors. The developed system only creates vibrations when the fingertip is moving. If the finger stops its movement the vibrations vanish. This behaviour does not reflect reality as surface features can still be distinguished in the real case without movement of the fingertip. Representing this static structure of the surface by vibrations is not a good solution as the vibrations do not create a static sensation but are always recognised as vibrations. A display recreating the height profile under the fingertip intended for the stimulation of the SA1 sensors together with vibrating elements for the stimulation of the PC sensors might create a more realistic sensation. The development of such a display could be an interesting research project for the future. In this monograph a system is described that simulates the tactile properties of fabrics. Although it is not perfectly realistic the sensations caused are relatively convincing and subjects were able to distinguish different fabrics and correctly rate their roughness. However, this is just a small step towards a completely convincing simulation of direct touch and a lot of research needs still to be done in this area.

References [1] Hollins, M., Faldowski, R., Rao, S., Young, F.: Perceptual dimensions of tactile surface texture: A multidimensional scaling analysis. Perception & Psychophysics 54(6), 697– 705 (1993) [2] Petit, G., Dufresne, A., Levesque, V., Hayward, V., Trudeau, N.: Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment. In: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, pp. 89–96. ACM, New York (2008) [3] Vidal-Verd´u, F., Hafez, M.: Graphical tactile displays for visually-impaired people. IEEE Transactions on Neural Systems and Rehabilitation Engineering 15(1), 119–130 (2007) [4] Wang, Q., Hayward, V.: Tactile synthesis and perceptual inverse problems seen from the view point of contact mechanics. ACM Transactions on Applied Perception 5(2), 1–19 (2008)

Appendix A

Fabrics

In this appendix all fabrics that were available for this work are described (cf. [1, 2]). The samples have been kindly provided by SmartWearLab at the Tampere University of Technology. The abbreviations of the contents are summarised in Table A.1. An image of the fabric has been included true to scale where available. The selection of the samples and their measurements is described in Chap. 4.

Table A.1 Abbreviations of the fabrics’ fibre contents. AF CA CLY CMD CO

Other fibres Acetat Lyocell Modal Cotton

CV EL JU LI PA

Viscose Elastane Jute Linen Polyamide

PAN PES PU SE WO

Polyacryl Polyester Polyurethane Silk Wool

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

1

Denim

100 CO

twill

380

1.6

2

Shirt cotton

100 CO

combined twill

120

0.61

3

Cord

100 CO

velveteen

330

1.76

4

Linen

100 LI

plain weave

250

1.09

5

Gabardine

100 WO

twill

175

0.55 A Fabrics

Continued on next page.

106

#

#

Description

Content %

Structure

6

Crepe

100 WO

7

Silk

8

Natural (bourette)

9

10

Image

Weight g/m2

Thickness mm

plain weave

145

0.93

100 SE

plain weave

15

0.1

100 SE

plain weave

150

0.8

Wild silk (dupion)

100 SE

plain weave

80

0.44

Jute

100 JU

plain weave

300

1.44

silk

107

Continued on next page.

A Fabrics

Continued from previous page.

108

Continued from previous page. #

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

11

Flannel

80/20 WO/PES

twill

290

1.53

12

Denim

62/35/3 PES/CO/EL

twill

275

1.13

13

Plaid

35/35/30 PES/AF/WO

twill

270

1.14

14

Tweed

66/14/10/10 AF/WO/PES/CMD

combined twill

270

3.9

15

Velvet

92/8 CO/CMD

velvet

300

1.88 A Fabrics

Continued on next page.

#

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

16

Lurex knit

70/30 PES/PA

held stitch knit

215

2.94

17

Crepe-jersey

85/15 PES/EL

rib knit

135

0.73

18

Motorcyclist. coated

72/28 PA/PU

plain weave

90

0.39

19

Woven easy care

65/35 PES/CO

twill

180

0.57

20

Warp knitted velour

90/10 PA/EL

warp knit velour

235

1.56

109

Continued on next page.

A Fabrics

Continued from previous page.

110

Continued from previous page. #

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

21

Weft knitted plain

98/2 CLY/EL

single jersey

172

1.21

22

Taffeta

100 CA

plain weave

125

0.33

23

Crepe

100 PES

plain weave

85

0.25

24

Satin

100 PES

satin

125

0.3

25

Felt

100 PES

nonwoven

155

1.25 A Fabrics

Continued on next page.

#

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

26

Organza

100 PES

plain weave

25

0.16

27

Fleece

100 PES

weft knit

250

3.99

28

Woven upholstery

100 PES

woven jacquard

600

2.38

29

Woven leisure

100 PES

plain weave

90

0.2

30

Tulle

100 PA

warp knitted tulle

10

0.3

111

Continued on next page.

A Fabrics

Continued from previous page.

112

Continued from previous page. #

Image

Content %

Structure

Weight g/m2

Thickness mm

31

Warp knitted tricotsatin

100 PA

warp knitted tricotsatin

100

0.4

32

Leather

100 leather

815

1.68

33

Men’s woven suit fabric (plain)

60/38/3 WO/PES/EL

plain weave

195

0.57

34

Men’s woven suit fabric (herringbone)

100 WO

broken twill

232

0.83

35

Men’s woven overcoat fabric

80/20 WO/PA

modified plain weave

324

2.64

Continued on next page.

A Fabrics

Description

#

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

36

Men’s woven overcoat fabric (twill)

59/25/11/5 CO/PAN/WO/PES

twill

460

3.23

37

Woven outdoor leisurewear fabric

100 PES

plain weave

98

0.26

38

Weft knitted jersey fabric

48/48/4 CO/CMD/EL

single jersey

208

1.09

39

Weft knitted terry fabric

55/45 CV/PES

weft knit terry

288

1.69

40

Warp knitted jerseybased fabric

100 PES

warp knit based

154

0.51

jersey-

113

Continued on next page.

A Fabrics

Continued from previous page.

114

Continued from previous page. #

Image

Description

Content %

Structure

Thickness mm

41

Warp knitted mesh fabric

100 PES

warp knit mesh

128

0.51

42

Brushed warp knitted fabric

100 PES

brushed warp knit

215

0.98

43

warp knit for car seats

89/11 PES/EL

pile warp knit

278

1.22

44

warp knit for car seats

100 PES

flattened pile warp knit

168

1.1

45

warp knit for car seats

100 PES

raised pile warp knit

147

1.6

Continued on next page.

A Fabrics

Weight g/m2

#

Image

Description

Content %

Structure

Thickness mm

46

warp knit for car seats

100 PA

raised pile warp knit

98

1.55

47

warp knit for car seats lamin

Face fabric: 100 PES (sample 42) Foam: E-35 PF (35 kg/m3 polyether) Backing: 100 PES (sample 52)

laminated warp knit

brushed

386

4.09

48

warp knit for car seats lamin

Face fabric: 89 PES/ 11 EL (sample 43) Foam: E-35 PF (35 kg/m3 polyether) Backing: 100 PA (sample 53)

laminated pile warp knit with

438

4.19

49

warp knit for car seats lamin

Face fabric: 100 PES (sample 44) Foam: E-35 PF (35 kg/m3 polyether). Backing: 100 PES (sample 52)

laminated flattened pile warp knit

335

4.11

50

warp knit for car seats lamin

Face fabric: 100 PES (sample 45) Foam: L30PS (30 kg/m3 polyester) Backing: 100 PES

laminated raised pile warp knit

332

3.54

Continued on next page.

115

Weight g/m2

A Fabrics

Continued from previous page.

116

Continued from previous page. #

Image

Description

Content %

Structure

Weight g/m2

Thickness mm

51

warp knit for car seats lamin

Face fabric: 100 PA (sample 46) Foam: E-25 TX (25 kg/m3 polyether)

laminated raised pile warp knit

170

4.03

52

backing

100 PES

warp knit

53

0.42

53

backing

100 PA

interlock

121

0.81

54

warp knit for car seats lamin

Face fabric: 89 PES/ 11 EL (sample 43. different colour) Foam: L30PS (30 kg/m3 polyester)

laminated pile warp knit

287

3.35

A Fabrics

References

117

References [1] M¨akinen, M., Meinander, H., Luible, C., Magnenat-Thalmann, N.: Influence of physical parameters on fabric hand. In: Haptex 2005 – Workshop on Haptic and Tactile Perception of Deformable Objects, Welfenlab, Universit¨at Hannover, pp. 8–16 (2005) [2] Varheenmaa, M., Meinander, H.: Mechanical properties as a base for haptic sensing of virtual fabrics. In: Proceedings of the Autex 2007 Conference, Tampere, Finland (2007)

Index

absolute threshold, 6 afferent neuron, 17 alternative hypothesis, 10 amplitude, 58 artificial surface, 68 autocorrelation, 62 autocorrelation matrix (ACM), 66 Bernoulli experiment, 9 bimorph, 29 binomial distribution, 9 Brownian surface, 70 confidence interval, 10 confidence level, 10 consistency, 98 correlation, 63, 66 correlation–restoration algorithm (CRA), 62 Craven’s distances, 12 data bus, 37 digital–analogue converter (DAC), 39 discrete Fourier transform (DFT), 58 discrimination methods, 8 drive electronics, 33 dual-layer approach, 94 efferent neuron, 17 end effector, 43 evaluation, 96 fabric, 50 fabric hand, 53 fabric samples, 52

false alarm rate, 11 fast Fourier transform (FFT), 58 fibre, 50 firmware, 35 force-feedback device, 42 frequency response, 30 geometrical roughness (SMD), 55 gram-force (gf), 53 HAPTEX project, 91 haptic loop, 43 haptic rendering, 94 Hausdorff dimension, 69 hit rate, 11 hypothesis testing, 10 intraneuron, 17 inverse piezoelectric effect, 29 just noticeable difference (jnd), 6 Kalman filter, 77 Kawabata evaluation system for fabrics (KES-F), 53 KES-F friction tester, 55 KES-F roughness tester, 54 knitting, 51 low-pass RC filter, 40 maximum likelihood estimator, 9 mechanoreceptive afferent, 18 Pacinian (PC), 19 rapidly adapting (RA), 19

120 slowly adapting type 1 (SA1), 19 slowly adapting type 2 (SA2), 19 mechanoreceptor, 18 model of human perception, 20 multidimensional scaling (MDS), 12 neural activity, 20 neuron, 15 non-woven, 51 null hypothesis, 10 odd-man-out forced-choice procedure, 9 operational amplifier (OPA), 41 phase, 58 physical simulation, 93 piezoelectric bimorph, 29 piezoelectric effect, 29 primitive, 64 psychometric function, 6 psychophysical scale, 7 psychophysics, 5 Q-factor, 31 realism, 98 repeatability, 98 response bias, 8 sensations, 20 shape from shading (SFS), 67 signal-detection theory, 10 spatial summation, 17

Index spring–damper model, 43 statistical feature matrix (SFM), 66 Stevens’ Power Law, 8 stimuli, 20 symbolic perception, 20 symmetry, 64 tactile colours, 86 tactile display, 25 tactile perception, 18 tactile rendering, 77 temporal summation, 17 tile, 64 tracking device, 77 translation, 66 triangular method, 9 USB controller, 34 variable voltage supply, 39 vibrotactile rendering, 85 virtual workspace, 76 warp, 50 weaving, 50 Weber’s law, 6 Weber-Fechner law, 7 weft, 50 window, 58 Blackman, 58 Hamming, 58 yarn, 50

Cognitive Systems Monographs Edited by R. Dillmann, Y. Nakamura, S. Schaal and D. Vernon Vol. 1: Arena, P.; Patanè, L. (Eds.): Spatial Temporal Patterns for Action-Oriented Perception in Roving Robots 425 p. 2009 [978-3-540-88463-7] Vol. 2: Ivancevic, T.T.; Jovanovic, B.; Djukic, S.; Djukic, M.; Markovic, S.: Complex Sports Biodynamics 326 p. 2009 [978-3-540-89970-9] Vol. 3: Magnani, L.: Abductive Cognition 534 p. 2009 [978-3-642-03630-9] Vol. 4: Azad, P.: Visual Perception for Manipulation and Imitation in Humanoid Robots 270 p. 2009 [978-3-642-04228-7] Vol. 5: de de Aguiar, E.: Animation and Performance Capture Using Digitized Models 168 p. 2009 [978-3-642-10315-5] Vol. 6: Ritter, H.; Sagerer, G.; Dillmann, R.; Buss, M.: Human Centered Robot Systems 216 p. 2009 [978-3-642-10402-2] Vol. 7: Levi, P.; Kernbach, S. (Eds.): Symbiotic Multi-Robot Organisms 467 p. 2010 [978-3-642-11691-9] Vol. 8: Christensen, H.I.; Kruijff, G.-J.M.; Wyatt, J.L. (Eds.): Cognitive Systems 491 p. 2010 [978-3-642-11693-3] Vol. 9: Hamann, H.: Space-Time Continuous Models of Swarm Robotic Systems 147 p. 2010 [978-3-642-13376-3] Vol. 10: Allerkamp, D.: Tactile Perception of Textiles in a Virtual-Reality System 120 p. 2010 [978-3-642-13973-4]

Tactile Perception of Textiles in a Virtual-Reality System (Cognitive Systems Monographs)

Cognitive processes in the perception of art

Spatial Temporal Patterns for Action-Oriented Perception in Roving Robots (Cognitive Systems Monographs)

Visual Perception for Manipulation and Imitation in Humanoid Robots (Cognitive Systems Monographs)

Human Centered Robot Systems: Cognition, Interaction, Technology (Cognitive Systems Monographs)

Perception (Vancouver Studies in Cognitive Science)

Cognitive Systems

Cognitive Systems: Human Cognitive Models In Systems Design

Joint Cognitive Systems: Patterns in Cognitive Systems Engineering

The Two Sides of Perception (Cognitive Neuroscience)

Oculomotor Systems and Perception

Advances in Cognitive Information Systems

System of Systems

System of Systems Engineering

Recycling in Textiles (Woodhead Publishing in Textiles)

Integrated Models of Cognitive Systems

Integrated Models of Cognitive Systems

Abductive Cognition: The Epistemological and Eco-Cognitive Dimensions of Hypothetical Reasoning (Cognitive Systems Monographs)

Symbiotic Multi-Robot Organisms: Reliability, Adaptability, Evolution (Cognitive Systems Monographs)

Joint Cognitive Systems

Space-Time Continuous Models of Swarm Robotic Systems: Supporting Global-to-Local Programming (Cognitive Systems Monographs)

Space-Time Continuous Models of Swarm Robotic Systems: Supporting Global-to-Local Programming (Cognitive Systems Monographs)

Digital Printing of Textiles (Woodhead Publishing in Textiles)

Biomechanical Engineering of Textiles and Clothing (Woodhead Publishing in Textiles)

Physical Testing of Textiles (Woodhead Publishing Series in Textiles)

Sensation and Perception: A History of the Philosophy of Perception

Digital Printing of Textiles (Woodhead Publishing in Textiles)

Language and Number: The Emergence of a Cognitive System

Language and Number: The Emergence of a Cognitive System

Handbook of Medical Textiles (Woodhead Publishing Series in Textiles)

Sensation and Perception: A History of the Philosophy of Perception

Tactile Perception of Textiles in a Virtual-Reality System (Cognitive Systems Monographs)

Cognitive processes in the perception of art

Spatial Temporal Patterns for Action-Oriented Perception in Roving Robots (Cognitive Systems Monographs)

Visual Perception for Manipulation and Imitation in Humanoid Robots (Cognitive Systems Monographs)

Human Centered Robot Systems: Cognition, Interaction, Technology (Cognitive Systems Monographs)

Perception (Vancouver Studies in Cognitive Science)

Cognitive Systems

Cognitive Systems: Human Cognitive Models In Systems Design

Joint Cognitive Systems: Patterns in Cognitive Systems Engineering

The Two Sides of Perception (Cognitive Neuroscience)

Oculomotor Systems and Perception

Advances in Cognitive Information Systems

System of Systems

System of Systems Engineering

Recycling in Textiles (Woodhead Publishing in Textiles)

Integrated Models of Cognitive Systems

Integrated Models of Cognitive Systems

Abductive Cognition: The Epistemological and Eco-Cognitive Dimensions of Hypothetical Reasoning (Cognitive Systems Monographs)

Symbiotic Multi-Robot Organisms: Reliability, Adaptability, Evolution (Cognitive Systems Monographs)

Joint Cognitive Systems

Space-Time Continuous Models of Swarm Robotic Systems: Supporting Global-to-Local Programming (Cognitive Systems Monographs)

Space-Time Continuous Models of Swarm Robotic Systems: Supporting Global-to-Local Programming (Cognitive Systems Monographs)

Digital Printing of Textiles (Woodhead Publishing in Textiles)

Biomechanical Engineering of Textiles and Clothing (Woodhead Publishing in Textiles)

Physical Testing of Textiles (Woodhead Publishing Series in Textiles)

Sensation and Perception: A History of the Philosophy of Perception

Digital Printing of Textiles (Woodhead Publishing in Textiles)

Language and Number: The Emergence of a Cognitive System

Language and Number: The Emergence of a Cognitive System

Handbook of Medical Textiles (Woodhead Publishing Series in Textiles)

Sensation and Perception: A History of the Philosophy of Perception

Recommend Documents