Associative Learning and Conditioning Theory: Human and Non-Human Applications

ASSOCIATIVE LEARNING AND CONDITIONING THEORY 00-Schachtman-Prelims.indd i 2/16/2011 5:45:16 PM This page intentiona...

Author: Todd R Schachtman PhD | Steve S Reilly PhD

203 downloads 1857 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

ASSOCIATIVE LEARNING AND CONDITIONING THEORY

00-Schachtman-Prelims.indd i

2/16/2011 5:45:16 PM

This page intentionally left blank

Associative Learning and Conditioning Theory Human and Non-Human Applications

Edited by

Todd R. Schachtman Steve Reilly

1

00-Schachtman-Prelims.indd iii

2/16/2011 5:45:16 PM

1

Oxford University Press, Inc., publishes works that further Oxford University’s objective of excellence in research, scholarship, and education. Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With ofﬁces in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam

Copyright © 2011 Oxford University Press Published by Oxford University Press, Inc. 198 Madison Avenue, New York, New York 10016 www.oup.com Oxford is a registered trademark of Oxford University Press All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of Oxford University Press. Library of Congress Cataloging-in-Publication Data Associative learning and conditioning theory: human and non-human applications/edited by Todd Schachtman and Steve Reilly. p. cm. Includes bibliographical references and index. ISBN 978-0-19-973596-9 (hardback) 1. Learning, Psychology of. 2. Classical conditioning. 3. Human behavior. 4. Paired-association learning. I. Schachtman, Todd R. II. Reilly, Steve. BF318.A85 2011 153.1’526—dc22 2010037600

1

3

5

7

9

8

6

4

2

Printed in the United States of America on acid-free paper

00-Schachtman-Prelims.indd iv

2/16/2011 5:45:16 PM

For Ariel and Becky (T.S.) For Elaine (S.R.)

00-Schachtman-Prelims.indd v

2/16/2011 5:45:16 PM

This page intentionally left blank

CONTENTS

Contributors

xi

PART I: OVERVIEW

1.

Things You Always Wanted to Know About Conditioning But Were Afraid to Ask Todd R. Schachtman and Steve Reilly

3

PART II: APPLICATIONS TO CLINICAL PATHOLOGY

2.

3.

4.

Fear Extinction and Emotional Processing Theory: A Critical Review Seth J. Gillihan and Edna B. Foa Fear Conditioning and Attention to Threat: An Integrative Approach to Understanding the Etiology of Anxiety Disorders Katherine Oehlberg and Susan Mineka Behavioral Techniques to Reduce Relapse After Exposure Therapy: Applications of Studies of Experimental Extinction Mario A. Laborda, Bridget L. McConnell, and Ralph R. Miller

27

44

79 104

5.

Learning and Anxiety: A Cognitive Perspective Peter F. Lovibond

6.

Trauma, Learned Helplessness, Its Neuroscience, and Implications for Posttraumatic Stress Disorder Vincent M. LoLordo and J. Bruce Overmier

121

Aberrant Attentional Processes in Schizophrenia as Reﬂected in Latent Inhibition Data Robert E. Lubow

152

7.

vii

00-Schachtman-Prelims.indd vii

2/16/2011 5:45:16 PM

viii

CONTENTS

8. Discrimination Learning Process in Autism Spectrum Disorders: A Comparator Theory Phil Reed

168

PART III: APPLICATIONS TO HEALTH AND ADDICTION

9. Conditioned Immunomodulation

191

Jennifer L. Szczytkowski and Donald T. Lysle

10. Learning, Expectancy, and Behavioral Control: Implications for Drug Abuse Muriel Vogel-Sprott and Mark T. Fillmore

213

11. Applications of Contemporary Learning Theory in the Treatment of Drug Abuse Danielle E. McCarthy, Timothy B. Baker, Haruka M. Minami, and Vivian M. Yeh

235

12. Internal Stimuli Generated by Abused Substances: Role of Pavlovian Conditioning and Its Implications for Drug Addiction Rick A. Bevins and Jennifer E. Murray

270

13. Learning to Eat: The Inﬂuence of Food Cues on What, When, and How Much We Eat Janet Polivy, C. Peter Herman, and Laura Girz

14. Conditional Analgesia, Negative Feedback, and Error Correction

290 305

Moriel Zelikowsky and Michael S. Fanselow

15. Incentives in the Modiﬁcation and Cessation of Cigarette Smoking

321

Edwin B. Fisher, Leonard Green, Amanda L. Calvert, and Russell E. Glasgow

PART IV: APPLICATIONS TO COGNITION, SOCIAL INTERACTION, AND MOTIVATION

16. Social Learning and Connectionism

345

Frank Van Overwalle

17. Application of Associative Learning Paradigms to Clinically Relevant Individual Differences in Cognitive Processing Teresa A. Treat, John K. Kruschke, Richard J. Viken, and Richard M. McFall

376

18. Evaluative Conditioning: A Review of Functional Knowledge and Mental Process Theories Jan De Houwer

399

19. Instrumental and Pavlovian Conditioning Analogs of Familiar Social Processes Robert Ervin Cramer and Robert Frank Weiss

417

20. The Impact of Social Cognition on Emotional Learning: A Cognitive Neuroscience Perspective Andreas Olsson

00-Schachtman-Prelims.indd viii

454

2/16/2011 5:45:16 PM

CONTENTS

ix

21. Effects of Conditioning in Advertising

481

Todd R. Schachtman, Jennifer Walker, and Stephanie Fowler

22. Applications of Pavlovian Conditioning to Sexual Behavior and Reproduction Michael Domjan and Chana K. Akins

507

23. Hot and Bothered: Classical Conditioning of Sexual Incentives in Humans Heather Hoffmann Index

00-Schachtman-Prelims.indd ix

532 551

2/16/2011 5:45:16 PM

This page intentionally left blank

CONTRIBUTORS

Chana K. Akins, Ph.D. Department of Psychology University of Kentucky Lexington, KY

Michael S. Fanselow, Ph.D. Department of Psychology University of California, Los Angeles Los Angeles, CA

Timothy B. Baker, Ph.D. Center for Tobacco Research & Intervention, and Department of Medicine University of Wisconsin School of Medicine and Public Health Madison, WI

Mark T. Fillmore, Ph.D. Department of Psychology University of Kentucky Lexington, KY

Rick A. Bevins, Ph.D. Department of Psychology University of Nebraska-Lincoln Lincoln, NE Amanda L. Calvert, A.M. Department of Psychology Washington University St. Louis, MO Robert Ervin Cramer, Ph.D. Department of Psychology California State University, San Bernardino San Bernardino, CA Jan De Houwer, Ph.D. Department of Psychology Ghent University Ghent, Belgium Michael Domjan, Ph.D. Department of Psychology The University of Texas at Austin Austin, TX

Edwin B. Fisher, Ph.D. Global Director, Peers for Progress American Academy of Family Physicians Foundation Professor, Health Behavior & Health Education Gillings School of Global Public Health University of North Carolina at Chapel Hill Chapel Hill, NC Edna B. Foa, Ph.D. Center for the Treatment and Study of Anxiety University of Pennsylvania Philadelphia, PA Stephanie Fowler, M.A. Department of Psychological Sciences University of Missouri Columbia, MO Seth J. Gillihan, Ph.D. Center for the Treatment and Study of Anxiety University of Pennsylvania Philadelphia, PA xi

00-Schachtman-Prelims.indd xi

2/16/2011 5:45:16 PM

xii

CONTRIBUTORS

Laura Girz, M.A. Department of Psychology University of Toronto Toronto, ON, Canada

Robert E. Lubow, Ph.D. Department of Psychology Tel Aviv University Tel Aviv, Israel

Russell E. Glasgow, Ph.D. Institute for Health Research Kaiser Permanente Colorado Denver, CO

Donald T. Lysle, Ph.D. Department of Psychology University of North Carolina at Chapel Hill Chapel Hill, NC

Leonard Green, Ph.D. Department of Psychology Washington University St. Louis, MO C. Peter Herman, Ph.D. Department of Psychology University of Toronto Toronto, ON, Canada Heather Hoffmann, Ph.D. Department of Psychology Knox College Galesburg, IL John K. Kruschke, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN Mario A. Laborda, M.S. Department of Psychology State University of New York— Binghamton Binghamton, NY Universidad de Chile Santiago, RM, Chile Vincent M. LoLordo, Ph.D. Department of Psychology Dalhousie University Halifax, Nova Scotia, Canada Peter F. Lovibond, Ph.D. School of Psychology University of New South Wales Sydney, Australia

00-Schachtman-Prelims.indd xii

Danielle E. McCarthy, Ph.D. Department of Psychology, Rutgers The State University of New Jersey New Brunswick, NJ Bridget L. McConnell, M.S. Department of Psychology State University of New York— Binghamton Binghamton, NY Richard M. McFall, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN Ralph R. Miller, Ph.D. Department of Psychology State University of New York— Binghamton Binghamton, NY Haruka M. Minami, M.S. Department of Psychology Rutgers, The State University of New Jersey New Brunswick, NJ Susan Mineka, Ph.D. Department of Psychology Northwestern University Evanston, IL Jennifer E. Murray, Ph.D. Department of Experimental Psychology University of Cambridge Cambridge, UK

2/16/2011 5:45:16 PM

CONTRIBUTORS

Katherine Oehlberg, M.S. Department of Psychology Northwestern University Evanston, IL Andreas Olsson, Ph.D. Karolinska Institutet Department of Clinical Neuroscience Psychology Unit Stockholm, Sweden

xiii

Jennifer L. Szczytkowski, Ph.D. Department of Psychology University of North Carolina at Chapel Hill Chapel Hill, NC Teresa A. Treat, Ph.D. Psychology Department University of Iowa Iowa City, IA

J. Bruce Overmier, Ph.D. Department of Psychology University of Minnesota Minneapolis, MN

Richard J. Viken, Ph.D. Department of Psychological and Brain Sciences Indiana University Bloomington, IN

Frank Van Overwalle, Ph.D. Department of Psychology Vrije Universiteit Brussel Brussels, Belgium

Muriel Vogel-Sprott, Ph.D. Department of Psychology University of Waterloo, Waterloo, ON, Canada

Janet Polivy, Ph.D. Department of Psychology University of Toronto Toronto, ON, Canada

Jennifer Walker, M.A. Department of Psychological Sciences University of Missouri Columbia, MO

Phil Reed, D.Phil. Department of Psychology Swansea University Swansea, Wales, UK

Robert Frank Weiss, Ph.D. Department of Psychology University of Oklahoma Norman, OK

Steve Reilly, D.Phil. Department of Psychology University of Illinois at Chicago Chicago, IL

Vivian M. Yeh, M.S. Department of Psychology Rutgers, The State University of New Jersey New Brunswick, NJ

Todd R. Schachtman, Ph.D. Department of Psychological Sciences University of Missouri Columbia, MO

Moriel Zelikowsky, M.A. Department of Psychology University of California, Los Angeles Los Angeles, CA

00-Schachtman-Prelims.indd xiii

2/16/2011 5:45:16 PM

This page intentionally left blank

PART I

Overview

01-Schachtman-01.indd 1

2/15/2011 7:41:40 PM

This page intentionally left blank

CHAPTER 1 Things You Always Wanted to Know About Conditioning But Were Afraid to Ask Todd R. Schachtman and Steve Reilly

This chapter describes signiﬁcant developments in the ﬁeld of conditioning and associative learning over the last 40 or so years (e.g., the emergence of cognitive views, potential associative structures, and the role of contextual factors) to clarify some issues that may confuse psychologists and other researchers or practitioners. Along the way, we shall illustrate some important and, in most cases, ongoing ﬁndings that explore the processes underlying associative learning and conditioning. Our intention is to help readers better appreciate the more detailed, application-focused chapters that follow in this volume.

INTRODUCTION This chapter has two primary aims, each of which will help expose those readers unschooled in conditioning and associative learning to this still-ﬂourishing and signiﬁcant area of research. First, we will describe some of the most important and exciting developments over the last 40 years or so, including the cognitive orientation of the ﬁeld, the possible associative structures underlying conditioning, and the role of contextual factors in conditioning. Second, we will clarify some speciﬁc issues that often plague psychologists and students of psychology. These issues include the following: (1) distinctions and similarities between classical (or Pavlovian) and instrumental (or operant) conditioning (e.g., when an animal [human or non-human] exhibits fear—is it an instrumental response or a classically conditioned response?); (2) the types of associations involved in the various forms of learning, for example, S-O (stimulus-outcome), R-O (response-outcome), and S-R (stimulusresponse) associations; (3) the similarities and differences among procedures such as avoidance,

punishment, and omission training; (4) differences between phenomena such as extinction, habituation, dishabituation, spontaneous recovery, and pseudoconditioning in classical conditioning. As might be expected due to the dynamic and evolving nature of the ﬁeld, these two purposes are intertwined with one another in the present chapter. A few disclaimers are warranted. We will report on those issues that strike us as most interesting; and this may be, in fact, because we are more familiar with these issues. Moreover, our own theoretical predilections will be exposed. For instance, much of our chapter focuses on classical conditioning. It should be clear that the discussion in this chapter is not intended to exhaust all of the topics that are of current interest in the associative conditioning literature. The present chapter, for these and others reasons, should not be taken as a substitute for a textbook on learning and conditioning because such a text would include many more issues (e.g., Bouton, 2007; Dickinson, 1980; Domjan, 2009; Frieman, 2002; Gluck, Mercado, & Myers, 2007; Mackintosh, 1974; Pearce, 2008); and each of the

3

01-Schachtman-01.indd 3

2/15/2011 7:41:40 PM

4

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

topics covered here requires much elaboration to fully do them justice. The present chapter aims to merely whet the appetite for the more detailed and often application-focused expositions found in the later chapters of this volume. Comments About Recent Developments in the Field of Conditioning

For many researchers, the ﬁeld has become increasingly more cognitive in its orientation and conceptual development (e.g., Bolles, 1972; Dickinson, 1980; Klein & Mowrer, 1989a, 1989b; Mackintosh, 1994; Mowrer & Klein, 2001; Rescorla, 1988; Wagner, 1976). There are several ﬁndings and theoretical advances that have shaped these changes. First, the ﬁeld has been largely dominated by the concepts of expectancy and prediction. The importance of expectancy was really thrust into the limelight with the discovery of the phenomenon of blocking (Kamin, 1968, 1969), and publication of the Rescorla-Wagner model (Rescorla & Wagner, 1972; Wagner & Rescorla, 1972) nearly 40 years ago. To say that after CS-US (conditioned stimulus-unconditioned stimulus) pairings, the CS allows an organism to expect the US1 or to be able to predict that the US will occur seems fairly obvious to many of us today; but, for many decades, conditioning was not often discussed within the framework of information processing and its closely related concepts of prediction and expectancy. Before discussing speciﬁc issues, we will point out two developments in the last 30 years that have greatly contributed to the changes in conditioning that we mentioned earlier. First, Rescorla’s work in the 1970s (to be discussed later) often used manipulations administered after an initial conditioning phase but prior to a ﬁnal test phase. This work by Rescorla was presumably inspired in part by Rozeboom (1958), who posited that treatments administered between conditioning and testing served to help determine the contents of the learned information (for examples of such ﬁndings, see sections on “The Role of Representations in Conditioning” and “Postconditioning Representations of the Unconditioned Stimulus”). A second issue that

01-Schachtman-01.indd 4

has been valuable is the distinction between what an organism knows and what the organism reveals that it knows by its performance. In other words, an organism does not always fully express what it actually knows. This learningperformance problem is an old one. Seventy or so years ago, Tolman’s work on latent learning showed that rats can be exposed to a maze without food present, and that they will explore the maze despite the absence of any obvious motivation to do so (Tolman & Honzik, 1930). During exploration, the rats do not reveal much evidence that they have learned about the ﬂoor plan of the maze. However, once these rats start true maze training in which food is presented at the end of the maze and the rats are now hungry, they show considerable “positive transfer” (beneﬁcial effects of earlier experience on a current task) and reveal that they learned a lot during the earlier, nonrewarded exposure to the maze. Another example of this kind of (i.e., unexpressed) learning is that which occurs during the initial phase of a sensory preconditioning experiment as will be discussed later (see section on “Compound Conditioning”). Relatedly, there is a distinction between poor conditioned responding that occurs due to an association never being formed, and poor conditioned responding that occurs because the association is formed and intact but is poorly retrieved or expressed at the time of test. Tulving (e.g., Tulving & Pearlstone, 1966) noted this same distinction while discussing human memory research when he distinguished between an unavailable memory (one that is not present or no longer present in memory) and an inaccessible memory (one that is located in memory but unable to be retrieved at a given point in time). How might one experimentally distinguish between an acquired, intact but unexpressed association and one that was never acquired? Several decades ago, researchers such as Ralph Miller, David Riccio, and Norman Spear administered conditioning trials to rats in which the animals had an opportunity to learn about, for example, a CS which predicted a US; but the animals did not show any evidence of such learning when tested (i.e., did not show evidence of the acquired CS-US association as demonstrated by

2/15/2011 7:41:40 PM

THINGS TO KNOW ABOUT CONDITIONING

a lack of a conditioned response [CR] to the CS). Then, in a second phase, the animals were given a treatment that presumably enhanced retrieval of the already existing association. This treatment facilitated retrieval, but it did not allow for any additional learning per se. When the rats were then tested, a CR did occur, showing that the retrieval facilitation procedure (sometimes called a “reminder treatment”) allowed a previously unexpressed association to then become evident in performance (e.g., Miller, Kasprow, & Schachtman, 1986; Spear, 1978; Spear & Riccio, 1994). When animals show no evidence of learning, and then some treatment—which of itself does not allow new learning to occur— causes this learning to be expressed, one can say that the association was latent or unexpressed prior to this reminder treatment. This ﬁnding illustrates several things. First, it shows how postconditioning manipulations can be important to reveal the content of learning as Rescorla and Rozeboom posited. By “content of learning” we mean the type of association that was acquired or whether an association was acquired at all. Second, it shows that conditioning research now uses very cognitive expressions such as the retrieval of information when discussing conditioned responding. Retrieval has been discussed in the work of many distinguished learning theorists, including Bouton, Hall, Hearst, Miller, Riccio, Spear, and others. Content and Structure of Learning

An instructor at a large Midwestern university was once giving a presentation on what it was like to teach an Introductory Psychology course to hundreds of students packed into a large lecture hall. The speaker made two main points. The ﬁrst one had to do with energy—using an animated voice, in-class demonstrations, and entertainment. The second point contained more shock value: He claimed, “You have to be able to lie.” He said that freshmen and sophomores are not ready to hear all the limitations of the theories that are presented, such as those of Piaget’s writings or the strengths and limitations of the various views of learned helplessness theory or many other ideas that have required

01-Schachtman-01.indd 5

5

complicated revisions over the empirical tests of time. The lesson: When starting out, one needs to keep things simple. Similarly, when lecturing on conditioning or discussing it in a textbook, one feels a need to keep things simple. However, most researchers recognize that in any given situation where conditioning is possible, there is the potential for both classical and instrumental conditioning to occur on the same conditioning trial. One’s task might be to give classical conditioning trials to a dog in which a tone (the CS) is followed by food (the US). This, of course, is the same type of procedure used by Pavlov when he discovered classical conditioning (Pavlov, 1927). The food, professors tell their students, promotes classical conditioning to occur for the tone CS; but the food is also capable of instrumentally reinforcing a response that may happen to occur around the time that the food is delivered. Thus, when the dog makes a salivation response, it is not entirely clear whether the salivation is a classically conditioned response occurring to the tone or whether the dog happened to be salivating at the time the food was ﬁrst delivered (and in the presence of the tone) such that instrumental conditioning occurred. That said, because it might be unlikely that salivation would happen to occur at the same time that the food was delivered (assuming the dog can’t smell the food coming), any salivation arising from the ﬁrst conditioning trial was probably not due to operant conditioning. However, if the dog makes a salivation response to the tone on the second classical conditioning trial, then the food delivery could reward that response as instrumental learning as well as further the classical conditioning to the tone. In this way, it is easy to see that a conditioning episode does not always render itself to a pure classical conditioning analysis or a pure instrumental analysis. Indeed, the overlap between the two types of conditioning may be even more pronounced. Some associative learning theorists (e.g., Mackintosh, 1983) have noted a “symmetry” of the potential associations underlying the two types of conditioning (i.e., the two associations having some similarity in their structure). Classical conditioning is viewed as involving a

2/15/2011 7:41:40 PM

6

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

stimulus-stimulus (S-S) association (i.e., an association between the CS and the US), such that pairings of a tone and food result in a tone-food association. Instrumental conditioning in which a response (e.g., a rat pressing a lever) is followed by food can be said to result in a responsestimulus (R-S) association in which the lever press response is associated with a food stimulus. In the literature, this food stimulus in both types of conditioning is sometimes called an outcome. Hence, a response-outcome (R-O) association is formed in which the outcome is a food stimulus; from this perspective, classical conditioning may be termed stimulus-outcome or S-O learning (see Table 1.1, which lists some of the more common nomenclature, terminology, and potential underlying processes thought to be involved in these two forms of associative learning). The symmetry view of conditioned associations acknowledges that classical conditioning using the example provided results in a tone-food association and instrumental conditioning results in a response-food association; and so the mechanisms underlying these two associations may be similar, with the exception that one type of conditioning involves a CS that is associated with food and the other involves a response that is

associated with food. According to this view, these associations may interact. This interaction could result in the two types of associations being in competition with each other on a given trial and producing effects such as “blocking,” which will be discussed later (also see Williams, 1999). We mentioned that instrumental conditioning can occur on a trial that uses a classical conditioning procedure and classical conditioning can occur on a trial that uses an instrumental conditioning procedure. If instrumentally and classically conditioned associations can compete, then it means that learning an association between a cue and the outcome will reduce the potential for R-O learning and vice versa. There is another parallel that exists between instrumental and classical conditioning. In instrumental conditioning, a stimulus (e.g., a tone) is often present at a time when a schedule of reinforcement is in place. In the case of positive reinforcement, the subject can learn that responding leads to reward when the tone is present. If the tone is absent, then responses have no consequences. The tone, in this example, is referred to as a “discriminative stimulus” in that it signals when the schedule of reinforcement is operating. A stimulus which signals that the response will

Table 1.1 Some Features of the Two Types of Associative Learning Classical Conditioning

Instrumental Learning Early in Training

After Extensive Training

Also called:

Pavlovian conditioning or Stimulus learning or Type I learning or Respondent conditioning

Operant conditioning or Response learning or Type II learning

Habit learning

Associations are commonly known as:

CS-US or S-S or S-S∗ or S-O

R-O or R-S∗ or A-O

S-R

Potential underlying processes:

Ability to predict future events

Modiﬁcation of voluntary behavior

Elicited by antecedent stimulus

A, action; CS, conditioned stimulus; O, outcome; R, response; S, stimulus; S∗, biologically signiﬁcant stimulus; US, unconditioned stimulus.

01-Schachtman-01.indd 6

2/15/2011 7:41:40 PM

THINGS TO KNOW ABOUT CONDITIONING

lead to a certain outcome is sometimes referred to as an SD or S+. Skinner (e.g., 1938), who coined the expression “SD,” also stated that the stimulus in this case “sets the occasion” for the response to result in the reward. When a stimulus signals that the response will not lead to the outcome (extinction), then such a stimulus is referred to as an S∆ (pronounced: S delta) or “S–.” These deﬁnitions illustrate the conditional relationships that events can have: A response can lead to a reward in the presence of the SD, but a response will not lead to a reward in the absence of the SD or, alternatively, in the presence of an S∆. Parallel phenomena exist in classical conditioning. A great deal of research in the past 25–30 years has focused on the conditional relationship among stimuli in classical conditioning. These phenomena are called “occasion setting” stimuli (Holland, 1992; Schmajuk & Holland, 1998). If a tone leads to food when it is accompanied by a light but the tone does not lead to food when the light is absent, then the light is said to “set the occasion” for when the tone will be followed by food. The light is referred to as a positive “occasion setter.” To reiterate for clarity, if lighttone-food trials occur along with tone–no-food trials, then the light becomes a “positive occasion setter” for the tone–food relationship. On the other hand, if light-tone–no-food trials occurred with tone-food trials, then the light could be said to be a “negative occasion setter.” The associative processes underlying classical and instrumental conditioning may extend into other conditioning phenomena. A pairing of the CS and US results in a CS-US association, and then extinction (a postacquisition phenomenon) involves the presentation of the CS without the US. Some conditioning theorists (and this issue will be brieﬂy discussed again later; see section on “Postconditioning Representations of the Unconditioned Stimulus”) have posited that such CS-only presentations after conditioning result in the learning of a CS-noUS association (an association reﬂecting that the CS is no longer being paired with the US) or, using other symbols, an S-noO associations (e.g., Bouton, 1993; Konorski, 1967; but see Rescorla, 2007). When instrumental conditioning occurs in which

01-Schachtman-01.indd 7

7

a response produces an outcome, an R-O association is formed; and extinction involves the occurrence of response without the outcome. One can say that a response–no-outcome (R-noO) association is formed during extinction in instrumental conditioning. Hence, parallel associative processes may occur for the two types of associations (classical and instrumental). Many other important conditioning phenomena (particularly those that stem from work using classical conditioning) such as blocking, overshadowing, potentiation, spontaneous recovery, and outcome preexposure effects may produce comparable empirical effects in instrumental and classical conditioning as well as similar underlying associative mechanisms. The extent that such parallels hold true will be determined by research ﬁndings, but mention of these issues at this time helps one appreciate the potential similarities of the different types of associative learning. Rescorla and Solomon (1967) showed that one can separately train a classically conditioned response and an instrumental response for a group of subjects and then combine the two training events together to observe what happens with this combination of the two types of training. Their ideas have been termed the “twoprocess theory” of conditioning. A subject (e.g., a dog) can learn that a tone leads to food, and thus the occurrence of the tone causes the dog to expect food. The same dog can also be trained to press a lever to produce food, and this dog will press the lever frequently because it expects that the lever press will produce food. Subsequently, when placed in front of the lever and presented with the tone, the dog will have two expectancies: one from the tone (classical conditioning) and one from the lever press (instrumental conditioning). The summation of expectancies can produce even greater lever pressing because the dog has an extremely high expectancy of food. Similar interactions can be examined for all combinations of predictors of the US and even for events that predict no US (e.g., a CS that signals the absence of the US) combined with a response that produces outcome avoidance. Two-factor theory (e.g., Mowrer, 1951) is also concerned with how classical and instrumental conditioning can interact. Mowrer pointed out

2/15/2011 7:41:41 PM

8

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

many years ago that if a subject must make a response to avoid an aversive event (called “active avoidance” or “negative reinforcement” as we will discuss in the next section), then at least two things can be learned on such a trial. First, the subject receives classical conditioning pairings of the environmental cues (i.e., the apparatus within which the experiment is conducted or, perhaps, a small light within the apparatus that is illuminated at the beginning of each trial signaling that an aversive event will occur if the target response does not occur) with the aversive outcome. That is, a CS-O association is potentially formed. These cues (the CS) become capable of producing fear in the subject. Thus, a fear CR occurs to the cues. Second, the subject learns to make the instrumentally conditioned response (such as ﬂeeing from the CS or the place that has been paired with shock) to prevent the aversive event from occurring. Therefore, subjects run because they have fear of the CS, which is due to classical conditioning; and they are making the response that will remove them from the aversive cues—the avoidance response (instrumental conditioning). Two-factor theory states that both classical and instrumental conditioning can occur during avoidance training. This example also reveals again how classical conditioning and instrumental conditioning can be intertwined—a CR to a cue (classical conditioning) can resemble and avoidance response (instrumental conditioning). Since classical and instrumental conditioning may overlap in terms of their associative mechanism (S-O and R-O associations, respectively), there has been interest in pitting the two types of conditioning against each other to document the roles of the two processes. The omission training procedure, which involves overlaying an instrumental contingency upon a classical contingency, has been useful in this regard (e.g., Shefﬁeld, 1965). In omission training, a cue (CS) is presented and the subject receives a US (e.g., food) only if it does not respond to the CS. To repeat, if the subject responds to the CS, the US is omitted. Thus, a CS-US pairing occurs only when the subject makes no response to the CS. As an instrumental procedure, however, the subject is rewarded for not responding. Again, if

01-Schachtman-01.indd 8

the subject does not respond to the cue, then the food is delivered and, consequently, a CS-US pairing is experienced. Thus, performance can be guided by both classical and instrumental conditioning in the omission procedure. As a concrete example, we can consider one of the original omission experiments done with pigeons in which the CS was a colored keylight that the birds can peck (Williams & Williams, 1969; also see Williams, 1981). When pigeons receive presentations of a keylight followed by food, they quickly learn through classical conditioning to respond by pecking that keylight (a classical conditioning phenomenon termed autoshaping or sign-tracking in which the CR to a light is a peck response; for reviews, see Hearst & Jenkins, 1974; Locurto, Terrace, & Gibbon, 1981). However, the operant (omission) contingency gives them food for not responding; and when they don’t respond they get a keylight-food pairing that makes them respond on the next trial. Both classical and operant conditioning will exert an effect on behavior in such situations: Responding is not as high as it might be because the omission contingency keeps responding low, but the CS-US pairings cause the animal to sometimes respond and, therefore, lose food presentations when they respond to the CS (the CS occurs without the US). Sixty or seventy years ago discussions of conditioning focused on S-R (stimulus-response) associations. Many readers of this chapter may have heard this expression and wondered about its contemporary status. Indeed, these are the associations that were purported to underlie conditioning when Thorndike began investigating associative learning in non-human animals at the turn of the previous century (e.g., Thorndike, 1898, 1911) and they were alleged to underlie both classical and instrumental conditioning throughout, at least, the 1950s. S-R associations involve an association between a (usually) environmental stimulus (such as a light, tone, or contextual stimulus) and the response. We say “usually” because one could have an “internal” stimulus (such as hunger pangs, moods and emotions, or states induced by the ingestion of a drug) as the stimulus of the S-R association, but we will keep things simple for this discussion.

2/15/2011 7:41:41 PM

THINGS TO KNOW ABOUT CONDITIONING

It is important to be clear as to what this association involves and how it differs from the contemporary view of associations (S-O and R-O) described earlier. An outcome is not part of, or encoded into, the S-R association. According to this older S-R view of conditioning, when an outcome (e.g., food) follows the occurrence of response (e.g., lever press) in the presence of a stimulus (e.g., the apparatus within which the experiment is being conducted), an association is formed between the stimulus (apparatus) and the response (lever press). That is, an S-R association is formed. The outcome (food) serves to facilitate the formation of the S-R association. According to this S-R association view of conditioning, the representation of the outcome in memory is not a part of the acquired association. Once the outcome has promoted the formation of the S-R association, its function is accomplished and it no longer has a role in further processing or performance regarding the S-R association. Work by Anthony Dickinson and Robert Rescorla and their colleagues and others challenged this view experimentally (e.g., Colwill & Rescorla, 1986; Dickinson, 1985; Dickinson & Balleine, 1994; Rescorla, 1991). If the outcome is no longer important after the instrumental association is acquired, then changing the value of the outcome for the subject after conditioning and prior to testing for the learned response should have no inﬂuence on performance. Work in Dickinson’s and Rescorla’s laboratories showed that changing the value of the outcome between conditioning and testing produced a marked change in the performance of the learned response (e.g., Adams & Dickinson, 1981; Colwill & Rescorla, 1985). This makes some intuitive sense. If you train a hungry rat to press a lever to obtain food pellets, it will learn to respond at a high rate to obtain the desired food. However, if you now devalue the food pellets (e.g., by conditioning a taste aversion to them) such that the rat dislikes them, it is not surprising that the rat will not press the lever much to obtain a disliked food. Yet earlier S-R theories did not predict such a change in behavior. A similar situation exists for a dog that might salivate to a signal for food—changing how the organism values

01-Schachtman-01.indd 9

9

the food after such training will attenuate the salivation response. The outcome seems to be part of the association. The subject has acquired an R-O association because the response activated the food representation in memory. Despite the strong focus on the S-O and R-O associations in this chapter, Dickinson and Rescorla have shown that S-R associations can be formed during conditioning in certain situations. Research in the last 30 years or so has shown that instrumental learning involves R-O associations early in training and can be said to be “goal directed” (the organism makes a response to obtain a desired goal or outcome). Moreover, this voluntary (i.e., intentional), goal-directed behavior is occurring in a speciﬁc context or in the presence of a speciﬁc stimulus (S). In certain circumstances, this stimulus, because it is invariably and repeatedly paired with the target response, may become associated with the response and an S-R association is also eventually acquired. Thus, presentation of the S triggers an activation of the response representation and the subject then responds habitually (i.e., automatically and usually without any conscious active representations), independent of the status (valued or devalued) of the outcome (Dickinson, Balleine, Watt, Gonzalez, & Boakes, 1995; but see Holland, 2004). By this analysis, habit (S-R) formation develops after, and overlays or masks, the initial goal-directed (R-O) behavior. Both associations involve the same instrumental response. As noted earlier, the S-R association, unlike the R-O association, is impervious to outcome devaluation (so clinical treatments aimed at changing behavior via some form of outcome devaluation may have limited utility if that behavior has become habitual; e.g., smoking or drinking). There is, moreover, evidence that each type of instrumental association (R-O and S-R) is subserved by an anatomically distinct brain system (Balleine, 2005; Balleine, Liljeholm, & Ostlund, 2009; Delamater, 2004; Yin & Knowlton, 2006). If these systems can be dissociated with drugs, there may be some hope that, with pharmacological intervention, extreme habits (S-R associations) can be alleviated.

2/15/2011 7:41:41 PM

10

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Instrumental Contingencies

We mentioned the terms avoidance and negative reinforcement in the preceding section. These and other instrumental contingencies (see Table 1.2 for a description of the four basic procedures and their effects on behavior) can be a source of confusion, especially since textbooks are not always consistent with the nomenclature applied to the various instrumental-conditioning manipulations. As most psychologists know, positive reinforcement occurs when there is a positive relationship (or positive contingency) between the production of a response and the presentation of an appetitive (pleasurable) outcome. When the expression “positive contingency” is used, it should be viewed as comparable to the expression “positive correlation” (and conversely for the expression “negative contingency”). That is, when the subject makes the response, something pleasurable (like food) reliably occurs. This positive contingency causes the response to increase in the future. Punishment involves a positive contingency between the response and an aversive outcome (something painful or unpleasant). When the subject makes this response, an aversive event occurs and, not surprisingly, this causes the response to decrease in frequency.

There is a caveat to mention that readers will likely wish that we did not bring up (this certainly should be spared from an undergraduate lecture on this topic). We make this cumbersome point because it will help when we make related points later. With positive reinforcement in which a lever press produces a pleasurable outcome, one could imagine what it would be like for the subject that has a lot of experience with this contingency to momentarily refrain from making a response and to therefore get no pleasurable outcome. That is, with positive reinforcement: No response leads to no reward. It involves a positive contingency between no response and no reward. One can think of it as an intrinsic, necessary feature of the positive contingency involving the response and reward occurrence. In truth, for any behavioral contingency (a response leads to an outcome), there are a total of four contingencies related to it because nonresponses and no outcomes can be viewed as events: A response has a positive contingency with the outcome, a nonresponse has a positive contingency with no outcome, a response has a negative contingency with no outcome, and a nonresponse has a negative contingency with the outcome. Four contingencies exist for a simple positive reinforcement contingency! But it is best to simply focus on

Table 1.2 Basic Instrumental Contingencies of Reinforcement and Punishment and Their Usual Effect on Behavior Outcome

Positive

Appetitive

Aversive

Reward (or positive reinforcement)

Punishment (or positive punishment)

Increase in behavior

Reduction in behavior

Example: Receive a kiss after complementing partner

Example: Drive too fast and get a speeding ticket

Omission (or negative punishment)

Avoidance (or negative reinforcement)

Reduction in behavior

Increase in behavior

Example: Driving license suspended following conviction for drunk driving

Example: Open umbrella to avoid getting rained on

Contingency

Negative

01-Schachtman-01.indd 10

2/15/2011 7:41:41 PM

THINGS TO KNOW ABOUT CONDITIONING

the fact that positive reinforcement is a positive contingency between producing the response and the presentation of a pleasurable outcome (and let the positive contingency between nonresponses and nonrewards just rest wherever they might rest and these other contingencies can rest there, too). One can say the same thing about punishment. Nonresponses will lead to the nonoccurrence of the aversive event. This is a positive contingency between nonresponding and no outcome. Nonresponses are preferred for the organism that is experiencing a punishment contingency. But again we will let these nonresponses and nonrewards slip away without further discussion and simply say that punishment involves an association between the occurrence of a response and the presentation of an aversive outcome. Avoidance involves a negative contingency between the response and the subsequent occurrence of an aversive outcome. If the subject makes a response, then the aversive event will not occur. This is referred to as negative reinforcement since the contingency increases the frequency of the behavior (the subject makes the response to prevent the aversive event from occurring). One key note is that whenever one sees the word reinforcement (either negative reinforcement or positive reinforcement), it means that the contingency causes the frequency of the response to increase. Hence, rewarding a behavior is positive reinforcement and avoidance is negative reinforcement, but the former is a positive contingency between the behavior and an appetitive outcome while the latter is a negative contingency between a behavior and an aversive outcome.2 Omission training is a negative contingency between the occurrence of a response and the presentation of an appetitive outcome. Thus, as mentioned before, a subject will receive food if it does not make a certain response. Avoidance and omission training are both negative contingencies, but the former is between a response and an aversive outcome and the latter is between a response and an appetitive outcome. Avoidance increases the rate of responding, whereas omission decreases response rate. It can be easy to get avoidance and punishment confused since they both involve an aversive

01-Schachtman-01.indd 11

11

outcome, and in both cases the subject is motivated to not experience the aversive event. It is good to remember that these two types of instrumental training involve different contingencies: negative and positive. That is, with avoidance the subject must make a certain response to prevent the aversive event from occurring; whereas with punishment the subject must not make a certain response to prevent the aversive event from occurring. Also, similar to the cumbersome digression expounded earlier, one can be tempted to think of an avoidance procedure as follows: A nonresponse produces an aversive event (a positive contingency between a nonresponse and an aversive event). This is true because it is the necessary converse of the following: A response prevents an aversive event—a negative contingency. But we encourage the reader to simply focus on the “occurrence of a speciﬁc response” when working through these contingencies. It is also easy to get confused about the difference between escape and avoidance. Escape does not ﬁt nicely into the scheme outlined earlier. In escape, the organism receives the aversive event no matter what, but it can terminate the existing aversive event by making a response. Hence, with a successful escape response, a response and an aversive event occur on the same trial (which suggests that a positive contingency is occurring as in punishment, but this is not true since the response certainly did not produce the aversive event). Indeed, since the response terminates the aversive event, it makes the escape contingency share properties with an avoidance contingency—the response gets rid of the aversive event. Another realm of confusion concerns the procedure termed “passive avoidance” (which is also known as “inhibitory avoidance”), a procedure that has become extensively used in recent neuroscience research. It is a task in which the subject receives an aversive event (e.g., a footshock) if it makes a certain response (e.g., step into one side of a two-compartment shuttle box). It should be clear at this point that this is a punishment procedure—there is a positive contingency between making a response (moving into the side of the chamber) and the occurrence

2/15/2011 7:41:41 PM

12

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of an aversive event. Although the names “passive avoidance” and “inhibitory avoidance” have been around for a long time, they are misnomers. They are not avoidance procedures. When scientists decided perhaps 25 years ago to favor the name “inhibitory avoidance” over “passive avoidance,” they did not correct the part of the name—avoidance—that has caused the greatest degree of confusion in the previous 70 years. While discussing instrumental work, we felt it was a good idea to illustrate some important and interesting issues with respect to schedules of reinforcement that might not always be fully appreciated. Nearly all psychologists have been exposed to the “classic” schedules of (intermittent) reinforcement: ﬁxed ratio, ﬁxed interval, variable ratio, and variable interval (Ferster & Skinner, 1957; see Table 1.3 for a description and some characteristics of performance on these four simple schedules of reinforcement). These are all schedules of positive reinforcement for the purposes of our exposition. Punishment contingencies can also be applied using these

schedules, but we will not consider them at this time. However, we want to describe a few of the properties of responding on these schedules, which many psychologists may not be aware of. First, an outcome is delivered on an interval schedule for making the ﬁrst response on the trial after the interval times out.3 We will assume it is a ﬁxed interval schedule for our discussion. Earlier responses (before the designated time interval elapses) are not necessary and, thus, a waste of energy. Hence, the most efﬁcient form of responding on a ﬁxed interval schedule is one in which only one response occurs per reinforcement. If there is a less-than-perfect positive contingency between responding and the reinforcement, it is because of the nonrewarded response “mistakes” on the part of the subject. The subject, in essence, determines the number of responses per reward based on how early in the trial he or she begins to respond (e.g., to what extent the responses are “early” and therefore not particularly functional). It is interesting that although a high response rate on an interval

Table 1.3 Characteristics of the Four Basic Types of Reinforcement Schedules Basis for Reward

Response

Fixed

Variable

Fixed ratio (FR)

Variable ratio (VR)

Reward delivered after ﬁxed, predictable number of responses

Reward delivered after variable, unpredictable number of responses

High rate of responding and pause after reward delivered

High, steady rate of responding and little or no pause after reward delivered

Example: Working on commission or on “piecework”

Example: Gambling (e.g., slot machine)

Fixed interval (FI)

Variable interval (VI)

Reward delivered for ﬁrst response after ﬁxed, predictable time period

Reward delivered for ﬁrst response after variable, unpredictable time period

Increasing rate of response and pause after reward delivered

Moderate, steady rate of responding and little or no pause after reward delivered

Example: Study patterns of students when examinations scheduled at regular intervals

Example: Trying to call someone on the phone when line is busy

Criterion

Time

01-Schachtman-01.indd 12

2/15/2011 7:41:41 PM

THINGS TO KNOW ABOUT CONDITIONING

schedule can be interpreted as indicative of a strong R-O or S-R association, it is highly inefﬁcient. On ratio schedules, every response “counts for something” and the subjects are free to respond at their own rate: quickly or slowly. From a “responses per outcome perspective,” all responses are valuable, all responding is equally efﬁcient, and all subjects end up with a comparable response/reinforcer ratio. Time is irrelevant in terms of the contingency. Thus, from a time perspective, an organism can be inefﬁcient on a ratio schedule but all responses count; but on a ﬁxed interval schedule an organism can be inefﬁcient from a response-wasting or energy-wasting standpoint, but time is only wasted if the subject is slow to respond after the interval times out. Burgeoning interest in the etiology, treatment, and neural underpinnings of addiction (and related issues) has rekindled interest in the motivation value of drugs and other highly desired stimuli. Although the schedules of reinforcement discussed earlier have been used in this enterprise, comparisons of the relative value of different outcomes may sometimes be problematic. As noted previously, rates of responding do not always provide an accurate index of the value of an outcome, particularly for stimuli such as psychoactive drugs, where ceiling effects in response rate may obscure detection of the true value of a reward. In this context, the progressive ratio schedule has proven to be a highly successful method to use (e.g., Hodos, 1961). These schedules differ from those just discussed in that progressive ratio schedules can have both a response and a time requirement. Although there are many variations, the essence of the progressive ratio schedule is this: As trials progress, more responses are required to obtain each successive outcome until the subject eventually fails to obtain the outcome within a speciﬁed time limit. For example, on a progressive ratio 3, three responses are required to obtain the ﬁrst outcome, six for the second, nine for the third, and so on. The number of responses produced to obtain the ﬁnal outcome (the ﬁnal outcome is the ratio value—it is the ﬁnal one because, after that trial, the ratio value is so high that the

01-Schachtman-01.indd 13

13

subject quits responding) is termed the “break point” and is an index of the value of the outcome (i.e., how hard the animal is willing to work to obtain the outcome). Progressive ratio schedules have been used to assess the “value” of outcomes such as gustatory stimuli (Reilly, 1999; Sclafani & Ackroff, 2003), electrical brain stimulation (Depoortere, Perrault, & Sanger, 1999; Keesey & Goldstein, 1968), and drugs of abuse (Ranaldi & Wise, 2000; Roberts, Morgan, & Liu, 2007). Another ﬁnding should be noted. We mentioned in an earlier section that research has found that changing the value of the instrumental outcome after conditioning and before testing can produce a marked change in the vigor or probability of occurrence of the instrumental response. It was noted that if you train a rat to press a lever to obtain food pellets and then you change the degree that the rat values the food pellets (i.e., pair it with a toxic drug to produce an aversion to the food pellet) such that the rat now dislikes food pellets, then the rat does not bar press much after this manipulation. Dickinson and colleagues (e.g., Adams, 1982: Dickinson, Nicholas, & Adams, 1983) found this to be more true for ratio than interval schedules (see Balleine, 2009, for a review of this literature). The Role of Representations in Conditioning

As mentioned earlier, the ﬁeld of conditioning and associative learning has become very cognitive in its focus in the past several decades. Given that the word representation is integral to a cognitive approach, the event-memory model put forth by Rescorla in the 1970s (e.g., Rescorla, 1973, 1974) also encouraged, we believe, conditioning to move in a cognitive direction by promoting the idea that an event can activate the mental (and neural) representation of itself and other associated events (see, e.g., Holland & Wheeler, 2009). The event-memory model stated that when pairings of a CS and US occur for an organism, the animal learns at least three things: (1) it forms a representation of the CS in memory; (2) it forms a representation of the US in

2/15/2011 7:41:41 PM

14

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

memory; and (3) it forms an association between these two representations in memory, a CS-US association. Once a CS is associated with a US, then presentation of the CS will activate a representation of that CS in memory and this will, via the CS-US association, cause an activation of the US representation. A dog salivates to the sound of a tone because of a previously acquired tone– food association. That is, detection of the CS activates the US representation and the dog salivates in anticipation of, and in preparation for, the expected food. In many cases, like the present example, the CR is similar, if not identical, to the unconditioned response (UR) triggered by the US. The UR to food is salivation (a dog salivates when food is placed in its mouth) and the CR to a CS that predicts food is also salivation. However, in other cases, particularly where the US is a drug state, the CR may actually be opposite of the UR (for further discussion, see Dworkin, 1993; Eikelboom & Stewart, 1982; Siegel & Ramos, 2002). Rescorla’s event-memory research focused a great deal on extinction (e.g., Rescorla, 1973; Rescorla, 1974; Rescorla & Heth, 1975). Once a CS-US association is formed as a result of CS-US pairings, CS-alone presentations (i.e., extinction trials) will result in a loss of conditioned responding to the CS. Rescorla stated that CS-alone presentations degrade the representation of the US because the US is no longer being presented at a time that it is expected. If a sufﬁcient number of extinction trials occurred, then not only was the US representation degraded, but the CS-US association will lose strength. Rescorla has conducted a considerable amount of research on extinction phenomena over the past few decades. His views of the underlying mechanisms of extinction in classical conditioning have changed greatly over the years (e.g., Rescorla 2001, 2004). In particular, although providing evidence that extinction does not involve unlearning of the original CS-US association (e.g., Rescorla, 1996), Rescorla may be less convinced than some contemporary theorists that extinction involves an association between the representation of the CS and the representation of the absence of the US (or no US; e.g., Rescorla, 2007).

01-Schachtman-01.indd 14

Two CSs can also be associated if they are presented together (two or more CSs presented together are collectively known as a stimulus compound or compound CS; this stimulus compound may be either simultaneous or serial depending, respectively, on whether the two CSs occur together at the same time or if one CS terminates before the other begins). Rescorla referred to these as within-compound associations (e.g., Rescorla & Cunningham, 1978; Rescorla & Durlach, 1981). Within-compound associations have important implications for many conditioning phenomena. For instance, let’s call one CS “A” and a second CS “B.” If A and B are each associated with the same US and A and B are also associated with each other (via a withincompound association), then what degree of conditioned responding might one expect if, say, A is presented? Well, A is associated with the US and so A can activate the US representation; therefore, we expect a CR to A based on this association. However, A also has the ability to activate a representation of B, and B has the ability to activate a representation of the US, and so A can produce a CR based on this sequence of activations as well (the A representation activates the B representation, this latter representation activates the US representation, and an extra vigorous CR results; e.g., Rescorla & Durlach, 1981). Within-compound associations can inﬂuence other conditioning effects. What if we have four CSs: A, B, C, and D? Let’s say that A is associated with B such that A has the ability to activate a representation of B in memory. Assume also that C and D are associated such that C has the ability to activate D in memory. Recent work by Holland and Sherwood (2008) and Dwyer, Mackintosh, and Boakes (1998) has found that if A and C are presented together, then an association can be formed between B and D because A has the ability to activate a representation of B on the AC trial and C has the ability to activate the D representation on the AC trial. Therefore, both the B and D representations are active in the organism’s memory at the same time on the AC conditioning trial. This is a very impressive ﬁnding because it shows two events becoming associated when neither is present on the trial

2/15/2011 7:41:41 PM

THINGS TO KNOW ABOUT CONDITIONING

that is responsible for their associative formation. Simply put, B and D become associated with each other even though they were never paired together. Along with phenomena like sensory preconditioning and higher order conditioning (to be discussed later, see section on “Compound Conditioning”), within-compound associations indicate a level of complexity and scope to classical conditioning that is not widely appreciated today. Postconditioning Presentations of the Unconditioned Stimulus

In support of his event-memory model, Rescorla (e.g., 1974) showed that one can change the representation of the US after conditioning and prior to testing. Rats were given CS-US pairings (e.g., tone paired with medium-intensity footshock) in phase 1. The CS was therefore associated with a representation of footshock which was moderate in intensity. Then in phase 2 Rescorla gave some rats a low-intensity footshock (just shocks presented in the experimental chamber— no CS was presented during this phase) in order to change the US representation from the previously formed “medium-intensity” to a new, low-intensity shock representation. These rats showed a smaller CR—which one would expect if the tone activated a low-intensity shock representation (the low-intensity shock representation replaced the older medium-intensity shock representation). Other rats received a higher intensity shock presented alone without the CS in phase 2. These rats now had a strong US representation associated with the tone (the strong shock representation replaced the medium one), so a stronger CR was expected; and, indeed, a stronger CR was observed. These results are fundamentally important because they demonstrate that, postconditioning, the representation of the USs can be changed (either deﬂated or inﬂated) in the absence of the CS. These ﬁndings reveal manipulations in classical conditioning that have yet to be fully exploited outside the learning laboratory for their clinical relevance. Another classical conditioning phenomenon has been used to document such processes. Counterconditioning is a treatment in which a

01-Schachtman-01.indd 15

15

CS is paired with a US of one affective valence (e.g., appetitive) in phase 1; and then during phase 2 (the counterconditioning treatment) the CS is paired with a US of the opposite affective valence (e.g., aversive). Readers may recognize that counterconditioning is one of the techniques used to deter alcoholics from drinking by pairing alcohol with an aversive experience (e.g., Revusky, 2009). Counterconditioning was also used to change the US representation associated with a CS for Rescorla’s rats that had received CS-footshock pairings earlier. Counterconditioning results in a lower CR to the CS. However, it is not clear what kind of associations underlie counterconditioning, that is, what the “contents” of learning might be. If a tone is ﬁrst paired with footshock and, subsequently, the tone is paired with food, then it is easy to imagine that the tone has an association with footshock as well as an association with food. If this is the case, then it is not clear that a footshock representation would truly be modiﬁed by the second US representation. Readers might ask whether these various postconditioning treatments (US representational deﬂation via counterconditioning and extinction) for decreasing the CR possess any similarities. As mentioned earlier, the associative structures underlying them are likely different. However, at least one similarity can be mentioned for counterconditioning and extinction: They both appear to be sensitive to context manipulations, as Bouton’s work demonstrates (e.g., Bouton, 1993, 2002). We will note one more ﬁnding before discussing context effects. This ﬁnding involves administering CS-alone extinction presentations after the CS-US conditioning pairings. Of course, extinction presentations will decrease the CR to the CS. Research conducted in the laboratories of Bouton, Miller, and Rescorla, among others, has examined the inﬂuence of the administration of USs after extinction and prior to testing the CS on performance. Similar to the ideas inherent in Rescorla’s event-memory model, extinction can be said to weaken (or deﬂate) the US representation. The US presentations occurring after extinction, then, can be said to increase (reestablish or reﬂate) the US representation.

2/15/2011 7:41:41 PM

16

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

This treatment is called “US reinstatement” and it does serve to increase the CR for a CS that has received extinction training. However, other interpretations of this effect exist. It has been argued that CS-US associations are learned during conditioning, a CS-noUS association is learned during extinction and the latter association causes interference with the former association, thus rendering it less retrievable. In other words, some have claimed that extinction reduces the retrievability of the CS-US association (and enhances the accessibility of a new CS-noUS association that is presumably learned during extinction); and that the US presentation during reinstatement causes rehearsal of the CS-US association, thereby making this CS-US association retrievable once again. Alternatively, it has also been pointed out that the US presentations can result in context conditioning (i.e., the formation of context-US associations) as a result of the context-US pairings at the time of the US reinstatement phase. If subjects are tested on the CS in the same context where the US reinstatement presentations occurred, then they are being tested on two events at once—the CS and the contextual cues, and both events are associated with the US, so a stronger CR is expected compared to subjects that did not receive US presentations in the context (where testing will occur) prior to the test (Bouton & Bolles, 1979). For the latter condition, only the CS has a strong association with the US. The former subjects are tested on two cues that are associated with the US. Of course, if all subjects are tested on the CS in a neutral context (a context where no other experimental treatments occurred although subjects may be previously acclimated to this context), then both groups are only tested on one event associated with the US. The Role of Contextual Cues During Conditioning

The importance of the conditioning of contextual cues has received a great deal of attention since the 1980s (see, e.g., the edited volume by Balsam & Tomie, 1985). We do not have space here to review the large amount of research ﬁndings on the role of these cues, but we will note

01-Schachtman-01.indd 16

two things. First, since pairings between a CS and US (i.e., classical conditioning) as well as instrumental learning do not occur in a vacuum, all types of conditioning procedures have the potential to produce associations involving contextual cues. These associations may involve context-CS associations (akin to withincompound associations mentioned earlier) or context-US associations. Second, there has been a lot of attention focused on the possibility that contextual cues can inﬂuence the retrieval of CS-US associations. The laboratories of Bouton, Miller, and Schachtman (Bouton, Garcia-Gutierrez, Zilski, & Moody, 2006; Chelonis, Calton, Hart, & Schachtman, 1999; Gunther, Denniston, & Miller, 1998) have shown that extinction is context speciﬁc. That is, if CS-US pairings occur in one context (Context A) and the CS-alone presentations (extinction) occur in a second context (Context B), then performance during those extinction trials will depend on where the CS is tested. After the extinction phase, if the CS is presented in the context where extinction occurred (Context B), then poor conditioned responding will occur. If the CS is tested in a context other than the extinction context (i.e., Context A or a third, neutral context), then the extinction treatment will not be expressed there and a strong CR to the CS will be seen (e.g., Bouton, 2004). This ﬁnding has a number of clinical implications not least of which concerns the efﬁcacy of treatments for phobias that utilize a desensitization treatment in a context other than that in which the original fear learning occurred or where the person will encounter the cue in the future (for further discussion, see Bouton, Westbrook, Corcoran, & Maren, 2006; Hermans, Craske, Mineka, & Lovibond, 2006; Redish, Jensen, Johnson, & Kurth-Nelson, 2007; Sotres-Bayon, Cain, & LeDoux, 2006). Another phenomenon that also shows great context speciﬁcity is latent inhibition (also known as the “CS preexposure effect”). Latent inhibition is the poor learning to a CS that is paired with a US after preconditioning exposure to the same CS in the absence of the US (Lubow, 1989, 2009; Lubow & Weiner, 2010). That is, in phase 1 the CS is presented several times on its

2/15/2011 7:41:41 PM

THINGS TO KNOW ABOUT CONDITIONING

own. Then the learning that stems from CS-US pairings in phase 2 is poor because of the phase 1 CS-alone trials. We should also point out that context-CS associations can presumably be formed even when a CS is presented alone in a context, and these associations can be important (Wagner, 1976). Finally, there has also been a lot of research examining the extent to which contextual cues function just like another CS or have distinctive properties of their own (Balsam & Tomie, 1985). Compound Conditioning

Another area of research that has produced a great deal of attention since the 1970s is compound conditioning—conditioning in which more than one CS is presented on a trial. Blocking was discovered in the late 1960s, and this ﬁnding stimulated an abundance of research that (along with other ﬁndings) gave rise to the RescorlaWagner (1972) model of classical conditioning. Blocking occurs when a weak CR is elicited by a CS (the “target CS”) that had been conditioned in the presence of a second CS (the “blocking CS”) that had been previously paired with the US. That is, in phase 1, the “blocking CS” (CS A) is paired with the US (A-US trials). In phase 2, the blocking CS and target CS (CS X) are presented together and paired with the US (AX-US trials). When the target CS (X) is tested, a weak CR occurs. The target CS does not produce a CR because the information that it provides (signaling the occurrence of the US) is redundant with the information already provided by the blocking CS (A). In phase 1, the subjects learn that the US is predicted by the blocking CS, A. In phase 2, the subject already expects the US because of the presence of CS A, and so no evidence of learning to CS X occurs. CS X is redundant and learning about it is “blocked.” The Rescorla-Wagner model highlights the important idea that CSs can compete for learning and predicts a large number of conditioning phenomena (Miller, Barnet, & Grahame, 1995). This model predicts another phenomenon in which CSs compete for learning— overshadowing,4 as well as many phenomena involving conditioned inhibitors5—stimuli that predict that the US will not occur.

01-Schachtman-01.indd 17

17

Some researchers have pointed out that CSs and instrumental responses may compete for learning. If a CS and a response both occur on a conditioning trial along with an outcome, then blocking or overshadowing of one of these events by the other event may occur (see Schachtman & Reed, 1998 for a review). If such competition occurs, then it is valuable to consider this effect because organisms are typically responding in some way at the time that a classical conditioning trial occurs, such that this behavior may compete with the CS for learning and attenuate the degree of manifest classical conditioning to the CS. Moreover, stimuli are typically present during instrumental conditioning (minimally, the cues of the experimental apparatus within which the subject is located), and these cues may compete with the response for learning and undermine the degree of expressed instrumental conditioning. Second-order conditioning and sensory preconditioning are two compound conditioning phenomena. Second-order conditioning involves pairing of a CS (A) with the US in phase 1 (A-US pairings) and then in phase 2 the target CS (X) is presented along with A but without the US (XAtrials; notice that a minus symbol is used to designate the absence of the US). A conditioned response occurs to X even though it had never been paired with the US. One early interpretation of this phenomenon by Rescorla (1980a, 1980b) is that the CR might occur to X because X is associated with A and A is associated with the US. Hence, X is capable of activating a representation of A in memory (via the A-X association acquired in phase 2) and this activation of A is capable of activating a representation of the US in memory (via the A-US association acquired in phase 1). X evokes a CR since the US representation has been activated when X is presented (via this sequence of associative linkages). Note that the association between A and X is similar to the within-compound associations mentioned earlier. Sensory preconditioning is similar in many ways to second-order conditioning. The primary difference is that the two training phases are reversed. In sensory preconditioning, then, XA– is presented in phase 1. Note that no US is

2/15/2011 7:41:41 PM

18

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

presented in this phase—just two CSs. In phase 2, A is paired with the US (A-US) and then, in the test phase, the occurrence of the CR to CSX is examined. A CR to X demonstrates that X can generate a CR without ever having been paired directly with the US. Instead, X was paired with A and an association was formed between them. CSA was paired with the US. Hence, a CR to X could occur if X activates a representation of A and A activates a representation of the US; and the active US representation results in a CR. However, other mechanisms may exist for why the CR to X occurs following sensory preconditioning training. These ﬁndings and related phenomena reveal interesting effects in classical conditioning and provide a glimpse at the theoretical approaches to understanding their underlying processes and structures. Distinctions Among Procedures in Which Single Conditioned Stimuli Are Presented

Researchers can have a difﬁcult time distinguishing among the many procedures that, for the most part, involve single-stimulus presentation. Habituation, pseudoconditioning, dishabituation, latent inhibition, disinhibition, sensitization, and extinction can easily be confused. Habituation and sensitization are easy to discuss because they involve the presentation of stimuli in a situation in which no conditioning trials occur. Habituation is the progressive decrease in responding to the stimulus if it is repeatedly presented. Sensitization is the increase in responding to a stimulus if it is repeatedly presented. Thus, in terms of behavioral responding, sensitization is the opposite of habituation. However, sensitization, unlike habituation, occurs when the subject, for any of a number of reasons, is in a state of high arousal. Habituation may be characterized as learning to ignore stimuli that have proven to be of no biological consequence. By helping us not to become distracted by harmless and meaningless stimuli, habituation is one of the most pervasive phenomena in our life. Sensitization, on the other hand, involves heightened vigilance to stimuli that might be dangerous. Indeed, it has been suggested that sensitization may contribute to a number of

01-Schachtman-01.indd 18

clinical conditions, including phobia and generalized anxiety disorder (e.g., Marks, 1987), as well as posttraumatic stress disorder (e.g., Dykman, Ackerman, & Newton, 1997). Although published many years ago, the dual-process theory of Groves and Thompson (1970; also see Thompson, 2009), which views habituation and sensitization as independent processes that function in parallel, remains one of the more prominent explanations of these two phenomena. Dishabituation is also easy to explain. This occurs when a stimulus (let’s say a tone) has already received habituation training so that the subject is no longer making a strong response to it. But some other “extraneous” stimulus occurs (say, a light) and this produces dishabituation to the tone in that a response suddenly occurs to the tone when it is presented again (especially if it is presented shortly after the light occurred). We have already discussed extinction—the decrease in CR when a CS (that has been previously conditioned using CS-US trials) is presented alone for many trials. Hence, a tone could be paired with a US and then given tone-alone extinction trials. If another “extraneous stimulus” occurs (say, a light), then the light might cause the CR to return to the CS. This increase in conditioned responding is called disinhibition. Therefore, dishabituation and disinhibition both involve an extraneous stimulus causing an increase in responding to a stimulus that had previously lost response-producing ability. Dishabituation involves a response that was not previously conditioned, whereas disinhibition involves a previously conditioned cue. Both habituation and extinction involve a decrease in responding (in the case of extinction, it is a CR that decreases; in the case of habituation, it is the waning of an unconditioned response to a stimulus). It is not clear if the mechanisms underlying extinction and habituation (as well as for disinhibition and dishabituation) are similar; however, it seems unlikely that the two phenomena are comparable. Latent inhibition, as we mentioned, involves CS-alone presentations prior to CS-US pairings, and the effect of these initial presentations of the CS is poor learning when the CS is paired with

2/15/2011 7:41:42 PM

THINGS TO KNOW ABOUT CONDITIONING

the US. Latent inhibition shares features with a few of the other phenomena presented in this section. The procedure for latent inhibition is the opposite of extinction: CS-alone presentations precede CS-US pairings for the former, whereas CS-alone presentations follow CS-US pairings for the latter. Despite this similarity (and difference) it seems unlikely that extinction and latent inhibition possess identical mechanisms; although, as mentioned earlier, both effects involve dependence on the contextual cues present as to whether the learning that occurs during the CS-alone phase will be expressed. Both phenomena may involve retrieval processes (Miller, Kasprow, & Schachtman, 1986). The ﬁrst phase of a latent inhibition procedure is similar to a habituation experiment in which the stimulus is simply presented repeatedly. Of course, a habituation experiment focuses on the response to the stimulus during this exposure phase, while latent inhibition focuses on conditioning to the stimulus in a subsequent phase; but, nonetheless, investigators have been curious as to whether similar processes may occur during the stimulus-alone phases during habituation and latent inhibition (e.g., a loss of attention). Pseudoconditioning is a procedure in which subjects receive presentations of the CS and US (e.g., a tone paired with shock) and it appears as though a CR is occurring to the CS due to conditioning (i.e., it appears as though learning has occurred), but what has really happened is that the US presentation on a previous trial has inﬂuenced the subject in such a way that they make a response (e.g., a startle response such as freezing) to the CS that looks like, but is not, the CR (e.g., a fear response such as freezing). Hence, conditioning has not occurred, but it appears as though it has occurred. Researchers sometimes use control conditions to identify (and therefore potentially rule out) the contribution that pseudoconditioning is making to manifest performance during conditioning. One reason that we have discussed the differences (and similarities) among these (primarily) single-stimulus treatments is that researchers and clinicians may become confused when administering a treatment. For instance, if someone has

01-Schachtman-01.indd 19

19

a fear of an untethered, growling, snarling dog and seeks treatment for this fear, then the clinician may administer a form of exposure therapy for the treatment. These therapeutic interventions (exposure therapy) are often viewed as extinction treatments. They are described this way for good reason in many cases. However, was the fear response to the dog acquired through conditioning or not? If the fear was acquired through conditioning, then certainly the use of ﬂooding or systematic sensitization should be considered extinction. But if there was no conditioning or learning of the fear (if the person has always had it), then the treatment could be viewed as habituation training. In both cases, we know that recovery of the fear can occur through spontaneous recovery (via time) or disinhibition/dishabituation (via an extraneous salient event) for the extinguished response. Both habituation and extinction can be reversed by a retention interval or an extraneous event. We also know that extinction (Bouton, Garcia-Gutierrez, Zilski, & Moody, 2006; Chelonis, Calton, Hart, & Schachtman, 1999; Gunther, Denniston, & Miller, 1998) and habituation (Marlin & Miller, 1981) are context speciﬁc; if you change contexts, the response can return. This chapter covered numerous instances of conditioning. The chapter sought to clarify some issues that may confuse psychologists and other researchers or practitioners, but it also illustrated some important ﬁndings that explore the processes and structures underlying associative learning. The following chapters examine the role of conditioning, based on contemporary research, in a number of domains, including connectionist modeling, psychoneuroimmunology, social inference and attribution, incentive learning, fears and phobias and anxiety, addictive behaviors, social learning, marketing, schizophrenia, learned helplessness, autism, analgesic responses, sexual behavior, evaluative conditioning, and consummatory behavior. These chapters were written by many of the premiere investigators in the ﬁeld, and we hope they illustrate that conditioning and associative learning theory and research continues to ﬂourish and be inﬂuential in many areas of our discipline.

2/15/2011 7:41:42 PM

20

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

ACKNOWLEDGMENTS Preparation of this chapter was supported in part by National Institutes of Deafness and Other Communication Disorders grant DC06456 to Steve Reilly. Thanks to Stephanie Fowler, Oskar Pineno, Jen Walker, and Jian-You Lin for their comments on this chapter.

NOTES 1. Note that the abbreviation “US” is used currently by most animal conditioning researchers rather than the more archaic “UCS” that one occasionally sees in some elementary textbooks. 2. It is a negative contingency because of the occurrence of the response means that the aversive event has not occurred and the occurrence of the aversive event means that the response did not occur. 3. A trial is deﬁned as the events that occur prior to each reward presentation. 4. Overshadowing, like blocking, results in weaker learning to one of the two elements of a compound CS. However, overshadowing is empirically deﬁned as the difference in conditioning to a CS that is paired with the US in the presence of a second CS relative to a group that receives conditioning to a CS paired with the US in the absence of a second CS. 5. A conditioned inhibitor is a cue that signals the absence of an otherwise expected US.

REFERENCES Adams, C. D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 34B, 77–98. Adams, C. D., & Dickinson, A. (1981). Instrumental responding following reinforcer devaluation. Quarterly Journal of Experimental Psychology, 33B, 109–121. Balleine, B. W. (2005). Neural bases of foodseeking: Affect, arousal and reward in corticostriatolimbic circuits. Physiology and Behavior, 86, 717–730. Balleine, B. W. (2009). Taste, disgust, and value: Taste aversion learning and outcome encoding in instrumental conditioning. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes

01-Schachtman-01.indd 20

(pp. 262–280). New York, NY: Oxford University Press. Balleine, B. W., Liljeholm, M., & Ostlund, S. B. (2009). The integrative function of the basal ganglia in instrumental conditioning. Behavioral Brain Research, 199, 43–52. Balsam, P. D., & Tomie, A. (Eds.). (1985). Context and learning. Hillsdale, NJ: Erlbaum. Bolles, R. C. (1972). Reinforcement, expectancy, and learning. Psychological Review, 79, 394–409. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E. (2002). Context, ambiguity and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry, 52, 976–986. Bouton, M. E. (2004). Context and behavioral processes in extinction. Learning and Memory, 11, 485–494. Bouton, M. E. (2007). Learning and behavior: A contemporary synthesis. Sutherland, MA: Sinauer. Bouton, M. E., & Bolles, R. C. (1979). Role of conditioned contextual stimuli in reinstatement of extinguished fear. Journal of Experimental Psychology: Animal Behavior Processes, 5, 368–378. Bouton, M. E., Garcia-Gutierrez, A., Zilski, J., & Moody, E. W. (2006). Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse. Behavioural Research and Therapy, 44, 983–994. Bouton, M. E., Westbrook, R. F., Corcoran, K. A., & Maren, S. (2006). Contextual and temporal modulation of extinction: Behavioral and biological mechanisms. Biological Psychiatry, 60, 352–360. Chelonis, J. J., Calton, J. L., Hart, J. A., & Schachtman, T. R. (1999). Attenuation of the renewal effect by extinction in multiple contexts. Learning and Motivation, 30, 1–14. Colwill, R. M., & Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11, 120–132. Colwill, R. M., & Rescorla, R. A. (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol 20, pp. 55–104). Orlando, FL: Academic Press. Delamater, A. R. (2004). Experimental extinction in Pavlovian conditioning: Behavioural and

2/15/2011 7:41:42 PM

THINGS TO KNOW ABOUT CONDITIONING

neuroscience perspectives. Quarterly Journal of Experimental Psychology, 57B, 97–132. Depoortere, R., Perrault, G., & Sanger, D. J. (1999). Intracranial self-stimulation under a progressive-ratio schedule in rats: Effects of strength of stimulation, d-amphetamine, 7-OH-DPAT and haloperidol. Psychopharmacology, 142, 221–229. Dickinson, A. (1980). Contemporary animal learning theory. Cambridge, UK: Cambridge University Press. Dickinson, A. (1985). Actions and habits: The development of behavioral autonomy. Philosophical Transactions of the Royal Society (London). B308, 67–78. Dickinson, A., & Balleine, B. (1994). Motivational control of behavior. Animal Learning and Behavior, 22, 1–18. Dickinson, A., Balleine, B., Watt, A., Gonzalez, F., & Boakes, R. A. (1995). Motivational control after extended instrumental training. Animal Learning and Behavior, 23, 197–206. Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983). The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 35B, 35–51. Domjan, M. (2009). The principles of learning and behavior (6th ed.). Belmont, CA: Wadsworth/ Thompson Learning. Dworkin, B. R. (1993). Learning and physiological regulation. Chicago, IL: University of Chicago Press. Dwyer, D. M., Mackintosh, N. J., & Boakes, R. A. (1998). Simultaneous activation of the representations of absent cues results in the formation of an excitatory association between them. Journal of Experimental Psychology: Animal Behavior Processes, 24, 163–171. Dykman, R. A., Ackerman, P. T., & Newton, J. E. (1997). Posttraumatic stress disorder: A sensitization reaction. Integrative Physiological and Behavioral Science, 32, 9–18. Eikelboom, R., & Stewart, J. (1982). Conditioning of drug-induced physiological responses. Psychological Review, 89, 507–528. Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York, NY: AppletonCentury-Crofts. Frieman, J. (2002). Learning and adaptive behavior. Belmont, CA: Wadsworth/Thompson Learning. Gluck, M. A., Mercado, E., & Myers, C. E. (2007). Learning and memory: From brain to behavior. New York, NY: Worth Publishers.

01-Schachtman-01.indd 21

21

Groves, P. M., & Thompson, R. F. (1970). Habituation: A dual-process theory. Psychological Review, 77, 419–450. Gunther, L. M., Denniston, J. C., & Miller, R. R. (1998). Conduct exposure treatment in multiple contexts can prevent relapse. Behaviour Research and Therapy, 36, 75–91. Hearst, E., & Jenkins, H. M. (1974). Sign–tracking: The stimulus–reinforcer relation and directed– action. Austin, TX: The Psychonomic Society. Hermans, D., Craske, M. G., Mineka, S., & Lovibond, P. F. (2006). Extinction in human fear conditioning. Biological Psychiatry, 60, 361–368. Hodos, W. (1961). Progressive ratio as a measure of reward strength. Science, 134, 943–944. Holland, P. C. (1992). Occasion-setting in Pavlovian conditioning. In D. L. Medin (Ed.), The psychology of learning and motivation (Vol. 28, pp. 69–125). San Diego, CA: Academic Press. Holland, P. C. (2004). Relations between Pavlovianinstrumental transfer and reinforcer devaluation. Journal of Experimental Psychology: Animal Behavior Processes, 30, 104–117. Holland, P. C., & Sherwood, A. (2008). Formation of excitatory and inhibitory associations between absent events. Journal of Experimental Psychology: Animal Behavior Processes, 34, 324–335. Holland, P. C., & Wheeler, D. S. (2009). Representation-mediated food aversions. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 196–225). New York, NY: Oxford University Press. Kamin, L. J. (1968). “Attention–like” processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation (pp. 9–31). Coral Gables, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York, NY: Appleton–Century–Crofts. Keesey, R. E., & Goldstein, M. D., (1968). Use of progressive ﬁxed-ratio procedures in the assessment of intracranial reinforcement. Journal of the Experimental Analysis of Behavior, 11, 293–301. Klein, S. B., & Mowrer, R. R. (Eds.). (1989a). Contemporary learning theories: Instrumental conditioning theory and the impact of biological constraints on learning. Hillsdale, NJ: Erlbaum.

2/15/2011 7:41:42 PM

22

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Klein, S. B., & Mowrer, R. R. (Eds.). (1989b). Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Erlbaum. Konorski, J. (1967). Integrative activity of the brain. Chicago, IL: University of Chicago press. Locurto, C. M., Terrace, H. S., & Gibbon, J. (Eds.). (1981). Autoshaping and conditioning theory. New York, NY: Academic Press. Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. Cambridge, England: Cambridge University Press. Lubow, R. E. (2009). Conditioned taste aversion and latent inhibition: A review. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 37–57). New York, NY: Oxford University Press. Lubow, R. E., & Weiner. I. (Eds.). (2010) Latent inhibition: Cognition, neuroscience, and applications to schizophrenia. New York, NY: Cambridge University Press. Mackintosh, N. J. (1974). The psychology of animal learning. London, England: Academic Press. Mackintosh, N. J. (1983). Conditioning and associative learning. New York, NY: Clarendon Press. Mackintosh, N. J. (Ed.). (1994). Animal learning and cognition. London, England: Academic Press. Marks, I. M. (1987). Fears, phobias and rituals: Panic, anxiety, and their disorders. New York, NY: Oxford University Press. Marlin, N. A., & Miller, R. R. (1981). Associations to contextual stimuli as a determinant of long-term habituation. Journal of Experimental Psychology: Animal Behavior Processes, 7, 313–333. Miller, R. R., Barnet, R. C., & Grahame, J. (1995). Assessment of the Rescorla-Wagner model. Psychological Bulletin, 117, 363–386. Miller, R. R., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Sources and consequences. American Journal of Psychology, 99, 145–218. Mowrer, O. H. (1951). Two-factor learning theory: Summary and comment. Psychological Review, 58, 350–354. Mowrer, R. R., & Klein, S. B. (Eds.). (2001). Handbook of contemporary learning theories. Hillsdale, NJ: Erlbaum. Pavlov, I. P. (1927). Conditioned reﬂexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). London, England: Oxford University Press. Pearce, J. M. (2008). Animal learning and cognition (3rd ed.). New York, NY: Psychology Press.

01-Schachtman-01.indd 22

Ranaldi, R., & Wise, R. A. (2000). Intravenous selfadministration of methamphetamine-heroin (speedball) combinations under a progressiveratio schedule of reinforcement in rats. Neuroreport, 11, 2621–2623. Redish, A., Jensen, S., Johnson, A., & Kurth-Nelson, A. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychological Review, 114, 784805. Reilly, S. (1999). Reinforcement value of gustatory stimuli determined by progressive ratio performance. Pharmacology Biochemistry and Behavior, 63, 301–311. Rescorla, R. A. (1973). Effect of US habituation following conditioning. Journal of Comparative and Physiological Psychology, 82, 137–143. Rescorla, R. A. (1974). Effect of inﬂation of the unconditioned stimulus value following conditioning. Journal of Comparative and Physiological Psychology, 86, 101–106. Rescorla, R. A. (1980a). Pavlovian second-order conditioning: studies in associative learning. Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1980b). Simultaneous and successive associations in sensory preconditioning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 207–216. Rescorla, R. A. (1988). Pavlovian conditioning. It’s not what you think it is. American Psychologist. 43, 151–160. Rescorla, R. A. (1991). Associative relations in instrumental learning: The eighteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology, 43B, 1–23. Rescorla, R. A. (1996). Preservation of Pavlovian associations through extinction. Quarterly Journal of Experimental Psychology, 49B, 245–258. Rescorla, R. A. (2001). Experimental extinction. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 119–154). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (2004). Spontaneous recovery. Learning and Memory, 11, 501–509. Rescorla, R. A. (2007). Renewal after overexpectation. Learning and Behavior, 35, 19–26. Rescorla, R. A., & Cunningham, C. L. (1978). Within-compound ﬂavor associations. Journal of Experimental Psychology: Animal Behavior Processes, 4, 267–275. Rescorla, R. A., & Durlach, P. J. (1981). Within-event learning in Pavlovian conditioning. In N. E. Spear & R. R. Miller (Eds.), Information processing

2/15/2011 7:41:42 PM

THINGS TO KNOW ABOUT CONDITIONING

in animals: Memory mechanisms (pp. 81–111) Hillsdale, NJ: Erlbaum. Rescorla, R. A., & Heth, D. C. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 1, 88–96. Rescorla, R. A., & Solomon, R. L. (1967). Twoprocess learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychological Review, 74, 151–182. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton–Century–Crofts. Revusky, S. (2009). Chemical aversion treatment of alcoholism. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 445–472). New York, NY: Oxford University Press. Roberts, D. C. S., Morgan, D., & Liu, Y. (2007). How to make a rat addicted to cocaine. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 31, 1614–1624. Rozeboom, W. W. (1958). “What is learned?” An empirical enigma. Psychological Review, 65, 22–33. Schachtman, T. R., & Reed, P. (1998). Optimization: Some factors that facilitate and hinder optimal performance in animals and humans. In W. O’Donahue (Ed.), Learning and behavior therapy (pp. 301–333). Needham Heights, MA: Allyn & Bacon. Schmajuk, N. A., & Holland, P. C. (1998). Occasion setting: Associative learning and cognition in animals. Washington, DC: American Psychological Association. Sclafani, A., & Ackroff, K. (2003). Reinforcement value of sucrose measured by progressive ratio operant licking in the rat. Physiology and Behavior, 79, 663–670. Shefﬁeld, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical conditioning: A symposium (pp. 302–322). New York, NY: Appleton-Century-Crofts. Siegel, S., & Ramos, B. M. C. (2002). Applying laboratory research: Drug anticipation and the treatment of drug addiction. Experimental and Clinical Psychopharmacology, 10, 162–183. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York, NY: AppletonCentury-Crofts.

01-Schachtman-01.indd 23

23

Sotres-Bayon, F., Cain, C. K., & LeDoux, J. E. (2006). Brain mechanisms of fear extinction: Historical perspectives on the contribution of prefrontal cortex. Biological Psychiatry, 60, 329–336. Spear, N. E. (1978). The processing of memories: Forgetting and retention. Hillsdale, NJ: Erlbaum. Spear, N. E., & Riccio, D. C. (1994). Memory: Phenomena and principles. Needham Heights, MA: Allyn & Bacon. Thompson, R. F. (2009). Habituation: A history. Neurobiology of Learning and Memory, 92, 127–134. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monograph 2 (4, Whole No. 8). Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York, NY: Macmillan. Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward and maze performance in rats. University of California Publications in Psychology, 4, 257–275. Tulving, E., & Pearlstone, Z. (1966). Availability versus accessibility of information in memory for words. Journal of Verbal Learning and Verbal Behavior, 5, 381–391. Wagner, A. R. (1976). Priming in STM: An information-processing mechanism for selfgenerated or retrieval-generated depression in performance. In T. J. Tighe & R. N. Leaton (Eds.), Habitation: Perspectives from child development, animal behavior, and neurophysiology (pp 95–128). Hillsdale, NJ: Erlbaum. Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and learning (pp. 301–336). New York, NY: Academic Press. Williams, B. A. (1999). Associative competition in operant conditioning: Blocking the responsereinforcer association. Psychonomic Bulletin and Review, 6, 618–623. Williams, D. R. (1981). Biconditional behavior: Conditioning without constraint. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 55–99). New York, NY: Academic Press. Williams, D. R., & Williams, H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511–520. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience. 7, 464–476.

2/15/2011 7:41:42 PM

This page intentionally left blank

PART II

Applications to Clinical Pathology

02-Schachtman-02.indd 25

2/15/2011 7:42:10 PM

This page intentionally left blank

CHAPTER 2 Fear Extinction and Emotional Processing Theory A Critical Review Seth J. Gillihan and Edna B. Foa

The process of fear extinction in animal studies bears strong resemblance to the process of reduction of pathological anxiety in humans via exposure therapy. Thus, ﬁndings emerging from experiments of extinction can inform us about the mechanisms of exposure therapy, which may lead to modifying the manner in which therapists conduct exposure and thereby improve treatment outcomes. In this chapter we use emotional processing theory as a framework to organize the knowledge about both exposure therapy and extinction. We examine whether hypotheses and suppositions derived from the theory are consistent with knowledge emerging from extinction experiments and from treatment studies of anxiety disorders; conversely, we examine how emotional processing theory can inform the questions that extinction research needs to address. We include an examination of the neural correlates of the reduction of pathological fear, which may allow us to expand our knowledge of mechanisms of exposure therapy by adding brain processes to the existing behavioral mechanisms implicated in extinction.

INTRODUCTION We view a parallel between exposure therapy and extinction paradigms in animals and humans and therefore take the position that ﬁndings emerging from experiments of extinction can inform us about the mechanisms of exposure therapy. It is our hope that this information will lead to modifying the manner in which therapists conduct exposure and thereby improve outcomes. As clinical researchers have long noted (Baum, 1970; Dollard & Miller, 1950; Stampﬂ & Levis, 1967), the process of fear extinction in animal studies bears strong resemblance to the process of reduction of pathological anxiety via exposure therapy, including the amelioration of symptoms of anxiety disorders. Indeed, patients with anxiety disorders tend to show deﬁcient fear extinction in laboratory studies (Lissek et al., 2005). Research into the phenomenon of fear

extinction therefore may have great relevance to the understanding of the learning processes that facilitate effective cognitive-behavioral treatment for anxiety disorders. In this chapter we use emotional processing theory (Foa & Kozak, 1985, 1986) as a framework to organize the knowledge about both exposure therapy and extinction. This approach will allow us to examine whether hypotheses and suppositions derived from the theory are consistent or in contradiction with knowledge emerging from extinction experiments and from treatment studies of anxiety disorders; conversely, we can examine how emotional processing theory can inform the questions that need to be asked by extinction research. Also, the review of the extinction literature may allow us to expand our knowledge of mechanisms of exposure therapy by adding brain processes to the existing behavioral mechanisms implicated in extinction.

27

02-Schachtman-02.indd 27

2/15/2011 7:42:10 PM

28

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

EMOTIONAL PROCESSING THEORY Foa and Kozak’s (1985, 1986) emotional processing theory presented a heuristic model for understanding pathological anxiety and the mechanisms involved in treatment of anxiety disorders. Expanding on Lang’s (1977, 1984) bioinformational model, the starting point of emotional processing theory is the supposition that fear is represented in memory as a cognitive structure that includes information about the fear stimuli, the fear responses, and their meaning. In contrast to Lang who emphasized the role of response representations, emotional processing theory places particular emphasis on the meaning representations of the stimuli and the responses. For example, a combat soldier in Vietnam may have a fear structure that includes representations of stimuli such as persons moving through the jungle and representations of responses such as heart beating fast and muscle tension. Of particular importance, however, is the meaning of the persons moving through the jungle as “those are Viet Cong soldiers and my life is in danger”; and the meaning of heart beating fast and muscle tension as “I am afraid.” The representations of the stimuli, responses, and their meaning in the structure are related to each other such that when a stimulus and/or response in the environment matches those represented in the fear structure, the entire structure is activated. Thus, seeing an enemy soldier in the jungle will activate the representation of a moving person, the meaning associated with that representation (“I’m in danger”), and the behavioral and physiological fear responses. In addition to emphasizing the crucial role of meaning representations, Foa and Kozak (1986) outlined the distinguishing features of normal and pathological fear structures. In the example mentioned earlier, the soldier’s fear structure is normal if it is restricted to the jungle setting during wartime; in these circumstances, activation of the fear structure will lead to adaptive responses such as staying low and avoiding enemy ﬁre. In contrast, the fear structure is pathological if it is activated by safe stimuli, for example when, upon returning to the United States

02-Schachtman-02.indd 28

the Vietnam veteran experiences fear and “takes cover” when seeing a person walking through the woods while vacationing with his family in the Great Smoky Mountains National Park. In other words, the pathological fear structure produces overgeneralization when safe stimuli are perceived as dangerous. Pathological fear structures also comprise excessive response elements (e.g., hypervigilance). According to emotional processing theory, two conditions are required for the modiﬁcation of pathological fear structures. First, the fear structure must be activated through exposure to information that is sufﬁciently similar to the information embedded in the fear structure. Second, information that is incompatible with the pathological elements of the fear structure must be available during exposure and incorporated into the fear structure. In the example of the combat veteran, the fear structure was activated when the veteran was exposed to persons moving through the woods; remaining in that situation and discovering that the persons do not harm him constitutes information that is incompatible with his perception that all moving persons in the woods are dangerous. The incorporation of this information into the existing pathological fear structure modiﬁes the pathological elements in the structure. This modiﬁcation, which is the essence of emotional processing, underlies the reduction in pathological fear. While it is not possible to directly observe fear structures and their modiﬁcation, Foa and Kozak (1986) postulated three indicators of successful emotional processing: activation of the fear structure, as indicated by both subjective and objective measures of fear; within-session habituation,1 or the reduction of anxiety within the course of a treatment session; and betweensession habituation, that is, lower peak anxiety to fear-related stimuli during successive treatment sessions. As Foa and Kozak (1986) suggested, many sources of data can be brought to bear to test the prediction that these three indicators of emotional processing are associated with the reduction of pathological fear. The original formulations of the theory incorporated existing research, which included behavioral and

2/15/2011 7:42:10 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

psychophysiological studies conducted primarily with human subjects; an update of the theory (Foa, Huppert, & Cahill, 2006) examined the degree to which the latest research lent support to the predictions of emotional processing theory, again focusing on clinical ﬁndings from human studies. The current chapter aims to evaluate the validity of emotional processing theory in light of basic research in learning processes. Learning research offers a powerful experimentally testable model of psychopathology and of treatment. It is particularly relevant for anxiety disorder because the biology and psychology of fear have been the focus of animal studies for decades; as such, ﬁndings from the learning literature may inform theoretical models of pathological anxiety and its treatment. Since the work of Ivan Pavlov (e.g., 1927), learning research has shed light on the processes that underlie the acquisition and retention of associations between stimuli and responses. Learning research has played a key role in our understanding of anxiety disorders, including panic disorder (Bouton, Mineka, & Barlow, 2001), posttraumatic stress disorder (PTSD; Rothbaum & Davis, 2003), and simple phobia (Davey, 1992). The applicability of learning principles to the study of anxiety is particularly salient in the context of PTSD because of the clear etiological factors associated with the development of the disorder. Indeed, PTSD is the sole anxiety disorder with diagnostic criteria that require an identiﬁable event that preceded the onset of the disorder. Much animal research has been done to model the fear responses seen after the experience of a trauma. Fear conditioning as an animal model of PTSD was discussed at length by Foa, Zinbarg, and Rothbaum (1992). Brieﬂy, Foa and colleagues argued that the behavioral responses seen in animals that are exposed to uncontrollable and unpredictable aversive stimuli parallel the clusters of symptoms seen in PTSD. For example, the heightened fear and arousal seen in fear-conditioned animals may be analogous to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) criterion of persistent arousal; similarly, animals tend to show avoidance of the fear-related stimuli, just

02-Schachtman-02.indd 29

29

as PTSD patients tend to avoid trauma-related stimuli. The authors also make the case that animal models of PTSD can be used to generate hypotheses about human trauma-related psychopathology. For instance, based on the animal literature they posit that traumatic events that lead to PTSD are perceived not only as more life threatening (e.g., Dunmore, Clark, & Ehlers, 1999; Resnick, Kilpatrick, Dansky, Saunders, & Best, 1993) but also as unpredictable or uncontrollable. Most individuals who experience a traumatic event experience fear-related symptoms that overlap with those of PTSD, including reexperiencing the event in response to reminders of it, hyperarousal, and avoidance of trauma-related stimuli (e.g., Breslau, Reboussin, Anthony, & Storr, 2005; Rothbaum, Foa, Riggs, Murdock, & Walsh, 1992). These symptoms ameliorate over time in most trauma survivors. When such reduction does not occur, chronic PTSD develops. Thus, PTSD can be viewed as a failure to extinguish conditioned fear responses. This perspective on the tenacious fear reactions in PTSD is supported by studies showing that patients with PTSD demonstrate deﬁcient fear extinction relative to controls (Blechert, Michael, Vriends, Margraf, & Wilhelm, 2007; Lissek et al., 2005; Peri, Ben-Shakhar, Orr, & Shalev, 2000). Earlier conceptualizations viewed extinction as a passive process of undoing learned associations, and little research was devoted to understanding the processes that facilitate or inhibit extinction. More recent research revealed that fear extinction is an active process that is distinct from the process of fear acquisition. Accordingly, an extinguished fear response (conditioned stimulus [CS]) may show spontaneous recovery (e.g., Leung & Westbrook, 2008), is “reinstated” when the unconditioned stimulus (US) is presented in the absence of the CS (e.g., Rescorla & Heth, 1975), and reemerges in contexts other than the extinction setting (e.g., Alvarez, Johnson, & Grillon, 2007). These ﬁndings suggest that extinction is not simply the erasure of the acquired fear response. Instead, extinction is thought to involve the inhibition of a fear association through the formation of new stimulus–response associations. As noted by

2/15/2011 7:42:11 PM

30

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Barad, Gean, and Lutz (2006), “extinction gates the expression of fear” (p. 326). This new conceptualization inﬂuenced Foa and McNally (1996) who updated emotional processing theory by suggesting that exposure therapy generates new fear structures rather than weakening the associations of the existing pathological fear structure. As discussed earlier, emotional processing theory posits three indicators that emotional processing of the fear structure has occurred (i.e., the pathological elements of the fear structure were modiﬁed): activation of the fear, and within-session and between-session habituation to fear-related stimuli. The hypotheses that are generated by the proposed indicators can be tested in fear extinction paradigms. First, the degree of fear activation during extinction training will be positively associated with a greater reduction in fear responses. Second, the degree of fear reduction within a fear extinction session will be associated with greater reduction at testing. Finally, lower peak responses in successive sessions of presentation of the previously conditioned stimulus will be associated with fear reduction at testing. We will address each of these hypotheses by examining the relevant research ﬁndings from human and animal studies from basic behavioral research to neural localization (i.e., identifying a brain area that is associated with a particular behavior or cognitive process) studies across a range of methodologies. We will focus on fear, although clearly PTSD comprises many other emotions such as anger or guilt; nevertheless, the majority of PTSD sufferers experience debilitatingly high levels of unrealistic fear. The fear is evident in the common symptoms of PTSD such as avoidance of trauma-related stimuli, stress reaction to trauma reminders, hypervigilance, and being easily startled. A Note About Terminology

For the sake of clarity we distinguish between the extinction phase, which involves presenting the CS in the absence of the US to previously fear conditioned organisms, and the test phase, which occurs at some point in time after the extinction phase.

02-Schachtman-02.indd 30

Evaluation of Prediction 1: Fear Activation During Extinction Behavioral Studies

During fear extinction paradigms, fear is activated by deﬁnition because the organism has been conditioned to experience fear when confronted with a CS that was associated with an aversive US. The fact that fear is activated as part of the extinction paradigm results in the restriction of range (i.e., low variability) in initial fear, which in turn renders the examination of the effects of level of fear activation on subsequent fear extinction problematic given that the detection of a signiﬁcant relationship between level of activation and degree of extinction requires variability in initial fear. Despite this inherent limitation, there is evidence that the level of fear activation during fear extinction learning predicts fear responding at test. Animals who were administered a chemical agent known to reduce fear activation (i.e., barbiturates) during fear extinction showed greater fear responses to later presentations of the CS during the test phase when the animals were not drugged (e.g., Barry, Etheredge, & Miller, 1965). Similarly, Bouton, Kenney, and Rosengard (1990) found that benzodiazepines commonly used in the treatment of anxiety disorders (chlordiazepoxide and diazepam) interfered with fear extinction learning. Bouton et al. paired a chamber with footshock and subsequently exposed the animals to the chamber in the absence of shock. Although rats in these experiments showed fear extinction during the extinction learning phase of the experiment irrespective of whether they received benzodiazepines, animals that had received the benzodiazepines during extinction showed signiﬁcantly greater fear responses during the test phase when the animals were exposed to the chamber in an undrugged state. Given that benzodiazepines and other sedating drugs reduce fear-related behavior, these data support the premise that fear activation during the extinction of conditioned fear is associated with greater reductions in fear at test. However, these studies did not speciﬁcally address variability in the animals’ fear responses during extinction learning as a predictor of

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

eventual fear extinction; rather, the studies focused on the differences among groups that received different drugs. Moreover, other interpretations of these results are possible, such as state-dependent learning—that is, that optimal recall during the test phase depends on the animal’s being in the same physiological state as during the learning acquisition phase (see Overton, 1985). Nevertheless, results from these and similar studies are consistent with the hypothesis that greater activation during exposure therapy is associated with greater reductions in anxiety. More direct behavioral evidence for the role of activation in fear extinction comes from later research demonstrating that direct pharmacologic manipulation of fear responses during extinction affects fear responding during the test phase. Cain, Blouin, and Barad (2004) administered yohimbine or propranolol to mice prior to fear extinction; yohimbine blocks the α2-receptor, resulting in increased adrenergic activity and anxiety, while propranolol blocks the β-receptor, leading to decreased adrenergic activity and anxiolysis. In light of the role of the adrenergic system in anxiety disorders (Bremner, Krystal, Southwick, & Charney, 1996) and the strong evidence for the role of norepinephrine in PTSD (Southwick et al., 1999), manipulations of this neurotransmitter system have important implications for the extinction of conditioned fear. Results from Cain et al. indicated that animals treated with yohimbine showed enhanced fear extinction during the test phase; in contrast, animals treated with propranolol exhibited impaired extinction. These ﬁndings again are consistent with the hypothesis that fear activation during exposure is associated with better treatment outcomes. In addition, these results provide compelling evidence that state-dependent learning per se cannot account for the observed effects of activation on fear reduction, given that neither the propranolol- nor the yohimbine-treated mice were in the same physiological state during the test phase as during the extinction learning phase. Morris and Bouton (2007) also performed multiple experiments to test the effect of yohimbine on conditioned fear extinction, measuring

02-Schachtman-02.indd 31

31

freezing response during extinction training under yohimbine versus saline administration. For the purpose of this discussion, the amount of freezing is taken to indicate level of fear activation during extinction training. Results indicated that a 1.0 mg/kg dose of yohimbine was associated with less fear activation during extinction, as well as less fear responding during the test phase. Given that yohimbine produces fearlike symptoms, the ﬁnding that the yohimbine group showed less fear during the test phase provides additional evidence that pharmacologically induced activation is associated with greater reductions in conditioned fear. However, because the yohimbine group did not show more fear response during the extinction learning phase than did the control group, this study does not present evidence that it was the greater initial fear activation which was responsible for the greater extinction learning. The activation hypothesis of emotional processing theory appears to be supported not only by extinction of conditioned fear but also of conditioned appetitive responses. In an elegant series of experiments, Rescorla (2000) paired various CSs with the delivery of food pellets. Animals that showed the greatest response during the extinction phase also showed the most profound degree of extinction during the test phase. These animals had been presented with compound CSs (noise plus light) during the extinction phase, which produced the greatest mean response during this phase; when presented subsequently with a single extinguished CS (noise or light) at test, the rats in this condition exhibited the lowest level of conditioned responses. These results were replicated and extended to a fear conditioning paradigm (Rescorla, 2006), further conﬁrming the importance of activation in extinction learning. Taken together, results from extinction studies demonstrate that greater activation during extinction is associated with greater decrements in the conditioned behavior. It follows that without activation of the relevant representations, extinction is unlikely to occur; this seems true for both fear and appetitive conditioning. Given the importance of activation, optimal extinction depends on the identiﬁcation of factors that promote

2/15/2011 7:42:11 PM

32

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

activation. On the basis of Rescorla’s ﬁndings (2000, 2006), the presentation of multiple CSs that are associated with the US are most likely to produce enhanced activation and thus greater subsequent extinction of the CS. Implications of this observation for exposure therapy will be discussed later. Comparatively little research has been conducted to test for the effects of pharmacologically manipulated activation on fear extinction in humans. Contrary to the ﬁndings for animals in which the β-blocker propranolol was associated with impaired fear extinction (Cain et al., 2004), Orr et al. (2006) found no signiﬁcant effect of propranolol on fear extinction among a group of males with PTSD. Conclusive evidence for the role of pharmacological agents in augmenting activation and thereby enhancing extinction awaits future research. Of note, a process complementary to extinction also demonstrates that alteration of fear memory requires activation. Several studies have shown that learned fear can be disrupted by blocking the reconsolidation of CS-US associations. In a typical experiment in this area, animals that had learned a CS-US pairing were presented with the CS alone some time after fear training. The retrieval of the fear memory in response to the CS presentation temporarily renders the memory labile and sensitive to disruption; evidence for this phenomenon comes from studies showing that interruption of the processes that are required for reconsolidation of the retrieved memory leads to a decrease in fear responding to the CS (e.g., Lee, Milton, & Everitt, 2006; Nader, Schafe, & LeDoux, 2000). Importantly, activation of the learned fear association through presentation of the CS must precede administration of the chemical agents that interrupt reconsolidation; without activation of the fear, there is no effect on the CS-US association (Nader et al., 2000). These ﬁndings provide further evidence for the critical role of fear activation in altering learned fear associations. Neural Studies

In addition to the behavioral data summarized earlier, many studies examining neural function have demonstrated that the extinction of

02-Schachtman-02.indd 32

conditioned fear is associated with the activation of brain structures that are involved in the representation of fear. The most widely studied brain structure in this context is the amygdala. The amygdala is an almond-shaped mass that is located in the limbic system; it plays a key role in the processing of emotional information, including emotional memory (Phelps, 2005) and making judgments about emotional facial expressions in humans (for a review see Sergerie, Chochol, & Armony, 2008). Importantly, the amygdala has been implicated in the acquisition of fear conditioning, both in animals (LeDoux, 2000) and in humans (LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998). Of direct relevance to the current chapter, patients with PTSD tend to show hyperactivity of the amygdala (e.g., Rauch et al., 2000; Shin et al., 2004), indicating the chronically heightened fear and arousal experienced by these individuals. Numerous experimental paradigms have demonstrated the involvement of the amygdala in fear extinction. In animals, the blockade of N-methyl-D-aspartate (NMDA) receptors by injection of NMDA antagonists, either systemically (Baker & Azorlosa, 1996) or into the amygdala, leads to poor extinction learning (Falls, Miserendino, & Davis, 1992; SotresBayon, Bush, & LeDoux, 2007). Conversely, infusion of the NMDA agonist D-cycloserine (DCS) either systemically or directly into the amygdala enhances the extinction of conditioned fear (Walker, Ressler, Lu, & Davis, 2002). The involvement of the amygdala in fear extinction has been demonstrated through tests of other neurotransmitter receptor systems, including the cannabinoid receptor type 1 (Chhatwal, Davis, Maguschak, & Ressler, 2005) and L-type voltage-gated calcium channels (e.g., Cain, Blouin, & Barad, 2002). In addition, fear extinction is associated with depotentiation of excitatory thalamic neurons that terminate at synapses on the amygdala (Kim et al., 2007). These studies indicated that amygdala activity is recruited during the extinction of conditioned fear. However, they do not tell us deﬁnitively whether the amygdala activity represents the activation of the existing fear structure, the modiﬁcation of the existing fear structure, or the

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

creation of an inhibitory extinction representation. A recent study by Herry et al. (2008) provides striking evidence that the basal amygdala is involved in both conditioned fear and its extinction. This research demonstrated that conditioned fear leads to the activation of “fear neurons” in response to the presentation of a CS. When the CS is presented in the absence of the US, “extinction neurons” increase their ﬁring rate; fear neurons subsequently reduce their ﬁring rate. Importantly, these neuronal changes preceded changes in freezing behavior. Crucially, the fear neurons continue to ﬁre at the beginning of extinction (see Herry et al., Fig. 3c), with extinction neurons starting to ﬁre just before fear neurons switch off. These results demonstrated that the neurons representing fear are active during the extinction of fear. Findings from the literature on fear memory reconsolidation also demonstrate the essential role of neurons in the amygdala in the modiﬁcation of learned fear associations. For example, reconsolidation can be inhibited by blocking protein synthesis in the amygdala (Nader et al., 2000); blocking NMDA receptors in the amygdala (Lee et al., 2006); blocking noradrenergic transmission in the amygdala (Debiec & LeDoux, 2004); the infusion of cannabinoid CB1 receptor agonists (Lin, Mao, & Gean, 2006); and by genetic disruption of transcription in the amygdala (Mamiya et al., 2009). Again, these effects hold only when presentation of the CS precedes administration of the chemical agents, that is, after activation of the fear memory. Thus, there is ample evidence that amygdala activation underlies the activation of fear memories and subsequent modiﬁcation of these memories. Human neuroimaging studies provide an essential test for the relevance of the animal ﬁndings for patients (Rauch, Shin, & Phelps, 2006). A related line of research has shown the involvement of the amygdala in human fear extinction. LaBar et al. (1998) found signiﬁcant amygdala activation during the early phase of fear extinction. Later studies replicated and extended these ﬁndings. Amygdala activation was found when the association between the US and the CS was altered, both during fear acquisition and extinction (Knight, Smith, Cheng, Stein, & Helmstetter,

02-Schachtman-02.indd 33

33

2004; Milad et al., 2007); moreover, the degree of fear extinction was strongly correlated with amount of amygdala activation (Phelps, Delgado, Nearing, & LeDoux, 2004). These data are consistent with Foa and Kozak’s (1986) emotional processing theory, which states that activation of the fear structure is necessary for its modiﬁcation and the subsequent alleviation of pathological fear levels present in anxiety disorders, including PTSD. If one can view exposure therapy as analogous to extinction and the reduction in different measures of PTSD as indicators of extinction, then there are several clinical studies that also lend support to the activation hypothesis (e.g., Foa, Riggs, Massie, & Yarczower, 1995; Kozak, Foa, & Steketee, 1988; Lang, Melamed, & Hart, 1970; Pitman et al., 1996). These studies assessed level of fear activation using different measures of fear (e.g., facial fear expression, heart rate, subjective ratings), thus demonstrating the robust effect of activation on extinction. The data from the fear extinction literature in animals and humans presented earlier provide a window into the mechanisms that underlie fear reduction and suggest a line of research that will combine fear extinction paradigms and studies on exposure therapy. Evaluation of Prediction 2: Within-Session Fear Reduction

According to Foa and Kozak (1986), the second indicator of emotional processing is withinsession fear reduction. It can be argued that within-session habituation would enhance extinction in the test phase. A small number of studies have tested this prediction, with mixed results. One approach that researchers have used is to compare the effects of massed versus spaced presentation of the CS during extinction training on fear responding during extinction and later during recall of extinction—that is, during presentation of the CS without the US some time after the extinction learning. Studies have demonstrated that massed presentations produce greater within-session fear extinction; at test 1 week later, massed CS presentations were associated with lower fear responding when animals were again presented with the CS alone

2/15/2011 7:42:11 PM

34

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(Cain, Blouin, & Barad, 2003). This ﬁnding is consistent with the emotional processing theory prediction. Two similar studies that examined appetitive extinction produced inconsistent results. Rescorla and Durlach (1987) obtained similar results to those of Cain et al. (2003), ﬁnding that extinction in pigeons was more durable when extinction trials were massed (intertrial interval of 10 sec) versus spaced (2 min). In contrast, Moody, Sunsay, and Bouton (2006) found no signiﬁcant effect of 60- versus 240-sec intertrial intervals; although the shorter interval produced greater within-session extinction, there was no signiﬁcant effect of interval at the test phase. Other studies demonstrated that the relationship between within-session fear reduction and extinction of conditioned fear during the test phase is more complicated. In a context conditioning paradigm, Li and Westbrook (2008) found that although spaced extinction trials produced less within-session fear reduction than massed extinction trials, the spaced trials resulted in greater fear extinction during the test phase. However, this study differed from that of Cain et al. (2003) in several important ways. First, Cain et al. used presentation of a discrete CS, whereas Li and Westbrook studied context conditioning, with each stint in the conditioning context representing a “session” of fear extinction learning. This distinction renders the Li and Westbrook design a test of between-session fear reduction, given that the animal was removed from the conditioned context for varying intervals between trials. Second, Cain et al.’s intertrial intervals ranged from 6 sec to 6 min, whereas those of Li and Westbrook were 4 min versus 24 hr. Therefore, the study by Li and Westbrook appears to apply to the spacing of exposure sessions rather than to the reduction of fear within a session. In summary, Foa and Kozak’s (1986) view that within-session habituation is an indicator of emotional processing has not received as strong support from studies of fear extinction as did their view of fear activation as an indicator of, as well as a necessary condition for, emotional processing. Similarly, exposure therapy studies with human beings also do not lend strong

02-Schachtman-02.indd 34

support to the role of within-session habituation in promoting emotional processing (e.g., Kozak et al., 1988; Riley et al., 1995; van Minnen & Foa, 2006). Thus, although patients often experience within-session habituation during exposure sessions, the degree of within-session habituation is not a consistent predictor of outcome. Evaluation of Prediction 3: Between-Session Fear Reduction Different Brain Mechanisms Involved in Within- and Between-Session Habituation

The third indicator of emotional processing in Foa and Kozak’s (1986) theory is betweensession habituation of fear. Emotional processing theory proposes that although betweensession fear reduction is related to within-session fear reduction, the two involve separate processes. In extinction paradigms, between-session fear reduction can be viewed as analogous to the retention of fear extinction between the extinction phase and the subsequent test phase, as distinct from the initial acquisition of fear extinction learning, which is akin to within-session fear reduction. Foa (1979) posited that different brain mechanisms are involved in within- and betweensession habituation in exposure therapy. Building on ﬁndings by Groves and Lynch (1972), she argued that a low-level brain structure (brain stem reticular formation) is involved in within-session habituation while between-session habituation relies on forebrain structures. Foa further argued that if forebrain structures are necessary for retention of habituation (extinction) across time, then higher level cognitive processes seem to be necessary for consolidation of extinction learning. Support for the view that two distinct learning processes are involved in short- and longterm extinction comes from studies of evaluative conditioning. In these studies there is a distinction between expectancy learning and evaluative learning; in the former, the organism learns that a certain event (stimulus) predicts another event, while in the latter the organism assigns a negative or positive valence to the CS (see Chapter 18; for a review see De Houwer, Thomas, & Baeyens, 2001). For example, an animal will

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

learn that a red light paired with a subsequent electric shock predicts the onset of the shock (expectancy learning); assigning a negative valence to the red light itself represents evaluative learning. Interestingly, Vansteenwegen et al. (2006) note that evaluative learning is more resistant to extinction than is expectancy learning. Accordingly, a subject may stop producing an electrodermal response following extinction of a previously conditioned stimulus that predicted electric shock but still will have a negative evaluation of the conditioned stimulus (e.g., Hermans, Vansteenwegen, Crombez, Baeyesn, & Eelen, 2002; Vansteenwegen, Francken, Vervliet, De Clercq, & Eelen, 2006). It is reasonable to assume that evaluative conditioning involves higher level cognitive operations and that those operations are more resistant to extinction, given the involvement of higher order brain areas (especially the prefrontal cortex; Zysset, Huber, Ferstl, & von Cramnon, 2002) in evaluative judgments. Indeed, there is evidence that evaluative conditioning is intact even in the absence of the amygdala (Coppens et al., 2006). These observations may suggest that extinction of expectancy learning is more similar to within-session extinction and that extinction of evaluative learning is more akin to between-session extinction. The distinctions between expectancy learning and evaluative learning may have important implications for exposure therapy (see Discussion). Taken together with Groves and Lynch (1972), the results from evaluative conditioning studies reinforce the view that separate brain areas and cognitive processes underlie within- and between-session habituation. Brain Mechanisms Underlying Retention of Fear Extinction

A large body of studies has conﬁrmed that distinct neural regions underlie the retention of fear extinction versus the acquisition of fear extinction. The vast majority of these studies have implicated the medial prefrontal cortex (mPFC) in the retention of extinguished fear in both animals and humans. In animals, many experimental paradigms have been used to examine the role of mPFC in extinction retention (for a review,

02-Schachtman-02.indd 35

35

see Quirk, Garcia, & González-Lima, 2006). In lesion studies, the destruction of mPFC structures is associated with normal within-session extinction, but the animals do not retain the extinction learning 24 hours later at test (e.g., Lebrón, Milad, & Quirk, 2004; Quirk, Russo, Barron, & Lebron, 2000). Disruption of the targets of inhibitory mPFC projections within the amygdala also has been shown to interfere with extinction recall (Chhatwal, Stanek-Rattiner, Davis, & Ressler, 2006; Likhtik, Popa, AspergisSchoute, Fidacaro, & Paré, 2008). Additional research has provided more precise estimates of the temporal course of mPFC involvement in extinction retention. Reversible pharmacological disruption of mPFC activity (by injection of PD098059, a mitogen-activated protein kinase inhibitor) immediately after fear extinction learning also produced deﬁcient extinction retention during the test phase (Hugues, Deschaux, & Garcia, 2004). Similar results were obtained via the infusion of an NMDA receptor antagonist immediately after extinction training (Burgos-Robles, Vidal-Gonzalez, Santini, & Quirk, 2007). Both of these studies found no signiﬁcant deﬁcit in extinction retention at test if the chemical agents were applied 2 hours after extinction training, suggesting that mPFC activity during this period is critical for extinction retention. Complementary paradigms have produced corroborating results; rather than examining the loss of function associated with destroying mPFC, Milad and Quirk (2002) measured neuronal activity in intact mPFC during fear conditioning, extinction training, and recall of extinction at test. They found that neurons in mPFC showed increased ﬁring rates only during the 24-hour recall of extinguished fear conditioning, again supporting the involvement of this region in extinction retention. Similar studies in humans have corroborated the results from animal studies showing that the mPFC is involved in extinction retention. Kalisch et al. (2006) paired presentation of faces (CS) with electric shock during the conditioning phase and then extinguished the CR by presenting the faces without shock. The following day during the test phase, presentation of the

2/15/2011 7:42:11 PM

36

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

extinguished CS produced signiﬁcant activation of mPFC. Milad et al. (2007) replicated these results using a slightly different paradigm, pairing electric shock with different visual contexts. In addition to ﬁnding signiﬁcant mPFC activations during recall of extinction, their results showed that the degree of extinction retention at test was positively correlated with percent signal change in mPFC. Phelps et al. (2004) also found that mPFC activity increased during the recall of extinction, and they reported a positive correlation between mPFC activity during extinction recall and prior success of extinction; that is, subjects who showed greater extinction during extinction training also had greater activation of mPFC during the recall of extinction at test. In addition to the functional associations between mPFC and extinction retention, structural variability within this region was found to be correlated with recall of extinction. Milad et al. (2005) found a positive correlation between thickness of mPFC and extinction retention. Retention of extinction learning has been consistently shown to be context dependent; that is, a CS that is extinguished in one context may provoke the CR when presented in a novel context at test (e.g., Bouton & Bolles, 1979). The hippocampus appears to play a crucial role in evaluating context in recall of extinction learning. Indeed, disruption of the hippocampus following fear extinction prevented the return of conditioned fear responses in a novel context (Corcoran & Maren, 2001), suggesting that hippocampal input is necessary for context sensitivity in extinction recall. Furthermore, the hippocampus has been shown to play a similar role in the contextual modulation of human extinction recall during the test phase (Kalisch et al., 2006; Milad et al., 2007). That is, if conditioning occurs in one context and extinction in another context, then the CR ordinarily would return if the subjects are tested in a novel context (Bouton & Bolles, 1979); such return of the CR does not occur in those with hippocampal lesions. Convergence Between Experimental Laboratory and Clinical Treatment Findings

In summary, it seems that between-session habituation (i.e., extinction retention) relies on

02-Schachtman-02.indd 36

higher cognitive processes more than does within-session habituation (extinction learning), given the higher level cortical areas that support between-session habituation. Supporting this conclusion are the ﬁndings that different brain structures are involved in each type of habituation, suggesting a distinction between extinction learning and extinction retention at test. The laboratory ﬁndings that extinction retention and extinction learning comprise different processes may explain why between-session habituation, which is akin to extinction retention, is related to treatment outcome (e.g., Kozak et al., 1988; Lang et al., 1970; Rauch et al., 2004), whereas within-session habituation generally is not. Moreover, the ﬁndings summarized earlier, suggesting the involvement of higher cognitive functioning in extinction retention, may further explain why between-session habituation is crucial for recovery from pathological anxiety. After all, to recover, patients need to remember the outcome of their previous experiences— namely, that their feared consequences were disconﬁrmed—and such memory relies on higher cognitive functioning.

DISCUSSION Implications of Fear Extinction Literature for Emotional Processing Theory

In this chapter our goal was to reexamine and extend emotional processing theory by considering three core tenets of the theory—activation, within-session habitation, and between-session habituation—in light of experimental extinction paradigms. Speciﬁcally, we have attempted to examine the body of information about extinction learning and retention and the related brain structures and functions in light of emotional processing theory with the goal of furthering our understanding of the mechanisms that support the reduction of pathological fear. Our review indicates that there is strong evidence from the extinction as well as from the treatment literature that initial fear activation is important in achieving extinction of conditioned fear and in reducing pathological anxiety after exposure therapy. These ﬁndings support the emotional

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

activation hypothesis of emotional processing theory. Behavioral, neural, and pharmacological data converge to indicate that the fear structure must be activated in order for the conditioned fear to be extinguished. These consistent ﬁndings from diverse areas will serve as a strong foundation for further research that will help us understand the basic mechanisms underlying fear extinction and exposure therapy. There have been fewer studies in the extinction literature that can shed light on the second hypothesis of emotional processing theory, that within-session fear extinction is related to outcome. This tenet of the theory has accrued less support in the clinical literature (for a review, see Craske et al., 2008), suggesting that withinsession fear reduction may not be required for fear to be extinguished or for successful exposure treatment to reduce pathological anxiety. However, the relative paucity of data argues for further examination of this hypothesis before a ﬁnal verdict is rendered. There is considerable support from the animal literature on extinction learning for the supposition of emotional processing theory that between-session habitation is strongly related to outcome of exposure therapy and therefore is a valid indicator of emotional processing. Of particular interest are the ﬁndings that activity in speciﬁc neural regions, namely the mPFC, is necessary for the maintenance of extinction learning between extinction training and subsequent test sessions. These data underscore the importance of between-session fear extinction and support the third tenet of emotional processing theory. The converging evidence from the clinical literature for the importance of between-session habituation on treatment outcome (e.g., Rauch et al., 2004) reinforces the importance of fear reduction across sessions as an indicator of emotional processing. Implications of Emotional Processing Theory for Extinction Research

As we noted in the Introduction, emotional processing theory and extinction paradigms in animal research may be mutually enriching in understanding the mechanisms involved in fear

02-Schachtman-02.indd 37

37

reduction and in the treatment of pathological anxiety. The literature reviewed in this chapter provides not only a test of the predictions put forward by emotional processing theory but also provides future directions for basic fear extinction research that may more directly inform the theory and treatment of anxiety. Few of the animal extinction studies reviewed herein were designed explicitly to test the tenets of emotional processing theory; more research in this area may further solidify the empirical basis of exposure treatment. Although the role of fear activation in extinction learning has been studied extensively and its importance is clearly supported by existing data, fewer studies have investigated the role of withinsession fear reduction on subsequent fear extinction. Similarly, although many studies have demonstrated that between-session fear reduction is a distinct process that appears to be related to extinction learning, few behavioral studies have tested the prediction that greater fear extinction across sessions is correlated with greater fear reduction at the test phase. Future studies designed speciﬁcally to bridge basic and clinical science must address issues that will help therapists modify exposure treatment in order to increase its efﬁcacy, such as optimal level of initial fear activation and the degree of withinsession fear reduction that is necessary for successful outcome. Treatment Implications

Findings from the fear extinction literature may have important implications for the practice of exposure therapy. As Foa and Kozak (1997) noted, the initial optimism that followed the advent of the ﬁrst behavioral therapies for anxiety gave way to the recognition that many patients fail to beneﬁt from the treatments, and that a signiﬁcant percentage of those that beneﬁt fail to maintain their gains. The authors argued that the apparent “efﬁcacy ceiling”—the leveling off of response rates to empirically tested cognitive and behavioral treatments—may be due to “an alienation from psychopathology and experimental psychology” (p. 606). They suggest that therapy researchers have not exploited ﬁndings

2/15/2011 7:42:11 PM

38

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

from basic research in psychopathology. In their view, greater communication between basic clinical scientists and therapy researchers has the potential to produce more powerful and more broadly applicable clinical treatments. Clinical application of the principles reviewed here in the context of emotional processing theory has the potential to raise the efﬁcacy ceiling on the treatment of anxiety disorders. For example, the results from Cain et al. (2004), who used pharmacologic agents to enhance the extinction of conditioned fear, suggest that exposure therapy may be delivered effectively across fewer sessions if activating agents are used to enhance the inhibitory learning processes. Existing research has shown that activating agents such as D-cycloserine enhance the effects of exposure therapy for anxiety disorders (e.g., Ressler et al., 2004). Some research laboratories currently are investigating the ability of agents such as yohimbine to enhance the efﬁcacy of exposure treatment for PTSD. If positive results are obtained, patients with PTSD may experience relief more quickly. Of particular importance is the possibility that people who might otherwise drop out before experiencing the effectiveness of the treatment might have quicker gains that encourage them to complete the treatment and reach full remittance of symptoms. Nonpharmacologic tools also may render exposure therapy more effective. For example, Powers and Emmelkamp (2008) in their metaanalysis found that virtual reality exposure (VRE) is more effective than in vivo exposure in the treatment of phobias. Virtual reality may increase the level of activation by producing a closer match with the fear structure. Effectiveness of VRE also may be driven by the ability to present multiple CSs (e.g., sounds of warfare; combat-related images) that deepen extinction (see Rescorla, 2000, 2006). In addition, VRE may help patients who are “under-engagers”; as Rothbaum (2009) points out, VR may help “patients who are reluctant to engage in recollections of feared memories or are not very good at imagining situations” (p. 210). Our current understanding of the role of within-session extinction does not provide clear guidance for the application of exposure therapy.

02-Schachtman-02.indd 38

Nevertheless, the ﬁnding that massed extinction sessions, which result in greater within-session extinction, are more effective in reducing fear than spaced sessions, suggests the importance of fear reduction within each exposure session. Among PTSD patients treated with exposure therapy, Pitman et al. (1996) found a trend for within-session HR habituation to be correlated with outcome. However, several clinical studies (e.g., Jaycox, Foa, & Morral, 1998; Kozak et al., 1988; van Minnen & Foa, 2006) have not found signiﬁcant associations between within-session habituation and outcome. Perhaps withinsession habituation is not a necessary condition for positive outcomes in exposure therapy for PTSD; alternatively, the subjective ratings of within-session distress may lack sensitivity for detecting associations between within-session experiences and outcome. Additional animal and neuroimaging studies may help resolve the question of whether within-session changes in emotional experience are important for longterm reduction in pathological fear reactions. The ﬁndings from evaluative conditioning may have implications for exposure therapy in that the negative evaluations of trauma-related stimuli such as automobiles (for motor vehicle accident survivors) or men (for rape survivors) likely will persist longer than will the fear responses to such stimuli. That is, while a patient with PTSD may no longer report distress in response to trauma-related stimuli because the expectancy has decreased in strength, the person may continue to avoid items on an exposure hierarchy in response to a persistent negative evaluation of these stimuli. An awareness of this phenomenon on the part of the therapist may be instrumental in helping the patient to overcome factors that encourage avoidance. Finally, the importance of consolidating fear extinction in order to retain the extinction learning (e.g., Hugues et al., 2004) also has important ramiﬁcations for exposure treatment. Patients should be encouraged strongly to abstain from activities, such as excessive drinking, that may interfere with memory consolidation processes, particularly in the immediate aftermath of an exposure session. Although abstaining may be more difﬁcult in the short term, especially for

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

patients who rely on chemical substances to numb their fear and anxiety, in the long run the patients will maintain their extinction learning and will experience more rapid relief from their anxiety.

NOTE 1. In this chapter we generally use the term extinction in the context of fear conditioning studies because it is clear what CS-US association is being extinguished. We use the less speciﬁc term habituation in the context of fear reduction during exposure therapy because in most cases it is unknown how the pathological fear originated.

REFERENCES Alvarez, R. P., Johnson, L., & Grillon, C. (2007). Contextual-speciﬁcity of short-delay extinction in humans: Renewal of fear-potentiated startle in a virtual environment. Learning and Memory, 14, 247–253. Baker, J. D., & Azorlosa, J. L. (1996). The NMDA antagonist MK-801 blocks the extinction of Pavlovian fear conditioning. Behavioral Neuroscience, 110, 618–620. Barad, M., Gean, P-W., & Lutz, B. (2006). The role of the amygdala in the extinction of conditioned fear. Biological Psychiatry, 60, 322–328. Barry, H., Etheredge, E. E., & Miller, N. E. (1965). Counterconditioning and extinction of fear fail to transfer from amobarbital to nondrug state. Psychopharmacologia (Berlin), 8, 150–156. Baum, M. (1970). Extinction of avoidance responding through response prevention (ﬂooding). Psychological Bulletin, 74, 276–284. Blechert, J., Michael, T., Vriends, N., Magraf, J., & Wilhelm, F. H. (2007). Fear conditioning in posttraumatic stress disorder: Evidence for delayed extinction of autonomic, experiential, and behavioural responses. Behaviour Research and Therapy, 45, 2019–2033. Bouton, M. E., & Bolles, R. C. (1979). Contextual control of the extinction of conditioned fear. Learning and Motivation, 10, 445–466. Bouton, M. E., Kenney, F. A., & Rosengard, C. (1990). State-dependent fear extinction with two benzodiazepine tranquilizers. Behavioral Neuroscience, 104, 44–55. Bouton, M. E., Mineka, S., & Barlow, D. H. (2001). A modern learning theory perspective on the

02-Schachtman-02.indd 39

39

etiology of panic disorder. Psychological Review, 108, 4–32. Bremner, J. D., Krystal, J. H., Southwick, S. M. & Charney, D. S. (1996). Noradrenergic mechanisms in stress and anxiety: II. Clinical studies. Synapse, 23, 39–51. Breslau, N., Reboussin, B. A., Anthony, J. C., & Storr, C. L. (2005). The structure of posttraumatic stress disorder: Latent class analysis in 2 community samples. Archives of General Psychiatry, 62, 1343–1351. Burgos-Robles, A., Vidal-Gonzalez, I., Santini, E., & Quirk, G. J. (2007). Consolidation of fear extinction required NMDA-dependent bursting in the ventromedial prefrontal cortex. Neuron, 53, 871–880. Cain, C. K., Blouin, A. M., & Barad, M. (2002). L-type voltage-gated calcium channels are required for extinction, but not for acquisition or expression, of conditional fear in mice. Journal of Neuroscience, 22, 9113–9121. Cain, C. K., Blouin, A. M., & Barad, M. (2003). Temporally massed CS presentations generate more fear extinction than spaced presentations. Journal of Experimental Psychology: Animal Behavior Processes, 29, 323–333. Cain, C. K., Blouin, A. M., & Barad, M. (2004). Adrenergic transmission facilitates extinction of conditional fear in mice. Learning and Memory, 11, 179–187. Chhatwal, J. P., Davis, M., Maguschak, K. A., & Ressler, K. J. (2005). Enhancing cannabinoid neurotransmission augments the extinction of conditioned fear. Neuropsychopharmacology, 30, 516–524. Chhatwal, J. P., Stanek-Rattiner, L., Davis, M., & Ressler, K. J. (2006). Amygdala BDNF signaling is required for consolidation but not encoding of extinction. Nature Neuroscience, 9, 870–872. Coppens, E., Vansteenwegen, D., Baeyens, F., Vandenbulcke, M., Van Paesschen, W., & Eelen, P. (2006). Evaluative conditioning is intact after unilateral resection of the anterior temporal lobe in humans. Neuropsychologia, 44, 840–843. Corcoran, K. A., & Maren, S. (2001). Hippocampal inactivation disrupts contextual retrieval of fear memory after extinction. Journal of Neuroscience, 21, 1720–1726. Craske, M. G., Kircanski, K., Zelikowsky, M., Mystkowski, J., Chowdhury, N., & Baker, A. (2008). Optimizing inhibitory learning during exposure therapy. Behaviour Research and Therapy, 46, 5–27.

2/15/2011 7:42:11 PM

40

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Davey, G. C. (1992). Classical conditioning and the acquisition of human fears and phobias: A review and synthesis of the literature. Advances in Behaviour Research and Therapy, 14, 29–66. Debiec, J., & LeDoux, J. E. (2004). Disruption of reconsolidation but not consolidation of auditory fear conditioning by noradrenergic blockade in the amygdala. Neuroscience, 129, 267–272. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy. New York, NY: McGraw-Hill. Dunmore, E., Clark, D. M., & Ehlers, A. (1999). Cognitive factors involved in the onset and maintenance of posttraumatic stress disorder (PTSD) after physical or sexual assault. Behaviour Research and Therapy, 37, 809–829. Falls, W. A., Miserendino, M. J., & Davis, M. (1992). Extinction of fear-potentiated startle: Blockade by infusion of an NMDA antagonist into the amygdala. Journal of Neuroscience, 12, 854–863. Foa, E. B. (1979). Failure in treating obsessivecompulsives. Behaviour Research and Therapy, 17, 169–176. Foa, E. B., Huppert, J. D., & Cahill, S. P. (2006). Emotional processing theory: An update. In B. O. Rothbaum (Ed.), The nature and treatment of pathological anxiety (pp. 3–24). New York, NY: Guilford Press. Foa, E. B., & Kozak, M. J. (1985). Treatment of anxiety disorders: Implications for psychopathology. In A. H. Tuma & J. D. Maser (Eds.), Anxiety and the anxiety disorders (pp. 421–452). Hillsdale, N J: Erlbaum. Foa, E. B., & Kozak, M. J. (1986). Emotional processing of fear: Exposure to corrective information. Psychological Bulletin, 99, 20–35. Foa, E. B., & Kozak, M. J. (1997). Beyond the efﬁcacy ceiling? Cognitive behavioral therapy in search of theory. Behavior Therapy, 28, 601–611. Foa, E. B., & McNally, R. J. (1996). Mechanisms of change in exposure therapy. In M. Rapee (Ed.), Current controversies in the anxiety disorders (pp. 329–343). New York, NY: The Guilford Press. Foa, E. B., Riggs, D. S., Massie, E. D., & Yarczower, M. (1995). The impact of fear activation and anger on the efﬁcacy of exposure treatment for posttraumatic stress disorder. Behavior Therapy, 26, 487–499.

02-Schachtman-02.indd 40

Foa, E. B., Zinbarg, R., & Rothbaum, B. O. (1992). Uncontrollability and unpredictability in posttraumatic stress disorder: An animal model. Psychological Bulletin, 112, 218–238. Groves, P. M., & Lynch, G. S. (1972). Mechanisms of habituation in the brain stem. Psychological Review, 79, 237–244. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Herry, C., Ciocchi, S., Senn, V., Demmou, L., Müller, C., & Lüthi, A. (2008). Switching on and off fear by distinct neuronal circuits. Nature, 454, 600–606. Hugues, S., Deschaux, O., & Garcia, R. (2004). Postextinction infusion of a mitogen-activated protein kinase inhibitor into the medial prefrontal cortex impairs memory of the extinction of conditioned fear. Learning and Memory, 11, 540–543. Jaycox, L. H., Foa, E. B., & Morral, A. R. (1998). Inﬂuence of emotional engagement and habituation on exposure therapy for PTSD. Journal of Consulting and Clinical Psychology, 66, 185–192. Kalisch, R., Korenfeld, E., Stephan, K. E., Weiskopf, N., Seymour, B., & Dolan, R. J. (2006). Contextdependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network. Journal of Neuroscience, 26, 9503–9511. Kim, J., Lee, S., Park, K., Hong, I., Song, B., Son, G.,. . .Choi, S. (2007). Amygdala depotentiation and fear extinction. Proceedings of the National Academy of Sciences, 104, 20955–20960. Knight, D. C., Smith, C. N., Cheng, D. T., Stein, E. A., & Helmstetter, F. J. (2004). Amygdala and hippocampal activity during acquisition and extinction of human fear conditioning. Cognitive, Affective, and Behavioral Neuroscience, 4, 317–325. Kozak, M. J., Foa, E. B., & Steketee, G. (1988). Process and outcome of exposure treatment with obsessive-compulsives: Psychophysiological indicators of emotional processing. Behavior Therapy, 19, 157–169. LaBar, K. S., Gatenby, C., Gore, J. C., LeDoux, J. E., & Phelps, E. A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: A mixed-trial fMRI study. Neuron, 20, 937–945.

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

Lang, P. J. (1977). Imagery in therapy: An information processing analysis of fear. Behavior Therapy, 8, 862–886. Lang, P. J. (1984). Cognition in emotion: Concept and action. In C. Izard, J. Kagan, & R. Zajonc (Eds.), Emotion, cognition and behavior (pp. 193–206). New York, NY: Cambridge University Press. Lang, P. J., Melamed, B. G., & Hart, J. (1970). A psychophysiological analysis of fear modiﬁcation using an automated desensitization procedure. Journal of Abnormal Psychology, 76, 220–234. Lebrón, K., Milad, M. R., & Quirk, G. J. (2004). Delayed recall of fear extinction in rats with lesions of ventral medial prefrontal cortex. Learning and Memory, 11, 544–548. LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 12, 155–184. Lee, J. L. C., Milton, A. L., & Everitt, B. J. (2006). Reconsolidation and extinction of conditioned fear: Inhibition and potentiation. Journal of Neuroscience, 26, 10051–10056. Leung, H. T., & Westbrook, R. F. (2008). Spontaneous recovery of extinguished fear responses deepens their extinction: A role for error-correction mechanisms. Journal of Experimental Psychology: Animal Behavior Processes, 34, 461–474. Li, S. H., & Westbrook, R. F. (2008). Massed extinction trials produce better short-term but worse long-term loss of context conditioned fear responses than spaced trials. Journal of Experimental Psychology: Animal Behavior Processes, 34, 336–351. Likhtik, E., Popa, D., Apergis-Schoute, J., Fidacaro, G. A., & Paré, D. (2008). Amygdala intercalated neurons are required for expression of fear extinction. Nature, 454, 642–645. Lin, H-C., Mao, S-C., & Gean, P-W. (2006). Effects of intra-amygdala infusion of CB1 receptor agnoists on the reconsolidation of fear-potentiated startle. Learning and Memory, 13, 316–321. Lissek, S., Powers, A. S., McClure, E. B., Phelps, E. A., Woldehawariat, G., Grillon, C., & Pine, D. S. (2005). Classical fear conditioning in the anxiety disorders: A meta-analysis. Behaviour Research and Therapy, 43, 1391–1424. Mamiya, N., Fukushima, H., Suzuki, A., Matsuyama, Z., Homma, S., Frankland, P. W., & Kida, S. (2009). Brain region-speciﬁc gene expression activation required for reconsolidation and extinction of contextual fear memory. Journal of Neuroscience, 29, 402–413.

02-Schachtman-02.indd 41

41

Milad, M. R., Quinn, B. T., Pitman, R. K., Orr, S. P., Fischl, B., & Rauch, S. L. (2005). Thickness of ventromedial prefrontal cortex in humans is correlated with extinction memory. Proceedings of the National Academy of Sciences, 102, 10706–10711. Milad, M. R., & Quirk, G. J. (2002). Neurons in medial prefrontal cortex signal memory for fear extinction. Nature, 420, 70–74. Milad, M. R., Wright, C. I., Orr, S. P., Pitman, R. K., Quirk, G. J., & Rauch, S. L. (2007). Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert. Biological Psychiatry, 62, 446–454. Moody, E. W., Sunsay, C., & Bouton, M. E. (2006). Priming and trial spacing in extinction: Effects on extinction performance, spontaneous recovery, and reinstatement in appetitive conditioning. Quarterly Journal of Experimental Psychology, 59, 809–829. Morris, R. W., & Bouton, M. E. (2007). The effect of yohimbine on the extinction of conditioned fear: A role for context. Behavioral Neuroscience, 121, 501–514. Nader, K., Schafe, G. E., & LeDoux, J. E. (2000). Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406, 722–726. Orr, S. P., Milad, M. R., Metzger, L. J., Lasko, N. B., Gilbertson, M. W., & Pitman, R. K. (2006). Effects of beta blockade, PTSD diagnosis, and explicit threat on the extinction and retention of an aversively conditioned response. Biological Psychiatry, 73, 262–271. Overton, D. A. (1985). Contextual stimulus effects of drugs and internal states. In P. D. Balsam & A. Tomie (Eds.), Context and Learning (pp. 357–384). Hillsdale, NJ: Lawrence Erlbaum. Pavlov, I. P. (1927). Conditioned reﬂexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). London, England: Oxford University Press. Peri, T., Ben-Shakhar, G., Orr, S. P., & Shalev, A. Y. (2000). Psychophysiologic assessment of aversive conditioning in posttraumatic stress disorder. Biological Psychiatry, 47, 512–519. Phelps, E. A. (2005). Emotion and cognition: Insights from studies of the human amygdala. Annual Review of Psychology, 57, 27–53. Phelps, E. A., Delgado, M. R., Nearing, K. I., & LeDoux, J. E. (2004). Extinction learning in humans: Role of the amygdala and vmPFC. Neuron, 43, 897–905.

2/15/2011 7:42:11 PM

42

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Pitman, R. K., Orr, S. P., Altman, B., Longpre, R. E., Poiré, R. E., Macklin, M. L.,. . . Steketee, G. S. (1996). Emotional processing and outcome of imaginal ﬂooding therapy in Vietnam veterans with chronic post-traumatic stress disorder. Comprehensive Psychiatry, 37, 409–418. Powers, M. B., & Emmelkamp, P. M. G. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders, 22, 561–569. Quirk, G. J., Garcia, R., & González-Lima, F. (2006). Prefrontal mechanisms in extinction of conditioned fear. Biological Psychiatry, 60, 337–343. Quirk, G. J., Russo, G. K., Barron, J. L., & Lebron, K. (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. Journal of Neuroscience, 20, 6225–6231. Rauch, S. A. M., Foa, E. B., Furr, J. M., & Filip, J. C. (2004). Imagery vividness and perceived anxious arousal in prolonged exposure treatment for PTSD. Journal of Traumatic Stress, 17, 461–465. Rauch, S. L., Shin, L. M., & Phelps, E. A. (2006). Neurocircuitry models of posttraumatic stress disorder and extinction: Human neuroimaging research—past, present, and future. Biological Psychiatry, 60, 376–382. Rauch, S. L., Whalen, P. J., Shin, L. M., McInerney, S. C., Macklin, M. L., Lasko, N. B.,. . . Pitman, R. K. (2000). Exaggerated amygdala response to masked facial stimuli in posttraumatic stress disorder: A functional MRI study. Biological Psychiatry, 47, 769–776. Rescorla, R. A. (2000). Extinction can be enhanced by a concurrent excitor. Journal of Experimental Psychology: Animal Behavior Processes, 26, 251–260. Rescorla, R. A. (2006). Deepened extinction from compound stimulus presentation. Journal of Experimental Psychology: Animal Behavior Processes, 32, 135–144. Rescorla, R. A., & Durlach, P. J. (1987). The role of context in intertrial interval effects in autoshaping. Quarterly Journal of Experimental Psychology, 39B, 35–40. Rescorla, R. A., & Heth, C. D. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 1, 88–96. Resnick, H. S., Kilpatrick, D. G., Dansky, B. S., Saunders, B. E., & Best, C. L. (1993). Prevalence of civilian trauma and posttraumatic stress disorder in a representative national sample

02-Schachtman-02.indd 42

of women. Journal of Consulting and Clinical Psychology, 61, 984–991. Ressler, K. J., Rothbaum, B. O., Tannenbaum, L., Anderson, P., Graap, K., Zimand, E.,. . .Davis, M. (2004). Cognitive enhancers as adjuncts to psychotherapy: Use of D-cycloserine in phobic individuals to facilitate extinction of fear. Archives of General Psychiatry, 61, 1136–1144. Riley, W. T., McCormick, M. G. F., Simon, E. M., Stack, K., Pushkin, Y., Overstreet, M. M.,. . . Magakian, C. (1995). Effects of alprazolam dose on the induction and habituation processes during behavioral panic induction treatment. Journal of Anxiety Disorders, 9, 217–227. Rothbaum, B. O. (2009). Using virtual reality to help our patients in the real world. Depression and Anxiety, 26, 209–211. Rothbaum, B. O., & Davis, M. (2003). Applying learning principles to the treatment of posttrauma reactions. Annals of the New York Academy of Sciences, 1008, 112–121. Rothbaum, B. O., Foa, E. B., Riggs, D. S., Murdock, T., & Walsh, W. (1992). A prospective examination of post-traumatic stress disorder in rape victims. Journal of Traumatic Stress, 5, 455–475. Sergerie, K., Chochol, C., & Armony, J. L. (2008). The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies. Neuroscience and Biobehavioral Reviews, 32, 811–830. Shin, L. M., Orr, S. P., Carson, M. A., Rauch, S. L., Macklin, M. L., Lasko, N. B.,. . .Pitman, R. K. (2004). Regional cerebral blood ﬂow in the amygdala and medial prefrontal cortex during traumatic imagery in male and female Vietnam veterans with PTSD. Archives of General Psychiatry, 61, 168–176. Sotres-Bayon, F., Bush, D. E., & LeDoux, J. E. (2007). Acquisition of fear extinction requires activation of NR2B-containing NMDA receptors in the lateral amygdala. Neuropsychopharmacology, 32, 1929–1940. Southwick, S. M., Bremner, J. D., Rasmusson, A., Morgan, C. A., Arnsten, A., & Charney, D. S. (1999). Role of norepinephrine in the pathophysiology and treatment of posttraumatic stress disorder. Biological Psychiatry, 46, 1192–1204. Stampﬂ, T. G., & Levis, D. J. (1967). Essentials of implosive therapy: A learning-theory-based psychodynamic behavioral therapy. Journal of Abnormal Psychology, 72, 496–503.

2/15/2011 7:42:11 PM

FEAR EXTINCTION AND EMOTIONAL PROCESSING THEORY

Van Minnen, A., & Foa, E. B. (2006). The effect of imaginal exposure length on outcome of treatment for PTSD. Journal of Traumatic Stress, 19, 427–438. Vansteenwegen, D., Francken, G., Vervliet, B., De Clercq, A., & Eelen, P. (2006). Resistance to extinction in evaluative conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 71–79. Walker, D. L., Ressler, K. J., Lu, K-T., & Davis, M. (2002). Facilitation of conditioned fear extinction

02-Schachtman-02.indd 43

43

by systemic administration or intra-amygdala infusions of D-cycloserine as assessed with fearpotentiated startle in rats. Journal of Neuroscience, 22, 2343–2351. Zysset, S., Huber, O., Ferstl, E., & von Cramnon, D. Y. (2002). The anterior frontomedian cortex and evaluative judgment: An fMRI study. Neuroimage, 15, 983–991.

2/15/2011 7:42:11 PM

CHAPTER 3 Fear Conditioning and Attention to Threat An Integrative Approach to Understanding the Etiology of Anxiety Disorders Katherine Oehlberg and Susan Mineka

In this chapter, we review the emerging literatures on conditioning of fear and anxiety and attentional biases for threat. First, we identify constructs common to both phenomena and suggest reasons for an approach that integrates both processes in the study of human anxiety disorders. We also review the current literature on individual differences in fear and anxiety conditioning. Next, we provide a current account of research on attentional biases for threat, which has focused on differentiating the components of attention affected by the threat relevance of stimuli and individual differences in anxiety. Finally, we review the growing number of studies that have combined conditioning and attention paradigms to investigate the relationships between these two phenomena. We conclude by indicating how these approaches to the study of conditioning and attentional processing in human anxiety might be integrated in the future.

RELEVANCE OF CONTEMPORARY LEARNING THEORY AND ATTENTIONAL PROCESSES IN MODELS OF ANXIETY DISORDERS Early theories of associative learning formed the basis for both research on fundamental conditioning processes as well as behavioral models for the acquisition and treatment of several human anxiety disorders (see Mineka & Zinbarg, 1996, 2006, for reviews). However, subsequent research has shifted the focus to cognitive and neurobiological variables that also play a role in both speciﬁc phobias and more complex disorders, such as generalized anxiety disorder, panic disorder, and posttraumatic stress disorder (see Chapter 6; also see, e.g., Bouton, Mineka, & Barlow, 2001; Mathews & MacLeod, 2005; Mineka & Zinbarg, 1996, 2006). Although modern research on conditioning processes continues to have theoretical importance and clinical relevance, we believe

that the continued vitality of this tradition (particularly in psychopathology) must consist of integrating conditioning theories with contemporary research on human cognition and its neurobiological underpinnings, with particular attention to individual differences (Mineka & Oehlberg, 2008). In this chapter, we explore the potential that this integrative approach may have by bringing current research ﬁndings on anxiety and fear learning to bear on one important cognitive phenomenon associated with anxiety, namely, attentional biases for threat. A great deal of research has indicated that our attentional systems are highly attuned for the rapid identiﬁcation of environmental threat (e.g., Esteves, Dimberg, & Öhman, 1994). Moreover, individual differences in attentional processing of threatening stimuli—typically referred to as attentional biases for threat-relevant stimuli— have been associated with both trait anxiety, a vulnerability factor for anxiety disorders, as well

44

03-Schachtman-03.indd 44

2/15/2011 7:42:31 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

as a range of clinical anxiety disorders such as speciﬁc phobias, generalized anxiety disorder (GAD), posttraumatic stress disorder (PTSD), and social phobias (see Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & van Ijzendoorn, 2007; Mathews & MacLeod, 2005, for reviews). Such biases are typically assessed using a behavioral response-time paradigm, in which a participant demonstrates selective attention toward a threatening stimulus (e.g., a threatening word, “disaster,” or an angry face) rather than a neutral stimulus (e.g., “vacuum,” or a neutral face). Several decades of research have now established that these biases appear to be multiply determined and may not be the exclusive purview of highly anxious individuals. Rather, there is evidence that biased attentional processing of threat stimuli may depend upon the intensity of the threat stimuli (Koster, Verschuere, Crombez, & Van Damme, 2005; Mogg et al., 2000; Wilson & MacLeod, 2003). Moreover, it appears that individuals low in trait anxiety tend to direct attention away from mild to moderate threat, but like highly traitanxious individuals direct attention toward highly threatening stimuli (see Bar-Haim et al., 2007; Frewen, Dozois, Joanisse, & Neufeld, 2008, for meta-analyses). A recent computational model of attentional biases for threat versus reward (Frewen et al., 2008), discussed later in this chapter, incorporates this ﬁnding as an essential feature of the anxiety-attentional bias link. Classical conditioning is widely considered to be a fundamental mechanism by which initially neutral environmental stimuli acquire the capacity to elicit fear and anxiety, and there is a long tradition of research relating classical conditioning of fear to the anxiety disorders (see Mineka, 1985; Mineka & Zinbarg, 1996, 2006 for reviews). Indeed, conditioning research has formed the basis of behavioral therapies for anxiety disorders, with successful treatment models being based upon extinction processes facilitated by gradual or massed exposures to feared stimuli. Such treatments, particularly with the speciﬁc phobias and panic disorder, represent some of the greatest successes of modern clinical psychology (Craske & Mystkowski, 2006).

03-Schachtman-03.indd 45

45

The degree to which a stimulus is perceived as threatening, as well as the subjective experience of anxiety or fear, represent areas of construct overlap between the domains of attentional threat biases and aversive conditioning. It is plausible, for instance, that the perceived threat value of a stimulus might be a direct consequence of basic associative learning processes. Given the signiﬁcance of perceived threat value in attentional biases, it makes sense to consider such processes in models of the development of attentional biases. Moreover, attention is an important moderator— if not mediator—of conditioning, and has been explicitly included in both traditional (e.g., Pearce & Hall, 1980) and current computational models of conditioning processes (Schmajuk, Lam, & Gray, 1996; Schmajuk & Larrauri, 2006). In addition, both attentional biases and learned classical associations are subject to modiﬁcation, so their study can elucidate mechanisms by which diathesis-stress interactions may occur. For instance, it has been found that modiﬁcation of attentional biases has little to no immediate effect on state anxiety, but that it has an effect on the amount of distress that an individual experiences when faced with a stressful event (see MacLeod, Rutherford, Campbell, Ebsworthy, & Holker, 2002; MacLeod, & Bridle, 2009; but also Amir, Beard, Burns, & Bomyea, 2009). The goal of this chapter is to review the emerging literatures in attentional biases for threat and conditioning of fear and anxiety, to identify constructs common to both phenomena (e.g., the stimulus threat value), and to suggest ways in which these approaches to the study of information processing in human anxiety might be integrated in the future. First, we distinguish between states of anxiety and fear, and brieﬂy review recent ﬁndings on the neural systems involved in the processing of threatening stimuli and their genetic correlates. We also discuss current directions in fear and anxiety learning that have particular relevance to human anxiety disorders. Second, we discuss ﬁndings and ongoing issues in attentional bias research. Third, we critically examine the small but growing literature directly relating attention to threat and conditioning processes. Finally, we suggest some directions for future research.

2/15/2011 7:42:31 PM

46

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

CLASSICAL CONDITIONING IN THE ANXIETY DISORDERS Starting with Watson and Rayner (1920), Pavlovian or classical conditioning has long been implicated in the origins of fears and phobias (e.g., see also Wolpe, 1968; Mowrer, 1939). Although early conditioning models assumed that traumatic conditioning experiences were both necessary and sufﬁcient for the development of phobic fears and other anxiety disorders, subsequent theorizing has expanded considerably upon this restrictive assumption. As discussed at length elsewhere (Mineka & Zinbarg, 1996; 2006), several important criticisms of these earlier simplistic conditioning views emerged and required careful analysis. Particularly as researchers focused on additional anxiety disorders (such as social phobia, panic disorder, and posttraumatic stress disorder), the importance of other forms of associative learning in the etiology of these disorders became evident (e.g., see Mineka, 1985; Mineka & Zinbarg, 1996). In this section we review highlights of the primary themes of contemporary research and thought on learning theory perspectives on several anxiety disorders. Distinctions Between Fear and Anxiety: Ethological, Clinical, and Neurobiological Evidence

Fear and anxiety, both common in the anxiety disorders, represent partially overlapping but also distinct emotional states. Traditionally, they were distinguished by whether there exists a clear and obvious source of danger that would be regarded as real by most people. Fear was thought to be experienced when the source of danger is obvious, and anxiety was experienced when one could not specify clearly what the danger was. In the past 20–25 years, however, many prominent researchers and theorists have proposed more fundamental distinctions between fear versus anxiety based on a strong and growing body of ethological, clinical, and neurobiological evidence (e.g., Barlow, 2002; Gray & McNaughton, 2000; Grillon, 2008). Although the theories vary in their speciﬁcs, there is general agreement on there being at least two negative threat-related

03-Schachtman-03.indd 46

emotional states. For example, Barlow proposed that fear or panic is a basic emotion involving activation of the “ﬁght-or-ﬂight” response of the sympathetic nervous system. By contrast, he characterized anxiety as a relatively diffuse, future oriented state consisting of a complex blend of negative emotions and cognitions. It involves a negative affective state, worry or anxious apprehension about possible future threat or danger, and a sense of being unable to predict or control the future threat. Physiologically anxiety often creates a state of tension, chronic overarousal, and vigilance for future threat, but no activation of the ﬁght-or-ﬂight response (Barlow, 1988, 2002). In this section, we brieﬂy outline several of the ways in which fear and anxiety have been distinguished. Ethological Evidence

From an evolutionary perspective, defensive systems must enable organisms to efﬁciently and adequately handle a variety of survival threats, including predators. A defensive fear system should motivate the animal to escape or avoid sources of imminent danger with very fast activation of defensive behaviors, but it must also allow an animal to deal with less explicit or more generalized threat cues with less effort or energy. For example, according to one prominent theory the defensive fear system in animals is organized such that different behaviors emerge depending on the animal’s psychological (and/or physical) distance from a “predator”—known as “predatory imminence” (Fanselow & Lester, 1988). That is, each type of defensive behavior has been selected through evolution to prevent movement to the next higher point on the imminence scale. If no threat exists, the animal goes about its usual daily activities. However, if predatory imminence begins to increase, even very slightly, behaviors optimal to the new situation are activated. For instance, when the predatory imminence is low but non-zero for rats, the animal’s eating patterns may change such that the animal shows heightened vigilance, and approach becomes more cautious, likely corresponding to the human affective states of anxiety and worry (Quinn & Fanselow, 2006). If a danger (like a predator) is actually detected but still at some

2/15/2011 7:42:31 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

distance, the animal engages in freezing behavior while its heart rate slows and respiration becomes shallower; this state seems to correspond to the human emotion of fear. As the threat proximity increases even further and attack is imminent, the animal may jump, ﬂee, or return the attack; this may correspond to human panic attacks. Thus, the speciﬁc physiological responses and overt behaviors engaged in at different levels of threat imminence differ not just quantitatively but also qualitatively. In this context, then, anxiety behaviors are prompted by less explicit or more generalized cues, and they involve physiological arousal and increased vigilance but often without organized functional behavior. By contrast, fear behaviors are hypothesized to be optimized for moderately imminent threat, and panic behaviors for immediately imminent threat. Clinical Evidence

Phenomenological evidence as well as psychometric analyses of clinical symptoms also indicate two relatively independent clusters of symptoms—fear and/or panic symptoms on the one hand, and a more general state of worry or anxious apprehension on the other hand (e.g., Barlow, 2002; Bouton et al., 2001; Brown, Chorpita, & Barlow, 1998). For example, structural equation modeling and factor analyses have uncovered two different factors when examining symptoms of panic, anxiety, and depression in clinical populations. One is exempliﬁed by the kinds of apprehension and worry that are characteristic of anxiety and the other is exempliﬁed by a sense of extreme fear or terror, strong autonomic arousal, and ﬁght-or-ﬂight action tendencies that are characteristic of panic or fear. The interrelationships between these two clusters of symptoms are surprisingly complex, and still a topic of active study. For instance, the autonomic arousal symptoms of panic seem to be inhibited in individuals with GAD, suggesting that worry functions to suppress heightened autonomic reactivity in this disorder (e.g., Borkovec, Alcaine, & Behar, 2004; Brown et al., 1998). However, there is also evidence from studies on panic-disordered individuals that panic attacks are actually potentiated when such an individual has a high level of anxiety

03-Schachtman-03.indd 47

47

(see Bouton et al., 2001, for a review). Thus, the effect of elevated anxiety on the occurance of panic appears to vary according to perameters not yet well speciﬁed. It is not yet clear how to reconcile such apparently paradoxical ﬁndings among the anxiety disorders. Neuroanatomical Evidence

Behavioral neuroscience research in animals also supports the hypothesis of two distinct aversive motivational systems involved in conditioning, although researchers do not agree on all the details. For example, according to Davis and colleagues (e.g., Davis & Shi, 1999; Davis, Walker, & Lee, 1997; Grillon, 2008), fear is a short-term state activated by discrete Pavlovian conditioned stimuli (CSs; see Table 3.1 for a summary of conditioning terminology), whereas anxiety is a longer term state that is activated by more diffuse cues that can be either unconditioned (such as darkness for humans) or conditioned (as in contextual cues associated with threat). These states appear to be related to two distinct neural systems, namely, the amygdala and the bed nucleus of the stria terminalis (BNST), which is immediately downstream from the basolateral amygdala. Although the amygdala mediates fear responses to explicit threatening stimuli, both the amygdala and the BNST are associated with more long-lasting aversive states not clearly linked to explicit threat cues. Both conditioned and unconditioned fear and anxiety states have been modeled in research on animals, and in many cases these experimental paradigms have been extended to research in humans. One of the principal experimental models today is the fear-potentiated startle effect (Davis, 1998, 2006), which is used in both animals and humans. This effect occurs when a greater magnitude startle reﬂex is elicited by a loud noise occurring 3–4 seconds after an independently established light CS+ (excitatory conditioned stimulus) has been presented, compared to when the light is a novel stimulus. One model of unconditioned anxiety is the light-enhanced startle effect, which occurs when a bright light (a mild unconditioned stimulus, or US, for anxiety in rats that prefer the dark) has been turned on 5–20 minutes prior to a loud noise; this also

2/15/2011 7:42:31 PM

48 Table 3.1

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS Terminology

Construct

Deﬁnition

Current Abbreviation

Alternative Abbreviations

Unconditioned stimulus

A biologically signiﬁcant stimulus or event; in this chapter, the Os of interest are drug related and include both direct effects of drug and the symptoms and signs of drug withdrawal

O

S, S∗, US

Conditioned stimulus

A previously neutral stimulus that obtains signiﬁcance through association with an unconditioned stimulus

S

S, CS

Conditioned response

A response elicited by a conditioned stimulus associated with a biologically signiﬁcant stimulus

R

R, CR

Pavlovian/classical conditioning

Learning in which individuals form associations between conditioned stimuli and unconditioned stimuli

S-O

S-S, S-S∗, CS-US

Instrumental/ operant conditioning

Learning in which individuals form associations between behavioral responses and outcomes

R-O

R-S

Habit learning

Learning in which stimuli come to evoke responses automatically even if the reinforcer is devalued

S-R

Incentive learning

Learning in which the value of a reinforcer is associated with certain stimuli or contexts, thus modulating the incentive value of Ss associated with the reinforcer

Occasion setter

A stimulus that signals whether a learned S-O association will hold in Pavlovian conditioning

Positive

Signals that O will follow S

S+

Negative

Signals that O will not follow S

S–

Discriminative stimulus

A stimulus that signals that a response will be reinforced in instrumental conditioning

Positive feature

A discriminative stimulus that indicates that responding will be reinforced

SD

S+

Negative feature

A discriminative stimulus that indicates that responding will not be reinforced

S∆

S–

Conditioned inhibitor

A conditioned stimulus that is associated with the absence of a biologically important stimulus. Unlike occasion setters and discriminative stimuli that are not related directly to the presence or absence of the unconditioned stimulus (O), conditioned inhibitors are directly associated with the absence of the O

S-

CS–

03-Schachtman-03.indd 48

2/15/2011 7:42:31 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

results in increased magnitude of the startle reﬂex in rats (Walker & Davis, 1997). It is referred to as an unconditioned anxiety effect, because it does not extinguish as a source of increased startle either within or across multiple test sessions (Walker & Davis, 1997). In humans, a similar increase in startle amplitude can be induced by exposure to the dark (Grillon, Pellowski, Merikangas, & Davis, 1997). What is common to the fear and anxiety paradigms is the dependent variable (an enhanced startle response), but the paradigms reﬂect unconditioned anxiety when the longer duration light stimulus is used, and conditioned fear when the brief presentation of a light CS+ is used in fearpotentiated startle paradigms. Lesions of the basolateral amygdala substantially reduce or abolish behavioral and autonomic responses to conditioned fear stimuli in a fear-potentiated startle task (e.g., LeDoux, 1995). However, such lesions do not reduce a rat’s preference for covered arms in a plus maze, another commonly used measure of unconditioned anxiety in rat research (Treit, Pesold, & Rotzinger, 1993). In addition, lesions of the central nucleus of the amygdala also do not reduce the lightenhanced startle reﬂex (Davis et al., 1997). Conversely, lesions of the BNST have no effect on the fear-potentiated startle effect but do signiﬁcantly reduce light-enhanced startle reﬂexes (see Grillon, 2008, for a review). Anxiety can also be a conditioned response evoked by long-duration CSs. For example, Waddell, Morris, and Bouton (2006) compared the effects of BNST lesions on aversive conditioning with short (e.g., 60 sec CS) or long (e.g., 600 sec CS) signals for shock. Prior to conditioning, some of the animals had received lesions of the BNST, while others received sham lesions. The BNST lesions had no effect on conditioning of fear with the short-duration CS, but for the long-duration CS, the BNST lesion signiﬁcantly reduced conditioning of anxiety. Therefore, it appears that the BNST is not only involved in the expression of unconditioned anxiety as seen with the light-induced startle effects, but it is also involved in the conditioning of anxious apprehension (as with the 600 sec CS). Finally, another model of conditioned anxiety, to be discussed

03-Schachtman-03.indd 49

49

later, occurs when contextual cues are associated with unpredictable threat. Fear and Anxiety Conditioning Abnormalities in Anxiety Disorders

If fear and anxiety conditioning processes account in part for the development of various anxiety disorders, then individual differences in personality or past experiences may serve as diatheses and these should be detectable using classical fear or anxiety conditioning paradigms measuring the degree and speed of fear acquisition and extinction. The possibility of individual differences in acquisition (ACQ) and extinction (EXT) has been explored in both psychophysiological experiments comparing anxious and nonanxious individuals, and it has begun to be investigated in behavioral and molecular genetic studies. Ideally, a deeper understanding of these issues would greatly beneﬁt from longitudinal prospective studies in which individual differences in conditionability of fear or anxiety could be used to predict which individuals would subsequently develop anxiety disorders following relevant learning experiences. Unfortunately, we are not aware of any such studies to date and so we will review highlights of the studies that do exist on individuals with several kinds of anxiety disorders. Conditioning Paradigms Used to Study Individual Differences

Two basic experimental designs have been employed in studies comparing conditioning in anxious and nonanxious groups (e.g., Lissek et al., 2005). In the ﬁrst (simple conditioning), participants in both groups are ﬁrst exposed to several CS-only trials (habituation) and are then subjected to repeated pairings of a neutral stimulus (CS) with an aversive stimulus (US). The development of a conditioned response (CR) is measured following the conditioning trials (or on nonreinforced test trials during acquisition) by measuring the CR to the CS presented alone and comparing it with responding during habituation; between-group differences are also examined. However, this simple conditioning paradigm is not ideal for measuring the amount of associative learning that has occurred because there

2/15/2011 7:42:31 PM

50

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

is no control for possible sensitization (nonassociative) effects. In the second type of paradigm (discriminative conditioning), two CSs are used, one paired with the aversive US (the CS+) and one explicitly unpaired with the aversive US (the CS–). By using within-subject comparisons of responding to the CS+ and CS–, one can infer whether the two groups differ in developing discriminative responding to the CS+ and CS–. Such paradigms are superior to simple conditioning paradigms because they better control for possible sensitization effects that should accrue as much to a CS– as to a CS+. Discriminative conditioning paradigms also allow for at least indirect comparisons of inhibitory as well as excitatory conditioning. There are several possible ways in which individual differences in conditioning as a function of anxiety might operate. With simple conditioning paradigms, anxious individuals, relative to nonanxious individuals, might either acquire CRs more rapidly or acquire greater magnitude CRs (to either discrete CSs paired with USs, or to contexts paired with USs). Because it is known that stronger associations extinguish more slowly (e.g., Annau & Kamin, 1961), another possibility is that individual differences might also emerge in rates of extinction. Indeed, EXT might be retarded by the presence of anxiety regardless of whether ACQ responding has been affected. With discriminative conditioning paradigms the possibilities are more complex. One theory proposes that greater differences in discriminative responding to CS+s and CS–s should emerge because of superior conditioning to the CS+ in anxious individuals (e.g., Orr et al., 2000). These differences may also be manifested in slower rates of EXT. However, before considering this issue, we ﬁrst describe several different theories reviewed by Lissek et al. (2005) that were developed to provide an explanation for differences in fear conditioning in anxious versus nonanxious individuals that are also relevant to predictions about potential group differences in discriminative conditioning. One theory, advanced by Orr and colleagues (Orr et al., 2000), speciﬁcally proposed that individual differences in conditionability to fear

03-Schachtman-03.indd 50

stimuli may help account for individual differences in anxiety. This theory predicted that anxious individuals would display stronger or more rapid acquisition of fear CRs than nonanxious individuals and therefore slower extinction. For example, GAD patients, compared with nonanxious individuals, showed similar ACQ but reduced EXT of skin conductance CRs to angry faces (Pitman & Orr, 1986). According to this theory anxious individuals, relative to nonanxious individuals, should also show a greater difference in discriminative responding to CS+s and CS–s. For example, in a study of trauma-exposed individuals, Orr et al. (2000) found that those who had developed PTSD showed greater discriminative responding during ACQ and EXT relative to those traumaexposed individuals who had not developed PTSD. Such results are consistent with the possibility that differences in diatheses for development of PTSD may be partly a function of differences in conditionability to fear cues, although a prospective design would be required to conclude this with certainty. By contrast, another theory proposed by Davis and colleagues (Davis, Falls, & Gewirtz, 2000) speciﬁes differences in inhibitory, rather than excitatory, conditioning as most relevant to the emergence of anxiety disorders. Speciﬁcally, they proposed that it is a failure to inhibit the fear CR in the presence of safety signals (for example, a CS–, which signals the absence of the aversive US) that is one mechanism by which clinical anxiety may develop. Consistent with this proposal, some studies show that anxiety patients (relative to controls) exhibit greater fear-potentiated startle during the CS- (Grillon & Ameli, 2001; Grillon & Morgan, 1999). In other studies they showed higher magnitude electrodermal responses during the CS– (e.g., Orr et al., 2000) or greater subjective anticipatory anxiety during a CS– (e.g., Hermann, Ziegler, Birbaumer, & Flor, 2002). Thus, results using all three kinds of fear measures converge in suggesting that the anxiety patients showed smaller magnitude inhibitory CRs to a CS– than controls. However, none of these studies provide deﬁnitive direct support for this theory about differences in inhibitory learning as a function of

2/15/2011 7:42:31 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

anxiety status, which would require direct assessment of the inhibitory power of the CS– in the two groups. Empirical Status of These Theories

Lissek et al. (2005) reviewed and performed a meta-analysis on 20 studies comparing fear conditioning in participants with (n = 453) and without (n = 455) an anxiety disorder (including panic disorder, PTSD, and GAD) in order to assess the empirical status of these theories. It was found that anxious individuals had stronger fear CRs during ACQ and EXT compared to healthy controls, although both effect sizes were small. Interestingly, these effects were found primarily in studies using a simple conditioning procedure; patients and controls showed similar rates of ACQ and EXT when discriminative conditioning procedures were used. Similar discriminative conditioning in patients and controls suggests that, in addition to stronger excitatory conditioning, anxious individuals may be less able than nonanxious individuals to suppress a fear CR in the presence of safety cues (CS–s). In fact, a number of the reviewed studies using discriminative conditioning paradigms (e.g., Grillon & Morgan, 1999; Peri, Ben-Shakhar, Orr, & Shalev, 2000) found elevated CRs in anxious individuals to both CS+ and CS– stimuli. These ﬁndings are not consistent with the Orr et al. (2000) hypothesis (or the Orr et al. results with PTSD), but they are consistent with the Davis et al. (2000) hypothesis that anxious individuals are less able to inhibit fear responding in the presence of safety cues. Again, however, such a conclusion would require an independent direct assessment of the inhibitory power of the CS– in the two groups. Researchers have recently begun addressing this question using methodology speciﬁcally designed to experimentally distinguish excitatory from inhibitory conditioning. For example, Myers and Davis (2004) adapted a discriminative conditioning paradigm called conditioned discrimination (Wagner & Rescorla, 1972), to explicitly test the inhibitory power of a CS– in rats. In this paradigm (abbreviated AX+/BX–), the excitatory A and inhibitory B stimuli are conditioned independently of one another, in

03-Schachtman-03.indd 51

51

compound with a third stimulus X. After AX+ and BX– trials, A was excitatory, and B inhibitory. Winslow, Noble, and Davis (2008) found parallel inhibitory effects when a very similar paradigm was used in rhesus monkeys. Moreover, one study on individuals with no history of any diagnosable psychological disorder also successfully demonstrated the inhibitory effects of B using such a paradigm (Jovanovic et al., 2005). An even more recent study (Jovanovic et al., 2009) used this paradigm to examine fear inhibition in patients with fairly severe PTSD and found that they failed to show any signiﬁcant fear inhibition (compared to those with mild PTSD or controls) on a compound trial. However, they were aware that a shock would not occur (i.e., they knew they were safe but they could not actually inhibit the response). It will be interesting to see whether other anxiety disorders are also characterized by failure to learn fear inhibition. In addition to the question of whether vulnerability to anxiety disorders reﬂects individual differences in either excitatory or inhibitory conditioning processes (or both), the question also remains as to whether individual differences in discrete-cue versus context conditioning, and fear versus anxiety conditioning, are relevant to the development of particular anxiety disorders. As discussed later, fear learning appears to be neurobiologically and behaviorally distinguishable from context conditioning (more akin to anxiety), which is the learning of associations between a US and its broader environmental context, rather than discrete cues upon which a US is contingent. If these two processes vary independently, individual differences in either or both may confer risk for speciﬁc anxiety disorders. Correlates of Individual Differences in Fear Conditioning: Personality, Neuroimaging, and Genes

If individual differences in either excitatory or inhibitory fear conditioning may act as diatheses for the development of anxiety disorders, it is sensible to ask what relationship they may have to other diatheses for anxiety. Personality factors such as neuroticism or high negative affectivity

2/15/2011 7:42:32 PM

52

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

have long been known to predict the development of a range of both anxiety and depressive disorders (e.g., Clark, Watson, & Mineka, 1994; Hayward, Killen, Kraemer, & Taylor, 1998, 2000). In addition, in the past 10 years there has been an explosion of ﬁndings using functional neuroimaging techniques that have uncovered individual differences in responsiveness to threat and punishment cues, and progress is being made toward associating differential neural responsivity in critical brain regions with individual differences in trait anxiety and other personality factors (see Hariri, 2009, for a review, but also Vul, Harris, Winkielman, & Pashler, 2009, for a methodological critique of this literature). Moreover, very exciting (but increasingly contested) associations have been found between speciﬁc genetic polymorphisms and personality traits, as well as clinical anxiety and mood disorders. In this section, we review research that relates individual differences in fear conditioning with personality and neural activity, as well as discussing emerging research investigating the heritability of fear conditioning and the association of fear conditioning with speciﬁc genetic polymorphisms. To prepare readers for this discussion, we begin by brieﬂy summarizing the current research on neuroimaging and genetics associated with threat responsiveness and anxiety. We focus in particular on polymorphisms in two particular genes, the serotonin transporter (5-HTT) and catechol-O-methyltransferase (COMT), which have received considerable attention in recent years for their putative relationships with trait anxiety, fear conditioning, and attentional processing of threat. Genetic and Neuroimaging Variation Associated with Anxiety: Serotonin Transporter Polymorphisms, Anxiety, and Amygdala Activation

In recent years, there has been tremendous excitement over the discovery of a possible relationship between anxious/depressive pathology and allelic differences in a variable repeat sequence of the promoter region of the serotonin transporter gene (5-HTT), particularly in relation to stressful life events. This gene encodes for a protein that regulates the reuptake of serotonin

03-Schachtman-03.indd 52

from the synaptic cleft, in turn affecting the synaptic concentrations of serotonin. Research has shown that having one or two short 5-HTT alleles, rather than two long alleles, is associated with as much as 50% less serotonin transporter availability, functionally increasing the synaptic concentration of serotonin (Heinz et al., 2000; Lesch et al., 1996). Initial work suggested that carriers of at least one short allele have higher levels of trait anxiety and neuroticism (Munafo, Clark, & Flint, 2004; Schinka, Busch, & Robichaux-Keene, 2004; Sen, Burmeister, & Ghosh, 2004), and increased risk for major depressive disorder after exposure to stressful life events (Caspi et al., 2003; Eley et al., 2004; Kaufman et al., 2004). With respect to the neuroanatomical correlates of these polymorphisms, healthy carriers of at least one short allele of the 5-HTT polymorphism, in comparison to those homozygous for the long allele, demonstrate greater limbic responsiveness to angry or fearful expressions (Bertolino et al., 2005; Hariri et al., 2002; Pezawas et al., 2005). Evidence also suggests an association between the presence of a short allele and stronger activation of the amygdala, together with greater coupling between the amygdala and ventromedial prefrontal cortex (vmPFC), to aversive images (Canli et al., 2005; Heinz et al., 2004). However, it is noteworthy that the analytic methodologies used in research relating neuroimaging results with self-reported mood and personality are presently under intense scrutiny (see Vul, Harris, Winkielman, & Pashler, 2009, for a pointed critique), so it is prudent to interpret such ﬁndings with caution. Enthusiasm over such ﬁndings is also currently waning in the light of several recent metaanalyses that call into question the relationship of the 5-HTT polymorphisms, particularly in interaction with life stress, with psychopathology and trait anxiety (Munafo, Durrant, Lewis, & Flint, 2009; Risch et al., 2009). However, as Monroe and Reid have argued (2008), the overwhelming majority of research on such gene– environment interactions has relied upon highly inconsistent measures of life stress across different studies, severely limiting the implications of meta-analyses on the literature to date. As this

2/15/2011 7:42:32 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

issue receives more attention, and the considerable research on life stress conceptualization and measurement is brought to bear on behavioral genetics research (Monroe, 2008), greater clarity will hopefully be brought to the role of this promising gene candidate. On a more positive note, there appears to be greater consistency in studies of the link between serotonin transporter genotype and amygdala activation (Munafo, Brown, & Hariri, 2008), although the authors of this meta-analysis and review warn that most studies to date are lacking in statistical power, rendering estimates of effect sizes premature. Genetic and Neuroimaging Variation Associated With Anxiety: COMT Polymorphisms, Anxiety, and Prefrontal Functioning

Another promising candidate genetic variant in anxiety is the catechol-O-methyltransferase gene (COMT), which encodes for an enzyme that degrades dopamine, epinephrine, and norephinephrine in the prefrontal cortex (Gogos et al., 1998; Tunbridege, Bannerman, Sharp, & Harrison, 2004), and hippocampus (Matsumoto et al., 2003). Variations in its functional polymorphism, COMT Val158Met, inﬂuence the levels of dopamine in these areas, with the Met allele associated with a third of the enzymatic activity of Val the allele in breaking down dopamine. Functionally, this results in the Met allele producing higher levels of synaptic dopamine in these brain areas. Although many contradictory ﬁndings have been published, preliminary evidence suggests that the Val158 allele is associated with inefﬁciency in cognitive control, whereas the Met allele has been related to anxiety related traits and disorders (Domschke et al., 2004; Enoch, Xu, Ferro, Harris, & Goldman., 2003; Stein, Fallin, Schork, & Gelernter, 2005). Personality and Temperament in Fear/ Anxiety Conditioning

As mentioned earlier, several stable temperamental and personality factors that are partially heritable have also been identiﬁed as diatheses for various anxiety disorders. One of these is the personality trait known as neuroticism or negative affectivity (Clark et al., 1994; Watson,

03-Schachtman-03.indd 53

53

Gamez, & Simms, 2005). In addition, trait anxiety—one of several facets of neuroticism— is also thought to be a vulnerability factor. Moreover, over the years numerous studies have shown that individuals high in trait anxiety show more rapid and stronger aversive conditioning (e.g., see Levy & Martin, 1981, for an early review; Lissek et al., 2005; Zinbarg & Mohlman, 1998). It is possible that such conditioning processes could be a mechanism through which high trait anxiety operates as a vulnerability factor for clinical anxiety disorders. In this regard it is also interesting to speculate that the partial overlap in heritability of anxiety disorders, such as speciﬁc phobias and panic disorder (e.g., Kendler et al., 1995), could be mediated by heritable differences in conditionability (see Bouton et al., 2001). Moreover, research on temperament in children indicates that young children with high levels of behavioral inhibition (a tendency to be shy, avoidant, and easily distressed by unfamiliar stimuli) in early childhood are at heightened risk for developing multiple speciﬁc phobias in childhood (Biederman et al., 1990), and social anxiety in adolescence when it is most likely to develop (e.g., Hayward et al., 1998; Schwartz, Snidman, & Kagan, 1999). Whether the effects of behavioral inhibition are mediated through differences in conditionability is not yet known, although this seems like a possibility worth investigating. Heritability and Genetic Findings in Fear Conditioning

There are several twin studies of other forms of learning, but we are only aware of one study that has speciﬁcally examined the heritability of fear conditioning. Hettema and colleagues (2003) investigated fear conditioning in 173 same-sex twin pairs (90 monozygotic and 83 dizygotic) using a standard discriminative conditioning procedure, in which pictorial stimuli (either fear-relevant [snakes and spiders] or fear-irrelevant [circles and triangles] stimuli) were paired with a mild electric shock US, measuring skin conductance (SCR) as the CR. Rates of habituation, acquisition, and extinction were all assessed in order to model the relative contributions of both associative and nonassociative processes.

2/15/2011 7:42:32 PM

54

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Moderate heritability was found for all three components, with additive genetic effects accounting for 34% to 43% of the total variance. Using structural equation modeling, they found that a two-factor model best explained the observed genetic variation in twin pairs. One factor accounted for variation in habituation, acquisition, and extinction (the nonassociative factor) and one factor accounted for variation in only acquisition and extinction (the associative factor). Interestingly, the investigators also found evidence that the heritability of associative fear learning using evolutionarily relevant stimuli, such as snakes and spiders, may be greater than that of fear-irrelevant stimuli, although statistical power was not sufﬁcient to draw this conclusion with certainty. A recent and exciting genetic study investigated the speciﬁc genetic polymorphisms 5-HTTLPR and COMT for their effects on conditioned fear acquisition and extinction (Lonsdorf et al., 2009). Forty-eight college students donated blood for DNA extraction and genotyping and then underwent a discriminative fear-conditioning paradigm using facial stimuli as CS+s and CS–s. There were nine CS+ trials, each ending with a 10 ms shock, and nine CS- trials ending with no shock. Startle probes were presented after 4–5 seconds on six of nine CS+ trials and on six of nine CS– trials; SCRs were also measured but the most important dependent variable was startle potentiation. On the following day there were 18 presentations of both the CS+ and the CS–, neither followed by a US, to assess extinction performance. As shown in Figure 3.1, results were striking for startle potentiation, but not SCRs, as the dependent measure. Carriers of one or two short alleles of the 5-HTT gene showed signiﬁcantly greater fear potentiation to the CS+ than did l/l homozygous carriers during acquisition, a pattern that persisted in extinction. By contrast, the two polymorphisms of the COMT Val158Met gene did not affect startle potentiation during acquisition. However, those with the homozygous met/met COMT Val158Met polymorphism showed much greater CS+ fear potentiation during extinction than those who were val-allele carriers. The participants who showed the most pronounced startle responding in extinction

03-Schachtman-03.indd 54

were those with the short allele(s) of the 5-HTT gene and the homozygous met allele of the COMT gene. These results suggest that polymorphisms in these two genes are differentially related to the ability to learn and maintain fear of aversive stimuli. More speciﬁcally, those with a short 5-HTT allele and two COMT met alleles are more likely to acquire fear of conditioned aversive stimuli, and less able to extinguish such learned responses. The additional ﬁnding that such results held only for the startle index of fear (and not for SCRs) is consistent with ﬁndings and ideas reviewed by Öhman and Mineka (2001) that there are two levels of fear learning: One is at a very basic emotional level and the other is at a cognitive level. Research shows that startle potentiation is a better index of a real emotional level of fear. SCRs, by contrast, are more an index of cognitive contingency learning. As Öhman and Mineka (2001) discuss, there are many examples in the literature of different parameters having selective effects on these two different indices of fear. Other Sources of Individual Differences in the Learning of Phobias and Other Anxiety Disorders Other Pathways to the Acquisition of Fear and Anxiety

So far we have only discussed traditional classical conditioning as a source of fear and anxiety. Yet not everyone developing phobias or other anxiety disorders seems to have undergone traumatic conditioning experiences. In many instances phobic fears instead may be acquired vicariously through simply watching another person (live or on a TV or movie screen) behaving fearfully with some object or situation. This is best illustrated by Mineka and Cook’s primate model of phobic fear acquisition from the 1980s (e.g., Cook & Mineka, 1991). They showed that laboratoryreared rhesus monkeys that were not initially afraid of snakes rapidly acquired an intense and long-lasting phobic-like fear of snakes after simply watching a wild-reared model monkey behaving fearfully on some trials when a toy or real snake was present and nonfearfully on other

2/15/2011 7:42:32 PM

FEAR CONDITIONING AND ATTENTION TO THREAT CS+

Startle blink magnitude (Difference from ITI; DT scores)

a

c

Conditioning: 5-HTTLPR 10

6 4

–2

6

*

–2

d 10

10

8

8

***

2

4

***

0

***

–2 s-allele carriers

10

**

long/long

val-allele carriers

Extinction: 5-HTTLPR s Carriers **

8 6

6

**

4

met/met

f Extinction: COMTval158met

Extinction: 5-HTTLPR

6

***

2

0 val-allele –2 carriers

met/met

*

4

***

0

b Startle blink magnitude (Difference from ITI; DT scores)

6

2 long/long

Conditioning: 5-HTTLPR s Carriers

8

4

s-allele carriers

e 10

*

0

Conditioning: COMTval158met

8

***

2

CS–

10

*

8

55

*

2

4 2

0

0

**

*

–2

–2 met/met

val-allele carriers

met/met

val-allele carriers

Figure 3.1 Potentiation of startle-response magnitudes as a function of genotype and stimulus. Black

bars show the difference between magnitude of the startle response elicited during the CS+ (the conditioned stimulus coupled to the unconditioned stimulus) and magnitude of the startle response elicited during the intertrial interval (ITI); white bars show the difference between magnitude of the startle response elicited during the CS− (the conditioned stimulus never coupled to the unconditioned stimulus) and magnitude of the response elicited during the ITI. Results are shown for 5-HTTLPR genotype groups (a) during conditioning and (b) during extinction, for COMTval158met genotype groups (c) during conditioning and (d) during extinction, and for COMTval158met genotype groups within 5-HTTLPR s-allele carriers (e) during conditioning and (f) during extinction. Error bars represent standard errors. Asterisks indicate signiﬁcant differences, ∗ p < 0.05, ∗∗ p < 0.01, ∗∗∗ p < 0.001. (From Lonsdorf, T. B., Weike, A. I., Nikamo, P., Schalling, M., Hamm, A. O., & Öhman, A. 2009. Genetic gating of human fear learning and extinction: Possible implications for gene-environment interaction in anxiety disorder. Psychological Science, 20(2), 198–206).

trials when a neutral object was present. Vicarious acquisition of aversive conditioning has also been demonstrated in humans using psychophysiological responses (such as electrodermal responses) as the dependent variable (Green & Osborne, 1985). Several studies have also supported the inﬂuence of parental modeling on increasing children’s fears for at least 1 week

03-Schachtman-03.indd 55

after the modeling experience (e.g., Askew & Field, 2007; Gerull & Rapee, 2002). Evidence is also accumulating of modeling of social anxiety in families of those who have developed social phobia (e.g., Bruch & Heimberg, 1994; Rapee & Melville, 1997). For example, both mothers and their socially phobic offspring reported more social avoidance in the parents

2/15/2011 7:42:33 PM

56

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

than seen in nonclinical control families. Moreover, parents of those who develop social phobia often seem to have discussed and thereby reinforced children’s avoidant tendencies in the context of threatening situations (e.g., Barrett, Rapee, Dadds, & Ryan, 1996). The Effects of Experiential Differences on Learning

In addition to genetic and personality differences having sometimes signiﬁcant effects on conditioning, it is also well known that prior learning experience can have powerful effects on the acquisition, maintenance, and extinction of fear and anxiety (Mineka, 1985; Mineka & Zinbarg, 1996, 2006). Indeed, there are a multitude of individual experiential differences that affect the outcome of aversive learning experiences. For example, the amount of exposure an individual has had with a potential CS before encountering it paired with a US very much affects the outcome of the conditioning experience. This phenomenon in classical conditioning is known as latent inhibition and illustrates that familiar stimuli or situations result in weaker conditioning than do novel or strange objects or situations (e.g., Lubow, 1998; see Kent, 1997, for a naturalistic example). Moreover, Mineka and Cook (1986) showed that monkeys who ﬁrst simply watched nonfearful monkeys behaving nonfearfully with snakes were immunized against later acquisition of a fear of snakes when they watched a fearful model behaving fearfully with snakes (which as discussed earlier ordinarily leads to strong and long-lasting conditioning). Finally, one study showed that rats initially exposed to escapable shocks later showed reduced fear conditioning both to a context and to a discrete CS relative to groups receiving no prior shocks or inescapable shocks (Baratta et al., 2007). Conversely, rats initially exposed to inescapable shocks later showed potentiated fear conditioning (see Chapter 6). Certain characteristics of a conditioning experience itself can also be important determinants of the level of fear that is conditioned. For example, the ability to escape (i.e., control) a traumatic experience dramatically reduces the magnitude of the level of fear that is conditioned

03-Schachtman-03.indd 56

to a CS+ relative to when the same intensity of trauma is inescapable or uncontrollable (e.g., Mineka et al., 1984; Mineka & Zinbarg, 1996, 2006). Animal research has also shown that social defeat (another uncontrollable stressor) leads to exaggerated fear-conditioned responses (e.g., Williams & Scott, 1989) as well as increased submissiveness to other conspeciﬁcs (e.g., Uhrich, 1938) as in social phobia. In addition, experiences that a person has following a conditioning experience may affect the strength and maintenance of conditioned fear. For example, the inﬂation effect, ﬁrst discovered by Rescorla (1974), suggests that someone who acquired a mild fear following a minor aversive experience with a CS might be expected to develop a more intense fear when later exposed to a much more aversive experience (even though it was not paired with the CS). These few examples of experiential factors that very much inﬂuence the onset and maintenance of fears and phobias are more complex than suggested by earlier simplistic conditioning views, although they are altogether consistent with contemporary research and theory on learning (Mineka & Zinbarg, 1996, 2006). Selective Associations in Fear Learning

Finally, some sources of important differences in who acquires phobias and other anxiety disorders reside in the nature of the objects or situations that come to be paired with aversive experiences. Indeed, human and non-human primates seem to be evolutionarily prepared to rapidly associate certain kinds of objects or situations—such as snakes, spiders, water, and enclosed spaces—with frightening or unpleasant events (e.g., Mineka & Öhman, 2002; Öhman, 1986; Öhman & Mineka, 2001). This inborn tendency to rapidly associate certain objects or situations that once posed real threats to our early ancestors probably occurred because organisms that did so would have enjoyed a selective advantage in the struggle for existence. In the social realm, the kinds of cues that are most likely to become the sources of fears are cues that signal dominance and aggression from conspeciﬁcs such as angry or threatening faces (e.g., Mineka & Öhman, 2002; Öhman & Mineka, 2001).

2/15/2011 7:42:33 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

A large amount of experimental evidence now supports this preparedness theory of phobias. One very important series of experiments using human participants conducted by Öhman and his colleagues has found that fear is conditioned more effectively to fear-relevant stimuli (slides of snakes, spiders, or angry faces) than to fearirrelevant stimuli (such as slides of ﬂowers, geometric objects, or happy faces) that are paired with mild electric shocks (e.g., see Öhman & Mineka, 2001, for a review). Indeed, even very brief subliminal presentations of such fearrelevant stimuli (but not fear-irrelevant stimuli) are sufﬁcient to evoke conditioned responses (including activation of the amygdala) either when presented in acquisition or in extinction (Carlsson et al., 2004; Öhman, Carlsson, Lundqvist, & Ingvar, 2007). This subliminal activation of responses to phobic stimuli may help to account for certain aspects of the irrationality of phobias because the fear may arise from cognitive structures not under conscious control (e.g., Öhman & Mineka, 2001). Another series of experiments on observational conditioning showed that lab-reared monkeys can easily acquire fears of fear-relevant stimuli such as toy snakes or toy crocodiles but not of fear-irrelevant stimuli such as ﬂowers or toy rabbits (Cook & Mineka, 1989, 1990). Thus, both monkeys and humans seem to selectively associate certain fear-relevant stimuli with threat or danger. Moreover, the lab-reared monkeys had no prior exposure to any of the stimuli involved (e.g., snakes or ﬂowers) before participating in these experiments. Thus, the monkey results support the evolutionarily based preparedness hypothesis even more strongly than do the human experiments. For example, human participants (unlike the lab-reared monkeys) might show superior conditioning to snakes or spiders because of ontogenetic factors such as preexisting negative associations to snakes or spiders, rather than because of evolutionary factors. These results also demonstrate that selective associations occur not only with mild and transient conditioning as seen in the human experiments but also with intense and longlasting phobic-like fears seen in the monkey experiments (Mineka & Ohman, 2002).

03-Schachtman-03.indd 57

57

Contextual Control of Conditioned Responses

Conditioned anxiety effects have also been observed using contextual cues associated with threat. Speciﬁcally, it has long been known that when unsignaled shocks are presented to rats in a distinctive environment, the environment itself acquires the capacity to elicit conditioned anxiety. Such contextual conditioning is important for several reasons. One reason why contextual conditioning is important stems from demonstrations that patients with PTSD and panic disorder show enhanced contextual modulation of baseline startle in experiments in which they know a fear-potentiated startle paradigm with an explicit threat cue will be included at some point (i.e., including delivery of shock) (Grillon & Ameli, 1998; Grillon & Morgan, 1999). However, anxiety-disordered individuals in these studies did not show exaggerated fear responses to explicit cues for imminent threat (as would be evidenced by greater fear-potentiated startle— that is, phasic fear to explicit cues) relative to controls. Thus, at least for people with panic disorder or PTSD, this elevated contextual modulation of startle seems to represent heightened anxiety in contexts in which something threatening may occur, but this is not accompanied by greater fear to more proximal discrete cues for threat (relative to controls). Very similar results have also been observed in children at familial risk for anxiety disorders (having a parent with an anxiety disorder) who also show enhanced contextual modulation of startle, but normal fear-potentiated startle (Grillon, Dierker, & Merikangas, 1998). Finally, individuals with high levels of neuroticism—a risk factor for most anxiety disorders—show a very similar pattern of results (Craske et al., 2009), that is, enhanced contextually modulated startle, but normal fearpotentiated startle relative to those with low or medium levels of neuroticism. Contextual conditioning is also important because of the critical role it plays in reinstatement of fear to a CS+ following the full or partial extinction that occurs when the CS is no longer followed by the US. Reinstatement is important in part because it may underlie the ﬂuctuating

2/15/2011 7:42:33 PM

58

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

course of symptoms often seen in anxiety disorders (e.g., Bouton et al., 2001; Mineka & Zinbarg, 1996). Reinstatement occurs when animals or humans whose fear of a CS has been extinguished show a return of that fear (“reinstatement”) after one or a few exposures to the US alone (not paired with the CS) (Rescorla & Heth, 1975). Subsequent work showed that reinstatement only occurs if the US is presented in the same context as where testing for reinstatement is to be conducted (e.g., Bouton, 1984). According to this and other research by Bouton and colleagues, when the reinstating US is presented, the animal must associate it with the current context. The presence of that contextual danger when the CS is next presented is thought to trigger fear of the previously extinguished CS. Moreover, evidence from Bouton’s lab has also conﬁrmed the prediction that lesions of the BNST signiﬁcantly attenuate reinstatement of fear, presumably because they blocked or reduced contextual conditioning of anxiety that would ordinarily occur with the reinstating US (e.g., Waddell et al., 2006). Unpredictability and Anxiety

Unpredictable aversive events have long been known to be more stressful than predictable aversive events, and exposure to unpredictability has been hypothesized to play a role in the development of panic disorder, GAD (with more minor events), and PTSD (with more severe events) (Mineka & Zinbarg, 1996, 2006). The most widely cited hypothesis offered to explain these effects is the safety-signal hypothesis (e.g., Seligman, 1968; Seligman & Binik, 1977). The idea is that when organisms are presented with signaled (predictable) aversive events they not only know when the event will occur (during the CS+) but also when the event will not occur (when the CS is not present—a safety signal). Safety signals allow them to relax and feel safe. By contrast, organisms exposed to unpredictable aversive events have no knowledge of when the threatening events will occur, or when they can relax and feel safe, and thus are in a state of chronic anxiety in this context. Early studies in rats used both stress-induced ulceration paradigms (Weiss, 1971) and conditioned suppression paradigms (Seligman, 1968).

03-Schachtman-03.indd 58

Recently this work has been replicated and extended using other measures of anxiety in human participants. For example, Grillon et al. (2004) modeled this situation in a human laboratory with normal participants who all experienced three conditions: one in which predictable shocks were given, one in which unpredictable shocks were given, and one in which no shocks were given. Results clearly showed that in the unpredictable shock condition participants showed both greater startle magnitude and higher subjective anxiety than in the other two conditions. Other studies by Grillon and colleagues have shown that in some situations what may be most important in determining contextual levels of anxiety is whether aversive stimuli are perceived to be predictable rather than whether they actually are predictable, with more conditioning to the context when the participants do not detect the CS-US contingency. So, for example, several studies have shown that participants in a discriminative conditioning procedure who failed to become aware of the CS-US contingency showed more contextual conditioning (i.e., potentiation of baseline startle responding in the startle context) than participants who were aware of the contingency. In the latter case they showed enhanced startle to their CS+ rather than more generalized contextual conditioning (Grillon, 2002). In one ingenious experiment by Grillon and colleagues (2006), participants were passively exposed to a virtual reality environment with three virtual rooms (one with no shock, one with predictable shocks, and one with unpredictable shocks). Participants later showed potentiated startle in the unpredictable context relative to the other two contexts, presumably because of greater contextual conditioning with the unpredictable shocks. Furthermore, when allowed to freely enter the three different rooms to ﬁnd monetary rewards, there was a strong preference for the no-shock context and avoidance of the unpredictable context. Recently Grillon and colleagues extended the study of unpredictability to patients with panic disorder, PTSD, and GAD to determine whether they show elevated reactivity to unpredictable aversive events. In one study on panic disorder,

2/15/2011 7:42:33 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

Grillon, Lissek et al. (2008) found that panic patients showed elevated contextually modulated startle when startle probes were delivered in a context where naturalistic unpredictable aversive events sometimes occurred (such as a white noise or a female scream) relative to a neutral context; startle responding in a predictable threat condition was intermediate. The healthy controls did not show this elevated startle potentiation in the unpredictable relative to the neutral condition. In another study the same investigators used nearly an identical paradigm but compared patients with PTSD, GAD and healthy controls (Grillon, Pine, Lissek, Rabin, & Bythilingam, 2009). The PTSD group showed elevated startle in the unpredictable context relative to the neutral and predictable contexts, but the other two groups showed elevated startle only in the predictable context. Thus, patients with both panic disorder and PTSD showed elevated reactivity to unpredictable aversive events relative to healthy controls and to patients with generalized anxiety. Summary

In this section we have brieﬂy reviewed multiple sources of individual differences in associative learning processes relevant to furthering our understanding of diathesis-stress perspectives on anxiety disorders. Thus, we discussed how high trait anxiety and clinical anxiety both affect acquisition and extinction of conditioned fear using simple conditioning paradigms, whereas discriminative conditioning does not seem to be affected—possibly because anxiety is associated with poor inhibitory conditioning. We also reviewed results of several studies suggesting that fear conditioning is moderately heritable and that individuals who have different combinations of polymorphisms of the 5-HTT and COMT genes show more or less robust fear conditioning, and faster or slower fear extinction. In addition, evidence that many individual experiential differences occurring before, during, or following real or vicarious conditioning trials affect the outcome of those conditioning trials was also reviewed. Finally, some sources of individual differences reside in the nature of the objects or situations that are associated with

03-Schachtman-03.indd 59

59

aversive consequences rather than in the individuals themselves as seen in research on selective associations. We also reviewed some of the evidence showing how individuals with certain anxiety disorders (PTSD and PD) show enhanced contextual conditioning in situations associated with uncertain threat and greater reactivity to unpredictable aversive events than do normal controls.

THREAT-RELEVANT ATTENTIONAL BIASES Although early models of anxiety disorders were primarily behavioral, many contemporary psychological models of the etiology and treatment of mood and anxiety disorders are more cognitive in nature. They posit that biased modes of processing affectively valenced material determine important characteristics of the emotional pathology (Beck, 1976; Eysenck, 1992; Williams, Watts, MacLeod, & Mathews, 1997). Within this tradition, attentional biases for emotional stimuli have been studied for over two decades and remain an exciting source of theoretical and applied questions in psychopathology. Most research on attentional biases has focused on their association with clinical anxiety and mood disorders, elevated trait anxiety, and dysphoria (Bar-Haim et al., 2007; Mogg & Bradley, 1998; Williams et al., 1997). Indeed, attentional biases for threat have been central to several etiological models of anxiety (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998; Williams et al., 1997). A smaller body of research has shown the following: (1) attentional biases in individuals at risk for emotional disorders (Joormann, Talbot, & Gotlib, 2007); (2) biases may predict negative reactions or responses to stress (MacLeod & Hagan, 1992; Mogg, Bradley, & Hallowell, 1994); (3) biases may persist beyond remission of clinical disorders (e.g., Joormann & Gotlib, 2007). Particularly interesting from a learning perspective are studies suggesting that bias modiﬁcation—the use of attentional training tasks to change response patterns—may reduce negative responses to stress and thereby indirectly inﬂuence mood states (e.g., Amir et al., 2009; Dandeneau et al., 2007; MacLeod et al., 2002).

2/15/2011 7:42:33 PM

60

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Many of the great challenges in this research are related to developing a deeper understanding of precisely what attentional phenomena are associated with mood and anxiety pathology, the causes of these associations, their consequences for other cognitive processes, and the etiology/ maintenance of clinical disorders. MacLeod, Mathews, and colleagues, who were responsible for the ﬁrst identiﬁcation of the association between trait anxiety and biased attention for threat (Mathews & MacLeod, 1985), have continued to explore the relationship between biased attention to threat and stress, as well as the effects of modifying attentional biases on reactions to stress (MacLeod et al., 2002; MacLeod, Soong, Rutherford & Campbell, 2007; See et al., 2009). Drawing from classic attentional models (Posner & Petersen, 1990), Fox and others have sought to understand the speciﬁc components and time courses of emotion-relevant attentional biases (orientation/shifting, disengagement). The work of Öhman and colleagues (e.g., Öhman & Mineka, 2001) has focused on attentional processing of evolutionary fear-relevant stimuli and their relationship to the development of clinical phobias. Moreover, great strides in understanding the neural and genetic underpinnings of threat perception and attention in trait anxiety/neuroticism have been made by several research groups (e.g., Canli et al., 2005; Hariri et al., 2002). In this section, we ﬁrst review the experimental paradigms typically used in attention and emotion research. Next, we summarize the most recent ﬁndings in attentional bias research, with particular focus on the concepts with potential relevance to conditioning processes. Methods of Attentional Bias Assessment

Attentional biases have traditionally been assessed using one of several experimental paradigms, including most prominently the emotional Stroop task, the dot-probe task, the exogenous spatial-cueing task, and the visual search task. All of these tasks share a common logic: Attentional biases are inferred by comparing an individual’s reaction times (RTs) on critical trials involving emotionally valenced stimuli to trials involving neutral stimuli (stimuli are variably words, photographs, or schematics of

03-Schachtman-03.indd 60

emotional faces, objects, or scenes). That is, individual biases are inferred when mean or median RTs to trials with emotionally valenced stimuli differ signiﬁcantly from those to trials with neutral stimuli. Researchers have debated and tested which of these paradigms best captures the biased attentional processing associated with psychopathology. In the following sections, we brieﬂy describe the most frequently employed paradigms, with particular emphasis on those that have been used in conditioning paradigms. Emotional Stroop Task

The emotional Stroop task (see Williams, Mathews, & MacLeod, 1996, for a review), although widely used in early research on attentional biases, has received substantial criticism as a pure measure of selective attention. In this task, participants are required to name the color that a word is printed in; trials in which the meaning of the word is emotionally valenced versus those in which the meaning is neutral are compared. Biased attention toward emotional words is inferred by slower reaction times to the emotional versus neutral words. Several researchers (e.g., Algom, Chajut, & Lev, 2004; Baldo, Shimamura, & Prinzmetal, 1998) have indicated that the emotional Stroop task represents a “generic slowdown,” or response bias, to emotionally valenced stimuli rather than a selective attention mechanism. Despite criticisms leveled against the emotional Stroop task, it continues to be used, and several of the conditioning studies discussed in the next section employ the measure. Dot-Probe Task

In the past 10 years, the dot-probe detection task (MacLeod, Mathews, & Tata, 1986) has emerged as the most widely used paradigm for assessing selective attentional biases. In the most common form of the task, trials consist of a neutral and a valenced stimulus brieﬂy presented (typically 15–1,250 ms) on each side of a central ﬁxation cross displayed on a computer monitor, followed immediately by a small dot probe that replaces one of the stimuli. The probe remains on the screen until participants indicate, by pressing a keyboard button, which side of the screen the

2/15/2011 7:42:33 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

probe appears on, and their latency to respond from the time of the dot probe onset is recorded as their reaction time. If a participant responds more quickly, on average, to probes replacing valenced than neutral stimuli, it is inferred that he or she was attending to the valenced stimulus at the time of the probe onset more frequently than to the neutral stimulus. Spatial-Cueing Task

Another task that is increasingly used is the exogenous spatial-cueing task, which was adapted by Fox and colleagues (2001) from the classic Posner cueing paradigm (Posner, 1980), in order to differentiate whether attentional biases associated with anxiety were due to facilitated orientation toward threat stimuli versus delayed disengagement from threat. Trials consist of a presentation of a single cue stimulus presented on the left or right side of a central ﬁxation cross on the screen for a brief period of time (typically 200–600 ms), followed by a screen with only the ﬁxation cross remaining, followed by a small target circle to which participants are required to respond with a spacebar press. Of these experimental trials, most (60%–75%) are valid (i.e., the target appears in the same location as the cue), whereas the remainder are either invalid (i.e., the target appears in the opposite location to the cue) or catch trials (no target appears after the cue). Equal numbers of trials are allotted to each cue stimulus type, and any particular stimulus has an equal probability of appearing in the left- and right-hand side boxes, and an equal probability of being followed by a valid, invalid, or catch trial. By comparing within-subject reaction times to threat versus neutral stimuli on validly cued trials, it is possible to infer that the speed of orientation is affected by the stimulus valence. By comparing reaction times to threat versus neutral stimuli on invalidly cued trials, it is possible to infer that the speed of disengagement is affected by the stimulus valence. At the longer stimulus exposure intervals (600 ms), this paradigm is used to assess the phenomenon of inhibition of return, in which all individuals are slower to reengage their attention toward a location or object that has been recently attended. Individual differences in the speed with which

03-Schachtman-03.indd 61

61

participants respond to a target that has been validly cued by a threatening versus neutral stimulus over this longer time frame are indicative of valence-dependent inhibition of return. Visual Search Task

The visual search task has been used less frequently in attentional bias research in clinical and trait anxiety (e.g., Fox et al., 2000; Öhman, Flykt, & Esteves, 2001; Tipples, Young, Quinlan, Broks, & Ellis, 2002), but it has been advocated as a more ecologically valid attention task than those discussed earlier (Weierich, Treat, & Hollingworth, 2008). It also has the advantage of a well-established body of normative research in visual cognition. In this task, participants view an array of visual stimuli on a computer screen and indicate whether a particular “target” stimulus is present among other distracter stimuli. The speed with which the target stimulus is detected is taken to indicate the efﬁciency with which attention is directed toward that stimulus. In anxiety research, arrays typically consist of fear-relevant and fear-irrelevant stimuli, and individual differences are assessed in the speed with which he or she is able to detect threatening stimuli among neutral arrays versus neutral stimuli among threatening stimulus arrays. Current Theories and Open Questions in Attentional Bias Research

In general, researchers have found statistically signiﬁcant within-subject differences to valenced and neutral stimuli primarily in participants with clinical anxiety or depression, or with high levels of trait anxiety or depressive symptoms. The majority of these effects have been found among individuals with anxiety disorders using threatening stimuli and shorter dot-probe or spatial-cueing task stimulus presentations (≤500 ms). Depression-related attentional biases (generally for depression-relevant stimuli, such as sad faces), by contrast, have only been consistently reported more recently, and only at exposure durations of greater than 1,000 ms (see Mogg & Bradley, 2005, for an overview). However, there are several exceptions to the failure to ﬁnd biases in nonanxious, nondepressed individuals.

2/15/2011 7:42:33 PM

62

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Several studies using dot-probe tasks have shown that at shorter exposure durations (100 ms), all individuals attend toward highly threatening stimuli (e.g., Cooper & Langton, 2006; Koster et al., 2005), while only anxious individuals continue to attend toward the threatening stimuli at exposure durations near 500 ms. Moreover, at least one study (Wilson & MacLeod, 2003) demonstrated that individuals both high and low in trait anxiety demonstrate a bias toward highly threatening angry faces, whereas only highly trait-anxious individuals demonstrate a bias toward moderately threatening faces. In addition, evidence has increasingly mounted for the notion that nonanxious individuals selectively attend away from mild to moderately threatening stimuli at longer exposure durations (see Bar-Haim et al., 2007; Frewen et al., 2008, for meta-analyses). Such ﬁndings point toward a rather complex interaction between individual differences in anxiety and depression, stimulus valence intensity, and stimulus exposure duration. As mentioned previously, several models of attentional biases in anxiety have been articulated by researchers (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998; Williams et al., 1997). In this section, we highlight the most important theoretical issues that researchers have tackled experimentally. Time Course of Attentional Biases: Orientation, Disengagement, and Avoidance

A recent debate in attentional bias research has concerned resolving apparently conﬂicting ﬁndings regarding whether high trait anxiety is associated with more rapid orientation toward threat or delayed disengagement from threat. In addition, there has also been related debate about whether initial attentional engagement with threat in trait anxiety is followed by avoidance of threat at longer stimulus exposure intervals. For instance, Mogg and Bradley (1998) argued that highly trait-anxious individuals direct their attention more rapidly toward threat, but later avoid it, resulting in a failure to habituate normally to threat-relevant stimuli. By contrast, Fox and colleagues (Fox, Russo, Bowles, & Dutton, 2001; Fox, Russo, & Dutton, 2002) demonstrated

03-Schachtman-03.indd 62

with the spatial-cueing task described earlier that highly trait-anxious individuals do not actually appear to orient more quickly toward a singly presented threatening cue, but they do tend to dwell on such stimuli longer than do low traitanxious individuals. Other researchers have found additional evidence for the delayed disengagement, or sustained attentional maintenance, hypothesis (e.g., Batty, Cave, & Pauli, 2003; Salemink, van den Hout, & Kindt, 2007). By approaching the literature from a more rigorous vision science perspective, Weierich, Treat, and Hollingworth (2008) have provided an excellent theoretical synthesis of these apparently disparate ﬁndings—that is, whether traitanxious individuals orient their attention more quickly toward threat versus whether they take longer to disengage from it, and whether these phenomena are followed by attentional threat avoidance. Their analysis is based on conceptual clariﬁcation of covert and overt attentional processes, close evaluation of the timescale at which biases are assessed, and differentiating between paradigms that require attention to a single stimulus versus competition between two stimuli (e.g., the exogenous cueing task, which uses a single stimulus, versus the dot-probe task, which presents two stimuli). In summary, they point out that these two views are compatible if it is assumed that threatening stimuli are more likely to be selected as targets of attention when in competition with other stimuli, but that the speed of covert attentional shifts toward stimuli is not inﬂuenced by threat relevance. That is, highly trait-anxious individuals may be more likely to attend to an angry face when it is presented together with a neutral face and be more likely to dwell upon it longer, but they are not faster than low trait-anxious individuals to covertly direct their attention toward the angry face. Moreover, evidence for overt and covert avoidance following disengagement from threat stimuli comes from studies assessing individual differences at longer stimulus exposure durations, particularly in eye-tracking experiments or spatial-cueing tasks showing interrupted inhibition of return among anxious individuals (Calvo & Avero, 2005; Fox et al., 2002; Pﬂugshaupt et al., 2005).

2/15/2011 7:42:33 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

Trait Anxiety, Threat Responsiveness, and the Amygdala

Several comprehensive theories of attention and anxiety (Mathews & Mackintosh, 1998; Mogg & Bradley, 1998) have argued that the perceived intensity of a threat cue mediates the relationship between trait anxiety and attentional biases. Evidence for this has been mixed, but it receives considerable support from the studies described in the next section on attentional bias and conditioning. Although explicit measures of selfreported threat intensity do not appear to be related to trait anxiety (Wilson & MacLeod, 2003), neuroimaging data increasingly suggests that trait anxiety is correlated with amygdala reactivity to emotional stimuli, even when presented subliminally (e.g., using backward masking and <15 ms stimulus exposure durations) (e.g., Etkin et al., 2004; Etkin & Wager, 2007). In further support of this hypothesis, amygdala reactivity during an attentional dot-probe task has also been found to be associated with behavioral measures of biased attention toward threat (Amin, Constable, & Canli, 2004; Armony & Dolan, 2002; Monk et al., 2004). As mentioned earlier, a novel neural-network model of attentional biases for threat versus positive stimuli using the dot-probe task has recently been published (Frewen et al., 2008). This model explicitly incorporates intensity appraisal, among other factors, in its connectionist architecture. According to the Frewen et al. model, sufﬁcient levels of perceived threat should receive preferred attention from all individuals. Thus, if trait-anxious individuals are more sensitive to threat stimuli, their threshold for directing attention toward a threatening stimulus should be lower than that for low trait-anxious individuals. It bears mentioning that a central assumption of this approach is that in order for low traitanxious individuals to selectively attend away, and high trait individuals to selectively attend toward, mildly to moderately threatening stimuli, it is necessary to introduce the idea that attention biases also have the normative function of orienting the organism toward potentially rewarding stimuli. Attention to threat would represent, in this model, an interruption of the

03-Schachtman-03.indd 63

63

normal operating mode. This notion dovetails nicely with theories of fear and anxiety discussed earlier (Quinn & Fanselow, 2006), which posit that the imminence of environmental threat serves to cue speciﬁc behavioral responses representing an interruption of the normal operating mode of reward-seeking behavior. Genetic and Neuroanatomical Correlates of Attention to Threat

Research on the genetics of attentional threat biases is only beginning to emerge, but recent publications have shown promising results that are consistent with the genetic ﬁndings on trait anxiety and anxiety disorders in general. As discussed earlier, behavioral genetics research has implicated both the COMT Val158 Met and 5-HTT promoter polymorphisms in reactivity in limbic and prefrontal areas of the brain, as well as anxiety and depression. To our knowledge, few published experiments have compared attentional biases for threat among individuals varying in one of these polymorphisms. In one experiment, Beevers, Gibb, McGeary, and Miller (2007) compared attentional biases for words related to anxious and dysphoric emotional states, using a standard dot-probe task, among groups of psychiatric inpatients who differed in the 5-HTT promoter polymorphism. Carriers of at least one short 5-HTT allele (the polymorphism associated with anxiety and depression) showed stronger attentional biases for anxious word stimuli as compared to carriers of two long alleles. In another recent study, Beevers and colleagues (2009) found that healthy individuals homozygous for the short allele, as well as those with a long allele variant (rs25531) that functions like a short allele, showed greater latency to disengage attention from sad, happy, and fearful faces than did individuals with the other allelic variations. Finally, Fox, Ridgewell, and Ashwin (2009) found that individuals with two long alleles (those at lowest risk for anxiety and depression) showed attentional biases away from negative and toward positive affective material. Further evidence for a possible link between genetics and attentional biases comes from the “neural endophenotype” approach, which compares neuroimaging patterns in behavioral tasks

2/15/2011 7:42:33 PM

64

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

among preselected individuals varying in theoretically signiﬁcant genotypes. Advocated by Canli (2008) and colleagues, this approach has also provided considerable insight into the mechanisms by which the phenotype of trait anxiety might relate to the phenomenon of biased attentional processing of threat. For instance, the COMT Val158 Met polymorphism has been implicated in individual differences in various cognitive functions, including attentional processes (Goldberg & Weinberger, 2004; Heinz & Smolka, 2006; Winterer & Goldman, 2003). In one functional magnetic resonance imaging (fMRI) study (Smolka et al., 2005), participants with one or more Met alleles had greater activation in limbic, prefrontal cortical regions, and regions associated with visuospatial attention when viewing negative, but not neutral or positively valenced images. Even more promising ﬁndings have been reported in regard to the 5-HTT polymorphism. As discussed earlier, presence of a short 5-HTT allele is associated with increased amygdala response to emotional faces relative to a visuospatial control task (Hariri et al., 2002). Moreover, Canli and colleagues (Canli et al., 2005; Canli et al., 2006) have provided evidence using an emotional Stroop task that greater amygdala activation to threat, relative to neutral stimuli, is associated with the presence of a short 5-HTT allele. Although this research is in its infancy, the published ﬁndings converge upon the notion that allelic variations among genes implicated in anxiety and depression may also be involved in attentional biases for emotional material. Further research is needed, however, to clarify the underlying mechanisms of these associations. Summary

Over the past 20 years, the question of how anxiety inﬂuences attention, speciﬁcally differential attention toward valenced stimuli, has received a great deal of interest and experimental investigation. In this section, we have reviewed the most recent ﬁndings in this area, which have had the beneﬁt of several new meta-analyses, critical methodological reviews, and a computational model. In short, it can be said that anxious, relative to nonanxious, individuals (1) do not orient

03-Schachtman-03.indd 64

toward threat stimuli more quickly, but they are more likely to select them as objects of attention relative to neutral stimuli, (2) tend to dwell upon threat stimuli for longer than neutral stimuli, and (3) are more likely to avoid reengaging with threat stimuli after disengagement than neutral stimuli. Recent neuroimaging and genetics ﬁndings in attentional biases have been largely consistent with the broader body of research on anxiety and threat processing, with associations found between biased attention for threat and amygdala reactivity, as well as speciﬁc genetic polymorphisms. Thus, this continues to be an exciting area worthy of further investigation.

RELATING FEAR CONDITIONING AND ATTENTIONAL BIASES As the previous sections have described, both fear/ anxiety conditioning and attentional biases for threatening stimuli have been implicated in the etiology and maintenance of anxiety disorders. Only recently, however, have researchers begun to assess how these two anxiety-related processes might relate to one another. To date, the published studies we have identiﬁed have focused on whether or how newly conditioned fear CSs may inﬂuence attention in normal (average traitanxious) participants. Here, we review the current body of research on the effect of fear conditioning on attentional bias for threat, research which stems from several theoretical perspectives. Many of these studies have been designed to explore the effect that conditioning of fear has upon attention to threat in normal (average trait-anxious) individuals, although the speciﬁc hypotheses being tested vary as will be described. The majority of these studies have a common design: Participants ﬁrst undergo a discriminative conditioning procedure using visual stimuli (either geometric ﬁgures or pictorial face stimuli) and an aversive US (white noise or mild electrodermal shock). They then complete an attentional bias task that tests whether the CS+ has indeed acquired the capacity to elicit a greater attentional bias than the CS–. However, as will be described later, successful conditioning of fear sufﬁcient to elicit attentional biases has also been obtained by interspersing the conditioning

2/15/2011 7:42:33 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

and attentional bias assessment phases, for example, by administering CS+–US and CS-–no US trials on intermixed blocks during the attentional bias task. Conditioned Fear and Attentional Shifting and Disengagement in Nonanxious Individuals

In two early studies, Stormark and colleagues (Stormark & Hugdahl, 1996; Stormark, Hugdahl, & Posner, 1999) used Posner’s (1980) exogenous spatial-cueing task to measure differential orientation (responding on validly cued trials) and shifting (responding on invalidly cued trials) following a discriminative conditioning paradigm in which certain highlighted rectangles (CS+) were explicitly paired with aversive white noise bursts (US), while other rectangles were presented with no noise (CS–). Differential conditioning to the CS+ was conﬁrmed in the conditioning group via electrodermal responses. Results showed that the conditioning group detected invalidly cued CS+ targets more quickly than invalidly cued CS– targets, indicating that participants shifted attention away from fearinducing stimuli more quickly than from neutral stimuli. No effect of discriminative conditioning was found for validly cued trials. These results suggest that nonanxious individuals do not tend to differ in orientation speed toward threatening versus nonthreatening stimuli, but they do appear to disengage from threat stimuli more quickly than neutral stimuli. However, it is important to note that this study used a relatively long stimulus exposure duration (600 ms). The ﬁndings provide support for the notion, described earlier, that nonanxious individuals may preferentially avoid conditioned fear stimuli; furthermore, the ﬁndings more speciﬁcally isolate this phenomenon to the disengagement stage of attention processing. Armony and Dolan (2002) adopted a somewhat different approach by combining the conditioning and attentional bias paradigms into a single task from the perspective of a participant. They compared the neural responses and reaction times of six healthy volunteers in an fMRI paradigm using an angry face CS+ paired with an aversive white noise, and an angry face

03-Schachtman-03.indd 65

65

CS– not paired with white noise. Trials consisting of a random presentation of (a) the CS– alone, (b) the CS+ alone, or (c) the CS+ with the US were followed immediately by a visual probe on either side of the screen. These conditioning trials were interspersed with typical dot-probe trials (using a 50 ms stimulus exposure duration) in which the CS+ and CS– were presented together, followed by a probe replacing either the CS+ (congruent) or CS– (incongruent trials). Participants were slower to respond to probes following incongruent trials than congruent trials, demonstrating the typical dot-probe bias effect. No reaction-time differences were observed when either the CS+ or CS– was presented singly. Thus, this study demonstrated that nonanxious individuals, at very short stimulus exposure durations, may indeed preferentially attend toward stimuli having acquired threat value from conditioning. Koster et al. (2004) followed this combinedtrials approach, using an exogenous spatialcueing task in which CSs were presented for 200 ms, rather than the dot-probe task employed by Armony and Dolan (2002). This task choice allowed Koster et al. to determine which aspects of attention were inﬂuenced by the acquisition of differential fear conditioning. Conditioning trials consisted of a white noise US paired with the CS+ and a neutral tone stimulus as a control to be paired with the CS–. Results indicated that the CS+ facilitated responding on validly cued trials and slowed responding on invalidly cued trials, suggesting that a signal of immediate threat relevance both captures attention more readily and holds it for longer than does a signal of safety in normal participants. Although these results appear to conﬂict with those of Stormark and colleagues described earlier, it is important to note that both the Armony and Dolan (2002) and Koster et al. (2004) studies used very short stimulus exposure durations. Thus, taken together, the studies suggest that nonanxious individuals appear to orient their attention more quickly toward threat-conditioned stimuli and disengage from them more slowly over very brief time courses, but that over a longer time course may show attentional avoidance of threat.

2/15/2011 7:42:34 PM

66

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

In a series of replications and extensions of these ﬁndings, Koster and colleagues (Koster, Crombez, Van Damme, Verschuere, & De Houwer, 2005; Van Damme, Crombez, Hermans, Koster, & Eccleston, 2006; Van Damme et al., 2004) also found the facilitated attentional capture and delayed disengagement observed during the acquisition phase diminished during an extinction phase, in which neither the aversive nor neutral USs were presented. Further evidence for the sensitivity of attentional biases to extinction processes can be found in an interesting study (Kelly & Forsyth, 2007) that investigated attentional biases in an observational fear-conditioning paradigm including acquisition and extinction. Moreover, as shown in Figure 3.2, in a paradigm using an aversive shock US (Van Damme et al., 2006), attentional biases to threat signals reemerged after extinction in a group undergoing reinstatement of the initial contingency, but not in a control group that did not undergo reinstatement conditioning.

Reinstatement group Control group 40

Attentional bias index (ms)

35 30 25 20 15 10 5 0 –5 –10 –15 –20 Baseline

Acquisition

Extinction Reinstatement

Figure 3.2 Mean and standard error of attentional bias index (difference scores of cue-validity index for CS+ minus cue-validity index for CS–) as a function of group (reinstatement/control) and experiment phase (baseline/acquisition/ extinction/reinstatement). (From Van Damme, S., Crombez, G., Hermans, D., Koster, E. H. W., & Eccleston, C. 2006. The role of extinction and reinstatement in attentional bias to threat: A conditioning approach. Behaviour Research and Therapy, 44(11), 1555–1563).

03-Schachtman-03.indd 66

Automaticity of Learning and Attentional Biases

Several studies have used conditioning and attentional bias designs to test the predictions of theories that posit automatic, preattentive processing of evolutionarily fear-relevant stimuli, such as angry faces, snakes, and spiders (e.g., Öhman, Lundqvist, & Esteves, 2001; Öhman & Mineka, 2001). Frequently, these studies have made reference to the two pathways of fear processing elucidated in the work of LeDoux and others (e.g., LeDoux, 1996), which differentiates two routes of threat-relevant sensory information to the amygdala: a “quick and dirty” input from the sensory thalamus, and a slower but more veridical representation from the sensory cortex. Attentional bias researchers have taken up the question of whether attentional biases might mediate one, but not the other, of these routes, in the studies described in the text that follows. In two experiments, Beaver, Mogg, and Bradley (2005) used dot-probe tasks following a conditioning paradigm to assess whether CSs conditioned without conscious awareness could bias attention without the confound of low-level perceptual differences between threat and nonthreat stimuli. In the ﬁrst study, a conditioning phase presented two masked fear-relevant stimuli (snakes and spiders) with either an aversive white noise US or a neutral tone; next, a dotprobe task presented CSs either unmasked (Experiment 1) or both masked and unmasked (Experiment 2), with fear-irrelevant ﬂower and mushroom stimuli paired with the fearconditioned stimuli for each dot-probe trial. A forced-choice recognition test conﬁrmed that participants were not aware of which stimuli had been paired with the aversive stimulus. Signiﬁcant attentional biases for the CS+ but not the CS– on the dot-probe task were found in both experiments for unmasked stimuli presented for 200 ms, and in the second experiment on masked trials as well. In the second experiment, it was also found that explicit ratings of the aversiveness of the US were predictive of differential attentional bias for both masked and unmasked CS+ versus CS–. Taken together, these results

2/15/2011 7:42:34 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

suggest that attentional bias depended upon the perceived aversiveness of the US and did not require conscious recognition of CSs during conditioning. In another set of experiments (Batty, Cave, & Pauli, 2005), visual search tasks were combined with an evaluative conditioning paradigm (in which aversive pictorial stimuli were used as USs rather than a shock or noise stimulus) to assess whether stimuli conditioned without conscious awareness could bias attention without the confound of low-level perceptual differences between threat and nonthreat stimuli. In these experiments, one abstract shape was paired with a highly aversive image (e.g., mutilated bodies) and one with a neutral image. Following each trial of the visual search task (each with 1, 3, 6, or 12 distracters which were similarly shaped ﬁgures) for which either one of the abstract shapes was the correct response, either the aversive or neutral image was shown brieﬂy. Successful evaluative conditioning was conﬁrmed using a modiﬁed Implicit Association Task (Greenwald, McGhee, & Schwartz, 1998), a frequently used measure of associations that develop under evaluative conditioning procedures. In the ﬁrst experiment, participants responded faster to threat-associated stimuli, but there was no evidence that they were detected preattentively, regardless of the trait anxiety of the participant. Similarly, the second experiment found that even individuals high in snake or spider fear showed no evidence of preattentive detection of threat-associated stimuli that had been associated with slides of snakes or spiders on conditioning trials. Although the ﬁndings of Batty, Cave, and Pauli (2005) and Beaver, Mogg, and Bradley (2005) appear to contradict one another in their ﬁndings for preattentive threat detection, it is important to underscore that Batty, Cave, and Pauli (2005) employed an evaluative rather than traditional fear-conditioning procedure. The relationship of Pavlovian conditioning with evaluative conditioning is still somewhat unclear. Indeed, this has been its own subject of investigation and debate, the scope of which is beyond the present chapter (see De Houwer, Thomas, & Baeyens, 2001, for a review). Although it has

03-Schachtman-03.indd 67

67

been found that variants of implicit association tasks are sensitive to changes in evaluative processes following evaluative conditioning, there is evidence to suggest that such paradigms may not capture the nature of threat associations resulting from classical, Pavlovian fear-conditioning paradigms. For instance, Boschen and colleagues (Boschen, Parker, & Neumann, 2007) found that despite successful discriminative conditioning as measured by SCR in a conditioning paradigm using angry face CSs and an aversive shock US, an implicit association task was not sensitive to these affective changes. However, a follow-up study (Pischek-Simpson, Boschen, Neumann, & Waters, 2009), found that the same conditioning procedure did lead to attentional biases as measured with a standard dot-probe task. Data from our own laboratory (unpublished) also suggest that the nature of the reactivity to threat associated with attentional biases is not well captured by implicit association measures. We administered a dot-probe task (using 300 ms and 500 ms exposure durations) together with self-report measures of trait anxiety and two implicit association measures (the Go/ No-Go Association Task [Nosek & Banaji, 2001] and the Single-Category Implicit Association Task [Karpinski & Steinman, 2006]) and found that although trait anxiety was signiﬁcantly correlated with attentional biases for angry faces, the implicit measures of threat association with these faces did not correlate signiﬁcantly with either trait anxiety or attention to threat. Taken together, these results suggest that the nature of the attentional bias to threat is affected by Pavlovian, rather than evaluative, conditioning mechanisms. Another recent study by Raes and colleagues (Raes, De Raedt, Fias, Koster, & Van Damme, 2009) investigated the inﬂuence that contingency awareness in associative learning may have on biased attention toward threat. Raes et al. used a combined spatial-cueing task (with 200 ms colored rectangle cue presentations) and an evaluative conditioning paradigm, in which either aversive pictures or neutral pictures were presented following half of the spatial-cueing trials. Participants were either told beforehand that some of the colored cue slides predicted an

2/15/2011 7:42:34 PM

68

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

unpleasant image and some predicted a neutral image, or they were told nothing. Participants receiving contingency instructions demonstrated slower disengagement from the cues predicting threat (slower responding on invalid CS+ than CS– trials), but no difference in orientation to threat cues. Perplexingly, the group not receiving contingency instructions did not differ in disengagement from CS+ and CS– cues, but showed faster orientation toward the CS– cues than the CS+ cues. However, the overall index of cue validity between the contingency instruction group and no instruction group was signiﬁcant, indicating that biased attentional processing of the threat cues occurred only when the participants were aware of this contingency. Because of the small sample size of the study (n = 40), the authors cautioned against weighting the orientation versus disengagement results too heavily, but they stressed the considerable overall difference in cue-validity effects between groups. Trait Anxiety and Conditioning in Attentional Biases

Given that the majority of attentional threat bias research has investigated the relationship of trait anxiety or other clinical symptoms to abnormal attentional processing, surprisingly few studies combining threat biases and conditioning have assessed the phenomena among clinical populations or assessed concurrent levels of trait anxiety or anxiety symptoms in a normal sample. In fact, we were only able to identify one study that speciﬁcally sought to assess trait anxiety associated with attentional biases following a conditioning procedure. In this experiment (Lee, Lim, Lee, Kim, & Choi, 2009), trait anxiety was measured by self-report among normal participants, who were then administered a discriminative fear-conditioning task using face stimuli as CSs and a shock US followed by an emotional Stroop task. As found in other studies, an emotional Stroop bias for CS+ versus CS– was found across all participants. Importantly, however, the acquired attentional bias was signiﬁcantly correlated with both the self-reported trait anxiety as well as the degree of self-reported aversiveness of the US.

03-Schachtman-03.indd 68

Summary

In our review of the literature speciﬁcally combining conditioning and attentional methodologies, we found that researchers have focused intently on the question of whether conditioned fear associations may be detected with attentional bias measures. The answer to this question appears from the majority of these studies to be that yes, conditioned fear associations may be detected with a variety of attentional bias paradigms. Exceptions to this appeared most consistently in studies employing evaluative procedures, in which the US is an aversive pictorial stimulus rather than an aversive noise or shock. This ﬁnding is particularly interesting given the failure of implicit association measures to detect associations conditioned through classical, but not evaluative, means. Speciﬁc questions that researchers have sought to explore using conditioning paradigms have varied from determining the speciﬁc components of attention that may be affected by newly conditioned threat stimuli to testing theories of the automaticity of attention to threat. However, we were surprised to ﬁnd so few studies explicitly incorporating trait-anxious or clinical populations, given that the majority of threat-relevant attentional bias research has focused on the relationship between subjective anxiety and attentional biases. We hope that future studies will move toward more rigorous testing of possible mediating relationships between acquisition of conditioned fear and biased attentional processing of threat. Also notable is the lack of studies employing contextual conditioning, which, according to the most recent conditioning research reviewed earlier, is more akin to states of anxiety rather than fear. This suggests another avenue that may be especially fruitful to pursue. Finally, we would like to underscore the fact that all of the studies identiﬁed tested the ability of conditioned threat stimuli to affect attentional biases. The possibility of the converse—that attentional biases for threat associated with trait anxiety may inﬂuence the nature of fear and anxiety conditioning—has not yet been explored, to our knowledge. Given that abnormal attentional processing of threat has been fairly consistently

2/15/2011 7:42:34 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

associated with trait anxiety, and that attention plays an important role in modern conditioning theories, it seems likely that such a relationship may exist.

CONCLUSION The study of fear and anxiety in humans has beneﬁtted considerably from diverse experimental perspectives and methodologies in the past century, including behavioral, cognitive, physiological, neuroanatomical, and genetic approaches. In this chapter, we have described some of the highlights and main ﬁndings in research on two phenomena closely linked with both trait anxiety and clinical anxiety disorders: biased attentional processing of threat, and conditioning of fear and anxiety. Although the experiments published to date speciﬁcally combining fear/anxiety conditioning and attentional bias paradigms are limited, this is an emerging literature and we hope this review will encourage researchers to explore further some of the ways in which attentional biases for threat may interact with conditioned fear and anxiety. In threat-relevant attentional bias research, a consensus appears to have been reached that anxious, relative to nonanxious, individuals (1) do not orient toward threat stimuli more quickly but are more likely to select them as objects of attention than neutral stimuli, (2) tend to dwell upon threat stimuli for longer than neutral stimuli, and (3) are more likely to avoid reengaging with threat stimuli after disengagement than neutral stimuli. Moreover, research suggests that attention to threat is associated with similar neuroanatomical areas and genetic polymorphisms implicated in trait anxiety. These genetic and neuroimaging ﬁndings are especially interesting to consider in light of the recent conditioning experiment described earlier (Lonsdorf et al., 2009), which found greater fear acquisition among carriers of one or two short 5-HTT alleles, and slower extinction among carriers of two COMT met alleles. We hope that juxtaposing these ﬁndings in this review will encourage future research that attempts to link attentional biases to conditioning through several such levels of analysis.

03-Schachtman-03.indd 69

69

In our review of studies linking conditioning and attentional biases, we found a growing body of evidence suggesting that a range of attentional bias measures are sensitive to conditioned fear associations. However, the majority of these studies were not directly aimed at investigating whether such conditioning processes may mediate the trait anxiety and attentional bias link, which we suggest is a promising area for future research. Moreover, despite the fact that conditioning abnormalities and attentional biases for threat are associated with trait anxiety, we were unable to ﬁnd any studies assessing whether individual differences in attention to threat may account for conditioning abnormalities. The relative inattention to personality variables such as trait anxiety reﬂects, perhaps, the common focus among researchers in both fear learning and cognitive processing biases upon experimental manipulation and causal mechanisms rather than sources of individual differences and personality traits (Revelle & Oehlberg, 2008). However, our review of current ﬁndings in fear conditioning, contextual conditioning, and attentional biases for threat suggest several areas of potential overlap. It is possible that the preferential attentional processing of threat stimuli by anxious individuals may help to account for these conditioning abnormalities; however, as of yet, no coherent theoretical framework or experimental data exist to describe the possible nature of this relationship. In the future, we hope there may be an attempt to integrate further the current ﬁndings of fear and anxiety conditioning with our considerable knowledge of threat-relevant attentional biases.

REFERENCES Algom, D., Chajut, E., & Lev, S. (2004). A rational look at the emotional Stroop phenomenon: A generic slowdown, not a Stroop effect. Journal of Experimental Psychology: General, 133(3), 15. Amin, Z., Constable, R. T., & Canli, T. (2004). Attentional bias for valenced stimuli as a function of personality in the dot-probe task. Journal of Research in Personality, 38(1), 15–23. Amir, N., Beard, C., Burns, M., & Bomyea, J. (2009). Attention modiﬁcation in individuals

2/15/2011 7:42:34 PM

70

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

with generalized anxiety disorder. Journal of Abnormal Psychology, 118(1), 28–33. Armony, J. L., & Dolan, R. J. (2002). Modulation of spatial attention by fear-conditioned stimuli: An event-related fMRI study. Neuropsychologia, 40(7), 817–826. Askew, C., & Field, A. P. (2007). Vicarious learning and the development of fears in childhood. Behaviour Research and Therapy, 45(11), 2616–2627. Baldo, J. V., Shimamura, A. P., & Prinzmetal, W. (1998). Mapping symbols to response modalities: Interference effects on Stroop-like tasks. Perception and Psychophysics, 60(3), 427–437. Bar-Haim, Y., Lamy, D., Pergamin, L., BakermansKranenburg, M. J., & van Ijzendoorn, M. H. (2007). Threat-related attentional bias in anxious and nonanxious individuals: A metaanalytic study. Psychological Bulletin, 133(1), 1–24. Baratta, M. V., Christianson, J. P., Gomez, D. M., Zarza, C. M., Amat, J., Masini, C. V., . . . Maier, S. F. (2007). Controllable versus uncontrollable stressors bi-directionally modulate conditioned but not innate fear. Neuroscience, 146(4), 1495–1503. Barrett, P. M., Rapee, R. M., Dadds, M. M., & Ryan, S. M. (1996). Family enhancement of cognitive style in anxious and aggressive children. Journal of Abnormal Child Psychology, 24(2), 187–203. Barlow, D. H. (1988). Anxiety and its disorders. New York, NY: Guilford Press. Barlow, D. H. (2002). Anxiety and its disorders: The nature and treatment of anxiety and panic. New York, NY: The Guilford Press. Batty, M. J., Cave, K. R., & Pauli, P. (2003). Attentional biases to stimuli associated with threat reﬂect difﬁculty in disengaging attention rather than preattentive identiﬁcation. Perception, 32 ECVP Abstract Supplement Batty, M. J., Cave, K. R., & Pauli, P. (2005). Abstract stimuli associated with threat through conditioning cannot be detected preattentively. Emotion, 5(4), 418–430. Beaver, J. D., Mogg, K., & Bradley, B. P. (2005). Emotional conditioning to masked stimuli and modulation of visuospatial attention. Emotion, 5(1), 67–79. Beck, A. T. (1976). Cognitive therapy and the emotional disorders. New York, NY: International Universities Press. Beevers, C. G., Gibb, B. E., McGeary, J. E., & Miller, I. W. (2007). Serotonin transporter

03-Schachtman-03.indd 70

genetic variation and biased attention for emotional word stimuli among psychiatric inpatients. Journal of Abnormal Psychology, 116(1), 208–212. Beevers, C. G., Wells, T., Ellis, A. J., & McGeary, J. E. (2009). Association of the serotonin transporter gene promoter region (5-HTTLPR) polymorphism with biased attention for emotional stimuli. Journal of Abnormal Psychology, 118(3), 670–681. Bertolino, A., Arciero, G., Rubino, V., Latorre, V., De Candia, M., Mazzola, V., . . . Scarabino, T. (2005). Variation of human amygdala response during threatening stimuli as a function of 5’HTTLPR genotype and personality style. Biological Psychiatry, 57(12), 1517–1525. Biederman, J., Rosenbaum, J. F., Hirshfeld, D. R., Faraone, S. V., Bolduc, E. A., Gersten, M., . . . Reznick, J. S. (1990). Psychiatric correlates of behavioral inhibition in young children of parents with and without psychiatric disorders. Archives of General Psychiatry, 47(1), 21–26. Borkovec, T. D., Alcaine, O., & Behar, E. (2004). Avoidance theory of worry and generalized anxiety disorder. In R. G. Heimberg, C. L. Turk, & D. S. Mennin (Eds.), Generalized anxiety disorder: Advances in research and practice (pp. 77–108). New York, NY: Guilford. Boschen, M. J., Parker, I., & Neumann, D. L. (2007). Changes in implicit associations do not occur simultaneously to Pavlovian conditioning of physiological anxiety responses. Journal of Anxiety Disorders, 21(6), 788–803. Bouton, M. E., Mineka, S., & Barlow, D. H. (2001). A modern learning theory perspective on the etiology of panic disorder. Psychological Review, 108(1), 4–32. Brown, T. A., Chorpita, B. F., & Barlow, D. H. (1998). Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology, 107(2), 179–192. Bruch, M. A., & Heimberg, R. G. (1994). Differences in perceptions of parental and personal characteristics between generalized and nongeneralized social phobics. Journal of Anxiety Disorders, 8(2), 155–168. Calvo, M., & Avero, P. (2005). Time course of attentional bias to emotional scenes in anxiety: Gaze direction and duration. Cognition and Emotion, 19(3), 433–451. Canli, T. (2008). Toward a “molecular psychology” of personality. In O. P. John, R. W. Robins, &

2/15/2011 7:42:34 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

L. A. Pervin (Eds.), Handbook of personality: Theory and research (Vol. 3, pp. 311–327). New York, NY: The Guilford Press. Canli, T., Omura, K., Haas, B. W., Fallgatter, A., Constable, R. T., & Lesch, K. P. (2005). Beyond affect: A role for genetic variation of the serotonin transporter in neural activation during a cognitive attention task. Proceedings of the National Academy of Sciences, 102(34), 12224–12229. Canli, T., Qiu, M., Omura, K., Congdon, E., Haas, B. W., Amin, Z., . . . Lesch, K. P. (2006). Neural correlates of epigenesis. Proceedings of the National Academy of Sciences, 103(43), 16033–16038. Carlsson, K., Petersson, K. M., Lundqvist, D., Karlsson, A., Ingvar, M., & Ohman, A. (2004). Fear and the amygdala: Manipulation of awareness generates differential cerebral responses to phobic and fear-relevant (but nonfeared) stimuli. Emotion, 4(4), 340–353. Caspi, A., Sugden, K., Mofﬁtt, T. E., Taylor, A., Craig, I. W., Harrington, H. . . .Poulton, R. (2003). Inﬂuence of life stress on depression: Moderation by a polymorphism in the 5-HTT gene. Science, 301, 386–389. Clark, L. A., Watson, D., & Mineka, S. (1994). Temperament, personality, and the mood and anxiety disorders. Journal of Abnormal Psychology, 103(1), 103–116. Cook, M., & Mineka, S. (1989). Observational Conditioning of Fear to Fear-Relevant Versus Fear-Irrelevant Stimuli in Rhesus Monkeys. Journal of Abnormal Psychology, 98(4), 448–459. Cook, M., & Mineka, S. (1990). Selective Associations in the Observational Conditioning of Fear in Rhesus Monkeys. Journal of Experimental Psychology: Animal Behavior Processes, 16(4), 372–389. Cook, M., & Mineka, S. (1991). Selective associations in the origins of phobic fears and their implications for behavior therapy. In P. Martin (Ed.), Handbook of behavior therapy and psychological science: An integrative approach (pp. 413–434). Oxford, England: Pergamon Press. Cooper, R. M., & Langton, S. R. H. (2006). Attentional bias to angry faces using the dot-probe task? It depends when you look for it. Behaviour Research and Therapy, 44(9), 1321–1329. Craske, M. G., & Mystkowski, J. L. (2006). Exposure therapy and extinction: Clinical studies. In M. G. Craske, D. Hermans, & D. Vansteenwegen (Eds.), Fear and learning: From basic processes to

03-Schachtman-03.indd 71

71

clinical implications (pp. 217–233). Washington, DC: APA Books. Craske, M. G., Waters, A. M., Nazarian, M., Mineka, S., Zinbarg, R. E., Grifﬁth, J. W., . . . Ornitz, E. M. (2009). Does neuroticism in adolescents moderate contextual and explicit threat cue modulation of the startle reﬂex? Biological Psychiatry, 65(3), 220–226. Dandeneau, S. D., Baldwin, M. W., Baccus, J. R., Sakellaropoulo, M., & Pruessner, J. C. (2007). Cutting stress off at the pass: Reducing vigilance and responsiveness to social threat by manipulating attention. Journal of Personality and Social Psychology, 93(4), 651–666. Davis, M. (1998). Are different parts of the extended amygdala involved in fear versus anxiety? Biological Psychiatry, 44(12), 1239–1247. Davis, M. (2006). Neural systems involved in fear and anxiety measured with fear-potentiated startle. Journal of Neuroscience, 22, 2343–2351. Davis, M., & Shi, C. (1999). The extended amygdala: Are the central nucleus of the amygdala and the bed nucleus of the stria terminalis differentially involved in fear versus anxiety? Annals of the New York Academy of Sciences, 877(1), 281–291. Davis, M., Walker, D. L., & Lee, Y. (1997). Amygdala and bed nucleus of the stria terminalis: Differential roles in fear and anxiety measured with the acoustic startle reﬂex. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 352(1362), 1675–1687. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127(6), 853–869. Domschke, K., Freitag, C. M., Kuhlenbäumer, G., Schirmacher, A., Sand, P., Nyhuis, P., . . . Deckert, J. (2004). Association of the functional V158M catechol-O-methyl-transferase polymorphism with panic disorder in women. The International Journal of Neuropsychopharmacology, 7(02), 183–188. Eley, T. C., Sugden, K., Corsico, A., Gregory, A. M., Sham, P., McGufﬁn, P., . . . Craig, I. W. (2004). Gene-environment interaction analysis of serotonin system markers with adolescent depression. Molecular Psychiatry, 9(10), 908–915. Enoch, M. A., Xu, K., Ferro, E., Harris, C. R., & Goldman, D. (2003). Genetic origins of anxiety in women: A role for a functional catechol-Omethyltransferase polymorphism. Psychiatric Genetics, 13(1), 33–41.

2/15/2011 7:42:34 PM

72

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Esteves, F., Dimberg, U., & Öhman, A. (1994). Automatically elicited fear: Conditioned skin conductance responses to masked facial expressions. Cognition and Emotion, 8(5), 393–413. Etkin, A., Klemenhagen, K. C., Dudman, J. T., Rogan, M. T., Hen, R., Kandel, E. R., & Hirsch, J. (2004). Individual differences in trait anxiety predict the response of the basolateral amygdala to unconsciously processed fearful faces. Neuron, 44(6), 1043–1055. Etkin, A., & Wager, T. D. (2007). Functional neuroimaging of anxiety: A meta-analysis of emotional processing in PTSD, social anxiety disorder, and speciﬁc phobia. American Journal of Psychiatry, 164(10), 1476–1488. Eysenck, M. W. (1992). Anxiety: The cognitive perspective. Hove, England: Erlbaum. Fanselow, M. S., & Lester, L. S. (1988). A functional behavioristic approach to aversively motivated behavior: Predatory imminence as a determinant of the topography of defensive behavior. In R. C. Bolles & M. D. Beecher (Eds.), Evolution and learning (pp. 185–212). Hinsdale, NJ: Lawrence Erlbaum. Fox, E., Lester, V., Russo, R., Bowles, R. J., Pichler, A., & Dutton, K. (2000). Facial expressions of emotion: Are angry faces detected more efﬁciently? Cognition and Emotion, 14(1), 61–92. Fox, E., Ridgewell, A., & Ashwin, C. (2009). Looking on the bright side: Biased attention and the human serotonin transporter gene. Proceedings of the Royal Society B: Biological Sciences, 276, 1747–1751. Fox, E., Russo, R., Bowles, R., & Dutton, K. (2001). Do threatening stimuli draw or hold visual attention in subclinical anxiety? Journal of Experimental Psychology: General, 130(4), 681–700. Fox, E., Russo, R., & Dutton, K. (2002). Attentional bias for threat: Evidence for delayed disengagement from emotional faces. Cognition and Emotion, 16(3), 355–379. Frewen, P. A., Dozois, D. J. A., Joanisse, M. F., & Neufeld, R. W. J. (2008). Selective attention to threat versus reward: Meta-analysis and neuralnetwork modeling of the dot-probe task. Clinical Psychology Review, 28(2), 308–338. Gerull, F. C., & Rapee, R. M. (2002). Mother knows best: Effects of maternal modelling on the acquisition of fear and avoidance behaviour in toddlers. Behaviour Research and Therapy, 40(3), 279–287.

03-Schachtman-03.indd 72

Gogos, J. A., Morgan, M., Luine, V., Santha, M., Ogawa, S., Pfaff, D., et al. (1998). Catechol-Omethyltransferase-deﬁcient mice exhibit sexually dimorphic changes in catecholamine levels and behavior. Proceedings of the National Academy of Sciences, 95(17), 9991–9996. Goldberg, T. E., & Weinberger, D. R. (2004). Genes and the parsing of cognitive processes. Trends in Cognitive Sciences, 8(7), 325–335. Gray, J. A., & McNaughton, N. (2000). The neuropsychology of anxiety. New York, NY: Oxford University Press. Green, G., & Osborne, J. G. (1985). Does vicarious instigation provide support for observational learning theories? A critical review. Psychological Bulletin, 97(1), 3–16. Greenwald, A. G., McGhee, D. E., & Schwartz, L. J. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464–1480. Grillon, C. (2002). Startle reactivity and anxiety disorders: aversive conditioning, context, and neurobiology. Biological Psychiatry, 52(10), 958–975. Grillon, C. (2008). Models and mechanisms of anxiety: Evidence from startle studies. Psychopharmacology, 199(3), 421–437. Grillon, C., & Ameli, R. (1998). Effects of threat and safety signals on startle during anticipation of aversive shocks, sounds, or airblasts. Journal of Psychophysiology, 12, 329–337. Grillon, C., & Ameli, R. (2001). Conditioned inhibition of fear-potentiated startle and skin conductance in humans. Psychophysiology, 38(05), 807–815. Grillon, C., Baas, J. M. P., Cornwell, B., & Johnson, L. (2006). Context conditioning and behavioral avoidance in a virtual reality environment: Effect of predictability. Biological Psychiatry, 60(7), 752–759. Grillon, C., Baas, J. P., Lissek, S., Smith, K., & Milstein, J. (2004). Anxious responses to predictable and unpredictable aversive events. Behavioral Neuroscience, 118(5), 916–924. Grillon, C., Dierker, L., & Merikangas, K. R. (1998). Fear-potentiated startle in adolescent offspring of parents with anxiety disorders. Biological Psychiatry, 44(10), 990–997. Grillon, C., Lissek, S., Rabin, S., McDowell, D., Dvir, S., & Pine, D. S. (2008). Increased anxiety during anticipation of unpredictable but not predictable aversive stimuli as a psychophysiologic

2/15/2011 7:42:34 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

marker of panic disorder. American Journal of Psychiatry, 165(7), 898. Grillon, C., & Morgan, C. A. (1999). Fearpotentiated startle conditioning to explicit and contextual cues in Gulf War veterans with posttraumatic stress disorder. Journal of Abnormal Psychology, 108, 134–142. Grillon, C., Pellowski, M., Merikangas, K. R., & Davis, M. (1997). Darkness facilitates the acoustic startle reﬂex in humans. Biological Psychiatry, 42(6), 453–460. Grillon, C., Pine, D. S., Lissek, S., Rabin, S., & Vythilingam, M. (2009). Increased anxiety during anticipation of unpredictable aversive stimuli in posttraumatic stress disorder but not in generalized anxiety disorder. Biological Psychiatry, 66(1), 47–53. Hariri, A. R. (2009). The neurobiology of individual differences in complex behavioral traits. Annual Review of Neuroscience, 32, 225–247. Hariri, A. R., Mattay, V. S., Tessitore, A., Kolachana, B., Fera, F., Goldman, D., . . . Weinberger, D. R. (2002). Serotonin transporter genetic variation and the response of the human amygdala. Science, 297(5580), 400–403. Hayward, C., Killen, J. D., Kraemer, H. C., & Taylor, C. (1998). Linking self-reported childhood behavioral inhibition to adolescent social phobia. Journal of American Academy of Child and Adolescent Psychiatry, 37(12), 1308–1316. Hayward, C., Killen, J. D., Kraemer, H. C., & Taylor, C. (2000). Predictors of panic attacks in adolescents. Journal of American Academy of Child and Adolescent Psychiatry, 39(2), 207–214. Heinz, A., Braus, D. F., Smolka, M. N., Wrase, J., Puls, I., Hermann, D., . . . Büchel, C. (2004). Amygdala-prefrontal coupling depends on a genetic variation of the serotonin transporter. Nature Neuroscience, 8(1), 20–21. Heinz, A., Jones, D. W., Mazzanti, C., Goldman, D., Ragan, P., Hommer, D., Weinberger, D. R. (2000). A relationship between serotonin transporter genotype and in vivo protein expression and alcohol neurotoxicity. Biological Psychiatry, 47(7), 643–649. Heinz, A., & Smolka, M. N. (2006). The effects of catechol O-methyltransferase genotype on brain activation elicited by affective stimuli and cognitive tasks. Reviews in the Neurosciences, 17(3), 359–367. Hermann, C., Ziegler, S., Birbaumer, N., & Flor, H. (2002). Psychophysiological and subjective

03-Schachtman-03.indd 73

73

indicators of aversive Pavlovian conditioning in generalized social phobia. Biological Psychiatry, 52(4), 328–337. Hettema, J. M., Annas, P., Neale, M. C., Kendler, K. S., & Fredrikson, M. (2003). A twin study of the genetics of fear conditioning. Archives of General Psychiatry, 60(7), 702–708. Joormann, J., & Gotlib, I. H. (2007). Selective attention to emotional faces following recovery from depression. Journal of Abnormal Psychology, 116(1), 80–85. Joormann, J., Talbot, L., & Gotlib, I. H. (2007). Biased processing of emotional information in girls at risk for depression. Journal of Abnormal Psychology, 116(1), 135–143. Jovanovic, T., Keyes, M., Fiallos, A., Myers, K. M., Davis, M., & Duncan, E. J. (2005). Fear potentiation and fear inhibition in a human fearpotentiated startle paradigm. Biological Psychiatry, 57(12), 1559–1564. Jovanovic, T., Norrholm, S. D., Fennell, J. E., Keyes, M., Fiallos, A. M., Myers, K. M., Davis, M., & Duncan, E. J. (2009). Posttraumatic stress disorder may be associated with impaired fear inhibition: Relation to symptom severity. Psychiatry Research, 167, 151–160. Karpinski, A., & Steinman, R. B. (2006). The single category implicit association test as a measure of implicit social cognition. Journal of Personality and Social Psychology, 91(1), 16–32. Kaufman, J., Yang, B. Z., Douglas-Palumberi, H., Houshyar, S., Lipschitz, D., Krystal, J. H., & Gelernter, J. (2004). Social supports and serotonin transporter gene moderate depression in maltreated children. Proceedings of the National Academy of Sciences, 101(49), 17316–17321. Kelly, M. M., & Forsyth, J. P. (2007). Observational fear conditioning in the acquisition and extinction of attentional bias for threat: An experimental evaluation. Emotion, 7(2), 324–335. Kendler, K. S., Walters, E. E., Neale, M. C., Kessler, R. C., Heath, A. C., & Eaves, L. J. (1995). The structure of the genetic and environmental risk factors for six major psychiatric disorders in women. Phobia, generalized anxiety disorder, panic disorder, bulimia, major depression, and alcoholism. Archives of General Psychiatry, 52(5), 374–383. Kent, G. (1997). Dental phobias. In G. C. Davey (Ed.), Phobias: A handbook of theory, research and treatment (pp. 107–127). Chichester, England: Wiley.

2/15/2011 7:42:34 PM

74

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Koster, E. H. W., Crombez, G., Van Damme, S., Verschuere, B., & De Houwer, J. (2004). Does imminent threat capture and hold attention. Emotion, 4(3), 312–317. Koster, E. H. W., Crombez, G., Van Damme, S., Verschuere, B., & De Houwer, J. (2005). Signals for threat modulate attentional capture and holding: Fear-conditioning and extinction during the exogenous cueing task. Cognition and Emotion, 19(5), 771–780. Koster, E. H. W., Verschuere, B., Crombez, G., & Van Damme, S. (2005). Time-course of attention for threatening pictures in high and low trait anxiety. Behaviour Research and Therapy, 43(8), 1087–1098. LeDoux, J. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York, NY: Simon & Schuster, Inc. LeDoux, J. E. (1995). Emotion: Clues from the brain. Annual Review of Psychology, 46(1), 209–235. Lee, T. H., Lim, S. L., Lee, K., Kim, H. T., & Choi, J. S. (2009). Conditioning-induced attentional bias for face stimuli measured with the emotional Stroop task. Emotion, 9(1), 134–159. Lesch, K. P., Bengel, D., Heils, A., Sabol, S. Z., Greenberg, B. D., Petri, S., . . . Murphy, D. L. (1996). Association of anxiety-related traits with a polymorphism in the serotonin transporter gene regulatory region. Science, 274, 1527–1531. Levy, M. N., & Martin, P. J. (1981). Neural regulation of the heart beat. Annual Review of Physiology, 43(1), 443–453. Lissek, S., Powers, A. S., McClure, E. B., Phelps, E. A., Woldehawariat, G., Grillon, C., & Pine, D. S. (2005). Classical fear conditioning in the anxiety disorders: A meta-analysis. Behaviour Research and Therapy, 43(11), 1391–1424. Lonsdorf, T. B., Weike, A. I., Nikamo, P., Schalling, M., Hamm, A. O., & Öhman, A. (2009). Genetic gating of human fear learning and extinction: Possible implications for gene-environment interaction in anxiety disorder. Psychological Science, 20(2), 198–206. Lubow, R. E. (1998). Latent inhibition and behavior pathology: Prophylactic and other possible effects of stimulus preexposure. In W. T. O’Donohue (Ed.), Learning and behavior therapy (pp. 107–121). Needham Heights, MA: Allyn & Bacon. MacLeod, C., & Hagan, R. (1992). Individual differences in the selective processing of threatening

03-Schachtman-03.indd 74

information, and emotional responses to a stressful life event. Behaviour Research and Therapy, 30(2), 151–161. MacLeod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal of Abnormal Psychology, 95(1), 15–20. MacLeod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective attention and emotional vulnerability: Assessing the causal basis of their association through the experimental manipulation of attentional bias. Journal of Abnormal Psychology, 111(1), 107–123. MacLeod, C., Soong, L. Y., Rutherford, E. M., & Campbell, L. W. (2007). Internet-delivered assessment and manipulation of anxiety-linked attentional bias: Validation of a free-access attentional probe software package. Behavior Research Methods, 39(3), 533–538. Mathews, A., & Mackintosh, B. (1998). A cognitive model of selective processing in anxiety. Cogntive Therapy and Research, 22(6), 539–560. Mathews, A., & MacLeod, C. (1985). Selective processing of threat cues in anxiety states. Behaviour Research and Therapy, 23(5), 563–569. Mathews, A., & MacLeod, C. (2002). Induced processing biases have causal effects on anxiety. Cognition and Emotion, 16(3), 331–354. Mathews, A., & MacLeod, C. (2005). Cognitive vulnerability to emotional disorders. Annual Review of Clinical Psychology, 1, 167–195. Matsumoto, M., Weickert, C. S., Beltaifa, S., Kolachana, B., Jingshan, C., Hyde, T. M., et al. (2003). Catechol O-methyltransferase (COMT) mRNA expression in the dorsolateral prefrontal cortex of patients with schizophrenia. Neuropsychopharmacology, 28(8), 1521–1530. Mineka, S. (1985). Animal models of anxiety-based disorders: Their usefulness and limitations. In A.H. Tuma & J.D. Maser (Eds.), Anxiety and the Anxiety Disorders (pp. 199–244). Hillsdale, NJ: Erlbaum. Mineka, S., & Cook, M. (1986). Immunization against the observational conditioning of snake fear in Rhesus monkeys. Journal of Abnormal Psychology, 95(4), 307–318. Mineka, S., Cook, M., & Miller, S. (1984). Fear conditioned with escapable and inescapable shock: effects of a feedback stimulus. Journal of experimental psychology: Animal behavior processes, 10(3), 307–323. Mineka, S. & Oehlberg, K.A. (2008). The relevance of recent developments in classical conditioning

2/15/2011 7:42:35 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

to understanding the etiology and maintenance of anxiety disorders. Acta Psychologica, 127(3), 567–580. Mineka, S., & Öhman, A. (2002). Phobias and preparedness: the selective, automatic, and encapsulated nature of fear. Biological Psychiatry, 52(10), 927–937. Mineka, S., & Zinbarg, R. (1996). Conditioning and ethological models of anxiety disorders: Stress-in-dynamic-context anxiety models. In D. Hope (Ed.), Perspectives on Anxiety, Panic, and Fear: Nebraska Symposium on Motivation. Lincoln: University of Nebraska Press. Mineka, S., & Zinbarg, R. (2006). A contemporary learning theory perspective on the etiology of anxiety disorders: It’s not what you thought it was. American Psychologist, 61(1), 10–26. Mogg, K., & Bradley, B. P. (1998). A cognitivemotivational analysis of anxiety. Behaviour Research and Therapy, 36(9), 809–848. Mogg, K., & Bradley, B. P. (2005). Attentional bias in generalized anxiety disorder versus depressive disorder. Cognitive Therapy and Research, 29(1), 29–45. Mogg, K., Bradley, B. P., Dixon, C., Fisher, S., Twelftree, H., & McWilliams, A. (2000). Trait anxiety, defensiveness and selective processing of threat: An investigation using two measures of attentional bias. Personality and Individual Differences, 28(6), 1063–1077. Mogg, K., Bradley, B. P., & Hallowell, N. (1994). Attentional bias to threat: Roles of trait anxiety, stressful events, and awareness. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 47A(4), 841–864. Monk, C. S., Nelson, E. E., Woldehawariat, G., Montgomery, L. A., Zarahn, E., McClure, E. B., . . . Pine, D. S. (2004). Experience-dependent plasticity for attention to threat: Behavioral and neurophysiological evidence in humans. Biological Psychiatry, 56(8), 607–610. Monroe, S. M. (2008). Modern approaches to conceptualizing and measuring human life stress. Annual Review of Clinical Psychology, 4, 33–52. Monroe, S. M., & Reid, M. W. (2008). Geneenvironment interactions in depression research: Genetic polymorphisms and life-stress polyprocedures. Psychological Science, 19(10), 947–956. Mowrer, O. H. (1939). A stimulus-response analysis of anxiety and its role as a reinforcing agent. Psychological Review, 46(6), 553–565.

03-Schachtman-03.indd 75

75

Munafo, M. R., Brown, S. M., & Hariri, A. R. (2008). Serotonin transporter (5-HTTLPR) genotype and amygdala activation: A metaanalysis. Biological Psychiatry, 63(9), 852–857. Munafo, M. R., Clark, T., & Flint, J. (2004). Does measurement instrument moderate the association between the serotonin transporter gene and anxiety-related personality traits? A metaanalysis. Molecular Psychiatry, 10(4), 415–419. Munafo, M. R., Durrant, C., Lewis, G., & Flint, J. (2009). Gene x environment interactions at the serotonin transporter locus. Biological Psychiatry, 65(3), 211–219. Myers, K. M., & Davis, M. (2004). AX+, BX-discrimination learning in the fear-potentiated startle paradigm: Possible relevance to inhibitory fear learning in extinction. Learning and Memory, 11(4), 464–475. Nosek, B. A., & Banaji, M. R. (2001). The go/nogo association task. Social Cognition, 19(6), 625–664. Öhman, A. (1986). Face the beast and fear the face: Animal and social fears as prototypes for evolutionary analyses of emotion. Psychophysiology, 23(2), 123–145. Öhman, A., Carlsson, K., Lundqvist, D., & Ingvar, M. (2007). On the unconscious subcortical origin of human fear. Physiology and Behavior, 92(1-2), 180–185. Öhman, A., Flykt, A., & Esteves, F. (2001). Emotion drives attention: Detecting the snake in the grass. Journal of Experimental Psychology: General, 130(3), 466–478. Öhman, A., Lundqvist, D., & Esteves, F. (2001). The face in the crowd revisited: A threat advantage with schematic stimuli. Journal of Personality and Social Psychology, 80(3), 381–396. Öhman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108(3), 483–522. Orr, S. P., Metzger, L. J., Lasko, N. B., Macklin, M. L., Peri, T., & Pitman, R. K. (2000). De novo conditioning in trauma-exposed individuals with and without posttraumatic stress disorder. Journal of Abnormal Psychology, 109(2), 290–298. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87(6), 532–552. Peri, T., Ben-Shakhar, G., Orr, S. P., & Shalev, A. Y. (2000). Psychophysiologic assessment of aversive

2/15/2011 7:42:35 PM

76

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

conditioning in posttraumatic stress disorder. Biological Psychiatry, 47(6), 512–519. Pezawas, L., Meyer-Lindenberg, A., Drabant, E. M., Verchinski, B. A., Munoz, K. E., Kolachana, B. S., . . . Weinberger, D. R. (2005). 5-HTTLPR polymorphism impacts human cingulate-amygdala interactions: A genetic susceptibility mechanism for depression. Nature Neuroscience, 8(6), 828–834. Pﬂugshaupt, T., Mosimann, U. P., Wartburg, R., Schmitt, W., Nyffeler, T., & Müri, R. M. (2005). Hypervigilance-avoidance pattern in spider phobia. Journal of Anxiety Disorders, 19(1), 105–116. Pischek-Simpson, L. K., Boschen, M. J., Neumann, D. L., & Waters, A. M. (2009). The development of an attentional bias for angry faces following Pavlovian fear conditioning. Behaviour Research and Therapy, 47(4), 322–330. Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of Experimental Psychology, 32(1), 3–25. Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13(1), 25–42. Quinn, J. J., & Fanselow, M. S. (2006). Defenses and memories: Functional neural circuitry of fear and conditional responding. In M. G. Craske, D. Hermans, & D. Vansteenwegen (Eds.), Fear and learning: From basic processes to clinical implications (pp. 55–74). Washington, DC: APA Books. Raes, A. K., De Raedt, R., Fias, W., Koster, E. H. W., & Van Damme, S. (2009). Does contingency awareness mediate the inﬂuence of emotional learning on the cueing of visual attention? Psychological Research, 73(1), 107–113. Rapee, R. M., & Melville, L. F. (1997). Recall of family factors in social phobia and panic disorder: Comparison of mother and offspring reports. Depression and Anxiety, 5(1), 7–11. Rescorla, R. A. (1974). Effect of inﬂation of the unconditioned stimulus value following conditioning. Journal of Comparative and Physiological Psychology, 86(1), 101–106. Rescorla, R. A., & Heth, C. D. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal behavior processes, 104(1), 88–96. Revelle, W., & Oehlberg, K. (2008). Integrating experimental and observational personality

03-Schachtman-03.indd 76

research - The contributions of Hans Eysenck. Journal of Personality, 76(6), 1387–1414. Risch, N., Herrell, R., Lehner, T., Liang, K. Y., Eaves, L., Hoh, J., . . . Merikangas, K. R. (2009). Interaction between the serotonin transporter gene (5-HTTLPR), stressful life events, and risk of depression: A meta-analysis. Journal of the American Medical Association, 301(23), 2462–2471. Salemink, E., van den Hout, M. A., & Kindt, M. (2007). Selective attention and threat: Quick orienting versus slow disengagement and two versions of the dot probe task. Behaviour Research and Therapy, 45(3), 607–615. Schinka, J. A., Busch, R. M., & Robichaux-Keene, N. (2004). A meta-analysis of the association between the serotonin transporter gene polymorphism (5-HTTLPR) and trait anxiety. Molecular Psychiatry, 9(2), 197–202. Schmajuk, N. A., Lam, Y. W., & Gray, J. A. (1996). Latent inhibition: A neural network approach. Journal of Experimental Psychology: Animal Behavior Processes, 22, 321–349. Schmajuk, N. A., & Larrauri, J. A. (2006). Experimental challenges to theories of classical conditioning: Application of an attentional model of storage and retrieval. Journal of Experimental Psychology: Animal Behavior Processes, 32(1), 1–20. Schwartz, C. E., Snidman, N., & Kagan, J. (1999). Adolescent social anxiety as an outcome of inhibited temperament in childhood. Journal of the American Academy of Child and Adolescent Psychiatry, 38(8), 1008–1015. See, J., MacLeod, C., & Bridle, R. (2009). The reduction of anxiety vulnerability through the modiﬁcation of attentional bias: A real-world study using a home-based cognitive bias modiﬁcation procedure. Journal of Abnormal Psychology, 118(1), 65–75. Seligman, M. E. P. (1968). Chronic fear produced by unpredictable electric shock. Journal of Comparative and Physiological Psychology, 66(2), 402–411. Seligman, M. E. P., & Binik, Y. M. (1977). The safety signal hypothesis. Operant-Pavlovian Interactions, 165–187. Sen, S., Burmeister, M. L., & Ghosh, D. (2004). Meta-analysis of the association between a serotonin transporter promoter polymorphism (5-HTTLPR) and anxiety-related personality traits. American Journal of Medical Genetics, 127B(1), 85–89.

2/15/2011 7:42:35 PM

FEAR CONDITIONING AND ATTENTION TO THREAT

Smolka, M. N., Schumann, G., Wrase, J., Grusser, S. M., Flor, H., Mann, K., . . . Heinz, A. (2005). Catechol-O-methyltransferase val158met genotype affects processing of emotional stimuli in the amygdala and prefrontal cortex. Journal of Neuroscience, 25(4), 836–842. Stein, M. B., Fallin, M. D., Schork, N. J., & Gelernter, J. (2005). COMT polymorphisms and anxietyrelated personality traits. Neuropsychopharmacology, 30(11), 2092–2102. Stormark, K. M., & Hugdahl, K. (1996). Peripheral cuing of covert spatial attention before and after emotional conditioning of the cue. International Journal of Neuroscience, 86(3), 225–240. Stormark, K. M., Hugdahl, K., & Posner, M. I. (1999). Emotional modulation of attention orienting: A classical conditioning study. Scandinavian Journal of Psychology, 40(2), 91–99. Tipples, J., Young, A. W., Quinlan, P., Broks, P., & Ellis, A. W. (2002). Searching for threat. The Quarterly Journal of Experimental Psychology: Section A, 55(3), 1007–1026. Treit, D., Pesold, C., & Rotzinger, S. (1993). Dissociating the anti-fear effects of septal and amygdaloid lesions using two pharmacologically validated models of rat anxiety. Behavioral Neuroscience, 107(5), 770–785. Tunbridge, E. M., Bannerman, D. M., Sharp, T., & Harrison, P. J. (2004). Catechol-omethyltransferase inhibition improves setshifting performance and elevates stimulated dopamine release in the rat prefrontal cortex. Journal of Neuroscience, 24(23), 5331–5335. Uhrich, J. (1938). The social hierarchy in albino mice. Journal of Comparative Psychology, 25, 373–413. Van Damme, S., Crombez, G., Hermans, D., Koster, E. H. W., & Eccleston, C. (2006). The role of extinction and reinstatement in attentional bias to threat: A conditioning approach. Behaviour Research and Therapy, 44(11), 1555–1563. Van Damme, S., Lorenz, J., Eccleston, C., Koster, E. H. W., De Clercq, A., & Crombez, G. (2004). Fear-conditioned cues of impending pain facilitate attentional engagement. Neurophysiologie Clinique/Clinical Neurophysiology, 34(1), 33–39. Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009). Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition 1. Perspectives on Psychological Science, 4(3), 274–290.

03-Schachtman-03.indd 77

77

Waddell, J., Morris, R. W., & Bouton, M. E. (2006). Effects of bed nucleus of the stria terminalis lesions on conditioned anxiety: Aversive conditioning with long-duration conditional stimuli and reinstatement of extinguished fear. Behavioral Neuroscience, 120(2), 324–336. Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R. A. Boakes & M. S. Halliday (Eds.), Handbook of contemporary learning theories (pp. 23–64). Hillsdale, NJ: Erlbaum. Walker, D. L., & Davis, M. (1997). Anxiogenic effects of high illumination levels assessed with the acoustic startle response in rats. Biological Psychiatry, 42(6), 461–471. Watson, D., Gamez, W., & Simms, L. J. (2005). Basic dimensions of temperament and their relation to anxiety and depression: A symptombased perspective. Journal of Research in Personality, 39(1), 46–66. Watson, J. B., & Rayner, R. (1920). Conditioned emotional responses. Journal of Experimental Psychology, 3, 1–14. Weierich, M. R., Treat, T. A., & Hollingworth, A. (2008). Theories and measurement of visual attentional processing in anxiety. Cognition and Emotion, 22, 985–1018. Weiss, J. M. (1971). Effects of coping behavior in different warning signal conditions on stress pathology in rats. Journal of Comparative and Physiological Psychology, 77, 1–13. Williams, J. L., & Scott, D. K. (1989). Inﬂuence of conspeciﬁc and predatory stressors and their associated odors on defensive burying and freezing responses. Animal Learning and Behavior, 17(4), 383–393. Williams, J. M. G., Mathews, A., & MacLeod, C. (1996). The emotional Stroop task and psychopathology. Psychological Bulletin, 120(1), 3–24. Williams, J. M. G., Watts, F. N., MacLeod, C., & Mathews, A. (1997). Cognitive psychology and emotional disorders (2nd ed.). Chichester, England: Wiley. Wilson, E., & MacLeod, C. (2003). Contrasting two accounts of anxiety-linked attentional bias: Selective attention to varying levels of stimulus threat intensity. Journal of Abnormal Psychology, 112(2), 212–218. Winslow, J. T., Noble, P. L., & Davis, M. (2008). AX+/BX discrimination learning in the fearpotentiated startle paradigm in monkeys. Learning and Memory, 15(2), 63.

2/15/2011 7:42:35 PM

78

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Winterer, G., & Goldman, D. (2003). Genetics of human prefrontal function. Brain Research Reviews, 43(1), 134–163. Wolpe, J. (1968). Psychotherapy by reciprocal inhibition. Integrative Psychological and Behavioral Science, 3(4), 234–240.

03-Schachtman-03.indd 78

Zinbarg, R. E., & Mohlman, J. (1998). Individual differences in the acquisition of affectively valenced associations. Journal of Personality and Social Psychology, 74(4), 1024–1040.

2/15/2011 7:42:35 PM

CHAPTER 4 Behavioral Techniques to Reduce Relapse After Exposure Therapy Applications of Studies of Experimental Extinction Mario A. Laborda, Bridget L. McConnell, and Ralph R. Miller

Pavlovian phenomena have long served as models for the etiology, treatment, and relapse from treatment of diverse disorders (e.g., phobias, addictions). Here we brieﬂy review Pavlovian conditioning models of anxiety disorders, experimental extinction models of exposure therapy, and recovery from extinction models of relapse following exposure therapy. We then focus on how research on experimental extinction has led to the development of speciﬁc behavioral techniques to reduce recovery from extinction and hence relapse from exposure therapy. These techniques include conducting extinction treatment in multiple contexts, giving a massive amount of extinction, increasing the time between extinction trials and between extinction sessions, administering extinction in the presence of a second excitor, and testing in the presence of a retrieval cue from extinction. It is concluded that these behavioral techniques, all of which were discovered in the experimental laboratory, are potent and important tools to be considered by psychotherapists trying to make their patients less susceptible to relapse.

The pioneering work of Pavlov (1927) laid the groundwork for the development of many successful models of select psychopathologies. Pavlovian phenomena have subsequently served as models of the etiology, treatment, and relapse of numerous disorders. In this chapter we brieﬂy review some of these roles of Pavlovian conditioning in modeling behavioral/mental disorders, and then focus on how the empirical study of extinction has resulted in the development of behavioral techniques that are applicable by clinicians who are trying to decrease relapse after exposure therapy. We will recurrently use fear and anxiety disorders as examples of how Pavlovian conditioning has been used as a model; however, these are only a few of the disorders for which these Pavlovian techniques are useful in preventing relapse after exposure therapy.

EARLY ASSOCIATIVE ACCOUNTS OF THE ETIOLOGY OF PSYCHOPATHOLOGY Pavlov’s (1927) discovery of so-called experimental neurosis represents one of the ﬁrst known experimental demonstrations of learned emotional responses, and it constitutes the ﬁrst associative model for the etiology of anxiety disorders. Pavlov and his colleagues showed how different environmental manipulations provoked learned emotional responses in dogs. In one exemplar study (Shenger-Krestinikova, 1921, cited in Pavlov, 1927), they showed that, after their subjects mastered a two-stimulus visual discrimination (i.e., training the subjects to respond to one of two visual stimuli for reinforcement), making this discrimination gradually more difﬁcult

79

04-Schachtman-04.indd 79

2/15/2011 7:42:52 PM

80

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(i.e., making the stimuli more similar) caused their subjects to display behaviors characteristic of human neurosis (e.g., erratic and aggressive behaviors). These results were soon extended using other manipulations (for a review of these different manipulations and their commonalities, see Mineka & Kihlstrom, 1978), and they were also replicated with children as experimental subjects (Krasnogorsky, 1925). Consequently, select psychopathologies were viewed, for the ﬁrst time, as conditions highly inﬂuenced by external stimuli and learning processes, discouraging the view that all psychopathology was based on internal states, structures, and hypothetical conﬂicts (e.g., the psychoanalytical point of view). As a testament of Pavlov’s inﬂuence, later models of the etiology and treatment of anxiety disorders were developed based on his early research (e.g., Wolpe, 1958). In a Pavlovian learning paradigm, two stimuli are paired; one of them elicits responding prior to any speciﬁc treatment (i.e., an unconditioned stimulus [US] that elicits an unconditioned response [UR]), and the other does not initially evoke the response in question (i.e., a neutral stimulus). However, when the stimuli are presented contiguously, the once neutral stimulus begins evoking a response [usually] similar to those evoked by the US. In other words, the neutral stimulus becomes a conditioned stimulus (CS) and begins eliciting a conditioned response (CR). This simple associative account of the acquisition of fear and phobias is exempliﬁed by Watson and Rayner’s (1920) demonstration of an infant’s learned fear response to a rat after the rat was paired with a loud noise. Here the rat served as a neutral stimulus that did not initially evoke a fear response by the infant. However, when the presentation of the rat (i.e., the CS) was followed by a stimulus that initially evoked a fear response (i.e., the loud noise, which served as the US), the infant seemingly associated them and began emitting conditioned fear responses upon presentation of the rat, similar to those evoked by the US. Generalizing from this demonstration, Watson and Rayner suggested that “It is probable that many of the phobias in psychopathology are true conditioned emotional reactions. . .” (p. 14). This simple

04-Schachtman-04.indd 80

conditioning model served as the foundation for more modern associative accounts of the etiology of fear and anxiety disorders, which has been expanded to include factors that can account for individual differences in associative learning, including personality/temperament factors, experiential factors, and evolutionary variables (e.g., Mineka & Oehlberg, 2008; Mineka & Sutton, 2006; Mineka & Zinbarg, 2006).

EXPERIMENTAL EXTINCTION AS A MODEL OF EXPOSURE THERAPY After an association between a CS and a US has been formed through contiguous activation of the mental representation of these stimuli, repeated presentations of the CS in the absence of the US decreases the conditioned responding it elicits. This manipulation is one of the most widely studied phenomena of associative learning and is referred to as experimental extinction (Pavlov, 1927; for reviews, see Delamater, 2004; Rescorla, 2001). In addition to the importance of the empirical study of extinction for assessment of general theories of associative learning (e.g., Gallistel & Gibbon, 2000; Rescorla & Wagner, 1972; Stout & Miller, 2007; Wagner, 1981), extinction has also been pivotal as an associative model of exposure therapies (e.g., Bouton, 2000; Bouton & Nelson, 1998; Hofmann, 2008). In exposure therapy for a speciﬁc phobia, the phobic object is considered a CS, which, if repeatedly presented without the aversive stimulus (i.e., the US) loses its potential to elicit a fear response. This associative model has received extensive attention from researchers and, as a result, today exposure therapy is one of the best empirically supported treatments for speciﬁc phobias and other anxiety disorders (Chambless et al., 1996; Chambless & Ollendick, 2001). Despite the success of extinction in decreasing conditioned responding, evidence indicates that extinction does not erase the original association between a CS and a US as some associative models have proposed (e.g., Rescorla & Wagner, 1972). Instead, experimental extinction is assumed to result in the formation of an inhibitory-like association (for reviews, see Bouton, 1993, 2000,

2/15/2011 7:42:52 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

2004), which has led researchers to return to Pavlov’s (1927) view of extinction as an inhibitory learning experience that creates a memory which competes with the original excitatory association for behavioral control. This view of extinction building new associations instead of erasing them has been incorporated into some associative models (e.g., Pearce & Hall, 1980). In the same manner that extinction does not erase the original learned associations, exposure therapy for anxiety disorders does not fully destroy the potential of the phobic object to elicit responding, and consequently relapse after treatment is common (Craske, 1999). This, of course, is problematic, given the long-term goals of therapy.

SOME ASSOCIATIVE MODELS OF RELAPSE AFTER EXPOSURE TREATMENT Several associative phenomena have been widely cited to support the new-learning (as opposite to erasure) account of extinction, and most of them also serve as evidence of the susceptibility of extinction to recovery. For example, postextinction presentations of the US often induce partial recovery of the extinguished CR (reinstatement; e.g., Bouton & Bolles, 1979b; Rescorla & Heth, 1975), and retraining of an extinguished cue is usually faster than training a novel cue (rapid reacquisition; e.g., Napier, Macrae, & Kehoe, 1992; Pavlov, 1927; Ricker & Bouton, 1996). More widely studied, a long delay between extinction treatment and testing has been seen to provoke partial recovery of the extinguished CR (spontaneous recovery; e.g., Brooks & Bouton, 1993; Pavlov, 1927), and partial recovery of the extinguished CR also occurs when testing takes place outside the extinction context (renewal; e.g., Bouton & Bolles, 1979a; Bouton & King, 1983; Bouton & Ricker, 1994). Given their similarities with relapse situations outside the laboratory (where relapse often occurs in a context different from the one used during treatment and often after a period of time has elapsed since the end of treatment), spontaneous recovery and renewal will be reviewed in more detail.

04-Schachtman-04.indd 81

81

There are three types of renewal that differ based on the contexts in which acquisition, extinction, and testing take place. ABA renewal is the recovery of an extinguished CR when subjects are tested in the acquisition context (A) after extinction treatment in a different context (B; e.g., Bouton & King, 1983). ABC renewal is the recovery of an extinguished CR that occurs when acquisition (Context A), extinction (Context B), and testing (Context C) all take place in different contexts (e.g., Bouton & Bolles, 1979a). AAC renewal is the recovery of an extinguished CR when acquisition and extinction occurs in the same context but testing occurs in a different context, which is associatively neutral (C; Bouton & Ricker, 1994). Most evidence suggests that ABA and ABC renewal result in more recovery from extinction than AAC renewal, which is sometimes completely ineffectual (e.g., Laborda, Witnauer, & Miller, in press; Rescorla, 2008; Tamai & Nakajima, 2000; Thomas, Larsen, & Ayres, 2003; Üngör & Lachnit, 2008). There are several theoretical accounts to explain why renewal of extinguished responses occurs when testing takes place outside the context of extinction. For example, according to Pearce’s (1987) conﬁgural model, subjects do not process stimuli elementally; rather, they process the entire perceptual ﬁeld as a single conﬁgural stimulus. Generalization occurs between conﬁgured stimuli to the extent that they are similar. In this framework, ABA renewal occurs because the target cue and the acquisition context are processed as a single conﬁgured cue that acquires an excitatory association with the US during acquisition treatment. Then, because the target cue provides similarity between conﬁgural units representing the acquisition and extinction trials, some of the strength of the excitatory conﬁgured cue generalizes to the extinction conﬁgured cue (i.e., the target and extinction contexts), creating a situation in which this second conﬁguration develops an inhibitory association with the US (because of the unfulﬁlled expectancy of US occurrence). When testing occurs in the acquisition context, the excitatory strength of the conﬁgured cue formed by the target cue and the acquisition context is only partially reduced by the generalization of inhibition from

2/15/2011 7:42:52 PM

82

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

the compound formed by the extinction context and the target cue, thereby producing a recovery of responding. Pearce’s model anticipates ABC renewal due to the inhibitory properties of the conﬁgured extinction cue (composed of the target cue and extinction context) not fully generalizing to the test situation, due to the absence of the extinction context at test. According to the Rescorla and Wagner (1972) model, during acquisition training in Context A both the target cue and the context acquire elemental excitatory associations with the US. Then, when subjects receive extinction training in a different context, the CS predicts the US and its absence causes the extinction context to develop an inhibitory relationship with the US. Importantly, summation of the excitation from the target cue with inhibition from the extinction context results in little stimulus control of behavior in the extinction context. Moreover, the inhibitory-like status of the extinction context protects the CS for further extinction (e.g., Lovibond, Davis, & O’Flaherty, 2000; McConnell & Miller, 2010; Rescorla, 2003). When subjects are tested back in the acquisition context, its excitatory association summates with the remaining excitatory association of the target cue provoking ABA renewal. Similarly, ABC renewal is anticipated because of the absence of the inhibitory status of Context B when testing in Context C, thereby allowing for more excitation relative to the ABB control group. The ABB group exhibits low responding presumably because at test it is under the inﬂuence of the inhibitory Context B. As stated elsewhere (Laborda et al., in press), although the comparator hypothesis (Stout & Miller, 2007) does not account for renewal (i.e., ABB < ABC; ABB < ABA; AAA < AAC) because it lacks of a rule to allow summation between the testing context and the target stimulus, there is no reason to think that such summation does not actually take place. The implementation of such summation rule would permit SOCR to predict renewal in a way similar to that of Rescorla and Wagner. From another perspective, Bouton’s (1993, 1994, 1997) retrieval model suggests that recovery from extinction occurs for two reasons: First, extinction learning is akin to inhibition learning, that is, learning when an outcome will not occur,

04-Schachtman-04.indd 82

and memories that support inhibitory learning are thought to be more labile than excitatory memories. Second, extinction learning involves the creation of ambiguity (because it contradicts the previously acquired knowledge concerning reinforcement of the cue), and ambiguity presumably fosters context-speciﬁc encoding of the extinction information as a means of resolving the ambiguity. That is, when two types of inconsistent information (i.e., reinforcement and nonreinforcement of a cue) are sequentially acquired, retrieval of the second-learned information depends on the spatial (renewal) and temporal (spontaneous recovery) test context matching that of the second-learned treatment (i.e., extinction). Thus, moving outside the context of second learning (i.e., the extinction context) should cause a failure to retrieve the second-learned information because it is inhibitory (in the case of extinction) and because it was coded as context speciﬁc due to its ambiguity (in contrast to the unambiguous ﬁrstlearned memory; i.e., according to Bouton’s model, ambiguous memories do not transfer well between contexts). These mechanisms conjointly account for ABA and ABC renewal. Rosas and colleagues (Rosas, Callejas-Aguilera, RamosÁlvarez, & Abad, 2006) recently revised the second mechanism of Bouton’s (1993, 1994, 1997) retrieval model. For them it is not ambiguity that makes second-learned information become context speciﬁc, but any manipulation that makes organisms to pay attention to the context (e.g., ambiguity, previous experience focusing on the context, instructions to pay attention to the context, informational value of the context, and the salience of the context relative to the target cue). According to Rosas and his colleagues, in the case of ABA and ABC renewal, the ambiguity experienced during extinction treatment (i.e., the CS acquiring a second meaning) makes subjects attend to the context, thereby encoding the extinction information as speciﬁc to the extinction context. When testing occurs in the acquisition context or a neutral context, extinction learning does not generalize to test because it was encoded as an “exception” occurring in the extinction context, thereby producing renewal. In general, Bouton’s approach has been successful

2/15/2011 7:42:52 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

in predicting several recovery phenomena and related data. For example, it accounts for AAC renewal being weaker than ABA and ABC renewal by noting that initial acquisition should interfere with the extinction context becoming a conditioned inhibitor for the AAC condition, but not the ABC or ABA conditions. Like renewal, there are several theoretical accounts to explain why spontaneous recovery occurs. For example, Skinner (1950) proposed that acquisition cues, such as handling of the experimental subjects, were not properly extinguished and those cues provoke the recovery of extinguished CRs at a delayed test. Robbins (1990) proposed another account in which a loss of attention to the CS during extinction wanes over a retention interval, thereby producing spontaneous recovery (but see Bouton & Peck, 1992, for difﬁculties with this account). More recently, Devenport (1998) used a temporal weighting rule to explain spontaneous recovery. This theory states that events are weighted differently depending on their recency. Recent events are given more weight in determining behavior than remote events. But as time elapses between extinction treatment and testing, the subject weights the acquisition and extinction phases more evenly. Thus, for an experimental group in which a long delay has been imposed between the extinction treatment and testing, subjects give relatively more weight to what was learned before extinction treatment (i.e., a high level of responding) than if no retention interval was imposed. Bouton (1993; 2010) proposed a related account in which spontaneous recovery occurs because changes of the so-called temporal context can produce similar effects to those produced by changes in physical contexts (i.e., renewal). For Bouton, renewal and spontaneous recovery are examples of the same phenomenon. In this framework, like renewal, spontaneous recovery is due to some combination of two factors. First, inhibitory-like information (extinction) does not generalize as well as excitation (from acquisition) to the time of test when testing is appreciably delayed. Also, second-learned information (extinction in this case) is temporally more context dependent than ﬁrst-learned information (acquisition in this case), making

04-Schachtman-04.indd 83

83

second-learned information less likely to transfer to a different temporal context. Rescorla (2005) suggested that the former was the major factor responsible for spontaneous recovery from extinction. But Sissons and Miller (2009) identiﬁed a lack of equivalence between Rescorla’s inhibitory information and his excitatory information. With this corrected (i.e., using the same number of trials in both phases of the experiment, using Pavlovian conditioned inhibition treatments in each phase as the nonreinforced treatment, and using a nontarget excitor for inhibition training to decrease the possibility of indirectly diminishing the inhibitory status of the stimulus trained as an inhibitor when nonreinforcing the target excitor in phase 2), Sissons and Miller’s data suggested that extinction being the second-learned information contributes more to spontaneous recovery than does the inhibitorylike nature of extinction. In addition to the vast number of reports of recovery of extinguished CRs in the (nonhuman) animal learning literature, these phenomena have also been observed in humans. For example, Vila and Rosas (2001; see also GarcíaGutiérrez & Rosas, 2003; Rosas & CallejasAguilera, 2006; Rosas, Vila, Lugo, & Lopez, 2001) reported recovery phenomena in a human contingency learning task. Using a ﬁctitious story relating a medicine (cue) with a deleterious side effect (outcome), they demonstrated acquisition of an excitatory relationship between the medicine and the side effect, extinction of this association, and then recovery of it when testing occurred in a context different than the extinction context (i.e., renewal) or when a retention interval was imposed between extinction and testing (i.e., spontaneous recovery). Similarly, Van Gucht, Vansteenwegen, Beckers, and Van den Bergh (2008) showed that chocolate cravings are susceptible to relapse when tested in the acquisition context after extinction in another context (i.e., ABA renewal). In a related line of research, Vansteenwegen et al. (2005; see also Effting & Kindt, 2007; Vansteenwegen et al., 2006; Vansteenwegen et al., 2007) reported recovery phenomena in a human fear-conditioning preparation. Using a differential conditioning paradigm, a picture of a face (i.e., the CS) was

2/15/2011 7:42:52 PM

84

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

paired with a loud aversive noise (i.e., the US). After training this association in one context and extinguishing it in another, returning subjects to the acquisition context to test responding to the CS resulted in renewal of the extinguished CRs, compared with a group in which all phases took place in the same context. In more applied situations, these extinction and the recovery phenomena have been replicated when social drinkers (Collins & Brandon, 2002; but see MacKillop & Lisman, 2008; Stasiewicz, Brandon, & Bradizza, 2007) and arachnophobic students (Mineka, Mystkowski, Hladek, & Rodriguez, 1999; Mystkowski, Craske, & Echiverri, 2002; Rodriguez, Craske, Mineka, & Hladek, 1999) were used as participants. For example, Collins and Brandon (2002) reported a recovery of extinguished reactivity of the urge to drink and extinguished salivatory responses to alcohol-related cues in undergraduate social drinkers. In this study all participants received extinction of alcohol-related cues (e.g., a beer can) in one context, decreasing salivation and urge to drink. Participants tested in a different context showed renewal of the extinguished CRs relative to participants tested in the extinction context. As previously mentioned, similar results have been found with subclinical participants. As an illustration, Mystkowski et al. (2002) conﬁrmed previous ﬁndings of recovery of extinguished fear responses in arachnophobic undergraduates (Mineka et al., 1999; Rodriguez et al., 1999). In this research, all participants received an exposure session in one context and were tested 1 week later in the same and a different context. An important feature of this study is that both contexts were fully counterbalanced real-world settings. Self-report data showed that fear was much higher when testing occurred in a context different from the one used during the extinction session, thereby extending the renewal and spontaneous recovery literature to phobic participants.

BEHAVIORAL TECHNIQUES TO REDUCE RECOVERY AFTER EXTINCTION The evidence summarized earlier is clear in suggesting that neither experimental extinction nor

04-Schachtman-04.indd 84

exposure therapy erases memories. Rather, they establish inhibitory-like associations that are more context dependent than the original excitatory association (e.g., Bouton, 1993). Relapse after exposure therapy is one of the greatest problems confronting psychotherapy. However, research concerning experimental extinction has identiﬁed some ways to prevent, or at least attenuate, the recovery of extinguished CRs. Recently, Boschen, Neumann, and Waters (2009, p. 97; see also Rachman, 1978) proposed 10 recommendations for preventing relapse after successful exposure treatment of anxiety-related disorders: (1) Extend the duration of exposure sessions past the point of habituation (i.e., past the point in which anxiety is no longer reported in a given exposure session); (2) Increase the overall number of sessions, continuing past successful extinction of the anxiety response across sessions (i.e., past the point in which anxiety is no longer reported at the beginning of the next therapy session); (3) Use massed exposure sessions with short durations between sessions; (4) Encourage patients to relinquish distraction techniques, and do not train patients in using these as part of treatment (i.e., during exposure treatment the anxiety-eliciting stimulus must be attended to; anxiety diminution could be reduced if patients are actively avoiding the situation); (5) Utilize an assortment of fear stimuli during exposure; (6) Conduct exposure in a variety of different environments and contexts; (7) Ensure that exposure tasks are sufﬁcient to elicit anxiety, but do not place excessive demand on the patient; (8) Homework tasks should be used to consolidate treatment and reduce probability of relapse; (9) Where possible, prior to reencountering the phobic stimulus, patients should attempt to recall (mentally reinstate) the treatment context; (10) Cognitive restructuring should be used to assist patients to recognize, reevaluate, and restructure associations between their phobic stimulus and negative outcomes. Most of these recommendations were directly or indirectly derived from experimental extinction research. Although this list is suggestive, most of the procedures have not yet been well researched, thereby raising doubts concerning which ones really work and under what conditions, if any, they are effective.

2/15/2011 7:42:53 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

Another promising area of research on behavioral techniques to prevent relapse is the study of pharmaceutical cognitive enhancers. For example, the use of d-cycloserine (DCS), a partial agonist of NMDA receptors, has been shown with non-human subjects to enhance extinction of freezing conditioned responses (Ledgerwood, Richardson, & Cranney, 2003), fear-potentiated startle (Walker, Ressler, Lu, & Davis, 2002), conditioned taste aversion (Davenport & Houpt, 2009), and cocaine-induced conditioned place preference (Thanos, Bermeo, Wang, & Volkow, 2009). Also, the use of DCS with humans has been reported to enhance the effects of exposure therapies on social phobia (Guastella et al., 2008; Hofmann et al., 2006), acrophobia (Ressler et al., 2004), and some cases of obsessive-compulsive disorder (Kushner et al., 2007). Adding to this body of data, there is also evidence of DCS attenuating reinstatement of extinguished fear CRs (Ledgerwood, Richardson, & Cranney, 2004) and extinguished cocaine-induced conditioned place preference (Paolone, Botreau, & Stewart, 2009). However, these successes are tempered by reported failures of DCS to enhance extinction of arachnophobia (Guastella, Dadds, Lovibond, Mitchell, & Richardson, 2007), human electrodermal responses (Guastella, Lovibond, Dadds, Mitchell, & Richardson, 2007), and some cases of exposure therapy for obsessivecompulsive disorder (Storch et al., 2007). Of more central interest here is that DCS has failed to prevent renewal of extinguished conditioned fear suppression (Bouton, Vurbic, & Woods, 2008; Woods & Bouton, 2006), and rapid reacquisition of extinguished freezing conditioned responses (Ledgerwood, Richardson, & Cranney, 2005) in rats. There is clearly a need to extend research on the effects of pharmaceutical cognitive enhancers on extinction and recovery situations; however, extended discussion of this area of investigation is beyond the scope of the present review. Next we review some behavioral techniques, discovered and extensively studied in the experimental laboratory, that have been shown to be robust and potentially useful tools to be considered by psychotherapists trying to make their treatments less susceptible to relapse.

04-Schachtman-04.indd 85

85

Massive Extinction

Denniston, Chang, and Miller (2003) compared the effects of moderate and massive extinction training in attenuating ABC and ABA renewal, using rats as subjects in a conditioned lick suppression paradigm. After a white noise (i.e., CS) was paired with a footshock (i.e., US) in one context (A), subjects received moderate (160) or massive (800) amounts of extinction trials in a second context (B). Notably, 160 extinction trials have been shown to be sufﬁcient to eliminate conditioned suppression in their preparation. When subjects were later tested for conditioned fear responses in a neutral context (C; Experiment 1; see Fig. 4.1A) and in the acquisition context (Experiment 2; see Fig. 4.1B), those that received a massive number of extinction trials showed attenuated renewal of the extinguished responses compared with subjects that received only a moderate number of extinction trials. These positive results were recently replicated in a similar experimental situation (Laborda & Miller, 2010). However, Thomas, Vurbic, and Novak (2009), Rauhut, Thomas, and Ayres (2001), and Tamai and Nakajima (2000) failed to see reduction in ABA renewal with increasing number of extinction trials, but their large number of trials (144, 100, and 112, respectively) were decidedly less than in the studies that obtained reduced renewal (800 and 810). This suggests that insufﬁcient extinction might account for the failures to obtain this effect. It should be noted that Tamai and Nakajima successfully reduced AAC renewal with only 112 extinction trials, which is in concordance with the evidence suggesting that this type of renewal is weaker than ABA and ABC renewal. Foa et al. (2005; see also Gillihan and Foa, this volume, chapter 2) observed reduced relapse with humans treated for posttraumatic stress disorder when prolonged exposure was used. Prolonged exposure can be viewed as having a similar effect as many extinction trials because both treatments increase time spent in the presence of the CS. Future research should determine whether massive extinction can prevent other recovery situations and whether these results can be replicated using a variety of experimental paradigms and subjects.

2/15/2011 7:42:53 PM

86

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS 2.7 2.5

Panel A

Mean time (log s)

2.3 2.1 1.9 1.7 1.5 1.3 1.1 0.9 0.7 No extinction (A–C)

Moderate extinction (ABC)

Massive extinction (ABC)

Moderate extinction (ABB)

Massive extinction (ABA)

Moderate extinction (AAA)

Groups 2.7 2.5

Panel B

Mean time (log s)

2.3 2.1 1.9 1.7 1.5 1.3 1.1 0.9 0.7 No extinction (A–A)

Moderate extinction (ABA) Groups

Figure 4.1 Panel A: Mean log time to complete 5 cumulative seconds of licking in the presence of the

target conditioned stimulus in a neutral context (C). Brackets represent standard error of the mean. Higher scores indicate more conditioned suppression. No extinction (A-C) = group that received no extinction trials and was tested in a neutral but familiar context (C); Moderate extinction (ABC) = group that received a moderate number of extinction trials (160) and was tested in a neutral but familiar context (C); Massive extinction (ABC) = group that received a massive number of extinction trials (800) and was tested in a neutral but familiar context (C); Moderate extinction (ABB) = group that received a moderate number of extinction trials (160) and was tested in the extinction context (B). See text for further explanation. (From J. C. Denniston, R. C. Chang, & R. R. Miller, 2003 Experiment 1, Learning and Motivation, 34, 68–86.) Panel B: Mean log time to complete 5 cumulative seconds of licking in the presence of the target conditioned stimulus in the acquisition context (A). Brackets represent standard error of the mean. Higher scores indicate more conditioned suppression. No extinction (A-A) = group that received no extinction trials; Moderate extinction (ABA) = group that received a moderate number of extinction trials (160) in a context different from the one used for both acquisition and testing; Massive extinction (ABA) = group that received a massive number of extinction trials (800) in a context different from the one used for both acquisition and testing; Moderate extinction (AAA) = group that received a moderate number of extinction trials (160) in the acquisition context. See text for further explanation. (From J. C. Denniston, R. C. Chang, & R. R. Miller. 2003. Experiment 2. Learning and Motivation, 34, 68–86).

Extinction in Multiple Contexts

Bouton (1991) suggested that another possible manipulation to reduce recovery after extinction could be to conduct extinction treatment in several contexts. He reasoned that if an association is extinguished in multiple contexts, the greater

04-Schachtman-04.indd 86

number of features present in multiple extinction contexts than in a single extinction context may help to generalize extinction learning to contexts other than those used for extinction treatment. In line with this operational suggestion, Gunther, Denniston, and Miller (1998) evaluated the effect of extinction treatment in

2/15/2011 7:42:53 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

the number of contexts used during acquisition and extinction were equated. This ﬁnding has implications for the treatment of patients who have suffered multiple traumas (e.g., war combatants and victims of multiple abuses) or consume drugs in different places and situations (e.g., smokers and drinkers). For these people, extinction in multiple contexts could prove hard to implement because of the number of extinction contexts that would be necessary to compensate for the multiple acquisition contexts. To date, Gunther et al.’s (1998, Experiment 1) results have been replicated several times and in a number of different situations (e.g., Bandarian Balooch & Neumann, 2011; Chaudhri, Sahuque, & Janak, 2008; Chelonis, Calton, Hart, & Schachtman, 1999; Glautier & Elgueta, 2009; Laborda & Miller, 2010; Neumann, 2006; Pineño & Miller, 2004; Thomas et al., 2009; Vansteenwegen et al., 2007; but see Betancourt et al., 2008; Bouton, García-Gutiérrez, Zilski, & Moody, 2006; Neumann, Lipp, & Cory, 2007,

2.7

2.7

2.5

2.5

2.3

2.3 Mean time (log s)

Mean time (log s)

multiple contexts on ABC renewal, using rats as subjects in a conditioned lick suppression paradigm. After a white noise (i.e., CS) and a mild shock (i.e., US) were paired in one context, subjects received extinction trials in one or three different contexts (all different from that of conditioning and that of testing). As depicted in Figure 4.2, rats that received extinction treatment in three different contexts showed less renewal of the extinguished CR when tested in an associatively neutral context (i.e., ABC renewal) than rats that received the same amount of extinction treatment but in only one context. Gunther et al. went a step further and also presented data delineating the situations in which this technique can be effective in attenuating renewal. In their Experiment 2, they found that the number of contexts in which acquisition took place limited the effectiveness of extinction in multiple contexts. As depicted in Figure 4.3, the effect of extinction in multiple contexts decreasing ABC renewal was eliminated when

87

2.1 1.9 1.7 1.5 1.3 1.1

2.1 1.9 1.7 1.5 1.3 1.1 0.9

0.9

0.7

0.7 No extinction (A[---]E)

Extinction-1 (A[BBB]E)

Extinction-3 (A[BCD]E)

Acquisition-1/Extinction-3

Acquisition-3/Extinction-3

Groups

Groups

Figure 4.3 Mean log time to complete 5 cumulaFigure 4.2 Mean log time to complete 5 cumula-

tive seconds of licking in the presence of the target conditioned stimulus in a neutral context (E) for all groups. Brackets represent standard error of the mean. Higher scores indicate more conditioned suppression. No extinction (A[—]E) = group that received no extinction trials; Extinction-1 (A[BBB] E) = group that received a moderate number of extinction trials (162) in only one context (B); Extinction-3 (A[BCD]E) = group that received a moderate number of extinction trials (162) in three different contexts (B, C, and D). See text for further explanation. (From L. M. Gunther, J. C. Denniston, & R. R. Miller. 1998. Experiment 1. Behaviour Research and Therapy, 36, 75–91).

04-Schachtman-04.indd 87

tive seconds of licking in the presence of the target conditioned stimulus in a neutral context (E) for all groups. Brackets represent standard error of the mean. Higher scores indicate more conditioned suppression. Acquisition-1 / Extinction-3 = group that received acquisition in one context and a moderate number of extinction trials (162) across three different contexts. Acquisition-3 / Extinction-3 = group that received the same amount of acquisition but in three different contexts and a moderate number of extinction trials (162) also across three different contexts. See text for further explanation. (From L. M. Gunther, J. C. Denniston, & R. R. Miller. 1998. Experiment 2. Behaviour Research and Therapy, 36, 75–91).

2/15/2011 7:42:53 PM

88

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

for negative results). For example, Chelonis et al. (1999) reported successful prevention of ABA renewal in a conditioned taste aversion preparation with rats as subjects. After a single pairing of sucrose (i.e., CS) with lithium chloride (i.e., US) in one context, subjects received extinction trials in one or three new contexts. The results indicated that extinction in only one context was more prone to renewal than when the same amount of extinction was administered in multiple contexts. Neumann extended the previous results to ABA and ABC renewal situations in a conditioned suppression task with humans as subjects. Moreover, two studies found reduced renewal from similar manipulations in arachnophobic participants. Vansteenwegen et al. (2007) reported successful attenuation of renewal of fear responses after extinction in multiple contexts. They exposed arachnophobic participants to a videotaped spider in one or three different locations within a house. When tested with the videotaped spider in a novel location of the house (i.e., an A[BCD]E-like design), participants who received exposure of a videotaped spider in only one location displayed more renewal of the extinguished CRs than participants who received the same amount of exposure to the videotaped spider in three different locations in the house. Finally, Rowe and Craske (1998b) showed attenuated return of fear using an interesting manipulation analogous to the extinction in multiple contexts procedure. The authors exposed arachnophobic participants to one or three different tarantulas during extinction treatment. When tested 3 weeks later, participants who were exposed to three different tarantulas during treatment showed less return of fear than participants that were exposed to only one type of tarantula. These results further support the view that extinction in the presence of multiple features can facilitate generalization of extinction learning to new situations. Overall, extinction treatment in multiple contexts, at least with some parameters, has been found to be an effective way to reduce relapse of extinguished CRs after context shifts (i.e., renewal) and delayed testing (i.e., spontaneous recovery). Future research should test the effectiveness of this manipulation in reducing

04-Schachtman-04.indd 88

relapse caused by other factors (e.g., reinstatement, rapid reacquisition), evaluate possible theoretical explanation of these results, and test its effectiveness in more clinical settings. Massive Extinction in Multiple Contexts

Thomas et al. (2009) recently reported that there is a summation of the effects of extinction in multiple contexts and large number of extinction trials in reducing renewal of extinguished CRs in conditioned barpress suppression preparation. After pairing the momentary absence of an otherwise present white noise (i.e., the CS) with a footshock (i.e., the US), rats received 36 or 144 extinction trials in Context A. Orthogonal to this manipulation of number of extinction trials, subjects received extinction in either one or three different contexts with the total number of extinction trials held constant (Experiment 2). The results suggested that extinction in multiple contexts was effective in attenuating renewal when subjects were tested in the acquisition context (i.e., A[BCD]A treatment resulted in less renewal than did A[BBB]A treatment), but only when a large number of extinction trials were administered. Thus, extinction in multiple contexts and massive extinction do appear to summate at least in some circumstances. This may offer an explanation for why some researchers have failed to observe reduced renewal after extinction in multiple contexts. For example, Bouton et al. (2006) gave 12 extinction trials compared to Gunther et al.’s (1998) 160 trials. Extending Thomas et al.’s (2009) results, Laborda and Miller (2010) have reported a similar summative effect of extinction in multiple contexts and massive extinction when a more naturalistic relapse situation was used. In therapeutic situations, relapse often occurs when subjects are exposed to the fearful stimulus in a context different than the one in which treatment took place, and after some time has elapsed since the last exposure session. In other words, relapse often occurs after a delayed context shift, which can be thought of as a situation in which the renewal effect summates with the spontaneous recovery of extinguished responses. Similar additive effects of context change and retention

2/15/2011 7:42:53 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

interval have previously been reported in latent inhibition (Rosas & Bouton, 1997), in an interference task in a human causal judgment preparation (Rosas et al., 2001), and after extinction of a conditioned taste aversion (Rosas & Bouton, 1998). Laborda and Miller extended these ﬁndings to a conditioned lick suppression situation. Their Experiment 2 tested the effectiveness of massive extinction, extinction in multiple contexts, and both manipulations together in reducing relapse after a delayed context shift. As indicated in Figure 4.4, the results suggested that extinction in multiple contexts alone and massive extinction alone decreased this strong recovery of extinguished CRs, but only marginally. However, when massive extinction trials (810) took place in three different contexts, the recovery of extinguished CRs was radically attenuated.

2.7 Moderate extinction Massive extinction

2.5 Mean time (log s)

2.3 2.1 1.9 1.7 1.5 1.3 1.1 0.9 0.7 Extinction-1 (A[BBB]---E)

Extinction-3 (A[BCD]---E) Condition

Figure 4.4 Mean log time to complete 5 cumula-

tive seconds of licking in the presence of the target conditioned stimulus in a neutral but familiar context (E) for all groups. Brackets represent standard error of the mean. Higher scores indicate more conditioned suppression. Extinction-1 (A[BBB]— E) = condition that received extinction trials in only one context (B); Extinction-3 (A[BCD]—E) = condition that received extinction trials in three different contexts (B, C, and D); Moderate extinction = condition that received a moderate number of extinction trials (162); Massive extinction = condition that received a massive number of extinction trials (810). See text for further explanation. (From M. A. Laborda & R. R. Miller. 2010. Experiment 2. Massive extinction in multiple contexts reduces fear recovery after a delayed context shift, Paper presented at the meeting of the Eastern Psychological Association, Brooklyn, NY).

04-Schachtman-04.indd 89

89

Although more research would be useful in understanding the underlying mechanisms of these techniques, the data available today strongly suggest that these conjoint behavioral manipulations are effective tools to attenuate relapse after successful extinction treatments. Retrieval Cues from Extinction

Another technique that has been proposed to reduce relapse is through the use of extinction cues (ECs; e.g., Brooks & Bouton, 1993). Typically, extinction cues (also known as retrieval cues for extinction) are stimuli that are presented shortly before the target CS on most CS extinction trials. Then at test, the EC is presented again just prior to the CS, which reduces responding (relative to, for example, a group in which no EC is presented at test) in a situation in which renewal would otherwise be expected (Brooks & Bouton, 1994; see Fig. 4.5). In other words, ECs are presented during most extinction trials and, when subjects are tested for responding to the extinguished CS in the presence of the EC, recovery phenomena are usually reduced. During extinction, the EC is typically not presented on all extinction trials in order to prevent it from becoming a conditioned inhibitor, which would protect the excitatory target CS from extinction (e.g., McConnell & Miller, 2010), or the subject from processing the EC and the target cue as a single conﬁgured cue. Beside those demonstrations in appetitive preparations (Brooks, 2000; Brooks & Bouton, 1993, 1994; Brooks & Bowker, 2001), there are reports of the effectiveness of ECs attenuating spontaneous recovery of alcohol tolerance (Brooks, Vaughn, Freeman, & Woods, 2004), conditioned taste aversion (Brooks, Palmatier, García, & Johnson, 1999), and directed swimming in a navigation task (Prados, Manteiga, & Sansa, 2003). Recently, Brooks (unpublished data) found that EC-alone presentations during the retention interval further attenuated spontaneous recovery. He speculated that this effect arises from the EC-alone trials extinguishing the EC’s second-order conditioned excitatory properties acquired by its having been paired with the target CS during extinction treatment.

2/15/2011 7:42:53 PM

90

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Percentage of initial performance

120.0

100.0 No extinction cue (ABA Renewal) 80.0

Extinction cue (ABA Renewal)

60.0

40.0

20.0

0.0 Last extinction trial

First renewal test

Figure 4.5 Performance in the presence of the

target conditioned stimulus expressed as a percentage of responding on the ﬁrst extinction trial. Brackets represent standard error of the mean. No extinction cue (ABA Renewal) = condition in which the target cue was tested for extinction and renewal in the absence of a cue from extinction treatment; Extinction cue (ABA Renewal) = condition in which the target cue was tested for extinction and renewal in the presence of a cue from extinction treatment; Last extinction trial = mean responding on the last extinction trial; First renewal test = mean responding on the ﬁrst test trial back in the acquisition context. See text for further explanation. (Adapted from D. C. Brooks & M. E. Bouton. 1994. Experiment 3. Journal of Experimental Psychology: Animal Behavior Processes, 20, 366–379).

In addition to the aforementioned demonstrations of ECs decreasing recovery of extinguished CRs with non-human subjects, similar effects have been reported with humans as participants. Dibbets, Havermans, and Arntz (2008) reported attenuated ABA renewal in a human fear-conditioning preparation. Participants received extinction trials with an EC and were then tested on the extinguished cue with or without the EC. Participants tested in the acquisition context with the EC present showed less recovery of US expectancy ratings than participants who were tested without the EC. Similar results were reported by Vansteenwegen et al. (2006), in which renewal of conditioned electrodermal responses and of expectancy ratings were attenuated when testing occurred in the presence of an

04-Schachtman-04.indd 90

EC. In a few instances the effectiveness of ECs to reduce recovery of extinguished CRs has been examined with subclinical and clinical samples. Collins and Brandon (2002) observed that a reduction in stimulus-induced alcohol craving in social drinkers produced by exposure therapy was more resistant to relapse when testing occurred in the presence of ECs, thereby extending the effect to clinical situations. In an interesting extension of the EC concept, Mystkowski, Craske, Echiverri, and Labus (2006) presented self-report data in which a verbal instruction to recall the context of treatment (what they called “mental reinstatement”) successfully diminished the return of fear after a context change (i.e., renewal), in arachnophobic participants. Thus, mental reinstatement of the circumstances of extinction may serve in place of or in addition to reinstatement of physical ECs. The evidence strongly supports the effectiveness of ECs in attenuating recovery of extinguished conditioned responses in different preparations and populations. Future research should focus on the mechanisms behind the success of ECs. Brooks (unpublished data) has argued that ECs differ mechanistically from both occasion setting (for properties, see Holland, 1992; Miller & Oberling, 1998) and conditioned inhibition, but the actual mechanism is still unclear. Likely, multiple processes drive effective ECs in clinical settings because the procedure that produces ECs resembles those commonly used to produce conditioned inhibition, occasion setting, and conﬁgural learning. Extinction in the Presence of a Second Excitor

Another possible manipulation to prevent relapse that is consistent with almost all theories of learning is to conduct extinction of the fearevoking target cue in the presence of a second fear-evoking cue. This should produce deeper (i.e., a greater degree of) extinction, which should, more arguably, attenuate recovery of extinguished responses. This expectation has been conﬁrmed several times (e.g., Rescorla, 2000, 2006; Thomas & Ayres, 2004; but see Pineño,

2/15/2011 7:42:53 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

Zilsky, & Schachtman, 2007). For example, Rescorla (2000) separately trained two CSs with an appetitive US (i.e., food). After learning was complete, subjects were extinguished on the target CS alone or in compound with the second excitatory CS. When the target CS was immediately tested for conditioned responding, rats that received extinction trials with both excitors together displayed greater extinction than rats that received extinction of the target CS alone. These results were later replicated in an aversively motivated task (Rescorla, 2006), using a relatively short retention interval of 7 days. Rescorla observed less spontaneous recovery when extinction treatment took place in the presence of a second excitor. However, Rescorla’s (2006) conclusions about spontaneous recovery must be qualiﬁed due to the absence of an adequate immediate testing control. Thomas and Ayres (2004) extended Rescorla’s results to an ABA design in a fear-conditioning experiment with rats, but because their design lacked of an appropriate experimental control for renewal (e.g., an ABB condition), their conclusions need to be qualiﬁed. In a fear-conditioning paradigm, Thomas and Ayres tested fear to the target cues back in the acquisition context (i.e., Context A) after administrating extinction trials with either the target cue alone or the target cue in compound with a second conditioned excitor in Context B. Their results were interpreted as showing more renewal after elemental than compounded extinction training, but, because of the lack of the appropriate comparisons (i.e., one in which the experimental groups are compared with groups tested in the extinction context), it remains unknown if better extinction could be obtained outside a renewal design (for further discussion, see Urcelay, Lipatova, & Miller, 2009). Despite the positive results just reviewed, there is evidence that extinction in the presence of an additional excitor is limited by generalization decrement. Urcelay, Lipatova, et al. (2009) observed enhanced extinction due to this manipulation, but its effects were almost completely obscured by generalization decrement going from elemental training to compound extinction and from compound extinction to elemental

04-Schachtman-04.indd 91

91

testing. That is, despite the deepened extinction effect induced by the added excitor, almost no beneﬁt was observed at test due to repeatedly changing the target by adding elements (going from acquisition to extinction) or subtracting elements (going from extinction to test). Presumably, the subjects conﬁgured the two CSs as a single cue during extinction, which limited the amount of generalization of learning that could occur between phases (i.e., perceiving the individual CS of training and testing as a component of the compound CS at extinction). This is consistent with work by Pearce and Wilson (1991) who also found a large generalization decrement effect such that extinction with a concurrent excitor actually resulted in less response decrement than elemental extinction. Moreover, conditioned taste aversion experiments with rats (e.g., Pineño et al., 2007) and fear-conditioning experiments with humans to date have so far failed to ﬁnd enhanced extinction in the presence of a second excitor (Lovibond et al., 2000; Vervliet, Vansteenwegen, Hermans, & Eelen, 2007). To reduce conﬁguring, Urcelay, Lipatova, et al. (2009) used cues with asynchronous onsets (a 15-s target that onset 5 s before a 10-s concurrent excitor) and of different modalities. These two manipulations collectively did result in a modest but signiﬁcant reduction of ABC renewal in subjects that were extinguished in the presence of a companion conditioned excitor (see Fig. 4.6). More speciﬁcally, the use of cues with asynchronous onsets and different sensory modalities reduced generalization decrement when testing elements following extinction trials with compound stimuli, thus supporting the view that the previous failures to ﬁnd such an effect were due to generalization decrement. Collectively, we see that extinction in the presence of other conditioned excitors has produced mixed results with respect to attenuating recovery phenomena. The results suggest that concurrent extinction of more than one excitor decreases relapse only slightly (e.g., Urcelay, Lipatova, et al., 2009). Experiments that appear to have yielded a large effect have lacked desirable experimental controls (e.g., Rescorla, 2006; Thomas & Ayres, 2004). In conclusion, the

2/15/2011 7:42:54 PM

92

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS 0.5

0.5 Suppression ratio

ABB (Extinction) ABC Renewal Suppression ratio

0.4

0.3

0.2

ABB (Extinction) ABA Renewal

0.4 0.3 0.2 0.1 0.0 No extinction

0.1

Massed extinction trials

Spaced extinction trials

Condition 0.0 Extinction with a second excitor

Gen-Dec control

Elemental extinction

Figure 4.6 Mean suppression ratio in the pres-

ence of the target conditioned stimulus. Brackets represent standard error of the mean. Lower scores indicate more conditioned suppression. Extinction with a second excitor = condition in which the target cue was extinguished in compound with a second excitatory cue; Gen-Dec control = condition in which the target cue was extinguished in compound with a neutral cue; Elemental extinction = condition in which the target cue was extinguished alone; ABB (Extinction) = condition in which extinction trials and testing occurred in a context different from the one used for acquisition; ABC Renewal = condition in which acquisition trials, extinction trials, and testing occurred in three different contexts. See text for further explanation. (From G. P. Urcelay, O. Lipatova, & R. R. Miller. 2009. Experiment 3. Learning and Motivation, 40, 343–363).

available data suggest only modest beneﬁts of this technique in reducing relapse. Spaced Training in Extinction

Urcelay, Wheeler, and Miller (2009) showed that spaced extinction trials produce more enduring extinction than do massed extinction trials. In their study, rats that received extinction with 6-s intertrial intervals showed renewal (Experiment 2, Fig. 4.7) and spontaneous recovery (Experiment 3, Fig. 4.8), whereas this recovery was attenuated in subjects that received extinction with 600-s intertrial intervals (a result recently extended to a situation in which ABC renewal and spontaneous recovery summates; Laborda, Miguez, &

04-Schachtman-04.indd 92

Figure 4.7 Mean suppression ratio in the pres-

ence of the target conditioned stimulus. Brackets represent standard error of the mean. Lower scores indicate more conditioned suppression. No extinction = condition that received no extinction trials; Massed extinction trials = condition that received extinction trials with a short (6 s) intertrial interval; Spaced extinction trials = condition that received extinction trials with a long (600 s) intertrial interval; ABB (Extinction) = condition that received extinction trials and was tested in a context different from the one used during acquisition; ABA Renewal = condition that received extinction trials in a context different from the one used for both acquisition and testing. See text for further explanation. (From G. P. Urcelay, D. S. Wheeler, & R. R. Miller. 2009. Experiment 2. Learning & Behavior, 37, 60–73).

Miller, 2011). This is consistent with evidence that spaced acquisition trials are more effective than massed acquisition trials (e.g., Barela, 1999; Barnet, Grahame, & Miller, 1995; Humphreys, 1940) and the view that extinction involves learning (i.e., acquisition) of a new association. Urcelay, Wheeler, et al.’s results suggest that spacing exposure trials can be an effective means of attenuating relapse after exposure treatments. In a related study using an appetitive preparation, Moody, Sunsay, and Bouton (2006) found that spacing the extinction trials was effective in decreasing recovery in a reinstatement design (Experiment 5b); however, they found no beneﬁt of spacing the extinction trials in preventing spontaneous recovery. Bjork and Bjork (1992, 2006) suggested that an expanding spaced schedule is more effective in producing robust

2/15/2011 7:42:54 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

Suppression ratio

0.5

Short delay Long delay

0.4 0.3 0.2 0.1 0.0 No extinction

Massed extinction trials

Spaced extinction trials

Condition

Figure 4.8 Mean suppression ratio in the presence

of the target conditioned stimulus. Brackets represent standard error of the mean. Lower scores indicate more conditioned suppression. No extinction = condition that received no extinction trials; Massed extinction trials = condition that received extinction trials with a short (6 s) intertrial interval; Spaced extinction trials = condition that received extinction trials with a long (600 s) intertrial interval; Short delay = condition tested after a short delay following extinction treatment; Long delay = condition tested after a long delay following extinction treatment. See text for further explanation. (From G. P. Urcelay, D. S. Wheeler, & R. R. Miller. 2009. Experiment 3. Learning & Behavior, 37, 60–73.).

extinction than trials spaced evenly. In this schedule, extinction trials are ﬁrst massed and then gradually spaced out. This is meant to produce fast extinction with the massed trials and robust extinction with the spaced trials. This idea has received support from human verbal learning studies (e.g., Fritz, Morris, Nolan, & Singleton, 2007; Landauer & Bjork, 1978), but there is also evidence that this expanding extinction procedure is not more effective than uniformspaced extinction when subjects are tested in a renewal or spontaneous recovery situation (e.g., Karpicke & Roediger, 2007; Orinstein, Urcelay, & Miller, 2010). Orinstein et al. observed faster extinction with this procedure, but they did not observe a difference at test between the expanding schedule and a uniform-spaced schedule in an ABA renewal design in a human contingency learning task. However, the fact that extinction proceeded faster with the expanding schedule may still be beneﬁcial for reducing

04-Schachtman-04.indd 93

93

drop-out rates in therapy, even if the end result is the same. Additional support for a beneﬁcial effect of spacing extinction trials on behavior during extinction learning has been reported using operant techniques with snails (Sangha, Scheibenstock, Morrow, & Lukowiak, 2003) and using conditioned taste aversion preparations with rats (Westbrook, Smith, & Charnock, 1985). However, there are also reports of massed extinction trials supporting better extinction than spaced extinction trials (Cain, Blouin, & Barad, 2003; Moody et al., 2006; Rescorla & Durlach, 1987). Likely, parametric differences in CS duration as well as trial spacing are responsible for these discrepancies. Notably, data originating from tests administered in the extinction context very soon after extinction treatment (as is often done when researchers assess the consequences of extinction treatment) are neither informative nor particularly relevant to exposure therapy because they only identify the ﬁnal level of behavioral control, which reﬂects not only associative extinction but also labile nonassociative endogenous and exogenous states immediately created by the extinction trials (Rescorla, 2004). Such data are not predictive of recovery after context changes or long retention intervals. Together these results suggest that there are likely beneﬁts of spacing extinction trials, but much more research is needed to delimit the scope and mechanisms of this manipulation. Spaced Extinction Sessions

As discussed earlier, spacing extinction trials has been found to be of some beneﬁt in preventing recovery from extinction. This suggests that spacing the extinction sessions might also support better extinction and/or promote less recovery. To test this prediction, Tsao and Craske (2000; also see Rowe & Craske, 1998a) compared massed sessions (i.e., four exposure sessions in a single day), uniform-spaced sessions (i.e., four exposure sessions with one every 5 days), and expanding-spaced sessions (i.e., four exposure sessions over a 16-day period, in an expanding schedule [days 1, 2, 6, and 16]) of exposure

2/15/2011 7:42:54 PM

94

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

therapy on reducing return of fear with anxious public speakers. Contrary to the author’s expectations, all three groups of participants showed similar reduction of fear at post treatment, a result consistent with Orinstein et al.’s (2010) results when using expanding and uniformly spaced intertrial intervals in a human contingency learning task. However, as hypothesized by Tsao and Craske, when the participants were tested 1 month later, the massed condition showed the greatest fear recovery. The uniformand expanding-spaced extinction sessions both attenuated return of the fear response relative to the massed session condition, but they did not differ between themselves (for negative results of a similar manipulation, see Lang & Craske, 2000). More recently, Laborda et al. (2011) reported a series of three fear-conditioning experiments in which the beneﬁt of spacing extinction trials and of spacing extinction sessions was evaluated, using rats as experimental subjects in a preparation in which renewal and spontaneous recovery summated. In brief, they found that the recovery of an extinguished association after a delayed context shift was reduced by spacing the extinction trials (600-s intertrial intervals vs. 6-s intertrial intervals; extending the results reported by Urcelay, Wheeler, et al., 2009) and by spacing the extinction sessions (7-d intersession intervals vs. 10-m intersession intervals). Moreover, simultaneously spacing the extinction trials and spacing the extinction sessions were shown to further attenuate strong recovery encouraged by a combination of renewal and spontaneous recovery. According to Bouton’s (1993, 2010) account, it is possible that, when using spaced sessions (or even spaced intertrial intervals), we are actually conducting extinction in multiple “temporal” contexts, which would put any beneﬁt in reducing recovery here on common ground with the reduction in recovery observed following extinction in multiple physical contexts. Extinction in a Context as Similar as Possible to That of the Precipitating Conditioning Event

Laborda et al. (in press) demonstrated that the weaker recovery of responding observed with

04-Schachtman-04.indd 94

AAC renewal relative to ABC renewal is due to extinction treatment in the acquisition context strengthening the association between the target cue and the excitatory acquisition context. Presumably, extinction in the acquisition context facilitates reduction of conditioned responding by not only reducing the effective CS-outcome association but also by increasing the expected background rate of outcome occurrence, which is mediated by CS-training context and training context-outcome associations (see Laborda et al., in press for details). This manipulation is essentially extinction of the target cue in the presence of an additional excitor, which in this case is the acquisition context. Extinguishing the acquisition context or overshadowing the targetcontext association with another cue presented during extinction both reduced initial extinction and increased AAC renewal. This provides a theoretical basis for the empirically supported view that exposure therapy in a context similar to that of the precipitating conditioning event often is more effective than when it occurs in a neutral context (Massad & Hulsey, 2006). Deeper extinction and less recovery can be expected when the extinction context is more similar to the acquisition context. Unconditioned Stimuli Presentations During Extinction

Considerable research has found that the unpaired presentations of USs during extinction treatment can enhance extinction and its resistance to recovery. As shown in Figure 4.9, Rauhut et al. (2001) attenuated renewal by presenting explicitly unpaired USs interspersed among the CS-alone trials of extinction treatment in contrast to administering only CS-alone trials. This created the conditions for combining extinction to the CS and habituation to the US (details of their data indicate that the treatment did not constitute inhibitory training of the target). Alternatively, Bouton, Rosengard, Achenbach, Peck, and Brooks (1993) have suggested that the unsignaled USs make the context of extinction more similar to that of acquisition, which, as proposed in the previous section, could attenuate relapse. In related research, Bouton, Woods,

2/15/2011 7:42:54 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

To our knowledge, the applicability of such intervention to clinical settings has not been yet studied.

0.5

Suppression ratio

0.4

0.3

The Acquisition-Extinction Interval

0.2 No USs during extinction 0.1

Unpaired USs during extinction-4 Unpaired USs during extinction-16

0.0 Extinction

ABA Renewal test

Figure 4.9 Mean suppression ratio in the pres-

ence of the target conditioned stimulus. Lower scores indicate more conditioned suppression. Brackets represent standard error of the mean. No USs during extinction = group that received conditioned stimulus-only extinction trials; Unpaired USs during extinction-4 = group that received extinction trials with four interspersed noncontingent unconditioned stimuli presentations; Unpaired USs during extinction-16 = group that received extinction trials with 16 interspersed noncontingent unconditioned stimuli presentations; Extinction = mean responding in the last ﬁve extinction trials; ABA renewal test = mean responding in the ﬁrst test trial back in the acquisition context. See text for further explanation. (Adapted from A. S. Rauhut, B. L. Thomas, & J. J. B. Ayres. 2001. Journal of Experimental Psychology: Animal Behavior Processes, 27, 99–114).

and Pineño (2004) observed slower reacquisition when occasional CS-US pairings occurred during extinction. They suggested that these occasional CS-US pairings re-created the context of acquisition, thereby constituting a condition more akin to an AAC renewal situation relative to the ABC situation created by continuous nonreinforcement, with AAC renewal being weaker than ABC renewal. However, a major challenge to translate this manipulation into practice is ethical problems in administering full-strength aversive USs in a therapeutic situation. An alternative approach is distantly suggested by McDonald and Siegel (2004), who determined that weak pharmacological USs can signal stronger versions of the same US (and might encourage habituation to the stronger US).

04-Schachtman-04.indd 95

95

A somewhat controversial variable of interest is the acquisition-to-extinction interval. Myers, Ressler, and Davis (2006) reported less reinstatement, renewal, and spontaneous recovery when this interval was very short relative to if it was long, a set of results that Myers et al. discussed in terms of the possible neural mechanisms underling extinction at various intervals. These data can also be interpreted within a behavioral framework based in Bouton’s (1997) context theory. As previously discussed, if the context is deﬁned by spatial and temporal cues, then one can think of a long retention interval as inducing a new temporal context to create a renewal-like situation. That is, imposing a long retention interval between acquisition and extinction and between extinction and testing causes the temporal context to be equivalent to an ABC-like renewal design. A short acquisitionto-extinction interval, however, is analogous to an AAC-like renewal design. As previously mentioned, AAC renewal tends to be much weaker than ABA or ABC renewal. Thus, Myers et al.’s data can be accounted for as an instance of AAC treatment producing poor renewal compared to a strong ABC renewal effect. However, this ﬁnding is controversial, because there are many reported failures to conﬁrm this effect (e.g., Alvarez, Johnson, & Grillon, 2007; Huff, Hernandez, Blanding, & LaBar, 2009; Kim & Richardson, 2009; Maren & Chang, 2006; Schiller et al., 2008; Woods & Bouton, 2008). Although there is no agreed-upon reason for the discrepant ﬁndings, it is likely related to how each researcher deﬁnes “long” and “short” when deciding upon parameters for the retention interval and is a function of the acquisition-to-test interval relative to other intervals rather than an absolute time. In brief, decreased recovery given short acquisition-to-extinction intervals is highly constrained by the temporal parameters used. A series of experiments recently reported by Johnson, Escobar, and Kimble (2010) began

2/15/2011 7:42:55 PM

96

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

clarifying the scope of the beneﬁt of immediate extinction. They found that a short acquisitionto-extinction interval decreased spontaneous recovery of extinguished fear responses in rats when the extinction-to-test interval was relatively long (a 3-d interval in Experiment 1 and a 7-d interval in Experiment 2), and increased recovery when the extinction-to-test interval was relatively short (a 2-d interval in Experiment 2), suggesting that it is an interaction between the acquisition-to-extinction and the extinction-totest intervals which determines the long-term effects of extinction.

CONCLUSIONS AND FINAL REMARKS: DEEPENING EXTINCTION AND/OR LINKING EXTINCTION AND TEST CONTEXTS Pavlov’s early studies (1927) and subsequent ﬁndings have been used to both understand acquired behavior and model human psychopathology. Research in the animal and human laboratory has found that Pavlovian conditioning provides a useful model of select mental disorders (e.g., anxiety, addictions) and some forms of cognitive-behavioral therapy. The study of experimental extinction has been fundamental for the development and success of exposure therapy. Unfortunately, despite extinction treatments and exposure therapies often immediately decreasing undesired behaviors, recovery and relapse commonly occur (e.g., Bouton, 2000). To decrease the possibility of recovery after successful extinction treatments, behavioral techniques have been developed based on associative models that speak to relapse prevention after exposure therapy. In the present chapter, we have reviewed these contemporary behavioral techniques to decrease recovery from extinction. Bouton, Woods, Moody, Sunsay, and García-Gutiérrez (2006) suggested that these techniques could be divided in two categories: techniques designed to deepen extinction learning, and techniques designed to enhance similarity between the context of extinction and the context of testing (i.e., linking or ‘bridging’ the extinction and test contexts). Dividing these

04-Schachtman-04.indd 96

strategies into these two categories appears inviting. However, upon careful inspection it becomes clear that almost none of them ﬁt exclusively into just one of these two categories. We instead suggest that these techniques can be categorized as deepening extinction, linking the extinction and test contexts, both, or none of the above, with most of the techniques likely belonging in part to both categories. Using a massive number of extinction trials and extinguishing the target cue with a second conditioned excitor can be considered exemplars of techniques that deepen extinction learning and clearly do not “bridge” the extinction and test contexts. The use of retrieval cues from extinction is the only technique that can be categorized as purely “bridging” the extinction and test contexts. In this technique a component of the extinction context is explicitly presented at test. Extinction in multiple contexts can be thought of as increasing the total number of contextual features associated with extinction, thereby increasing the possibility of generalizing extinction learning to other contexts. But an alternative account of any reduction in recovery from extinction following extinction in multiple contexts is that renewal in each new extinction context may increase fear during extinction, thereby facilitating deeper extinction (Rescorla, 2001; but see Bouton, García-Gutiérrez, et al., 2006). Spaced extinction trials and sessions might reduce recovery of extinguished CRs because each trial and session can be considered as occurring in different “temporal” contexts, and in this way working as extinction in multiple “temporal” contexts. But spaced training has been showed to improve learning during acquisition (e.g., Barela, 1999) and extinction (e.g., Westbrook et al., 1985); consequently, we could think of these techniques as deepening extinction by enhancing the inhibitory-like learning that occurs during extinction. US-alone presentations interspersed during extinction treatment can be characterized as improving extinction learning because these US presentations either make the extinction context excitatory, thereby creating the conditions for explicitly unpaired conditioned inhibition training of the target cue (which is usually

2/15/2011 7:42:55 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

more effective in reducing responding than extinction) or fostering habituation of the US. Alternatively, unsignaled USs could make the context of extinction more similar to that of acquisition, thereby creating an AAC renewal situation, which is known to support less recovery from extinction than ABA or ABC renewal treatments. Bouton, Woods, et al. (2006) contend that efforts to produce deeper extinction are ineffectual; we respectfully disagree. To date the evidence is still unclear in several cases, but in some of them (e.g., massive extinction), deepening extinction is clearly effective in attenuating relapse. In our view, a more useful distinction is to think of behavioral interventions occurring at the time of extinction or at the time of testing. Those interventions in which the critical manipulation takes place during the extinction trials (e.g., massive extinction in multiple contexts) are potentially more useful to clinicians than manipulations that depend on interventions at the time of testing (e.g., extinction cues) because the clinician can more readily control what occurs during therapy than what occurs at the moment of potential relapse. Despite our tentative conclusions, the results reviewed in this chapter strongly suggest that more research is needed concerning the parameters that optimize these techniques and their underlying mechanisms. Lastly, additional research evaluating the translation of these laboratory techniques to clinical settings is essential so that they can be used to enhance psychotherapy.

ACKNOWLEDGMENTS National Institute of Mental Health grant 33881 supported preparation of this chapter. The authors thank Shannon Gaynor, Jason King, Gonzalo Miguez, Jun Park, Cody Polack, Lee Villinsky, and James Witnauer for their comments on an earlier version. Mario Laborda was supported by the Comisión Nacional de Investigación Cientíﬁca y Tecnológica (CONICYT-Chile) and the Department of Psychology of the Universidad de Chile. Inquiries concerning this research should be addressed to Ralph R. Miller, Department of Psychology,

04-Schachtman-04.indd 97

97

SUNY- Binghamton, Binghamton, NY 139026000; e-mail: [email protected].

REFERENCES Alvarez, R. P., Johnson, L., & Grillon, C. (2007). Contextual-speciﬁcity of short-delay extinction in humans: Renewal of fear-potentiated startle in a virtual environment. Learning and Memory, 14, 247–253. Bandarian Balooch, S., & Neumann, D. L. (2011). Effect of multiple contexts and context similarity on the renewal of extinguished conditioned behaviour in an ABA design with humans. Learning and Motivation, 42, 53–63. Barela, P. B. (1999). Theoretical mechanisms underlying the trial-spacing effect in Pavlovian fear conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 25, 177–193. Barnet, R. C., Grahame, N. J., & Miller, R. R. (1995). Trial spacing effects in Pavlovian conditioning: A role for local context. Animal Learning and Behavior, 23, 340–348. Betancourt, R., Corada, L., Dominichetti, J., Laborda, M. A., Martínez, G., & Miguez, G. (2008). Efecto de la extinción en múltiples contextos sobre la renovación de la tolerancia a las drogas. Psicothema, 20, 279–283. Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus ﬂuctuation. In A. F. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.), From learning processes to cognitive processes: Essays in honor of William K. Estes (Vol. 2, pp. 35–67). Hillsdale, NJ: Erlbaum. Bjork, R. A., & Bjork, E. L. (2006). Optimizing treatment and instruction: Implications of a new theory of disuse. In L-G. Nilsson & N. Ohta (Eds.), Memory and society: Psychological perspectives (pp. 109–133). New York, NY: Psychology Press. Boschen, M. J., Neumann, D. L., & Waters, A. M. (2009). Relapse of successfully treated anxiety and fear: Theoretical issues and recommendations for clinical practice. Australian and New Zealand Journal of Psychiatry, 43, 89–100. Bouton, M. E. (1991). A contextual analysis of fear extinction. In P. R. Martin (Ed.), Handbook of behavior therapy and psychological science: An integrative approach (pp. 435–453). Elmsford, NY: Pergamon Press. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigm of

2/15/2011 7:42:55 PM

98

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E. (1994). Conditioning, remembering, and forgetting. Journal of Experimental Psychology: Animal Behavior Processes, 20, 219–231. Bouton, M. E. (1997). Signals for whether versus when an event will occur. In M. E. Bouton & M. S. Fanselow (Eds.), Learning, motivation, and cognition: The functional behaviorism of Robert C. Bolles (pp. 385–409). Washington, DC: American Psychological Association. Bouton, M. E. (2000). A learning theory perspective on lapse, relapse, and the maintenance of behavior change. Health Psychology, 19, 57–63. Bouton, M. E. (2004). Context and behavioral processes in extinction. Learning and Memory, 11, 485–494. Bouton, M. E. (2010). The multiple forms of “context” in associative learning theory. In B. Mesquita, L. F. Barrett, & E. R. Smith (Eds.), The mind in context (pp. 233–258). New York, NY: Guilford Press. Bouton, M. E., & Bolles, R. C. (1979a). Contextual control of the extinction of conditioned fear. Learning and Motivation, 10, 445–466. Bouton, M. E., & Bolles, R. C. (1979b). Role of conditioned contextual stimuli in reinstatement of extinguished fear. Journal of Experimental Psychology: Animal Behavior Processes, 5, 368–378. Bouton, M. E., García-Gutiérrez, A., Zilski, J., & Moody, E. W. (2006). Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse. Behaviour Research and Therapy, 44, 983–994. Bouton, M. E., & King, D. A. (1983). Contextual control of the extinction of conditioned fear: Tests for the associative value of the context. Journal of Experimental Psychology: Animal Behavior Processes, 9, 248–265. Bouton, M. E., & Nelson, J. B. (1998). The role of context in classical conditioning: Some implications for cognitive behavior therapy. In W. O’Donohue (Ed.), Learning and behavior therapy (pp. 59–84). Boston, MA: Allyn and Bacon. Bouton, M. E., & Peck, C. A. (1992). Spontaneous recovery in cross-motivational transfer (counterconditioning). Animal Learning and Behavior, 20, 313–321. Bouton, M. E., & Ricker, S. T. (1994). Renewal of extinguished responding in a second context. Animal Learning and Behavior, 22, 317–324. Bouton, M. E., Rosengard, C., Achenbach, G. G., Peck, C. A., & Brooks, D. C. (1993). Effects of

04-Schachtman-04.indd 98

contextual conditioning and unconditioned stimulus presentation on performance in appetitive conditioning. Quarterly Journal of Experimental Psychology, 46B, 63–95. Bouton, M. E., Vurbic, D., & Woods, A. M. (2008). D-cycloserine facilitates context-speciﬁc fear extinction learning. Neurobiology of Learning and Memory, 90, 504–510. Bouton, M. E., Woods, A. M., Moody, E. W., Sunsay, C., & García-Gutiérrez, A. (2006). Counteracting the context-dependence of extinction: Relapse and tests of some relapse prevention methods. In M. G. Craske, D. Hermans, & D. Vansteenwegen (Eds.), Fear and learning: From basic processes to clinical implications (pp. 175–196). Washington, DC: American Psychological Association. Bouton, M. E., Woods, A. M., & Pineño, O. (2004). Occasional reinforced trials during extinction can slow the rate of rapid reacquisition. Learning and Motivation, 35, 371–390. Brooks, D. C. (2000). Recent and remote extinction cues reduce spontaneous recovery. Quarterly Journal of Experimental Psychology, 53B, 25–58. Brooks, D. C., & Bouton, M. E. (1993). A retrieval cue for extinction attenuates spontaneous recovery. Journal of Experimental Psychology: Animal Behavior Processes, 19, 77–89. Brooks, D. C., & Bouton, M. E. (1994). A retrieval cue for extinction attenuates response recovery (renewal) caused by a return to the conditioning context. Journal of Experimental Psychology: Animal Behavior Processes, 20, 366–379. Brooks, D. C., & Bowker, J. L. (2001). Further evidence that conditioned inhibition is not the mechanism of an extinction cue’s effect: A reinforced cue prevents spontaneous recovery. Animal Learning and Behavior, 29, 381–388. Brooks, D. C., Palmatier, M. I., García, E. O., & Johnson, J. L. (1999). An extinction cue reduces spontaneous recovery of a conditioned taste aversion. Animal Learning and Behavior, 27, 77–88. Brooks, D. C., Vaughn, J. M., Freeman, A. J., & Woods, A. M. (2004). An extinction cue reduces spontaneous recovery of ataxic ethanol tolerance in rats. Psychopharmacology, 176, 256–265. Cain, C. K., Blouin, A. M., & Barad, M. (2003). Temporally massed CS presentations generate more fear extinction than spaced presentations. Journal of Experimental Psychology: Animal Behavior Processes, 29, 323–333.

2/15/2011 7:42:55 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

Chambless, D. L., & Ollendick, T. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716. Chambless, D. L., Sanderson, W. C., Shoham, V., Bennett Johnson, S., Pope, K. S., CritsChristoph, P.,. . .McCurry, S. (1996). An update on empirically validated therapies. Clinical Psychologist, 49, 5–18. Chaundhri, N., Sahuque, L. L., & Janak, P. H. (2008). Context-induced relapse of conditioned behavioral responding to ethanol cues in rats. Biological Psychiatry, 64, 203–210. Chelonis, J. J., Calton, J. L., Hart, J. A., & Schachtman, T. R. (1999). Attenuation of the renewal effect by extinction in multiple contexts. Learning and Motivation, 30, 1–14. Collins, B. N., & Brandon, T. H. (2002). Effects of extinction context and retrieval cues on alcohol cue reactivity among nonalcoholic drinkers. Journal of Consulting and Clinical Psychology, 70, 390–397. Craske, M. (1999). Anxiety disorders: Psychological approaches to theory and treatment. Boulder, CO: Westview Press. Davenport, R. A., & Houpt, T. A. (2009). D-cycloserine enhances short-delay, but not long-delay, conditioned taste aversion learning in rats. Pharmacology, Biochemistry and Behavior, 91, 596–603. Delamater, A. R. (2004). Experimental extinction in Pavlovian conditioning: Behavioural and neuroscience perspectives. Quarterly Journal of Experimental Psychology, 57B, 97–132. Denniston, J. C., Chang, R. C., & Miller, R. R. (2003). Massive extinction attenuates the renewal effect. Learning and Motivation, 34, 68–86. Devenport, L. D. (1998). Spontaneous recovery without interference: Why remembering is adaptive. Animal Learning and Behavior, 26, 172–181. Dibbets, P., Havermans, R., & Arntz, A. (2008). All we need is a cue to remember: The effect of an extinction cue on renewal. Behaviour Research and Therapy, 46, 1070–1077. Effting, M., & Kindt, M. (2007). Contextual control of human fear associations in a renewal paradigm. Behaviour Research and Therapy, 45, 2002–2018. Foa, E. B., Hembree, E. A., Cahill, S. P., Rauch, S. A. M., Riggs, D. S., Feeny, N. C., & Yadin, E. (2005). Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: Outcome

04-Schachtman-04.indd 99

99

at academic and community clinics. Journal of Consulting and Clinical Psychology, 73, 953–964. Fritz, C. O., Morris, P. E., Nolan, D., & Singleton, J. (2007). Expanding retrieval practice: An effective aid to preschool children’s learning. Quarterly Journal of Experimental Psychology, 60, 991–1004. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289–344. García-Gutiérrez, A., & Rosas, J. M. (2003). Empirical and theoretical implications of additivity between reinstatement and renewal after interference in causal learning. Behavioural Processes, 63, 21–31. Glautier, S., & Elgueta, T. (2009). Multiple cue extinction effect on recovery of responding in causal judgments. International Journal of Comparative Psychology, 22, 254–270. Guastella, A. J., Dadds, M. R., Lovibond, P. F., Mitchell, P., & Richardson, R. (2007). A randomized controlled trial of the effect of D-cycloserine on exposure therapy for spider fear. Journal of Psychiatric Research, 41, 466–471. Guastella, A. J., Lovibond, P. F., Dadds, M. R., Mitchell, P., & Richardson, R. (2007). A randomized controlled trial of the effect of D-cycloserine on extinction and fear conditioning in humans. Behaviour Research and Therapy, 45, 663–672. Guastella, A. J., Richardson, R., Lovibond, P. F., Rapee, R. M., Gaston, J. E., Mitchell, P., & Dadds, M. R. (2008). A randomized controlled trial of D-cycloserine enhancement of exposure therapy for social anxiety disorder. Biological Psychiatry, 63, 544–549. Gunther, L. M., Denniston, J. C., & Miller, R. R. (1998). Conducting exposure treatment in multiple contexts can prevent relapse. Behaviour Research and Therapy, 36, 75–91. Hofmann, S. G. (2008). Cognitive processes during fear acquisition and extinction in animals and humans: Implications for exposure therapy of anxiety disorders. Clinical Psychology Review, 28, 199–210. Hofmann, S. G., Meuret, A. E., Smits, J. A. J., Simon, N. M., Pollack, M. H., Eisenmenger, K.,. . . Otto M. W. (2006). Augmentation of exposure therapy with D-cycloserine for social anxiety disorder. Archives of General Psychiatry, 63, 298–304. Holland, P. C. (1992). Occasion setting in Pavlovian conditioning. In D. L. Medin (Ed.), The psychology

2/15/2011 7:42:56 PM

100

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of learning and motivation (Vol. 28, pp. 69–125). San Diego, CA: Academic Press. Huff, N. C., Hernandez, J. A., Blanding, N. Q., & LaBar, K. S. (2009). Delayed extinction attenuates conditioned fear renewal and spontaneous recovery in humans. Behavioral Neuroscience, 123, 834–843. Humphreys, L. G. (1940). Distributed practice in the development of the conditioned eyelid reaction. Journal of General Psychology, 22, 379–385. Johnson, J. S., Escobar, M., & Kimble, W. L. (2010). Long-term maintenance of immediate or delayed extinction is determined by the extinction-test interval. Learning and Memory, 17, 639–644. Karpicke, J. D., & Roediger, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57, 151–162. Kim, J. H., & Richardson, R. (2009). Expression of renewal is dependent on the extinction-test interval rather than the acquisition-extinction interval. Behavioral Neuroscience, 123, 641–649. Krasnogorsky, N. I. (1925). The conditioned reﬂexes and children’s neuroses. American Journal of Diseases of Children, 30, 753–768. Kushner, M. G., Kim, S. W., Donahue, C., Thuras, P., Adson, D., Kotlyar, M.,. . .Foa, E. B. (2007). D-cycloserine augmented exposure therapy for obsessive-compulsive disorder. Biological Psychiatry, 62, 835–838. Laborda, M. A., Miguez, G., & Miller, R. R. (2011). Additive effect of spacing extinction trials and spacing extinction sessions in preventing recovery from extinction. Paper presented at the meeting of the Eastern Psychological Association, Cambridge, MA. Laborda, M. A., & Miller, R. R. (2010). Massive extinction in multiple contexts reduces fear recovery after a delayed context shift. Paper presented at the meeting of the Eastern Psychological Association, Brooklyn, NY. Laborda, M. A., Witnauer, J. E., & Miller, R. R. (in press). Contrasting AAC and ABC renewal: The role of context associations. Learning and Behavior. Landauer, T. K., & Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory (pp. 625–632). London, England: Academic Press. Lang, A. J., & Craske, M. G. (2000). Manipulations of exposure-based therapy to reduce return

04-Schachtman-04.indd 100

of fear: A replication. Behaviour Research and Therapy, 38, 1–12. Ledgerwood, L., Richardson, R., & Cranney, J. (2003). Effects of D-cycloserine on extinction of conditioned freezing. Behavioral Neuroscience, 117, 341–349. Ledgerwood, L., Richardson, R., & Cranney, J. (2004). D-cycloserine and the facilitation of extinction of conditioned fear: Consequences for reinstatement. Behavioral Neuroscience, 118, 505–513. Ledgerwood, L., Richardson, R., & Cranney, J. (2005). D-cycloserine facilitates extinction of learned far: Effects on reacquisition and generalized extinction. Biological Psychiatry, 57, 841–847. Lovibond, P. F., Davis, N., & O’Flaherty, A. S. (2000). Protection from extinction in human fear conditioning. Behaviour Research and Therapy, 38, 967–983. MacKillop, J., & Lisman, S. A. (2008). Effects of a context shift and multiple context extinction on reactivity to alcohol cues. Experimental and Clinical Psychopharmacology, 16, 322–331. Maren, S., & Chang, C. (2006). Recent fear is resistant to extinction. Proceedings of the National Academy of Sciences of the United States of America, 103, 18020–18025. Massad, P. M., & Hulsey, T. L. (2006). Exposure therapy renewed. Journal of Psychotherapy Integration, 16, 417–428. McConnell, B. L., & Miller, R. R. (2010). Protection from extinction provided by a conditioned inhibitor. Learning and Behavior, 38, 68–79. McDonald, R. V., & Siegel, S. (2004). Intraadministration associations and withdrawal symptoms: Morphine-elicited morphine withdrawal. Experimental and Clinical Psychopharmacology, 12, 3–11. Miller, R. R., & Oberling, P. (1998). Analogies between occasion setting and Pavlovian conditioning. In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 3–35). Washington, DC: American Psychological Association. Mineka, S., & Kihlstrom, J. (1978). Unpredictable and uncontrollable aversive events. Journal of Abnormal Psychology, 87, 256–271. Mineka, S., Mystkowski, J. L., Hladek, D., & Rodriguez, B. I. (1999). The effects of changing contexts on return of fear following exposure therapy for spider fear. Journal of Consulting and Clinical Psychology, 67, 599–604. Mineka, S., & Oehlberg, K. (2008). The relevance of recent developments in classical conditioning to

2/15/2011 7:42:56 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

understanding the etiology and maintenance of anxiety disorders. Acta Psychologica, 127, 567–580. Mineka, S., & Sutton, J. (2006). Contemporary learning theory perspective on the etiology of fears and phobias. In M. G. Craske, D. Hermans, & D. Vansteenwegen (Eds.), Fear and learning: From basic processes to clinical implications (pp. 75–97). Washington, DC: American Psychological Association. Mineka S., & Zinbarg, R. (2006). A contemporary learning theory perspective on anxiety disorders: It’s not what you thought it was. American Psychologist, 61, 10–26. Moody, E. W., Sunsay, C., & Bouton, M. E. (2006). Priming and trial spacing in extinction: Effects on extinction performance, spontaneous recovery, and reinstatement in appetitive conditioning. Quarterly Journal of Experimental Psychology, 59, 809–829. Myers, K. M., Ressler, K. J., & Davis, M. (2006). Different mechanisms of fear extinction dependent on length of time since fear acquisition. Learning and Memory, 13, 216–223. Mystkowski, J. L., Craske, M. G., & Echiverri, A. M. (2002). Treatment context and return of fear in spider phobia. Behavior Therapy, 33, 399–416. Mystkowski, J. L., Craske, M. G., Echiverri, A. M., & Labus, J. S. (2006). Mental reinstatement of context and return of fear in spider-fearful participants. Behavior Therapy, 37, 49–60. Napier, R. M., Macrae, M., & Kehoe, E. J. (1992). Reacquisition after extinction in the rabbit nictitating membrane response. Journal of Experimental Psychology: Animal Behavior Processes, 18, 182–192. Neumann, D. L. (2006). The effects of physical context changes and multiple extinction contexts on two forms of renewal in a conditioned suppression task with humans. Learning and Motivation, 37, 149–175. Neumann, D. L., Lipp, O. V., & Cory, S. E. (2007). Conducting extinction in multiple contexts does not necessarily attenuate the renewal of shock expectancy in a fear-conditioning procedure with humans. Behaviour Research and Therapy, 45, 385–394. Orinstein, A., Urcelay, G. P., & Miller, R. R. (2010). Expanding the intertrial interval during extinction: Response cessation and recovery. Behavior Therapy, 41, 14–29. Paolone, G., Botreau, F., & Stewart, J. (2009). The facilitative effects of D-cycloserine on extinction of a cocaine-induced conditioned place

04-Schachtman-04.indd 101

101

preference can be long lasting and resistant to reinstatement. Psychopharmacology, 202, 403–409. Pavlov, I. P. (1927). Conditioned reﬂexes (G. V. Anrep, Ed. & Trans.). London, England: Oxford University Press. Pearce, J. M. (1987). A model for stimulus generalization in Pavlovian conditioning. Psychological Review, 94, 61–73. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532–552. Pearce, J. M., & Wilson, P. N. (1991). Effects of extinction with a compound conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 17, 151–162. Pineño, O., & Miller, R. R. (2004). Signaling a change in cue-outcome relations in human associative learning. Learning and Behavior, 32, 360–375. Pineño, O., Zilsky, J. M., & Schachtman, T. R. (2007). Second-order conditioning during a compound extinction treatment. Learning and Motivation, 38, 172–192. Prados, J., Manteiga, R. D., & Sansa, J. (2003). Recovery effects after extinction in the Morris swimming pool navigation task. Learning and Behavior, 31, 299–304. Rachman, S. J. (1978). Fear and courage. San Francisco, CA: Freeman. Rauhut, A. S., Thomas, B. L., & Ayres, J. J. B. (2001). Treatments that weaken Pavlovian conditioned fear and thwart its renewal in rats: Implications for treating human phobias. Journal of Experimental Psychology: Animal Behavior Processes, 27, 99–114. Rescorla, R. A. (2000). Extinction can be enhanced by a concurrent excitor. Journal of Experimental Psychology: Animal Behavior Processes, 26, 251–260. Rescorla, R. A. (2001). Experimental extinction. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 119–154). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (2003). Protection from extinction. Learning and Behavior, 31, 124–132. Rescorla, R. A. (2004). Spontaneous recovery. Learning and Memory, 11, 501–509. Rescorla, R. A. (2005). Spontaneous recovery of excitation but not inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 31, 277–288. Rescorla, R. A. (2006). Deepened extinction from compound stimulus presentation. Journal of

2/15/2011 7:42:56 PM

102

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Experimental Psychology: Animal Behavior Processes, 32, 135–144. Rescorla, R. A. (2008). Within-subject renewal in sign tracking. Quarterly Journal of Experimental Psychology, 61, 1793–1802. Rescorla, R. A., & Durlach, P. J. (1987). The role of context in intertrial interval effects in autoshaping. Quarterly Journal of Experimental Psychology, 39B, 35–48. Rescorla, R. A., & Heth, C. D. (1975). Reinstatement of fear to an extinguished conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 1, 88–96. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current theory and research (pp. 64–99). New York, NY: Appleton-Century Crofts. Ressler, K. J., Rothbaum, B. O., Tannenbaum, L., Anderson, P., Graap, K., Zimand, E.,. . . Davis, M. (2004). Cognitive enhancers as adjuncts to psychotherapy: Use of D-cycloserine in phobic individuals to facilitate extinction of fear. Archives of General Psychiatry, 61, 1136–1144. Ricker, S. T., & Bouton, M. E. (1996). Reacquisition following extinction in appetitive conditioning. Animal Learning and Behavior, 24, 423–436. Robbins, S. J. (1990). Mechanisms underlying spontaneous recovery in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 16, 235–249. Rodriguez, B. I., Craske, M. G., Mineka, S., & Hladek, D. (1999). Context-speciﬁc of relapse: Effects of therapist and environmental context on return of fear. Behaviour Research and Therapy, 37, 845–862. Rosas, J. M., & Bouton, M. E. (1997). Additivity of the effects of retention interval and context change on latent inhibition: Toward resolution of the context forgetting paradox. Journal of Experimental Psychology: Animal Behavior Processes, 23, 283–294. Rosas, J. M., & Bouton, M. E. (1998). Context change and retention interval can have additive, rather than interactive, effects after taste aversion extinction. Psychonomic Bulletin and Review, 5, 79–83. Rosas, J. M., & Callejas-Aguilera, J. E. (2006). Context switch effects on acquisition and extinction in human predictive learning. Journal of

04-Schachtman-04.indd 102

Experimental Psychology: Learning, Memory, and Cognition, 32, 461–474. Rosas, J. M., Callejas-Aguilera, J. E., RamosÁlvarez, M. M., & Abad, M. J. F. (2006). Revision of retrieval theory of forgetting: What does make information context-speciﬁc? International Journal of Psychology and Psychological Therapy, 6, 147–166. Rosas, J. M., Vila, N. J., Lugo, M., & López, L. (2001). Combined effect of context change and retention interval on interference in causality judgment. Journal of Experimental Psychology: Animal Behavior Processes, 27, 153–164. Rowe, M. K., & Craske, M. G. (1998a). Effects of an expanding-spaced vs. massed exposure schedule on fear reduction and return of fear. Behaviour Research and Therapy, 36, 701–717. Rowe, M. K., & Craske, M. G. (1998b). Effects of varied-stimulus exposure training on fear reduction and return of fear. Behaviour Research and Therapy, 36, 719–734. Sangha, S., Scheibenstock, A., Morrow, R., & Lukowiak, K. (2003). Extinction requires new RNA and protein synthesis and the soma of the cell right pedal dorsal 1 in Lymnaea stagnalis. Journal of Neuroscience, 23, 9842–9851. Schiller, D., Cain, C. K., Curley, N. G., Schwartz, J. S., Stern, S., LeDoux, J. E., & Phelps, E. A. (2008). Evidence for recovery of fear following immediate extinction in rats and humans. Learning and Memory, 15, 394–402. Sissons, H. T., & Miller, R. R. (2009). Spontaneous recovery of excitation and inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 35, 419–426. Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57, 193–216. Stasiewicz, P. R., Brandon, T. H., & Bradizza, C. M. (2007). Effects of extinction context and retrieval cues on renewal of alcohol-cue reactivity among alcohol-dependant outpatients. Psychology of Addictive Behaviors, 21, 244–248. Storch, E. A., Merlo, L. J., Bengtson, M., Murphy, T. K., Lewis, M. H., Yang, M. C.,. . . Goodman, W. K. (2007). D-cycloserine does not enhance exposure-response prevention therapy in obsessive-compulsive disorder. International Clinical Psychopharmacology, 22, 230–237. Stout, S. C., & Miller, R. R. (2007). Sometimescompeting retrieval (SOCR): A formalization of the comparator hypothesis. Psychological Review, 114, 759–783.

2/15/2011 7:42:56 PM

BEHAVIORAL TECHNIQUES TO REDUCE RELAPSE

Tamai, N., & Nakajima, S. (2000). Renewal of formerly conditioned fear in rats after extensive extinction training. International Journal of Comparative Psychology, 13, 137–147. Thanos, P. K., Bermeo, C., Wang, G., & Volkow, N. D. (2009). D-cycloserine accelerates the extinction of cocaine-induced conditioned place preference in C57bL/c mice. Behavioural Brain Research, 199, 345–349. Thomas, B. L., & Ayres, J. J. B. (2004). Use of the ABA fear renewal paradigm to assess the effects of extinction with co-present fear inhibitors or excitors: Implications for theories of extinction and for treating human fears and phobias. Learning and Motivation, 35, 22–52. Thomas, B. L., Larsen, N., & Ayres, J. J. B. (2003). Role of context similarity in ABA, ABC, and AAB renewal paradigms: Implications for theories of renewal and for treating human phobias. Learning and Motivation, 34, 410–436. Thomas, B. L., Vurbic, D., & Novak, C. (2009). Extensive extinction in multiple contexts eliminates the renewal of conditioned fear in rats. Learning and Motivation, 40, 147–159. Tsao, J. C. I., & Craske, M. G. (2000). Timing of treatment and return of fear: Effects of massed, uniform-, and expanding-spaced exposure schedules. Behavior Therapy, 31, 479–497. Üngör, M., & Lachnit, H. (2008). Dissociations among ABA, ABC, and AAB recovery effects. Learning and Motivation, 39, 181–195. Urcelay, G. P., Lipatova, O., & Miller, R. R. (2009). Constraints on enhanced extinction resulting from extinction treatment in the presence of an added excitor. Learning and Motivation, 40, 343–363. Urcelay, G. P., Wheeler, D. S., & Miller, R. R. (2009). Spacing extinction trials alleviates renewal and spontaneous recovery. Learning and Behavior, 37, 60–73. Van Gucht, D., Vansteenwegen, D., Beckers, T., & Van den Bergh, O. (2008). Return of experimentally induce chocolate craving after extinction in a different context: Divergence between craving for and expecting to eat chocolate. Behavior Research and Therapy, 46, 375–391. Vansteenwegen, D., Hermans, D., Vervliet, B., Francken, G., Beckers, T., Baeyens, F., & Eelen, P. (2005). Return of fear in a human differential conditioning paradigm caused by a return to the original acquisition context. Behaviour Research and Therapy, 43, 323–336.

04-Schachtman-04.indd 103

103

Vansteenwegen, D., Vervliet, B., Hermans, D., Beckers, T., Baeyens, F., & Eelen, P. (2006). Stronger renewal in human fear conditioning when tested with an acquisition retrieval cue than with an extinction retrieval cue. Behaviour Research and Therapy, 44, 1717–1725. Vansteenwegen, D., Vervliet, B., Iberico, C., Baeyens, F., Van den Bergh, O., & Hermans, D. (2007). The repeated confrontation with videotapes of spiders in multiple contexts attenuates renewal of fear in spider-anxious students. Behaviour Research and Therapy, 45, 1169–1179. Vervliet, B., Vansteenwegen, D., Hermans, D., & Eelen, P. (2007). Concurrent excitors limit the extinction of conditioned fear in humans. Behaviour Research and Therapy, 45, 375–383. Vila, N. J., & Rosas, J. M. (2001). Renewal and spontaneous recovery after extinction in a causal-learning task. Mexican Journal of Behavior Analysis, 27, 79–96. Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. Spear & R. R. Miller (Eds.), Information processing in animals: Memory mechanisms (pp. 5–47). Hillsdale, NJ: Erlbaum. Walker, D. L., Ressler, K. J., Lu, K., & Davis, M. (2002). Facilitation of conditioned fear extinction by systemic administration or intra-amygdala infusions of D-cycloserine as assessed with fearpotentiated startle in rats. Journal of Neuroscience, 22, 2343–2351. Watson, J. B., & Rayner, R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3, 1–14. Westbrook, R. F., Smith, F. J., & Charnock, D. J. (1985). The extinction of an aversion: Role of the interval between non-reinforced presentations of the averted stimulus. Quarterly Journal of Experimental Psychology, 37B, 255–273. Wolpe, J. (1958). Psychotherapy by reciprocal inhibition. Stanford, CA: Stanford University Press. Woods, A. M., & Bouton, M. E. (2006). D-cycloserine facilitates extinction but does not eliminate renewal of the conditioned emotional response. Behavioral Neuroscience, 120, 1159–1162. Woods, A. M., & Bouton, M. E. (2008). Immediate extinction causes a less durable loss of performance than delayed extinction following either fear or appetitive conditioning. Learning and Memory, 15, 909–920.

2/15/2011 7:42:56 PM

CHAPTER 5 Learning and Anxiety A Cognitive Perspective Peter F. Lovibond

Traditionally, it has been assumed that associative learning is carried out by a low-level, reﬂexive, unconscious system, and accordingly associative explanations of clinical disorders have focused on irrational aspects of those disorders. However, research on human associative learning suggests that it depends critically on high-level, propositional, conscious processes. This perspective opens the door for an associative account of cognitive features of disorders, such as beliefs (rational and irrational). In this chapter I explore such an account, with a focus on anxiety. I argue that learned anxiety involves the development of threat beliefs regarding associations between antecedent stimuli and harmful outcomes. Performance is an automatic expectancy-based process. Threat expectancy and anxiety are also modulated by instrumental actions such as avoidance and safety behaviors. Threat beliefs may be established by direct experience, observation, language, and inference. Effective treatment can best be achieved by a coordinated combination of these same pathways.

In the mid 1980s I started working in human conditioning after 10 years of research in nonhuman animal conditioning. By that time I was already persuaded of the centrality of the role of expectancy in both learning and performance, and I was an enthusiast of informational, S-S models of conditioning (e.g., Kamin, 1969; Rescorla & Wagner, 1972). Therefore, I could be said to have had a cognitive orientation to nonhuman animal learning. Nonetheless, I was quite unprepared for what I observed in the human conditioning laboratory. I had assumed that associative learning was a fundamental ability that conferred so much adaptive advantage that it would be robust and ubiquitous. Instead, my human research participants found associative learning quite difﬁcult, and they would regularly fail to notice what seemed to me to be simple and obvious contingencies such as differential conditioning. Even more surprising was that there was a clear positive relationship between their verbally

expressed knowledge and their conditioned responses (CRs). I had fully expected that their CRs would serve as an index of associative links formed by a conditioning system quite separate from language and conscious belief. I soon discovered that my observations were far from atypical. It all started to make sense when I read a chapter by Michael Dawson and Anne Schell (1985), which carefully reviewed the human learning literature and concluded that associative learning in humans met the criteria for a controlled cognitive process. Before long I realized that the connection between associative learning in non-human animals and high-level cognitive processes in humans opened up far greater applications for associative models than the more traditional view I had grown up with. My subsequent research on human conditioning and anxiety over the following 20 years has explored these connections and applications. In this chapter I will brieﬂy summarize laboratory

104

05-Schachtman-05.indd 104

2/16/2011 5:48:10 PM

LEARNING AND ANXIETY

research on human associative learning, outline a propositional model of learning my colleagues and I have recently developed, and then consider the applications of this model to anxiety, anxiety disorders, and treatment.

THE ROLE OF COGNITIVE PROCESSES IN HUMAN ASSOCIATIVE LEARNING There are three main empirical ﬁndings that demonstrate the critical role of cognitive processes in human associative learning. First, contrary to popular belief, the development of Pavlovian conditioned responses (CRs) is closely associated with the development of explicit contingency knowledge—that is, consciously available and verbally expressible knowledge about the relationships between the conditioned stimulus (CS) and the unconditioned stimulus (US). Every systematic review of the human Pavlovian conditioning literature has reached this conclusion (e.g., Boakes, 1989; Brewer, 1974; Dawson & Schell, 1985; De Houwer, 2009; Lovibond & Shanks, 2002; Shanks, 2010). A similar pattern is observed for instrumental learning (Dawson & Schell, 1987; Shanks, Green, & Kolodny, 1994). Second, associative learning is strongly inﬂuenced by verbal instruction, for example, regarding acquisition and extinction (e.g., Grings, Schell, & Carey, 1973). Third, associative learning is subject to principles of reasoning. For example, Kamin’s (1969) blocking effect is only strongly observed when blocking is a permissible inference, both in the case of causal judgment (De Houwer, Beckers, & Glautier, 2002; Lovibond, Been, Mitchell, Bouton, & Frohart, 2003) and Pavlovian fear conditioning (Mitchell & Lovibond, 2002). The Dual-System Model of Associative Learning

Of course, nobody would deny that humans possess high-level cognitive abilities, nor that such abilities could be applied to the problem of learning associations. Therefore, the most common reaction to the extensive body of empirical evidence outlined in the previous section has been to propose that humans can

05-Schachtman-05.indd 105

105

achieve associative learning through two independent systems, one high level and one low level. The high-level system is generally considered to be symbolic and representational in nature, to depend on a limited capacity working memory mechanism, to be closely associated with language and consciousness, and to have emerged late in evolution. The low-level system is generally considered to involve excitatory and inhibitory links, to operate automatically and unconsciously, and to have emerged early in evolution. This proposed “dual-system” architecture is pervasive within the ﬁelds of learning, cognition, and neuroscience, and it has parallels in virtually every ﬁeld of psychology (e.g., Ashby & Maddox, 2005; Dienes & Perner, 1999; Evans, 2008; Kahneman & Frederick, 2002; Olson & Fazio, 2001; Reber, 1989; Sloman, 1996; Squire, 1994; Wilson, Lindsey, & Schooler, 2000). However, there are serious limitations to the ability of dual-system models to account for the phenomena of human associative learning. Empirical Evidence Against the DualSystem Model of Associative Learning

The ﬁrst difﬁculty faced by dual-system models is empirical. The clearest prediction from such models is that manipulations involving distraction and cognitive load should impact preferentially on the limited-capacity highlevel system, thus allowing the operation of the proposed low-level unconscious system to be observed. It should be common, for example, to observe development of CRs despite a lack of conscious awareness. In fact, however, the consistent outcome is that distraction and cognitive load impair both conscious contingency knowledge and CRs (e.g., Dawson & Biferno, 1973; Ross & Nelson, 1973). Similarly, verbal instruction should impact directly on the highlevel cognitive system and not on CRs, yet it affects both. A second difﬁculty for dual-system models concerns testability. Most dual-system models are implicitly endorsed rather than explicitly formulated, and hence they are not speciﬁed with sufﬁcient detail as to make clear testable predictions beyond the basic ones noted earlier

2/16/2011 5:48:10 PM

106

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(see Mitchell, De Houwer, & Lovibond, 2009b). In particular, there are no published dual-system models of learning that provide a complete speciﬁcation of how the two systems interact. Does the high-level system suppress output from the low-level system if their outcomes are in conﬂict? If both systems reach the same conclusion, is the behavioral output increased? Some dualsystem models, such as Squire’s (1994) declarative/procedural theory, allow that both systems can produce CRs. Such models could in principle account for the ability of verbal instruction to produce CRs, but they cannot account for the ability of verbal instruction to inhibit CRs generated by the low-level procedural system without additionally specifying a top-down inhibitory link. But if the high-level system is given too much control over the low-level system, the dual-system model effectively reduces to a single-system model because the two systems are no longer independent. A ﬁnal and related problem for dual-system models concerns parsimony. It will not have escaped the reader’s attention that all of the empirical evidence reviewed to this point could be explained by postulating a single system, namely the cognitive system. The clear prediction of dual-system models that unconscious learning should be commonplace is not supported by the evidence. These models are forced to suggest that the strong relationship between contingency awareness and CRs is simply a coincidence, and one that is all the more astonishing given the radically different architectures attributed to the cognitive and reﬂexive systems. Many examples of empirical dissociations in the literature that have been classically interpreted in dual-system terms are equally amenable to a single-system account (Chater, 2003; Kinder & Shanks, 2001; Newell & Dunn, 2008). If dualsystem models do not provide a substantial explanatory beneﬁt to justify their greater complexity, then a single-system model is to be preferred on the grounds of parsimony. I do not expect that all readers will be persuaded at this point. I have recently experienced the strength of commitment to dualsystem thinking in the reaction of commentators to a paper in which my colleagues and I have put

05-Schachtman-05.indd 106

the case for a single-system propositional model (Mitchell, De Houwer, & Lovibond, 2009a). It is likely that many readers will have their own favorite example of apparent unconscious learning. And there are certainly some data that are open to alternative interpretations, such as Perruchet’s (1985) gambler’s fallacy effect in eyeblink conditioning (see Weidemann, Tangen, Lovibond, & Mitchell, 2009). Nonetheless, I hope that readers will ﬁnd it valuable to explore the novel implications of a single-system approach to learning for anxiety and anxiety disorders, a ﬁeld that has been dominated by dual-system theorizing. At the very least, they may be persuaded that relatively more of the interesting phenomena of learning and anxiety can be understood in cognitive terms, and fewer by appealing to an unconscious link-based system. Accordingly, I will brieﬂy outline a single-system cognitive model of associative learning that my colleagues and I have recently developed (Mitchell et al., 2009a), before moving on to the applications to anxiety.

A SINGLE-SYSTEM PROPOSITIONAL MODEL OF ASSOCIATIVE LEARNING The essential features of our single-system model of associative learning are as follows. First, learning is a form of reasoning that is effortful and attention demanding. Second, learning gives rise to associative knowledge that is represented in a symbolic, propositional form, such as a causal belief, that is accessible to consciousness and to language. Finally, performance involves the utilization of acquired associative knowledge in ﬂexible ways. Knowledge of action-outcome contingencies can guide voluntary (instrumental) behavior. Knowledge of stimulus-outcome contingencies can guide expectancies (beliefs about future states of affairs), which in turn elicit speciesspeciﬁc anticipatory behaviors (CRs). Associative knowledge can also be combined with other knowledge to guide inference and further action. For a more complete presentation of this model, see Mitchell et al. (2009a). At this stage, however, it is important to dispense with some mischaracterizations of single-system models, and our model in particular.

2/16/2011 5:48:10 PM

LEARNING AND ANXIETY

First, our single-system model does not deny the role of lower level, unconscious processes. Indeed, it depends critically on processes such as perception, object recognition, and episodic memory. The critical difference between our model and dual-system models is that we deny that low-level unconscious processes constitute an independent associative system capable of directly driving behavior. Rather, we argue that low-level processes work cooperatively with higher level symbolic processes to achieve learning and performance, and that the output of this coordinated system is available to consciousness. Second, our model does not assume that all behavioral output is voluntary. There is good evidence that the performance of Pavlovian CRs is automatic, as shown for example by research on omission schedules in appetitive conditioning (Williams & Williams, 1969). However, automaticity is not in itself a reason to retreat to a reﬂexive, link-based model of learning. There are many examples of high-level cognitive processes that are automatic (for example, reading, as in the color Stroop task). Furthermore, the automatic nature of anticipatory behavior is not restricted to learning based on direct exposure to stimulus contingencies; it applies equally to knowledge gained symbolically—witness, for example, the emotional reaction of a person who receives bad news by verbal means. Similarly, our propositional model does not imply that all learning is logical, normative, and adaptive. There may well be biases, errors, and confusions that lead to faulty attributions and false beliefs. Finally, our model does not imply that nonhuman animals learn through a different system from humans, nor does it imply conversely that animals must have conscious beliefs of a human type. The similarity in behavioral response of humans and laboratory animals leads us to suppose that there is a great deal of overlap in mechanism. Accordingly we propose that laboratory animals such as rats have a precursor to the human cognitive system. Across the evolution of humans, we assume an increase in the sophistication of the cognitive system, leading to a progressively richer capacity to represent the environment, eventually allowing an artiﬁcial symbolic system (language) to emerge. Such an

05-Schachtman-05.indd 107

107

evolutionary pathway is far more plausible biologically than the traditional dual-system model, which assumes that humans inherited almost unchanged a primordial associative link system, and at the same time suddenly acquired a unique cognitive system that had little or no precedent in our immediate ancestors (for further discussion, see Mitchell, De Houwer, & Lovibond, 2009b).

APPLICATIONS OF THE SINGLE-SYSTEM PROPOSITIONAL MODEL TO ANXIETY AND ANXIETY DISORDERS Expectancy, Threat Beliefs, and Anxiety

The ﬁrst and most compelling advantage of a single-process, propositional model of associative learning is its compatibility with the “cognitive content” of learned anxiety. In the laboratory, participants who learn positive or negative contingencies between CSs and aversive USs such as electric shock are able to describe these relationships verbally, and when presented with CSs they can rate their expectancy of the US (e.g., Dawson & Biferno, 1973). These contingency beliefs and expectancy ratings are highly correlated with anxiety-related CRs such as skin conductance, heart rate, and startle modulation (e.g., Purkis & Lipp, 2001). We have recently extended these procedures to provide a laboratory model of instrumental avoidance learning (Lovibond, 2006; Lovibond, Saunders, Weidemann, & Mitchell, 2008). In this model, participants ﬁrst learn that a Pavlovian CS is a predictor of shock; that is, they receive CS-shock pairings. They then learn that an instrumental response (pressing a particular button) is able to cancel the shock predicted by the Pavlovian CS. As their belief in the efﬁcacy of the avoidance response increases, both their rated expectancy of shock and their skin conductance responses decline. But if response availability is suddenly withdrawn, there is an immediate return of shock expectancy and skin conductance responses. These laboratory data are consistent with the single-system view that expectancy of a harmful outcome is the critical cognitive precursor

2/16/2011 5:48:10 PM

108

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

to anxiety. According to this view, Pavlovian conditioning involves a progressive increase in the strength of a causal belief regarding the contingency between a stimulus and a harmful outcome. Such beliefs are stored in long-term memory. Presentation of the stimulus activates the belief, leading to an expectation of the harmful outcome, which in turn activates the anxiety system. The opportunity to perform an instrumental avoidance response can attenuate the expectation of harm and hence reduce anxiety. This view is highly consistent with cognitive appraisal models of anxiety that have been developed entirely independently by researchers working with anxiety and stress in naturalistic settings. For example, the classic cognitive model of Lazarus and Folkman (1984) proposes an initial “threat appraisal” component that involves the rapid detection of threat in the environment. Threat appraisal may, however, be moderated by “coping appraisal,” the perception that one has a response available that can avoid or mitigate the impact of the threatened event. The overall level of anxiety or stress that is elicited will be determined by the combination of these two forms of appraisal. The notion of coping appraisal also has parallels with Bandura’s (1977) concept of self-efﬁcacy, and (inversely) with Seligman’s (1975) concept of learned helplessness. The phenomenology of clinical anxiety disorders is likewise dominated by threat beliefs and expectancy of harm. In addition to the deﬁning diagnostic feature of anxious emotional arousal, patients with anxiety disorders report a wide variety of speciﬁc threat beliefs and expectancies that are consistent with the stimuli they fear and the behaviors in which they engage. For example, height phobics report exaggerated perceptions of the risk of falling and being injured (Menzies & Clark, 1995a); social phobics report a preoccupation with rejection and negative appraisal (Rapee, 1995), and patients with panic disorder report believing they they are going to collapse, go crazy, suffocate, or have a heart attack (Clark, 1986). According to a cognitive, propositional model of learning, the symptoms of anxiety and panic follow directly from these exaggerated threat beliefs. According to dual-system models,

05-Schachtman-05.indd 108

however, threat beliefs are causally irrelevant to anxiety—they either arise coincidentally from the operation of an independent cognitive system, or secondarily from interpretation of anxiety symptoms that are driven by an unconscious conditioning mechanism (Bouton, Mineka, & Barlow, 2001; Öhman & Mineka, 2001). The idea that a person’s conviction of his or her own imminent death has no casual role in anxiety whatsoever is hard to accept at face value. But if the concession is made that cognitive appraisal can contribute to anxiety, then the value of the dual-system model evaporates. Why postulate two systems to explain a phenomenon when one will do, especially when there is so little evidence for a separate unconscious system in the laboratory? More direct evidence for the causal role of threat beliefs comes from studies of the interaction between anxiety and coping behavior. Clinicians have noted that patients with anxiety disorders will often engage in “safety behaviors” in an attempt to forestall or minimize the catastrophic outcomes they fear. The behaviors that patients choose align closely with the outcomes they fear (Salkovskis, Clark, & Gelder, 1996). For example, social phobics who are concerned about blushing will wear high-necked clothing, and panic patients who fear collapsing will seek physical support. Furthermore, it has been shown that allowing patients to engage in safety behaviors interferes with exposure therapy. This type of therapy involves exposing patients to antecedent stimuli they fear, and it has been shown to be a powerful method for reducing both anxiety symptoms and threat beliefs. According to the single-system model, exposure works by the same mechanism as extinction in the laboratory: Experience with the antecedent stimulus (CS) in the absence of harm (US) contradicts and thereby weakens the threat belief, which in turn generates less anxiety on subsequent encounters. The reason that safety behaviors interfere with the extinction process is that they provide an alternative causal explanation for the absence of the harmful outcome, thus protecting the initial threat belief. Indeed, patients often report thinking that if they hadn’t engaged in a particular protective strategy, the

2/16/2011 5:48:10 PM

LEARNING AND ANXIETY

harmful event they fear would surely have eventuated. Clinical research has shown that asking patients to refrain from safety behaviors during exposure can increase the beneﬁts they obtain from this therapy (e.g., Salkovskis, Clark, Hackmann, Wells, & Gelder, 1999). A similar pattern emerges from an older literature on obsessive-compulsive disorder, in which patients engage in a variety of “neutralizing behaviors” that are closely tied to their obsessions—for example, excessive washing in the case of patients concerned with contamination, or cognitive rituals such as thinking “good thoughts” in the case of patients who believe their “bad thoughts” will cause harm to themselves or others. Clinical research with this disorder has shown that exposure therapy is most effective when combined with “response prevention”; in other words, asking the patient to experience the antecedent stimuli he or she fears (e.g., touching a door handle) while refraining from neutralizing behavior (Abramowitz, 1997). The single-system propositional model provides a coherent account of the strong interrelationships observed between threat beliefs, safety behavior, and anxiety. Traditional dualsystem models, by contrast, attribute anxiety to an unconscious Pavlovian conditioning system, safety behavior to an unconscious instrumental response-shaping system, and threat beliefs to a causally irrelevant cognitive system. Any coherence among these phenomena is essentially coincidental. To examine more directly the role of safety and threat beliefs in the extinction of Pavlovian conditioned fear, we have carried out a number of studies on “protection from extinction” in the laboratory. This phenomenon refers to the ability of an inhibitory CS to interfere with extinction of an excitatory CS when the inhibitory CS is presented at the same time as the excitatory CS during nonreinforced exposure (Rescorla, 2003; Soltysik, Wolfe, Nicholas, Wilson, & Garcia-Sanchez, 1983). We have shown this effect both with a Pavlovian CS (Lovibond, Davis, & O’Flaherty, 2000) and more recently with an instrumental avoidance response (Lovibond, Mitchell, Minard, Brady, & Menzies, 2009). In each case the effect was observed on

05-Schachtman-05.indd 109

109

both self-report US expectancy and on skin conductance responding. The parallel with the ability of safety stimuli and safety behaviors to interfere with exposure therapy in patients is striking. In both of these situations, a source of safety (in the environment, or arising from the person’s own actions) prevents the reduction in threat value of a feared stimulus when it is presented without any adverse consequence. Furthermore, the explanations given by learning theorists and clinicians are highly similar. For example, the well-established Rescorla-Wagner (1972) model of associative learning explains protection from extinction in terms of the inhibitory CS canceling the expectation of the US that is elicited by the excitatory CS, thus eliminating the “surprisingness” of the nonoccurrence of the US, and leaving the excitatory strength of the excitatory CS unchanged. This explanatory convergence, coupled with the congruence that is observed between anxiety and threat beliefs in both the laboratory and the clinic, strongly suggests that the Rescorla-Wagner model is capturing critical functional characteristics of a propositional learning system. In other words, there is good reason to consider that associative theories such as the Rescorla-Wagner model, and the non-human animal conditioning data they were designed to account for, have relevance not for supposedly unconscious processes in humans but for mainstream conscious cognition—reasoning and belief formation (Mitchell et al., 2009a). Irrational Anxiety

If learning in humans is a form of reasoning, as proposed by a single-system propositional model, an important question that arises is how learning processes could explain irrational behavior observed in patients. One answer to this question is the same as that provided by traditional dual-system models—namely, unfortunate learning experiences. An obvious example is exposure to a traumatic event or a sustained period of abuse or mistreatment. But there is opportunity for maladaptive learning even in more common situations. For example, a child whose ﬁrst experience with public speaking leads to embarrassment and humiliation may

2/16/2011 5:48:11 PM

110

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

represent social situations and their potential threat in a way that is resistant to subsequent corrective information. Processes such as situational avoidance and within-situation safety behaviors may further protect these initial threat beliefs from disconﬁrmation. A second way in which irrational behavior can arise in patients is through faulty reasoning. As noted earlier, the acquisition and modiﬁcation of beliefs is not necessarily a cold and logical process. A popular example in the reasoning literature is the Wason (1966) selection task, which is answered incorrectly by more than 90% of university students (Johnson-Laird & Wason, 1970). Reasoning errors often arise from selection of the wrong strategy, for example, a shortcut heuristic that has worked in similar situations in the past but is not applicable to a target situation (Tversky & Kahneman, 1974). Social psychologists have identiﬁed a wide range of shortcut strategies (for example, conformity, stereotyping) that are self-serving or save mental work but lead to unfortunate consequences such as intolerance (e.g., Hilton & von Hippel, 1996). In clinical populations, examples of faulty reasoning are similarly abundant. The majority of contemporary models of psychopathology identify faulty reasoning and maladaptive beliefs as the core locus of disturbance—for example, threat beliefs in anxious patients, beliefs about body shape in patients with eating disorders, and beliefs in one’s own inadequacy in depression (e.g., Beck, 1967; Clark, 1986; Waller, Ohanian, Meyer, & Osman, 2000). As discussed further in the sections that follow, clinical interventions similarly focus on the correction of maladaptive beliefs and the reasoning errors that serve to maintain them. Although it is certainly the case that cognitive psychologists themselves often appeal to unconscious processes with an associative ﬂavor to explain errors in reasoning (e.g., Sloman, 1996), a viable alternative is that human reasoning itself is an imperfect process (see Evans, 2008). Routes to Anxiety

It is well accepted in the clinical literature that anxiety reactions may arise through at least three routes: associative learning, observation, and

05-Schachtman-05.indd 110

language (e.g., Rachman, 1977). The singlesystem propositional model of learning readily accommodates these three etiological pathways, because they all provide information that is relevant to the development of a causal representation of the environment in general, and potential threat in particular. Dual-system theories, by contrast, typically assert that anxiety arising from associative learning and observational learning is mediated by a low-level unconscious system, whereas anxiety arising from language is mediated by the high-level cognitive system (e.g., Le Doux, 1996; Öhman & Mineka, 2001). Thus, evidence for integration between sources of information, particularly between associative learning and language, is problematic for the dual-system model. Lovibond (2003) used a retrospective revaluation design in autonomic fear conditioning to test for integration of this type in the laboratory. The design required two pieces of information (AB+ trials and A– trials), given in successive phases, to be combined in order to “solve” the task, namely to revalue upward the causal status of stimulus B. Participants were able to perform this revaluation when the two pieces of information were both provided by direct exposure to the contingencies, when they were both provided by verbal instruction, and critically, when one piece of information was provided by direct exposure and the other by instruction. The latter result demonstrates successful integration of causal knowledge about threat acquired through direct experience and through language, implying that associative learning gives rise to information at a symbolic level similar to that conveyed by language. The single-system propositional model suggests a further route to anxiety that has similarities with an associative learning episode but does not involve direct experience with harm, namely the “near-miss” experience. The most harmful event that can befall any organism is its own death, but by deﬁnition any organism that experiences this outcome is not going to learn about it! However a close brush with death is a different matter. A subset of traumatic events that lead to posttraumatic stress syndrome (PTSD) involve no direct physical harm, but

2/16/2011 5:48:11 PM

LEARNING AND ANXIETY

they do involve the perception on the part of individuals that they were going to die or could easily have died. A traditional reﬂexive theory (e.g., Hull, 1943; Levis, 1989) would attribute learning in such a case to the formation of a stimulus-response (S-R) association between stimuli present at the time and the fear response. However, there is little direct evidence for S-R learning in the laboratory in either non-human animals or humans, and good evidence for Pavlovian S-S learning, that is, learning about the actual stimulus relationships in the environment (e.g., Dickinson, 1980). The propositional model offers an alternative account of near-miss learning, based on reasoning on the part of the individual that he or she nearly experienced a catastrophic outcome and could actually experience that outcome if exposed to the same situation again. This account offers the ability to account for the cognitive content of near-miss learning (e.g., rumination), as well as circumstances in which fear learning does not occur, such as when the person attributes the nonoccurrence of harm to a stable source of safety (Lovibond, 2001). A ﬁnal route to anxiety that needs to be considered is the genetic route. Disentangling the roles of genetic and learning processes in fear and anxiety has proven to be surprisingly difﬁcult. One relatively clear ﬁnding is that trait anxiety, the general tendency to be chronically anxious or be easily provoked to anxiety, has a strong genetic basis. This trait is associated with anxiety disorders, as well as depressive disorders (Hettema, Neale, & Kendler, 2001). It is generally considered that environmental triggers work additively or multiplicatively with trait anxiety to determine the level of state anxiety at any time (e.g., Barlow, 2002). One environmental factor that has received a great deal of attention, of course, is associative learning. Since Watson and Rayner (1920), learning theorists have argued that fear reactions, especially speciﬁc phobias that are tied to particular stimuli such as insects, heights, or water, have an associative basis. The primary support for this idea is that conditioning with an aversive US is sufﬁcient to establish a fear reaction to any arbitrary stimulus. Therefore, it is a plausible hypothesis that an observed fear

05-Schachtman-05.indd 111

111

reaction to a speciﬁc stimulus may have had its origin in conditioning. However, it became apparent that the range of stimuli that supported phobic reactions is very restricted, and not in a way that would be predicted by conditioning theory. For example, the majority of phobic stimuli are naturally occurring rather than human made, yet many human-made stimuli are associated with pain and trauma. Thus, genetic factors may again play a role, but this time in relation to speciﬁc stimuli rather than general trait anxiety. The ﬁrst person to articulate this possibility in a formal model was Seligman (1971) in his classic “preparedness” theory of phobias. Seligman did not abandon associative learning in favor of genetic inﬂuences; rather, he supplemented associative learning with the idea of biological belongingness. According to this idea, potentially phobic stimuli such as insects are initially neutral, like other stimuli, but are particularly predisposed or prepared to enter into fear learning if paired with an aversive event in a particular individual. If the aversive event is sufﬁciently traumatic, a single pairing with a prepared stimulus was thought to be able to establish a long-lasting phobia. Seligman’s (1971) model is an important one to consider in the context of unconscious conditioning and the single-/dual-system debate, because Seligman suggested that prepared associations may be formed by an especially primitive and cognitively inpenetrable conditioning mechanism. Furthermore, speciﬁc phobia is often presented as a clear example of irrational learning (e.g., Öhman & Mineka, 2001). One immediate problem with the direct conditioning model of speciﬁc phobia, including Seligman’s (1971) preparedness model, is that patients with speciﬁc phobia often cannot recall a conditioning episode in their past (e.g., Menzies & Clarke, 1995b; Öst, 1991). At face value, it seems extremely unlikely that an event sufﬁciently traumatic to condition a long-lasting phobia would be forgotten by the patient. Furthermore, people who are exposed to a documented traumatic event, if they develop any long-lasting symptoms at all, tend to show a distinct pattern of symptoms known as posttraumatic stress disorder (PTSD) rather than

2/16/2011 5:48:11 PM

112

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

speciﬁc phobia (Bryant, 2003). Far from being unable to recall their traumatic event, patients with PTSD experience intrusive thoughts and ﬂashbacks of the trauma. Thus, while associative processes are clearly implicated in PTSD, the research on traumatic events does not support a conditioning account of speciﬁc phobia. One proposal for rescuing the conditioning model of speciﬁc phobia appeals to the phenomenon of infantile amnesia. Since most speciﬁc phobias are present from childhood, it has been suggested that the relevant traumatic conditioning experience may have occurred in the ﬁrst few years of life and subsequently been forgotten (e.g., Öst & Hugdahl, 1981). However, there are ﬁndings that are inconsistent with this proposal. For example, parents of young children with strong fears, such as of water, often report no adverse experience of the type expected by conditioning theory, such as a near-drowning (Menzies & Clark, 1993). A particularly interesting study tested the conditioning hypothesis by making use of an existing longitudinal data set, the Dunedin Multidisciplinary Health and Development Study (Poulton, Davies, Menzies, Langley, & Silva, 1998). The researchers reasoned that participants who reported a strong fear of heights should, according to the conditioning hypothesis, have experienced a greater frequency of injuries arising from falls during early childhood, compared to a matched nonfearful control group. Astonishingly, the Dunedin dataset contained the information they needed to test this hypothesis. When Poulton and colleagues examined the records, they discovered that not only did the height fearful group not have more falls or injuries, they actually had signiﬁcantly fewer such events. Poulton et al. (1998) interpreted their results in terms of a nonassociative model of phobia (Menzies & Clark, 1995b; Poulton & Menzies, 2002). According to this model, the stimuli that support phobic reactions do indeed have a genetic prepotency to elicit fear. However, the degree of fear to particular stimuli will show genetic variation across individuals. Furthermore, latent fear reactions can be activated by nonspeciﬁc stress or traumatic events (sensitization), or attenuated by prolonged exposure (habituation).

05-Schachtman-05.indd 112

Menzies and Poulton argued that many patients report no onset experience at all; they have had their fear as long as they can remember. Of those who do report a clear point of onset, the circumstances tend to ﬁt the sensitization model (stressor preceding exposure to the eliciting stimulus) better than the associative model (stressor closely following exposure to the eliciting stimulus). Although there have been critiques of the nonassociative account as the sole route to phobia (e.g., Mineka & Öhman, 2002), it appears to offer a viable alternative to the traditional direct pairing conditioning model. The nonassociative account of phobia can also be applied to laboratory research on autonomic conditioning in humans with prepared or “fear-relevant” stimuli such as pictures of snakes and spiders (Öhman, Erixon, & Lofberg, 1975). In this preparation, threat of electric shock, in the absence of an actual pairing with shock, is sufﬁcient to markedly enhance skin conductance responses to fear-relevant stimuli by comparison to fear-irrelevant control stimuli (Öhman, Eriksson, Fredriksson, Hugdahl, & Olofsson, 1974). When fear-relevant stimuli are paired with shock, they do not show superior conditioning, but they do tend to show a resistance to extinction effect, which is consistent with Seligman’s preparedness model (McNally, 1987). However, some of my own work suggests that this effect may be due to a combination of normal associative learning and selective sensitization to fear-relevant stimuli (Lovibond, Siddle, & Bond, 1993), in line with the nonassociative model proposed by Poulton and Menzies (2002). Ironically, then, the anxiety disorder for which conditioning seemed the perfect explanation, speciﬁc phobia, has yielded relatively little evidence in support of an associative mechanism. It is likely, however, that associative experiences play a more substantial role in other anxiety disorders. For example, socially based fears offer many more opportunities for relevant adverse events such as negative evaluation to have occurred. Panic disorder provides potential pairings between panic attacks and speciﬁc predictive cues. And as noted earlier, associative processes are clearly involved in posttrauma reactions such as PTSD. In all these cases,

2/16/2011 5:48:11 PM

LEARNING AND ANXIETY

the propositional model can not only account for the role of associative events, it can also account for the threat beliefs that closely track the anxiety reactions, as well as the circumstances under which they emerge. For example, a person who attributes a panic attack to having consumed too much coffee will show very different reactions in the future than someone who attributes a panic attack to a brain tumor or a weak heart (Clark, 1986). Regardless of the role played by associative learning episodes in particular anxiety disorders, learning retains an important role in research because it provides a ﬂexible laboratory model for the study of anxiety. If the dual-system model is correct, the study of learning can inform us only about unconsciously elicited anxiety reactions. If, however, the propositional model is correct, then the study of learned anxiety is a model preparation for the study of expectancy, reasoning, threat beliefs, and voluntary safety behaviors. Insights from the study of these processes can then be applied to anxiety reactions in the clinic, independently of whether these reactions arose from a conditioning episode, observational learning, a near-miss experience, or symbolic transmission. Implications for Treatment

If associative learning is a sufﬁcient basis for establishing an anxiety/fear reaction, then it follows that associative procedures such as extinction may be effective in ameliorating such reactions. This insight provided the basis for one of the earliest examples of behavior therapy, namely exposure therapy (see McNally, 2007). There is a further insight that arises speciﬁcally from a propositional model of associative learning. That is, all of the routes to anxiety discussed previously lead to the development of associative knowledge that is represented in the same way, as a causal belief linking particular antecedent stimuli to particular harmful outcomes. Accordingly, it is possible to use one or more of these routes to reverse an anxiety reaction, regardless of the route(s) by which it was established. Furthermore, the most effective strategy would be to combine multiple routes in a coordinated fashion so as to target the speciﬁc

05-Schachtman-05.indd 113

113

dysfunctional causal belief (Bandura, 1977). It is important to note that the dual-system perspective does not make this prediction. Instead, the dual-system model asserts that the therapeutic route has to match the acquisition route. For example, it predicts that it would be ineffective to use exposure to try to reverse a fear reaction that had been established symbolically or to use verbal therapy to reverse a fear reaction established by associative learning (e.g., Brewin, 1989). Yet “treatment matching” of this (or any other) type has yielded disappointing results (e.g., Haaga, Rabois, & Brody, 1999; Heather, 2008), and contemporary cognitive-behavioral therapies (CBT) routinely combine verbal and experience-based procedures in a coordinated fashion (e.g., Dobson & Dobson, 2009; Hawton, Salkovskis, Kirk, & Clark, 1989). In this respect, the single-system propositional model of learning is consistent with the dominant empirically validated approach to psychopathology and intervention, CBT. The “cognitive” part of CBT derives, not surprisingly, from cognitive therapy, an approach that was developed after behavior therapy, but not from laboratory work in cognition. Instead, it emerged from the clinical ﬁeld (e.g., Beck, 1967) and was based on essentially a lay perspective on the role of thoughts and beliefs in clinical disorders, although it was later supplemented by constructs from cognitive psychology. The term cognitive in cognitive therapy has two primary meanings. First, it conveys that the target of therapy is a maladaptive cognitive structure, or set of beliefs. In this regard, it is consistent with a propositional model of associative learning. Second, the term cognitive refers to a therapeutic procedure that is delivered verbally. The effectiveness of verbal intervention is again consistent with a propositional associative model; we have already seen that in the laboratory, verbal instruction can establish and reverse anxiety reactions. However, according to the propositional model, the use of the term cognitive to refer to verbal intervention is too narrow, because other routes such as direct experience of a contingency, or observational learning, are also symbolic and hence cognitive. The term behavioral also smacks of dual-system thinking and

2/16/2011 5:48:11 PM

114

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

fails to convey the critical element of such interventions, namely the use of direct exposure to new, corrective associations. Thus, I have argued that it would be preferable to use more descriptive, theoretically neutral terms to describe the mode of interventions, such as “verbal” or “symbolic” in place of cognitive, and “experiencebased” in place of behavioral (Lovibond, 1993). A good example of the interplay between verbal and experience-based procedures is in the delivery of exposure therapy. A classic behavioral approach to exposure is based on maximizing the opportunity for extinction of conditioned anxiety, usually considered as arising from a low-level, unconscious system. However, no therapist will deliver exposure without some verbal accompaniment or explanation, and this verbal information will often include elements of safety, control, and lack of danger. Similarly, although cognitive therapists sometimes use purely verbal interventions (e.g., Jones & Menzies, 1998), they will more often incorporate some element of direct exposure. This exposure will generally take the form of a “behavioral experiment,” designed to test and thereby refute the patient’s excessive belief of danger. The most effective interventions are those that combine exposure and verbal information in a coordinated, synergistic fashion (Hawton et al., 1989). Thus, the patient’s attention is drawn to the contradiction between the outcome of exposure (no harm) and his or her prior beliefs. This process also allows potentially counterproductive alternative attributions to be identiﬁed and dealt with, both verbally and through additional tailored experiences. We have already seen one example of this process in the role of safety and neutralizing behaviors, where the patient attributes the absence of harm to his or her own protective actions, rather than changing his or her appraisal of the target fear-eliciting stimuli. Another example is attribution to an external source of safety, or to luck, as in the “near-miss” appraisal sometimes seen in panic patients (Salkovskis et al., 1996). An important question to ask at this point is whether the different routes to changing associative beliefs are all equally effective. There is good reason to believe they are not. It has commonly

05-Schachtman-05.indd 114

been suggested that direct experience is more effective than verbal information, with observational learning in between (e.g., Reiss, 1980). Although these differences could be explained in terms of discrete learning systems, a plausible alternative explanation within the single-system propositional model is the personal relevance of the information. Direct experience is concrete and immediately relevant—if something has happened to you before, it can happen again. Experiences that occur to others are potentially relevant, but they might not apply to oneself. Finally, language is often the least reliable source—many things we are told turn out not to be true. Conversely, there are other advantages to language that allow it to play an important role in an overall intervention. It is a highly ﬂexible method that allows delivery of rich and detailed information. When it is only possible to provide limited direct experience, language can help ensure the most adaptive interpretation of that experience, for example, to help a patient generalize from one positive experience to a feeling of mastery and control. The direct experience may be necessary to challenge ingrained irrational beliefs, but the verbal procedures can help direct the patient to develop more positive and adaptive beliefs (Lovibond, 1993). The single-system propositional model also offers a new perspective on the role of observational learning in anxiety. According to this model, when we observe a harmful outcome occur to another person, we draw an inference of danger along the lines of “If it happened to that person in this situation, it could happen to me in a similar situation.” The strength of this inference may be attenuated in proportion to perceived dissimilarity of oneself to the other person. As with direct experience, this type of learning is likely to be adaptive or normative in most circumstances, the exception being when unusually extreme or unlikely outcomes are observed. However, the majority of research on observational learning has examined a different type of observational learning scenario, in which the observer is exposed to another person displaying a fear reaction to a particular stimulus. From a propositional perspective, this situation is more complex and may have greater potential for

2/16/2011 5:48:11 PM

LEARNING AND ANXIETY

excessive threat appraisal. First, the observer has to correctly infer the external stimulus that is causing fear in the other person. Second, the potential harmful outcome is not directly speciﬁed, and it must also be inferred. In the case where the observer is a child, and the other person is an adult such as a parent, the inference is presumably along the lines that “I don’t know what it is that this stimulus can do, but if it is scaring my dad so much, it must be terrible!” It is easy to see how a process of this type could yield a causal belief that is not only exaggerated and generalized but also contentless and therefore hard to correct with subsequent exposure—a “nameless dread.” It would be useful to directly compare these two types of observational learning opportunity and to assess the interpretations that each induces. Another ﬁeld in which a propositional perspective may yield new insights and applications concerns the phenomenon of relapse after treatment. There has now been considerable research in both non-human animals and humans showing that extinction of learned fear is fragile, and that the original fear memory (or belief) may return to control behavior as a result of stress, context change, or the passage of time (Bouton, 2002; Hermans, Craske, Mineka, & Lovibond, 2006). The return of learned fear offers a laboratory model for the relapse sometimes shown by patients after successful treatment such as exposure. What the propositional model predicts is that the return of fear, and the range of circumstances in which it occurs, will depend on inferential processes. For example, a change of context may raise the (understandable) doubt that the recent benign experience with a previously feared stimulus is unrepresentative and may not apply in a new situation. Similarly, the passage of time may lead to discounting of the current relevance of the exposure experience. The experience of harm, especially in the same domain as the feared stimulus (i.e., exposure to the US alone), may likewise cast doubt on the idea that the previous threat has permanently disappeared. If return of fear involves inferential reasoning, then it follows that interventions should focus on establishing causal beliefs on the part of the patient that are

05-Schachtman-05.indd 115

115

realistic and that provide a clear basis for objective threat appraisal across changes in context and time. For example, if negative events of the type feared by the patient (e.g., criticism and rejection in the case of social phobia) are in fact likely to occur at some point in the future, then it would be advantageous not only to expose the patient to scenarios of this type, but to train the patient in how to interpret these experiences in a nonthreatening way. Future Directions

One important direction for future cognitive research in anxiety is to explore the interface between cognitive and biological factors. In my previous work, I had assumed that innate fear reactions were directly coded in an S-R fashion (e.g., snake → fear), and that exposure therapy for such fear reactions operated not by reduction of threat appraisal but by a separate process such as habituation (e.g., Lovibond, 1993). However, there are some lines of evidence that suggest there may be considerable overlap in the mechanisms underlying innate and learned fear. I have already noted the evidence for an innate basis for speciﬁc phobia, yet speciﬁc phobics display exaggerated threat beliefs and expectancies in line with their anxiety levels (Menzies & Clarke, 1995a, 1995b). If the fear is innate, where do these cognitive appraisals come from? A classic study by Killcross and Balleine (1996) provides a possible laboratory model for the innate representation of biologically signiﬁcant events. In that study, preexposure of a neutral stimulus led to latent inhibition (interference with subsequent associative learning involving that stimulus; see Lubow & Moore, 1959) only when the US it was paired with matched the motivational state experienced during preexposure. Thus, rats only showed latent inhibition for pairings between a CS and a food US when they had been preexposed to the CS while hungry, and only showed latent inhibition for pairings with a water US when they had been preexposed to the CS while thirsty. Yet neither food nor water weas presented during preexposure. Killcross and Balleine (1996, p. 41) concluded: “Clearly, animals learn that the preexposed stimulus is unrelated to events of relevance to their current motivational state.”

2/16/2011 5:48:11 PM

116

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Two more recent non-human animal studies can also be interpreted in a similar way. Rescorla (2008) used his compound test procedure (Rescorla, 2002) to test both appetitive and aversive conditioning to stimuli with differing natural (innate) levels of attractiveness such as sucrose and quinine. He showed that stimuli whose innate level of attractiveness was opposite to the valence of the US showed the greatest associative change, the same pattern as for stimuli whose level of attractiveness had been established by previous associative learning. This outcome suggests that innate stimulus value of a to-be-CS acts in a similar way as learned stimulus value in modulating further learning. A study by Pineño, ZilskiPineño, and Miller (2008) is even more directly relevant to the present chapter. These researchers showed that habituation of an innate fear/avoidance reaction, taste neophobia, could be blocked by the presence of a “safe” taste, that is, one that had previously been trained as a conditioned inhibitor for illness. In other words, they showed a “protection from habituation” effect, suggesting that habituation of an innate fear response may involve a mechanism similar to that underlying extinction of a learned fear response. All of these studies suggest that innate emotional reactions may be coded in a similar way to learned emotional reactions. If, as I have argued throughout this chapter, learned emotion arises from a propositional expectancy-based mechanism, then the intriguing possibility arises that our genes might encode cognitive appraisals such as beliefs or expectancies regarding biologically signiﬁcant classes of event. Although it is hard to imagine the mechanism whereby stimulus representations can be encoded in genes, results such as these suggest that genetic inﬂuences may be mediated through the cognitive representational system, rather than directly activating response systems. In the case of anxiety, this suggests that the relevance of particular stimuli for particular classes of harmful events may turn out to be coded in our genes.

model of learning that has dominated previous attempts to apply principles of associative learning to the clinical domain. In particular, this model fails to account for the strong relationship between conditioned responding and conscious propositional knowledge; the effect of verbal instruction; and the role of reasoning processes. When applied to clinical anxiety disorders, the dual-system model fails to account for the pivotal role of reasoning and contingency beliefs in both the acquisition and the remediation of anxiety reactions, it relies too heavily on direct CS-US pairings in the etiology of anxiety reactions, and it fails to account for the efﬁcacy of verbal (“cognitive”) interventions. As an alternative to the traditional dualsystem model of associative learning, I brieﬂy described a single-system propositional model of associative learning and considered its implications for anxiety and anxiety disorders. According to the single-system propositional model, learning consists of the strengthening and weakening of causal beliefs concerning the relationship between antecedent events (stimuli and actions) and harmful outcome events. Performance arises when antecedent stimuli signaling harm are present, leading to an expectation of harm that activates the anxiety system. This model is consistent with the cognitive content of anxiety and anxiety disorders. It suggests that a variety of sources of information, including direct experience, observation, language, near-miss experiences, and possibly genetic inﬂuences, can all feed into an inferential reasoning process that leads to the establishment of causal beliefs regarding threat. Several of these routes can also be used to reduce excessive threat beliefs, and the single-system propositional model suggests the most effective strategy is to combine these sources, regardless of the initial route of acquisition. The model provides guidance in combining different sources of information in a systematic, coordinated fashion to target speciﬁc abberant beliefs in therapy.

CONCLUSIONS

REFERENCES

In this chapter I have brieﬂy summarized the empirical evidence against the generic dual-system

Abramowitz, J. S. (1997). Effectiveness of psychological and pharmacological treatments for

05-Schachtman-05.indd 116

2/16/2011 5:48:11 PM

LEARNING AND ANXIETY

obsessive-compulsive disorder: A quantitative Review. Journal of Consulting and Clinical Psychology, 65, 44–52. Ashby, F. G., & Maddox, W. T. (2005). Human category learning. Annual Review of Psychology, 56, 149–178. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice-Hall. Barlow, D. H. (2002). Anxiety and its disorders: The nature and treatment of anxiety and panic (2nd ed.). New York, NY: Guilford Press. Beck, A. T. (1967). Depression: Clinical, experimental, and theoretical aspects. New York, NY: Hoeber. Boakes, R. A. (1989). How one might ﬁnd evidence for conditioning in adult humans. In T. Archer & L-G. Nilsson (Eds.), Aversion, avoidance and anxiety: Perspectives on learning and memory (pp. 381–402). Hillsdale, NJ: Erlbaum. Bouton, M. E. (2002). Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction. Biological Psychology, 52, 976–986. Bouton, M. E., Mineka, S., & Barlow, D. H. (2001). A modern learning theory perspective on the etiology of panic disorder. Psychological Review, 108, 4–32. Brewer, W. F. (1974). There is no convincing evidence for operant or classical conditioning in adult humans. In W. B. Weimer & D. S. Palermo (Eds.), Cognition and the symbolic processes (pp. 1–42). Hillsdale, NJ: Erlbaum. Brewin, C. R. (1989). Cognitive change processes in psychotherapy. Psychological Review, 96, 379–394. Bryant, R. A. (2003). Early predictors of posttraumatic stress disorder. Biological Psychiatry, 53, 789–795. Chater, N. (2003). How much can we learn from double dissociations? Cortex, 39, 167–169. Clark, D. M. (1986). A cognitive approach to panic. Behaviour Research and Therapy, 24, 461–470. Dawson, M. E., & Biferno, M. A. (1973). Concurrent measurement of awareness and electrodermal classical conditioning. Journal of Experimental Psychology, 101, 55–62. Dawson, M. E., & Schell, A. M. (1985). Information processing and human autonomic classical conditioning. In P. K. Ackles, J. R. Jennings, & M. G. H. Coles (Eds.), Advances in psychophysiology (Vol., 1, pp. 89–165). Greenwich, CT: JAI Press. Dawson, M. E., & Schell, A. M. (1987). Human autonomic and skeletal classical conditioning: The role of conscious cognitive factors. In G. Davey (Ed.), Cognitive processes and Pavlovian

05-Schachtman-05.indd 117

117

conditioning in humans (pp. 27–55). Oxford, England: John Wiley & Sons. De Houwer, J. (2009). The propositional approach to associative learning as an alternative for association formation models. Learning and Behavior, 37, 1–20. De Houwer, J., Beckers, T., & Glautier, S. (2002). Outcome and cue properties modulate blocking. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 55A, 965–985. Dickinson, A. (1980). Contemporary animal learning theory. Cambridge, England: Cambridge University Press. Dienes, Z., & Perner, J. (1999). A theory of implicit and explicit knowledge. Behavioral and Brain Sciences, 22, 735–808. Dobson, D., &Dobson, K. S. (2009). Evidencebased practice of cognitive-behavioral therapy. New York, NY: Guilford Press. Evans, J. S. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255–278. Grings, W. W., Schell, A. M., & Carey, C. A. (1973). Verbal control of an autonomic response in a cue reversal situation. Journal of Experimental Psychology, 99, 215–221. Haaga, D. A. F., Rabois, D., & Brody, C. (1999). Cognitive behavior therapy. In M. Hersen & A. S. Bellack (Eds.), Handbook of comparative interventions for adult disorders (2nd ed., pp. 48–61). Hoboken, NJ: John Wiley & Sons Inc. Hawton, K., Salkovskis, P. M., Kirk, J., & Clark, D. M. (Eds.). (1989). Cognitive behaviour therapy for psychiatric problems: A practical guide. New York, NY: Oxford University Press. Heather, N. (2008). UK alcohol treatment trial: Client-treatment matching effects. Addiction, 103, 228–238. Hermans, D., Craske, M. G., Mineka, S., & Lovibond, P. F. (2006). Extinction in human fear conditioning. Biological Psychiatry, 60, 361–368. Hettema, J. M., Neale, M. C., & Kendler, K. S. (2001). A review and meta-analysis of the genetic epidemiology of anxiety disorders. American Journal of Psychiatry, 158, 1568–1578. Hilton, J. L., & von Hippel, W. (1996). Stereotypes. Annual Review of Psychology, 47, 237–271. Hull, C. L. (1943). Principles of behavior. New York, NY: Appleton-Century-Crofts. Johnson-Laird, P. N., & Wason, P. C. (1970). A theoretical analysis of insight into a reasoning task. Cognitive Psychology, 1, 134–148.

2/16/2011 5:48:11 PM

118

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Jones, M. K., & Menzies, R. G. (1998). Danger ideation reduction therapy (DIRT) for obsessive-compulsive washers: A controlled trial. Behaviour Research and Therapy, 36, 959–970. Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. In T. D. Gilovich, D. W. Grifﬁn, & D. Kahneman (Eds.), Heuristics and biases (pp. 49–81). New York, NY: Cambridge University Press. Kamin, L. J. (1969). Predictability, surprise attention and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York, NY: Appleton-Century-Crofts. Killcross, S., & Balleine, B. (1996). Role of primary motivation in stimulus preexposure effects. Journal of Experimental Psychology: Animal Behavior Processes, 22, 32–42. Kinder, A., & Shanks, D. R. (2001). Amnesia and the declarative/nondeclarative distinction: A recurrent network model of classiﬁcation, recognition, and repetition priming. Journal of Cognitive Neuroscience, 13, 648–669. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal and coping. New York, NY: Springer. Le Doux, J. (1996). The emotional brain. New York, NY: Simon & Schuster. Levis, D. J. (1989). The case for a two-factor theory of avoidance: A failure of nonfear interpretations. In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 227–277). Hillsdale, NJ: Erlbaum. Lovibond, P. F. (1993). Conditioning and cognitive-behavior therapy. Behavior Change, 10, 119–130. Lovibond, P. F. (2001). The “near miss” as a fourth pathway to anxiety. Behavioural and Cognitive Psychotherapy, 29, 35–43. Lovibond, P. F. (2003). Causal beliefs and conditioned responses: Retrospective revaluation induced by experience and by instruction. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 97–106. Lovibond, P. F. (2006). Fear and avoidance: An integrated expectancy model. In M. G. Craske, D. Hermans, & D. Vansteenwegen (Eds.), Fear and learning: From basic processes to clinical implications (pp. 117–132). Washington, DC: American Psychological Association. Lovibond, P. F., Been, S. L., Mitchell, C. J., Bouton, M. E., & Frohardt, R. (2003). Forward and

05-Schachtman-05.indd 118

backward blocking of causal judgment is enhanced by additivity of effect magnitude. Memory and Cognition, 31, 133–142. Lovibond, P. F., Davis, N. & O’Flaherty, A. (2000). Protection from extinction in human fear conditioning. Behaviour Research and Therapy, 38, 967–983. Lovibond, P. F., Mitchell, C. J., Minard, E., Brady, A., & Menzies, R. G. (2009). Safety behaviours preserve threat beliefs: Protection from extinction of human fear conditioning by an avoidance response. Behaviour Research and Therapy, 47, 716–720. Lovibond, P. F., Saunders, J. C., Weidemann, G., & Mitchell, C. J. (2008). Evidence for expectancy as a mediator of avoidance and anxiety in a laboratory model of human avoidance learning. Quarterly Journal of Experimental Psychology, 61, 1199–1216. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Lovibond, P. F., Siddle, D. A. T., & Bond, N. W. (1993). Resistance to extinction of fear-relevant stimuli: Preparedness or selective sensitization? Journal of Experimental Psychology: General, 122, 449–461. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition: The effect of nonreinforced preexposure to the conditional stimulus. Journal of Comparative and Physiological Psychology, 52, 415–419. McNally, R. J. (1987). Preparedness and phobias: A review. Psychological Bulletin, 101, 283–303. McNally, R. J. (2007). Mechanisms of exposure therapy: How neuroscience can improve psychological treatments for anxiety disorders. Clinical Psychology Review, 27, 750–759. Menzies, R. G., & Clarke, J. C. (1993). The etiology of childhood water phobia. Behaviour Research and Therapy, 31, 499–501. Menzies, R. G., & Clarke, J. C. (1995a). Danger expectancies and insight in acrophobia. Behaviour Research and Therapy, 33, 215–221. Menzies, R. G., & Clarke, J. C. (1995b). The etiology of phobias: A nonassociative account. Clinical Psychology Review, 15, 23–48. Mineka, S., & Öhman, A. (2002). Born to fear: Non-associative vs associative factors in the etiology of phobias. Behaviour Research and Therapy, 40, 173–184.

2/16/2011 5:48:11 PM

LEARNING AND ANXIETY

Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009a). The propositional nature of human associative learning. Behavioral and Brain Sciences, 32, 183–198. Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009b). Link-based learning theory creates more problems than it solves. Behavioral and Brain Sciences, 32, 230–246. Mitchell, C. J., & Lovibond, P. F. (2002). Backward and forward blocking in human electrodermal conditioning: Blocking requires an assumption of outcome additivity. The Quarterly Journal of Experimental Psychology, 55B, 311–329. Newell, B. R., & Dunn, J. C. (2008). Dimensions in data: Testing psychological models using statetrace analysis. Trends in Cognitive Sciences, 12, 285–290. Öhman, A., Eriksson, A., Fredriksson, M., Hugdahl, K., & Olofsson, C. (1974). Habituation of the electrodermal orienting reaction to potentially phobic and supposedly neutral stimuli in normal human subjects. Biological Psychology, 2, 85–93. Öhman, A., Erixon, G., & Lofberg, I. (1975). Phobias and preparedness: Phobic versus neutral pictures as conditioned stimuli for human autonomic responses. Journal of Abnormal Psychology, 84, 41–45. Öhman, A., & Mineka, S. (2001). Fears, phobias and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108, 483–522. Olson, M. A., & Fazio, R. H. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 12, 413–417. Öst, L-G. (1991). Acquisition of blood and injection phobia and anxiety response patterns in clinical patients. Behaviour Research and Therapy, 29, 323–332. Öst, L-G., & Hugdahl, K. (1981). Acquisition of phobias and anxiety response patterns in clinical patients. Behaviour Research and Therapy, 19, 439–447. Perruchet, P. (1985). A pitfall for the expectancy theory of human eyelid conditioning. Pavlovian Journal of Biological Science, 20, 163–170. Pineño, O., Zilski- Pineño, J. M., & Miller, R. R. (2008). Habituation of unconditioned fear can be attenuated by the presence of a safe stimulus: Assessment using the neophobic response of the rat. Behavioural Processes, 77, 55–60. Poulton, R., Davies, S., Menzies, R. G., Langley, J. D., & Silva, P. A. (1998). Evidence for a nonassociative model of the acquisition of a fear of

05-Schachtman-05.indd 119

119

heights. Behaviour Research and Therapy, 36, 537–544. Poulton, R., & Menzies, R. G. (2002). Non-associative fear acquisition: A review of the evidence from retrospective and longitudinal research. Behaviour Research and Therapy, 40, 127–149. Purkis, H. M., & Lipp, O. V. (2001). Does affective learning exist in the absence of contingency awareness? Learning and Motivation, 32, 84–99. Rachman, S. (1977). The conditioning theory of fear-acquisition: A critical examination. Behaviour Research and Therapy, 15, 375–387. Rapee, R. M. (1995). Descriptive psychopathology of social phobia. In R. G. Heimberg, M. R. Liebowitz, D. A. Hope, & F. R. Schneier, (Eds.), Social phobia: Diagnosis, assessment, and treatment (pp. 41–66). New York, NY: Guilford Press. Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psychology: General, 118, 219–235. Reiss, S. (1980). Pavlovian conditioning and human fear: An expectancy model. Behavior Therapy, 11, 380–396. Rescorla, R. A. (2002). Savings tests: Separating differences in rate of learning from differences in initial levels. Journal of Experimental Psychology: Animal Behavior Processes, 28, 369–377. Rescorla, R. A. (2003). Protection from extinction. Learning and Behavior, 31, 124–132. Rescorla, R. A. (2008). Conditioning of stimuli with nonzero initial value. Journal of Experimental Psychology: Animal Behavior Processes, 34, 315–323. Rescorla, R. A., & Wagner, A. R. (1972) A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II (pp. 64–99). New York, NY: Appleton-Century-Crofts. Ross, L. E., & Nelson, M. N. (1973). The role of awareness in differential conditioning. Psychophysiology, 10, 91–94. Salkovskis, P. M., Clark, D. M., & Gelder, M. G. (1996). Cognition-behaviour links in the persistence of panic. Behaviour Research and Therapy, 34, 453–458. Salkovskis, P. M., Clark, D. M., Hackmann, A., Wells, A., & Gelder, M. G. (1999). An experimental investigation of the role of safetyseeking behaviours in the maintenance of panic disorder with agoraphobia. Behaviour Research and Therapy, 37, 559–574.

2/16/2011 5:48:11 PM

120

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Seligman, M. E. (1971). Phobias and preparedness. Behavior Therapy, 2, 307–320. Seligman, M. E. P. (1975). Helplessness: On depression, development, and death. New York, NY: W. H. Freeman. Shanks, D. R. (2010). Learning: From association to cognition. Annual Review of Psychology, 61, 273–301. Shanks, D. R., Green, R. E., & Kolodny, J. (1994). A critical examination of the evidence for nonconscious (implicit) learning. In C. Umlita & M. Moscovitch (Eds.), Attention and performance XV: Conscious and nonconscious information processing (pp. 837–860). Cambridge, MA: MIT Press. Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119, 3–22. Soltysik, S. S., Wolfe, G. E., Nicholas, T., Wilson, W. J., & Garcia-Sanchez, J. L. (1983). Blocking of inhibitory conditioning within a serial conditioned stimulus-conditioned inhibitor compound: Maintenance of acquired behavior without an unconditioned stimulus. Learning and Motivation, 14, 1–29. Squire, L.R. (1994). Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory. In D. L. Schacter & E. Tulving (Eds.), Memory systems. (pp. 203–231). Cambridge, MA: MIT Press.

05-Schachtman-05.indd 120

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131. Waller, G., Ohanian, V., Meyer, C., & Osman, S. (2000). Cognitive content among bulimic women: The role of core beliefs. International Journal of Eating Disorders, 28, 235–241. Wason, P. C. (1966). Reasoning. In B. M. Foss (Ed.), New horizons in psychology (pp. 135–151). Harmondsworth, England: Penguin. Watson, J. B., & Rayner, R. R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3, 1–14. Weidemann, G., Tangen, J., Lovibond, P. F., & Mitchell, C. J. (2009). Is Perruchet’s dissociation between eyeblink conditioned responding and outcome expectancy evidence for two learning systems? Journal of Experimental Psychology: Animal Behavior Processes, 35, 169–176. Williams, D. R., & Williams H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511–520. Wilson, T. D., Lindsey, S., & Schooler, T. Y. (2000). A model of dual attitudes. Psychological Review, 107, 101–126.

2/16/2011 5:48:11 PM

CHAPTER 6 Trauma, Learned Helplessness, Its Neuroscience, and Implications for Posttraumatic Stress Disorder Vincent M. LoLordo and J. Bruce Overmier

Forty years ago it was discovered that animals that had previously experienced inescapable aversive events, but not ones that had learned to escape these events, subsequently failed to learn to escape aversive events in a new situation with new task demands. This ﬁnding gave rise to a major theoretical development, the learned helplessness hypothesis, and stimulated an enormous amount of research on the varied effects of lack of control. One focus of this chapter is research from the Maier-Watkins laboratory on the neural basis of learned helplessness effects, culminating in recent work on the role of medial prefrontal cortex in mediating the differential effects of escapable versus inescapable aversive events. Then applications of learned helplessness to human psychopathology are considered, with the focus on parallels between learned helplessness and posttraumatic stress disorder (PTSD). These include Pavlovian fear conditioning to stimuli associated with the trauma, as well as sensitization, whereby learned helplessness rats and PTSD patients acquire new fears readily, show deﬁcits in extinction of fear, and also exhibit exaggerated startle responses, an aspect of hyperarousal. Parallels between the neural bases of learned helplessness and PTSD are considered; the most studied are enhanced activation of the amygdala and attenuated activation of medial prefrontal cortex in learned helplessness/PTSD.

LEARNED HELPLESSNESS Aversive experiences and even trauma are a common part of life. Whether these experiences change our future lives depends on how we cope with them initially. This is not speculation but is based on knowledge derived from experiments, experiments that arose from early research on trauma-induced interference with adaptive behavior and the learned helplessness hypothesis developed to account for that interference. Wide-ranging investigations over the last 40 years, and continuing today, have illuminated the behavioral and physiological foundations of the trauma-induced dysfunctions that have implications for depression and, even more, for posttraumatic stress disorder (PTSD) that is so much in the news today.

A quick search of the literature on PsychINFO reveals nearly 3,000 papers that deal with the phenomena of learned helplessness or use the construct to account for other phenomena in a variety of applied settings. Here we shall trace the development of some of the lines of research and applications. To truly understand where the construct applies, one needs a solid understanding of its experimental bases and proposed mechanisms. We begin with a review of the core critical experiments, some controversies, and analyses of the underlying behavioral, anatomical, and neurophysiological mechanisms. Then we turn to applications to human psychopathology, primarily to PTSD. Some of the exposition is rather detailed because in this sort of research seemingly small differences in procedure are critical, and interpretations may

121

06-Schachtman-06.indd 121

2/15/2011 7:43:32 PM

122

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

turn on the details. It is a long story but an intriguing one.

THE INTERFERENCE EFFECT AND THE LEARNED HELPLESSNESS HYPOTHESIS Systematic research on what has come to be called “learned helplessness” began with an experiment (Overmier & Seligman, 1967) in which some dogs received a large number of unsignaled, inescapable, and unavoidable painful shocks to the hind paws while they were restrained in a hammock, and others were simply restrained. A day later all the dogs were given escape/avoidance training in a shuttle box. The dogs could escape painful shocks delivered through a grid ﬂoor by jumping a shoulder-high barrier separating the two compartments of the shuttle box. The shocks were signaled by a visual stimulus, and the dogs could also avoid the shocks by jumping in the presence of the warning signal. The outcome was striking. Most of the dogs that had been shocked on the previous day failed to learn to escape in the shuttle box. On the other hand, dogs that had only been restrained on the previous day readily learned to jump to turn off the shock; their escape latencies decreased markedly over trials. This interference with escape, which was robust if the test occurred 1 day after the induction phase, did not occur in other groups that were tested 2, 3, or 6 days after inescapable shock. However, interference with avoidance behavior lasted a month (see reanalysis of Overmier & Seligman data by Overmier,

Table 6.1

Patterson, & Wielkiewicz, 1980, p.11). Seligman and Groves (1970) found that the interference effect on escape behavior persisted at least a week following four sessions of inescapable shock distributed over 8 days. Overmier and Seligman (1967) proposed that the inescapability of the shocks in the hammock was responsible for the interference effect. To test this hypothesis, Seligman and Maier (1967) conducted an experiment with three groups of dogs. All were placed in the hammock with Plexiglas panels on both sides of their heads. One group could turn off shock by pressing either panel once. For a second group the panels were nonfunctional. For each dog in this group the duration of shock on a trial was matched (called yoked) to that received by a dog in the escapable shock group. Thus, the two groups received the same pattern of shock durations across trials. Operationally, only the escapability of shocks distinguished the two groups. The third group was just restrained in the harness. This is called the triadic design, which is illustrated in Table 6.1. When the dogs were tested in the shuttle box a day later, most of the previously inescapably shocked dogs failed to learn to escape shock. In contrast, nearly all dogs in the other two groups learned to escape. Thus, the inescapability of the shocks in the hammock was presumed responsible for the interference effect in the shuttle box. Overmier, Seligman, and Maier (Overmier & Seligman, 1967; Seligman & Maier, 1967) proposed the learned helplessness hypothesis to account for this interference effect. This was a

The Triadic Design

Day 1 Restrained in hammock

Day 2 In shuttle box

Panel press turns off shocks (ES)

Crossing barrier turns off shocks Nearly all dogs learn to escape

Yoked, inescapable shocks (IS)

Crossing barrier turns off shocks Most dogs fail to learn to escape

No shocks

Crossing barrier turns off shocks Nearly all dogs learn to escape

ES, escapable shock; IS, inescapable shock.

06-Schachtman-06.indd 122

2/15/2011 7:43:32 PM

TRAUMA AND LEARNED HELPLESSNESS

cognitive account; it maintained that the inescapably shocked dogs learned that none of the responses they made in the hammock affected the likelihood that shock would end and thus came to expect that their behaviors would not turn off shock. The next day, when shock occurred in an otherwise very different situation, this expectancy of helplessness was retrieved, and thus the dogs did not attempt to escape shock. In addition to this motivational or response initiation deﬁcit, there was an associative or cognitive deﬁcit. If on a trial the dog did jump the barrier and turn off shock, it would learn little from this coincidence because it had already learned over 64 trials in the hammock that there was no relation between its behaviors and shock termination. Of course, if a given dog never escaped in the shuttle box, then its behavior would provide no evidence for the associative deﬁcit (Klosterhalfen & Klosterhalfen, 1983). There was also an emotional deﬁcit. After a few shocks in the shuttle box, the previously inescapably shocked dogs seemed to become apathetic, passively accepting the shocks (Seligman, Maier, & Solomon, 1971). Dogs that had learned to press a panel to escape shock in the hammock could not acquire the expectation that their behaviors would not turn off shock, and thus would not exhibit these deﬁcits. Hereafter, effects like the interference effect described earlier, which depend on the operational inescapability of the shocks or other stressors, will be called learned helplessness effects (Maier & Watkins, 2005). This term simply labels behavioral outcomes that differ in magnitude in previously escapably versus inescapably shocked animals. Use of the term does not imply acceptance of the learned helplessness hypothesis, which has always been controversial.

GENERALITY OF THE INTERFERENCE EFFECT An important, though sometimes overlooked, feature of these experiments is that the induction and test phases occurred in very different situations; in the induction phase the dogs were suspended in a hammock and restrained, but in the escape/avoidance phase they could move around

06-Schachtman-06.indd 123

123

in a large box in a different room. It is likely that the experimenters, whose research agenda was driven by learning theory, conducted the two phases in different contexts because they wanted to demonstrate the generality of the interference effect, generality not only in the sense that the effect was transsituational but also in the sense that it occurred even when the required escape response was not a behavior that the animal had emitted in the induction phase. Dess and Overmier (1989; also see LoLordo & Taylor 2001) noted that, if we substitute conditioned stimuli (CSs) for responses and unconditioned stimuli (USs) for shocks, the learned helplessness hypothesis can be seen as anticipating the related hypothesis that if an animal has had prior experience with uncorrelated presentations of the CS and US and then those stimuli are repeatedly paired, acquisition of a conditioned response to the CS will be retarded because the animal has learned the lack of contingency between the two events. This was called learned irrelevance (Baker, 1976; Baker & Mackintosh, 1979; Mackintosh, 1973), and its existence is still debated (e.g., Bonardi, Hall, & Ong, 2005). In any case, learned helplessness postulates a broader range of deﬁcits than learned irrelevance; the latter would not be formally equivalent to learned helplessness unless it entailed learning that CSs in general are uncorrelated with a US. Dess and Overmier (1989) and Linden, Savage, and Overmier (1997) noted deficits in Pavlovian conditioning when a target CS was paired with a US following uncorrelated presentations of nontarget CSs and that US. This outcome suggests that animals can learn that CSs in general are uncorrelated with a US, but more research is needed. Even broader would be the claim that organisms can learn/infer that CSs in general are uncorrelated with USs in general, and the analogous claim that they can learn/infer that responses in general are uncorrelated with outcomes in general. Although Williams and Maier (1977) noted that the learned helplessness hypothesis was vague concerning transfer rules for responses and outcomes, it can be construed as completely general, and there are results that are consistent with such a broad formulation. In an early

2/15/2011 7:43:32 PM

124

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

experiment Overmier (1968) showed that dogs that had been inescapably shocked in the hammock were slower than restrained controls to learn a signaled avoidance task in the shuttle box, where jumping in the presence of the warning signal turned it off and prevented shock, and where in the absence of an avoidance response on a trial the subsequent shock was too brief to be escaped. To account for the observed interference effect, the learned helplessness hypothesis would have to argue that the animal’s expectation of lack of control extended beyond lack of control during shock. Two other examples would call for similar reasoning. Lee and Maier (1988) tested groups of rats on choice escape from room-temperature water after they had been exposed to inescapable shock (IS), shock that could be escaped by turning a wheel (escapable shock [ES]), or no shock. Inescapable shocks resulted in an interference effect on water-escape choice accuracy, relative to the other groups, if the rats had to learn a position discrimination with brightness cues irrelevant. Rosellini (1978) used the triadic design to show that rats acquired a lever-press response that was followed by food after a 1-second delay more slowly if they had a history of inescapable shock, rather than no shock or shock that could be escaped by jumping onto a platform. In both of these experiments the two phases occurred in different contexts. In the 1970s rats became the most common species in research on the interference effect. The most widely used procedure with rats has been one developed by Maier, Albin, and Testa (1973). In the triadic design, semirestrained groups of rats with shock electrodes attached to their tails have access to a small wheel mounted in the wall. From 80 to 100 intense shocks are given in a single session of 80–100 minutes. Rats in the escapable shock group can turn the wheel with their paws to turn off shock, and the response requirement is adjusted so that (1) the response is an operant rather than a reﬂexive response for the escape group, and (2) yoked rats receive enough shock to produce the deﬁcits; escape latencies at the end of the session were around 5 seconds (Maier & Jackson, 1977). Shock durations for rats in the inescapable shock group are yoked to those received by rats in the escapable

06-Schachtman-06.indd 124

shock group. The third group is simply restrained. Twenty-four hours later the rats receive escape/ avoidance training in a shuttle box. On the ﬁrst 1–5 trials in various experiments the rat has to cross to the other side once to turn off shock (FR-1 trials). Thereafter the rat must cross and then come back to escape (FR-2 trials). Typically the groups do not differ on FR-1 trials, but the previously inescapably shocked rats have much longer escape latencies than the others on FR-2 trials. A broadly construed learned helplessness hypothesis would suggest that a prior history of control over shock could immunize an animal against learned helplessness resulting from inescapable shock, even when the immunization and test phases occur in different situations and require dissimilar escape responses. Williams and Maier (1977) demonstrated such immunization; prior experience with wheel-turn escape markedly reduced the ability of inescapable tail shocks in a restraining tube to produce escape latency deﬁcits in a shuttle box (also see Amat et al., 2005; Christianson et al., 2008, Nakajima, Nakajima, & Imada, 1999). Immunization may have implications for PTSD (Basoglu et al., 1997). If learned helplessness effects are important for PTSD, then one factor that might distinguish those who develop PTSD following a trauma from those who do not is the presence of immunizing experiences in the lives of the latter. Many other learned helplessness effects have been demonstrated in experiments in which the test phase does not require the rat to learn some instrumental response. These effects will be described in later sections of this paper. The next section will focus on an important characteristic of learned helplessness effects.

DURATION OF VARIOUS LEARNED HELPLESSNESS EFFECTS How long does the effect last? Maier, Coon, McDaniel, Jackson, and Grau (1979) found that rats that received 80 5-sec, inescapable tail shocks while restrained in a tube showed a marked escape deﬁcit, compared to restrained controls, if they were tested in the shuttle box a day later.

2/15/2011 7:43:32 PM

TRAUMA AND LEARNED HELPLESSNESS

However, if 48 or 168 hours separated the phases, there was no deﬁcit. This result parallels the earlier ﬁnding with dogs. In a second experiment, designed to assess the contribution of inactivity to the escape deﬁcit, Maier et al. (1979) replaced the FR-2 escape/avoidance trials in the shuttle box with trials on which the 5-sec signal was followed by a 30-sec inescapable footshock. They measured activity in terms of the number of times the rats made two or more crosses during shock. The results mirrored those of their ﬁrst experiment; rats that were tested 1 day after receiving inescapable shocks were much less active than restrained controls, but rats tested 48 or 168 hours after inescapable shock showed no activity deﬁcit. Several other behaviors that are differentially affected by inescapable versus escapable shock show a similar time course. Jackson, Maier, and Coon (1979) demonstrated another striking consequence of inescapable shock. After the ﬁve FR-1 trials, the FR-2 trials in the shuttle box were replaced by the tail-ﬂick test of pain sensitivity, a test that measures the latency of a discrete tail twitch in response to heat applied to the tail and thus requires little activity. The rats that had received inescapable shock on the previous day showed longer tail-ﬂick latencies than rats that had learned to escape shock or restrained controls. Maier et al. (1979) observed that this analgesic effect of inescapable shock was greatly reduced if the test occurred 2 days after the shocks, and it disappeared altogether if a week separated the two phases. It now seems unlikely that either reduced activity during shock in the shuttle box or the analgesia that can be demonstrated after the FR-1 trials is responsible for the shuttle box escape deﬁcit. MacLennan et al. (1982) showed that neither removal of the pituitary or injection of dexamethasone, which blocks the stress-induced release of adrenocorticotropic hormone (ACTH) and beta-endorphin from the anterior pituitary a few hours before the inescapable shocks had any effect on the escape learning deﬁcit a day later, although both manipulations completely prevented the longterm analgesia. So the analgesia is not necessary for the escape deﬁcit. Drugan and Maier (1983) subsequently showed that dexamethasone given

06-Schachtman-06.indd 125

125

a few hours before inescapable shocks signiﬁcantly reduced the activity deﬁcit in the presence of 30-sec shocks 24 hours later. The activity was measured in the open ﬁeld, and shocks were tail shocks, rather than shocks from a grid ﬂoor. Drugan and Maier concluded that the activity deﬁcit is not necessary for the escape deﬁcit in the shuttle box (also see Lawry et al., 1978; LoLordo & Taylor, 2001, and Maier & Seligman, 1976, for a comprehensive early review of the relation between activity and escape and other theoretical issues). Minor (1990) observed that, compared to ES and restrained control groups, rats given IS showed an enhanced neophobic reaction to a novel peppermint odor. This neophobia, which was reﬂected by suppression of drinking, did not occur if the interval between IS and test was 72 hours. Short and Maier (1993), using the triadic design and a 24-hour interval between the induction and test phases, found that relative to escapable shock or restraint, inescapable shock resulted in a decrease in the amount of social interaction (e.g., snifﬁng, following, grooming) when two similarly treated adult male rats were placed together in a novel context for 10 minutes. In a second experiment that included only inescapably shocked and restrained groups, separate groups were tested 24, 48, 72, or 168 hours after the ﬁrst treatment. Inescapable shock suppressed social interaction 24 or 48 hours later, but not at longer intervals. Christianson, Paul, et al. (2008) have recently shown that at induction-test intervals of 12 or 36 hours an inescapably shocked male rat interacts less with a naïve juvenile male than do escapably shocked or restrained rats. This result reﬂects the adults from the IS group actively disengaging from the juvenile. Sutton, Lea, et al. (1997) used the triadic design to demonstrate that prior inescapable shock potentiated the analgesic response to a low dose of morphine (1 mg/kg) in the tail-ﬂick test when 24 or 48 hours elapsed between the induction phase and the morphine injection, but not when that interval was 72 hours. Morphine can be used to condition place preferences in rats. Will, Watkins, and Maier (1998) showed that,

2/15/2011 7:43:32 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

relative to escapable shock and restraint, inescapable shocks potentiated morphine-based place preference conditioning when the interval between the ﬁrst treatments and the start of the 2-day place preference conditioning treatment was 24 hours. In another experiment with just inescapable shock and home cage control groups (HC), the potentiation again occurred with a 24-hour interval between the two treatments, was weaker if the interval was 4–6 days, and did not occur with a 14-day interval. Another consequence of the inescapable shock treatment described earlier is exaggerated freezing in the shuttle box following shocks on one or two FR-1 escape trials. To study the relationship between fear, as measured by freezing, and escape behavior, Maier (1990) modiﬁed the shuttle box test procedure. The ﬁrst FR-1 trial was delayed for 10 minutes so that fear that generalized from the wheel turn box to the shuttle box could be assessed. It is known that inescapable shocks condition more fear to the shock context (e.g., Mineka, Cook, & Miller, 1984; also see Minor & LoLordo, 1984) than escapable shocks, so one might expect more generalized fear in the shuttle box in the group that received inescapable shock on the previous day than in the rats that had received escapable shock. Maier observed this, but the absolute levels of freezing were low, and in several replications no freezing was seen in either group (Amat et al., 2005; Amat, Paul, Watkins, & Maier, 2008; Greenwood et al., 2003). After the ﬁrst two FR-1 shock escape trials, freezing was observed for another 20 minutes. Both groups (ES and yoked IS) that had been shocked in the wheel-turn box froze more than the restrained controls, and the yoked IS group froze much more than the ES group (see Fig. 6.1). Maier argued that although the greater preshock freezing in inescapably shocked than in escapably shocked rats reﬂected differences in the amount of Pavlovian fear conditioning in the wheel-turn box, which generalized to the shuttle box, the exaggerated postshock freezing in the IS group reﬂected greater potentiation by the IS of fear conditioning to the shuttle box resulting from the shock on the FR-1 trial. This potentiation was said to be a nonassociative effect, sensitization. A second experiment showed that while

06-Schachtman-06.indd 126

14 12 Mean freezing

126

10 8

R E Y

6 4 2 0 1 2 3 4 5 6 7 8 9 10 2 Min blocks

Figure 6.1 Mean number of observation periods

for which freezing was observed after two shocks in the shuttle box, across 2-minute blocks, for previously restrained (R), escapably shocked (E), and yoked (Y) inescapably shocked rats in Experiment 1 of Maier (1990). (Reprinted with the author’s permission from Maier, S. F. 1990. The role of fear in mediating the shuttle escape learning deﬁcit produced by inescapable shock. Journal of Experimental Psychology: Animal Behavior Processes, 16, 137–150).

the generalized fear survived a 72-hour delay between IS and the test, the exaggerated postshock freezing completely disappeared, that is, such freezing was no greater than in the ES group. Recently Baratta et al. (2007) conducted a formally similar, but parametrically different experiment and obtained quite different results. Rats received either ES or yoked IS in the wheelturn chamber, or were HC controls. A week later all received a single pairing of a tone and a foot shock in a different context. On the following day all were tested for fear, as assessed by freezing, in the conditioning context, and for fear of the tone in a third, novel context. Relative to HC controls, IS potentiated, and ES suppressed, fear in both the conditioning context and in the presence of tone (see Fig. 6.2). Although the prophylactic effect of ES is exciting, our focus has been on the effects of IS, and no more will be said about this effect of ES here. In another experiment the treatments in the wheel-turn box occurred a day after context fear conditioning, and context fear was assessed a week later in a series of extinction tests. IS rats froze more than HC controls. The authors argued that because IS occurred a day after fear conditioning, it must

2/15/2011 7:43:32 PM

TRAUMA AND LEARNED HELPLESSNESS Context test

A

Percent freezing

100 75 ES 50

IS HC

25 0

B Percent freezing

100

Tone test

75 50

ES IS

25

HC

0 Pretone

Tone

Figure 6.2 (A) Mean percent freezing in the con-

ditioning context for groups of rats given escapable shock (ES), inescapable shock (IS), or no shock (HC) before fear conditioning. (B) Mean percent freezing to the altered experimental context (pretone) and to the tone conditioned stimulus for groups given ES, IS, or HC before fear conditioning. (Reprinted with the author’s permission from Baratta, M. V. et al. 2007. Controllable versus uncontrollable stressors bidirectionally modulate conditioned but not innate fear. Neuroscience, 146, 1495–1503).

have affected the expression of the conditioned fear, rather than the strength of the association between context and shock. The extent to which this relatively long-lasting sensitization effect and the more transient one described by Maier (1990) are different in kind, rather than just quantitatively, remains to be seen. In an effort to ﬁnd a learned helplessness effect that would reﬂect a cognitive/associative deﬁcit, Jackson, Alexander, and Maier (1980) developed a choice escape task. After shock exposures according to the triadic design, the next day the rat was placed in one arm of a Y-maze in a dark room containing few extramaze cues, footshock was administered, and the

06-Schachtman-06.indd 127

127

rat had to turn in a particular direction to end the shock. If the rat was required to turn right, and it instead turned left, it would then have to turn right to end the shock. Thus, there was no particular place that was consistently safe. The authors argued that if the previously inescapably shocked rats had learned that there was no contingency between their behaviors in the wheelturn apparatus and shock termination, then they would be slow to associate a speciﬁc turn with shock termination in the Y-maze, and thus would make more errors than the ES and restrained groups. Jackson et al. (1980) obtained this result. In a subsequent experiment they showed that administering the choice escape task a week after the induction phase did not reduce the IS rats’ choice escape deﬁcit relative to restrained controls. The robust retention of the effect is consistent with the argument that it reﬂects a cognitive/associative learning deﬁcit. In later research prompted by several failures to replicate these ﬁndings, Minor, Jackson, and Maier (1984) demonstrated that the learned helplessness effect on choice escape accuracy only occurred if the test had two characteristics: (1) a short, variable delay of shock termination following the correct response, and (2) the presence of task-irrelevant external cues that varied from trial to trial but could not be used to solve the problem. These cues were lights that came on behind one of the three end walls of the maze, with the location of the light on a given trial uncorrelated with the location of the arm into which a correct turn would take the rat. Again IS rats had a higher percentage of trials with an error than ES rats or restrained controls (also see Minor, Pelleymounter, & Maier, 1988).

SIMILARITY OF INDUCTION AND TEST ENVIRONMENTS AFFECTS DURATION OF LEARNED HELPLESSNESS EFFECTS All the experiments discussed so far administered the induction and test treatments in different environments. Prompted by ﬁndings from other laboratories that administered the two treatments in the same environment and found long-lasting escape learning deﬁcits

2/15/2011 7:43:32 PM

128

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(e.g., Greenwood, Strong, Dorey, & Fleshner, 2007; Hannum, Rosellini, & Seligman, 1976; Malberg & Duman, 2003), Maier and Watkins (2005) examined escape learning in the shuttle box at various time intervals after the rats received inescapable tail shocks in a restraint tube on a table or while they were freely moving in one side of the shuttle box. The rats that had received tail shocks in the tube 24 or 48 hours before the tests showed a marked escape learning deﬁcit, but the IS 72-hour and 168-hour groups looked just like the restrained controls. On the other hand, all the rats that had been given tail shocks in the shuttle box showed an escape learning deﬁcit, which was as great 168 hours after IS as it was 24 hours after IS (see Fig. 6.3). In a related experiment, Greenwood and Fleshner (2008) looked at freezing and escape learning in the shuttle box a week after rats had received tail shocks in a restraint tube on a table, or in the shuttle box with their front paws in contact with the grid. At the beginning of the session in the shuttle box, before any shocks were administered, neither the no-shock controls nor the rats given IS outside the shuttle box showed any generalized freezing, but the rats given IS in the shuttle box initially froze on about 30% of the

observations. Furthermore, there was a substantial escape latency deﬁcit in the rats that received tail shock inside the shuttle box, while the ones that had received tail shock outside the shuttle box learned to escape as well as the no-shock controls. It is likely that the increased generalized fear in the shuttle box in rats given IS in the shuttle box inhibited the escape response. Their FR-2 escape latencies were much longer than in no-shock controls from the very beginning. On the other hand, the initial latencies in rats that had been given IS outside the shuttle box 24 or 48 hours earlier and eventually showed the escape deﬁcit were no longer than those in controls. In summary, what may seem to be a small procedural difference has profound effects on behavior, presumably because an additional process is in play when animals are tested in the induction context.

REMINDER CUES EXTEND THE DURATION OF LEARNED HELPLESSNESS EFFECTS In a most important demonstration, Maier (2001) showed that the duration of the learned helplessness effect on shuttle box escape latencies could

Same

Different 30

20 24 Hr 15

48 Hr 72 Hr

10

168 Hr 5

Control

Mean latency (sec)

Mean latency (sec)

25

25 24 Hr 20

48 Hr 72 Hr

15

168 Hr Control

10 5

0 1

2

3

4

Blocks of 5 trials

5

1

2

3

4

5

Blocks of 5 trials

Figure 6.3 Mean shuttle box escape latencies for groups (n = 8) given inescapable tailshocks and tested

in the shuttle box either 24, 48, 72, or 168 hours later. The control was restrained in the apparatus and tested 24 hours later. The top panel (Different) shows the results for groups that received inescapable shock in a restraining tube, whereas the bottom panel (Same) shows the results for groups that received inescapable shock in one side of the shuttle box. (Reprinted with the author’s permission from Maier, S. F., & Watkins, L. R. 2005. Stressor controllability and learned helplessness: The roles of the dorsal raphe nucleus, serotonin, and corticotropin-releasing factor. Neuroscience and Biobehavioral Reviews, 29, 829–841).

06-Schachtman-06.indd 128

2/15/2011 7:43:32 PM

TRAUMA AND LEARNED HELPLESSNESS

be extended if the rats were reminded of the stressful situation. Without reminders, the learned helplessness effect lasted 2 days, but less than 4 days. However, if the rats that had been inescapably shocked in a restraining tube were reconﬁned in the tube for 10 minutes without shock 2 days later, then they did show the escape latency deﬁcit 4 days after shock. If the reminder occurred 4 days after IS, and the test was 2 days later, then there was no deﬁcit 6 days after IS. However, a deﬁcit was observed on that day if reminders had occurred both 2 and 4 days after IS. If eight reminder treatments were given, on days 2, 4, 6, 8, 10, 12, 14, and 16 after IS, then the escape latency deﬁcit occurred if the test occurred 2, 4, or 6 days after the last reminder, but not if it occurred 30 days after the last reminder. The duration of the reminder treatments mattered; a single 90-minute reminder 2 days after IS extended the deﬁcit as much as a 10-min reminder, but multiple 90-minute reminders did not extend the duration of the effect. We need to know whether the duration of the other short-lived learned helplessness effects described earlier can be extended by reminder cues. This is important because the ﬂashbacks that occur in PTSD can be seen as providing such reminders. If the life of diverse learned helplessness effects is extended by reminders, then such effects are more likely to be applicable to PTSD.

NOT ALL EFFECTS OF INESCAPABLE SHOCK ARE LEARNED HELPLESSNESS EFFECTS Not all of the effects of inescapable shock in one context upon later behavior in another context are learned helplessness effects. First, Grahn, Kalman, Brennan, Watkins, and Maier (1995), using the triadic design, found that when prior inescapable shocks resulted in a reduction in the time rats spent on the open arms of an elevated plus maze, a commonly used measure of increased anxiety, prior escapable shocks had the same effect. Second, relative to both restrained and home cage controls, rats that had learned to turn a wheel to avoid tail shocks and their yoked, inescapably shocked counterparts showed equivalent reductions in preference for a familiar 2%

06-Schachtman-06.indd 129

129

sucrose solution over water (Christianson, Paul, et al., 2008). Third, as compared with home cage controls, rats that had learned to turn a wheel to avoid tail shocks and their yoked, inescapably shocked counterparts showed equivalent increases in unconditioned fear of the odor of a ferret, as measured by time spent in the part of an open ﬁeld that contained a towel impregnated with the odor (Baratta et al., 2007). Fourth, Woodmansee, Silbert, and Maier (1993) showed that running in a wheel in the home cage was depressed for from 9–12 days by prior inescapable shock, relative to the performance of restrained controls. However, prior escapable shock resulted in the same reduction in activity. Some of the physiological effects of inescapable shock have also been shown not to be helplessness effects. Maier, Ryan, Barksdale, and Kalin (1986) examined plasma ACTH and corticosterone levels at times from 0–24 hours after inescapable versus escapable shocks. Both treatments quickly and equally elevated levels of both ACTH and corticosterone, and these declined back to baseline in 60–150 minutes (but see Dess, Linwick, Patterson, Overmier, & Levine, 1983). In a second experiment, plasma levels of corticosterone and ACTH were assayed following ﬁve 5-sec shocks in the shuttle box that occurred a day after IS, ES, or time in the home cage. Corticosterone levels rose after the ﬁve shocks regardless of the rats’ histories, peaked at 30 minutes, and then declined. For ACTH levels, the rise immediately after the ﬁve shocks was greater for both IS and ES groups than for the home cage controls. Again IS and ES groups did not differ. So changes in these key parameters of the HPA axis response to stressors were not learned helplessness effects.

ALTERNATIVE PERSPECTIVES ON THE NATURE OF LEARNED HELPLESSNESS The original conception of learned helplessness was a cognitive one; it hypothesized that extended exposure to uncontrollable, unpredictable aversive events established an expectancy that future responses would be useless, and so helplessness was observed. Of course, this is not

2/15/2011 7:43:33 PM

130

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

the only possible account. For examples, Black (1977) argued that the interference effect could result from response competition, for example, learning to hold still (but see Lawry et al., 1978), and Levis (1980) offered a frustration account. Neither account was sustained over time. This section will focus on two newer, alternative conceptualizations about the nature of learned helplessness. The ﬁrst shift in conceptualization stems from the observation described earlier that a prior history of inescapable shock in the wheelturn box, as compared with ES or restraint, results in retarded learning to turn right in a Y-maze to escape shock (Minor, Jackson, & Maier, 1984). Minor et al. (1984) argued that during the IS treatment rats stopped paying attention to stimuli produced by their responses and thus increased their attention to external stimuli. Lee and Maier (1988) tested this hypothesis by training rats on choice escape from roomtemperature water after they had received ES, yoked IS, or restraint in a wheel-turn box. Prior IS interfered with choice accuracy if the rats had to learn a position discrimination with brightness cues irrelevant, but it had no effect if both arms were black, or both white, on a trial. Strikingly, IS facilitated performance if rats had to learn a brightness discrimination with position irrelevant. These results are consistent with an “attentional-shift” hypothesis. The results of other experiments that can be construed as tests of the attentional-shift hypothesis are mixed (e.g., Rodd, Rosellini, Stock, & Gallup, 1997; Testa, Juraska, & Maier,1974; see LoLordo & Taylor, 2001, for a detailed review). The second development concerns a new way of thinking about learned helplessness per se that was suggested by Overmier (1988; see Minor, Dess, & Overmier, 1991) roughly 20 years ago. On this view, the IS treatment (exposure to shocks that are both uncontrollable in any way and unpredictable) is considered the baseline stress condition that has a range of debilitating effects on the organism, both physiological and behavioral. To the extent that the situation may afford any kind of behavioral control over duration, intensity, or quality of the aversive events, that would serve to reduce the severity of the

06-Schachtman-06.indd 130

stress. Similarly, to the extent that the situation may afford any kind of prediction about onset, duration, termination, or future likelihood of the aversive events, that too would serve to reduce the severity of the stress. The presence of either of these two types of coping opportunities (which are related to instrumental and classical conditioning, respectively) modulates the psychological effects of the baseline stress challenge. Dess et al. (1983) undertook an experiment to assess the stress level as indexed by corticoid levels in dogs. The experiment had two phases, and the corticoid levels were measured in each. In an extended phase 1, shocks were either predictable and controllable, controllable only, predictable only, or neither. Phase 1 was arranged so that all dogs in the four groups received the same numbers of shocks in the same temporal distribution. Phase 2 was the next day; all dogs were placed in a shuttle box and given standard discrete-trial avoidance training with trials signaled by a cue novel to them. During Phase 1, dogs with control over the shocks had signiﬁcantly lower levels of corticoids that than those without control, and there was no signiﬁcant effect of predictability in this phase (although the dogs with predictability did have numerically lower scores than their counterparts). During Phase 2, dogs that had control in phase 1 had cortisol levels that were slightly but nonsigniﬁcantly lower than those without control. In contrast, the levels of cortisol in phase 2 were dramatically and signiﬁcantly lower as a result of having received predictable shocks in phase 1. This pattern of results conﬁrmed that both control and prediction had powerful effects in reducing stress—and presumably the fear level, but the reduction of adrenocortical responses from these two forms of coping occurred in different phases (see Overmier, 1985, for another example of predictability and controllability having different loci of effects). Returning to the Maier paradigm with rats that we have been following, it is relevant to look at the effects on learned helplessness of manipulations that are known to reduce conditioned fear. For example, adding the escape contingency is seen as a treatment that may modulate the

2/15/2011 7:43:33 PM

TRAUMA AND LEARNED HELPLESSNESS

effects of IS because it also reduces conditioned fear (Mineka, Cook, & Miller, 1984). Maier, Amat, Baratta, Paul, and Watkins (2006) noted that, on this view, the presence of the escape contingency is the “active ingredient.” As has been shown earlier, it did modulate the severity of learned helplessness effects, which led investigators to wonder about what feature of escapability was responsible for the modulation. Escape responses produce stimuli, and perhaps these stimuli are responsible because they predict the end of the shock. The initial research looked at the effect of adding a backward conditioned stimulus to the inescapable shocks. Such a stimulus would come on as the shock ended and stay on for the initial part of the intertrial interval. Researchers asked whether the backward CS modulates the effect of IS in the same way that adding an escape contingency does. Like adding an escape contingency, adding a backward CS reduces fear of the shock context (Maier & Keith, 1987), so long as the minimum time to the next shock is not too short (Rosellini, DeCola, & Warren, 1986). Adding a backward CS to inescapable shock also attenuated the subsequent shuttle box escape learning deﬁcit (Maier & Warren, 1988; also see DeCola, Rosellini, & Warren, 1988), and recently Christianson, Benison, et al. (2008) observed that, like adding an escape contingency to shock, adding a backward CS prevented the subsequent reduction in time spent investigating a juvenile male rat otherwise induced by IS. Are the modulating effects of adding an escape contingency and a backward CS the same, that is, results of the same process? Several ﬁndings suggest that they are not. First, unlike adding a backward CS, adding an escape contingency wipes out the exaggerated context fear resulting from shock even when there is a short minimum time between the escape response or CS and the next shock (Rosellini, Warren, & DeCola, 1987). Second, in a three-stage immunization procedure, prior escapable shocks, but not inescapable shocks followed by a CS, immunize the stage-3 shuttle box escape learning against the proactive effects of stage-2 IS (Maier & Warren, 1988). Moreover, Christianson, Benison, et al. (2008) showed that while the reduction in time spent

06-Schachtman-06.indd 131

131

investigating a juvenile rat that resulted from prior IS did not occur if the shocks were escapable or if a backward CS followed each inescapable shock, the neural mechanisms of the two effects were different. This point will be discussed in a later section. This line of research linking levels of fear to manifest learned helplessness then turned to cessation signals, stimuli that consistently occur shortly before shock ends and thus signal its termination. Minor, Trauner, Lee, and Dess (1990) showed that adding a cessation signal to IS alleviated the shuttle box escape learning deﬁcit that would otherwise have occurred; this happened even when the minimum time to the next shock was short. Moreover, in the three-stage design the cessation signal partially immunized shuttle box escape learning against the proactive effects of IS, and a cessation signal in conjunction with a physically different backward CS produced complete immunization (see Fig. 6.4). We do not know whether the choice escape deﬁcit described earlier would be attenuated by the addition of a cessation signal to inescapable shock. If the cessation signal had this effect, and to the same extent as adding an escape contingency, then as LoLordo and Taylor (2001) noted, it will be very difﬁcult to attribute the cognitive deﬁcit to inescapability of the shocks. Even though adding a backward CS and adding an escape contingency do not engage the same neural mechanism, it is still possible that the cessation signal and escapability do engage the same mechanisms, given that the behavioral effects of the two are so similar. In summary, learned helplessness appears not to depend only on the uncontrollability of the aversive events. Unpredictability, too, plays some role.

THE NEURAL MECHANISMS OF LEARNED HELPLESSNESS EFFECTS There has been an enormous amount of research on the neurophysiology of learned helplessness effects, beginning with the work of Weiss, which focused on the noradrenergic system (e.g., Weiss et al., 1981), and including sustained research efforts in the laboratories of Anisman, Minor,

2/15/2011 7:43:33 PM

Mean escape latency (sec)

132 40 30

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

II CI CB-I RR

20 10 0 FR-1

1

2

3

4

5

Blocks of 5 trials

Figure 6.4 Mean shuttle box escape latencies in blocks of ﬁve trials for all groups in Experiment 6 of Minor et al. (1990). During immunization training, rats were exposed to variable-duration inescapable tail shock (II), inescapable shock in conjunction with a compound cessation signal (CI), inescapable shock in which a tone was presented during the last 3 seconds of each trial and darkness occurred during the ﬁrst 3 seconds of the intertrial interval (CB-I), or restraint (RR) in the wheel-turn chambers. On day 2, the three groups that had received shock in the ﬁrst phase were exposed to ﬁxed-duration inescapable tailshocks in tubes. All rats were tested for shuttle-escape performance on day 3. (Reprinted with the author’s permission from Minor, T. R., Trauner, M. A., Lee, C., & Dess, N. K. 1990. Modeling signal features of escape response: Effects of cessation conditioning in “Learned Helplessness” paradigm. Journal of Experimental Psychology: Animal Behavior Processes, 16, 123–136).

Petty, Martin, and all their colleagues (e.g., Anisman, Zalcman, Shanks & Zacharko,1991; Martin & Puech, 1996; Neumaier, Petty, et al., 1997; Woodson, Minor, & Job, 1998) The most persistent and systematic pursuit of physiological mechanisms has been done by Maier and Watkins and their colleagues, and page limitations force us to restrict our discussion to (some of) their research. The Dorsal Raphe Nucleus and Its Projections

Maier et al. (1993) examined the effects of brain lesions on fear, as indexed by freezing, and the shock escape learning deﬁcit resulting from IS, using the triadic design and test procedure

06-Schachtman-06.indd 132

developed by Maier (1990). Lesions of the basolateral region and central nucleus of the amygdala eliminated both the preshock fear and the shockinduced fear resulting from a few FR1 shockescape trials in the shuttle box a day after IS, but did not reduce the escape learning deﬁcit produced by IS. In contrast, lesions of the dorsal raphe nucleus (DRN), a small brainstem structure that projects widely to limbic and cortical structures, did not reduce the fear that generalizes to the shuttle box after IS, but did eliminate the enhanced fear conditioning in the shuttle box following IS as well as the escape deﬁcit. The authors suggested that the DRN, which projects to the lateral and basal nuclei of the amygdala, modulates the activity of the amygdaloid structures involved in the production of fear. A single session of IS may sensitize the DRN to its inputs for up to 48–72 hours. The sensitized DRN (after IS) exerts a stronger activating effect on the amygdala than does the nonsensitized DRN of control rats, and so IS enhances fear conditioning resulting from the FR-1 escape trials. Since amygdala lesions do not reduce the interference with escape learning following IS, the DRN-amygdala projection cannot be responsible for the interference with escape behavior. The DRN also projects to the dorsal periaqueductal gray (DPAG), which mediates unconditioned motor responses to shock and other aversive stimuli. That serotonergic (5-HT) pathway is known to be inhibitory (Kiser, Brown, Sanghera, & German, 1980; Schutz, DeAguiar, & Graeff, 1985). Maier et al. suggested that the shuttle box escape deﬁcit results from inhibition of the DPAG produced by the sensitized DRN. This reasoning is consistent with the conclusion drawn by Graeff, Guimares, De Andrade, and Deakin (1996) that stimulation of serotonergic neurons within the DRN results in fear via projections to the amygdala as well as inhibition of active defense responses via projections to DPAG. Once the DRN had been identiﬁed as a critical structure in the circuit mediating the escape deﬁcit, Maier and his colleagues began to microinject drugs into the DRN to work out the neurophysiological basis of the deﬁcit. A brief description of some of the anatomy of the DRN

2/15/2011 7:43:33 PM

TRAUMA AND LEARNED HELPLESSNESS

should make these experiments easier to understand. First, the 5-HT neurons in the DRN, which are the ones that project to amygdala and DPAG, are tonically inhibited by GABA interneurons (Tao, Ma, & Auerbach, 1996). Benzodiazepines are known to bind to the GABAA receptor complex and thus increase this inhibitory effect on 5-HT neurons. Second, when the 5-HT neurons are activated, 5-HT is released in the vicinity of those 5-HT neurons, and stimulates 5-HT1A receptors located on their cell bodies and dendrites. Those receptors inhibit the activity of the 5-HT neurons, that is, they are inhibitory autoreceptors. These receptors are easy to desensitize (Kennett, Marcou, Dourish, & Curzon, 1987), and when they are desensitized the 5-HT neurons produce an exaggerated output in response to an input. Maier and his colleagues (e.g., Maier & Watkins, 1998) suggested that IS desensitizes the inhibitory autoreceptors for a few days, so that the FR-1 shocks in the shuttle box result in exaggerated output from the 5-HT neurons in DRN, and thus greater excitation of the amygdala and greater inhibition of the DPAG. Maier, Kalman, and Grahn (1994) began to explore the effects of benzodiazepines infused into DRN upon the learned helplessness effects. There were three IS and three restrained groups in a design like Maier (1990). One pair of groups got the benzodiazepine chlordiazepoxide (CDP) infused into the DRN 10 min before IS or restraint, and vehicle before the test in the shuttle box. Another pair of groups received vehicle before IS or restraint and CDP before test, and a third pair received vehicle on both days. There was very little generalized freezing in the shuttle box prior to the FR-1 trials. The drug given either before IS or before test eliminated both the excess freezing after two shocks and the escape deﬁcit resulting from IS. The authors argued that microinjection of benzodiazepines into the DRN before IS increases inhibition of the 5-HT neurons in DRN and thus prevents the hyperactivity of those neurons (relative to the activity that ES induces) that would otherwise lead to desensitization of the inhibitory autoreceptors. Infusion of the drug before testing directly reduces 5-HT activity in DRN. The reduced DRN activity results in (1) a smaller

06-Schachtman-06.indd 133

133

excitatory effect on the amygdala, and thus less postshock freezing, and (2) a smaller inhibitory effect upon the DPAG, and thus less interference with escape behavior. Analysis of Serotonergic Effects in the Dorsal Raphe Nucleus

If activation of 5-HT neurons within the DRN is an important mediator of the behavioral consequences of IS, then IS should evoke a greater increase in the level of extracellular 5-HT in the DRN than ES. Maswood, Barter, Watkins, and Maier (1998; see also Amat et al., 2001), using microdialysis, observed that administration of IS, but not ES, produced a marked increase in the level of extracellular 5-HT in the DRN, which reached a peak 75 minutes into the shock session and returned to baseline 30 minutes after the end of the treatment. Grahn et al. (1999) conﬁrmed this result with a different technique, ﬁnding greater Fos-like immunoreactivity in 5-HT neurons in the middle and caudal DRN after IS than after ES. Maximal c-Fos expression occurred 2 hours after the shocks. Takase et al. (2005) replicated this effect and showed that both the escape deﬁcit and the effect on Fos-like immunoreactivity occurred after 50 or 100 inescapable shocks, but not after 10. Moreover, Fos-like immunoreactivity among noradrenergic neurons in the locus coeruleus (LC) showed a similar relation to the number of inescapable shocks. Amat, Matus-Amat, Watkins, and Maier (1998) directly tested the hypothesis that the IS potentiates subsequent fear conditioning because it activates the DRN 5-HT neurons more than ES does, and thus it causes a greater release of 5-HT in the amygdala. IS but not ES resulted in increased levels of extracellular serotonin in the basolateral amygdala, relative to levels in restrained controls. This effect persisted for 140 min after the inescapable shock session. Moreover, a day later basal levels of serotonin in the amygdala were still elevated in the IS group, and that group had an exaggerated 5-HT response to two footshocks. Postshock levels of 5-HT in basolateral amygdala were positively correlated with postshock freezing. Benzodiazepine inverse agonists such as DMCM (a β-carboline) are assumed to bind to

2/15/2011 7:43:33 PM

134

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

the benzodiazepine receptor and thereby interfere with GABAergic inhibition. So if IS produces learned helplessness effects by reducing the level of such inhibition of 5-HT neurons in DRN, then infusion of DMCM by itself should produce the helplessness effects. Maier, Busch, Maswood, Grahn, and Watkins (1995) microinjected DMCM into the DRN before restraint. It produced exaggerated postshock fear and escape deﬁcits in the shuttle box 24 hours later. Why does IS interfere with GABAergic inhibition? Perhaps because it activates inhibitory opioid input to the GABA neurons in DRN and also leads to the release of an endogenous inverse agonist, which activates 5-HT cells in DRN. In accord with this hypothesis, intra-DRN infusion of benzodiazepine receptor antagonists like ﬂumazenil and CGS-8216 before IS acts like infusion of an agonist, facilitating the inhibitory effect of GABA and so blocking exaggerated fear and interference with escape (Maier, Grahn, Maswood, & Watkins, 1995). Intra-DRN infusions of naloxone have the same effect, presumably by inhibiting the opioid inﬂuence upon GABA neurons, thus allowing the inhibitory effect of those GABA neurons on the 5-HT neurons to be manifested. Maier, Grahn, and Watkins (1995) followed up the suggestion that increased 5-HT activity in the DRN is important for the escape deﬁcit. 8-OH-DPAT is a partial 5-HT1A agonist. Experiments assessed its impact on the enhanced fear conditioning and escape deﬁcit that follow exposure to IS. In Experiment 1, 1 µg of the drug or vehicle was injected into DRN 10 min before IS or restraint. A day later, freezing in the shuttle box following the ﬁrst two shocks was enhanced by IS in the rats that received vehicle, and the drug eliminated this effect. The drug also eliminated the escape deﬁcit found in IS rats. Injecting the drug intothe DRN 5–10 min before the shuttle box escape test had the same effects. The authors suggested that the microinjected drug probably had its effect by activating the inhibitory somatodendritic 5-HT1A autoreceptors in the DRN, thereby reducing serotonergic neurotransmission. The effects of the microinfusions described earlier were replicated by Maier and Watkins (2005), who also showed that the

06-Schachtman-06.indd 134

drug had no effect if the two phases of the experiment occurred in the same environment, in which case learning processes that did not involve 5-HT, for example, stimulus generalization of Pavlovian fear conditioning, must have been responsible for the escape deﬁcit. Norepinephrine, Corticotrophin-Releasing Hormone, and Opioids

Other inputs to the DRN have been shown to be important for the learned helplessness effects discussed earlier. These include norepinephrine (NE), corticotrophin-releasing hormone (CRH), and opioids. Grahn et al. (2002) injected the selective α-1 adrenoreceptor antagonist benoxathian into the DRN before IS. The escape latency deﬁcit was completely eliminated and the exaggerated freezing after two FR-1 trials was signiﬁcantly reduced. A major source of NE input to the DRN is the LC. Weiss and his colleagues have shown that NE outputs from LC are affected by IS and are important for the ensuing behavioral depression, as indexed by an increased proportion of time spent ﬂoating in the forced swim test (Simson, Weiss, Ambrose, & Webster, 1986; Weiss & Simson, 1985). These facts tempt one to conclude that the neurophysiology described by Weiss is part of the explanation of the learned helplessness effects studied in the Maier-Watkins laboratory. However, both the induction procedures and the test task are quite different in the two lines of research, making such a conclusion risky (also see Greenwood & Fleshner, 2008; LoLordo & Taylor, 2001; Weiss et al., 2005). Sutton et al.(1997) showed another learned helplessness effect, that is, that analgesia in response to 1 mg/kg of morphine was potentiated by IS, but not by ES, given 24 or 48 hours earlier. This effect could be blocked by intraDRN administration of the opioid antagonist naltrexone 15 minutes before IS. The effects of corticotropin-releasing hormone (CRH) on the DRN and learned helplessness effects have been explored more thoroughly. In summary, injection of a CRH-antagonist into DRN before IS blocked the exaggerated postshock freezing and escape deﬁcit seen after IS, and CRH infused into DRN mimicked the effects

2/15/2011 7:43:33 PM

TRAUMA AND LEARNED HELPLESSNESS

of IS (Hammack et al., 2002). Moreover, a selective CRH-1 receptor antagonist had no effect on either exaggerated freezing or the escape deﬁcit after IS, but a selective CRH-2 antagonist dosedependently blocked both effects (Hammack et al., 2003). A highly selective CRH-2 agonist substituted for IS in producing the learned helplessness effects, and it also increased fos-like immunoreactivity in 5-HT neurons in DRN and the level of extracellular 5-HT in basolateral amygdala. The anatomical sources of these inputs are currently under investigation. The bed nucleus of the stria terminalis (BNST) can be considered the rostral part of the extended amygdala. Davis has suggested that it mediates anxiety, which has as its hallmark a long-lasting response to an unpredictable event (e.g., Walker, Toufexis, & Davis, 2003). Hammack, Richey, Watkins, and Maier (2004) noted that the inescapable shocks used in the experiments from the Watkins-Maier laboratory are unpredictable and that the learned helplessness effects they induce last a couple of days. So what they had been calling “anxiety” also met Davis’ deﬁnition of the term, suggesting that the BNST might be involved in the learned helplessness effects. They made NMDA lesions of the BNST or sham lesions, gave groups either IS or kept them in the HC, and then tested the rats in the shuttle box the next day. The lesions prevented both the exaggerated postshock freezing and the escape deﬁcit (also see Bangasser, Santollo, & Shors, 2005). The BNST is a major source of CRH, and Hammack et al. suggested that it might be inﬂuencing the 5-HT neurons in DRN through its effect on CRH-2 receptors on those neurons. Maier and Minor (1993) extended the pharmacological analysis to an accuracy measure of escape learning, which presumably reﬂects associative or cognitive factors. They used the procedure developed by Minor, Jackson, and Maier (1984), which was described earlier. The effects of their manipulations suggest that the impairment in escape choice accuracy and the escape latency deﬁcit following IS result from different processes. In Experiment 1, 5 or 10 mg/kg diazepam ip prior to the IS treatment failed to improve choice accuracy. However, the drug did

06-Schachtman-06.indd 135

135

eliminate the escape latency deﬁcit that otherwise resulted from prior IS. In Experiment 2, administration of the β-carboline FG-7142 (a benzodiazepine receptor inverse agonist), which is known to induce a brief period of intense fear or anxiety, to restrained rats instead of giving them IS had no effect on escape accuracy in the Y-maze a day later. However, 10 mg/kg of the drug did result in long escape latencies (which eventually improved to control levels). Subcutaneous doses of naltrexone ranging from 3.5 to 14.0 mg/kg given 10 min before IS had no effect on subsequent escape accuracy in the Y-maze. The two highest doses did eliminate the escape latency deﬁcit of IS rats. Maier and Minor suggested that although the escape latency deﬁcit resulted from intense anxiety sensitizing the DRN (see earlier discussion), the reduced choice accuracy resulted from a change in attentional bias toward external stimuli (see Lee & Maier, 1988; Minor et al., 1984) and suggested that perhaps the LC-dorsal noradrenergic bundle system is implicated (Minor et al., 1988). The Prefrontal Cortex and Learned Helplessness Effects

The DRN is a small brainstem structure, and it is unlikely to be the structure that encodes whether a stressor is escapable. A likely candidate for such a role is the prefrontal cortex, which is known to be important for executive functions. Moreover, glutamatergic projections from the infralimbic (IL) and prelimbic (PL) regions of the ventral medial prefrontal cortex (mPFCv) synapse onto GABAergic neurons within the DRN (Gabbott, Warner, Jays, Salway, & Busby, 2005; Jankowski & Sesack, 2004). These in turn have inhibitory links to 5-HT neurons in DRN. This information led Amat et al. (2005) to investigate mPFCv inﬂuences on activation of 5-HT neurons in DRN and exaggerated release of 5-HT in the DRN, as well as on learned helplessness effects observed in the shuttle box a day after the induction phase. In a 2 x 3 design, vehicle or the GABAA receptor agonist muscimol was microinjected into the mPFCv (at the IL/PL border) an hour before groups of rats received ES or yoked IS in wheel-turn boxes or served as

2/15/2011 7:43:33 PM

136

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

HC controls. Infusion of muscimol would result in inhibition of the glutamatergic neurons in mPFCv that project to DRN. Muscimol did not affect learning of the wheel-turn escape response; the muscimol-ES and vehicle-ES groups learned equally well. In one experiment, rats were sacriﬁced 2 hours after the induction phase, and c-fos expression in 5-HT neurons in the DRN was assessed. Among the vehicle groups, IS, compared with ES and HC, led to a two-fold increase in the percentage of 5-HT neurons in the caudal DRN that expressed c-fos. Importantly, muscimol abolished this difference, and it did so by increasing c-fos expression in ES rats to the level in IS rats. In a separate experiment, in vivo microdialysis was used to measure 5-HT efﬂux in caudal DRN, which reﬂects activity of the 5-HT neurons there before, during, and after the induction phase. Among the vehicle groups, IS produced an increase in the level of extracellular 5-HT in DRN that was sustained across the 100-minute stress session, whereas for the ES group the increase lasted only 20-40 minutes. Muscimol had no effect on either HC or IS groups. Importantly, and in accord with the effects on c-fos expression, muscimol and ES produced a sustained increase in 5-HT efﬂux, mimicking the pattern of results in the IS groups (see Fig. 6.5). In a separate experiment with the same design, the behaviors typically seen in the shuttle box a day after IS, exaggerated freezing after two FR-1 trials and a deﬁcit in escape learning, were also seen in the ES group that had been given muscimol shortly before ES. The authors asserted that all the results converged on the conclusion that shocks drive the DRN and that the addition of an escape contingency enhances the glutamatergic output from mPFCv to the DRN, output that results in 5-HT activity in DRN and its projection regions being inhibited. When that happens, behaviors like exaggerated postshock freezing and a deﬁcit in shuttle box escape learning that depend on a high level of 5-HT release in these projection regions do not occur (see Fig. 6.6). If the level of activation of the glutamatergic outputs from mPFCv while the stressor is occurring is critical for the learned helplessness effects, then if the strength of this activation could be

06-Schachtman-06.indd 136

increased during IS, the rats should treat the inescapable shocks as though they were escapable. Amat, Paul, Watkins, and Maier (2008) microinjected the GABA antagonist picrotoxin into the mPFCv, a procedure that increases its glutamatergic output, an hour before groups were given IS, ES, or HC treatments. The drug markedly suppressed both the initial rise in extracellular 5-HT otherwise caused by both IS and ES and the sustained increment in 5-HT that would otherwise be caused by IS. This effect on IS rats was site speciﬁc. Picrotoxin also prevented the exaggerated postshock freezing and the escape learning deﬁcit in the shuttle box that would otherwise be produced by IS, without affecting the behavior of ES or HC groups (but see Hunter, Balleine, & Minor, 2003, and Minor, Plumb, Schell, & Pham, 2010, for a different view of the role of mPFCv in responses to IS). Experiments analogous to these have looked at two other learned helplessness effects: reduced time spent exploring a juvenile rat (Christianson, Thompson, Watkins, & Maier, 2008), and enhanced morphine-based conditioned place preference (Rozeske, Der-Avakian, Bland, Beckley, Watkins, & Maier., 2009). Among rats that had received vehicle, IS resulted in reduced time exploring the juvenile and increased preference for the place paired with morphine. Muscimol before ES caused ES to function like IS, and picrotoxin before IS caused IS to function like ES. The mPFCv also plays a role in the immunization effect that was discussed earlier. Amat, Paul, Zarza, Watkins, and Maier (2006) reasoned that escapable shocks in the ﬁrst phase of the three-phase experiment would activate the mPFCv at that time, and they speculated that mPFCv activity would become associated with some aspect of the shock experience so that the mPFCv would also become activated by the inescapable shocks in the second phase a week later. Output from the mPFCv should inhibit the activation of the 5-HT system in the DRN that would otherwise result from IS and thus the rats should not show the exaggerated postshock freezing and escape deﬁcits in the shuttle box the next day. A comparison of vehicle groups demonstrated that prior ES blocked IS-induced

2/15/2011 7:43:33 PM

TRAUMA AND LEARNED HELPLESSNESS Inescapable 320 280 240 200 160 120 80 40 0

0

0

22

0

20

0

0

0

0

18

16

14

12

10

60

80

40

0

–6

20

Injection

0 –4 0 –2 0

5-HT (Percentage of baseline)

a

137

320 280 240 200 160 120 80 40 0

0 22

0

0

18

14

0 10

60

20

Injection

Time (min)

0

0

0

0

0

0 22

20

18

16

14

12

0 10

80

60

40

20

0

0

–2

0

Injection

–4

–6

0

5-HT (Percentage of baseline)

–6

0

Escapable

320 280 240 200 160 120 80 40 0

0

b

–2

5-HT (Percentage of baseline)

Time (min)

Time (min)

Figure 6.5 5-HT (mean +/- SEM) in the dorsal raphe nucleus as a percentage of baseline. (a) Groups that

received inescapable shock and controls: Open circles represent rats that had received vehicle before inescapable shock, ﬁlled circles represent rats that had received muscimol before inescapable shock, and the dashed line represents controls. (b) Groups that received escapable shock and controls. Open circles represent rats that had received vehicle before escapable shock, ﬁlled circles represent rats that had received muscimol before escapable shock, and the dashed line represents controls. The gray bar represents the period of stressor exposure in the wheel-turn boxes. The insert shows nonshocked home cage controls that received either muscimol or vehicle into the ventral medial prefrontal cortex. (Reprinted with the author’s permission from Amat, J., Baratta, M. V., Paul, E., Bland, S. T., Watkins, L. R., & Maier, S. F. 2005. Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nature Neuroscience, 8, 365–371).

activation of the 5-HT system in the DRN, as revealed by c-fos expression and levels of extracellular 5-HT, and also prevented the exaggerated freezing and escape deﬁcit in the shuttle box on the next day. Administration of muscimol an hour before the ES prevented all of these effects. So did muscimol just before the IS in the second phase. Thus, mPFCv activity during both the ﬁrst and second phases is necessary for immunization. In related experiments, Amat, Paul, Watkins, and Maier (2008) used picrotoxin to

06-Schachtman-06.indd 137

activate the mPFCv during ﬁrst-stage IS and found that this treatment prevented the sustained rise in the level of extracellular 5-HT during second-phase IS as well as the escape deﬁcit in the shuttle box. So IS accompanied by increased output from mPFCv to DRN functioned like ES in its proactive effect when IS occurred again, in a different place, a week later. Earlier it was noted that adding a backward CS or safety signal to inescapable shock mimicked some of the effects of adding an escape

2/15/2011 7:43:33 PM

138

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS vmPFC

CONTROL -

GABA

-

GLU

+ DRN LH EFFECTS GABA

-

5-HT

+ SHOCKS

Figure 6.6 A representation, based on recent

research from the Maier-Watkins laboratory described in the text, of the neural circuitry in the dorsal raphe nucleus (DRN) and ventral medial prefrontal cortex (vmPFC) that subserves learned helplessness effects. Shocks activate 5-HT neurons in the DRN, which ultimately leads to learned helplessness effects, for example, an escape latency deﬁcit. If escapability of the shocks is added, then the net effect is reduced inhibition of glutamatergic neurons in the vmPFC. Since these neurons have an excitatory effect on GABAergic neurons in the DRN, which in turn have an inhibitory effect on the critical 5-HT neurons, activation of the latter will be inhibited, and learned helplessness effects will be mitigated. Adding an infusion of the GABA antagonist picrotoxin into vmPFC before inescapable shock has the same effect as adding escapability.

contingency. Adult rats spent much less time exploring a juvenile male rat a day after they had received IS, compared to HC controls. However, if the lights were turned off for 5 seconds beginning at the end of each inescapable shock (backward CS), then the reduction in social exploration did not occur. This outcome was not mediated by a reduction in activation of the DRN serotonin system. The backward CS group and a group that received the same number of shocks and CSs, but with no contingency between them, both showed the same sustained increase in extracellular 5-HT during the stressor, and the same higher proportion, compared to HC rats, of 5-HT neurons in DRN containing c-fos. Moreover, inhibition of the mPFCv with muscimol did not affect the relative increase in

06-Schachtman-06.indd 138

social exploration that stemmed from adding a backward CS (Christianson, Benison, et al., 2008). Thus, although adding an escape contingency and adding a backward CS to shock both prevent the reduction of social exploration that would otherwise occur, the two effects are mediated by different neural mechanisms. Christianson et al. suggested that the posterior insular cortex might have the capacity to process the multimodal inputs and their temporal arrangement. Bilateral lesions of the posterior insula had no effect on the baseline level of social interaction, but the lesions did prevent the backward CS from exerting its ameliorative effect on social exploration. Importantly, the ameliorative effects of adding an escape contingency were unaffected by the lesion.

APPLICATIONS TO HUMAN PSYCHOPATHOLOGY Among the researchers who conducted the ﬁrst experiments on helplessness in dogs, Overmier and Maier continued to focus on the implications of the phenomenon for learning theory, while Seligman(1975) developed a very inﬂuential model of human reactive depression that was formally identical to the learned helplessness hypothesis. The modeling relation between learned helplessness and depression has been well reviewed elsewhere (e.g., see LoLordo, 2001; Overmier & LoLordo, 1998; and Peterson, Maier, & Seligman, 1993). Some extensions need to be noted. In 1978, Abramson, Seligman, and Teasdale published a reformulation that changed the focus of research on the relation between learned helplessness and depression. They asserted that you need to understand the attributions a person makes about his or her lack of control of an event in order to predict its effect on future behavior. Speciﬁcally, whether helplessness is attributed to a stable or an unstable factor will determine how long the expectation of future helplessness and the ensuing motivational and cognitive deﬁcits will last. Whether the lack of control is attributed to a global or to a speciﬁc factor will determine how wide ranging the expectation of future helplessness will be.

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

Moreover, if the attribution is to an internal factor, then self-esteem will suffer, and this will be among the symptoms of depression. In 1989, Abramson, Metalsky, and Alloy further altered the model, hypothesizing a subtype of depression they called hopelessness depression, in which both the expectation of negative outcomes and the expectation of inability to affect those outcomes are required for hopelessness and depression. The authors argued that the negativity of the person’s attributional style interacts with the negativity of the precipitating event to determine the likelihood of hopelessness, so that it might not require a very negative event to induce hopelessness in someone with a very depressogenic attributional style. The hopelessness theory continues to generate fruitful research (e.g., Gibb, Alloy, Abramson, & Marx, 2003; Gibb & Alloy, 2006). Most of the animal research that has been described in this chapter was not conducted with an applied goal in mind (e.g., Maier & Watkins, 2005). However, in recent years several applied researchers have suggested that various learned helplessness effects are more relevant for PTSD, a diagnosis that was not available when learned helplessness research began, than for depression (Foa, Zinbarg, & Rothbaum, 1992). First, the salient features of PTSD will be described. Then the learned helplessness effects that best model these features will be considered.

139

reexperiencing must occur. Avoidance refers to avoidance of stimuli associated with the traumatic event and numbing of general responsiveness. Seven sorts of avoidance are listed, and at least three are required for inclusion. They include efforts to avoid thoughts or feelings associated with the trauma and also people and places that might cue memories of it, as well as inability to recall aspects of the trauma. Other aspects of avoidance include loss of interest in formerly engaging activities, feelings of detachment, loss of affect, and the sense that there is not much of a future. Arousal includes irritability, having a difﬁcult time falling asleep or staying asleep, having trouble concentrating, being hypervigilant, and exhibiting an exaggerated startle response. At least two of these symptoms are required for inclusion. In addition, the reexperiencing, avoidance, and hyperarousal must last at least 1 month and must cause signiﬁcant impairment of important activities. Only a minority of persons exposed to severe trauma are diagnosed as having PTSD (Nemeroff et al., 2006), and genetic factors, as well as the effects of experiences prior to, at around the time of, and in the month after trauma have been discussed as sources of these individual differences (e.g., Mineka & Oehlberg, 2008; Nemeroff et al., 2006).

Posttraumatic Stress Disorder

Parallels Between Learned Helplessness Effects and Symptoms of Posttraumatic Stress Disorder

The Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV, 1994) lists several criteria for PTSD. First, several characteristics of the precipitating event are stipulated. It must involve actual or threatened death or serious injury, and it must evoke intense fear, horror, or helplessness. Next, three classes of symptoms are described: reexperiencing, avoidance, and arousal. Reexperiencing includes having upsetting recollections of the event, having nightmares of the event, reliving the event, becoming very distressed in response to stimuli that resemble some aspect of the event, and showing heightened physiological responses to such stimuli. At least one of these forms of

Pavlovian conditioned fear seems to operate similarly in learned helplessness and PTSD. Contexts and cues paired with IS evoke more conditioned fear than those paired with ES, and this conditioning is long lasting. Moreover, after IS there will be more generalized conditioned fear in situations somehow resembling the trauma setting. Paralleling these ﬁndings, people with PTSD become very distressed in response to stimuli that resemble some aspect of the traumatic event, and they show exaggerated physiological responses to these stimuli as well. These aspects of reexperiencing are Pavlovian conditioned responses (e.g., Rasmusson & Charney, 1997).

06-Schachtman-06.indd 139

2/15/2011 7:43:34 PM

140

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

The sensitization that occurs in learned helplessness also has a parallel in PTSD. IS sensitizes a circuit in the DRN so that even in a novel situation where there is no Pavlovian conditioned fear a single shock will evoke exaggerated freezing. Maier (1990) called this sensitized state anxiety and suggested that it facilitated fear conditioning to a novel context. The effect was short lived but could be greatly extended by repeated reminder cues. Recently Baratta et al. (2007, 2008) described a more long-lasting effect whereby a single fear conditioning trial to a novel cue in a novel context resulted in greater expression of fear to both context and cue after IS than after ES or no shock. Moreover, the rats responded more fearfully on a series of extinction trials following IS. The circuitry of this longer lasting sensitization effect has not been worked out. This sensitization ﬁnds a parallel in PTSD; patients are said to acquire new fears readily and to show a deﬁcit in extinction of fear. Blechert, Michael, Vriends, Margraf, and Wilhelm (2007) looked at conditioned skin conductance responses during discriminative aversive conditioning, in which one conditioned stimulus (CS+) was paired with the unconditioned stimulus and a second conditioned stimulus (CS–) was not, and subsequent extinction, and also measured reported valence of the CSs and expectancy of the US. CSs were inkblots and the US was unpleasant electric shock. Only participants who were aware of the contingencies were included in the analyses. The most striking effects were found in reactions to the former CS+ in extinction. The PTSD group showed no decline in rated expectancy of shock in this CS, whereas the other groups showed a marked decline. Similar effects, though not as strong, were found for valence of the CS+ and conditioned skin conductance response (SCR). The authors noted that this pattern of results was reminiscent of the failure of the emotional reactions to cues associated with the trauma to decline over time in PTSD. Orr et al. (2000) also found that persons with PTSD showed a deﬁcit in fear extinction learning, as well as exaggerated fear conditioning. Another property of sensitization in the animal laboratory is that it will enhance startle

06-Schachtman-06.indd 140

responding to stimuli that occur in dangerous contexts but are not themselves followed by aversive events. If sensitization functions analogously in learned helplessness and PTSD, then the latter should be characterized by exaggerated startle, an aspect of hyperarousal, in the presence of such stimuli. Grillon and Morgan (1999), using discriminative Pavlovian conditioning with shock to the wrist as the US, found that the startle was potentiated in both CS+ and CS– in the PTSD group, but only in CS+ in controls. So the PTSD group could not inhibit the startle even though they knew the CS– was safe. Jovanovic et al. (2009) extended this research by using a modiﬁed Pavlovian conditioned inhibition procedure. Conditioned stimuli were differently colored lights, and the US was an air blast to the larynx. During conditioning one pair of serially presented lights was followed by the air blast (call this AX+), whereas when the X stimulus occurred along with a third light, B, there was no air blast (so BX–). Then there were test trials with AB and AC, where C was a novel stimulus. If B had become a conditioned inhibitor, then AB should evoke less fear, and a smaller startle response, than AX had evoked. Responding to AC should also be less than to AX if the novel C has a distracting effect, which Pavlov (1927) called external inhibition. During each light, the participants, healthy controls, and PTSD groups that had high versus low levels of symptoms in the previous month could respond on a keypad to indicate whether they expected the air blast. During conditioning all groups showed fearpotentiated startle to AX. The controls and low-symptom PTSD groups showed signiﬁcantly less potentiation to BX than AX, but the highsymptom PTSD group did not discriminate between the two. Moreover, the controls and low-symptom PTSD groups responded less to AB and AC than to AX, but the high-symptom PTSD group showed similar potentiation to all three compounds, even though the rated expectancy of the air blast was lower in AB than AX in all groups, and lower in AC than AX in the controls and high-symptom groups. So Vietnam veterans with PTSD who are currently suffering more symptoms do not seem to be able to inhibit

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

the fear response to stimuli that they know are safe, whereas those suffering fewer symptoms do show inhibition of fear. Rats with a history of IS show greater neophobia than ES or restrained control groups (Minor, 1990). Enhanced neophobia can be seen as a parallel to hypervigilance, one of the aspects of hyperarousal observed in PTSD. We have seen that dogs and rats with a history of IS seem to passively accept shock in the shuttle box, and that stress-induced analgesia is greater after IS than ES. Among other learned helplessness effects are avoidance of social interaction with another adult rat in a novel environment or with a juvenile male rat in a familiar environment. In the latter case the adult actively disengages from the juvenile. These learned helplessness effects, which may be attributed to the anxiety generated by IS, parallel avoidance symptoms of PTSD. The passive acceptance of shock and the potentiated stress-induced analgesia resemble emotional numbing in PTSD. Indeed, PTSD patients have been shown to be hypoalgesic (Geuze et al., 2007). The avoidance of social interaction can be seen as parallel to the detachment and persistent loss of interest in what would otherwise be an engaging activities that are both seen in PTSD. It remains to be seen whether the duration of these phenomena in the learned helplessness model can be extended by reminder cues or multiple sessions of IS. Parenthetically, a common source of PTSD, combat experience, is a series of spaced traumatic events. Thus, it would be reasonable to ask whether multiple sessions of IS would be a more useful model. Recall that a choice escape procedure in which task-irrelevant cues are present and escape follows the correct choice after a short, variable delay reveals slower learning even a week after yoked IS, as compared to ES or a no-shock control treatment (Jackson et al., 1980). Does this choice escape deﬁcit resemble any of the aspects of PTSD? The parallel would be impaired executive functioning, manifested as long-term failure to make adaptive choices to deal with aversive events, especially when even the best choices do not terminate the aversion immediately

06-Schachtman-06.indd 141

141

and potentially distracting external stimuli abound. Parallels in the Neural Bases of the Phenomena

Another place to look for parallels between learned helplessness effects and PTSD is in the neural bases of the phenomena. We have seen that IS conditions stronger fear to the trauma situation than ES, and it is known that a circuit in the amygdala mediates the effect (e.g., Wilensky, Schafe, Kristensen, & LeDoux, 2006). So there should be greater activation of the amygdala in response to trauma-related cues in PTSD patients than in controls. Moreover, inhibitory inputs from mPFCv to amygdala are important in the extinction of fear (e.g., Quirk, Russo, Barron, & Lebron, 2000; Sierra-Mercado, Corcoran, Lebron, & Quirk, 2006), so we should see reduced mPFCv activity in PTSD patients relative to controls during extinction of fear. In general, neuroimaging research has supported this view, ﬁnding that participants with PTSD, as compared with those without it, show greater activation of the amygdala and attenuated activation of mPFC to reminders of traumatic events as compared to control conditions. For example, in an early PET study, Bremner et al. (1999) found that Vietnam veterans with PTSD showed decreased blood ﬂow, a correlate of neural activity, in mPFC in response to combat-related sights and sounds, whereas veterans without PTSD did not show this effect (also see, e.g., Lindauer et al., 2004). Liberzon et al.(1999) compared the responses to combat sounds and white noise in combat veterans with PTSD, combat veterans without PTSD, and healthy controls who had not been exposed to combat. Only the veterans with PTSD showed greater blood ﬂow in the left amygdala in response to combat sounds versus white noise. Shin et al. (2004), using PET and script-driven imagery, showed decreases in blood ﬂow in mPFC for traumatic versus neutral material in veterans with PTSD but not in those without PTSD. Moreover, there was an inverse correlation between changes in activity in the mPFC and in activity in the left amygdala and the right amygdala/periamygdaloid cortex.

2/15/2011 7:43:34 PM

142

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

The short-lived, at least in the absence of reminders, enhancement of fear conditioning/ expression studied in the Maier-Watkins laboratory results from IS sensitizing serotonergic neurons in the DRN that have excitatory effects on the amygdala. If the IS are accompanied by excitatory glutamatergic outputs from mPFCv to GABA neurons in DRN, the sensitization does not occur, and the expression of newly conditioned fear is no greater than in the ES group. The parallel ﬁnding in PTSD would be that PTSD patients show enhanced activation in the DRN and amygdala, as well as decreased activation of mPFCv, relative to trauma-exposed but non-PTSD controls, when they perceive a stimulus to which fear has been conditioned, even when that stimulus and the aversive event are unrelated to the original trauma. Because of its small size, there has been little neuroimaging of the DRN, but for the amygdala and mPFCv, there are data in accord with this prediction. In a positron emission tomography (PET) study, women with early sexual abuse and PTSD and healthy, nonabused women received some unreinforced presentations of a blue square, followed by pairings of that CS with an annoying shock, and then some extinction trials with the square (Bremner et al., 2005). In another session participants experienced unpaired presentations of the blue square and shocks followed by extinction trials with the blue square. The PTSD group acquired increased anxiety as a result of the conditioning, and to a much greater extent than controls. Moreover, the PTSD group had increased blood ﬂow in the amygdala and decreased blood ﬂow in the mPFCv/anterior cingulate in acquisition (paired condition minus unpaired condition) and decreased blood ﬂow in the mPFC in extinction (extinction after paired condition minus extinction after unpaired condition), relative to controls. Medial prefrontal blood ﬂow during extinction and anxiety were negatively correlated in the PTSD group. The hypervigilance characteristic of PTSD might also be a consequence of the heightened amygdala activity characteristic of the fear and anxiety that accompany learned helplessness. Stimulation of the amygdala increases vigilance and arousal. For example, electrical stimulation

06-Schachtman-06.indd 142

of its central nucleus in rabbits results in cholinergically mediated arousal in the sensory cortex (Kapp, Supple, & Whalen, 1994; also see a review by Davis & Whalen, 2001). This widespread effect should lower thresholds for sensory stimuli and increase their impact, and it is a mechanism of vigilance. If sensory stimuli are “ﬂooding in,” then the trouble concentrating that is characteristic of PTSD might be one result. Rauch, Shin, Whalen, and Pitman (1998) argued that in PTSD the hippocampus would be hypoactive, and that such hypoactivity would be responsible for the characteristic memory deﬁcits and the diminished control by context, for example, responding to a loud noise at home as if in a combat situation. Studies of the hippocampus in PTSD have focused primarily on (a) hippocampal volume and (b) verbal declarative memory, which is known to depend on the hippocampus (e.g., Tulving & Markowitsch, 1998). With respect to hippocampal volume, recent meta-analyses have shown that right and left hippocampal volumes are reduced in both men and women with PTSD (Kitayama, Vaccarino, Kutner, Weiss, & Bremner, 2005; Smith, 2005). Regarding declarative memory, a meta-analysis of studies on memory for emotionally neutral information in PTSD, Brewin, Kleiner, Vasterling, and Field (2007) concluded that adults with PTSD showed a small to moderate deﬁcit in verbal memory. Bremner et al. (2003) found that women who had suffered early childhood sexual abuse and had PTSD had smaller hippocampal volumes than women who had been abused but were without PTSD. Moreover, when they were asked to recall a paragraph that had been read to them, women without PTSD showed increased blood ﬂow in the hippocampus, but those with PTSD did not (also see Shin et al., 2004). To the extent that PTSD and learned helplessness reﬂect similar processes, there should be hippocampal dysfunction in learned helplessness. There has been relatively little research on the hippocampus in learned helplessness. However, Bland, Schmid, Greenwood, Watkins, and Maier (2006), using the triadic design, found that IS, but not ES, suppressed neural progenitor cell proliferation in the dentate gyrus

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

of hippocampus and also decreased the survival of neurons there 28 days after stress. In addition, ES, but not IS, increased the expression of ﬁbroblast growth factor 2 (FGF-2), a neuroprotective factor that induces differentiation of neural progenitor cells into neurons in the hippocampus. As far as we know, there are no studies that relate learned helplessness effects on hippocampal parameters to effects on aspects of memory. A Cautionary Note

The discussion thus far has treated the symptoms of PTSD, for example, hyperarousal or a smaller hippocampus, as though they were caused by the trauma, in conjunction with some risk factors. However, it may be the case that persons with smaller hippocampi or hyperarousal are more affected by trauma, and thus more likely to be diagnosed with PTSD. In a widely cited study, Gilbertson et al. (2002) asked whether the hippocampal volume differences reﬂect the response to trauma or instead a preexisting disposition to react to trauma pathologically. Participants were Vietnam War combat veterans and their identical twins with no combat experience. Some of the combat veterans had developed chronic PTSD, while others had never developed PTSD. Severity of symptoms in the veterans with PTSD was signiﬁcantly negatively correlated with their hippocampal volume (adjusted for total brain volume) and also with the hippocampal volume of their non-combat-exposed twins. Moreover, combat veterans who developed more severe PTSD (the highest 12 scorers of 17 men with PTSD) had smaller hippocampal volumes than veterans who did not develop PTSD, but the unexposed twins of the former also had smaller hippocampal volumes than the twins of the latter. Thus, smaller hippocampal volume in PTSD may reﬂect a preexisting vulnerability factor. In another study with the same participants, Gilbertson et al. (2006) observed an analogous result: poorer verbal memory and executive function, and lower IQ, for combat veterans with PTSD and their twins than for combat veterans without PTSD and their twins. Scores on tests of verbal memory, executive function, and IQ were negatively correlated with

06-Schachtman-06.indd 143

143

the severity of PTSD symptoms, but scores on tests of visual memory and visuospatial ability were uncorrelated with symptom severity. The authors argued that the high functioning tapped by the ﬁrst set of tests acted to protect against pathological responses to trauma. The contribution of preshock cognitive functioning to vulnerability to learned helplessness effects has not been studied in rats, though it has been shown that a noncognitive characteristic, the degree of neophobia in an open ﬁeld, modulates the effect of 60 inescapable shocks upon subsequent escape performance (Minor, Dess, Ben-David, & Chang, 1994). It is not possible in a short chapter to be fully comprehensive; there are several important aspects of learned helplessness and its relation to PTSD that we have not discussed. Although we have noted that trauma victims vary in their susceptibility to PTSD, we have not discussed the full range of factors that inﬂuence the severity of the response to trauma (but see Mineka & Oehlberg, 2008, for a thorough review). Nor have we discussed the effects of perceived control of the trauma or of events in general on PTSD (but see, e.g., Basoglu, Livanou, & Crnobaric, 2007; Basoglu et al., 1997; Engelhard, Macklin, McNally, van den Hout, & Arntz, 2001). Some Final Thoughts

It should be noted that learned helplessness is just one of many putative animal models of PTSD (see reviews by Foa et al., 1992; Rasmusson & Charney, 1997; Siegmund & Wotjak, 2006; Stam, 2007; Yehuda & Antelman, 1993). Most of these models rely on both fear conditioning and nonassociative sensitization processes (e.g., Cohen, Liberzon, & Richter-Levin, 2009; Siegmund & Wotjak, 2007). There have been few experimental comparisons of different induction procedures (e.g., Murison & Overmier, 1998), and only a few of the models have looked at a wide range of dependent variables and modulating factors (notable is the research from H. Cohen’s laboratory; e.g., Cohen et al., 2009). The sensitizing effect of IS versus ES and HC on later fear conditioning that Maier and his colleagues observed after 100 inescapable shocks has been

2/15/2011 7:43:34 PM

144

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

observed in other laboratories with fewer shocks (e.g., Rau, DeCola, & Fanselow, 2005; Shors & Servatius, 1997; Siegmund & Wotjak, 2007; van der Kolk, Greenberg, Boyd, & Krsytal, 1985; van Dijken, van der Heyden, Mos, & Tilders, 1992). For example, Rau and Fanselow (2009) gave groups of rats 4 or 15 brief, inescapable shocks in one context, and then 90 days later placed them in a new context. They showed more freezing than no-shock controls; that is, context fear showed increased generalization over time, a ﬁnding in itself relevant for PTSD. The rats were left in the context until the generalized freezing extinguished, and then they were given a single shock. In a subsequent test, the 4-shock and 15-shock groups froze much more than the noshock controls, which did not differ from each other. So a few shocks seem, in some cases, to be as effective as many in inducing the sensitization of subsequent fear conditioning. Now with just a few shocks it is unlikely that their escapability matters. However, with many shocks it matters a great deal; recall that Baratta et al. (2007, 2008) found that prior escapable shocks actually reduced the effectiveness of fear conditioning conducted a week later below the level of a HC control. Whether traumas change our future lives depends on how we cope with them, and that coping likely depends upon prior mastery learning with other aversive experiences. Gaining control (and perhaps prediction) over trauma may to some extent immunize against the development of new fears.

REFERENCES Abramson, L. Y., Metalsky, G., & Alloy, L. B. (1989). Hopelessness depression: A theory-based subtype of depression. Psychological Review, 96, 358–372. Abramson, L. Y., Seligman, M. E. P., & Teasdale, J. (1978). Learned helplessness in humans: Critique and reformulation. Journal of Abnormal Psychology, 87, 49–74. Amat, J., Baratta, M. V., Paul, E., Bland, S. T., Watkins, L. R., & Maier, S. F. (2005). Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe action. Nature Neuroscience, 8, 365–371.

06-Schachtman-06.indd 144

Amat, J., Matus-Amat, P., Watkins, L. R., & Maier, S. F. (1998). Escapable and inescapable stress differentially alter extracellular levels of 5-HT in the basolateral amygdala of the rat. Brain Research, 783, 113–120. Amat, J. Paul, E., Watkins, L. R., & Maier, S. F. (2008). Activation of the ventral medial prefrontal cortex during an uncontrollable stressor reproduces both the immediate and long-term protective effects of behavioral control. Neuroscience, 154, 1178–1186. Amat, J., Paul, E., Zarza, C., Watkins, L. R., & Maier, S. F. (2006). Previous experience with behavioral control over stress blocks the behavioral and dorsal raphe nucleus activating effects of later uncontrollable stress: Role of the ventral medial prefrontal cortex. Journal of Neuroscience, 26, 13264–13272. Amat, J., Sparks, P. D., Matus-Amat, P., Griggs, J., Watkins, L. R., & Maier, S. F. (2001). The role of the habenular complex in the elevation of dorsal raphe nucleus serotonin and the changes in the behavioral responses produced by uncontrollable stress. Brain Research, 917, 118–126. Anisman, H., Zalcman, S., Shanks, N., & Zacharko, R. M. (1991). Multi-system regulation of performance deﬁcits induced by stressors: An animal model of depression. In A. Boulton, G. Baker, & M. T. M. Iverson (Eds.), Animal models in psychiatry II (pp. 1–60). Clifton, NJ: Humana. Baker, A. G. (1976). Learned irrelevance and learned helplessness: Rats learn that stimuli, reinforcers, and responses are uncorrelated. Journal of Experimental Psychology: Animal Behavior Processes, 2, 130–141. Baker, A. G., & Mackintosh, N. J. (1979). Preexposure to the CS alone, US alone, or CS and US uncorrelated: Latent inhibition, blocking by context, or learned irrelevance? Learning and Motivation, 10, 278–294. Bangasser, D. A., Santollo, J., & Shors, T. J. (2005). The bed nucleus of the stria terminalis is critically involved in enhancing associative learning after stressful experience. Behavioral Neuroscience, 119, 1459–1466. Baratta, M. V., Christianson, J. P., Gomez, D. M., Zarza, C. M., Amat, J., Masini, C. V., Watkins, L. R., & Maier, S. F. (2007). Controllable versus uncontrollable stressors bi-directionally modulate conditioned but not innate fear. Neuroscience, 146, 1495–1503. Baratta, M. V., Lucero, M. L., Amat, J., Watkins, L. R., & Maier, S. F. (2008). Role of the ventral medial

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

prefrontal cortex in mediating behavioral control-induced reduction of later conditioned fear. Learning and Memory, 15, 84–87. Basoglu, M., Mineka, S., Paker, M., Aker, T., Livanou, M., & Gok, S. (1997). Psychological preparedness for trauma as a protective factor in survivors of torture. Psychological Medicine, 27, 1421–1433. Basoglu, M., Livanou, M., & Crnobaric, C. (2007). Torture versus other cruel, inhuman and degrading treatment: Is the distinction real or apparent? Archives of General Psychiatry, 64, 277–285. Black, A. H. (1977). Comments on “Learned helplessness: Theory and evidence” by Maier and Seligman. Journal of Experimental Psychology: General, 106, 41–43. Bland, S. T., Schmid, M. J., Greenwood, B. N., Watkins, L. R., & Maier, S. F. (2006). Behavioral control of the stressor modulates stress-induced changes in neurogenesis and ﬁbroblast growth factor-2. Neuroreport, 17, 593–597. Blechert, J., Michael, T., Vriends, N., Margraf, J., Wilhelm, F. H. (2007). Fear conditioning in posttraumatic stress disorder: Evidence for delayed extinction of autonomic, experiential, and behavioural responses. Behaviour Research and Therapy, 45, 2019–2033. Bonardi, C., Hall, G., & Ong, S. Y. (2005). Analysis of the learned irrelevance effect in appetitive Pavlovian conditioning. Quarterly Journal of Experimental Psychology: Section B, 58B, 141–162. Bremner, J. D., Staib, L. H., Kaloupek, D., Southwick, S. M., Soufer, R., & Charney, D. S. (1999). Neural correlates of exposure to traumatic pictures and sound in Vietnam combat veterans with and without posttraumatic stress disorder: A positron emission tomography study. Biological Psychiatry, 45, 806–816. Bremner, J. D., Vermetten, E., Schmahl, C., Vaccarino, V., Vythilingam, M., Afzal, N., . . . Charney, D. S. (2005). Positron emission tomographic imaging of neural correlates of a fear acquisition and extinction paradigm in women with childhood sexual-abuse-related post-traumatic stress disorder. Psychological Medicine, 35, 791–806. Bremner, J. D., Vythilingam, M., Vermetten, E., Southwick, S. M., McGlashan, T., Staib, L. H., . . .Charney, D. S. (2003). Neural correlates of declarative memory for emotionally valenced words in women with posttraumatic stress

06-Schachtman-06.indd 145

145

disorder related to early childhood sexual abuse. Biological Psychiatry, 53, 879–889. Brewin, C. R., Kleiner, J. S., Vasterling, J. J., & Field, A. P. (2007). Memory for emotionally neutral information in post-traumatic stress disorder: A meta-analytic investigation. Journal of Abnormal Psychology, 116, 448–463. Christianson, J. P., Benison, A. M., Jennings, J., Sandsmark, E. K., Amat, J., Kaufman, R. D., . . .Maier, S. F. (2008). The sensory insular cortex mediates the stress buffering effects of safety signals but not behavioral control. Journal of Neuroscience, 28, 13703–13711. Christianson, J. P., Paul, E. D., Irani, M., Thompson, B. M., Kubala, K. H., Yirmiya, R., . . . Maier, S. F. (2008). The role of prior stressor controllability and the dorsal raphe nucleus in sucrose preference and social exploration. Behavioural Brain Research, 193, 87–93. Christianson, J. P., Thompson, B. M., Watkins, L. R., & Maier, S. F. (2008). Medial prefrontal cortical activation modulates the impact of controllable and uncontrollable stressor exposure on a social exploration test of anxiety in the rat. Stress, 1, 1–6. Cohen, H., Liberzon, I., & Richter-Levin, G. (2009). Exposure to extreme stress impairs contextual odour discrimination in an animal model of PTSD. International Journal of Neuropsychopharmacology, 12, 291–303. Davis, M., & Whalen, P. J. (2001). The amygdala: Vigilance and emotion. Molecular Psychiatry, 6, 13–34. DeCola, J. P., Rosellini, R. A., & Warren, D. A. (1988). A dissociation of the effects of control and prediction. Learning and Motivation, 19, 269–282. Dess, N. K., Linwick, D., Patterson, J., Overmier, J. B., & Levine, S. (1983). Immediate and proactive effects of controllability and predictability on plasma cortisol responses to shock in dogs. Behavioral Neuroscience, 97, 1005–1016. Dess, N. K., & Overmier, J. B. (1989). General learned irrelevance: Proactive effects on Pavlovian conditioning in dogs. Learning and Motivation, 20, 1–14. Drugan, R. C., & Maier, S. F. (1983). Analgesic and opioid involvement in the shock-elicited activity and escape deﬁcits produced by inescapable shock, Learning and Motivation, 14, 30–47. Engelhard, I. M., Macklin, M. L, McNally, R. J., van den Hout, M. A., & Arntz, A. (2001). Emotion- and intrusion-based reasoning in

2/15/2011 7:43:34 PM

146

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Vietnam veterans with and without chronic posttraumatic stress disorder. Behaviour Research and Therapy, 39, 1339–1348. Foa, E., Zinbarg, R., & Rothbaum, B. (1992). Uncontrollability and unpredictability in posttraumatic stress disorder: An animal model. Psychological Bulletin, 112, 218–238. Gabbott, P. L., Warner, T. A., Jays, P. R., Salway, P., & Busby, S. J. (2005). Prefrontal cortex in the rat: Projections to subcortical autonomic, motor, and limbic centers. Journal of Comparative Neurology, 492, 145–177. Geuze, E., Westenberg, H. G., Jochims, A., deKloet, C. C., Bohus, M., Vermetten, E., & Schmahl, C. (2007). Altered pain processing in veterans with posttraumatic stress disorder. Archives of General Psychiatry, 64, 76–85. Gibb, B. E., & Alloy, L. B. (2006). A prospective test of the hopelessness theory of depression in children. Journal of Clinical Child and Adolescent Psychology, 35, 264–274. Gibb, B. E., Alloy, L. B., Abramson, L. Y., & Marx, B. P. (2003). Childhood maltreatment and maltreatment-speciﬁc inferences: A test of Rose and Abramson’s (1992) extension of the hopelessness theory. Cognition and Emotion, 17, 917–931. Gilbertson, M. W., Paulus, L. A., Williston, S. K., Gurvits, T. V., Lasko, N. B., Pitman, R. K., & Orrr, S. P. (2006). Neurocognitive function in monozygotic twins discordant for combat exposure: Relationship to posttraumatic stress disorder. Journal of Abnormal Psychology, 115, 484–495. Gilbertson, M. W., Shenton, M. E., Ciszewski, A., Kasai, K., Lasko, N. B., Orr, S. P., & Pitman, R. K. (2002). Smaller hippocampal volume predicts pathologic vulnerability to psychological trauma. Nature Neuroscience, 5, 1242–1247. Graeff, F. G., Guimaress, F. S., De Andrade, T. G. C. S., & Deakin, J. F. W. (1996). Role of 5-HT in stress, anxiety and depression. Pharmacology, Biochemistry and Behavior, 54, 129–141. Grahn, R. E., Hammack, S. E., Will, M. J., O’Connor, K. A., Deak, T., Sparks, P. D., . . . Maier, S. F. (2002). Blockade of alpha1 adrenoreceptors in the dorsal raphe nucleus prevents enhanced conditioned fear and impaired escape performance following uncontrollable stressor exposure in the rat. Behavioural Brain Research, 134, 387–392. Grahn, R. E., Kalman, B. A., Brennan, F. X., Watkins, L. R., & Maier, S. F. (1995). The elevated

06-Schachtman-06.indd 146

plus maze is not sensitive to the effect of stressor controllability in rats, Pharmacology, Biochemistry and Behavior, 52, 565–570. Grahn, R. E., Will, M. J., Hammack, S. E., Maswood, S., McQueen, M. B., Watkins, L. R., & Maier, S. F. (1999). Activation of serotoninimmunoreactive cells in the dorsal raphe nucleus in rats exposed to an uncontrollable stressor. Brain Research, 826, 35–43. Greenwood, B. N., & Fleshner, M. (2008). Exercise, learned helplessness, and the stress-resistant rat. Neuromolecular Medicine, 10, 81–98. Greenwood, B. N., Foley, T. E., Day, H. E., Campisi, J., Hammack, S. H., Maier, S. F., & Fleshner, M. (2003). Freewheel running prevents learned helplessness/behavioral depression: Role of dorsal raphe serotonergic neurons. Journal of Neuroscience, 23, 2889–2898. Greenwood, B. N., Strong, P. V., Dorey, A. A., & Fleshner, M. (2007). Therapeutic effects of exercise: Wheel running reverses stress-induced interference with shuttle box escape. Behavioral Neuroscience, 121, 992–1000. Grillon, C., & Morgan, C. A., III. (1999). Fearpotentiated startle conditioning to explicit and contextual cues in Gulf War veterans with posttraumatic stress disorder. Journal of Abnormal Psychology, 108, 134–142. Hammack, S. E., Richey, K. J., Schmid, M. J., LoPresti, M. L., Watkins, L. R., & Maier, S. F. (2002). The role of corticotropin releasing hormone in the dorsal raphe nucleus in mediating the behavioral consequences of uncontrollable stress. Journal of Neuroscience, 22, 1020–1026. Hammack, S. E., Richey, K. J., Watkins, L. R., & Maier, S. F. (2004). Chemical lesion of the bed nucleus of the stria terminalis blocks the behavioral consequences of uncontrollable stress. Behavioral Neuroscience, 118, 443–448. Hammack, S. E., Schmid, M. J., LoPresti, M. L., Der-Avakian, A., Pellymounter, M. A, Foster, A. C., . . . Maier, S. F. (2003). Corticotropin releasing hormone type 2 receptors in the dorsal raphe nucleus mediate the behavioral consequences of uncontrollable stress. Journal of Neuroscience, 23, 1019–1025. Hannum, R. D., Rosellini, R. A., & Seligman, M. E. P. (1976). Learned helplessness in the rat: Retention and immunization. Developmental Psychology, 12, 449–454. Hunter, A. M., Balleine, B. W., & Minor, T. R. (2003). Helplessness and escape performance: Glutamate-adenosine interactions in

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

the frontal cortex. Behavioral Neuroscience, 117, 123–135. Jackson, R. L., Alexander, J. H., & Maier, S. F. (1980). Learned helplessness, inactivity, and associative deﬁcits: The effects of inescapable shock on response choice escape learning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 1–20. Jackson, R. L., Maier, S. F., & Coon, D. J. (1979). Long-term analgesic effects of inescapable shock and learned helplessness. Science, 206, 91–94. Jankowski, M. P., & Sesack, S. R. (2004). Prefrontal cortical projections to the rat dorsal raphe nucleus: Ultrastructural features and associations with serotonin and gamma-aminobutyric acid neurons. Journal of Comparative Neurology, 468, 518–529. Jovanovic, T., Norrholm, S. D., Fennell, J. E., Keyes, M., Fiallos, A. M., Myers, K. M., . . . Duncan, E. J. (2009). Posttraumatic stress disorder may be associated with impaired fear inhibition: Relation to symptom severity. Psychiatry Research, 167, 151–160. Kapp, B. S., Supple, W. F. Jr., & Whalen, P. J. (1994). Effects of electrical stimulation of the amygdaloid central nucleus on neocortical arousal in the rabbit. Behavioral Neuroscience, 108, 81–93. Kennett, G. A., Marcou, M., Dourish, C. T., & Curzon, G. (1987). Single administration of 5-HT1A agonists decrease 5-HT1A presynaptic, but not postsynaptic receptor mediated responses: Relationship to antidepressant-like actions. European Journal of Pharmacology, 138, 56–60. Kiser, R. S., Brown, C. A., Sanghera, M. K., & German, D. C. (1980). Dorsal raphe nucleus stimulation reduces centrally-elicited fearlike behavior. Brain Research, 191, 265–272. Kitayama, N., Vaccarino, V., Kutner, M., Weiss, P., & Bremner, J. D. (2005). Magnetic resonance imaging (MRI) measurement of hippocampal volume in posttraumatic stress disorder: A meta-analysis. Journal of Affective Disorders, 88, 79–86. Klosterhalfen, W., & Klosterhalfen, S. (1983). A critical analysis of the animal experiments cited in support of learned helplessness. Psychologische Beitrage, 25, 436–458. Lawry, J. A., Lupo, V., Overmier, J. B., Kochevar, J., Hollis, K. L., & Anderson, D. C. (1978). Interference with avoidance behavior as a function of qualitative properties of inescapable shocks. Animal Learning and Behavior, 6, 147–154.

06-Schachtman-06.indd 147

147

Lee, R. K. K., & Maier, S. F. (1988). Inescapable shock and attention to internal versus external cues in a water escape discrimination task. Journal of Experimental Psychology: Animal Behavior Processes, 14, 302–311. Levis, D. J. (1980). The learned helplessness effect: An expectancy, discrimination deﬁcit, or motivational-induced persistence? Journal of Research in Personality, 14, 158–169. Liberzon, I., Taylor, S. F., Amdur, R., Jung, T. D., Chamberlain, K. R., Minoshima, S., . . .Fig, L. M. (1999). Brain activation in PTSD in response to trauma-related stimuli. Biological Psychiatry, 45, 817–826. Lindauer, R. J., Booij, J., Habraken, J. B., Uylings, H. B., Olff, M., Carlier, I. V., . . .Gersons, B. P. (2004). Cerebral blood ﬂow changes during scriptdriven imagery in police ofﬁcers with posttraumatic stress disorder. Biological Psychiatry, 56, 853–861. Linden, D. R., Savage, L. M., & Overmier, J. B. (1997). General learned irrelevance: A Pavlovian analogue to learned helplessness. Learning and Motivation, 28, 230–247. LoLordo, V. M. (2001). Learned helplessness and depression. In M. E. Carroll & J. B. Overmier (Eds.), Animal research and human health (pp. 63–77). Washington, DC: American Psychological Asociation. LoLordo, V. M., & Taylor, T. L. (2001). Effects of uncontrollable aversive events: Some unsolved puzzles. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 469–504). Mahwah, NJ: Lawrence Erlbaum Associates. Mackintosh, N. J. (1973). Stimulus selection: Learning to ignore stimuli that predict no change in reinforcement. In R. A. Hinde & J. S. Hinde (Eds.), Constraints on learning (pp. 75–100). London, England: Academic Press. MacLennan, A. J., Drugan, R. C., Hyson R. L., Maier, S. F., Madden, J., IV, & Barchas, J. D. (1982). Hypophysectomy and dexamethasone block the analgesic but not the shuttlebox escape learning consequences of inescapable shock. Journal of Comparative and Physiological Psychology, 96, 904–912. Maier, S. F. (1990). The role of fear in mediating the shuttle escape learning deﬁcit produced by inescapable shock, Journal of Experimental Psychology: Animal Behavior Processes, 16, 137–150. Maier, S. F. (2001). Exposure to the stressor environment prevents the temporal dissipation of

2/15/2011 7:43:34 PM

148

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

behavioral depression/learned helplessness. Biological Psychiatry, 49, 763–773. Maier, S. F., Albin, R. W., & Testa, T. J. (1973). Failure to learn to escape in rats previously exposed to inescapable shock depends on nature of escape response. Journal of Comparative and Physiological Psychology, 85, 581–592. Maier, S. F., Amat, J., Baratta, M. V.,Paul, E., & Watkins, L. R. (2006). Behavioral control, the medial prefrontal cortex, and resilience. Dialogues in Clinical Neuroscience, 8, 397–406. Maier, S. F., Busch, C. R., Maswood, S., Grahn, R. E., & Watkins, L. R. (1995). The dorsal raphe nucleus is a site of action mediating the behavioral effects of the benzodiazepine receptor inverse agonist DMCM. Behavioral Neuroscience, 109, 759–766. Maier, S. F., Coon, D. J., McDaniel, M. A., Jackson, R. L., & Grau, J. (1979). The time course of learned helplessness, inactivity and nociceptive deﬁcits in rats. Learning and Motivation, 10, 467–487. Maier, S. F., Grahn, R. E., Kalman, B. A., Sutton, L. C., Wiertelak, E. P., & Watkins, L. R. (1993). The role of the amygdala and dorsal raphe nucleus in mediating the behavioral consequences of inescapable shock. Behavioral Neuroscience, 107, 377–388. Maier, S. F., Grahn, R. E., Maswood, S., & Watkins, L. R. (1995). The benzodiazepine receptor antagonists ﬂumazenil and CGS8216 block the enhancement of fear conditioning and interference with escape behavior produced by inescapable shock. Psychopharmacology, 121, 250–259. Maier, S. F., Grahn, R. E., & Watkins, L. R. (1995). 8-OH-DPAT microinjected in the region of the dorsal raphe nucleus blocks and reverses the enhancement of fear conditioning and interference with escape produced by exposure to inescapable shock. Behavioral Neuroscience, 109, 404–412. Maier, S. F., & Jackson, R. L. (1977). The nature of the initial coping response and the learned helplessness effect. Animal Learning and Behavior, 5, 407–414. Maier, S. F., Kalman, B. A., & Grahn, R. E. (1994). Chlordiazepoxide microinjected into the region of the dorsal raphe nucleus eliminates the interference with escape responding produced by inescapable shock whether administered before inescapable shock or escape testing. Behavioral Neuroscience, 108, 121–130.

06-Schachtman-06.indd 148

Maier, S. F., & Keith, J. (1987). Shock signals and the development of stress-induced analgesia. Journal of Experimental Psychology: Animal Behavior Processes, 13, 226–238. Maier, S. F., & Minor, T. R. (1993). Dissociation of interference with the speed and accuracy of escape produced by inescapable shock. Behavioral Neuroscience, 107, 139–146. Maier, S. F., Ryan, S. M., Barksdale, C. M., & Kalin, N. H. (1986). Stressor controllability and the pituitary-adrenal system, Behavioral Neuroscience, 100, 669–674. Maier, S. F., & Seligman, M. E. P. (1976). Learned helplessness: Theory and evidence. Journal of Experimental Psychology: General, 105, 3–46. Maier, S. F., & Warren, D. A. (1988). Controllability and safety signals exert dissimilar proactive effects on nociception and escape performance. Journal of Experimental Psychology: Animal Behavior Processes, 14, 18–25. Maier, S. F., & Watkins, L. R. (1998). Stressor controllability, anxiety, and serotonin. Cognitive Therapy and Research, 22, 595–613. Maier, S. F., & Watkins, L. R. (2005). Stressor controllability and learned helplessness: The roles of the dorsal raphe nucleus, serotonin, and corticotropin-releasing factor. Neuroscience and Biobehavioral Reviews, 29, 829–841. Malberg, J. E., & Duman, R. S. (2003). Cell proliferation in adult hippocampus is decreased by inescapable stress. Reversal by ﬂuoxetine treatment. Neuropsychopharmacology, 28, 1562–1571. Martin, P., & Puech, A. J. (1996). Antagonism by benzodiazepines of the effects of serotonin-, but not norepinephrine-, uptake blockers in the learned helplessness paradigm in rats. Biological Psychiatry, 39, 882–890. Maswood, S., Barter, J. E., Watkins, L. R., & Maier, S. F. (1998). Exposure to inescapable but not escapable shock increases extracellular levels of 5-HT in the dorsal raphe nucleus of the rat. Brain Research, 783, 115–120. Mineka, S., Cook, M., & Miller, S. (1984). Fear conditioned with escapable and inescapable shock: Effects of a feedback stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 10, 307–323. Mineka, S., & Oehlberg, K. (2008). The relevance of recent developments in classical conditioning to understanding the etiology and maintenance of anxiety disorders. Acta Psychologica, 127, 567–580.

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

Minor, T. R. (1990). Conditioned fear and neophobia following inescapable shock. Animal Learning and Behavior, 18, 212–226. Minor, T. R., Dess, N. K., Ben-David, E., & Chang, W-C. (1994). Individual differences in vulnerability to inescapable shock in rats. Behavioral Neuroscience, 20, 402–412. Minor, T. R., Dess, N. K., & Overmier, J. B. (1991). Inverting the traditional view of “learned helplessness.” In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 87–133). Hillsdale, NJ: Erlbaum. Minor, T. R., Jackson, R. L., & Maier, S. F. (1984). Effects of task irrelevant cues and reinforcement delay on choice escape learning following inescapable shock: Evidence for a deﬁcit in selective attention. Journal of Experimental Psychology: Animal Behavior Processes, 10, 543–556. Minor, T. R., & LoLordo, V. M. (1984). Escape deﬁcits following inescapable shock: The role of contextual odor. Journal of Experimental Psychology: Animal Behavior Processes, 10, 168–181. Minor, T. R., Pelleymounter, M. A., & Maier, S. F. (1988). Uncontrollable shock, forebrain norepinephrine, and stimulus selection during choice escape learning, Psychobiology, 16, 135–145. Minor, T. R., Plumb, T. N., Schell, C. J., & Pham, A. K. (2010). Brain adenosine signaling in psychological trauma and comorbid depression. In L. Sher & A. Vilens (Eds.) Neurobiology of PTSD (pp. 229–257). New York, NY: Nova Science Publishers. Minor, T. R., Trauner, M. A., Lee, C., & Dess, N. K. (1990). Modeling signal features of escape response: Effects of cessation conditioning in “Learned Helplessness” paradigm. Journal of Experimental Psychology: Animal Behavior Processes, 16, 123–136. Murison, R., & Overmier, J. B. (1998). Comparison of different animal models of stress reveals a non-monotonic effect. Stress, 2, 227–230. Nakajima, M., Nakajima, S., & Imada, H. (1999). General learned irrelevance and its prevention. Learning and Motivation, 30, 265–280. Nemeroff, C. B., Bremner, J. D., Foa, E. B., Mayberg, H. S., North, C. S., & Stein, M. B. (2006). Posttraumatic stress disorder: A state of the science review. Journal of Psychiatric Research, 40, 1–21. Neumaier, J. F., Petty, F., Kramer, G. L., Szot, P., Hamblin, M. W. (1997). Learned helplessness increases 5-hydroxytryptamine 1B receptor

06-Schachtman-06.indd 149

149

mRNA levels in the rat dorsal raphe nucleus. Biological Psychiatry, 41, 668–674. Orr, S. P., Metzger, L. J., Lasko, N. B., Macklin, M. L., Peri, T., & Pitman, R. K. (2000). De novo conditioning in trauma-exposed individuals with and without posttraumatic stress disorder. Journal of Abnormal Psychology, 109, 290–298. Overmier, J. B. (1968). Interference with avoidance behavior: Failure to avoid traumatic shock. Journal of Experimental Psychology, 78, 340–343. Overmier, J. B. (1985). Toward a reanalysis of the causal structure of the learned helplessness syndrome. In F. R. Brush & J. B. Overmier (Ed.), Affect, conditioning, and cognition: Essays on the determinants of behavior (pp. 211–227). Hillsdale, NJ: Erlbaum. Overmier, J. B. (1988). Psychological determinants of when stressors stress. In D. H. Hellhammer, I. Florin, & H. Weiner (Eds.). Neurobiological approaches to the human disease (pp. 235–259). Toronto, ON: Hans Huber. Overmier, J. B., & LoLordo, V. M. (1998). Learned helplessness. In W. O’Donohue (Ed.), Learning and behavior therapy (pp. 352–373). Boston, MA: Allyn and Bacon. Overmier, J. B., Patterson, J., & Wielkiewicz, R. M. (1980). Environmental contingencies as sources of stress in animals. In S. Levine & H. Ursin (Eds.), Coping and health. (pp. 1–38). New York, NY: Plenum. Overmier, J. B., & Seligman, M. E. P. (1967). Effects of inescapable shock upon subsequent escape and avoidance learning. Journal of Comparative and Physiological Psychology, 63, 28–33. Pavlov, I. P. (1927). Conditioned reﬂexes. Oxford, England: Oxford University Press. Peterson, C., Maier, S. F., & Seligman, M. E. P. (1993). Learned helplessness: A theory for the age of personal control. New York, NY: Oxford University Press. Quirk, G. J., Russo, G. K., Barron, J. L., & Lebron, K. (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. Journal of Neuroscience, 20, 6225–6231. Rasmusson, A. M., & Charney, D. S. (1997). Animal models of relevance to PTSD. Annals of the New York Academy of Sciences, 821, 332–350. Rau, V., DeCola, J. P., & Fanselow, M. S. (2005). Stress-induced enhancement of fear learning: An animal model of posttraumatic stress. Neuroscience and Biobehavioral Reviews, 29, 1207–1223.

2/15/2011 7:43:34 PM

150

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Rau, V., & Fanselow, M. S. (2009). Exposure to a stressor produces a long lasting enhancement of fear learning in rats. Stress, 12, 125–133. Rauch, S. L., Shin, L. M., Whalen, P. J., & Pitman, R. K. (1998). Neuroimaging and the anatomy of PTSD. CNS Spectrums, 3(Suppl. 2), 30–41. Rodd, Z. A., Rosellini, R. A., Stock, H. S., & Gallup, G. G., Jr. . (1997). Learned helplessness in chickens (Gallus gallus): Evidence for attentional basis. Learning and Motivation, 28, 43–55. Rosellini, R. A. (1978). Inescapable shock interferes with the acquisition of an appetitive operant. Animal Learning and Behavior, 6, 155–159. Rosellini, R. A., DeCola, J. P., & Warren, D. A. (1986). The effect of feedback stimuli on contextual fear depends upon the length of the intertrial interval. Learning and Motivation, 17, 229–242. Rosellini, R. A., Warren, D. A., & DeCola, J. P. (1987). Predictability and controllability: Differential effects upon contextual fear, Learning and Motivation, 18, 392–420. Rozeske, R. R., Der-Avakian, A., Bland, S. T., Beckley, J. T., Watkins, L. R., & Maier, S. F. (2009). The medial prefrontal cortex regulates the differential expression of morphineconditioned place preference following a single exposure to controllable or uncontrollable stress. Neuropsychopharmacology, 34, 834–843. Schutz, M. T. B., DeAguiar, J. C., & Graeff, F. G. (1985). Antiaversive role of serotonin in the dorsal periaqueductal gray matter. Psychopharmacology, 85, 340–345. Seligman, M. E. P. (1975) Helplessness: On depression, dying, and death. San Francisco, CA: Freeman. Seligman, M. E. P., & Groves, D. (1970). Nontransient learned helplessness. Psychonomic Science, 19, 191–192. Seligman, M. E. P., & Maier, S. F. (1967). Failure to escape traumatic shock. Journal of Experimental Psychology, 74, 1–9. Seligman, M. E. P., Maier, S. F., & Solomon, R. L. (1971). Unpredictable and uncontrollable events. In F. R. Brush (Ed.), Aversive conditioning and learning (pp. 347–400). New York, NY: Academic Press. Shin, L. M., Orr, S. P., Carson, M. A., Rauch, S. L., Macklin, M. L., Lasko, N. B., . . .Pitman, R. K. (2004). Regional cerebral blood ﬂow in the amygdala and medial prefrontal cortex during traumatic imagery in male and female Vietnam

06-Schachtman-06.indd 150

veterans with PTSD. Archives of General Psychiatry, 61, 168–176. Shors, T. J., & Servatius, R. J. (1997). The contribution of stressor intensity, duration, and context to the stress-induced facilitation of associative learning. Neurobiology of Learning and Memory, 67, 92–96. Short, K. R., & Maier, S. F. (1993). Stressor controllability, social interaction, and benzodiazepine systems. Pharmacology, Biochemistry, and Behavior, 45, 1–9. Siegmund, A., & Wotjak, C. T. (2006). Toward an animal model of posttraumatic stress disorder. Annals of the New York Academy of Sciences, 1071, 324–334. Siegmund, A., & Wotjak, C. T. (2007). Hyperarousal does not depend on trauma-related contextual memory in an animal model of posttraumatic stress disorder. Physiology and Behavior, 90, 103–107. Sierra-Mercado, D., Corcoran, K. A., Lebron, K., & Quirk, G. J. (2006). Inactivation of ventromedial prefrontal cortex reduces expression of conditioned fear and impairs subsequent recall of extinction. European Journal of Neuroscience, 24, 1751–1758. Simson, P. G., Weiss, J. M., Ambrose, M. J., & Webster, A. (1986). Infusion of a monoamine oxidase inhibitor into the locus coeruleus can prevent stress-induced behavioral depression. Biological Psychiatry, 21, 724–734. Smith, M. E. (2005). Bilateral hippocampal volume reduction in adults with post-traumatic stress disorder: A meta-analysis of structural MRI studies. Hippocampus, 15, 798–807. Stam, R. (2007). PTSD and stress sensitization: A tale of brain and body Part 2: Animal models. Neuroscience and Biobehavioral Reviews, 31, 558–584. Sutton, L. C., Lea, S. E., Will, M. J., Schwartz, B. A., Hartley, C. E., Poole, J. C., . . .Maier, S. F. (1997). Inescapable shock-induced potentiation of morphine analgesia. Behavioral Neuroscience, 111, 1105–1113. Takase, L. F., Nogueira, M. I., Bland, S. T., Baratta, M., Watkins, L. R., Maier, S. F., . . .Jacobs, B. L. (2005). Effect of number of tailshocks on learned helplessness and activation of serotonergic and noradrenergic neurons in the rat. Behavioural Brain Research, 162, 299–306. Tao, R., Ma, Z., & Auerbach, S. B. (1996). Differential regulation of 5-hydroxytryptamine release by GABAA and GABAB receptors in

2/15/2011 7:43:34 PM

TRAUMA AND LEARNED HELPLESSNESS

midbrain raphe nuclei and forebrain in rats. British Journal of Pharmacology, 119, 1375–1384. Testa, T. J., Juraska, J. M., & Maier, S. F. (1974). Prior exposure to inescapable electric shock in rats affects extinction behavior after the successful acquisition of an escape response. Learning and Motivation, 3, 380–392. Tulving, E., & Markowitsch, H. J. (1998). Episodic and declarative memory: Role of the hippocampus. Hippocampus, 8, 198–204. Van der Kolk, B., Greenberg, M., Boyd, H., & Krystal, J. (1985). Inescapable shock, neurotransmitters, and addiction to trauma: Toward a psychobiology of post traumatic stress. Biological Psychiatry, 20, 314–325. Van Dijken, H. H., van der Heyden, J. A. M., Mos, J., & Tilders, F. J. (1992). Inescapable footshocks induce progressive and long-lasting behavioral changes in male rats. Physiology and Behavior, 51, 787–794. Walker, D. L., Toufexis, D. J., & Davis, M. (2003). Role of the bed nucleus of the stria terminalis versus the amygdala in fear, stress and anxiety. European Journal of Pharmacology, 463, 199–216. Weiss, J. M., Boss-Williams, K., Moore, J. Demetrikopoulos, M. K., Ritchie, J. C., & West, C. H. (2005). Testing the hypothesis that locus coeruleus hyperactivity produces depression-related changes via galanin. Neuropeptides, 39, 281–287. Weiss, J. M., Goodman, P. A., Losito, B. G., Corrigan, S., Charry, J. M., & Bailey, W. H. (1981). Behavioral depression produced by an uncontrollable stressor. Relationship to norepinephrine, dopamine and serotonin levels in

06-Schachtman-06.indd 151

151

various regions of rat brain. Brain Research Reviews, 69, 167–205. Weiss, J. M., & Simson, P. G. (1985). Neurochemical basis of stress-induced depression, Psychopharmacological Bulletin, 21, 447–457. Wilensky, A. E., Schafe, G. E., Kristensen, M. P., & LeDoux, J. E. (2006). Rethinking the fear circuit: The central nucleus of the amygdala is required for the acquisition, consolidation, and expression of Pavlovian fear conditioning. Journal of Neuroscience, 26, 12387–12396. Will, M. J., Watkins, L. R., & Maier, S. F. (1998). Uncontrollable stressors potentiate the rewarding properties of morphine. Pharmacology, Biochemistry and Behavior, 60, 655–664. Williams, J. L., & Maier, S. F. (1977). Transituational immunization and therapy of learned helplessness in the rat. Journal of Experimental Psychology: Animal Behavior Processes, 3, 240–252. Woodmansee, W. W., Silbert, L. H., & Maier, S. F. (1993). Factors that modulate inescapable shock-induced reductions in daily activity in the rat. Pharmacology, Biochemistry and behavior, 45, 553–559. Woodson, J. C., Minor, T. R., & Job, R. F. S. (1998). Inhibition of adenosine deaminase by erythro9-(2-hydroxy-3-nonyl)adenine (EHNA) mimics the effect of inescapable shock on escape learning in rats. Behavioral Neuroscience, 112, 399–409. Yehuda, R., & Antelman, S. M. (1993). Criteria for rationally evaluating animal models of posttraumatic stress disorder. Biological Psychiatry, 33, 479–486.

2/15/2011 7:43:34 PM

CHAPTER 7 Aberrant Attentional Processes in Schizophrenia as Reﬂected in Latent Inhibition Data Robert E. Lubow

Latent inhibition (LI) is demonstrated when a previously unattended stimulus is less effective in a new learning situation than a novel stimulus. Since LI is reduced by dopamine agonists and increased by dopamine antagonists, and schizophrenic patients often display attentional impairments, LI has come to play an important role in the investigation of information processing deﬁcits in schizophrenia. The present chapter reviews the rationale for this approach and summarizes the LI data from schizophrenia patients and healthy groups that are self-rated on traits related to schizophrenia (schizotypality). The review suggests that schizophrenia patients with positive symptoms exhibit attenuated LI, whereas those with negative symptoms show normal or potentiated LI. These effects are accounted for by differences in the ability to shift attention from controlled to automatic processing and the manner in which such shifts are affected by the masking task load.

INTRODUCTION Any discourse that involves schizophrenia must begin by addressing issues of classiﬁcation. In this regard, it is generally agreed that the deﬁnition of a disease requires a coherent set of symptoms and, in many cases, a reliable response to a speciﬁc treatment. As such, schizophrenia does not qualify as a homogeneous diagnostic category. Indeed, according to the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV, 2000), a diagnosis of schizophrenia is assigned if a person has two or more of a variety of symptoms over a speciﬁed period of time. Although the list of symptoms is quite diverse, they have been grouped into three categories: positive, negative, and cognitive disorganization. Positive symptoms reﬂect unusual increases in normal behavior, and they include delusions, hallucinations, thought disorder, the latter of which can be expressed in incoherent speech,

use of neologisms, and disorders of movement, such as tics and uncontrolled grotesque mannerisms. On the other hand, negative symptoms represent abnormal decreases in normal behavior, including poverty of speech, ﬂat expression of affect, inability to ﬁnd pleasure, and social isolation, all of which are also consistent with a diagnosis of depression. The third category, cognitive disorganization, not always independent of positive and negative symptoms, may include disabilities in executive functioning, maintenance of attention, and working memory. Irrespective of questions regarding diagnostic reliability and validity, there is little doubt that the symptoms ascribed to schizophrenia, whether individually or in clusters, are powerfully debilitating. As such, schizophrenia poses a major health problem. While accurate ﬁgures are difﬁcult to obtain because of the widely different diagnostic criteria, it is clear that the numbers are distressingly high, typically cited as 1% (e.g., Regier et al., 1993), although more recent studies

152

07-Schachtman-07.indd 152

2/15/2011 7:43:48 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

suggest that between 0.4% and 0.6% of the population is affected (Bhugra, 2006; Goldner, Hsu, Waraich, & Somers, 2002). The devastating personal, social, and economic consequences of these symptoms, the limited ability to control them, and the dearth of viable explanations and theories have more than justiﬁed the rapid expansion of schizophrenia-related research. As such, the ﬁeld has expanded far beyond traditional psychiatry and psychology, and has embraced almost all aspects of neuroscience, including neuropsychology, neurochemistry, neurophysiology, pharmacology, and genetics, to the point that one may ask if there is still a place for the pure behavioral approach of traditional experimental psychology, or even better to ask how behavioral psychology is situated within the ever-broadening context of schizophrenia research. A complete exploration of this question would be a daunting task, requiring an examination of the many different subdisciplines and experimental paradigms within psychology that have been drawn into schizophrenia-related research, including areas such as perception, memory, attention, and learning. However, the present volume has a narrower scope, namely to focus on applications of conditioning theory and methodology, and, in the current chapter, to relate their contributions to furthering our understanding of the schizophrenia pathologies. This chapter, then, will examine the nexus between classical conditioning and schizophrenia. Although, readers of this volume need no introduction to classical conditioning, it is still worth noting that classical conditioning is deﬁned operationally as the pairing of an initially “neutral” conditioned stimulus (CS) with an unconditioned stimulus (US) that elicits an unconditioned response (UR), such that after one or more CS-US pairings the CS comes to elicit a response (CR) that was not present on ﬁrst presentation. Traditionally, the target CR was identiﬁed as a replica of the UR, as in Pavlov’s (1927) classical experiments with dogs, where the US was the application of a mild acid solution to the mouth, the UR a gush of saliva, and the CR a smaller salivary spurt. However, in many current classical conditioning preparations, the UR is not directly observed, and therefore the

07-Schachtman-07.indd 153

153

CR may or may not be a reproduction of some aspect of the UR. Instead, the UR and CR gain their designated status by virtue of theoretical considerations, as in the conditioned suppression and taste aversion paradigms. The aforementioned distinction becomes important when one examines the manner in which classical conditioning has been applied to schizophrenia. On the one hand, it might be asked whether patients with schizophrenia have deﬁcits in their ability to acquire speciﬁc CRs, the answer to which requires the use of a traditional paradigm, such as eyeblink conditioning (for a recent review, see Lubow, 2009). On the other hand, more general deﬁcits as in the ability to acquire associations can be assessed by means of protocols that do not directly observe CRs and URs. The former approach makes few demands on the theoretical basis of classical conditioning, whereas the latter requires a number of assumptions. However, there is yet a third application, one that uses the relatively simple empirical laws of classical conditioning as the basis for illuminating a process that underlies a particular behavior that appears, for example, to be abnormal in patients with schizophrenia. The present chapter, focusing on the third approach, will review the evidence that schizophrenia is characterized by an impaired ability to process irrelevant information. For patients with positive symptoms this is manifest in supersensitivity to irrelevant stimuli, which, in turn, can enhance the processing of those stimuli when they become relevant. For patients with negative symptoms, the sensitivity to irrelevant stimuli may be subnormal, a condition that would impede processing of those stimuli when they become relevant. Although the data are not completely consistent, it will be shown that these proposals are largely in accord with the data from latent inhibition (LI) experiments. To this end, the chapter will describe the rationale for studying aberrant LI effects in schizophrenia patients, following which the LI data from schizophrenia patients and from healthy individuals who have responded on selfreport schizotypal scales will be summarized. As will be seen, there is evidence that suggests that schizophrenia patients with positive symptoms

2/15/2011 7:43:48 PM

154

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

exhibit attenuated LI whereas those with negative symptoms show normal or potentiated LI. These effects will be accounted for in terms of differences in the ability to shift attention from controlled to automatic processing and in the manner in which such shifts are affected by the load of the masking task (the procedure used to divert attention from the to-be-conditioned preexpossed stimulus).

LATENT INHIBITION AND SCHIZOPHRENIA The Rationale for Exploring Latent Inhibition Effects in Schizophrenia

Our studies, initiated many years ago and independent of any interest in schizophrenia, began with the discovery that a stimulus that is repeatedly presented in such a manner that it is not attended (i.e., it is not followed by a consequence) subsequently becomes impaired in its ability to enter into or express new associations (Lubow & Moore, 1959). Since that time, this phenomenon, termed latent inhibition (LI), has been widely explored in animal and humans with a variety of learning and conditioning tasks (for reviews, see Lubow, 1989; Lubow & Gewirtz, 1995; Lubow & Weiner, 2010). The facts that LI appears to protect the organism from information overload by attenuating the processing of previously irrelevant stimuli, and that many schizophrenic patients are highly distractible, displaying an inability to focus attention on task-relevant information (e.g., Barch, Carter, Hachten, Usher, & Cohen, 1999; McGhie & Chapman, 1961; Ohman, Nordby, & d’Elia, 1986), led to our efforts to link LI and schizophrenia (for recent reviews, Lubow, 2005; Lubow & Weiner, 2010). We began by focusing on the distractibility symptom of schizophrenia, and we used conditioning methodology and some aspects of conditioning theory to explain the effects from that symptom on performance with the LI protocol. The rationale for exploring a LI–schizophrenia linkage received additional support from several factors. The ﬁrst includes several interrelated pharmacological effects: (1) amphetamine, an indirect dopamine agonist, which can induce

07-Schachtman-07.indd 154

positive symptoms in normal subjects (e.g., Zahn, Rappaport, & Thompson, 1981) and exacerbates such symptoms in schizophrenia patients (e.g., Sato, Numachi, & Hamamura, 1992), attenuates LI in rats (e.g., Weiner, Lubow, & Feldon, 1984, 1988); (2) on the other hand, dopamine-receptor antagonists, such as chlorpromazine and haloperidol, which are effective neuroleptics, reverse this attenuation and produce a super-LI effect in rats (e.g., Solomon et al., 1981; Weiner, Feldon, & Katz, 1987); (3) in healthy humans, low doses of amphetamine attenuate LI (e.g., Gray, Pickering, Hemsley, Dawling, & Gray, 1992; Thornton et al., 1996); (4) likewise, chlorpromazine produces a super-LI effect (e.g., McCarten et al., 2001), as do low doses of haloperidol (e.g., Williams et al., 1997); (5) the correspondence between dopamine agonists that produce psychotic-like effects in humans and reduce LI, and dopamine antagonists that counteract the effects of dopamine agonists and produce super-LI, also can be found with atypical antipsychotic drugs such as clozapine, olanzipine, remoxipride, and rispiridone (e.g., Moran, Fischer, Hitchcock, & Moser, 1996; Gosselin, Oberling, & Di Scala, 1996; Trimble, Bell, & King, 1997; Alves, Delucia, & Silva, 2002, respectively). These well-documented effects have been extensively reviewed (e.g., Moser, Hitchcock, Lister, & Moran, 2000; Weiner, 1990, 2003; Weiner & Arad, 2010). These ﬁndings, plus the facts that the same basic procedure could be used for animals and humans, and that attenuated LI, which is the predicted effect of schizophrenia (at least until recently, see section on “Latent Inhibition Effects in Patients With Schizophrenia”), is indexed by better learning by patients compared to controls, placed the LI paradigm at the forefront of research relating attentional dysfunction to schizophrenia. The Basic Latent Inhibition Procedure with Humans

The exploration of an LI–schizophrenia connection was also encouraged by a theory of LI, based on conditioning principles in animals, that emphasized the role of the conditioning of inattention (Lubow, Weiner, & Schnur, 1981), and which was later expanded to account for

2/15/2011 7:43:48 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

human LI (e.g., Lubow, 1989; Lubow, 2005). A description of an early human LI experiment (Ginton, Urca, & Lubow, 1975), which provided the protocol for many of the LI studies with schizophrenia patients (e.g., Gray, Hemsley, & Gray, 1992; for additional references, see section on “Latent Inhibition Effects in Patients with Schizophrenia”), illustrates the basic LI procedure. The Ginton et al. (1975) LI procedure, schematically illustrated in Table 7.1, consists of two stages, preexposure and test, and two groups of subjects, stimulus preexposed (PE) and not preexposed (NPE). In the preexposure stage, both groups are required to attend to a continuous stream of meaningless syllables (the masking task) and to count the number of times that the list, or a speciﬁc item in the list, repeats itself. The purpose of the masking task is to divert attention from the nominal target stimulus (to-be-CS, hereafter called CS-0), usually a white noise or tone, which is presented to the PE group but not to the NPE group. Thus, concurrently with the syllables, the PE group is exposed to a number of CS-0 trials, whereas the NPE group only hears the syllables. In the test stage, the subject’s task is to predict the onset of a reinforcer (change in a counter value) that is always preceded by the CS. With such a procedure, NPE groups learn the association more quickly than PE groups, an LI effect in humans that has been replicated many times (for reviews, Lubow, 2005; Lubow, 2010; also see next paragraph). Evidence That a Masking Task Is Required to Produce Latent Inhibition in Human Subjects

Importantly, with the exception of autonomic conditioning test procedures (e.g., Lipp & Vaitl,

155

1992), and others in which the response elicited in the preexposure stage is the same or related to the response in the test stage (e.g., Escobar, Arcediano, & Miller, 2003), LI in adult humans cannot be generated without the use of a masking task. Thus, the vast majority of experiments that have successfully elicited LI in adults have preexposed CS-0 while the subject was occupied with a masking task (e.g., Gray, Hemsley, & Gray, 1992; Gray, Pickering, et al., 1992; Lubow, Ingberg-Sachs, Zalstein-Orda, & Gewirtz, 1992; Pineno et al., 2006). The masking task requirement is strengthened even more by the results of studies that have explicitly compared masked and nonmasked conditions; LI has been obtained with the former but not with the latter (Braunstein-Bercovitz & Lubow, 1998; De la Casa & Lubow, 2001; Ginton, Urca, & Lubow, 1975; Graham & McLaren, 1998; Lubow, Caspy, & Schnur, 1982). The role of the masking task in LI can be accounted for by the assumption that it diverts attention or processing resources from the preexposed stimulus. This proposal is supported by studies that, among others, demonstrate that LI is a function of the cognitive load of the masking task (Braunstein-Bercovitz & Lubow, 1998; Braunstein-Bercovitz, Hen, & Lubow, 2004; for a detailed presentation of the evidence, see Lubow, 2005; Lubow, 2010). These ﬁndings also implicate the operation of attention-related processing of CS-0 during the preexposure stage 1 (see section on “The Masking Task Ensures Automatic Processing of CS-0”). As already suggested, the part played by attention in the acquisition of LI has direct implications for schizophrenia. Patients who are highly distractible, unlike healthy controls, will continue to allocate attention to the irrelevant CS-0 during the preexposure stage. As a consequence, a highly distractible schizophrenic PE group,

Table 7.1 Prototypical Experimental Design for a Two-Stage Latent Inhibition Experiment with Adult Humans Group

Preexposure Stage

Aquisition/Test Stage

PE

CS-0 and masking task in Context A

CS+ in Context A

NPE

0-0 and masking task in Context A

CS+ in Context A

07-Schachtman-07.indd 155

2/15/2011 7:43:48 PM

156

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

as compared to a healthy control PE group, will begin the test phase with a higher level of attention allocated to the previously irrelevant stimulus, with the result that they will learn the new association faster than the healthy control group. 2 Latent Inhibition Effects in Patients with Schizophrenia

In short, patients with schizophrenia (at least those who are abnormally distractable) should exhibit attenuated LI. However, the data are not entirely consistent. As described in Table 7.2, many reports do indicate that LI loss is associated with schizophrenia (Baruch, Hemsley, & Gray, 1988a; Gray, Hemsley, & Gray, 1992; Gray, Pilowsky, Gray, & Kerwin, 1995; Rascle et al., 2001; Vaitl et al., 2002). However, it should not come as a surprise that some studies have failed to ﬁnd this effect (Swerdlow, Braff, Hartston, Perry, & Geyer, 1996; Swerdlow et al., 2005; Williams et al., 1998; for reviews see Lubow, 2005; Lubow, 2010; Swerdlow, 2010). The heterogeneity of the patient population, due to symptom variation, drug interventions, hospitalization, diagnostic problems, and the difﬁculties in obtaining sufﬁciently large samples of patients, as well as variations in LI procedures, all may contribute to the lack of uniformity in the results. Perhaps even more important, there is evidence that LI disruption may take the form of either attenuation or persistence, a distinction that may point to underlying differences in the sources of the pathology (Weiner, 2003; Weiner, 2010; Weiner & Arad, 2010). Despite these concerns, Table 7.2 offers some tentative generalizations. It would appear that LI is reduced or abolished in recently medicated, acute, positive symptom (e.g., agitation, delusions, hallucinations, cognitive disorganization) schizophrenia patients as compared to longer term, medicated, chronic schizophrenia patients and to normals (Baruch, Hemsley, & Gray, 1988a; Gray, Hemsley, et al., 1992; Gray et al., 1995; Rascle et al., 2001; Sitskoorn, Salden, & Kahn, 2001; Vaitl et al., 2002; but see Swerdlow et al., 1996; Williams et al., 1998; also see Kumari & Ettinger, 2010). As opposed to this, chronic,

07-Schachtman-07.indd 156

medicated patients with schizophrenia, for whom negative symptoms (e.g., emotional apathy, absence of volition) are usually dominant, exhibit normal LI (Baruch et al., 1988a; Leumann, Feldon, Vollenweider, & Ludewig, 2002; Lubow, Weiner, Schlossberg, & Baruch, 1987; Serra, Jones, Toone, & Gray, 2001), or even potentiated or persistent LI, as reported for chronic medicated schizophrenia patients with high levels of negative symptoms, either by themselves (Gal et al., 2009; Rascle et al., 2001), or only when the negative symptoms are combined with low levels of positive symptoms (Cohen et al., 2004). Latent Inhibition Effects in Healthy Subjects Who Differ on Measures of Schizotypality

Since distractibility is associated with the positive symptoms of schizophrenia, the present theoretical position would proﬁt from being able to attribute the differences in LI effects across the various studies to the distinction between patients with positive symptoms and those with negative symptoms. Unfortunately, this distinction frequently covaries with chronicity and medication, which themselves are often correlated. Thus, chronic patients are most likely to be medicated, and acute patients, if tested early in their hospitalization, are frequently free of drug effects (it is generally agreed that about 2 weeks of administration is required for antipsychotic drugs to reduce symptom intensity). More important, while the acute phase of the pathology is often dominated by positive symptoms, chronic patients are more likely to display negative symptoms (for review, see, McGlashan, 1998). To overcome these difﬁculties, and other potential confounds that may accompany hospitalized psychotic patients, LI has been studied in healthy individuals who are differentiated on the basis of high and low scores on self-report questionnaires that assess schizotypality. These studies presume that psychotic tendencies lie on a continuum, with a normal population at one extreme, and a hospitalized patient group at the other extreme, a position that is supported by a variety of evidence (for summaries, see

2/15/2011 7:43:48 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

157

Table 7.2 Latent Inhibition Studies with Schizophrenic Patients as a Function of Dependent Variable, Diagnosis, Gender, Medication, and Magnitude of Latent Inhibition (LI) in the Schizophrenia (Sz) Groups Relative to Healthy Control Groups Experiment

Dependent Variable

Group

M/F

Medicated

Results

Baruch et al. (1988a)

Correct R

Control

24/29

—

LI

Acute Sz

18/8

Yes

Reduced LI

Chronic Sz

16/11

Yes

LI

RT visual search

Control

16/14

—

LI

Sz

24/6

Yes

LI a

RT b

Control

5/15

—

LI–Block 1, not 2

Sz

10/9

—

LI–Block 2, not 1 c

Control

10/10

—

LI

Acute Sz

12/4

No d

No LI (facilitation)

Chronic Sz

12/4

Yes

LI

Control

7/6

—

LI

Acute Sz

2/4

No

Reduced LI

Chronic Sz

3/4

No

LI

Control

13/5

—

LI

Sz

8/2

Yes-Typ.

LI

Sz

11/1

Yes-Atyp.

LI

Control

20/28

—

LI

Paranoid Sz

18/2

Yes

LI

Non-Para Sz

12/7

Yes

LI

Control

15/17

—

LI

Sz

17/15

Yes

LI

Control

14/26

—

LI

Acute Sz

23/12

Yes

No LI

Chronic Sz

21/9

Yes

Super LI

Control

13/14

—

LI

15/6

Yes

No LI

6/13

—

No LI

9/13

—

No LI

Cohen et al. (2004)

Gal et al. (2009)

Gray, Hemsley et al. (1992)

Gray et al. (1995)

Correct R

Correct R e

f

Leumann et al. (2002)

Lubow et al. (1987)

Lubow et al. (2000)

Rascle et al. (2001)

Serra et al. (2001)

Correct R

Correct R

RT visual search Correct R

Correct R

Chronic Sz NonSchizotypal Schizotypal g

g

(Continued )

07-Schachtman-07.indd 157

2/15/2011 7:43:48 PM

158

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Table 7.2 (Continued) Experiment

Dependent Variable

Group

M/F

Medicated

Results

Swerdlow et al. (1996, Exp. 1 and 2)

Correct R

1 Control

63 h

—

LI

Swerdlow et al. (2005)

Correct R

Vaitl et al. (2002)

GSR, RT e

Williams et al. (1998)

Yogev et al. (2004)

Correct R

R switch k

1 Acute Sz

24

h

No

1 Chronic Sz

40 h

Yes

2 Control

44 i

—

h

LI LI LI

2 Acute

18

i

i

No

2 Chronic

33 i

Yes

LI

Control

21/39

—

LI

Sz

16/4

Yes

LI

Control

8/8

—

LI

Sz

8/8

No

No LI j

Sz

8/8

Yes

LI j

Control

39/34

—

LI

Sz

14/9

No

LI

Sz

28/6

Yes

Reduced LI

Control

6/18

—

LI

Neg Sz

8/5

No

LI

Pos Sz

15/13

No

Reduced LI

LI

a

Overall LI effect was due to super-LI in patients with a combination of low-positive and high-negative symptoms. The other three symptom combination groups did not have signiﬁcant LI. b “Hits, commissions, and total responses” also were analyzed. However, since the test task required that the subject respond as quickly as possible to the appearance of the target, which was cued by the PE and NPE stimuli, RTs are the only meaningful data. c On the basis of the comparisons with the control group, the chronic schizophrenic group displayed persistent LI. d Patients were tested within 14 days of start of neuroleptics medication; they were drug-free at least 6 months prior to that. e PE/NPE, within-subject design. f Deﬁned as illness duration of less than 12 months. g First-degree relatives divided into low and high schizotypals. h As reported by Swerdlow et al. (1996) in their Figure 1 caption. i As reported by Swerdlow et al. (1996) in their Figure 3 caption. j GSR; conclusions compromised by absence of PE/NPE x Group interaction. k In the test, the preexposed tone signaled a change in the visual stimulus that would be correct. The dependent variable was the number of sets of same correct stimulus trials that were presented before the subject changed the “sorting” criterion.

e.g., Claridge & Broks 1984; Vollema & van der Bosch, 1995). A review of the schizotypal literature clearly indicates that high- as compared to low-schizotypal subjects exhibit reduced LI (e.g., Allan et al., 1995; Baruch, Hemsley, & Gray,

07-Schachtman-07.indd 158

1988b; Braunstein-Bercovitz & Lubow, 1998; Lubow & De la Casa, 2002). Furthermore, several studies have taken advantage of the fact that self-report schizotypal questionnaires, such as SPQ (Raine, 1991) and O-LIFE (Mason, Claridge, & Jackson, 1995), are composed of subscales

2/15/2011 7:43:48 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

that differentiate between positive and negative symptom types. Parallel to the trends with schizophrenic patients, LI attenuation tends to be associated with scale-scores that reﬂect positive, but not negative, symptoms (e.g., Burch, Hemsley, & Joseph, 2004; Evans, Gray, & Snowden, 2007; Gray, Fernandez, Williams, Ruddle, & Snowden, 2002; Schmidt-Hansen, Killcross, & Honey, 2009; Shrira & Tsakinakos, 2009). Relating the Direction of Abnormal Latent Inhibition to Symptoms of Schizophrenia

In summary, LI appears to be attenuated in acute, nonmedicated schizophrenia patients, comprised mostly of patients with positive symptoms, but not in chronic, medicated patients, who primarily have negative symptoms and normal or even potentiated LI. Healthy subjects who differ on the positive-negative dimensions of schizotypy exhibit a similar pattern of LI effects. Taken together, these provisional conclusions suggest an expansion of the initial proposal regarding the relationship between attenuated LI and the high distractibility of those with schizophrenia. The LI anomalies in schizophrenia patients and high-schizotypal normals, namely attenuated and persistent LI, may clarify some of the experiential aspects of the disease. For example, the patient’s inability to ignore irrelevant stimuli, which accounts for attenuated LI, may also be the basis for the perceptual chaos that characterizes positive symptomology. The inability to dampen irrelevant stimuli also can be a cause of further disorientation and confusion, which, in turn, would increase anxiety and exacerbate the original problem. Indeed, anxiety and stress, independently of psychopathology, attenuate LI in humans (Braunstein-Bercovitz, 2000; Braunstein-Bercovitz, Dimentman-Ashkenazi, & Lubow, 2001) and rats (e.g., Hellman, Crider, & Solomon, 1983; Shalev, Feldon, & Weiner, 1998; for reviews, Lubow, 2005; Pryce & Feldon, 2003). Within this framework, positive symptoms such as delusions and hallucinations can be viewed as being adaptive, imposing some degree of order on an anarchic array of stimuli, thereby reducing anxiety and suspending an otherwise devastating iterative process.

07-Schachtman-07.indd 159

159

Alternatively, the storm of irrelevant stimuli also can be blunted by negative symptoms, such as apathy and withdrawal. Thus, in general, schizophrenia, whether accompanied by positive or negative symptoms, can be regarded as a defense against a system breakdown that would result in conscious experience being inundated with phenomenally novel, meaningless stimuli. Frith (1979) described this collapse in similar terms, referring to the inability of schizophrenia patients to limit the contents of consciousness. Abnormal LI effects in schizophrenia patients appear to reﬂect the outcome of processes that are associated with containment of this state, with attenuated LI being associated with positive symptoms, and potentiated/persistent LI with negative symptoms.

ACCOUNTING FOR HUMAN LATENT INHIBITION The earlier description, while intuitively satisfying, remains speculative, not the least because there is little agreement as to how to explain normal LI. The next several paragraphs will outline an explanation of human LI that emphasizes the role of the masking task. Within that framework, the behavioral mechanisms that can account for pathologically attenuated and potentiated LI will be described. Some Comments on Theories of Latent Inhibition

In general, older theories of associative learning made use, either explicitly or implicitly, of attentional constructs to account for the reduced processing of the CS-0 that purportedly leads to attenuated associability (A-theories; e.g., Lubow, Weiner, & Schnur, 1981; Mackintosh, 1975; Pearce & Hall, 1980; Wagner, 1978). In each case, the impaired processing of the preexposed stimulus and its resulting decline in associability is an outcome from repeated CS-0 pairings that lead to a loss of CS salience. As a result, the attention allocated to a preexposed stimulus at the beginning of the test stage is less than that given to a novel stimulus, thereby accounting for the relatively slow acquisition of associative strength

2/15/2011 7:43:48 PM

160

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

by the preexposed stimulus. In addition, the inability to obtain LI in adult humans in the absence of a masking task during the preexposure stage is attributed to the maintenance of attention to CS-0 (Lubow & Gewirtz, 1995). Very simply, when the adult subject is not occupied with a task that diverts attention from CS-0, he or she will continuously look for some explanation as to why that stimulus keeps appearing (see next section). Although some theories of LI shift the burden for explaining the LI effect from attention-related processes in the preexposure stage to expression- or retrieval-related processes in the test stage (e.g., Bouton, 1993; Miller, Kasprow, & Schachtman, 1986; Weiner, 1990), more recent theories tend to adopt a compromise, acknowledging contributions from either or both stages (e.g., Hall & Rodriguez, 2010; Lubow, 2010; Schmajuk, 2010, Weiner, 2010). As will be seen, either of the two more recent positions can largely accommodate the LI-schizophrenia data. Despite the variety of theoretical mechanisms, both between and within the broader conceptual frameworks, the present approach will emphasize the role of the masking task, which by purportedly modulating attention to the preexposed stimulus provides the bridge between LI and schizophrenia symptoms. The Masking Task Ensures Automatic Processing of CS-0

The function of the masking task in producing LI in adult humans, and its apparent irrelevance for infrahumans, can be understood by considering the distinction between automatic and controlled processing (Schneider & Shiffrin, 1977; Shiffrin & Schneider, 1977). Automatic processes, common to animals and humans, are preattentive (not available to conscious awareness), relatively effortless and rapid, and operate in a parallel mode. Controlled processes, arguably limited to humans, are attention demanding, resource limited, and effortful, operating relatively slowly and serially. In addition, humans have a controlled processing bias, which when fully engaged overrides automatic processing. In an LI experiment with human subjects, controlled attentional processing is brieﬂy initiated

07-Schachtman-07.indd 160

when the preexposed stimulus is ﬁrst presented in stage 1. At that time, the preexposed stimulus is novel, and, as such, it elicits an orienting response that acts as a “call for processing resources” (Ohman, 1979). When the CS-0 is presented repeatedly while the subject is engaged in controlled processing of the masking task stimuli, the “call” for additional CS-0 processing goes unanswered, and CS-0 processing is switched to the automatic mode. Accordingly, the function of the masking task in generating LI in adults is to engage controlled attention, thereby diverting processing resources from the preexposed stimulus and allowing CS-0 to be processed in the default automatic mode. The same description can be applied to context stimuli, which, when novel, elicit exploratory behavior. Thus, normal LI is preceded by automatic processing of stimuli in the preexposure stage (CS-0 and context) leading to the encoding of their relatively primitive properties, 3 an effect that is self-evident from the fact that LI is stimulus and context speciﬁc (e.g., for early reviews, see Lubow, 1989, pp. 58–59, 74–80, respectively). It is proposed, then, that distraction represents a failure to shift from controlled to automatic processing under conditions of repeated presentations of stimuli that are not followed by consequences. For the “positive” schizophrenia patient, the irrelevant stimuli attract attention throughout the preexposure stage; they continue to be processed in the controlled mode, thereby overloading short-term memory/conscious awareness. As previously described, the positive symptoms of schizophrenia subjects are congruent with the notion that their conscious experience (shortterm or working memory) is “preoccupied” with irrelevant stimuli. In short, normal LI depends on the processing-shift from controlled to automatic. The disrupted LI in “positive” schizophrenia patients and high-schizotypal normals (as well as in normal adults without a masking task) is due to a failure to make that shift. The Role of Masking Task Load

Evidence to support this contention, in its general form, comes from a study by BraunsteinBercovitz and Lubow, (1998) with low- and high-schizoptypal subjects, that manipulated

2/15/2011 7:43:48 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

Median trials to criterion

300

NPE PE

250 200 150 100 50 0

Low schiz. High schiz. Masked

Low schiz. High schiz. Nonmasked

Figure 7.1 Median number of trials to reach the

learning criterion and interquartile range in the test phase for low- and high-schizotypal participants in the four preexposure conditions: nonpreexposed (NPE), preexposed (PE), masked, and nonmasked. In groups where Mdn = 256, only part of the range is presented. (Experiment 1; from Braunstein-Bercovitz & Lubow, 1998).

07-Schachtman-07.indd 161

300 Median trials to criterion

the load of the masking task. In Experiment 1 (see Figure 7.1), with zero load (i.e., no masking task), neither low nor high schizotypals exhibited LI, as would be expected since the absence of a masking task causes attention to be maintained to the preexposed stimulus. On the other hand, in Experiment 2 (see Figure 7.2), a lowload masking task (judgments of letter-pair similarity-differences) produced LI in low but not high schizotypals, as found in many other studies (see earlier). Experiment 2 also compared low- and high-schizotypal groups on two masking task conditions, low load, as in the ﬁrst experiment, and high load (letter orientations varied among the four cardinal positions). As in Experiment 1, with low load, low but not high schizotypals showed an LI effect. One might expect that the high-load masking task would deplete attentional resources, thereby precluding any processing of the preexposed irrelevant stimulus, with the consequence that the preexposed CS would be functionally novel in the test stage, and LI would not be obtained. However, once again, the results were affected by whether the participants were low or high schizotypals. Unlike with low load, where low schizotypals showed a larger LI effect than high schizotypals, with high load the opposite effect was achieved. High schizotypals demonstrated a pronounced

161 NPE PE

250 200 150 100 50 0

Low schiz. High schiz. Low load

Low schiz. High schiz. High load

Figure 7.2 Median number of trials to reach the

learning criterion and interquartile range in the test phase for low- and high-schizotypal participants in the four preexposure conditions: nonpreexposed (NPE), preexposed (PE), low load, and high load. In groups where Mdn= 256, only part of the range is presented. (Experiment 2; from Braunstein-Bercovitz & Lubow, 1998).

LI effect, whereas LI was abolished in low schizotypals (for related ﬁndings, see BraunsteinBercovitz, Hen, & Lubow, 2004; Della Casa, Höfer, Weiner, & Feldon, 1999; Höfer, Della Casa & Feldon, 1999). Attenuated and Potentiated Latent Inhibition and Their Relationship to Positive and Negative Symptoms

Although the analysis in the previous section does not take into account types of symptoms, positive-symptom subjects as suggested earlier show attenuated LI because they maintain attention to the preexposed stimulus, which, in turn, directly affects stimulus associability and/or the retrieval of the one or more associations acquired during the preexposure stage. On the other hand, for the subjects with negative symptoms, LI may be potentiated because the preexposed stimulus is relatively unattended from the very beginning of the preexposure stage, thereby giving these subjects more opportunities to acquire the CS-0 (and/or CS-context) associations. Relatedly, recent rat-LI data collected by Weiner and her associates indicate that disruption of LI, normally exhibited as an attenuation of LI, can also be manifest as an abnormal persistence of LI (for reviews of these data and a detailed description of the underlying brain

2/15/2011 7:43:48 PM

162

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

circuitry and pharmacology, see Weiner, 2010 Weiner & Arad, 2010). In brief, attenuated and potentiated LI can be mapped onto increased and decreased mesolimbic dopamine transmission, respectively. Furthermore, given that both dopamine hyperfunction and hypofunction may be present in schizophrenia (e.g., Moore, West & Grace, 1999; O’Donnell & Grace, 1998; Weiner & Joel, 2002), the LI abnormalities should be state dependent, being present or absent, depending on the underlying level of dopamine. In general, the animal LI data have been used to suggest that attenuated LI can be expected in the acute positive symptom stage of schizophrenia, which is associated with the hyperdopaminergic state, and that normal or potentiated LI should accompany the chronic state, in which patients with predominantly negative symptoms are characterized by increased attentional perseveration, which may have a basis in structural abnormalities in the brain, and lowered rather than increased dopamine levels (Weiner, 2003; Weiner, 2010) Weiner & Arad, 2010). As described in earlier paragraphs, these propositions are beginning to receive support from studies with schizophrenia patients and high-schizotypal normals.

SUMMARY AND CONCLUSIONS The preceding review of the literature indicates that LI research, although originally developed from interests in classical animal associationist theories, may have much to offer in increasing our understanding of dysfunctional attentional processes in schizophrenia. The data from LI studies with healthy humans point to the important role of attentional processes, particularly during the stimulus preexposure stage. Related studies with schizophrenia patients and normal subjects who are rated low and high on schizotypal symptoms suggest that LI may be attenuated or potentiated depending on whether the symptom type is positive or negative, respectively. In brief, it is proposed that the schizophrenia spectrum disorders can be differentiated into two mutually exclusive attentional deﬁcits: (1) a hyperdopaminergic state characterized by an attentional system that maintains controlled

07-Schachtman-07.indd 162

processing of irrelevant stimuli, leading to high distractibility, reduced LI, and positive symptoms; (2) a hypodopaminergic state distinguished by overly inert attention, ﬁxation on salient stimuli, and negative symptoms. Despite the fact that the symptoms that deﬁne schizophrenia are not readily apparent in animal behavior, the conﬂuence of new empirical ﬁndings and new theories of LI may provide important insights into the processes that underlie this devastating pathology.

NOTES 1. Although these studies indicate that LI is modulated by attentional processes operating in the preexposure stage, LI may also be affected by test stage retrieval factors, as suggested by LI experiments that have examined the effects of post-preexposure variables such as context change and retention interval. 2. One of the advantages of the LI protocol is that the attenuated LI that is associated with some schizophrenia groups is a result of that group learning better than the preexposed normal group, an effect that cannot be attributed to a generalized deﬁcit state that often accompanies psychopathology. 3. Humans can encode a number of primitive properties of unattended stimuli, including ﬁgure-ground segmentation (Kimchi & Peterson, 2008), grouping (e.g., Russell & Driver, 2005), surface completion (Moore, Grosjean, & Lleras, 2003), and direction of background motion (Watanabe, Nanez, & Sasaki, 2001). 4. Table 7.2 omits the two LI studies that compared event-related potentials in schizophrenia patients and healthy controls, because the results were not clear (Guterman, Josiassen, Bashore, Johnson, & Lubow, 1996; Kathmann, von Recum, Haag, & Engel, 2000).

REFERENCES Allan, L. M., Williams, J. H., Wellman, N. A., Tonin, J., Taylor, E., Feldon, J., & Rawlins, J. N. P. (1995). Effects of tobacco smoking, schizotypy and number of preexposures on latent inhibition in healthy subjects. Personality and Individual Differences, 19, 893–902. Alves, C. R. R., Delucia, R., & Silva, M. T. A. (2002). Effects of fencamfamine on latent inhibition.

2/15/2011 7:43:49 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

Progress in Neuro-psychopharmacology and Biological Psychiatry, 26, 1089–1093. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Barch, D. M., Carter, C. S., Hachten, P. C., Usher, M., & Cohen, J. D. (1999). The “beneﬁts” of distractibility: Mechanisms underlying increased Stroop effects in schizophrenia. Schizophrenia Bulletin, 25, 749–762. Baruch, I., Hemsley, D. R., & Gray, J. A. (1988a). Differential performance of acute and chronic schizophrenics in a latent inhibition task. Journal of Nervous and Mental Disease, 176, 698–600. Baruch, I., Hemsley, D. R., & Gray, J. A. (1988b). Latent inhibition and “psychotic proneness” in normal subjects. Personality and Individual Differences, 9, 777–783. Bhugra, D. (2006). The global prevalence of schizophrenia. PLoS Medicine, 2, 372–373. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Braunstein-Bercovitz, H. (2000). Is the attentional dysfunction in schizotypy related to anxiety? Schizophrenia Research, 46, 255–267. Braunstein-Bercovitz, H., Dimentman-Ashkenazi, I., & Lubow, R. E. (2001). Stress affects the selection of relevant from irrelevant stimuli. Emotion, 1, 182–192. Braunstein-Bercovitz, H., Hen, I., & Lubow, R. E. (2004). Masking task load modulates latent inhibition: Support for a distraction model of irrelevant information processing by high schizotypal and anxious participants. Cognition and Emotion, 18, 1135–1144. Braunstein-Bercovitz, H., & Lubow, R. E. (1998). Are high schizotypal normal participants distractible or limited in attentional resources? A study of LI as a function of masking task load and schizotypy. Journal Abnormal Psychology, 107, 659–670. Burch, G. St. J, Hemsley, D. R., & Joseph, M. H. (2004). Trials-to-criterion latent inhibition in humans as a function of stimulus pre-exposure and positive-schizotypy. British Journal of Psychology, 95, 179–196. Claridge, G. S., & Broks, P. (1984). Schizotypy and hemisphere function-I: Theoretical considerations and the measurement of schizotypy. Personality and Individual Differences, 5, 633–648.

07-Schachtman-07.indd 163

163

Cohen, E., Sereni, N., Kaplan, O., Weizman, A., Kikinzon, L., Weiner, I., & Lubow, R. E. (2004). The relation between latent inhibition and symptom-types in young schizophrenics. Behavioural Brain Research, 149, 113–122. De la Casa, G., & Lubow, R. E. (2001). Latent inhibition with a response time measure from a within-subject design: Effects of number of preexposures, masking task, context change and delay. Neuropsychology, 15, 244–253. Della Casa, D., Höfer, I., Weiner, I., & Feldon, J. (1999). Effects of smoking status and schizotypy on latent inhibition. Journal of Psychopharmacology, 13, 45–57. Escobar, M., Arcediano, F., & Miller, R. R. (2003). Latent inhibition in human adults without masking. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 1028–1040. Evans, L., Gray, N. S., & Snowden, R. J. (2007). A new continuous within-participant latent inhibition task: Examining associations with schizotypy dimensions, smoking and gender. Biological Psychology, 74, 365–373. Frith, C. D. (1979). Consciousness, information processing, and schizophrenia. British Journal of Psychiatry, 134, 225–235. Gal, G., Barnea Y., Biran, L., Mendlovic, S., Gedi, T., Halavy, M., Feldon, J., Fennig, S., & Levkovitz, Y. (2009). Enhancement of latent inhibition in patients with chronic schizophrenia. Behavioural Brain Research, 197, 1–8. Ginton, A., Urca, G., & Lubow, R. E. (1975). The effects of preexposure to a nonattended stimulus on subsequent learning: Latent inhibition in adults. Bulletin of the Psychonomic Society, 5, 5–8. Goldner, E. M., Hsu, L., Waraich, P., & Somers, J. M. (2002). Prevalence and incidence studies of schizophrenic disorders: A systematic review of the literature. Canadian Journal of Psychiatry, 47, 833–843. Gosselin, G., Oberling, P., & Di Scala, G. (1996). Antagonism of amphetamine-induced disruption of latent inhibition by the atypical antipsychotic olanzipine in rats. Behavioural Pharmacology, 7, 820–826. Graham, S., & McLaren, I. P. L. (1998). Retardation in human discrimination learning as a consequence of preexposure: Latent inhibition or negative priming? Quarterly Journal of Experimental Psychology, 51B, 155–172. Gray, N. S., Fernandez, M., Williams, J., Ruddle, R. A., & Snowden, R. J. (2002). Which schizotypal

2/15/2011 7:43:49 PM

164

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

dimensions abolish latent inhibition. British Journal of Clinical Psychology, 41, 271–284. Gray, N. S., Hemsley, D. R., & Gray, J. A. (1992). Abolition of latent inhibition in acute, but not chronic, schizophrenics. Neurology, Psychiatry, and Brain Research, 1, 83–89. Gray, N. S., Pickering, A. D., Hemsley, D. R., Dawling, S., & Gray, J. A. (1992). Abolition of latent inhibition by a single 5 mg dose of d-amphetamine in man. Psychopharmacology, 107, 425–430. Gray, N. S., Pilowsky, L. S., Gray, J. A., & Kerwin, R. W. (1995). Latent inhibition in drug naive schizophrenics: Relationship to duration of illness and dopamine D2 binding using SPET. Schizophrenia Research, 17, 95–107. Guterman, Y., Josiassen, R., Bashore, R. E., Johnson, M., & Lubow, R. E. (1996). Latent inhibition effects reﬂected in event-related brain potentials in healthy controls and schizophrenics. Schizophrenia Research, 20, 315–326. Hall, G., & Rodriguez, G. (2010). Associative and nonassociative processes in latent inhibition: An elaboration of the Pearce-Hall model. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 114–136). New York, NY: Cambridge University Press. Hellman, P. A., Crider, A., & Solomon, P. R. (1983). Interaction of tail-pressure stress and d-amphetamine in disruption of the rat’s ability to ignore an irrelevant stimulus. Behavioral Neuroscience, 97, 1017–1021. Höfer, I., Della Casa, V., & Feldon, J. (1999). The interaction between schizotypy and latent inhibition: Modulation by experimental parameters. Personality and Individual Differences, 26, 1075–1088. Kathmann, N., von Recum, S., Haag, C., & Engel, R. R. (2000). Electrophysiological evidence for reduced latent inhibition in schizophrenic patients. Schizophrenia Research, 45, 103–114. Kimchi, R., & Peterson, M. A. (2008). Figureground segmentation can occur without attention. Psychological Science, 19, 660–668. Kumari, V., & Ettinger, U. (2010). Latent inhibition in schizophrenia and schizotypy: A review of the empirical literature. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 419–447). New York, NY: Cambridge University Press.

07-Schachtman-07.indd 164

Leumann, J., Feldon, J., Vollenweider, F., & Ludewig, K. (2002). Effects of typical and atypical antipsychotics on prepulse inhibition and latent inhibition in chronic schizophrenia. Biological Psychiatry, 52, 729–739. Lipp, O. V., & Vaitl, D. (1992). Latent inhibition in human Pavlovian differential conditioning: Effect of additional stimulation after preexposure and relation to schizotypal traits. Personality and Individual Differences, 13, 1003–1012. Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. Cambridge, England: Cambridge University Press. Lubow, R. E. (2005). Construct validity of the animal latent inhibition model of selective attention deﬁcits in schizophrenia. Schizophrenia Bulletin, 31, 1–15. Lubow, R. E. (2009). Classical eyeblink conditioning and schizophrenia: A short review. Behavioural Brain Research, 202, 1–4. Lubow, R. E. (2010). Latent inhibition in schizophrenia: The ins and outs of context. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 500–530). New York, NY: Cambridge University Press. Lubow, R. E., Caspy, T., & Schnur, P. (1982). Latent inhibition and learned helplessness in children: Similarities and differences. Journal of Experimental Child Psychology, 34, 231–256. Lubow, R. E., & De la Casa, G. (2002). Latent inhibition as a function of schizotypality and gender: Implications for schizophrenia. Biological Psychology, 59, 69–86. Lubow, R. E., & Gewirtz, J. (1995). Latent inhibition in humans: Data, theory, and implications for schizophrenia. Psychological Bulletin, 117, 87–103. Lubow, R. E., Ingberg-Sachs, Y., Zalstein-Orda, N., & Gewirtz, J. C. (1992). Disruption of latent inhibition in high psychotic-prone normal subjects. Personality and Individual Differences, 13, 563–572. Lubow, R. E., Kaplan, O., Abramovich, P., Rudnick, A., & Laor, N. (2000). Visual search in schizophrenia: Latent inhibition and novel popout effects. Schizophrenia Research, 45, 145–156. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition: The effect of non-reinforced pre-exposure to conditional stimulus. Journal of Comparative and Physiological Psychology, 52, 416–419. Lubow, R. E., & Weiner, I. (Eds.) (2010). Latent inhibition: Data, theories, and applications to

2/15/2011 7:43:49 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

schizophrenia. New York, NY: Cambridge University Press. Lubow, R. E., Weiner, I., Schlossberg, A., & Baruch, I. (1987). Latent inhibition and schizophrenia. Bulletin of the Psychonomic Society, 25, 464–467. Lubow, R. E., Weiner, I., & Schnur, P. (1981). Conditioned attention theory. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 15, pp. 1–49). New York, NY: Academic Press. Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298. Mason, O., Claridge, G., & Jackson, M. (1995). New scales for the assessment of schizotypy. Personality and Individual Differences, 18, 7–13. McCartan, D., Bell, R., Green, J. F., Campbell, C., Trimble, K., Pickering, A., & King, D. J. (2001). The differential effects of chlorpromazine and haloperidol on latent inhibition in healthy volunteers. Journal of Psychopharmacology, 15, 96–104. McGhie, A., & Chapman, J. (1961). Disorders of attention and perception in early schizophrenia. British Journal of Medical Psychology, 34, 103–116. McGlashan, T. H. (1998). The proﬁles of clinical deterioration in schizophrenia. Journal of Psychiatric Research, 32, 133–141. Miller, R. R., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Sources and consequences. American Journal of Psychology, 99, 145–218. Moore, C. M., Grosjean, M., & Lleras, A. E. (2003). Using inattentional blindness as an operational deﬁnition of unattended: The case of surface completion. Visual Cognition, 10, 299–318. Moore, H., West, A. R., & Grace, A. A. (1999). The regulation of forebrain dopamine transmission: Relevance to the pathophysiology and psychopathology of schizophrenia. Biological Psychiatry, 46, 40–55. Moran, P. M., Fischer, T. R., Hitchcock, J. M., & Moser, P. C. (1996). Effects of clozapine on latent inhibition in the rat. Behavioural Pharmacology, 7, 42–48. Moser, P. C., Hitchcock, J. M., Lister, S., & Moran, P. M. (2000). The pharmacology of latent inhibition as an animal model of schizophrenia. Brain Research Reviews, 33, 275–307. O’Donnell, P., & Grace, A. A. (1998). Dysfuncrtions in multiple interrelated systems as the

07-Schachtman-07.indd 165

165

neurobiological bases of schizophrenic symptom clusters. Schizophrenia Bulletin, 24, 267–293. Ohman, A. (1979). The orienting response, attention, and learning: An information-processing perspective. In E. D. Kimmel, E. H. van Olst, & J. F. Orlebeke (Eds.), The orienting reﬂex in humans (pp. 443–471). Hillsdale, NJ: Erlbaum. Ohman, A., Nordby, H., & d’Elia, G. (1986). Orienting and schizophrenia: Stimulus signiﬁcance, attention, and distraction in a signaled reaction time task. Journal of Abnormal Psychology, 95, 326–334. Pavlov, I. P. (1927). Conditioned reﬂexes. London, England: Oxford University Press. Pearce, J. M., & Hall, G. (1980). A model of Pavlovian conditioning: Variations in the effectiveness of conditioned but not unconditioned stimuli. Psychological Review, 87, 332–352. Pineno, O., De la Casa, L. G., Lubow, R. E., & Miller, R. R. (2006). Some determinants of latent inhibition in human predictive learning. Learning and Motivation, 37, 42–65. Pryce, C. R., & Feldon, J. (2003). Long-term neurobehavioural impact of the postnatal environment in rats: Manipulations, effects and mediating mechanisms. Neuroscience and Biobehavioral Reviews, 27, 57–71. Raine, A. (1991). The SPQ: A scale for the assessment of schizotypal personality based on DSM-III-R criteria. Schizophrenia Bulletin, 17, 555–564. Rascle, C., Mazas, O., Vaiva, G., Tournant, M., Raybois, O., Goudemand, N., & Thomas, P. (2001). Clinical features of latent inhibition in schizophrenia. Schizophrenia Research, 51, 149–161. Regier, D. A., Narrow, W. E., Rae, D. S., Manderscheid, R. W., Locke, B. Z., & Goodwin, F. K. (1993). The de facto US mental and addictive disorders service system. Epidemiologic catchment area prospective 1-year prevalence rates of disorders and services. Archives of General Psychiatry, 50, 85–94. Russell, C., & Driver, J. (2005). New indirect measures of “inattentive” visual grouping in a change-detection task. Perception & sychophysics, 67, 606–623. Sato, M., Numachi, Y., & Hamamura, T. (1992). Relapse of paranoid psychotic state in methamphetamine model of schizophrenia. Schizophrenia Bulletin, 18, 115–122. Schmajuk, N. A. (2010). From latent inhibition to retrospective revaluation: An attentionalassociative model. In R. E. Lubow & I. Weiner

2/15/2011 7:43:49 PM

166

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 137–162). New York, NY: Cambridge University Press. Schmidt-Hansen, M., Killcross, A. S., & Honey, R. C. (2009). Latent inhibition, learned irrelevance and schizotypy: Assessing their relationship. Cognitive Neuropsychiatry, 14, 11–29. Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: Detection, search and attention. Psychological Review, 84, 1–66. Serra, A. M., Jones, S. H., Toone, B., & Gray, J. A. (2001). Impaired associative learning in chronic schizophrenics and their ﬁrst-degree relatives: A study of latent inhibition and the Kamin blocking effect. Schizophrenia Research, 48, 273–289. Shalev, U., Feldon, J., & Weiner, I. (1998). Latent inhibition is disrupted by acute and repeated administration of corticosterone. International Journal of Neuropsychopharmacology, 1, 103–113. Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic attentional processing II: Perceptual learning, automatic attending and a general theory. Psychological Review, 84, 127–190. Shrira, A., & Tsakanikos, E. (2009). Latent inhibition as a function of positive and negative schizotypal symptoms: Evidence for a bi-directional model. Personality and Individual Differences, 47, 434–438. Sitskoorn, M. M., Salden, M., & Kahn, R. S. (2001). Latent inhibition in ﬁrst-episode patients with schizophrenia. Schizophrenia Research, 49, 145–146 (abstract). Solomon, P. R., Crider, A., Winkelman, J. W., Turi, A., Kamer, R. M., Kaplan, L. J. (1981). Disrupted latent inhibition in the rat with chronic amphetamine or haloperidol-induced supersensitivity: Relationship to schizophrenic attention disorder. Biological Psychiatry, 16, 519–537. Swerdlow, N. R. (2010). A cautionary note about latent inhibition in schizophrenia: Are we ignoring relevant information. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 448–456). New York, NY: Cambridge University Press. Swerdlow, N. R., Braff, D. L., Hartston, H., Perry, W., & Geyer, M. A. (1996). Latent inhibition in schizophrenia. Schizophrenia Research, 20, 91–103.

07-Schachtman-07.indd 166

Swerdlow, N. R., Stephany, N., Wasserman, L. C., Talledo, J., Sharp, R., Minassian, A., & Auerbach, P. P. (2005). Intact visual latent inhibition in schizophrenia patients in a withinsubject paradigm. Schizophrenia Research, 72, 169–183. Thornton, J. P., Dawe, S., Lee, C., Capstick, C., Corr, P. J., Cotter, P., Frangou, S., Gray, N. S., Russell, M. A. H., & Gray, J. A. (1996). Effects of nicotine and amphetamine on latent inhibition in normal human subjects. Psychopharmacology, 127, 164–173. Trimble, K. M., Bell, R., & King, D. J. (1997). Enhancement of latent inhibition in the rat by the atypical antipsychotic agent remoxipride. Pharmacology, Biochemistry and Behavior, 56, 809–816. Vaitl, D., Lipp, O., Bauer, U., Schuler, G., Stark, R., Zimmermann, M., & Kirsch, P. (2002). Latent inhibition and schizophrenia: Pavlovian conditioning of autonomic responses. Schizophrenia Research, 55, 147–158. Vollema, M. G., & van den Bosch, R. J. (1995). The multidimensionality of schizotypy. Schizophrenia Bulletin, 21, 19–31. Wagner, A. R. (1978). Expectancies and the priming of STM. In S. H. Hulse, H. Fowler, & W. K. Honig (Eds.), Cognitive processes in animal behavior (pp. 177–209). Hillsdale, NJ: Lawrence Erlbaum. Watanabe, T., Nanez, J.E., & Sasaki, Y. (2001). Perceptual learning without perception. Nature, 413, 844–848. Weiner, I. (1990). The neural substrates of latent inhibition. Psychological Bulletin, 108, 442–461. Weiner, I. (2003). The “two-headed” latent inhibition model of schizophrenia: Modeling positive and negative symptoms and their treatment. Psychopharmacology, 169, 257–2297. Weiner, I. (2010). What the brain teaches us about latent inhibition (LI): The neural substrates of the expression and prevention of LI. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 372–418). New York, NY: Cambridge University Press. Weiner, I., & Arad, M. (2010). The pharmacology of latent inhibition: Modeling domains of pathology in schizophrenia and their treatment. In R. E. Lubow & I. Weiner (Eds.), Latent inhibition: Data, theories, and applications to schizophrenia (pp. 276–318). New York, NY: Cambridge University Press.

2/15/2011 7:43:49 PM

ABERRANT ATTENTIONAL PROCESSES IN SCHIZOPHRENIA

Weiner, I., Feldon, J., & Katz, Y. (1987). Facilitation of the expression but not the acquisition of latent inhibition by haloperidol in rats. Pharmacology, Biochemistry and Behavior, 26, 241–246. Weiner, I., & Joel, D. (2002). Dopamine in schizophrenia: Dysfunctional information processing in basal ganglia- thalamocortical split circuits. In G. Di Chiara (Ed.), Handbook of experimental pharmacology: Dopamine in the CNS (pp. 417–472). Berlin, Germany: Springer. Weiner, I., Lubow, R. E., & Feldon, J. (1984). Abolition of the expression but not the acquisition of latent inhibition by chronic amphetamine in rats. Psychopharmacology, 83, 194–199. Weiner, I., Lubow, R. E., & Feldon, J. (1988). Disruption of latent inhibition by acute administration of low doses of amphetamine. Pharmacology, Biochemistry, and Behavior, 30, 871–878.

07-Schachtman-07.indd 167

167

Williams, J. H., Wellman, N. A., Geaney, D. P., Cowen, P. J., Feldon, J., & Rawlins, J. N. P. (1998). Reduced latent inhibition in people with schizophrenia: An effect of psychosis or of its treatment. British Journal of Psychiatry, 172, 243–249. Williams, J. H., Wellman, N. A., Geaney, D. P., Feldon, J., Cowen, P. J., & Rawlins, J. N. P. (1997). Haloperidol enhances latent inhibition in visual tasks in healthy people. Psychopharmacology, 133, 262–268. Yogev, H., Sirota, P., Gutman, Y., & Hadar, U. (2004). Latent inhibition and over-switching in schizophrenia. Schizophrenia Bulletin, 30, 713–726. Zahn, T., Rappaport, J. L., & Thompson, C. L. (1981). Autonomic effects of dextroamphetamine in normal men: Implications for hyperactivity and schizophrenia. Psychiatry Research, 4, 39–47.

2/15/2011 7:43:49 PM

CHAPTER 8 Discrimination Learning Process in Autism Spectrum Disorders A Comparator Theory Phil Reed

The current chapter reviews evidence that suggests that there is a deﬁcit in discrimination learning in individuals with autism spectrum disorders (ASD), for both simple and conditional discriminations. These deﬁcits are linked to several of the core problems seen in many individuals with ASD, and they are discussed with reference to the concept of overselectivity. A new conceptualization of discrimination learning for those with ASD, based on a comparator theory, is postulated; this theory includes the notion that the comparator is sensitive to relative, rather than absolute, differences in stimulus strength, and that the comparator in individuals with ASD is hypersensitive to such differences. Several predictions derived from this model are shown to be substantiated, including postconditioning revaluation effects; enhanced overselectivity with subasymptotic learning; and greater sensitivity to slight differences between stimuli in ASD. The implications of this application of learning theory to interventions for overselectivity in an ASD population are discussed.

Autism spectrum disorders (ASD) encompass a wide-ranging collection of problems, which include the diagnoses of autistic disorder, Asperger’s syndrome, Rett’s disorder, childhood disintegrative disorder, and pervasive developmental disorder not otherwise speciﬁed (Diagnostic and Statistical Manual of Mental Disorders, fourth edition [DSM-IV]). Although each of these speciﬁc problems has a unique diagnosis, these disorders are generally characterized by pronounced deﬁcits in social-emotional reciprocity; that is, individuals with an ASD often have difﬁculty interpreting social situations and emotions in themselves and others. However, individuals with any of these speciﬁc problems can display a very wide spectrum of symptoms, including impairment in social interactions, communication difﬁculties, limited spontaneous pretend and imaginative play, and restricted,

repetitive, and stereotyped patterns of behaviors and interests (DSM-IV). In addition, it is not uncommon for there to be substantial levels of challenging behaviors, including large degrees of externalizing behaviors, which is recognized by the International Classiﬁcation of Diseases, 10th edition (ICD-10) classiﬁcation system. The challenge posed by ASD for psychology is two-fold: most immediately is the need to ﬁnd strategies to help manage the problems associated with the disorder and to promote levels of independent functioning in those with ASD; but also these strategies must be based on a ﬁrm theoretical base that allows an understanding of ASD and its diverse symptomatology. Both of these tasks are, to some extent, hindered by the wide-ranging and disparate nature of ASD, which makes single-solution suggestions to either of these challenges less than attractive.

168

08-Schachtman-08.indd 168

2/15/2011 7:44:06 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

However, learning theory offers a strong opportunity to supply some speciﬁc suggestions about individual elements of the autism spectrum, regarding both of these aforementioned questions. In part, this is because learning theory tends to offer abstract conceptualizations that can be applied to speciﬁc behaviors, rather than attempting to provide disorder-speciﬁc theories that apply to more generalized behaviors as does cognitive psychology; and in part this is because much work on both the applied, and basic, aspects of ASD has already been undertaken within this learning-theoretic framework. This chapter will examine the application of learning theory to one particular aspect of ASD; that is, the potentially critical issue of discrimination learning. It is well established that the ability to learn discriminations is a key component in many interventions for children with ASD (see Dube, 2009, for a review). It has also long been known that children with ASD display great difﬁculty in mastering these discriminations (e.g., Lovaas & Schriebman, 1971). Experimentally, this is often studied through the investigation of stimulus overselectivity (see Koegel & Wilhelm, 1973; Lovaas, Schreibman, Koegel, & Rehm, 1971; Reed & Gibson, 2005), which occurs when one aspect of the environment comes to control behavior at the expense of other equally salient events within the environment (e.g., Lovaas & Schreibman, 1971). Due to the limited range of components that come to control behavior, overselectivity often hampers learning. For example, when teaching an individual to discriminate between a knife and a fork, elements such as shape, color, and texture are all taken into consideration. If only the color element, and not the shape, of the object controls behavior (i.e., if only one element of the several elements presented has been acquired in the learning process), a distinction between a knife and fork cannot be made. A number of suggestions regarding why individuals with ASD, and other learning problems, should have this difﬁculty in mastering discriminations have been made, including the postulation of deﬁcits in the attentional (e.g., Dube et al., 1999; Lovaas et al., 1971), encoding (e.g.,

08-Schachtman-08.indd 169

169

Boucher & Warrington, 1976; Reed & Gibson, 2005), and retrieval (e.g., Leader, Loughnane, McMoreland, & Reed, 2009) stages of learning. Similarly, based on these suggestions, a number of intervention techniques to support such discrimination learning have been developed, including prompting techniques, where a response to the correct alternative may be guided by physical or verbal means (e.g., Repp, Karsh, & Lenz, 1990; Schreibman, Charlop, & Koegel, 1982); observing response procedures, where the individual is required to identify all aspects of the discrimination contingency prior to making a response (e.g., Constantine & Sidman, 1975; Dube & McIlvane, 1999); and extinction procedures, where aspects of the environment that exert control over behavior at the expense of other equally important aspects are extinguished (Reed, Broomﬁeld, McHugh, McCausland, & Leader, 2009). The current chapter brieﬂy reviews some of the evidence on discrimination learning in individuals with ASD (to highlight the nature of the problem) and then attempts to ﬁt these ﬁndings, and some new developments in interventions, into a relatively novel conceptual learning theory framework based on a comparator hypothesis. The version of comparator theory to be presented in this chapter does not correspond precisely to any previous version of this theory, although it is based, to some extent, on those outlined by Miller and Schachtman (1985) and by Vinogradova (2001).

COGNITIVE APPROACHES Of course, it should be acknowledged that a signiﬁcant amount of research effort, especially over the last three decades, has been devoted to establishing whether a single core cognitive deﬁcit may underlie the symptoms presented by individuals diagnosed with ASD. A brief overview of this cognitive literature serves two purposes: It highlights some of the behaviors that need to be accommodated with respect to ASD; and it highlights the relative lack of progress in producing actionable, generalized cognitive theories about ASD. To this end, a number of views that can broadly be classed as “cognitive” will be examined in turn.

2/15/2011 7:44:06 PM

170

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Theory of Mind

This is a very broad set of theories, of slightly differing types (see Heyes, 1998), but they all derive originally from the work of Premack and Woodruff (1978) with chimpanzees. For example, Baron-Cohen, Leslie, and Frith (1985) postulated that individuals with ASD display severe deﬁcits in theory of mind; that is, they have an inability to understand the possible thoughts of others. Classically, this ability is demonstrated by using experimental procedures like the “SallyAnne” task (Wimmer & Perner, 1983; although there are other similar tasks as well). In such tasks, a child may be shown a doll (“Sally”) placing an object under a bowl. This doll is then withdrawn, and the child sees a second doll (“Anne”) removing the object from the bowl and placing it under a box. The observing child is then asked where the departed puppet (“Sally”) would look for the object. The correct answer involves stating: “under the bowl,” or pointing at the bowl (as this is where the doll would believe the object to be, but not where the observing child knows the object to be). Findings have indicated that children diagnosed with ASD show poor performance on the task, often giving the location that they know to be true, and not the response based on the doll’s “belief ”; however, typically developing children, and children with Down syndrome, can easily answer questions from Sally’s perspective (e.g., Baron-Cohen et al., 1985). However, it is important to note that, while theory of mind has gained wide currency in discussions about ASD, it is not the only view of the central deﬁcits involved in ASD. Moreover, while a view of ASD based on theory of mind deﬁcits can accommodate the profound social and emotional problems experienced in this population, it struggles to accommodate some of the other aspects of the disorder noted earlier, such as the typically noted restricted range of interests often manifest in high levels of stereotypical behaviors (see Kennedy, Meyer, Knowles, & Shukla, 2000; Lord, Cook, Leventhal, & Amaral, 2000). Executive Dysfunction

In an attempt to accommodate these problems, other theories with a strong cognitive component,

08-Schachtman-08.indd 170

such as the executive dysfunction account, have been proposed (e.g., Ozonoff, Pennington, & Rogers, 1991; Rumsey, 1985). Executive function is a term used for a wide variety of cognitive functions, including planning, impulse control, working memory, inhibition, mental ﬂexibility, and the monitoring of action (see Baddeley, 1991, for an overview). People with ASD have often been noted to suffer from problems with many aspects of executive function, in that they perform worse in such tasks than matched control groups (e.g., Hughes, Russell, & Robbins, 1994; Ozonoff et al., 1991). However, there are many clinical groups that also show executive dysfunction on a wide variety of these tasks, such as individuals with schizophrenia (Frith, 1992), and those with some forms of acquired brain injury, especially involving the frontal lobes (see Shallice, 1988). It is very important to note that these groups of individuals do not necessarily show the social deﬁcits noted in those with ASD, even though they have profound executive function disorders. This fact has suggested to some researchers that the executive dysfunctions noted in ASD may, themselves, result from another central deﬁcit. Furthermore, it is suggested that this underlying central deﬁcit may also be the source of the theory of mind deﬁcits. Thus, neither executive dysfunction, nor theory of mind, may be causally implicated in these disorders, but it may be the result of a further underlying problem (Baron-Cohen, 1997). Central Coherence Theory

A third cognitive view of ASD, which has received a great deal of attention in the literature, is termed “central coherence theory” (see Frith, 1989). There are both strong (e.g., Frith, 1989) and weak (e.g., Happé & Frith, 2006) versions of this theory, but whichever version of this theory is adhered to, both suggest that individuals with ASD display difﬁculties in utilizing context in controlling their behavior. For example, Happé (1995) reported that individuals with ASD may mispronounce homophones. A common example involves the word “tear,” such as in the case of: “There was a tear in her eye,” which can often be misread to sound as if it appeared in the

2/15/2011 7:44:06 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

sentence: “There was a tear in her dress” (see Happé, 1995). This error depends on the individual not using the context of the whole sentence to control the pronunciation. Conversely, individuals with ASD often show a performance advantage in tasks that depend on speciﬁc aspects of a situation, rather than the overall aspects of a situation, to control behavior. That is, such individuals tend to focus on the “local” or elemental aspects of a stimulus, rather than on the “global” or conﬁgural aspects of the stimulus. For example, individuals with ASD often show enhanced recognition of embedded ﬁgures (Shah & Frith, 1983; but not, it should be noted, under all circumstances, Ozonoff, Strayer, McMahon, & Filloux, 1994). A central coherence account that rests on the presumed tendency to focus on speciﬁc aspects of the situation, and/or an inability to use the whole situation, to control behavior has the potential to explain several aspects of both the impaired and enhanced performances seen in those with ASD. Moreover, it can be applied to understand some of the deﬁcits in social skills and language that are extremely common in ASD. These latter areas often involve integrating information across a number of sources (see Jordan & Thomas, 2001), or they rely on the ability of the context to facilitate understanding (McHugh & Reed, 2007). As will be seen later in this chapter, the central coherence accounts actually resemble several of the views that derive from learning theory, especially those that focus on “overselective” responding. However, there are two problematic issues for central coherence theories that should be mentioned. First, the mechanism that drives the focus on local or elemental aspects of a complex stimulus is not clear; it is not speciﬁed whether this is a deﬁcit in contextual control per se or an enhancement of elemental control. Secondly, it is also unclear how central coherence theories, as single explanations for ASD, could accommodate the wide range of behavioral problems often seen within ASD. As with the executive dysfunction hypothesis, it is the case that these deﬁcits are not unique to ASD (which is not necessarily a major issue), and it also appears to be the case that not all situations

08-Schachtman-08.indd 171

171

in which a central coherence deﬁcit might be expected to produce a performance deﬁcit actually produce problems for those with ASD (see Baron-Cohen, 1997). Summary

Despite the considerable research effort that has been devoted to discovering the central cognitive deﬁcit underlying ASD, it is clear that no current proposal adequately accommodates the behaviors that are associated with ASD. Moreover, it is becoming increasingly apparent that a single cognitive central deﬁcit will not be adequate to accommodate the wide range of problems exhibited by individuals with ASD (see Goodman, 1989; Happe, Ronald, & Plomin, 2006). The huge variations in behaviors shown by this population suggest that conceptualizing ASD as a unique, and internally coherent, disorder may be misplaced, and that a focus upon the speciﬁc behaviors that are often exhibited by those diagnosed with ASD, rather than upon the disorder as a whole, may be more proﬁtable (see Happe et al., 2006; Osborne & Reed, 2009). That is, ASD may be better conceptualized as a broad syndrome of behavioral problems, rather than a disorder caused by a single central deﬁcit (see Lord & McGee, 2001, for a discussion). In fact, such a conceptualization of a psychopathology would not be novel, and it is often applied in the case of schizophrenia (e.g., Bentall, 1990). Moreover, and perhaps more important, it is also clear that the vast amount of cognitive research into the core deﬁcits of ASD has brought only very minimal gains in terms of treatment of this debilitating and lifelong disorder (see Osborne & Reed, 2009).

LEARNING THEORY AND AUTISM SPECTRUM DISORDERS It is in both of these latter areas that an approach based on conditioning, in general, and behavior analysis, in particular, has produced some pronounced beneﬁts over cognitive approaches. It is certainly the case that behavioral psychology has developed some of the most effective and scientiﬁcally well-validated approaches to the management of ASD; most notably in terms of the

2/15/2011 7:44:06 PM

172

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

applied behavior analytic (ABA) approaches to its treatment (e.g., Howard, Sparkman, Cohen, Green, & Stanislaw, 2005; Lovaas, 1987; Reed, Osborne, & Corness, 2007). The ABA approaches, documented by Lovaas (1987), Greer and Keohane (2009), and Sundberg and Michael (2001), among others, have been found to produce clinically signiﬁcant gains, not only for speciﬁc behaviors but also in terms of overall functioning of individuals with ASD (see Eldevik et al., 2009; Makrygianni & Reed, 2010, for metaanalytic reviews). Individuals with ASD who are given ABA interventions often show improvements in their general intellectual functioning and in their adaptive behaviors, which, more often than not, surpass those gains produced by other forms of intervention (see Howard et al., 2005; Reed et al., 2007). These ABA approaches appear effective precisely because they focus on speciﬁc behaviors, rather than focusing upon a disorder as a whole (see discussion by Lord & McGee, 2001), and the role of such functional analysis is a topic that requires a great deal more investigation. These applications of learning theory to the treatment of ASD have been particularly well documented and reviewed in numerous sources (see Eldevik et al., 2009; Howard et al., 2005; Reed et al., 2007). However, learning theory has also offered important insights into the processes that control some of the speciﬁc behaviors that are typically seen in individuals with ASD, rather than attempting to accommodate the overarching conceptualizations of the disorder. It is these applications of learning theory to understanding the processes governing the speciﬁc behaviors of those with ASD that are the focus of the current chapter; in particular, the difﬁculties, and sometimes the advantages, seen in learning about which aspects of the environment control behavior by those with ASD. As noted earlier, the performance of those with ASD on discrimination learning tasks and situations is particularly important to understand. This ability relates to many of the core features of ASD, such as an overfocus on some aspects of the environment at the expense of other equally important aspects, and the inability to integrate information about complex cues. In fact, the application of learning

08-Schachtman-08.indd 172

theory to exploring the causes of such speciﬁc behavioral differences in ASD has a long and underexplored history, and this has been particularly pronounced in respect to those ASD behaviors that involve the discrimination and generalization of stimuli and events.

DISCRIMINATION LEARNING AND AUTISM SPECTRUM DISORDERS The notion of discrimination learning is a very simple one; it refers to the acquisition of differential control over behavior by various aspects of the environment. Of course, many conditioning experiments have focused on trying to understand the processes involved in producing discrimination between stimuli, and the broader aspect of perceptual learning (for reviews, see Hall, 1991; Pearce, 1987). Many of these studies reinforce one stimulus (A), while not reinforcing another stimulus (B), in a particular context (X). Thus, this procedure can be conceptualized as AX+ BX–. Despite the simplicity of the issue, there are large numbers of theories devoted to explaining this phenomenon. For example, the “conditioningextinction” theories derived from the work reported by Pavlov (1926), and then developed by Hull (1952) and Spence (1956). This view suggests that the stimuli associated with reinforcement (e.g., A) come to possess excitatory properties, and the stimuli associated with nonreward (e.g., B) acquire inhibitory properties, while those stimuli that are non–differentially associated with the outcomes (e.g., X) come to have no associative strength. Although clearly a simpliﬁcation, such a view also forms a basis for more contemporary accounts, such as those presented by Pearce (1994). Other views suggest that discrimination learning occurs as a result of changes in the associability of the relevant and irrelevant stimuli (for a review, see Mackintosh, 1983). The literature on this theoretical topic is vast and way beyond the scope of this chapter to review, nor is the precise outcome of this debate particularly relevant, at the current time, for understanding the nature of ASD (although it may become so, when this topic has been studied

2/15/2011 7:44:06 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

in more detail in the context of ASD). At present, the task for this chapter is to show how learning theory explanations can illuminate the understanding of the nature of this particular clinical problem. In fact, all of these theories of discrimination learning focus attention on two aspects of the problem: ﬁrst, how the cues that signal the occurrence or nonoccurrence of an outcome (e.g., A and B) come to acquire excitatory or inhibitory strength over the control of behavior (stimulus discrimination); and secondly, they focus on what happens to the stimuli that may be associated with target cues (e.g., X; which form part of the reinforced or nonreinforced compound) but that may not be contingently related to the outcome (stimulus differentiation). The terms “discrimination” and “stimulus differentiation” (or “overselectivity”) will be used throughout this chapter to refer to these two phenomena, respectively. Although these two terms are not entirely coextensive with the literature concerning learning to discriminate between different stimuli, and learning to discriminate between the elements within a compound stimulus (often referred to as perceptual learning), mapping the current discrimination/ differentiation distinction onto the between/ within-stimulus learning literatures may open possibilities for reconceptualizing the nature of this problem in ASD. Hence, while not wishing to make speciﬁc theoretical claims based on this mapping, drawing such a parallel between literatures may offer beneﬁcial heuristics in understanding these aspects of learning for ASD, and it may suggest potentially new areas of exploration into the nature of ASD. Importance of Discrimination Learning for Autism Spectrum Disorders and Learning Disabilities

The importance of discrimination learning for those with ASD (and learning disabilities, more generally) can be evidenced by the more clinically oriented literature. The problems experienced by those with ASD in terms of discrimination learning have been recently reviewed by Dube (2009), who gives a thorough account of this issue. In general, those with ASD

08-Schachtman-08.indd 173

173

experience similar problems in discrimination to a wide range of individuals with other developmental and intellectual disabilities; that is, they ﬁnd learning discriminations harder than mental-age-matched controls. This is not a novel suggestion, and there were a series of seminal studies of this issue concerned with those with learning disabilities provided nearly 50 years ago by Zeaman and House (1963). These researchers documented that discrimination learning was impaired in those with learning disabilities, not in terms of the asymptotic levels of discrimination reached, but rather in terms of the time taken to acquire this asymptotic level of discrimination. In particular, in these studies, and the mathematical models derived from them, it took much longer for individuals with learning disabilities to attend to the relevant cues, and for these cues to come to control behavior, than in those without such a disability. That is, Zeaman and House (1963) suggested that there was little discriminated responding for longer periods of time in those with learning disabilities, but when the discrimination started to occur, it occurred at the same rate as for those without the disability. Although more recent evidence has shown that there may also be a contribution of learning rate (see Okada, 1978; Reed, 2006), the basic impairment in discrimination is well established. In fact, a key aspect of clinical practice, as it is related to those with ASD, and to those with learning disabilities, involves implementing procedures for promoting discrimination performance. The promotion of improved discrimination is taken to be essential to encourage more independent functioning in these individuals. The results obtained from the study of these procedures can often demonstrate that discrimination problems can result from stimulus differentiation problems. Some of these clinical training procedures were mentioned brieﬂy earlier, and they will be discussed in some detail later in this chapter. These procedures include various prompting techniques (e.g., Repp et al., 1990; Schreibman et al., 1982) and observing response procedures (e.g., Constantine & Sidman, 1975; Dube & McIlvane, 1999; Wyckoff, 1952). Focusing on

2/15/2011 7:44:06 PM

174

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

the former technique, there is a literature on the comparison of various methods for prompting correct responses in a discrimination learning task for those with ASD, which has some particular relevance to the current chapter. Some prompting methods involving the addition of prompts that are external to the stimulus (extrastimulus), such as pointing to the stimulus, and some procedures involve within-stimulus prompts, such as making the stimulus more salient. For example, Schreibman (1975) compared the effects of these two types of stimulus prompts for promoting discrimination learning in those with ASD, and he has shown that, under some circumstances, the within-stimulus prompt appears to promote better discrimination than the extra-stimulus prompts. It has been suggested that such results show that those with ASD are particularly sensitive to interference from extra-stimulus prompts. However, other reports have shown that, sometimes, the internal features of the stimulus can mutually interfere with one another during the course of the discrimination. For example, Ray and Sidman (1970) demonstrated that previously established discrimination performance between two lines, one oriented to the right, and one oriented to the left, collapsed when the distance from the vertical became equal in the two cases. This result was taken to show that the distance from the vertical, and not the orientation of the lines, was controlling behavior. Although the precise nature of these experiments is not the key issue, they are reported because they show that a variety of aspects of the stimulus can control behavior during a discrimination learning task, and that these aspects of the stimulus can compete with one another. Thus, it is clear that discrimination learning is an important issue for those with ASD. A very selective review can highlight some of the important facets of the area, and it can also help illuminate the nature of discrimination learning in ASD. The two aspects of this broad aspect of conditioning highlighted earlier in the context of understanding ASD are, ﬁrst, the relatively retarded speed of forming both simple and conditional discriminations between stimuli exhibited by those with ASD; and secondly, the

08-Schachtman-08.indd 174

paradoxically enhanced differentiation between the elements within a particular stimulus (overselectivity) that is often shown by individuals with ASD. Several of the studies reported in the sections that follow have speciﬁcally attempted to explore which aspects of these discrimination learning processes appear altered in those with ASD (e.g., Leader et al., 2009; Lovaas et al., 1971; Reed & Gibson, 2005). Simple Discrimination Learning

The fact that individuals with ASD speciﬁcally take longer than controls to learn simple discriminations has, in fact, been explored only in relatively few experiments. Leader et al. (2009) presented children with ASD, and a control group of typically developing children, closely matched in terms of their chronological age to the mental age of the group with ASD, with a simple discrimination task: AB+ versus CD– (see top panel of Fig. 8.1 for a representation of the stimuli involved). Participants were simply required to point to the reinforced stimulus (AB) 10 times in a row. All participants successfully learned this simple discrimination, but the participants with ASD took a mean of 28.7 trials to reach criterion, whereas the control group only took a mean of 14.1 trials, a difference that was statistically signiﬁcant. Further evidence of a speciﬁc problem with discrimination learning emerged from a series of experiments conducted in the Swansea laboratory by Reed, Stott, and Staytom (unpublished data). These studies exposed children with ASD, Down syndrome, and mental-age-matched typically developing controls to a sequence of learning tasks, suggested by McDaniel (2001; see Table 8.1), in order to explore the capacities of children with learning disabilities. This sequence of tasks was based on Thomas’ (1996) learning hierarchy, which suggests that some learning tasks are easier or simpler to acquire than others. Thomas (1996) and McDaniel (2001) noted that the degree of intellectual impairment in a range of developmental, and learning, disabilities predicted the ability of individuals to complete the tasks speciﬁed in this series. The greater the degree of impairment, the fewer tasks in the sequence the individual could achieve.

2/15/2011 7:44:06 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

175

Figure 8.1 Examples of the stimuli used in the two-card discrimination procedure. Top panel shows the training with two, two-element cards, and the bottom panel shows an example test trial.

Reed et al. (unpublished data) found that whereas Down syndrome children started to show deﬁcits at the level of basic instrumental learning (i.e., stage 3 in the model), relative to children with ASD and typically developing controls, children with ASD only showed a deﬁcit at the stage of concurrent discrimination learning

Table 8.1 A Suggested Adaptation of Thomas’ Synthesis (1996) for Use with Cognitively Impaired Human Populations 1. Habituation-sensitization 2. Signal learning (classical conditioning) 2A. Observational learning 3. Stimulus-response learning (instrumental conditioning) 4. Chaining 5. Concurrent discrimination learning

(stage 5 in the model). The number of trials to criterion for mastering that discrimination stage of the sequence (a simple S+ S– task), for all three groups, is shown in Figure 8.2. Although not using a population with ASD, impaired discrimination learning has been shown in a group with social phobia, a quality often displayed by those with ASD (Sachs, Anderer, Doby, Saletu, & Dantendorfer, 2003). In this study, an eye-blink procedure was used for social-phobic adults and matched controls who were exposed to a simple S+, S– discrimination procedure. Both groups showed a signiﬁcant difference between responding to the S+ and S–, but, for the social phobics, the level of conditioned responding to the S+ did not increase during the course of the training, whereas the control group did show increasing conditioned discrimination; this ﬁnding was exactly in line with the predictions of Yarmolenko (1926).

5A. Simple strategic rule (reversal) learning

Conditional Discrimination Learning

5B. Learning set formation

In addition to a retardation of simple discrimination learning, Rodgers (2000) suggested that hierarchical relational learning (conditional discriminations) also may be attenuated in people with ASD. Relational control can be studied using a match-to-sample (MTS) procedure, as relational control is required for performance

6. Class concepts absolute and relative 7. Relational concepts I: conjunctive, disjunctive conditional concept 8. Relational concepts II: biconditional concepts

08-Schachtman-08.indd 175

2/15/2011 7:44:06 PM

20 19 18 17 16 15 14 13 12 11 10

comparison at 0s, 15s, and 60s delays between the presentation of the sample and comparisons. The ASD group performed with about 67% accuracy, as opposed to around 90% in the control children at 0s delay, and the accuracy performance of the two groups diverged as the samplecomparison interval lengthened (see Fig. 8.3), indicating both impaired conditional discrimination ability and overselectivity in the ASD condition. Given the earlier discussion about the reasons for poor performance by those with ASD on simple discrimination tasks, the role of the sample/S– relations should be mentioned in the context of MTS, because these relations appear critical to account for the discrimination deﬁcits within an MTS procedure (Dube, 2009). Accurate MTS performance includes selecting correct comparisons and rejecting incorrect ones; overselectivity impairs MTS accuracy due to failures to reject incorrect comparison stimuli (see also Reed & Gibson, 2005). According to Dube (2009), this may happen because the common stimulus features controlling behavior are shared by both correct and incorrect comparisons, and individuals with ASD often have difﬁculty with this discrimination (see Plaisted, O’Riordan, & Baron-Cohen, 1998; Reed & Gibson, 2005). Another set of issues, concerned with the contextual control of behavior, have also been

ASD

Control

Downs

Figure 8.2 Mean trials to criterion in three groups

of participants for a simple discrimination learning task reported by Reed, Stott, and Staytom (unpublished data). ASD, autism spectrum disorder children; Control, mental-age-matched typically developing children; Downs, Down syndrome children.

on this task in that it involves relations between the sample and comparison stimuli (see Dube, 2009), such as sample/S+ and sample/S– (Dube & McIlvane, 1996). Typically developing individuals often have high accuracy scores on MTS tasks, but those with ASD often perform poorly (e.g., Dickson, Deutsch, Wang, & Dube, 2006). In a typical MTS task, participants may be presented with one of three, three-element compound stimuli (e.g., ABC, DEF, GHI) and are then shown (as the comparison stimulus) one of the sample stimuli along with two of the other stimuli (e.g., if ABC was shown as a sample, either “A,” “B,” or “C” would then be shown with two of the other unpresented symbols, such as DE or GF). According to Dube (2009), scores of around 67% on this task indicate overselectivity, because stimulus control by only one sample (i.e., overselectivity) would mean that on half the trials that stimulus would appear as a comparison, and the participant would always be correct. On the other trials, that stimulus would not appear, and correct responding by chance would be 33%. Combining these types of trial would produce an accuracy score of about 67%. Broomﬁeld et al. (unpublished data) presented a match-to-sample task to children with ASD, and to a mental-age-matched group of typically functioning children. Each group received exposure to a three-element sample and

08-Schachtman-08.indd 176

Percentage accuarcy

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Trials to criterion

176

100 90 80 70 60 50 40 30 20 10 0

ASD Control 0s

15s

60s

Retention interval

Figure 8.3 Overall accuracy in a three-element

match to sample procedure for children with autism spectrum disorders (ASD) and mentalage-matched controls (control) for three samplecomparison delays in a study by Broomﬁeld, Simpson, McHugh, and Reed (unpublished data).

2/15/2011 7:44:06 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

177

suggested to relate to the development of language that is often impoverished in children with ASD (McHugh & Reed, 2008), and subsequently to the poor social behaviors that often derive from the ability to engage in particular forms of language (Conallen & Reed, unpublished data). It might be noted that, in order to understand longer sentences, a complex hierarchical discrimination may be required, in which hierarchical dependencies may serve to connect parts of a sentence. For example, such hierarchical dependencies help to give meaning to inserted subordinate clauses: “The song [that the boy sang] pleased the teacher.” If discrimination learning is retarded, then such contextual control may be lacking, impacting on the ability to formulate long sentence structures that rely on the ability to use hierarchical relationships to give meaning to the terms in those sentences.

(the local aspects), rather than for the overall form of the compound stimulus (e.g., O’Riordan & Plaisted, 2001). These types of effect were discussed as central issues for the central coherence theory of ASD mentioned earlier (see Frith, 1989). Plaisted et al. (1998) suggested that such effects could be regarded as “enhanced discrimination,” in the sense of an increased ability to detect the presence of a part of a stimulus complex. Although such overselectivity is not unique to the ASD population (Reed & Gibson, 2005), and it has been seen in those with acquired neurological damage (see Wayland & Taplin, 1985) and in older people (McHugh & Reed, 2007), it is clearly an important aspect of discrimination learning in those with ASD. It suggests a reason why S+ versus S– discrimination may be poor, while some elements of the S+ appear to exert excessive control over behavior (Dube, 2009).

“Within-Stimulus” Learning

Overselective Responding in Autism Spectral Disorders

The lower levels of simple and conditional discriminations noted in these experiments for individuals with ASD are one aspect of poor discrimination for this population, but there is also another aspect to this area that requires comment. Although overall levels of discrimination between stimuli are poor, the relative control of behavior by one stimulus element of the S+ compared to another can be quite pronounced; that is, although performance to the S+ in those with ASD can be low, relative to that seen in control subjects, the degree to which some aspects of the S+ control behavior, compared to other aspects of the S+, can often be more pronounced in those with ASD. For example, although those with ASD may not learn an AB+ CD– discrimination quickly, they appear to be better able to discriminate between the components of the reinforced compound (A and B) better than those without ASD. Local and Global Processing

As noted earlier, individuals with ASD often show overselective responding to one element of the S+ (e.g., Broomﬁeld et al., under review; Dube, 2009), and they also show better detection of the individual elements of a stimulus

08-Schachtman-08.indd 177

Some discussion of this effect was given in the earlier section on MTS procedures. However, another example of this effect in those with ASD can be seen in the report by Leader et al. (2009). Following training on a simple AB+ CD– discrimination, as described earlier, the participants were presented with a choice between the elements of the reinforced compound and the elements from the nonreinforced compound, so either stimulus “A” or stimulus “B” from the previously reinforced compound (AB) was paired with either stimulus “C” or stimulus “D” from the previously nonreinforced (CD) compound (see bottom panel of Fig. 8.2 for a representation of such a test trial). There were ﬁve test trials for each combination of previously positively reinforced and punished components (i.e., “A vs. C,” “A vs. D,” “B vs. C,” “B vs. D”). No feedback was provided during test trials. The participants with ASD selected the elements from the previously reinforced compound, when presented in this test, approximately equally often as one another (100% for the most-selected stimulus at test, irrespective of whether this was “A” or “B,” and about 95% for the least-selected stimulus). However, although to choose one of these stimuli more often than the other appears

2/15/2011 7:44:07 PM

178

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

to have no basis in the associative values of these stimuli, most participants with ASD selected one of the elements stimulus (whether A or B) much more than the other at test (see Fig. 8.4). These ﬁndings suggest that individuals with ASD may show enhanced discrimination (differential responding) between the within-compound elements of the S+ (which may acquire different associative strengths to one another), even though they show retarded levels of discrimination between S+ and S– stimuli. This effect may help to explain the lack of generalization between training and test stimuli in many instances for those with ASD; if only one aspect of several potentially important aspects of a cue are acquired, then other similar cues sharing some, but not all of these elements, may control behavior in the expected manner. To continue from an earlier example, if a person is taught how to use a fork of a particular color and shape, but only the color controls behavior due to overselective learning about the within-stimulus element, then another fork of the same shape but a different color may not be used appropriately. Summary

This very selective and brief review of the discrimination learning literature involving those with ASD suggests a number of effects. In some cases, there is an inability to display good control

100 Percentage chosen

90

Most selected Least selected

80 70

of behavior by stimuli in the environment, especially in terms of simple, and conditional, discriminations. In contrast, there are some cases in which there is an enhanced discrimination between the elements within a stimulus, often resulting in a lack of generalization from one situation to another. The notion of overselectivity appears well ﬁtted to accommodate these results, but the issue remains regarding how to best explain overselectivity and to demonstrate how this knowledge can be put to use in an applied setting.

A COMPARATOR THEORY OF AUTISM SPECTRAL DISORDERS One perspective derived from learning theory, which may go some way to explain these differences in the manner in which individuals with ASD perform on discrimination tasks, is based on a version of a comparator hypothesis. There have been numerous elaborations of such models (e.g., Miller & Schachtman, 1985), and some of the basic suggestions made by such elaborations will be adopted and modiﬁed. In particular, the notion of the comparator as a gating mechanism proposed by Vinogradova (2001) will be used in this context, along with some more speciﬁc suggestions regarding the workings of the comparator mechanism (i.e., that it assesses relative, and not absolute, differences in the strength of the stimuli), and how this mechanism may operate differently in individuals with ASD. This comparator theory of ASD functioning actually allows a large number of the discrimination learning ﬁndings discussed earlier to be accommodated.

60

A Comparator Model

50 40 30 ASD

Control

Figure 8.4 Mean percentage choice for the most-

and least-selected stimuli for the groups with autism spectrum disorder (ASD) and the mentalage-matched controls in an experiment reported by Leader, Loughnane, Mc Moreland, and Reed (2009).

08-Schachtman-08.indd 178

According to a comparator model, a comparison is made, at the time of performance, between the strengths of the various stimuli in regard to predicting the appropriate outcome. The cues that have the greatest strengths, in terms of their predictive values with respect to the outcome, are selected to control behavior by the comparator, and performance on the basis of those stimuli with relatively weaker strengths is inhibited.

2/15/2011 7:44:07 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

08-Schachtman-08.indd 179

study by Leader et al., 2009). In a subsequent phase, for one group of subjects, the overselected stimulus for each individual was identiﬁed, and extinguished, by pairing it with a novel and reinforced cue (e.g., two trial types, A– E+). A second group did not receive this treatment. The elements of the previously reinforced (AB) and nonreinforced (CD) compounds were then retested. After this previously overselected stimulus (i.e., either A or B) was extinguished, behavioral control by the previously overselected stimulus reduced, and control by the underselected cue emerged, without any direct training of this latter stimulus. This was not the case for the group lacking the extinction of the previously overselected stimulus (see panel of Fig. 8.5). Figure 8.5 shows the change in the percentage of times that each stimulus was chosen in the follow-up test relative to the ﬁrst test. This ﬁnding suggests that overselectivity, a key phenomenon in explaining discrimination performance in those with ASD, could be explained as a postacquisition phenomenon and could be explained by the comparator theory. Comparator Predictions for Autism Spectral Disorders

That overselectivity is much more pronounced in individuals with ASD than in typically developing individuals (Leader et al., 2009; Lovaas et al., 1971)

% change pre and post extinction

At any given time, there are many stimuli that could potentially form the basis for action, and the comparator system allows a decision to be made about which are the most important. This hypothesis makes one quite important prediction not made by many other theories of learning (Aiken & Dickinson, 2005; Dickinson & Burke, 1996); speciﬁcally, in situations of cue competition, such as between the elements of a reinforced compound (AB+), the stimulus that controls performance most powerfully does so, not because the other stimuli present have not been learned about, but rather, because it has a relatively greater strength at the time of test. Thus, the comparator theory is a theory regarding performance, rather than initial learning. Differences in levels of conditioned responding may reﬂect postacquisition effects. This suggestion implies that reducing the strength of the controlling cue would improve the control exerted by the apparently weaker cues, despite no direct change having been made to the strength of these latter cues; this is often referred to as a revaluation effect, and it has been shown a number of times in various contexts with various species (e.g., see Kaufman & Bolles, 1981; Matzel, Schachtman, & Miller, 1985; Van Hamme & Wasserman, 1994). In fact, such an effect has been noted with some individuals with ASD, where devaluing an element of a compound that controls behavior results in the emergence of greater behavioral control than previously shown by the remaining element in the absence of direct training for that latter element. This ﬁnding suggests that the comparator theory is a potential candidate for use in this context, and, more speciﬁcally, in explaining the overselectivity phenomenon. Reed et al. (2009) reported an experiment that examined whether overselectivity in individuals with ASD could be the product of a postacquisition performance mechanism, rather than the more usual suggestion that individuals with ASD show an attentional deﬁcit, and do not encode all the elements of the S+ (e.g., Dube et al., 1999; Lovass et al., 1971). In this study, children with ASD were presented with a trial-and-error discrimination task, using two, two-element stimuli (i.e. AB+ versus CD–), and overselectivity was noted (as described earlier for the ﬁrst part of the

179

40 30

Most

Least

20 10 0 –10

Experimental

Control

–20 –30 –40 –50

Figure 8.5 Mean difference in pre- and postex-

tinction scores for the most- and least-chosen stimuli for the experimental group (most-selected stimulus extinguished) and control group (no extinction). (From an experiment reported by Reed, Broomﬁeld, McHugh, McCausland, and Leader, 2009).

2/15/2011 7:44:07 PM

180

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

can be explained by two aspects of the comparator theory as outlined in this chapter. The ﬁrst potential explanation follows from the notion that the comparator operates on the relative strengths of the stimuli, rather than their absolute strengths. This would suggest that in situations where learning is not as strong, or complete (subasymptotic), that the relative differences between the stimuli will be greater. For example, if two cues reach 100% and 90% of their possible strengths, the relative difference between them is 11% (the 10% difference is 11% of 90%); but if the cues reach 60% and 50% of their potential strengths, the relative difference is 20% (note that the absolute difference is the same in both cases). This possibility would not necessarily be unique to those with ASD, but it would suggest such an effect with any group or situation where learning is weak. The second suggestion is speciﬁc to those with ASD, and it is that the comparator mechanism is oversensitive in individuals with ASD. That is, when presented with a range of stimuli that are all potentially important for predicting an outcome, slight differences between those stimuli are noted by the oversensitive comparator in individuals with ASD. As a result, only a subset of those stimuli will control behavior. For example, as in the discrimination task discussed earlier (Reed et al., 2009), only one out of two possible predictive elements subsequently controlled behavior. In contrast, a less sensitive comparator would not detect such differences, and all stimuli may control behavior. Thus, there are two potential causes of overselectivity, the ﬁrst not unique to those with ASD, the second potentially unique to this population. That overselectivity is not uniquely displayed by those with ASD (e.g., McHugh & Reed, 2007; Reed & Gibson, 2005) is accounted for by the ﬁrst suggestion. That those with ASD have both of these potential causes of overselectivity accounts for the greater levels of this effect found in those individuals even compared to other overselective groups (e.g., Lovaas et al., 1971). Overselectivity and Subasymptotic Performance

The comparator theory of ASD allows an explanation of the apparently discrepant ﬁndings,

08-Schachtman-08.indd 180

noted earlier, that, although discrimination learning is poorer overall in those with ASD, “withincompound” discrimination (differentiation) in responding to the elements is more pronounced. Individuals with ASD, as well as many individuals with learning difﬁculties, show reduced levels of discrimination learning (see Dube, 2009). This would suggest that the relative strength of the S+ is lower than in typically developing subjects. Under these conditions, a version of the comparator view, such as outlined earlier, in which the relative differences between the elements of the S+ are critical, would make the prediction that differentiation between the elements of the S+ would be greater; that is, overshadowing between the element of the S+ will be stronger if the associative strengths of the elements are weak. This phenomenon has been observed in a number of learning theory experiments, especially those concerned with overshadowing. For example, Reed and Gibson (2005) noted that when two, two-element stimulus compounds were presented in series prior to the delivery of food, rats showed more conditioned responding to the compound presented in closer temporal proximity to food than to the compound presented at a greater temporal distance from food. This effect is not surprising, but when the strength of the individual elements of both compound stimuli were tested in extinction, the relative difference between the elements from the weaker compound was greater than the difference between the elements from the stronger compound; this suggests greater levels of overshadowing within compound when conditioning levels were lower than when they were greater. A somewhat similar effect was noted by Stout, Arcediano, Escobar, and Miller (2003) in terms of amount of training in a second-order conditioning procedure. In this study, overshadowing was attenuated as training progressed, when the associative strengths of the stimuli might be expected to be greater. Somewhat relatedly, Mackintosh (1976), when comparing the degrees of reciprocal overshadowing, found greater degrees of reciprocal overshadowing between stimuli when they were less salient and might be

2/15/2011 7:44:07 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

expected to have lower associative strengths. Moreover, Reed and Gibson (2005) noted that adults, with no learning disabilities or head injury, displayed overselectivity in a typical AB+ CD– discrimination training procedure when they were presented with a concurrent load task (memorizing the location of objects in a 4 x 4 grid). This was taken to reduce the degree to which the discrimination task could be learned about because there was an interfering concurrent load task, and this would lead to the prediction of overselectivity produced by subasymptotic stimuli.

given roughly similar associative strengths. However, in individuals with ASD, an overselective comparator would assess small differences as important, predictively, and one element of the stimulus would come to control behavior to a greater extent than another. To investigate this prediction, Leader et al. (2009) exposed participants to a trial-and-error discrimination study, which was described earlier. One group of participants had ASD, and the other was a mental-age-matched control group. Participants were presented with a simultaneous discrimination procedure, comprising two, twoelement stimuli, each compound stimulus comprising two colored circles (see left panel of Fig. 8.6). Two sets of such cards were used for each participant: One set of cards could display a blue circle and a red circle on one card, and green circle and a yellow circle on the other card. The second set could be brown and gray on one card, and orange and pink on the other card. The exact combination of colors was different for each participant. The intensity of the four colors displayed on one of the sets of the cards was equal to one another (as determined both by the amount of pixilation of color in the circle and preexperimental subjective ratings). For the other set of cards, however, three colors were of

Overselectivity and Sensitive Comparator

An overselective comparator may explain differences in reactivity to stimulus salience (Kemner & Van Engeland, 2006; Leader et al., 2009), stimulus novelty (O’Riordan, Plaisted, Driver, & Baron-Coehn, 2001), as well as stimulus strength, which characterize discrimination performance in those with ASD. In individuals with a typically functioning comparator system, small differences between the elements of a stimulus would not lead to one particular element controlling behavior to a much greater extent than another element; rather each element would control performance to the same degree as one another

Percentage

181

90 80 70 60 50 40 30 20 10 0

Most Least

Eq

Diff Autistic

Eq

Diff Control

Figure 8.6 Left panel shows an example of the two-card discrimination tasks, which show elements of equal salience (top) and when one element is less salient (bottom). Right panel shows the mean percentage of trials in which a component of the reinforced stimulus complex was chosen in preference to a component from the nonreinforced complex for the participants with autism spectrum disorders (ASD) and those without, for both conditions. Experiment reported by Leader, Loughnane, Mc Moreland, and Reed (2009).

08-Schachtman-08.indd 181

2/15/2011 7:44:07 PM

182

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

equal salience, whereas the fourth was less salient. After a criterion of 10 correct answers in a row was reached during each training task (making it likely that both stimuli were at asymptotic strength), the participants were given a nonreinforced test session. During this test they were shown the individual elements from the cards (one element from the reinforced and one from the nonreinforced compound), and the degree to which each element was selected was calculated. These data are shown in the right panel of Figure 8.6 and reveal that the control participants displayed little overselectivity when confronted with the stimuli of equal salience, and only marginal amounts of overselectivity when the salience of the stimuli was slightly altered. In contrast, there was some overselectivity for the equal salience stimuli in the participants with ASD, but overselectivity dramatically increased when the stimuli became discrepant in salience, even though this difference did not trigger overselectivity in mental-age-matched control. These data suggest a speciﬁc sensitivity to salience differences in children with ASD relative to mental-age-matched controls. This effect may, of course, reﬂect differing sensitivity to aspects of the visual environment as suggested earlier (see also Kemner & van Engeland, 2006), or it may reﬂect differing sensitivity to novelty (O’Riordan et al., 2001); in this latter context, the less salient cue may be novel because it was the only stimulus of this (low) salience. However, given that the less salient, or novel, cue was not selected, these explanations would have to suggest that this triggered the operation of a rule “select opposite” to the cue that triggers attention, which, although possible, seems unlikely in children with ASD who have a tendency to perseverate within set (Hughes et al., 1994). Certainly, previous research suggests that their behavior tends to follow the stand-out cues (Russell, Mauthner, Sharpe, & Tidswell, 1991). The results are, however, entirely consistent with the operation of a sensitive comparator mechanism. Slight differences in salience, not typically enough to trigger differential responding to the cues (overshadowing-like effects), may be

08-Schachtman-08.indd 182

discriminated by a sensitive comparator and lead to such differences in performance based on these cues. This suggestion explains the degree to which stimulus overselectivity emerges in individuals with ASD.

IMPLICATIONS FOR INTERVENTION One reason why learning theory explanations have proved long lasting, relative to the changing nature of cognitive explanations, is that they tend to provide both a ﬁrm theoretical base for understanding a phenomenon and also allow action to be taken on the basis of this understanding. The comparator view of ASD is no exception to this suggestion, and it provides the basis for a novel intervention for overselectivity in ASD. This is not to say that existing interventions are inappropriate, or ineffective, but that an additional tool in this particular arsenal is not unhelpful. Observing Responses

An intervention for overselectivity that has received investigation is the use of observing response procedures. This method ensures that all stimuli present are sampled, overcoming potential attentional deﬁcits. An observing response procedure achieves this by bringing sensory receptors into contact with all of the environmental stimuli (Wyckoff, 1952). An observing response is one designed to allow such contact to be made (e.g., pointing to all elements of the stimulus present). Previous ﬁndings have shown that the use of observing response procedures can alleviate the problem of overselectivity (Constantine & Sidman, 1975, Dube & McIlvane, 1999). Broomﬁeld, McHugh, and Reed (2008) investigated this procedure for a nonclinical sample using an MTS task. It was found that, regardless of the type of observing response used (naming or pointing), overselectivity was reduced by an observing response procedure. However, when the observing response procedure was removed, there was no maintenance of the reduction of overselectivity; the observing response conditions did not differ signiﬁcantly

2/15/2011 7:44:07 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

Extinction to Combat Overselectivity

The use of extinction to combat overselectivity has not previously been explored in the literature. However, given the relative failure of an observing response procedure with this population, developing alternative methods of intervention would seem to be important. Moreover, the theoretical suggestions of the comparator hypothesis, and empirical ﬁndings outlined here and elsewhere (Broomﬁeld et al., 2008), suggest that extinction of the overselected stimulus may be a plausible alternative. It is a truism to say that extinction, as a procedure, has been widely used to decrease problem behaviors within a clinical context. For example, Iwata, Pace, Kalsher, Cowdey, and Cataldo (1990) found that self-injurious behavior, in three subjects, was decreased to almost zero levels after the ﬁve, 15-minute sessions of extinction. Koegel, Egel, and Williams (1980)

08-Schachtman-08.indd 183

implemented the time-out procedure to reduce aggression from a child with ASD at school. Thus, there are numerous examples of the use of extinction to decrease clinical problem behaviors, including those occurring at a high rate and that are disruptive or injurious to an individual. It is possible to conceptualize overresponding to a particular aspect of the environment in that manner, and, to that extent, such behavior is a prime target for extinction. The use of extinction to reduce overselectivity was explored by Leader et al. (2009) and by Reed et al. (2009), using Reed and Gibson’s (2005) discrimination task. As noted earlier, once the overselected cue was established, it was extinguished by pairing it with novel stimuli. The results, presented in Figure 8.6, illustrate that there are some circumstances under which extinction of the previously overselected stimulus resulted in both a reduction in the level of responding to the previously overselected, and, importantly, to a reemergence of responding to a previously underselected stimulus. Both of these effects, of course, would be necessary for any clinical intervention to be truly useful. However, it is worth noting that such effects were only obtained with a relatively highfunctioning group of individuals with ASD, and not with a lower functioning set of individuals (see Fig. 8.7), which may limit the usefulness of such a procedure. Moreover, a series of studies

% change pre and post extinction

from the control conditions. That is, overselectivity returned. This pattern of ﬁndings replicated those reported by Dube and McIlvane (1999). The fact that the observing response intervention did not maintain its gains postintervention could have possible implications for developing it for use in a clinical population, and this was explored by Broomﬁeld et al. (unpublished data). When the observing response procedure was applied to a group of children with ASD, results did not reveal the beneﬁt seen by Broomﬁeld et al. (2008) in a typically developing population. In fact, there was very little evidence of the effectiveness of an observing response procedure at all with the clinical population in this context. It was found that, even when the observing response was in place, there was no signiﬁcant difference between the levels of overselectivity in the observing response group compared with the control condition. Naturally, any failure to demonstrate an effect needs to be taken with caution (it is always difﬁcult to interpret a null result). However, the issue of maintenance of the effect, and the null results, together call into question how effective the observing response procedure is as an intervention in this population.

183

30 20 10 0 –10

High functioning

Low functioning

–20 –30 –40 –50

Most

Least

–60

Figure 8.7 Mean difference in pre- and postextinc-

tion scores for the most and least chosen stimuli for the high-functioning group and low-functioning group in an experiment reported by Leader, Loughnane, Mc Moreland, and Reed (2009).

2/15/2011 7:44:07 PM

184

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

by Broomﬁeld, McHugh, and Reed (unpublished data) also highlight several important issues for the use of such a procedure, if it is to be developed into a clinical procedure. For example, it appeared only to work well where there were initially high levels of overselectivity. A large overselectivity effect is needed, initially, to produce the extinction-induced effects discussed earlier. One potential reason for this is that a large difference gives room to see an emergence of control by the underselected stimulus. Another reason is that, in the absence of an overselectivity effect, it is possible that the elements are, in fact, seen as a compound. This would mean that, when extinction takes place, there would be a reduction in responding to both the targeted overselected stimuli, and to the underselected cues, which may well be associated with those overselected cues, and, through second-order extinction, also suffer a reduction in their power to control behavior. Also, if both of the cues are attended to initially, resulting in neither acquiring overselective behavioral control, it would mean, of course, that what would otherwise be the “underselected” cue simply cannot “emerge,” because it is already controlling behavior; in other words, there is a ceiling effect. Thus, there is a suggestion that extinction may offer some individuals with ASD some beneﬁt. This stands in contrast to the ﬁnding that observing responses, at least in this context, do not offer beneﬁt to individuals with ASD in terms of overselectivity. It has been suggested that the extinction results are compatible with a comparator interpretation, because reduction in the value of the previously overselected stimulus can result in an increase in control over behavior by the previously underselected stimulus. While this is true, the discussion of clinical interventions raises another possibility, which is worth mentioning in this context. The use of extinction as a clinical tool (and in the laboratory) has been found to increase variability in behavior, at least on its initial application. Pear (1985) found that when pigeons were reinforced for pecking a key every 15 seconds, they stayed close to the key and emitted routine patterns of head and body movements. When the animals were placed on a similar schedule, but one that was only reinforced

08-Schachtman-08.indd 184

once every 5 minutes, they strayed farther from the key. Both of these patterns happened during extinction; but as extinction continued, behavior became much more variable. Ducker and Van Lent (1991) increased the number of gesture requests exhibited by individuals with developmental disabilities by withdrawing reinforcement for high-rate requests. Similarly, Lalli, Zanolli, and Wohn (1994) found that when previously reinforced topographies of toy play were placed under extinction, induction of untrained topographies occurred. These effects raise another possibility. It may be that increased variability introduced by extinction could lead to the greater sampling of previously underselected stimuli, and it is this increased sampling, rather than any comparator process, that is responsible for the current results. Although this is certainly worth further investigation, there is one reason (discussed later in detail) that may make this suggestion less attractive. That is, although extinguishing responding to the previously overselected stimulus (responses to that stimulus are no longer followed by a “yes” response), this procedure does not necessarily involve a reduction of overall reinforcement rate, because responses to a novel cue are reinforced. Potentially, participants could very quickly regain rates of reinforcement. It is unclear, under these circumstances, whether increased variability in responding would occur.

SUMMARY The discussion in this chapter is based on the idea that some differences in the discrimination learning performance of people with ASD are not based on acquisition deﬁcits but result from postlearning, or comparator processes, connected to an oversensitive comparator mechanism. This problem may not be unique to people with ASD, but it appears to play a role in determining their expressed behavioral proﬁle. In addition to the theoretical implications, the current discussion suggests that an intervention of extinguishing overselected cues may aid subsequent performance in children with ASD. Thus, one important feature of the current learning-theoretic approach to overselectivity is

2/15/2011 7:44:08 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

its potential utility in the design of intervention programs for individuals with ASD for whom these important skills appear to be absent.

ACKNOWLEDGMENTS Thanks are due to Lisa A. Osborne for her support and help with the development of these ideas. Correspondence concerning this article should be sent to Phil Reed, Department of Psychology, Swansea University, Singleton Park, Swansea, SA2 8PP, United Kingdom (e-mail: p.reed@ swansea.ac.uk).

REFERENCES Aiken, M. R. F., & Dickinson, A. (2005). Simulations of a modiﬁed SOP model applied to retrospective revaluation of human causal learning. Learning and Behavior, 33, 147–159. Baddeley, A. (1991). Human memory: Theory and practice. Hove, England: Lawrence Erlbaum Associates. Baron-Cohen, S. (1997). Theory of mind in autism: Its relationship to executive function and central coherence. In D. Cohen, & F. Volkmar (Eds.), Handbook of autism and pervasive developmental disorders. Chichester, England: John Wiley and Sons. Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985). Does the autistic child have a “theory of mind”? Cognition, 21, 37–46. Bentall, R. P. (1990). Reconstructing schizophrenia. London, England: Routledge. Boucher J, & Warrington, E. K. (1976). Memory deﬁcits in early infantile autism: Some similarities to the amnesiac syndrome. British Journal of Psychology, 67, 76–87. Broomﬁeld, L., McHugh, L., & Reed, P. (2008). The effect of observing response procedures on the reduction of over -selectivity in a match to sample task: Immediate but not long term beneﬁts. Research in Developmental Disabilities, 29, 217–234. Constantine, B., & Sidman, M. (1975). The role of naming in delayed matching to sample. American Journal of Mental Deﬁciency, 79, 680–689. Dickinson, A., & Burke, J. (1996). Within compound associations mediate the retrospective revaluation of causality judgements. Quarterly Journal of Experimental Psychology, 49, 60–80.

08-Schachtman-08.indd 185

185

Dickson, C. A., Deutsch, C. K., Wang, S. S., & Dube, W. V. (2006). Matching-to sample assessment of stimulus overselectivity in students with intellectual disabilities. American Journal on Mental Retardation, 111, 447–453. Dube, W. V. (2009). Stimulus overselectivity in discrimination learning. In P. Reed (Ed.), Behavioural theories and interventions for autism. New York, NY: Nova Science Publishers. Dube, W. V., Lombard, K. M., Farren, K. M., Flusser, D. S., Balsamo, L. M., & Fowler, T. R. (1999). Eye tracking assessment of stimulus overselectivity in individuals with mental retardation. Experimental Analysis of Human Behavior Bulletin, 17, 8–14. Dube, W. V., & McIlvane, W. J. (1996). Some implications of a stimulus control topography analysis for emergent stimulus classes. In T. R. Zentall & P. M. Smeets (Eds.), Stimulus class formation in humans and animals (pp. 197–218). Amsterdam, Netherlands: Elsevier. Dube, W. V., & McIlvane. W. J (1999). Reduction of stimulus overselectivity with non verbal differential observing responses. Journal of Applied Behavior Analysis, 32, 25–33. Duker, P. C., & van Lent, C. (1991). Inducing variability in communicative gestures used by severely retarded individuals. Journal of Applied Behavior Analysis, 24, 379–386. Eldevik, S., Hastings, R. P., Hughes, J. C., Jahr, E., Eikeseth, S., & Cross, S. (2009). Meta-analysis of early intensive beheavioral intervention for children with autism. Journal of Clinical Child and Adolescent Psychology, 38, 1–12. Frith, U. (1989). Autism: Explaining the enigma. Oxford, England: Blackwell. Frith, C. (1992). The cognitive neuropsychology of schizophrenia. Hillsdale, NJ: Lawrence Erlbaum Associates. Goodman, R. (1989). Infantile autism: A syndrome of multiple primary deﬁcits. Journal of Autism and Developmental Disorders, 19, 409–424. Greer, R. D., & Keohane, D-D. (2009). CABAS® contributions to identifying, inducing, and sequencing verbal development: The functions of CABAS® schools: Learner driven research and demonstration centers. In P. Reed (Ed.), Behavioral theories and treatments for autism (pp. 169–203). New York, NY: Nova Science Publishers. Hall, G. (1991). Perceptual and associative learning. Oxford, England: Clarendon. Happé, F., & Frith, U. (2006). The weak coherence account: Detail-focused cognitive style in autism

2/15/2011 7:44:08 PM

186

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

spectrum disorders. Journal of Autism and Developmental Disorders, 36, 5–25. Happe, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9, 1218–1220. Heyes, C. M. (1998). Theory of mind in nonhuman primates. Behavioral and Brain Sciences, 21, 101–114. Howard, J. S., Sparkman, C. R., Cohen, H. G., Green, G., & Stanislaw, H. (2005). A comparison of intensive behavior analytic and eclectic treatments for young children with autism. Research in Developmental Disabilities, 26, 359–383. Hughes, C., Russell, J., & Robbins, T., (1994). Speciﬁc planning deﬁcit in autism: Evidence of a central executive dysfunction. Neuropsychologia, 3, 477–492. Hull, C.L. (1952). A behavior system. New Haven, CT: Yale University Press. Iwata, B.A., Pace, G.M., Kalsher, M.J., Cowdery, G.E., & Cataldo, M.F. (1990). Experimental analysis and extinction of self-injurious escape behavior. Journal of Applied Behavior Analysis, 23, 11–27. Jolliffe, T., & Baron-Cohen, S. (1999). A test of central coherence theory: Linguistic processing in high-functioning adults with autism or Asperger syndrome: Is local coherence impaired? Cognition, 71, 149–185. Jordan, T. R., & Thomas, S. M. (2001). Effects of horizontal viewing angle on visual and audiovisual speech recognition. Journal of Experimental Psychology: Human Perception and Performance, 27, 1386–1403. Kaufman, M. A., & Bolles, R. C. (1981). A nonassociative aspect of overshadowing. Bulletin of the Psychonomic Society, 18, 318–320. Kemner, C., & van-Engeland, H. (2006). ERPs and eye movements reﬂect atypical visual perception in pervasive developmental disorder. Journal of Autism and Developmental Disorders, 36, 45–54. Kennedy, C. H., Meyer, K. A., Knowles, T., & Shukla, S. (2000). Analyzing the multiple functions of stereotypical behavior for students with autism: Implications for assessment and treatment. Journal of Applied Behavior Analysis, 33, 559–571. Koegel, R. L., Egel, A. L., & Williams, J. (1980). Behavioral contrast and generalization across settings in treatment of autistic children. Journal of Experimental Child Psychology, 30, 422–437.

08-Schachtman-08.indd 186

Koegel, R. L., & Wilhelm, H. (1973). Selective responding to multiple visual cues by autistic children. Journal of Experimental Child Psychology, 15, 442–453. Lalli, J. S., Zanolli, K., & Wohn, T. (1994). Using extinction to promote response variability in toy play. Journal of Applied Behavior Analysis, 27, 735–736. Leader, G., Loughnane, A., Mc Moreland, C., & Reed, P. (2009). The effect of stimulus salience on over-selectivity. Journal of Autism and Developmental Disorders, 39, 330–338. Lord, C., Cook, E., Leventhal, B., & Amaral, D. (2000). Autism spectrum disorders. Neuron, 28, 355–363. Lord, C., & McGee, J. (2001). Educating children with autism. Washington, DC: National Academy Press. Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3–9. Lovaas, O. I., & Schreibman, L. (1971). Stimulus overselectivity of autistic children in a two stimulus situation. Behavior Research and Therapy, 9, 305–310. Lovaas, O. I., Schreibman, L., Koegel, R., & Rehm, R. (1971). Selective responding by autistic children to multiple sensory input. Journal of Abnormal Psychology, 77, 211–222. Mackintosh, N. J. (1976). Overshadowing and stimulus intensity. Animal Learning and Behavior, 4, 186–192. Mackintosh, N. J. (1983). Conditioning and associative learning. New York, NY: Oxford University Press. Makrygianni, M., & Reed, P. (2010). A meta-analytic review of the effectiveness of behavioral early intervention programs for children with Autistic Spectrum Disorders. Research in Autism Spectrum Disorders, 4, 577–593. Matzel, L. D., Schachtman, T. R., & Miller, R. R. (1985). Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. Learning and Motivation, 16, 398–412. McDaniel, W. F., (2001). A simple strategy for the qualitative assessment of learning capacity of clients with mental retardation or other severe cognitive deﬁcits. Developmental Disabilities Bulletin, 29, 1–22. McHugh, L., & Reed, P. (2007). Age trends in stimulus over-selectivity. Journal of the Experimental Analysis of Behavior, 88, 369–380.

2/15/2011 7:44:08 PM

AUTISM SPECTRUM DISORDERS AND DISCRIMINATION

McHugh, L., & Reed, P. (2008). Using relational frame theory to build grammar in children with autistic spectrum conditions. The Journal of Speech-Language Pathology and Applied Behavior Analysis, 2.4–3.1, 60–77. Miller, R. R., & Schachtman, T. R. (1985). The several roles of context at the time of retrieval. In P. D. Balsam & A. Tomie (Eds.), Context and learning. Hillsdale, NJ: Laurence Erlbaum. O’Riordan M., & Plaisted, K. (2001). Enhanced discrimination in autism. The Quarterly journal of experimental psychology. A, Human experimental psychology 2001;54(4):961–79. O’Riordan M. A., Plaisted, K. C., Driver, J., & BaronCohen, S. (2001). Superior visual search in autism. Journal of Experimental Psychology. Human Perception and Performance, 27, 719–730. Osborne, L. A., & Reed, P. (2009). The role of the family in autistic spectrum conditions: Theory and practical implications. In P. H. Krause, & T. M. Dailey (Eds.), Handbook of parenting: Styles, stresses, and strategies (pp. 259–289). New York, NY: Nova Science Publishers. Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1991). Executive function deﬁcits in highfunctioning autistic individuals: Relationship to theory of mind. Journal of Child Psychology and Psychiatry, 32, 1081–1105. Ozonoff, S., Strayer, L., McMahon, A., & Filloux, F. (1994). Executive function abilities in autism and Tourette syndrome: An information processing approach. Journal of Child Psychology and Psychiatry, 35, 1015–1032. Pavlov, I. P. (1926). Conditioned reﬂexes. New York, NY: Dover. Pear, J. J. (1985). Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 44, 217–231. Pearce, J. M. (1987). A model of stimulus generalization for Pavlovian conditioning. Psychological Review, 94, 61–73. Pearce, J. M. (1994). Similarity and discrimination: a selective review and a connectionist model. Psychological Review, 101, 587–607. Plaisted, K., O’Riordan, M., & Baron-Cohen, S. (1998). Enhanced discrimination of novel, highly similar stimuli by adults with autism during a perceptual learning task. Journal of Child Psychology and Psychiatry, 39, 765–775. Premack, D. G., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1, 515–526.

08-Schachtman-08.indd 187

187

Ray, B. A., & Sidman, M. (1970). Reinforcement schedules and stimulus control. In W. N. Schoenfeld (Ed.), The theory of reinforcement schedules (pp. 187–214). New York, NY: Appleton. Reed, P. (2006). The effect of retention interval on stimulus over-selectivity using a matching to sample procedure. Journal of Autism and Developmental Disorders, 36, 1115–1121. Reed, P., Broomﬁeld, L., McHugh, L., McCausland, A., & Leader, G. (2009). Extinction of overselected stimuli causes re-emergence of previously under selected stimuli in higher functioning children with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 39, 290–298. Reed, P., & Gibson. E. (2005). The effects of concurrent task load on stimulus overselectivity. Journal of Autism and Developmental Disorders, 35, 601–614. Reed, P., Osborne, L. A., & Corness, M. (2007). The real-world effectiveness of early teaching interventions for children with autistic spectrum disorders. Exceptional Children, 73, 417–433. Repp, A. C., Karsh, K. G., & Lenz, M. W. (1990). Discrimination training for persons with developmental disabilities: A comparison of the task demonstration model and the standard prompting hierarchy. Journal of Applied Behavior Analysis, 23, 43–52. Rodgers, J. (2000). Visual perception and Asperger syndrome: Central coherence deﬁcit or hierarchization deﬁcit? Autism, 4, 321–329. Rumsey, J. M. (1985). Conceptual problem-solving in highly verbal, nonretarded autistic men. Journal of Autism and Developmental Disorders, 15, 23–36. Russell, J., Mauthner, N., Sharpe, S., & Tidswell, T. (1991). The “windows task” as a measure of strategic deception in preschoolers and autistic subjects. British Journal of Developmental Psychology, 9, 331–349. Sachs, G., Anderer, P., Doby, D., Saletu, B., & Dantendorfer, K. (2003). Impaired conditional discrimination learning in social phobia. Neuropychobiology, 47, 66–72. Schreibman, L., Charlop, M. H., & Koegel, R. L. (1982). Teaching autistic children to use extrastimulus prompts. Journal of Experimental Child Psychology, 33, 475–491. Shah, A., & Frith, U. (1983). An islet of ability in autism: A research note. Journal of Child Psychology and Psychiatry, 24, 613–620.

2/15/2011 7:44:08 PM

188

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Shallice, T. (1988). From neuropsychology to mental structure. Cambridge, England: Cambridge University Press. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press. Stout, S., Arcediano, F., Escobar, M., & Miller, R. R. (2003). Overshadowing as a function of trial number: Dynamics of ﬁrst- and second-order comparator effects. Learning and Behavior, 31, 85–97. Sundberg, M. L., & Michael, J. (2001). The beneﬁts of Skinner’s analysis of verbal behavior for children with autism. Behavior Modiﬁcation, 25, 698–724. Thomas, R. K. (1996). Investigating cognitive abilities in animals: Unrealized potential. Cognitive Brain Research, 3, 157–166. Van Hamme, L. J., & Wasserman, E. A. (1994). Cue competitions in causality judgments: The role of nonpresentation of compound stimulus elements. Learning and Motivation, 25, 127–151. Vinogradova, O. S. (2001). Hippocampus as comparator system: Two inputs and two outputs of the hippocampus. Hippocampus, 11, 578–598.

08-Schachtman-08.indd 188

Wayland, S., & Taplin, J. E. (1985). Featureprocessing deﬁcits following brain injury. Overselectivity in recognition memory for compound stimuli. Brain and Cognition, 4, 338–355. Wimmer, H., & Perner, J. (1983). Beliefs about beliefs: Representations and constraining functions of wrong beliefs in young children’s understanding of deception. Cognition, 13, 103–128. Wyckoff, L. B. (1952). The role of observing responses in discriminative learning, part 1. Psychological Review, 59, 431–442. Yarmolenko, A.V. (1926). [Change of environment as a factor determining an anomaly of behavior (autism in uncontrolled children)]. Изменение окружающей среды ка к ф актор, определяющий аномалию поведения (аутизм в б езудержных де тях)] Voprosy Izucheniya i Vospitaniya Lichnosti, 1, 38–46. Zeaman, D., & House, B. J. (1963). The role of attention on retardate discrimination learning. In N. R. Ellix (Ed.), Handbook of mental deﬁciency. New York, NY: McGraw-Hill.

2/15/2011 7:44:08 PM

PART III

Applications to Health and Addiction

09-Schachtman-09.indd 189

2/17/2011 11:17:18 AM

This page intentionally left blank

CHAPTER 9 Conditioned Immunomodulation Jennifer L. Szczytkowski and Donald T. Lysle

It was originally believed that the immune system functioned independently from other regulatory systems of the body; however, recent research has shown that the cells and processes of the immune system are highly inﬂuenced by both neural and endocrine factors. It is now well established that the neural processes involved in learned behaviors can inﬂuence the immune system and thus alter susceptibility to disease. This chapter focuses on the regulation of the immune response by Pavlovian conditioning utilizing such immunomodulatory unconditioned stimuli as illicit and immunosuppressive drugs, aversive stimuli, and immunostimulatory agents. Research indicates that pairing a neutral stimulus that does not evoke changes in immune status with a stimulus that is itself immunomodulatory may result in the formerly neutral stimulus acquiring immune-altering properties. Accordingly, Pavlovian conditioning of the immune response is a mechanism by which an organism can anticipate environmental events and alter the response of the immune system.

INTRODUCTION TO CONDITIONED IMMUNOMODULATION The ﬁelds of neuroscience and immunology evolved independently of one another throughout most of their history. It was originally believed that the immune system functioned in isolation from the rest of the body, and it was considered to be an autonomous, self-regulating entity responsible for the prevention of infection and the maintenance of health. Recently it has been shown that the cells and processes of the immune system are highly inﬂuenced by both internal and external factors that can alter the immune response, thereby modulating the susceptibility of the host to pathogens and the progression of disease (e.g., Straub & Besedovsky, 2003; Vishwanath, 1996). Changes in the characteristics of an immune response can signiﬁcantly impact the ability of an organism to neutralize a potentially harmful microbial attack and theoretically either speed or prohibit recovery. It is

now well recognized that cells involved in an immune response interact in an endogenous chemical/hormonal environment, so that any alteration in the neurochemical or hormonal milieu can potentially change the nature of the immune response. Factors that may modulate the neural and hormonal chemistry and thereby impact immune function include behavior, the use of drugs (both illicit and therapeutic), antigenic challenge, and changes in the external environment. Certain behaviors may induce alterations in hormone secretion via activation of the hypothalamic-pituitary-adrenal (HPA) axis. Upon activation of this pathway, the paraventricular nucleus of the hypothalamus produces corticotrophin-releasing hormone (CRH), which stimulates release of adrenocorticotropic hormone (ACTH) from the pituitary gland which results in the release of glucocorticoids such as cortisol from the adrenal glands. Adrenocortical hormones have been shown to modulate lymphocyte function, providing evidence for the 191

09-Schachtman-09.indd 191

2/17/2011 11:17:18 AM

192

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

inﬂuence of the HPA axis on immunoregulation (Crabtree, Gillis, Smith, & Munck, 1980; Onsrud & Thorsby, 1981). In addition, the sympathetic division of the nervous system densely innervates both primary and secondary lymphoid tissues and releases molecules that interact with receptors on immune cells (Felten et al., 1984; Williams et al., 1981). Furthermore, radioligand binding studies have demonstrated the presence of both α- and β-adrenergic binding sites on the surface of immune cells (Fuchs, Campbell, & Munson, 1988; McPherson & Summers, 1982; Williams, Snyderman, & Lefkowitz, 1976), and in vitro studies indicate that these binding sites are truly functional adrenergic receptors (Hadden, Hadden, & Middleton, 1970; Sanders & Munson, 1985). The communication between the nervous system and the immune system is bidirectional; activation of the immune system is perceived by the brain, which can result in behavioral and neuroendocrine changes. For example, hypothalamic ﬁring rates increase in response to antigenic challenge (Besedovsky, Sorkin, Felix, & Haas, 1977), and infection with the inﬂuenza virus leads to increased plasma concentrations of corticosterone in mice (Dunn, Powell, Meitin, & Small, 1989). Interleukin-1 (IL-1) acts both to coordinate the peripheral immune response and to signal the central nervous system. IL-1 has been shown to be involved in the induction of fever (Dinarello, 1999), as well as the regulation of sleep (Imeri & Opp, 2009). The HPA axis has been shown to become stimulated in response to lipopolysaccharide (LPS) and/or cytokines, with IL-1 being the most potent activator (Beishuizen & Thijs, 2003; Besedovsky et al., 1991). LPS is a component of the gram-negative bacterial cell wall utilized experimentally to induce activation of select immune parameters, including the production of nitric oxide. The injection of IL-1 or LPS either peripherally or directly into the central nervous system induces increases in proinﬂammatory cytokine expression in the brain followed by a vagal-mediated display of sickness behaviors (Bluthe et al., 1994; Dantzer et al., 1998; Goehler et al., 1999). Although the effects are complex, there is evidence that depressive symptomatology is associated with changes in the level of IL-1 in plasma of

09-Schachtman-09.indd 192

patients with depression (e.g., Ovaskainen et al., 2009). Figure 9.1 provides a common illustration of the nature of the interactions between the central nervous system and the immune system.

PAVLOVIAN CONDITIONING AS EVIDENCE FOR NEUROIMMUNE INTERACTIONS Factors such as an individual’s psychological state or learned behaviors have been shown to inﬂuence the functioning of the immune system. In fact, Pavlovian conditioning of the immune response is a well-documented phenomenon that can be accomplished with a wide variety of immunomodulatory stimuli providing some of the most concrete evidence for communication between the nervous and immune systems. Research has shown that the pairing of a neutral stimulus that does not evoke changes in immune status (e.g., a sound, taste, smell, or physical context) with a stimulus that is itself immunomodulatory results in the formerly neutral stimulus acquiring immune-altering properties. In the typical experimental design, conditioned alterations of immune status are determined when the immune system is challenged in conjunction with exposure to the conditioned stimulus. There is evidence for both conditioned immune enhancement and conditioned immunosuppression. Although more research is required, the direction of the conditioned alterations is primarily dependent upon the speciﬁc properties of the unconditioned stimulus (Ader & Cohen, 1975; MacQueen & Siegel, 1989). Most investigators would agree that Pavlovian conditioning is an associative learning process by which an organism may become better prepared to adapt its behavior as necessary (e.g., Rescorla, 1988). The Pavlovian conditioning of the immune response may be viewed as a means by which an organism learns the predictive value of a stimulus that has become associated with an immunologically relevant outcome such that it may adapt not only its behavior but the response of its immune system as well. The ability of the organism to learn associations between stimuli predictive of immunological challenge

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

193

Neural immune interactions

Hypothalamus CRH

ol

Locus coeruleus

r tis

Co

Nucleus of the tractus solitarius

ACTH Pituitary gland

Adrenal glands

SNS Vagus nerve

Cytokines

Cortisol & Epinephrine Immune organs

Neutrophil Immune cells

APC

T-cell NK cell

Mast cell

B-cell

Figure 9.1 Behavioral changes in an organism may result in the activation of the hypothalamic-pituitary-adrenal (HPA) axis and/or the sympathetic nervous system. Activation of the HPA axis results in the release of corticotrophin-releasing hormone (CRH) from the hypothalamus. CRH acts upon the pituitary gland, stimulating the release of ACTH and resulting in the production of glucocorticoids, such as cortisol, by the adrenal glands. Cortisol, along with epinephrine, which may also be released from the adrenal glands, has direct effects on the immune system by binding to receptors on immune cells. Sympathetic innervation of immune organs, such as the spleen, may also cause changes in immune functioning by the releasing of norepinephrine. Cytokines released from cells of the immune system feed back to the nervous system and may modulate the activity of the hypothalamus.

may allow the host to alter immune function in anticipation of these challenges in an attempt to more adequately respond to environmental events.

CONDITIONED IMMUNOMODULATION: HISTORICAL PERSPECTIVES The modern history of conditioned immunomodulation began with the serendipitous discovery

09-Schachtman-09.indd 193

by Robert Ader (1974, 2000) that conditioning processes could regulate immune responses. While studying conditioned taste aversion in rats, Ader paired the nausea-inducing, immunomodulatory drug cyclophosphamide (CY) with varied amounts of a sweet-tasting saccharin solution. Conditioned taste aversion is a well-documented example of one-trial passive avoidance learning in which the taste of a neutral stimulus (i.e., saccharin) is paired with a stimulus that induces gastric upset (Garcia & Koelling, 1967).

2/17/2011 11:17:18 AM

194

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Historically, toxins such as lithium chloride, apomorphine, and CY have been used as the unconditioned stimuli in taste-aversion paradigms. In the case of CY, the drug is not only capable of inducing gastrointestinal malaise, but it is also an immunomodulatory agent. Following just a single pairing of the administration of CY with the saccharin solution, the animals did in fact acquire an aversion to the saccharin, which was evidenced by a reduction in the volume of solution ingested during testing that was related to the volume consumed during the conditioning trial. In addition, continued exposure to the conditioned stimulus (i.e., the saccharin solution) without presentation of the unconditioned stimulus (i.e., CY) led to the extinction of the avoidance behavior. Ader (1974) also observed that some of the animals that had received pairings of CY and saccharin died when repeatedly reexposed to the saccharin solution alone. Furthermore, mortality rates for these animals correlated with the volume of saccharin solution ingested during the conditioning trial. It appeared possible from these results that the taste of saccharin took on immunosuppressive properties of its own through its association with CY; animals exposed to the saccharin solution during testing were more susceptible to pathogens in the environment, which led to increased mortality rates among these animals. This was conﬁrmed by a study in which rats underwent either one or two pairings of CY (50 mg/kg) with saccharin solution and 3 days later were administered an injection of the antigen, sheep red blood cells (SRBC). The experimental groups reexposed to the conditioned stimulus prior to testing not only demonstrated an aversion to the saccharin solution but also exhibited a reduction in anti-SRBC antibody titers compared to the control groups (Ader & Cohen, 1975). Pairing of saccharin and lithium chloride, which induces nausea without altering immune function, did not result in a conditioned immunosuppression, suggesting the changes are neither stress induced nor dependent upon the induction of taste aversion alone (Ader, Cohen, & Grota, 1979; MacQueen & Siegel, 1989). The conditioned suppression of anti-SRBC antibodies

09-Schachtman-09.indd 194

was still present up to 15 days after challenge with the antigen, evidence that these effects are not transient but long lasting (Ader, Cohen, & Bovbjerg, 1982; Neveu, Dancer, & Le, 1986). The conclusion reached by Ader and Cohen was that the reduction in antibodies was a result of “behaviorally conditioned immunosuppression” brought about by the learned association between the CY and the saccharin solution. These experiments were replicated shortly thereafter by other laboratories that reported similar ﬁndings (e.g., Rogers, Reich, Strom, & Carpenter, 1976; Wayner, Flannery, & Singer, 1978). Taken together, these results provided concrete evidence that conditioned immunosuppression is possible and suggested that there must be a connection between the nervous and immune systems mediating these effects. While the experiments conducted by Ader and Cohen brought the study of conditioned immunomodulation into the realm of mainstream scientiﬁc inquiry, these were not the ﬁrst investigations of their type. Recorded clinical observations of a possible link between immunological activity and learned associations date back at least to the previous century. An 1886 article in the American Journal of Medical Science details one physician’s report of a patient’s allergic response to an artiﬁcial rose (MacKenzie, 1886). In this instance, the patient was a woman suffering from debilitating allergic symptoms brought about by a variety of stimuli, one of which was roses. Upon the presentation of an artiﬁcial rose, the woman responded with allergic symptoms similar to those seen upon the presentation of the actual ﬂower. Since that time several others have documented similar observations of conditioned allergic reactions (Dekker & Groen, 1956; Hill, 1930). While observations of these phenomena may predate the 20th century, the actual investigation of conditioned immunomodulation has its roots in the 1920s experiments conducted by Soviet scientists. In these experiments administration of a foreign substance into the peritoneum of guinea pigs was paired repeatedly with a tactile stimulation. Upon the presentation of the tactile stimulation alone, the animals showed similar

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

increases in leukocyte numbers in the peritoneum to animals receiving injection of the foreign substances. While the methodology of these experiments would not pass the rigorous standards of modern experimental design, these results still provided some of the ﬁrst evidence that the immune system can in fact be conditioned in a manner agreeable with the classic work of Ivan Pavlov (Metal’nikov & Chorine, 1926, 1928; Pavlov, 1928). The studies conducted by Ader and Cohen initially, and by others subsequently, have utilized similar experimental designs. The experimental group receives exposure to the immunomodulatory unconditioned stimulus (US) paired with the neutral conditioned stimulus (CS) either repeatedly or, in the case of taste–immune associations, only one pairing is necessary. At some later point, which varies between experiments, the experimental group is reexposed to the CS without the US and at this time may also receive an immunological challenge to provide a sensitive test of an immunological conditioned response. The expression of a conditioned response (CR) following exposure to the CS indicates the acquisition of a learned association between the CS and US such that the CS alone is now sufﬁcient to elicit this response (i.e., the conditioned response) that was previously limited to the US (i.e., the unconditioned response). The speciﬁcity of the response is assured by the use of a comprehensive set of control groups, including a nonconditioned group of animals that are exposed to the CS as well as the US but in a noncontingent (e.g., unpaired) manner. These animals are subsequently also reexposed to the CS during testing to conﬁrm the lack of an immunological effect of the CS. Other control groups include animals that undergo the conditioning paradigm but are not reexposed to the CS during testing. This group serves as a control for any possible effects of the US that may still be present at the time of testing and for potential immunomodulatory effects of the conditioned procedure itself. A group exposed to the US alone allows for the measurement of the unconditioned effects of the immunomodulatory agent (the unconditioned stimulus), which can then be compared to the conditioned effects observed

09-Schachtman-09.indd 195

195

in the experimental group. Some experiments also include a “placebo group,” which receives exposure to the CS but never the US. Still another set of control groups may be included in which animals receive pre- or postconditioning exposure to the CS, which should attenuate the conditioned response and substantiate the effects as dependent upon associative learning processes. The development and persistence of the CR is dependent on several factors, and certain experimental manipulations may produce a weakening of this response. Two of the most widely studied experimental paradigms leading to reduced expression of the CR are extinction and latent inhibition. Extinction is evident when, after conditioning has taken place, repeated exposure to the CS without the US decreases the CR. Latent inhibition is a process by which repeated nonreinforced exposure to a stimulus prior to conditioning will inhibit the formation of a CR to that stimulus (Lubow & Moore, 1959). Extinction and latent inhibition have been studied extensively within models of conditioning to test hypotheses concerning retroactive and proactive stimulus interference, respectively (Pineno & Miller, 2005). Despite the wealth of literature in this area, it is as yet unclear whether all immune responses may be conditioned and research continues to unveil the breadth and complexity of conditioned immunomodulation.

CONDITIONING WITH IMMUNOSUPPRESSIVE AGENTS Studies demonstrating conditioned immunomodulation have used a variety of stimuli as the unconditioned stimulus. There are numerous experiments that have utilized pharmacological agents as the US paired with gustatory stimuli as the CS, similar to the initial studies by Ader and Cohen. Cyclophosphamide (Cytoxan) was one of the ﬁrst immunomodulatory agents to be extensively studied within the ﬁeld of conditioned immunomodulation. Originally used in the conditioned taste-aversion paradigm because of its ability to induce gastrointestinal upset (Elkins, 1974), CY, once converted by the liver to its biologically active metabolite, is a DNA

2/17/2011 11:17:18 AM

196

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

alkylating agent with cytotoxic effects (Bach, 1975; Shand, 1979). The drug acts through a p450 pathway to cross-link and alkylate DNA, rendering a cell incapable of replication (Moore, 1991). Due to this mechanism of action, CY nonspeciﬁcally alters the functioning of the immune system, and it has been used clinically in the treatment of cancer, autoimmune disorders, and as an anti-rejection drug following organ transplantation. Following the initial investigations, several more studies were conducted using CY paired with saccharin to induce alterations in immune parameters. A 1986 study found a decrease in the number of splenic plaque-forming cells in animals reexposed to a previously CY-paired stimuli, indicating immune functioning at the cellular level may be inﬂuenced by behavioral conditioning (McCoy, Roszman, Miller, Kelly, & Titus, 1986). Plaque-forming cells refer to cells that produce antibody speciﬁc to sleep red blood cell (SRBC), an antigen commonly used in the 1980s. More speciﬁcally, Gorczynski (1987) showed that conditioned reductions (due to pairings of a CS with CY) in IgG antibody–producing cells following immunization with SRBC were dependent upon modulation of T cells. Cohen, Ader, Green, and Bovbjerg (1979) utilized the CYsaccharin model to demonstrate conditioning of a thymus-independent, primarily humoral antibody response in mice following immunization with the hapten trinitrophenyl (TNP) coupled to lipopolysaccharide (LPS), a type-1, T cell–independent antigen. Conditioned animals exposed to saccharin exhibited a suppressed anti-TNP antibody response. Other studies failed to show alterations in antibody responses to type-2, thymus-independent antigens such as pneumococcal polysaccharide (Schulze, Benson, Paule, & Roberts, 1988), indicating that the ability to condition alterations in immunity may be dependent upon the type of antigen utilized for the activation of the immune response. In addition, these results may be speciﬁc to the US because low doses of CY have actually been shown to increase splenic antibody production against unrelated antigens (Portiansky, Gonzalez, & Laguens, 1996), suggesting that the drug is not exclusively immunosuppressive but may exhibit

09-Schachtman-09.indd 196

a diversity of immunomodulatory effects. In addition to the conditioned effects on antibody production, research has also shown effects on other immune processes. For example, pairing of a saccharin solution with CY administration results in a conditioned suppression of natural killer (NK) cell cytotoxicity upon the re-presentation of the saccharin alone (O’Reilly & Exon, 1986). NK cells are a component of the innate immune system and are involved in the destruction of tumor cells and cells infected by viruses. The activity of these cells is measured by their cytotoxicity, the ability to induce cell death in target cells via the release of specialized proteins. In addition, mice (Neveu et al., 1986) and rats (Kusnecov, Husband, & King, 1988) conditioned to a saccharin solution paired with CY exhibit reduced lymphocyte proliferation in the spleen in response to mitogens following exposure to the CS. Thus, there is evidence that conditioned stimuli based on CY as an unconditioned stimulus can induce widespread effects on immune status. There are several circumstances in which immunomodulation, speciﬁcally immunosuppression, is a desired clinical outcome, such as in the treatment of autoimmune disease, prevention of transplanted organ rejection, cancer chemotherapy, and the management of allergic conditions. CY’s immunosuppressive effects have been conditioned in a model of graft-versus-host response, an example of cell-mediated immunity that serves as an animal model of tissue transplant. In these experiments, exposure to the saccharin solution that had previously been paired with CY administration signiﬁcantly reduced the popliteal node weights of rats injected with donor splenic leukocytes in comparison to the control groups (Bovbjerg et al., 1982). Further reports indicated that this conditioned suppression is susceptible to extinction because animals given repeated exposure to the saccharin solution alone after conditioning no longer differed from controls in the graft versus host response (Bovbjerg, Ader, & Cohen, 1984). This was some of the ﬁrst evidence that conditioned immune alterations may be clinically signiﬁcant, because reducing the immune response to foreign cells is beneﬁcial in an individual receiving transplanted tissue.

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

Following the work by Bovberg and colleagues, another laboratory demonstrated that suppression of adjuvant-induced arthritis can be achieved following presentation of a saccharin solution that had been paired with CY (Klosterhalfen & Klosterhalfen, 1983). This study was important because it was the ﬁrst to show that the severity of an animal model of a human autoimmune disorder might be modulated by behavioral conditioning. Another experiment documenting the possible clinical relevance of conditioned immunomodulation was conducted for the treatment of systemic lupus erythmatosus. CY has been a commonly prescribed treatment for lupus because of its ability to extend survival and delay the onset of disease symptoms; however, because of its lack of toxic speciﬁcity the drug has numerous unwanted side effects. For this reason, it would be beneﬁcial to reduce the amount of medication administered while still sustaining the effects of the drug therapy. Lupus-prone female New Zealand mice were administered a saccharin solution once a week, with the experimental group receiving an injection of CY immediately following half of the saccharin administrations (Ader & Cohen, 1982). The animals showed signiﬁcant delays in the onset of proteinuria and mortality compared to unconditioned animals receiving the same amount of drug. These results were interesting because they provided evidence that a regimen of drug treatment that is by itself incapable of altering the course of an autoimmune disease may be used to delay its onset when combined with the presentation of a CS without producing the unwanted side effects of the drug. In addition, animals that were exposed to the CS after drug treatment had been discontinued survived longer than control animals that were presented with neither the CS nor the active drug (Ader & Cohen, 1982). This same experimental design was utilized in an actual clinical case study of a child with systemic lupus erythmatosus (Olness & Ader, 1992). The patient received administration of CY paired with both gustatory (cod liver oil) and olfactory (rose perfume) conditioned stimuli. Following the conditioning trials, the patient received half of the prescribed doses of CY, and the other half

09-Schachtman-09.indd 197

197

of the doses was replaced by exposure to the CS. Despite the reduction in active drug treatment, the patient showed clinical improvement. In line with this study, Goebel et al. (2002) demonstrated behaviorally conditioned immunosuppression in humans by pairing cyclosporine-A (CsA) with a distinctly ﬂavored beverage. When given the drink 1 week later with a placebo pill rather than active CsA treatment, the subjects still showed suppressed cytokine production and lymphocyte proliferation. CsA, like CY, is an immunomodulatory agent. However, unlike CY, CsA speciﬁcally suppresses T cell–mediated immune responses by binding to cyclophilins, thereby reducing the expression of the cytokines interleukin-2 and interferon-γ (Bukrinsky, 2002). Because the majority of the initial studies utilized the nonspeciﬁc immunosuppressive agent CY as the US, it began to be questioned whether the use of nonnoxious stimulus and/or a stimulus with more speciﬁc immunomodulatory action would still create a conditioned immunomodulation. To answer this question, Kusnecov et al. (1983) administered rabbit anti-rat lymphocyte serum, a biological immunosuppressant agent that speciﬁcally targets lymphocytes without altering other biological events. Animals receiving pairings of anti-rat lymphocyte serum with a saccharin solution that were subsequently reexposed to the saccharin alone exhibited reduced lymphocyte responsiveness to allogeneic spleen cells. These results demonstrated that conditioned regulation of the immune system is not restricted to nonspeciﬁc, noxious stimuli such as CY but extends to more targeted immunomodulatory stimuli. Conditioned immunomodulation involves the learning of an association between the neutral conditioned stimulus (CS, e.g., ﬂavored solution) and the immunomodulatory unconditioned stimulus (US, e.g., the immunomodulatory effects of cyclosporine-A), such that the CS may take on the immunomodulatory properties of the US. This learned association has so far proven similar to many other forms of learning and as such may involve similar brain structures. Though the exact neural mechanisms by which conditioned immunomodulation occurs is as yet poorly understood, some insights have

2/17/2011 11:17:18 AM

198

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

been gained. For example, lesion studies have shown a role for the insular cortex and in both the acquisition and expression of conditioned taste aversions (e.g., Garcia, Lasiter, BermudezRattoni, & Deems, 1985; Reilly & Schachtman, 2009), and thus the possible involvement of the region in conditioned immunosuppressive responses involving a gustatory CS was investigated. Ramirez-Amaya et al. (1996) created bilateral lesions of both the insular and parietal cortex prior to the pairing of CY with a gustatory CS. If either of these regions is involved in the acquisition of the association between the CS and the US, then the lesion will disrupt the expression of the conditioned response. Upon presentation of the CS alone, animals that had received sham lesions or lesions of the parietal cortex exhibited decreased antibody titers to an injection of SRBC as well as decreased IgM production following immunization with ovalbumin. These results suggest that the parietal cortex is not involved in the acquisition of the conditioned response since the response was still present following the lesioning of this area. However, the animals receiving insular cortex lesions did not show these reductions, suggesting a role for this brain region in the acquisition of conditioned immunomodulation involving a gustatory stimulus. Conversely, NMDA-induced lesions of the amygdala attenuated only the acquisition of conditioned immunosuppression in this model, evidence that the expression and acquisition of the conditioned effect may be mediated through different neural mechanisms (Ramirez-Amaya, varez-Borda, & Bermudez-Rattoni, 1998). While elucidating the neural mechanisms mediating conditioned immunomodulation is imperative, the search for the peripheral mediators of these effects is equally important. Evidence suggests a possible role for a number of potentially immunomodulatory chemical mediators released under the control of the nervous system. For example, the immunosuppressive effects of CsA on cytokine production can be conditioned to a gustatory stimulus and this conditioned effect is blocked by denervation of the spleen (Exton et al., 1998), suggesting involvement of the sympathetic nervous system. In addition, exposure to a stimulus previously paired with

09-Schachtman-09.indd 198

CsA increased survival rates in a model of graftversus-host disease possibly through the inhibition of cytokine production. Sympathetic denervation of the spleen blocked this effect, further suggesting that the conditioned effects of CsA are mediated by sympathetic innervation of immune organs such as the spleen (Exton et al., 1999). Interestingly, the conditioned suppression of a contact hypersensitivity response was not altered by denervation of the spleen (Exton et al., 2000), suggesting that there is no single mechanism through which conditioned immunomodulation occurs but that the mechanism may vary depending upon the immune parameters being studied and the US utilized.

CONDITIONING USING IMMUNOSTIMULATORY AGENTS A central question is whether one can condition a response to a speciﬁc antigen or to an immunoenhancing US. In the 1950s, researchers reported the ﬁrst controlled conditioning study in which antibody production was elicited as a CR (Zeitlenok & Bychkova, 1954). Rats received pairings of a tactile stimulus with the subsequent injection of inﬂuenza A virus. Exposure to the CS elicited an antibody production as the CR. Since these early investigations, several other laboratories have reproduced similar ﬁndings. For example, Jenkins and colleagues (1983) reported elicited antibody production as a CR using a protocol in which intraperitoneal injection of sheep red blood cells was employed as the US. Antibody titers to sheep red blood cells was signiﬁcantly elevated in conditioned animals compared to groups following exposure to the CS. Conditioned anybody production has also been demonstrated using keyhole limpet hemocyanin (KLH) as the US (Ader, Kelly, Moynihan, Grota, & Cohen, 1993). In these experiments, a gustatory stimulus (chocolate milk solution) was used as the CS and was followed by the immunization with KLH. Interestingly, the CR was manifest by a signiﬁcant enhancement of anti-KLH antibody production when the CS was combined with a small dose of antigen at the time of exposure. In yet another demonstration, an Australian group of investigators reported a conditioned

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

enhancement of antibody production following a conditioning protocol using ovalbumin (OVA) (Husband et al., 1993). In these studies rats received injection of OVA after the presentation of a saccharin solution as the CS. The results showed that anti-OVA IgG titers were higher in conditioned animals than in controls. Similarly, two independent laboratories demonstrated an increase in IgM and/or IgG antibodies in animals exposed to a CS that had previously been paired with the antigen hen egg lysosome (Alvarez-Bord, Ramirez-Amaya, Perez-Montfort, & BermudezRattoni, 1995; Madden et al., 2001). Collectively, the results of the study show that a CS can acquire the ability to regulate antibody production similar to that of the antigen used as the US. In related procedures, there have been a number of studies demonstrating that allergic reactions can also be conditioned to neutral stimuli. For example, Russell and colleagues (1984) paired an olfactory CS with bovine serum albumin. On test trials, reexposure to the olfactory CS resulted in a release of histamine, an important mediator of allergic reactions. Later investigations showed that this effect could be extinguished by the unreinforced exposure to the CS, indicating that the conditioned effect followed the principles of Pavlovian conditioning (Dark, Peeke, Ellman, & Salﬁ, 1987). Similarly, MacQueen and Siegel (1989) measured rat mast cell protease II in serum, an enzyme that indicates mast cell activation. Exposure to a CS that had been paired with egg albumin resulted in the elevation of this protease relative to baseline levels. These results, along with those of other laboratories, provide strong evidence that conditioning processes can regulate a speciﬁc mediator of mast cell function involved in allergic responses. Thus, the control of immune function by CS extends to a number of cell types. This claim is also supported by an extensive amount of research investigating the CR that developed when polyinosinic:polycytidylic (poly I:C) is used as the US. Poly I:C is a synthetic polyribonucleotide that mimics the action of a virus and increases NK cell activity and interferon production. In an extensive series of experiments, Hiramoto and colleagues demonstrated in mice that reexposure to an odor previously

09-Schachtman-09.indd 199

199

paired with poly I:C induces an increase in NK cell activity (e.g., Ghanta, Hiramoto, Solvason, & Spector, 1985; Solvason, Ghanta, Lorden, Soon, & Hiramoto, 1991). More recently, Hsueh and colleagues replicated the poly I:C induced conditioning of NK cell activity and extended it to demonstrate the conditioned enhancement of neutrophil activity (Chao, Hsu, Yuan, Jiang, & Hsueh, 2005; Hsueh, Chen, Lin, & Chao, 2002). These experiments clearly demonstrate that conditioning processes acting through the central nervous system can regulate speciﬁc immune cells in the periphery. The studies are clearly valuable to the understanding of how the central nervous system regulates the immune system, but surprisingly little is known about the exact mechanisms by which these diverse types of conditioning come to control speciﬁc immune responses. In other words, there is little known about the mechanisms by which these effects occur and what common factors may be involved in the effects. However, research has revealed several possible mechanisms by which these effects may be mediated. For example, both plasma ACTH and IFN-alpha in the spleen are increased during the expression of conditioned NK cell activity enhancement (Hsueh, Tyring, Hiramoto, & Ghanta, 1994). In addition, dopamine and norephinephrine were found to be increased in the brains of animals conditioned with immunologically relevant compounds, and central disruption of either the dopaminergic or noradrenergic systems blocked the conditioned increase in NK cell activity (Hsueh et al., 1999). Animals that had received pairings of ovalbumin with a saccharin solution exhibited increased antibody production and c-Fos expression in the insular cortex when exposed to the saccharin solution alone (Chen et al., 2004), suggesting that the brain area may be important for both conditioned immunoenhancement and conditioned immunosuppresssion using a gustatory stimulus.

CONDITIONING USING AVERSIVE STIMULI A major focus in the area of neuroimmune interactions is the effect of stressful stimuli on

2/17/2011 11:17:18 AM

200

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

immune function. This literature has provided some of the best evidence for the impact of psychological states on the immune system. For instance, chronic stress delays the healing of wounds in individuals responsible for the care of an Alzheimer’s patient, and it reduces the antibody response to the inﬂuenza vaccine (KiecoltGlaser, Marucha, Malarkey, Mercado, & Glaser, 1995; Vedhara et al., 1999). In animals, exposure to aversive stimuli has also been shown to have a profound impact on immune status. For example, mice subjected to loud noise exhibit alterations in cytotoxic responses mediated by lymphocytes and changes in splenic cell counts (Monjan & Collector, 1977). Furthermore, rats presented with electric footshocks display reductions in mitogeninduced lymphocyte proliferative responses (Keller, Weiss, Schleifer, Miller, & Stein, 1981, 1983; Lysle, Lyte, Fowler, & Rabin, 1987), NK cell cytotoxicity (Shavit, Lewis, Terman, Gale, & Liebeskind, 1984), and the production of antiKLH antibodies (Laudenslager et al., 1988). The investigation of the role of conditioning in the effects of stressful events has made signiﬁcant contributions to the literature in this area. First, the evaluation of the effects of an aversive conditioned stimulus on immune status provides an animal model of psychological stress. The conditioning model is one in which an aversive stimulus is presented that does not have physically aversive properties, but acquired this property by association. In other words, the aversive CS provides an excellent model of the stress that humans may experience which is more psychological in nature and does not necessarily involve physically aversive events. Moreover, conditioning models have been used to determine whether the immunological effects of aversive stimuli can be conditioned to neutral stimuli, and they have been used to investigate the neural mechanisms involved in neuroimmunomodulatory effects. Some of the earliest investigations were conducted in our laboratory and demonstrated that a stimulus previously paired with electric shock treatment will induce a number of immune alterations, including decreased NK cell activity, suppressed lymphocyte responsiveness to mitogens, decreased number of antibody-producing cells, and depressed cytokine

09-Schachtman-09.indd 200

production (Luecken & Lysle, 1992; Lysle, Cunnick, Fowler, & Rabin, 1988; Perez & Lysle, 1995). Furthermore, the conditioned response was attenuated by both pre- and postconditioning exposure to the CS (i.e., extinction and latent inhibition; Lysle et al., 1988). Taken together, these results indicate that stimuli previously associated with an aversive event or stimulus induce activation of neural circuits capable of modulating immune responses. Although these exact neural circuits are as yet unknown, research has identiﬁed speciﬁc regions of the brain that might be involved in the expression of conditioned immunomodulation to aversive stimuli. Pezzone et al. (1992, 1993) demonstrated an increase in c-Fos immunoreactivity in the paraventricular nucleus of the hypothalamus, the periaquaductal gray, the serotoninergic neurons of the dorsal raphe nuclei, and the noradrenergic neurons of the locus ceruleus in response to exposure to conditioned aversive stimuli. These data suggest involvement of the autonomic nervous system in the mediation of conditioned behaviors to aversive stimuli, possibly including conditioned alterations in immunity. In support of this ﬁnding, previous research from our laboratory shows that the suppression of splenic mitogenic responsiveness observed following exposure to a conditioned aversive stimulus can be blocked by administration of the beta-adrenergic receptor antagonist, propanolol (Lysle, Cunnick, & Maslonek, 1991). The conditioned suppression of splenic T cell proliferation and Con-A induced gamma-interferon production is dose dependently attenuated by administration of either selective beta 1- or beta 2-receptor antagonists. However, the fact that neither antagonist affected the conditioned suppression of NK cell activity, blood lymphocyte mitogenic responsiveness, LPS-induced B cell proliferation, or interleukin-2 production suggests that multiple mechanisms might be at play. In fact, conditioned immunomodulation to aversive stimuli such as electric shock was found to also involve the endogenous opioid system as evidenced by the ability of naltrexone to block the conditioned response (Lysle, Luecken, & Maslonek, 1992; Perez & Lysle, 1997). Opioid receptors have been identiﬁed on immune cells,

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

providing a direct mechanism of action for immunomodulation via endogenous opioids (Carr et al., 1989; Madden, Donahoe, Zwemer-Collins, Shafer, & Falek, 1987; Wybran, Appelboom, Famaey, & Govaerts, 1979). However, Lysle and Coussons-Read (1995) demonstrated that the conditioned effects of aversive stimuli on NK cell activity and proliferative responses of splenocytes are modulated via centrally located opioid receptors. In this experiment, animals received an injection of naltrexone or N-methylnaltrexone prior to exposure to the conditioned stimulus that had previously been paired with an aversive event. N-methylnaltrexone is a quaternary form of naltrexone that does not readily cross the blood– brain barrier. The inability of N-methylnaltrexone to block the conditioned effects of an aversive stimulus on immunity indicates that central, rather than peripherally located receptors are involved in these effects. More speciﬁcally, reductions in immunity are mediated by mu-opioid receptors because intracerebroventricular administration of a mu-1-selective opioid receptor antagonist, but not a kappa- or delta-selective antagonist, blocks electrical shock-induced conditioned alterations in immunity (Perez & Lysle, 1997). Collectively, these results indicate that aversive conditioned stimuli modulate immune function through an opioid mechanism. This ﬁnding helped lead to the investigation of the conditioned effects of exogenous opioids on immune status, which is discussed in the next section.

CONDITIONING USING DRUGS OF ABUSE Several investigators have shown that many of the physiological and behavioral responses to drugs of abuse, including changes in immune functioning, may be conditioned to stimuli predictive of drug availability. For example, environmental stimuli that had previously been paired with morphine administration can elicit such morphine-like effects as hyperthermia when presented in the absence of morphine (Bardo & Valone, 1994; Eikelboom & Stewart, 1979; Miksic, Smith, Numan, & Lal, 1975; Schwarz & Cunningham, 1990; Wikler & Pescor, 1967).

09-Schachtman-09.indd 201

201

Similar results have been seen with the conditioning of morphine’s analgesic properties such that exposure to morphine-associated stimuli induces a conditioned analgesic response in the absence of drug (Bardo & Valone, 1994; Miller, Kelly, Neiswander, McCoy, & Bardo, 1990). There has been increasing evidence from studies conducted on human subjects showing that drug-paired stimuli can cause intense craving, feelings of being “high,” galvanic skin responses, autonomic arousal, and altered neural activity in drug users (Ehrman, Robbins, Childress, & O’Brien, 1992; Foltin & Haney, 2000; Sideroff & Jarvik, 1980). In fact, exposure to drug cues is one of the key contributing factors to relapse mainly because these cues induce a wide variety of complex, classically conditioned physical and behavioral responses (Derbas & al-Haddad, 2001; Unnithan, Gossop, & Strang, 1992). Current research in the area of drug abuse has focused on the critical task of reducing relapse rates by attempting to reduce cue-induced drugseeking behavior; however, it is also imperative to take into account the widespread effects these cues may have on immune function and the potential effects on resistance to infectious diseases. Our laboratory provided some of the ﬁrst evidence that morphine-induced alterations of immune status can be conditioned to environmental stimuli that have been paired with morphine administration (Coussons, Dykstra, & Lysle, 1992; Coussons-Read, Dykstra, & Lysle, 1994a, 1994b). In these investigations rats were given subcutaneous injections of morphine upon placement into a distinctive environment. Following reexposure to the previously morphine-paired environment in the absence of drug, animals exhibited alterations in immune parameters similar to those observed with actual morphine administration, including decreased lymphocyte responsiveness to mitogens, reduced interleukin-2 production, and suppressed activity of splenic NK cells. More recently we have demonstrated that heroin’s immunomodulatory effects may also be conditioned to environmental stimuli (Lysle & Ijames, 2002). Heroin (diacetylmorphine) is a semisynthetic derivative of morphine that produces many of the same

2/17/2011 11:17:18 AM

202

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

biological effects as morphine but also possesses certain properties that distinguish it from morphine. Heroin has a more rapid onset of action when compared to morphine (Kaiko, Wallenstein, Rogers, Grabinski, & Houde, 1981; Scott & Orr, 1969). Early reports indicate that heroin produces a greater degree of euphoria and fewer side effects than morphine (Dundee, Loan, & Clarke, 1966; Seevers & Pfeiffer, 1936; Twycross, 1973). Differences in potency between heroin and morphine also are well documented. For example, heroin is as much as 16 times more potent than morphine in producing reinforcing effects in animals (Harrigan & Downs, 1978; van Ree, Slangen, & De, 1978) and subjective effects in humans (Jasinski & Nutt, 1972; Kaiko et al., 1981; Martin & Fraser, 1961; Seevers & Pfeiffer, 1936). The high prevalence of opportunistic infections among heroin users has long suggested that heroin use impairs the ability of the host to combat infectious disease (Hussey & Katz, 1950; Louria, Hensle, & Rose, 1967; Luttgens, 1949). Heroin has been shown to alter a number of basic immune parameters, including NK cell activity, circulating lymphocyte numbers, antibody-dependent cellular cytotoxicity, cytokine production, and nitric oxide production (Bencsics, Elenkov, & Vizi, 1997; Chao, Molitor, Close, Hu, & Peterson, 1993; Govitrapong, Suttitum, Kotachabhakdi, & Uneklabh, 1998; Lysle & How, 2000; Nair, Laing, & Schwartz, 1986; Paciﬁci, di Carlo, Bacosi, Pichini, & Zuccaro, 2000). Nitric oxide is a critical component of the innate immune system, and the effects of heroin on nitric oxide have been shown to be susceptible to conditioning with environmental stimuli. Produced by many cells of the immune system, this molecule mediates diverse biological functions, including vasodilation, the cytotoxic activity of macrophages, and the inhibition of platelet adhesion and aggregation (Suschek, Schnorr, & Kolb-Bachofen, 2004; Tuteja, Chandra, Tuteja, & Misra, 2004). Nitric oxide is a gaseous molecule with a short half-life, making it difﬁcult to measure directly. However, nitric oxide may be quantiﬁed indirectly by analyzing the expression of inducible nitric oxide synthase (iNOS), one of the enzymes responsible

09-Schachtman-09.indd 202

for the synthesis of nitric oxide. Our laboratory has demonstrated that the effects of heroin on nitric oxide may be conditioned to environmental stimuli predictive of drug availability. In these investigations rats received subcutaneous injections of heroin (1 mg/kg) upon placement into a distinctive environment. When rats were subsequently reexposed to the environment without heroin administration, the production of nitric oxide was suppressed to a level similar to that observed upon actual heroin administration (Lysle & Ijames, 2002). Further experimentation revealed that the suppressive effects of heroin on the proinﬂammatory cytokines, TNF-α and IL1-β, may also be conditioned to environmental stimuli (Szczytkowski & Lysle, 2008). Similar to nitric oxide, interleukin-1 beta (IL-1β) and tumor necrosis factor-alpha (TNF-α) both play critical roles in the body’s defense against infectious challenge. Both IL-1β and TNF-α are proinﬂammatory cytokines produced primarily by activated macrophages. When secreted, IL-1β and TNF-α have widespread effects in the host, including altering the thermoregulatory setting in the hypothalamus to produce fever and increasing the expression of adhesion factors on endothelial cells to promote the transmigration of leukocytes to sites of infection. Recently it has been shown that conditioned immunomodulation by drugs of abuse is not conﬁned to opioids; in fact, the impact of other drugs of abuse on immune function may also be conditioned. For example, Kubera et al. (2008) demonstrated that rats presented with a cue that had previously been paired with cocaine administration exhibit changes in select measures of immune status, including splenocyte proliferation and cytokine production. These same effects, among others, that were not found to be susceptible to conditioning, were exhibited following active administration of the drug. The susceptibility of heroin-induced conditioned immune alterations to the effects of extinction and latent inhibition demonstrate that this conditioning paradigm follows accepted principles of learning. To test this, both pre- and postconditioning exposure to the conditioned stimulus (i.e., conditioning chamber) signiﬁcantly reduced the conditioned response

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

09-Schachtman-09.indd 203

iNOS copy number

20000

HC Control CS Exposed

15000

10000

5000

0 Conditioned

15000 iNOS copy number

(Szczytkowski & Lysle, 2007). In these experiments, rats received ﬁve training sessions in which heroin was administered immediately upon placement into a standard conditioning chamber which served as the CS. For the extinction study, animals were reexposed to the conditioning chamber without further drug administration for 10 consecutive days following the ﬁnal conditioning session to produce extinction. For the latent inhibition study, animals received 10 consecutive days of nonreinforced preexposure to the chamber to induce latent inhibition. The results from these studies are shown in Figure 9.2. Overall, these data suggest that the conditioned effects of heroin on nitric oxide involve a learned association between the drug and the environment in which it is delivered. These results are important, not only because they provide the ﬁrst demonstration that immunologic alterations can be conditioned to drug cues paired with heroin administration but also because they have profound implications for opioid–immune interactions. These results suggest that former and current opiate users may be more susceptible to infection not only because of the direct action of opiates on target tissues but also because their body associates previously drugpaired cues with changes in immunity. One key implication is that the host response to drugs of abuse is not static but changes as the organism learns to predict the administration of the drug on the basis of distinctive stimuli in the environment. Human brain imaging data have shown changes in neural activity within speciﬁc brain regions following exposure to drug cues in former drug users (Sell et al., 2000). Several investigators have demonstrated a role of the basolateral amygdala (BLA) in associative learning and speciﬁcally in the formation of new, and in the utilization of already established, stimulus–reward associations in drug abuse models. In addition, research has demonstrated an increase in c-Fos expression in the lateral habenula, basolateral amygdala complex, prelimbic cortex, and nucleus accumbens core in rats exposed to drugassociated CS (Miller & Marshall, 2005; Zhang et al., 2005). Animals receiving morphine paired with a distinct environment exhibit Fos protein

203

Conditioned + Extinction

HC Control CS Exposed

10000

5000

0 Conditioned

Conditioned + Latent inhibition

Figure 9.2 Effect of extinction (top) and latent inhibition (bottom) procedures on the expression of heroin-induced conditioned immunomodulation. Animals exposed to the conditioned stimulus (CS) on test day (white bars) exhibit signiﬁcantly lower levels of inducible nitric oxide synthase (iNOS) than home cage controls (black bars). The expression of this conditioned response is attenuated by both the extinction and latent inhibition procedures (second group of bars in each panel). The data are expressed as iNOS copy number per 10 ng cDNA based on a standard curve using Roche LightCycler software. The error bars represent the standard error of the mean.

expression, a marker of neural activation, within both the BLA and the central nucleus of the amygdala during a test of conditioned place preference (Harris & Aston-Jones, 2003). Similarly, cocaine-associated cues elicited neural activity within the BLA nearly identical to that seen upon intravenous delivery of cocaine in rats taught to self-administer (Carelli, Williams, & Hollander, 2003). Inactivation of the BLA inhibits the reacquisition of heroin-conditioned place preference (Rizos, Ovari, & Leri, 2005) and abolishes the

2/17/2011 11:17:18 AM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

expression of CS-induced reinstatement of heroin-seeking behavior in rats (Fuchs & See, 2002). Recent investigations in our laboratory have provided evidence that inactivation of the BLA can reduce or even eliminate the conditioned suppressive effects of heroin on nitric oxide induction and on the expression of the proinﬂammatory cytokines, TNF-α and IL-1β (Szczytkowski & Lysle, 2008). In these experiments animals received intra-BLA infusion of a combination of GABA agonists to temporarily inactivate the region prior to testing of the conditioned response. Moreover, to ensure that these results are speciﬁc to the BLA, a control experiment was conducted in which an area of the caudate situated dorsal to the BLA was inactivated. The inactivation of this region of the caudate did not produce an attenuation of the conditioned response, demonstrating that the results are speciﬁc to the BLA. The dopaminergic system within the BLA has been shown to be involved in learning and memory and speciﬁcally the associative learning that underlies classical conditioning. For example, microdialysis studies revealed an increase in dopamine and its metabolites in the BLA during the learning of a discriminative task in rats (Hori, Tanaka, & Nomura, 1993). Dopaminergic signaling within the BLA appears to be particularly important for the acquisition and expression of conditioned responses to drugs of abuse. Sitespeciﬁc administration of dopamine antagonists into the BLA, but not the central amygdala, blocks alcohol-induced conditioned place preference in mice (Gremel & Cunningham, 2008). In addition, activation of dopamine D1 receptors in the BLA was found to be necessary for the expression of cue-induced reinstatement of cocaineseeking behavior in rats (See, Kruzich, & Grimm, 2001). There is also evidence for a differential involvement of dopamine receptor subtypes within the BLA in the acquisition and expression of responses to conditioned drug cues. In contrast to the role of D1 receptors, D2 receptors in the BLA appear to be involved in the acquisition of associations between drugs and the conditioned cues that guide subsequent cocaine-seeking behavior (Berglind et al., 2006). Recently we have shown that antagonism of dopamine D1,

09-Schachtman-09.indd 204

but not D2, receptors within the BLA attenuates the conditioned effects of heroin on nitric oxide (see Fig. 9.3). In these experiments animals received intra-BLA microinfusions of the dopamine D1 selective antagonist, SCH23390, prior to reexposure to the CS on test day. D1 receptor antagonism blocked the conditioned reduction in nitric oxide. Furthermore, both conditioned and unconditioned effects of morphine appear to be mediated via dopamine D1 receptors in the nucleus accumbens, as antagonism of these receptors block morphine- and morphine-associated stimuli induced suppression of NK cell activity (Saurer, Carrigan, Ijames, & Lysle, 2006a; Saurer, Ijames, Carrigan, & Lysle, 2008). Pyramidal cell glutamatergic projections from the BLA have been shown to synapse in close proximity to dopamine axons on medium spiny neurons of the nucleus accumbens (Johnson, Aylward, Hussain, & Totterdell, 1994; Kelley, Domesick, & Nauta, 1982; Robinson & Beart, 1988), thus establishing a possible neural circuit through

40000

iNOS copy number

204

HC Control CS Exposed

30000

20000

10000

0 Saline

SCH23390

Figure 9.3 Intraamygdala administration of a selective D1 receptor antagonist attenuates the expression of heroin-induced conditioned (HC) immunomodulation. Animals reexposed to the conditioned stimulus (CS) on test day (white bars) exhibit a signiﬁcant suppression of inducible nitric oxide synthase (iNOS) mRNA levels in the spleen as compared to control animals (black bars). However, animals receiving intra-BLA microinfusions of the SCH23390 compound (second set of bars) do not exhibit this reduction. Data are expressed as iNOS copy number per 10 ng cDNA based on a standard curve using Roche LightCycler software. The error bars represent the standard error of the mean.

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

which opiate-induced conditioned immunomodulation may be mediated. While the exact peripheral mechanisms modulating the alterations in proinﬂammatory mediators observed in response to exposure to drug-associated cues are unknown, research has revealed potential roles for several neuroendocrine factors. For example, both the conditioned and unconditioned effects of opiates on the cytotoxicity of NK cells appear to be mediated by peripheral neuropeptide Y, a peptide transmitter coreleased with norepinephrine by the sympathetic nervous system (Saurer, Ijames, & Lysle, 2006; Saurer et al., 2008). In addition, the expression of a subset of morphine-induced conditioned immune alterations are blocked by the systemic administration of the nonselective peripheral β-adrenergic receptor antagonist nadolol (Coussons-Read et al., 1994b), further suggesting a role for the sympathetic nervous system in the mediation of these conditioned effects. Both the acquisition and expression of morphine-induced conditioned suppression of immune functioning are attenuated by antagonism of opioid receptors, suggesting that endogenous opioids play a role in opioid-induced conditioned immunomodulation (Coussons-Read et al., 1994a). Cocaine-associated cues induce alterations in immune functioning, including TNF-α production, and these alterations accompany elevations in corticosterone levels, suggesting the HPA axis may be involved in drug cue–mediated conditioned immunomodulation (Kubera et al., 2008). The continued search for the exact peripheral mediators modulating the conditioned effects of opioid drugs on immune parameters is critical for understanding how to control these effects in recovering drug users and reduce the susceptibility to infection in these individuals.

CONCLUSIONS In summary, there is now substantial evidence that conditioning processes can play a major role in the regulation of the immune response. Systematic studies have shown that a wide variety of immunomodulatory agents can be employed as unconditioned stimuli for the conditioning of

09-Schachtman-09.indd 205

205

immunomodulatory properties to originally neutral stimuli. Research is now beginning to determine the neural, endocrine, and immune mechanisms involved in this wide array of conditioned effects. Furthermore, a number of studies have shown that conditioned immunomodulation can have a signiﬁcant impact on disease outcome. Future studies are required for a more complete understanding of the mechanisms involved in the conditioned effects and to explore the potential clinical relevance of the effects and the beneﬁcial use of conditioning in the clinical setting.

REFERENCES Ader, R. (1974). Letter: Behaviorially conditioned immunosuppression. Psychosomatic Medicine, 36, 183–184. Ader, R. (2000). On the development of psychoneuroimmunology. European Journal of Pharmacology, 405, 167–176. Ader, R., & Cohen, N. (1975). Behaviorally conditioned immunosuppression. Psychosomatic Medicine, 37, 333–340. Ader, R., & Cohen, N. (1982). Behaviorally conditioned immunosuppression and murine systemic lupus erythematosus. Science, 215, 1534–1536. Ader, R., Cohen, N., & Bovbjerg, D. (1982). Conditioned suppression of humoral immunity in the rat. Journal of Comparative and Physiological Psychology, 96, 517–521. Ader, R., Cohen, N., & Grota, L. J. (1979). Adrenal involvement in conditioned immunosuppression. International Journal of Immunopharmacology, 1, 141–145. Ader, R., Kelly, K., Moynihan, J. A., Grota, L. J., & Cohen, N. (1993). Conditioned enhancement of antibody production using antigen as the unconditioned stimulus. Brain, Behavior, and Immunity, 7, 334–343. Alvarez-Borda, B., Ramirez-Amaya, V., PerezMontfort, R., & Bermudez-Rattoni, F. (1995). Enhancement of antibody production by a learning paradigm. Neurobiology of Learning and Memory, 64, 103–105. Bach, J. F. (1975). The mode of action of immunosuppressive agents. Frontiers of Biology, 41, 1–374. Bardo, M. T., & Valone, J. M. (1994). Morphineconditioned analgesia using a taste cue: Dissociation of taste aversion and analgesia. Psychopharmacology (Berling), 114, 269–274.

2/17/2011 11:17:18 AM

206

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Beishuizen, A., & Thijs, L. G. (2003). Endotoxin and the hypothalamo-pituitary-adrenal (HPA) axis. Journal of Endotoxin Research, 9, 3–24. Benscsics, A., Elenkov, I. J., & Vizi, E. S. (1997). Effect of morphine on lipopolysaccharide induced tumor necrosis factor alpha in vivo: Involvement of the sympathetic nervous system. Journal of Neuroimmunology, 73(1–2), 1–6. Berglind WJ, Case JM, Parker MP, Fuchs RA, See RE. (2006). Dopamine D1 or D2 receptor antagonism within the BLA differentially alters the acquisition of cocaine-cue associations necessary for cue-induced reinstatement of cocaineseeking. Neuroscience, 137(2):699–706. Besedovsky, H., Sorkin, E., Felix, D., & Haas, H. (1977). Hypothalamic changes during the immune response. European Journal of Immunology, 7, 323–325. Besedovsky, H. O., del Rey A., Klusman, I., Furukawa, H., Monge, A. G., & Kabiersch, A. (1991). Cytokines as modulators of the hypothalamus-pituitary-adrenal axis. Journal of Steroid Biochemitry and Molecular Biology, 40, 613–618. Bluthe, R. M., Walter, V., Parnet, P., Laye, S., Lestage, J., Verrier, D.,…Dantzer, R. (1994). Lipopolysaccharide induces sickness behaviour in rats by a vagal mediated mechanism. Comptes Rendus de l’Academie des Sciences – Series III, 317, 499–503. Bovbjerg, D., Ader, R., & Cohen, N. (1982). Behaviorally conditioned suppression of a graftversus-host response. Proceedings of the National Academy of Sciences USA, 79, 583–585. Bovbjerg, D., Ader, R., & Cohen, N. (1984). Acquisition and extinction of conditioned suppression of a graft-vs-host response in the rat. Journal of Immunology, 132, 111–113. Bukrinsky, M. (2002). Cyclophilins: Unexpected messengers in intercellular communications. Trends in Immunology, 23, 323–325. Carelli, R. M., Williams, J. G., & Hollander, J. A. (2003). Basolateral amygdala neurons encode cocaine self-administration and cocaineassociated cues. Journal of Neuroscience, 23, 8204–8211. Carr, D. J., DeCosta, B. R., Kim, C. H., Jacobson, A. E., Guarcello, V., Rice, K. C., & Blalock, J. E. (1989). Opioid receptors on cells of the immune system: Evidence for delta- and kappa-classes. Journal of Endocrinology, 122, 161–168. Chao, C. C., Molitor, T. W., Close, K., Hu, S., & Peterson, P. K. (1993). Morphine inhibits the

09-Schachtman-09.indd 206

release of tumor necrosis factor in human peripheral blood mononuclear cell cultures. International Journal of Immunopharmacology, 15, 447–453. Chao, H. J., Hsu, Y. C., Yuan, H. P., Jiang, H. S., & Hsueh, C. M. (2005). The conditioned enhancement of neutrophil activity is catecholamine dependent. Journal of Neuroimmunology, 158, 159–169. Chen, J., Lin, W., Wang, W., Shao, F., Yang, J., Wang, B.,…Ju, G. (2004). Enhancement of antibody production and expression of c-Fos in the insular cortex in response to a conditioned stimulus after a single-trial learning paradigm. Behavior and Brain Research, 154, 557–565. Cohen, N., Ader, R., Green, N., & Bovbjerg, D. (1979). Conditioned suppression of a thymusindependent antibody response. Psychosomatic Medicine, 41, 487–491. Coussons, M. E., Dykstra, L. A., & Lysle, D. T. (1992). Pavlovian conditioning of morphineinduced alterations of immune status. Journal of Neuroimmunology, 39, 219–230. Coussons-Read, M. E., Dykstra, L. A., & Lysle, D. T. (1994a). Pavlovian conditioning of morphineinduced alterations of immune status: Evidence for opioid receptor involvement. Journal of Neuroimmunology, 55, 135–142. Coussons-Read, M. E., Dykstra, L. A., & Lysle, D. T. (1994b). Pavlovian conditioning of morphineinduced alterations of immune status: Evidence for peripheral beta-adrenergic receptor involvement. Brain, Behavior, and Immunity, 8, 204–217. Crabtree, G. R., Gillis, S., Smith, K. A., & Munck, A. (1980). Mechanisms of glucocorticoid-induced immunosuppression: Inhibitory effects on expression of Fc receptors and production of T-cell growth factor. Journal of Steroid Biochemistry and Molecular Biology, 12, 445–449. Dantzer, R., Bluthe, R. M., Laye, S., Bret-Dibat, J. L., Parnet, P., & Kelley, K. W. (1998). Cytokines and sickness behavior. Annals of the New York Academy of Sciences, 840, 586–590. Dark, K., Peeke, H. V., Ellman, G., & Salﬁ, M. (1987). Behaviorally conditioned histamine release. Prior stress and conditionability and extinction of the response. Annals of the New York Academy of Sciences, 496, 578–582. Dekker, E., & Groen, J. (1956). Reproducible psychogenic attacks of asthma; A laboratory study. Journal of Psychosomatic Research, 1, 58–67. Derbas, A. N., & al-Haddad, M. K. (2001). Factors associated with immediate relapse among

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

Bahraini heroin abusers. Eastern Mediterranean Health Journal, 7, 473–480. Dinarello, C. A. (1999). Cytokines as endogenous pyrogens. Journal of Infectious Diseases., 179 (Suppl. 2), S294–S304. Dundee, J. W., Loan, W. B., & Clarke, R. S. (1966). Studies of drugs given before anaesthesia. XI. Diamorphine (heroin) and morphine. British Journal of Anaesthesiology, 38, 610–619. Dunn, A. J., Powell, M. L., Meitin, C., & Small, P. A., Jr. (1989). Virus infection as a stressor: Inﬂuenza virus elevates plasma concentrations of corticosterone, and brain concentrations of MHPG and tryptophan. Physiology and Behavior, 45, 591–594. Ehrman, R. N., Robbins, S. J., Childress, A. R., & O’Brien, C. P. (1992). Conditioned responses to cocaine-related stimuli in cocaine abuse patients. Psychopharmacology (Berlin), 107, 523–529. Eikelboom, R., & Stewart, J. (1979). Conditioned temperature effects using morphine as the unconditioned stimulus. Psychopharmacology (Berlin), 61, 31–38. Elkins, R. (1974). Conditioned ﬂavor aversions to familiar tap water in rats: An adjustment with implications for aversion therapy treatment of alcholism and obestity. Journal of Abnormal Psychology, 83, 411–417. Exton, M. S., Elfers, A., Jeong, W. Y., Bull, D. F., Westermann, J., & Schedlowski, M. (2000). Conditioned suppression of contact sensitivity is independent of sympathetic splenic innervation. American Journal of Physiology - Regulatory, Integrative, and Comparative Physiology, 279, R1310–R1315. Exton, M. S., Schult, M., Donath, S., Strubel, T., Bode, U., del Rey, A., … Schedlowski, M. (1999). Conditioned immunosuppression makes subtherapeutic cyclosporin effective via splenic innervation. American Journal of Physiology, 276, R1710–R1717. Exton, M. S., von, H. S., Schult, M., Voge, J., Strubel, T., Donath, S., … Schedlowski, M. (1998). Behaviorally conditioned immunosuppression using cyclosporine A: Central nervous system reduces IL-2 production via splenic innervation. Journal of Neuroimmunology, 88, 182–191. Felten, D. L., Livnat, S., Felten, S. Y., Carlson, S. L., Bellinger, D. L., & Yeh, P. (1984). Sympathetic innervation of lymph nodes in mice. Brain Research Bulletin, 13, 693–699.

09-Schachtman-09.indd 207

207

Foltin, R. W., & Haney, M. (2000). Conditioned effects of environmental stimuli paired with smoked cocaine in humans. Psychopharmacology (Berlin), 149, 24–33. Fuchs, B. A., Campbell, K. S., & Munson, A. E. (1988). Norepinephrine and serotonin content of the murine spleen: Its relationship to lymphocyte beta-adrenergic receptor density and the humoral immune response in vivo and in vitro. Cellular Immunology, 117, 339–351. Fuchs, R. A., & See, R. E. (2002). Basolateral amygdala inactivation abolishes conditioned stimulus- and heroin-induced reinstatement of extinguished heroin-seeking behavior in rats. Psychopharmacology (Berlin), 160, 425–433. Garcia, J., & Koelling, R. A. (1967). A comparison of aversions induced by x-rays, toxins, and drugs in the rat. Radiation Research Supplement, 7, 439–450. Garcia, J., Lasiter, P. S., Bermudez-Rattoni, F., & Deems, D. A. (1985). A general theory of aversion learning. Annals of the New York Academy of Sciences, 443, 8–21. Ghanta, V. K., Hiramoto, R. N., Solvason, H. B., & Spector, N. H. (1985). Neural and environmental inﬂuences on neoplasia and conditioning of NK activity. Journal of Immunology, 135, 848s–852s. Goebel, M. U., Trebst, A. E., Steiner, J., Xie, Y. F., Exton, M. S., Frede, S., … Schedlowski, M. (2002). Behavioral conditioning of immunosuppression is possible in humans. FASEB Journal, 16, 1869–1873. Goehler, L. E., Gaykema, R. P., Nguyen, K. T., Lee, J. E., Tilders, F. J., Maier, S. F., & Watkins, L. R. (1999). Interleukin-1beta in immune cells of the abdominal vagus nerve: A link between the immune and nervous systems? Journal of Neuroscience, 19, 2799–2806. Gorczynski, R. M. (1987). Analysis of lymphocytes in, and host environment of, mice showing conditioned immunosuppression to cyclophosphamide. Brain, Behavior, and Immunity, 1, 21–35. Govitrapong, P., Suttitum, T., Kotchabhakdi, N., & Uneklabh, T. (1998). Alterations of immune functions in heroin addicts and heroin withdrawal subjects. Journal of Pharmacology and Experimental Therapeutics, 266, 417–423. Gremel, C. M., & Cunningham, C. L. (2008). Involvement of amygdala dopamine and nucleus accumbens NMDA receptors in ethanol-seeking behavior in mice. Neuropsychopharmaocology, 34, 1443–1453.

2/17/2011 11:17:18 AM

208

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Hadden, J. W., Hadden, E. M., & Middleton, E. Jr. (1970). Lymphocyte blast transformation. I. Demonstration of adrenergic receptors in human peripheral lymphocytes. Cellular Immunology, 1, 583–595. Harrigan, S. E., & Downs, D. A. (1978). Selfadministration of heroin, acetylmethadol, morphine, and methadone in rhesus monkeys. Life Sciences, 22, 619–623. Harris, G. C., & Aston-Jones, G. (2003). Enhanced morphine preference following prolonged abstinence: Association with increased Fos expression in the extended amygdala. Neuropsychopharmacology, 28, 292–299. Hill, L. (1930). Philosophy of a biologist. London, England: Arnold. Hori, K., Tanaka, J., & Nomura, M. (1993). Effects of discrimination learning on the rat amygdala dopamine release: A microdialysis study. Brain Research, 621, 296–300. Hsueh, C. M., Chen, S. F., Lin, R. J., & Chao, H. J. (2002). Cholinergic and serotonergic activities are required in triggering conditioned NK cell response. Journal of Neuroimmunology, 123, 102–111. Hsueh, C. M., Kuo, J. S., Chen, S. F., Huang, H. J., Cheng, F. C., Chung, L. J., & Lin, R. J. (1999). Involvement of catecholamines in recall of the conditioned NK cell response. Journal of Neuroimmunology, 94, 172–181. Hsueh, C. M., Tyring, S. K., Hiramoto, R. N., & Ghanta, V. K. (1994). Efferent signal(s) responsible for the conditioned augmentation of natural killer cell activity. Neuroimmunomodulation., 1, 74–81. Husband, A., Lin, W., Madsen, G., & King, M. G. (1993). A conditioning model for immunostimulation: Enhancement of the antibody responses to ovalbumin by behavioral conditioning in rats. In (Husband, AJ, editor), Psychoimmunology (pp. 139–148). Boca Raton, FL: CRC Press. Hussey H. H., & Katz, S. (1950). Infections resulting from narctotic addiction: Report of 102 cases. American Journal of Medicine, 9, 186–993. Imeri, L., & Opp, M. R. (2009). How (and why) the immune system makes us sleep. Nature Reviews Neuroscience, 10, 199–210. Jasinski, D. R., & Nutt, J. G. (1972). Report of the Thirty-Fourth Annual Scientiﬁc Meeting Committee on Problems of Drug Dependence, Michigan. Ann Arbor, :National Institute on Mental Health.

09-Schachtman-09.indd 208

Jenkins, P., Chadwick, R., & Nevin, J. (1983). Classically conditioned enhancement of antibody production. Bulletin of the Psychonomic Society, 21, 485–487. Johnson, L. R., Aylward, R. L., Hussain, Z., & Totterdell, S. (1994). Input from the amygdala to the rat nucleus accumbens: Its relationship with tyrosine hydroxylase immunoreactivity and identiﬁed neurons. Neuroscience, 61, 851–865. Kaiko, R. F., Wallenstein, S. L., Rogers, A., Grabinski, P., & Houde, R. W. (1981). Relative analgesic potency of intramuscular heroin and morphine in cancer patients with postoperative pain and chronic pain due to cancer. NIDA Research Monographs, 34, 213–219. Keller, S. E., Weiss, J. M., Schleifer, S. J., Miller, N. E., & Stein, M. (1981). Suppression of immunity by stress: Effect of a graded series of stressors on lymphocyte stimulation in the rat. Science, 213, 1397–1400. Keller, S. E., Weiss, J. M., Schleifer, S. J., Miller, N. E., & Stein, M. (1983). Stress-induced suppression of immunity in adrenalectomized rats. Science, 221, 1301–1304. Kelley, A. E., Domesick, V. B., & Nauta, W. J. (1982). The amygdalostriatal projection in the rat—an anatomical study by anterograde and retrograde tracing methods. Neuroscience, 7, 615–630. Kiecolt-Glaser, J. K., Marucha, P. T., Malarkey, W. B., Mercado, A. M., & Glaser, R. (1995). Slowing of wound healing by psychological stress. Lancet, 346, 1194–1196. Klosterhalfen, W., & Klosterhalfen, S. (1983). Pavlovian conditioning of immunosuppression modiﬁes adjuvant arthritis in rats. Behavioral Neuroscience, 97, 663–666. Kubera, M., Filip, M., Budziszewska, B., BastaKaim, A., Wydra, K., Leskiewicz, M., … Lason, W. (2008). Immunosuppression induced by a conditioned stimulus associated with cocaine selfadministration. J.Pharmacol.Sci., 107, 361–369. Kusnecov, A. W., Husband, A. J., & King, M. G. (1988). Behaviorally conditioned suppression of mitogen-induced proliferation and immunoglobulin production: Effect of time span between conditioning and reexposure to the conditioning stimulus. Brain, Behavior, and Immunity, 2, 198–211. Kusnecov, A. W., Sivyer, M., King, M. G., Husband, A. J., Cripps, A. W., & Clancy, R. L. (1983). Behaviorally conditioned suppression of the

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

immune response by antilymphocyte serum. Journal of Immunology, 130, 2117–2120. Laudenslager, M. L., Fleshner, M., Hofstadter, P., Held, P. E., Simons, L., & Maier, S. F. (1988). Suppression of speciﬁc antibody production by inescapable shock: Stability under varying conditions. Brain, Behavior, and Immunity, 2, 92–101. Louria, D. B., Hensle, T., & Rose, J. (1967). The major medical complication of heroin addiction. Annals of Internal Medicine, 67, 1–27. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition: The effect of nonreinforced pre-exposure to the conditional stimulus. Journal of Comparative and Physiological Psychology, 52, 415–419. Luecken, L. J., & Lysle, D. T. (1992). Evidence for the involvement of beta-adrenergic receptors in conditioned immunomodulation. Journal of Neuroimmunology, 38, 209–219. Luttgens, W. F. (1949). Endocarditis in “main line” opium addicts. Archives of Internal Medicine, 83, 653–664. Lysle, D. T., & Coussons-Read, M. E. (1995). Mechanisms of conditioned immunomodulation. International Journal of Immunopharmacology, 17, 641–647. Lysle, D. T., Cunnick, J. E., Fowler, H., & Rabin, B. S. (1988). Pavlovian conditioning of shockinduced suppression of lymphocyte reactivity: Acquisition, extinction, and preexposure effects. Life Science, 42, 2185–2194. Lysle, D. T., Cunnick, J. E., & Maslonek, K. A. (1991). Pharmacological manipulation of immune alterations induced by an aversive conditioned stimulus: Evidence for a beta-adrenergic receptor-mediated Pavlovian conditioning process. Behavioral Neuroscience, 105, 443–449. Lysle, D.T., & How, T. (2000). Heroin modulates the expression of inducible nitric oxide synthase. Immunopharmacology, 46, 181–192. Lysle, D. T., & Ijames, S. G. (2002). Heroin-associated environmental stimuli modulate the expression of inducible nitric oxide synthase in the rat. Psychopharmacology (Berlin), 164, 416–422. Lysle, D. T., Luecken, L. J., & Maslonek, K. A. (1992). Modulation of immune status by a conditioned aversive stimulus: Evidence for the involvement of endogenous opioids. Brain, Behavior, and Immunity, 6, 179–188. Lysle, D. T., Lyte, M., Fowler, H., & Rabin, B. S. (1987). Shock-induced modulation of lymphocyte reactivity: Suppression, habituation, and recovery. Life Sciences, 41, 1805–1814.

09-Schachtman-09.indd 209

209

MacKenzie, J. (1886). The production of the so-called “rose cold” by means of an artiﬁcial rose. American Journal of Medical Science, 9, 45–57. MacQueen, G. M., & Siegel, S. (1989). Conditional immunomodulation following training with cyclophosphamide. Behavioral Neuroscience, 103, 638–647. Madden, J. J., Donahoe, R. M., Zwemer-Collins, J., Shafer, D. A., & Falek, A. (1987). Binding of naloxone to human T lymphocytes. Biochemical Pharmacology, 36, 4103–4109. Madden, K. S., Boehm, G. W., Lee, S. C., Grota, L. J., Cohen, N., & Ader, R. (2001). One-trial conditioning of the antibody response to hen egg lysozyme in rats. Journal of Neuroimmunology, 113, 236–239. Martin, W. R., & Fraser, H. F. (1961). A comparative study of physiological and subjective effects of heroin and morphine administered intravenously in postaddicts. Journal of Pharmacology and Experimental Therapeutics, 133, 388–399. McCoy, D. F., Roszman, T. L., Miller, J. S., Kelly, K. S., & Titus, M. J. (1986). Some parameters of conditioned immunosuppression: Species difference and CS-US delay. Physiology and Behavior, 36, 731–736. McPherson, G. A., & Summers, R. J. (1982). Characterization and localization of [3H]clonidine binding in membranes prepared from guinea-pig spleen. Clinical and Experimental Pharmacology and Physiology, 9, 77–87. Metal’nikov, S., & Chorine, V. (1926). Role des reﬂexes conditionnels dans l’immunite. Annals L’Institute Pasteur, 40, 893–900. Metal’nikov, S., & Chorine, V. (1928). Role des reﬂexes conditionnels dans la formation des anticorps. Comptes Rendus des Seances de la Societe de Biologie et de Ses Filiales, 102, 133–134. Miksic, S., Smith, N., Numan, R., & Lal, H. (1975). Acquistion and extinction of a conditioned hyperthermic response to a tone paired with morphine administration. Neuropsychobiology, 1, 277–283. Miller, C. A., & Marshall, J. F. (2005). Altered Fos expression in neural pathways underlying cueelicited drug seeking in the rat. European.Journal of Neuroscience, 21, 1385–1393. Miller, J. S., Kelly, K. S., Neisewander, J. L., McCoy, D. F., & Bardo, M. T. (1990). Conditioning of morphine-induced taste aversion and analgesia. Psychopharmacology (Berlin), 101, 472–480.

2/17/2011 11:17:18 AM

210

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Monjan, A. A., & Collector, M. I. (1977). Stressinduced modulation of the immune response. Science, 196, 307–308. Moore, M. J. (1991). Clinical pharmacokinetics of cyclophosphamide. Pharmacokinet, 20, 194–208. Nair, M. P. N., Laing, T. J., & Schwartz, S. A. (1986). Decreased natural and antibody-dependent cellular cytotoxic activities in intravenous drug users. Clinical Immunology, 38, 68–78. Neveu, P. J., Dantzer, R., & Le, M. M. (1986). Behaviorally conditioned suppression of mitogen-induced lymphoproliferation and antibody production in mice. Neuroscience Letters, 65, 293–298. O’Reilly, C. A., & Exon, J. H. (1986). Cyclophosphamide-conditioned suppression of the natural killer cell response in rats. Physiology and Behavior, 37, 759–764. Olness, K., & Ader, R. (1992). Conditioning as an adjunct in the pharmacotherapy of lupus erythematosus. Journal of Developmental and Behavioral Pediatrics, 13, 124–125. Onsrud, M., & Thorsby, E. (1981). Inﬂuence of in vivo hydrocortisone on some human blood lymphocyte subpopulations. I. Effect on natural killer cell activity. Scandinavian Journal of Immunology, 13, 573–579. Ovaskainen, Y., Koponen, H., Jokelainen, J., Keinanen-Kiukaanniemi, S., Kumpusalo, E., & Vanhala, M. (2009). Depressive symptomatology is associated with decreased interleukin-1 beta and increased interleukin-1 receptor antagonist levels in males. Psychiatry Research, 167, 73–79. Paciﬁci, R., di Carlo, S., Bacosi, A., Pichini, S., Zuccaro, P. (2000). Pharmacokinetics and cytokine production in heroin and morphinetreated mice. International Journal of Immunopharmacology, 22, 603–614. Pavlov, I. (1928). Lectures on conditioned reﬂexes. New York, NY: Liveright. Perez, L., & Lysle, D. T. (1995). Corticotropinreleasing hormone is involved in conditioned stimulus-induced reduction of natural killer cell activity but not in conditioned alterations in cytokine production or proliferation responses. Journal of Neuroimmunology, 63, 1–8. Perez, L., & Lysle, D. T. (1997a). Conditioned immunomodulation: Investigations of the role of endogenous activity at mu, kappa, and delta opioid receptor subtypes. Journal of Neuroimmunology, 79, 101–112.

09-Schachtman-09.indd 210

Pezzone, M. A., Lee, W. S., Hoffman, G. E., Pezzone, K. M., & Rabin, B. S. (1993). Activation of brainstem catecholaminergic neurons by conditioned and unconditioned aversive stimuli as revealed by c-Fos immunoreactivity. Brain Research, 608, 310–318. Pezzone, M. A., Lee, W. S., Hoffman, G. E., & Rabin, B. S. (1992). Induction of c-Fos immunoreactivity in the rat forebrain by conditioned and unconditioned aversive stimuli. Brain Research, 597, 41–50. Pineno, O., & Miller, R. R. (2005). Primacy and recency effects in extinction and latent inhibition: A selective review with implications for models of learning. Behavioural Processes, 69, 223–235. Portiansky, E. L., Gonzalez, P. H., & Laguens, R. P. (1996). Increase of murine splenic natural antibody-secreting cells after cyclophosphamide treatment. Clinical Immunology and Immunopathology, 81, 215–217. Ramirez-Amaya, V., varez-Borda, B., & BermudezRattoni, F. (1998). Differential effects of NMDA-induced lesions into the insular cortex and amygdala on the acquisition and evocation of conditioned immunosuppression. Brain, Behavior, and Immunity, 12, 149–160. Ramirez-Amaya, V., Alvarez-Borda, B., Ormsby, C. E., Martinez, R. D., Perez-Montfort, R., & BermudezRattoni, F. (1996). Insular cortex lesions impair the acquisition of conditioned immunosuppression. Brain, Behavior, and Immunity, 10, 103–114. Reilly, S., & Schachtman, T.R. (2009) Conditioned taste aversions: Behavioral and neural mechanisms. New York, NY: Oxford University Press. Rescorla, R. A. (1988). Behavioral studies of Pavlovian conditioning. Annual Review of Neuroscience, 11, 329–352. Rizos, Z., Ovari, J., & Leri, F. (2005). Reconditioning of heroin place preference requires the basolateral amygdala. Pharmacology Biochemistry and Behavior, 82, 300–305. Robinson, T. G., & Beart, P. M. (1988). Excitant amino acid projections from rat amygdala and thalamus to nucleus accumbens. Brain Research Bulletin, 20, 467–471. Rogers, M. P., Reich, P., Strom, T. B., & Carpenter, C. B. (1976). Behaviorally conditioned immunosuppression: Replication of a recent study. Psychosomatic Medicine, 38, 447–451.

2/17/2011 11:17:18 AM

CONDITIONED IMMUNOMODULATION

Russell, M., Dark, K. A., Cummins, R. W., Ellman, G., Callaway, E., & Peeke, H. V. (1984). Learned histamine release. Science, 225, 733–734. Sanders, V. M., & Munson, A. E. (1985). Role of alpha adrenoceptor activation in modulating the murine primary antibody response in vitro. Journal of Pharmacology and Experimental Therapy, 232, 395–400. Saurer, T. B., Carrigan, K. A., Ijames, S. G., & Lysle, D. T. (2006). Suppression of natural killer cell activity by morphine is mediated by the nucleus accumbens shell. Journal of Neuroimmunology, 173, 3–11. Saurer, T. B., Ijames, S. G., Carrigan, K. A., & Lysle, D. T. (2008). Neuroimmune mechanisms of opioid-mediated conditioned immunomodulation. Brain, Behavior, and Immunity, 22, 89–97. Saurer, T. B., Ijames, S. G., & Lysle, D. T. (2006). Neuropeptide Y Y1 receptors mediate morphine-induced reductions of natural killer cell activity. Journal of Neuroimmunology, 177, 18–26. Schulze, G. E., Benson, R. W., Paule, M. G., & Roberts, D. W. (1988). Behaviorally conditioned suppression of murine T-cell dependent but not T-cell independent antibody responses. Pharmacology, Biochemistry and Behavior., 30, 859–865. Schwarz, K. S., & Cunningham, C. L. (1990). Conditioned stimulus control of morphine hyperthermia. Psychopharmacology (Berlin), 101, 77–84. Scott, M. E., & Orr, R. (1969). Effects of diamorphine, methadone, morphine, and pentazocine in patients with suspected acute myocardial infarction. Lancet, 1, 1065–1067. See, R. E., Kruzich, P. J., & Grimm, J. W. (2001) Dopamine, but not glutamate, receptor blockade in the BLA attenuates conditioned reward in a rat model of relapse to cocaine-seeking behavior. Psychopharmacology (Berlin), 154, 301–310. Seevers, M. H., & Pfeiffer, C. C. (1936). A study of the analgesia, subjective depression, and euphoria produced by morphine, heroine, dilaudid and codeine in the normal human subject Journal of Pharmacology and Experimental Therapeutics, 56, 166–187. Sell, L. A., Morris, J. S., Bearn, J., Frackowiak, R. S., Friston, K. J., & Dolan, R. J. (2000). Neural responses associated with cue evoked emotional

09-Schachtman-09.indd 211

211

states and heroin in opiate addicts. Drug and Alcohol Dependence, 60, 207–216. Shand, F. L. (1979). The immunopharmacology of cyclophosphamide. International Journal of Immunopharmacology, 1, 165–171. Shavit, Y., Lewis, J. W., Terman, G. W., Gale, R. P., & Liebeskind, J. C. (1984). Opioid peptides mediate the suppressive effect of stress on natural killer cell cytotoxicity. Science, 223, 188–190. Sideroff, S. I., & Jarvik, M. E. (1980). Conditioned responses to a videotape showing heroin-related stimuli. International Journal of Addiction, 15, 529–536. Solvason, H. B., Ghanta, V. K., Lorden, J. F., Soong, S. J., & Hiramoto, R. N. (1991). A behavioral augmentation of natural immunity: Odor speciﬁcity supports a Pavlovian conditioning model. International Journal of Neuroscience, 61, 277–288. Straub, R. H., & Besedovsky, H. O. (2003). Integrated evolutionary, immunological, and neuroendocrine framework for the pathogenesis of chronic disabling inﬂammatory diseases. FASEB Journal, 17, 2176–2183. Suschek, C. V., Schnorr, O., & Kolb-Bachofen, V. (2004). The role of iNOS in chronic inﬂammatory processes in vivo: Is it damage-promoting, protective, or active at all? Current Molecular Medicine, 4, 763–775. Szczytkowski, J. L., & Lysle, D. T. (2007). Conditioned effects of heroin on the expression of inducible nitric oxide synthase in the rat are susceptible to extinction and latent inhibition. Psychopharmacology (Berlin), 191, 879–889. Szczytkowski, J. L., & Lysle, D. T. (2008). Conditioned effects of heroin on proinﬂammatory mediators require the basolateral amygdala. European Journal of Neuroscience., 28, 1867–1876. Tuteja, N., Chandra, M., Tuteja, R., & Misra, M. K. (2004). Nitric oxide as a unique bioactive signaling messenger in physiology and pathophysiology. Journal of Biomedicine and Biotechnology, 2004, 227–237. Twycross, R. G. (1973). Stumbling blocks in the study of diamorphine. Postgraduate Medical Journal, 49, 309–313. Unnithan, S., Gossop, M., & Strang, J. (1992). Factors associated with relapse among opiate addicts in an out-patient detoxiﬁcation programme. British Journal of Psychiatry, 161, 654–657. van Ree, J. M., Slangen, J. L., & De, W. D. (1978). Intravenous self-administration of drugs in rats.

2/17/2011 11:17:19 AM

212

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Journal of Pharmacology and Experimental Therapeutics, 204, 547–557. Vedhara, K., Cox, N. K., Wilcock, G. K., Perks, P., Hunt, M., Anderson, S., … Shanks, N. M. (1999). Chronic stress in elderly carers of dementia patients and antibody response to inﬂuenza vaccination. Lancet, 353, 627–631. Vishwanath, R. (1996). The psychoneuroimmunological system: A recently evolved networking organ system. Medical Hypotheses, 47, 265–268. Wayner, E. A., Flannery, G. R., & Singer, G. (1978). Effects of taste aversion conditioning on the primary antibody response to sheep red blood cells and Brucella abortus in the albino rat. Physiology and Behavior, 21, 995–1000. Wikler, A., & Pescor, F. T. (1967). Classical conditioning of a morphine abstinence phenomenon, reinforcement of opioid-drinking behavior and “relapse” in morphine-addicted rats. Psychopharmacologia., 10, 255–284. Williams, J. M., Peterson, R. G., Shea, P. A., Schmedtje, J. F., Bauer, D. C., & Felten, D. L. (1981). Sympathetic innervation of murine

09-Schachtman-09.indd 212

thymus and spleen: Evidence for a functional link between the nervous and immune systems. Brain Research Bulletin, 6, 83–94. Williams, L. T., Snyderman, R., & Lefkowitz, R. J. (1976). Identiﬁcation of beta-adrenergic receptors in human lymphocytes by (-) (3H) alprenolol binding. Journal of Clinical Investigation 57, 149–155. Wybran, J., Appelboom, T., Famaey, J. P., & Govaerts, A. (1979). Suggestive evidence for receptors for morphine and methionine-enkephalin on normal human blood T lymphocytes. Journal of Immunology, 123, 1068–1070. Zeitlenok, N. A., & Bychkova, E. N. (1954). [Role of the higher nervous function in infection and immunity.]. Zhurnal vyssheĭ nervnoĭ deiatelnosti imeni I P Pavlova, 4, 267–281. Zhang, F., Zhou, W., Liu, H., Zhu, H., Tang, S., Lai, M., & Yang, G. (2005). Increased c-Fos expression in the medial part of the lateral habenula during cue-evoked heroin-seeking in rats. Neuroscience Letters, 386, 133–137.

2/17/2011 11:17:19 AM

CHAPTER 10 Learning, Expectancy, and Behavioral Control Implications for Drug Abuse Muriel Vogel-Sprott and Mark T. Fillmore

This chapter reviews research demonstrating that learned expectancies mediate behavior. A drug-taking situation illustrates how associative learning develops drug-related expectancies. Experiments are described to show that manipulating individual expectancies can reveal their causal inﬂuence on the intensity of the drug effect and the type of behavioral response to the drug. These results are also related to other evidence that implicates behavioral disinhibition and impulsivity in the risk for drug abuse. Evidence is presented to show how a drug user’s expectancy about the disinhibiting effects of a drug can alter the response to the drug. Taken together, the ﬁndings provide new information on how drug-related expectancies affect basic mechanisms of behavioral control, and they offer new insights into how expectancies can mediate the well-known association between disinhibition and risk for drug abuse.

FOREWORD Regrettably, Dr. Muriel Vogel-Sprott passed away shortly after completing the ﬁnal draft of this chapter. Dr. Vogel-Sprott’s research on the behavioral effects of alcohol spanned over 50 years, producing nearly 200 scientiﬁc publications, two books, and numerous chapters. Her ideas and scientiﬁc ﬁndings have had considerable impact on our understanding of the behavioral effects of alcohol and the factors that lead to alcohol abuse. A major thrust of her work concerned the question of why a drinker’s behavioral tolerance can vary, and why the drinker’s history of alcohol exposure does not necessarily predict the amount of behavioral tolerance displayed. Her work was guided by the theory that environmental events known to affect learning of new behaviors would also inﬂuence tolerance, and she proposed that learned expectancies about the behavioral effects of alcohol affected the degree of tolerance that a drinker displayed. Her experiments provided convincing demonstrations that

learned expectancies played a critical role in the development of alcohol tolerance. Such evidence for the role of learning was groundbreaking because it challenged the traditional assumption that tolerance was based solely on the prior exposure to alcohol. Her work also had considerable social relevance. By showing how intoxicated behavior responded to environmental consequences in the same fashion as sober behavior, her work challenged the social view that intoxication can somehow excuse antisocial and other problem behaviors. In many respects this chapter represents the culmination of Dr. Vogel-Sprott’s research, for which I am grateful to have been a part, both as her former student and as her colleague. Mark T. Fillmore

INTRODUCTION Evidence that addictive drugs can activate basic biological properties common to all people has led some investigators to ask why all drug users 213

10-Schachtman-10.indd 213

2/15/2011 7:45:08 PM

214

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

do not become addicted (Altman et al., 1996). The effects of the socially used drug, alcohol, raise a similar puzzle because the drug can affect different people in different ways, and it can even affect the same person differently on separate occasions (Kerr & Hindmarch, 1998). Basic neurobiological processes or constitutional and personal traits of an individual may inﬂuence the behavioral effect of alcohol; but they are constant factors that seem unlikely to explain why the excessive use of the drug occurs on some occasions and not on others, or why the behavioral effect of alcohol on an individual varies in different situations, or suddenly changes within an individual during a given occasion. Behavioral shifts toward sobriety have been frequently reported during police assessments of impairment in drinking drivers (e.g., Goldberg & Harvard, 1968; Langenbucher & Nathan, 1983; Wells, Green, Foss, Ferguson, & Williams, 1997). It appears that symptoms of alcohol-induced intoxication become difﬁcult to detect when sobriety is advantageous. More generally, it seems that the expected outcome for behavior under alcohol may inﬂuence the effects of the drug. “Expectancy” refers to the anticipation of some event. If “X” is reliably followed by “Y,” then the occurrence of “X” will result in the expectation of “Y.” The concept of expectancy was ﬁrst introduced to account for research ﬁndings in animal learning (Tolman, 1932). When learning is viewed as a process of gathering information about relationships between events, expectancy is an intervening explanatory variable that represents this information. The acquisition of information conveyed by an expectancy is attributed to associative learning processes of classical and instrumental conditioning (Bolles, 1972, 1979). Bolles identiﬁed two sources of expectancies: the association of a neutral stimulus with a biologically important stimulus (S-S∗), and the association of a response with a biologically important stimulus outcome (R-S∗). The R-S∗ expectancy was considered to have a motivating, guiding inﬂuence on behavior. Animal studies have provided support for these assumptions (e.g., Colwill & Rescorla, 1986; Rescorla, 1990). When events are arranged to permit the acquisition of an expectancy, behavior in the

10-Schachtman-10.indd 214

situation is predicted by this expectancy. Learned expectancies also are retained and function to guide behavior in other environments that are perceived to be similar. Much research by cognitive psychologists has investigated the occurrence and nature of individuals’ self-reported expectancies about alcohol. The results indicate that people report expectancies about receiving alcohol, the types of effects the expectancies exert, and consequences of these effects (e.g., Brown, 1993). An expectancy based on information about the association between events in the environment apparently can be acquired through ﬁrst-hand experience, by verbal instruction, or by observing other drug users (Goldman, Brown, & Christiansen, 1987). A particularly important ﬁnding is that people who report expecting more pleasant effects of alcohol also report higher consumption of the drug (Goldman, Del Boca, & Darkes, 1999). These correlations are consistent with the prediction that expectancies can guide behavior, and with the suspicion that alcohol-related expectancies might be associated with the risk of excessive drinking (Marlatt & Gordon, 1985). However, correlations cannot demonstrate a causal inﬂuence of drug-related expectancies. A different experimental design is needed in which the content of an expectancy is manipulated and the resulting response is measured. Such research can test the implication that differences among individuals in their alcohol expectancies could have broad effects, ranging from performance under the drug to alcohol abuse. This chapter reviews the research ﬁndings on the causative effects of expectancies on these aspects of alcohol-related behavior. Additional details on the experimental procedures and results of manipulating alcohol-related expectancies have been presented elsewhere (VogelSprott, 1992, 2004). We begin by illustrating how the expectancy model can be applied to identify the S-S∗ and R-S∗ expectancies in a drug-taking situation.

DRUG-RELATED EXPECTANCIES A learning analysis of a drug-taking situation focuses on the stimuli and responses in the drug

2/15/2011 7:45:08 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

setting in order to identify associations between speciﬁc events that give rise to particular expectancies. The model involves four events that could occur regularly when an individual uses a drug. These events are illustrated in Figure 10.1 and identiﬁed as some environmental stimulus cue for the drug (S), a drug stimulus (S∗d), a response to the drug (Rd), and an environmental outcome (S∗). The events occur in this strict temporal order and their repetition in a drugtaking situation provides an opportunity to associate three successive pairs of events, yielding three different expectancies. The ﬁrst pair of events consist of a stimulus (S) that precedes the drug stimulus (S∗d). This (S- S∗d) association provides an opportunity to learn what stimulus event predicts the drug. When this relationship is reliable, the stimulus (S) functions as a cue to signal the administration of the drug (S∗d) and the individual expects the drug when this cue is presented. The drinking environment provides a rich source of cues for alcohol, ranging from the setting of a bar, bottles of beer, and the scent of alcohol in a beverage. The second expectancy is based on the association between a drug stimulus, (S∗d), and the

Events:

Pairs of Events:

Cue for drug

S

Drug stimulus

S*d

Expectancies:

Receiving a drug

Type of effect Response to drug

Rd Consequence

Outcome

S*

Figure 10.1 Model of a drug-taking situation

consisting of some environmental stimulus cue (S) preceding the drug stimulus (S∗d). The drug is followed in turn by a behavioral response (Rd) and some environmental outcome (S∗). Events accompanied by an asterisk identify stimuli that have some impact that is important to an individual. This symbolization follows that of Bolles (1972), who used the asterisk to identify biologically important stimuli, such as food or shock.

10-Schachtman-10.indd 215

215

response to the drug, (Rd). The widespread action of a drug allows it to affect various biological, affective, and behavioral reactions. The ongoing activity of an individual in the situation determines the particular type of behavioral response associated with the drug effect. This relationship, S∗d- Rd, provides an opportunity to learn what type of behavioral effect the drug exerts. Repeated pairing of these events generates an expected effect of the drug that corresponds to the effect the drug has on the activity. For example, repeated success in playing darts after a few beers should lead a person to expect that alcohol improves this activity. Figure 10.1 shows the last pair of events consist of the response to the drug (Rd) and its environmental outcome (S∗). When this Rd - S∗ association is reliable, it conveys information that leads to a third expectancy: the expected consequence of the response to a drug. Of course, the particular outcome depends on the environmental context. Thus, a person can learn to expect that a given response leads to a favorable outcome in one situation, and a very unfavorable one in another situation. In theory, the expected consequence (S∗) affects the occurrence of the Rd response, making it more likely to be displayed when its expected outcome is desirable. For example, drinkers might expect alcohol to impair their driving skill. However, if they also expect that resisting this impairment yields a more desirable (safer) outcome, they may increase their care and attention to driving and exhibit less impairment than would otherwise be displayed. Thus, the expected consequence may function adaptively to modify the nature or intensity of the response to a drug in order to maximize the desired outcome. Up to this stage, our discussion of the threeexpectancy model has concerned predicting the behavioral response to a drug. But the model applies to placebo responses as well. When some particular response to a drug is expected, such as impairment, then that response should also be observed when a person expects the drug but a placebo is substituted. This is because the placebo provides the expectation of receiving the drug (S- S∗d) that leads to the expected drug effect (S∗d-Rd) and the expected consequence (Rd-S∗).

2/15/2011 7:45:08 PM

216

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

The behavioral effect of a placebo should be consistent with the expected effect of alcohol whenever the expected outcome of the behavior is favorable. It is also important to note that drugopposite, better performance under a placebo could also occur whenever the behavioral effect of the drug is expected to yield undesirable consequences. For example, a drinker who expects unfavorable consequences for impaired performance under alcohol may attempt to compensate by increasing effort and attention to the task. This would result in better than normal performance under a placebo because the impairing effect of the drug is absent. These examples serve to emphasize that a person’s response to cues for a drug is not solely determined by expecting the drug; the response also can be inﬂuenced by the other two expectancies. When these expectancies are learned, the presentation of a placebo with cues for a drug gives rise to these other expectancies. Their occurrence should affect the response to a placebo in a fashion similar to that shown under the drug. If alcohol is expected to impair behavior, performance under a placebo should become poorer. However, if resisting impairment is expected to yield a more favorable outcome, performance under a placebo might improve. In summary, the expectation of receiving a drug (S- S∗d) is commonly included as a control in studies of drug effects so that the pharmacological inﬂuence of the drug can be distinguished from the effect of simply expecting the drug. However, our expectancy model shown in Figure 10.1 reveals that the expectation of receiving a drug is a prerequisite for the occurrence of two other expectancies (S∗d-Rd and Rd-S∗) about the type of drug effect and the outcome of the response to a drug. The sequence of events shows how their occurrence could lead to the acquisition of three different expectancies and how the last event in one expectancy provides the cue for the next expectation so that they act sequentially to inﬂuence the response to a drug or a placebo. Much research on placebos has focused on the expectation of receiving a drug. The identiﬁcation of the expected type of effect and the expected consequence is a novel contribution of the learning analysis, and this chapter focuses

10-Schachtman-10.indd 216

largely on these two expectancies and their effects on psychomotor and cognitive performance. Expected Type of Effect (S∗d -Rd)

A number of reviews have assessed research on the effect of a moderate dose of alcohol on simple and complex psychomotor skills and cognitive tasks (e.g., Holloway, 1995). The results of these studies typically reveal that the performance of social drinkers under alcohol tends to be degraded and impaired, but the degree of impairment differs to some extent among individuals. There is some reason to suspect that alcohol-related expectancies could contribute to individual differences in response to the drug. The normal course of social drinking could provide an individual with many opportunities to experience some alcohol-induced impairment in cognitive and motor activities. This association between the drug stimulus and the response to the drug, illustrated by S∗d- Rd in Figure 10.1, provides the basis for learning to expect some drug-induced effect on performance. However, the particular activities performed will not be identical for all individuals. Therefore, drinkers may differ in the degree of alcohol impairment they expect on any given task. The expectancy model predicts that individual differences in the expected degree of alcohol impairment should determine the degree of impairment displayed: Drinkers who expect more impairment should perform more poorly under alcohol when there are no consequences of impairment. Various tasks, ranging from complex psychomotor skills to speed of information processing, have been used to test the relation between social drinkers’ expected and actual impairment under alcohol (Fillmore, Carscadden, & Vogel-Sprott, 1998; Fillmore & Vogel-Sprott, 1994, 1995c). To remove and control the inﬂuence of the expected consequence of alcohol effects on performance, participants completed trials on a task alone in a laboratory room where no environmental event (i.e., S∗) was associated with performance. After they had learned the task, they rated the expected effect of a moderate dose of alcohol on their performance (e.g., two beers drunk in an hour), using a scale that ranged from extreme impairment to

2/15/2011 7:45:08 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

extreme improvement. The drinking habits of the participants did not differ, but their ratings of the expected alcohol effect varied considerably, ranging from slight improvement to moderate impairment. Participants subsequently performed the task under a moderate dose of alcohol (0.62 g/kg), and other groups performed under a placebo, or no-beverage. The results showed that the drinker’s expected degree of impairment predicted the amount of impairment displayed under alcohol, and those who expected greater impairment from alcohol performed more poorly under a placebo. The no-beverage treatment excluded alcohol-related expectancies and therefore eliminated any relation between a drinker’s expected impairment and performance in the no-treatment group. The contribution of the expected type of effect to individual differences in motor skill and cognitive functioning under alcohol may be quite general because this S∗d- Rd expectancy has been shown also to inﬂuence the response to combining caffeine with alcohol, to caffeine alone, and to antidepressants, such as Prozac (Fillmore & Vogel-Sprott, 1992, 1994, 1995b; Kirsch & Saperstein, 1999; Kirsch & Weixel, 1988). Some studies indicate that individuals can hold expectancies about the effect of alcohol without any personal use of the drug (e.g., Christiansen, Goldman, & Inn, 1982). The accuracy of those S∗d-Rd expectancies is not known, but the learning model predicts that the accuracy should increase with drinking experience. When social drinking commences, the personal experience of drug effects provides information that allows the acquisition of an expected type of effect which more closely corresponds to the person’s own response to the drug. The strengthening of this experience should depend in part on the repetition and length of time that a drinker has been using alcohol regularly. This possibility has been tested in a study that classiﬁed social drinkers in terms of the time they had been drinking regularly (Fillmore & VogelSprott, 1995a). A group of “novice” drinkers had been drinking for 20 months or less, and an “experienced” group had been drinking socially for 24 months or more. The groups did not differ in age or in drinking habits, and all participants

10-Schachtman-10.indd 217

217

were familiarized with a motor skill before they rated the degree to which they expected alcohol (0.56 g/kg) to impair their task performance. The results showed that experienced drinkers who expected more impairment performed more poorly under alcohol. In contrast, no relationship was obtained between the expected and actual alcohol impairment of novice drinkers. These ﬁndings indicate that individuals require some history of drinking experience in order to develop a predictive relationship between their expected and actual behavioral effect of alcohol. These associations between expectancies and drug-related behavior are consistent with the assumption that learned expectancies mediate this behavior. But correlations cannot exclude the possibility that some other factor is responsible for these expectancy–behavior relationships. To demonstrate a causal effect, the expected type of drug effect must be manipulated experimentally and shown to change behavior. One method of altering the expected type of effect is to present new information about the drug effect to the drug user. This association is represented by (S∗d- Rd) in Figure 10.1. Several studies have used verbal information to manipulate the expected type of drug effect on motor skills (Fillmore, Mulvihill, & Vogel-Sprott, 1994; Fillmore & Vogel-Sprott, 1992). In this research, participants were well trained on the task before any information about drug effects was introduced. To exclude the pharmacological effects of a drug, two groups of participants received alcohol placebos, and two other groups received caffeine placebos. The groups expecting caffeine were informed that it would either enhance or impair their motor skill. Similarly, the groups expecting alcohol were told that the drug would either enhance or impair their task performance. Tests were conducted with no outcome for performance to ensure the absence of an expected consequence of performance (Rd -S∗). Manipulating the expected type of drug effect changed the task performance of each group. The caffeine-placebo groups expecting enhancement displayed better performance than did those expecting impairment. The performance of the alcohol-placebo groups also altered, but those expecting enhancement under alcohol

2/15/2011 7:45:08 PM

218

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

performed more poorly and those expecting impairment showed improvement. Although better task performance by the alcohol-placebo group that expected alcoholinduced impairment might seem counterintuitive, these observations could be attributed to an expectation of a more favorable outcome for resisting the impairing effect of the drug. This is the third expectancy in our model (Rd - S∗). Even though the research setting provided no environmental outcome for task performance (i.e., no S∗), it is possible that drinkers still entered the experiment with the expectation that resisting alcohol impairment of a motor skill is generally appropriate and desirable. If this is the case, they should also resist impairment induced by the actual administration of alcohol in this experimental setting that withheld an outcome for performance. Additional research has conﬁrmed this prediction (Fillmore & Vogel-Sprott, 1996). Accordingly, when alcohol was received, those led to expect impairment performed better (i.e., showed less impairment) than those who had received no information about impairment. This research also showed that individuals differed in the degree to which the expected type of alcohol effects could be manipulated. Social drinkers who had been using alcohol regularly for a longer period of time were less susceptible to the expectancy manipulation and less likely to change their response to alcohol. This resistance to change is consistent with the strengthening effect of practice on a response and suggests that alcohol-related expectancies of more experienced drinkers might be more difﬁcult to alter with new information. Research in this section was designed to assess the behavioral effect of the expected type of response to a drug. To demonstrate the separate inﬂuence of this particular expectancy, the studies provided no environmental consequence for behavior under the drug. This did eliminate the expectation of an environmental outcome for response to drug in the research setting, but the results showed that previously acquired expected outcomes that seem relevant to a new setting may still be retained and applied to tasks that appear similar (e.g., Fillmore & Vogel-Sprott, 1996). A situation in which the type of response to alcohol has no consequence is also a rather

10-Schachtman-10.indd 218

artiﬁcial and rare situation because drinkers usually engage in a variety of activities that could have important consequences. Expected Outcome (Rd -S∗)

The association between a response to alcohol (Rd) and its outcome (S∗) provides an opportunity to learn what consequences to expect for the response. Manipulating the value of the outcome has a reliable effect on the response, and the motivation for the response is attributed to the incentive value of the expected consequence (e.g., Dickinson & Balleine, 1995). In our model of expectancies the Rd–S∗ relationship predicts the display of a particular response to a drug depends on the expected presence or absence of a favorable outcome. To test this hypothesis, the experimental procedure is designed to hold constant the expectancy of receiving the drug (S-S∗d) and the drug effect (S∗d-Rd), and to only manipulate the outcome of the response to a drug. The expectation of a desirable consequence for resisting alcohol-induced impairment may be learned when some outcome, like money or verbal praise, is reliably contingent on unimpaired performance. The effect of this expectancy should be revealed by a reduction or complete absence of task impairment under a moderate dose of the drug. When complex psychomotor tasks, like driving, require some learned skills, research has shown that the effect of the newly acquired response–outcome expectancy is exhibited gradually with task practice under a few doses of alcohol (e.g., Vogel-Sprott, 1992). These results bear a resemblance to the gradual reduction in the intensity of alcohol effects that are observed with habitual drug use. This phenomenon, referred to as tolerance, has been attributed to a compensatory response that counteracts the drug effect (e.g., Kalant, 1989; Siegel, 1989). Behavioral tolerance is inferred from a reduction in the disrupting effect of alcohol on performance. Alcohol tolerance may, in some instances, be due to physiological changes induced by repeated drug exposures. However, the expectancy model predicts that a reduced behavioral effect of alcohol that resembles tolerance should be displayed without requiring

2/15/2011 7:45:08 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

22 10 18 16 14 12 10 8 6 4 2 0 –2 –4 –6

group who were rewarded for unimpaired performance with money and verbal information, and a group who received no money or information. Drinkers receiving money and information displayed progressively less impairment on each alcohol session and performance became comparable to the drug-free level. Finally, when alcohol was expected but a placebo session was administered, only the group receiving money and information exhibited signiﬁcant improvement above the drug-free standard of performance. This compensatory enhancement of task performance is consistent with the results of other studies showing that the expectancy of a favorable outcome for resisting impairment improves performance under a placebo, above the standard drug-free level of achievement (e.g., Sdao-Jarvie & Vogel-Sprott, 1991). A period of abstinence has usually been considered necessary for behavioral tolerance to alcohol to subside. However, studies have shown that this apparent alcohol tolerance can be eliminated swiftly, by merely withholding a favorable outcome for resisting impairment that drinkers had come to expect (Mann & Vogel-Sprott, 1981).

Consequence Money plus information No money or information Impairment

Change in performance (seconds)

long-term habitual use of the drug. What is required is the expectation that resisting alcohol effects yields a favorable outcome. In the absence of this particular expectancy, little reduction in the drug effect should be displayed. Considerable research testing these hypotheses and their implications has been presented elsewhere (Vogel-Sprott, 1992, 2004), and a brief summary of the ﬁndings is presented here. The experiments typically include groups who receive the same doses of alcohol and equal practice trials on a task under each dose of the drug. One group receives a favorable outcome, such as money or verbal praise, contingent on unimpaired performance, whereas control groups receive either no outcome or an equivalent number of favorable outcomes administered randomly with respect to performance (e.g., Beirness & Vogel-Sprott, 1984). These control conditions provide no opportunity to acquire this expectancy and do not systematically reduce the behavioral effects of alcohol. It is the expectation of a favorable outcome for resisting impairment that reliably diminishes the alcohol effect. Figure 10.2 illustrates the results of a

219

Session 1 Alcohol

Drug-free baseline

2 Alcohol

3 4 Alcohol Alcohol

Figure 10.2 Change in the performance of a motor skill task in groups with, or without, information

and monetary reward. Tests occurred on weekly sessions under alcohol. A ﬁnal session administered a placebo. Zero represents the drug-free baseline. Positive scores indicate slower, impaired performance, and negative scores show improvement. Vertical bars show standard errors of the mean. (Adapted from Sdao-Jarvie, K., & Vogel-Sprott, M. 1991. Response expectancies affect the acquisition and display of behavioral tolerance. Alcohol, 8, 491–498).

10-Schachtman-10.indd 219

2/15/2011 7:45:09 PM

220

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Because drinkers’ tolerance extinguished even though alcohol doses continued to be repeated, an interval of alcohol abstinence apparently is not always necessary to reduce the resistance to alcohol effects. Other evidence strengthening the interpretation of the effect of expected outcomes has shown that behavioral tolerance acquired in a particular setting will generalize and transfer to another drinking situation where a similar type of task and outcome for the response to alcohol is provided (Rawana & Vogel-Sprott, 1985). Drinkers who expected and received a rewarding outcome for displaying tolerance on one motor skill task readily displayed tolerance on a second similar task performed for the ﬁrst time under alcohol. The results illustrated in Figure 10.2 have been obtained in many studies of behavioral tolerance to alcohol in motor skill tasks. Taken together, they indicate that the effect of expecting a favorable outcome for resisting motor skill impairment emerges gradually as doses are repeated a few times. This gradual reduction in impairment is unlikely to be due to some physiological changes that weaken the drug effect because individuals with no expected outcome of performance continue to show a fairly stable level of impairment under these doses. An alternative explanation is suggested by the resemblance between the progressive recovery from impairment as the task is performed under repeated alcohol doses, and the gradual improvement with task practice that characterizes the learning of a new motor skill. Research tends to support this learning hypothesis (Easdon & Vogel-Sprott, 1996; Zinatelli & Vogel-Sprott, 1993). In these experiments, two groups learned a complex motor skill and then both received additional drug-free practice with a reward for maintaining good performance. An experimental group worked under environmentally induced impairment created by a “fog” that degraded the display of test stimuli and slowed responding. This impairment in the experimental group diminished progressively with task practice, so it appeared that tolerance to this impairment developed through learning some new behavioral coping strategies. Meanwhile, the control group continued with equivalent practice trials

10-Schachtman-10.indd 220

in the standard environment. Thus, both groups had the same number of practice trials on the task at the conclusion of the practice. The experimental group only differed by having some of this practice under environmental conditions that impaired task performance. The possibility that this practice overcoming environmental impairment could provide an opportunity to learn some alternative motor skill strategies to counteract alcohol impairment was subsequently tested by administering a dose of alcohol to both groups who performed the task in the standard environment. Although the alcohol tests were identical for both groups, the experimental group displayed no alcohol impairment, whereas the control group exhibited considerable alcohol-induced impairment. Because the experimental group only differed by having a prior history of drug-free training to overcome environmentally induced impairment, it seems that some learned motor skill to counteract impairment, coupled with some rewarding consequence for performing well, can almost immediately generate behavioral tolerance to alcohol. Moreover, if these motor skills have not been learned in advance, the gradual emergence of tolerance when a task is performed under repeated doses of alcohol may reﬂect the development of this learning. The various ways in which expectancy effects are exhibited, as well as their broad generality, also merit comment. First, expected outcomes regarding drugged behavior (Rd-S∗) appear to inﬂuence a wide range of behaviors that are affected by alcohol. Much of this work has concerned tolerance to the impairing effects of alcohol on motor skills. However, studies of the alcohol effects on cognitive activities also show tolerance when drinkers expect rewarding outcomes for nonimpaired performance (for a review, see Vogel-Sprott & Fillmore, 2002). This research used tasks developed in cognitive science to assess information processing, working memory, and intentional control of behavior. All tasks have shown that expecting a desirable outcome of cognitive performance diminished or eliminated the impairing effect of an initial dose of alcohol, even at peak blood alcohol levels in the range of 80 mg/100 ml (e.g., Fillmore &

2/15/2011 7:45:09 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

Vogel-Sprott, 1997; Fillmore, Vogel-Sprott, & Gavrilescu, 1999; Grattan & Vogel-Sprott, 2001; Grattan-Miscio & Vogel-Sprott, 2005). However, unlike motor skills whereby resistance to alcohol effects may be increased through repeated task practice under the drug, tolerance to impaired cognitive functions appears under an initial dose of alcohol almost immediately upon the expectation of a rewarding outcome. Since these sorts of mental tasks do not require any new motor skill, merely a button press or a verbal response, the expectation of a rewarding outcome for resisting impairment should, alone, diminish the impairing effect of an initial dose of alcohol. Indeed, practice of a motor skill under repeated doses of alcohol might only be needed to learn some new motor skills to overcome impairment. Secondly, the generality of expectancy effects on behavior show that the expected consequences do not always produce drug tolerance. The expected outcome can also result in an intensiﬁcation of the drug effect (i.e., sensitization). With its important potential relationship to drug abuse and addiction, behavioral tolerance to alcohol has attracted much more research interest than has sensitization. Yet the expectancy model indicates that sensitization to alcohol impairment should increase as readily as tolerance. It only requires the expectation of a favorable outcome of behavioral impairment. A series of studies have shown that sensitization is displayed by drinkers who learn to expect a favorable consequence for impairment under alcohol while others who expect the same outcome for resisting impairment display tolerance (Zack & Vogel-Sprott, 1995, 1997). It seems that drunk or sober behavior under alcohol depends on which one is expected to yield a favorable consequence.

221

can counteract alcohol-induced impairment. Identifying and understanding how these expectancies can mediate behavior under a drug has potential safety implications because drinkers engage in many types of motor and cognitive activities that could be impaired and might contribute to risky behavior such as driving, aggression, or excessive drinking. Risks affecting the amount of alcohol consumed have also been identiﬁed in an experiment (Sharkansky & Finn, 1998) where participants had an ad lib drinking session before they completed a task. The type of alcohol effect (S-S∗d) was manipulated at the outset of the drinking session by telling one group of drinkers that alcohol would cause intense impairment on the task and informing the other group that alcohol would have no appreciable effect. All individuals expected to earn money for efﬁcient performance, and this ensured that the expected consequence of displaying good performance under alcohol (Rd - S∗) was held constant. The results showed that those who expected considerable impairment on the task drank signiﬁcantly less alcohol during the ad lib session than did those who expected little impairment. This research reveals the expected effect of alcohol can have a powerful moderating inﬂuence on alcohol consumption when social drinkers expect some desirable consequence for good performance. While an understanding of how these expectancies achieve these effects requires other research, the observations clearly indicate that expectancies may contribute signiﬁcantly to understanding the occurrence of excessive drinking and drug abuse. The next section describes research that offers some insights on those issues.

Summary

BEHAVIORAL CONTROL, EXPECTANCY, AND ACUTE DRUG EFFECTS

This section reviewed some of the research with motor skill and cognitive tasks that shows how individual differences in drug-related expectancies can be acquired and can affect behavior under the drug or placebo. Studies also demonstrated how the motivating effect of expecting a favorable consequence for good performance

Traditionally, the abuse potential of a drug has been attributed to its positive rewarding effects, which are assumed to reﬂect direct drug action of reward centers of brain areas, such as the nucleus accumbens (DiChiara, Acquas, & Tanda, 1996; Koob, 2003). By contrast, the degree to which a drug impairs behavioral functioning,

10-Schachtman-10.indd 221

2/15/2011 7:45:09 PM

222

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

such as motor coordination, was thought to have little to do with its addictive potential. However, that viewpoint has changed somewhat in recent years, since more is known about how drugs alter cognitive functions in ways that could contribute to an increased use of drugs, possibly leading to abuse and dependence (Fillmore, 2003; Lyvers, 2000). A number of models have been advanced to study how drugs alter cognitive processes involved in the control and regulation of behavior, and how such drug-induced alterations could promote drug addiction. However, to date, little work has examined how drugrelated expectancies might operate in such cognitive systems. The following sections describe a cognitive model that concerns drug effects on basic inhibitory processes critical for suppressing behavioral impulses. In describing the model we identify how expectancies concerning appropriate responses play a key role in the control of behavior and how such expectancies mediate the inﬂuence of drugs on behavioral control. Cognitive Control as the Inhibition and Activation of Behavior

There is growing interest in identifying and characterizing the basic neurocognitive mechanisms that underlie the regulation of behavior (Goschke, 2003; Miller & Cohen, 2001). Several theories in cognitive neuroscience postulate that the control of behavior is governed by distinct inhibitory and activational systems (Fowles, 1987; Gray, 1976; Logan & Cowan, 1984). Despite the complexity of the central nervous system (CNS), much of its inﬂuence can be reduced to the summed effects of opposing inhibitory and excitatory neural activity. Studies in neuropharmacology and neuroanatomy have identiﬁed distinct neural systems that implicate separate inhibitory and activational processes in the control of behavior (Fillmore, 2003; Jentsch & Taylor, 1999; Leigh & Zee, 1999; Lyvers, 2000). The relative strength of these systems is assumed to determine behavioral control. Extreme or disinhibited behavior may arise from either a weakened inhibitory system or from a heightened activation system. This means that responding and inhibiting must each be measured to identify the source of impaired control.

10-Schachtman-10.indd 222

Disinhibited behavior is considered to be an important risk factor for the onset of drug abuse, and it has received considerable attention in drug abuse research. Research in cognitive neuroscience has focused on the processes that govern behavioral inhibition, suggesting that impairment in this system underlies many deﬁcits of self-control. Much of this research has been inﬂuenced by a “stop-signal” model of behavioral control (Logan & Cowan, 1984). According to the model, control is determined by a competitive race between activating and inhibiting processes. Go and stop signals elicit these activating and inhibitory processes, and the time in which each process is completed determines the behavioral outcome. If the inhibiting processes are completed ﬁrst, the response is withheld, but if the activating processes ﬁnish ﬁrst, the response is executed (see Fig. 10.3). The stop-signal model is a reaction time task that measures the countervailing inﬂuences of inhibitory and activational mechanisms. Individuals are required to quickly activate a response to a go signal and to inhibit a response when a stop signal occasionally occurs. Activation is typically measured by the speed of responding to go signals, and inhibition to stop signals is assessed by the probability of suppressing the response or by the time needed to suppress the response. Inhibition is usually required in a context in which there is a strong tendency to respond to a stimulus (i.e., a prepotency), thus making inhibition difﬁcult. These tasks require quick,

Environmental stimulus Go signal

Stop signal

Response outcome

Process Time

Execution

Activation Time

Inhibitation

Suppression

Figure 10.3 Race model of behavioral control. Go and stop signals elicit the activating and inhibitory processes, and the time in which each process is completed determines the behavioral outcome. If the inhibiting processes are completed ﬁrst, the response is withheld, but if the activating processes ﬁnish ﬁrst, the response is executed.

2/15/2011 7:45:09 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

accurate choice responses to go signals and the sudden inhibition of these responses during occasional presentations of stop signals. The go signal is a letter (e.g., O or X) visually presented one at a time on a computer monitor. Participants respond to the letters by pressing one of two keys on a computer keyboard. Subjects are required to inhibit the response when stop signals (e.g., brief auditory tones) occasionally accompany go-signal presentations. The presentation of a go target on every trial produces a prepotency (i.e., instigation) to respond. Subjects must overcome the prepotent tendency in order to suppress responses when stop signals occur. This stop signal presumably initiates the inhibitory process. The basic premise of this task is that two separate but interactive processes—a response execution or “go” process and a response inhibition or “stop” process—are engaged in an interactive “race” every time one wishes to inhibit a response (Logan, 1994). If the stop process runs to completion before the go process ﬁnishes, then the response is successfully inhibited; otherwise a response occurs. Inherent in this model is the notion that these processes occur at different rates and thus require different amounts of time to complete. More speciﬁcally, the stop or inhibitory process must be faster than the go process; otherwise responses could not be successfully inhibited. Stop signal reaction time (SSRT) is deﬁned as the difference between the time the stop signal is presented and the end of the inhibitory process, and it can be estimated mathematically based on the probability of successfully inhibiting a response when the stop signal is presented at various delays following the go signal (see Logan, 1994). Longer SSRTs indicate weaker inhibitory control over behavior, perhaps due to a slow inhibitory process. Stop-signal tasks have been useful in showing that deﬁcient or impaired inhibitory control is implicated in the display of impulsivity and disorders of self-control. Aggressive and impulsive behaviors that characterize disorders, such as antisocial personality, obsessive-compulsive, and attention deﬁcit/hyperactivity disorders (ADHD) have been attributed to impaired inhibitory mechanisms (Barkley, 2006; Nigg, 2006). The model is sensitive to inhibitory deﬁcits characteristic of

10-Schachtman-10.indd 223

223

brain injury (Cremona-Meteyard & Geffen, 1994; Malloy, Bihrle, Duffy, & Cimino, 1993) and trait-based impulsivity (Logan, Schachar, & Tannock, 1997). Several studies have employed stop-signal tasks to examine the acute effects of moderate doses of alcohol on behavioral control in healthy adults (e.g., de Wit, Crean, & Richards, 2000; Easdon & Vogel-Sprott, 2000; Fillmore & Blackburn, 2002; Fillmore & Vogel-Sprott, 1999, 2000; Mulvihill, Skilling, & Vogel-Sprott, 1997). These studies found that moderate doses of alcohol selectively reduced drinkers’ ability to inhibit their behavior while leaving their ability to activate behavior unaffected. Impaired inhibition under alcohol was evident by reduced frequency of inhibiting a response to a stop signal (e.g., Fillmore & VogelSprott, 1999) and by increased time needed to inhibit, as measured by SSRT (de Wit et al., 2000). An advantage of this approach is that it examines the effects of alcohol on individual “building blocks” of behavior and can observe some degree of independence between inhibitory and activational processes. In particular, it appears that alcohol can disrupt inhibition at blood alcohol concentrations that are insufﬁcient to disrupt response activation. This suggests that inhibitory mechanisms might be more vulnerable than activational mechanisms to the impairing effects of alcohol. The proposal also has important implications for understanding how alcohol impairs behavioral control. Many fundamental cognitive and perceptual processes, such as inhibitory mechanisms, are considered to operate in a “bottom-up” fashion to exert increasing inﬂuence at each stage of higher order attentional and cognitive functions (McClelland & Rumelhart, 1981). The efﬁciency of working memory, decision making, and other complex cognitive operations likely depend on the ability to suppress any ongoing unnecessary actions and distractions in order to free up maximal cognitive resources for the task at hand. Thus, any alcohol-induced impairments of the ability to inhibit unnecessary action might operate in a bottom-up manner to produce pronounced impairments of higher cognitive operations (e.g., decision making, planning) that depend on inhibitory control.

2/15/2011 7:45:09 PM

224

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Alcohol-Related Expectancy and Inhibitory Control

Given the potential role of inhibitory control in drug abuse, a better understanding is needed of how alcohol-related expectances might alter the drinker’s inhibitory control in the drug-taking situation. The advantage of behavioral control models, like the stop-signal task, is that they separate the speciﬁc mechanisms of control (e.g., inhibition versus activation) and provide an opportunity to study how a drug-related expectancy might affect these mechanisms differently. Studies examining drinkers’ expected outcomes (Rd-S∗) concerning alcohol-induced impairment of response inhibition and response activation have shown that each mechanism is highly sensitive to a drinker’s expectancy (Fillmore & Vogel-Sprott, 1999, 2000). In these studies, drinkers received alcohol or placebo and performed the stop-signal task. Some groups received a favorable outcome (money) contingent upon maintaining the same proportion of sober response inhibitions after drinking. Other groups received money contingent upon maintaining a sober level of response activation (i.e., speed) after drinking. Thus, the favorable expected outcome was speciﬁc to only one mechanism of behavioral control (inhibition or activation). Performance was tested under alcohol and placebo conditions. The results showed that resistance to alcohol impairment was evident and was speciﬁc to the mechanism associated with a favorable outcome. Those who expected rewarding outcomes for maintaining unimpaired response inhibition displayed tolerance to the drug and maintained inhibitory control. Likewise, those who expected a favorable outcome for maintaining quick response activation displayed tolerant levels of response speed under the drug. Under the placebo, the expectancies produced mechanism-speciﬁc compensatory reactions and better than sober-baseline performance. Also noteworthy was that the tolerance and compensatory reactions were evident immediately upon the ﬁrst test, which is similar to the immediacy with which expectancies produced alcohol tolerance and compensatory performance on other cognitive tasks (Fillmore &

10-Schachtman-10.indd 224

Vogel-Sprott, 1997; Fillmore, Vogel-Sprott, & Gavrilescu, 1999; Grattan & Vogel-Sprott, 2001; Grattan-Miscio & Vogel-Sprott, 2005). Given that the expected outcome could affect speciﬁc mechanisms of behavioral control in the drinking situation, Fillmore and Blackburn (2002) examined how manipulating the expected type of effect (i.e., S∗d - Rd) might also affect speciﬁc mechanisms of control under alcohol. In that study, drinkers received information leading them to expect intense alcohol impairment of one aspect of behavioral control on the stopsignal task: either response activation or response inhibition. Speciﬁcally, some groups expected that alcohol would have intense impairing effects on the ability to quickly execute responses (i.e., impaired activation) and other groups expected impairment of the ability to suppress responses (impaired inhibition). They found that subjects compensated for the expected impairment by showing less impairment under alcohol and an above-baseline, compensatory response to a placebo. Moreover, the compensatory effects were mechanism speciﬁc in that drinkers who expected impaired inhibition showed compensation of inhibitory control (i.e., became more inhibited) and those who expected impaired activation compensated by speeding response activation (i.e., became more activated). The results also showed a trade-off between the two mechanisms such that compensating for one aspect of behavioral control (e.g., inhibition) resulted in increased impairment of the other mechanism (activation). The ﬁndings from these studies are important because they demonstrate how alcoholrelated expectancies can affect individual aspects of behavioral control to produce resistance to alcohol effects that are speciﬁc to either inhibition or response activation. However, it is also important to recognize that the expectancymediated drug resistance displayed in these studies does not necessarily mean that drinkers’ overall level of behavioral functioning was restored. Rather, tolerance to alcohol-induced impairment in one mechanism appeared to occur at the cost of the other mechanism. Indeed, the stop-signal model demonstrates reciprocity of the countervailing inﬂuences of inhibition

2/15/2011 7:45:09 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

and activation underlying behavioral control. Essentially a trade-off can occur between inhibition and activation because of an inherent reciprocity between the speed of responding to go signals and the ability to inhibit responses to stop signals that suddenly occurs (Fillmore, & Vogel-Sprott, 2000; Logan & Cowan, 1984). Consequently, increased response inhibition is accompanied by slower response times, and enhanced response speed is typically accompanied by poorer inhibitory control. Thus, an expectancy-guided compensatory reaction in one system could compromise the functioning of the other system. Response-Appropriate Expectancies in Behavioral Control

Other studies of alcohol effects on behavioral control have addressed the question of expectancy from a somewhat different vantage point by examining how environmental cues in a situation can produce expectancies concerning the appropriate action to be taken, and how these acquired expectancies can facilitate behavioral control and reduce the disruptive inﬂuence of drugs, such as alcohol (Fillmore, 2003). Speed of action is critical for effective behavioral control (Logan, 1994). Consequently, basic acts of control can be anticipatory, whereby individuals prepare to act in response to the expectation that a particular response will be required in the situation. Stimulus cues often precede signals to inhibit or respond in the natural environment. Such predictive cues can produce an expectancy about the appropriate response that will likely be required (i.e., inhibition or activation). These response-appropriate expectancies can facilitate the activation and inhibition of behavior by allowing the individual to prepare for the execution or suppression of an action (Duncan, 1981; Posner, 1980; Posner, Snyder, & Davidson, 1980). Laboratory tasks that incorporate responseappropriate expectancies have considerable ecological relevance for assessing behavioral control. Outside a laboratory, cues to prepare for appropriate actions are commonplace. A familiar real-life example is the use of the amber trafﬁc light to allow drivers time to prepare to stop

10-Schachtman-10.indd 225

225

their vehicle in anticipation of the ensuing red light. Behavioral control tasks similar to the stopsignal task have been developed to examine the inﬂuence of response-appropriate expectancies on the individual mechanisms of behavioral control. For example, the “cued” go/no-go task presents antecedent cues to the subject to generate expectancies about whether to inhibit or execute a response. These tasks typically present a stimulus cue followed by a go or no-go target stimulus that requires a response to be either executed (go) or inhibited (no-go). The cue provides information concerning the probability that a go or no-go target will be presented. The cue–target relationship is manipulated so that cues have a high probability of correctly signaling a target and a low probability of incorrectly signaling a target. Correct (i.e., valid) cues produce response-appropriate expectancies and therefore tend to facilitate response execution and response inhibition. For example, responses to go targets are faster when they are preceded by a go cue. Similarly, the likelihood of suppressing a response to a no-go target is greater when it is preceded by a no-go cue (Miller, Schaffer, & Hackley, 1991). The expectancy-supported facilitation of response execution and response suppression is attributed to advanced preparatory processing that occurs before the target actually appears (Duncan, 1981; Posner, 1980; Posner et al., 1980). Once the target appears, less processing of the appropriate response is required. A recent line of research using the cued go/no-go task has shown that such responseappropriate expectancies play an important role in determining the degree to which alcohol impairs mechanisms of behavioral control (for reviews, see Fillmore, 2003, 2007). In one study (Marczinski & Fillmore, 2003) subjects performed a cued go/no-go task after receiving each of three doses of alcohol (0.0, 0.45, and 0.65 g/kg) administered on separate days. A white rectangle-shaped cue was visually presented on a computer screen in either an upright or ﬂat orientation. The go and no-go targets were displayed as a color change of the rectangle to either green (go target) or blue (no-go target). Subjects were required to respond to the green, go target

2/15/2011 7:45:09 PM

226

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

mechanisms of behavioral control are greatly facilitated when subjects expected the appropriate response. Failures to inhibit responses to no-go targets were less likely on trials in which subjects expected that inhibition would be required. Similarly, speed of activation was much faster on trials in which subjects expected response activation. By contrast, inhibition and activation were much poorer on trials when expectancies were inappropriate. Inhibitory failures were more frequent on trials in which subjects incorrectly expected response activation, and response activation was slower on trials in which subjects incorrectly expected response inhibition. These expectancies also had a pronounced inﬂuence on impairment under alcohol. Indeed, the magnitude of alcohol impairment on the inhibition or the execution of responses depended entirely upon whether subjects held

by pressing a key, and to inhibit any response when a blue, no-go target appeared. The orientation of the white rectangle (upright or ﬂat) signaled the probability that it would change to either a go (green) or no-go (blue) target. Upright cues led subjects to expect that response activation would be required because they preceded go targets on 80% of the time and only preceded no-go targets 20% of the time. Flat cues led subjects to expect that response inhibition would be required because they preceded no-go targets 80% of the time and preceded go targets only 20% of the time. Figure 10.4 shows the results of that study. The two primary measures of interest were the participants’ failures to inhibit responses to no-go targets (failures of response inhibition) and their speed of responding to go targets (response activation). First, the ﬁgure shows that

Response required Inhibition

Activation

310

0.25 295

0.15

0.1 0.05 0

0.0 g/kg

280

265

250 0 0.45 g/kg

0.65 g/kg

Impairment

Mean RT (ms)

0.2

Impairment

Mean proportion of inhibition failures

0.3

Response expected Inhibition Activation

0.0 g/kg

0.45 g/kg

0.65 g/kg

Dose

Figure 10.4 Effect of response-appropriate expectancies on the degree to which inhibitory and activational mechanisms are impaired under three alcohol dose conditions: 0.0 g/kg (placebo), 0.45 g/kg, and 0.65 g/kg. (Left) Impaired response inhibition measured as mean proportion of failures to inhibit responses to no-go targets following go and no-go cues. (Right) Impaired response activation measured as mean reaction time (RT) to respond to go targets following go and no-go cues. Vertical bars show standard errors of the mean. (Adapted from Marczinski, C. A., & Fillmore, M. T. 2003. Pre-response cues reduce the impairing effects of alcohol on the execution and suppression of responses. Experimental and Clinical Psychopharmacology, 11, 110–117).

10-Schachtman-10.indd 226

2/15/2011 7:45:09 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

response-appropriate expectations during those trials. With regard to response inhibition, alcohol dose-dependently increased inhibitory failures to no-go targets only when drinkers incorrectly expected response activation. When these drinkers expected response inhibition, no impairing effect was observed. Similarly, with regard to response activation, alcohol dosedependently slowed response time only when drinkers incorrectly expected to inhibit the response. When they correctly expected to execute the response, no impairing effect of alcohol was observed. These ﬁndings demonstrate how a responseappropriate expectancy in a drinking situation can offer some protection from the impairing effects of alcohol on behavioral control. The expectancy likely offers protection from the impairing effects of alcohol because it allows the drinker to prepare the appropriate response (i.e., to activate or to inhibit) in advance before the actual response is required. This processing “head-start” reduces the amount of information that needs to be processed when the actual response is required (Fillmore, 2004; Marczinski & Fillmore, 2003). Similar evidence for the involvement of response-appropriate expectancy has been found in a study that examined the effects of d-amphetamine on behavioral control in stimulant abusers (Fillmore, Rush, & Marczinski, 2003). That study also used the cued go\no-go task and found that d-amphetamine produced a dose-dependent increase in inhibitory failures following invalid go cues, but had no effect on inhibitory failures following valid no-go cues. Thus, like alcohol, d-amphetamine increased the degree to which inhibitory control depended on the user’s expectancy about the appropriate behavior in the situation. Behavioral Control and Conﬂicting Expectancies

The studies described earlier also demonstrate the selectivity of expectancies as evidenced by the mechanism-speciﬁc reactions they evoke in individuals under alcohol. However, some situations may not contain sufﬁcient information about which aspects of behavioral control are

10-Schachtman-10.indd 227

227

expected to be compromised by a drug, or which aspects of control should be maintained to yield rewarding consequences. Furthermore, some settings might contain information leading to conﬂicting expectancies concerning aspects of behavioral control. Deﬁned broadly, conﬂict refers to the simultaneous occurrence of any two competing response tendencies (e.g., approach versus avoidance). Conﬂict between approach and avoidance tendencies is commonly referred to as inhibitory response conﬂict and has been of particular interest in the study of self-regulation and behavioral control (Fowles, 1987; Logan & Cowan, 1984). Expectations of rewarding outcomes are associated with approach tendencies, whereas expected punishing outcomes are associated with avoidance tendencies. As many behaviors have the potential to be both rewarding and punishing, activation and inhibition are often simultaneously evoked, resulting in conﬂict. Observed behavior is assumed to be the result of whichever system is more strongly activated (Fowles, 1987; Gray, 1976, 1977; Logan & Cowan, 1984). An early example of research on alcohol effects on response conﬂict is Conger’s (1956) seminal study of approach-avoidance behavior in rats. In this study, pressing a lever was associated with both the rewarding outcome of a food pellet and the punishing outcome of an electric shock. Thus, an approach-avoidance conﬂict was created. Conger (1956) found that administration of alcohol served to resolve this conﬂict, presumably by reducing the fear associated with (and therefore avoidance of) the negative outcome, which led rats to engage in the approach behavior. Subsequent studies that followed continued to show that intoxicated lab animals would no longer avoid noxious stimuli in pursuit of positive reinforcement (e.g., Barry, Wagner, & Miller, 1962). The possibility that alcohol’s “disinhibiting” effects might resolve many approach-avoidance conﬂicts in favor of approach behaviors could explain the drug’s association with a host of inappropriate and seemingly impulsive behaviors, such as aggression, risk taking, intoxicated driving, and risky sexual activity. For example, alcohol intoxication could increase risk of

2/15/2011 7:45:10 PM

228

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

unprotected sex because the drinker’s expectation of immediate sexual gratiﬁcation outweighs any conﬂicting expectancy of a punishing or negative outcome, such as contracting a sexually transmitted disease. Indeed, studies of humans have examined the inﬂuence of alcohol in social situations involving response conﬂict. One common example is aggression. For instance, an individual might be inclined to aggress against another person because of some expected positive outcome, such as gaining favor among peers and increased social status. However, at the same time, such aggression likely is associated with conﬂicting expectancies of negative outcomes, such as arrest for assault and the accompanying legal sanctions. In two meta-analyses, Steele and Southwick (1985) and Ito et al. (1996) reviewed several studies of alcohol effects in humans and noted that behavioral responses to the drug tended to be more pronounced in situations of inhibitory conﬂict whereby a response is instigated by one set of cues and inhibited by another set. Thus, clashing expectancies about desirable outcomes for responding or inhibiting a response may intensify the risk of alcohol impairment. Recent studies using stop-signal and cued go/ no-go tasks show that conﬂicting expectancies can intensify the disinhibiting effects of alcohol (Fillmore, Blackburn, & Harrison, 2008; Fillmore & Vogel-Sprott, 2000). Conﬂict was increased by leading subjects to expect monetary reinforcement contingent on quickly activating responses to go targets and also on suddenly inhibiting responses to no-go targets. These studies showed that greatest disinhibiting effects of alcohol were observed when drinkers’ expected positive outcomes for inhibition and activational aspects of behavioral control. Similar effects of conﬂicting expectancies have been observed in situations involving driving behavior, whereby drivers expected equal monetary incentives for slow, careful driving but also for fast driving to shorten the trip duration (Fillmore et al., 2008). Alcohol adversely affected several aspects of driving performance, but the presence of conﬂicting expectancies exacerbated impaired and risky driving behaviors, such as failures to stop at red lights. Taken together, the evidence suggests that

10-Schachtman-10.indd 228

a high-risk scenario for disinhibited and impulsive behaviors while drinking might occur when the expected consequences for behavioral inhibition and behavioral activation are salient and equally motivating (i.e., create maximum conﬂict). Summary

This section described how drug-related expectancies can be studied in terms of their effects on speciﬁc behavioral mechanisms involved in selfregulation. Although a fairly recent endeavor, the study of expectancies in terms of their effects on speciﬁc activation and inhibition processes of behavioral control has yielded some important ﬁndings. The studies reviewed here demonstrate how individuals can hold speciﬁc expectancies about how drugs affect the ability to inhibit or to execute behaviors. Moreover, these expectancies exert a causal inﬂuence on behavior. Expectancies can reduce or intensify the degree of druginduced behavioral impairment, and they have the potential to be mechanism-speciﬁc, compromising or enhancing the inhibitory or activational tendencies in a situation. With particular regard to inhibitory control, evidence that drugrelated expectancies can augment or temper the expression of disinhibited behavior in a drugtaking situation is novel and might have important implications for understanding drug abuse. Situations that give rise to conﬂicting expectancies regarding the effects or outcomes of a drugged response represent high-risk scenarios for the display of behavioral disinhibition. When studied as a risk factor for drug abuse, disinhibition is typically considered a static characteristic of the individual indicative of a personality trait, such as impulsivity and sensation seeking. It also has been attributed to a direct pharmacological effect of a CNS-depressant drug, such as alcohol. However, the ﬁndings reviewed here suggest that learned expectancies in a drinking situation can greatly inﬂuence the degree of inhibitory control that the drinker displays. Taken together, the evidence suggests that impulsivity could be determined as much by expectancies in the drugtaking environment as it is by the personality of the drug taker or by the pharmacological properties of the drug taken.

2/15/2011 7:45:10 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

DISCUSSION Alcohol Use and Abuse

Evidence that alcohol-related expectancies can inﬂuence the extent to which response inhibition or activation is expressed in a drug-taking situation could help to understand drug use and abuse. This chapter showed that expectancy effects on activation and inhibition processes are speciﬁc, and the effect on one can occur at the cost of the other. This has some important practical implications beyond the laboratory. These expectancies could play an important role in determining the behavior of individuals in a drinking situation. Expectancies that strengthen inhibition might lead to the display of highly restrained, reserved behavior, whereas expectancies that strengthen activation could result in impulsive or reckless behavior. This implies that certain alcohol-related expectancies held by the drinker could further compromise a drinker’s inhibitory control. Examples may include drinking situations where binge drinking and accompanying risky behaviors are expected to yield more favorable outcomes than the inhibition of such actions. Such activities might include driving while intoxicated to get to the next party or engaging in unprotected sexual activity. Simply put, the risk for disinhibited behavior under alcohol might intensify in settings where the motivational valence of expectancies in the situation favors activation over inhibition. Indeed, there has been much interest in the possibility that behavioral disinhibition and impulsivity contribute to the risk for alcohol abuse. These traits are typically measured by self-reports of undercontrolled behaviors, such as an inability to delay gratiﬁcation and acting without forethought or consideration of potential consequences. Studies have found that impulsive or disinhibited individuals tend to drink more frequently and in larger amounts during drinking episodes (e.g., Cherpitel, 1993; Marczinski, Combs, & Fillmore, 2007). In terms of an acute reaction, disinhibition might be especially problematic for the drinker once the drinking episode has begun because the drug itself can impair inhibitory control even in low doses

10-Schachtman-10.indd 229

229

(Fillmore, 2003, 2007). Any impairment of normal inhibitory control resulting from an initial dose of alcohol could compromise the ability to stop additional alcohol administration in a drinking situation, leading to a binge. Many drinkers report intentions to limit their alcohol use to one or two drinks only to fail and instead drink excessively (Collins, 1993). Studies of drug abusers also have documented their poor performance on cognitive and neuropsychological tasks (Jentsch & Taylor, 1999). Prolonged drug abuse is associated with widespread neuropsychological deﬁcits involving memory, attention, learning, problem solving, and perceptual motor speed. The results of some recent studies measuring expectancy acquisition in drug abusers suggest that the crux of the problem may be maladaptive learning of expectancies about behaviors and their ensuing consequences. For example, some research has associated risk of alcohol abuse with weak classical conditioning of signals for punishment (Finn, Kessler, & Hussong, 1994). Studies using associative learning tasks also ﬁnd that drug abusers display impaired acquisition of new expectancies, especially when such learning involves abandoning associations previously learned, but no longer valid. For example, prolonged exposure to drugs, such as cocaine, is associated with impaired discrimination-reversal performance in which previously learned expectancies must be abandoned in order to acquire new appropriate expectations (e.g., Fillmore & Rush, 2006; Jentsch, Olausson, De La Garza, & Taylor, 2002). These deﬁcits in discrimination-reversal learning might reﬂect impaired inhibition whereby a response to a previously associated stimulus cannot be withheld in order to learn a new association between the response and another stimulus (Fillmore, 2003; Jentsch & Taylor, 1999). The result of a failure to inhibit previously learned response–outcome expectancies is a pattern of persistent, inﬂexible behavior that fails to adapt to changing environmental contingencies. Interestingly, such preservative behavioral patterns in the attainment of drugs have long been considered a key characteristic of addictive behavior (Koob et al., 1998).

2/15/2011 7:45:10 PM

230

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Other Settings and Situations

This chapter applied an analysis of learned expectancies to the problem of alcohol abuse partly because of the important harmful effects of alcohol misuse for individuals and society, and partly because research on alcohol-related expectancies has generated many ﬁndings with important implications for understanding drug use and abuse. Although interest to date has primarily focused on alcohol-related expectancies, a learning analysis could equally well be applied to expectancies related to other social or illicit drugs. Given that other drugs may have different intoxicating effects and consequences for behavioral use, investigations of learned expectancies associated with such drugs could reveal their distinctive expected effects and expected consequences of drug use. An example of this type of research is the chapter in the present text by McCarthy et al. (see Chapter 11), which examines expectancies about stimulus–outcome and response–outcome learning in nicotine use and smoking behavior, and how the resulting learned expectancies inﬂuence drug use. The analysis of learned expectancies also can be extended to situations or problematic behavior that involves no drug at all. Abstinencerelated expectancies have rarely been investigated. However, research ﬁndings could identify the types of expected effects and expected consequences of abstinence that might contribute importantly to understanding factors that strengthen or weaken resistance to relapse. Another example candidate for a learned expectancy analysis is obsessivecompulsive behavior. This behavior is problematic because it is displayed so frequently that it intrudes and disrupts normal daily activities. In this regard, obsessive-compulsive behavior resembles drug addiction, but without any drugs. The sheer frequency of the ritualistic behavior is its unusual feature, and this generates questions about the effect of extensive practice on learned expectancies. Learning research indicates that highly practiced behavior occurs swiftly, with little attention or awareness. These attributes are considered to typify automatic responses that are no longer deliberate or intentional (e.g., Jacoby, 1991). Some investigators have suggested

10-Schachtman-10.indd 230

that expectancies also are not necessarily always deliberate and could be automatic (e.g., Kirsch, 1985, 1999). No research has investigated this prospect, although some observations suggest that practice strengthens an expectancy and efforts to change expectancies underlying automatic and intentional responses may not be equally successful (e.g. Fillmore & Vogel-Sprott, 1996; Grattan & Vogel-Sprott, 2001). These possibilities have broad potential implications for treatments that aim to change problematic behavior. The use of an expectancy model representing the expectancies in a situation requiring a given response could be used as a prototype to examine how much repetition is required before intentional responding becomes automatic, as well as whether, or how, each of the guiding expectancies can be changed. It seems that future research applying expectancy analyses may garner large beneﬁts in understanding behavior in many types of situations.

CONCLUSION This chapter presented a review of research demonstrating that learned expectancies mediate behavior. A drug-taking situation illustrated the associative learning processes that gave rise to a sequence of three expectancies: receiving a drug, S-S∗d; type of drug effect, S∗d-Rd; and the consequence, Rd -S∗. Experiments that manipulated single expectancies revealed their causal inﬂuence on the intensity of the drug effect and the type of behavioral response. Overall, evidence from various laboratories and surveys implicates behavioral disinhibition and impulsivity in the risk for drug abuse. The research reviewed in this section highlighted the role of expected consequences of behavior and demonstrates how particular expectations give rise to disinhibition and impulsivity. The mediating role of expectancy in disinhibition and impulsivity was considered both in terms of the acute reactions to a drug (e.g., impaired inhibitory control) and in terms of stable personality characteristics of the individual drug abuser (e.g., impulsiveness). These ﬁndings provide new perspectives on how drugrelated expectancies affect basic mechanisms of

2/15/2011 7:45:10 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

behavioral control and inﬂuence the risk for drug abuse. The chapter also called attention to the very broad general potential application of a learning analysis of expectancies to other types of drugs and beyond drug-taking situations. Such future research was considered to broaden the opportunity to obtain new information and insight on other types of problematic behavior.

ACKNOWLEDGMENTS This work was supported by grants R01 AA12895 from the National Institute on Alcohol Abuse and Alcoholism, and by grants from the Alcoholic Beverage Medical Research Foundation and the Natural Sciences and Engineering Research Council of Canada. The authors would like to thank Jaime Blackburn for her editorial assistance in preparing this chapter.

REFERENCES Altman, J., Everitt, B. J., Glautier, S., Markou, A., Nutt, D., Oretti, R. Phillips, G. D., & Robbins, T. W. (1996). The biological, social and clinical bases of drug addiction: Commentary and debate. Psychopharmacology, 125, 285–345. Barkley, R. (2006). Attention-deﬁcit hyperactivity disorder (3rd ed.). A handbook for diagnosis and treatment. New York, NY: Guilford Press. Barry, H., Wagner, A. R, & Miller, N. E. (1962). Effects of alcohol and amobarbital on performance inhibited by experimental extinction. Journal of Comparative and Physiological Psychology, 55, 464–468. Beirness, D., & Vogel-Sprott, M. (1984). Alcohol tolerance in social drinkers: Operant and classical conditioning effects. Psychopharmacology, 84, 393–397. Bolles, R. C. (1972). Reinforcement, expectancy and learning. Psychological Review, 79, 394–409. Bolles, R. C. (1979). Learning theory (2nd ed.). New York, NY: Holt, Reinhart and Winston. Brown, S. A. (1993). Drug effect expectancies and addictive behavior change. Special Section: Motivation and addictive behaviours. Experimental and Clinical Psychopharmacology, 1, 55–67. Cherpitel, C. J. (1993). Alcohol, injury, and risktaking behavior: Data from a national sample.

10-Schachtman-10.indd 231

231

Alcohol: Clinical and Experimental Research, 17, 762–766. Christiansen, B. A., Goldman, M. S., & Inn, A (1982) The development of alcohol-related expectancies in adolescents: Separating pharmacological from social learning inﬂuences. Journal of Consulting and Clinical Psychology, 50, 336–344. Collins, R. L. (1993). Drinking restraint and risk for alcohol abuse. Experimental and Clinical Psychopharmacology, 1, 44–54. Colwill, R. M., & Rescorla, R. A. (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 20, pp. 55–104). New York, NY: Academic Press. Conger, J. J., (1956). Reinforcement theory and the dynamics of alcoholism. Quarterly Journal of Studies on Alcohol, 17, 296–305. Cremona-Meteyard, S. L, & Geffen, G. M. (1994). Event-related potential indices of visual attention following moderate to severe closed head injury. Brain Injury, 8, 541–558. de Wit, H., Crean, J., & Richards, J. B. (2000). Effects of d-amphetamine and ethanol on a measure of behavioral inhibition in humans. Behavioral Neuroscience, 114, 830–837. DiChiara, G., Acquas, E., & Tanda, G. (1996). Ethanol as a neurochemical surrogate of conventional reinforcers: The dopamine-opiod link. Alcohol, 13, 13–17. Dickinson, A., & Balleine, B. (1995). Motivational control of instrumental action. Current Directions in Psychological Science, 4, 162–167. Duncan, J. (1981). Directing attention in the visual ﬁeld. Perception and Psychophysics, 30, 90–93. Easdon, C. M., & Vogel-Sprott, M. (1996). Drugfree behavioral history affects social drinkers’ tolerance to a challenge dose of alcohol. Journal of Studies on Alcohol, 57, 591–597. Easdon, C. M., & Vogel-Sprott, M. (2000). Alcohol and behavioral control: Impaired response inhibition and ﬂexibility in social drinkers. Experimental and Clinical Psychopharmacology, 8, 387–394. Fillmore, M. T. (2003). Drug abuse as a problem of impaired control: Current approaches and ﬁndings. Behavioral and Cognitive Neuroscience Reviews, 2, 179–197. Fillmore, M. T. (2004). Environmental dependence of behavioral control mechanisms: Effects of alcohol and information processing demands.

2/15/2011 7:45:10 PM

232

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Experimental and Clinical Psychopharmacology, 12, 216–223. Fillmore, M. T. (2007). Acute alcohol-induced impairment of cognitive functions: Past and present ﬁndings. International Journal on Disability and Human Development, 6, 115–125. Fillmore, M. T., & Blackburn, J. (2002). Compensating for alcohol-induced impairment: Alcohol expectancies and behavioral disinhibition. Journal of Studies on Alcohol, 63, 237–246. Fillmore, M. T., Blackburn, J. S., & Harrison, E. L. (2008). Acute disinihibiting effects of alcohol as a factor in risky driving behavior. Drug and Alcohol Dependence, 95, 97–106. Fillmore, M. T., Carscadden, J., & Vogel-Sprott, M. (1998). Alcohol, cognitive impairment and expectancies. Journal of Studies on Alcohol, 59, 174–179. Fillmore, M. T., Mulvihill, L. E., & Vogel-Sprott, M. (1994). The expected drug and its expected effect interact to determine placebo responses to alcohol and caffeine. Psychopharmacology, 115, 383–388. Fillmore, M. T., & Rush, C. R. (2006). Polydrug abusers display impaired discrimination-reversal learning in a model of behavioral control. Journal of Psychopharmacology, 20, 24–32. Fillmore, M. T., Rush, C. R., & Marczinski, C. A. (2003). Effects of d-amphetamine on behavioral control in stimulant abusers: The role of prepotent response tendencies. Drug and Alcohol Dependence, 71, 143–152. Fillmore, M. T., & Vogel-Sprott, M. (1992). Expected effect of caffeine on motor performance predicts the type of response to placebo. Psychopharmacology, 106, 209–214. Fillmore, M., & Vogel-Sprott, M. (1994). Psychomotor performance under alcohol and under caffeine: Pharmacological and expectancy effects. Experimental and Clinical Psychopharmacology, 2, 319–328. Fillmore, M., & Vogel-Sprott, M. (1995a). Behavioral effects of alcohol in novice and experienced drinkers: Alcohol expectancies and impairment. Psychopharmacology, 122, 175–181. Fillmore, M., & Vogel-Sprott, M. (1995b). Behavioral effects of combining alcohol and caffeine: The contribution of drug-related expectancies. Journal of Experimental and Clinical Psychopharmacology, 3, 33–38.

10-Schachtman-10.indd 232

Fillmore, M., & Vogel-Sprott, M. (1995c). Expectancies about alcohol-induced motor impairment predict individual differences in responses to alcohol and placebo. Journal of Studies on Alcohol, 56, 90–98. Fillmore, M. T., & Vogel-Sprott, M. (1996). Evidence that expectancies mediate behavior under alcohol. Journal of studies on Alcohol, 57, 598–603. Fillmore, M. T., & Vogel-Sprott, M. (1997). Resistance to cognitive impairment under alcohol: The role of environmental consequences of cognitive performance. Experimental and Clinical Psychopharmacology, 5, 251–255. Fillmore, M. T., & Vogel-Sprott, M. (1999). An alcohol model of impaired inhibitory control and its treatment in humans. Experimental and Clinical Psychopharmacology, 7, 49–55. Fillmore, M. T., & Vogel-Sprott, M. (2000). Response inhibition under alcohol: Effects of cognitive and motivational conﬂict. Journal of Studies on Alcohol, 61, 239–246. Fillmore, M. T., Vogel-Sprott, M., & Gavrilescu, D. (1999). Alcohol effects on intentional behavior: Automatic and controlled processes. Experimental and Clinical Psychopharmacology, 7, 372–378. Finn, P. R., Kessler, D. N., & Hussong, A. M. (1994). Risk for alcoholism and classical conditioning to signals for punishment: Evidence for a weak behavioral inhibition system? Journal of Abnormal Psychology, 103, 293–301. Fowles, D. C. (1987). Application of a behavioral theory of motivation to the concepts of anxiety and impulsivity. Journal of Research on Personality, 21, 417–435. Goldberg, L., & Havard, J. (1968). Research on the effects of alcohol and drugs on driver behaviour and their importance as a cause of road accidents. Paris, France: Organization for Economic Co-operation and Development. Goldman, M., Brown, S., & Christiansen, B. (1987). Expectancy theory: Thinking about drinking. In H. Blane & K. Leonard (Eds.), Psychological theories of drinking and alcoholism (pp 181–226). New York, NY: Guilford Press. Goldman, M. S., Del Boca, F. K., & Darkes, J. (1999). Alcohol expectancy theory: The application of cognitive neuroscience. In: K. E. Leonard & H. T. Blane (Eds.), Psychological theories of drinking and alcoholism (2nd ed., pp. 203–246). New York, NY: Guilford Press.

2/15/2011 7:45:10 PM

LEARNING, EXPECTANCY, AND BEHAVIORAL CONTROL

Goschke, T. (2003). Voluntary action and cognitive control from a cognitive neuroscience perspective. In: S. Massen, W. Prinz, & G. Roth (Eds.), Voluntary action: Brains, minds, and sociality (pp. 49–85). New York, NY: Oxford University Press. Grattan, K., & Vogel-Sprott, M. (2001). Maintaining intentional control of behavior under alcohol. Alcoholism: Clinical and Experimental Research, 25, 192–197. Grattan-Miscio, K., & Vogel-Sprott, M. (2005). Alcohol, intentional control, and inappropriate behavior: Regulation by caffeine or an incentive. Experimental and Clinical Psychopharmacology, 13, 48–55. Gray, J. A. (1976). The behavioral inhibition system: A possible substrate for anxiety. In: M. P. Feldman & A. Broadhurst (Eds.), Theoretical and experimental bases of behavior therapies (pp. 3–41). London, England: Wiley. Gray, J. A. (1977). Drug effects of fear and frustration. Possible limbic site of action of minor tranquilizers. In: L. L. Iverson, S. D. Iverson, & S. H. Snyder (Eds.), Handbook of psychopharmacology (Vol. 8, pp. 433–529). New York, NY: Plenum. Holloway, F. A. (1995). Low-dose alcohol effects on human behavior and performance. Alcohol Drugs Driving, 11, 39–56. Ito, T. A., Miller, N., & Pollock, V. E. (1996). Alcohol and aggression: A meta-analysis on the moderating effects of inhibitory cues, triggering events, and self-focused attention. Psychological bulletin, 120, 60–82. Jacoby, L. L. (1991). A process-dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory and Language, 30, 513–541. Jentsch, J. D., Olausson, P., De La Garza, R., & Taylor, J. R. (2002). Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology, 26, 183–190. Jentsch, J. D., & Taylor, J. R. (1999). Impulsivity resulting from frontostriatal dysfunction in drug abuse: Implication for the control of behavior by reward-related stimuli. Psychopharmacology, 146, 373–390. Kalant, H. (1989). Drug tolerance and sensitization: A pharmacological overview. In A. J. Goudie & M. W. Emmett-Oglesby (Eds.), Psychoactive

10-Schachtman-10.indd 233

233

drugs: Tolerance and sensitization (pp. 547–577). Clifton, NJ: Humana Press. Kerr, J. S., & Hindmarch, I. (1998). The effects of alcohol alone or in combination with other drugs on information processing, task performance and subjective responses. Human Psychopharmacology, 13, 1–9. Kirsch, I. (1985). Response expectancy as a determinant of experience and behavior. American Psychologist, 40, 1189–1202. Kirsch, I. (1999). How expectancies shape experience. Washington, DC: American Psychological Association. Kirsch, I., & Saperstein, G. (1999). Listening to Prozac but hearing placebo. In I. Kirsch (Ed.), How expectancies shape experience (pp. 303–320). Washington, DC: American Psychological Association. Kirsch, I., & Weixel, L. (1988). Double-blind versus deceptive administration of a placebo. Behavioral Neuroscience, 102, 319–323. Koob, G. F. (2003). Neuroadaptive mechanisms of addiction: Studies on the extended amygdala. European Neuropsychopharmacology, 13, 442–452. Koob, G. F., Rocio, M., Carrera, A., Gold, L. H., Heyser, C. J., Maldonado-Irizarry, C.,. . .Weisz, F. (1998). Substance dependence as a compulsive behavior. Journal of Psychopharmacology, 12, 39–48. Langenbucher, J. W., & Nathan, P. E. (1983). Psychology, public policy, and the evidence for alcohol intoxication. American Psychologist, 38, 1070–1077. Leigh, R. J., & Zee, D. S. (1999). The neurology of eye movements (3rd ed.). New York, NY: Oxford University Press. Logan, G. D. (1994). On the ability to inhibit thought and action: A user’s guide to the stopsignal paradigm. In: D. Dagenbach & T. H. Carr (Eds.), Inhibitory processes in attention, memory, and language. San Diego, CA: Academic Press. Logan, G. D., & Cowan, W. B. (1984). On the ability to inhibit thought and action: A theory of an act of control. Psychological Review, 91, 295–327. Logan, G. D., Schachar, R. J., & Tannock, R. (1997). Impulsivity and inhibitory control. Psychological Science, 8, 60–64. Lyvers, M. (2000). “Loss of control” in alcoholism and drug addiction: A neuroscientiﬁc interpretation. Experimental and Clinical Psychopharmacology, 8, 225–249.

2/15/2011 7:45:10 PM

234

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Malloy, P., Bihrle, A., Duffy, J., & Cimino, C. (1993). The orbitomedial frontal syndrome. Archives of Clinical Neuropsychology, 8, 185–201. Mann, R. E., & Vogel-Sprott, M. (1981). Control of alcohol tolerance by reinforcement in nonalcoholics. Psychopharmacology, 75, 315–320. Marczinski, C. A., Combs, S. W., & Fillmore, M. T. (2007). Increased sensitivity to the disinhibiting effects of alcohol in binge drinkers. Psychology of Addictive Behaviors, 21, 346–354. Marczinski, C. A., & Fillmore, M. T. (2003). Pre-response cues reduce the impairing effects of alcohol on the execution and suppression of responses. Experimental and Clinical Psychopharmacology, 11, 110–117. Marlatt, G. A., & Gordon, J. R. (Eds.). (1985). Relapse prevention: Maintenance strategies in the treatment of addictive behaviors. New York, NY: Guilford Press. McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: I. An account of basic ﬁndings. Psychological Review, 88, 375–407. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. Miller, J., Schaffer, R., & Hackley, S. A. (1991). Effects of preliminary information in a go versus no-go task. Acta Psychologica, 76, 241–292. Mulvihill, L. E., Skilling, T. A., & Vogel-Sprott, M. (1997). Alcohol and the ability to inhibit behavior in men and women. Journal of Studies on Alcohol, 58, 600–605. Nigg, J. T. (2006). What causes ADHD? Understanding what goes wrong and why. New York, NY: Guilford Press. Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3–25. Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980) Attention and the detection of signals. Journal of Experimental Psychology: General, 109, 160–174. Rawana, E., & Vogel-Sprott, M. (1985). The transfer of alcohol tolerance, and its relation to reinforcement. Drug and Alcohol Dependence, 16, 75–83. Rescorla, R. A. (1990). The role of information about the response-outcome relation in instrumental discrimination learning. Journal of Experimental Psychology: Animal behavior processes, 16, 262–270.

10-Schachtman-10.indd 234

Sdao-Jarvie, K., & Vogel-Sprott, M. (1991). Response expectancies affect the acquisition and display of behavioral tolerance. Alcohol, 8, 491–498. Siegel, S. (1989). Pharmacological conditioning and drug effects. In A. J. Goudie & M. W. EmmettOglesby (Eds.), Psychoactive drugs: Tolerance and sensitization (pp. 115–180). Clifton, NJ: Humana Press. Sharkansky, E. J., & Finn, P. R. (1998). Effects of outcome expectancies and disinhibition on ad lib alcohol consumption. Journal of Studies on Alcohol, 59, 198–206. Steele, C., & Southwick, L. (1985). Alcohol and social behavior: I. The psychology of drunken excess. Journal of Personality and Social Psychology, 48, 18–34. Tolman, E. C. (1932). Purposive behavior in animals and men. New York, NY: Century. Vogel-Sprott, M. (1992). Alcohol tolerance and social drinking: Learning the consequences. New York, NY: Guilford Press. Vogel-Sprott, M. (2004). Drugs, behavior and environmental sources of motivation: Bridging a gap. In R. Bevans & M. Bardo (Eds.), Motivational factors in the etiology of drug abuse (Vol 50, (pp. 235–260). Lincoln, NB: University of Nebraska Press. Vogel-Sprott, M., & Fillmore, M. (2002). Expectancy and behavioral effects of socially used drugs. In I. Kirsch (Ed.), How expectancies shape experience (pp. 215–232). Washington, DC: American Psychological Association. Wells, J., Greene, M., Foss, R., Ferguson, S., & Williams, A. (1997). Drivers with high BACs missed at sobriety checkpoints. Journal of Studies on Alcohol, 58, 513–517. Zack, M., & Vogel-Sprott M. (1995). Behavioral tolerance and sensitization to alcohol in humans: The contribution of learning. Experimental and Clinical Psychopharmacology, 3, 396–401. Zack, M., & Vogel-Sprott, M. (1997). Drunk or sober? Learned conformity to behavioral standards. Journal of Studies on Alcohol, 58, 495–501. Zinatelli, M., & Vogel-Sprott, M. (1993). Behavioral tolerance to alcohol in humans is enhanced by prior drug-free treatment. Journal of Experimental and Clinical Psychopharmacology, 1, 194–199.

2/15/2011 7:45:10 PM

CHAPTER 11 Applications of Contemporary Learning Theory in the Treatment of Drug Abuse Danielle E. McCarthy, Timothy B. Baker, Haruka M. Minami, and Vivian M. Yeh

Addictive drug use can be viewed as an overlearned avoidance behavior maintained on a variable reinforcement schedule. This chapter explores the implications of this conceptualization of drug use and contemporary models of avoidance learning, and stimulus and response learning more generally, for drug treatment in humans. Existing treatments are analyzed in terms of their ability to alter the learning that sustains chronic use and prompts relapse. The relevance of stimulus and response learning is discussed, along with the applicability of animal learning research to human drug use. Suggestions for novel treatments are outlined based on basic research on Pavlovian and operant conditioning and extinction. The chapter focuses on tobacco addiction but has implications for other drugs of abuse.

Drug and alcohol use are learned behaviors that have enormous societal and individual costs. In 2005, addictive drug use cost federal, state, and local governments an estimated $468 billion, most of this going to cover the health-related consequences of use (National Center on Addiction and Substance Abuse, 2009). Additional costs in health care and lost productivity are borne by private entities and individuals. Substance abuse also causes hundreds of thousands of premature deaths among Americans each year. In 2000, tobacco use was the cause of an estimated 435,000 deaths, alcohol use accounted for 85,000 additional deaths, and illicit drug use led to 17,000 more deaths (Mokdad, Marks, Stroup, & Gerberding, 2004). The public health impact of drug abuse is enormous. Changing these costly behaviors has proven difﬁcult, and relatively little of the money spent on drug use is spent on treatment (National Center on Addiction and Substance Abuse, 2009). This chapter will focus on ways that treatment may be improved based on a richer theoretical

understanding of what sustains drug use and how to modify it. The learning that sustains drug use and dependence has been subjected to decades of careful study (e.g., O’Brien, 1976; Wikler, 1948). For instance, researchers have studied the associative bases of important addictive phenomena, such as tolerance, withdrawal, and craving (e.g., Koob & Le Moal, 1997, 2008; Siegel, 1979; Siegel, Baptista, Kim, McDonald, & Weise-Kelly, 2000; Volkow, Fowler, & Wang, 2003; Volkow, Fowler, Wang, Baler, & Taleng, 2009) and have explored the roles of dopamine, norepinephrine, and glutamate in mediating drug-induced neural plasticity (Hyman, 2005). These data show that drug addiction induces robust learning reinforced by the direct effects of drugs on the brain (Hyman, 2005). Debate continues, however, about the nature of the learning that is central to addiction. Whereas some hold that reinforcement learning, particularly negative reinforcement through withdrawal relief, is the dominant determinant of continued drug use 235

11-Schachtman-11.indd 235

2/15/2011 7:45:33 PM

236

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(Baker, Piper, McCarthy, Majeskie, & Fiore, 2004; Solomon, 1977; Wikler, 1948), other scholars focus on incentive learning and assert that drugs alter the reward learning system and inﬂate the incentive value of drug-related cues (de Wit & Stewart, 1981; Robinson & Berridge, 2003). In the former case, the emphasis is on the consequences of drug use behavior, whereas in the latter case, the emphasis is on the antecedents of the behavior and the direct effect of drugs on stimulus learning. Each of these models accounts for some important, documented phenomena in addicted individuals and makes testable predictions, some of which overlap across models. The aim of this chapter is not to review these models in detail or to evaluate their respective merits and ability to account for extant observations (see Baker et al., 2004 for such an analysis, Everitt et al., 1999). Instead, the aim of the current chapter is to examine what integrated models of learning can tell us about how to change addictive drug use. That is, we focus less on controversies regarding the etiology of drug abuse and dependence and focus more on ways to alter the associative processes that sustain dependent substance use. We ﬁrst review contemporary models of Pavlovian conditioning and instrumental or operant conditioning to highlight the learning principles and characteristics likely to have relevance to addiction. The terms and abbreviations that we use throughout the chapter are deﬁned in Table 11.1. We stray from conventional abbreviation systems (also in Table 11.1) slightly by using the term outcome (and abbreviation O) to describe the unconditioned stimulus of drug effects. We do this because we want to emphasize that the unconditioned stimuli central to drug addiction are the reinforcing effects of the drug on the central nervous system. After this brief review of Pavlovian and instrumental/operant learning, we then review the literature on relapse phenomena that occur and what is currently known about ways to prevent such relapses by altering the parameters of extinction training. We recognize that it is not possible to attribute drug self-administration to a particular set of associative motivational processes, as we lack information about the underlying mechanisms

11-Schachtman-11.indd 236

that sustain drug use. We will, however, identify inﬂuences on drug motivation and selfadministration and derive theoretical and clinical implications from these. To that end, we will classify and analyze existing drug treatment approaches based on the learning mechanisms that they are likely to activate. We will use basic animal research, clinical research, and contemporary learning theory to analyze existing treatments for addictive disorders and to suggest novel treatment approaches. We will focus on tobacco addiction, but we believe that the change processes discussed in the context of tobacco cessation are likely relevant to other substance use disorders as well.

TYPES OF LEARNING Current understanding of learning processes suggests that instrumental behavior is not solely a function of reinforcement history but is also inﬂuenced by stimulus learning (Bolles, 1972; Bouton, 2007). Indeed, drug use appears to be inﬂuenced by a complex mix of reinforcement, Pavlovian conditioning, and psychomotor activation (Everitt et al., 1999). Instrumental behavior is no longer thought to be exclusively inﬂuenced by behavioral consequences that simply stamp in stimulus-response associations (Thorndike, 1911). Instead, operant/instrumental conditioning, which we call response learning, after Bouton (2007), is viewed as a complex type of learning that is inﬂuenced by both the antecedents and consequences of behavior. An example of the diverse associations learned by smokers is shown in Figure 11.1. Stimulus Learning

As shown in Figure 11.1, classical or Pavlovian conditioning, which we call stimulus learning, also after Bouton (2007), occurs in parallel with response learning in instrumental conditioning paradigms and in human drug addiction (Everitt et al., 1999; Hyman, 2005). This is represented by the stimulus and outcome (S-O) associations and stimulus and response (S-R) associations shown in Figure 11.1. For example, smokers learn that the act of smoking is followed by

2/15/2011 7:45:33 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE Table 11.1

237

Terminology Legend

Construct

Deﬁnition

Current Abbreviation

Alternative Abbreviations

Unconditioned stimulus

A biologically signiﬁcant stimulus or event; in this chapter, the Os of interest are drug related and include both direct effects of drug and the symptoms and signs of drug withdrawal

O

S, S∗, US

Conditioned stimulus

A previously neutral stimulus that obtains signiﬁcance through association with an unconditioned stimulus

S

S, CS

Conditioned response

A response elicited by a conditioned stimulus associated with a biologically signiﬁcant stimulus

R

R, CR

Pavlovian/classical conditioning

Learning in which individuals form associations between conditioned stimuli and unconditioned stimuli

S-O

S-S, S-S∗, CS-US

Instrumental/operant conditioning

Learning in which individuals form associations between behavioral responses and outcomes

R-O

R-S

Habit learning

Learning in which stimuli come to evoke responses automatically even if the reinforcer is devalued

S-R

Incentive learning

Learning in which the value of a reinforcer is associated with certain stimuli or contexts, thus modulating the incentive value of Ss associated with the reinforcer

Occasion setter

A stimulus that signals whether a learned S-O association will hold in Pavlovian conditioning

Positive

Signals that O will follow S

S+

Negative

Signals that O will not follow S

S-

Discriminative stimulus

A stimulus that signals that a response will be reinforced in instrumental conditioning

Positive feature

A discriminative stimulus that indicates that responding will be reinforced

SD

S+

Negative feature

A discriminative stimulus that indicates that responding will not be reinforced

S∆

S-

A conditioned stimulus that is associated with the absence of a biologically important stimulus. Unlike occasion setters and discriminative stimuli that are not related directly to the presence or absence of the unconditioned stimulus (O), conditioned inhibitors are directly associated with the absence of the O.

S-

CS-

Conditioned inhibitor

11-Schachtman-11.indd 237

2/15/2011 7:45:33 PM

238

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

S1Stimuli • Cigarette • Lighter • Car • Alcohol • Negative affect

S1-O1 S1-R1

R1 Conditioned Responses: • Seek cigarettes • Anticipatory pleasure • Smoke cigarettes • Compensatory response

SNegative occasion setter

R1-O1

O1 Nicotine Outcomes: • Relief from craving, withdrawal • Pleasure • Improved concentration • Enhanced incentive salience

SD Discriminative Stimuli • Time since last cigarette • Context

S2-O2 S2 Withdrawal Stimuli R Alternative Responses: R2-O2 • Smoking restrictions S2-R2 •2 Pursuit of other • Illness symptoms reinforcers • Restlessness • Coping • Anxiety

O2 Abstinence Outcomes: • Craving • Withdrawal • Increased incentive salience of nicotine cues • Money savings • Better health

Figure 11.1 Learned associations between stimuli (S), responses (R), and biologically important out-

comes (O) in a hypothetical smoker. Learned associations inﬂuence both smoking behavior (top) and alternative, abstinence behaviors (bottom). On the appetitive side (top) the smoker learns to associate both exteroceptive stimuli (S1, e.g., lighters) and interoceptive stimuli (e.g., deprivation effects) with both a set of behaviors (R1) that culminate in nicotine self-administration, and the effects of nicotine on the central nervous system (O1). The S-R connection is the basis of habit learning. The smoking response (R1) is reinforced by the positive consequences of smoking (O1). Occasion setters may moderate R1-O1 associations, such that smoking is only expected to result in positive consequences in certain situations (e.g., when sufﬁcient time has passed since the last cigarette was smoked). On the withdrawal side (bottom), smokers learn to associate stimuli (S2) with alternative responses (R2, e.g., coping, eating) and both the short-term, negative effects and long-term, positive effects of abstinence (O2). Smoking (R1) serves as a negative occasion setter that informs the smoker that the warning stimuli (S2, i.e., early signs of withdrawal or stimuli associated with withdrawal) are not going to lead to the feared outcomes of withdrawal (O2). In this way, smoking serves as a safety signal that reduces fear of withdrawal, in addition to being reinforced directly (O1).

particular consequences or reinforcing outcomes (Os, e.g., short-term relief of craving and withdrawal, improvement in concentration, sensory pleasure), represented by the R-O association in Figure 11.1, and also learn that stimuli such as lighters and contexts (e.g., the car, the back porch) are associated with both the act of smoking (R) and these desired effects of nicotine (O). The S-O linkage may cause drug-associated Ss to induce expectancies of reward through Pavlovian conditioning, or stimulus learning (Bolles, 1972).

11-Schachtman-11.indd 238

As such, drug users learn to expect drug effects when in the presence of Ss that have been paired with these Os through associative learning mechanisms. Drug effects may directly promote long-term memory or consolidation of these associations (e.g., Berke, 2003; Fuji, Jia, Yang, & Sumikawa, 2000; Mansvelder & McGehee, 2000; Matsuyama, Matsumoto, Taira, & Tomoyuki, 2000), as shown by the feedback arrow representing incentive sensitization and cognitive facilitation in Figure 11.1.

2/15/2011 7:45:33 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

Expectancies about S-O and R-O associations appear to be importantly modulated by contexts that serve as occasion setters that convey information about whether the S-O relationship will hold (in Pavlovian conditioning) or discriminative stimuli that indicate whether the R-O relationship will hold (in instrumental/operant conditioning). Studies have demonstrated that animals that learn to press a lever for food when sated do not lever press when hungry in a subsequent extinction test (Balleine, 1992). This suggests that the incentive learning that prompts lever pressing, presumably based on the R-O expectation that lever pressing will deliver the rewarding food pellets and associated sensory experiences, is speciﬁc to the hunger/satiety state (S) in which training was completed. In this way, R-O learning may be modulated by discriminative stimuli (SDs) present during acquisition. Humans show similar modulation of responding by SDs. Individuals who have experienced drug-related reward in the context of withdrawal, and formed an R-O expectancy that using drugs will reduce withdrawal, exhibit increased attention to and responding for drugrelated SDs, relative to subjects who have not formed this expectancy (Gloria et al., 2009; Hutcheson, Everitt, Robbins, & Dickinson, 2001). A similar effect has been shown in nicotine; conditioned Rs to drug Ss seems to depend on explicit awareness of the contingency and expectancies that the drug O will follow the S or R (Hogarth & Duka, 2006). Withdrawal thus serves as an SD for drug use for individuals who recognize that drug use is more rewarding in withdrawn than in nonwithdrawn states. In humans, a simple verbal instruction that responding will be rewarded, without any prior training, is sufﬁcient to evoke responding in a cognitively mediated manner (Lovibond & Shanks, 2002). Expectancies shaped by experience or instruction thus appear to play an important role in inﬂuencing goal-directed behavior. Vogel-Sprott and Fillmore (Chapter 10, this volume) discuss the types of expectancies that develop in addiction and inﬂuence behavior in greater detail. Drugs may inﬂuence responding via another mechanism as well. The direct physiological

11-Schachtman-11.indd 239

239

effects of drugs on dopaminergic transmission may heighten the incentive salience of Ss that have been paired with O (Berridge, 2007; Robinson & Berridge, 1993, 2003). This incentive sensitization process is distinct from the processes of hedonic valuation of O, and learning of S-O associations, and seems to account for the ability of drug cues to serve as “motivational magnets” (Berridge, 2007, p. 409) that elicit powerful wanting (whether conscious or not) of drugs (Berridge, 2007). Drugs may inﬂate CS incentive salience directly through dopaminergic mechanisms that are separable from associative learning mechanisms (Berridge, 2007; Robinson & Berridge, 1993). This incentive sensitization may make drug effects particularly motivating or appealing to the individual, so that drug Ss and anticipated drug Os can eclipse natural rewards in terms of salience and incentive value (Berridge, 2007; Hyman, 2005). In contrast to the expectancy-driven learning described earlier, incentive sensitization can be implicit and does not appear to depend on awareness (Berridge, 2007) or to be dependent on context (Mead, Crombag, & Rocha, 2004). The degree to which such implicit incentive learning causes drug-seeking and drug-use Rs remains somewhat unclear, however (Hogarth & Duka, 2006). Another, somewhat controversial form of learning that is tied closely to experience and not mediated by declarative memory is called habit learning. In habit learning, consequent to repeated pairings with drug use, Ss provoke motivation to use drugs that is relatively impervious to devaluation of the reinforcement (O) after initial acquisition of the drug seeking and use Rs (Bouton, 2007). That is, habit learning would support automatic, reﬂexive execution of the response even if the contingent outcome value changed (e.g., by inducing selective satiation through a nicotine infusion and then assessing smoking behavior in response to cue exposure, or by pairing smoking with an aversive taste or nausea in addicted smokers; Berridge, 2007). Dickinson and colleagues (Dickinson, Wood, & Smith, 2002) have conducted animal research that suggests that S-R habit learning may be more likely to drive responding for ethanol than

2/15/2011 7:45:34 PM

240

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

food in animals, whereas cognitive expectancies of R-O associations remain more important in promoting seeking of natural rewards such as food. Thus, addictive drugs may be more likely to induce habit learning than are natural rewards, although this may not appear to hold for cocaine or heroin (Hutcheson et al., 2001). Such habit learning is thought to be gradual and incremental via learning of associations between Ss and Rs, often through trial-and-error learning that can be shown to occur without consciousness in amnestic patients (Bayley, Frascino, & Squire, 2005; Knowlton, Mangels, & Squire, 1996; Yin, Knowlton, & Balleine, 2005). Interval schedules of reinforcement (in which delivery O is contingent on time and behavior) may facilitate habit learning more than ratio schedules of reinforcement (in which delivery of O is contingent solely on behavior), according to some animal research (Dickinson, Nicholas, & Adams, 1983). Habit learning is controversial because it is difﬁcult to demonstrate, even in tightly controlled animal studies (Hutcheson et al., 2001; Olmstead, Lafond, Everitt, & Dickinson, 2001), but some research suggests that this type of learning occurs and is mediated by neural substrates (putamen, caudate) that are distinct from those that undergird response learning and declarative memory (hippocampus, amygdala) (Bayley et al., 2005; Knowlton et al., 1996). Recent evidence suggests that habit learning may be associated with compulsive behavior symptoms in clinical disorders, including Tourette syndrome in which S-R learning appears to be impaired (Marsh et al., 2004). The existence and signiﬁcance of habit learning in human addiction is less clear, although Tiffany (1990) proposed that automatic action schemata (akin to habits) are important in addiction, even while cognitive action plans remain important (akin to expectancy-mediated R-O learning). Thus, even if the S-R connections that are learned in addiction do not become truly independent of O devaluation, these associations are likely to develop such that Rs are evoked by Ss that have been paired with the R. This may help to account for instances of automatic use (as when the smoker reﬂexively reaches for cigarettes in response to smoking Ss without being aware of

11-Schachtman-11.indd 240

wanting to smoke) and may have important implications for treatment. For example, treatments that attempt to devalue drug Os by pairing use with aversive stimuli (e.g., Cannon, Baker, Gino, & Nathan, 1986; Howard, 2001), for example, are not likely to reduce habit (S-R) strength, which may reduce the efﬁcacy of such interventions. Incentive sensitization and habit learning may exert complementary effects on drug motivation and execution of self-administration Rs. For instance, there is evidence that incentive effects are modulated by the induction of drug deprivation. That is, there is evidence of greater motivational responding to drug Ss in the context of drug deprivation (Hogarth & Duka, 2006; McCarthy, Gloria, & Curtin, 2009). However, habit learning would produce execution of drug self-administration even in the context of recent drug use. Together these different inﬂuences might explain why smokers show stronger motivation to smoke while in withdrawal but will nevertheless reach for a cigarette immediately after extinguishing a previous one (Gloria et al., 2009; Zinser, Fiore, Davidson, & Baker, 1999). Another way in which stimulus learning may inﬂuence drug use behavior is through Pavlovianinstrumental transfer. This paradigm typically involves three phases: one in which the S is paired with an O, one in which instrumental responding for the reinforcer is established through R-O training, and one in which instrumental responding is tested in the presence or absence of the S from phase 1, usually in the absence of the O. In this paradigm, which has been tested in both animals (Dickinson & Balleine, 2002) and humans (Talmi, Seymour, Dayan, & Dolan, 2008), prior Pavlovian conditioning of an S with delivery of an O (e.g., noticing that smoking among your peers is associated with pleasant social interactions) increases the vigor of instrumental responses that are established via later instrumental learning (e.g., smoking yourself). Thus, one would expect that you would smoke more vigorously with peers than when alone due to Pavlovian instrumental transfer. That is, Ss that have been associated with Os passively (without involving instrumental responding) can later modulate instrumental

2/15/2011 7:45:34 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

responding for the same O. The S may serve as a discriminative stimulus that increases the availability of the representation of the O in working memory and therefore also increases the tendency to emit Rs that trigger delivery of the O. Holland (2004) suggested that this type of transfer becomes more important with extensive training in the instrumental response, when S-R associations grow stronger. This learning phenomenon may help us to make sense of some of the variation in drug seeking and use that we see across contexts. That is, some of the variability in drug use and craving observed across situations and individuals may reﬂect modulation of responding through Pavlovian conditioning history. Incentive sensitization by dopaminergic pathways may also account for variability in drug craving and use across situations. Ss that are present when dopamine activity increases (e.g., after eating, sex, or drug use) acquire greater incentive salience than they would have if presented without elevated dopamine (Berridge, 2007). This process appears to modulate strength of drug motivation when drug signals or cues (Ss) are presented in the context of an agent that increases dopamine activity in brain systems known to modulate cue incentive value (even if the activity was not induced by drug per se). For instance, if an animal is exposed to both an S previously paired with an addictive drug that stimulates increased dopamine activity, and is concurrently given a dose of the drug, it will demonstrate stronger drug motivational/ incentive effects to the drug cue than with either event by itself (drug or S presentation; e.g., Tindell, Berridge, Zhang, Pecina, & Adridge, 2005). This may help to account for both the excessive salience acquired by addictive drug Ss, and for the elevated salience of drug cues in the context of natural rewards (e.g., smoking after meals). Two other characteristics of stimulus learning merit mention here. First, it is important to note that learning about stimulus associations has less than perfect speciﬁcity. Generalization across stimulus gradients is a well-documented phenomenon. That is, animals and humans do not exhibit conditioned responding only in the

11-Schachtman-11.indd 241

241

presence of the speciﬁc S in whose presence the S-R or S-O associations were acquired. Pigeons, for example, will exhibit Rs to many stimuli that share common elements with the original S. Responding to stimuli tends to peak when the test trial stimulus is most like the S, except when a discriminative stimulus (S∆) that indicates that the reinforcer will not be delivered has also been trained. In the latter case, the peak in responses is shifted away from the exact S used in training, in the direction opposite to that of the training S∆ (Weiss & Weissman, 1992). That is, responding will be greatest for stimuli that are even more different from the S∆ than the S. This phenomenon has been observed in both Pavlovian and instrumental learning (Weiss & Weisman, 1992). For instance, if symptoms of nicotine toxicity signal a lack of reward availability, this might increase smoking during periods of especially heightened withdrawal (to the extent that cues of heightened withdrawal are perceived as highly dissimilar to nicotine overdose effects). Generalization can increase as a function of time since learning, presumably reﬂecting forgetting, such that responding begins to occur to a greater range of stimuli following initial training (Riccio, Rabinowitz, & Axelrod, 1994). These generalization phenomena may have important implications for drug treatment. For example, this and the Pavlovian-instrumental transfer and dopaminergic effects described earlier suggest that counseling treatments should incorporate a broader deﬁnition of triggers (Ss) than is typical. Because of these processes that modulate incentive value and associative learning, Ss that can serve as triggers for relapse are not likely to be limited to the speciﬁc cues that have been paired with drug use in the past. This may be especially likely long after a person has stopped using the drug, when drug-conditioned associations might be elicited by more diffuse stimuli than would occur shortly after drug withdrawal. Recognition of these effects may help individuals in treatment to better anticipate and prepare for potential relapse risks. Second, stimulus learning is inﬂuenced by exposure to the stimuli that occurs prior to drug conditioning (e.g., via latent inhibition and blocking effects). In latent inhibition, pre-exposure to

2/15/2011 7:45:34 PM

242

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

an S reduces the formation of associations between the S and O in acquisition (Lubow & Moore, 1959). A similar effect occurs with preexposure to the O prior to conditioning (Domjan & Best, 1980). An implication of these wellknown phenomena is that novel, salient stimuli are likely to facilitate stronger learning than are familiar stimuli that do not elicit as much attentional processing. This has been most thoroughly documented in Pavlovian conditioning, but given the associations that form between S and R and between S and O in response learning, these phenomena are also important in instrumental/ operant conditioning. Blocking is a similar effect that reduces the strength of associative learning as a function of experience. This effect, demonstrated by Kamin (1969), refers to the fact that a new stimulus (S2) will not come to elicit strong conditioned responding if it signals an O that is already effectively predicted by a different stimulus (S1). For example, if a drug is reliably signaled by a visual cue (S1), an auditory cue (S2) will acquire little associative strength if it is later paired with the visual cue and both precede the drug; the auditory cue will be redundant and will fail to elicit strong drug-conditioned Rs. This is thought to reﬂect the fact that presentation of S1 primes retrieval of representations of the R or O. Because the R or O is already in working memory when the S2 appears, it receives less attention and processing when it is introduced in the compound conditioning trials (Wagner, 1981). The playing ﬁeld is not equal for all stimuli and attention to a stimulus appears to be important in establishing associations with it. This suggests that attempts to bring substance use under tight stimulus control may be undermined by blocking effects. Inveterate drug users likely have learned to predict both drug effects and withdrawal effects very well based on existing interoceptive and exteroceptive Ss. Thus, attempting to impose stimulus control, with the hope of reducing drug use by eventually reducing exposure to the CS, may be blocked by this previous learning. In summary, basic animal and human research indicates the cues paired with important outcome stimuli, such as drug effects, develop motivational signiﬁcance by nature of

11-Schachtman-11.indd 242

their association with the stimuli. This learning reﬂects distinct forms of learning (i.e., S-O expectancy learning, incentive sensitization, habit learning) with different neural substrates. The different forms of learning will also show differential sensitivity to modulation by both internal and external stimuli (e.g., occasion setters, discriminative stimuli). It is important to recognize that debates about the nature of drug learning and motivation are not settled, and this lack of understanding no doubt hinders attempts to devise treatments that are designed to inhibit or undermine this learning. Learning is modiﬁed in many important ways by many parameters (e.g., length of training, number of reinforcers available, stimulus exposure history, deprivation states, etc.), even in tightly controlled laboratory settings (e.g., Holland, 2004). These basic ﬁndings have important implications for treatment development. For example, devaluation of the Os associated with smoking is not likely to affect smoking behavior if it primarily reﬂects habit learning, whereas such devaluation may reduce the likelihood of smoking maintained by learned cognitive S-O representations. Response Learning

Instrumental/operant conditioning models emphasize the importance of consequences in shaping behavior. Behaviors (Rs) that are paired with favorable consequences (Os) are more likely to be repeated than behaviors that are paired with unfavorable consequences. Skinner referred to this as selection of behaviors by consequences (1969). Those behaviors that are positively reinforced with access to a positive stimulus (i.e., a reward) and those that are negatively reinforced with removal of a negative stimulus (i.e., escape from an aversive stimulus) are more likely to recur than are behaviors that are followed by positive punishment (i.e., introduction of a negative stimulus) or negative punishment (i.e., loss of a positive stimulus). However, instrumental/operant behavior is modulated by many factors, such as the type and magnitude of Ss involved, the latency between the R and O, the schedule of Os, deprivation, availability of other reinforcers, stimulus

2/15/2011 7:45:34 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

learning, and prior learning and reinforcement history (e.g., Capaldi, 1994; Catania, 1998; Flaherty, 1996; Herrnstein, 1970; Williams, 1988). For example, basic laboratory research has demonstrated that response learning is strongest when the consequences (O) of the operant behavior (R) are large in magnitude or highly salient or when the latency between the R and the O is short (e.g., Bower, 1961; Marlin, 1981). Indeed, long latencies in O delivery sometimes lead to what appears to be self-defeating behavior. Long latencies between R and O can lead to a pattern of responding for “specious reward,” as described by Ainslie (1975). In this situation, an individual responds in a way that confers the worse of two possible Os because of a delay in the delivery of the better O (Ainslie, 1975; Rachlin & Green, 1972). As such, behavior is not simply a function of the magnitude of reinforcement; it is also inﬂuenced by its immediacy. This process has been shown to be at work in drug users in laboratory research (e.g., Bickel & Johnson, 2003; Bickel, Odum, & Madden, 1999) and may help to explain the intransigence of drug use despite the fact that the reinforcement it provides (whether positive or negative) is modest compared to the long-term punishments associated with smoking (e.g., morbidity and mortality). Different schedules of reinforcement alter learning and extinction curves for operant behaviors. For example, ﬁxed ratio reinforcement schedules (in which a constant number of Rs is needed to elicit the O) tend to elicit faster acquisition than variable ratio schedules (in which the number of Rs needed to elicit the O varies across trials), whereas variable schedules retard extinction relative to ﬁxed schedules (Rescorla, 1999). With extended training, variable interval schedules (in which reinforcement is contingent on the timing of Rs) tend to lead to more robust habit (S-R) learning, as discussed earlier, than do ratio schedules (in which reinforcement is dependent on the number of Rs) (Dickinson et al., 1983). This may be relevant to addiction, because not every obtained drug O may be equally effective. For example, among smokers, research suggests that self-administration schedules (e.g., smoking a pack of cigarettes

11-Schachtman-11.indd 243

243

per day) yield nicotine dosing that exceeds receptor binding capacity in the brain (Brody et al., 2006). The ﬁrst few cigarettes of the day occupy critical receptors for several hours (Brody et al., 2006; Valette et al., 2003), which produces direct drug effects but also antagonizes (blocks) the effects of subsequent nicotine doses. This jibes well with smoker self-reports that indicate that the ﬁrst cigarette smoked of the day is often the most valued (Mehringer, Pomerlau, Snedcor, & Finkenauer, 2008). As such, clinical and pharmacological evidence suggests that smoking is maintained on a variable interval reinforcement schedule. (This makes clear that the O is actually not the drug per se, but rather its effects [Stewart, de Wit, & Eikelboom, 1984].) This is an interval schedule rather than a ratio schedule because reinforcement potential depends on drug metabolism and receptor availability rather than behavioral responses exclusively. Thus, basic research suggests that the reinforcement history of smokers may help to explain some of the difﬁculty in extinguishing the self-administration R (i.e., smoking). Another complication in response learning is the blurring of the nature of the consequences of a behavior (Os). Although research on appetitive behavior and avoidance behavior has proceeded along largely parallel tracks, some research suggests that the same Ss can elicit both approach and avoidance Rs under certain circumstances, as described by Daly and Daly (1991). For example, Ss associated with appetitive Os tend to elicit approach behaviors when the actor has a recent history of receiving expected or better than expected Os for engaging in approach Rs. Cues linked to appetitive stimuli can elicit avoidance Rs, however, if the actor has recently been frustrated by lack of reward or worse-than-expected Os on recent trials. Daly and Daly (1991) referred to this latter process as aversive counterconditioning of approach behaviors and studied this phenomenon extensively in animals. Amsel’s (1992) frustration theory described this phenomenon and highlighted the importance of learned expectancies in shaping behavioral Rs to discrete Ss and contexts. Additional study has established the existence and signiﬁcance of contrast effects in which responding to stimuli

2/15/2011 7:45:34 PM

244

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

varies as a function of past reinforcement (e.g., such that the same outcome can be disappointing if it is worse than expected or elating if it is better than expected) (Flaherty, 1996). This may have implications for drug users; doses of drug that yield disappointing effects (e.g., cigarettes smoked late in the day) may engender frustration and may actually enhance the rewarding effects of initial cigarettes smoked the next day, when nicotine effects are restored due to overnight abstinence. A similar phenomenon blends the role of approach and avoidance motivation in aversive conditioning paradigms. Scholars (e.g., Daly & Daly, 1991; McAllister & McAllister, 1991) have argued that aversive conditioning paradigms that establish Pavlovian associations between Ss (including purely temporal cues as in the Sidman, 1953 avoidance paradigm) and aversive Os (e.g., electric shock, pharmacologically induced sickness) can elicit approach behaviors. In this case, the actor is approaching safety signal Ss that have been paired with escape and avoidance of the aversive outcome in previous trials. Execution of avoidance Rs and exposure to Ss that have been associated with escape/avoidance in the past can elicit relief Os in the short-term and relaxation Os in the longer term and, thus, motivate future approach behaviors. Thus, escape/avoidance Rs are not just triggered by Ss of threat, but they can also be elicited by Ss that have been associated with relief/relaxation in the past and that begin to serve as attractive safety signals (McAllister & McAllister, 1991). Among drug users, this may mean that appetitive motivation for cigarettes may reﬂect negative reinforcement and avoidance learning. Smokers, for example, may be attracted to smoking-related cues and experience conditioned relief and relaxation in their presence, not just because cigarettes induce pleasure and heighten incentive salience but also because cigarettes provide relief from withdrawal. Thus, the conditioning and counterconditioning processes that support drug use may engender a complex blend of both approach and avoidance Rs that may help to explain the ambivalence and difﬁculty changing that smokers often experience. This may also mean that effective change efforts may need to

11-Schachtman-11.indd 244

address the multiple motives that sustain continued drug use. This brief review of response learning research highlights the complex nature of this important form of learning. A comprehensive review of the factors that modulate response learning is beyond the scope of this chapter. We cite illustrative examples to highlight the multitude of inﬂuences on instrumental behavior, but we will also focus on one particularly problematic area of response learning, avoidance learning, to highlight its relevance to drug treatment. Avoidance learning is of particular interest in our discussion of drug addiction because continued drug use among addicted individuals can be conceived of as an avoidance behavior reinforced by withdrawal relief, as we have argued previously (Baker et al., 2004; McCarthy, Curtin, Piper, & Baker, 2010). Avoidance learning has posed a challenge for learning theorists, and its study has prompted many important reﬁnements in our understanding of how learning unfolds. For example, the absence of observable distress prior to execution of an avoidance R in a well-trained individual was long thought to undermine the claim that the avoidance behaviors were alleviated by distress relief (McAllister & McAllister, 1991). If negative reinforcement is important, why do individuals continue to exhibit avoidance behavior even as their observable fear diminishes over training trials (McAllister & McAllister, 1991)? This question has clear relevance to individuals who use drugs, because most drug selfadministrations do not occur in the presence of frank, observable misery, but rather as routine behaviors that occur in positive, neutral, and negative mood states. If drug self-administration were reinforced via relief of the aversive withdrawal syndrome, why would the smoker not smoke only when showing signiﬁcant withdrawal distress? It is important to recognize that the avoidance R itself can come to serve as a negative occasion setter that signals nonoccurrence of an aversive O, an O that would occur if the avoidance response were not executed (De Houwer, Crombez, Baeyens, & 2005). Thus, an S previously paired with shock may elicit little fear if the avoidance R has effectively prevented

2/15/2011 7:45:34 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

the shock in the past (De Houwer et al., 2005). Negative occasion setters, such as the avoidance R, and conditioned inhibitors may help account for another curious characteristic of avoidance— its resistance to extinction (Rescorla, 2003). Extinction is slower and less robust for avoidance Rs than for appetitive Rs (Rescorla, 2003). In addition, punishment of avoidance Rs tends to strengthen the avoidance R rather than decrease it, as one would expect based on research on appetitive responding. This “selfpunitive behavior” effect (Dean & Pittman, 1991; Mowrer, 1947) is observed, for example, when rats trained to run away from a start box in a maze that has been previously paired with electric shock continue to run and exhibit fear when they were shocked during their escape from the start box during extinction (when the best response option would be to stay in the start box). This form of punished extinction tends to elicit greater fear than regular extinction. This fear presumably increases motivation to engage in the Rs previously associated with escape from this fear (e.g., running), even though this R is ineffective in the new contingency. This effect is not permanent; the avoidance R will eventually extinguish, but the rate of extinction is slowed by punishment (Dean & Pittman, 1991). This ﬁnding, too, has implications for substance abuse change efforts and may help explain why change is so difﬁcult to maintain. Social opprobrium, guilt, worry, and illness are consequences of smoking that may, in essence, punish this avoidance response and paradoxically make it more resistant to extinction. In summary, response learning is more complex than a superﬁcial analysis of the laws of effect (Thorndike, 1911) would suggest. Contemporary learning theory recognizes that many factors modulate the acquisition and expression of learned behavior. An important implication of this is that learned responses are not as predictable as they might seem based upon a molar view of an individual’s learning history. Rather, the expression of learned behavior is likely to be modulated by many factors, including the current availability of and latency for alternative Os, recent learning trials that may establish contrast effects, and mood state (that may serve as

11-Schachtman-11.indd 245

245

a discriminative stimuli for avoidance behavior). In addition, our previous review of stimulus learning suggests that behavior is inﬂuenced in important ways not just by past response contingencies but also by Ss that can alter outcome expectancies and modulate motivational states (see Chapter 10, this volume, for more on the role of expectancies in inﬂuencing complex behavioral responses to alcohol). Next, we will review the basic literature on extinction because this has critical relevance to behavior change efforts in addicted individuals and others seeking to control overlearned responses.

EXTINCTION Extinction has been the focus of rigorous and elegant research for several decades. This research has helped to establish some of the mechanisms by which dissociating S, R, and/or O can alter behavior. Extinction is now understood to be an active form of learning that establishes new, inhibitory connections, rather than simply a form of forgetting. Forgetting is an important phenomenon in its own right and the study of forgetting has elucidated the important roles that attention and elaborative processing of stimuli play in memory formation and the importance of retrieval cues in assessing learning and memory (Bouton, 1993; Mensink & Raajimakers, 1988; Miller, Kasprow, & Schachtman, 1986). Like extinction, this has implications for treatment of substance abuse (i.e., we should minimize interference and maximize retrieval of new coping skill learning in recovery, for example). Extinction is not just forgetting or unlearning, however, as has been well documented in laboratory research paradigms that show that both humans and animals retain learning that can be elicited under the right conditions, even after lengthy delays (Bouton, 2002). Bouton and his colleagues have collected extensive data demonstrating this in animal models of Pavlovian conditioning as well as appetitive and aversive instrumental Rs. Indeed, extinction learning seems to be prone to the same relapse phenomena as counterconditioning in which an S that has been paired with one O in the ﬁrst phase of learning is paired with

2/15/2011 7:45:35 PM

246

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

a different O (often of opposite valence) in the second phase of learning (Bouton, 2002). For example, one may countercondition a light S that was paired with an appetitive O, such as a sucrose solution, by later pairing the light with an aversive S (e.g., a quinine solution). Both extinction and counterconditioning thus entail new learning that must compete with existing learning to inﬂuence Rs. Research has documented several ways in which conditioned responding may return after extinction training that is designed to reduce such responding through repeated presentations of the S in the absence of the R or O (see Bouton, 2002 for an excellent discussion of this research and its implications for optimizing extinction). First, reacquisition occurs when extinction is followed by renewed pairings of the S and O (Bouton, 2002). Interestingly, the rate of responding during reacquisition is not always faster than acquisition, as one would expect given the now-established fact that extinction does not abolish the original learning acquired in training. Reacquisition can sometimes be slower than acquisition, particularly if extinction (S only) trials are occasionally interrupted by acquisition (S-O) trials or when the O is presented noncontingently during extinction training (Bouton, 2002). Occasional reinforced trials and noncontingent O presentations can slow reacquisition relative to typical extinction involving S exposure without the O alone as well (Bouton, Woods, & Pineño, 2004). For instance, it might be that exposure to environmental cues of smoking (e.g., coffee, a context paired with smoking breaks) will fail to extinguish as rapidly as would otherwise occur if a person used nicotine replacement products that would deliver nicotine noncontingent with the smoking ritual. (This interference with extinction would be reduced, in theory, if the O of nicotine replacement was markedly different from that produced by smoking.) Aversive or programmed smoking treatments that occur after a quit day could be considered reacquisition trials in that the smoking ritual is paired with nicotine effects in widely spaced, infrequent episodes. It is possible that these prescribed returns to smoking could protect against relapse of the R. On the other hand,

11-Schachtman-11.indd 246

spontaneously occurring lapses appear to have a disastrous effect on the likelihood of subsequent relapse (Kenford et al., 1994). It may be that the rewarding effects of lapses are so powerful (perhaps due to withdrawal relief) that they negate any reduction in reacquisition that would occur due to such occasional R-O pairings. Second, reinstatement occurs when an extinguished conditioned R recurs following noncontingent reexposure to the O (Bouton, 2002). This has been studied extensively in drug-addicted animals exposed to priming doses of drug. Conditioned responding for drug increases following a priming drug dose even after thorough extinction (de Wit & Stewart, 1981; Shaham, Adamson, Grocki, & Corrigall, 1997; Shaham, Shalev, Lu, de Wit, & Stewart, 2003). This effect holds if O reexposure occurs in the same context as the original conditioning, but it often does not hold if the testing context is different from the original conditioning context (Bouton, 2002). Reinstatement occurs in avoidance learning as well as in appetitive conditioning. Thus, repeated exposure to a tone S will extinguish fear and avoidance Rs previously established via tone (S) and shock (O) pairings. However, unpaired presentations of the shock O during extinction will reinstate the fear and avoidance Rs. The fact that this effect is stronger in the context of the original learning suggests that conditioning of the context (S) with the shock O may account for reinstatement. Pairing the extinction context with an aversive stimulus can also induce reinstatement of the R in the presence of the O, so the effect is not limited to the original context but may occur in other “dangerous” contexts as well (Bouton, 2002). Reinstatement may be relevant in drug use; negative affect (like that which occurs in withdrawal) may reinstate smoking to an S (e.g., a convenience store) following extinction (e.g., past trips to the convenience store without buying cigarettes or smoking). This could be problematic in addiction treatment, to the extent that withdrawal-like states of negative affect (recall that generalization often occurs across stimuli) are unavoidable and that any context in which drug has reduced withdrawal can induce reinstatement.

2/15/2011 7:45:35 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

A third type of relapse that occurs is also inﬂuenced by contextual conditioning. Renewal occurs when the R reemerges following extinction in response to a change in context (Bouton, 2002). Renewal can occur when an individual returns to the context in which acquisition occurred, or when an individual is tested in a novel context. It seems as though the absence of the extinction context as an S is important in this renewal effect because it can occur even when acquisition and extinction occur in the same context, but testing is done in a different context. In this way, contexts may serve as occasion setters or discriminative stimuli that signal to an individual whether the original learning will apply, or rather the exception to this established in extinction will apply (Bouton, 2002). A change of context, therefore, may be all that is needed for responding to reemerge. This effect has been documented for both appetitive and avoidance behaviors using diverse paradigms and both animal and human subjects (Bouton, 2002). In our clinical work with smokers we often hear that relapse was precipitated by a trip to either a familiar or new location, and this may be due to renewal. Renewal has implications for treatment because narrow treatment contexts may limit the robustness of the new learning established through extinction dissociating S, R, and O. This contextual effect may provide a rationale for giving clients recovery “homework” assignments to complete outside of the formal treatment context. It is important to remember that context can be deﬁned broadly, and it encompasses more than just the conﬁguration of physical cues in the external environment. Time and internal state are also important aspects of the context, as has been demonstrated in animal models using pharmacological manipulation of internal state, for example (Bouton, 2002). This has clear implications for treatment as well. Learning to dissociate drinking coffee and smoking may be easy enough when an individual is calm and may result in reduced smoking motivation in the presence of coffee in the short term, but this may not persist as a person’s mood changes and time passes. This suggests that conducting extinction in multiple contexts in spaced practice sessions may reduce the risk of renewal effects.

11-Schachtman-11.indd 247

247

Spontaneous recovery refers to recovery of conditioned responding that occurs when an S is presented following a signiﬁcant postextinction delay (Bouton, 2002). This effect depends upon the duration of the postextinction delay rather than on a change in context. This effect highlights the fragility of the learning established by extinction because changes in the temporal context are sufﬁcient to induce R recovery. Spontaneous recovery is reduced by spaced rather than massed extinction sessions, suggesting that exposure-based treatments (e.g., for anxiety treatment) might be most effective if they were conducted with relatively long intersession intervals. There is considerable evidence that smoking cessation treatment effects are restricted to the period of time that treatment is ongoing (McCarthy et al., 2008; Piasecki, Fiore, McCarthy, & Baker, 2002). To the extent that treatment effects could involve extinction of smoking-related conditioned responding, it is possible that spaced treatment sessions that extend well into the postquit period could increase long-term success. Recent reports of highly successful long-term therapy for smoking are consistent with this hypothesis (e.g., Hall et al., 2009). A ﬁnal relapse phenomenon that poses a challenge for change attempts is called resurgence. Resurgence refers to recovery of one conditioned response after an alternative response is no longer reinforced (Lieving & Lattal, 2003). This is usually demonstrated in a multiphase experiment in which an organism is ﬁrst trained to exhibit one conditioned response (R1), then R1 is extinguished, and the organism is trained to exhibit a different response (R2), and then is subjected to a period of nonreinforcement through extinction or extension of the reinforcement intervals. Recovered responding of the initial conditioned response (R1) is termed resurgence (Epstein & Skinner, 1980; Lieving & Lattal, 2003). Resurgence of alcohol seeking has been demonstrated in animals following nonreinforcement of food-seeking responses (Podlesnik, JimenezGomez, & Shahan, 2006). This suggests that training addicted individuals to exhibit a competing R (e.g., abstaining to earn the opportunity to earn monetary rewards, as in contingency

2/15/2011 7:45:35 PM

248

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

management programs) may only work while the reinforcement for the competing R is being offered, which is consistent with observed data (Pendergrast, Podus, Finney, Greenwell, & Roll, 2006). Resurgence could play a role in relapse to the extent that individuals develop new strategies (Rs) to deal with stress or negative affect, and their old drug use Rs extinguish. However, in the face of high levels of distress that do not respond to nondrug coping Rs, the individual might revert to the drug use R. The research on extinction summarized earlier suggests several ways to optimize extinction learning (Bouton, 2002). One approach is to increase the intensity of extinction training by conducting a massive number of trials (e.g., 800 extinction trials following 8 acquisition trials) in an effort to promote interference in retrieval of the original, excitatory learning (Denniston, Chang, & Miller, 2003). This has yielded reductions in renewal in both novel and original learning contexts in animal models (Denniston et al., 2003). In one sample massive extinction experiment, there were 100 times more extinction trials than acquisition trials, and this signiﬁcantly reduced renewal upon testing. This type of ratio may be very difﬁcult to achieve in addiction treatment, however. Consider that many smokers who seek treatment have been smoking for decades. A pack-a-day smoker who has been smoking for 20 years has had well over 1 million puffs from cigarettes, many of which served as acquisition trials on a variable reinforcement schedule. To increase this intensity of training 100-fold in clinical treatment settings would be impossible. To increase the number of extinction trials over acquisition trials by even 1 trial, or to extend the duration of new learning beyond that of the original learning, as some recommend (e.g., Driskell, Willis, & Copper, 1992), would also seem almost impossible, except for individuals who are neophyte drug users or smokers. Despite the difﬁculty of conducting more extinction than acquisition trials in addicted humans, there may be beneﬁts of massive extinction training to induce overlearning of a new response (R2) (Lang, Craske, & Bjork, 1999) with repeated training even after perfect performance is observed (because performance

11-Schachtman-11.indd 248

and learning are separate phenomena, repeated practice can result in improved learning and no change in performance [Bjork & Bjork, 1992]). The beneﬁts of overlearning may depend on the schedule of learning, however. As noted earlier, research suggests that spacing extinction trials or using variable intertrial intervals might make extinction learning more robust. There is little evidence from human clinical research that supports this claim, however (Cain, Blouin, & Barad, 2004; Craske et al., 2008). This approach to changing behavior has not yielded robust effects in either animals (Cain et al., 2004) or humans (e.g., Chambless, 1990; Foa, Jameson, Turner, & Payne, 1980). It is possible that manipulating factors such as the duration of S presentation during extinction could enhance extinction (e.g., to match S duration during acquisition and extinction; Cain et al., 2004). However, based on the evidence, it seems unlikely that this strategy would yield markedly stronger extinction effects. In addition, while this strategy may work in the laboratory where precise S durations are used during acquisition trials, it is unclear how relevant this would be to real-world conditions with uncontrolled exposures to smoking Ss. Bjork and Bjork (1992) assert that a variable schedule in which the intervals between extinction trials grows during training may optimize learning of the new inhibitory associations, even as it makes performance more difﬁcult. According to their theory of disuse, this is so because increases in permanent storage strength are greatest when temporary, limited retrieval strength is low. That is, extinction trials that require an individual to retrieve a memory that is not already activated yield stronger increments in learning (Bjork & Bjork, 1992; Lang et al., 1999). The validity of these predictions in humans has not been established, however, and more research in this area is required (see Craske et al., 2008). Conducting extinction in multiple contexts would also seem to hold promise for increasing the inhibition of a previously learned response (Bjork & Bjork, 1992; Bouton, 2002; Craske et al., 2008). Research shows, however, that this is not always so (Bouton, García-Gutiérrez, Zilski, & Moody, 2006); evidence suggests that

2/15/2011 7:45:35 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

this will help only if extinction occurs in more contexts than were linked to the original learning (Gunther, Denniston, & Miller, 1998). In the case of substance abuse, drug self-administration may occur in numerous, highly varied contexts. This is especially true of drugs that support frequent self-administration (e.g., tobacco, alcohol). This makes it unlikely that conducting extinction in additional contexts will meaningfully increase the generalization of extinction. There are strategies that seem to hold promise for enhancing extinction, but they are also in need of study. One strategy is to use retrieval cues or mental rehearsal strategies to help bridge inhibitory learning across contexts, and the second strategy is to habituate individuals to the O (Craske et al., 2008). The ﬁrst strategy might involve conducting extinction in the presence of a salient retrieval cue (e.g., a token of sobriety) and then testing in the presence of that cue (even in a different spatial and temporal context). Animal research supports the efﬁcacy of this strategy (Brooks & Bouton, 1994; Bouton et al., 2006). Collins and Brandon (2002) also found evidence to support the efﬁcacy of retrieval cues in reducing renewal effects following extinction training. College-aged heavy drinkers completed alcohol cue-exposure treatment without drinking in the presence of a salient environmental cue (a pencil with a teddy bear eraser and a neon orange clipboard) in one context and were then tested in a second context. Those who had the teddy bear pencil and clipboard retrieval cues during testing showed lower levels of selfreported craving and salivation than did those who completed extinction in the absence of the cues (Collins & Brandon, 2002). This did not work as well in a sample of alcohol-dependent individuals, however, as these subjects did not show the traditional renewal effect with a change of context (room) following a 1-day delay after extinction (Stasiewicz, Brandon, & Bradizza, 2007). Mental rehearsal of context features has been shown to reduce fear responding following a change of context after spider phobia extinction (exposure and response prevention) training (Mystkowski, Craske, Echiverri, & Labus, 2006). As such, there is some evidence that techniques that help individuals retrieve extinction

11-Schachtman-11.indd 249

249

training (i.e., in working memory) in novel contexts or those previously paired with the S and O can help to reduce renewal effects that may lead to relapse in addicted individuals. Another strategy might be based on the observation that exposure to the O may induce habituation to the O and thus reduce the motivational signiﬁcance of the stimulus (Craske et al., 2008). This strategy seems likely to have limited applicability in drug use, however, because sensitization, rather than habituation, to drug incentive effects occurs with greater exposure (Robinson & Berridge, 2003), and because sensitization to withdrawal effects has been demonstrated as well (e.g., for ethanol; Veatch & Becker, 2002). In summary, evidence regarding extinction training (and other forms of learning after acquisition of S-R, S-O, or R-S associations) suggests that whatever is learned second (e.g., extinction following acquisition) is more context bound than the original learning. This may reﬂect an increase in the ambiguity of stimulus meaning (negation of the organism’s causal or prediction model) when new learning suggests that the original learning is unreliable (Bouton, 2002). Whatever the reason, these data suggest that attempting to alter the learned associations depicted in Figure 11.1 among addicted individuals is an uphill battle, not just because drugs have potent, direct effects on the nervous system, but because diverse types of learning suggest that that which is learned ﬁrst is, in a sense, privileged. The original associations (even if inhibitory) are less contextually bound than are those learned in extinction and counterconditioning (Bouton, 2002). Thus, it is easier to fail to store and to fail to retrieve this new learning than it is to fail to retrieve the earlier acquired, strongly stored associations between stimuli and overlearned drug-seeking and drug-taking responses (Bjork & Bjork, 1992). In considering the ﬁndings reviewed earlier, it is important to recognize that some have argued that the simple extinction paradigms described earlier have modest direct relevance for addicted humans. One concern is that the simple experimental paradigms used to investigate extinction fail to capture the complexity of contingencies faced by humans. Hughes (2002),

2/15/2011 7:45:35 PM

250

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

for example, asserted that the simple means of dissociating S and R and R and O in animals are not equivalent to the complex situations faced by addicted individuals, in which drug use responses are often punished and alternative reinforcers are also contingent on inhibiting drug use responses. Although this criticism warrants consideration, it nevertheless seems prudent to explore the extent to which research on basic learning and extinction processes has heuristic value in suggesting intervention strategies or can account for important phenomena in the course or presentation of addictive disorders. In the next section, we will use the learning theories and models reviewed earlier to both posit mechanisms for existing substance abuse treatments and also propose variations of such treatments that might enhance their effects.

TREATMENT Existing substance abuse treatments tend to fall into a few general (and nonexhaustive) categories, in no particular order. First, there are drug replacement paradigms such as nicotine replacement and methadone maintenance, and mixed agonist/antagonist therapies (e.g., varenicline for smoking) that provide some agonist effects while blocking the reinforcing effects of drug use. Second, there are aversive conditioning treatments (e.g., rapid smoking, emetic aversion therapy) that are designed to link drug taking with aversive consequences (i.e., to punish drug use). As such, these are essentially counterconditioning paradigms. Third, there are drug antagonist treatments that are designed to extinguish drug seeking and use by blocking the reinforcing effects of drug use (e.g., naltrexone for alcohol or opioids, mecamylamine for nicotine). Fourth, some strategies attempt to shift behavioral contingencies so that nonuse delivers reinforcement that can compete effectively with the immediate reinforcement offered by drug use (e.g., contingency management programs). Fifth, cue exposure treatments are designed to extinguish drug motivational responses to cues that have been paired with drug use in the past. Sixth, and most commonly used, are treatment strategies that aim to enhance cognitive control over drug use

11-Schachtman-11.indd 250

through diverse means, such as problem solving or coping skill training, resolution of motivational ambivalence and value-behavior discrepancies, treatment of underlying emotion regulation difﬁculties, changing social contingencies (so that nonuse is supported more than use), and coaching individuals to reward themselves for abstinence and change. These counseling approaches are offered in diverse formats and modalities, but all of them encourage individuals to make choices that support change. Some treatments may fall into more than one of these categories, and many treatment programs will contain components from multiple categories. For example, varenicline functions as a mixed agonist and antagonist of nicotinic drug receptors and is often offered in conjunction with counseling for smoking cessation. We will look at each of these six categories of treatment approaches in terms of their possible effects on S-R, R-O, and S-O associations. Drug Replacement

Drug replacement is designed to attenuate withdrawal and to provide some of the appetitive reinforcement that maintains drug use, but in a manner that does not create addiction to a new self-administration regimen or create toxic or iatrogenic effects. Although nicotine replacement is designed to be used short term to attenuate withdrawal and craving during the worst of withdrawal, methadone is typically used as a long-term harm-reduction strategy well beyond the end of opiate withdrawal precipitated by falling drug levels. As such, this category is not homogenous. Even within the nicotine replacement therapies, the drugs have very different pharmacokinetic proﬁles and means and schedules of administration. Slow-releasing formulations, such as the nicotine patch, are ideal for reducing withdrawal, whereas the oral and nasal replacement therapies (the nicotine lozenge, gum, or spray) may work by serving as a coping response that substitutes for smoking, and that may provide some of its conditioned and unconditioned reinforcement. The unconditioned reinforcement delivered by these replacement products is greater when their pharmacokinetics

2/15/2011 7:45:35 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

resemble those of smoking. Nicotine-delivering medications roughly double the odds of quitting smoking for 6 months or more relative to placebo, and combinations of these therapies result in even greater efﬁcacy: High-dose monotherapies do not similarly boost success rates (Fiore et al., 2008). This suggests that these treatments may work by different mechanisms (e.g., attenuating withdrawal and providing a coping behavior). The means by which these medications achieve these effects are not fully known, although they have been shown to reduce withdrawal and craving (Ferguson, Shiffman, & Gwaltney, 2006). In this way, these drugs may promote abstinence, in part, by altering Ss that trigger addictive drug seeking and use (R). Indeed, high enough doses of the replacement medications may induce a state that is similar to that induced by the abused drug itself, rather than a state that resembles withdrawal. This may reduce motivation to use drugs to the extent that the outcomes delivered by the replacement match expectancies. If this occurs, then the replacement drug may become associated with its own conditioned stimuli signaling its availability and likely effects. This is akin to providing an attractive alternative reinforcer that can compete with the drug. These medications may affect drug use through additional routes. Using the abused drug while the replacement therapies are present may help to extinguish the drug-seeking response. Recent evidence in smokers suggests that smoking while wearing nicotine patches for 2 weeks before attempting to quit smoking increases later abstinence rates (Rose, Behm, Westman, & Kukovich, 2006; Shiffman & Ferguson, 2008). This effect appears to be even stronger if an individual smokes nicotine-free cigarettes while wearing the nicotine patch, perhaps because smoking these placebo cigarettes reduces the strength of R-O associations. Wearing the patch, in contrast, may blunt the (withdrawal) stimulus features that prod craving and drug use and using the replacement formulation maps an alternative R to the nicotine O. An incentive sensitization account of drug motivation, however, would suggest that replacement merely reduces the deprivation state that inﬂates

11-Schachtman-11.indd 251

251

the value of the incentive stimuli and does not depend on associative learning (Berridge, 2007; Robinson & Berridge, 1993). In addition, an individual who smokes while his/her nicotine receptors are already occupied by patch-delivered nicotine is not likely to experience reinforcement and may even experience punishment (although some conditioned reinforcement associated with the behavioral routine of smoking is likely and may take weeks to extinguish; see Baker, Japuntich, Hogle, McCarthy, & Curtin, 2006). In this way, saturating drug receptors may help some drug replacement strategies have the same effects as drug antagonists (discussed later). This should induce new conditioned inhibition of smoking while wearing a patch (i.e., the patch signals that smoking will not be reinforced). If this is the mechanism by which pre-cessation patch use works, however, the treatment is most likely to be effective while patch use is ongoing. When this cue is removed, prior expectancies that smoking will alleviate withdrawal distress, induce relaxation, and so on may resurface. This may help account for the decline in efﬁcacy after the end of treatment that is observed in most treatment trials (i.e., treatment is effective while ongoing but does not seem to induce permanent changes in S-R, S-O, R-O learning; Piasecki et al., 2002). Habit learning could account for the intransigence of drug use, but this account seems less consistent with the observation of rather immediate decreases in smoking that occur when nicotine replacement is used (Kenford et al., 1994). In addition, to the extent that drug replacements modulate dopaminergic tone (EppingJordan, Watkins, Koob, & Markou, 1998) they may enhance the anticipation or experience of pleasure to nondrug stimuli. Withdrawal may produce a pervasive anhedonia for nondrug activities and events (Epping-Jordan et al., 1998). The presence of replacement or agonist drugs in the body may increase anticipation and pleasure of alternative reinforcers and promote nondrug rewarding activities. Nicotine seems to serve as both a primary reinforcer and as an enhancer of the incentive and reinforcing effects of Ss paired with nicotine use (Caggiula et al., 2009). Nicotine replacement may likewise help to maintain or

2/15/2011 7:45:35 PM

252

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

strengthen the incentive and reinforcing effects of nonpharmacologic stimuli that are encountered during treatment. Methadone maintenance, which is typically longer in duration than nicotine replacement therapy (Mattick, Breen, Kimber, Davoli, & Breen, 2003), may work in a similar fashion. This treatment is not likely to promote robust extinction or counterconditioning of S-R and S-O associations because the cues and responses associated with methadone (e.g., going to a clinic every morning, drinking the methadone) are so different from the cues associated with heroin (e.g., meeting a dealer, cooking up, shooting up). Although some counterconditioning of subtle opioid withdrawal signs may occur, such that expectancies of withdrawal-relief are now induced by methadone-associated cues, the cues that were uniquely associated with heroin are likely to retain their motivational signiﬁcance. As such, this treatment, too, seems to induce temporary rather than permanent changes in the mapping of drug-related S, R, and O among addicted individuals. In summary, drug replacement strategies do not seem to induce new learning about S, R, and O that is likely to compete effectively with original learning beyond the end of treatment. For this reason, the long-term efﬁcacy of these treatments likely reﬂects the combined effects of the replacement therapies and of the other adjuvant treatment components (e.g., coping training, social support, environmental change, accountability) as well as nontreatment effects (e.g., experience of dramatically improved health upon cessation). Indeed, offering adjunct counseling to smokers using pharmacotherapies is considered the standard of care in smokingcessation treatment (Fiore et al., 2008). Counseling often encourages individuals to be aware of triggers for use (i.e., Ss), to avoid these triggers (i.e., change contexts) to reduce relapse risk (i.e., renewal), and to develop alternative coping behaviors (i.e., map new Rs to old Ss). There is clear evidence that such adjuvant strategies boost the effects of replacement pharmacotherapies (Fiore et al., 2008), and perhaps this occurs via altered S-O and R-O associations. Finally, researchers should explore further the

11-Schachtman-11.indd 252

use of adjuvant behavioral strategies (e.g., extensive smoking of nicotine-free cigarettes while using nicotine replacement) in order to determine whether the effectiveness of this strategy can be enhanced. Aversive Conditioning

Various forms of punishment and aversive conditioning have been tested as treatments for drug abuse. Early work in this area presented aversive stimuli during drug use (e.g., emetic therapies that pair drinking alcohol with nausea and malaise) or attempted to associate drinking with aversive stimuli later (e.g., showing alcoholics videotapes of themselves while intoxicated). These paradigms are similar to taste aversion training in animal models and trace conditioning, respectively. Evidence suggests that such strategies have some efﬁcacy (Cannon et al., 1986; Elkins, 1991; Howard & Jensen, 1990), but limited reach, because they are labor intensive and unpleasant (Wilson, 1991). Less aversive treatments based on similar principles have been developed, however. In tobacco treatment, several aversive smoking procedures have been tested. Rapidly inhaling cigarette smoke (e.g., one puff every 6 seconds) is highly aversive, even for heavy daily smokers, and can induce nausea, vomiting, dizziness, burning and dryness in the mouth and throat, and uncomfortable levels of arousal. Meta-analyses suggest that rapid smoking procedures have a modest but signiﬁcant positive effect on abstinence in smoking cessation (Fiore et al., 2008; Hajek & Stead, 2001). Some evidence suggests that the smoking need not even be rapid to achieve the desired effect (Hall, Rugg, Tunstall, Jones, 1984); it may be that simply smoking on a schedule makes smoking mildly aversive or at least less reinforcing. Indeed, Cinciripini and colleagues (Cinciripini et al., 1995; Cinciripini, Wetter, & McClure, 1997) have argued that scheduled smoking reduction may be an effective way for people to quit, in part because smoking on a schedule is nonreinforcing and may even be unpleasant. Milder versions of the rapid smoking procedure that have fewer risks have been developed, but these are less studied and available data suggest that they have little beneﬁt

2/15/2011 7:45:35 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

in terms of abstinence. In alcohol treatment, using disulﬁram may promote abstinence by establishing the expectancy that drinking will induce nausea and discomfort. In this way, disulﬁram may effect change through a topdown, cognitive route, even if no conditioning trials (pairing alcohol and aversive effects such as nausea) have taken place (Hughes & Cook, 1997). Expectancy change is another mechanism by which aversive techniques may improve treatment outcomes (Hogarth & Duka, 2006). In terms of learning theory, these aversive treatment approaches attempt to countercondition the Ss and Rs associated with smoking so that they are now paired with unpleasant sensory and emotional outcomes that will evoke avoidance rather than approach behaviors. As noted earlier, counterconditioning is prone to the same relapse phenomena as extinction. As such, these aversive conditioning treatment procedures are likely to induce learning that is context speciﬁc and prone to reacquisition (e.g., following a lapse), reinstatement (e.g., following exposure to second-hand smoke or conditioned dopamine releases to drug-related stimuli), renewal in changed contexts, spontaneous recovery after delays, and resurgence if alternative reinforcers become inaccessible. Many trials of aversive conditioning are likely necessary to effectively counter the years of learning that most addicted individuals bring to treatment. Such trials are inherently unpleasant. Given this, high and sustained levels of motivation are necessary for individuals to engage in aversive treatment to a degree that is likely to have clinically signiﬁcant beneﬁt. As such, motivational enhancement techniques (Miller & Rollnick, 2002) may be important adjuncts to aversive treatments. There are likely several sound reasons that aversive treatments have largely fallen by the wayside in recent decades (e.g., the risks involved, the need for close supervision, the lack of reach due to expense and associated personnel and supervision requirements, and the ﬂeeting nature of the effects). Despite these concerns, aversive techniques may still be useful to study in order to learn basic information about how counterconditioning works in clinical populations and to

11-Schachtman-11.indd 253

253

reﬁne treatments for the select population of addicted individuals who seek such treatment. Given that extinction is safer, it may make more sense to focus efforts in this area in the future, however. Some procedures, like the scheduled smoking reduction proposed by Cinciripini and colleagues (1995, 1997), may facilitate both extinction and counterconditioning by prohibiting smoking at times when the client is most motivated to smoke, and when it would be most reinforcing to do so, and by mandating clients to smoke when they do not want to do so. By taking control of smoking from the client and randomly assigning smoking times at ever-longer intervals, scheduled smoking may break associations between S and R and may link new, aversive Os with R (Cinciripini et al., 1995, 1997). This approach is safer than rapid smoking and is likely to be more acceptable to both providers and clients as well. In recent years evidence on the health risks posed by second-hand smoke have led to widespread restrictions on contexts in which smoking can occur (e.g., bans on workplace smoking). These restrictions are associated with increased cessation success (Longo, Johnson, Kruse, Brownson, & Hewett, 2001), as are personal restrictions reported by smokers (restrictions on smoking in the home; e.g., Bolt et al., 2009). Perhaps this link with successful cessation is due to the mechanisms cited earlier (e.g., breaking S-R bonds). Modiﬁed, hybrid approaches like scheduled smoking that combine extinction and counterconditioning might be a rich area to explore in future treatment development research. Drug Antagonists

Another means of inducing extinction is to block drug reinforcement delivery by administering antagonist drugs. Effective blockade of drug effects should facilitate the extinction of drug self-administration by inhibiting the R-O association learned during use (which should also occur with mixed agonists such as varenicline and buprenorphine). Mecamylamine is a blood pressure–lowering drug that is also a nicotine antagonist. It has been studied as a stand-alone treatment and in conjunction with nicotine replacement.

2/15/2011 7:45:35 PM

254

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

At high doses, mecamylamine has adverse effects that limit its clinical utility as a stand-alone treatment (Lancaster & Stead, 1998). Smaller doses offered in conjunction with nicotine seem to be somewhat better tolerated and may yield small beneﬁts in terms of abstinence rates (Lancaster & Stead, 1998). Bupropion is an FDA-approved, ﬁrst-line pharmacotherapy for smoking cessation (Fiore et al., 2008) that acts as a noncompetitive nicotinic acetylcholinergic receptor antagonist (Fryer & Lukas, 1999; Slemmer, Martin, & Damaj, 2000), in addition to reducing dopamine and norepinephrine reuptake (Ascher et al., 1995). In general, it doubles abstinence rates over those achieved with placebo treatment (Fiore et al., 2008). Its success may be due, in part, to its antagonism of the effects of nicotine self-administration (i.e., smoking). In this way, bupropion (most often prescribed in a sustained release formulation) may facilitate the extinction of R-O learning. Recent evidence, however, suggests that bupropion SR effects tend to last only as long as treatment is ongoing (McCarthy, Jorenby, Minami, & Yeh, 2009) and that treatment effects diminish during posttreatment follow-up. This may indicate that bupropion SR acts as a negative discriminative stimulus (S∆) of R-O outcome expectancies, and that, when bupropion SR is removed and the individual returns to the original learning context, self-administration resumes (see Chapter 10, this volume). Once again, the inhibition conferred by treatment appears to be more temporary and fragile than the original learning. The newest FDA-approved medication for tobacco cessation is varenicline, an α4ß2 nicotinic acetylcholine receptor partial agonist, which may work in two ways. First, the agonist effects of varenicline may function like a nicotine replacement therapy and help to attenuate withdrawal and craving (as with the ﬁrst category of treatments reviewed earlier); evidence from early clinical trials supports this. Second, the antagonist effects of this agent may also block reinforcement stemming from nicotine selfadministration. This treatment, offered in conjunction with standard smoking-cessation counseling, is currently the best available

11-Schachtman-11.indd 254

treatment for tobacco dependence (Fiore et al., 2008). Data suggest that smokers taking varenicline report not only decreased withdrawal symptoms due to its agonist effects but also decreases in the pleasure and reward associated with smoking (e.g., Gonzales et al., 2006; Jorenby et al., 2006), consistent with its antagonist effects. The latter should facilitate the extinction of R-O associations. Over time, when the O is blocked in the presence of Ss previously paired with R and O, and R diminishes in frequency, this should inhibit both S-O and S-R associations shown in Figure 11.1. Such inhibition may prove to be conditioned to the presence of varenicline, however, and thus may prove temporary, as with bupropion SR. Contingency Management

An alternative approach that has been used extensively in opioid addiction is to countercondition responses through contingency management. Contingency management often involves the provision of attractive, salient, nondrug reinforcers, such as cash, contingent on abstinence. This procedure is effective in promoting abstinence from opioids, cocaine, and nicotine (Prendergast, Podus, Finney, Greenwell, & Roll, 2006) and does not require that every nonuse response be immediately reinforced with an appetitive stimulus. Indeed, probabilistic outcomes such as earning lottery tickets seem to have similar beneﬁcial effects (Petry & Martin, 2002). Contingency management may work via multiple mechanisms. Presenting alternative reinforcers that are sufﬁciently attractive may not alter drug-relevant associations but may reﬂect learning about the value and time course of delivery of these alternatives. Having an attractive alternative that is readily available may be enough to reduce drug-taking behaviors associated with specious reward (Ainslie, 1975) and counter delay-discounting effects (Bickel, DeGrandpre, & Higgins, 1995). Repeated trials in which subjects refrain from drug use in the presence of drug cues in order to obtain an alternative reinforcer deemed more valuable may also alter the associative strength of drug Ss and both use and nonuse Rs. In a connectionist model, the connection between R and S may

2/15/2011 7:45:35 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

change as a function of increases in abstinence Rs paired with drug-related Ss, and decreases in drug use Rs paired with drug-related Ss. As we have proposed elsewhere (McCarthy, Curtin, et al., 2010), this type of training may result in greater levels of response conﬂict in the presence of drug-related Ss, and this conﬂict may trigger the recruitment of cognitive control to resolve the conﬂict. This will evoke awareness of the desire to use (craving) and the use of higher order cortical processes to decide how to proceed (i.e., to use or not to use becomes the question). Increasing the associative strength between alternative Rs and drug-related Ss thus has the potential to both alter the responses executed automatically (without conscious awareness of motivation) as a function of stimulus learning, and to alter the process of decision making (i.e., by turning off the auto-pilot and recruiting effortful control). To the extent that such procedures encourage the routine recruitment of cognitive control, they may sow the seeds of durable treatment effects without relying exclusively on extinction per se. Unfortunately, there is evidence that contingency management procedures are primarily effective while in place, like so many other treatments for addictive disorders. That is, once the experimentally arranged contingency is discontinued, individuals are likely to return to substance use (Pendergrast et al., 2006). Relatively little work has attempted to pair contingency management procedures with cognitive control training or other strategies designed to impart more durable effects. Thus, at present little is known about how to sustain the early effects of this intervention. Cue Exposure

Basic research on extinction and on addicted individuals’ reactivity to drug-related Ss (cues) in terms of craving, affect, and drug use suggests that cue exposure may be an effective addiction treatment. Cue exposure treatment consists of extinction trials in which addicted individuals encounter Ss (drug cues) without using the drug. The explicit goal of this treatment is to inhibit the S-R and S-O associations depicted in Figure 11.1. Consistent with the animal literature

11-Schachtman-11.indd 255

255

demonstrating the contextual speciﬁcity of extinction (Bouton, 2002), this treatment yields transient beneﬁts and does not appear to induce lasting behavior change (Conklin & Tiffany, 2002). Although craving and other measures of motivation (e.g., salivation; Collins & Brandon, 2002) diminish rapidly during the cue exposure extinction trials, renewal effects are observed with a change of context (Collins & Brandon, 2002). Several methodological factors may have contributed to the disappointing results for this theory-based treatment approach (Conklin & Tiffany, 2002). As discussed previously, the robustness of extinction can be inﬂuenced by context (broadly conceived and encompassing temporal, exteroceptive, and interoceptive Ss), the spacing of trials, the schedule of trials (e.g., interspersing S-only trials with occasional S-O or O-only trials appears to facilitate extinction; Rauhut, Thomas, & Ayres, 2001), and the presence of retrieval cues. The choice of S no doubt has an effect of extinction as well. Most studies of cue exposure conducted to date have focused on exteroceptive cues (e.g., pictures of drugs, drug paraphernalia) that are presented to the subject or evoked in mental imagery using imaginal inductions. Tailored Ss are often used to ensure that the cues are relevant to each individual subject (e.g., Conklin, 2006). Exposure to interoceptive cues is far less common, despite the fact that these cues may be the most potent instigators of drug motivation (Baker et al., 2004; McCarthy, Curtin, et al., 2010). Some scholars are beginning to focus on interoceptive cues, however, and are developing treatments that will alter associations between internal sensations and responses. Otto, Pollack, and colleagues have developed an interoceptive exposure paradigm (Otto, Powers, & Fischman, 2005) and applied it to treatment-resistant heroin abusers and found preliminary evidence supporting interoceptive cue exposure, versus intensive treatment in a control condition, for women, but not for men (Pollack et al., 2002). This approach may have interesting parallels with treatments that are being developed to improve distress tolerance in addicted individuals (Brown et al., 2008) because learning to tolerate distress

2/15/2011 7:45:35 PM

256

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

may, in a way, be a form of extinction. That is, tolerating distress may require withholding avoidance Rs to Ss associated with drug use. Cue exposure approaches may beneﬁt from systematic investigation of the parameters that optimize extinction learning in addicted humans (Conklin & Tiffany, 2002), and from careful evaluation of the relative importance of interoceptive and exteroceptive cues; some basic research suggests that interoceptive cues can overshadow exteroceptive cues in conditioning (Kim, Siegel, & Patenall, 1999; Razran, 1961). A ﬁnal note regarding extinction: While formal extinction procedures may not have shown signiﬁcant beneﬁt, this does not mean that extinction as a process is ineffective. For instance, it is possible that treatments such as nicotine replacement, which produce signiﬁcantly elevated levels of short-term abstinence, may produce some of their effects via extinction. That is, as individuals go about their daily lives without smoking, they are loosening the bonds between smoking and environmental Ss. If this is true, it suggests that extinction may be effective if it can be instituted so that it occurs pervasively in individuals’ daily lives. This is speculative, but the broad meaning of this observation is that our failed attempts to harness a mechanism of change may speak more to our poor implementation than to the potential relevance of the targeted process. Counseling

Myriad, diverse psychosocial treatment approaches for addiction have been developed. These treatments vary along countless dimensions (e.g., duration and dose schedule, modality, theoretical orientation, speciﬁc content). Our objective here is not to compare and contrast these treatments. Instead, our objective is to focus on the commonalities across treatments, speciﬁcally with regard to the learning that these treatments attempt to alter. We have foreshadowed this discussion in the preceding text. Whether the treatment involves extended residence in an intervention community or a brief, court-mandated motivational interviewing session, counseling treatment effects rely on cognitive control in addition to encouraging

11-Schachtman-11.indd 256

individuals to reduce S exposure and modify their Rs. One could argue that the most intensive treatments that involve the most contact also establish strong contingencies between abstinence and positive attention (e.g., social approval), and between drug use and negative attention (e.g., rejection) from those in the treatment milieu. Although counseling treatments sometimes establish new contingencies that may enhance response conﬂict and shape behavior, counseling strives primarily to facilitate effective, reliable cognitive control over drug-use behavior. Cognitive control is important in overriding automatic, overlearned response tendencies (i.e., habits). Cognitive control may help users overcome associative learning via numerous routes. For example, cognitive control helps individuals resolve response conﬂicts (e.g., ambivalence), and it may also help individuals protect against relapse by helping them to avoid triggers (Ss) that elicit drug motivation. In addition, cognitive control strategies may help individuals to focus on the beneﬁts of abstinence in order to overcome delay discounting tendencies that favor immediate relief and satisfaction over long-term health. Cognitive control may further support the development of new responses and the learning of new S-R, R-O, and S-O associations through deliberate behavioral practice. In many therapies, cognitive control is conferred by verbal means (e.g., through didactics and talk therapy), although behavioral practice and cognitive rehearsal are increasingly being incorporated into treatment programs in recognition of the importance of rehearsal in memory consolidation and performance (e.g., Abramowitz, 2005; Hoffman et al., 1995). With the possible exceptions noted, the desired effects of counseling on behavior change are mediated by conscious, effortful control. As such, these treatments have in common a reliance on limited cognitive control resources, and therefore, a proneness to failure (McCarthy, Curtin, et al., 2010). We have outlined boundary conditions in which cognitive control is most likely to be recruited (e.g., in situations of response conﬂict, distress, novel and salient stimuli) in a recent chapter (Curtin, McCarthy, Piper, & Baker, 2006) and have suggested that

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

cognitive control recruitment is a likely mediator of subjective craving. We have also asserted that cognitive control does not always support abstinence. An individual who becomes aware of the desire to use may, in fact, elect to use, despite the specious nature of the drug reward. (The reward is specious because the long-term costs of use are greater than the short-term beneﬁts.) Cognitive control may be shaped, however, by counseling (and other experiences, such as health crises or epiphanies; Miller & C’deBaca, 2001) so that individuals develop greater motivation and capacity to resist urges and to maintain abstinence. While strategies designed to enhance cognitive control might seem orthogonal to a conditioning-based approach to addiction treatment, in fact, the two perspectives might yield complementary effects. For example, counseling techniques can be used to promote cognitive control of S exposure likelihood (e.g., staying away from bars while trying to quit smoking). Standard drug counseling often involves helping clients to identify “triggers” for use. These triggers are Ss that have been paired with drug use Rs and positive drug Os so that the Ss now habitually evoke the R or expectancies of reward. By drawing attention to triggers, counselors may help clients to be forewarned about the motivational signiﬁcance of these cues and to behave in a way that reduces exposure to them or strengthens precommitment strategies (Rachlin & Green, 1972) that reduce the likelihood of responding habitually to them. Vogel-Sprott and Fillmore (see Chapter 10) provide compelling evidence that individuals can modulate their responses to drug Ss and Os based on task demands in the laboratory. This may extend beyond the laboratory as well, and it may suggest ways to help addicted individuals resist triggers to smoke. Avoidance of triggers has the potential to protect individuals from relapse in the short term, but prolonged avoidance is both difﬁcult to achieve and not especially productive from a learning theory perspective. Avoidance leaves the S-R and S-O associations intact and unchallenged by extinction trials. Many counseling interventions do not promote avoidance exclusively, however. Indeed, newer approaches that cultivate mindfulness

11-Schachtman-11.indd 257

257

(Davis, Fleming, Bonus, & Baker, 2007) foster distress tolerance (Brown et al., 2008) or promote acceptance and commitment (Gifford et al., 2004) encourage full engagement with the Ss that evoke craving and prompt drug use. In these treatments, individuals learn to observe the internal states and external Ss that evoke R motivation and O expectancies, and they learn to respond to these Ss with acceptance and abstinence rather than drug use. Coping and problem-solving training are additional, efﬁcacious treatment components (Fiore et al., 2008). Meta-analyses suggest that these components of smoking-cessation counseling are among the most efﬁcacious counseling strategies (Fiore et al., 2008). First, coping may enable extinction. Individuals may employ behavioral coping strategies (e.g., deep breathing) and/or cognitive strategies (e.g., focusing on the beneﬁts of abstaining, telling oneself that smoking is not an option) to resist smoking in situations in which smoking is perceived as possible. In this way, coping supports the uncoupling of S and R during recovery and thus facilitates extinction of habit learning. Second, coping and problem-solving training may help train up competing Rs to Ss so as to induce response conﬂict in high-risk situations and, therefore, increase the likelihood that cognitive control resources will be recruited. By creating viable competing response tendencies, behavioral skill training can decrease the likelihood that habitual responding will be evoked when drug Ss are encountered. Response conﬂict creates a window of opportunity for inhibitory control that would be shut in the absence of response conﬂict. This control is thought to be mediated by activity in the prefrontal cortices (Botvinick, Braver, Barch, Carter, & Cohen, 2001; Miller & Cohen, 2001). Without a competing response, drug use is likely to occur automatically with limited awareness in the presence of S, due to the strength of S-R learning acquired over years or even decades of use. The effectiveness of such prefrontal control is likely dependent on many additional factors (e.g., self-efﬁcacy, knowledge), but the control must be recruited before these important variables can exert their inﬂuence.

2/15/2011 7:45:36 PM

258

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Third, counseling may activate top-down mechanisms that alter drug use expectancies and responses. Psychoeducation about such things as the harmful consequences of drug use may be enough for some people to alter S-O expectancies (see Chapter 10). For example, when a patient learns that she has chronic obstructive pulmonary disease with permanent loss of lung function and risk of a degenerating, fatal course of illness if she continues to smoke, this information may be enough to shift her perception and experience of smoking so that old S-O and R-O expectancies are overwhelmed by new, cognitively mediated associations between smoking and catastrophic illness and death. In this way, an acute psychoeducational intervention may establish associations that overshadow even the overlearned associations at work in drug use. Something similar may occur in women who stop smoking during pregnancy but relapse postpartum. In this case, however, the healthrelated incentives are temporary and pregnancy serves as a discriminative stimulus (S∆) that signals that smoking will be followed by worry for the fetus, self-recrimination, or opprobrium, rather than relaxation and pleasure. Novel Treatments

We have now reviewed the major types of addiction treatments. We note that few of these treatments directly target the learning that motivates continued drug use and relapse, and that none addresses all of the key associative paths directly and simultaneously. To optimize treatment, it may be necessary to combine multiple treatment components targeting different aspects of learning, in addition to attempting to broaden the generalizability of the new learning through the use of appropriately spaced and scheduled extinction training and salient, portable retrieval cues (Bouton, 2002). S-R Learning

Enhanced cue exposure techniques may help to inhibit habit (S-R) learning. As noted earlier, the cue exposure strategies tested to date have tended to focus on exteroceptive cues and have not fully harnessed the knowledge about extinction that

11-Schachtman-11.indd 258

has been gained through systematic animal research (Bouton, 2002; Conklin & Tiffany, 2002). We have argued that interoceptive cues of negative affect are the central Ss in drug addiction (Baker et al., 2004). Therefore, effective extinction of S-R will likely involve induction of negative affect via drug deprivation and stress manipulations. This seems particularly important, because the learning obtained through extinction training will be of greatest use when the change attempt begins. This is also the time that withdrawal distress will be at its peak. Thus, it makes little sense to conduct cue exposure training in the context of euthymia and drug satiety, when the testing context (e.g., the quit day) will likely be marked by dysthmia and deprivation. Given the context-dependent nature of extinction training, this approach would seem to be a recipe for failure. Repeated, extended practice periods of withdrawal and abstinence prior to the formal cessation attempt may help prepare an individual for a change attempt in several ways, including by conducting extinction trials in the context in which the individual will be tested and habituating the individual to aversive withdrawal symptoms that are avoided through drug use. Such practice quitting could result in more robust extinction if it were paired with a salient retrieval cue (e.g., a token like that used in 12-step programs, a bracelet worn only on trial abstinence days) and if clients were instructed in mental rehearsal strategies (e.g., picturing the room in which extinction occurred) to improve retrievability of the extinction context when the quit day arrives. Bjork and Bjork’s (1992) theory of disuse also suggests that the practice quitting sessions should be difﬁcult, but achievable, because the storage strength of learning increases more when retrieval strength is low, and retrieval difﬁcult, than when retrieval is easy. This would seem to contradict the previous recommendation that retrieval cues be incorporated in extinction to prevent renewal. This discrepancy may reﬂect the distinction between learning and performance that Bjork and Bjork (1992) make. The retrieval cue may facilitate performance but may slow learning in the sense that storage strength for the new inhibitory learning will increase

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

more slowly if retrieval cues that enhance performance are provided. Thus, retrieval cues should not be so salient that they render the retrieval task easy. The proper balance between difﬁculty and retrieval cuing is not obvious and may need to be studied systematically in order to identify retrieval cue strategies that optimize both learning and performance. Treatment providers should be careful not to protect clients from extinction inadvertently by providing conditioned inhibitors that suppress responding during extinction but do not establish learning that extends to the absence of the conditioned inhibitor. Rescorla (2003) pointed out that a therapist, a therapy ofﬁce, or other cues associated with counseling can come to serve as conditioned inhibitors that signal to the individual that the previously learned associations do not apply in the presence of the inhibitor. When the inhibitor is removed, however, responding rebounds to prior levels. Thus, the presence of discrete cues during extinction training can reduce the effectiveness of extinction training (Rescorla, 2003). This is further justiﬁcation for the recommendation that extinction training occur in the client’s day-to-day environments. This can be accomplished via interventions that produce high rates of initial abstinence (e.g., contingency management, medications) in outpatient contexts and to ensure that the individual achieves extensive extinction training both before and after the quit day. Even though this would not abolish renewal effects, it may reduce the risk of protection from extinction. As noted earlier, creating response conﬂict by mapping drug-related Ss to incompatible, nondrug-use Rs may also facilitate change by increasing the likelihood that cognitive control will be recruited. This will thus create the opportunity for more extinction trials if the individual chooses to engage in the nonuse response rather than use. This form of counterconditioning is slightly different from extinction training because the response in extinction is often passive (although some individuals use active distraction or self-soothing in response to frustration; Mischel, Ebbesen, & Zeiss, 1972). In this strategy, a speciﬁc response is trained up in the

11-Schachtman-11.indd 259

259

presence of the Ss that usually evoke drug use. Because the extent of mapping increases response conﬂict, rehearsal of a particular strategy should help it to compete more effectively with use than would a scattershot of responses. The response could be cognitive or behavioral. Frustration and negative affect are likely to be evoked by prevention of the habitual response (drug use) because individuals appear to experience withdrawal when deprived of the conditioned effects of the self-administration ritual (Baker et al., 2006). This ritual comes to be reinforcing in its own right after sufﬁcient pairings with drug delivery, and prevention of self-administration (even by one’s own will) is likely to induce withdrawal-like symptoms and frustration (Baker et al., 2006). As such, the alternative response that may best compete with drug use would be another self-soothing or self-stimulatory behavior that might help to alleviate some of the distress induced by blocking drug use. R-O Learning

Drug antagonists that are not readily discriminable from placebo (Conklin & Tiffany, 2002) and placebo drugs (e.g., denicotinized cigarettes) may facilitate extinction of R-O associations. In this form of extinction, the individual executes the R but ﬁnds that the O is not as expected. The lack of delivery of the expected reinforcement induces frustration. As noted earlier, such frustration could feed back on motivation to use in a way that makes maintaining abstinence more challenging, given the strong association between negative affect (S) and use (R). Given this potential for an unintended effect, it is important to ensure that the withdrawal avoidance response is not reinforced. Drug antagonists rarely block all effects of self-administration. Even small decrements in distress caused by drug use may be enough to strengthen rather than inhibit negative reinforcement learning during extinction. As such, it is important to make sure that receptor availability is limited to the greatest extent possible by antagonist medications prior to extinction. Even placebo treatments (e.g., nicotine-free cigarettes) are not likely to protect against reinforcement of self-administration

2/15/2011 7:45:36 PM

260

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

early in extinction. The self-administration ritual itself is rewarding. Studies using denicotinized cigarettes indicate that smoking can reduce craving for at least 11 days (Donny, Houtsmuller, & Stitzer, 2006), despite the fact that these cigarettes deliver no or negligible amounts of nicotine. The Ss that have been paired with nicotine delivery in the million or so trials that precede extinction render the self-administration ritual robustly rewarding and difﬁcult to extinguish. These ﬁndings suggest that prolonged extinction training may be necessary to inhibit R-O associations in addicted individuals. Psychoeducation and counseling may be useful adjuncts in this type of treatment because maintaining the motivation to adhere to a frustrating and prolonged treatment such as this may be a challenge. Providing a compelling rationale and support throughout treatment may improve adherence and therefore also the effectiveness of such treatments. S-O Learning

Aversive counterconditioning and R-O extinction may be useful strategies in reducing the triggering of positive expectancies of drug use by Ss. Conducting rapid smoking in the presence of key triggers may invoke more negative expectancies in the presence of these triggers in the future, for example. Rapid smoking has typically been conducted in controlled laboratory contexts due to the risks associated with the procedure. Using milder, but still unpleasant, versions of the task or pairing smoking with other unpleasant outcomes (e.g., social rejection, a foul taste) may help to alter a smoker’s expectations of smoking. The effects of this type of counterconditioning are likely to be ﬂeeting, however, so the treatment parameters will have to be considered carefully and informed by the best available information about contextual conditioning and how to optimize generalization across contexts. Extinction of discriminative stimuli could also be accomplished during R-O extinction if the R-O extinction is conducted in the presence of key Ss. Interoceptive Ss of negative affect may be essential cues to include in R-O extinction trials because these cues may serve as discriminative

11-Schachtman-11.indd 260

stimuli (SDs) that indicate that smoking will be rewarded. Both incentive sensitization and negative reinforcement models of drug motivation assert that distress, particularly withdrawal distress, modulates the incentive salience of drugrelated stimuli. Distress may signal to a user that drugs would be especially helpful at that moment. If so, changing the value of the reinforcement (or incentive tone) offered in the presence of this SD should also reduce the discriminative value of the S on subsequent trials. Distress will no longer convey useful information about the probability of reward following such extinction training. Conducting the R-O extinction in the absence of such distress, however, will leave the SD intact and may leave an individual vulnerable to relapse in distress. Another strategy is to reduce the number of contexts in which drug use can occur. Due to recent smoking ban laws and policies, smoking is no longer permitted in movie theaters, airplanes, ofﬁces, and hospitals. As such, these environments serve as conditioned inhibitors that signal to a smoker that smoking is not permitted, even in the presence of excitatory Ss and positive expectancies of smoking that prompt motivation to smoke. Not only might these policies encourage successful cessation, but smokers are now often encouraged in smoking-cessation counseling to extend further environmental restrictions on their smoking prior to attempting to quit (e.g., to stop smoking in the car). The goal is to begin to extinguish smoking in response to Ss in multiple contexts prior to attempting to quit for good. This strategy may be augmented by pairing drug replacements or placebo drugs (e.g., nicotine-free cigarettes) with several contexts prior to a quit attempt as well. Pharmacotherapy

Recent research suggests that pharmacological agents modulate the strength of the inhibitory learning targeted in extinction training. For example, the NMDA-receptor agonist D-cycloserine promotes long-term-potentiation and may help to make extinction learning more robust (although this treatment does not eliminate relapse effects; Ledgerwood, Richardson,

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

& Cranney, 2005; Richardson, Ledgerwood, & Cranney, 2004). In addition, AM404 is a cannabanoid reuptake inhibitor that facilitates extinction in animals (Bitencourt, Pamplona, & Takahashi, 2008; Chhatwal, Davis, Maguschak, & Ressler, 2005). Another, more toxic agent, yohimbine, an α2 adrenergic antagonist, also facilitates extinction (Cain et al., 2004). A bettertolerated version of this medication may also help to promote the new, beneﬁcial learning that counters associations that sustain addictive behaviors. Although these medications are not suitable as stand-alone treatments and do not eliminate relapse phenomena following extinction, they may augment the learning-based strategies outlined earlier.

CONCLUSIONS Drug use is a learned behavior that reﬂects the inﬂuence of multiple forms of associative learning (Fig. 11.1). At this point, we lack information that will help us to identify which types or aspects of learning are most critical in sustaining drug use or most likely to respond to treatment. In the absence of such information, we must consider all possibilities and systematically investigate multiple learning pathways and mechanisms of addiction and change. This chapter endeavored to highlight diverse constructs in contemporary learning theory that may have relevance for addiction. We reviewed multiple forms of stimulus learning, including Pavlovian conditioning, habit learning, and incentive sensitization, in an effort to illustrate the mechanisms via which diverse stimuli might acquire motivational and behavioral signiﬁcance during addictive drug use. We discussed how modulators of stimulus learning (e.g., stimulus generalization, latent inhibition, and blocking effects) may be relevant to the development of treatment protocols (e.g., so as to extinguish drug use in response to a range of similar stimuli and to choose novel, salient stimuli to pair with abstinence). We then reviewed the instrumental/ operant conditioning literature to highlight the relevance of learning conditions and parameters on later behavioral performance. We also noted that drug use can serve as both an appetitive and

11-Schachtman-11.indd 261

261

avoidant behavior, which may have implications for treatment, given that avoidance learning seems to have unique properties (e.g., strengthened responding in the face of punishment) that differentiate it from approach behaviors. We reviewed current knowledge about extinction and counterconditioning paradigms, which are critical to therapeutic change, and highlighted the fragility of these types of learning and the equivocal results of attempts to optimize extinction in basic and clinical research. We discussed models of relapse documented in basic learning research, and we linked these to the clinical phenomenon of relapse. Finally, we brieﬂy analyzed existing treatments that target the stimulus and response learning that contributes to addiction, and we suggested directions for novel treatments and treatment combinations for future research. The aim of this chapter is to stimulate thinking, discussion, and research regarding ways to optimize addiction treatment based on analysis of basic, contemporary models of learning. Although not all of the learning phenomena reviewed here will inspire highly effective or readily disseminable treatments, the model of addiction learning presented here can provide a theoretical basis for treatment development, reﬁnement, and combination in the future. Our current treatments for drug abuse lack clarity in terms of mechanistic objectives. Future treatment research may beneﬁt from asking the following questions about treatments: What part of the learning that sustains drug motivation is this treatment designed to change? What does basic learning research say about the optimal way to change this type of learning? What other learning processes should we target in order to maximize the likelihood of beneﬁt? As answers accrue to these questions, a clearer picture regarding the critical processes activated by speciﬁc treatment components may emerge, and more focused questions to be addressed in subsequent research may be formulated. In this way, attention to treatment-activated learning processes may help us understand more about how change happens and how best to promote it. In conclusion, the interrelatedness of S, R, and O in drug learning (Fig. 11.1) argues for an integrated treatment approach that will address

2/15/2011 7:45:36 PM

262

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

these relations in a coordinated, simultaneous manner. A piecemeal approach is likely to have only modest effects that are fragile and generalize poorly (e.g., existing cue exposure treatments). An integrated approach is more likely to work, although more basic information about how to optimize extinction and counterconditioning is needed. Such optimization may be accomplished through parametric variation of the paradigms, psychosocial interventions to enhance cognitive control strategies, and/or direct pharmacological intervention (e.g., with D-cycloserine). Embracing the complexity of contemporary learning theory may help us both to understand drug addiction better and to develop new and better treatments that will directly target the learning that maintains addictive behavior. Learning theory continues to develop apace, largely in parallel with addiction treatment development. Bridging these two areas of research has the potential to raise important questions about the adequacy of animal models of learning that may lead to innovation in that ﬁeld while at the same time providing a rational basis for combining, modifying, and developing new clinical treatments.

ACKNOWLEDGMENTS Timothy B. Baker has served as a consultant, given lectures sponsored by, or has conducted research sponsored by GlaxoSmithKline, Nabi Biopharmaceuticals, Pﬁzer, and SanoﬁSynthelabo. Dr. Baker received support for this work from the University of Wisconsin Transdisciplinary Tobacco Use Research Center grant P50DA19706 from the National Institute on Drug Abuse and 1K05CA139871 from the National Cancer Institute. Correspondence regarding this chapter should be addressed to Danielle E. McCarthy, Department of Psychology, Rutgers, The State University of New Jersey, 152 Frelinghuysen Rd., Piscataway, NJ 08854; e-mail: [email protected]

REFERENCES Abramowitz, J. S. (2005). Obsessive-compulsive disorder. Cambridge, MA: Hogrefe.

11-Schachtman-11.indd 262

Ainslie, G. (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 82, 463–496. Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge, England: Cambridge University Press. Ascher, J. A., Cole, J. O., Colin, J. N., Feighner, J. P., Ferris, R. M., Fibiger, H. C., … Richelson, E. (1995). Bupropion: A review of its mechanisms of antidepressant activity. Journal of Clinical Psychiatry, 56, 395–401. Baker, T. B., Japuntich, S. J., Hogle, J. M., McCarthy, D. E., & Curtin, J. J. (2006). Pharmacologic and behavioral withdrawal from addictive drugs. Current Directions in Psychological Science, 15, 232–236. Baker, T. B., Piper, M. E., McCarthy, D. E., Majeskie, M. R., & Fiore, M. C. (2004). Addiction motivation reformulated: An affective processing model of negative reinforcement. Psychological Review, 111, 33–51. Balleine, B. W. (1992). Instrumental performance following a shift in primary motivation depends on incentive learning. Journal of Experimental Psychology: Animal Behavior Processes, 18, 236–250. Bayley, P. J., Frascino, J, C., & Squire, L. R. (2005). Robust habit learning in the absence of awareness and independent of medial temporal lobe. Nature, 436, 550–553. Berke, J. D. (2003). Learning and memory mechanisms involved in compulsive drug use and relapse. In J. Q. Wang (Ed.), Methods in molecular medicine (Vol. 79, pp. 75–101). Totowa, NJ: Humana Press. Berridge, K. (2007). The debate over dopamine’s role in reward: The case for incentive salience. Psychopharmacology, 191, 391–431. Bickel, W. K., DeGrandpre, R. J., & Higgins, S. T. (1995). The behavioral economics of concurrent drug reinforcers: A review and reanalysis of drug self-administration research. Psychopharmacology, 118, 250–259. Bickel, W. K., & Johnson, M. W. (2003). Delay discounting: A fundamental behavioral process of drug dependence. In G. Loewenstein, D. Read, & R. F. Baumeister (Eds.), Time and decision: Economic and psychological perspectives on intertemporal choice (pp. 419–440). New York, NY: Russell Sage Foundation. Bickel, W. K., Odum, A. L., & Madden, G. J. (1999). Impulsivity and cigarette smoking: Delay

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

discounting in current, never, and ex-smokers. Psychopharmacology, 146, 447–454. Bitencourt, R. M., Pamplona, F. A., & Takahashi, R. N. (2008). Facilitation of contextual fear memory extinction and anti-anxiogenic effects of AM404 and cannabidiol in conditioned rats. European Neuropsychopharmacology, 18, 849–859. Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus ﬂuctuation. In A. Healy, S. Kosslyn, & R. Shiffrin (Eds.), From learning processes to cognitive processes: Essays in honor of William K. Estes (Vol. 2, pp. 35–67). Hillsdale, NJ: Edrlbaum. Bolles, R. C. (1972). Reinforcement, expectancy, and learning. Psychological Review, 77, 32–48. Bolt, D. M., Piper, M. E., McCarthy, D. E., Japuntich, S. J., Fiore, M. C., Smith, S. S., & Baker, T. B. (2009). The Wisconsin Predicting Patients’ Relapse (WI-PREPARE) Questionnaire. Nicotine and Tobacco Research, 11, 481–492. Botvinick, M., Braver, T., Barch, D., Carter, C., & Cohen, J. (2001). Conﬂict monitoring and cognitive control. Psychological Review, 108, 624–652. Bouton, M. E. (1993). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80–99. Bouton, M. E. (2002). Context, ambiguity and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry, 52, 976–986. Bouton, M. E. (2007). Learning and behavior: A contemporary synthesis (1st ed.). Sutherland, MA: Sinauer. Bouton, M. E., Garcia-Gutiérrez, A., Zilski, J., & Moody, E. W. (2006). Extinction in multiple contexts does not necessarily make extinction less vulnerable to relapse. Behaviour Research and Therapy, 44, 983–994. Bouton, M. E., Woods, A. M., & Pineño, O. (2004). Occasional reinforced trials during extinction can slow the rate of rapid reacquisition. Learning and Motivation, 35, 371–390. Bower, G. H. (1961). A contrast effect in differential conditioning. Journal of Experimental Psychology, 62, 196–199. Brody, A. L., Mandelkern, M. A., London, E. D., Olmstead, R. E., Farahi, J., Scheibal, D., … Mukhin, A. G. (2006). Cigarette smoking saturates brain α4β2 nicotinic acetylcholine receptors. Archives of General Psychiatry, 63, 907–915.

11-Schachtman-11.indd 263

263

Brooks, D. C., & Bouton, M. E. (1994). A retrieval cue for extinction attenuates response recovery (renewal) caused by a return to the conditioning context. Journal of Experimental Psychology: Animal Behavior Processes, 20, 366–379. Brown, R. A., Palm, K. M., Strong, D. R., Lejuez, C. W., Kahler, C. W., Zvolensky, M. J., … Gifford, E. V. (2008). Distress tolerance treatment for early lapse smokers: Rationale, program description and preliminary ﬁndings. Behavior Modiﬁcation, 32, 302–332. Caggiula, A. R., Donny, E. C., Palmatier, M. I., Liu, X., Chaudhri, N., & Sved, A. F. (2009). In R. A. Bevins & A. R. Caggiula (Eds.), Nebraska Symposium on Motivation, 55. The motivational impact of nicotine and its role in tobacco use. (pp. 91–212). New York, NY: Springer Science. Cain, C. K., Blouin, A. M., & Barad, M. (2004). Adrenergic transmission facilitates extinction of conditional fear in mice. Learning and Memory, 11, 179–187. Cannon, D. S., Baker, T. B., Gino, A., & Nathan, P. E. (1986). Alcohol-aversion therapy: Relation between strength of aversion and abstinence. Journal of Consulting and Clinical Psychology, 54, 825–830. Capaldi, E. J. (1994). The sequential view: From rapidly fading stimulus traces to the organization of memory and the abstract concept of number. Psychonomic Bulletin and Review, 1, 156–181. Catania, A. C. (1998). Learning (4th ed.). Upper Saddle River, NJ: Prentice Hall. Chambless, D. L. (1990). Spacing of exposure sessions in treatment of agoraphobia and simple phobia. Behavior Therapy, 21, 217–229. Chhatwal, J. P., Davis, M., Maguschak, K. A., & Ressler, K. J. (2005). Enhancing cannabinoid neurotransmission augments extinction of conditioned fear. Neuropsychopharmacolgy, 30, 516–524. Cinciripini, P. M., Lapitsky, L., Seay, S., Wallﬁsch, A., Kitchens, K., & Van Vunakis, H. (1995). The effects of smoking schedules on cessation outcome: Can we improve on common methods of gradual and abrupt nicotine withdrawal? Journal of Consulting and Clinical Pscyhology, 63, 388–399. Cinciripini, P. M., Wetter, D. W., & McClure, J. B. (1997). Scheduled reduced smoking: Effects on smoking abstinence and potential mechanisms of action. Addictive Behaviors, 22, 759–767.

2/15/2011 7:45:36 PM

264

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Collins, B. N. & Brandon, T. H. (2002). Effects of extinction context and retrieval cues on alcohol cue reactivity among nonalcoholic drinkers. Journal of Consulting and Clinical Psychology, 70, 390–397. Conklin, C. A. (2006). Environments as cues to smoke: Implications for human extinctionbased. Research and Treatment Experimental and Clinical Psychopharmacology, 14, 12–19. Conklin, C. A., & Tiffany, S. T. (2002). Applying extinction research and theory to cue-exposure addiction treatments. Addiction, 97, 155–167. Craske, M. G., Kircanski, K., Zelikowsky, M., Mystkowski, J., Chowdhury, N., & Baker, A. (2008). Optimizing inhibitory learning during exposure therapy. Behaviour Research and Therapy, 46, 5–27. Curtin, J., McCarthy, D. E., Piper, M. E., & Baker, T. B. (2006). Implicit and explicit drug motivational processes: A model of boundary conditions. In R. W. Wiers & A. W. Stacy (Eds.), Handbook of implicit cognition and addiction (pp. 233–250). Thousand Oaks, CA: Sage. Daly, H. B., & Daly, J. T. (1991). Aversive learning: Review and extensions of DMOD. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 135–163). Hillsdale, NJ: Erlbaum. Davis, J. M., Fleming, M. F., Bonus, K. A., & Baker, T. B. (2007). A pilot study on mindfulness based stress reduction for smokers. BMC Complementary and Alternative Medicine, 7, 2. De Houwer, J., Crombez, G., & Baeyens, F. (2005). Avoidance behavior in research and theories of human contingency learning. Quarterly Journal of Experimental Psychology, 4B, 289–310. de Wit, H., & Stewart, J. (1981). Reinstatement of cocaine-reinforced responding in the rat. Psychopharmacology, 75, 134–143. Dean, S. J., & Pittman, C. M. (1991). Self-punitive behavior: A revised analysis. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 135–163). Hillsdale, NJ: Erlbaum. Denniston, J. C., Chang, R. C., & Miller, R. R. (2003). Massive extinction treatment attenuates the renewal effect. Learning and Motivation, 34, 68–86. Dickinson, A., & Balleine, B. W. (2002). The role of learning in motivation. In C. R. Gallistel (Ed.), Steven’s handbook of experimental psychology: Learning, motivation and emotion (Vol. 3, 3rd ed., pp. 497–533). New York, NY: John Wiley & Sons.

11-Schachtman-11.indd 264

Dickinson, A., Nicholas, D. J., & Adams, C. D. (1983). The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 35B, 35–51. Dickinson, A., Wood, N., & Smith, J. W. (2002) Alcohol seeking by rats: Action or habit. The Quarterly Journal of Experimental Psychology, 55B, 331–348. Domjan, M., & Best, M. R. (1980). Interference with ingestional aversion learning produced by preexposure to the unconditioned stimulus: Associative and nonassociative aspects. Learning and Motivation, 11, 522–537. Donny, E. C., Houtsmuller, E., & Stitzer, M. L. (2006). Smoking in the absence of nicotine: Behavioral, subjective and physiological effects over 11 days. Addiction, 102, 324–334. Driskell, J. E., Willis, R. P., & Copper, C. (1992). Effect of overlearning on retention. Journal of Applied Psychology, 77, 615–622. Elkins, R. L. (1991). An appraisal of chemical aversion (emetic therapy) approaches to alcoholism treatment. Behaviour Research and Therapy, 29, 387–413. Epping-Jordan, M. P., Watkins, S. S., Koob, G. F., & Markou, A. (1998) Dramatic decreases in brain reward function during nicotine withdrawal. Nature, 393, 76–79. Epstein, R., & Skinner, B. F. (1980). Resurgence of responding during the cessation of responseindependent reinforcement. Proceedings of the National Academy of Sciences USA, 77, 6251–6253. Everitt, B. J., Parkinson, J. A., Olmstead, M. C., Arroyo, M., Robledo, P., & Robbins, T. W. (1999). Associative processes in addiction and reward: The role of amygdala- ventral striatal subsystems. Annals of the New York Academy of Sciences, 877, 412–438. Ferguson, S. G., Shiffman, S., & Gwaltney, C. J. (2006). Does reducing withdrawal severity mediate nicotine patch efﬁcacy? A randomized clinical trial. Journal of Consulting and Clinical Psychology, 74, 1153–1161. Fiore, M. C., Jaén, C. R., Baker, T. B., Bailey, W. C., Benowitz, N., Curry, S. J.… & Wewers, M. E. (2008). Treating tobacco use and dependence: 2008 update. Clinical practice guideline. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service. Flaherty, C. F. (1996). Incentive relativity. (pp. 19–53) New York, NY: Cambridge University Press.

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

Foa, E. B., Jameson, J. S., Turner, R. M., & Payne, L. L. (1980). Massed vs. spaced exposure sessions in the treatment of agoraphobia. Behaviour Research and Therapy, 18, 333–338. Fryer, J. D., & Lukas, R. J. (1999). Noncompetitive functional inhibition at diverse, human nicotinic acetylcholine receptor subtypes by bupropion, phencyclidine, and ibogaine. Journal of Pharmacology and Experimental Therapeutics, 288, 88–92. Fuji, S., Jia, Y., Yang, A., & Sumikawa, K. (2000). Nicotine reverses GABAergic inhibition of longterm potentiation induction in the hippocampal CA1 region. Brain Research, 863, 259–265. Gifford, E. V., Kohlenberg, B. S., Hayes, S. C., Antonuccio, D. O., Piasecki, M. M., RasmussenHall, M. L., & Palm, K. M. (2004). Acceptancebased treatment for smoking cessation. Behavior Therapy, 35, 689–705. Gloria, R., Angelos, L., Schaefer, H. S., Davis, J. M., Majeskie, M., Richmond, B. S., … Baker, T. B. (2009). An fMRI investigation of the impact of withdrawal on regional brain activity during nicotine anticipation. Psychophysiology, 46, 681–693. Gonzales, D., Rennard, S. I., Nides, M., Oncken, C., Azoulay, S., Billing, C. B., … Varenicline Phase 3 Study Group. (2006). Varenicline, an alpha4beta2 nicotinic acetylcholine receptor partial agonist, vs sustained-release bupropion and placebo for smoking cessation: A randomized controlled trial. Journal of the American Medical Association, 296, 47–55. Gunther, L. M., Denniston, J. C., & Miller, R. R. (1998). Conducting exposure treatment in multiple contexts can prevent relapse. Behaviour Research and Therapy, 36, 75–91. Hajek, P., & Stead, L. F. (2001). Aversive smoking for smoking cessation. Cochrane Database of Systematic Reviews, Issue 3. Art. No. CD000546. DOI: 10.1002/14651858.CD000546.pub2. Hall, S. M., Humﬂeet, G. L., Muñoz, R. F., Reus, V. I., Robbins, J. A., & Prochaska, J. J. (2009). Extended treatment of older cigarette smokers. Addiction, 104, 1043–1052. Hall, S. M., Rugg, D., Tunstall, C., & Jones, R. T. (1984). Preventing relapse to cigarette smoking by behavioral skill training. Journal of Consulting and Clinical Psychology, 52, 372–382. Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243–266. Hoffman, J. A., Caudill, B. D., Landry, M., & Associates, Inc. (1995). Living in balance: A comprehensive

11-Schachtman-11.indd 265

265

substance abuse treatment and relapse prevention manual. Washington, DC: Koba Associates, Inc. Hogarth, L., & Duka, T. (2006). Human nicotine conditioning requires explicit contingency knowledge: Is addictive behavior cognitively mediated? Psychopharmacology, 184, 553–566. Holland, P. C. (2004). Relations between Pavlovianinstrumental transfer and reinforcer devaluation. Journal of Experimental Psychology: Animal Behavior Processes, 30, 104. Howard, M. O. (2001). Pharmacological aversion treatment of alcohol dependence. I. Production and prediction of conditioned alcohol aversion. American Journal of Drug and Alcohol Abuse, 27(3), 561–585. Howard, M. O., & Jenson, J. M. (1990). Chemical aversion treatment of alcohol dependence. I. Validity of current criticisms. Substance Use and Misuse, 25, 1227–1262. Hughes, J. C., & Cook, C. H. C. (1997). The efﬁcacy of disulﬁram: A review of outcome studies. Addiction, 92, 381–395. Hughes, J. R. (2002). Is extinction in animals the same as abstinence in humans? Addiction, 97, 1219. Hutcheson, D. M., Everitt, B. J., Robbins, T. W., & Dickinson, A. (2001). The role of withdrawal in heroin addiction: Enhances reward or promotes avoidance? Nature Neuroscience, 4, 943–947. Hyman, S. E. (2005) Addiction: A disease of learning and memory. American Journal of Psychiatry, 162, 1414–1422. Jorenby, D. E., Hays, J. T., Rigotti, N. A., Azoulay, S., Watsky, E.J., Williams, K. E., … Phase 3 Study Group. (2006). Efﬁcacy of varenicline, an alpha 4beta 2 nicotinic acetylcholine receptor partial agonist, vs placebo or sustainedrelease bupropion for smoking cessation: A randomized controlled trial. Journal of the American Medical Association, 296, 56–63. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment an aversive behavior (pp. 279–296). New York, NY: Appleton-Century-Crofts. Kenford, S. L., Fiore, M. C., Jorenby, D. E., Smith, S. S., Wetter, D., & Baker, T. B. (1994). Predicting smoking cessation: Who will quit with and without the nicotine patch. Jounral of the American Medical Association, 271, 589–594. Kim, J. A., Siegel, S., & Patenall, V. R. A. (1999). Drug-onset cues as signals: Intra-administration

2/15/2011 7:45:36 PM

266

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

associations and tolerance. Journal of Experimental Psychology: Animal Behavior Processes, 25, 491–504. Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996). A neostriatal habit learning system in humans. Science, 273, 1399–1402. Koob, G. F., & Le Moal, M. (1997). Drug abuse: Hedonic homeostatic dysregulation. Science, 278, 52–58. Koob, G. F., & Le Moal, M. (2008). Addiction and the brain antireward system. Annual Review of Psychology, 59, 29–53. Lancaster, D. S. T., & Stead, L. F. (1998). Mecamylamine (a nicotine antagonist) for smoking cessation. Cochrane Database of Systematic Reviews, Issue 2. Art. No. CD001009. DOI: 10.1002/14651858.CD001009. Lang, A. J., Craske, M. G., & Bjork, R. A. (1999). Implications of a new theory of disuse for the treatment of emotional disorders. Clinical Psychology: Science and Practice, 6, 80–94. Ledgerwood, L., Richardson, R., & Cranney, J. (2005). D-cycloserine facilitates extinction of learned fear: Effects on reacquisition and generalized extinction. Biological Psychiatry, 57, 841–847. Lieving, G. A., & Lattal, K. A. (2003). Recency, repeatability, and reinforce retrenchment: An experimental analysis of resurgence. Journal of the Experimental Analysis of Behavior, 80, 217–233. Longo, D. R., Johnson, J. C., Kruse, R. L., Brownson, R. C., & Hewett, J. E. (2001). A prospective investigation of the impact of smoking bans on tobacco cessation and relapse. Tobacco Control, 10, 267–272. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition; The effect of nonreinforced preexposure to the conditioned stimulus. Journal of Comparative and Physiological Psychology, 52, 415–419. Mansvelder, H. D., & McGehee, D. S. (2000). Longterm potentiation of excitatory inputs to brain reward areas by nicotine. Neuron, 27, 349–357. Marlin, N. A. (1981). Contextual associations in trace conditioning. Animal Learning and Behavior, 9, 519–523. Marsh, R., Alexander, G. M., Packard, M. G., Zhu, H., Wingard, J. C., Quackenbush, G., &

11-Schachtman-11.indd 266

Peterson, B. S. (2004). Habit learning in Tourette syndrome a translational neuroscience approach to a developmental psychopathology. Archives of General Psychiatry, 61, 1259–1268. Matsuyama, S., Matsumoto, A., Taira, E., & Tomoyuki, N. (2000). Activation of nicotinic acetylcholine receptors induces long-term potentiation in vivo in the intact mouse dentate gyrus. European Journal of Neuroscience, 12, 3741–3747. Mattick, R. P., Breen, C., Kimber, J., Davoli, M., & Breen, R. (2003). Methadone maintenance therapy versus no opioid replacement therapy for opioid dependence. Cochrane Database of Systematic Reviews. Issue 2. Art. No. CD002209. DOI: 10.1002/14651858.CD002209. McAllister, W. R., & McAllister, D. E. (1991). Fear theory and aversively motivated behavior: Some controversial issues. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 135–163). Hillsdale, NJ: Erlbaum. McCarthy, D. E., Curtin, J. J., Piper, M. E., & Baker, T. B. (2010). Negative reinforcement: Possible clinical implications of an integrative model. In J. Kassel (Ed.), Substance abuse and emotion. (pp. 15–42), Washington, DC: American Psychological Association. McCarthy, D. E., Gloria, R., & Curtin, J. J. (2009). Attention bias in nicotine withdrawal and under stress. Psychology of Addictive Behaviors, 23, 77–90. McCarthy, D. E., Jorenby, D. E., Minami, H. M., & Yeh, V. M. (2009). Treatment options in smoking cessation: What place for bupropion sustained-release? Clinical Medicine: Therapeutics, 1, 683-696. McCarthy, D. E., Piasecki, T. M., Lawrence, D. L., Jorenby, D. E., Shiffman, S., Fiore, M. C., & Baker, T. B. (2008). A randomized, controlled clinical trial of bupropion SR and individual smoking cessation counseling. Nicotine and Tobacco Research, 10, 717–729. Mead, A. N., Crombag, H. S., & Rocha, B. A. (2004). Sensitization of psychomotor stimulation and conditioned reward in mice: Differential modulation by contextual learning. Neuropsychopharmacology, 29, 249–258. Mehringer, A. M., Pomerleau, C. S., Snedecor, S. M., & Finkenauer, R. (2008). Favorite cigarette of the day in a random sample of women smokers. Addictive Behavior, 33, 848–852. Mensink, G-J. M., & Raaijmakers, J. G. (1988). A model for interference and forgetting. Psychological Review, 95, 434–455.

2/15/2011 7:45:36 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

Miller, E., & Cohen, J. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. Miller, R. M., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Sources and consequences. American Journal of Psychology, 99, 145–218. Miller, W. R., & C’deBaca, J. (2001). Quantum change: When epiphanies and sudden insights transform ordinary lives. New York, NY: Guilford Press. Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people to change (2nd ed.). New York, NY: Guilford Press. Mischel, W., Ebbesen, E. B., & Zeiss, A. R. (1972). Cognitive and attentional mechanisms in delay of gratiﬁcation. Journal of Personality and Social Psychology, 21, 204–218. Mokdad, A. H., Marks, J. S., Stroup, D. F., & Gerberding, J. L. (2004). Actual causes of death in the United States, 2000. Journal of the American Medical Association, 291, 1238–1245. Mowrer, O. H. (1947). On the dual nature of learning: A re-interpretation of “conditioning” and “problem-solving.” Harvard Educational Review, 17, 102–148. Mystkowski, J. L., Craske, M. G., Echiverri, A. M., & Labus, J. S. (2006). Mental reinstatement of context and return of fear in spider-fearful participants. Behavior Therapy, 37, 49–60. National Center on Addiction and Substance Abuse (CASA) at Columbia University. (2009). Shoveling up II: The impact of substance abuse on federal, state and local budgets. New York, NY: Author. O’Brien, C. P. (1976). Experimental analysis of conditioning factors in human narcotic addiction. Pharmacological Reviews, 27, 533–534. Olmstead, M. C., Lafond, M. V., Everitt, B. J., & Dickinson, A. (2001). Cocaine-seeking by rats is a goal directed action. Behavioral Neuroscience, 115, 394–402. Otto, M. W., Powers, M. B., & Fischman, D. (2005). Emotional exposure in the treatment of substance use disorders: Conceptual model, evidence, and future directions. Clinical Psychology Review, 25, 824–839. Pendergrast, M., Podus, D., Finney, J., Greenwell, L., & Roll, J. (2006). Contingency management for treatment of substance use disorders: A metaanalysis. Addiction, 101, 1546–1560. Petry, N. M. & Martin, B. (2002). Low-cost contingency management for treating cocaine- and

11-Schachtman-11.indd 267

267

opioid-abusing methadone patients. Journal of Consulting and Clinical Psychology, 70, 398–405. Piasecki, T. M., Fiore, M. C., McCarthy, D. E., & Baker, T. B. (2002). Have we lost our way? The need for dynamic formulations of smoking relapse proneness. Addiction, 97, 1093–1108. Podlesnik, C. A., Jimenez-Gomez, C., & Shahan, T. A. (2006). Resurgence of alcohol seeking produced by discontinuing non-drug reinforcement as an animal model of drug relapse. Behavioural Pharmacology, 17, 369–374. Pollack, M. H., Penaca, S. A., Bolton, E., Worthington III, J. J., Allen, G. L., Farach, F. J., Jr., & Otto, M. W. (2002). A novel cognitivebehavioral approach for treatment-resistant drug dependence. Journal of Substance Abuse Treatment, 23, 335–342. Prendergast, M., Podus, D., Finney, J., Greenwell, L., & Roll, J. (2006). Contingency management for treatment of substance use disorders: A metaanalysis. Addiction, 101, 1546–1560. Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of Experimental Analysis and Behavior, 17, 15–22. Rauhut, A. S., Thomas, B. L., & Ayres, J. J. B. (2001). Treatments that weaken Pavlovian conditioned fear and thwart its renewal in rats: Implications for treating human phobias. Journal of Experimental Psychology: Animal Behavior Processes, 27, 99–114. Razran, G. (1961). The observable unconscious and the inferable unconscious in current Soviet psychophysiology: Interoceptive conditioning, somatic conditioning, and the orienting reﬂex. Psychological Review, 68, 81–147. Rescorla, R. A. (1999). Partial reinforcement reduces the associative change produced by nonreinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 25, 403–414. Rescorla, R. A. (2003). Protection from extinction. Learning and Behavior, 31, 124–132. Riccio, D. C., Rabinowitz, V. S., & Axelrod, S. (1994). Memory: When less is more. American Psychologist, 49, 917–923. Richardson, R., Ledgerwood, L., & Cranney, J. (2004). Facilitation of fear extinction by D-cycloserine: Theoretical and clinical implications. Learning and Memory, 11, 510–516. Robinson T. E., & Berridge K. C. (1993). The neural basis of drug craving: An incentive sensitization theory of addiction. Brain Research Reviews, 18, 247–291.

2/15/2011 7:45:36 PM

268

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Robinson, T. E., & Berridge, K. C. (2003). Addiction. Annual Review of Psychology, 54, 25–53. Rose, J. E., Behm, F. M., Westman, E. C., & Kukovich, P. (2006). Precessation treatment with nicotine skin patch facilitates smoking cessation. Nicotine and Tobacco Research, 8, 89–101. Shaham, Y., Adamson, L. K., Grocki, S., & Corrigall, W. A. (1997). Reinstatement and spontaneous recovery of nicotine seeking in rats. Psychopharmacology, 130, 396–403. Shaham, Y., Shalev, U., Lu, L., de Wit, H., & Stewart, J. (2003). The reinstatement model of drug relapse: History, methodology and major ﬁndings. Psychopharmacology, 168, 3–20. Shiffman, S., & Ferguson, S. G. (2008). Nicotine patch therapy prior to quitting smoking: A meta-analysis. Addiction, 1103, 557–563. Sidman, M. (1953). Avoidance conditioning with brief shock and no exteroceptive warning signal. Science, 118, 157–158. Siegel, S. (1979). The role of conditioning in drug tolerance and addiction. In J. D. Keehn (Ed.), Psychopathology in animals: Research and treatment implications (pp. 143–168). New York, NY: Academic Press. Siegel, S., Baptista, M. A. S., Kim, J. A., McDonald, R. V., & Weise-Kelly, L. (2000). Pavlovian psychopharmacology: The associative basis of tolerance. Experimental and Clinical Psychopharmacology, 8, 276–293. Skinner, B. F. (1969). Contingencies of reinforcement: A theoretical analysis. New York, NY: Appleton-Century-Crofts. Slemmer, J. E., Martin, B. R., & Damaj, M. I. (2000). Bupropion is a nicotinic antagonist. Journal of Pharmacology and Experimental Therapeutics, 295, 321–327. Solomon, R. L. (1977). An opponent-process theory of acquired motivation: The affective dynamics of addiction. In J. D. Maser & M. E. P. Seligman (Eds.), Psychopathology: Experimental models (pp. 66–103). San Francisco, CA: W. H. Freeman. Stasiewicz, P. R., Brandon, T. H., & Bradizza, C. M. (2007). Effects of extinction context and retrieval cues on renewal of alcohol-cue reactivity among alcohol-dependent outpatients. Psychology of Addictive Behaviors, 2, 244–248. Stewart, J., de Wit, H., & Eikelboom, R. (1984). Role of unconditioned and conditioned drug effects in the self-administration of opiates and stimulants. Psychological Review, 91, 251–268.

11-Schachtman-11.indd 268

Talmi, D., Seymour, B., Dayan, P., & Dolan, R. J. (2008). The human Pavlovian-instrumental transfer. Journal of Neuroscience, 28, 360–368. Thorndike, E. L. (1911) Animal intelligence: Experimental studies. New York, NY: Macmillan. Tiffany, S. T. (1990). A cognitive model of drug urges and drug-use behavior: Role of automatic and nonautomatic processes. Psychological Review, 97, 147–168. Tindell, A. J., Berridge, K. C., Zhang, J., Pecina, S., & Adridge, J. W. (2005). Ventral pallidal neurons code incentive motivation: Ampliﬁcation by mesolimbic sensitization and amphetamine. European Journal of Neuroscience, 22, 2617–2634. Valette, H., Bottlaender, M., Dolle, F., Coulon, C., Ottaviani, M., & Syrota, A. (2003). Long-lasting occupancy of central nicotinic acetylcholine receptors after smoking: A PET study in monkeys. Journal of Neurochemistry, 84, 105–111. Veatch, L. M., & Becker, H. C. (2002). Electrographic and behavioral indices of ethanol withdrawal sensitization. Brain Research, 946, 272–282. Volkow, N. D., Fowler, J. S., & Wang, G. J. (2003). The addicted human brain: Insights from imaging studies. Journal of Clinical Investigations, 111, 1444–1451. Volkow, N. D., Fowler, J. S., Wang, G. J., Baler, R., & Telang, F. (2009). Imaging dopamine’s role in drug abuse and addiction. Neuropharmacology, 56, S3–S8. Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. Spear & R. R. Miller (Eds.), Information processing in animals: Memory mechanisms (pp. 5–47). Hillsdale, NJ: Lawrence Erlbaum. Weiss, S. J., & Weissman, R. D. (1992). Generalization peak shift for autoshaped and operant key pecks. Journal of the Experimental Analysis of Behavior, 57, 127–143. Wikler, A. (1948). Recent progress in research on the neurophysiological basis of morphine addiction. American Journal of Psychiatry, 105, 329–338. Williams, B. A. (1988). Reinforcement, choice, and response strength. In R. C. Atkinson & R. J. Herrnstein (Eds.), Stevens’ handbook of experimental psychology, Vol. I. Perception and motivation; Vol. 2. Learning and cognition (2nd ed., pp. 167–244). New York, NY: Wiley. Wilson, G. T. (1991). Chemical aversion conditioning in the treatment of alcoholism: Further comments. Behaviour Research and Therapy, 29, 415–419.

2/15/2011 7:45:37 PM

LEARNING THEORY IN THE TREATMENT OF DRUG ABUSE

Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2005). Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. European Journal of Neuroscience, 22, 505–512.

11-Schachtman-11.indd 269

269

Zinser, M. C., Fiore, M. C., Davidson, R. J., & Baker, T. B. (1999). Manipulating smoking motivation: Impact on an electrophysiological index of approach motivation. Journal of Abnormal Psychology, 108, 240–254.

2/15/2011 7:45:37 PM

CHAPTER 12 Internal Stimuli Generated by Abused Substances Role of Pavlovian Conditioning and Its Implications for Drug Addiction Rick A. Bevins and Jennifer E. Murray

Consideration of the importance of Pavlovian conditioning involving interoceptive stimuli to healthrelated issues dates back to Pavlov. Despite this long history and its likely importance, the preponderance of empirical and theoretical effort in the drug abuse ﬁeld has been on exteroceptive conditioning with the drug conceptualized as the unconditioned stimulus. This chapter reviews what research has been done on Pavlovian conditioning involving the interoceptive effects of abused drugs as stimuli (i.e., conditioned stimuli or occasion setters). That research indicates that conditioning not only alters behavior evoked or modulated by drug stimuli, but that it alters the drug state in a manner that likely contributes to addiction. For instance, nicotine and diazepam acquire conditioned reinforcing value by virtue of being repeatedly paired with an appetitive event. Throughout the chapter we highlight translational links between preclinical research on interoceptive drug stimuli and drug addiction, as well as identify gaps in the scientiﬁc literature.

INTRODUCTION Upon quick reﬂection, every reader of this chapter will hopefully recognize and admit familiarity with internal or interoceptive stimuli. These may be the warmth of the throat and stomach after drinking hot chocolate on a wintery day, the mental fog and drowsiness from cold medication, the twinge of pain from the sprained ankle, or the twitchy jittery feeling of drinking too much caffeinated coffee. Like external stimuli such as the smell of the hot chocolate or the sight of the cold medication packet, these internal stimuli are available to serve as stimuli and guide or control learned behavior. Given the focus of the present chapter on Pavlovian (classical) conditioning, it is notable that Pavlov (1927) recognized this function of internal stimuli and attributed some importance to them in human

disorders (see later). In wrapping up his discussion on conditioned stimuli and trace reﬂexes, Pavlov (1927) put forward the following deﬁnition: To conclude this part of our discussion I shall suggest the following modiﬁcation and ampliﬁcation of our deﬁnition of agencies which can become conditioned, viz. that innumerable individual ﬂuctuations in the external and internal environment of the organism may, each and all of them, singly or collectively, being reﬂected in deﬁnite changes in the cells of the cerebral cortex, acquire the properties of a conditioned stimulus. (p. 43, italics added for emphasis)

The early interoceptive conditioning research used experimenter-generated internal stimuli that ranged from muscle/joint stimulation, irrigation of the stomach, electric stimulation of a speciﬁc brain region, and injected drug (e.g., Bykov, 1957; Cook, Davidson, Davis, & Kelleher,

270

12-Schachtman-12.indd 270

2/16/2011 5:50:29 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

1960; Doty, 1961; Pavlov, 1927). Of particular interest to the present chapter is the research on the ability of an injected ligand—especially a drug of abuse—to serve as an interoceptive stimulus in a Pavlovian conditioning situation. There is growing literature indicating that drugs of abuse such as methamphetamine, nicotine, and ethanol appear to serve as conditioned stimuli and/or Pavlovian features (terms such as facilitator, occasion setters, and modulator have also been used in place of feature). In the narrative that follows, we will ﬁrst provide working deﬁnitions of key terms used throughout this chapter. We will then describe different approaches that have been adopted by investigators to study interoceptive Pavlovian conditioning using drugs of abuse. Following this discussion, we will focus our attention on research suggesting that Pavlovian conditioning not only modiﬁes behavior evoked by the interoceptive stimulus effects of a drug, but that such conditioning alters the drug state in a way that might be important for addiction. Further, we will highlight translational links between preclinical research on interoceptive drug stimuli and drug use and addiction in humans. In doing so, we hope to identify for the reader some gaps in the scientiﬁc literature requiring empirical attention, as well as increase awareness and enthusiasm for the importance of understanding basic conditioning processes involved in major public health problems such as drug addiction.

CONCEPTS AND METHODS Focusing on the procedural aspects of the conditioning situation in the laboratory for the moment, and ignoring potential theoretical assumptions or biases, a drug state may be paired reliably with another drug state, including a higher dose of the same drug, or an exteroceptive event such as food or footshock. In this situation, the initial drug state is considered the conditioned stimulus (CS) and the subsequent event is thought of as the unconditioned stimulus (US). Conditioning is evidenced by the drug CS acquiring the ability to evoke a new response or modify an ongoing response. As a Pavlovian feature (see Fig. 12.1), the drug state signals

12-Schachtman-12.indd 271

271

when a second stimulus (i.e., the CS) such as illumination of a light or a novel taste will be reinforced (feature positive; FP) or not reinforced (feature negative, FN). In the absence of the drug state, the opposite relation exits. For example, in the FP situation there are two session types—drug and nondrug (e.g., saline). On drug sessions, the CS is presented for some time before the US occurs. On saline sessions, the CS also occurs, but the US is withheld. The CS comes to evoke a conditioned response (CR) only on drug sessions. For FN training, the CS is followed by the US only on saline sessions, and CS-evoked conditioned responding is isolated to nondrug sessions. Thus, in either training situation the drug state disambiguates whether the CS-US relation is in force. Drug States as Pavlovian Stimuli: A Working Model

At any one moment, it is a challenge for learning researchers to know exactly the nature of the stimulus used in their tasks. A stimulus as seemingly simple as a pure tone can vary with

Feature Positive drug

*

*

*

*

*

*

*

*

*

*

*

*

*

salin e

Feature Negative drug salin e

*

denotes brief exteroceptive stimulus such a 15-sec illumination of a light * denotes delivery of the US such as 4-sec access to a sucrose solution

Figure 12.1 This graphic provides an exemplar from our laboratory on the method for training the interoceptive stimulus effects of a drug as a feature-positive (FP) or a feature-negative (FN) occasion setter. In brief, the drug stimulus disambiguates when a discrete cue such a brief light will be followed (FP) or not followed (FN) by the reinforcer (see text for details). Drugs such as nicotine and methamphetamine readily function as FP and FN occasion setters using these procedures (see Bevins et al. (2006); Reichel et al (2007b); and later section on Discriminated Goal Tracking).

2/16/2011 5:50:29 PM

272

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

environmental features (e.g., material used to construct the chamber, speaker quality, etc.) and the organism (location within apparatus, wellness, learning history, etc.). Internal stimuli generated by abused substances are no exception, and they have the added complexity that the central and peripheral nervous system processes underlying their often widespread effects are not fully understood or even known. Despite these difﬁculties, we ﬁnd it instructional to provide at least a working deﬁnition. A drug stimulus is a complex polymodal event in which its elements reﬂect the relevant neurobiological process on which the drug or its metabolite acts directly or indirectly (i.e., downstream effects) at any given time. This deﬁnition attempts to capture some important aspects of a drug stimulus. For instance, the nature of the drug stimulus changes with time due to absorption, distribution, metabolisms, and so forth. In fact, several research groups have shown convincingly that the early “low” dose effects of a drug can serve as a CS for its later “high” dose effects of the drug (Greeley, Poulos, & Cappell, 1984; Kim, Seigel, & Patenall, 1999; see Within Drug Conditioning in next section for more detail). Further, metabolites of some drugs are active and may also contribute to the drug stimulus at later time points. A drug may also bind to several different receptor subtypes. These subtypes may have different binding afﬁnity for the drug and therefore require different synaptic drug levels for activation of the receptor. In other words, receptor speciﬁcity can vary with drug dose. Thus, the neurobiological processes recruited by a low dose of a drug can differ from a high dose not only in intensity or salience but also in the neurobiological elements that contribute to its stimulus effects. A further consideration is that receptor subtypes and their density vary with location in the nervous system. Because function varies with location, a drug’s stimulus effects are dependent on location of relevant receptors within the nervous system, as well as their interconnection with other physiological processes. Finally, drugs act on a physiology that evolved to survive in a particular environment, adapt to ﬂuctuations in that environment, and so forth.

12-Schachtman-12.indd 272

The impact of drugs should be considered within this context. Thus, if a drug acts predominately within the neural systems responsible for social afﬁliation, then the typical internal processes associated with this system could be altered long term (see Pankseep, 1998). This perspective also suggests that individual learning history and its neurophysiological underpinnings can alter the nature of a drug stimulus (see Acquired Excitatory Properties). A thorough description of the basic neuropharmacological principles discussed in this paragraph is well beyond the scope of this chapter. As such, we refer the reader to Feldman et al. (1997) and Koob and Le Moal (2006). These excellent reference sources provide more detailed and thorough description of these and other important processes involved in drug action. Drug States as Pavlovian Stimuli: Some Examples

Over the years, various conditioning tasks that have their historical origin in the associative or Pavlovian conditioning literature have been developed to study the interoceptive stimulus properties of abused substances. These tasks have been used to a varying degree of success. To facilitate later discussion of the contribution of this research to our understanding of drug stimuli and the addiction process, we ﬁrst describe a subset of these tasks in more detail. Between Drug Conditioning

One drug state can function as the CS for a different drug state (i.e., the US). For instance, amphetamine, a classic psychomotor stimulant, has an unconditioned effect of increasing heart rate in rats. Taking advantage of this, Revusky and Reilly (1990) administered pentobarbital (32 mg/kg, IP) 30 min before amphetamine (16 mg/kg, intramuscular). After several pairings of the pentobarbital CS with the amphetamine US, they found that pentobarbital evoked an increased heart rate in the absence of amphetamine. Rats that received equal exposure to the drugs in an explicitly unpaired fashion (separated by ∼24 hr) did not show a change in heart rate in the pentobarbital-alone test. In another

2/16/2011 5:50:30 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

study, the same dose of pentobarbital was repeatedly paired with nicotine (6 mg salt/kg, SC). Similar to the study with amphetamine as the US, the pentobarbital CS came to evoke an increased heart rate in the absence of nicotine; rats with unpaired presentations did not display this effect (Reilly & Revusky, 1992). Notably, such learning history with a drug CS has been shown to affect its ability to later function as a US in a different Pavlovian conditioning task (Revusky & Reilly, 1990). Brieﬂy, the pentobarbital CS was ﬁrst paired with the amphetamine US as described previously. Then, rats were shifted to a taste conditioning task in which a saccharin solution was paired with pentobarbital. The ability of pentobarbital to condition avoidance of the saccharin solution was reduced relative to a control group that had had pentobarbital and amphetamine unpaired in the earlier phase. This result indicates that the aversive properties of pentobarbital were attenuated by its association with amphetamine. Perhaps pentobarbital acquired some appetitive properties of amphetamine in the drug-drug conditioning phase (see Bevins, 2009; Revusky & Reilly, 1990). This conclusion has important implications for drug abuse liability and will be discussed in more detail in the section on “Acquired Excitatory Properties.” Within Drug Conditioning (Conditioned Tolerance)

Conditioned drug tolerance refers to a drugopposite CR that appears to “prepare” the animal for ensuing drug administration. This effect has been implicated in the ability of drug users to ingest ever-increasing doses of drugs. Further, it is the failure of conditioned tolerance to occur that has been implicated in drug “overdose” deaths (Siegel, 1984; Siegel & Ellsworth, 1986). In the now classic study of this effect using rats (Siegel, Hinson, Krank, & McCully, 1982), a distinct environment (i.e., context) CS was paired repeatedly with increasing doses of heroin. For a subset of the rats, there was a shift in the environment during the subsequent test of drug tolerance. This shift resulted in rats dying from a dose of heroin that on previous conditioning trials was well tolerated, and was

12-Schachtman-12.indd 273

273

still tolerated by rats that were tested in the original conditioning context. Siegel and Ellsworth (1986) report a case study on the death of a cancer patient that is strikingly similar to the experimental situation just described. The cancer patient had always received morphine administered in his darkened bedroom. Approximately 4 weeks into treatment, he died after he received the same dose of morphine as the day before, but administered in his living room that was well lit by the sun. The reaction to, and subsequent death from, this morphine administration was remarkably similar to an overdose, and it suggested to Siegel and Ellsworth that shifting the room (context) meant that there was no conditioned drug tolerance to counter the effects of the morphine. An important variant of this conditioned tolerance task uses a small dose of drug, rather than an external context, as the CS for a larger dose of the drug (i.e., the US). For example, in one study, rats received a low dose of alcohol (0.8 g/kg) paired 60 min later with a high dose of alcohol (2.5 g/kg). Another group of rats had the two doses unpaired throughout the study. In a test of conditioned tolerance to the hypothermic effects of alcohol, rats in the paired group showed a substantially lower hypothermic response to the high test dose of alcohol when presented with the low-dose alcohol CS (Greeley et al., 1984). Similarly, the early interoceptive effects of a single administration of morphine (i.e., the CS) can signal the later and more profound effects of that drug (i.e., the US). This association that occurs naturally within a single administration has come to be called an “intra-administration association” to distinguish it from the explicit pairings of separate small and large doses of the drug (see Greeley et al., 1984). Rats that received morphine infusions over 25–30 min displayed tolerance to the analgesic effects of morphine when tested with this longduration infusion. However, the same dose over a short period of 14–17 sec did not evoke any tolerance. Presumably, the shift in infusion speed changed the nature of the drug CS (see earlier); thus, no conditioned tolerance was evident (Kim et al., 1999). Additionally, when a small “probe” dose of morphine was administered, rats that

2/16/2011 5:50:30 PM

274

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

had been given the long infusions showed a hyperalgesic response in a tail-ﬂick test compared to rats that had been given the much shorter infusions of the same dose. Again, the CR evoked by the drug CS was attenuated once its temporal parameter was shifted away from that experienced during training. Because these “intra-administration associations” are possible in all drug-taking situations, their potential importance to continued use and extent of relapse is signiﬁcant (see McDonald & Siegel, 2004, and later section on “Interdrug Associations”). Within Drug Conditioning (Conditioned Neurotransmitter Release)

The addictive liability of cocaine is thought to arise primarily from its ability to increase dopamine concentration and hence, dopamine receptor binding, in the terminal ﬁelds of the mesocorticolimbic system of the brain (Caine & Koob, 1994; de Wit & Wise, 1977; Goeders & Smith, 1993). In cocaine-experienced animals, the initial rise of dopamine in this brain region appears glutamate mediated, whereas the later increase in dopamine results from cocaine inhibiting dopamine transporter function (You, Wang, Zitzman, Azari, & Wise, 2007). The glutamatemediated rise in dopamine has been attributed to environmental cues paired with cocaine triggering the release of glutamate onto dopaminergic neurons (Schultz, 1998; You et al., 2007). One prediction of this work is that the relatively immediate interoceptive effects of the drug, before cocaine reaches the central nervous system, may be among the environmental CSs evoking glutamate release. In a set of elegantly conducted studies, Wise and colleagues (2008) tested this prediction. Rats were trained to self-administer cocaine (1 mg/ kg/infusion) via pressing a lever. Following acquisition, the lever pressing was extinguished by replacing the cocaine with saline. Following extinction was a test of cocaine reinstatement of responding. Rats primed with cocaine (10 mg/ kg, IP) reinstated drug-associated lever pressing and showed a concomitant increase in glutamate release. The same pattern was achieved when rats were primed with cocaine methiodide (MI; 13 mg/kg, IP), a cocaine hydrochloride (HCl)

12-Schachtman-12.indd 274

analog. Cocaine MI does not cross the blood– brain barrier, yet it has the same action as cocaine HCl in the periphery. There was also no effect of an IP saline injection on lever pressing or glutamate release. Finally, reinstatement of lever pressing in rats primed with cocaine MI was blocked by infusing a glutamate antagonist into the ventral tegmental area of the mesocorticolimbic system. Combined, this pattern of results suggests that the initial peripheral drug effects of cocaine function as a CS to evoke glutamate release (i.e., the CR), and it appears to play a key role in this widely used model of relapse. Fear Conditioning

Pavlovian fear conditioning involving exteroceptive stimuli (e.g., light, tone, and situational cues) and an aversive outcome such as a footshock US has been widely studied in rodents (Bevins & Ayres, 1995; Estes & Skinner, 1941). There have been a few published examples of a drug state acquiring the ability to evoke a conditioned fear response. For example, Overton et al. (1993) using water-restricted rats examined whether pentobarbital (15 mg/kg, IP), PCP (3 mg/kg, IP), morphine (6 mg/kg, IP), or pentylenetetrazol (10 mg/kg, IP) could function as a CS; the US was a series of electric footshocks (0.9 mA), each lasting 0.5 sec and occurring every 10 sec over a 5-min period. Within an average of eight conditioning sessions, each of the drugs functioned as an effective CS+ coming to evoke conditioned suppression of drinking; the drugs did not affect drinking when trained as the CS–. This latter ﬁnding also suggests that suppression of drinking was not an unconditioned drug effect. Bormann and Overton (1993), using an extensive set of controls, conﬁrmed the observation of Overton et al. (1993) that morphine functioned as a CS in this fear-conditioning task and eliminated alternative nonassociative accounts. A similar effect was found by Turner and Altshuler (1976) using a lever-press conditioned suppression task with amphetamine as the CS. Sign Tracking

Sign tracking (also referred to as autoshaping) refers to the approach and interaction with the CS (i.e., the sign) developing through repeated

2/16/2011 5:50:30 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

pairings of the CS with a US. For example, pigeons that receive brief illumination of a keylight CS paired with access to grain will come to peck the key even though grain delivery does not require key pecking (Brown & Jenkins, 1968; Hearst & Jenkins, 1974). This autoshaped pecking response can come under the control of a drug state. In one such study by Parker and colleagues (1994), there were two distinct keylights, one white and one white with a black vertical line foreground. For a given pigeon, when methadone (2 mg/kg) was administered, one key-light (e.g., black line) was paired with food access; the other key-light (e.g., white) was unpaired with food access. On nondrug sessions, the relation between the key-lights and food access was reversed. Responding came under control of the interoceptive state with key-pecking occurring mostly to the key-light CS correlated with food in the methadone versus nondrug state. In a separate set of studies, these researchers found that a lower dose of methadone (1.7 mg/kg) or phencyclidine (PCP; 1 mg/kg) could also function as a positive feature in this sign-tracking task (Parker, Schaal, & Miller, 1994). More recently, a sexual conditioning task with Japanese quail has been used to demonstrate discriminative control of sign-tracking by an interoceptive drug state. In this conditioning task, male quail received brief presentation of a CS (e.g., plush toy dog or wood block) repeatedly paired with a copulatory opportunity with a sexually receptive female quail. The CS came to evoke approach behavior after repeated pairings; the same toy CS did not elicit this approach if it was explicitly unpaired with copulatory opportunity (Akins, Domjan, & Gutierrez, 1994; Domjan, O’Vary, & Green, 1988; see Chapter 22, this volume, for further discussion of this topic). Troisi and Akins (2004) used this sexual conditioning task to ascertain whether a cocaine drug state could function as an FP or FN occasion setter. In that study, male Japanese quail were presented with a cylindrical block CS. On reinforced sessions, the block CS presentation was followed by access to a sexually receptive female; on nonreinforced sessions, there was no access to a female. Cocaine (10 mg/kg, IP) was used as an FP in one group of quail, indicating that the

12-Schachtman-12.indd 275

275

block CS would be followed by access to the female (i.e., US); on saline sessions the block was presented, but access to the female was withheld. In a second group of quail, the cocaine state was used as an FN, indicating the absence of the female US following block presentations; the block CS was paired with copulatory opportunity only on saline sessions. Interoceptive conditioning was evidenced by male quail approaching the vicinity where the CS was presented only in the drug state for FP-trained quail or nondrug state for FN-trained quail. Discriminated Taste Aversion

As noted earlier, a conditioned taste aversion involves pairing a distinct taste CS with an illness-inducing US (e.g., Garcia & Koelling, 1966; Reilly & Schachtman, 2009). A drug state can function as an interoceptive stimulus disambiguating when a taste–illness pairing will (FP) or will not (FN) occur. For example, Jaeger and Mucha (1990) pretreated rats with fentanyl (0.04 mg/kg, SC) or saline before access to a novel taste (i.e., the CS). On saline sessions, the taste CS was followed by an illness-inducing injection of LiCl (90 mg/kg, IP; the US). On fentanyl sessions, the same taste was not followed by LiCl. Within four training sessions, a discriminated taste aversion (DTA) developed, with rats consuming more of the taste CS on drug than saline sessions. In this situation, fentanyl was serving as an FN inhibiting the avoidance CR. Likewise, when fentanyl was given on sessions in which the taste CS was followed by the LiCl, and saline was given on sessions in which the taste CS was not followed by LiCl, rats consumed more of the taste CS on saline rather than drug sessions. This effect indicates that fentanyl was also able to function as an FP, facilitating the avoidance CR to the taste CS. Morphine has also been widely studied as an FP and FN occasion setter in this DTA task (see Skinner, Goddard, & Holland, 1998, for an excellent review). Discriminated Goal Tracking

As noted earlier, animals will readily track stimuli (e.g., illuminated key-light) associated with appetitive USs, and a drug state can function as an FP or FN stimulus for this sign-tracking

2/16/2011 5:50:30 PM

276

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

behavior. Rats will also increase approach behavior to a place where an appetitive outcome has occurred in the past. Although this behavior clearly requires the animal to approach signs of the reward, when these stimuli are the location of the US (e.g., food cup, grain hopper, or dipper receptacle), the term goal-tracking is used to describe the behavior (Boakes, 1977; Farwell & Ayres, 1979). Like sign-tracking, drug states can acquire the ability to modulate a goal-tracking CR. For example, Maes and Vossen (1997) found that midazolam (0.1 mg/kg, SC) and amphetamine (0.5 mg/kg, SC) were effective FP occasion setters in rats. In these experiments, there were ﬁve 30-sec CS presentations (either light or light+tone) per session. Each CS presentation was paired with a food pellet on drug sessions. On intermixed saline sessions, the CS was not followed by food pellets (recall Fig. 12.1). Following six intermixed training sessions (i.e., three drug, three saline) was a drug and saline test session in which the CS was presented without the US. Whether amphetamine or midazolam was trained as the positive drug feature, the drug state facilitated the goal-tracking CR (i.e., shorter latencies) to the CS relative to saline. Interestingly, in these experiments separate groups were trained in which the drug state occasioned when the CS would not be followed by food (i.e., FN training). In the test, these rats had shorter response latencies in the presence of the CS regardless of interoceptive state (drug or saline). Thus, under these training conditions, amphetamine or midazolam did not function as an FN stimulus. These studies only used six sessions (i.e., 30 total trials). Although this was sufﬁcient for the FP situation, perhaps more training was needed to train a drug as a negative feature in a discriminated goal-tracking (DGT) task. Recent research from our laboratory lends some support to this suggestion. In that study, methamphetamine (0.5 mg/kg, IP) was trained as either an FP or FN occasion setter (Reichel, Wilkinson, & Bevins, 2007b). For FP rats, methamphetamine indicated that eight 15-sec light CS presentations across a 20-min session would each be followed by access to a sucrose solution for 4 sec; on intermixed saline sessions, no sucrose was delivered. Training conditions

12-Schachtman-12.indd 276

were reversed for the FN rats. The discrimination developed quickly in the FP condition (see Fig. 12.2). Relative to the saline (nondrug) state, methamphetamine facilitated CS-evoked goal-tracking (i.e., head entries into a dipper receptacle) within four intermixed sessions (i.e., 24 trials). However, FN discrimination did not stabilize until approximately 12 intermixed sessions (i.e., 96 trials). Of note, Troisi and Akins (2004), if anything, found that the FN discrimination was acquired faster than the FP discrimination using cocaine and sign tracking to a wood block CS presented in a location just adjacent to where the male quail had access to the receptive female. Clearly, much more research is required to determine the generality and signiﬁcance of such differences (e.g., drug type, extent of training, species, US type, CR form, etc.). Regardless, in addition to methamphetamine, published studies from our laboratory using this DGT task have shown that amphetamine (1 mg/kg, IP), bupropion (10 mg/kg, IP), caffeine (10 mg/kg, IP), chlordiazepoxide (5 mg/kg, IP), and nicotine (0.4 mg base/kg, SC) function as FP occasion setters (Murray, Palmatier, & Bevins, 2007; Palmatier & Bevins, 2007, 2008; Palmatier, Peterson, Wilkinson, & Bevins, 2004; Palmatier, Wilkinson, & Bevins, 2005; Wilkinson, Li, & Bevins, 2008). In addition, nicotine (0.4 mg base/kg, SC) served a negative drug feature in the DGT task (Bevins, Wilkinson, Palmatier, Seibert, & Wiltgen, 2006). Our laboratory has also studied the interoceptive CS effects of nicotine in the DGT task. In this conditioning task, rats are administered nicotine or saline before placement in a conditioning chamber. On nicotine sessions, access to sucrose is provided intermittently across the 20-min session, and there is no discrete CS preceding sucrose delivery; sucrose is withheld on saline sessions. The primary measure of conditioning is the rate of dipper entries before the ﬁrst sucrose delivery on nicotine sessions (or equivalent time on saline sessions). Nicotine comes to acquire control of conditioned responding as evidenced by an increase in goal tracking on nicotine compared to saline sessions. We have made a good start in elucidating the behavioral processes mediating the CS effects

2/16/2011 5:50:30 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES 10

0.20

FP

277 10

FN

Saline 8

Methamphetamine 0.15

6

6 4

4

0.10

2

2 0

#

#

#

# #

# #

#

# # #

4

6

8

10

# # #

–2 0

2

12

14

# #

# 0.05

#

#

// 0.00 16 0

# # # #

Elevation Scores

Elevation Scores

8

# # #

0 –2

2

4

6

8

10

12

14

16

Session

Figure 12.2 The left panel depicts acquisition using methamphetamine as a feature-positive (FP) occasion setter in a discriminated goal-tracking task. The right panel shows acquisition when methamphetamine was a feature-negative (FN) occasion setter in the same task. Elevation score (±1 SEM) is the difference between head entries during the discrete light cue and the equivalent time period just before presentation of that cue. Thus, a positive value indicates an increase responding during the light cue. The discrimination was acquired in both cases, although stable responding developed faster in the FP situation. # indicates signiﬁcant difference between methamphetamine and saline (p < 0.05). (Figure adapted from Reichel, C. M., Wilkinson, J. L., & Bevins, R. A. 2007b. Methamphetamine functions as a positive and negative drug feature in a Pavlovian appetitive discrimination task. Behavioural Pharmacology, 18, 755–765).

of nicotine. A range of nicotine doses (0.1 to 0.4 mg base/kg) are effective CSs (Besheer, Palmatier, Metschke, & Bevins, 2004; Murray & Bevins, 2007a, 2007b; Reichel, Linkugel, & Bevins, 2007a; Wilkinson et al., 2006). Generalization to other nicotine doses and duration of the CS effects also vary with the salience (dose) of the nicotine CS used in training (e.g., Murray & Bevins, 2007a, 2007b). Asymptotic levels of conditioned responding to the nicotine CS increase as a function of sucrose US concentration (Murray, Penrod, & Bevins, 2009) and number of nicotine CS-sucrose US pairings per session (Wilkinson et al., 2006). When the sucrose US is withheld from nicotine sessions (i.e., extinction), CR magnitude decreases across repeated extinction sessions (Besheer et al., 2004). Persistence of conditioned responding in extinction increases with salience (dose) of the nicotine CS (Murray & Bevins, 2007b) and the number of sucrose US deliveries per nicotine session (Wilkinson et al.,

12-Schachtman-12.indd 277

2006). Reacquisition following extinction is very rapid (Murray & Bevins, 2007b; Wilkinson et al., 2006). Following acquisition of DGT to nicotine, introduction of sucrose deliveries on saline sessions disrupt the discrimination by increasing goal tracking on saline sessions, likely reﬂecting a switch of stimulus control from the nicotine drug state to the exteroceptive chamber cues (Wilkinson et al., 2006). Importantly, rats that acquired an appetitive dipper entry response in either a nicotine or saline state did not show disrupted responding when tested in the alternate state (Bevins, Penrod, & Reichel, 2007). This ﬁnding strains any state-dependent learning or recall account of nicotine-evoked conditioned responding and supports the conclusion the interoceptive stimulus effects of nicotine are functioning as a CS. Finally, rats that receive nicotine CS− training in this DGT task (i.e., sucrose only on saline sessions) withhold responding on nicotine sessions but display the

2/16/2011 5:50:30 PM

278

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

goal-tracking CR on intermixed saline sessions (Besheer et al., 2004). In the research described in the previous paragraph, nicotine-administered SC is conceptualized as a relatively long-lasting interoceptive contextual CS—nicotine has a half-life of approximately 50 min in rats (Ghosheh, Dwoskin, Li, & Crooks, 1999). More recently, we extended this research to intravenous (IV) infusions of nicotine (Murray & Bevins, 2009). The idea was to create a more discrete interoceptive stimulus in which conditioned responding could be monitored on a trial-by-trial basis. Success at doing so would open up all sorts of research not possible with subcutaneously administered nicotine (see Murray & Bevins, 2009). In this variant of the DGT task, the session duration was increased from 20 min to 2 hr and there were 10 IV nicotine infusions (0.03 mg base/kg, 1 sec). Each infusion was followed 30 sec later by 4-sec access to sucrose. With repeated nicotine–sucrose pairings, there was a clear increase in goal tracking during the 30 sec postinfusion. This increase was not a nonspeciﬁc increase in dipper entries resulting from drug infusion; a control condition with explicitly unpaired nicotine infusions and sucrose deliveries displayed no such increase in responding. The exact nature of this nicotine CS in this task is an interesting empirical and theoretical question. The IV nicotine conditioning protocol can be conceptualized as a cumulative dosing regimen given the length of nicotine’s half-life. As such, the nicotine CS in this task is likely the early interoceptive nicotine effects that accompany the 1-sec infusion (see McDonald & Siegel, 2004; see section on “Within Drug Conditioning”). Interestingly, this is not completely unlike what we expect is happening with chronic smokers. That is, with perhaps the exception of the ﬁrst cigarette of the day, each cigarette consumed produces an increase in the level of nicotine in the nervous system above some level present from earlier cigarettes.

INTEROCEPTIVE CONDITIONING AND DRUG ADDICTION Relative to the other tasks, the DGT and DTA protocol have been used to elucidate the

12-Schachtman-12.indd 278

neuropharmacological processes underlying the stimulus effects of some drug stimuli. The “workhorse” in this area, however, has been the twolever operant drug discrimination task, and we refer the reader to this rich and extensive literature for more information on such processes (see Porter & Prus, 2009). There has been mostly good agreement between the two-lever drug discrimination task and the DGT and DTA tasks regarding neuropharmacological processes (but see Murray & Bevins, 2007a; Murray, Wells, Lyford, & Bevins, 2009). However, the goal of the present chapter is not to review this research. Rather, we will spend the remainder of the chapter discussing research that suggests that Pavlovian conditioning not only alters or modiﬁes behavior evoked by the interoceptive stimulus effects of a drug, but that such conditioning alters the function of that drug state in a way that is likely to be fundamentally important for addiction. Surprisingly, given the potential health relevance of these lines of inquiry, there has been very little research in some areas. Thus, some of the following sections will be more of a call for research than a discussion of current knowledge. Acquired Excitatory Properties

The preponderance of drug conditioning research and theory conceptualizes the drug as a US that becomes associated with environmental stimuli (drug paraphernalia, fellow user, place, etc.) that reliably occur in the drug-taking situation. These stimuli are thought to accrue the ability to evoke drug-related CRs that prompt continued drug use and relapse (e.g., Koob, 2004; Robinson & Berridge, 1993). Further, these stimuli should also function as conditioned reinforcers. This appears to be the case. The ﬁnding that smokers given denicotinized cigarettes for 11 days only reduced intake by a small amount is a very potent example of this view and the importance of learning history on continued drug use (Donny, Houtsmuller, & Stitzer, 2007). This should also hold for drug stimuli. That is, drug use often co-occurs with other affective stimuli—appetitive (positive) or aversive (negative). If these stimuli are reliably and sufﬁciently appetitive, then one

2/16/2011 5:50:30 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

can imagine that the drug state could accrue conditioned reinforcing value that increases the likelihood of future drug use and speeds the trajectory to chronic use and addiction (see Alessi, Roll, Reilly, & Johanson, 2002; Bevins, 2009; Bevins & Palmatier, 2004). The following is an excellent example from Alessi et al. (2002): Similarly, imagine having to attend an early morning meeting each Friday. On several occasions you are so tired that you cannot make any meaningful contributions. However, if you take a caffeine pill 30 min before the meeting, you ﬁnd that you are able to be an active participant and as such are able to obtain reinforcement that was previously unavailable. An arrangement such as this could result in the caffeine acquiring the properties of a conditioned reinforcer, which may in turn further increase your likelihood of using caffeine. (p. 82)

As this hypothetical scenario suggests, acquired reinforcing value may be an important contributor to drug abuse. Despite its potential importance, very little research has been conducted to directly test this idea (see earlier discussion of Revusky & Reilly, 1990). One important published example was conducted by Alessi et al. (2002). Brieﬂy, they recruited healthy volunteers to the laboratory and conducted a two-phase study. In the ﬁrst phase, they gave participants placebo or diazepam (5 mg) in capsules on separate sessions and then allowed them to choose which they preferred. Consistent with past research, 5 of the 6 individuals preferred placebo over diazepam. In the second phase, the researchers then paired the nonpreferred state (diazepam for 5 of the participants) with high monetary payoff in a computer task; the preferred state was paired with low monetary payoff. Notably, payoff was not contingent on the behavior in this computer task. Following this conditioning phase, they reassessed the preference and found that all individuals that preferred placebo had switched their preference to diazepam. Further, the subjective reports of liking switched to diazepam after the appetitive conditioning history. Although a control for shift due to mere drug exposure was not included in the study, the authors convincingly argue that

12-Schachtman-12.indd 279

279

the shift in preference likely reﬂects the drug state acquiring conditioned reinforcing value. Our lab has begun to test this idea using IV nicotine as the CS. In our preliminary study, rats (n = 4 per group) received 0.01 mg/kg IV nicotine either paired or unpaired with brief access to sucrose for 10 sessions (see Murray & Bevins, 2009; see section on “Discriminated Goal Tracking”). Rats that had nicotine paired with sucrose developed a goal-tracking CR in the time period immediately after the infusion; goal tracking did not develop in the unpaired group. We interpret the control of goal tracking by nicotine in the paired group as reﬂecting acquisition of conditioned appetitive properties to the nicotine drug state. If so, this low dose of nicotine should be more rewarding in other behavioral tasks that measure conditioned reward. To test this idea, we followed CS training with IV nicotine place conditioning for four conditioning trials. Place conditioning is a widely used model to study the conditioned motivational effects of a drug. In a typical place conditioning experiment, drug is paired with one distinct environment; placebo is administered in a second distinct environment (see Fig. 12.3A for an image of our place conditioning apparatus). If the drug had rewarding effects, then a preference would be shown for the paired compartment on a drug-free choice test (see Bardo & Bevins, 2000; Bevins & Cunningham, 2006). Previous research from our laboratory has shown that 0.01 mg/kg IV nicotine does not condition a place preference after four conditioning trials (Wilkinson & Bevins, 2008). Consistent with this earlier ﬁnding, the 0.01 mg/kg dose did not condition a preference in the group that had received nicotine explicitly unpaired with sucrose; only one rat spent more time in the nicotine-paired compartment (see Fig. 12.3B; left-most bars). In contrast, 0.01 mg/kg nicotine was able to condition a place preference in rats that previously had this dose repeatedly paired with sucrose (right-most bars). In fact, all four rats in this group spent more time in the paired than unpaired compartment on this test. This outcome is consistent with the notion than an appetitive learning history with a drug state can alter its motivational valence (see also Revusky & Reilly, 1990).

2/16/2011 5:50:30 PM

280

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Like Alessi et al. (2002), we believe that such conditioning has major implications for drug use, addiction, relapse, and its treatment. For instance, it could provide insight into individual differences in the trajectory from experimentation to chronic drug use seen in the population. Individuals that experiment with a drug in situations containing a greater number of appetitive or reinforcing events may be more vulnerable to reusing the drug and eventually developing dependence. Further, such conditioning history could help explain how drugs that appear to have weak, if any, direct reinforcing effects in animal models (e.g., LSD, valium, nicotine) could have signiﬁcant addictive liability. Admittedly, at this time, these examples are

highly speculative and await programmatic research in human and non-human animal laboratory studies. Interdrug Associations

Pavlovian conditioning in which two drugs reliably co-occur has not been widely studied (e.g., Clements, Glautier, Stolerman, White, & Taylor, 1996; Li, He, & Mead, 2009; Reilly & Revusky, 1992; Revusky, Davey, & Zagorski, 1989; Revusky & Reilly, 1990). This deﬁcit is somewhat surprising given the prevalence of polydrug abuse. For instance, in 2002, 56% of all individuals entering publically funded treatment programs report abusing at least two drugs (Substance Abuse and

A

B Time (sec) spent in compartment

400

Place conditioning compartment Unpaired

Paired

300 #

200

100

0 Unp:Nicotine/ /Sucrose

Paired: Nicotine--Sucrose

Previous conditioning history with nicotine

Figure 12.3 (A) Our place-conditioning apparatus that has two distinct end compartments separated by

a smaller placement area. The end compartments vary as to the ﬂooring: 340 1.3 cm holes drilled into a 16-gauge aluminum sheet versus two (1 cm diameter) rods mounted side by side with the adjacent rod pairs separated by 1 cm. (B) The results of a preliminary study conducted by Jamie Wilkinson in the laboratory. Rats that had 0.01 mg/kg IV nicotine paired with sucrose tended to be more sensitive than unpaired controls to the subsequent rewarding effects as measured by place conditioning (see text for detailed description; # denotes paired t-test, p = 0.07, two-tailed rejection region).

12-Schachtman-12.indd 280

2/16/2011 5:50:30 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

Mental Health Services Administration, 2005). Notably, when looking at this data by age, the greatest polydrug abuse (65%) was reported by treatment seekers 19 years old or younger; polydrug abuse then decreased with age. Another example not necessarily picked up by Treatment Episode Data Set is the signiﬁcant comorbidity between a diagnosis with an alcohol drinking problem and smoking (Heffner, Barrett, & Anthenelli, 2007; Schmitz, Sayre, Hokanson, & Spiga, 2003). There has been a recent increase in research on the behavioral and neural effects of co-treatment with nicotine and ethanol, as well as the impact of pretreatment with one on later effects of the other (e.g., Leeman et al., 2008; Prendergast, Harris, Mayer, & Littleton, 2000). However, this research has not typically approached the problem from the perspective detailed in this chapter. That is, does alcohol function as a US conferring conditioned reinforcing value to the nicotine CS? Alternatively, perhaps alcohol functions as an FP occasioning when a nicotine CS-socializing US relation will be in force. A similar paucity of research exists for studying the import of conditioned associations involving early drug effects. Nonreinforcement Not Always Sufﬁcient

Cue-exposure and related therapies for drug abuse are based on the assumption that drugrelated stimuli function as “simple” excitatory CSs evoking conditioned responses that drive drug taking and precipitate relapse (see Carter & Tiffany, 1999; Dadds, Bovbjerg, Redd, & Cutmore, 1997; Niaura et al., 1999; O’Brien, Childress, McLellan, & Ehrman, 1992; Siegel & Ramos, 2002). In this treatment approach, the nonreinforcement (extinction) of these CSs should be effective in treating drug addiction and preventing relapse. Although there has been some success with this approach, its effectiveness is often limited (e.g., Carter & Tiffany, 1999; Conklin & Tiffany, 2002). Conklin and Tiffany (2002) provide an excellent discussion of this literature and possible reasons for this limited success. We would add at least one more reason to their list. That is, typical cue-exposure therapy does not consider the drug state as a CS in the

12-Schachtman-12.indd 281

281

same manner as drug paraphernalia or situational cues. Consistent with this suggestion, Siegel and Ramos (2002) noted that “cue exposure treatments sometimes may be ineffective because there is no attempt to extinguish these highly salient DOCs (drug onset cues) . . .it is possible that mere exposure to predrug environmental cues may not effectively extinguish the association between predrug cues and the drug effect” (p. 173, italic words added for clarity). Considering the potential learning involving the interoceptive drug state suggests several important factors that may contribute to the difﬁculty of quitting drug use and the high rate of relapse seen with most abused substances. For instance, whether a drug state serves as a CS for rewarding events, or an FP occasion setter signaling some other appetitive CS-US relation, the positive experiences associated with the interoceptive effects of the drug likely make it more difﬁcult to quit. Similarly, this associative learning would make it more difﬁcult to return to abstinence after one had lapsed during a quit attempt. This could be a couple of puffs of a friend’s cigarette for a smoker or a glass of wine at a reception for an alcoholic. If the drug state serves as a CS, then presenting the drug state in the absence of any reinforcers (i.e., extinction) should produce interfering or competing associations that would aid in the cessation attempt. As noted earlier, previous research in our laboratory has shown that conditioned responding evoked by a nicotine CS in the DGT task decreases when the sucrose US is withheld (Besheer et al., 2004). Further, conditioned responding is more resistant to extinction with higher training doses (i.e., more salient CS) and with increased number of nicotine-sucrose pairings in the acquisition phase (Murray & Bevins, 2007b; Wilkinson et al., 2006). Magnitude of the US also affects extinction of conditioned responding with goaltracking evoked by the nicotine CS (0.4 mg base/ kg) being more persistent at the higher sucrose US concentrations (unpublished data). These ﬁndings are consistent with research using more traditional exteroceptive CSs and suggest that extinction of a drug CS could be a treatment option. Indeed, Sirtharthan et al.

2/16/2011 5:50:31 PM

282

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(1997) included priming doses of alcohol (i.e., drug CS) as part of the stimuli in cue-exposure therapy for alcoholics and found that it can be an effective intervention strategy (see also Rankin, Hodgson, & Stockwell, 1983). Albeit potentially effective and theoretically possible, there are signiﬁcant challenges with implementing such a strategy. For instance, exposing an addict to the substance of addiction is fraught with ethical concerns, and the approach is likely limited by the drug of abuse. That is, it may be perceived as acceptable to expose a smoker to nicotine (e.g., nicotine patch or gum), but the conversation shifts when giving heroin to a heroin addict while the individual is engaged in a serious quit attempt. Perhaps methadone could be used as a partial substitute for the heroin state if such a treatment strategy were adopted. To this point, our discussion of extinction has focused on the drug as a CS. What if critical learning for addiction involves the drug functioning as an occasion setter? In Pavlovian conditioning research using exteroceptive stimuli, the modulating function of an occasion setter is often not susceptible to extinction (Miller & Oberling, 1998; Swartzentruber, 1995). That is, a discrete stimulus trained as an FP occasion setter for a CS-US association will still prompt conditioned responding to the CS even after the FP stimulus has been presented repeatedly without the US. Such an effect suggests that its modulating function does not come from the summation of excitatory conditioning between the FP stimulus and the CS. We have conﬁrmed this observation with drug states trained as FP occasion setters in the DGT task (Palmatier & Bevins, 2007). In that research, rats were trained with amphetamine (1 mg/kg), chlordiazepoxide (5 mg/kg), or nicotine (0.4 mg/kg) as an FP occasion setter. Then, each drug state was repeatedly presented alone—no sucrose or discrete CS in the extinction sessions. This extinction training did not disrupt the ability of any drug to later facilitate responding to the discrete CS and suggests that the drug states did not modulate the CR via a direct association with sucrose. Rather, the drug state appears to facilitate conditioned responding through a higher order or nonassociative mechanism. One implication of this

12-Schachtman-12.indd 282

ﬁnding is that simple extinction of the drug state may not be sufﬁcient to alter its control over behavior. As such, more elaborate and individually tailored strategies that provide a competing learning history involving the drug feature may be required (Bevins & Palmatier, 2004; Palmatier & Bevins, 2007). Functional versus Pharmacological Substitution

As noted in the previous section, the ability of an FP drug state to modulate responding in the DGT task despite undergoing extensive extinction suggests a higher order or nonassociative process. Stated in a different way, there is more than a direct association between the drug state and the reinforcer involved in controlling responding. If so, the Pavlovian conditioning literature on occasion setting also suggests that drug states trained as positive features for separate CS-US associations should substitute for one another based on this associative learning history (e.g., Bonardi & Hall, 1994) rather than any common neuropharmacological process. Research from our laboratory has conﬁrmed this prediction in the DGT task (Palmatier & Bevins, 2008). For instance, nicotine was trained as an FP occasion setter for one discrete CS (cue light) as described earlier. In the same rats, chlordiazepoxide was trained as an FP occasion setter using a different CS (white noise). When performance stabilized on both discriminations, rats were tested with the different drug state and CS combinations. Each FP drug state transferred its ability to modulate conditioned responding to the CS that occurred in the other drug state (compare Transfer Test to Training State in Fig. 12.4). Such facilitation was not seen to a novel CS that had been partially reinforced before testing. Importantly, these drug states are pharmacologically unrelated in that chlordiazepoxide does not substitute for a nicotine FP occasion setter unless both are trained as occasion setters (Palmatier et al., 2005). Further, treatment with amphetamine, a novel drug state, did not facilitate conditioned responding (see Novel [Amp] Test in Fig. 12.4). This result indicated that training two discriminations

2/16/2011 5:50:31 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES 10

Elevation Score

8

Training State

6 4 2 0 Transfer Test

Novel (Amp) Test

Figure 12.4 This ﬁgure summarizes the ﬁnding

from Palmatier and Bevins (2008) investigating the transfer of occasion setting across drug states. The solid line with the juxtaposed dashed lines reﬂects the mean elevation score (±1 SEM) evoked by the drug feature and conditioned stimulus (CS) used in the training phase. When a novel combination of drug feature and CS was tested, there was complete transfer of the feature-positive occasion-setting function of the drug state (Transfer Test). Merely testing an untrained and novel drug state (i.e., amphetamine) did not prompt an increase in conditioned responding to the CS (Novel [Amp] Test).

in the same rat does not simply produce a default “rule” of respond when in a drug state. Interestingly, similar speciﬁcity does not occur in the DTA task. Skinner et al. (1998) have shown that an FP drug state in this procedure reduced intake of novel and familiar tastes that have not been explicitly trained in FP discrimination. They suggest that a drug state trained as an FP in DTA task likely occasions a more general consummatory response–illness US association rather than CS–US association as in the discriminated goal tracking task. Regardless, under some conditions a drug’s ability to modulate conditioned responding transfers its control to a separate CS-US relation modulated by a pharmacologically unrelated drug (Palmatier & Bevins, 2008). Thus, generalization of responding is based on overlap in learning histories (functional substitution) rather than shared pharmacological properties such as binding to nicotinic acetylcholine receptors. To us, this ﬁnding clearly indicates the profound effect learning may have on the

12-Schachtman-12.indd 283

283

functional role of drugs and extends the classes of drugs that can substitute for each other and hence prompt conditioned withdrawal, craving, and so on that lead to continued use, as well as prompt relapse. For example, there are some studies indicating that steady or increased tobacco consumption can trigger relapse in recovering alcoholics (e.g., Friend & Pagano, 2005). Perhaps some of these relapses could be accounted for by functional transfer of occasion setting between nicotine and ethanol. Clearly much more research is needed in this area to assess the potential impact and import of such shared learning histories on addiction. Alteration of Stimulant Effects

Many drugs of abuse have powerful stimulant effects. The neurobiological processes underlying these effects overlap with those involved in its rewarding properties and likely play a role in abuse liability (see Robinson & Berridge, 1993; Wise & Leeb, 1993). One such drug is methamphetamine. As an example, rats were treated IP with 1 mg/kg methamphetamine or saline for 8 days (see Bevins & Peterson, 2004, Experiment 1). Activity, measured as infrared beam breaks, was recorded in circular locomotor chambers for 30 min. The average activity counts on day 8 were 840 for the methamphetamine rats; the average for the saline rats was 364. This reﬂects a nearly 130% increase in activity induced by methamphetamine over that of controls. The psychomotor effects of methamphetamine appeared altered when methamphetamine was trained as an FN (Reichel et al., 2007b). Brieﬂy, one group of rats was trained with methamphetamine (0.5 mg/kg, IP) as FP occasion setter indicating when a 15-sec light CS was paired with sucrose. The other group had methamphetamine trained as an FN occasion setter; light was not reinforced when in the methamphetamine state. Following acquisition of the discrimination was an intermixed series of 4-min generalization tests using a range of methamphetamine doses (0.025 to 1 mg/kg). Not surprisingly, the occasion-setting function of methamphetamine was dose dependent with lower doses of methamphetamine having less stimulus control over

2/16/2011 5:50:31 PM

284

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

behavior. Of interest in the present discussion was the ﬁnding that the locomotor stimulant effects of methamphetamine were blunted in rats trained with methamphetamine as FN stimulus. For instance, despite the test only being 4 min, FP-trained rats treated with 1 mg/kg methamphetamine in the generalization test displayed a statistically signiﬁcant 48% increase in activity above its own saline baseline. In contrast, the FN-trained rats’ activity on 1 mg/kg methamphetamine did not differ from saline— only a 27% increase in activity induced by methamphetamine. This pattern was not speciﬁc to methamphetamine. Substitution tests revealed methamphetamine-like stimulus effect of cocaine and bupropion. Notably, the locomotor activating effects of these two stimulant drugs were signiﬁcantly attenuated in FN-trained rats (see Fig. 12.5). Recall that FN-trained rats withhold dipper entries on methamphetamine sessions; thus, head entries into the receptacle were not interfering with moving around the chamber on the test days. Beyond documenting this difference, little is known about the generality of this effect or the mediating processes. Although the implications of such a ﬁnding require further research, it seems important that the

120

Cocaine y = 7.7x + 70.4

110

Bupropion y = 4.1x + 71.8 FP FN

Activity

100 Cocaine y = 1.6x + 71.8

90

Bupropion y = 0.5x + 74.0

80 70 60 0 0.1 1

3

5

10 Drug Doses (mg/kg)

20

Figure 12.5 This ﬁgure plots the line of best ﬁt

for the locomotor activity data reported and described in more detail in Reichel et al. (2007b). When methamphetamine was trained as a featurenegative (FN) occasion setter, the locomotor stimulant effects of cocaine and bupropion in brief substitution tests were blunted relative to rats that had methamphetamine trained as a feature-positive (FP) occasion setter.

12-Schachtman-12.indd 284

psychomotor stimulant effects of methamphetamine, cocaine, and bupropion were blunted when methamphetamine was trained as FN occasion setter. Conditioned Inhibitory Properties

Wasserman et al. (1974) found that pigeons moved and remained away from a stimulus that signaled the absence of food. We suggested in an earlier paper that methamphetamine trained as FN might control similar avoidance or withdrawal from the location of the sucrose receptacle and CS lights (located above and to each side of the dipper), hence interfering with the psychomotor effects of the drug (Reichel et al., 2007b). Albeit speculative, this possibility suggests that the FN methamphetamine may function as a conditioned inhibitor (see Bevins et al. 2006; Skinner, Martin, Howe, Pridgar, & van der Kooy, 1995). To demonstrate that the interoceptive effects of a drug have acquired inhibitory properties and function as a conditioned inhibitor, an FN drug state must pass the summation and retardation tests (Pavlov, 1927; Rescorla, 1969). To pass the summation test, a putative conditioned inhibitor has to inhibit conditioned responding to a CS that has been trained as an excitor separate from any occasion-setting function. To pass the retardation test, later acquisition of an excitatory association (i.e., CR) to the putative conditioned inhibitor will be impaired (retarded) relative to controls. Of note, the morphine drug state passes the summation test when trained as an FN occasion setter in the DTA task (Skinner et al., 1995). Brieﬂy, in the initial training phase a morphine drug state occasioned when a vinegar solution (i.e., the CS) would not be paired with illness. When placebo (nondrug) was given, rats had access to the same vinegar solution, but the vinegar CS was paired with illness. Thus, rats avoided consumption of the vinegar solution only on nondrug trials. In the summation test, the negative morphine feature diminished avoidance of saccharin solution that had previously been paired with illness. It does not appear that the retardation-of-acquisition test can be conducted within the DTA task. Doing so would

2/16/2011 5:50:31 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

require somehow making the interoceptive morphine stimulus a taste CS. Experiments are underway in our laboratory to test whether nicotine trained as an FN occasion setter in the DGT task (see Bevins et al., 2006) passes these tests of conditioned inhibition. Although our early indication is that nicotine does pass these tests, additional controls are required before making any conclusion as to whether an FN drug state acquires conditioned inhibitory properties.

CLOSING THOUGHTS The preponderance of research on Pavlovian conditioning involving abused drugs conceptualizes the drug as a US modifying responding evoked by exteroceptive environmental stimuli. The preclinical and clinical literature certainly indicates that these conditioned associations with the drug as a US play an important role in the addiction processes. The research discussed in this chapter further indicates that the interoceptive stimulus effects of these drugs may function as Pavlovian stimuli—CS+, CS−, FP occasion setter, and FN occasion setter. As such, these interoceptive drug stimuli come to acquire or modulate control over behaviors they previously did not control. Drug stimuli appear to acquire conditioned excitatory strength that alters their perceived hedonic valence. As occasion setters, their facilitating effects appear to resist extinction, and modulation can transcend drug classes with a shared associative learning history. In brief, we believe the evidence supports the conclusion that Pavlovian conditioning involving the interoceptive stimulus effects of a drug can change the function and meaning of that drug state in a manner that contributes to early drug use, later chronic use and abuse, as well as relapse. The exact nature of such changes and their relative import will require much more research. Even if the reader only accepts part of the conclusions in the previous paragraph, the accumulating research discussed in the chapter clearly indicates that theories of drug use and addiction must move beyond simple excitatory conditioning in which exteroceptive stimuli enter into direct association with the drug.

12-Schachtman-12.indd 285

285

Associative processes involving the internal states produced by drugs either directly or indirectly must be considered in a comprehensive theory of drug addiction. Pavlov (1927) clearly appreciated the potential and far-reaching role of interoceptive stimuli to human psychopathology. For instance, in the closing paragraph of Lecture XX on pathological disturbances (p. 360), Pavlov notes the following: If one bases conclusions on the experiments as they stand, the motor area of the cortex must be thought of as an analyser of the impulses from muscles and joints (proprioceptive), exactly as other areas are analysers of impulses from stimuli acting on the organism from the outside (exteroceptive). From this point of view the entire cortex represents a complex system of analysers of the internal as well as of the external environment of the organism. Obviously, if one accepts this hypothesis in relation to the motor activity, there is good reason to extend it to the activity of most, if not all, other tissues of the organism. The important role played by autosuggestion with all its extraordinary aspects, as, for example, imaginary pregnancy, and all sorts of imaginary diseases, can be understood from the physiological point of view only if we admit the existence of corresponding cortical analysers, even though they may be only little differentiated and indeﬁnite.

Hopefully this chapter has convinced the reader to follow Pavlov’s lead and extend the role of associative learning processes with interoceptive states to the devastating health problems society faces, including the primary topic of this chapter, drug addiction.

ACKNOWLEDGMENTS Thanks to everyone in the Behavioral Neuropharmacology Laboratory over the years that have worked so tirelessly on the research discussed in this chapter. R. A. Bevins and some of the research described in this chapter were supported by DA018114 and DA023951. J. E. Murray was supported by DA025399. Correspondence concerning this chapter should be addressed to Rick A. Bevins, Ph.D., Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE 68588-0308; e-mail: [email protected].

2/16/2011 5:50:31 PM

286

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

REFERENCES Akins, C. K., Domjan, M., & Gutiérrez, G. (1994). Topography of sexually conditioned behavior in male Japanese quail (Coturnix japonica) depends on the CS-US interval. Journal of Experimental Psychology: Animal Behavior Processes, 20, 199–209. Alessi, S. M., Roll, J. M., Reilly, M. P., & Johanson, C-E. (2002). Establishment of a diazepam preference in human volunteers following differential-conditioning history of placebo versus diazepam choice. Experimental and Clinical Psychopharmacology, 10, 77–83. Bardo, M. T., & Bevins, R. A. (2000). Conditioned place preference: What does it add to our preclinical understanding of drug reward? Psychopharmacology, 153, 31–43. Besheer, J., Palmatier, M. I., Metschke, D. M., & Bevins, R. A. (2004). Nicotine as a signal for the presence or absence of sucrose reward: A Pavlovian drug appetitive conditioning preparation in rats. Psychopharmacology, 172, 108–117. Bevins, R. A. (2009). Altering the motivational function of nicotine through conditioning processes. In R. A. Bevins & A. R. Caggiula (Eds.), The motivational impact of nicotine and its role in tobacco use. The 55th Nebraska Symposium on Motivation (pp. 111–129). New York, NY: Springer. Bevins, R. A., & Ayres, J. J. B. (1995). One-trial context fear conditioning as a function of the interstimulus interval. Animal Learning and Behavior, 23, 400–410. Bevins, R. A., & Cunningham, C. L. (2006). Place conditioning: A methodological analysis. In M. Anderson (Ed.), Tasks and techniques: A sampling of methodologies for the investigation of animal learning, behavior, and cognition (pp. 99–110). Hauppauge, NY: Nova Science Publishers, Inc. Bevins, R. A., & Palmatier, M. I. (2004). Extending the role of associative learning processes in nicotine addiction. Behavioral and Cognitive Neuroscience Reviews, 3, 143–158. Bevins, R. A., Penrod, R. D., & Reichel, C. M. (2007). Nicotine does not produce statedependent effects on learning in a Pavlovian appetitive goal-tracking task in rats. Behavioural Brain Research, 177, 134–141. Bevins, R. A., & Peterson, J. L. (2004). Individual differences in rats’ reactivity to novelty and

12-Schachtman-12.indd 286

the unconditioned and conditioned locomotor effects of methamphetamine. Pharmacology, Biochemistry, and Behavior, 79, 65–74. Bevins, R. A., Wilkinson, J. L., Palmatier, M. I., Seibert, H. L., & Wiltgen, S. M. (2006). Characterization of nicotine’s ability to serve as a negative feature in a Pavlovian appetitive conditioning task in rats. Psychopharmacology, 184, 470–481. Boakes, R. A. (1977). Performance on learning to associate a stimulus with positive reinforcement. In H. Davis & H. M. B. Hurwitz (Eds.), Operant-Pavlovian interactions (pp. 67–97). Hillsdale, NJ: Erlbaum. Bonardi, C., & Hall, G. (1994). Occasion-setting training renders stimuli more similar: Acquired equivalence between the targets of featurepositive discriminations. Quarterly Journal of Experimental Psychology B, Comparative and Physiological Psychology, 47, 63–81. Bormann, N. M., & Overton, D. A. (1993). Morphine as a conditioned stimulus in a conditioned emotional response paradigm. Psychopharmacology, 112, 277–284. Brown, P. L., & Jenkins, H. M. (1968). Autoshaping of the pigeon’s key-peck. Journal of the Experimental Analysis of Behavior, 11, 1–8. Bykov, K. M. (1957). The cerebral cortex and the internal organs. New York, NY: Chemical Publishing Company. Caine, S. B., & Koob, G. F. (1994). Effects of mesolimbic dopamine depletion on responding maintained by cocaine and food. Journal of the Experimental Analysis of Behavior, 61, 213–221. Carter, B. L., & Tiffany, S. T. (1999). Meta-analysis of cue-reactivity in addiction research. Addiction, 94, 327–340. Clements, K., Glautier, S., Stolerman, I. P., White, J-A. W., & Taylor, C. (1996). Classical conditioning in humans: Nicotine as CS and alcohol as US. Human Psychopharmacology: Clinical and Experimental, 11, 85–95. Conklin, C. A., & Tiffany, S. T. (2002). Applying extinction research and theory to cue-exposure addiction treatments. Addiction, 97, 155–167. Cook, L., Davidson, A., Davis, D. J., & Kelleher, R. T. (1960). Epinephrine, norepinephrine, and acetylcholine as conditioned stimuli for avoidance behavior. Science, 131, 990–991. Dadds, M. R., Bovbjerg, D., Redd, W., & Cutmore, T. (1997). Imagery in human classical conditioning. Psychological Bulletin, 122, 89–103.

2/16/2011 5:50:31 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

de Wit, H., & Wise, R. A. (1977). Blockade of cocaine reinforcement in rats with the dopamine receptor blocker pimozide, but not with the noradrenergic blockers phentolamine or phenoxybenzamine. Canadian Journal of Psychology, 31, 195–203. Doty, R. W. (1961). Conditioned reﬂexes formed and evoked by brain stimulation. In D. E. Sheer (Ed.), Electrical stimulation of the brain: An interdisciplinary survey of neurobehavioral integrative systems (pp. 397–412). Austin TX: University of Texas Press. Domjan, M., O’Vary, D., & Green, P. (1988). Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. Journal of the Experimental Analysis of Behavior, 50, 505–519. Donny, E. C., Houtsmuller, E., & Stitzer, M. L. (2007). Smoking in the absence of nicotine: Behavioral, subjective, and physiological effects over 11 days. Addiction, 102, 324–334. Estes, W. K., & Skinner, B. F. (1941). Some quantitative properties of anxiety. Journal of Experimental Psychology, 29, 390–400. Farwell, B. J., & Ayres, J. J. B. (1979). Stimulusreinforcer and response-reinforcer relations in the control of conditioned appetitive headpoking (“goal tracking”) in rats. Learning and Motivation, 10, 295–312. Feldman, R. S., Meyer, J. S., & Quenzer, L. F. (1997). Principles of neuropsychopharmacology. Sunderland, MA: Sinauer Associates. Friend, K. B., & Pagano, M. E. (2005). Changes in cigarette consumption and drinking outcomes: Findings from Project MATCH. Journal of Substance Abuse Treatment, 29, 221–229. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Goeders, N. E., & Smith, J. E. (1983). Cortical dopaminergic involvement in cocaine reinforcement. Science, 221, 773–775. Greeley, J., Lê, D. A., Poulos, C. X., & Cappell, H. (1984). Alcohol is an effective cue in the conditional control of tolerance to alcohol. Psychopharmacology, 83, 159–162. Ghosheh, O., Dwoskin, L. P., Li, W-K., & Crooks, P. A. (1999). Residence times and halflives of nicotine metabolites in rat brain after acute peripheral administration of [2’-14C]nicotine. Drug Metabolism and Disposition, 27, 1448–1455. Hearst, E., & Jenkins, H. M. (1974). Sign-tracking: The stimulus-reinforcer relation and directed action. Austin, TX: The Psychonomic Society, Inc.

12-Schachtman-12.indd 287

287

Heffner, J. L., Barrett, S. W., & Anthenelli, R. M. (2007). Predicting alcohol misusers’ readiness and ability to quit smoking: A critical review. Alcohol and Alcoholism, 42, 186–195. Jaeger, T. V., & Mucha, R. F. (1990). A taste aversion model of drug discrimination learning: Training drug and condition inﬂuence rate of learning, sensitivity and drug speciﬁcity. Psychopharmacology, 100, 145–150. Kim, J. A., Siegel, S., & Patenall, V. R. A. (1999). Drug-onset cues as signals: Intra administration associations and tolerance. Journal of Experimental Psychology: Animal Behavior Processes, 25, 491–504. Koob, G. F. (2004). Allostatic view of motivation: Implications for psychopathology. In R. A. Bevins & M. T. Bardo (Eds.), Motivational factors in the etiology of drug abuse. The 50th Nebraska Symposium on Motivation (pp. 1–18). Lincoln, NE: University of Nebraska Press. Koob, G. F., & Le Moal, M. (2006). Neurobiology of addiction. New York, NY: Academic Press. Leeman, R. F., McKee, S. A., Toll, B. A., KrishnanSarin, S., Cooney, J. L., Makuch, R. W., & O’Malley, S. S. (2008). Risk factors for treatment failure in smokers: Relationship to alcohol use and to lifetime history of an alcohol use disorder. Nicotine and Tobacco Research, 10, 1793–1809. Li, M., He, W., & Mead, A. (2009). An investigation of the behavioral mechanisms of antipsychotic action using a drug-drug conditioning paradigm. Behavioural Pharmacology, 20, 184–194. Maes, J. H. R., & Vossen, J. M. H. (1997). Conditional control by midazolam and amphetamine in a rapid appetitive discrimination procedure. European Journal of Pharmacology, 319, 5–11. McDonald, R. V., & Siegel, S. (2004). Intraadministration associations and withdrawal symptoms: Morphine-elicited morphine withdrawal. Experimental and Clinical Psychopharmacology, 12, 3–11. Miller, R. R., & Oberling, P. (1998). Analogies between occasion setting and Pavlovian conditioning. In P. C. Holland & N. A. Schmajuk (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 3–35). Washington, DC: American Psychological Association. Murray, J. E., & Bevins, R. A. (2007a). Behavioral and neuropharmacological characterization of nicotine as a conditional stimulus. European Journal of Pharmacology, 561, 91–104. Murray, J. E., & Bevins, R. A. (2007b). The conditional stimulus effects of nicotine vary as a function of

2/16/2011 5:50:31 PM

288

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

training dose. Behavioural Pharmacology, 18, 707–716. Murray, J. E., & Bevins, R. A. (2009). Acquired appetitive responding to intravenous nicotine reﬂects a Pavlovian conditioned association. Behavioral Neuroscience, 123, 97–108. Murray, J. E., Li, C., Palmatier, M. I., & Bevins, R. A. (2007). The interoceptive Pavlovian stimulus effects of caffeine. Pharmacology, Biochemistry and Behavior, 86, 838–846. Murray, J. E., Penrod, R. D., & Bevins, R. A. (2009). Nicotine-evoked conditioned responding is dependent on concentration of sucrose unconditioned stimulus. Behavioural Processes, 81, 136–139. Murray, J. E., Wells, N. R., Lyford, G. D., & Bevins, R. A. (2009) Investigation of endocannabinoid modulation of conditioned responding evoked by a nicotine CS and the Pavlovian stimulus effects of CP 55,940 in adult male rats. Psychopharmacology, 205, 655–665. Niaura, R. S., Abrams, D. B., Shadel, W. G., Rohsenow, D. J., Monti, P. M., & Sirota, A. D. (1999). Cue exposure treatment for smoking relapse prevention: A controlled clinical trial. Addiction, 94, 685–695. O’Brien, C. P., Childress, A. R., McLellan, A. T., & Ehrman, R. (1992). A learning model for addiction. In C. P. O’Brien & J. H. Jaffe (Eds.), Addictive states (pp. 157–177). New York, NY: Raven Press. Overton, D. A., Shen, C. F., & Tatham, T. A. (1993). Centrally acting drugs act as conditioned stimuli in a conditioned suppression of drinking task. Psychopharmacology, 112, 270–276. Palmatier, M. I., & Bevins, R. A. (2007). Facilitation by drug states does not depend on acquired excitatory strength. Behavioural Brain Research, 176, 292–301. Palmatier, M. I., & Bevins, R. A. (2008). Occasion setting by drug states: Functional equivalence following similar training history. Behavioural Brain Research, 195, 260–270. Palmatier, M. I., Peterson, J. L., Wilkinson, J. L., & Bevins, R. A. (2004). Nicotine serves as a feature-positive modulator of Pavlovian appetitive conditioning in rats. Behavioural Pharmacology, 15, 183–194. Palmatier, M. I., Wilkinson, J. L., & Bevins, R. A. (2005). Stimulus properties of nicotine, amphetamine, and chlordiazepoxide as positive features in a Pavlovian appetitive discrimination task in rats. Neuropsychopharmacology, 30, 731–741.

12-Schachtman-12.indd 288

Panksepp, J. (1998). Affective neuroscience: The foundations of human and animal emotions. New York, NY: Oxford University Press. Parker, B. K., Schaal, D. W., & Miller, M. (1994). Drug discrimination using a Pavlovian conditional discrimination paradigm in pigeons. Pharmacology, Biochemistry, and Behavior, 49, 955–960. Pavlov, I. P. (1927). Conditioned reﬂexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). New York, NY: Dover Publications. Porter, J. H., & Prus, A. J. (2009). Drug discrimination: 30 years of progress. Psychopharmacology, 203, 189–191. Prendergast, M. A., Harris, B. R., Mayer, S., & Littleton, J. M. (2000). Chronic, but not acute, nicotine exposure attenuates ethanol withdrawal-induced hippocampal damage in vitro. Alcoholism, Clinical and Experimental Research, 24, 1583–1592. Rankin, H., Hodgson, R., & Stockwell, T. (1983). Cue exposure and response prevention with alcoholics: A controlled trial. Behaviour Research and Therapy, 21, 435–446. Reichel, C. M., Linkugel, J. D., & Bevins, R. A. (2007a). Nicotine as a conditioned stimulus: Impact of attention deﬁcit/hyperactivity disorder medication. Experimental and Clinical Psychopharmacology, 15, 501–509. Reichel, C. M., Wilkinson, J. L., & Bevins, R. A. (2007b). Methamphetamine functions as a positive and negative drug feature in a Pavlovian appetitive discrimination task. Behavioural Pharmacology, 18, 755–765. Reilly, S., & Revusky, S. (1992). Drug-drug heart rate conditioning in rats: Effective USs when pentobarbital is the CS. Pharmacology, Biochemistry and Behavior, 42, 633–643. Reilly, S., & Schachtman, T. R. (Eds.). (2009). Conditioned taste aversion: Behavioral and neural processes. New York, NY: Oxford University Press, Inc. Rescorla, R. A. (1969). Pavlovian conditioned inhibition. Psychological Bulletin, 72, 77–94. Revusky, S., Davey, V., & Zagorski, M. (1989). Heart rate conditioning with pentobarbital as a conditioned stimulus and amphetamine as an unconditioned stimulus. Behavioral Neuroscience, 103, 296–307. Revusky, S., & Reilly, S. (1990). When pentobarbital is the conditioned stimulus and amphetamine is the unconditioned stimulus, conditioning depends on the type of conditioned response. Behavioral Neuroscience, 104, 693–703.

2/16/2011 5:50:31 PM

INTERNAL STIMULI GENERATED BY ABUSED SUBSTANCES

Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: An incentivesensitization theory of addiction. Brain Research Reviews, 18, 247–291. Schmitz, J. A., Sayre, S. L., Hokanson, P. S., & Spiga, R. (2003). Assessment of the relative reinforcement value of smoking and drinking using a multiple-choice measurement strategy. Nicotine and Tobacco Research, 5, 729–734. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1–27. Siegel, S. (1984). Pavlovian conditioning and heroin overdose: Reports by overdose victims. Bulletin of the Psychonomic Society, 22, 428–430. Siegel, S., & Ellsworth, D. W. (1986). Pavlovian conditioning and death from apparent overdose of medically prescribed morphine: A case report. Bulletin of the Psychonomic Society, 24, 278–280. Siegel, S., Hinson, R. E., Krank, M. D., & McCully, J. (1982). Heroin “overdose” death: contribution of drug-associated environmental cues. Science, 216, 436–437. Siegel, S., & Ramos, B. M. C. (2002). Applying laboratory research: Drug anticipation and the treatment of drug addiction. Experimental and Clinical Psychopharmacology, 10, 162–183. Sirtharthan, T., Sitharthan, G., Hough, M. J., & Kavanagh, D. J. (1997). Cue exposure in moderation drinking: A comparison with cognitive-behavior therapy. Journal of Consulting and Clinical Psychology, 65, 878–882. Skinner, D. M., Goddard, M. J., & Holland, P. C. (1998). What can nontraditional features tell us about conditioning and occasion setting? In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 113–144). Washington, DC: American Psychological Association. Skinner, D. M., Martin, G. M., Howe, R. D., Pridgar, A., & van der Kooy, D. (1995). Drug discrimination learning using a taste aversion paradigm: An assessment of the role of safety cues. Learning and Motivation, 26, 343–369. Substance Abuse and Mental Health Services Administration. (2005). The DASIS report.

12-Schachtman-12.indd 289

289

Retrieved from http://www.oas.samhsa.gov/ 2k5/polydrugTX/polydrugTX.htm Swartzentruber, D. (1995). Modulatory mechanisms in Pavlovian conditioning. Animal Learning and Behavior, 23, 123–143. Trosi, J. R., & Akins, C. (2004). The discriminative stimulus effects of cocaine in a Pavlovian sexual approach paradigm in male Japanese quail. Experimental and Clinical Psychopharmacology, 12, 237–242. Turner, E. G., & Altshuler, H. L. (1976). Conditioned suppression of an operant response using d-amphetamine as the conditioned stimulus. Psychopharmacology, 50, 139–143. Wasserman, E. A., Franklin, S. R., & Hearst, E. (1974). Pavlovian appetitive contingencies and approach versus withdrawal to conditioned stimuli in pigeons. Journal of Comparative Physiology and Psychology, 86, 616–627. Wilkinson, J. L., & Bevins, R. A. (2008). Intravenous nicotine conditions a place preference in an unbiased design with rats. Pharmacology, Biochemistry and Behavior, 88, 256–264. Wilkinson, J. L., Li, C., & Bevins, R. A. (2008). Pavlovian drug discrimination with bupropion as a feature positive occasion setter: Substitution by methamphetamine and nicotine, but not cocaine. Addiction Biology, 14, 165–173. Wilkinson, J. L., Murray, J. E., Li, C., Wiltgen, S. M., Penrod, R. D., Berg, S. A., & Bevins, R. A. (2006). Interoceptive Pavlovian conditioning with nicotine as the conditional stimulus varies as function of number of conditioning trials and unpaired sucrose deliveries. Behavioural Pharmacology, 17, 161–172. Wise, R. A., & Leeb, K. (1993). Psychomotorstimulant sensitization: A unitary phenomenon? Behavioural Pharmacology, 4, 339–349. Wise, R. A., Wang, B., & You, Z.-B. (2008). Cocaine serves as a peripheral interoceptive conditioned stimulus for central glutamate and dopamine release. PLoS ONE, 3, e2846. You, Z-B., Wang, B., Zitzman, D., Azari, S., & Wise, R. A. (2007). A role for conditioned ventral tegmental glutamate release in cocaine-seeking. Journal of Neuroscience, 27, 10546–10555.

2/16/2011 5:50:31 PM

CHAPTER 13 Learning to Eat The Inﬂuence of Food Cues on What, When, and How Much We Eat Janet Polivy, C. Peter Herman, and Laura Girz

Humans must learn what to eat, when to eat, and how much to eat. We review the evidence that sensory food cues such as the sight and smell of food become conditioned stimuli for the foods, the unconditioned stimuli, with which they are paired. Research on children and adults supports the contention that various kinds of food cues signal what, when, and how much to eat on any given occasion, and that some types of individuals, namely obese people and chronically dieting restrained eaters, are more responsive to these cues. The super-abundance of food cues in our current environments may well be signaling us to eat more often and larger amounts than we need, and they may be contributing to increased obesity.

Eating seems like an innate, natural activity that shouldn’t require much learning. But beyond mother’s milk, humans must learn ﬁrst how to eat, and then, more important, what, when, and how much to eat. One factor that has been shown to have a broad inﬂuence on food intake is the presence of sensory food cues such as the sight or smell of food, which both signal that we can eat (i.e., that food is present) and tell us what to eat (Weingarten, 1985). These cues are associated with food or signal food. Therefore, these cues could be considered conditioned stimuli (CSs) and the foods that they have been paired with can be seen as unconditioned stimuli (USs). Some aspects of eating do not have to be learned. For example, infants have an innate preference for sweet and salty tastes, and they reject sour and bitter tastes (Beauchamp, Cowart, Manella, & Marsh, 1994). In addition, children are inclined to reject new foods (food neophobia), probably because consuming new and unfamiliar substances can be risky (Birch & Fisher, 1998). One thing that children are predisposed to

learn is the connection between the taste of foods (CSs) and the postingestive consequences of eating (USs). Thanks to this predisposition, food and ﬂavor preferences are learned from experience. Through associative conditioning, food cues related to these foods direct consumption (Birch, 1998). In this chapter, we discuss how food cues direct our consumption, telling us what, when, and how much we should eat on a given occasion.

WHAT WE EAT—HOW DO FOOD CUES HELP TO TEACH US WHAT TO EAT? One way that we learn what to eat is by being given new substances by our parents or caregivers. For example, Sullivan and Birch (1994) gave infants 10 opportunities to consume a particular vegetable; they found that the infants signiﬁcantly increased their intake of a vegetable to which they were repeatedly exposed and appeared (to their parents) to like the previously

290

13-Schachtman-13.indd 290

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

eaten vegetable more than an unfamiliar vegetable. So children’s food preferences are strongly inﬂuenced by their exposure to particular foods (Birch, 1998). Children (and adults too) learn to recognize foods that they have eaten (and enjoyed or disliked) before, and they react accordingly when encountering these foods. Exposure per se may lead to liking of (or preference for) initially neutral foods, but certain foods are inherently palatable (or unpalatable) and we react accordingly even on our ﬁrst exposure. We learn very quickly to eat foods that are palatable, that is, that taste good. Moreover, students given a taste of a palatable food (either ice cream or pizza) subsequently ate more of the food that they had tasted than did those who did not taste anything or who had tasted the alternative food (Cornell, Rodin, & Weingarten, 1989). Food cues thus induce desire for and consumption of the speciﬁc food that was cued rather than merely a general desire to eat. Other research has shown that access to well-liked foods increases preference for them and intake of them (Cottone, Sabino, Steardo, & Zorilla, 2008). We also learn what to eat by watching others eat. Young children aged 2–5 years ate more of a novel food if they saw an adult eating a food that looked like their (novel) food than if the adult either did not eat anything or ate a different colored food (Adessi, Galloway, Visalberghi, & Birch, 2005). Other people serve as models inﬂuencing what sorts of foods we eat, owing to our tendency to imitate people in our immediate environments (Birch, 1998). In particular, both children and adults are more likely to model their eating behavior on others who are familiar to them (e.g., Clendenen, Herman, & Polivy, 1994; Salvy, Vartanian, Coelho, Jarrina, & Pliner, 2008). Advertising is another inﬂuence on what people eat. For example, 49% of Saturdaymorning children’s-TV advertisements were for food, and 64% of these ads were for breakfast cereals/cereal bars, restaurants, or snack foods, and 91% were for high-fat, high-sodium, or high-sugar foods or beverages (Batada, Seitz, Wootan, & Story, 2008). A recent large-scale survey of middle and high school students found that those who watched the most television had

13-Schachtman-13.indd 291

291

the worst eating habits 5 years later, eating the fewest fruits and vegetables and the most trans fats, fast foods, high-fat snacks, and sugary beverages (Barr-Anderson, Larson, Nelson, & Neumark-Sztainer, 2009). The authors concluded that exposure to the barrage of advertising for unhealthy foods inﬂuenced the food choices of the young adults in the study, although we must keep in mind that these ﬁndings are merely correlational so other factors may also be operating. Fisher and Birch (1999) showed that restricting access to a particular (palatable) food had the seemingly paradoxical effect of increasing children’s tendency to choose and eat that food. The authors concluded that restricting access to certain foods makes children focus their attention more strongly on the “forbidden” foods, and it teaches them to want to obtain and consume these foods that are now more desirable. In fact, imposing stringent controls on children’s eating can increase their preferences for restricted highfat foods, and it can limit their willingness to consume a variety of foods (Birch & Fisher, 1998). On the other hand, children are also more likely to eat fruits and vegetables if they are made available and accessible at home or school (Hearn et al., 1998). That is, restricting access to preferred foods increases the preference for them, but exposure to less familiar foods also increases consumption of these. Even being exposed to a food cue as remote as lists of food words can affect people. Oakes and Slotterback (2000) found that showing students a list of food words that they were asked to rate for nutritional value increased their hunger, desire to eat, and the number of foods they wanted to eat compared to controls who did not read the list and rate the food words. Obesity and Responsiveness to Food Cues

Research by Schachter and his colleagues in the 1960s and 1970s demonstrated that obese individuals are more externally responsive than are the nonobese, and that they respond more strongly to food cues (see e.g., Schachter, 1971 for a review). For example, obese individuals drink more of a good-tasting milkshake and less

2/15/2011 7:46:13 PM

292

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of a bitter, quinine-laced milkshake than normal-weight participants. Rodin (1981) argued that people who are more externally responsive are particularly sensitive to food cues and have to exercise restraint to avoid eating and gaining weight whenever attractive food cues are present (because they are so much more likely to respond by eating the food that is cued). Herman, Olmsted, and Polivy (1983) exposed overweight and normal-weight diners in a restaurant to verbal food cues (an orally presented dessert menu), social food cues (the waitress recommended her “favorite” dessert), or visual food cues (the waitress carrying one of the desserts) and found that either seeing the food or having it recommended as a favorite or both increased ordering of that particular dessert by obese patrons, but it did not affect which desserts the nonobese patrons ordered. The food cues did not increase dessert ordering overall, but they did direct obese eaters to order the desserts that they saw or heard about rather than those simply listed on the menu. More recently, Braet and Crombez (2003) found that obese children are also hypersensitive to food cues. Restrained Eating and Responsiveness to Food Cues

The Fisher and Birch (1999) ﬁnding that restriction increased selection and eating of a food is reminiscent of the effects of the personality variable of chronic dietary restraint, or restrained eating. Research has shown that chronic dieters/ restrained eaters are more responsive to palatable food cues (Higgs, 2007; Kauffman, Herman, & Polivy, 1995). Kauffman et al. found a stronger taste by deprivation interaction for restrained eaters than for unrestrained eaters, although they did not ﬁnd a simple restraint by taste interaction. Higgs (2007) found that restrained eaters were more reactive to food cues in the sense that thinking about food impaired their cognitive performance, but there was no measure of food intake. When stressed or dysphoric, restrained and/or emotional eaters also appear to shift their food selection from meal-type foods, fruits, and vegetables to sweets, snacks, and bread (Oliver & Wardle, 1999) or to high-fat,

13-Schachtman-13.indd 292

high-calorie meals (Oliver, Wardle, & Gibson, 2000). Habhab, Sheldon, and Loeb (2009) found that restrained women preferred more sweet, high-fat food and ate more high-fat food than did unrestrained women, and even when they were not stressed. Even simply studying lists of food words makes restrained eaters think about eating (Boon, Stroebe, Schut, & Jansen, 1998). Mere exposure to palatable food cues makes restrained eaters feel hungry, salivate, and eat the food (Rogers & Hill, 1989), as well as crave the food more (Fedoroff, Polivy, & Herman, 1997). Moreover, the craving and increases in consumption are speciﬁc to the food cues, as was shown in the Cornell et al. (1989) study described earlier. When exposed to thoughts and smell of freshly baking pizza, (female) restrained eaters craved and ate more pizza, but not chocolatechip cookies. Conversely, when exposed to chocolate-chip-cookie cues, restrained eaters craved and consumed more chocolate-chip cookies, but not pizza (Fedoroff, Polivy, & Herman, 2003). Unrestrained eaters did not show a signiﬁcant response to the food cues (Fedoroff et al., 1997; 2003), even when the overall analysis indicated a main effect of food cues on eating (for all participants); the effect was actually carried by the heightened responsiveness of the restrained eaters to these food cues. People thus appear to learn to desire and eat foods that they have been exposed to and liked previously. Restrained eaters (chronic dieters) and obese individuals appear to be even more responsive than are normal, unrestrained eaters to these cues, thereby directing which food to eat on a given occasion. It has been argued that obese and restrained normal-weight individuals resemble each other behaviorally because they are both inhibiting their intake of preferred, high-calorie foods. The reason why these individuals are “externally bound” (Schachter, 1971) and thus hyperresponsive to food cues remains an intriguing question.

WHEN WE EAT—HOW DO FOOD CUES TEACH US WHEN TO EAT? Learning when to eat should be pretty simple: We eat when we are hungry and stop when we

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

are full, right? Food cues can increase the desire to eat and produce corresponding physiological effects. According to Powley (1977), the presence of food or food cues triggers cephalic-phase responses, which are a cascade of reactions within the body, including the secretion of saliva, gastric juices, pancreatic enzymes, and insulin. These processes prepare the body to ingest and digest food. Cephalic-phase responses may be produced by the sight and smell of food, by other conditioned food cues, or even by thoughts of food. Thus, it appears that we are programmed to respond to food cues by eating, as is reﬂected in the fact that food cues cause our bodies to prepare to digest the food (Woods, 1991), and this physiological preparation may be experienced as hunger that we try to assuage by eating (Weingarten, 1985). The timing of eating episodes, then, is actually a lot more complicated than merely responding to hunger and satiety. To begin with, many of us appear to learn to ignore hunger and satiety cues at an early age. Our parents may feed us either on a rigid schedule unconnected to our internal states of hunger or satiety, or feed us whenever we cry, or try to reduce our intake for (often misguided, in the case of infants and children) aesthetic reasons or health reasons, or otherwise attempt to control our eating without consulting us (e.g., Birch & Fisher, 1998; Bruch, 1973). As Birch and Fisher note, some well-meaning, concerned parents may think that children are not capable of knowing what, when, or how much they should eat. These parents therefore step in to control their children’s eating and do not allow the child himor herself to control consumption, putting the child completely at the mercy of external factors such as food cues to indicate when it is appropriate or possible or desirable to eat. In fact, a study by Birch, McPhee, Sullivan, and Johnson (1989) showed that learned cues can overcome satiety and induce further eating in preschool children. The children participated in seven conditioning trials, during which eating was conditioned using location, visual, and auditory cues. In the cues-present condition, the children were fed snacks in a room with music playing and a set of ﬂashing lights. The cuesabsent condition took place in a second room

13-Schachtman-13.indd 293

293

where the children did not eat and a different set of music and ﬂashing lights was presented (so the eating cues were absent). During two sets of test trials the children were taken to a third room where they were ﬁrst given enough ice cream to satiate them. Then they were presented with a snack while the conditioned food cues were either absent or present. When the conditioned cues were present, all 15 children ate immediately on the ﬁrst trial and 14 of the children ate immediately on the second trial. In contrast, when the conditioned cues were absent, only three children ate immediately on the ﬁrst trial and only one ate immediately on the second trial. These results clearly show that learned cues can initiate eating even in the absence of hunger. Self-Reported Reasons for Eating

So when do we eat, and if we are at the mercy of food cues, why do we think we are eating? Several questionnaire studies have examined the reasons that people give for deciding to eat a snack or meal. An inventory created from responses to the question, “What kinds of feelings, thoughts, or circumstances typically prompt you (or others) to eat or want to eat (besides simply feeling hungry/being hungry)?” found that the reasons cited could be combined into four broad categories (Jackson, Cooper, Mintz, & Albino, 2003). Coping motivations (e.g., “stress/depressed”) accounted for 48% of responses, social motivations (e.g., “as a social event”) accounted for 13%, compliance motivations (e.g., “someone saying ‘you have to try this’”) accounted for 4%, pleasure motivations (e.g., “love the taste of food”) accounted for 16%, and nonpsychological reasons (e.g., “keeping energy levels up”) accounted for 19%. It is important to note, however, that these percentages represent the proportion of people who selected each motivation, not the frequency with which these motivations (were perceived to) drive eating. Diary studies can better address the question of which motivations most often initiate eating. In a 7-day food-diary study, Adriaanse, de Ridder, and de Wit (2009) asked normal-weight female students to record their motivations for

2/15/2011 7:46:13 PM

294

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

each unhealthy snack eaten. “Having an appetite,” “enjoyment,” “feeling bored,” and “to be social” were the motivational cues cited most often. These motivations seem to ﬁt well within the inventory created by Jackson et al. (2003), with “enjoyment,” “feeling bored,” and “to be social” corresponding, respectively, to pleasure motivations, coping motivations, and social motivations. However, because these reasons were recorded only in response to eating unhealthy snacks, it is not clear whether similar motivations underlie eating more generally. A 24-hour diary study by Tuomisto, Tuomisto, Hetherington, and Lappalainen (1998) examined reasons for eating in response to a wider set of eating occasions. Obese participants completed self-monitoring diaries of food intake in which they chose among 26 reasons for eating and a ﬁll-in option. “It was mealtime” was the reason most often cited (32.7%), followed by “hunger” (20.9%), “regular lifestyle” (13.1%), and because the participant “fancied food” (7.8%). Although “fancying food” falls well within the category of pleasure motivations, the “mealtime” and “regular lifestyle” responses appear to represent a new category of motivations. It is not clear whether such motivations are more common among obese individuals or whether these reasons did not come up in the Adriaanse et al. study because only snack-eating behavior was reported by the normal-weight participants. It is probably worth noting that the reasons that people offer for their own food intake are not necessarily accurate. Vartanian, Herman, and Wansink (2008) asked people why they had eaten the way they did in various experimental situations. Although the experimentally manipulated variables (e.g., social inﬂuence) had a powerful effect on food intake, participants systematically failed to identify the true cause of why they ate more or ate less. Accordingly, we prefer to focus on manipulated (or at least measured) variables that trigger eating rather than to rely on the eater’s own perception of those triggers. Presentation (or Mere Presence) of Food

When we are presented with food, or food is present, does that automatically mean that we should eat? If we are hungry, the answer is easy;

13-Schachtman-13.indd 294

yes, we should eat. But what if we are not hungry? Cornell, Rodin, and Weingarten (1989) examined the reactions of hungry and sated individuals to the presentation of palatable foods. Participants in the satiety condition received a complete lunch and were told to eat until they felt full, whereas those in the hunger condition did not eat. Participants then tasted either pizza or ice cream, after having rated their desire to eat the target food. Next, they were informed that they could eat as much of the tasted food as they wanted. Despite the fact that the sated participants had eaten an average of 1,020 calories for lunch, they still consumed an additional 300 calories of the target food, with no differences found in the consumption of pizza versus ice cream. Although the intake of the sated participants was lower than that of the hungry participants, the sated participants consumed a signiﬁcant amount of food when full, indicating that presenting palatable food can induce eating in the absence of hunger. Fisher and Birch (2002) examined a group of girls at age 5 and then again at age 7. During both testing periods, the girls were fed a standard ad libitum lunch and reported whether they were “hungry,” “half-full,” or “full.” Only those who reported themselves to be full were included in the analyses. The girls were then asked to rate small samples of 10 snack foods. After this, they were told that the experimenter had to do some work in the room next door and that they should feel free to play with the toys provided or eat the snack foods. Despite being “full,” the girls ate an average of 125 calories at age 5 and an average of 169 calories at age 7. Moens and Braet (2007) examined the amounts eaten by normal-weight and overweight boys and girls aged 7–13 who had just ﬁnished a family meal. After eating the meal the children were left for 20 minutes in a room with toys and snacks and were told that they could play with the toys or eat the snacks. Two-thirds of the children ate snacks, and the average consumption was 68 grams. “Mealtime” as a Cue to Eat (or Not)

The “it was mealtime” reason for eating can either induce eating or prevent people from eating,

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

as when they are offered food just before a meal and don’t want to interfere with the meal. Schachter and Gross (1968) brought students to the laboratory at 5 pm and left them for 30 minutes with a clock that ran either fast or slow, so that they believed that it was either 5:15 (before dinnertime) or 6:00 (dinnertime). The participants were then left to ﬁll out questionnaires for 10 minutes, during which they were told to help themselves to crackers from a box left on the table with them. Obese participants ate more if they thought that it was mealtime than if they thought that it was earlier; normal-weight participants did the opposite, eating more if they thought that there was still time before dinner than if they thought that it was dinnertime, when they thought that eating in the lab would presumably ruin their dinner. In either case, students were clearly “told” by the clock/time cue whether they should eat the crackers. Conversely, when time cues are removed completely, meal timing appears to be at least partially determined by the size of the previous meal (Bernstein, Zimmerman, Czeisler, & Weitzman, 1981; Green, Pollak, & Smith, 1987). If you have just completed a meal, it is not time to eat again; a snack, however, does not interfere with eating a meal. Pliner and Zec (2007) showed that participants who ate a portion of food in three courses at a table with cutlery and napkins saw the food as a meal and ate less when food was presented soon after the “meal” than participants who ate the same food as 29 small portions while standing at a kitchen counter (and who saw it as a snack rather than a meal). Thus, once you have had a meal, it is no longer mealtime. Can thinking it is mealtime make sated individuals eat a meal? Rozin, Dow, Moscovitch, and Rajaram (1998) examined the willingness of individuals with amnesia to accept (i.e., eat) additional meals shortly after eating a ﬁrst meal. Ten to thirty minutes after completing lunch, two individuals with amnesia were presented with a second meal along with the phrase, “here’s lunch.” This procedure was then repeated with a third meal. One participant ate 81%–87% of each meal on each of the three testing sessions. The other ate the ﬁrst two meals and was

13-Schachtman-13.indd 295

295

prevented from ﬁnishing the third meal during two of the sessions, and ate his ﬁrst two meals and rejected the third in the other session. These results suggest that the presentation of a meal is a strong cue to eat, even when individuals are not hungry. Likewise, this study makes it clear that memory plays a role in decisions to eat; if you cannot remember having recently eaten, you are more likely to eat. Individuals asked to recall what they ate for lunch ate fewer snacks than did control participants who did not recall what they ate (Higgs, 2005), whereas manipulations (e.g., watching a video while eating) that disrupt the encoding of memories about the eating experience tend to increase subsequent snacking (Higgs, 2008).

HOW MUCH WE EAT—HOW DO FOOD CUES TEACH US HOW MUCH TO EAT? If food cues teach us not only what to eat, but even override hunger and satiety to tell us when to eat, how important are they in determining how much we eat on any given occasion? According to the research evidence, food cues are very important in determining how much we eat. The cue-reactivity model of overeating proposed by Jansen and colleagues (e.g., Jansen, et al., 2003) suggests that salient food cues cause individuals who are especially sensitive to the presence of these cues to eat a lot (more than if cues were not present), and it is this heightened eating that contributes to obesity. Moreover, as in the case of when to eat, food cues even overcome hunger and satiety in their inﬂuence on the quantities that we consume (e.g., Polivy, Herman, & Coelho, 2008). Sensory Food Cues

The power of food cues may well lie in their ability to evoke the pleasurable aspects of food by reminding us about the positive qualities of the food. Thus, the aroma (smell), attractive appearance, and most of all good taste of a food are all sensory food cues that are likely to induce us to eat more of a food exhibiting these properties. Possibly the most basic sensory food cue is the

2/15/2011 7:46:13 PM

296

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

food’s taste or palatability. We have already pointed out that we learn fairly quickly and easily to eat palatable foods. Bobroff and Kissileff (1986) point out that the amount people eat is directly related to how good-tasting or palatable the food is. Palatable or good-tasting foods also increase people’s self-reports of appetite and/or hunger, and tasting a little of a palatable food heightens one’s desire to eat that food (Yeomans, Blundell, & Leshem, 2004). Everyone eats more palatable than unpalatable foods. Palatable foods are generally higher in calories and thus evolutionarily desirable (Ulijaszek, 2002), so it is not surprising that we eat more of them, given that humans evolved in environments wherein food was not always available in unlimited quantities. It has been argued that the unpredictability of the food supply caused humans to evolve in such a way that we are essentially programmed to eat as much as we possibly can when food is readily available in order to store any excess food as fat so as to buffer us against later shortages (Pinel, Assanand, & Lehman, 2000). It is thus adaptive for humans to respond strongly to good-tasting foods and eat more of them (Polivy & Herman, 2006). For example, human infants fed sweetened baby formula ate more of it than of the normal formula (Nisbett & Gurwitz, 1970). This was the case for both sexes but especially for female babies. Adult French participants kept food diaries that included both palatability and amount of the foods that they ate for 4-week periods. Higher palatability was related to (a) larger amounts eaten at a given meal (which also tended to last longer) and (b) greater hunger (DeCastro, Bellisle, & Dalix, 2000). Similarly, in the laboratory, Bobroff and Kissileff’s participants drank 146% more of a highly palatable yogurt drink than of the same drink made unpalatable by adding cumin to it. Adding quinine to palatable ice cream reduced consumption from about 109 grams to only 42 grams (Woody, Costanza, Leifer, & Conger, 1981). In an experiment reported by Spiegel, Shrager, and Stellar (1989), people ate more than twice as much when they rated the food as palatable as they did when the food was deemed unpalatable. Restrained and unrestrained college

13-Schachtman-13.indd 296

students given either good-tasting, freshly-baked chocolate chip cookies or gray, salty, barely sweetened cookies (which grandma literally refused to bake for the experimenters) ate plenty of the good cookies but only minimal amounts of the bad ones (Polivy, Herman, & McFarlane, 1994). Intake by college students of a palatable, a less palatable, and an unpalatable soup was governed completely by the palatability of the soup, with consumption of the less and unpalatable soups being 65% and 40%, respectively, of the amount of palatable soup consumed (DeGraaf, DeJong, & Lambers, 1999). Wansink and Kim (2005) found that students ate more than 35% less popcorn if it was stale and unpalatable than if it was fresh and tasty. Sensory cues other than taste also affect food intake. Rogers and Hill (1989) showed that the exposure to the sight and smell of palatable (but not nonpreferred) food increases food intake in restrained eaters (chronic on-again-off-again dieters) but not in unrestrained eaters. Similarly, restrained eaters ate more after they smelled palatable foods than after no smell/food cues (Jansen & Van den Hout, 1991; Fedoroff et al., 1997; 2003). Finally, overweight children increased their food intake after either eating a small preload of appetizing food or after merely smelling palatable food (without eating it) (Jansen et al., 2003). Painter, Wansink, and Hieggelke (2002) placed containers of chocolate candy kisses in ofﬁces; the candy containers were positioned on the desk for 5 days (making the candies both visible and convenient), in a desk drawer for 5 days (so that they were convenient but not visible), and on a shelf two meters away from the desk for 5 days (so they were visible but not convenient). When the candies were visible and convenient, more candies were eaten; when the candies were more difﬁcult to get to, fewer candies were eaten. The visible and easily attainable food cues increased how much of the candy was eaten. The food cues that lead to eating can even be merely auditory descriptions of food. Rodin (1975) found that obese men who listened to 50-minute tapes that were either boring, interesting, or food-related drank signiﬁcantly more milkshake after listening to the food-related tape

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

than after listening to either the boring or interesting tape. This was not the case for normalweight men, who ate the same amount regardless of what tape they listened to. DeCastro’s (1990; DeCastro & Brewer, 1991) time-extension model of the social facilitation of eating posits that the reason people eat more when they eat in groups than when they eat alone is that they spend more time at the table (with food in front of them). They have found this ﬁnding repeatedly in food-diary studies. An experiment examining their model by comparing the effects of number of people eating together and the time spent eating showed that, in fact, in contrast to the de Castro model, it is increased time spent eating, rather than the number of people present, that leads to greater intake (Pliner, Bell, Hirsch, & Kinchla, 2006). The authors concluded that being in the presence of food (cues) causes people to eat, and the longer they spend with the food, the more of it they consume. Several of the studies reviewed earlier indicated that restrained eaters or obese individuals were either more responsive to the sensory food cues or were the only ones who responded. There is a fair amount of data suggesting that these types of people are generally more responsive to sensory food cues than are normal-weight or unrestrained eaters (Herman & Polivy, 2008). Normative Food Cues

We have recently distinguished between sensory and normative food cues as inﬂuences on eating behavior (Herman & Polivy, 2008). Normative cues are environmental stimuli that give an indication of what and/or how much to eat in a given situation, whereas sensory cues are properties of the food itself. Some normative cues, however, are themselves food cues, such as the portion size served or the number of pieces of food someone else is eating. So normative food cues indicate what or how much food it is appropriate to eat, whereas sensory cues are more likely to indicate what is “good.” There are a multitude of studies on social inﬂuences on how much people eat showing that social cues (such as the behavior of another eater, i.e., “modeling,” the presence of other eaters, i.e., “social facilitation,”

13-Schachtman-13.indd 297

297

and the use of amount eaten to convey information about the self, i.e., “impression formation”) have strong inﬂuences on the amounts that people eat (see Herman, Roth, & Polivy, 2003, for a review). In the current chapter, we will focus on normative cues that involve the actual presence of food, such as portion size. The effects of portion size on the amount people eat have received a fair amount of attention recently, and inﬂated portion sizes are being blamed for contributing to the “obesity epidemic” currently afﬂicting Western culture (e.g., Brownell & Horgen, 2004; Young & Nestle, 2002). The effects of portion size have been extensively studied in children and adults by Rolls, Birch, and colleagues (e.g., Rolls, Engell, & Birch, 2000; Rolls, Morris, & Roe, 2002). Children aged 3 were not responsive to portion size, eating the same whether the portion they were served was small, medium, or large; but 5-year-olds, who are presumably now old enough to respond to social/normative cues, ate signiﬁcantly more of the large than the small portion (and a moderate amount of the medium-sized portion) (Rolls et al., 2000). For adults, as portion sizes increase, so does the amount that people consume (regardless of weight or restrained eating status) (Rolls et al., 2002). DiLiberti, Bordi, Conklin, Roe, and Rolls (2004) manipulated portion sizes in cafeteria servings of pasta, and found that those served larger portions ate 43% more. Similarly, Levitsky and Youn (2004) gave students either the amount they had eaten at a baseline meal, or 25% or 50% more than baseline; as the amount served increased, so did the amount that the students ate. Wansink has also done several studies on the effects of portion size. The Wansink and Kim (2005) study described earlier using good- and bad-tasting popcorn included a manipulation of container size, which also yielded a main effect such that people ate more (of either the good- or bad-tasting popcorn) when it was served in a larger container. Wansink (2005) gave students access to snacks served in medium- or largesized containers and found that males and females served themselves more and ate more of the snacks when they were served in large bowls than when the same amount was presented in

2/15/2011 7:46:13 PM

298

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

twice as many smaller bowls. Perhaps the most creative portion-size study was Wansink, Painter, and North’s (2005; see Herman, 2005) “bottomless bowl” study, in which bowls of soup were rigged so that they imperceptibly reﬁlled themselves as participants ate from them. The visual food cues related to “portion size” had a powerful effect on how much people ate, with those exposed to accurate cues (the bowl became depleted as they ate) eating a normal amount while those eating from the self-reﬁlling bowls eating 73% more than those with normal bowls. Of even more interest is the fact that those who ate from the bottomless bowls did not report feeling any more full or sated than did those eating from normal bowls (and eating much less). Finally, the effects of portion size may be at least in part a reﬂection of what Geier and colleagues (2006) called “unit bias.” A unit (one candy bar, two cookies) of certain foods may be seen as the appropriate amount to eat, so that one will eat more when presented with a large Tootsie Roll bar than when presented with a smaller Tootsie Roll bar. The authors found that when serving themselves M&M candies with a large spoon, people took and ate more than if given a small spoon with which to serve themselves. In some ways this calls to mind the ﬁnding by Nisbett (1968) that when served one sandwich, obese males ate only the one sandwich despite the presence of a refrigerator full of extra sandwiches in the room with them, but when served three sandwiches, the obese males ate well over two sandwiches, on average. Normal-weight participants, however, ate about two sandwiches regardless of what was served to them, possibly because a “unit bias” says that lunch consists of two sandwiches for these individuals. Overall, then, the evidence thus strongly supports the notion that portion size serves as a food cue telling people how much they should eat in a given eating situation. Larger portions produce greater food intake than do smaller ones across a wide variety of eating situations, attesting to the strength of this cue. Variety of Foods Present

The variety or number of different foods present on an eating occasion has long been known to

13-Schachtman-13.indd 298

affect the amount eaten, and it is known as the “variety effect” (e.g., Raynor & Epstein, 2001; Remick, Polivy, & Pliner, 2009). Raynor and Epstein reviewed the literature and concluded that when there is more dietary variety within a meal, people eat more. For example, Rolls, Rolls, Rowe, and Sweeney (1981) found that participants who were given a single food for lunch ate more in an unexpected second course when presented with a new food as compared to when more of the same food was offered. Remick et al. extended this conclusion to encompass the variety of foods presented across more than one meal. We will not repeat the work of these two recent reviews by re-reviewing this extensive literature, but we will reiterate the conclusions of these reviews that a variety of foods serves as a cue that increases food consumption. Several authors have studied the limits of this phenomenon and tried to ﬁnd the underlying mechanism. Kahn and Wansink (2004) showed in a series of studies that the degree of variability that one perceives in a food assortment determines the extent of overeating. Raynor and Epstein (2001) posited that variety overcomes the sensory-speciﬁc satiety that usually reduces the palatability of a food as it is being eaten (the deﬁning feature of sensory-speciﬁc satiety being the reduction in palatability as a food is being eaten), leading people to stop eating. Normally, sensory-speciﬁc satiety develops in response to eating a particular food; its palatability declines as one eats and one’s intake slows; but when a new food is introduced, sensory-speciﬁc satiety is overcome by the novelty (and correspondingly high palatability) of the new food (e.g., Rolls et al., 1981). For example, Pliner, Polivy, Herman, and Zakalusny (1980) gave students tasty pizza rolls to eat until they reported being sated. They then brought out either more of the same food or a different kind of snack. The students presented with a different food (greater variety) ate much more than those given the same food again. Temple, Giacomelli, Roemmich, and Epstein (2008) studied a possible mechanism for how sensory-speciﬁc satiety is overcome by variety, namely habituation to food cues. They gave children either the same or different foods, measured

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

salivation in response to the same or different food cues, and then measured consumption of the foods. Participants who were shown the same food cues habituated faster to the food cues, as reﬂected by reduced salivation compared to those given a variety of cues. As in previous studies, the children given the variety of foods consumed more than did those given only one food. Memories of Food Eaten

Thinking about (i.e., remembering) what we have eaten clearly entails thinking about food. So remembering what we have eaten is a food cue. We discussed the effects of memory on eating when we talked about remembering when to eat and when we last ate. The Rozin et al. (1998) study, for example, showed that without any memory for having eaten, people ate whenever a meal was presented to them. This study also showed, however, that the mere presentation of food can make people eat, or eat more, even when they are not hungry. Similarly, as we discussed earlier, having a food preload that seems like a snack rather than a meal (because it is eaten standing up at a counter, out of a container rather than sitting at a table eating with utensils from a plate) increases how much people eat in a subsequent meal (Pliner & Zec, 2007). Conversely, remembering what one ate a few hours ago at lunch reduces the amount people eat in the laboratory later in the afternoon (Higgs, 2005). Mere memories of foods we have previously eaten act as food cues that affect current consumption. Disinhibition of Restrained Eating

Restrained eaters are chronic dieters who are trying to suppress their eating in order to lose weight or maintain a lowered weight. Unfortunately for them, however, many factors appear to disrupt their attempts at restraint and lead to disinhibited (over-)eating. For example, Herman and Mack (1975) found that after consuming either no preload, a preload of one milkshake, or a preload of two milkshakes and then being asked to taste and rate ice cream, restrained eaters behaved very differently than did unrestrained

13-Schachtman-13.indd 299

299

(nondieting) eaters. A strong restraint by condition interaction was found. Restrained eaters who consumed either one milkshake or two milkshakes ate signiﬁcantly more ice cream than did restrained eaters who did not consume a preload. On the other hand, unrestrained eaters in the no preload condition ate the most ice cream, whereas those in the two-milkshakepreload condition ate the least ice cream. Overall, it seems that restrained eaters’ eating became disinhibited after their diets were broken by a milkshake preload. Once disinhibited, they ate a large amount of ice cream regardless of the size of the preload. In contrast, the unrestrained eaters ate in a regulatory manner: the larger the preload, the smaller the amount of ice cream they subsequently ate. The self-control of restrained eaters over their eating is disrupted not only by actual highcalorie preloads, but by the mere belief that one has broken one’s diet (e.g., Polivy, 1976; Spencer & Fremouw, 1979), the expectation that one will have to overeat later in the day (Ruderman, Belzer, & Halperin, 1985), anticipating having to diet later (Urbszat, Herman, & Polivy, 2002), or even thinking that what one has eaten is “forbidden” (Knight & Boland, 1989). For example, Knight and Boland (1989) gave female participants either no preload, one chocolate milkshake, two chocolate milkshakes, one bowl of cottage cheese and fruit, or two bowls of cottage cheese and fruit (i.e., there were four conditions). Although each milkshake was equivalent in calories to a bowl of cottage cheese and fruit, the restrained eaters ate more calories in the milkshake conditions than in the cottage-cheese conditions. They also ate more than the unrestrained participants did in the milkshake conditions but not more calories than the unrestrained group ate in the cottage-cheese conditions. It seems that restrained eaters view their diets as intact even when relatively large portions of food are consumed, as long as the food is perceived as “healthy,” but as soon as they consider their diets to have been broken, they become disinhibited and eat more, rather than less. Emotional distress is another contributor to increased eating by restrained eaters. Distress generally reduces eating in normal-weight and

2/15/2011 7:46:13 PM

300

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

unrestrained eaters, but it tends to increase eating in overweight and restrained eaters (e.g., Herman & Polivy, 1975; Herman, Polivy, Lank, & Heatherton, 1987; Ruderman, 1985; Schachter, Goldman, & Gordon, 1968). However, the type of distress also appears to be important: Physical fear has been found to decrease eating in normalweight individuals and unrestrained eaters, but to have no signiﬁcant effect on the intake of overweight individuals or restrained eaters. Conversely, ego threats have been found to have little effect on the intake of normal-weight individuals and unrestrained eaters, but to increase eating in overweight individuals and restrained eaters (Heatherton, Herman, & Polivy, 1991). Overall, there are a variety of situations that can disinhibit restrained eaters’ eating and lead them to increase their food intake. Since unrestrained eaters are not attempting to restrict their intake, they are not susceptible to disinhibition. Restrained eaters thus seem to eat, and eat a lot, in response to a larger range of cues that serve as conditioned stimuli than do unrestrained eaters.

CONCLUSION Given the ubiquity of attractive foods and food cues in our environment, it is not surprising that we are suffering from an “obesity epidemic.” The attractive and abundant food cues surrounding us tell us that even if we are not hungry, we should eat these foods, eat them now, and eat a lot of them. From an evolutionary perspective, this does make sense. In an environment in which food availability ﬂuctuates, it is important to take advantage of the presence of food (as signaled by the various food cues that we have discussed) and eat while one can (e.g., Polivy & Herman, 2006). It has been further argued that to get through times of food scarcity, it is important to control one’s intake and eat less than one could conceivably consume, in order to husband one’s food supply through the times of scarcity (Polivy & Herman, 2006). The main mechanisms posited to promote selfcontrol involve changes in the palatability of foods such that foods that are being eaten (during the course of a meal) as well as foods that are

13-Schachtman-13.indd 300

eaten often become less palatable over the course of the meal (i.e., sensory-speciﬁc satiety) or over time as they are presented repeatedly (i.e., monotony). Thus, humans are programmed to eat in response to the palatability of food. So does this mean that now that we have developed reliable accessibility to palatable food, we are all doomed to become obese? Not necessarily. Remember, we are also adapted to exert self-control (although we do not seem to be as good at that as we are at eating when we sense the presence of food). Some hope is offered by a series of recent studies. Papies, Stroebe, and Aarts (2008) report two studies examining selfregulatory success in dieting. They point out that despite previous research indicating that restrained eaters (i.e., chronic dieters) respond (by eating) to attractive food cues that trigger cravings for food and interfere with their diets, in some dieters, these temptation cues may actually activate the diet goal and thus facilitate self-regulation (e.g., Fishbach, Friedman, & Kruglanski, 2003). Papies et al. found that experiencing self-regulatory success moderates the effect of food cues on restrained eaters such that attractive food cues serve to activate the diet goal in successful restrained eaters and interfere with dieting in unsuccessful restrained eaters. Thus, it is only the unsuccessful restrained eaters who overeat in response to food cues. The mechanism for this correlational ﬁnding is not clear, but this ﬁnding suggests that it may be possible to learn to overcome our mindless dependence on food cues that tell us what, when, and how much to eat, and learn to use food cues simply as cues that we may eat but not that we must.

REFERENCES Addessi, E., Galloway, A. T., Visalberghi, E., Birch, L. L. (2005). Speciﬁc social inﬂuences on the acceptance of novel foods in 2-5-year-old children. Appetite, 45, 264–271. Adriaanse, M. A., de Ridder, D. T. D., & de Wit, J. B. F. (2009). Finding the critical cue: Implementation intentions to change one’s diet work best when tailored to personally relevant reasons for unhealthy eating. Personality and Social Psychology Bulletin, 35, 60–71.

2/15/2011 7:46:13 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

Barr-Anderson, D. J., Larson, N. I., Nelson, M. C., & Neumark-Sztainer, D. (2009). Does television viewing predict dietary intake ﬁve years later in high school students and young adults? International Journal of Behavioral Nutrition and Physical Activity, 6, 1–8. Batada, A., Seitz, M. D., Wootan, M. G., & Story, M. (2008). Nine out of 10 food advertisements shown during Saturday morning children’s television programming are for foods high in fat, sodium, or added sugars, or low in nutrients. Journal of the American Dietetic Association, 108, 673–678. Beauchamp, G. K., Cowart, B. J., Menella, J. & Marsh, R. R. (1994) Infant salt taste: Developmental methodology and contextual factors. Developmental Psychobiology, 27, 353–365. Bernstein, I. L., Zimmerman, J. C., Czeisler, C. A., & Weitzman, E. D. (1981). Meal patterns in “free-running” humans. Physiology and Behavior, 27, 621–623. Birch, L. L. (1998). Psychological inﬂuences on childhood diet. The Journal of Nutrition, 128, 407S–410S. Birch, L. L., & Fisher, J. O. (1998). Development of eating behaviors among children and adolescents. Pediatrics, 101(Suppl.), 539–549. Birch, L. L., McPhee, L., Sullivan, S., & Johnson, S. (1989). Conditioned meal initiation in young children. Appetite, 13, 105–113. Bobroff, E. M., & Kissileff, H. R. (1986). Effects of changes in palatability on food intake and the cumulative food intake curve in man. Appetite, 7, 85–96. Boon, B., Stroebe, W., Schut, H., & Jansen, A. (1998). Food for thought: Cognitive regulation of food intake. British Journal of Health Psychology, 3, 27–40. Braet, C., & Crombez, G. (2003). Cognitive interference due to food cues in childhood obesity. Journal of Clinical Child and Adolescent Psychology, 32, 32–39. Brownell, K. D., & Horgen, K. B. (2004). Food ﬁght. New York, NY: McGraw Hill. Bruch, H. (1973). Eating disorders: Obesity, anorexia and the person within. New York, NY: Basic Books, Inc. Clendenen, V. L., Herman, C. P., & Polivy, J. (1994). Social facilitation of eating. An experimental study. Appetite, 23, 1–13. Cornell, C. E., Rodin, J., & Weingarten, H. (1989). Stimulus-induced eating when satiated. Physiology and Behavior, 45, 695–704.

13-Schachtman-13.indd 301

301

Cottone, P., Sabino, V., Steardo, L., & Zorilla, E. P. (2008). Intermittent access to preferred food reduces the reinforcing efﬁcacy of chow in rats. American Journal of Physiology-Regulatory Integrative and Comparative Physiology, 295, R1066–R1076. de Castro, J. M. (1990). Social facilitation of duration and size but not rate of the spontaneous meal intake of humans. Physiology and Behavior, 47, 1129–1135. de Castro, J. M., Bellisle, F., & Dalix, A. M. (2000). Palatability and intake relationships in freeliving humans: Measurement and characterization in the French. Physiology and Behavior, 68, 271–277. de Castro, J. M., & Brewer, E. M. (1991). The amount eaten in meals by humans is a power function of the number of people present. Physiology and Behavior, 51, 121–125. DeGraaf, C., DeJong, L. S., & Lambers, A. C. (1999). Palatability affects satiation but not satiety. Physiology and Behavior, 66, 681–688. DiLiberti, N., Bordi, P. L., Conklin, M. T., Roe, L. S., & Rolls, B. J. (2004). Increased portion size leads to increased energy intake in a restaurant meal. Obesity Research, 12, 562–568. Fedoroff, I., Polivy, J., & Herman, C. P. (1997). The effect of pre-exposure to food cues on the eating behavior of restrained and unrestrained eaters. Appetite, 28, 33–47. Fedoroff, I., Polivy, J., & Herman, C. P. (2003). The speciﬁcity of restrained versus unrestrained eaters’ responses to food cues: General desire to eat, or craving for the cued food? Appetite, 41, 7–13. Fishbach, A., Friedman, R. S., & Kruglanski, A. W. (2003). Leading us not into temptation: Momentary allurements elicit overriding goal activation. Journal of Personality and Social Psychology, 84, 296–309. Fisher, J. O., & Birch, L. L. (1999). Restricting access to palatable foods affects children’s behavioral response, food selection, and intake. American Journal of Clinical Nutrition, 69, 1264–1272. Fisher, J. O., & Birch, L. L. (2002). Eating in the absence of hunger and overweight in girls from 5 to 7 y of age. American Journal of Clinical Nutrition, 76, 226–231. Geier, A., Rozin, P., & Doros, G. (2006). Unit bias: A new heuristic that helps explain the effect of portion size on food intake. Psychological Science, 17, 521–525. Green, J., Pollak, C. P., & Smith, G. P. (1987). Meal size and intermeal interval in human subjects in

2/15/2011 7:46:14 PM

302

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

time isolation. Physiology and Behavior, 41, 141–147. Habhab, S., Sheldon, J. P., & Loeb, R. C. (2009). The relationship between stress, dietary restraint, and food preferences in women. Appetite, 52, 437–444. Hearn, M. D., Baranowski, T., Baranowski, J., Doyle, C., Smith, M., Lin, L. S., & Resnicow, K. (1998). Environmental inﬂuences on dietary behavior among children: Availability and accessibility of fruits and vegetables enable consumption. Journal of Health Education, 29, 26–32. Heatherton, T. F., Herman, C. P., & Polivy, J. (1991). Effects of physical threat and ego threat on eating behavior. Journal of Personality and Social Psychology, 60, 138–143. Herman, C. P. (2005). Lessons from the bottomless bowl. Obesity Research, 13, 2. Herman, C. P., & Mack, D. (1975). Restrained and unrestrained eating. Journal of Personality, 43, 647–660. Herman, C. P., Olmsted, M. P., & Polivy, J. (1983). Obesity, externality, and susceptibility to social inﬂuence: An integrated analysis. Journal of Personality and Social Psychology, 45, 926–934. Herman, C. P., & Polivy, J. (1975). Anxiety, restraint, and eating behavior. Journal of Abnormal Psychology, 84, 666–672. Herman, C. P., & Polivy, J. (2008). External cues in the control of food intake in humans: The sensory-normative distinction. Physiology & Behavior, 94, 722–728. Herman, C. P., Polivy, J., Lank, C. N., & Heatherton, T. F. (1987). Anxiety, hunger, and eating behavior. Journal of Abnormal Psychology, 96, 264–269. Herman, C. P., Roth, D., & Polivy, J. (2003). Effects of the presence of others on food intake: A normative interpretation. Psychological Bulletin, 129, 873–886. Higgs, S. (2005). Memory and its role in appetite regulation. Physiology and Behavior, 85, 67–72. Higgs, S. (2007). Impairment of cognitive performance in dietary restrained women when imagining eating is not affected by anticipated consumption. Eating Behaviors, 8, 157–161. Higgs, S. (2008). Cognitive inﬂuences on food intake: The effects of manipulating memory for recent eating. Physiology and Behavior, 94, 734–739. Jackson, B., Cooper, M. L., Mintz, L., & Albino, A. (2003). Motivations to eat: Scale development

13-Schachtman-13.indd 302

and validation. Journal of Research in Personality, 37, 297–318. Jansen, A., Theunissen, N., Slechten, K., Nederkoorn, C., Boon, B., Mulkens, S., & Roefs, A. (2003). Overweight children overeat after exposure to food cues. Eating Behaviors, 4, 197–209. Jansen, A. & Van den Hout, M. (1991). On being led into temptation: “Counterregulation” of dieters after smelling a preload. Addictive Behaviors, 16, 247–253. Kahn, B. E., & Wansink, B. (2004). The inﬂuence of assortment structure on perceived variety and consumption quantities. Journal of Consumer Research, 30(4), 519–533. Kauffman, N., Herman, C. P., & Polivy, J. (1995). Hunger induced ﬁnickiness in humans. Appetite, 24, 203–218. Knight, L. J., & Boland, F. J. (1989). Restrained eating: An experimental disentanglement of the disinhibiting variables of perceived calories and food type. Journal of Abnormal Psychology, 98, 412–420. Levitsky, D. A., & Youn, T. (2004). The more food young adults are served, the more they overeat. Journal of Nutrition, 134, 2546–2549. Moens, E., & Braet, C. (2007). Predictors of disinhibited eating in children with and without overweight. Behaviour Research and Therapy, 45, 1357–1368. Nisbett, R. E. (1968). Determinants of food intake in human obesity. Science, 159, 1254–1255. Nisbett, R. E., & Gurwitz, S. B. (1970). Weight, sex, and the eating behavior of human newborns. Journal of Comparative and Physiological Psychology, 73, 245–253. Oakes, M. E., & Slotterback, C. S. (2000). Selfreported measures of appetite in relation to verbal cues about many foods. Current Psychology, 19, 137–142. Oliver, G., & Wardle, J. (1999). Perceived effects of stress on food choice. Physiology and Behavior, 66, 511–515. Oliver, G., Wardle, J., & Gibson, E. L. (2000). Stress and food choice: A laboratory study. Psychosomatic Medicine, 62, 853–865. Painter, J. E., Wansink, B., & Hieggelke, J. B. (2002). How visibility and convenience inﬂuence candy consumption. Appetite, 38, 237–238. Papies, E. K., Stroebe, W., & Aarts, H. (2008). Healthy cognition: Processes of self-regulatory success in restrained eating. Personality and Social Psychology Bulletin, 34, 1290–1300.

2/15/2011 7:46:14 PM

LEARNING TO EAT: THE INFLUENCE OF FOOD CUES

Pinel, J. P. J., Assanand, S., & Lehman, D. R. (2000). Hunger, eating, and ill health. American Psychologist, 55, 1105–1116. Pliner, P., Bell, R., Hirsch, E. S., & Kinchla, M. (2006). Meal duration mediates the effect of “social facilitation” on eating in humans. Appetite, 46, 189–198. Pliner, P., Polivy, J., Herman, C. P., & Zakalusny, I. (1980). Short-term intake of overweight individuals and normal weight dieters and nondieters with and without choice among a variety of foods. Appetite, 1, 203–213. Pliner, P., & Zec, D. (2007). Meal schemas during a preload decrease subsequent eating. Appetite, 48, 278–288. Polivy, J. (1976). Perception of calories and regulation of intake in restrained and unrestrained subjects. Addictive Behavior, 1, 237–243. Polivy, J., & Herman, C. P. (2006). Restrained eating in modern society: An evolutionary perspective on dieting. Appetite, 47, 30–35. Polivy, J., Herman, C. P., & Coelho, J. (2008). Caloric restriction in the presence of attractive food cues: External cues, eating, and weight. Physiology and Behavior, 94, 729–733. Polivy, J., Herman, C. P., & McFarlane, T. (1994). Effects of anxiety on eating: Does palatability moderate distress-induced overeating in dieters? Journal of Abnormal Psychology, 103, 505–510. Powley, T. L. (1977). The ventromedial hypothalamic syndrome, satiety, and a cephalic phase hypothesis. Psychological Review, 84, 89–126. Raynor, H. A., & Epstein, L. H. (2001). Dietary variety, energy regulation, and obesity. Psychological Bulletin, 127(3), 325–341. Remick, A. K., Polivy, J., & Pliner, P. (2009). Internal and external moderators of the effect of variety on food intake. Psychological Bulletin, 135, 434–451. Rodin, J. (1975). Causes and consequences of time perception differences in overweight and normal weight people. Journal of Personality and Social Psychology, 31, 898–904. Rogers, P. J., & Hill, A. (1989). Breakdown of dietary restraint following mere exposure to food stimuli: Interrelationships between restraint, hunger, salivation and food intake. Addictive Behaviors, 14, 387–397. Rolls, B. J., Engell, D., & Birch, L. L. (2000). Serving portion size inﬂuences 5-year-old but not 3-year-old children’s food intakes. Journal of the American Dietetic Association, 100, 232–234.

13-Schachtman-13.indd 303

303

Rolls, B. J., Morris, E. L., & Roe, L. S. (2002). Portion size of food affects energy intake in normal-weight and overweight men and women. American Journal of Clinical Nutrition, 76, 1207–1213. Rolls, B. J., Rolls, E. T., Rowe, E. A., & Sweeney, K. (1981). Sensory speciﬁc satiety in man. Physiology and Behavior, 27, 137–142. Rozin, P., Dow, S., Moscovitch, M., & Rajaram, S. (1998). What causes humans to begin and end a meal? A role for memory for what has been eaten, as evidenced by a study of multiple meal eating in amnesiac patients. Psychological Science, 9, 392–396. Ruderman, A. J. (1985). Dysphoric mood and overeating: A test of restraint theory’s disinhibition hypothesis. Journal of Abnormal Psychology, 94, 78–85. Ruderman, A. J., Belzer, L. J., & Halperin, A. (1985). Restraint, anticipated consumption, and overeating. Journal of Abnormal Psychology, 94, 547–555. Salvy, S. J., Vartanian, L. R., Coelho, J., Jarrina, D., & Pliner, P. (2008). The role of familiarity on modeling of eating and food consumption in children. Appetite, 50, 514–518. Schachter, S. (1971). Some extraordinary facts about obese humans and rats. American Psychologist, 26, 129–144. Schachter, S., Goldman, R., & Gordon, A. (1968). Effects of fear, food deprivation, and obesity of eating. Journal of Personality and Social Psychology, 10, 91–97. Schachter, S., & Gross, L. P. (1968). Manipulated time and eating behavior. Journal of Personality and Social Psychology, 10, 98–106. Spencer, J. A., & Fremouw, W. J. (1979). Binge eating as a function of restraint and weight classiﬁcation. Journal of Abnormal Psychology, 88, 262–267. Spiegel, T. A., Shrager, E. E., & Stellar, E. (1989). Responses of lean and obese subjects to preloads, deprivation, and palatability. Appetite, 13, 45–69. Sullivan, S. A., & Birch, L. L. (1994). Infant dietary experience and acceptance of solid foods. Pediatrics, 93, 271–277. Temple, J. L., Giacomelli, A. M., Roemmich, J. N., & Epstein, L. H. (2008). Dietary variety impairs habituation in children. Health Psychology, 27, S10–S19. Tuomisto, T., Tuomisto, M. T., Hetherington, M., & Lappalainen, R. (1998). Reasons for initiation

2/15/2011 7:46:14 PM

304

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and cessation of eating in obese men and women and the affective consequences of eating in everyday situations. Appetite, 30, 211–222. Ulijaszek, S. J. (2002). Human eating behaviour in an evolutionary ecological context. Proceedings of the Nutrition Society, 61, 517–524. Urbszat, D., Herman, C. P., & Polivy, J. (2002). Eat, drink, and be merry, for tomorrow we diet: Effects of anticipated deprivation on food intake in restrained and unrestrained eaters. Journal of Abnormal Psychology, 111(2), 396–401. Vartanian, L. R., Herman, C. P., & Wansink, B. (2008). Are we aware of the external factors that inﬂuence our food intake? Health Psychology, 27, 533–538. Wansink, B. (2004). Environmental factors that increase the food intake and consumption volume of unknowing consumers. Annual Review of Nutrition, 24, 455–479. Wansink, B. (2005). Super bowls: Serving bowl size and food consumption. Journal of the American Medical Association, 293, 1727–1728. Wansink, B., & Kim, J. (2005). Bad popcorn in big buckets: Portion size can inﬂuence intake as

13-Schachtman-13.indd 304

much as taste. Journal of Nutrition Education and Behavior, 37, 242–245. Wansink, B., Painter, J. E., & North, J. (2005). Bottomless bowls: Why visual cues of portion size may inﬂuence intake. Obesity Research, 13, 93–100. Weingarten, H. P. (1985). Stimulus control of eating: Implications for a two-factor theory of hunger. Appetite, 6, 387–401. Woods, S. C. (1991). The eating paradox: How we tolerate food. Psychological Review, 98, 488–505. Woody, E. Z., Costanzo, P. R., Liefer, H., & Conger J. (1981). The effects of taste and caloric perceptions on the eating behavior of restrained and unrestrained subjects. Cognitive Therapy and Research, 5, 381–390. Yeomans, M. R., Blundell, J. E., & Leshem, M. (2004). Palatability: Response to nutritional need or need-free stimulation of appetite? British Journal of Nutrition, 92, S3–S14. Young, L. R., & Nestle, M. (2002). The contribution of expanding portion sizes to the US obesity epidemic. American Journal of Public Health, 92, 246–249.

2/15/2011 7:46:14 PM

CHAPTER 14 Conditional Analgesia, Negative Feedback, and Error Correction Moriel Zelikowsky and Michael S. Fanselow

Error correction has played an important role in the development of theories of Pavlovian conditioning, and it continues to be a central force driving research on the processes underlying learning and memory. This chapter highlights some of the key aspects of error correction—from the original groundwork that helped formulate the way we think about error correction to recent work looking at some of the physiological mechanisms that comprise error correction. In particular, we focus on the perceptualdefensive-recuperative and negative-feedback models for Pavlovian conditioning, as well as the role of conditional analgesia, attention, and dopamine in error-correction-based processes. We discuss speciﬁc applications of error-correction principles in conditioning, but we also try to stress a comprehensive role for error correction, particularly in the selection of appropriate brain circuits for speciﬁc functions.

INTRODUCTION The idea that we correct for the errors we make is as fundamental to behavior as the idea that we learn at all. As animals, it is in our nature to adapt, and integral to this is our ability to respond to our environment. All mammals must be able to continuously update the information they have stored and adjust their behaviors accordingly. This might be as simple as a squirrel learning to correct for the errors it makes jumping from branch to branch or as complicated as a man buying his wife ﬂowers to correct for forgetting the last time. Of great interest—though perhaps not what comes to mind when one intuitively thinks about error and correction—is the idea that the brain physiologically corrects for errors at the mechanistic or structural level. Research aimed at uncovering the processes involved in how the brain performs errorcorrection calculations has begun to push forward our ideas about error correction and establish it as

an extremely rich, deep phenomenon that seems to emerge in every corner of behavior. While recent, exciting discoveries have brought new attention to the ﬁeld of error correction (e.g., Fiorillo, Tobler, & Schultz, 2003; Tobler, Fiorillo, & Schultz, 2005), these studies are essentially the relaunching of a long-standing issue in learning theory: What processes support and limit what and how we learn? One powerful idea, ﬁrst formulated by Leon Kamin (1968, 1969), is that learning is driven by the surprisingness of a reinforcer. This notion of surprise has since been incorporated into a number of theories, several of which describe learning as being regulated by a form of error correction. Notably, the concept was captured more formally by Rescorla and Wagner (1972), who characterized changes in conditioning as a function of the difference between an obtained and an expected reinforcer. In this chapter, we will focus on particular components of the error-correction model as well as its development.

305

14-Schachtman-14.indd 305

2/15/2011 7:46:27 PM

306

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

We begin with a model of error correction— the perceptual-defensive-recuperative model (Bolles & Fanselow, 1980), which suggests that the notion of surprise could be conceived as error correction through negative feedback. The perceptual-defensive-recuperative model was the ﬁrst to suggest differential, competing roles for fear and pain, which function and interact in an error-correction-type fashion for the regulation of defensive behavior. In particular, we highlight the role of conditional analgesia as fundamental to the functioning of this model and future error-correction-based models of fear learning. We will focus on the idea that Pavlovian conditioning is regulated by a negative-feedback mechanism that allows for the calculation and correction of errors at the circuit level. Additionally, we touch upon the nature of errors that produce decrements in responding (“negative” errors), the role of attention in error correction, and recent research suggesting that midbrain dopamine neurons perform errorcorrection-type functions. We will close by suggesting a novel application of error-correction principles in the selection of particular brain circuits appropriate for speciﬁc types of learning.

THE PERCEPTUAL-DEFENSIVERECUPERATIVE MODEL All animals respond with a wide variety of behaviors in reaction to a traumatic event. For example, an animal may identify the presence of a predator, defend itself from attack, or perform recuperative behaviors if injured. The more successful an animal is in performing each of these behaviors, respectively, the more likely that animal is to survive. However, an animal’s success is largely determined by its ability to perform each of these behaviors at the appropriate point in time. For instance, licking a wound clean may be extremely beneﬁcial, but only if this behavior is performed after any immediate threat has passed. Indeed, while under attack, an animal must instead focus on defending itself rather than on mending its wounds. Therefore, the ability to select a diverse range of behaviors in response to a traumatic event is important, but the ability to select when it is appropriate to perform each behavior is crucial.

14-Schachtman-14.indd 306

The perceptual-defensive-recuperative (PDR) model (Bolles & Fanselow, 1980) explains the course of an animal’s behavior in a traumatic situation. At the core of this model is the distinction between fear and pain—a distinction that casts the two as opposing and competitive motivational systems, serving entirely different functions. According to the PDR model, fear, the emotion produced by stimuli that signal noxious events, activates defensive mechanisms, such as freezing and ﬂight behaviors. On the other hand, pain, the sensation produced by noxious stimulation, results in recuperative behaviors, such as resting or tending to an injury. According to this model, it is because fear and pain represent distinct motivational systems that they can generate completely different classes of behavior. By making this distinction, the PDR model utilized the emerging evidence on the existence of “antipain” mechanisms in the brain to explain the complexity and time course of animal behavior during a traumatic event. By 1980, the ﬁnding that pain-inhibiting peptides exist endogenously in the brain had already opened up an entire area of research on stress-induced analgesia (Cannon, Liebeskind, & Frenk, 1978; Sherman & Liebeskind, 1980). These opioid neuropeptides (e.g. endorphins, enkephalins) were found to act much like morphine in that they produce an analgesic state and effectively inhibit pain. At the time, the idea that pain mechanisms are highly adaptive and functionally useful was well established, however, the notion that an antipain system could be equally advantageous was quite novel. The potential advantages of these pain-inhibiting endorphins and enkephalins became the subject of much theory and research (Bolles & Fanselow, 1982). The PDR model offered an explanation for the functional purpose of having both a pain and an antipain system. According to the perceptual-defensiverecuperative model, pain and pain inhibition work hand in hand to produce adaptive behaviors in traumatic and peritraumatic situations. More speciﬁcally, the PDR model suggests that pain and pain-related behaviors (i.e., recuperation) are distinct and work in opposition to fear and fear-related behaviors (i.e., defense).

2/15/2011 7:46:27 PM

ERROR CORRECTION PROCESSES

307

The key to this distinction is that the model classiﬁes pain inhibition as a defensive behavior. That is, not only will an animal freeze, ﬁght, or ﬂee in defense of itself, but that animal will also inhibit any pain that it has incurred from being injured. Pain inhibition as a form of defense is important because it enables an animal to continue to actively defend itself while under attack. By inhibiting pain, an animal can perform defensive behaviors that would be otherwise impossible after a serious injury. Hence, the PDR model gives a functional signiﬁcance to endorphins: They enhance an animal’s ability to defend itself successfully. Thus, pain and pain inhibition are nothing more than the manifestation of painand fear-related behaviors. In its entirety, the PDR model distinguishes three stages of animal behavior in the face of a traumatic event: the perceptual phase, the defensive phase, and the recuperative phase (Fig. 14.1). In the ﬁrst phase, an animal perceives a threatening stimulus—a cue or environment that has come to predict the occurrence of a traumatic event. In Pavlovian fear-conditioning terms, the perceptual phase is when an animal encounters an initially neutral, conditional stimulus (CS), which it then learns signals the occurrence of an aversive, unconditional stimulus (US). Thus, the perceptual phase establishes the encoding of a CS-US relationship such that after conditioning, perception of the CS results in expectancy of the US. The emphasis on US expectancy is a vital component of the PDR model. In the second, defensive phase, this US expectancy activates the motive state of fear. Once activated,

DEFENSIVE SYSTEM

+

Threat-signaling conditional stimuli

the fear system motivates a host of defensive behaviors. It is important to note that it is not the US itself that activates a state of fear; rather, it is the expectancy of the US—elicited upon perception of the CS—that activates the fear state. This contrasts with the view that a state of fear is automatically triggered in direct response to a noxious stimulus. Indeed, the PDR model breaks with this latter notion of fear, instead classifying fear as a motivational state that is triggered by the occurrence of a CS that predicts a US, rather than by the US directly. This break is important because it shifts fear from being an automatic “reﬂexive” response to being an anticipatory central motive state that is responsible for the orchestration of a host of defensive behaviors. In addition, by proposing that fear and pain function in opposition, the PDR model contrasts with the previous two-factor theory (Miller, 1948, 1951; Mowrer, 1939, 1951) that viewed fear as the conditioned form of pain. Instead, the PDR model describes fear as having its own, distinct functional importance. More speciﬁcally, the PDR model states that fear functions by organizing an animal’s species-speciﬁc defense reactions (SSDRs), or the unique behaviors all members of a particular species will exhibit in their defense (Bolles, 1970). For example, freezing, the lack of all movement except that necessitated by respiration, is an SSDR rodents perform when afraid. Freezing reduces the likelihood that rodents will be detected and/or attacked by a predator. Importantly, all rodents will freeze despite the fact that the environmental circumstances signaling threat may be extremely

+

Species-specific defensive reactions

+ –

Nociceptive unconditional stimuli

+

Perception of nociceptive input

+

Recuperative behaviors

Figure 14.1 Diagram of the PDR model. (Adapted from Fanselow, M. S. (1986). Conditioned fearinduced opiate analgesia: A competing motivational state theory of stress analgesia. Annals of the New York Academy of Science, 467, 40–54).

14-Schachtman-14.indd 307

2/15/2011 7:46:27 PM

308

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

both across rodents and even within a single rodent’s experience. The reason why all members of a species uniformly perform a particular defense reaction is that any environment or cue that signals threat is able to activate the motive state of fear, which in turn will be able to generate a ﬁxed SSDR such as freezing. Thus, fear links and organizes environmental input with the appropriate behavioral output. Importantly, fear also produces an endogenous opioid-mediated state of analgesia, which is key to the inhibition of the perception of nociceptive input. The release of endorphins, much like freezing, is selected to occur through activation of the fear state. Thus, analgesia, like freezing, is simply part of the repertoire of defensive behavior and as such is not an automatic response but rather a conditional response. The idea that endogenous opioids inhibit pain so as to enhance defensive behaviors is supported by experimental work showing that along with defensive behavior, fear produces a loss of integrated painelicited responses (Fanselow & Baackes, 1982; Fanselow, Calcagnetti, & Helmstetter, 1989; Fanselow & Helmstetter, 1988). Thus, the second phase of the PDR model is primarily concerned with the central motive state of fear and in particular, the way in which fear organizes defensive behaviors and inhibits pain. This inhibition turns out to serve an important error correction purpose, which we will turn to shortly. The third, recuperative phase focuses on that which was inhibited during the defensive phase: pain. After the perception of threat has passed and fear has subsided, an animal shifts from defending itself to performing recuperative or healing behaviors to any injury it has sustained. An animal that is injured in a conﬂict will eventually shift from fear-related defensive behaviors to pain-related recuperative behaviors. Because the pain system receives input from noxious stimuli that cause tissue damage, an animal will begin to perform recuperative behaviors, unless the pain system is being inhibited by the fear system (i.e., unless the animal’s “perception” of pain is altered) (Fanselow & Baackes, 1982). For example, in the presence of more immediate threats to survival, the pain system may be inhibited through conditional analgesic.

14-Schachtman-14.indd 308

A chief role of conditional analgesia is to provide an inhibitory link from the defensive system onto the nociceptive system (Fanselow, 1986) (see Fig. 14.1). More speciﬁcally, fear-evoked analgesia serves to inhibit the “detection” of noxious stimuli, offering a clear advantage to an injured animal. This would be an example of defensive behavior taking precedence over recuperative behavior. The idea being that if an animal does not fully “feel” the pain inﬂicted by a predator, that animal has more of a chance of defending itself in that situation. Thus, our perception of pain is quite distinct from the actual noxious stimulus administered. This gap between what we “feel” and what is actually delivered turns out to be of great importance to error-correction models of traumatic events.

ANALGESIA Liebeskind and colleagues reported that brain stimulation of the periacqueductal gray (PAG) in rats resulted in a state of analgesia (Mayer, Wolﬂe, Akil, Carder, & Liebeskind, 1971). This ﬁnding was consistent with a similar observation made by Reynolds (1969). Liebeskind and colleagues noted many similarities between opiate and stimulation produced analgesia, notably cross-tolerance, which seemed to suggest that they were stimulating an endogenous opiate-like pathway. This, in part, led to a search for the opioid receptor (Pert & Snyder, 1973a; 1973b) and the natural opiate ligand (Kosterlitz & Hughes, 1975). Additionally, Liebeskind and colleagues discovered the existence of “stress-induced analgesia,” which similarly exhibited cross-tolerance with opiates as well as with PAG stimulation (Akil, Mayer, & Liebeskind, 1972b, 1972a, 1976; Mayer et al., 1971). This discovery led to research focused on the functional role of analgesia. The role of endorphins in eliciting an analgesic state was largely supported by studies looking at shock-induced analgesia. Notably, Lewis and colleagues (Lewis, Cannon, & Liebeskind, 1980; Lewis, Cannon, Stapleton, & Liebeskind, 1980; Lewis, Slater, Hall, Terman, & Liebeskind, 1982; Lewis, Tordoff, Sherman, & Liebeskind, 1982) established the existence of footshockinduced opioid analgesia. Though this endorphinmediated analgesic response was shown to be

2/15/2011 7:46:27 PM

ERROR CORRECTION PROCESSES

sensitive to training parameters (e.g., shocks had to be discontinuous), these studies helped establish the idea that the brain indeed has a mechanism by which stressful stimuli such as shock could be countered or dampened down by an opposing analgesia. Work by Maier et al. (1980) demonstrated that shock-induced analgesia was eliminated using the opioid antagonist naltrexone and the induction of analgesia was shown to partially depend on the uncontrollability of shock. Again, this work showed that it is not shock per se but a state associated with shock that engages the analgesic response. This research served to solidify the role of endorphins in the mediation of a footshock-triggered analgesic state. The ﬁrst suggestion that analgesia might play an important, functional role in fear conditioning came in 1978, when Chance et al. demonstrated that, like stress, conditional fear could also produce a state of analgesia (Chance, White, Krynock, & Rosencrans, 1978). Meanwhile, Fanselow and Bolles (1979b) reported that the opioid antagonist naloxone blocked the preference for signaled shock that rats normally show. Furthermore, Fanselow and Bolles (1979b) demonstrated that animals trained to fear contextual cues showed enhanced levels of fear at test if they had been conditioned in the presence of the opioid antagonist naloxone. In other words, by removing the pain-dampening effect of analgesia, naloxone made the shock a more effective US. This suggests that an analgesia signal, makes shock less aversive, which is consistent with the idea that conditional analgesia is a component of defensive behavior. Analgesia as a defensive mechanism was further demonstrated by studies looking at the ability of opioid antagonism to restore recuperative behaviors in the presence of fear (Fanselow & Baackes, 1982). Fanselow and Baackes (1982) examined formalin-induced recuperative behavior wherein a rat injected with a dilute formalin solution into its hind paw would subsequently lift and lick the paw (recuperative behavior). This recuperative behavior was suppressed if a fear-eliciting conditional stimulus was simultaneously presented (presumably due to conditional analgesia dampening down nociceptive

14-Schachtman-14.indd 309

309

input). However, if the opioid antagonist naltrexone was administered in addition to the presentation of the CS, formalin-induced recuperative behavior was restored. This ﬁnding solidiﬁed a role for conditional analgesia in the maintenance of “circa-strike” defensive behavior (Fanselow & Lester, 1988) and the regulation of recuperative behavior. Furthermore, it suggested a valuable theoretical implication for conditional analgesia, namely, that it provides an inhibitory link from the defensive system onto the recuperative system. Importantly, this laid the initial groundwork for a more general model of negative feedback.

NEGATIVE-FEEDBACK REGULATION OF PAVLOVIAN FEAR CONDITIONING In biology, the notion of negative feedback— that a structure may receive “negative” or opposing information from a source to which it ordinarily sends positive information—is most often used to describe how systems maintain homeostasis. One example of negative feedback can be seen in the role that conditional analgesia plays in fear conditioning. As described previously, fear conditioning a stimulus (CS) leads to the production of a number of conditional fear responses that may be elicited by that CS. The CS activates the motivational state of fear, which triggers the defensive system and a host of defensive behaviors. For example, a rodent presented with a fearful CS will both freeze and become analgesic. This conditional analgesia causes a reduction in the impact of nociceptive input, therein causing suppression of recuperative behaviors (see Fig. 14.2). A further consequence is the reduction in the ability for that rodent to subsequently condition fear. In other words, because an animal is analgesic, a US that would otherwise be “painful” no longer is. This reduction in conditioning is revealed by studies which show that conditional fear is enhanced by treatment with the opioid antagonist naloxone (Fanselow, 1981). Similarly, an animal will not condition fear to a CS that is paired with a US if that animal is already analgesic.

2/15/2011 7:46:27 PM

310

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS Light-CS

Registration of Light CS

Negative Feedback

Site of CS-US association formation

Conditional fear responses (e.g. freezing)

Footshock-US Registration of Footshock US

Figure 14.2 The negative-feedback model of fear conditioning. CS, conditioned stimulus; US, unconditioned

stimulus.

For this reason, after the ﬁrst CS-US conditioning trial, any subsequent exposure to that CS (including subsequent CS-US trials) results in the production of conditional analgesia (Fanselow & Bolles, 1979a, 1979b). In turn, this generates a graded reduction in the ability to condition that CS that is proportional to the amount of conditioning that has already accrued to the CS (e.g., Young & Fanselow, 1992). Importantly, the same reduced conditioning would apply to a novel stimulus being paired with the US if an animal is simultaneously exposed to a previously conditioned CS (i.e. “blocking,” Fanselow & Bolles, 1979b; McNally, Pigg, & Weidemann, 2004a). Thus, conditional analgesia reﬂects the amount of fear conditioning that has accrued to a CS (similar to any other measure of fearrelated conditional behavior, e.g., freezing). This information is then fed back to the very structures involved in detecting nociception, which results in an overall dampening of perceived nociceptive input and hence a reduction in subsequent conditioning. In other words, conditional analgesia provides a descending negative feedback onto the ascending reinforcing input responsible for the acquisition of fear (Fanselow, 1986, 1998) (see Fig. 14.2). This negative-feedback model of conditioning provides a physiological mechanism by which Rescorla-Wagner-type calculations can be made. For example, the negative-feedback model physiologically explains US-limited phenomena such as blocking, similar to how the

14-Schachtman-14.indd 310

Rescorla-Wagner model handles such phenomenon conceptually. In blocking (Kamin, 1968, 1969), conditioning of a CS, such as a light, is reduced if the light is presented in compound with another, previously conditioned CS, such as a tone. Prior conditioning of the tone blocks subsequent conditioning of the light. Kamin’s discovery of the phenomenon of “blocking” established that contiguity, or pairing of a CS and US, was not sufﬁcient to produce conditioning. Kamin concluded that what mattered was not contiguity, but instead the surprising aspect of the US. If the US was surprising, then conditioning of the CS would occur. Thus, the blocked light-CS fails to condition because the US is not surprising. Rescorla and Wagner took this notion of surprise further by developing a mathematical model of Pavlovian conditioning in which US processing played a central role (Rescorla & Wagner, 1972). This equation (∆V = αβ(λ – ΣV)) simply states that on a given conditioning trial, the change in the associative strength of a particular CS is equivalent to the amount of surprise on that trial—where surprise can be thought of as the difference between what you get (λ) minus what you expected to get (ΣV)—multiplied by the salience of the CS and the US. For our purposes, the term of interest here is the surprise term or prediction error (λ – ΣV). It is this term for which the negative-feedback model of fear conditioning offers a physiological mechanism. In the model, λ refers to the actual, physical intensity of the US, which gets registered

2/15/2011 7:46:27 PM

ERROR CORRECTION PROCESSES

14-Schachtman-14.indd 311

100

Tone Fear

Context Fear

75 % Freezing

by the central nervous system (e.g., the dorsal horn of the spinal cord), and ΣV refers to the amount of conditioning that has accrued to all stimuli present. We emphasize this pathway because it is the best documented and most thoroughly implicated (Basbaum & Fields, 1984). However, any conditioning-dependent response that mitigates the impact of the US would operate in this manner. Since there is already evidence for the conditioning of analgesia and subsequent reduction in the amount of nociception detected, conditional analgesia perfectly ﬁts as the physiological correlate to the ΣV term in the Rescorla-Wagner model. Thus, the magnitude of conditional analgesia (ΣV), which gets subtracted from the nociceptive value of an aversive US (λ) in the Rescorla-Wager model is in line with analgesia’s physiological role of providing negative feedback onto the area registering nociceptive input. Rescorla-Wagner’s λ – ΣV term stands for surprise, but it is also an error term. Surprise is nothing more than the prediction error made when you get something you did not expect to get. In the negative-feedback model, analgesia is the mechanism regulating such errors. For this reason, saying that an aversive US is fully expected is to simply say that the CS is fully conditioned, and the amount of conditional analgesia is sufﬁcient to cancel out the impact of the US. When analgesia is not sufﬁcient to cancel out US impact (i.e., λ > ΣV), there is error. This resulting error serves as the reinforcing signal. Thus, the greater the analgesic feedback, the smaller the error. The model can be tested by blocking endogenous opioids with antagonists (e.g. naloxone or naltrexone). Without endogenous opioidmediated conditional analgesia, the model predicts that ∆V = αβ(λ – ΣV) becomes ∆V = αβ(λ). There is extensive evidence for this prediction. For instance, a number of ﬁndings demonstrate that opioid antagonists such as naloxone attenuate blocking if they are administered in the second phase of a blocking experiment (Fanselow & Bolles, 1979a, 1979b; Galli et al., 2009; McNally et al., 2004a) (see Fig. 14.3). In addition, Young and Fanselow (1992) showed that administration of naloxone prior to conditioning results in

311

50

25

0 Forward

Backward Forward Training condition Saline

Backward

Naloxone

Figure 14.3 Naloxone causes unblocking in a

one-trial context-blocking experiment. During Phase 1, rats received either 15 forward or 15 backward pairings of a 30-sec tone and shock. In Phase 2, the rats were given an injection of saline or naloxone and placed in a novel context. There they received a single presentation of the tone followed by shock. The left panel (Tone Fear) shows that there was more conditioning to the forwardthan backward-paired tone and naloxone did not alter the expression of this fear. The right panel (Context Fear) shows fear conditioning to the context by the single shock. In saline-treated rats, the reduced context conditioning of the forwardtrained group relative to the backward-trained group indicates blocking. Naloxone prevented this blocking effect. (Adapted from Fanselow, M. S., & Bolles, R. C. (1979b). Triggering of the endorphin analgesic reaction by a cue previously associated with shock - reversal by naloxone. Bulletin of the Psychonomic Society, 14(2), 88–90).

increased conditioning asymptotes, thereby concluding that naloxone may function to lift the limits on the US’s ability to condition in a manner analogous to increasing the intensity of the actual shock itself. More recently, we have shown that the opioid antagonist naltrexone attenuates overshadowing—another Pavlovian phenomenon thought to occur due to limitations on the US’s ability to support conditioning (Zelikowsky & Fanselow, 2010). In overshadowing, a highly salient CS reduces conditioning to a concurrently presented low-salience CS (Pavlov, 1927),

2/15/2011 7:46:27 PM

312

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

again suggesting that the US is limited in the amount of conditioning it can support. However, if naltrexone is administered to an animal prior to training, overshadowing of the less salient CS is signiﬁcantly attenuated (Zelikowsky & Fanselow, 2010 ). The fact that naltrexone allows for the conditioning of the low-salience CSs gives further evidence that naltrexone may work to lift the limits off of the US’s ability to condition. According to the negative-feedback model, overshadowing, like blocking, occurs because conditional analgesia is elicited. The more salient CS will have a faster rate of acquisition, and hence rapidly generates a conditional analgesia that blocks conditioning to the less salient, slow to condition, CS.

DECREMENTAL ERROR CORRECTION Initial tests of the negative-feedback model speciﬁcally addressed error correction when the expectation is less than the received reinforcer (e.g., acquisition). In this case, the error term signals increments in associative strength. Another type of error correction is when the expectation is greater than the reinforcer— an error signal that leads to decrements in responding (e.g., extinction). A programmatic series of studies by McNally and colleagues (McNally, Pigg, & Weidemann, 2004b; McNally et al., 2004a) has shown that opioid antagonists also block this latter type of error correction. McNally and colleagues (2004a, 2004b) found that there are situations in which conditional analgesia exceeds the amount needed to fully cancel the reinforcing aspects of the US (i.e., when the prediction error, (λ – ΣV), is negative). In these cases, administration of an opioid antagonist blocks these “inhibitory” forms of learning such as extinction (McNally & Westbrook, 2003) and Pavlovian overexpectation (McNally et al., 2004a). In overexpectation, two CSs that have each been independently trained with a US are presented together and reinforced with the same size US such that what is expected is “double” what is actually received and hence λ – ΣV is negative. Similarly, in extinction, the CS is repeatedly presented in the

14-Schachtman-14.indd 312

absence of the US and hence what is expected is greater than what is received. These data ﬁt nicely with the negativefeedback model, if one assumes that there is some baseline level of activity (resting ﬁring rate of neurons) in ascending pain pathways under unstimulated conditions. The resting ﬁring rate would not produce changes in conditioning alone (i.e., would not support reinforcement). On the other hand, unpredicted painful events would increase ﬁring rate and support fear acquisition. However, if activity in the descending (analgesic) arm of the circuit was greater than needed to reduce painful input, the ﬁring rate in the pathway should drop below the resting rate (see Fig. 14.2). Such a condition would be met, for example, when a CS is presented without a US, as is the case in extinction. Instances in which the ﬁring rate slips below baseline would promote decreases in associative strength. Consequently, opioid antagonists that prevent analgesia would hinder such decrements in associative strength. This model is physiologically plausible because morphine not only suppresses pain-induced activity of dorsal horn neurons but also suppresses the spontaneous ﬁring rate of these neurons (Einspahr & Piercey, 1980). The application to fear conditioning is supported by the ﬁnding that naloxone—at least under some circumstances—can prevent extinction (McNally & Westbrook, 2003). The idea of a negative-feedback model of conditioning is extremely powerful in that it offers a physiological mechanism by which perception of the US changes as conditioning progresses in a manner analogous to that described so elegantly by the Rescorla-Wagner model. This model is further emboldened by the existence of anatomically independent negative-feedback loops in other forms of Pavlovian conditioning (e.g., eyeblink conditioning; Kim, Krupa, & Thompson, 1998).

ERROR CORRECTION AND ATTENTION While we have focused on error correction as conceived by US-processing models of conditioning, it should also be noted that there is a

2/15/2011 7:46:28 PM

ERROR CORRECTION PROCESSES

14-Schachtman-14.indd 313

Tone Fear 35 Aquisition rate score

large body of literature focused on the role of CS associability in conditioning (Mackintosh, 1975b; Pearce & Hall, 1980). While US processing models suggest that the error-correction signal is a reinforcement signal, CS processing models (e.g., Mackintosh and Pearce-Hall) suggest that error-correction signals adjust “associability,” which in turn has an effect on a constant reinforcement signal. In particular, CS-processing views of conditioning lay the success or failure of conditioning on the amount of attention the CS is or is not able to garner. Most often, the more attention paid to a CS, the more it can successfully be conditioned. Although attentional theories are not necessarily in agreement regarding the factor most likely to generate an attention-grabbing CS (i.e., the CS is a good predictor of a US, a novel predictor, or simply innately salient), they agree that conditioning depends on whether attention is paid to the CS. Thus, these theories explain phenomena such as overshadowing not in terms of US limitations, but in terms of properties of the CS. One notable advantage of associability models is that they are able to explain latent inhibition. In latent inhibition, a stimulus that has been preexposed is subsequently retarded in its ability to be conditioned (Lubow, 1973, 1989). This slower rate of acquisition can be accounted for using an associability model, which focuses on the CS and its salience, where CS pre-exposure serves to reduce the salience of a CS and hence the rate of acquisition. However, because latent inhibition occurs in the absence of a reinforcer, explaining it in terms of the negativefeedback model is problematic. Indeed, Young and Fanselow (1992) failed to block latent inhibition with an opioid antagonist (see Fig. 14.4). However, there are phenomena—one-trial blocking—that cannot be explained by associability models but can be accounted for by US-processing models. In one-trial blocking (Cole & McNally, 2007; Mackintosh, 1975a) one conditioning trial with a single stimulus is followed by a conditioning trial with a compound stimulus. The stimulus introduced in the compound is blocked. Since blocking consists of earlier training experience (the pre-compound

313

30 25 20 15 15 5 0 Latent inhibition Saline

Negative-transfer Naloxone

Figure 14.4 Naloxone blocks Hall-Pearce nega-

tive transfer, but not latent inhibition. During Phase 1, rats received either one exposure to a 64-sec tone only (latent inhibition groups) or one pairing of a 64-sec tone followed by a 1-sec lowintensity footshock (negative-transfer groups) per day over 10 days. In Phase 2, rats were given an injection of saline or naloxone and received a single tone presentation followed by a high-intensity footshock per day over 2 days. The graph displays difference scores for freezing to the tone on the ﬁrst versus second day of Phase 2. Low scores indicate the slow acquisition expected of latent inhibition and Hall-Pearce negative transfer. Saline groups showed latent inhibition and negative-transfer effects. However, naloxone prevented negative transfer but left latent inhibition intact. (Adapted from Young, S. L., & Fanselow, M. S. (1992). Associative regulation of Pavlovian fear conditioning: Unconditional stimulus intensity, incentive shifts, and latent inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 18(4), 400–413).

conditioning phase), a US-processing model, such as Rescorla-Wagner, can easily account for one-trial blocking. Indeed, one-trial blocking is prevented by the administration of an opioid antagonist (Cole & McNally, 2007; Fanselow & Bolles, 1979a). However, associability models, which depend on previous experience with the CS, cannot explain one-trial blocking. Thus, it is likely that changes in both US processing and changes in CS associability contribute to Pavlovian conditioning. For example, the slow rate of learning that follows after a CS has

2/15/2011 7:46:28 PM

314

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

been previously conditioned with a weak US (negative-transfer; Hall & Pearce, 1979) is blocked by naloxone (Young & Fanselow, 1992). These ﬁndings are summarized in Figure 14.4. Thus, by integrating CS-associability and US-processing views of conditioning, a wide breath of Pavlovian phenomena can be accounted for. Lastly, it should be noted that one-trial overshadowing cannot be accounted for by any of these models. One-trial overshadowing (Mackintosh, 1971) is a variant of the basic overshadowing effect; however the effect is achieved with a single conditioning trial of a compound CS. The occurrence of one-trial overshadowing is problematic for US-processing models such as the negativefeedback model because negative feedback is only generated after the ﬁrst conditioning trial. Similarly, associability models also require prior experience to drive interaction between stimuli. This suggests that initial competition between stimuli may be driven from a purely perceptual or attentional level. Thus, in addition to US-processing and CS-associability factors, raw attentional factors may also play an important role in Pavlovian conditioning. A number of studies have provided evidence for the role of dopamine in the regulation of attentional factors in Pavlovian phenomenon. In most of these studies, administration of a dopamine (DA) agonist often attenuates the Pavlovian phenomenon of interest. For example, amphetamine (which releases DA) has been shown to disrupt blocking (Crider, Solomon, & McMahon, 1982; Ohad, Lubow, Weiner, & Feldon, 1987) as well as overshadowing (O’Tuathaigh & Moran, 2002). Further studies have narrowed this effect down to the role of the DA D1 receptor in attentional processes, as the selective D1 agonist SKF 38393 attenuates overshadowing (O’Tuathaigh & Moran, 2002; Zelikowsky & Fanselow, 2010). Importantly, the indirect dopamine (DA) agonist D-amphetamine sulphate was shown to disrupt both blocking and overshadowing within a single study (O’Tuathaigh et al., 2003). In a separate task sensitive to attentional factors, Granon et al. (2000) showed that injecting SKF 38393 directly into the medial prefrontal cortex (mPFC) enhanced attentional performance in this task, suggesting

14-Schachtman-14.indd 314

that dopamine in the mPFC may play a role in the regulation of attentional processing in Pavlovian conditioning. An account of Pavlovian processes that attributes conditioning on the ﬁrst trial to attentional factors, subsequent trials to a negative-feedback mechanism, and previous CS exposure to an associability model, would have a good chance of encompassing many of the phenomenon that occur in Pavlovian conditioning. The fact that both the opioid antagonist naltrexone and the dopamine D1 agonist SKF 38393 attenuate Pavlovian overshadowing (albeit differently), suggests that multiple mechanisms may indeed contribute to the same Pavlovian phenomena (Zelikowsky & Fanselow, 2010). These multiple mechanisms may work hand in hand in a temporal fashion and/or may even mutually compensate for each other.

RECENT ADVANCES IN ERROR CORRECTION: DOPAMINE NEURONS While error correction–calculating circuits have been described for fear and eyeblink conditioning (Fanselow, 1998; Kim et al., 1998), recent work has suggested that in positive reinforcement learning, certain groups of neurons respond as though they detect mismatches between earned and expected rewards. More speciﬁcally, a number of studies from Schultz and collaborators have suggested that ﬁring of midbrain dopamine neurons operate according to error-correction-type rules in the regulation of reward learning (Fiorillo et al., 2003; Hollerman & Schultz, 1998; Schultz, 1997, 1998; Schultz, Dayan, & Montague, 1997; Tobler, Dickinson, & Schultz, 2003; Tobler et al., 2005; Waelti, Dickinson, & Schultz, 2001). These studies ﬁnd that burst activity of midbrain dopamine neurons—that is, the “phasic” dopamine response—can be seen following food or liquid rewards. However, if a reward is already predicted by a cue (i.e., a stimulus has been well conditioned to predict a food US), this burst activity does not occur, and if an expected reward is omitted, activity in these neurons is depressed (see Schultz, 2007 for a review).

2/15/2011 7:46:28 PM

ERROR CORRECTION PROCESSES

Schultz and colleagues interpret the behavior of these neurons as demonstrative of encoding the discrepancy between a predicted reward and the reward actually received. Thus, the output of these midbrain dopamine neurons seem to behave much like an error signal—with positive errors correlated to increased activity in these neurons and negative errors with depressed activity. In Rescorla-Wagner terminology, the response of these dopamine neurons is meant to represent the surprise term (λ – ΣV). Thus, Schultz and colleagues suggest that these dopamine neurons represent unexpected reinforcers and therefore act as a signal for reinforcement. This role is consistent with the long-standing view that dopamine acts as the brain’s reward system. It also implicates dopamine in the regulation of both prediction error and attention in Pavlovian conditioning. However, there are critical outstanding issues surrounding this view. First, unlike the more fully understood fear and motor learning systems, we do not know how these neurons actually calculate error. A second issue is that after conditioning, midbrain dopamine neurons will also react to a predictive CS with a phasic response. Thus, these neurons seem to both generate an expectancy type signal (RescorlaWagner’s V term) as well as an error signal (Rescorla-Wagner’s λ – V term). However, an expectancy signal drives your response based on what you have learned (V), whereas an error signal drives your learning based on how you have responded (λ – V). These are very different actions and require different computations. How are the neurons that receive these signals to discriminate between these two different meanings? A third issue, noted by Redgrave and Gurney (2006), lies in the fact that the occurrence of the phasic dopamine response has a very short latency (70–100 ms) from stimulus onset (Schultz, 1998). So short in fact, that it occurs during an animal’s “preattentive” processing phase—in other words, before the animal could actually identify a reward and/or its value. Thus, it becomes less clear what exactly these neurons contribute. In an alternative account, Redgrave and Gurney (2006) suggest that instead of signaling

14-Schachtman-14.indd 315

315

an unpredicted reward, the phasic dopamine response signals an animal to “reselect” an action that was immediately followed by an unpredicted biologically signiﬁcant event. According to this “reselection hypothesis,” the phasic dopamine response plays much more of a causal role. It allows an agent to recognize that a particular action it performed in a particular contextual backdrop preceded an unexpected biologically salient event and hence may be a probable cause. According to this hypothesis, an animal uses the phasic dopamine signal to differentiate between events for which it is responsible from events for which it is not, regardless of any immediate reward value (Redgrave & Gurney, 2006; Redgrave, Gurney, & Reynolds, 2008). This account is further supported by experiments from Winterbauer and Balleine (2007), showing that amphetamine enhances performance on a response (lever pressing) that was followed by the delivery of a simple visual stimulus. This solidiﬁes a role for dopamine in the reselection of a response, despite the absence of any reward contingency. Taken together, these reselection studies suggest that instead of signaling reward values, dopamine neurons may signal events that should be attended to, which dovetails nicely with our earlier discussion of the role of dopamine in selective attention. Certainly stimuli that you have learned about (V) and stimuli that signal surprise (λ – V) should be attended to. Thus, the actual proﬁle of responding of these neurons is more in line with an attentional view.

CIRCUIT SELECTION AND ERROR CORRECTION Thus far, we have described the manner by which particular circuits in the brain operate to calculate and correct for errors. We have also discussed the behavioral implications of error correction, namely that error-correction-type rules can be used to explain a wide range of Pavlovian phenomenon (e.g., overshadowing and blocking). We covered evidence consistent with a role for US limitations and negativefeedback circuits as well as the role of attention

2/15/2011 7:46:28 PM

316

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and dopamine in the regulation of these phenomena. However, we would like to take the notion of error correction one step farther. We propose that the very same error-correction rules that govern “stimulus selection” may also regulate how the brain selects circuits. The case of contextual fear memory is a particular poignant example of how such “circuit selection” may be occurring in the brain. In contextual fear conditioning, an animal learns to fear an environment in which it has received an aversive US (e.g., footshock). The memory of the context is initially stored in the hippocampus for a period of time, as lesions of the hippocampus immediately following contextual fear conditioning result in a complete loss of memory (Anagnostaras, Maren, & Fanselow, 1999; Kim & Fanselow, 1992). However, it has been shown that if damage to the hippocampus is sustained prior to training, animals are able to condition fear to a context (Frankland, Cestari, Filipkowski, McDonald, & Silva, 1998; Maren, Aharonov, & Fanselow, 1997; Wiltgen, Sanders, Anagnostaras, Sage, & Fanselow, 2006). Such data suggest that hippocampal damage produces retrograde amnesia but does not necessarily produce anterograde amnesia. It appears that although an animal may “normally” use its hippocampus to learn and store a representation of a place, in the absence of the hippocampus animals can compensate and form a representation of that place. Thus, when the primary, hippocampus-based circuit is compromised, an alternate circuit may be “selected” by the brain (Fanselow, 2010). However, this alternate circuit does not learn if the hippocampus is already engaged in learning. The interesting question remains as to the source and nature of this compensation. Retrograde amnesia studies tell us that the hippocampus— and not the alternate circuit—will normally form a conﬁgural representation of a place (see Fanselow, 2000). On the other hand, anterograde studies tell us that the alternate circuit may be utilized when the hippocampus is compromised. Just as the hippocampus has been shown to be important for context learning and memory, the basolateral amygdala (BLA) has been found

14-Schachtman-14.indd 316

to be vital for fear learning and memory (e.g., see Fanselow & LeDoux, 1999; Gale et al., 2004). Similar to the case with the hippocampus, a rodent with lesions or inactivation of the BLA may compensate and demonstrate fear learning and memory, provided that a strong regimen is used for training (Maren, 1999; Ponnusamy, Poulos, & Fanselow, 2007; Poulos et al., 2010). The same pattern holds for fear learning by subnuclei within the BLA complex (AngladaFigueroa & Quirk, 2005). Thus, fear learning and memory, like context learning and memory, seem to follow a similar pattern: A particular structure and circuit are normally used, but if they are damaged prior to—but not subsequent to—learning, then an alternate pathway may compensate. A remaining question is why the alternate circuit does not learn when the primary circuit is learning. One solution is that perhaps circuits, just like regular discrete cues, behave according to associative learning rules such as those that govern Pavlovian overshadowing. According to this idea, “salient” circuits would be selected for conditioning, while others would be overshadowed in a manner similar to discrete cues (see Fanselow, 2010). And, as is the case with an overshadowed discrete cue, an overshadowed circuit may be given the chance to learn if the limits on the amount of learning normally supported (i.e., λ) are removed or lifted. Thus, because an opioid antagonist such as naltrexone attenuates the overshadowing of a cue, by presumably lifting the limits on a US’s ability to condition (Zelikowsky & Fanselow, 2010), the same effect should be translatable to the selection of an “overshadowed” circuit. Taking the rules of associative learning and competition and applying them more broadly to circuit selection has powerful implications. Notably, it suggests that circuits, like discrete stimuli, can be learned about, despite being weaker or less salient, provided the amount of learning that is supported can be increased. This has important practical repercussions regarding patients suffering from some form of brain damage in which a primary pathway for learning and memory is compromised. It also has theoretical implications in that it suggests the depth and

2/15/2011 7:46:28 PM

ERROR CORRECTION PROCESSES

breadth of error-correction is quite wide. Error correction is not simply a mechanism for enabling ﬁne motor movements or predicting a reward; it forms the very basis and framework for how of our brains select appropriate circuits for speciﬁc types of learning.

CONCLUSIONS In this chapter, we have tried to present a picture of how error-correction processes can drive and mold the way we learn and behave. From the idea that a key component of defense is the successful inhibition of recuperation (PDR model), to more mechanistic notions of negative feedback and dopamine signaling, it seems that conditioning is driven incrementally by discrepancies between what actually happens in our environment and what we expect to happen. Whether this discrepancy is more sensitive to factors such as attention, environmental limitations on what can be learned, or what direction an error occurs (i.e., incremental vs. decremental), the basic idea remains the same: Our behavior is a result of what we expect about our environment compared to what we do not. In this chapter, we have emphasized particular mechanisms by which such errors may be calculated (e.g., analgesiamediated negative feedback or dopamine signaling and reward). Additionally, we suggest that error correction may in fact comprise a much more global mechanism. Namely, that the rules that underlie error correction are ubiquitous in the brain; they are used by speciﬁc brain circuits to perform particular functions, and they are also used by the brain overall to select circuits for more complex and integrated functions.

REFERENCES Akil, H., Mayer, D. J., & Liebeskind, J. C. (1972a). Comparison chez le rat entre l’analgesie induite par stimulation de la substance grise peri-aqueducale et l’analg&ie mor- phinique. [Comparison in the rat between analgesia induced by stimulation of periacqueducal gray matter and morphine analgesia]. Comptes Rendus Hebdomadaires des Seances de l Academie des Sciences D, 274(26), 3603–3605.

14-Schachtman-14.indd 317

317

Akil, H., Mayer, D. J., & Liebeskind, J. C. (1972b). [Monoaminergic mechanisms involved in analgesia caused by stimulation of the brain stem in rats]. Journal of Physiology (Paris), 65, Suppl:185A. Akil, H., Mayer, D. J., & Liebeskind, J. C. (1976). Antagonism of stimulation-produced analgesia by naloxone, a narcotic antagonist. Science, 191(4230), 961–962. Anagnostaras, S. G., Maren, S., & Fanselow, M. S. (1999). Temporally graded retrograde amnesia of contextual fear after hippocampal damage in rats: within-subjects examination. Journal of Neuroscience, 19(3), 1106–1114. Anglada-Figueroa, D., & Quirk, G. J. (2005). Lesions of the basal amygdala block expression of conditioned fear but not extinction. Journal of Neuroscience, 25(42), 9680–9685. Basbaum, A. I., & Fields, H. L. (1984). Endogenous pain control systems: Brainstem spinal pathways and endorphin circuitry. Annu Rev Neurosci, 7, 309–338. Bolles, R. C. (1970). Species-speciﬁc defence reactions and avoidance learning. Psychological Review, 77, 32–48. Bolles, R. C., & Fanselow, M. S. (1980). A perceptual-defensive-recuperative model of fear and pain. Behavioral and Brain Sciences, 3(2), 291–301. Bolles, R. C., & Fanselow, M. S. (1982). Endorphins and behavior. Annual Review of Psychology, 33, 87–101. Cannon, J. T., Liebeskind, J. C., & Frenk, H. (1978). Neural and neurochemical mechanisms of pain inhibition. In R. A. Sternbach (Ed.), The psychology of pain (pp. 27–47). New York, NY: Raven Press. Chance, W. T., White, A. C., Krynock, G. M., & Rosencrans, J. A. (1978). Conditional fear-induced antinociception and decreased binding of [3H]N-leu-enkephalin to rat brain. Brain Research, 141(2), 371–374. Cole, S., & McNally, G. P. (2007). Opioid receptors mediate direct predictive fear learning: Evidence from one-trial blocking. Learning and Memory, 14(4), 229–235. Crider, A., Solomon, P. R., & McMahon, M. A. (1982). Disruption of selective attention in the rat following chronic d-amphetamine administration: Relationship to schizophrenic attention disorder. Biological Psychiatry, 17(3), 351–361. Einspahr, F. J., & Piercey, M. F. (1980). Morphine depresses dorsal horn neuron responses to

2/15/2011 7:46:28 PM

318

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

controlled noxious and non-noxious cutaneous stimulation. Journal of Pharmacology and Experimental Therapeutics, 213(3), 456–461. Fanselow, M. S. (1981). Naloxone and Pavlovian fear conditioning. Learning and Motivation, 12, 398–419. Fanselow, M. S. (1986). Conditioned fear-induced opiate analgesia: A competing motivational state theory of stress analgesia. Annals of the New York Academy of Science, 467, 40–54. Fanselow, M. S. (1998). Pavlovian conditioning, negative feedback, and blocking: Mechanisms that regulate association formation. Neuron, 20(4), 625–627. Fanselow, M. S. (2000). Contextual fear, gestalt memories, and the hippocampus. Behavioural Brain Research, 110(1–2), 73–81. Fanselow, M. S. (2010). From contextual fear to a dynamic view of memory systems. Trends in Cognitive Sciences, 14, 7–15. Fanselow, M. S., & Baackes, M. P. (1982). Conditioned fear-induced opiate analgesia on the formalin test: Evidence for two aversive motivational systems. Learning and Motivation, 13, 200–221. Fanselow, M. S., & Bolles, R. C. (1979a). Naloxone and shock-elicited freezing in the rat. Journal of Comparative and Physiological Psychology, 93(4), 736–744. Fanselow, M. S., & Bolles, R. C. (1979b). Triggering of the endorphin analgesic reaction by a cue previously associated with shock - reversal by naloxone. Bulletin of the Psychonomic Society, 14(2), 88–90. Fanselow, M. S., Calcagnetti, D. J., & Helmstetter, F. J. (1989). Role of mu and kappa opioid receptors in conditional fear-induced analgesia: The antagonistic actions of nor-binaltorphimine and the cyclic somatostatin octapeptide, Cys2Tyr3Orn5Pen7-amide. Journal of Pharmacology and Experimental Therapeutics, 250(3), 825–830. Fanselow, M. S., & Helmstetter, F. J. (1988). Conditional analgesia, defensive freezing, and benzodiazepines. Behavioral Neuroscience, 102(2), 233–243. Fanselow, M. S., & LeDoux, J. E. (1999). Why we think plasticity underlying Pavlovian fear conditioning occurs in the basolateral amygdala. Neuron, 23(2), 229–232. Fanselow, M. S., & Lester, L. S. (1988). A functional behavioristic approach to aversively motivated behavior: Predatory imminence as a determinant

14-Schachtman-14.indd 318

of the topography of defensive behavior. In R. C. Bolles & M. D. Beecher (Eds.), Evolution and learning (pp. 185–212). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299(5614), 1898–1902. Frankland, P. W., Cestari, V., Filipkowski, R. K., McDonald, R. J., & Silva, A. J. (1998). The dorsal hippocampus is essential for context discrimination but not for contextual conditioning. Behavioral Neuroscience, 112(4), 863–874. Gale, G. D., Anagnostaras, S. G., Godsil, B. P., Mitchell, S., Nozawa, T., Sage, J. R., . . . Fanselow, M. S. (2004). Role of the basolateral amygdala in the storage of fear memories across the adult lifetime of rats. Journal of Neuroscience, 24(15), 3810–3815. Galli, G., Hancock, K., Hoschler, K., DeVos, J., Praus, M., Bardelli, M., . . .Stephenson, I. (2009). Fast rise of broadly cross-reactive antibodies after boosting long-lived human memory B cells primed by an MF59 adjuvanted prepandemic vaccine. Proceedings of the National Academy of Sciences USA, 106(19), 7962–7967. Granon, S., Passetti, F., Thomas, K. L., Dalley, J. W., Everitt, B. J., & Robbins, T. W. (2000). Enhanced and impaired attentional performance after infusion of D1 dopaminergic receptor agents into rat prefrontal cortex. Journal of Neuroscience, 20(3), 1208–1215. Hall, G., & Pearce, J. M. (1979). Latent inhibition of a CS during CS-US pairings. Journal of Experimental Psychology: Animal Behavior Processes, 5(1), 31–42. Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1(4), 304–309. Kamin, L. J. (1968). “Attention-like” processes in classical conditioning. In M. R. Jones (Ed.), Miami Symposium on the Prediction of Behavior: Aversive stimulation (pp. 9–33). Miami, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. C. R. M. Church (Ed.), Punishment and aversive behavior (pp. 9–31). New York, NY: Appleton-Century-Crofts. Kim, J. J., & Fanselow, M. S. (1992). Modalityspeciﬁc retrograde amnesia of fear. Science, 256(5057), 675–677.

2/15/2011 7:46:28 PM

ERROR CORRECTION PROCESSES

Kim, J. J., Krupa, D. J., & Thompson, R. F. (1998). Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science, 279(5350), 570–573. Kosterlitz, H. W., & Hughes, J. (1975). Some thoughts on the signiﬁcance of enkephalin, the endogenous ligand. Life Sciences, 17(1), 91–96. Lewis, J. W., Cannon, J. T., & Liebeskind, J. C. (1980). Opioid and nonopioid mechanisms of stress analgesia. Science, 208(4444), 623–625. Lewis, J. W., Cannon, J. T., Stapleton, J. M., & Liebeskind, J. C. (1980). Stress activates endogenous pain-inhibitory systems: Opioid and non-opioid mechanisms. Proceedings of the Western Pharmacology Society, 23, 85–88. Lewis, J. W., Slater, S. J., Hall, J. L., Terman, G. W., & Liebeskind, J. C. (1982). Chronic stress enhances morphine analgesia in the rat. Proceedings of the Western Pharmacology Society, 25, 137–140. Lewis, J. W., Tordoff, M. G., Sherman, J. E., & Liebeskind, J. C. (1982). Adrenal medullary enkephalin-like peptides may mediate opioid stress analgesia. Science, 217(4559), 557–559. Lubow, R. E. (1973). Latent inhibition. Psychology Bulletin, 79(6), 398–407. Lubow, R. E. (1989). Latent inhibition and conditioned attention theory. Cambridge, England: Cambridge University Press. Mackintosh, N. J. (1971). An analysis of overshadowing and blocking. Quarterly Journal of Experimental Psychology, 23, 118–125. Mackintosh, N. J. (1975a). Blocking of conditioned suppression: Role of the ﬁrst compound trial. Journal of Experimental Psychology: Animal Behavior Processes, 1(4), 335–345. Mackintosh, N. J. (1975b). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298. Maren, S. (1999). Neurotoxic basolateral amygdala lesions impair learning and memory but not the performance of conditional fear in rats. Journal of Neuroscience, 19(19), 8696–8703. Maren, S., Aharonov, G., & Fanselow, M. S. (1997). Neurotoxic lesions of the dorsal hippocampus and Pavlovian fear conditioning in rats. Behavioural Brain Research, 88(2), 261–274. Mayer, D. J., Wolﬂe, T. L., Akil, H., Carder, B., & Liebeskind, J. C. (1971). Analgesia from electrical stimulation in the brainstem of the rat. Science, 174(16), 1351–1354. McNally, G. P., Johnston, I. N., & Westbrook, R. F. (2000). A peripheral, intracerebral, or intrathecal

14-Schachtman-14.indd 319

319

administration of an opioid receptor antagonist blocks illness-induced hyperalgesia in the rat. Behavioral Neuroscience, 114(6), 1183–1190. McNally, G. P., Pigg, M., & Weidemann, G. (2004a). Blocking, unblocking, and overexpectation of fear: A role for opioid receptors in the regulation of Pavlovian association formation. Behavioral Neuroscience, 118(1), 111–120. McNally, G. P., Pigg, M., & Weidemann, G. (2004b). Opioid receptors in the midbrain periaqueductal gray regulate extinction of pavlovian fear conditioning. Journal of Neuroscience, 24(31), 6912–6919. McNally, G. P., & Westbrook, R. F. (2003). Opioid receptors regulate the extinction of Pavlovian fear conditioning. Behavioral Neuroscience, 117(6), 1292–1301. Miller, N. E. (1948). Studies of fear as an acquirable drive .1. Fear as motivation and fear-reduction as reinforcement in the learning of new responses. Journal of Experimental Psychology, 38(1), 89–101. Miller, N. E. (1951). Learnable drives and rewards. In S. S. Stevens (Ed.). Handbook of experimental psychology, 435–472. New York: John Wiley & Sons. Mowrer, O. H. (1939). A stimulus-response analysis of anxiety and its role as a reinforcing agent. Psychological Review, 46(6), 553–565. Mowrer, O. H. (1951). Two-factor learning theory: Summary and comment. Psychological Review, 58(5), 350–354. O’Tuathaigh, C. M., & Moran, P. M. (2002). Evidence for dopamine D(1) receptor involvement in the stimulus selection task: Overshadowing in the rat. Psychopharmacology (Berl), 162(3), 225–231. O’Tuathaigh, C. M., Salum, C., Young, A. M., Pickering, A. D., Joseph, M. H., & Moran, P. M. (2003). The effect of amphetamine on Kamin blocking and overshadowing. Behavioural Pharmacology, 14(4), 315–322. Ohad, D., Lubow, R. E., Weiner, I., & Feldon, J. (1987). The effects of amphetamine on blocking. Psychobiology, 15, 137–143. Pavlov, I. P. (1927). Conditioned reﬂexes (G. V. Anrep, Trans.). London: Oxford University Press. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87(6), 532–552.

2/15/2011 7:46:28 PM

320

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Pert, C. B., & Snyder, S. H. (1973a). Opiate receptor: Demonstration in nervous tissue. Science, 179(77), 1011–1014. Pert, C. B., & Snyder, S. H. (1973b). Properties of opiate-receptor binding in rat brain. Proceedings of the National Academy of Sciences USA, 70(8), 2243–2247. Ponnusamy, R., Poulos, A. M., & Fanselow, M. S. (2007). Amygdala-dependent and amygdalaindependent pathways for contextual fear conditioning. Neuroscience, 147(4), 919–927. Poulos, A. M., Ponnusamy, R., Dong, H. W., & Fanselow, M. S. (2010). Compensation in the neural circuitry of fear conditioning awakens learning circuits in the bed nuclei of the stria terminalis. Proc Natl Acad Sci U S A, 107(33), 14881–14886. Redgrave, P., & Gurney, K. (2006). The shortlatency dopamine signal: A role in discovering novel actions? Nature Reviews Neuroscience, 7(12), 967–975. Redgrave, P., Gurney, K., & Reynolds, J. (2008). What is reinforced by phasic dopamine signals? Brain Research Review, 58(2), 322–339. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In W. F. Prokasy (Ed.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton-Century-Crofts. Reynolds, D. V. (1969). Surgery in the rat during electrical analgesia induced by focal brain stimulation. Science, 164(878), 444–445. Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Current Opinions in Neurobiology, 7(2), 191–197. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1–27. Schultz, W. (2007). Behavioral dopamine signals. Trends in Neuroscience, 30(5), 203–210. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599. Sherman, J. E., & Liebeskind, J. C. (1980). An endorphinergic, centrifugal substrate of pain

14-Schachtman-14.indd 320

modulation: Recent ﬁndings, current concepts, and complexities. Research Publications Association for Research in Nervous and Mental Disease, 58, 191–204. Tobler, P. N., Dickinson, A., & Schultz, W. (2003). Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. Journal of Neuroscience, 23(32), 10402–10410. Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005). Adaptive coding of reward value by dopamine neurons. Science, 307(5715), 1642–1645. Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412(6842), 43–48. Wiertelak, E. P., Roemer, B., Maier, S. F., & Watkins, L. R. (1997). Comparison of the effects of nucleus tractus solitarius and ventral medial medulla lesions on illness-induced and subcutaneous formalin-induced hyperalgesias. Brain Research, 748(1–2), 143–150. Wieseler-Frank, J., Maier, S. F., & Watkins, L. R. (2005). Immune-to-brain communication dynamically modulates pain: Physiological and pathological consequences. Brain, Behavior, and Immunity, 19(2), 104–111. Wiltgen, B. J., Sanders, M. J., Anagnostaras, S. G., Sage, J. R., & Fanselow, M. S. (2006). Context fear learning in the absence of the hippocampus. Journal of Neuroscience, 26(20), 5484–5491. Winterbauer, N. E., & Balleine, B. W. (2007). The inﬂuence of amphetamine on sensory and conditioned reinforcement: Evidence for the re-selection hypothesis of dopamine function. Front Integr Neurosci, 1, 9. Young, S. L., & Fanselow, M. S. (1992). Associative regulation of Pavlovian fear conditioning: Unconditional stimulus intensity, incentive shifts, and latent inhibition. Journal of Experimental Psychology: Animal Behavior Processes, 18(4), 400–413. Zelikowsky, M., & Fanselow, M. S. (2010). Opioid regulation of Pavlovian overshadowing in fear conditioning. Behav Neurosci, 124(4), 510–519.

2/15/2011 7:46:28 PM

CHAPTER 15 Incentives in the Modiﬁcation and Cessation of Cigarette Smoking Edwin B. Fisher, Leonard Green, Amanda L. Calvert, and Russell E. Glasgow

We review research on the effectiveness of incentives in general health promotion, and in interventions for smoking and other drug addictions in particular. Consistent with basic principles of learning and reinforcement, and from a behavioral economic perspective, we ﬁnd that (1) incentives are effective in encouraging smoking cessation and other health behaviors; (2) incentives are effective while in place but not after they are terminated; and (3) nonsmoking may be encouraged by increasing the availability of reinforcing activities that substitute for the reinforcement from nicotine. Incentives may be especially effective when smoking cessation is a priority for a speciﬁc period of time, such as during pregnancy. Incentive programs appear to inﬂuence otherwise “hard-to-reach” groups. The impact of incentives is enhanced when implemented in the context of broader programs promoting smoking cessation, and incentives applied to populations may be cost efﬁcient by achieving modest effects for large numbers of individuals. Basic principles of reinforcement and the use of incentives understood within a behavioral economic framework should continue to inform public-health interventions and may lead to new insights into effective approaches for inﬂuencing health behavior.

INTRODUCTION The good news is that the prevalence of smoking among adults has declined from 42% in 1965 to 20.8% in 2008 (Centers for Disease Control [CDC], 2008). Of all adults in this country who have smoked, 49% have quit (CDC, 2002). The bad news is that despite these impressive strides, one-ﬁfth of the adult population continues to smoke, and the percentage of teenagers who smoke was also 20% in 2007 (Centers for Disease Control Ofﬁce on Smoking and Health, 2007). Thus, this widespread, severe public health problem will persist for at least another generation. Moreover, the prevalence of smoking remains especially high in several groups of special concern: low-income groups, minorities, those with smoking-related disease, and low-income women of childbearing age (Gilpin & Pierce, 2002).

The need for focused and vigorous approaches to reducing smoking among these groups remains especially acute. Early research on the effectiveness of smoking-cessation programs often considered the role of incentives. A review of worksite smoking programs by Orleans and Shipley (1982) included a number of case-study and anecdotal reports of successful incentive programs. They advocated controlled investigations of how incentives might enhance worksite programs. The investigation of incentives also was encouraged by a more general recognition of the role that positive reinforcement plays in smoking (Pomerleau, Collins, Shiffman, & Pomerleau, 1993), including how genetics may inﬂuence the effectiveness of nicotine metabolism as a reinforcer (Lerman et al., 1999). Additionally, the ﬁeld of behavioral economics has increased our understanding of

321

15-Schachtman-15.indd 321

2/15/2011 7:46:42 PM

322

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

how different incentives for drug use interact with each other and with the contexts in which they operate (e.g., Bickel & McLellan, 1996; Kagel, Battalio, & Green, 1995). These literatures raise the prospect of more rational and effective incentive programs to encourage nonsmoking and smoking cessation. Since Orleans and Shipley’s review (1982), however, research on the use and impact of incentives in smoking cessation has been modest. We conducted a series of Medline searches of journal articles published between 1966 and 2009, using cognates of reinforcer, contingency, reward, incentive, lottery, and contest. The vast majority of papers identiﬁed that also dealt with smoking were not intervention studies. Despite this limited research on incentives in smoking cessation, there are substantial literatures on incentives in general health promotion and in treating other types of drug abuse. Accordingly, we include in this chapter a brief review of incentives in other areas of health promotion along with review of the use of incentives in treating addictions to other drugs as well as to nicotine. There are two broad approaches in the use of incentives for promoting appropriate health behavior. The ﬁrst approach includes incentives applied directly to the target behavior, for example, providing reinforcers for maintaining abstinence from smoking for several days. A second approach entails incentives for other behaviors that may compete with the target behavior, what in behavioral terms is referred to as “differential reinforcement of alternative behavior.” Consequently, the ﬁrst part of this chapter addresses incentives applied directly to behaviors, and the second addresses incentives applied indirectly.

INCENTIVES APPLIED DIRECTLY TO BEHAVIOR Following a brief presentation of the use of incentives in health promotion in general and in the treatment of drug use other than smoking, we discuss the use of incentives in smoking cessation.

15-Schachtman-15.indd 322

Incentives in Health Promotion

For some time, reports have indicated that incentives may be successful in a wide variety of areas of health promotion: child immunizations (Achat, McIntyre, & Burgess, 1999) and vaccinations (LeBaron, Starnes, Dini, Chambliss, & Chaney, 1998); adherence to antihypertensive medications (Feldman, Bacher, Campbell, Drover, & Chockalingman, 1998); return for reading of tuberculosis (TB) skin tests among drug users (Fitzgerald et al., 1999; Malotte, Hollingshead, & Rhodes, 1999); compliance with TB drug regimens among homeless adults (Tulksy et al., 2004); adherence to the hepatitis B vaccine regimen among injection drug users (Seal et al., 2003); promoting mammography (Janz et al., 1997); maintenance of breast self-examination (Solomon et al., 1998); reducing loss to follow-up among women with abnormal Pap tests (Marcus et al., 1998); and, of pertinence to the present chapter, reducing carbon monoxide levels (levels of carbon monoxide correlate with tobacco smoking) among individuals with chronic obstructive pulmonary disease (Crowley, MacDonald, Zerbe, & Petty, 1991). Based on results reported in individual studies and ﬁndings from other systematic reviews and meta-analyses, recent papers (Marteau, Ashcroft, & Oliver, 2009; Sutherland, Christianson, & Leatherman, 2008) have summarized the evidence for the effectiveness of incentives in health promotion. A comprehensive review of studies evaluating the role of incentives on risky behaviors, preventive care, and adherence to recommended treatment (Sutherland et al., 2008) concluded: “The ﬁndings of studies reviewed in this article suggest that ﬁnancial incentives, even rather small ones, can inﬂuence health behaviors” (p. 74S). This conclusion is probably an understatement. The review documents evidence for the effects of incentives in exercise promotion, improving lipid metabolism, having a mammogram (with the odds ratio for those given incentives being 2.7; that is, the likelihood of having a mammogram was 2.7 times greater among those offered incentives

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

than among those not offered incentives) (Stone et al., 2002), follow-up after positive PAP tests, screening for colon cancer, screening and treatment for tuberculosis, prenatal and postnatal care, as well as outpatient HIV testing and attendance at classes addressing HIV prevention (see pp. 63S–73S). Providing incentives for immunizations has produced especially clear results: “In all reviews, patient-targeted incentives, used alone or in combination with other interventions, were found to be effective in increasing uptake of immunizations in both children and adults” (p. 66S). In particular, a meta-analysis that included 81 studies of incentives for immunizations reported an odds ratio of 3.4 for incentives (Stone et al., 2002). The review evaluated price reductions and reductions in copays among incentives, including, for example, price reductions that increased choices of healthy foods (p. 43S) or reductions of out-of-pocket costs that increased the likelihood of getting vaccinated (p. 66S). The use of incentives for producing weight loss has provided mixed results: “The evidence on the effect of incentives on weight loss, either in the community or at worksites, is less conclusive” than that for other health behaviors (Sutherland et al., 2008, p. 65S). As noted in reviews, methodological problems have contributed to this lack of conclusiveness, especially the use of designs in which incentives are combined with other interventions (e.g., behavioral counseling, health-promotion classes, or selfmonitoring of weight), complicating the evaluation of the effect of the incentive itself. Incentives do not preclude but rather may complement other program components. For example, either a personal trainer or monetary incentives were effective in increasing exercise in a weight-management program, and the combination was more effective than either alone (Jeffery, Wing, Thorson, & Burton, 1998). A similar pattern emerged from a review of programs designed to promote adherence to TB treatment. The odds ratios comparing program components to control conditions were 1.6 for monetary incentives, 1.2 for health education, and 2.4 for the combination of the two (Volmink & Garner, 1997).

15-Schachtman-15.indd 323

323

Incentives for Drug-Use Problems Other Than Smoking

Various forms of treatment programs have used incentives in treating problems other than cigarette smoking. Some of these include contingency contracting, in which the patient and the treatment provider agree that speciﬁed consequences (i.e., incentives) will be contingent upon appropriate behaviors (Prendergast, Podus, Finney, Greenwell, & Roll, 2006; Silverman et al. 1996); vouchers retrievable for goods and services (Bigelow, Brooner, & Silverman, 1998; Jones, Haug, Silverman, Stitzer, & Svikis, 2001; Plebani et al., 2006; Robles et al., 2000); and the receipt of privileges, such as the freedom to take doses of methadone at home rather than at a clinic (contingent on drug-free urine tests) (Chutuape, Silverman, & Stitzer, 1999a). Additional research has demonstrated the beneﬁts of using escalating schedules of reinforcement (e.g., Roll & Shoptaw, 2006; Silverman, Chutuape, Bigelow, & Stitzer, 1997). In escalating schedules, the criterion for earning the next reinforcer is higher than that for the previous reinforcer. Using this type of schedule to deliver reinforcers makes it more likely that individuals will make contact with the reinforcer early in treatment, a variable that has been shown to be related to treatment success (Kirby, Marlowe, Festinger, Lamb, & Platt, 1998). Moreover, because the number of reinforcers delivered decreases with time, an escalating schedule may also be cost efﬁcient because it delivers more incentives early in the program, when they are likely to have a greater effect, and fewer later on. A set of several incentives among which individuals can choose can also serve as an extra incentive. Choice between a take-home dose of methadone or a voucher worth $25.00 contingent on drug-free urine tests resulted in more drug-free urine samples, greater latencies to drug-positive urine samples, and longer sustained abstinence than did standard care, which lacked these features (Chutuape, Silverman, & Stitzer, 1999b). As with health promotion in general, incentives do not preclude the effective use of other

2/15/2011 7:46:43 PM

324

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

interventions. Contingent delivery of vouchers signiﬁcantly improved success when added to a drug-treatment program that already included a system of contingent privileges for such things as take-home medications and individual counseling (Iguchi, Belding, Morral, Lamb, & Husband, 1997). The literature on incentives in the treatment of drug abuse reveals an important ﬁnding: Incentives may be especially effective for those who are resistant to treatment or otherwise hard to reach. In one study, for example, individuals who had achieved fewer than 4 out of 13 weeks of cocaine abstinence in a methadone maintenance program were considered treatment resistant and eligible for a study of voucher magnitude (Silverman, Chutuape, Bigelow, & Stitzer, 1999). In counterbalanced order, the individuals received three, 9-week programs that differed in the total dollar value of the vouchers earned: $0, $382, or $3480. The $3480 total incentive amount resulted in 10 of the 22 patients (45%) achieving abstinence in ≥ 4 out of a possible 9 weeks. Only one patient in the $382 phase and none in the $0 phase achieved more than 2 weeks of cocaine abstinence. The high reward magnitude condition also resulted in a signiﬁcantly higher percentage of cocaine-negative urine samples (p < 0.01). In addition to showing how incentives may be effective in reaching otherwise treatment-resistant individuals, the study also demonstrated another important ﬁnding, namely the importance of amount of reinforcement. Incentives for Smoking Cessation

Initial work by Stitzer, Bigelow, and their colleagues demonstrated that monetary incentives could reduce smoking (Stitzer & Bigelow, 1982). Importantly, incentive effects were speciﬁc to the contingencies imposed, such as with contingencies on carbon monoxide (CO) in expired air, an indicator of recent smoking. When the target CO level was 8 ppm or less, 45% of participants reduced their CO to that level, in comparison to 0% when the target CO was 16 ppm (Henningﬁeld, Stitzer, & Grifﬁths, 1980; Stitzer & Bigelow, 1985). (It is to be noted that 8 ppm or

15-Schachtman-15.indd 324

lower is generally taken as indicative of abstinence.) Having shown that smoking could be brought under the control of incentives, Stitzer, Rand, Bigelow, and Mead (1986) secured short-term smoking abstinence with payments contingent on reduction and cessation in conjunction with worksite and home CO monitoring. Extending these ﬁndings, contingent payment ($4 twice weekly) led to greater abstinence over 3 months than noncontingent payment (Rand, Stitzer, Bigelow, & Mead, 1989). These results were obtained without any provision of cessation strategies, coping skills training, or programmed social reinforcers to the participants, thus underscoring the positive impact that incentives have on behavior change. A critically important ﬁnding, one that will be emphasized in this chapter, is that incentives appear to be effective only while they remain in effect. Once the incentive is discontinued, the change in behavior typically ceases. A review of the literature conducted by the Cochrane Collaboration on smoking-cessation programs that use incentives and competitions found these programs to be effective, but only while they are in force (Cahill & Perera, 2008). Long-term maintenance of behavior change requires continuation of incentives, either through continuation of programs providing them or through generalizing program-speciﬁc reinforcers to those occurring naturally. In addition to modifying smoking among users of other drugs, incentives also have been shown to reduce smoking among schizophrenics (Roll, Higgins, Steingard, & McGinley, 1998). This ﬁnding is important because of the heightened prevalence of and difﬁculty of treating smoking among patients with schizophrenia as well as depression (Covey, Glassman, & Stetner, 1997; El-Khorazaty et al., 2007) and other psychological problems (Hughes, Hatsukami, Mitchell, & Dahlgren, 1986). Clinical Impact

The studies reviewed were designed to examine whether incentives would inﬂuence smoking, but they did not address the issue of whether clinically signiﬁcant, sustained changes in smoking would be achieved. A study by Fortman and

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

Killen (1995) demonstrated the potential contributions of incentives in smoking cessation. Participants who had quit for at least 24 hours were recruited by random phone surveys for a study of self-help materials and nicotine gum. All participants were offered $100 if they were able to remain abstinent at 6 months follow-up. Remarkably, even a control group that received neither self-help materials nor nicotine gum, but did receive $100 if they remained abstinent for 6 months, achieved 20% and 16% abstinence at 6- and 12-month follow-up, respectively. Of course, the sample was a volunteer sample of those who had already quit (albeit for only a day). However, the proportion of smokers identiﬁed from the random phone surveys who qualiﬁed for and volunteered to participate in the study—14.6% (1,044 out of 7,135)—was still much higher than the percentage of smokers who volunteer for other, non-incentives-based treatment programs. This ﬁnding suggests that the $100 incentive was quite inﬂuential in promoting both enrollment as well as cessation among those who joined. More recently, Volpp and colleagues studied the effect of incentives on smoking cessation in a population of lower-income smokers in a Veterans Administration hospital (Volpp et al., 2006) and middle-income employees in a worksite-based cessation program (Volpp et al., 2009). Results from these randomized, controlled trials showed a signiﬁcantly lower rate of smoking at 1 and 12 months following program completion, respectively, for the groups that were given incentives contingent on smoking cessation. Return of Deposit

A popular form of the use of incentives to change behavior has been requiring a deposit of money or valued possessions for entry into a program and then returning the deposit contingent on progress in the program. Paxton (1980) showed that return of monetary deposits increased the efﬁcacy of a smoking-cessation program, and that, consistent with the literature on reinforcement effects, the amount of deposit returned (although not the frequency by which deposits were returned) increased the short-term impact on smoking abstinence (Paxton, 1981).

15-Schachtman-15.indd 325

325

An obvious concern with this type of incentive implementation is whether requiring a deposit for entry into a program reduces participation. However, collecting a cash deposit in four, weekly installments rather than one initial payment increased participation rates with no lesser impact on abstinence from smoking (Paxton, 1983). In a study of the trade-off between increased efﬁcacy and decreased participation (Jeffery, Hellerstedt, & Schmid, 1990), participants either paid $5 to enter a 6-month correspondence program or gave a $60 deposit, one-sixth of which was refunded for each month of smoking abstinence. In terms of participation, the version with the $5 entrance fee was far more popular (in terms of enrollment) than that with the $60 deposit by a ratio of about 5 to 1. However, abstinence at 6-month follow-up in the return-of-deposit condition was 20% versus 9% in the $5 condition. A similar pattern of results was found in a second group of individuals recruited for a weight-loss program in which the return of deposit was based on preset monthly weight-loss goals. Contests and Lotteries

As would be expected from research on the effects of extinction on behavior, contests and lotteries have been shown to have appreciable impact on abstinence while those contingencies are in place, but lesser impact on long-term abstinence from smoking after the contingencies have been removed (Matson, Lee, & Hopp, 1993). For example, a “Quit-to-win” contest achieved abstinence rates of 56%, 27%, and 21% at 6-week, 6-month, and 12-month follow-up evaluations (Leinweber, Macdonald, & Campbell, 1994). As with return-of-deposit programs, evaluation of lotteries and contests also must examine effects on both participation and efﬁcacy (Matson et al., 1993). In one study, adding competitions to worksite smoking-cessation programs increased participation without reducing the cessation rate among those who participated. Thus, the competition worksites achieved cessation among an estimated 16% of all smoking employees, versus 7% in worksites that did not incorporate competitions (Klesges, Vasey, &

2/15/2011 7:46:43 PM

326

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Glasgow, 1986). However, participation and cessation rates may diverge, as demonstrated in a community study that compared participation and cessation rates among those receiving smoking-cessation classes, self-help materials, or contests (Altman, Flora, Fortmann, & Farquhar, 1987). The smoking-cessation classes achieved the highest cessation rates but reached the fewest people and were the least cost efﬁcient. The selfhelp materials had the lowest cessation rate, percentage-wise, but reached the largest numbers of people and were more cost efﬁcient than the classes. Finally, the contests fell in between the other two approaches (i.e., the smokingcessation classes and self-help materials) in cessation rate, numbers of participants, and cost efﬁciency. A program for smokers with chronic obstructive pulmonary disease demonstrated the importance of the amount of incentive, the potential of combining several different types of incentives, and the utility of variation in procedures to enhance their salience (Crowley et al., 1991). When up to three public lottery tickets were contingent on CO levels below 10 ppm, there was no change in smoking behavior from baseline. When the incentive was increased to ﬁve lottery tickets combined with the use of nicotine gum, CO levels were initially reduced but recovered over time. The lottery tickets were eventually presented contingent on reduced CO levels but following random checks of CO levels (i.e., a variable ratio, VR, reinforcement schedule). This resulted in CO levels that were abruptly reduced and remained low as long as the contingencies remained in place. A VR schedule is one that produces high, constant rates of responding, in part because the organism cannot predict which response will produce reinforcement. The VR schedule may be beneﬁcial in this type of program because participants need to maintain abstinence throughout the duration of the program in order to receive the lottery tickets when they are (unpredictably) made available.

in smoking in California and Massachusetts (CDC, 1996; Pierce et al., 1998). Because such an approach reaches entire populations, even a small impact can have substantial beneﬁt. In California, reductions in consumption attributable to a $0.25 tax increase were greater than those attributable to an anti-smoking media campaign (Hu, Sung, & Keeler, 1995). (Such effects, of course, are dependent on the tax amount and the level of investment in media campaigns.) Uptake among youth appears especially price sensitive and therefore susceptible to the imposition of taxes. Results from surveys among adult and teen smokers in Massachusetts indicated reduced smoking following a $0.25 tax increase. Reductions were most pronounced among lowincome smokers, especially low-income teens (Biener, Aseltine, Cohen, & Anderka, 1998). Analyses of changes in smoking rates among Canadian provinces that did and did not institute reductions in cigarette taxes indicated that the tax cuts were associated with greater uptake of smoking and lower rates of quitting (Hamilton, Levinton, St-Pierre, & Grimard, 1997). The effects of taxes can be quite complex. For example, increasing taxes on cigarettes has been shown to increase use of smokeless tobacco. Presumably, the relative cost reduction for smokeless tobacco led to increased consumer choice. However, this effect was not reciprocal. Increasing taxes on smokeless tobacco did not increase consumption of cigarettes (Ohsfeldt, Boyle, & Capilouto, 1997). This nonsymmetrical result mirrors an effect found in rats: When food was restricted, rats increased the amount of water they drank (above baseline, nonrestricted levels), but when water was restricted, the rats did not increase their food consumption (relative to baseline) (Rachlin & Krasnoff, 1983). Such a ﬁnding ﬁts within a behavioral economics framework in which issues of substitutability and complementarity are addressed (see Green & Freed, 1993, 1998). These issues will be discussed in more detail in the following sections.

Taxes

Combination of Social and Monetary Incentives

Increases in excise taxes and the use of those proceeds for smoking-prevention programs have been associated with substantial statewide reductions

15-Schachtman-15.indd 326

In their review, Orleans and Shipley (1982) called for research into how incentives might

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

potentiate the inﬂuence of other components of worksite programs, such as social support among employees. The possibility of incentives enhancing other treatment components is not limited to worksites, of course. Donatelle, Prows, Champeau, and Hudson (2000) reported impressive effects from the combination of interpersonal support and voucher incentives for smoking cessation among pregnant recipients of WIC services. Every participant initially received educational materials about quitting smoking and also identiﬁed a “social supporter, preferably a female non-smoker with whom the participant had a regular, close, positive association” (p. iii67). Additionally, for those in the voucher condition, participants were eligible to earn department store vouchers worth $50 per month over 10 months, for a total of $500 in vouchers, contingent on monthly self-reports of abstinence conﬁrmed by biological assessments (i.e., salivary cotinine and thiocyanate levels). The quit rate during the eighth month of gestation for those women who received vouchers plus interpersonal support and educational materials (voucher group) was 32%, as compared to 9% for those who received only the support and educational materials (control group). At 2 months postpartum, 21% of the voucher group had remained abstinent, whereas only 6% of the control group had done so. As reviewed by Matson and colleagues (1993), several studies have demonstrated impacts of combining support groups, competitions, and incentives in worksite smoking programs. For example, one worksite-based study (Jason, Jayaraj, Blitz, Michaels, & Klett, 1990) included the following: • Direct incentives—$10 for each of 14 meetings attended (independent of smoking status), $1 per each day that the individual was abstinent for up to 6 months following completion of the program, $30 for each period of 30 consecutive days’ abstinence, and chances in a cash lottery • Availability of support groups—participants could complete smoking-cessation procedures on their own or with a group • Availability of team competition—participants could form teams of three smokers and

15-Schachtman-15.indd 327

327

compete for a cash prize of $300 to be given to the team with the most number of days abstinent Relative to a control worksite that received no intervention, the combination of the support group, incentives, competition, and cash prize resulted in substantially greater numbers of participants who were abstinent at the end of the program (49% versus 9%), 6-month follow-up (42% versus 13%), and at 12-month follow-up (36% versus 16%). The success of this combination of intervention tactics raises interest in further research examining sequences and combinations of such approaches. For example, a program might begin with ﬁnancial incentives and then phase them out while introducing social incentives that may be more sustainable and also may be more able to link individuals to naturally occurring reinforcers among friends and families.

INFLUENCING PROBLEM BEHAVIORS INDIRECTLY THROUGH INCENTIVES FOR OTHER BEHAVIORS Behavioral economics identiﬁes several ways in which incentives for one behavior may inﬂuence the likelihood of other behaviors. One is substitutability in which changes in the price or the availability of one good leads to opposite or compensating changes in consumption of another good. As mentioned earlier, an example would be an increase in the price of cigarettes leading to reduced consumption of cigarettes but increases in the consumption of smokeless tobacco. In this case, smokeless tobacco and cigarettes would be substitutable for each other. Another way in which consumption of two goods can be linked is complementarity, in which changes in the price or availability of one produces similar directional changes in the consumption of the other good. For example, an increase in the price of cigarettes may lead to reduced purchases of cigarettes as well as reduced purchases of coffee. Drug taking has been shown to be substitutable with other reinforcing activities. This ﬁnding is

2/15/2011 7:46:43 PM

328

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

in contrast to the typical image of drugs as blinding the individual to other reinforcers, thereby keeping the addict from seeking other sources of reinforcement. The substitutability of drug taking was demonstrated in an experimental, controlled economy in which participants lived and had unconstrained access to marijuana. Participants received no money other than that earned by making belts while in the experiment. In this closed economy, increases in the wage for belt making reduced marijuana smoking during the experimental period (Kagel, Battalio, & Miles, 1980; Miles et al., 1974), consistent with the view that money is substitutable for marijuana. Pleasant activities and social relationships also are substitutable for drug taking. High frequency of reinforcing activities unrelated to drug taking (Correia, Simons, Carey, & Borsari, 1998), pleasant events (Van Etten, Higgins, Budney, & Badger, 1998), and social engagement and activity (Audrain-McGovern et al., 2004; Vuchinich & Tucker, 1988) are related to lower levels of smoking, drug taking, and alcohol consumption. Similarly, a combined take-home methadone and voucher treatment showed greater cocaine abstinence as well as greater enjoyment of daily activities among methadone-maintained cocaine abusers (Rogers et al., 2008). Nondrug social reinforcers have the potential for enhancing the effectiveness of a treatment program. A review of cocaine treatment studies found that the availability of alternative, nondrug reinforcers enhanced the effects of pharmacological treatments for cocaine abuse (Higgins, 1997; LeSage, Stafford, & Glowa, 1999). The demonstration of substitutability of other reinforcers for drug taking has important implications for the promotion of abstinence. From the perspective of behavioral economics, low prices of alternative reinforcers relative to the price of drugs should increase consumption of those alternatives and decrease consumption of the drugs. For example, cigarette puffs decreased as their price increased relative to either nicotine gum or money (Johnson & Bickel, 2003). In contrast, drug taking will be encouraged if drug reinforcers are more readily available than other reinforcers, such as a rewarding job and a

15-Schachtman-15.indd 328

supportive family and friends. For those under conditions of economic disadvantage, or when opportunities to obtain these reinforcers are blunted by prejudice, access to these non-drugrelated reinforcers may be limited. This type of situation is more likely to be found among many of the subpopulations with high rates of drug use. The effect of increasing alternative activities and goods in people’s lives as a way to reduce drug taking or to help sustain abstinence has received little attention in programs for smoking cessation or other addictions. The implications of this approach may be especially important in light of the greater prevalence of addiction problems among economically and educationally disadvantaged groups. Whatever the reasons for an individual becoming addicted, the limited access to goods and pleasurable activities associated with educational and economic disadvantage, as well as the erosion of other sources of reinforcement that follow multiple drug dependencies, may make abstinence especially difﬁcult. Increasing other reinforcers may encourage steps toward abstinence. Community Reinforcement Approach (CRA) programs have been found to reduce alcohol consumption (Smith, Meyers, & Delaney, 1998) and opiate use (Abbott, Weller, Delaney, & Moore, 1998) through a variety of structured behavioral skills sessions focused on problem solving, abstinence, and communication skills. CRA programs often include structured drug- and alcohol-free social events. These social events may be thought of as situations in which non-drug-related social reinforcers are made accessible. The emphasis on relationships and pleasurable activities does not preclude the use of tangible reinforcers. Indeed, voucher incentives in conjunction with a CRA program were shown to produce signiﬁcantly greater abstinence than the CRA alone (Higgins et al., 1994). Vouchers for work and work-related training in a therapeutic workplace for pregnant women led to 59% abstinence relative to 33% among controls (Silverman, Svikis, Robles, Stitzer, & Bigelow, 2001). Monetary reinforcement for abstinence and for attendance at a prenatal care and drug counseling program for cocaine-dependent

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

pregnant women led to higher rates of attendance and, strikingly, 0% as opposed to 80% adverse perinatal outcomes (Elk, Mangus, Rhoades, Andres, & Grabowski, 1998). Reinforcement of behaviors other than drug taking also can be approached through family and friends. A number of years ago, Azrin and colleagues (Hunt & Azrin, 1973) demonstrated the beneﬁcial effects of educating spouses in providing reinforcement for behavior other than drinking and extinction for drinking and related behaviors. An intervention that trained concerned family members and signiﬁcant others in reinforcing the drug user’s entering treatment and ceasing drug use, as well as reinforcing behaviors inconsistent with drug taking, increased the likelihood of the drug user’s entering treatment relative to an Alanon self-help program (Kirby, Marlowe, Festinger, Garvey, & LaMonica, 1999).

THEORETICAL ISSUES AND KEY FACTORS IN INCENTIVE PROGRAMS The results from the studies reviewed are consistent with the conclusion that, while they are in effect, incentives have reliable effects on health behaviors, including addictive behaviors and, in particular, cigarette smoking. In addition to demonstrating these effects in laboratory or experimental settings, they have been shown in a variety of clinical and prevention programs in diverse settings and with diverse populations. They also are consistent with an enormous literature on the effects of incentives/reinforcers on diverse human and animal behaviors (Rachlin, 1991). The answer to “Should we pay the patient?” (Giuffrida & Torgerson, 1997) is “Yes”— reinforcement works. That said, there still are issues to be considered in fully evaluating the effectiveness of incentives in reducing smoking. Amount and Delay of Reinforcement

Increases in the amount of reinforcement have been associated with reductions in smoking and increases in the percentage of individuals achieving a criterion of 50% of baseline CO

15-Schachtman-15.indd 329

329

levels (Stitzer & Bigelow, 1983), increases in time since last cigarette smoked (Correia & Benson, 2006), and increases in consecutive abstinent CO readings (Roll, Higgins, & Badger, 1996). In programs for other problem behaviors, amount of reinforcement was associated with frequency of drug-free urine tests among cocaine abusers (Higgins et al, 2006; Petry et al., 2004), treatment attendance (Jones, Haug, Stitzer, & Svikis, 2000; Svikis, Lee, Haug, & Stitzer, 1997), reductions in heroin use in opioid-dependent individuals (Comer et al., 1998), and returning for reading of TB skin tests (Malotte, Rhodes, & Mais, 1998). However, as noted earlier in the discussion of contests and lotteries and as conﬁrmed by experimental studies, delay to reinforcement reduces the effectiveness of a reinforcer (see, e.g., Lattal, 1987) and has been shown to inﬂuence the choice to smoke cigarettes (Roll, Reilly, & Johanson, 2000). Of course, the relatively immediate reinforcement of smoking, 7–9 seconds from inhaling to the time that nicotine reaches the central nervous system, has long been implicated in the strength of addiction to nicotine (Henningﬁeld & Keenan, 1993). Greater Effectiveness of Cash Than Other Incentives

Ten dollars was more effective than the equivalent amount in grocery store coupons, bus tokens, or fast-food coupons in reinforcing return for reading of TB skin tests (Malotte et al., 1999). Similarly, cash was more effective than nonmonetary incentives in reinforcing attendance at a clinic for sexually transmitted diseases (Kamb et al., 1998) and was more effective than self-reward (i.e., a small reward that the individual administers, such as “enjoy some quiet time” or “buy yourself ﬂowers”) in promoting breast self-examination (Solomon et al., 1998). Interestingly, the “urn” lottery, developed by Petry and colleagues (Petry, Martin, Cooney, & Kranzler, 2000) has been shown to be more effective in producing drug abstinence and more cost efﬁcient than vouchers among opioiddependent individuals (Petry, Alessi, Marx, Austin, & Tardif, 2005). In this lottery system,

2/15/2011 7:46:43 PM

330

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

abstinence results in an opportunity to draw a slip of paper from a bowl, where each slip states the prize to be received. Prizes often range in value from very small to large, and the probability of drawing a prize is inversely proportional to its value. Research should compare the efﬁcacy of monetary and other rewards when applied to smoking cessation. Effects of Context on Incentives

Tangible incentives operate within broader contexts of other program components. The importance of other program components was shown in analyses of contests within the COMMIT program, a major trial of community-based programs and activities to promote nonsmoking (The COMMIT Research Group, 1995a, 1995b). The best predictor of positive smoking outcomes for a community was the amount of money invested in the contest program (e.g., media, staff, and labor costs) that did not include the contest prizes themselves (Shipley, Hartwell, Austin, Clayton, & Stanley, 1995). This result underlines the importance of investing resources in planning and announcing programs, working with individuals, and employing appropriately trained staff, in addition to the money invested in the actual incentives, an aspect that is often overlooked in the literature. A number of other studies have evaluated the impact of the context of lotteries or contests in promoting cessation. In the Minnesota Heart Disease Prevention Program, intensive promotion of a statewide contest in Bloomington resulted in participation by about 1.06% of eligible smokers, substantially higher than the 0.2% participation in other suburbs of the Twin Cities area. With the 37% long-term abstinence rate obtained in Bloomington, this translates to a total reduction of the smoking rate in the community of about 0.39% (1.06 × 0.37). Although the longterm abstinence rate among participants in the other Twin Cities suburbs was somewhat higher (45%) than the 37% in Bloomington, the difference in participation resulted in a net reduction in the overall percentage of smokers of 0.09% in those other suburbs (0.2 × 0.45), relative to the 0.39% in Bloomington (Lando, Loken,

15-Schachtman-15.indd 330

Howard-Pitney, & Pechacek, 1990). Despite the small percentage difference, notice that the intensively promoted program had a four-fold greater reduction in smoking, a not-inconsequential effect when considered at the population level. Combinations of contests and incentives with group smoking-cessation programs and promotional campaigns to encourage quitting have been reported to lead to 12-month abstinence rates of 36% (Jason et al., 1990) and 50% (Maheu, Gevirtz, Sallis, & Schneider, 1989). The North Karelia project in Finland (Puska, Vartiainen, Tuomilehto, Salomaa, & Nissinen, 1998) also demonstrated the importance of broader community support and promotion in enhancing the beneﬁts of a nationwide combination of a contest and an 8-installment television program promoting smoking cessation. Relative to the city of Turku, in which community support was less intensive, rates of viewing the program, participating in the contest, attempted quits among viewers, and abstinence rates 6 months after the program all favored the region of North Karelia (Korhonen et al., 1992). Nicotine Replacement as Substitutability

In a sense, the demonstrated success of nicotine replacement (e.g., nicotine patch, nicotine gum) reﬂects the substitutability of one source of nicotine for another (e.g., Buchkremer, Minneker, & Block, 1991; Fiore, Smith, Jorenby, & Baker, 1994). This is joined with the ﬁnding of substitutability of cigarettes and smokeless tobacco (Ohsfeldt et al., 1997). Of course, the choice of nicotine replacement over smoking cigarettes will be inﬂuenced by their relative prices and the levels of inconvenience associated with obtaining them. A behavioral economic perspective on relatively safe sources of nicotine as a substitute for cigarettes that contain nicotine is congruent with current movements to make varied sources of nicotine readily available and competitively priced. Social Support and Interactions as Incentives Substitutable for Nicotine

A key factor in the success of smoking cessation may be the presence of supportive people around

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

the smokers. For example, cessation rates were higher among the 60% of participants in a Minnesota community-based contest who designated a “support person” (e.g., a friend, spouse, or other family member) than among those who did not (Pirie, Rooney, Pechacek, Lando, & Schmid, 1997). This difference was especially pronounced among those who reported that their spouses (who were not designated as the “support person”) were either smokers or nonsupportive. More generally, substantial research indicates that smoking is more likely among socially isolated individuals and that social support from friends and family is associated with greater likelihood of successful quitting (Fisher, Brownson, Heath, Luke, & Sumner, 2004). Reﬂecting these ﬁndings, the 2008 guidelines on smoking cessation of the Department of Health and Human Services, Treating Tobacco Use and Dependence (Fiore et al., 2008), reported that the provision of social support along with the number of contacts and the total duration of smoking-cessation interventions are all predictive of greater success. Nevertheless, there have been mixed ﬁndings regarding efforts to enhance smoking cessation with social support, such as reported in an inﬂuential review by Lichtenstein, Glasgow, and Abrams (1986). In his review of what may explain such mixed ﬁndings, Fisher (1997) emphasized a behavioral economic perspective in which social support is viewed as a reinforcer that may substitute for nicotine. From this perspective, several problems with social support interventions that were unsuccessful and several possible changes to these interventions based on behavioral economics were noted: 1. Social support was often terminated at the end of treatment. However, the ex-smokers may still need the reinforcement that social support provides as a substitute for nicotine. If support is to be an effective alternative to nicotine, then its availability needs to be sustained. 2. Some interventions focused on teaching individuals how to obtain support for quitting rather than providing social reinforcement to the quitter. Other interventions were

15-Schachtman-15.indd 331

331

too inﬂexible in the ways in which the social support could be provided to the individuals. It seems reasonable, then, that these social interactions may have been less enjoyable and, thus, less reinforcing. 3. Some supportive interventions emphasized teaching participants how to obtain support rather than simply providing them with sources of support. The latter is more effective if support is to serve as an incentive that substitutes for nicotine. “Overjustiﬁcation Effect” and Intrinsic Motivation

A recurrent concern about the use of incentives has centered on the possibility of “overjustiﬁcation effects” in which salient, extrinsic incentives might undermine intrinsic motives for behavior. However, a critical review of the empirical literature demonstrated that such effects are, in fact, minimal in real-world settings in which manipulated incentives are not very large, in which extrinsic incentives are not administered in a manner so as to obscure the salience of other incentives for desirable behavior, or in which newly introduced reinforcers do not interfere with already existing reinforcers of established behavior (e.g., by introducing prizes for practicing the piano that impose a new requirement of keeping records that, in turn, interferes with already established reinforcers in the practice routine) (Cameron & Pierce, 1994; Fisher, 1979). In their analysis of over a quarter century of research, Cameron and Pierce (2002) found little evidence that reinforcement reduces intrinsic task interest. Reinforcement does not appear to reduce intrinsic motivation; on the contrary, Cameron, Banko, and Pierce (2001) observed that when reinforcement is linked to level of performance, intrinsic motivation increases or shows no change (see also Eisenberger & Cameron, 1996; Eisenberger, Pierce, & Cameron, 1999). The same perspective that distinguishes intrinsic and extrinsic incentives and gives rise to the hypothesized overjustiﬁcation effect also leads to hypotheses that programs aimed at bolstering enduring intrinsic motivation will be

2/15/2011 7:46:43 PM

332

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

more successful than those that address extrinsic rewards and incentives for desired behavior. There is remarkably limited evidence to support this claim. Seattle-area smokers were offered extrinsic rewards or individually tailored feedback (i.e., a personal analysis of the participant’s progress throughout the intervention) contingent on returning a baseline questionnaire and progress reports within a self-help program (Curry, Wagner, & Grothaus, 1991). Those given the feedback were more likely to use the self-help materials, report short-term abstinence (3 months), and be abstinent (validated by measuring cotinine levels, a salivary by-product of smoking tobacco) at 12-months follow-up. These ﬁndings might be interpreted as showing that increasing intrinsic motivation is better accomplished with personalized feedback rather than with extrinsic rewards. However, consideration of the details of the procedures suggests an alternative explanation. The extrinsic reward entailed a “secret gift” (a ceramic coffee cup) along with entries into drawings for three prizes, a 1-week vacation in Hawaii, a weekend at a resort on the San Juan Islands outside of Seattle, or a weekend at a deluxe hotel in Seattle. Thus, individually tailored feedback did produce better results than did receipt of a coffee cup and chances among 607 other participants to win one of three vacation prizes. Considering the evidence regarding the importance of amount and probability of reward, the results from this study may be seen as indicating the greater impact on smoking cessation of tailored feedback than of one small prize and low odds of winning, rather than a more general advantage of intrinsic rewards. The Use of Incentives for High-Priority Behaviors During Limited Time Periods

The reviews by Marteau et al. (2009) and Sutherland et al. (2008) make clear that incentives have only a modest long-term impact on behavior after the incentives are no longer in place. As Sutherland et al. noted: “ . . . research evidence suggests that incentives can increase adoption of healthy behaviors but that positive effects may diminish over time” (p. 65S).

15-Schachtman-15.indd 332

Although many might interpret such a conclusion as evidence for the ineffectiveness of incentives, the reader of this volume is well aware that extinction is an established phenomenon and that diminution of beneﬁts of incentives following their termination—in the absence of other incentives to maintain the behavior—reﬂects the adaptability of behavior to its context, not a failure of incentives to change behavior. Therefore, the use of incentives might be highly recommended (a) for increasing key behaviors that need to occur only once or relatively few times, or (b) for increasing the likelihood of behaviors in particular settings or for particularly crucial periods. The ﬁrst point is illustrated by the evidence for the use of incentives to increase immunizations mentioned earlier (Sutherland et al., 2008). A noteworthy illustration of the second point (inﬂuencing behavior in certain contexts or during critical periods) would be nonsmoking during pregnancy, when the development of the fetus and newborn are crucially affected by smoking. For example, reduction of smoking among pregnant women can reduce the risk of a low birth-weight child by 45% (Ershoff, Quinn, Mullen, & Lairson, 1990). Thus, incentive programs may be especially appropriate for promoting nonsmoking during pregnancy. Little research has explored incentives for this group, but women taking part in an incentivesbased smoking-cessation program during pregnancy and postpartum achieved a higher rate of abstinence up to 24 weeks after delivery (Higgins et al., 2004) and importantly, increased fetal weight during pregnancy (Heil et al., 2008). Incentives for nonsmoking among pregnant women could achieve substantial savings in costs such as those for caring for low birth-weight babies (Adams et al., 2002; Marks, Koplan, Hogue, & Dalmat, 1990). It may be expected that extinction of the appropriate, desired behavior will occur when incentives are withdrawn. However, such an expectation is not reason to be discouraged nor is this the only way to terminate incentivebased smoking-cessation programs. In addition to providing extrinsic incentives contingent on cessation, interventions also must plan for the

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

generalization of program-based reinforcers to reinforcers that occur naturally contingent on smoking cessation (e.g., money savings, social approval, better health). Generalization must occur gradually, where the most effective, most easily controlled program-based reinforcers are delivered early in the intervention in order to achieve immediate cessation. Later, then, the programmed reinforcers can be faded out, and the naturally occurring reinforcers faded in. It is a failure of many incentives-based smokingcessation programs to not plan for this needed generalization.

CONCLUSIONS The literature reviewed above provides the basis for the following conclusions for intervention programs, research, and public health policies. The Use of Incentives in Programs That Promote Smoking Cessation

Incentives do reduce drug taking, including cigarette smoking, at least while they are in effect. As with other reinforcers, amount, probability, and delay of incentives are important: Increasing amount and probability of incentives, and decreasing delay to their receipt, typically increases their effectiveness. Incentives are most promising when smoking cessation is an especially high priority for a deﬁned period of time (e.g., among pregnant women, patients recovering from a heart attack, patients preparing for and recovering from cardiac or cancer surgery and treatment). The effects of incentive programs for cessation of substance use are not explained by other aspects of the intervention programs (e.g., educational components, interaction with intervention providers), although the beneﬁts of incentives are often enhanced when combined with these other components. Incentives alone may have little long-term or “carry-over” effects once they are terminated. This ﬁnding is consistent with most basic and applied research on reinforcement processes. Reinforcers must be continued as long as the target behavior is desired (that is, indeﬁnitely, for

15-Schachtman-15.indd 333

333

substance-use cessation programs). More practically, interventions that incorporate naturally occurring reinforcers into the contingency are more likely to produce long-term behavior change. As Baer, Wolf, and Risley (1968) noted in a key paper on incentives in behavior modiﬁcation over 40 years ago, “Generalization [or maintenance of behavior change] should be programmed, rather than expected or lamented” (p. 97). External reinforcers, such as incentives given in smoking-cessation programs, should be tapered toward the end of treatment and replaced with naturally occurring reinforcers before the intervention is terminated. This needs to be a planned component of the treatment program so that the appropriate behavior will be maintained and generalized, rather than a hoped-for result. A variety of goods may function as incentives, including nicotine replacement, social interaction, and feedback of progress. Incentives could be used to increase participation in a program that already achieves acceptable rates of smoking cessation, or they could be made contingent upon the use of nicotine replacement. Other reinforcing activities can be substitutable for nicotine use, such as positive social interaction, physical activity, and other activities that generally support higher levels of health. Based on research on other types of substance abuse, the use of incentives may be especially cost efﬁcient. Cost efﬁciency will depend on program objectives. A program that reduces smoking during a limited period of time during which continued smoking would lead to appreciable health costs (e.g., during pregnancy or following surgery) may be quite cost efﬁcient. Shifting from a tightly deﬁned target group to a population perspective, modest incentives for smoking cessation deployed to large numbers of individuals can be quite cost efﬁcient even if only a small percentage of the participants quit smoking. Recommendations for Research

The aforementioned conclusions as well as comments and recommendations from several thoughtful reviews in the ﬁeld (e.g.,

2/15/2011 7:46:43 PM

334

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Bigelow et al., 1998) identify priorities for research in several areas, including reﬁning the use of incentives in smoking cessation, and integrating incentives into broader programs for health promotion and quality of life. These are presented in the following sections. Reﬁning the Use of Incentives in Smoking Cessation

One important aspect of the use of incentives is to gain a greater understanding of the boundary conditions under which they are effective. What is the smallest amount or largest delay between the response and receipt of the incentive that can still produce satisfactory effects? If incentives are delivered intermittently, what is the most cost-efﬁcient probability (as in a VR or escalating schedule) for delivering incentives contingent on abstinence? How effective is the use of money versus other incentive types? These parameters will be important in designing smokingcessation programs that take into account a speciﬁc population, funding opportunities, and logistical constraints. As already noted, there may be great cost efﬁciency in applying modest incentives to large populations of smokers, even if such interventions achieve only modest cessation rates. Dallery and colleagues have developed a very promising Web-based contingency management program for smoking cessation that could be very effective in reaching large numbers of smokers with minimal cost to both the agency and the individual (Dallery, Meredith, & Glenn, 2008; Reynolds, Dallery, Shroff, Patak, & Leraas, 2008). Large-scale studies should investigate the application to a wider population and to speciﬁc target groups. Another aspect of incentive implementation that needs to be explored further is the application of shaping procedures for the acquisition and maintenance of behavior change. Although a central concept in the psychological tradition from which incentive programs emerged, the shaping of behavior is rarely emphasized in this literature. For example, one problem with the use of incentives contingent upon abstinence is that many individuals fail to achieve the minimal level drug-negative samples (e.g., in urine

15-Schachtman-15.indd 334

samples or CO breath levels) that are required to receive the incentive, and therefore never make contact with the reinforcement contingency. As a way to increase contact with the incentives, the use of percentile schedules of reinforcement may be especially helpful for those individuals who are more resistant to behavior change (Lamb, Morral, Kirby, Iguchi, & Galbicka, 2004). In the study of percentile schedules by Lamb and colleagues (2004), smokers were randomly assigned to conditions in which reinforcement was contingent on breath CO levels being less than the lowest 1, 3, 5, or 7 out of their previous 10 samples. That is, the several different conditions made reinforcement contingent on the current sample being less than the 10th, 30th, 50th, or 70th percentile of the previous 10 samples. Thus, the conditions differed appreciably in their stringency requirement. In the 10th percentile condition, the participant received reward only if the most recent sample was lower than 9 of the previous 10 samples. In contrast, the 70th percentile condition was quite lenient in that reinforcement was provided for any sample lower than 3 of the previous 10. It is to be noted, however, that even in this lenient condition, meeting that contingency would gradually lower the 70th percentile, moving the contingency inexorably toward zero. Results showed that CO levels were signiﬁcantly lower in the 70th percentile group as the 3-month study progressed. Furthermore, for those smokers classiﬁed as “hard to treat,” the 70th percentile schedule was more effective in producing immediate CO reductions and maintaining lower CO levels than any of the other schedules. This success among hard-to-treat individuals is especially noteworthy given that the prevalence of smoking continues to decline, leaving still smoking those who are often most challenged by cessation. Fading procedures also must be explored more fully. As mentioned, incentives are effective when they are in effect. Therefore, interventions must include a procedure in which the programmed reinforcers (e.g., vouchers, lottery tickets) are faded out and replaced with reinforcers that occur naturally in an individual’s life (e.g., social approval, money savings,

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

better health). In order to use naturally occurring reinforcers most effectively, however, there must be a comprehensive conceptualization of the role of social inﬂuences as incentives. Several studies indicate that social support, social interaction, or feedback of progress may function as reinforcers for participation in programs as well as for not smoking. This view of social support as an incentive differs from the more traditional view in which social support has been conceptualized as an inﬂuence that enhances an individual’s skill or performance (Fisher, 1996). A fuller understanding of social inﬂuence will increase the ability to deploy it effectively. Integrating Incentives into Broader Programs for Health Promotion and Quality of Life

If incentives for nonsmoking are integrated with other program components, then nonsmoking incentives might encourage participation in other health-promotion programs (e.g., preventive care, weight loss, disease management). For example, inclusion of incentives for not smoking may increase the overall attractiveness of the smoking-cessation programs in which they are included (Klesges et al., 1986) or for broader programs such as those for cardiovascular risk reduction or rehabilitation. This is especially important given the need to reduce the socioeconomic disparities in health surrounding smoking, obesity, physical activity, and the many diseases such as diabetes that are tied to lifestyle risks. Reaching disadvantaged groups is a high priority (Glasgow, Vogt, & Boles, 1999), and including incentives in health-promotion programs may assist in pursuing it. Incentives also may provide incremental utility when added to programs with already documented beneﬁts (e.g., smoking-cessation counseling for pregnant women). When added to otherwise successful interventions, the beneﬁcial effect of incentives may be difﬁcult to detect. This problem is statistical, not conceptual, in nature. It is not that the effect of incentives does not exist when other programs also are being implemented, but rather that the additional effect that is unique to the incentives may be too small to be detected statistically. A key proviso

15-Schachtman-15.indd 335

335

for such research is that it be designed with adequate power to detect what might often be subtle or modest additive effects. Implications for Public Policy

Several aspects of the use of incentives to promote smoking cessation have implications for public policy. As already noted, these programs appear to be most appropriate for high-risk, highpriority, and hard-to-reach patients or for those for whom the importance of smoking cessation is heightened during a deﬁned period of time. The application of incentive-based programs to these groups could be especially effective. Few studies have employed incentive programs with large populations (Morris, Flores, Olinto, & Medina, 2004), but such programs could be especially effective in reaching large groups of people (e.g., all women of childbearing age) as opposed to smaller, more selected populations for whom behavior-change interventions are most often applied. For example, incentive programs might be incorporated within primary care or general health care or ﬁnancing programs, such as Medicaid and Medicare. Behavioral economic considerations of a broad range of incentives and disincentives for healthy behavior have gained increasing attention in discussion of national health care reform. Incorporating a reinforcement and behavioral economic framework in the design and implementation of health interventions offers the promise of bringing to bear the two major ﬁndings of this review: (1) for smoking cessation and other important health behaviors, incentives do work; and (2) consistent with perhaps the most reliable observations in all of psychology, for incentives to have sustained effects, the incentives themselves must be sustained.

ACKNOWLEDGMENTS This chapter is based on a review commissioned by the Robert Wood Johnson Foundation to assist in the development of Foundation-sponsored projects addressing smoking cessation. We gratefully acknowledge the assistance of Daniel Holt in the preparation of that review.

2/15/2011 7:46:43 PM

336

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Preparation of the chapter was supported by the Peers for Progress program of the American Academy of Family Physicians Foundation, supported by the Eli Lilly and Company Foundation, to E. Fisher, and by National Institutes of Health grant MH055308 to L. Green.

REFERENCES Abbott, P. J., Weller, S. B., Delaney, H. D., & Moore, B. A. (1998). Community reinforcement approach in the treatment of opiate addicts. American Journal of Drug and Alcohol Abuse, 24, 17–30. Achat, H., McIntyre, P., & Burgess, M. (1999). Health care incentives in immunization. Australian and New Zealand Journal of Public Health, 23, 285–288. Adams, E. K., Miller, V. P., Ernst, C., Nishimura, B. K., Melvin, C., & Merritt, R. (2002). Neonatal health care costs related to smoking during pregnancy. Health Economics, 11, 193–206. Altman, D. G., Flora, J. A., Fortmann, S. P., & Farquhar, J. W. (1987). The cost-effectiveness of three smoking cessation programs. American Journal of Public Health, 77, 162–165. Audrain-McGovern, J., Rodriguez, D., Tercyak, K. P., Epstein, L. H., Goldman, P., & Wileyto, E. P. (2004). Applying a behavioral economic framework to understanding adolescent smoking. Psychology of Addictive Behaviors, 18, 64–73. Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97. Bickel, W. K., & McLellan, A. T. (1996). Can management by outcome invigorate substance abuse treatment? Micro-incentives for the quality of care. The American Journal on Addictions, 5, 281–291. Biener, L., Aseltine, R. H., Cohen, B., & Anderka, M. (1998). Reactions of adult and teenaged smokers to the Massachusetts Tobacco Tax. American Journal of Public Health, 88, 1389–1391. Bigelow, G. E., Brooner, R. K., & Silverman, K. (1998). Competing motivations: Drug reinforcement vs. non-drug reinforcement. Journal of Psychopharmacology, 12, 8–14. Buchkremer, G., Minneker, E., & Block, M. (1991). Smoking-cessation treatment combining transdermal nicotine substitution with behavioral therapy. Pharmacopsychiatrist, 24, 96–102.

15-Schachtman-15.indd 336

Cahill, K., & Perera, R. (2008). Competitions and incentives for smoking cessation. Cochrane Database of Systematic Reviews, 3, Art. No. CD004307. Cameron, J., Banko, K. M., & Pierce, W. D. (2001). Pervasive negative effects of rewards on intrinsic motivation: The myth continues. The Behavior Analyst, 24, 1–44. Cameron, J., & Pierce, W. D. (1994). Reinforcement, reward, and intrinsic motivation: A meta-analysis. Review of Educational Research, 64, 363–423. Cameron, J., & Pierce, W. D. (2002). Rewards and intrinsic motivation: Resolving the controversy. Westport, CT: Bergin & Garvey. Centers for Disease Control and Prevention (CDC). (1996). Cigarette smoking before and after an excise tax increase and an antismoking campaign - Massachusetts, 1990 - 1996. Morbidity and Mortality Weekly Reports, 45, 966–970. Centers for Disease Control and Prevention (CDC). (2002). Cigarette smoking among adults— United States, 2000. Morbidity and Mortality Weekly Reports, 51, 642–645. Centers for Disease Control and Prevention (CDC). (2008). Current smoking. Retrieved from http:// www.cdc.gov/tobacco/data_statistics/tables/ index.htm Centers for Disease Control Ofﬁce on Smoking and Health. (2007). Cigarette use among high school students–United States, 1991-2007. Morbidity and Mortality Weekly Reports, 57, 689–691. Chutuape, M. A., Silverman, K., & Stitzer, M. (1999a). Contingent reinforcement sustains postdetoxiﬁcation abstinence from multiple drugs: A preliminary study with methadone patients. Drug and Alcohol Dependence, 54, 69–81. Chutuape, M. A., Silverman, K., & Stitzer, M. (1999b). Use of methadone take-home contingencies with persistent opiate and cocaine users. Journal of Substance Abuse Treatment, 16, 23–30. Comer, S. D., Collins, E. D., Wilson, S. T., Donovan, R., Foltin, R. W., & Fischman, M. W. (1998). Effects of an alternative reinforcer on intravenous heroin self-administration by humans. European Journal of Pharmacology, 345, 13–26. The COMMIT Research Group. (1995a). Community intervention trial for smoking cessation (COMMIT): I. Cohort results from a four-year community intervention. American Journal of Public Health, 85, 183–192. The COMMIT Research Group. (1995b). Community intervention trial for smoking

2/15/2011 7:46:43 PM

INCENTIVES IN SMOKING CESSATION

cessation (COMMIT): II. Changes in adult cigarette smoking prevalence. American Journal of Public Health, 85, 193–200. Correia, C. J., & Benson, T. A. (2006). The use of contingency management to reduce cigarette smoking among college students. Experimental and Clinical Psychopharmacology, 14, 171–179. Correia, C. J., Simons, J., Carey, K. B., & Borsari, B. E. (1998). Predicting drug use: Application of behavioral theories of choice. Addictive Behaviors, 23, 705–709. Covey, L. S., Glassman, A. H., & Stetner, F. (1997). Major depression following smoking cessation. American Journal of Psychiatry, 154, 263–265. Crowley, T. J., MacDonald, M. J., Zerbe, G. O., & Petty, T. L. (1991). Reinforcing breath carbon monoxide reductions in chronic obstructive pulmonary disease. Drug and Alcohol Dependence, 29, 47–62. Curry, S., Wagner, E., & Grothaus, L. (1991). Evaluation of intrinsic and extrinsic motivation interventions with a self-help smoking cessation program. Journal of Consulting and Clinical Psychology, 59, 318–324. Dallery, J., Meredith, S., & Glenn, I. M. (2008). A deposit contract method to deliver abstinence reinforcement for cigarette smoking. Journal of Applied Behavior Analysis, 41, 609–615. Donatelle, R. J., Prows, S. L., Champeau, D., & Hudson, D. (2000). Randomized controlled trial using social support and ﬁnancial incentives for high risk pregnant smokers: Signiﬁcant other supporter (SOS) program. Tobacco Control, 9, iii67–iii69. Eisenberger, R., & Cameron, J. (1996). The detrimental effects of reward: Myth or reality? American Psychologist, 51, 1153–1166. Eisenberger, R., Pierce, W. D., & Cameron, J. (1999). Effects of reward on intrinsic motivation— negative, neutral, and positive: Comment on Deci, Koestner, and Ryan (1999). Psychological Bulletin, 125, 677–691. Elk, R., Mangus, L. A., Rhoades, H., Andres, R., & Grabowski, J. (1998). Cessation of cocaine use during pregnancy: Effects of contingency management interventions on maintaining abstinence and complying with prenatal care. Addictive Behaviors, 23, 57–64. El-Khorazaty, M. N., Johnson, A. A., Kiely, M., El-Mohandes, A. A. E., Subramanian, S., Laryea, H. A., Murray, K. B., Thornberry, J. S., & Joseph, J. G. (2007). Recruitment and retention of low-income minority women in a

15-Schachtman-15.indd 337

337

behavioral intervention to reduce smoking, depression, and intimate partner violence during pregnancy. BioMed Central Public Health, 7, 233. Ershoff, D., Quinn, V., Mullen, P., & Lairson, D. (1990). Pregnancy and medical cost outcomes of a self-help prenatal smoking cessation program in an HMO. Public Health Reports, 105, 340–347. Feldman, R., Bacher, M., Campbell, N., Drover, A., & Chockalingman, A. (1998). Adherence to pharmacologic management of hypertension. Canadian Journal of Public Health, 89, I16–I18. Fiore, M. C., Jaén, C. R., Baker, T. B., Bailey, W. C., Benowitz, N. L., Curry, S. J., . . .Wewers, M. E. (2008). Treating tobacco use and dependence: 2008 update. Clinical practice guideline. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service. Fiore, M. C., Smith, S. S., Jorenby, D. E., & Baker, T. B. (1994). The effectiveness of the nicotine patch for smoking cessation. A metaanalysis. Journal of the American Medical Association, 271, 1940–1947. Fisher, E. B. (1979). Overjustiﬁcation effects in token economies: Replicability and the importance of similar, complex functional relations. Journal of Applied Behavior Analysis, 12, 407–415. Fisher, E. B. (1996). A behavioral-economic perspective on the inﬂuence of social support on cigarette smoking. In L. Green & J. H. Kagel (Eds.), Advances in behavioral economics (Vol. 3, pp. 207–236). Norwood, NJ: Ablex. Fisher, E. B., Jr. (1997). Two approaches to social support in smoking cessation: Commodity model and nondirective support. Addictive Behaviors, 22, 819–833. Fisher, E. B., Brownson, R. C., Heath, A. C., Luke, D. A., & Sumner, W. II. (2004). Cigarette smoking. In T. J. Boll, J. Raczynski, & L. Leviton (Eds.), Handbook of clinical health psychology (Vol. 2, pp. 75–120). Washington, DC: American Psychological Association. Fitzgerald, J. M., Patrick, D. M., Strathdee, S., Rekart, M., Elwood, R. K., Schecter, M. T., Montaner, J., & O’Shaughnessy, M. (1999). Use of incentives to increase compliance for TB screening in a population of intravenous drug users. Vancouver injection drug use study group. International Journal of Tuberculosis and Lung Disease, 3, 153–155. Fortman, S. P., & Killen, J. D. (1995). Nicotine gum and self-help behavioral treatment for smoking

2/15/2011 7:46:44 PM

338

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

relapse prevention: Results from a trial using population-based recruitment. Journal of Consulting and Clinical Psychology, 63, 460–468. Gilpin, E. A., & Pierce, J. P. (2002). Demographic differences in patterns of the incidence of smoking cessation: United States 1950-1990. Annals of Epidemiology, 12, 141–150. Giuffrida, A., & Torgerson, D. J. (1997). Should we pay the patient? Review of ﬁnancial incentives to enhance patient compliance. British Medical Journal, 315, 703–707. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322–1327. Green, L., & Freed, D. E. (1993). The substitutability of reinforcers. Journal of the Experimental Analysis of Behavior, 60, 141–158. Green, L., & Freed, D. (1998). Behavioral economics. In W. T. O’Donohue (Ed.), Learning and behavior therapy (pp. 274–300). Boston, MA: Allyn & Bacon. Hamilton, V. H., Levinton, C., St-Pierre, Y., & Grimard, F. (1997). The effect of tobacco tax cuts on cigarette smoking in Canada. Canadian Medical Association Journal, 156, 187–191. Heil, S. H., Higgins, S. T., Berstein, I. M., Solomon, L. J., Rogers, R. E., Thomas, C. S., Badger, G. J., & Lynch, M. E. (2008). Effects of voucher-based incentives on abstinence from cigarette smoking and fetal growth among pregnant women. Addiction, 103, 1009–1018. Henningﬁeld, J. E., & Keenan, R. M. (1993). Nicotine delivery kinetics and abuse liability. Journal of Consulting and Clinical Psychology, 61, 743–750. Henningﬁeld, J. E., Stitzer, M. L., & Grifﬁths, R. R. (1980). Expired air carbon monoxide accumulation and elimination as a function of number of cigarettes smoked. Addictive Behaviors, 5, 265–272. Higgins, S. T. (1997). The inﬂuence of alternative reinforcers on cocaine use and abuse: A brief review. Pharmacology Biochemistry and Behavior, 57, 419–427. Higgins, S. T., Budney, A. J., Bickel, W. K., Foerg, F. E., Donham, R., & Badger, G. J. (1994). Incentives improve outcome in outpatient behavioral treatment of cocaine dependence. Archives of General Psychiatry, 51, 568–576. Higgins, S. T., Heil, S. H., Dantona, R., Donham, R., Matthews, M, & Badger, G. J. (2006). Effects of

15-Schachtman-15.indd 338

varying the monetary value of voucher-based incentives on abstinence achieved during and following treatment among cocaine-dependent outpatients. Addiction, 102, 271–281. Higgins, S. T., Heil, S. H., Solomon, L. J., Bernstein, I. M., Plebani Lussier, J., Abel, R. L., Lynch, M. E., & Badger, G. J. (2004). A pilot study on voucher-based incentives to promote abstinence from cigarette smoking during pregnancy and postpartum. Nicotine and Tobacco Research, 6, 1015–1020. Hu, T. W., Sung, H. Y., & Keeler, T. E. (1995). Reducing cigarette consumption in California: Tobacco taxes vs. an anti-smoking media campaign. American Journal of Public Health, 85, 1218–1222. Hughes, J. R., Hatsukami, D. K., Mitchell, J. E., & Dahlgren, L. A. (1986). Prevalence of smoking among psychiatric outpatients. American Journal of Psychiatry, 143, 993–997. Hunt, G., & Azrin, N. (1973). A community reinforcement approach to alcoholism. Behaviour Research and Therapy, 11, 91–104. Iguchi, M. Y., Belding, M. A., Morral, A. R., Lamb, R. J., & Husband, S. D. (1997). Reinforcing operants other than abstinence in drug abuse treatment: an effective alternative for reducing drug abuse. Journal of Consulting and Clinical Psychology, 65, 412–428. Janz, N. K., Schottenfeld, D., Doerr, K. M., Selig, S. M., Dunn, R. L., Strawderman, M., & Levine, P. A. (1997). A two-step intervention to increase mammography among women aged 65 and older. American Journal of Public Health, 87, 1683–1686. Jason, L., Jayaraj, S., Blitz, C., Michaels, M., & Klett, L. (1990). Incentives and competition in a worksite smoking cessation intervention. American Journal of Public Health, 80, 205–206. Jeffery, R. W., Hellerstedt, W. L., & Schmid, T. L. (1990). Correspondence programs for smoking cessation and weight control: A comparison of two strategies in the Minnesota Heart Health Program. Health Psychology, 9, 585–598. Jeffery, R. W., Wing, R. R., Thorson, C., & Burton, L. R. (1998). Use of personal trainers and ﬁnancial incentives to increase exercise in a behavioral weight-loss program. Journal of Consulting and Clinical Psychology, 66, 777–783. Johnson, M. W., & Bickel, W. K. (2003). The behavioral economics of cigarette smoking: The concurrent presence of a substitute and an independent reinforcer. Behavioral Pharmacology, 14, 137–144.

2/15/2011 7:46:44 PM

INCENTIVES IN SMOKING CESSATION

Jones, H. E., Haug, N., Silverman, K., Stitzer, M., & Svikis, D. (2001). The effectiveness of incentives in enhancing treatment attendance and drug abstinence in methadone-maintained pregnant women. Drug and Alcohol Dependence, 61, 297–306. Jones, H. E., Haug, N. A., Stitzer, M. L., & Svikis, D. S. (2000). Improving drug treatment outcomes for pregnant drug-dependent women using low-magnitude voucher incentives. Addictive Behavior, 25, 263–267. Kagel, J. H., Battalio, R. C., & Green, L. (1995). Economic choice theory. New York, NY: Cambridge University Press. Kagel, J. H., Battalio, R. C., & Miles, C. G. (1980). Marijuana and work performance: Results from an experiment. Journal of Human Resources, 15, 373–395. Kamb, M. L., Rhodes, F., Hoxworth, T., Rogers, J., Lentz, A., Kent, C., MacGowen, R., & Peterman, T. A. (1998). What about money? Effect of small monetary incentives on enrollment, retention, and motivation to change behaviour in an HIV/STD prevention counselling intervention. The project RESPECT study group. Sexually Transmitted Infections, 74, 253–255. Kirby, K. C., Marlowe, D. B., Festinger, D. S., Garvey, K. S., & LaMonica, V. (1999). Community reinforcement training for family and signiﬁcant others of drug abusers: A unilateral intervention to increase treatment entry of drug users. Drug and Alcohol Dependence, 56, 85–96. Kirby, K. C., Marlowe, D. B., Festinger, D. S., Lamb, R. J., & Platt, R. J. (1998). Schedule of voucher delivery inﬂuences cocaine abstinence. Journal of Consulting and Clinical Psychology, 66, 761–767. Klesges, R. C., Vasey, M. W., & Glasgow, R. E. (1986). A worksite smoking modiﬁcation competition: Potential for public health impact. American Journal of Public Health, 76, 198–200. Korhonen, H. J., Niemensivu, H., Piha, T., Koskela, K., Wiio, J., Johnson, C. A., & Puska, P. (1992). National TV smoking cessation program and contest in Finland. Preventive Medicine, 21, 74–87. Lamb, R. J., Morral, A. R., Kirby, K. C., Iguchi, M. Y., Galbicka, G. (2004). Shaping smoking cessation using percentile schedules. Drug and Alcohol Dependence, 76, 247–259. Lando, H. A., Loken, B., Howard-Pitney, B., & Pechacek, T. (1990). Community impact of a

15-Schachtman-15.indd 339

339

localized smoking cessation contest. American Journal of Public Health, 80, 601–603. Lattal, K. A. (1987). Considerations in the experimental analysis of reinforcement delay. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 107–123). Hillsdale, NJ: Erlbaum. LeBaron, C. W., Starnes, D., Dini, E. F., Chambliss, J. W., & Chaney, M. (1998). The impact of interventions by a community-based organization on inner-city vaccination coverage. Archives of Pediatrician Adolescent Medicine, 152, 327–352. Leinweber, C. E., Macdonald, J. M., & Campbell, H. S. (1994). Community smoking cessation contests: An effective public health strategy. Canadian Journal of Public Health, 85, 95–98. Lerman, C., Caporaso, N. E., Audrain, J., Main, D., Bowman, E. D., Lockshin, B., Boyd, N. R., & Shields, P. G. (1999). Evidence suggesting the role of speciﬁc genetic factors in cigarette smoking. Health Psychology, 18, 14–20. LeSage, M. G., Stafford, D., & Glowa, J. R. (1999). Preclinical research on cocaine selfadministration: Environmental determinants and their interaction with pharmacological treatment. Neuroscience and Biobehavioral Reviews, 23, 717–741. Lichtenstein, E., Glasgow, R. E., & Abrams, D. B. (1986). Social support in smoking cessation: In search of effective interventions. Behavior Therapy, 17, 607–619. Maheu, M., Gevirtz, R., Sallis, J., & Schneider, N. (1989). Competition/cooperation in worksite smoking cessation using nicotine gum. Preventive Medicine, 18, 867–876. Malotte, C. K., Hollingshead, J. R., & Rhodes, F. (1999). Monetary versus nonmonetary incentives for TB skin test reading among drug users. American Journal of Preventive Medicine, 16, 182–188. Malotte, C. K., Rhodes, F., & Mais, K. E. (1998). Tuberculosis screening and compliance with return for skin test reading among active drug users. American Journal of Public Health, 88, 792–796. Marcus, A. C., Kaplan, C. P., Crane, L. A., Berek, J. S., Bernstein, G., Gunning, J. E., & McClatchey, M. W. (1998). Reducing loss-to-follow-up among women with abnormal Pap smears. Results from a randomized trial testing an intensive

2/15/2011 7:46:44 PM

340

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

follow-up protocol and economic incentives. Medical Care, 36, 397–410. Marks, J. S., Koplan, J. P., Hogue, C. J. R., & Dalmat, M. E. (1990). A cost-beneﬁt/cost effectiveness analysis of smoking cessation for pregnant women. American Journal of Preventative Medicine, 6, 282–289. Marteau, T. M., Ashcroft, R. E., & Oliver, A. (2009). Using ﬁnancial incentives to achieve healthy behaviour. British Medical Journal, 338, 983–985. Matson, D. M., Lee, J. W., & Hopp, J. W. (1993). The impact of incentives and competitions on participation and quit rates in worksite smoking cessation programs. American Journal of Health Promotion, 7, 270–280, 295. Miles, C. G., Congreve, G. R. S., Gibbins, R. J., Marshman, J., Devenyi, R., & Hicks, R. C. (1974). An experimental study of the effects of daily cannabis smoking on behavior patterns. Acta Pharmacologica et Toxocologica, 34(Suppl. 1), 1–44. Morris, S. S., Flores, R., Olinto, P., Medina, J. M. (2004). Monetary incentives in primary health care and effects on use and coverage of preventative health care interventions in rural Honduras: Cluster randomized trial. Lancet, 364, 2030–2037. Ohsfeldt, R. L., Boyle, R. G., & Capilouto, E. (1997). Effects of tobacco excise taxes on the use of smokeless tobacco products in the USA. Health Economics, 6, 525–531. Orleans, C. S., & Shipley, R. H. (1982). Worksite smoking cessation initiatives: Review and recommendations. Addictive Behaviors, 7, 1–16. Paxton, R. (1980). The effects of a deposit contract as a component in a behavioural programme for stopping smoking. Behaviour Research and Therapy, 18, 45–50. Paxton, R. (1981). Deposit contracts with smokers: Varying frequency and amount of repayments. Behaviour Research and Therapy, 19, 117–123. Paxton, R. (1983). Prolonging the effects of deposit contracts with smokers. Behaviour Research and Therapy, 21, 425–433. Petry, N. M., Alessi, S. M., Marx, J., Austin, M., & Tardif, M. (2005). Vouchers versus prizes: Contingency management treatment of substance abusers in community settings. Journal of Consulting and Clinical Psychology, 73, 1005–1014. Petry, N. M., Martin, B., Cooney, J. L., & Kranzler, H. R. (2000). Give them prizes and they will come: Variable-ratio contingency management for treatment of alcohol dependence.

15-Schachtman-15.indd 340

Journal of Consulting and Clinical Psychology, 70, 250–257. Petry, N. M., Tedford, J., Austin, M., Nich, C., Carroll, K. M., & Rounsaville, B. J. (2004). Prize reinforcement contingency management for treating cocaine users: How low can we go, and with whom? Addiction, 99, 349–360. Pierce, J. P., Gilpin, E. A., Emery, S. L., White, M. M., Rosbrook, B., Berry, C. C., & Farkas, A. J. (1998). Has the California tobacco control program reduced smoking? Journal of the American Medical Association, 280, 893–899. Pirie, P. L., Rooney, B. L., Pechacek, T. F., Lando, H. A., & Schmid, L. A. (1997). Incorporating social support into a community-wide smoking-cessation contest. Addictive Behaviors, 22, 131–137. Plebani Lussier, J., Heil, S. H., Mongeon, J. A., Badger, G. J., & Higgins, S. T. (2006). A metaanalysis of voucher-based reinforcement therapy for substance use disorders. Addiction, 101, 192–203. Pomerleau, O. F., Collins, A. C., Shiffman, S., & Pomerleau, C. S. (1993). Why some people smoke and others do not: New perspectives [review]. Journal of Consulting and Clinical Psychology, 61, 723–731. Prendergast, M., Podus, D., Finney, J., Greenwell, L., & Roll, J. (2006). Contingency management for treatment of substance use disorders: A metaanalysis. Addiction, 101, 1546–1560. Puska, P., Vartiainen, E., Tuomilehto, J., Salomaa, V., & Nissinen, A. (1998). Changes in premature deaths in Finland: Successful long-term prevention of cardiovascular diseases. Bulletin of the World Health Organization, 76, 419–425. Rachlin, H. (1991). Introduction to modern behaviorism (Vol. 3). San Francisco, CA: Freeman. Rachlin, H., & Krasnoff, J. (1983). Eating and drinking: An economic analysis. Journal of the Experimental Analysis of Behavior, 39, 385–404. Rand, C. S., Stitzer, M. L., Bigelow, G. E., & Mead, A. M. (1989). The effects of contingent payment and frequent workplace monitoring on smoking abstinence. Addictive Behaviors, 14, 121–128. Reynolds, B., Dallery, J., Shroff, P., Patak, M., & Leraas, K. (2008). A web-based contingency management program with adolescent smokers. Journal of Applied Behavior Analysis, 41, 597–601. Robles, E., Silverman, K., Preston, K., Cone, E. J., Katz, E., Bigelow, G. E., & Stitzer, M. L. (2000).

2/15/2011 7:46:44 PM

INCENTIVES IN SMOKING CESSATION

The brief abstinence test: Voucher-based reinforcement of cocaine abstinence. Drug and Alcohol Dependence, 58, 205–212. Rogers, R. E., Higgins, S. T., Silverman, K., Thomas, C. S., Badger, G. J., Bigelow, G., & Stitzer, M. (2008). Abstinence-contingent reinforcement and engagement in non-drug-related activities among illicit drug abusers. Psychology of Addictive Behaviors, 22, 544–550. Roll, J. M., Higgins, S. T., & Badger, G. J. (1996). An experimental comparison of three different schedules of reinforcement of drug abstinence using cigarette smoking as an exemplar. Journal of Applied Behavior Analysis, 29, 495–505. Roll, J. M., Higgins, S. T., Steingard, S., & McGinley, M. (1998). Use of monetary reinforcement to reduce the cigarette smoking of persons with schizophrenia: A feasibility study. Experimental and Clinical Psychopharmacology, 6, 157–161. Roll, J. M., Reilly, M. P., & Johanson, C. E. (2000). The inﬂuence of exchange delays on cigarette versus money choice: A laboratory analog of voucher based reinforcement therapy. Experimental and Clinical Psychopharmacology, 8, 366–370. Roll, J. M., & Shoptaw, S. (2006). Contingency management: Schedule effects. Psychiatry Research, 144, 91–93. Seal, K. H., Kral, A. H., Lorvick, J., McNees, A., Gee, L., & Edlin, B. R. (2003). A randomized controlled trial of monetary incentives vs. outreach to enhance adherence to the hepatitis B vaccine series among injection drug users. Drug and Alcohol Dependence, 71, 127–131. Shipley, R. H., Hartwell, T. D., Austin, W. D., Clayton, A. C., & Stanley, L. C. (1995). Community stop-smoking contests in the COMMIT Trial: Relationship of participation to costs. Preventive Medicine, 24, 286–292. Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1997). Reinforcement of cocaine abstinence in treatment-resistant patients: Effects of reinforcer magnitude. NIDA Monograph, 174. Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1999). Voucher-based reinforcement of cocaine abstinence in treatmentresistant methadone patients: Effects of reinforcement magnitude. Psychopharmacology, 146, 128–138. Silverman, K., Svikis, D., Robles, E., Stitzer, M. L., & Bigelow, G. E. (2001). A reinforcement-based therapeutic workplace for the treatment of

15-Schachtman-15.indd 341

341

drug abuse: Six-month abstinence outcomes. Experimental and Clinical Psychopharmacology, 9, 14–23. Silverman, K., Wong, C. J., Higgins, S. T., Brooner, R. K., Montoya, I. D., Contoreggi, C., Umbricht-Schneiter, A., Schuster, C. R., & Preston, K. L. (1996). Increasing opiate abstinence through voucher-based reinforcement therapy. Drug and Alcohol Dependence, 41, 157–165. Smith, J. E., Meyers, R. J., & Delaney, H. D. (1998). The community reinforcement approach with homeless alcohol-dependent individuals. Journal of Consulting and Clinical Psychology, 66, 541–548. Solomon, L. J., Flynn, B. S., Worden, J. K., Mickey, R. M., Skelly, J. M., Geller, B. M., Peluso, N. W., & Webster, J. A. (1998). Assessment of self-reward strategies for maintenance of breast self-examination. Journal of Behavioral Medicine, 21, 83–102. Stitzer, M. L., & Bigelow, G. E. (1982). Contingent reinforcement for reduced carbon monoxide levels in cigarette smokers. Addictive Behaviors, 7, 403–412. Stitzer, M. L., & Bigelow, G. E. (1983). Contingent payment for carbon monoxide reduction: Effects of pay amount. Behavior Therapy, 14, 647–656. Stitzer, M. L., & Bigelow, G. E. (1985). Contingent reinforcement for reduced breath carbon monoxide levels: Target-speciﬁc effects on cigarette smoking. Addictive Behaviors, 10, 345–349. Stitzer, M. L., Rand, C. S., Bigelow, G. E., & Mead, A. M. (1986). Contingent payment procedures for smoking reduction and cessation. Journal of Applied Behavior Analysis, 19, 197–202. Stone, E. G., Morton, S. C., Hulscher, M. E., Maglione, M. A., Roth, E. A., Grimshaw, J. M., . . .Shekelle, P. G. (2002). Interventions that increase use of adult immunization and cancer screening services: A meta-analysis. Annals of Internal Medicine, 136, 641–651. Sutherland, K., Christianson, J. B., & Leatherman, S. (2008). Impact of targeted ﬁnancial incentives on personal health behavior: A review of the literature. Medical Care Research Reviews, 65(Suppl. 6), 36S–78S. Svikis, D. S., Lee, J. H., Haug, N. A., & Stitzer, M. L. (1997). Attendance incentives for outpatient treatment: Effects in methadone- and nonmethadonemaintained pregnant drug dependent women. Drug and Alcohol Dependence, 48, 33–41.

2/15/2011 7:46:44 PM

342

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Tulsky, J. P., Hahn, J. A., Long, H. L., Chambers, D. B., Robertson, M. J., Chesney, M. A., & Moss, A. R. (2004). Can the poor adhere? Incentives for adherence to TB prevention in homeless adults. The International Journal of Tuberculosis and Lung Disease, 8, 83–91. Van Etten, M. L., Higgins S. T., Budney, A. J., & Badger, G. J. (1998). Comparison of the frequency and enjoyability of pleasant events in cocaine abusers vs. non-abusers using a standardized behavioral inventory. Addiction, 93, 1669–1680. Volmink, J., & Garner, P. (1997). Systematic review of randomised controlled trials of strategies to promote adherence to tuberculosis treatment. British Medical Journal, 315, 1403–1406.

15-Schachtman-15.indd 342

Volpp, K. G., Gurmankin Levy, A., Asch, D. A., Berlin, J. A., Murphy, J. J., Gomez, A., Sox, H., Zhu, J., & Lerman, C. (2006). A randomized controlled trial of ﬁnancial incentives for smoking cessation. Cancer Epidemiology, Biomarkers and Prevention, 15, 12–18. Volpp, K. G., Troxel, A. B., Pauly, M. V., Glick, H. A., Puig, A., Asch, D. A., Galvin, R., . . .AudrainMcGovern, J. (2009). A randomized, controlled trial of ﬁnancial incentives for smoking cessation. The New England Journal of Medicine, 360, 699–709. Vuchinich, R. E., & Tucker, J. A. (1988). Contributions from behavioral theories of choice to an analysis of alcohol abuse. Journal of Abnormal Psychology, 97, 181–195.

2/15/2011 7:46:44 PM

PART IV

Applications to Cognition, Social Interaction, and Motivation

16-Schachtman-16.indd 343

2/15/2011 7:46:58 PM

This page intentionally left blank

CHAPTER 16 Social Learning and Connectionism Frank Van Overwalle

This chapter reviews evidence to demonstrate that many judgments and biases in social cognition can be understood from a connectionist perspective. A basic feature of connectionist modelling is that many social judgments are driven by basic associative learning processes, most often by an error-minimizing algorithm as illustrated in the delta learning algorithm. Two major emergent properties falling naturally out from this learning algorithm are acquisition (sample size effects) and competition (discounting and augmentation). These properties are unique to error minimizing algorithms like delta learning. Empirical evidence is reviewed showing that causal en dispositional attributions are strongly determined by these emergent properties. In addition, a number of simulations are reviewed to illustrate that many other social judgments and biases might result from such connectionist learning processes. These simulations include person impression formation, assimilation and contrast, illusory correlations in groups, subtyping of extreme dissidents, cognitive dissonance, attitude formation through persuasive communication, and recent ﬁndings of brain imaging research on person perception. The common theme in this chapter is that a single connectionist learning mechanism—the delta algorithm—is capable of producing emerging properties that explain a rich set of empirical data in social cognition.

INTRODUCTION How might associative learning advance our understanding of social processes? Can a connectionist approach that grew out of and extended classic associative learning tell us something new about social cognition? Social cognition is a subﬁeld of social psychology concerned with the question of how we perceive and interpret the behavior of other human beings in terms of their motives, traits, social constraints, and so on. The aim of this chapter is to demonstrate that social connectionism brings a deeper understanding to the ﬁeld of social cognition. For one thing, connectionist models may provide a common framework for learning about human beings, inspired on the neural working of the brain, that the traditional approach in social psychology lacks and that explains why this ﬁeld is currently replete with many unrelated, fragmentary, and

ad-hoc perspectives and theories. Moreover, given the nascent interest in social neuroscience and the growing empirical evidence on the location and timing of brain activation using novel brain-imaging techniques, there is a need to understand how these processes are shaped in the brain. Given their neurological inspiration, connectionist models may ﬁll this gap. They explain content (what is learned and memorized) and process (how it is learned) by a single mechanism, unlike earlier approaches that often see these aspects as driven by different processes taking place at different stages. Perhaps most fundamentally, connectionism views social cognition as a constant learning and adaptation in a changing environment, and our memories, inferences, and judgments as a natural outcome of that process. This chapter is subdivided into three main sections. The ﬁrst section introduces the basics

345

16-Schachtman-16.indd 345

2/15/2011 7:46:58 PM

346

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of associative or connectionist learning and illustrates how important properties in social reasoning emerge from this. The second section demonstrates how these properties are revealed in empirical research on causal and dispositional attribution that my research team has conducted in the last 10 years. The last section involves simulations of other important empirical ﬁndings in social psychology, to demonstrate the breath and value of the connectionist framework.

WHAT IS CONNECTIONIST LEARNING? In social cognition, information processing is often explained in terms of spreading activation models, in which social concepts such as persons, groups, behaviors, traits, and attitudes are represented by highly interconnected units, and social judgments are explained by the output of spreading activation between these units. In connectionist networks, these units are typically grouped together in layers, most often comprising at least an input layer and an output layer. Figure 16.1A demonstrates this architecture in the most simple model with only forward connections between the input and output layer (i.e., feedforward model), whereas Figure 16.1B demonstrates a more complex model that includes all connections between units, in both a forward and backward order (i.e., recurrent model). To provide some ﬂesh and blood to these rather abstract models, Figure 16.1C depicts a generic example of how such a model might look like in a simulation of social phenomena. As we will see, many simulations in this chapter deal with agents (often involving a target person, and sometimes also another person, object or group, or situational constraints that limit the actions of the target) that produce some effect that can be observed in the agents’ behaviors or other characteristics (e.g., traits, preferences). The connections in this example reﬂect the tendency of the agents to engage in a given behavior or to possess a given trait. Common to all these models in Figure 16.1 is that the strength of the connections determines long-term memory storage and that the ﬂow of activation along the connections reﬂects

16-Schachtman-16.indd 346

processing this information. Consequently, these connectionist models are more powerful than earlier spreading activation models. These earlier models leave the development of connections and the ﬂow of activation often unspeciﬁed. Even in more recent developments of this notion in the format of constraint satisfaction models, the strength of the connections is determined a priori by the researcher and judgment is believed to result solely from how the activation ﬂows and settles in the network. In contrast, the feedforward en recurrent connectionist models shown in Figure 16.1 and applied throughout this chapter not only specify the ﬂow of activation but, more crucially, also include a learning mechanism that determines how the connections between the units can change so that they provide a ﬂexible storage for long-term memory. These two processes are done in parallel by the operation of the interconnected units themselves, so that connectionist systems have no need for a central executive. This capacity to self-learning and self-organization allows the connectionist approach to get rid of the problem of the “homunculus” in the brain that makes our mental decisions. Although these learning processes are, in principle, working to accurately understand the human environment, they sometimes lead astray into biases and shortcomings of social reasoning, some of which we review in this chapter. Learning and Adaptation

Unlike spreading activation models, most connectionist networks are able to learn over time, usually by means of a simple learning algorithm that progressively modiﬁes the strength of the connections between the units making up the network. Learning is modeled as a process of on-line adaptation of existing knowledge to novel information provided by the environment. Speciﬁcally, the network changes the weights of the connections between units (e.g., between an agent and his or her characteristics, or between causes and their effects) and so reﬂects the accumulated history of co-occurrences between these stimuli. Because the weights of connections change slowly, the connections are conservative

2/15/2011 7:46:58 PM

SOCIAL LEARNING AND CONNECTIONISM

347

A. Feedforward

B. Recurrent

Output

Output

Internal input

External input

External input C. Generic Example Effects:

Agents:

Trait

Behavior

Target Person

Other/ Group

Situational Context

Figure 16.1 (A and B) A feedforward and recurrent network. The output layer is depicted at the top, and

the input layer is depicted at the bottom. External activation is fed in the network at the input layer and ﬂows along the connections to the output layer directly (feedforward and recurrent network) or indirectly via internal connections (recurrent network only; additional connections are indicated by dotted arrows). (Reprinted with permission from Figure 3.1 of Van Overwalle, F. 2007. Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press). (C) A generic example of many simulations that are addressed in this chapter, embedded in a feedforward architecture. Most simulations use one or more of the units depicted here. The arrows (i.e., connections) in this example reﬂect the tendency of an agent to engage in a given behavior or to possess a given trait.

and reﬂect past as well as novel co-occurrences. As such, they are the repository of the network’s long-term memory, modiﬁed only in part by recent learning. The model’s memory is retrieved, not by passively reading connection weights, but by reactivating some units of interest by current inputs. As a consequence, retrieval is a reconstructive blend of input as well as a mixture of old and recent information embedded in the weights. The historic predecessors of the connectionist approach, associative learning models, also represent causal strength in memory as an association

16-Schachtman-16.indd 347

between two stimuli and describe how the weight of this association is adjusted on-line as new information is received (for an overview, see Allan, 1993; Shanks, 1995). One of the most popular associative learning models by Rescorla and Wagner (1972) is, in fact, identical to the delta learning algorithm used in many connectionist models (McClelland & Rumelhart, 1988). This delta learning algorithm is used throughout all simulations in this chapter. Given that associative learning models were designed to explain also animal learning and conditioning, they provide the connectionist perspective with an

2/15/2011 7:46:58 PM

348

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

evolutionary perceptive and an extensive research base from which social researchers can draw a lot of interesting data. Thus, the phenomena presented in this chapter are naturally developed from old learning or conditioning processes that other organisms besides humans also exhibit.

more fundamental associative learning processes. We discuss two properties in more detail, because they form the basis of all empirical studies and simulations in this chapter.

EMERGENT PROPERTIES OF ACQUISITION AND COMPETITION

We have seen that, because a connectionist network gradually reduces errors, it slowly approaches the statically predicted weight between input and output, or covariation. Consider a simple case in which cause A is repeatedly presented with effect E (see Fig. 16.2A). In the network, the activation of each unit is turned on (set to +1) and activation spreads from A to E. Because initially, the A→E connection or causal strength is 0, cause A does not predict effect E at all and there is a large error. This error is gradually reduced each time A and E are presented together, so that the A→E weight increases slowly and converges toward the statistical norm of +1, at which point cause A fully explains or predicts the effect E. In contrast, when A is no longer followed by the effect (e.g., see A° from trial 9 onward), then its weight starts to decrease and converge toward 0, at which point A° does not explain or predict the effect. Thus, the network eventually learns the best weight of the connections that predict most accurately when and to what degree an output (e.g., effect E) will occur when an input (e.g., cause A) is present. This process is consistent with our intuitions and experimental evidence. Often, we do not jump immediately to conclusions; rather, we build on several experiences to shape our judgments and estimates. This property of gradual acquisition, which slowly strengthens our judgments, is also known as the sample size effect. An effect of sample size has been documented in many areas of social judgment. For instance, when receiving more supportive information, people tend to hold more extreme impressions about other persons (Anderson, 1967, 1981), make more polarized group decisions (Ebbesen & Bowers, 1974; Fiedler, 1996), endorse hypotheses more ﬁrmly (Fiedler, Walther, & Nickel, 1999), make more extreme predictions (Manis, Dovalina, Avis, & Cardoze, 1980), agree more

The delta algorithm represents learning as an adaption of the internal representation of the mental system to match the external environment. This internal representation or internal activation is determined by the activation ﬂow in the network. Speciﬁcally, all interconnected units sent activation to each other in proportion to the weight of their connections, and the received activations are accumulated (e.g., summed) in each unit. This accumulated internal activation is compared with the external activation received from the outside environment. Differences between these two activation levels reﬂect errors in the system’s representation, and they are gradually reduced to reach a more accurate representation. This error reduction is determined by a learning rate that controls the speed of learning (typically around .10 in symbolic representation architectures where each unit represents a meaningful high-level concept like person, trait, etc.). Although the delta learning algorithm has no other goal than error minimization, given sufﬁcient learning experiences, the weights between input and output converge to the probabilistic norm of contingency or covariation (see Chapman & Robbins, 1990; Van Overwalle, 1996). Thus, by reducing error gradually, the delta learning algorithm is sensitive to the accurate covariation between stimuli, such as causes and effects. Gradual learning and error minimization is crucial, because otherwise new learning would erase old memories dramatically (a phenomenon known as catastrophic interference; McCloskey & Cohen, 1989), so that covariation estimation over longer time periods would be impossible. By learning in this manner, the delta learning algorithm reveals other properties that can explain social human behavior on the basis of

16-Schachtman-16.indd 348

Acquisition and Sample Size

2/15/2011 7:46:58 PM

SOCIAL LEARNING AND CONNECTIONISM A. Acquisition

349

Causal strength

1.0

E

A

A

0.8 0.6 0.4 0.2 A° 0.0 0

B. Competition

8 Trials

12

16

B

Causal strength

1.0

E

A

4

A

0.8 0.6 0.4

B

0.2 0.0 0

4

8 Trials

12

16

Figure 16.2 (A) Example of acquisition of cause A. (B) Example of competition between a stronger cause A and a weaker cause B. Gray denotes an active unit. Full lines denote strong connectionist weights, and broken lines denote weaker connection weights. A° denotes that cause A does not co-occur any more with the effect E (i.e., E is not activated any more during these trials). The learning rate in these examples is .20. (Reprinted with permission from Figure 3.2 of Van Overwalle, F. 2007. Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press).

with persuasive messages (Petty & Cacioppo, 1984), and make more extreme causal or dispositional judgments (Baker, Berbier, & ValléeTourangeau, 1989; Shanks, 1985, 1995; Shanks, Lopez, Darby, & Dickinson, 1996; Van Overwalle, 2003; Van Overwalle & Van Rooy, 2001a). Competition, Discounting, and Augmentation

Discounting is a general tendency where “the role of a given cause in producing a given effect is discounted if other plausible causes are also present” (Kelley, 1971, p. 8). One of the most common examples of discounting in social cognition is when internal attributions to the actor are discounted given evidence on the potent inﬂuence of external pressures. The opposite tendency is described in the augmentation principle. This principle speciﬁes that “if for a given

16-Schachtman-16.indd 349

effect, both a plausible inhibitory cause and a plausible facilitatory cause are present, the role of the facilitative cause in producing the effect will be judged greater” (Kelley, 1971, p. 12). For instance, a person’s success is more strongly attributed to internal capacities when the task was hard rather than easy (for an overview, see McClure, 1998). In a connectionist network, discounting and augmentation are natural consequences of the emergent property of competition. The term “competition” stems from the associative learning literature (Rescorla & Wagner, 1972). One prime example of discounting in the associative literature is blocking. Blocking predicts that when one stimulus already accurately predicts an effect through a strong A→E connection, then the development of additional connections of other stimuli with E is blocked. The reason is that prediction in the system is

2/15/2011 7:46:58 PM

350

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

determined by the summed activations from all causes. The accurate prediction from the strong A→E connection leaves no error in the system, so that no further learning takes place. As another example, Figure 16.2B depicts a type of overshadowing from the conditioning literature. In this example, cause A is always presented with the effect E, either alone or in compound with cause B. Speciﬁcally, while cause A always co-occurs with the effect E, cause B co-occurs only in half of the cases. Given the summed activations from both causes when the compound AB is presented, at a certain moment, the effect E is overestimated (in trial 10 the internal activation sums to 1.08), resulting in a downward adjustment of the connection weights. In the long run, the decrement is most detrimental for the weaker cause B, while the strength of A slowly increases at the expense of B (i.e., A overshadows B). Competition can be understood as though the two explanations compete for the available strength, which is limited to +1 or the maximum magnitude of the effect to explain. The opposite effect of augmentation is also a direct consequence of the competition property and is also known as superconditioning in the associative learning literature. When one of two causes is inhibitory, then the other is cause is highly facilitatory in order to compensate for the inhibitory effect. That is, when there is a negative A→E connection, given that the summed activations from both causes predict the effect, then the cause B develops a strong positive connection with E so that an accurate joint prediction of E can be reached. The notion of competition is consistent with the tendency of people to prefer a simple, single explanation. Competition is a robust ﬁnding in empirical research on human causal attribution (Hansen & Hall, 1985; Kruglanski, Schwartz, Maides, & Hamel, 1978; Shanks, 1985; Van Overwalle & Van Rooy, 2001b; Wells & Ronis, 1982) and impression formation (Gilbert & Malone, 1995; Trope & Gaunt, 2000; Van Overwalle, 2006). Besides stating that competition exists, many earlier models in social cognition are incapable of providing a processing mechanism for it (e.g., Kelley, 1971). And even though a Hebbian connectionist learning algorithm is capable of

16-Schachtman-16.indd 350

simulating sample size differences in person and group impressions (Kashima & Kerekes, 1994; Kashima, Woolcock, & Kashima, 2000), it cannot produce competition. Because connection weights in the Hebbian algorithm depend on the accumulated number of co-occurrences of stimuli without any upper bound that limits the predicted output, it does not incorporate a competition property. Other Emergent Properties

Intriguingly, the properties we just discussed are not directly built into the model or delta learning algorithm; rather, they fall naturally out of speciﬁc learning histories and are therefore termed emergent. That is, in the model there are no a priori built-in weights or other parameters that weaken or strengthen the weights of some units, but not others. For instance, the effect of sample size depends solely on an increasing number of learning experiences. Likewise, the competition effect follows from a learning history involving strong and weak connections. This contrasts with less powerful models such as constraint satisfaction, which often require a priori built-in inhibitory connections between stimuli to produce competition between them (e.g., Read & Miller, 1993, p. 535). There are a number of other emergent properties. Information in connectionist models can be represented at a symbolic level (i.e., localist representation), where each unit represents a meaningful concept (e.g., person, cause, trait, and so on). Conversely, information can also be represented at a lower, subsymbolic level, where each unit has no symbolic meaning as such but instead represents features of it (e.g., parts of characters or letters from verbal input, perceptual features from visual input). Consequently, each symbolic concept is reﬂected by a pattern of activation across a large set of units representing subsymbolic features (i.e., distributed representation). This makes connectionist models even more powerful. A distributed representation solves the problem of how new concepts are represented in the system—not by new units but by redeploying existing units. This also leads to a number of other emergent properties such as the

2/15/2011 7:46:59 PM

SOCIAL LEARNING AND CONNECTIONISM

ability to extract prototypes from a number of exemplars because these exemplars share prototypical features (prototype extraction), to recognize exemplars based on the observation of incomplete features because features are strongly interconnected so that missing information is “ﬁlled in” by activation ﬂow (pattern completion), to generalize knowledge about features to similar exemplars because these new exemplars share similar features (generalization), and to lose stored knowledge only partially after damage because each concept is represented by a large amount of features, some of which may even be redundant (graceful degradation; for an accessible introduction, see McLeod, Plunkett, & Rolls, 1998). To keep things simple and easy understandable, however, this chapter avoids a distributed representation. We focus on a localist representation and the properties that emerge from this alone, namely, the properties of acquisition and competition. Given that earlier models in social cognition have difﬁculty explaining these properties, exploring these properties in social judgments can convincingly demonstrate that connectionist principles also underlie social cognition and thinking. In the next section, we turn to a series of studies that tested whether social judgments are consistent with these connectionist properties.

STUDIES EXPLORING ACQUISITION AND COMPETITION IN SOCIAL JUDGMENTS Do people obey the property of acquisition (sample size) and competition (discounting) when they make judgments of social events? Most studies with humans typically used experimental tasks that contained little social material to demonstrate an effect of sample size (Baker et al., 1989; Shanks, 1985, 1987, 1995; Shanks et al., 1996) and competition (Shanks, 1985; Van Hamme, 1994; Williams & Docking, 1995). It is thus necessary to test the robustness of these effects in the social domain. We tested these predictions in our lab using social judgments and compared the obtained results with connectionist

16-Schachtman-16.indd 351

351

simulations of the experimental manipulations. Because these studies focused on empirical replication, we used the simplest, feedforward network for these simulations. We directed our attention to two types of judgments that dominate the social cognition literature, in particular, causal and dispositional attributions. What are these social judgments? Causal and Dispositional Attributions Inferred from Covariation Information

When trying to understand an actor’s behavior (e.g., Sally stepped on Peter’s shoes during the foxtrot), observers often seek the cause of the behavior. These causes can involve temporary and speciﬁc aspects of the actor (e.g., Sally did not pay attention); of the object—in this case, the other person (e.g., Peter made a silly remark); or of the situation (e.g., the dancing hall was too crowded). They can also refer to enduring or general dispositional traits of the actor (e.g., Sally is clumsy), of the other person (e.g., Peter is nasty), or of the situation (e.g., the foxtrot is difﬁcult). Whereas causal attributions refer to all possible explanations, dispositional attributions refer to the subset of causes involving only enduring traits of a person. This distinction is important. Although resulting from a similar process, attributions to traits versus mere causes (i.e., that are no traits) have very different social consequences. For instance, trait attributions have strong implications on how we evaluate the other person and future interactions with him or her, whereas situational explanations have fewer consequences for the actor. Given the great variety of possible causes, however, research investigating causal attributions often limits possible answers to general categories, phrased as “something about” the person, the object, or the situation (Kelley, 1967). This is also how it was done in our studies. Other research investigates person impressions (Anderson, 1981). This typically involves general evaluative impression of people (e.g., are they likeable?) based on their behaviors or trait information. In person impression research, these behavioral or trait descriptions refer uniquely

2/15/2011 7:46:59 PM

352

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and directly to the actor, so that the causal role of the actor is taken for granted. In contrast, in attribution research, observers have to infer the actor’s potential causal role from relevant covariation information. In our research presented in this chapter, we focus on consensus and distinctiveness covariation information (Kelley, 1967). In one of the most inﬂuential theories of causal attribution in social psychology, Kelley proposed that these are among the prime social sources from which people infer covariation. Consensus refers to the extent to which behaviors or outcomes of an actor generalize to other, similar actors, whereas distinctiveness refers to the extent to which outcomes given an object do not generalize to other, similar objects. High covariation of an actor is implied given low consensus (i.e., only this actor behaved in this manner; e.g., Sally stepped on Peter’s shoes whereas others did not) and this leads to strong attributions to the actor (e.g., Sally). Similarly, high covariation of an object is implied given high distinctiveness (i.e., the behavior occurred only with this object; e.g., Sally stepped on Peter’s shoes but not those of other dancers) and leads to strong attributions to the object (e.g., Peter). In contrast, low covariation is implied given the reversed patterns of high consensus or low distinctiveness. That is, high consensus (i.e., everybody behaved in this manner; e.g., Sally and many others stepped on their partners’ shoes) implies little actor causality. Low distinctiveness (the behavior generalized across stimuli; e.g., Sally stepped on Peter’s shoes and those of all her dance partners) implies low object causality. Sample Size in Causal Attribution

While many connectionist models predict an effect of sample size, surprisingly, most attribution studies and models in social cognition around the turn of the millennium ignored the question of sample size (e.g., Cheng & Novick, 1990; Försterling, 1989; Hewstone & Jaspars, 1987; Hilton & Slugoski, 1986; Kelley, 1967; Orvis, Cunningham, & Kelley, 1975; Read & Marcus-Newhall, 1993; but see Försterling, 1992). These models took a statistical approach

16-Schachtman-16.indd 352

to causal learning and accounted only for the ﬁnal end result of the process. They implied that the number of observations should not affect people’s causal estimates. Some models attempted to overcome this limitation by incorporating an updating rule that made them sensitive to sample size (Busemeyer, 1991; Hogarth & Einhorn, 1992). However, a major restriction was that the proposed rule involved only a single cause and did not take into account the inﬂuence of alternative causes (e.g., competition). Given a single cause, these models are actually mathematically identical to the delta algorithm (Wasserman, Kao, Van Hamme, Katagiri, & Young, 1996; Appendix D) and are therefore not considered here. In sum, whereas earlier prominent statistics-based models of causality predict no effect of sample size, connectionist models predict that subjects incrementally adjust social ratings in the direction of the true covariation between stimulus and effect, the more observations are made. In one of our earliest studies (Van Overwalle & Van Rooy, 2001a), we addressed the question of sample size in causal attribution. Our basic idea was simple. We repeated information on the covariation between a cause and an effect (see Fig. 16.2A), without changing the level of covariation. By simply providing the same information repeatedly or only once, this design allows analyzing the effect of sample size while controlling for covariation. In an early acquisition experiment (Van Overwalle & Van Rooy, 2001a, Experiment 1), participants (n = 97) received two different levels of covariation (0% or 100%) by manipulating consensus and distinctiveness, and this was repeated across six blocks of trials. To illustrate with the distinctiveness manipulation, the participants read information that “Jasmine deceived her friend, Corinne.” Low (0%) covariation of the target object, Corinne, was obtained when the outcome of a comparison object was identical: “Jasmine deceived her friend, Karen”; and high (100%) covariation was obtained when the outcome was absent: “Jasmine did NOT deceive her friend, Karen.” Each trial was displayed consecutively on a separate screen in a random order for each participant. After this block of

2/15/2011 7:46:59 PM

SOCIAL LEARNING AND CONNECTIONISM

16-Schachtman-16.indd 353

70

100%

60 50 40 30

0%

(

Attribution Rating Means, ‘ ’ Simulation)

80

20

2

4

6

8

10

12

Trial

Attribution Rating Means, ‘ ’ Simulation)

80 70

(

two trials, the participants rated the causal inﬂuence of the target. For instance, they rated how much inﬂuence “something special about Corinne” had on the outcome, using an 11-point rating scale ranging from 0 (absolutely no inﬂuence) to 100 (very strong inﬂuence), with midpoint 50 (partial inﬂuence). This was repeated for each block involving the same target object (i.e., Corinne) and a novel comparison object (i.e., a novel name instead of Karen). The manipulation of consensus was similar, and it involved the presentation of the same target actor and different comparison actors (i.e., covariation of the agent) across blocks. The data, collapsed across consensus and distinctiveness information conditions, are depicted in Figure 16.3 (top panel). Consistent with the sample size predictions, the ratings show a steady increase over trials in the 100% covariation condition and a steady decrease over trials in the 0% condition. Consistent with a delta error-correcting algorithm, the curves in the ﬁgure depict a linear trend indicating a signiﬁcant increase or decrease across trials, together with a smaller quadratic trend indicating that this linear change becomes smaller toward the end (i.e., showing less learning closer to asymptote when the error reaches zero, a pattern that reﬂects a kind of error-minimizing process predicted by the delta algorithm). Note in this and subsequent studies that judgments start off at midrange scale values for the ﬁrst ratings, which might be plausible given the little information participants have at that moment. We saw in this experiment that observers progressively adjusted their causal ratings when subsequent covariation information conﬁrmed their initial causal judgments. A different question is whether they would also adjust their judgments when subsequent information conﬂicts with their initial judgments, and how far this correction would go. This is an important question, because normatively, causal ratings should return to baseline when they are contradicted by the same amount of novel data. This question was addressed in a second acquisition experiment (Van Overwalle & Van Rooy, 2001a, Experiment 3). Participants (n = 101) were given an initial block of four trials in which a given

353

20

100% 50%

60 50 40

0%

30 Control 4

5

6

7

8

Trial

Figure 16.3 Causal attribution ratings (indicated by full lines) and feedforward simulation (indicated by dotted lines) in function of covariation level and number of trials. Covariation is denoted as 100% (= Low Consensus/High Distinctiveness), 50% (= Midrange Consensus/Distinctiveness), or 0% (= High Consensus/Low Distinctiveness). (Reprinted with permission from Figures 1 and 3 of Van Overwalle, F., & Van Rooy, D. 2001a. When more observations are better than less: A connectionist account of the acquisition of causal strength. European Journal of Social Psychology, 31, 155–175).

target always covaried with the outcome (to built up sufﬁcient causal strength), and then received information in a second block of four trials where the same amount of 100% covariation was given (conﬁrming initial judgments) or was lowered to 50% (partly disconﬁrming initial judgments) or to 0% (entirely disconﬁrming initial judgments). The latter 0% condition was compared with a control baseline condition in which

2/15/2011 7:46:59 PM

354

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

subjects received 0% covariation all the time. As depicted in Figure 16.3 (bottom panel), the results conﬁrm the connectionist predictions. The ratings show an increase over trials in the 100% condition, a marginal decrease in the 50% condition, and a substantial decrease in the disconﬁrming 0% condition, which closely approaches the control condition at the last trial. This latter result conﬁrms that ratings are brought back at the baseline level, when initial impressions are immediately contradicted. The jagged-like pattern of the 50% condition is due to the fact that 50% covariation was reached on even trials only. Again, all the conditions revealed a signiﬁcant linear increase or decrease of the target ratings, and also a marginal to signiﬁcant quadratic trend indicating that this change became smaller at the end. To evaluate how closely a connectionist framework can approximate our data, we ran simulations using a feedforward network, with exactly the same input information and order of trials as in the experiments (Van Overwalle & Van Rooy, 2001a). The network consisted of a target unit and a comparison unit representing either agents (for consensus) or objects (for distinctiveness), and an outcome unit. When a target or comparison agent/object was present, the unit was turned on (activation = +1) and when absent, the unit was turned off (activation = 0). Similarly, when the outcome was obtained, the activation was set to +1, and when the outcome was absent, the activation was set to 0. The connections were adjusted after each trial. The target→outcome connection represents the causal inﬂuence of the target in producing the outcome. Note that in this and all other simulations in this chapter, to visually match the simulation results with the observed data, the simulated values were regressed on the observed ratings. Of importance are not the exact simulation values, but their pattern. As can be seen in Figure 16.3, these simulations closely match the observed data. This is conﬁrmed by the high overall correlations between empirical and simulated data; r = .99 for the ﬁrst and second experiment. As a way of comparison, we also ran simulations of two major statistical models of Cheng and Novick

16-Schachtman-16.indd 354

(1990) and Försterling (1992), extended with additional parameters that take into account sample size. We did our very best to come up with extensions that were reasonable and effective. Nonetheless, the ﬁt was generally poor for these two statistical models, because some data sets revealed zero correlations. The reason is that these sample size extensions are artiﬁcial additions to these models and are not an inherent part of it, as is the case for connectionist models. Sample Size in Dispositional Attribution

In a subsequent acquisition experiment, Van Overwalle (2003; Experiment 2) explored whether the sample size effect would also be revealed for dispositional attributions. As in the previous experiments, participants were given high or low levels of covariation, and this information was repeated for six blocks while keeping the level of covariation constant. To obtain strong and unbiased manipulations of dispositional attributions, consensus and distinctiveness were manipulated simultaneously (see Van Overwalle (1997), resulting in either 100% covariation for the target person (“Target” condition; e.g., only Jasmine and nobody else cheats all her friends), 100% for the other comparison person (“Other” condition; e.g., everybody cheats only Corinne and no other friends), or intermediate covariation levels where covariation with the actor and object are both high (“Both” condition reﬂecting an interaction between persons; e.g., only Jasmine cheats Corinne and nobody else) or both low (“None” condition; e.g., everybody cheats everybody else). As before, each trial was displayed consecutively on a separate screen in a random order for each participant. The acquisition of dispositional attributions was monitored by requesting dispositional trait ratings after each block of trials. For instance, the participants had to judge “to what extent is Jasmine untrustworthy” on an 11-point scale (0 = not at all untrustworthy to 10 = very much untrustworthy), and they also had to judge “to what extent is Corinne naive” on a similar 11-point scale (0 = not at all naive to 10 = very much naive). The results are depicted in Figure 16.4 (top panel) and demonstrate that dispositional

2/15/2011 7:46:59 PM

SOCIAL LEARNING AND CONNECTIONISM 9

Target Dispostion

8 Target 7 None 6 5

Both

4 3

Other 1

2

3

4

5

6

Block - Repeated 9

Target Dispostion

8

Target

Target 7

None

None 6 Both 5

Both Other

4 3

1

2

Other

3

4

5

6

Block - Reversed

Figure 16.4 Dispositional ratings of the target in function of covariation pattern. (Top) Repeated condition. (Bottom) Reversed condition. Covariation is denoted as target (100% for the target person), other (100% for the comparison person), none (0% for both target and comparison persons), or both (100% for both). (Reprinted with permission from Figure 2 of Van Overwalle, F. 2006. Acquisition of dispositional attributions: Effects of sample size and covariation. European Journal of Social Psychology, 33, 515–533).

attributions are sensitive to sample size in line with connectionist predictions. There is a steady increase when covariation is 100% for the target (see line on the top denoted “Target”), and there is a steady decrease when covariation is 100% for the other comparison person (see line on the bottom denoted “Other”). For instance, when only Jasmine cheats her friends (100% “target” covariation), participants describe her as increasingly untrustworthy, whereas if everyone cheats

16-Schachtman-16.indd 355

355

Corinne and nobody else (100% “other” covariation), Jasmine is seen as decreasingly untrustworthy. This linear trend was signiﬁcant in both conditions. In contrast, for the intermediate conditions, dispositional judgments remained relatively ﬂat and ended at an intermediate level, also in line with connectionist predictions. There was a signiﬁcant linear decrease when both factors covaried (see line denoted “Both”), but only a marginal linear trend when none of them covaried with the outcome (see line denoted “None”). I simulated these manipulations using a feedforward network and using exactly the same architecture, information input, activation settings, and order of trials as in the experiment. However, because the two information conditions were manipulated simultaneously, there were now separate units representing the actor (for consensus) and the object (for distinctiveness) and their respective comparison cases. The simulations showed a strong ﬁt with the empirical data with a correlation of r = .91. The extended statistical models described earlier, obtained much lower correlations of r = .37 (Cheng & Novick, 1990) and r = .24 (Försterling, 1992). Van Overwalle (2003; Experiment 2) also ran another condition in which the original covariation was reversed and disconﬁrmed in the second half of the experiment. Normatively, this should lead to a null effect at the end. As can be seen in Figure 16.4 (bottom panel), this prediction was supported because there were no signiﬁcant differences between all four conditions after the last block. Crucially, in line with an acquisition property, the ratings ﬁrst increase or decrease in the ﬁrst half of the experiment, and then return to their baseline in the second half. Discounting and Augmentation in Causal Attribution

Now that we have seen that causal and dispositional attributions obey the acquisition property emerging from connectionist learning, would they also reveal the competition properties of discounting and augmentation? If the number of observations for a competing explanation

2/15/2011 7:46:59 PM

356

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

goes up, would a target explanation be more discounted or augmented? It seems intuitively plausible that, for instance, when there is growing evidence indicating that a successful task was easy, then the person’s ability is more discounted. Conversely, the greater the evidence that the task was hard, the more the person’s ability is augmented. As indicated earlier, most learning or attribution models in psychology fail to incorporate competition, especially when combined with the property of acquisition. In contrast, the connectionist delta algorithm predicts that competition against a target cause should become stronger whenever the alternative cause becomes more facilitatory (leading to increased discounting) or more inhibitory (leading to increased augmentation). To test this connectionist prediction, Van Overwalle and Van Rooy (2001b) capitalized on the earlier sample size manipulation. They induced changes in the discounting and augmentation of a target cause by varying the number of observations (or sample size) of the alternative cause, while keeping its degree of covariation constant. For instance, consider that Theo has several tennis team players of whom ﬁve (large size) or only one (small size) won a singles tennis match earlier on. Next, Theo and his team players win a series of doubles matches. The contribution of Theo tends to be discounted given the earlier win of his team player(s), and especially so when ﬁve rather than one team player won, because in the former case one’s conﬁdence in the tennis talents of the team players is much higher. Conversely, consider that Theo’s team players lost either ﬁve singles matches (large size) or one singles match (small size) before the successful doubles matches with Theo. In that case, Theo’s contribution for winning the doubles matches is augmented, and even more so when ﬁve rather than one team players lost their singles match, because in the former case one is much more certain about the team players’ poor tennis talents. In sum, competition effects are increased given a large rather than a small sample size of the competing cause. In a ﬁrst competition experiment (Van Overwalle & Van Rooy, 2001b, Experiment 2), participants (n = 115) received information about

16-Schachtman-16.indd 356

the outcomes of an alternative cause and the joint outcome of a target and alternative cause, much like in the example of Theo. Interestingly, this information was presented not only sequentially (i.e., one trial after the other) as in the previous acquisition experiments but also in a summary format during one screen shot. Thus, for instance, discounting of one (versus ﬁve) stimulus was described in the summary format “Annie and one (ﬁve) other salesgirl(s) attained high sales ﬁgures for perfumes,” whereas in the sequential format this information was presented trial by trial for each salesgirl separately. Both formats have ecological validity. A sequential format resembles how perceivers receive information from direct observation, whereas a summary format resembles how perceivers pick up information during social conversation. After all information was provided on a target person, participants rated how much inﬂuence “something special about [the target]” had on the outcome, using an 11-point rating scale ranging from 0 (absolutely no inﬂuence) to 100 (very strong inﬂuence), with midpoint 50 (partial inﬂuence). The results are depicted in Figure 16.5 (top panel). Consistent with a connectionist perspective, in comparison with a small sample size of the alternative cause, given a large size, the target cause is signiﬁcantly more augmented or discounted. This is the case for both a sequential and a summary format (although in the sequential format, some amount of competition is already seen given a small size). Connectionist simulations using a feedforward network and using exactly the same input and order of trials as in the experiment show a strong ﬁt with the empirical data with a correlation of r = 1.00 with either the discounting or augmentation data. The extended statistical models described earlier obtained much lower mean correlations of r = .50 (Cheng & Novick, 1990) and r = .00 (Försterling, 1992). Discounting and Augmentation in Dispositional Attribution

Van Overwalle (2006; Experiment 2) investigated whether and how discounting and augmentation of dispositional and causal attributions differ.

2/15/2011 7:46:59 PM

SOCIAL LEARNING AND CONNECTIONISM

Target Cause

90

Summary

70 Augmentation Discounting

50

30

9

Target Cause

Sequential

Small

Large

Small

Dispositions

Large

Causes

7

Augmentation Discounting

5

3 Small

Large

Small

Large

Figure 16.5 Dispositional and causal ratings after

discounting and augmentation as a function of a small or large sample size. (Top) Causal attributions given a sequential trial-by-trial or summary format. (Reprinted and adapted with permission from Figure 3 of Van Overwalle, F., & Van Rooy, D. 2001b. How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626) (Bottom) Dispositional and causal attributions. (Reprinted with permission from Figure 2 of Van Overwalle, F. 2006. Discounting and augmentation of dispositional and causal attributions. Psychologica Belgica, 46, 211–234).

As in the previous experiment (see top panel of Fig. 16.5), the strength of a causal or dispositional attribution to a target actor (or object) was varied by manipulating the number of observations (i.e., sample size) of an alternative actor (or object). Participants (n = 126) viewed this material presented in a sequential format and then rated each actor on causal and dispositional ratings as described earlier. The results are depicted in Figure 16.5 (bottom panel) and

16-Schachtman-16.indd 357

357

indicate that a greater sample size of the alternative actor (or object) resulted in greater discounting or augmentation of the target, and that this effect was alike for causal and dispositional attributions. Again, these results are consistent with a connectionist approach. Although the results thus far indicate that the competition effect is a robust phenomenon, an important limitation is that the information was relatively well structured. In all previous experiments, information on a speciﬁc target actor always appeared in one or several successive blocks of trials, while information on other target actors appeared in other successive blocks. The trials were randomized only within each block. Would these competition effects survive when information about the target is presented in a much less structured and chaotic manner, as in real life where we pick up information piece by piece about varying persons at different moments in time? Interestingly, in a follow-up experiment, Van Overwalle (2006; Experiment 3) made the encoding of this information more difﬁcult and ecologically realistic by randomly shufﬂing all trial information on all stories of all target causes of the previous experiment, and presenting all this information in a single block before ratings were made. If, as suggested by a connectionist approach, dispositions and causes are developed on-line rather than by explicitly estimating frequencies and making calculations on them as statistical models suggest, then the effect of sample size and competition should also appear when it is much more difﬁcult to extract these frequencies. The results showed that discounting and augmentation were revealed for causal attributions, but not for dispositional attributions. In contrast to the pattern revealed in Figure 16.5 (bottom left), in this latter case, the lines representing discounting and augmentation were essentially ﬂat and overlapped each other. This is an unexpected result. One potential explanation for the lack of competition between dispositional attributions under difﬁcult encoding conditions is the fundamental attribution error. This bias indicates that in explaining someone’s behavior, perceivers often emphasize too much an actor’s dispositions and ignore situational information

2/15/2011 7:46:59 PM

358

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(e.g., competing factors). Research has shown that this occurs most often under processing constraints (Gilbert & Malone, 1995; Trope & Gaunt, 2000). Thus, the processing difﬁculty in this experiment may have led participants to focus more on trait-relevant information while ignoring (competing) situational information. However, this argument in itself does not explain why this attribution error occurred only for dispositional inferences and not for causal attributions. Perhaps, adjusting dispositional inferences in the light of alternative situational explanations might consume more cognitive effort than causal attributions, so that they are more vulnerable to manipulations that render the extraction of information more difﬁcult. Is there any ground for such argument? There is. Hilton, Smith, and Kim (1995) and Van Overwalle (1997) found that because dispositions refer to stable and enduring characteristics, perceivers rely more on Kelley’s (1967) covariation evidence that reﬂects generalization across comparison cases such as low distinctiveness (i.e., identical behaviors across different situations). In contrast, for causal attributions, they rely more on differences such as low consensus (i.e., differences between actors’ behaviors). By relying more on the generalization of a single actor’s behavior across different situations for making dispositional attributions and less on differences between situations, attention to situationspeciﬁc information may have been limited so that competition failed.

SIMULATIONS OF SOCIAL JUDGMENTS AND BIASES Having demonstrated that connectionist networks provide a valid framework to account for various social judgments like causal and dispositional attributions, we now turn to a series of simulations in which the connectionist approach was further explored for a number of important phenomena and ﬁndings in the social cognition literature. By exploring these major ﬁndings and modeling them from a connectionist perspective, the scope and usefulness of this framework can be fully appreciated. I begin where we left off in the previous section, that is, with person

16-Schachtman-16.indd 358

impression formation. Next I turn to biases in group judgments and then discuss attitude formation. Finally, I end with a simulation of recent social neuroimaging data. An overview of these topics and ﬁndings is listed in Table 16.1, together with the properties that drive the simulations. All the simulations are introduced in a nontechnical manner, mostly by describing or picturing the assumed properties of acquisition and competition underlying the social process, so that it only requires an intuitive understanding of how the simulations are run. Before embarking on some speciﬁc simulation, it is informative to describe a number of general characteristics of all simulations. All simulations used a recurrent network because this architecture allows reproducing more complex social effects, except for the feedforward simulations of cognitive dissonance (Van Overwalle & Jordens, 2002) that were published earlier. In each simulation, a simulated learning history replicates the exact number and order of trials in an experiment, or it makes some simplifying but reasonable assumptions on the minimal number of trials necessary to develop weak or strong connection weights to mimic how participants built up prior knowledge in their lives. Each condition is run in a separate simulation. Moreover, often the simulation of all conditions is repeated 50 or 100 times in which each “run” represents a single “participant.” This repetition introduces some realistic noise in the input data by mimicking differences between real participants in the order of trials (i.e., when randomized in the actual experiment) or in their prior knowledge (i.e., by providing slightly different initial connection weights or activation levels). The results of the simulations are often directly compared with some empirical data, by projecting the simulated values on the empirical data so that the ﬁt in the pattern of simulated and observed data can be immediately observed and evaluated. Person Impression Formation Competition

To introduce how our simulation approach works, consider the last empirical study from the previous section (Van Overwalle, 2006). This study

2/15/2011 7:47:00 PM

SOCIAL LEARNING AND CONNECTIONISM

359

Table 16.1 Overview of the Simulated Social Cognition Topics and the Underlying Connectionist Properties Topic

Findings

Property

Person Impression Formation Discounting and sample size

Discounting of an actor’s trait when there is more evidence on an alternative actor

Acquisition of alternative→trait link which leads to competition against target→trait link

Assimilation and contrast

Priming with. . . -a trait leads to assimilation of that trait -an exemplar leads to contrasting away from the implied trait

Acquisition: Additional trait activation is linked to the actor Competition: Exemplar→trait link competes with target→trait link

Illusory correlation

A minority group is seen as more negative despite the fact that the proportion of positive and negative behaviors is identical to a majority group. Better memory (shorter assignment latencies) for items from a minority group

Acquisition: greater sample size of desirable and undesirable traits in majority group Competition: greater sample size in majority group (or stronger majority→trait link) competes with episodic weights (or behavior→trait links)

Stereotype change

Group stereotype changes more if stereotype-inconsistent information is dispersed across many members rather than concentrated in a few

Competition: less discounting of inconsistent trait when dispersed among many members

Cognitive dissonance: Prohibition

Mild threat has more behavioral effects than severe threat.

Competition: Mild threat (or mild activation of inhibitory threat→play link) requires less compensatory augmentation of the toy→play link

Persuasion

Deliberate attitudes are determined by expectancy of effects x the value of these effects

Acquisition of values and of valued information

Covariation patterns (LLH & HLH) predicted to yield strong dispositional attributions recruit the temporo-parietal junction and medial prefrontal cortex

Acquisition: activation of brain areas mainly determined by covariation with target actor

Group Biases

Attitude Change

Brain Imaging Dispositional attribution based on covariation information

Note. None of the competition effects can be explained by information loss exemplar models (Fiedler, 1996; Smith, 1991) or connectionist models on the basis of a Hebbian learning algorithm (Kashima, Woolcock, & Kashima, 2000) without complementary assumptions.

explored discounting of dispositions and illustrated the emergent property of acquisition (e.g., to develop weak versus strong trait inferences about an alternative actor by manipulating sample size) and competition (e.g., to block the

16-Schachtman-16.indd 359

development of similar trait inferences about a target actor). We presented the network with a learning history in which a competing actor is ﬁrst engaged in a trait-implying behavior alone, and then together with the target actor (#5;

2/15/2011 7:47:00 PM

360

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

where # indicates the number of trials). Because the competing actor covaries with the traitimplying behavior before the target actor, this (stronger) competing cause→effect connection creates competition against the (weaker) target cause→effect connection. The crucial difference between conditions is the amount of competition exerted by the competing connection. This was manipulated by the sample size of the competing cause, or the number of times (#1 or #5) the competing actor engages in the behavior alone. As can be seen in Figure 16.6, the simulation (dotted line) demonstrates that in the large as opposed to small size condition, the perceiver attributes stronger trait attributions to the competing actor (see left portion of the ﬁgure). This leads to more competition such that fewer trait attributions are made to the target actor (see right portion). As can be seen, the simulation closely matches the empirical data. Note that for the simulation to work, it assumes that the competing information is received ﬁrst to build up a strong competing connection that exerts a blocking effect on the target connection. If the competing information is received afterwards (as was the case in some conditions of the experiment),

8.5 Small Size Large Size Simulation

Trait Inference

8.0

7.5

7.0

6.5 Competing Actor

Target Actor

Figure 16.6 Discounting and sample size: Observed data from Van Overwalle (2006) are indicated by bars and recurrent simulation results (general learning rate = .13, for competing actor = .08) by dotted lines. (Reprinted with permission from Figure 9 of Van Overwalle, F., & Labiouse, C. 2004. A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61).

16-Schachtman-16.indd 360

it is assumed that perceivers readjust their initial target attribution by simply reevaluating the target, that is, as though they reran that part of the simulation where information on the target actor is processed. In this manner, the strong connection of the competing actor is taken into account. Assimilation and Contrast

In a series of simulations on person attribution and impression formation, Van Overwalle and Labiouse (2004) presented many other simulations in which they modeled important ﬁndings of primacy and recency in impression formation (Hogarth & Einhorn, 1992; Kashima & Kerekes, 1994), asymmetric diagnosticity of ability- and morality-related behaviors such that estimated ability is determined more by high than low performance (e.g., sports) while estimated morality is determined more by the occurrence of low rather than high moral behaviors (e.g., lying; Skowronski & Carlston, 1987), increased recall for trait-inconsistent information (Hamilton, Katz, & Leirer, 1980), and assimilation and contrast effects in priming (Stapel, Koomen, & van der Pligt, 1997). I choose the last simulation as an illustration because it is fairly easy to understand. There is an abundance of social cognition research indicating that we often ﬁll in unobserved characteristics of another person by temporary assumptions we make about them, and this process is termed assimilation. Thus, when primed (i.e., temporarily activated) with “violent,” we judge a nondescript or ambiguous target person as more hostile, and when primed with “friendly,” we judge that same target as less hostile. However, under some circumstances, the opposite effect may occur and leads to contrast rather than assimilation. For instance, when primed with the exemplar “Gandhi,” people may judge a target person as relatively more hostile, whereas primed with “Hitler,” they may judge the same target as relatively less hostile. Under these conditions, the exemplars Gandhi and Hitler serve as an anchor against which the target is judged, and so lead to contrast effects (Stapel, Koomen, & van der Pligt, 1997). What produces assimilation or contrast? The properties leading to each of these effects are

2/15/2011 7:47:00 PM

SOCIAL LEARNING AND CONNECTIONISM

schematically depicted on top of Figure 16.7, and a simulation and empirical data are shown in the bottom panel (Van Overwalle & Labiouse, 2004). The network consists of units representing a target actor, the extreme exemplars like Gandhi and Hitler, and two traits (friendly and hostile). Person impressions are expressed by the target→trait connections. The network ﬁrst builds up background knowledge about the extreme exemplars by linking Gandhi with the friendly trait and Hitler with the hostile trait (each #10). Because the target person is not described, no prior learning is assumed for this agent. Now comes the essential manipulation, which is schematically depicted by showing only

Trait

Trait

+

+ Impression of Ambigious Person

4.5

Target

Target

Exemplar Positive Prime Negative Prime Simulation

4.0

3.0

Person Exemplar

Trait Prime

Figure 16.7 Assimilation and contrast effects

after priming with a trait or person. (Top) A schematic illustration of the acquisition property generating assimilation and the competition property generating contrast. The primed stimulus is indicated with a plus sign. The weight of the connections is depicted in an increasing order by dotted–broken–full arrows. (Bottom) Observed data from Stapel, Koomen, and van der Pligt (1997, Experiment 3) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (Reprinted and adapted with permission from of Van Overwalle, F., & Labiouse, C. 2004. A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61).

16-Schachtman-16.indd 361

the essential units and connections in the top panel of Figure 16.7. As shown on the top left, when a trait concept is primed, this trait is assumed to be still active (denoted by +) when the actor is presented (#1). Through their co-occurrence and the property of acquisition, this leads to stronger positive or negative actor→trait connections in line with the prime, consistent with the empirical results (see bottom left portion of the ﬁgure). In contrast, as shown on the top right portion of the ﬁgure, when an exemplar such as Gandhi is primed (denoted by +), competition arises between this primed exemplar and the target actor in their connection to the friendly trait. The competition between the stronger Gandhi→trait connection (#10) and the weaker target→trait connection (#1) leads to discounting of the target→trait connection. This results in a contrast effect where the target is seen as less positive after comparison with a positive exemplar (Gandhi), and more positive after comparison with a negative exemplar (Hitler), which is consistent with the empirical ﬁndings (see bottom right portion of Fig. 16.7). Group Biases

3.5

2.5

361

In a series of simulations on group impression formation, Van Rooy, Van Overwalle, Vanhoomissen, Labiouse, and French (2003) modeled major biases and stereotypes in group judgments from a connectionist perspective. These biases were in the areas of group impression formation (also denoted as stereotyping) such as illusory correlation (deﬁned later; Hamilton & Gifford, 1976), group differentiation (or the accentuation of differences between groups and the opposite tendency for differences within groups; Eiser, 1971), stereotype change under dispersed versus concentrated distribution of inconsistent information (deﬁned later; Hewstone, Macrae, Grifﬁths, & Milne, 1994; Johnston & Hewstone, 1992; Weber & Crocker, 1983), and group homogeneity (or the tendency to see minority groups as less variable; see Simon & Brown, 1987). I illustrate this approach with the important phenomenon of illusory correlation.

2/15/2011 7:47:00 PM

362

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Illusory Correlation

This bias occurs when perceivers erroneously see a relation between categories that are actually independent. For instance, minorities or outgroups are often stereotyped with bad characteristics, although these characteristics are sometimes present just as often in the perceiver’s ingroup. The earliest demonstration of illusory correlation in a group context comes from a study by Hamilton and Gifford (1976). Participants read about members of two groups A and B who engaged in the same ratio of desirable to undesirable behaviors, but much more behaviors were performed by members of group A than by members of group B. Although there was no objective correlation between group membership and desirability of behavior, participants showed greater liking for the majority group A than for the minority group B (for reviews see Hamilton & Sherman, 1989; Mullen & Johnson, 1990).

Illusory correlation can be explained by the acquisition property. How does it work? Consider a network with two group units (group A and B), two trait units (desirable and undesirable), and a set of units each representing a single behavior (of less importance here). Group impressions are represented by the group→trait connections, as schematically shown in the top panel of Figure 16.8. Because more behaviors are performed by members of the majority group A than by members of the minority group B, there is a larger sample size in group A (#8 and #4 for desirable and undesirable behaviors, respectively) than in group B (#4 and #2). Consequently, the group→trait connections of the majority group A are stronger at the end of learning than the corresponding connections of the minority group B. This is illustrated in the bottom panel of Figure 16.8. As a result, the relative proportion of desirable versus undesirable information is more clearly encoded in the group→trait connections for majority group A (denoted Da) than

Desirable

Undesirable

Group

0.8

Group A

Group B

Simulated Evalution

0.74 0.6

Da

0.51

0.36

0.4

Db 0.25

Desirable Undesirable 0.2

0.0

0 1 2 3 4 5 6 7 8 Statements

0 1 2 3 4

0 1 2 3 4 0 1 2 Statements

Figure 16.8 Illusory correlation. (Top) A schematic illustration of the acquisition property generating stronger desirable than undesirable trait connections for each group. (Bottom) Simulated evaluative strength in an illusory correlation design (Da,b indicates the difference between desirable and undesirable evaluation for group A and B, respectively) in which two desirable and one undesirable behavior were alternately presented to the network (learning rate = .15). (Reprinted and adapted with permission from Figure 3 of Van Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).

16-Schachtman-16.indd 362

2/15/2011 7:47:00 PM

SOCIAL LEARNING AND CONNECTIONISM

16-Schachtman-16.indd 363

Group

Group

A

(strong)

2.7

B

(weak) Behavior

Trait

Correct Assignment RT sec)

for minority group B (denoted Db). This results in a more favorable impression overall for the majority group A. Note that the evaluations after four trials differ between groups A (.36) and B (.51) because the lateral connections between the units also differ in number between groups (i.e., there is a minor effect of the behavioral units). As can be seen in the ﬁgure, the connectionist network predicts that with little training, illusory correlation will not appear. Interestingly, it also makes the prediction that with continued training, the weights will asymptote at the same values, and illusory correlation will disappear in the model. This prediction has recently been conﬁrmed (Murphy, Schmeer, ValléeTourangeau, Mondragon, & Hilton, 2009). Besides a decreased evaluation for minority group B, illusory correlation is often accompanied with increased source memory for undesirable group B behaviors in a task where participants have to assign each behavior to the correct group (Hamilton, Dugan, & Trollier, 1985; McConnell, Sherman, & Hamilton, 1994; Stroessner, Hamilton, & Mackie, 1992). This memory advantage might be produced by the competition property. Source memory is expressed by the behavioral→group connections. However, as illustrated in Figure 16.9 (top panel), the trait→group connections compete against these behavior→group connections. Given that the trait→group connections of group A are stronger than group B, these behavior→group connections are more discounted for group A behaviors than for group B behaviors. Based on the same logic (not illustrated), given that the trait→group connections of positive traits are stronger than negative traits (since there are typically more positive behaviors than negative behaviors), these behavior→group connections are more discounted for positive behaviors than for negative behaviors. The bottom panel of Figure 16.9 illustrates a successful simulation of this decreased memory for majority and positive behaviors (McConnell et al., 1994, Experiment 2). In this study, participants had to remember to which group each behavior belonged (i.e., group assignment), and faster responses on this task reﬂect stronger behavior→group links in memory. As can be

363

Behavior

Trait

Majority Group A Majority Group B Simulation

2.8 2.9 3.0 3.1 3.2

A+

A–

B+

B–

Condition

Figure 16.9 Illusory correlation and memory.

(Top) A schematic illustration of the competition property generating better memory after strong versus weak trait–group links. The weight of the connections is depicted in an increasing order by coarsely dotted–dotted–broken–full arrows. (Bottom) Observed data from McConnell, Sherman, and Hamilton (1994, Experiment 2, Table 5) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (The scale is reversed so that higher values reﬂect better memory and, consequently, faster latencies.) (Reprinted and adapted with permission from Figure 5 of Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).

seen, memory for group afﬁliation was lowest for positive behaviors of group A (A+) and highest for negative behaviors of group B (B–). It is important to note that this illusory correlation simulation of group assignment excludes other well-known theoretical accounts of group biases such as information loss exemplar models (Fiedler, 1996; Smith, 1991), which assume that noise at the exemplar level is reduced the more exemplars are accumulated (e.g., summed) in an aggregated group judgment. It also excludes connectionist models on the basis of a Hebbian learning algorithm (Kashima et al., 2000), which simply accumulates the number of co-occurrences of

2/15/2011 7:47:00 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

two stimuli in the weights of the connections. Because these models are based on accumulation without an upper bound rather than error minimization, which limits the amount of learning (like the delta algorithm), none of them incorporates the essence of competition and thus cannot explain the opposite relation between strong favoritism for group A and less memory for the behaviors of that group. Subtyping

The illusory correlation effect demonstrates how perceivers may make initial impressions about minority groups in society that are biased. An interesting question, then, is how can these stereotypes be abolished? Research has demonstrated that the best tactic to change group stereotypes is to distribute disconﬁrming information among as many group members as possible. If disconﬁrming information is not distributed but rather concentrated in a few members, these members are subtyped as extreme disconﬁrmers. Subtyping insulates the group from their extreme dissenting members, so that the disconﬁrming information is attributed to these extremists only and the content of the existing group stereotype is preserved (Hewstone et al., 1994; Johnston & Hewstone, 1992; Weber & Crocker, 1983). A connectionist framework can simulate subtyping through the property of competition. The network consisted of a group unit, several units representing individual members, and two trait units (consistent and inconsistent traits). Group stereotyping is represented by the group→trait connections. To built up an initial group stereotype, the network ﬁrst receives preexperimental learning experiences on stereotypical (i.e., consistent) beliefs of the group (#10). Next, information is provided that is consistent (#12) and inconsistent (#12) with the stereotype. Crucially, in the concentrated condition, all inconsistent information is concentrated in two disconﬁrming group members, whereas in the dispersed condition, inconsistent information appears in six disconﬁrming members. As can be seen in Figure 16.10 (top panel), when stereotypeinconsistent information is concentrated in a few members, each of these disconﬁrming

16-Schachtman-16.indd 364

Trait

Trait (few)

Group

Trains Ratings on Whole Group

364

Member

(many) Group

6

Member

Concentrated Dispersed Simulation

5

4

3

2 Consistert

Inconsistert Traits

Figure 16.10 Dispersed versus concentrated stereotype-inconsistent information. (Top) A schematic illustration of the competition property generating less group trait inconsistency after strong versus weak member–trait links given a few versus many inconsistent members. The weight of the connections is depicted in an increasing order by coarsely dotted–dotted–broken–full arrows. (Bottom) Observed data from Johnston and Hewstone (1992, Experiment 1, Table 3) are indicated by bars and simulation results (learning rate = .15) by dotted lines. (Reprinted and adapted with permission from Figure 8 of Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. 2003. A recurrent connectionist model of group biases. Psychological Review, 110, 536–563).

members engages in many inconsistent behaviors (each #6) and so develops strong member→ inconsistent trait connections that compete against and discount the group→inconsistent connections. In contrast, when the stereotypeinconsistent information is dispersed across multiple members, each individual member engages in few inconsistent behaviors (each #2) and thus develops weaker member→inconsistent connections, so that less competition arises against the group→inconsistent connection. Consequently, the group→inconsistent connection is more discounted by a few disconﬁrming members in the concentrated condition than by multiple members in the dispersed condition, leading to a conservation of stereotypical perceptions of the group

2/15/2011 7:47:01 PM

SOCIAL LEARNING AND CONNECTIONISM

as a whole. The result of this simulation is shown in the bottom panel of Figure 16.10. Other theories such as the exemplar-based model by Fiedler (1996) and connectionist models using the Hebbian learning algorithm (Kashima et al., 2000) do not posses the competition property, and hence cannot make this prediction except by adding auxiliary assumptions (see e.g., Kashima et al., 2000, p. 931). Attitude Change Cognitive Dissonance

In a series of simulations, Van Overwalle and Jordens (2002) presented a connectionist implementation of cognitive dissonance, that is, a state where a person acts against his or her personal attitudes. In their network model, an attitude is represented by the connection between an attitude object and behavioral-affective outcomes. One way that dissonance can arise is when circumstantial constraints induce a mismatch between the model’s prediction based on what is known about the person and discrepant behavior or affect. A series of feedforward simulations successfully replicated several classical dissonance paradigms, including prohibition (or greater behavioral effect of mild versus severe threats; Freedman, 1965), initiation (or more liking for a group following harsher hazing; Gerard & Mathewson, 1966), forced compliance (or greater tendency to change one’s opinion in the absence of external constraints; Calder, Ross, & Insko, 1973; Collins & Hoyt, 1972; Linder, Cooper, & Jones, 1967; Sherman, 1970), free choice (or greater preference for an item chosen rather than forgone; Shultz, Léveillé, & Lepper, 1999), and misattribution (or decreased behavioral change after attributing dissonance feelings to external factors, e.g., medication; Higgins, Rhodewalt, & Zanna, 1979). As an illustration of dissonance, I now explore the effects of prohibiting a desired action (Freedman, 1965). School children were forbidden to play with an attractive toy (a robot) under either mild or severe threat of punishment, without subsequent surveillance. Actual play with the forbidden toy about 40 days later in the absence of the experimenter or any threat revealed greater

16-Schachtman-16.indd 365

365

derogation of the toy in the mild than in the severe threat condition. The attributional explanation for these results is that mild threat alone provides insufﬁcient justiﬁcation for the counterattitudinal behavior of not playing with the attractive toy, and thus creates high dissonance that is reduced by lowering the attraction for the toy. In contrast, the high threat provides sufﬁcient justiﬁcation for not playing with the toy, and thus creates little dissonance and little attitude change. How can a connectionist framework model this process of dissonance reduction? Figure 16.11 (top panel) depicts a simulation of this effect. Three units are included in the network: toy, threat, and playing. The liking for the toy is expressed by the toy→playing connection. For the preexperimental learning phase, we assume that the most natural and most often occurring situation for the child is to play with an attractive toy, a pleasant experience (#10). In contrast, if children are severely threatened not to play with a forbidden toy, we assume that they would not play with it (#4). Figure 16.11 (acquisition curve in the bottom panel) shows the development of a facilitatory toy→play connection followed by an inhibitory threat→play connection given this learning history. At the end, these two connections have about equal but opposing strength that keeps them in balance. When severe threat is applied in the experimental condition (#1), there is no change because the network’s learning error is negligible (see middle portion of acquisition curve). However, when mild threat is simulated by activating the threat unit by only half its typical activation level (#1), this results in weaker inhibitory activation received at the play unit (–.2; see right portion of top panel in Fig. 16.11). Consequently, this leads to less compensation for the inhibitory threat→play connection or less augmentation (or a decrease) of the toy→play connection (implying that the child likes the toy less; see right portion of acquisition curve). In other terms, this weight change justiﬁes the unexpected behavior. Contrary to an earlier constraint satisfaction approach (Shultz & Lepper, 1996), the present connectionist model changes not only temporary activation of attitude objects but also long-term connection

2/15/2011 7:47:01 PM

366

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Play

Play

A

B –.2 (mild)

–.4 (severe) Threat

Toy

Toy

Threat

1.0

0.8

Prior Learning

Severe Mild Threat Threat

0.6 Connection Weight

Experimental Learning

0.4

Toy

0.2

0.0

–0.2

Threat

–0.4 0

2

4

6

8 10 12 14 Trials

01

01

Figure 16.11 Prohibition and the effect of severe

versus mild threat. (Top) A schematic illustration of the competition property generating more decrease of the toy–play link due to weaker activation received given mild versus severe threat. The weight of the connections is depicted in an increasing order by dotted–broken–full arrows. (Bottom) Changes in connection weights after each trial in the prior learning history (trials 0–14 left) and in each of the experimental conditions (trials 0–1 middle and right; learning rate = .30; feedforward network). The numbers next to the connections reﬂect the activation received at the play unit through these connections, not the connection weights themselves. (Reprinted and adapted with permission from Figure 2 of Van Overwalle, F., & Jordens, K. 2002. An adaptive connectionist model of cognitive dissonance. Personality and Social Psychology Review, 6, 204–231).

weights, so that it can explain why after 40 days, the dissonance manipulation was still effective. Persuasive Communication

Although the effects of dissonance are counterintuitive and highly intriguing, attitudes are more often changed through persuasive arguments, such

16-Schachtman-16.indd 366

as campaigns and advertisement. Van Overwalle and Siebler (2005) replicated a number of important ﬁndings in this literature that are typically explained by dual-process approaches of attitudes (Chaiken, 1987; Petty & Cacioppo, 1981, 1986). A recurrent network was applied to well-known experiments involving deliberative attitude formation (deﬁned later) as well as the use of heuristics of length, consensus, expertise, and mood (Chaiken & Maheswaran, 1994; Maheswaran & Chaiken, 1991; Petty & Cacioppo, 1984; Petty, Schumann, Richman, & Strathman, 1993). These heuristics reﬂect the ﬁnding that when people are not capable or motivated to attend to the message content, they are easily swept by the length of the arguments, the apparent consensus with other members of the audience, the expertise of the source, or their own moods rather than the strength of the message arguments. All these empirical phenomena were successfully reproduced in the simulation. Let us focus on the simplest case of deliberative attitude formation. Perhaps the most inﬂuential model of attitude formation that describes this sort of deliberative weighting of all salient alternatives and consequences is the theory of reasoned action by Fishbein and Ajzen (1975). According to this theory, an attitude is a function of the expectation that the behavior leads to certain consequences or outcomes (e.g., a car is fast and keeps you dry during the rain but also pollutes the air) and the person’s evaluation of these outcomes (e.g., fast and dry is good, pollution is bad). The attitude is the outcome of this weighting process, and it is computed by multiplying the expectancy and value components associated with each outcome and summing up these products. This formula of attitude formation has received considerable empirical support in many studies (see Ajzen, 1991; Ajzen & Madden, 1986; Fishbein & Ajzen, 1975). However, a limitation is that the theory remains vague about the psychological integration process underlying this formula. Can a connectionist approach mimic this integration in line with the predictions of the theory of reasoned action (Fishbein & Ajzen, 1975)? Figure 16.12 depicts a network architecture that was applied by Van Overwalle and

2/15/2011 7:47:01 PM

SOCIAL LEARNING AND CONNECTIONISM

367

Evaluative

Fast

Dry

Neg

Pos

Prediction Simulation

Pollutes

Attitude

Car

Attitude

Cognitive

Car Bicycle Bus

Figure 16.12 Attitude formation. (Left) Network architecture with one attitude object (car) connected to three cognitive nodes (fast, dry, and pollutes) and two valence nodes. All nodes are interconnected to all other nodes, but for a clear understanding of the major mechanisms underlying attitude formation, only the most important (feedforward) connections are shown. Contrary to all previous simulations, there were two cycles to spread activation around the network so that activation at the cognitive attributes could spread further to the valence units (learning rate = .35). (Right) Predicted data from Fishbein and Ajzen (1975) are indicated by bars and simulation results by dotted lines. (Reprinted and adapted with permission from Figures 2 and 4 of Van Overwalle, F., & Siebler, F. 2005. A connectionist model of attitude formation and change. Personality and Social Psychology Review, 9, 231–274).

Siebler (2005). The expectancy variable in the Fishbein and Ajzen (1975) formula is determined by the frequency that an attitude object cooccurs with a cognitive attribute, which leads to stronger object→attribute connections. The evaluation variable is determined by the frequency of satisfaction or dissatisfaction experienced when that attribute is present, and it is stored in attribute→valence connections. In the simulation, during a prior learning phase, valences are developed (#15) and afterward, during the main learning phase, the attributes of each object (e.g., transportation vehicle) are learned. The number of trials is determined by the expectancy that the attributes are present, and a greater expectancy is reﬂected in a higher number of trials. Crucially, this learning does not only determine the weight of the object→attribute and attribute→valence connections through the property of acquisition, but at the same time it also shapes the direct object→valence connection, which reﬂects the attitude. This latter connection represents the attitude. As can be seen in

16-Schachtman-16.indd 367

the inset of Figure 16.12, the simulated and predicted data from Fishbein and Ajzen’s (1975) formula match almost perfectly in this example. Dispositional Attributions in the Brain

We end this chapter with a simulation of recent neuroimaging ﬁndings on dispositional attributions. In a study on trait inferences using functional magnetic resonance imaging (fMRI), Harris, Todorov, and Fiske (2005) explored which dimensions of covariation information determine activity in social areas of the brain. The authors gave their participants short stories involving several combinations of Kelley’s covariation dimensions. As noted earlier, consensus reﬂects comparisons between the behavior of the actor and others, and distinctiveness denotes comparisons between actors’ goal objects. Consistency denotes comparisons over time. To illustrate, a strong trait-implying story reads: “John laughs at the comedian [target behavior]. Hardly anyone else laughs at the comedian [low consensus].

2/15/2011 7:47:01 PM

368

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

John also laughs at every other comedian [low distinctiveness]. In the past, John would always laugh at the comedian [high consistency].” While reading these stories, brain activity was measured using fMRI. As can be seen in Figure 16.13, the brain imaging data revealed that behaviors implying the strongest actor traits according to Kelley’s (1967) covariation model (denoted as LLH; see earlier example) showed the largest increase in activation at the temporo-parietal junction (TPJ) and the medial prefrontal cortex (mPFC), two brain areas typically recruited when judging and reasoning about other people. The only exception was that consensus information was underutilized, which is a typical ﬁnding in behavioral research (see also Wells & Harvey, 1977). Consequently, the HLH condition also produced very high activation in these areas. Can a connectionist simulation reproduce these brain activation results? I tested this in a simulation. The network consists of four units (e.g., actors, objects, time, and trait-implying behaviors) and four trials for each dimension, resulting in 43 or 64 pieces of behavioral information for a story

representing all possible (high versus low) combinations of consensus, distinctiveness, and consistency. I explored various alternative coding schemes to represent this information in an increasingly simpler manner, involving fewer covariation dimensions (see Table 16.2 for an example). A ﬁrst simpliﬁcation was to drop the consensus information (incomplete coding) because it had little effect, leaving only 16 trials. Because the information in Harris et al. (2005) was actually presented in a summary format, a graded coding schema was then developed in which the degree of activation rather than the number of trails reﬂects the frequency by which an actor engages in a behavior, and this information was only presented once. The best match between the simulation and the fMRI activation in Harris et al. (2005) was obtained if activation in both TPJ and mPFC areas contribute about equally, and therefore this activation was simply summed before comparing it with the simulation data. As can be seen in Figure 16.13, the simulation under a complete trial-by-trial coding scheme is almost perfect,

0.2 TPJ

Joint activity of TPJ + mPFC

0.1

mPFC

0.0 –0.1 Coding: Trial-by-trial Complete Trial-by-trial Incomplete Graded Consistency

–0.2 –0.3 –0.4 –0.5

LLH

HLH

LHH

HHH LLL Covariation

HLL

LHL

HHL

Figure 16.13 Simulation of traits following varying covariation information. Empirical data (the summed functional magnetic resonance imaging [fMRI] activity of the temporo-parietal junction [TPJ] and medial prefrontal cortex [mPFC] from Harris, Todorov, & Fiske, 2005) are indicated by bars and the simulation results (learning rate = .04) by dotted lines. The bottom axis reﬂects consensus, distinctiveness, and consistency, respectively, all high (H) or low (L). The inset shows the approximate location of the brain areas involved on the right hemisphere (TPJ) and the medial brain (mPFC).

16-Schachtman-16.indd 368

2/15/2011 7:47:01 PM

16-Schachtman-16.indd 369

Table 16.2

Different Coding Schemes Illustrated for Consensus and Consistency Coding Scheme TbT - Complete Frequency

TbT - Incomplete

Graded - Incomplete

Graded - Consistency

Actor

Behavior

Actor

Behavior

Actor

Behavior

Actor

Behavior

Low Consensus Target Actor

#16/16

1

1

1

1

16

1

4

1

Actor 2

#16/16

1

0

—

—

—

—

—

—

Actor 3

#16/16

1

0

—

—

—

—

—

—

Actor 4

#16/16

1

0

—

—

—

—

—

—

High Consensus Target Actor

#16/16

1

1

1

1

16

1

4

1

Actor 2

#16/16

1

1

—

—

—

—

—

—

Actor 3

#16/16

1

1

—

—

—

—

—

—

Actor 4

#16/16

1

1

—

—

—

—

—

—

Low Consistency Target Time

#16/4

1

1

1

1

4

1

1

1

Time 2

#16/4

1

0

1

0

12

0

3

0 (Continued)

2/15/2011 7:47:02 PM

16-Schachtman-16.indd 370

Table 16.2

(Continued) Coding Scheme TbT - Complete Frequency

TbT - Incomplete

Graded - Incomplete

Graded - Consistency

Actor

Behavior

Actor

Behavior

Actor

Behavior

Actor

Behavior

Time 3

#16/4

1

0

1

0

—

—

—

—

Time 4

#16/4

1

0

1

0

—

—

—

—

High Consistency Target Time

#16/4

1

1

1

1

16

1

4

1

Time 2

#16/4

1

1

1

1

—

—

—

—

Time 3

#16/4

1

1

1

1

—

—

—

—

Time 4

#16/4

1

1

1

1

—

—

—

—

Note. Cells entries denoted the external activation. TbT = trial by trial; Incomplete = without consensus information; Consistency = only consistency information; = number of trials (#16 or # 4 for complete and incomplete trial-by-trial coding, respectively; #1 everywhere for graded coding). The same logic applies for distinctiveness and can be applied by changing “time” into “object.”

2/15/2011 7:47:02 PM

SOCIAL LEARNING AND CONNECTIONISM

and it reaches a signiﬁcant correlation with the fMRI data of r = .99, immediately followed by a signiﬁcant correlation of the incomplete coding scheme (without consensus), r = .98. For the graded input coding scheme, the correlation is again signiﬁcant for incomplete information (without consensus), r = .96, but consistencyonly information also yields an almost perfect correlation, r = .98. The strong correlations of the graded coding scheme suggest that summary information was perhaps encoded in much the same way as assumed by this coding scheme, that is, by roughly estimating the frequency of co-occurrences and adjusting judgments on the basis of these estimates. Moreover, the high correlation of the consistency-only coding suggests that focusing only on the approximate number of times the actor was paired with the target behavior, and ignoring all other covariation information, seems sufﬁcient for this process to reach adequate results. Evidently, these are preliminary conclusions that need to be conﬁrmed by subsequent research and future simulations on different information patterns. Graded coding of summarized social behavior has been hereto largely ignored in connectionist simulations, not in the least because it departs radically from the typical trial-by-trial input format in associative learning. Surprisingly, the simulation matches better the brain data than participants’ trait ratings, as the correlation of trait ratings with complete trial-by-trial coding was much lower, r = .71, although still signiﬁcant.

CONCLUSION This chapter reviewed support for the idea that many social judgments and biases can be proﬁtably viewed from a common connectionist framework that sees these social judgments as arising from basic associative learning processes. I provided empirical evidence to demonstrate that social judgments on causes and traits are best explained by connectionist approaches. Using connectionist simulations, I demonstrated that many other social judgments and biases might result from such basic learning processes. If there is one point that stands out and should be remembered from all this research, it is that a

16-Schachtman-16.indd 371

371

single connectionist learning mechanism—the delta algorithm—is capable of producing a rich set of emerging properties to explain a wealth of empirical data. As far as I am aware, no other theory in social psychology is capable of doing that. Instead of developing isolated and fragmentary social psychological theories that lack psychological generality, this common framework promotes the view that social behavior is just one of many processes of a human mind, that does many other nonsocial things, such as perception and emotion, interpretation and evaluation, judgment and action. If the ﬁeld of social psychology is to grow in the coming decades, it is time to leave behind our narrow pet theories, to cross the borders of our social discipline, and to look out for a broader and cumulative perspective that encompasses many ﬁelds in psychology. The next step in advancing our understanding of social learning or learning at large is not simulating the brain, but looking into the brain itself, using state-of-the-art imaging and timing techniques, and see where and when social processes and judgments are computed in the real brain. Fortunately, scientists of many disciplines are now working together in this endeavor. Collaboration is necessary because the technical obstacles are very complex, but it is also needed because the brain itself presents us with many surprises and riddles, which can only be resolved if we work together. To illustrate, on the one hand, it is clear that several parts of the brain are preferentially used for various social judgments (for reviews, see Van Overwalle, 2008; Van Overwalle & Baetens, 2009). On the other hand, it becomes increasingly clear that these social brain areas also have crucial nonsocial functions (e.g., Mitchell, 2008), perhaps because social processes may have evolved on top of more basic nonsocial computations. Understanding the evolution and interaction between social and nonsocial processes in the brain is an important question for future research, which requires psychologists from various backgrounds to provide different pieces of the puzzle. Looking only at the brain areas recruited by social tasks will not reveal to us how the brain computes these social functions (just like looking only at the steering wheel does not tell how a car works).

2/15/2011 7:47:02 PM

372

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Another point that becomes increasingly clear is that brain imaging can tell us which areas are involved in psychological functioning at a higher level, whereas single-cell recording reveals to us how individual brain cells respond to stimuli at a lower level. However, to combine both levels and obtain an integrated understanding of functioning and interaction in tightly coupled brain networks, we must probably return to simulations. Given the enormous impact and insights the associative or connectionist approach has provided in the past, ultimately it may be the only way to get order and insight in the growing amount of brain data that we see coming out from research laboratories today.

REFERENCES Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211. Ajzen, I., & Madden, T. J. (1986). Prediction of goal-directed behavior: Attitudes, intentions, and perceived behavioral control. Journal of Experimental Social Psychology, 22, 453–474. Allan, L. G. (1993). Human contingency judgments: Rule based or associative? Psychological Bulletin, 114, 435–448. Anderson, N. H. (1967). Averaging model analysis of set size effect in impression formation. Journal of Experimental Psychology, 75, 158–165. Anderson, N. H. (1981). Foundations of information integration theory. New York, NY: Academic Press. Baker, A. G., Berbier, M. W., & Vallée-Tourangeau, F. (1989). Judgments of a 2 x 2 contingency table: Sequential processing and the learning curve. The Quarterly Journal of Experimental Psychology, 41B, 65–97. Busemeyer, J. R. (1991) Intuitive statistical estimation. In N. Anderson (Ed.) Contributions to information integration theory, Vol. 1. Cognition (pp. 189–215). Hillsdale, NJ: Erlbaum. Calder, B. J., Ross, M., & Insko, C. A. (1973). Attitude change and attitude attribution: Effects of incentive, choice, and consequences. Journal of Personality and Social Psychology, 25, 84–99. Chaiken, S. (1987). The heuristic model of persuasion. In M. P. Zanna, J. M. Olson, & C. P. Herman (Eds.), Social inﬂuence: The Ontario Symposium (Vol. 5., pp. 3–39). Hillsdale, NJ: Erlbaum.

16-Schachtman-16.indd 372

Chaiken, S., & Maheswaran, D. (1994). Heuristic processing can bias systematic processing: Effects of source credibility, argument ambiguity, and task importance on attitude judgment. Journal of Personality and Social Psychology, 66, 460–473. Chapman, G. B., & Robbins, S. J. (1990). Cue interaction in human contingency judgment. Memory and Cognition, 18, 537–545. Cheng, P. W., & Novick, L. R. (1990). A probabilistic contrast model of causal induction. Journal of Personality and Social Psychology, 58, 545–567. Collins, B. E., & Hoyt, M. F. (1972). Personal responsibility-for-consequences: An integration and extension of the “forced compliance” literature. Journal of Experimental Social Psychology, 8, 558–593. Ebbesen, E. B., & Bowers, R. J. (1974) Proportion of risky to conservative arguments in a group discussion and choice shifts. Journal of Personality and Social Psychology, 29, 316–327. Eiser, J. R. (1971). Enhancement of contrast in the absolute judgment of attitude statements. Journal of Personality and Social Psychology, 17, 1–10. Fiedler, K. (1996). Explaining and simulating judgment biases as an aggregation phenomenon in probabilistic, multiple-cue environment. Journal of Personality and Social Psychology, 103, 193–214. Fiedler, K., Walther, E., & Nickel, S. (1999). The auto-veriﬁcation of social hypotheses: Stereotyping and the power of sample size. Journal of Personality and Social Psychology, 77, 5–18. Fishbein, M., & Ajzen, I. (1975). Belief attitude, intention and behavior: An introduction to theory and research. London, England: AddisonWesley. Försterling, F. (1989). Models of covariation and attribution: How do they relate to the analogy of analysis of variance? Journal of Personality and Social Psychology, 57, 615–625. Försterling, F. (1992). The Kelley model as an analysis of variance analogy: How far can it be taken? Journal of Experimental Social Psychology, 28, 475–490. Freedman, J. L. (1965). Long-term behavioral effects of cognitive dissonance. Journal of Experimental Social Psychology, 1, 145–155. Gerard, H. B., & Mathewson, G. C. (1966). The effects of severity of initiation on liking for a

2/15/2011 7:47:02 PM

SOCIAL LEARNING AND CONNECTIONISM

group: A replication. Journal of Experimental Social Psychology, 2, 278–287. Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38. Hamilton, D. L., Dugan, P. M., & Trollier, T. K. (1985). The formation of stereotypic beliefs: Further evidence for distinctiveness-based illusory correlation. Journal of Personality and Social Psychology, 48, 5–17. Hamilton, D. L., & Gifford, R. K. (1976). Illusory correlation in interpersonal perception: A cognitive basis of stereotypic judgments. Journal of Experimental Social Psychology, 12, 392–407. Hamilton, D. L., Katz, L. B., & Leirer, V. O. (1980). Cognitive representation of personality impressions: Organizational processes in ﬁrst impression formation. Journal of Personality and Social Psychology, 39, 1050–1063. Hamilton, D. L., & Sherman, J. W. (1989). Illusory correlations: Implications for stereotype theory and research. In D. Bar-Tal, C. F. Graumann, A. W. Kruglanski, & W. Stroebe (Eds.), Stereotype and Prejudice: Changing conceptions (pp. 55–92). New York, NY: Springer. Hansen, R. D., & Hall, C. A. (1985). Discounting and augmenting facilitative and inhibitory forces: The winner takes all. Journal of Personality and Social Psychology, 49, 1482–1493. Harris, L. T., Todorov, A., & Fiske, S. T. (2005). Attributions on the brain: Neuro-imaging dispositional inferences, beyond theory of mind. NeuroImage, 28, 763–769. Hewstone, M., & Jaspars, J. (1987). Covariation and causal attribution: A logical model of the intuitive analysis of variance. Journal of Personality and Social Psychology, 53, 663–673. Hewstone, M., Macrae, C. N., Grifﬁths, R., & Milne, A. (1994). Cognitive models of stereotype change (5): Measure, developments and consequences of subtyping. Journal of Experimental Social Psychology, 30, 505–526. Higgins, E. T., Rhodewalt, F., & Zanna, M. P. (1979). Dissonance motivation: Its nature, persistence, and reinstatement. Journal of Experimental Social Psychology, 15, 16–34. Hilton, D. J., & Slugoski, B. R. (1986). Knowledgebased causal attribution: The abnormal conditions focus model. Psychological Review, 93, 75–88. Hilton, D. J., Smith, R. H., & Kim S. H. (1995). The process of causal explanation and dispositional attribution. Journal of Personality and Social Psychology, 68, 377–387.

16-Schachtman-16.indd 373

373

Hogarth, R. M., & Einhorn, H. J. (1992) Order effects in belief updating: The belief-adjustment model. Cognitive Psychology, 24, 1–55. Johnston, L., & Hewstone, M. (1992). Cognitive models of stereotype change: (3) Subtyping and the perceived typicality of disconﬁrming group members. Journal of Experimental Social Psychology, 28, 360–386. Kashima, Y., & Kerekes, A. R. Z. (1994). A distributed memory model of averaging phenomena in person impression formation. Journal of Experimental Social Psychology, 30, 407–455. Kashima, Y., Woolcock, J., & Kashima, E. S. (2000). Group impression as dynamic conﬁgurations: The tensor product model of group impression formation and change. Psychological Review, 107, 914–942. Kelley, H. H. (1967). Attribution in social psychology. Nebraska Symposium on Motivation, 15, 192–238. Kelley, H. H. (1971). Attribution in social interaction. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 1–26). Morristown, NJ: General Learning Press. Kruglanski, A. W., Schwartz, S. M., Maides, S., & Hamel, I. Z. (1978). Covariation, discounting, and augmentation: Towards a clariﬁcation of attributional principles. Journal of Personality, 76, 176–189. Linder, D. E., Cooper, J., & Jones, E. E. (1967). Decision freedom as a determinant of the role of incentive magnitude in attitude change. Journal of Personality and Social Psychology, 6, 245–254. Maheswaran, D. & Chaiken, S. (1991). Promoting systematic processing in low-motivation settings: Effect of incongruent information on processing and judgment. Journal of Personality and Social Psychology, 61, 13–25. Manis, M., Dovalina, I., Avis, N. E., & Cardoze, S. (1980). Base rates can affect individual predictions. Journal of Personality and Social Psychology, 38, 231–248. McClelland, J. L., & Rumelhart, D. E. (1988). Explorations in parallel distributed processing: A handbook of models, programs and exercises. Cambridge, MA: Bradford. McCloskey, M., & Cohen N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning and Motivation, 24, 109–165.

2/15/2011 7:47:02 PM

374

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

McClure, J. (1998). Discounting of causes of behavior: Are two reasons better than one? Journal of Personality and Social Psychology, 74, 7–20. McConnell, A. R., Sherman, S. J., & Hamilton, D. L. (1994). Illusory correlation in the perception of groups: An extension of the distinctivenessbased account. Journal of Personality and Social Psychology, 67, 414–429. McLeod, P., Plunkett, K., & Rolls, E. T. (1998). Introduction to connectionist modeling of cognitive processes. Oxford, England: Oxford University Press. Mitchell, J. P. (2008) Activity in right temporoparietal junction is not selective for theory of mind. Cerebral Cortex, 18, 262–271. Mullen, B., & Johnson, C. (1990). Distinctivenessbased illusory correlations and stereotyping: A meta-analytic integration. British Journal of Social Psychology, 29, 11–28. Murphy, R., Schmeer, S., Vallée-Tourangeau, F., Mondragon, E., & Hilton, D. (2009). Making the illusory correlation effect appear and then disappear: The effects of increased learning. Quarterly Journal of Experimental Psychology, under revision. Orvis, B. R., Cunningham, J. D., & Kelley, H. H. (1975). A closer examination of causal inference: The role of consensus, distinction and consistency information. Journal of Personality and Social Psychology, 32, 605–616. Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and persuasion: Central and peripheral routes to attitude change. New York, NY: Springer. Petty, R. E., & Cacioppo, J. T. (1984). The effects of involvement on responses to argument quantity and quality: Central and peripheral routes to persuasion. Journal of Personality and Social Psychology, 46, 69–81. Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 19, pp. 123–205). San Diego, CA: Academic Press. Petty, R. E., Schumann, D. W., Richman, S. A., & Strathman, A. J. (1993). Positive mood and persuasion: Different roles for affect under highand low-elaboration conditions. Journal of Personality and Social Psychology, 64, 5–20. Read, S. J., & Marcus-Newhall, A. (1993) Explanatory coherence in social explanations: A parallel distributed processing account. Journal of Personality and Social Psychology, 65, 429–447.

16-Schachtman-16.indd 374

Read, S. J., & Miller, L. C. (1993) Rapist or “regular guy”: Explanatory coherence in the construction of mental models of others. Personality and Social Psychology Bulletin, 19, 526–541. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–98). New York, NY: AppletonCentury-Crofts. Shanks, D. R. (1985). Forward and backward blocking in human contingency judgment. Quarterly Journal of Experimental Psychology, 37b, 1–21. Shanks, D. R. (1987). Acquisition functions in contingency judgment. Learning and Motivation, 18, 147–166. Shanks, D. R. (1995). Is human learning rational? Quarterly Journal of Experimental Psychology, 48a, 257–279. Shanks, D. R., Lopez, F. J., Darby, R. J., & Dickinson, A. (1996). Distinguishing associative and probabilistic contrast theories of human contingency judgment. In D. R. Shanks, K. J. Holyoak, & D. L. Medin (Eds.), The psychology of learning and motivation (Vol. 34, pp. 265–311). New York, NY: Academic Press. Sherman, S. J. (1970). Attitudinal effects of unforeseen consequences. Journal of Personality and Social Psychology, 16, 510–520. Shultz, T., & Lepper, M. (1996). Cognitive dissonance reduction as constraint satisfaction. Psychological Review, 2, 219–240. Shultz, T. R., Léveillé, E., & Lepper, M. R. (1999). Free choice and cognitive dissonance revisited: Choosing “lesser evils” versus “greater goods.” Personality and Social Psychology Bulletin, 25, 40–48. Simon, B., & Brown, R. J. (1987). Perceived homogeneity in minority-majority contexts. Journal of Personality and Social Psychology, 53, 703–711. Skowronski, J. J., & Carlston, D. E. (1987). Social judgment and social memory: The role of cue diagnosticity in negativity, positivity and extremity biases. Journal of Personality and Social Psychology, 52, 689–699. Smith, E. R. (1991). Illusory correlation in a simulated exemplar-based memory. Journal of Experimental Social Psychology, 27, 107–123. Stapel, D. A., Koomen, W., & van der Pligt, J. (1997). Categories of category accessibility: The impact of trait concept versus exemplar priming

2/15/2011 7:47:02 PM

SOCIAL LEARNING AND CONNECTIONISM

on person judgments. Journal of Experimental Social Psychology, 33, 47–76. Stroessner, S. J., Hamilton, D. L., & Mackie, D. M. (1992). Affect and stereotyping: The effect of induced mood on distinctiveness-based illusory correlations. Journal of Personality and Social Psychology, 62, 564–576. Trope, Y., & Gaunt, R. (2000). Processing alternative explanations of behavior: Correction or integration? Journal of Personality and Social Psychology, 79, 344–354. Van Hamme, L. J. (1994). Associative and statistical accounts of cue competition in causality judgments. Unpublished doctoral dissertation, University of Iowa, Iowa City. Van Overwalle, F. (1996). The relationship between the Rescorla-Wagner associative model and the probabilistic joint model of causality. Psychologica Belgica, 36, 171–192. Van Overwalle, F. (1997) Dispositional attributions require the joint application of the methods of difference and agreement. Personality and Social Psychology Bulletin, 23, 974–980. Van Overwalle, F. (2003) Acquisition of dispositional attributions: Effects of sample size and covariation. European Journal of Social Psychology, 33, 515–533. Van Overwalle, F. (2006) Discounting and augmentation of dispositional and causal attributions. Psychologica Belgica, 46, 211–234. Van Overwalle, F. (2007). Social connectionism: A reader and handbook for simulations. New York, NY: Psychology Press. Van Overwalle, F. (2009). Social cognition and the brain: A meta-analysis. Human Brain Mapping, 30, 829–858. Van Overwalle, F., & Baetens, K. (2009). Understanding others’ actions and intentions by mirror and mentalizing systems: A meta-analysis. Manuscript submitted for publication. Van Overwalle, F., & Jordens, K. (2002). An adaptive connectionist model of cognitive dissonance. Personality and Social Psychology Review, 6, 204–231.

16-Schachtman-16.indd 375

375

Van Overwalle, F., & Labiouse, C. (2004). A recurrent connectionist model of person impression formation. Personality and Social Psychology Review, 8, 28–61. Van Overwalle, F., & Siebler, F. (2005). A connectionist model of attitude formation and change. Personality and Social Psychology Review, 9, 231–274. Van Overwalle, F., & Van Rooy, D. (2001a). When more observations are better than less: A connectionist account of the acquisition of causal strength. European Journal of Social Psychology, 31, 155–175. Van Overwalle, F., & Van Rooy, D. (2001b). How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626. Van Rooy, D., Van Overwalle, F., Vanhoomissen, T., Labiouse, C., & French, R. (2003). A recurrent connectionist model of group biases. Psychological Review, 110, 536–563. Wasserman, E. A., Kao, S-F., Van Hamme, L., Katagiri, M., & Young, M. E. (1996) Causation and association. The psychology of learning and motivation, 34, 207–264. Weber, R., & Crocker, J. (1983). Cognitive processes in the revision of stereotypic beliefs. Journal of Personality and Social Psychology, 45, 961–977. Wells, G. L., & Harvey, J. H. (1977). Do people use consensus information in making causal attributions? Journal of Personality and Social Psychology, 35, 279–293. Wells, G. L., & Ronis, D. L. (1982). Discounting and augmentation: Is there something special about the number of causes? Personality and Social Psychology Bulletin, 8, 566–572. Williams, D. A., & Docking, G. L. (1995). Associative and normative accounts of negative transfer. The Quarterly Journal of Experimental Psychology, 48A, 976–998.

2/15/2011 7:47:02 PM

CHAPTER 17 Application of Associative Learning Paradigms to Clinically Relevant Individual Differences in Cognitive Processing Teresa A. Treat, John K. Kruschke, Richard J. Viken, and Richard M. McFall

This chapter presents our efforts to examine clinically relevant individual differences in category learning with complex, socially relevant stimuli, highlighting the generalizability of cognitive scientists’ models and paradigms for the investigation of normative learning processes with simple, artiﬁcial stimuli. We document the substantial inﬂuence of individual variability in perceived dimensional salience on category learning, which extends the well-established link between normative perceived salience and category learning. It also appears that the complex, socially relevant stimuli of interest to some clinical researchers may be processed in a more holistic fashion than the artiﬁcial stimuli of primary interest to cognitive researchers, which typically are processed more separably. More integral processing may diminish the role of attentionshifting mechanisms in clinically relevant category learning, suggesting the importance of future research on the enhancement of attention shifting. More generally, the present work illustrates the utility of translating associative-learning paradigms to address applied questions about clinically and socially relevant processing of complex stimuli.

Individual differences in cognitive processing have been implicated in a range of clinical problems, such as depression, anxiety, schizophrenia, sexual aggression, and disordered eating (e.g., Beck, 1976; Kelly, 1955; McFall, 1990). Clinical scientists have been slow, however, to capitalize on the wealth of theories, measurement strategies, and analytical approaches developed by quantitative cognitive scientists, even though these models and methods seem promising for the advancement of cognitive theories of psychopathology (Treat et al., 2007). Cognitive scientists also have been slow to evaluate the generalizability of their models and methods to the more complex circumstances characteristic of “real-world” processing. The limited exploration of the integrative area of quantitative clinical-cognitive science reﬂects, in part,

two fundamental differences between cognitive and clinical science. First, quantitative cognitive scientists typically focus on the development and evaluation of formal mathematical models of the normative operation of component cognitive processes, such as attention, memory, and learning. Clinical scientists, in contrast, often study individual differences in abnormal processing. Second, quantitative cognitive scientists commonly study processing of simple, artiﬁcial stimuli that vary along readily identiﬁable dimensions, that are perceived similarly across persons, and with which persons frequently have limited experience. In contrast, clinical scientists more frequently study individual differences in processing of more complex, socially and emotionally relevant stimuli that vary along numerous dimensions, that may be perceived

376

17-Schachtman-17.indd 376

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

quite differently across persons, and with which persons often have prior experience. Thus, the application of cognitive science methods to clinical problems necessitates addressing the representation and assessment of individual differences, as well as far greater stimulus complexity. This chapter provides an overview of our efforts to draw on contemporary cognitive science to examine clinically relevant individual differences in category learning with complex, socially relevant stimuli. Cognitive scientists long have recognized the fundamental importance of category learning as a core cognitive process (see Ashby & Maddox, 2005; Kruschke, 2005a, for reviews). Clinical researchers tend to focus on static characterizations of cognitive processing, but examination of the way in which processing changes through time could be quite fruitful, whether the variation occurs naturally over different time scales, in response to feedback, or as a function of a theoretically relevant manipulation that is associated with exacerbation or amelioration of the clinical phenomenon. We focus in this chapter on learning as a function of feedback, because socially relevant learning processes presumably inﬂuence the development, maintenance, and modiﬁcation of our interpersonal beliefs and behaviors. Moreover, the provision of trial-by-trial feedback in structured category-learning protocols ultimately may serve as a useful prevention or intervention strategy. The approach that we adopt treats participants’ perceptual organization of stimuli as a representational base for the operation of learning processes. Thus, characterization of individual differences in perceptual organization is central to our investigations of individual differences in category learning. We focus in particular on individual differences in the perceived salience of stimulus dimensions as an important determinant of individual differences in category learning with socially relevant stimuli. We document that participants show far better performance on category structures that are congruent with their underlying perceptual organization, and they struggle to learn category structures that are incongruent with the perceptual organization that they bring to the task.

17-Schachtman-17.indd 377

377

The ﬁrst section provides a detailed overview of our use of cognitive-science methods to characterize perceptual organization and categorylearning processes with complex stimuli. The second and third sections describe our use of these methods to characterize (a) individual differences in men’s perceptual organization of and learning about women’s affect (positive or negative) and physical appearance (physically exposed or not), with implications for our understanding of sexual aggression; and (b) individual differences in women’s perceptual organization of and learning about other women’s facial affect (happy or sad) and body size (heavy or light), with implications for our understanding of disordered eating.

PERCEPTUAL ORGANIZATION AND CATEGORY LEARNING Perceptual Organization

Perceptual organization refers to the representation and organization of incoming stimuli in terms of their underlying psychological attributes. We most commonly assess perceptual organization using a similarity-ratings paradigm, in which participants judge the similarity of pairs of stimuli on a scale anchored by “very different” and “very similar.” For example, participants might view numerous pairs of photographs of women who vary in terms of their affect (negative to positive) and physical exposure (covered to exposed). On a single similarityratings trial, the participant might evaluate the similarity of a physically exposed and happy woman to a physically unexposed and happy woman. Because these two photos differ only on degree of exposure, not on affect, a rating of “very different” would suggest that the participant perceives physical exposure to be more salient than affect, whereas a rating of “very similar” would suggest the opposite. Participants are told that there are no right or wrong answers and are encouraged to respond quickly with their ﬁrst impression of the photo pair’s similarity. Note that this task neither speciﬁes the stimulus attributes of interest nor directs participants to attend to particular stimulus attributes, thus

2/17/2011 10:59:41 AM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

17-Schachtman-17.indd 378

2

1 Stimulus C Affect

providing a relatively implicit assessment of participants’ perceptual organizations. A multidimensional scaling (MDS) analysis of participants’ similarity ratings provides a spatial representation of the group-level perceptual organization or “psychological space,” in which the perceived similarity between two stimuli is modeled as a decreasing function of the distance between the perceived values of two stimuli (Davison, 1992; Treat et al., 2002). Thus, two stimuli that are judged to be very similar are scaled much closer in the psychological space than two stimuli that are judged to be very dissimilar. For example, the upper panel of Figure 17.1 displays the psychological space of 24 photo stimuli that present women varying along physical exposure and affect dimensions. Stimuli A and B, which depict physically exposed women expressing negative affect, are scaled close together, reﬂecting the participant’s perception that they are very similar to one another. In contrast, this participant judged both stimuli to be very dissimilar to stimulus C, a physically unexposed woman expressing positive affect, who is scaled across the psychological space from stimuli A and B. The metric used to compute the distance between stimuli reﬂects assumptions about the extent to which the stimulus dimensions are processed in a more separable or integral manner. The Euclidean metric is used when the stimulus dimensions are processed more holistically, or integrally, such as when evaluating color patches that vary in hue and saturation (Nosofsky & Palmeri, 1996; Shepard, 1964). The Euclidean distance between two stimuli is simply the length of a straight line between them. In contrast, the city-block metric is used when the stimulus dimensions are perceived more distinctively, or separably, such as when evaluating objects that vary in size and orientation. The city-block distance between two stimuli is the sum of the distances between them along each stimulus dimension. In our experience, the better ﬁtting metric for the more complex, ecologically valid stimulus sets of interest to clinical researchers often is Euclidean, rather than city-block, because the dimensions are difﬁcult to isolate. The worse ﬁt

0

–1

Stimuli A & B Category J

–2 –2

–1

Category F 0

1

2

Physical Exposure 2

Stimulus C

1 Affect

378

0 Stimuli A & B

–1 Category J –2 –2

–1

Category F 0

1

2

Physical Exposure

Figure 17.1 Multidimensional spatial representation of perceptual organizations of participants who perceive exposure to be relatively more salient than affect (top) and who perceive affect to be relatively more salient than exposure (bottom). See text for further information.

of the city-block metric suggests that the dimensions may be difﬁcult to attend to selectively (Nosofsky & Palmeri, 1996), since the city-block metric entails independent analysis of the dimensions. In contrast, the appropriate metric for the simpler, artiﬁcial stimuli of interest to cognitive scientists typically is city block, and it is much easier to attend selectively to separably processed dimensions. As we will see, the extent to which stimulus dimensions are processed separably versus integrally has signiﬁcant implications for the operation of category-learning processes with more complex stimuli. In the weighted MDS model, individual differences in participants’ similarity ratings are

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

modeled as participant-speciﬁc weighting of the stimulus dimensions (Carroll & Chang, 1970). Conceptually, these “salience weights” stretch and shrink the dimensions of the group psychological space. The upper panel of Figure 17.1 presents the psychological space of a participant who is inﬂuenced much more by women’s physical exposure than by their affect. The large salience weight for physical exposure serves to increase the distance between the more and less exposed women, which reﬂects this participant’s perception that more and less exposed women are very dissimilar. In contrast, the small salience weight for affect shrinks the distance between the women displaying positive versus negative affect, consistent with the participant’s judgment that women displaying positive and negative affect are not particularly dissimilar. The lower panel of Figure 17.1 depicts the psychological space of a participant who is inﬂuenced more by women’s affect than by physical exposure. Thus, the physically exposed and unexposed women are scaled closer together than the women exhibiting positive and negative affect. A particularly nice feature of the weighted MDS model is its simultaneous representation of both groupand participant-speciﬁc aspects of perceptual organization: Both the dimensions spanning the psychological space and the organization of the stimuli within each dimension are assumed to be shared by participants, whereas the relative salience of each dimension is allowed to vary across participants. It is important to distinguish perceived dimensional salience from selective attention to dimensions. Perceived dimensional salience is relatively static and enduring, indicating the default allocation of attention to the dimensions within the context of a particular stimulus set. Selective attention, on the other hand, refers to relatively dynamic reallocation of attention, whereby processing of dimensions might be ampliﬁed or attenuated selectively over a brief time span. For example, a person might display a perceptual organization in which affect is highly salient and exposure is relatively ignored. This relative salience would manifest itself in similarity ratings. The same person, however, might be able to reallocate attention selectively

17-Schachtman-17.indd 379

379

if a different task demanded it (e.g., when learning an exposure-relevant category structure). Reallocation of attention might prove to be difﬁcult for these stimuli, however, because scaling studies suggest that the dimensions are integral. We also have relied on a prototypeclassiﬁcation task to provide estimates of the perceived salience of stimulus dimensions, since providing similarity ratings for all possible pair of stimuli (including prototypes) places a signiﬁcant burden on participants (e.g., judging the similarity of all possible pairs of 24 stimuli takes approximately 30 minutes). In the classiﬁcation task, participants ﬁrst view two prototypical photo stimuli that vary along both theoretical dimensions of interest. For example, a “Type D woman” might express positive affect and be normatively heavy, whereas a “Type K woman” might express negative affect and be normatively light. Participants study the two prototypes for 10–15 seconds, then freely classify each of the remaining stimuli as an example of a Type D or a Type K woman, without corrective feedback. Because the prototypes vary along both theoretically relevant dimensions, participants can base their classiﬁcations on either or both dimensions. Here, too, the task provides a relatively implicit assessment of participants’ perceptual organizations, because it neither speciﬁes the stimulus attributes of interest nor directs participants to attend to particular stimulus attributes. Multiple logistic-regression techniques are used to estimate individual differences in perceptual organization during the classiﬁcation task. A participant’s classiﬁcation judgments are regressed onto normative data for the stimulus dimensions (e.g., a separate undergraduate sample’s average judgments of the affect and body size of the women depicted in the photos). In other words, the normative data for affect and body size serve as two predictors of each participant’s dichotomous classiﬁcation decisions. The slope estimates from these analyses reﬂect the change in the probability of classifying a stimulus into a particular category that can be predicted by the normative stimulus values for affect or body size. For instance, each participant’s slope (or utilization coefﬁcient) for affect reﬂects the expected increase in the probability

2/17/2011 10:59:41 AM

380

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of classifying a stimulus with the positive-affect prototype for every unit increase in the normative value of affect for the stimulus. Viken, Treat, Nosofsky, McFall, and Palmeri (2002) demonstrated a strong association between utilization coefﬁcients (based on a prototype-classiﬁcation task) and the corresponding salience weights (based on a similarity ratings task) for body size and affect, with an average correlation of 0.75. Thus, we treat the utilization estimates from the prototype-classiﬁcation tasks as indicators of perceived dimensional salience. Category Learning

Category learning in the present context refers to the placement of stimuli into categories with feedback about the accuracy of classiﬁcation, although experimenter instruction as to the stimulus characteristics on which to base classiﬁcations remains absent. To date, we have relied on category structures deﬁned by a single central boundary along a single dimension. For example, a participant might view photos of women who vary along physical exposure and affect dimensions, classify the woman depicted in each photo as a member of one of two categories with arbitrary labels (e.g., “Category F” and “Category J”), and then receive feedback on the accuracy of his classiﬁcations (e.g., “Correct! She is a member of Category J.”). Note that both panels of Figure 17.1 include a category boundary indicated by a dashed line that is perpendicular to the physical exposure axis of the psychological space. In this case, the participant would be learning to classify physically exposed women as members of Category F and physically unexposed women as members of Category J. Participants are told that initially they will be guessing, because they have not been told the basis for the feedback. Participants also are informed that the basis for the feedback might change during the course of the task, and that they should attempt to learn the new category labels for the stimuli if this occurs. If a shift to an affect category structure occurred, then the participant might have to learn to classify women expressing positive affect as members of Category F and women expressing negative affect as members of Category J.

17-Schachtman-17.indd 380

The formal process models of category learning upon which we rely assume that the underlying spatial representation of the stimuli (i.e., the perceptual organization) is the foundation for the category-learning processes (e.g., Kruschke, 1992; Kruschke & Johansen, 1999; Nosofsky, 1992). These models predict that participants will learn a category structure based on a particular stimulus feature more quickly when stimuli in different categories are perceived to be very dissimilar and stimuli in the same category are perceived to be very similar. This prediction follows purely from generalization of learning from one exemplar to another in close proximity. In particular, when a category distinction aligns with a salient dimension, the distinction should be learned rapidly, because the stretching of a salient dimension increases the distance between stimuli in different categories. In our applications with complex and socially relevant stimuli, individual differences in category learning should be related to individual differences in the perceived salience of stimulus dimensions. Note that, in this analysis, dimensional salience always is a perceived, rather than an intrinsic, property of a stimulus dimension. Consider, for example, differences in the expected rate at which an exposure category structure is learned by the participants whose perceptual organizations are depicted in Figure 17.1. The participant for whom exposure is more salient than affect (in the upper panel) should acquire the exposure category structure rapidly, because the stimuli falling into the same category are perceived to be quite similar (i.e., they are close to one another in the psychological space), whereas the stimuli falling in different categories are perceived to be very dissimilar. In contrast, the participant for whom affect is more salient than exposure (in the lower panel) should learn the exposure category structure much more slowly, because the structure is far less congruent with his perceptual organization. In both of the studies described in this chapter, we examine whether category-learning performance is congruent with participants’ perceptual organizations. We also ﬁt Kruschke and Johansen’s (1999) connectionist model of category learning, RASHNL, which implements

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

three mechanisms that may underlie participants’ responses. The ﬁrst mechanism sets the initial relative perceived salience of the psychological dimensions of physical exposure and facial affect. For example, participants who initially perceive affect to be more salient than exposure should be at a distinct advantage over their counterparts when learning the affect category structure, because of the greater perceived similarity of the stimuli within the same category and the greater perceived dissimilarity of the stimuli in different categories. The second mechanism is shifting of attention toward relevant dimensions and away from irrelevant dimensions. This attentional shifting allows participants to learn category structures by modifying their perceptual organization to be more consistent with the demands of the category structure. In other words, participants who initially perceive exposure to be more salient than affect could learn the affect category structure by increasing their attention to affect and decreasing their attention to exposure, thus modifying their perceptual organization to make it more similar to that of an initially affect-oriented participant. This shift in dimensional attention would minimize intracategory distances and maximize intercategory distances, and it could happen quite rapidly. The third mechanism is gradual strengthening of associations between regions of the psychological space and correct category responses. The association-learning mechanism produces incremental improvement in performance for speciﬁc exemplars and their nearby neighbors, whereas attention shifting affects the relative distribution of all exemplars simultaneously. Distinguishing these mechanisms in category learning could be beneﬁcial for our understanding of cognitive processing of these complex, socially relevant stimuli. As discussed earlier, the initial perceived salience of the psychological dimensions (i.e., the ﬁrst mechanism mentioned) is already a well-established predictor of category learning with simple, artiﬁcial stimuli. Consideration of category learning about more complex stimuli with which participants have prior experience allows us to investigate the potential inﬂuence of systematic individual

17-Schachtman-17.indd 381

381

differences in perceived dimensional salience on category learning as well. Extensive evidence within cognitive science also supports a role for attention shifting (the second mechanism) as a primary strategy when learning category structures with artiﬁcial stimuli that are composed of separable dimensions (e.g., Kruschke, 1993, 1996). Nosofsky and Palmeri (1996) demonstrated that attention shifting played a far less central role, however, when learning structures with artiﬁcial stimuli composed of integral dimensions (e.g., color patches that vary in hue and saturation). Thus, category-learning studies have demonstrated that artiﬁcial stimuli that are best scaled by Euclidean metrics in similarity rating are best modeled with little attention shifting in category learning. Because the dimensions of the more ecologically valid stimuli of interest to clinical researchers frequently will be processed integrally, rather than separably, this may diminish the role of attention shifting in category learning with these stimuli. Moreover, a relative inability to shift attention rapidly might enhance the importance of individual differences in perceived dimensional salience for applied category learning. In this case, gradual changes in the associations between regions of the psychological space and the correct category label (the third mechanism) may become more central to the acquisition of applied category structures, particularly those that are incongruent with a person’s initial allocation of attention across dimensions. Thus, the relative importance of the three learning mechanisms instantiated in RASHNL may differ meaningfully as a function of the nature of the stimuli of primary interest to cognitive and clinical scientists.

INDIVIDUAL DIFFERENCES IN MEN’S PERCEPTIONS OF AND LEARNING ABOUT WOMEN Background

Social information-processing models of sexually coercive or aggressive behavior between acquaintances specify a critical role for the way in which men process information about women (e.g., Farris, Treat, Viken, & McFall, 2008; Johnston &

2/17/2011 10:59:41 AM

382

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Ward, 1996; McFall, 1990; Segal & Stermac, 1990). More speciﬁcally, theorists suggest that men at greater risk of exhibiting sexual aggression toward acquaintances attend relatively less to information about women’s affect or sexual interest and relatively more to women’s physical sexual attributes (e.g., physical exposure, sensuality, provocativeness, and sexual attractiveness). In 2001, we conducted a study to evaluate this hypothesized link between men’s risk status and their relative attention to affect and physical exposure dimensions of full-body photos of women in newsstand magazines (Treat, McFall, Viken, & Kruschke, 2001). This study provided an opportunity to examine individual differences in participants’ performance when learning category structures that were based on either women’s affect or physical exposure. This allowed us to evaluate whether individual differences in perceptual organization facilitated or inhibited performance on the category-learning tasks, depending on the congruence of the participant’s perceptual organization with the category structure to be learned. We also ﬁt Kruschke and Johansen’s (1999) RASHNL model to the category-learning data, so that we could evaluate the role of the three mechanisms described earlier. We hoped that attention shifting would play a central role in young men’s category learning about women’s affective and appearance-based cues, because a participant’s ability to shift attention between these cues in an artiﬁcial categorylearning task might indicate that his relative attention to such dimensions in “real-world” social environments would be malleable as well. Methods

Stimuli were 26 color slides of Caucasian women who appeared either in newsstand magazines or mass-marketing catalogs. A separate sample of undergraduate males rated these stimuli along 10-point scales for several relevant dimensions, including affect and exposure. The average ratings along the affect and exposure dimensions for each stimulus served as “normative ratings” for the stimuli. Only 14 of the 26 stimuli were used in the similarity-ratings task, given the prohibitively large number of all possible pairs

17-Schachtman-17.indd 382

that would need to be rated if we had included all 26 stimuli in this task. All 26 stimuli were used in the category-learning tasks. The normative ratings for affect and exposure were used to classify the stimuli as having high or low values on the exposure and affect dimensions for purposes of providing feedback in the two categorylearning tasks. Seventy-one undergraduate males ﬁrst completed a similarity-ratings task, in which they judged the similarity of all possible pairs of 14 photos of women on a 10-point scale ranging from 0 = very different to 9 = very similar. After ﬁnishing an implicit classiﬁcation task that is not relevant to the current discussion, participants then completed two category-learning tasks. They viewed individual photos of 26 women, judged whether each woman did or did not have an unspeciﬁed characteristic, and received trialby-trial feedback on the accuracy of their classiﬁcations. The feedback was based on the woman’s normative affect in one task and on the woman’s level of physical exposure in the other task. Four blocks of trials were completed for each category structure. Finally, participants responded to the Heterosocial Perception Survey (McDonel & McFall, 1991), which indexed a participant’s perception of the justiﬁability of a man continuing to make sexual advances in the face of increasing resistance by a female acquaintance. Participants whose justiﬁability ratings declined less rapidly as the woman’s negativity increased were presumed to be at higher risk of exhibiting sexually aggressive behavior toward acquaintances. This Rape Justiﬁability Score was computed for all participants, with higher values indicating a greater propensity toward sexual aggression. Results

Weighted multidimensional scaling techniques were used to quantify individual differences in the relative perceived salience of the physicalexposure and facial-affect dimensions. A twodimensional conﬁguration was imposed, with the stimulus coordinates constrained to be equal to the normative ratings for exposure and affect,

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

17-Schachtman-17.indd 383

1

Proportion Correct

and salience weights for the two dimensions were estimated for each participant. Average salience weights for physical exposure and affect indicated that physical exposure was perceived to be far more salient than affect in this particular stimulus set. A single index of the relative salience of exposure versus affect was constructed from the two salience weights (MacCallum, 1977). Marked variability in relative salience scores was observed, as expected. Participants whose relative salience score fell in the upper and lower terciles were classiﬁed as exposure oriented (n = 24) and affect oriented (n = 24), respectively. As expected, exposure-oriented participants showed signiﬁcantly higher Rape Justiﬁability Scores than affect-oriented participants. Exposure-oriented participants’ perception of the justiﬁability of unwanted sexual advances depended less on the degree of negative reaction from the hypothetical woman than it did for affect-oriented participants. Thus, those participants who perceived affect to be relatively more salient on the similarity-ratings tasks demonstrated greater sensitivity to the negativity of a woman’s affect in the justiﬁability rating task. Proportion correct was computed for each of the four blocks of both the exposure and affect category-learning tasks. Preliminary analyses indicated that the order in which the two category structures was completed did not inﬂuence the ﬁndings, so it was not included in subsequent analyses. A repeated-measures analysis with a robust estimator was applied to the data with Group (exposure oriented or affect oriented), Task (exposure or affect), and Block as factors. A signiﬁcant Task effect emerged, Wald c 2 (1) = 27.39, p < 0.001, with average performance on the exposure-relevant structure (M = 0.90) far exceeding performance on the affectrelevant structure (M = 0.81). The predicted Group x Task interaction also emerged, Wald c 2 (1) = 4.01, p < 0.05; both affect-oriented and exposure-oriented participants showed better performance across blocks when learning the category structure that was more congruent with their perceptual organization. These effects are illustrated in Figure 17.2. In sum, both average perceived salience and individual differences in perceived salience predicted performance

383

0.9

0.8 AO-Affect Task EO-Affect Task AO-Exposure Task EO-Exposure Task

0.7

0.6 1

2

3

4

Block

Figure 17.2 Perceptual organization and category

structure (Task) inﬂuence performance on exposure and affect category-learning tasks in men’s learning study. AO. affect oriented; EO, exposure oriented. Bars correspond to standard error of the means.

on the exposure and affect category-learning tasks. We ﬁt RASHNL (Kruschke & Johansen, 1999) to the proportion-correct values of the exposure- and affect-oriented groups on each block in the learning task, using a stepwise search algorithm to minimize the root-mean-squared deviation between the observed and predicted proportion-correct values. The Euclidean metric was used to compute interstimulus distances, because preliminary model ﬁts indicated that participants perceived the stimulus dimensions integrally, rather than separably. As expected, the RASHNL model ﬁt best with group-speciﬁc estimates of initial perceived salience, such that exposure-oriented participants had a larger exposure/affect salience ratio than affect-oriented participants. These model results are consistent with the ﬁndings described earlier; however, ﬁtting a process model to the data also provided group-speciﬁc estimates of the perceived relative salience of the stimulus dimensions during category learning. Unexpectedly, the RASHNL model suggested that reallocating attention toward relevant dimensions and away from irrelevant dimensions did not contribute to participants’ learning. The best-ﬁtting attention-shift rate was zero,

2/17/2011 10:59:41 AM

384

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and the best-ﬁtting association learning rates did not differ for exposure- and affect-oriented groups. In summary, ﬁts by the RASHNL model revealed that the exposure-oriented and affectoriented groups differed only in the relative perceived salience of the dimensions. The groups learned at the same rate, and they did not learn to attend selectively to the dimensions. Conclusions

As expected, young men’s perceptions of young women’s physical-exposure and affective information showed strong relationships with men’s performance on exposure and affect category structures. Overall performance was better on the exposure than the affect category structure, consistent with the far greater average salience of exposure than affect in weighted multidimensional scaling analyses. Participants also learned a category structure much more rapidly when it was congruent with their underlying perceptual organization. That is, exposure-oriented participants performed better on the exposure category structure than affect-oriented participants, whereas affect-oriented participants performed better on the affect category structure than exposure-oriented participants. Fitting RASHNL to the category-learning data indicated that adaptive shifting of attention toward relevant stimulus dimensions and away from irrelevant stimulus dimensions did not play a role in participants’ learning, because the best-ﬁtting value for the attention shifting parameter was zero. Rather, participants gradually learned to associate regions of the psychological space with the correct category label. The lack of attentional shifting presumably reﬂects the integral, rather than separable, nature of the stimulus dimensions in the current application. Holistic processing of stimulus dimensions implies that it is difﬁcult to shift attention toward or away from speciﬁc dimensions (Nosofsky & Palmeri, 1996). It would be useful in everyday life, however, if people were able to attend selectively to these dimensions, such as when a person needs to shift attention away from a potential sexual partner’s physical exposure and toward the partner’s affective expressions of

17-Schachtman-17.indd 384

sexual interest. Thus, it would be proﬁtable for future research to investigate the conditions under which the distribution of attention across dimensions exhibits greater ﬂexibility and malleability, even when the stimulus dimensions are perceived integrally. Overall, these ﬁndings suggest that the wellestablished normative link between dimensional salience and category learning with simple, artiﬁcial stimuli (e.g., Kruschke, 1992; Kruschke & Johansen, 1999) indeed generalizes to the association between individual differences in perceptual organization and learning with more complex, socially and emotionally relevant stimuli, although the processes underlying learning may differ. Numerous theories suggest that the etiology or course of clinical phenomena is inﬂuenced by features of participants’ perceptual organizations, such as attention to stimulus dimensions (e.g., Beck, 1976; Kelly, 1955; McFall, 1990). The present study provides further support for this link by demonstrating that young men who perceived women’s physical characteristics to be more salient than their affect also perceived continued sexual advances in the face of a woman’s increasing resistance to be more justiﬁed, relative to participants who perceived women’s affect to be more salient. Clinical researchers have tended to focus far more on the role that attentional processes play in psychopathology, but the present work highlights the potential utility of translating the models and methods of associative learning to clinically relevant investigations as well. Performance on category-learning paradigms with more complex stimuli may provide a window into the operation of “real-world” social learning processes that presumably underlie socially relevant attitudes and behaviors.

INDIVIDUAL DIFFERENCES IN WOMEN’S PERCEPTIONS OF WOMEN Background

Clinical researchers increasingly have focused on the role of cognitive factors, such as distorted processing of shape- and weight-related

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

information, in the etiology and maintenance of eating-disorder symptoms and in the development of cognitive-behavioral treatments for these symptoms (e.g., Cooper 2005; Fairburn, Cooper, & Shafran, 2003; Lee & Shafran, 2004). Vitousek and colleagues, for example, proposed that increased attention to and memory for shape-, weight-, and eating-related information inﬂuence the development and maintenance of eating-disorder symptoms (e.g., Vitousek, 1996; Vitousek & Hollon, 1990). Decreased attention to and memory for affective information also might play a signiﬁcant role. Many women with eating disorders display marked deﬁcits in interpersonal problem solving and emotion regulation, and they indicate that negative mood and social interactions are common triggers for eating-disordered behaviors (e.g., Lingswiler, Crowther, & Stephens, 2006; McFall, Eason, Edmondson, & Treat, 1999; Smyth et al., 2007). Such difﬁculties may reﬂect, in part, impoverished processing of affective information. In prior work, we have demonstrated that young women who report clinically signiﬁcant disordered eating patterns (“High-Symptom women”) show altered processing of other women’s weightand affect-related information, as presented in full-body photos (Treat, Viken, Kruschke, & McFall, 2010; Viken et al., 2002). High-Symptom women, relative to Medium- and Low-Symptom women, showed greater attention to body size and less attention to affect in both similarityratings and prototype-classiﬁcation tasks. HighSymptom women also showed better memory for body size and worse memory for affect, relative to the remaining participants, in a recognition-memory task. Thus, the operation of other higher order component cognitive processes, such as category learning, also merits investigation. The present work examines young women’s perceptions of full-body photos of other women. The women depicted in the photos varied in affect (negative to positive) and in body size (lighter to heavier). We assessed the perceived salience of the photo dimensions, and then we examined how perceived salience inﬂuenced learning in a multiphase category-learning task. The phases of the category-learning task were based loosely on

17-Schachtman-17.indd 385

385

learning experiences that young women might encounter in the real world. Our society inundates young women with social environments in which thinness is glamorized and increased body size is a strong negative predictor of a variety of indicators of success. Moreover, women who struggle with disordered eating self-expose themselves to television shows and magazines that promote a thin ideal at greater rates than their peers (e.g., Botta, 2003; Stice, Schupak-Neuberg, Shaw, & Stein, 1994; Tiggemann, 2003). Thus, women’s immediate social environments provide highly inﬂuential “feedback” on the relevance of body size to happiness. Only gradually and later might affect emerge as a better indicator of happiness. In particular, if a young woman later undergoes therapy for an eating disorder, the therapy might include, implicitly or explicitly, attempts to reorient attention away from body size. In summary, a young women initially might experience real-world “training” that body size is relevant, with affect being only a later-learned cue. Finally, the young woman might experience some sort of training to reorient attention away from the initially learned cue. Participants were assigned randomly to one of four learning conditions, each of which contained multiple phases of training. Table 17.1 presents the conditions and learning phases. Because we wanted to counterbalance which dimension initially was relevant, some participants learned an initial category structure for which body size was relevant (the “Body-Size Initial” condition). Other participants learned an initial structure for which affect was relevant (the “Affect Initial” condition). The “Body-Size Initial” structure is shown in Figure 17.3. Each panel of Figure 17.3 shows the two-dimensional stimulus space, with affect plotted horizontally and body size plotted vertically. In the Initial Phase of learning in the BodySize Initial condition, photos of lighter women displaying neutral affect were to be classiﬁed into Category F, and photos of heavier women displaying neutral affect were to be classiﬁed into Category J. Only stimuli drawn from these two regions of the stimulus space were shown in the Initial Phase of the category-learning task. Participants received feedback on their

2/17/2011 10:59:41 AM

386

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Table 17.1 Design of Phased-Learning Task in Women’s Learning Study Learning Phase Condition/Group

Initial

Redundant

Transfer

Shift

N

Affect Initial

Affect

Affect + Body Size

Transfer

Body Size

63

Affect Control

—

Affect + Body Size

Transfer

Body Size

59

Body-Size Initial

Body Size

Affect + Body Size

Transfer

Affect

58

Body-Size Control

—

Affect + Body Size

Transfer

Affect

62

classiﬁcations, but they were not told the basis for the feedback. Thus, body size was relevant to the category label in the Initial Phase for participants in the Body-Size Initial condition, whereas variability in affect was highly restricted and irrelevant to the category label. We expected that performance in the Initial Phase would be highly congruent with participants’ perceptual organizations (e.g., participants who perceived body size to be more salient than affect would show far better initial performance in the Body-Size Initial condition). In the second Redundant Phase of learning for those in the Body-Size Initial condition, affect became a redundant relevant cue, in that body size and affect both perfectly predicted the category label. For example, you can see in the second panel of Figure 17.3 (under “Redundant”) that participants could make correct classiﬁcations based on either the woman’s affect or body

Body Size

Body Size

J

F

J OR F Affect

Affect

?

?

? Affect

F

Shift Body Size

?

J

Affect

Transfer Body Size

Body Size

Redundant

Initial

J

J

F

F Affect

Figure 17.3 Phased-learning schemata, for Body-

Size Initial condition in women’s learning study.

17-Schachtman-17.indd 386

size. Only stimuli drawn from the two regions of the stimulus space indicated by the letters “F” and “J” were shown in one of two versions of the Redundant Phase. Participants again received feedback on their classiﬁcations but were not told the basis for the feedback. We anticipated that the newly relevant dimension (i.e., affect) would not be learned as well, because the previously learned dimension (i.e., body size) already had proven to be a perfectly predictive cue for the category label. In other words, we anticipated that learning about the redundant relevant cue of affect would be “blocked” in the Body-Size Initial condition (e.g., Denton & Kruschke, 2006; Kamin, 1969). In contrast, participants in the Body-Size Control condition would not be expected to show blocking, because they did not experience the Initial Phase of learning. The third Transfer Phase in the categorylearning task presented photos from all four corners of the stimulus space, so we could assess utilization of the two dimensions. This Transfer Phase presented photos from the four regions of the stimulus space labelled with “?” in Figure 17.3 and recorded participants’ classiﬁcations. Critically, participants did not receive corrective feedback during the transfer trials. Suppose that a participant had experienced the Redundant phase in which she learned to classify lighter, unhappy women into Category F and heavier, happy women into Category J. If she were relying purely on body size when making these judgments, then we would expect her to place all lighter women in Category F and all heavier women in Category J, regardless of their affect. Alternatively, if she were using only affect as a basis for her judgments in the Redundant phase,

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

then she would place all unhappy women in Category F and all happy women in Category J, regardless of their body size. Thus, the pattern of classiﬁcations in the Transfer Phase was diagnostic of a participant’s utilization of bodysize and affect information. We anticipated that participants in the Body-Size Initial condition would transfer much more strongly to body size than to affect, because of the blocking of affect. Participants in the Body-Size Control condition, in contrast, were expected to show less transfer to body size, because affect would not have been blocked for them. The ﬁnal Shift Phase in the category-learning task made relevant the dimension that was irrelevant in the Initial Phase. Thus, for participants in the Body-Size Initial condition, women displaying positive affect were to be classiﬁed into Category F, whereas women displaying negative affect were to be classiﬁed into Category J. You can see that one way to perform well on this task is to ignore the initial dimension of body size and make classiﬁcation decisions based on affect. Photos of women from the four regions of the stimulus space labeled in the Shift panel in Figure 17.3 were presented, and participants received corrective feedback. This ﬁnal phase was intended to encourage a shift of attention to the initially irrelevant dimension. We expected that participants in the Body-Size Initial condition would show worse performance in the Shift Phase, relative to those in the Body-Size Control condition, because participants in the former condition would be attempting to learn a category structure based on a blocked cue. Overall, we anticipated that the blocking manipulation in the Redundant Phase would result in suppressed learning about the blocked cue for participants in the experimental conditions (i.e., the Affect Initial and Body-size Initial conditions), as assessed in the Transfer and Shift phases. Notably, this prediction presupposes that the current design is analogous to the classic blocking design, in which the blocked cue does not appear at all in the initial phase. In the current design, in contrast, the blocked cue appears in the initial phase with a middling value (i.e., either neutral affect or moderate body size).

17-Schachtman-17.indd 387

387

According to RASHNL, either attentional or associative learning mechanisms could underlie the predicted blocking effects. More speciﬁcally, among participants experiencing the Initial Phase of training, decreased transfer to the blocked cue and suppressed shift learning about the blocked cue could reﬂect (a) only generalization from the items learned in the Initial Phase, or (b) generalization and a shift of attention toward the initially relevant unblocked cue and away from the initially irrelevant blocked cue. The predicted main effects of blocking are qualitatively similar regardless of whether attention shifting plays a role, and one of the key beneﬁts of quantitative modeling is parametric estimation of the magnitude of attention shifting. We hoped to demonstrate that attention shifting played a role in learning but recognized that integral processing of the stimulus dimensions might preclude this possibility. Perceptual organization also was predicted to exert a strong inﬂuence on initial learning, transfer performance, and later learning, whereby performance was superior on category structures that were congruent with the participants’ perceptual organization. Thus, we expected that the blocking manipulation would not eliminate the inﬂuence of perceptual organization on later transfer and learning. Finally, we explored the interaction between perceptual organization and the blocking manipulation. Research on blocking in human category learning with simple artiﬁcial stimuli demonstrates that it is harder to block a more salient cue (Denton & Kruschke, 2006). To the extent that the classic blocking design and the current design operate analogously, we hoped to demonstrate that the blocking effect would be weaker when the to-be-blocked dimension was more salient. This prediction also hinged on separable processing of the stimulus dimensions, which presumably would be necessary to support learning via attention-shifting mechanisms. On the other hand, it might be the case that the stimulus dimensions were processed holistically, such that learning was driven only by associative mechanisms. In this case, the current design (involving middling values on the “blocked” dimension) and classic designs are

2/17/2011 10:59:41 AM

388

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

not analogous. Predictions in this case could be derived accurately only from model simulations, but qualitatively we would expect that if the initially irrelevant dimension were salient, then learning of the initial categories would be relatively difﬁcult, and transfer would be less consistent with the initially relevant dimension, compared to when the initially irrelevant dimension was not salient. The magnitude of the difference between experimental and control groups could not be predicted in advance, but it could be assayed via best-ﬁtting parameter values in the RASHNL model. No clinical information was obtained from participants in this study, because a prohibitively large number of participants would be necessary to examine the extent to which disordered eating status moderates the hypothesized effects. We know from our earlier work that variation in women’s perceptual organization of body-size and affect information is a reliable correlate of disordered eating patterns, however. Thus, a strong connection between individual differences in perceptual organization and category-learning performance would highlight the potential utility of examining the clinical relevance of individual differences in young women’s learning about body-size and affect categories structures in future research. Methods Participants

Two hundred forty-four undergraduate females received partial course credit for their participation in the study. Mean age was 19.47 (SD = 1.82), and 83.1% of participants self-identiﬁed their ethnicity as White/Caucasian. Photo Stimuli

Stimuli were pictures of 58 paid female models recruited from the university population. Each model was photographed in a white t-shirt and black stretch pants in front of a neutral background. Body size varied naturally, and each model was instructed to display both sad and happy expressions. A sample of 60 undergraduate women provided normative ratings of the photos along affect (unhappy to happy) and body

17-Schachtman-17.indd 388

size (slender to plump) dimensions. Mean normative ratings were used to classify each woman as light, moderate, or heavy along the body-size dimension. Women whose body size was judged to be in the lower third across all three facial expressions were classiﬁed as “light” (n = 15), and women whose body size was rated in the middle or upper third were classiﬁed as “moderate” (n = 12) or “heavy” (n = 18), respectively. Body-size classiﬁcations for the 13 remaining unique women varied across their facial expressions. The woman in each photo was classiﬁed as exhibiting “negative,” “neutral,” or “positive” affect if the mean normative rating of her affect fell in the bottom, middle, or upper third, respectively, of all mean affect ratings. Prototype-Classiﬁcation Tasks

Participants performed two classiﬁcation tasks, which were presented in a counterbalanced order across participants. At the beginning of the ﬁrst task, participants studied two prototypical photos that varied along both affect and bodysize dimensions (e.g., a “Type D woman” received normative ratings toward the extreme “heavy” end of the body-size dimension and toward the extreme “happy” end of the facial-affect dimension, whereas a “Type K woman” received normative ratings indicating that she was viewed as “light” and “unhappy”). After participants inspected the two prototypes, they classiﬁed each of 20 remaining photo stimuli as examples of one of the two types of women. Next, participants completed the same task with two new prototypes (e.g., a happy-light, “Type V” woman, and a sad-heavy, “Type N” woman). The paucity of photo stimuli precluded the use of different stimuli in the prototype-classiﬁcation and phased-learning tasks. Therefore, the stimuli used in the classiﬁcation task were the same as those viewed by participants in the Redundant, Transfer, and Shift Phases of the phased-learning task. This constraint necessitated construction of four versions of the classiﬁcation task, depending on whether affect and body size correlated positively or negatively in the Redundant Phase for participants in the ﬁrst two and the last two groups in the phased-learning task. The prototypes

2/17/2011 10:59:41 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

viewed in the prototype-classiﬁcation tasks were the same across all four versions of the tasks, however, and the prototypes were not used in the learning task, given their markedly greater familiarity. Phased-Learning Task

In all but the Transfer Phase of the categorylearning task, participants classiﬁed individual photos as members of Category F or Category J and received accurate trial-by-trial feedback on their classiﬁcations. Participants were assigned randomly to one of four learning conditions, which are displayed in Table 17.1. A schematic depiction of the learning phases for participants in one of the four learning conditions, the Body-Size Initial condition, also is provided in Figure 17.3. Stimulus presentation order was randomized separately for each participant within each block. Eight unique stimuli were presented in each of 10 blocks in the Initial Phase of learning. Four of the stimuli received mean normative ratings in the upper third of the distribution of mean ratings for affect or body size for all 174 potential stimuli, respectively, whereas the remaining four stimuli received mean ratings in the lower third. The mean ratings for the eight stimuli along the dimension that was not the basis for the category structure fell in the middle third of the distribution of normative ratings. For example, the eight stimuli viewed in the Initial Phase of the Body-Size Initial condition received mean normative ratings in the middle third of the distribution of affect ratings, as depicted in Figure 17.3. None of the photos presented in the Initial Phase had been seen previously by participants. Participants completed eight blocks of eight trials apiece in the Redundant Phase of learning. The shift to the Redundant Phase for participants in the Affect Initial and Body-Size Initial conditions was unannounced. In the Body-Size Initial and Body-Size Control conditions, four stimuli received mean normative ratings in the upper third of the ratings distribution for body size, and four stimuli received mean ratings in the lower third of the body-size-ratings distribution. Mean affect ratings for these two groups of

17-Schachtman-17.indd 389

389

stimuli fell in either the upper or lower third of the affect ratings distribution. As depicted in Figure 17.3, affect and body size correlated positively for half of the participants in each condition (i.e., only happy-heavy and sad-light women were presented) and negatively for the remaining participants (i.e., only happy-light and sadheavy women were presented). Analogously, in the Affect Initial and Affect Control conditions, four stimuli were judged normatively to be happy and four to be unhappy; average ratings for these stimuli along the body-size dimension fell in either the upper or lower third of the distribution of body-size ratings. Additionally, body-size and affect correlated positively for half of the participants and negatively for the other half. Structurally, the two control conditions were identical, although the stimuli used in the two conditions varied. All women presented in the Redundant Phase differed from those presented in the Initial Phase. All participants classiﬁed 16 stimuli twice and received no feedback in the Transfer Phase, as shown in the lower left corner of Figure 17.3 for the Body-Size Initial condition. According to normative-ratings distributions, four stimuli were happy-heavy, four were happy-light, four were sad-heavy, and four were sad-light. Half of the stimuli had been viewed in the Redundant Phase. In the ﬁnal Shift Phase of learning, participants in the Body-Size Initial and Body-Size Control conditions completed seven blocks of an effect category structure, which is shown in the lower right corner of Figure 17.3. Four of the stimuli presented on each block were judged to be happy, and four were judged to be unhappy; the body size of two of the former stimuli and two of the latter stimuli was classiﬁed as heavy, and the remaining stimuli were classiﬁed as light. Conversely, participants in the other two conditions learned a body-size category structure, in which four stimuli were light, four stimuli were heavy, and the affect classiﬁcations for these stimuli were orthogonal to the body-size classiﬁcations. All women presented in the Shift Phase had not been seen in earlier learning phases, although they had been viewed in the prototypeclassiﬁcation tasks.

2/17/2011 10:59:41 AM

390

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Procedure

Participants ﬁrst read and signed the consent form and then were seated in front of a computer in a subject-running booth. Participants entered their age and ethnicity and then completed the two prototype-classiﬁcation tasks and the phased-learning task, as described earlier. Finally, participants were debriefed and thanked for their participation. The experiment lasted approximately 55 minutes. Data Preparation Prototype-Classiﬁcation Data

Logistic-regression techniques were used to estimate individual differences in perceptual organization during the classiﬁcation task. The following logistic function was ﬁt to each participant’s classiﬁcation judgments for each of the two classiﬁcation tasks: P ( “T Type K” or “T Type V” | A =

1 1 + exp(−[(ak * AFF ) (b ( k*

and BS values ) ) + c])

,

where k = 1 or 2, depending on the task, and AFF and BS refer to standardized normative scale values for the stimuli along affect and body-size dimensions. The absolute value of the ratio of ak to bk indicated the relative perceived salience of affect and body size. As bk approached zero, values of this ratio became extreme and unstable. Thus, the arctangent of the ratio was taken. This value has a simple geometric interpretation in the stimulus space as the best-ﬁtting angle of a line that separates the two category responses. The ﬁnal measure of relative salience of affect and body size, that is, the perceptual organization score, was the average of these transformed ratios for the two tasks: Perceptual Organization score ⎛a ⎞ ⎛a ⎞ arctan ⎜ 1 ⎟ + arctan ⎜ 2 ⎟ ⎝ b1 ⎠ ⎝ b2 ⎠ = 2

Values for this measure ranged from 0.00 radians (or 0 degrees), when the participant relied exclusively on body size in making her

17-Schachtman-17.indd 390

classiﬁcation judgments, to 1.57 radians (or 90 degrees), when the participant utilized only affect when classifying the stimuli. Participants were classiﬁed as affect oriented if their relative salience score exceeded 1.22 radians (or 70 degrees), as body size oriented if their score was less than 0.349 radians (or 20 degrees), or as both oriented if their score lay between these two extremes. Over twice as many participants were classiﬁed as affect oriented (n = 107, 44.2%) rather than as body size oriented (n = 53, 21.9%), and one-third of the participants were classiﬁed as both oriented (n = 82, 33.9%). A chi-square test supported the independence of perceptual organization classiﬁcation and learning group, χ2 (6) = 4.337, p > 0.50, indicating that the three classes of perceptual organizations were distributed uniformly across the four learning groups. Transfer Data

Analogous logistic-regression techniques were used to quantify each participant’s relative utilization of affect and body size on the transfer trials in the phased-learning task. The formula shown earlier was ﬁt to each participant’s classiﬁcations of all 32 stimuli, the arctangent of the absolute value of the ratio of a to b was calculated, and the arctangent was transformed into degrees. The resulting values ranged between 0, which indicated perfect transfer to body size, and 90, which indicated perfect transfer to affect. Finally, these values were reﬂected for participants in the Body-Size Initial and Body-Size Control groups, so that values of 90 indicated perfect transfer to the initial cue and 0 indicated perfect transfer to the shift cue. Learning Data

Average percent correct was calculated for each participant for both the Initial and Shift Phases of learning. In the latter case, percent correct was calculated across only the ﬁrst three blocks, because effects on the ﬁrst three blocks were expected to be more revealing than when accuracies converged to ceiling levels in the later blocks. The pattern of results described later was similar when analogous analyses were conducted on average performance across all blocks, rather

2/17/2011 10:59:42 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

than the ﬁrst three blocks, but the effects were smaller in magnitude. Additionally, individual participants’ performance was examined on the last two blocks of the Redundant Phase, to determine whether their performance surpassed what would be expected on the basis of chance alone. The worst observed performance was 81.25% (i.e., correct response on 13 of 16 trials; n = 3). This would be expected less than 1.05% of the time, if the participant responded randomly, assuming a binomial distribution (n = 16 and p = 0.5). Thus, all participants’ data were retained for further analysis. Resampling Approach to Statistical Analyses

Resampling methods of statistical inference derive sampling distributions empirically, rather than theoretically, by sampling with replacement repeatedly from the observed sample and calculating the relevant test statistic on each resample (Good, 2006). The resulting distribution of test statistics serves as the empirically derived sampling distribution, and critical values can be obtained by reading off the relevant value in this distribution at the percentile of interest. In contrast, parametric statistical approaches specify the sampling distribution and relevant critical values by making stringent theoretical assumptions about the moments of the population distribution. Resampling approaches to statistical inference are preferred when the assumptions of parametric statistical approaches are violated severely. In the present case, a resampling approach was adopted for two primary reasons: one, most distributions were either bimodal or so severely skewed that they could not be transformed to normality without discretization and resulting loss of information; and two, variances tended to differ dramatically across the cells of the analyses. In each resampling-based comparison of J group means based on a total of N scores, 50,000 resamples with replacement of size N were obtained from the concatenated distribution of all J groups’ observed data, and these resampled data were distributed into J groups with

17-Schachtman-17.indd 391

391

the appropriate sample sizes nj. To create the relevant sampling distributions to evaluate main effect and interaction questions, the relevant F statistics were calculated for each resample. Appropriate percentiles from this distribution then served as critical values against which the F statistics based on the sample data could be compared, and the p-value indicated the proportion of the bootstrapped sampling distribution that exceeded the observed test statistic. Pairwise mean comparisons then were conducted for signiﬁcant omnibus tests, using analogous procedures to obtain two-tailed, bootstrapped p-values for t statistics. All computations were executed using Resampling Stats in MATLAB (Kaplan, 1999). Results

Does the congruence of perceptual organization with the category structure inﬂuence performance in the Initial Phase of learning? Performance in the Initial Phase of learning was expected to vary as a function of the congruence of participants’ perceptual organization classiﬁcations with the category structure. To increase the power of our analyses, participants in the Affect Initial and Body-Size Initial Groups were classiﬁed as exhibiting either high, medium, or low congruence with the category structure, depending on their perceptual organization classiﬁcation. Thus, affect-oriented participants in the Affect Initial Group and body-size-oriented participants in the Body-Size Initial Group were classiﬁed as “high congruence,” whereas body-size-oriented participants in the Affect Initial Group and affect-oriented participants in the Body-Size Initial Group were classiﬁed as “low congruence.” The remaining both-oriented participants in both groups were classiﬁed as “medium congruence.” Figure 17.4 depicts average performance in the Initial Learning Phase as a function of the congruence of participant perceptual organization with the initial category structure. The resampling-based omnibus evaluation of whether at least two of the group means differed was signiﬁcant, F = 15.29, p < 0.001. All follow-up evaluations of pair-wise differences between means were signiﬁcant: high versus

2/17/2011 10:59:42 AM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS 90

1.00 0.96 0.95

0.91

0.90 0.83

0.85 0.80

Transfer (in degrees) to initial category structure

Mean Proportion Correct

392

80

82.84

78.53

Experimental Group Control Group

70.95

70 60

51.70 42.18

50 40 30 20

11.26

10 0

0.75 High

Medium

Low

Congruence of Perceptual Organizational Classification with initial Category structure

High

Medium

Low

Congruence of Perceptual Organizational Classification with initial Category structure

Figure 17.5 The blocking manipulation and the Figure 17.4 The congruence of perceptual orga-

nization with the initial category structure inﬂuences performance in the initial learning phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

congruence of perceptual organization with the initial category structure inﬂuence performance in the transfer phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

medium, t = –3.03, p < 0.01; high versus low, t = −5.12, p < 0.001; medium versus low, t = –2.95, p < 0.01. Thus, participants performed markedly better in the Initial Phase when learning a category structure that was congruent with their perceptual organization classiﬁcation and struggled markedly when learning an incongruent category structure. Do the blocking manipulation and the congruence of perceptual organization with the initial category structure inﬂuence performance in the Transfer Phase? Performance in the Transfer Phase was expected to vary as a function of both the blocking manipulation and the congruence of participants’ perceptual organization classiﬁcations with the initial category structure. As in the previous analysis, participants in the Affect Initial and Body-Size Initial Groups were classiﬁed as exhibiting either high, medium, or low congruence with the initial category structure on the basis of their perceptual organization classiﬁcation. Additionally, participants in both the Affect Initial and Body-Size Initial Groups were classiﬁed as members of the experimental group for the blocking manipulation, and the remaining participants were classiﬁed as members of the control group. Figure 17.5 depicts average performance in the Transfer Phase as a function of the congruence of participant perceptual organization with the initial category structure and the blocking manipulation.

Larger values indicate greater transfer to the initial cue, with values of 90 indicating perfect transfer to the initial cue and values of 0 indicating perfect transfer to the shift cue. The main effect of the blocking manipulation was signiﬁcant, F = 57.19, p < 0.001, and the difference between the experimental and control means—the overall “blocking effect”—was substantial at 25.3 degrees. As expected, the experimental group was much more likely to transfer to the initial cue, presumably because either participants shifted attention toward the initial cue and away from the blocked cue or because participants generalized from the initial-phase items. In contrast, the control group was more likely to show transfer patterns that were purely congruent with their perceptual organization. The main effect of the congruence of participants’ perceptual organization classiﬁcations with the initial category structure also was signiﬁcant, F = 77.43, p < 0.001. Follow-up evaluations indicated that all pair-wise differences were signiﬁcant: high versus medium, t = 5.79, p < 0.001; high versus low, t = 12.30, p < 0.001; medium versus low, t = 50.01, p < 0.001. In other words, regardless of the blocking manipulation, participants transferred more strongly to the cue that was congruent with their perceptual organization. A signiﬁcant interaction between the blocking manipulation and the congruence of participants’

17-Schachtman-17.indd 392

2/17/2011 10:59:42 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

17-Schachtman-17.indd 393

cue than control participants, because transfer was at a functional ceiling among control participants, thereby deﬂating the difference between experimental and control groups. Do the blocking manipulation and the congruence of perceptual organization with the initial category structure inﬂuence performance in the Shift Phase of learning? Performance in the Shift Learning Phase was expected to vary as a function of both the blocking manipulation and the congruence of participants’ perceptual organization classiﬁcations with the initial category structure. As in the previous analyses, participants in the Affect Initial and Body-Size Initial Groups were classiﬁed as exhibiting high, medium, or low congruence with the initial category structure, depending on their perceptual organization classiﬁcation. We anticipated that low-congruence participants would perform substantially better in the shift phase than high-congruence participants, because participants with low congruence between their perceptual organization and the initially relevant cue had high congruence with the shift-phase relevant cue. Participants also were classiﬁed as members of the experimental or control groups for the blocking manipulation, as in the previous analyses. Figure 17.6 depicts average performance in the Shift Learning Phase as a function of the congruence of participant perceptual organization with the initial

1.00 Mean Proportion Correct First Three Blocks

perceptual organization classiﬁcations with the initial category structure emerged, F = 10.07, p < 0.001. Follow-up comparisons of the blocking effect for each level of the congruence factor indicated a signiﬁcant blocking effect for the low-congruence group, t = 5.68, p < 0.001, and the medium-congruence group, t = 4.04, p < 0.001, and a nonsigniﬁcant trend for the high-congruence group, t = .60, p < 0.10. For example, you can see in Figure 17.5 that there was little effect of the blocking manipulation when the initial category structure was highly congruent with the perceptual organization classiﬁcation; in both cases, participants transferred overwhelmingly to the dimension underlying the initial category structure. In contrast, when the initial category structure was less congruent with the perceptual organization classiﬁcation, the effect of the blocking manipulation on participants’ transfer patterns was substantial (i.e., the experimental group showed much stronger transfer to the dimension underlying the initial category structure than the control group). Overall, therefore, the blocking effect was of similar strength for the medium- and low-congruence groups and signiﬁcantly stronger than for the high-congruence group, which showed a nonsigniﬁcant blocking effect. Based on analogy to previous research with blocked, separably processed cues that had no middling value in the Initial Phase (Denton & Kruschke, 2006), we had anticipated that the low-congruence group would show the weakest blocking effect, because the to-be-blocked cue was highly salient, whereas the high-congruence group would have shown the strongest blocking effect, secondary to the weak perceived salience of the to-be-blocked cue. The results seem to indicate instead that the Initial-Phase items had a strong inﬂuence in the Transfer Phase, producing, in the LowCongruence group, marked generalization to the initial category structure, thereby inﬂating the difference between experimental and control groups. In the high-congruence condition, on the other hand, the inﬂuence of perceptual organization on initial learning was so substantial that it would have been almost impossible for experimental participants in the high-congruence group to shift more strongly to the initial

393

0.90

0.93

Experimental Group Control Group 0.82

0.81

0.80 0.70 0.70 0.60

0.59 0.57

0.50 High

Medium

Low

Congruence of Perceptual Organizational Classification with initial Category structure

Figure 17.6 The blocking manipulation and the

congruence of perceptual organization with the initial category structure inﬂuence performance on the ﬁrst three blocks of the shift learning phase of the women’s learning study. Bars correspond to bootstrapped standard error of the means.

2/17/2011 10:59:42 AM

394

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

category structure and the blocking manipulation. Analyses were conducted on average performance in the ﬁrst three blocks only; performance was near ceiling for many participants in later blocks, and early learning performance was expected to be more diagnostic of the effects of interest. The main effect of the blocking manipulation was signiﬁcant, F = 14.31, p < 0.001, indicating that later learning about the blocked cue was attenuated for the experimental group relative to the control group. This main effect echoes previous ﬁndings that learning about a blocked cue is retarded relative to learning about nonblocked control cues (Kruschke, 2005b; Kruschke & Blair, 2000). The main effect of the congruence of participants’ perceptual organization classiﬁcations with the initial category structure also was signiﬁcant, F = 61.62, p < 0.001. Follow-up evaluations indicated that all pair-wise differences were signiﬁcant: low versus medium, t = 4.14, p < 0.001, low versus high, t = 11.97, p < 0.001, medium versus high, t = 6.37, p < 0.001. In other words, regardless of the blocking manipulation, participants learned more rapidly a category structure that was congruent with their initial perceptual organization. A signiﬁcant interaction between the blocking manipulation and the congruence of participants’ perceptual organization classiﬁcations with the initial category structure emerged, F = 4.50, p < 0.05. Follow-up comparisons of the blocking effect for each level of the congruence factor indicated a signiﬁcant blocking effect for the low-congruence group, t = –3.66, p < 0.001, and the medium-congruence group, t = –3.85, p < 0.01, and a nonsigniﬁcant effect for the high-congruence group, t = 0.47, n.s. Inspection of Figure 17.6 facilitates understanding of this interaction. Participants in the low-congruence group by deﬁnition perceived the shift cue to be much more salient than the initial cue. These participants showed a strong blocking effect, such that those in the control group learned quickly about the shift cue (which was not blocked), whereas those in the experimental group learned far more slowly about the shift cue (which was blocked). Participants in the medium-congruence group showed a similar

17-Schachtman-17.indd 394

pattern. But participants in the high-congruence group—who perceived the shift cue to be far less salient than the initial cue—showed no evidence of blocking. This pattern of ﬁndings was inconsistent with the analogy to previous research involving blocked cues (with no initially middling values and attentionally separable dimensions; Denton & Kruschke, 2006). This prior work suggested that it would be harder to block a more salient shift cue, such that participants in the high-congruence group would have shown the strongest blocking effect. Instead, the results are consistent with the hypothesis that the Initial-Phase items have continuing inﬂuence throughout subsequent phases and attention is not shifted easily from the enduring perceived salience of these holistically processed dimensions. Conclusions

This study evaluated three potential inﬂuences on women’s learning about other women’s affective or weight-related information: individual differences in the perceived salience of affect and body-size information, an experimental blocking manipulation, and the interaction between these two factors. We also used formal modeling techniques to evaluate whether shifts of attention toward relevant stimulus dimensions played a role in participants’ learning. As expected, congruence of individual differences in perceptual organization with the experienced category structure exerted a strong inﬂuence on performance throughout the phasedlearning task, regardless of whether participants were in an experimental or control condition for the blocking manipulation. In the Initial Learning Phase, participants who experienced a category structure that was congruent with their perceptual organization showed percent-correct scores that were 13 percentage points higher than participants who experienced an incongruent category structure. In the Transfer Phase, participants showed much stronger transfer to a cue that was congruent with their perceptual organization (82.84 degrees) than to a cue that was incongruent with their perceptual organization (51.70 degrees). And in the Shift Learning

2/17/2011 10:59:42 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

Phase, participants learning a category structure that was congruent with their perceptual organization showed a 29-point advantage in percent correct, relative to participants learning an incongruent category structure. The greater overall perceived salience of affect than body size also inﬂuenced category learning. Average performance on the initial affect structure (M = 0.94, SD = 0.09) was signiﬁcantly better than average performance on the body-size structure (M = 0.87, SD = 0.13), t(119) = 3.52, p < 0.01. Performance on the later affect structure (M = 0.80, SD = 0.20) also was superior to that on the later body-size structure (M = 0.66, SD = 0.20), t(240) = 5.72, p < 0.001. Thus, both average perceptual organization across participants (i.e., the greater perceived salience of affect than body size overall) and individual differences in perceptual organization inﬂuenced category learning, providing further evidence of the generalizability of the normative perceptual organization-learning relationship that previously has been observed with simple, artiﬁcial stimuli. The blocking manipulation produced a notable reduction in transfer to and later learning about a blocked cue. Whereas control participants on average showed moderate transfer to the dimension underlying the initial category structure (43.99 degrees), experimental participants showed much stronger transfer to the (unblocked) dimension underlying the initial structure (68.49 degrees). Experimental participants also showed attenuated shift learning of the blocked cue relative to control participants (70% versus 78% correct). The blocking effect on transfer and learning could reﬂect either shifting of attention away from the blocked cue or a remapping of the associations between regions of the psychological space and the category labels in the Initial Learning Phase. We ﬁt RASHNL to the choice data from the four groups simultaneously to evaluate whether attention shifting played a role in learning. The best-ﬁtting parameters showed an attentional shift rate of zero. This lack of attentional learning is consistent with the stimulus dimensions being integral, not separable, as was suggested by the fact that the best ﬁtting

17-Schachtman-17.indd 395

395

similarity-scaling metric was Euclidean, not city block. The modeling suggests conclusions here that echo those made for men’s perceptions of women in the previous section of the chapter. Women’s learning about other women’s affect and body size is inﬂuenced strongly by the observer’s perceived salience of the dimensions. It is difﬁcult to shift attention rapidly between these dimensions, even when performance on a category-learning task would be facilitated by doing so. The congruence of perceptual organization with the initial category structure interacted with the blocking manipulation. In general, when the initial category structure was congruent with the perceptual organization that a participant brought to the task, then performance on the shift category structure was very poor indeed. When, initially, participants experienced a category structure that was incongruent with their perceptual organization, they showed an average of 83% correct. When participants only later were exposed to a category structure that was incongruent with their perceptual organization—after having experienced a category structure that either was congruent or could be perceived as congruent (the Redundant Relevant Cues phase of learning, in the case of control participants)—they showed an average of 59% correct (or 68% correct, if all blocks in the shift learning phase are considered). This disparity in performance as a function of learning history and perceptual organization congruence is quite worrisome, given the simplicity of the category structures being presented (e.g., heavy vs. light or happy vs. sad). Thus, it will be of particular interest in future work to evaluate whether women who report disordered eating patterns are particularly likely to struggle when trying to learn an affect category structure at all, and especially after experiencing a body-size category structure. If so, then the blocking paradigm might serve as a useful analog for a real-life circumstance in which the social environment’s reinforcement of young women’s preoccupation with body-size information as a royal road to happiness makes it extremely difﬁcult to learn about alternative predictors of success.

2/17/2011 10:59:42 AM

396

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

CLOSING COMMENTS Category-learning processes play a central role in everyday life, and cognitive scientists have worked for decades to develop valid models and paradigms for the investigation of normative learning processes with simple, artiﬁcial stimuli. The work described in this chapter highlights the generalizability of these models and methods to the study of clinically relevant individual differences in category-learning processes with far more complex, socially and emotionally relevant information. Accounting for individual differences in category learning with complex stimuli necessitates an increased focus on individual differences in the perceived salience of stimulus dimensions. The present work documents the substantial inﬂuence of across-individual variability in perceived dimensional salience on category learning, such that participants learn category structures that are more congruent with their underlying perceptual organizations far more quickly than category structures that are less congruent. This faster learning of salience-congruent structures persists even after learning a salience-incongruent structure, most likely because the socially relevant stimuli have integral dimensions that prevent shifts of attention between dimensions. This observed link between individual differences in perceived salience and individual differences in category learning extends the well-established link between normative perceived salience and category learning. The dimensions of the complex, socially relevant stimuli of interest to some clinical researchers also may be processed in a more holistic fashion than the components of the artiﬁcial stimuli of primary interest to cognitive researchers, which typically are processed more separably. Prior research with artiﬁcial stimuli has demonstrated that attention-shifting mechanisms play a far less signiﬁcant role in category learning with integral-dimension stimuli (Nosofsky & Palmeri, 1996) than separabledimension stimuli. The research described herein extends this ﬁnding to applied research with far more complex stimuli. In both reported

17-Schachtman-17.indd 396

studies, formal ﬁts of RASHNL (Kruschke & Johansen, 1999) to the category-learning data indicated that learning resulted not from shifting attention toward relevant stimulus dimensions and away from irrelevant dimensions (because the best-ﬁtting attentional shift rate was zero), but rather from individual differences in the initial perceived salience of stimulus dimensions and the gradual strengthening of associations between regions of the stimulus space and category labels. The evident difﬁculties in dynamic reallocation of attention for integral stimuli enhance the signiﬁcance of individual differences in perceptual organization in applied social learning. They also highlight the importance of future research on the conditions associated with increased dimensional attention shifting, even when the stimulus dimensions are perceived more holistically, because the ability to shift attention rapidly could be quite adaptive in social perceptual learning. More generally, the present work illustrates the utility of translating associative-learning paradigms to address applied questions about clinically and socially relevant processing of complex stimuli. These paradigms may offer useful analogs of “real-world” social learning environments that presumably contribute to the development of attitudes and beliefs of interest to clinical researchers. Formal modeling techniques then could be used to elucidate the mechanisms underlying learning, which not only might enhance clinical scientists’ understanding of the role of more dynamic aspects of processing in clinical phenomena but also might contribute to the development of novel prevention or intervention strategies that directly target deﬁcient or maladaptive cognitive processing. Current cognitively oriented treatments rely primarily on verbally mediated techniques that emphasize the identiﬁcation and modiﬁcation of speciﬁc distorted thoughts and beliefs (Treat et al., 2007), but cognitive therapy might be augmented usefully by drawing to a greater degree on the plethora of associative-learning paradigms to modify problematic processing patterns or to facilitate the acquisition of important category structures.

2/17/2011 10:59:42 AM

APPLICATION OF ASSOCIATIVE LEARNING PARADIGMS

REFERENCES Ashby, F. G., & Maddox, W. T. (2005). Human category learning. Annual Review of Psychology, 56, 149–178. Beck, A. T. (1976). Cognitive theory and the emotional disorders. New York, NY: International Universities Press. Botta, R. A. (2003). For your health? The relationship between magazine reading and adolescents’ body image and eating disturbances. Sex Roles, 48, 389–399. Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an N-way generalization of “EckartYoung” decomposition. Psychometrika, 35, 283–320. Cooper, M. J. (2005). Cognitive theory in anorexia nervosa and bulimia nervosa: Progress, development and future directions. Clinical Psychology Review, 25, 511–531. Davison, M. L. (1992). Multidimensional scaling. Malabar, FL: Krieger Publishing Company. Denton, S. E., & Kruschke, J. K. (2006). Attention and salience in associative blocking. Learning and Behavior, 34, 285–304. Fairburn, C. G., Cooper, Z., & Shafran, R. (2003). Cognitive behaviour therapy for eating disorders: A “transdiagnostic” theory and treatment. Behaviour Research and Therapy, 41, 509–528. Farris, C. A., Treat, T. A., Viken, R. J., & McFall, R. M. (2008). Sexual coercion and the misperception of sexual intent. Clinical Psychology Review, 28, 48–66. Good, P. I. (2006). Resampling methods: A practical guide to data analysis (3rd ed.). Boston, MA: Birkhäuser. Johnston, L., & Ward, T. (1996). Social cognition and sexual offending: A theoretical framework. Sexual Abuse: Journal of Research and Treatment, 8, 55–80. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment (pp. 279–296). New York, NY: Appleton-Century-Crofts. Kaplan, D. T. (1999). Resampling Stats in MATLAB. Arlington, VA: Resampling Stats, Inc. Kelly, G. A. (1955). The psychology of personal constructs. New York, NY: Norton. Kruschke, J. K. (1992). ALCOVE: An exemplarbased connectionist model of category learning. Psychological Review, 99, 22–44.

17-Schachtman-17.indd 397

397

Kruschke, J. K. (1993). Human category learning: Implications for back propagation models. Connection Science, 5, 3–36. Kruschke, J. K. (1996). Dimensional relevance shifts in category learning. Connection Science, 8, 225–247. Kruschke, J. K. (2005a). Category learning. In K. Lamberts & R. L. Goldstone (Eds.), The handbook of cognition (pp. 183–201). London, England: Sage. Kruschke, J. K. (2005b). Learning involves attention. In G. Houghton (Ed.), Connectionist models in cognitive psychology (pp. 113–140). Hove, England: Psychology Press. Kruschke, J. K., & Blair, N. J. (2000). Blocking and backward blocking involve learned inattention. Psychonomic Bulletin and Review, 7, 636–645. Kruschke, J. K., & Johansen, M. K. (1999). A model of probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1083–1119. Lee, M., & Shafran, R. (2004). Information processing biases in eating disorders. Clinical Psychology Review, 24, 215–238. Lingswiler, V. M., Crowther, J. H., & Stephens, M. A. P. (2006). Affective and cognitive antecedents to eating episodes in bulimia and binge eating. International Journal of Eating Disorders, 8, 533–539. MacCallum, R. C. (1977). Effects of conditionality on INDSCAL and ALSCAL weights. Psychometrika, 42, 297–305. McDonel, E. C., & McFall, R. M. (1991). Construct validity of two heterosocial perception skill measures for assessing rape proclivity. Violence and Victims, 6, 17–30. McFall, R. M. (1990). The enhancement of social skills: An information-processing analysis. In W. L. Marshall & D. R. Laws (Eds), Handbook of sexual assault: Issues, theories, and treatment of the offender (pp. 311–330). New York, NY: Plenum Press. McFall, R. M., Eason, B. J., Edmondson, C. B., & Treat, T. A. (1999). Social competence and eating disorders: Development and validation of the anorexia and bulimia problem inventory. Journal of Psychopathology and Behavioral Assessment, 21, 365–394. Nosofsky, R. M. (1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43, 25–53.

2/17/2011 10:59:42 AM

398

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Nosofsky, R. M., & Palmeri, T. J. (1996). Learning to classify integral-dimension stimuli. Psychonomic Bulletin and Review, 3, 222–226. Segal, Z. V., & Stermac, L. E. (1990). The role of cognitions in sexual assault. In W. L. Marshall, D. R. Laws, & H. E. Barbaree (Eds.), Handbook of sexual assault: Issues, theories and treatment of the offender (pp. 161–174). New York, NY: Plenum. Shepard, R. N. (1964). Attention and the metric structure of the stimulus space. Journal of Mathematical Psychology, 1, 54–87. Smyth, J. M., Wonderlich, S. A., Heron, K. E., Sliwinski, M. J., Crosby, R. D., Mitchell, J. E., & Engel, S. G. (2007). Daily and momentary mood and stress are associated with binge eating and vomiting in bulimia nervosa patients in the natural environment. Journal of Consulting and Clinical Psychology, 75, 629–638. Stice, E., Schupak-Neuberg, E., Shaw, H. E., & Stein, R. I. (1994). Relation of media exposure to eating disorder symptomatology: An examination of mediating mechanisms. Journal of Abnormal Psychology, 103, 836–840. Tiggemann, M. (2003). Media exposure, body dissatisfaction and disordered eating: Television and magazines are not the same! European Eating Disorders Review, 11, 418–430. Treat, T. A., McFall, R. M., Viken, R. J., & Kruschke, J. K. (2001). Using cognitive science methods to assess the role of social information processing in sexually coercive behavior. Psychological Assessment, 13, 549–565. Treat, T. A., McFall, R. M., Viken, R. J., Kruschke, J. K., Nosofsky, R. M., & Wang, S. S.

17-Schachtman-17.indd 398

(2007). Clinical-cognitive science: Applying quantitative models of cognitive processing to examination of cognitive aspects of psychopathology. In R. W. J. Neufeld (Ed.), Advances in clinical-cognitive science: Formal modeling and assessment of processes and symptoms (pp. 179–205). Washington, DC: APA Books. Treat, T. A., McFall, R. M., Viken, R. J., Nosofsky, R. M., MacKay, D. B., & Kruschke, J. K. (2002). Assessing clinically relevant perceptual organization with multidimensional scaling techniques. Psychological Assessment, 14, 239–252. Treat, T. A., Viken, R.J., Kruschke, J.K., & McFall, R. M. (2010). The role of attention, memory, and correlation-detection processes in eating disorders. Journal of Mathematical Psychology, 54, 184–195. Viken, R. J., Treat, T. A., Nosofsky, R. M., McFall, R. M., & Palmeri, T. (2002). Bulimics and controls’ differential attention to and classiﬁcation of body-size and affect stimulus information. Journal of Abnormal Psychology, 111, 598–609. Vitousek, K. B. (1996). The current status of cognitive-behavioral models of anorexia nervosa and bulimia nervosa. In P. M. Salkovskis (Ed.), Frontiers of cognitive therapy (pp. 383–418). New York, NY: Guilford Press. Vitousek, K. B., & Hollon, S. D. (1990). The investigation of schematic content and processing in eating disorders. Cognitive Therapy and Research, 14, 191–214.

2/17/2011 10:59:42 AM

CHAPTER 18 Evaluative Conditioning A Review of Functional Knowledge and Mental Process Theories Jan De Houwer

Pavlovian conditioning can be deﬁned as a change in behavior that is due to the pairing of stimuli. Evaluative conditioning is a subclass of Pavlovian conditioning effects in that it refers to the effect of stimulus pairings on liking. As is the case with other instances of Pavlovian conditioning, two questions can be asked about evaluative conditioning: (1) What elements in the environment moderate the effect of stimulus pairings on liking? (2) Which mental processes mediate the effect of stimulus pairings on liking? In this chapter, I present a brief overview of the literature pertaining to these two questions.

Applying Pavlovian conditioning to a phenomenon in daily life always boils down to the following question: Does the phenomenon qualify as an instance of Pavlovian conditioning? As is evidenced by different chapters in this book, many phenomena have been considered as instances of Pavlovian conditioning. In the present chapter, I examine whether changes in liking can also be understood from this perspective. Before I review the evidence on this topic, I ﬁrst consider in more detail the meaning of the term “Pavlovian conditioning” because this determines what phenomena can be seen as instances of Pavlovian conditioning and thus how Pavlovian conditioning can be applied.

WHAT IS PAVLOVIAN CONDITIONING? In some textbooks, Pavlovian conditioning is deﬁned in a very narrow manner as the unconscious formation of associations that results from the pairing of a conditioned stimulus (CS) and an unconditioned stimulus (US) and that

leads to changes in physiological responses (e.g., salivation) (e.g., Arnould, Price, & Zinkhan, 2004; Evans, Jamal, & Foxall, 2006). Such a deﬁnition is narrow in that it limits conditioning to changes in one particular class of responses that are due to one particular type of mental process. Other researchers impose other restrictions on the deﬁnition of Pavlovian conditioning, for instance, when arguing that “true” Pavlovian conditioning always involves biologically relevant USs (e.g., Miller & Matute, 1996). In this section, I will argue that it makes more sense to deﬁne Pavlovian conditioning in the broadest possible manner, that is, as a change in behavior that is due to the pairing of stimuli. Imposing restrictions on the nature of the changes, the nature of the responses, the nature of the stimuli, or the nature of the underlying mental processes is unnecessary and can lead to deleterious effects (see De Houwer, 2007, 2009, for an in-depth discussion). Let us consider the necessity of restrictions regarding the nature of the responses. There is no a priori reason why only changes in one type 399

18-Schachtman-18.indd 399

2/16/2011 5:53:50 PM

400

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

of responses (e.g., physiological responses) should be considered as possible instances of Pavlovian conditioning. It is true that Pavlov’s (1927) seminal studies focused on changes in a physiological response (salivation) but since then many studies on Pavlovian conditioning have looked at changes in other types of behavior. For example, in studies on autoshaping, food is presented to pigeons each time a key lights up. As a result of this contingency, the pigeons start pecking the key (e.g., Brown & Jenkins, 1968). There is no logical reason why a change in salivation could count as Pavlovian conditioning but a change in key pecking could not. More generally, there is no logical reason why any type of response should be excluded on an a priori basis from the realm of Pavlovian conditioning (Eelen, 1980). There is also no a priori reason why Pavlovian conditioning should be restricted to changes in behavior that are due to a particular process like the (unconscious) formation of associations in memory. It is true that some researchers (used to) believe that conditioned changes in behavior are due to the unconscious formation of associations (e.g., Thorndike, 1911; Watson, 1913). But this was simply one possible theory about the processes that produce conditioning effects. There are also other theories of classical conditioning according to which the formation of associations does depend on awareness (e.g., Dawson & Schell, 1987) or that do not refer to the formation of associations at all. For instance, Mitchell, De Houwer, and Lovibond (2009a) have argued that Pavlovian conditioning results from the conscious formation and evaluation of propositions about CS-US relations. Why should some theories be dismissed on an a priori basis? In addition to the fact that there are no good reasons to limit Pavlovian conditioning to only a subset of effects of the pairing of stimuli, doing so can have several deleterious consequences (see De Houwer, 2007). First, it could prevent researchers from acknowledging and studying the similarities and differences between different effects of stimulus pairings. The mere fact of using different labels for different effects can draw attention away from similarities in the conditions under which the effects occur. Moreover, knowledge about the similarities and differences

18-Schachtman-18.indd 400

between different effects of stimulus pairings can provide crucial information about the processes by which the pairing of stimuli inﬂuences behavior. Second, deﬁning Pavlovian conditioning in terms of particular processes hampers the study of conditioning because a particular change in behavior can be considered as an instance of conditioning only when it can be established that it is produced by certain processes. This is problematic because it is often extremely difﬁcult to determine whether a particular process is responsible for a certain change in behavior. The same problem arises with other criteria that are difﬁcult to verify (e.g., whether stimuli are biologically relevant). Finally, a deﬁnition of Pavlovian conditioning in terms of mental processes holds the risk that doubts about the role of those mental processes lead to doubts about the existence of Pavlovian conditioning effects (Eelen, 1980). For instance, the downfall of behaviorist theories of conditioning led to a dramatic slowdown in research on conditioning effects because many researchers inferred that humans do not show “true” Pavlovian conditioning effects (i.e., effects that are due to unconscious association formation; e.g., Brewer, 1974). More generally, deﬁning effects in terms of processes violates the scientiﬁc principle that theories (i.e., the explanans) need to be separated from the effects that they aim to explain (i.e., the explanandum). Because of these reasons, I prefer to deﬁne Pavlovian conditioning as a change in behavior that results from the pairing of stimuli. The only criterion that is left for labeling a change in behavior as an instance of Pavlovian conditioning is that the change is due to the pairing of stimuli rather than other factors such as genetic makeup (e.g., changes in behavior due to maturation), the simple repeated experience of a stimulus (e.g., habituation), or the relation between a behavior and a stimulus (e.g., operant conditioning; De Houwer, 2009). In laboratory situations, this criterion can be veriﬁed by implementing appropriate control conditions (e.g., by presenting stimuli in an unpaired manner). Although it might not always be easy to determine the environmental cause of a change in behavior, applying the minimal deﬁnition of

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

Pavlovian conditioning will always be more straightforward than applying other deﬁnitions in which other criteria are added to the minimal deﬁnition (e.g., criteria regarding the nature of the response, the nature of the US, or the type of underlying process). Some readers might not be willing to accept this minimal deﬁnition, perhaps because of historical reasons. All deﬁnitions are subject to matters of convention and everyone is thus free to add criteria to the minimal deﬁnition. If one chooses to do so, it should, however, be made explicit what the additional criteria are, why those criteria are thought to be crucial, and how it can be veriﬁed whether a certain effect of stimulus pairings meets those criteria. One should also be aware of and be willing to accept the limitations that such additional criteria impose on Pavlovian conditioning research. I prefer the minimal deﬁnition because it maximizes the scope of Pavlovian conditioning research, because it is logically coherent, and because it is the easiest to verify.

WHAT IS EVALUATIVE CONDITIONING AND HOW CAN WE STUDY IT? Adopting a minimal deﬁnition of Pavlovian conditioning allows one to fully engage in the question of whether certain changes in liking are instances of Pavlovian conditioning. The only criterion that needs to be satisﬁed in order to categorize a change in liking as an instance of Pavlovian conditioning is that the change is due to a relation between stimuli rather than to other environmental factors such as the simple repeated presentation of a single stimulus (as is the case in mere exposure effects; see Bornstein, 1989). A change in liking that is due to the pairing of stimuli is typically referred to as an evaluative conditioning (EC) effect (De Houwer, 2007; Levey & Martin, 1975). EC effects are thus a subset of Pavlovian conditioning effects, the only distinction being that EC effects always involve changes in liking, whereas Pavlovian conditioning effects can relate to a change in any type of observable response. The procedures that are used to study evaluative conditioning are

18-Schachtman-18.indd 401

401

also very similar to the procedures that are used to study other forms of Pavlovian conditioning: Stimuli are paired in a certain way under certain conditions. The only systematic difference between EC procedures and other Pavlovian conditioning procedures is that all EC procedures measure on changes in liking (De Houwer, 2007). As is the case with other instances of Pavlovian conditioning (and thus all possible applications of Pavlovian conditioning), several questions can be raised regarding EC (De Houwer, 2009). A ﬁrst question concerns the procedural conditions under which the pairing of stimuli results in changes in liking. These conditions refer either to the abstract nature of the relation between stimuli or to the concrete way in which the relation is implemented. The abstract nature of the relation refers to the statistical properties of the relation between stimuli (e.g., contiguity, contingency) and possible changes in those properties (e.g., extinction). Studies that examine the impact of these procedural features inform us about which aspects of the relation are crucial in establishing EC. The concrete implementation of the relation requires choices regarding the stimuli that are presented, the type of liking response that is observed, the organism that experiences the relation, the context in which the relation is present, and the way in which information about the relation is communicated. By manipulating each of these procedural elements, knowledge can be gained about those aspects of the procedure or environment that determine whether and to what extent a relation between stimuli inﬂuences the liking of those stimuli. I will refer to this knowledge as functional knowledge (De Houwer, in press). Note that functional knowledge about EC goes beyond mere empirical knowledge. First, it involves more than the description of an event in that it offers hypotheses about which elements in the procedure are assumed to exert an impact on behavior. Like all hypotheses, functional knowledge needs to be backed up by (empirical and logical) arguments. For instance, the empirical observation that the liking of a particular CS changes after it has been paired 10 times with a particular US can be explained in terms of the

2/16/2011 5:53:50 PM

402

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

mere co-occurrence of the CS and US or in terms of the positive statistical contingency between the CS and US (i.e., the fact that the CS and US often co-occur and never occur separately). Which procedural explanation is correct needs to be determined on the basis of additional research (e.g., by manipulating the number of CS-only and US-only trials). Second, functional knowledge goes beyond the individual event in that hypotheses about the role of procedural elements are thought to be valid in more than one situation. For instance, the hypothesis that CS-US co-occurrences are sufﬁcient for EC is thought to be true not only for one particular CS-US pair in one speciﬁc context. Functional knowledge is therefore more than a collection of empirical ﬁndings (see De Houwer, 2009). The second question that can be raised regarding EC concerns the nature of the mental processes that are responsible for EC. Theories about the mental processes that underlie EC aim to explain functional knowledge about EC, that is, why certain aspects of the EC procedure determine the magnitude and direction of EC effects. For instance, some mental process models predict that EC will depend primarily on the number of CS-US co-occurrences, whereas other mental process models predict that the statistical contingency is important (i.e., that also the number of CS-only and US-only trials count). The merits of different mental process theories can be evaluated by examining (a) how well they can account for the available functional knowledge (i.e., the heuristic function of process theories) and (b) the extent to which they predict novel functional knowledge (i.e., the predictive function of process theories). In the remainder of this chapter, I use this framework to review the literature on EC (see De Houwer, Thomas, & Baeyens, 2001, and Field, 2005, for other reviews). The ﬁrst section summarizes ﬁndings regarding the effects of the abstract nature of the relation and the concrete implementation of the relation. The second section provides an overview of mental process theories of EC. The review is not intended to be exhaustive in the sense that all relevant studies will be discussed or even cited. I will, however, attempt to present a summary of what I believe

18-Schachtman-18.indd 402

to be the main ﬁndings and insights that are currently available in the literature on EC.

FUNCTIONAL KNOWLEDGE The Abstract Nature of the Relation The Statistical Properties of the CS-US Relation

The presence of stimuli can be related in many different ways. Therefore, the value of the claim that a change in behavior is due to a CS-US relation depends heavily on how much is known about the exact properties of the CS-US relation that determine its effect on behavior. We will consider three statistical properties of the CS-US relation: co-occurrence, contingency, and redundancy. Co-occurrence Several studies suggest that EC becomes stronger as the number of CS-US pairings increases, that is, the more often the CS and US co-occur in space and time (e.g., Baeyens, Eelen, Crombez, & Van den Bergh, 1992a; BarAnan, De Houwer, & Nosek, in press). However, after more than 10 pairings, there is no further increase (Bar-Anan et al., in press) or even a slight decrease in EC (Baeyens et al., 1992a). Although co-occurrence is important, it is not a necessary condition for EC. EC can occur even when the CS and US have never co-occurred but are related in an indirect manner through co-occurrences with a third stimulus. In a seminal study on this topic, Hammerl and Grabitz (1996) ﬁrst presented pairs of neutral stimuli (e.g., A-B). Afterward, one of the stimuli of each pair was presented together with a US (e.g., B-US). During a test phase, it was found that the latter pairings changed not only the liking of the CS that was paired with the US (e.g., B) but also the liking of the other neutral stimulus (e.g., A). This effect, which is known as sensory preconditioning, has been replicated and examined in more detail by Walther (2002). Contingency Baeyens, Hermans, and Eelen (1993) examined the impact of the statistical contingency between the CS and the US by intermixing CS-US trials with trials on which only the CS or only the US occurred. Unlike what is typically found in Pavlovian conditioning

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

research, the magnitude of EC was not affected by the number of trials on which only the CS or only the US was presented. However, the failure of Baeyens et al. to ﬁnd an impact of contingency might have been due to the low power of their statistical tests. More research on this issue is thus needed. Redundancy A relation between a CS and a US can also be described in terms of its redundancy, that is, the extent to which the CS-US relation overlaps with other CS-US relations. We know of only two studies that looked at the impact of the redundancy of the CS-US relation on EC. The relation between a CS A and the US can be described as redundant when CS A always co-occurs together with a second CS B that also co-occurs with the US. Dwyer, Jarratt, and Dick (2007) found that the change in liking of A was as large after A-US pairings (i.e., A+ trials) as after AB-US pairings (i.e., AB+ trials). That is, they failed to observe an overshadowing effect. Lipp, Neumann, and Mason (2001) on the other hand, observed a bigger change in liking of A when the A-US relation was redundant (i.e., after AB+ and B+ trials) than when the A-US relation was not redundant (i.e., after AB+ and B- trials). Again, more research on this topic is needed before clear conclusions can be drawn regarding the impact of redundancy on EC. Changes in the Statistical Properties of the CS-US Relation

The relation between a CS and US does not necessarily remain stable over context or time. The presence of a CS-US relation can be preceded or followed by the absence of the same relation. It can also vary according to the broader context in which the organism is placed. I will now discuss the impact of these various procedural elements. Presentations of the CS-Only and the US-Only Most research about changes in the CS-US relation focused on the impact of CS postexposure trials. These trials are presented after the CS-US trials and contain only the CS. Studies on Pavlovian conditioning typically show that a conditioned change in behavior can be reversed by presenting the CS on its own after

18-Schachtman-18.indd 403

403

the CS-US trials. Surprisingly, this phenomenon, which is known as extinction, has not been found in a number of EC studies (e.g., Baeyens, Crombez, Van den Bergh, & Eelen, 1988; Baeyens, Eelen, Van den Bergh, & Crombez, 1989a; De Houwer, Baeyens, Vansteenwegen, & Eelen, 2000; Diaz, Ruiz, & Baeyens, 2005) even under conditions in which extinction of Pavlovian conditioning was observed (e.g., Vansteenwegen, Francken, Vervliet, De Clercq, & Eelen, 2006). Lipp, Oughton, and Lelievre (2003; also see Lipp & Purkis, 2006, and Blechert, Michael, Vriends, Margraf, & Wilhelm, 2007), on the other hand, did ﬁnd evidence for extinction in EC when they measured CS liking during the extinction phase rather than only after the extinction phase. In a more recent study, however, Blechert, Michael, Williams, Purkis, and Wilhelm (2008) failed to ﬁnd extinction even when liking of the CSs was measured during the extinction phase. In sum, the available evidence suggests that CS-only trials have little effect on conditioned changes in liking. Only two studies have examined the effects of CS-only trials that are presented before the CS-US trials (De Houwer et al., 2000; Stuart et al., 1987). In both studies, CS-only trials seemed to interfere with EC, that is, with the effect of subsequent CS-US trials. This effect of CS preexposure trials is typically referred to as latent inhibition (e.g., Lubow & Gewirtz, 1995). Finally, there is one study that looked at the effects of US-only trials. In that study, Hammerl, Bloch, and Silverthorne (1997) found that US-only trials also delayed EC, both when presented before and after the CS-US trials. Occasion Setting In the studies discussed so far, the change in the CS-US relation occurred over time. A CS-US relation can, however, also depend on the physical context that is present (e.g., a second stimulus or the color of a room). In that case, the physical context can function as an occasion setter that signals when the CS-US relation holds. Baeyens and colleagues (Baeyens, Crombez, De Houwer, & Eelen, 1996; Baeyens, Hendrickx, Crombez, & Hermans, 1998) studied occasion setting in EC by using the color of drinks as a signal for when a particular fruit

2/16/2011 5:53:50 PM

404

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

ﬂavor would be followed by a bad aftertaste. For instance, a fruit ﬂavor was followed by the bad aftertaste only in green drinks but not in blue drinks. During a test phase in which none of the drinks had the bad aftertaste, drinks with the fruit ﬂavor that was previously paired with the bad aftertaste were liked less than drinks with other ﬂavor, regardless of the color of the drink. Hence, the studies of Baeyens and colleagues did not provide evidence to support the conclusion that conditioned changes in liking depend on the presence of the context in which the CS-US pairings were presented. Note that Hardwick and Lipp (2000) did observe occasion setting when using modulation of the startle response as an index of learning. However, it has been argued that modulation of the startle responses does not provide a good index of evaluative conditioning because it can also be affected by factors other than the valence of the CSs (see section on “The Nature of the Evaluative Responses”). The Concrete Implementation of the CS-US Relation The Nature of the Stimuli

The Modality and Semantic Category of the CSs and USs EC effects have been observed with a wide range of visual stimuli such as photographs of human faces as CSs and USs (e.g., Baeyens et al., 1992a), paintings as CSs and USs (Levey & Martin, 1975), outdoor sculptures as CSs and USs (e.g., Hammerl & Grabitz, 1996), nonsense words as CSs and valenced words as USs (e.g., Staats & Staats, 1957), cartoon characters as CSs and valenced words and photographs as USs (e.g., Olson & Fazio, 2001), and names of ﬁctitious products as CSs and valenced pictures as USs (e.g., Pleyers, Corneille, Luminet, & Yzerbyt, 2007; Stuart et al., 1987). EC effects have also been found with nonvisual stimuli. For instance, gustatory (taste) stimuli have been used as CSs (e.g., artiﬁcial fruit ﬂavors) and USs (e.g., sugar or a chemical substance that produces a bad aftertaste; e.g., Baeyens, Eelen, Van den Bergh, & Crombez, 1990b; Lamote, Baeyens, Hermans, & Eelen, 2004; Zellner, Rozin, Aron, & Kulish, 1983). Olfactory (odor) stimuli have also been used successfully

18-Schachtman-18.indd 404

as USs (e.g., Hermans, Baeyens, Lamote, Spruyt, & Eelen, 2005; Todrank, Byrnes, Wrzesniewski, & Rozin, 1995) and as CSs and USs (e.g., Stevenson, Boakes, & Wilson, 2000). Other studies demonstrated EC with somatosensory stimuli such as the touch of objects as CSs and USs (e.g., Hammerl & Grabitz, 2000) or mild electric shocks as USs (e.g., Hermans et al., 2002). Auditory stimuli such as pieces of music have also been employed successfully as USs (e.g., Bierley, McSweeney, & Vannnieuwkerk, 1985). The Valence and Identity of the US Although EC can be characterized as a general phenomenon that can involve many different kinds of stimuli, the available evidence suggests that properties of the stimuli do determine the magnitude and direction of EC. First and foremost, the valence of the US determines the direction of the change in valence of the CS. That is, a CS that is paired with a positively valenced US tends to become more positive, whereas a CS that is paired with a negatively valenced US tends to become more negative (e.g., Baeyens et al., 1992a). The importance of US valence rather than the speciﬁc identity of the US is also illustrated by the fact that EC is found not only when a CS is repeatedly paired with a single (positive or negative) US but also when it is paired with different USs that all share the same valence (e.g., Olson & Fazio, 2001; Stahl & Unkelbach, 2009). There is less certainty about the effects of the extremity of the valence of the USs. On the one hand, Baeyens et al. (1988) found equally large effects with strongly valenced USs (e.g., mutilated faces) than with mildly valenced USs (e.g., disliked photographs of intact human faces). In a more recently reported study, Jones, Fazio, and Olson (2009) selected USs on the basis of a pilot study in which the valence of each US had to be determined as quickly as possible. USs that were classiﬁed quickly (and tended to be strongly valenced) resulted in smaller EC effects than USs that were classiﬁed slowly (and tended to be mildly valenced). Although more studies are needed to examine the impact of US extremity, the available evidence gives little reason to believe that more extreme USs lead to more extreme EC. If anything, the reverse seems to be true.

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

There are some indications that positive USs are less effective than negative USs (e.g., Baeyens, Eelen, & Van den Bergh, 1990a). One explanation might be that positive USs tend to be less extremely valenced than negative USs. As we discussed in the previous paragraph, there is, however, little evidence to support the idea that more extreme USs produce stronger EC. A second possible explanation is that people seem to differ more in their appreciation of potentially positive stimuli than in their liking of potentially negative stimuli (Peeters & Czapinski, 1990). This would result in more variability (and thus statistically smaller effects) when positive USs are used than when negative USs are used. Finally, organisms might be genetically prepared to learn more quickly about relations that involve negative stimuli than about relations that involve positive stimuli (Peeters & Czapinski, 1990). More research is needed before a deﬁnite conclusion regarding this issue can be reached. Changes in the Valence and Identity of the US Assume that a CS is paired with a positive US, and that this results in an increase in the liking of the CS. Research showed that when the positive US is afterward made negative (e.g., by pairing it with negative stimuli), the CS will also become more negative. This ﬁnding is known as US revaluation and has been observed in a number of studies (e.g., Baeyens, Eelen, Van den Bergh, & Crombez, 1992b; Walther, Gawronski, Blank, & Langer, 2009). It should be noted, however, that Baeyens, Vanhouche, Crombez, and Eelen (1998) failed to observe a US-revaluation effect in a study where artiﬁcial fruit ﬂavors were used as CSs and a soap-like aftertaste was used as the US. They attributed this failure to the speciﬁc nature of their US. In studies on US revaluation, the identity of the US with which a CS is paired is kept stable, whereas the valence of the US is changed. In studies on counterconditioning, both US identity and US valence are changed. For instance, after a CS has been paired with a positively valenced US, it is paired with a different US with a negative valence. Results have shown that the second pairing undoes and sometimes even

18-Schachtman-18.indd 405

405

reverses the change in liking that was produced (e.g., Baeyens et al., 1989a). The Intrinsic Relation Between CS and US Apart from possible main effects of the nature of the CS and the nature of the US, EC can also depend on the intrinsic relation between the CS and US, that is, the interaction of the nature of the CS and the nature of the US. For instance, Baeyens et al. (1990a) found that pairing the color of a drink with a bad aftertaste did not result in a change in liking of other drinks with that color, whereas pairing the ﬂavor of a drink with a bad aftertaste did change the liking of other drinks with that ﬂavor. Hence, it seems that the relation between the color and the aftertaste has less effect on the liking of drinks than the relation between the ﬂavor and the aftertaste. Likewise, Todrank et al. (1995) found that pairings of neutral photographs of human faces as CSs with odors as USs inﬂuenced the liking of the photographs only if the odors were “plausibly human” (e.g., sweat or fragrances). Such ﬁndings suggest that EC depends on the intrinsic relation between the CS and the US (see Garcia & Koelling, 1966, for similar ﬁnding in the context of other types of Pavlovian conditioning). Researchers have also looked at the impact of the perceptual similarity between the CSs and USs. Martin and Levey (1978) found that EC was stronger for CSs that were paired with perceptually similar USs than for CSs that were paired with perceptually dissimilar USs. This effect was replicated by Field and Davey (1999) but shown to be an artifact in that it remained present even when the CS-US pairs were never presented. The artifact can arise when researchers select CSs on the basis of unreliable evaluative ratings that a participant gives at the start of the experiment. It is, for instance, possible that evaluative ratings of some CSs change simply as the result of seeing other stimuli that are used in the experiment. This is most likely to happen for CSs that resemble other positive or negative stimuli in the stimulus list (see Field & Davey, 1999, for more details). Baeyens, Eelen, Van den Bergh, and Crombez (1989b) manipulated the perceptual similarity between the CS and US independently of the evaluative ratings of participants and failed

2/16/2011 5:53:50 PM

406

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

to ﬁnd stronger EC for perceptually similar CS-US pairs. In sum, there is little evidence to support the conclusion that EC depends on the perceptual similarity between the CS and US. The Manner in Which the CS and US Are Presented Once the stimuli have been selected, they need to be presented in a certain manner. This requires decisions about the time and location at which the stimuli occur, their size, luminance, and so on. Results suggest that EC effects become stronger the more the CS and US are presented in close temporal and spatial proximity (e.g., Jones, Fazio, & Olson, 2009). Even though EC has been observed when the US always is presented before the CS (Martin & Levey, 1987; Stuart, Shimp, & Engle, 1987), effects in those situations appear to be weaker than when the presentation of the CS and US overlap or when the CS is presented brieﬂy before the US. Finally, when using pictures as CSs and USs, Jones et al. (2009) recently found that EC effects increase in magnitude with increases in the size of the CSs. The Nature of the Evaluative Response

What sets EC apart from other forms of Pavlovian conditioning is that it involves changes in evaluative responses, that is, responses that are assumed to reﬂect the liking of objects. Most often, changes in direct measures of liking are examined. Such direct measures require the participant to self-assess his or her liking of the CSs and USs, for instance, by selecting a number on a Likert scale or by sorting the stimuli into separate piles for liked, neutral, or disliked pictures (e.g., Baeyens et al., 1992a; Levey & Martin, 1975). In more recent studies, indirect measures of liking have also been used, in which liking is inferred from performance during reaction time tasks (e.g., De Houwer, Hermans, & Eelen, 1998; Hermans, Vansteenwegen, Crombez, Baeyens, & Eelen, 2002; Kerkhof et al., 2009; Mitchell, Anderson, & Lovibond, 2003), from physiological responses (e.g., Vansteenwegen, Crombez, Baeyens, & Eelen, 1998), and from neurological responses (e.g., Klucken et al., 2009). One should, however, be aware of the fact that a change in a response can be labeled as EC only if it can be argued that the response is

18-Schachtman-18.indd 406

an evaluative response, that is, if it can be argued that the response provides an index of liking. For instance, some physiological responses such as skin conductance and modulation of the startle response seem to be determined by arousal level of stimuli rather than by the evaluative properties of stimuli and therefore do not qualify as indices of liking (e.g., Vansteenwegen et al., 1998). Nevertheless, the available evidence supports the conclusion that EC has been observed in a variety of evaluative responses. Nature of the Organism That Experiences the CS-US Relation

Species Many studies on EC have been conducted with human samples (see De Houwer et al., 2001, for a review). Some studies could be described as studies on EC in non-human animals (e.g., Boakes, Albertella, & Harris, 2007; Capaldi, 1992; Delamater, Campese, LoLordo, & Sclafani, 2006). Unfortunately, the literature on human and non-human EC has developed independently, perhaps because it is not clear whether (the determinants of) evaluative responses in human and non-humans are comparable. Societal Status and Age Most studies on EC involved psychology students, but some studies involved children (e.g., Baeyens, Eelen, Crombez, & De Houwer, 2001; Field, 2006; Fulcher, Mathews, & Hammerl, 2008) and community samples (e.g., Baeyens, Wrzesniewski, De Houwer, & Eelen, 1996). However, little is known about whether EC depends on the societal status or age of the participants. Personality and Mental Disorders Surprisingly few EC studies have taken into account the personality of the participants. One exception is a study by Baeyens et al. (1992a) who measured the “evaluative style” of their participants but failed to ﬁnd differences in EC depending on whether participants were “feelers” or “thinkers.” We know of one study that compared EC in patients and healthy controls. In this study, Blechert et al. (2007) observed that patients with poststraumatic stress disorder showed delayed extinction of EC compared to a healthy control group. This ﬁnding is somewhat puzzling given the fact that extinction of EC is

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

rarely observed in groups of healthy participants (see earlier discussion). Neurological and Genetic Properties Neurological research on EC is still in its infancy. There have been two studies on the role of the amygdaloid nuclear complex (ANC), a structure that is critically involved in Pavlovian conditioning of fear responses. Whereas Johnsrude, Owen, White, Zhao, and Bohbot (2000) found impaired EC in individuals with unilateral damage to the ANC, Coppens et al. (2006) did ﬁnd intact EC in these individuals. More research is thus needed to clarify the involvement of this and other brain regions in EC. To the best of my knowledge, there are no studies on the effects of the genetic makeup of individuals or the use of chemical substances on EC. Nature of the Context in Which the Relation Is Presented

Other Tasks A relation between stimuli cannot be presented in a vacuum but always occurs in a broader context in which other regularities are present. For instance, a particular CS-US relation can be present in a context in which participants are asked to fulﬁll certain tasks. Several studies conﬁrm that tasks that are present brieﬂy before or during the presentation of the CS-US relation can inﬂuence EC. Corneille, Yzerbyt, Pleyers, and Mussweiler (2009) recently found that EC effects were stronger when, brieﬂy before the presentation of the CS-US pairs, participants were asked to detect similarities between various kinds of pictures compared to when their task was to detect differences between pictures. Tasks that are present during the presentation of the CS-US pairs also seem to be able to inﬂuence EC. Whereas some researchers found that the presence of an attention-demanding task facilitates EC (e.g., Fulcher & Hammerl, 2001; Walther, 2002), others found that it reduces the size of EC (e.g., Field & Moore, 2005; Pleyers, Corneille, Yzerbyt, & Luminet, 2009). Finally, Baeyens, Eelen, and Van den Bergh (1990b) found that instructing participants to detect which CS goes together with which US did not inﬂuence the magnitude of EC. If anything, EC effects were smaller in the condition where participants were

18-Schachtman-18.indd 407

407

asked to discover the CS-US pairs. In sum, although it is clear that the broader context in which CS-US pairs are presented does inﬂuence EC, it is not clear how other regularities in the environment (such as secondary tasks) affect EC. Other Effects of the CS-US Relation The context in which a CS-US relation is presented consists not only of other contingencies and their effects on the organism but also of other effects of that CS-US relation. This consideration leads to the question of whether the effects that a CS-US relation has on the liking of the CS (i.e., EC) are somehow related to other effects of that CS-US relation (e.g., effects on physiological responses or on conscious knowledge). Research on this question has focused mainly on effects of the CS-US relation on responses that can be seen as indices of awareness of the CS-US relation. The results of this research are, however, mixed. Some studies suggest that EC is independent of awareness of the CS-US relation (see De Houwer et al., 2001; Field, 2005, for reviews). For instance, Baeyens et al. (1990a; also see Dickinson & Brown, 2007, but see Wardle, Mitchell, & Lovibond, 2007) found that a contingency between a ﬂavor and a bad aftertaste led to a change in liking of the ﬂavor, even though participants could not indicate which ﬂavor was paired with the bad aftertaste. When, however, there was a relation between the color of the drinks and the bad aftertaste, participants were able to indicate which color was paired with the bad aftertaste, but they did not change their liking of the drinks with that color. A similar dissociation between EC and awareness of the CS-US relation was found by Fulcher and Hammerl (2001). They found that manipulations that increased awareness of the CS-US relations (e.g., instructions to detect the contingencies, blockwise presentations of the CS-US pairs) actually decreased the magnitude of EC. In many other studies, however, a close link between awareness of the CS-US contingencies and EC has been observed. In a particularly convincing study, Pleyers et al. (2007) calculated for each participant the EC effect for CSs for which the participants could indicate the valence of the US with which it was paired and the EC effect for CSs for which

2/16/2011 5:53:50 PM

408

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

participants did not remember the valence of the associated US. Signiﬁcant EC occurred only for the former set of CSs. Similar results were found in several recent studies (e.g., Dawson, Rissling, Schell, & Wilcox, 2007; Stahl & Unkelbach, 2009; Stahl, Unkelbach, & Corneille,2009; Wardle et al., 2007). Nature of the Way in Which the CS-US Relation Is Communicated

In most EC studies, the CS and US stimuli are physically present and are thus experienced directly by the participants. However, a CS-US relation in the world can have an impact even when the organism does not experience the CS and US stimuli directly. First, EC can result also from observing other organisms that do directly experience the CS-US relation. In studies on observational EC, Baeyens, Vansteenwegen et al. (1996) videotaped a child who drank little cups of water, some of which also contained a product that resulted in a bad, soap-like aftertaste. The model always displayed a negative facial expression after drinking cups of water with a bad aftertaste and a positive facial expression after drinking water that did not have the bad aftertaste. Other children watched the video while drinking cups of water simultaneously with the model. None of the drinks of the observers contained the bad aftertaste. Instead, each drink contained one of two neutral fruit ﬂavors. The order of the drinks was arranged in such a way that the model always displayed a positive facial expression after the observers drank a cup of water with one ﬂavor (e.g., apricot). When the observers drank water with the second ﬂavor (e.g., lychee), the model always displayed a negative expression. Afterward, the observers reported that they liked the ﬁrst ﬂavor better than the second one. One explanation for this ﬁnding is that the facial expression of the model functioned as a US. From this perspective, the participants did experience the relation between the CS (ﬂavor) and US (facial expression) directly. Another explanation is that the observers were inﬂuenced by how the model reacted to the relation between the CS (ﬂavor) and US (bad aftertaste of the drink). A second way of presenting information about CS-US relations indirectly is by giving

18-Schachtman-18.indd 408

verbal instructions. For instance, Gregg, Seibt, and Banaji (2006) told some participants that members of a ﬁctitious social group called “Nifﬁtes” generally behaved in a positive manner, whereas members of a different ﬁctitious social group called “Luupites” generally behaved in a negative manner. Simply providing this information was sufﬁcient to create a preference for Nifﬁtes compared to Luupites, regardless of whether liking was measured using rating scales or derived from performance in a reaction time task. In a related study, De Houwer (2006) told participants the names of Nifﬁtes would be paired with positive stimuli, whereas names of Luupites would be paired with negative stimuli (or vice versa). Even though the stimuli were never actually presented, a reaction time measure of liking (i.e., the Implicit Association Test) showed that participants did like the Nifﬁtes names better than the Luupites names. Although there have been few studies that directly compare the effects of direct experiences of CS-US relations with the effects of verbal information about the same CS-US relations, the available evidence does allow for the conclusion that both ways of presenting information can lead to EC.

MENTAL PROCESS THEORIES Until now we have considered only the impact of elements of the procedure on EC. Our review shows that a lot can be learned about the determinants of EC at this level of explanation. In this section, we will try to explain this functional knowledge by describing mental processes that might underlie EC. The aim of these process theories is to explain how the pairing of stimuli can lead to changes in liking and why these changes depend on the nature of the procedure. The Conceptual-Categorization Account

According to Davey (1994), the pairing of a CS and a US can result in a change in the liking of the CS because it makes salient those features of the CS that it has in common with the US. For example, assume that an evaluatively neutral face has the features of brown eyes, long shape, full lips, and long hair. Also assume that this

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

neutral face is repeatedly presented together with a liked US that has the features of blue eyes, round shape, full lips, and long hair. The CS-US pairings are assumed to increase the salience of the features that the CS has in common with the US, that is, full lips and long hair. As a result, the CS is more likely to be categorized as a liked stimulus. Note that this explanation of EC does not refer to the existence of associations in memory but does attribute EC to the CS-US pairings. The model of Davey (1994) correctly predicts that EC should depend mainly on the number of co-occurrences of the CS and US because it is on these trials that the salience of the CS features can change. Once the salience of certain CS features has been increased, these changes in salience (and thus liking) might persist even when the CS or US is subsequently presented on its own. However, the model has difﬁculties explaining a number of other ﬁndings. First, EC has been found even when the CS and US belong to different modalities and therefore do not have features in common. Second, the model cannot explain the fact that revaluation of a particular US after the CS-US pairings inﬂuences the liking of the CS with which it was paired but not the liking of other CSs (Baeyens et al., 1992b; Walther et al., 2009). US revaluation might inﬂuence the nature of the features that a participant regards as typical for liked or disliked stimuli. This should, however, inﬂuence the liking of all CSs, not only the CS that was paired with that speciﬁc US. Third, the model does not provide an explanation for how merely instructing participants about a CS-US relation can result in EC. The Holistic Account

Martin and Levey (1978, 1994; Levey & Martin, 1975) postulated that the co-occurrence of a CS and a US automatically results in the formation of a holistic representation that encodes stimulus elements of both the CS and US, as well as the valence of the US. Once the holistic representation has been formed, the CS can activate this representation and thus the evaluation that was associated with the US. The holistic model correctly predicts that conditioned changes in liking should depend

18-Schachtman-18.indd 409

409

mainly on CS-US co-occurrences because these trials result in the formation of the holistic representation. Subsequent CS-only trials should not alter the holistic representation and thus should also not inﬂuence the conditioned change in liking. Hence, the model is in line with those studies that failed to ﬁnd extinction of EC. The model can explain the effect of US revaluation if it is assumed that the US can activate the holistic representation on the US-revaluation trials and if the new valence of the US can be integrated in the holistic representation. The holistic account also predicts that EC can occur in the absence of awareness. Some argue, however, that unconscious EC has still not been demonstrated conclusively (e.g., Dawson et al., 2007; Lovibond & Shanks, 2002). The model cannot explain that, in many cases, EC does occur only when participants are aware of the CS-US contingencies (e.g., Pleyers et al., 2007). It also fails to provide an account of how EC can occur on the basis of instructions, in the absence of any CS-US pairings. The Misattribution Account

Recently, Jones et al. (2009) proposed a misattribution theory according to which the evaluative reaction that is evoked by the US can be become associated with the CS on trials where the CS and US co-occur. In line with early behaviorist theories of conditioning (e.g., Thorndike, 1911; Watson, 1913), it is assumed that these S-R associations can be formed in the absence of awareness of the CS-US relation. Jones et al. do postulate, however, that the formation of an S-R association depends on what they call an “implicit misattribution” of the evaluative response to the CS. That is, participants need to (incorrectly) assume that the evaluation that they experience is caused by the CS rather than by the US. This misattribution can occur implicitly in that it does not depend on a conscious evaluation of the CS or US. Nevertheless, any variable that inﬂuences the likelihood that the US valence is misattributed to the CS should inﬂuence EC. Jones et al. indeed observed an impact of a number of these variables, including the size of the CS (feelings are more likely to be

2/16/2011 5:53:50 PM

410

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

attributed to large and thus salient CSs) and spatial proximity (feelings are more likely to be attributed to CSs that are close to a US). However, such effects could be explained also without invoking the assumption that EC depends on a misattribution of the evaluation that is evoked by the US (e.g., CS size and CS-US proximity could as such inﬂuence the formation of CS-US associations). The most striking support for the misattribution theory, however, comes from the ﬁnding that mildly valenced USs result in stronger EC effects than strongly valenced USs. Jones et al. (2009) explain this ﬁnding by assuming that the feeling evoked by strongly valenced USs is more likely to be correctly attributed to the US and thus less likely to be misattributed to the CS. However, the effect that Jones et al. observed was small (i.e., only marginally signiﬁcant despite a large sample) and present only in participants who were classiﬁed as unaware. Moreover, Baeyens et al. (1988) failed to ﬁnd an effect of US extremity. More research on this topic is clearly needed. The misattribution theory cannot explain the effects of US revaluation on EC because the representation that is assumed to underlie EC does not contain information about the stimulus properties of the US. Because of this, the US cannot activate the CS representation during the revaluation trials. The model can also not explain EC in the absence of CS-US co-occurrences, such as with indirect CS-US relations (i.e., sensory preconditioning) or EC as the result of instructions. The Referential Account

Baeyens et al. (1992b; Baeyens & De Houwer, 1995) postulated that there are two types of learning. The ﬁrst type concerns the learning of predictive relations by which the CS becomes a signal for the upcoming presentation of the US. This type of signal learning is assumed to underlie most cases of Pavlovian conditioning, that is, most effects of the pairing of stimuli (also see Rescorla, 1988). The second type concerns the learning of referential relations by which the CS becomes a stimulus that simply refers to (i.e., makes one think of) the US without

18-Schachtman-18.indd 410

becoming a signal for the actual presentation of the US. EC is assumed to depend on the second type of learning. Whereas Baeyens et al. (1992b) seemed to assume that the two types of learning actually depend on the formation of different types of CS-US associations in memory, De Houwer (1998; De Houwer et al., 2001) suggested that signal and referential learning depend on a single learning mechanism that produces only one type of CS-US association. According to De Houwer, signal and referential learning differ because the CS-US associations have a different effect on preparatory responses than on evaluative responses. Because referential learning is thought to be driven by the co-occurrence of stimuli, the referential model can explain why EC seems to be resistant to extinction (i.e., impervious to the effects of CS-only trials that are presented after CS-US trials). The redundancy of the CS-US relation should also not have an effect. Moreover, the model can explain the presence of US-revaluation effects because the change in liking of the US is assumed to be mediated by the activation of the US representation. Changing this representation during the US-revaluation trials should thus also affect the liking of the CS. Just like the holistic and the misattribution accounts, the referential account postulates that referential learning is independent of awareness of CS-US contingencies and should thus occur also in the absence of contingency awareness. However, the evidence on unaware EC is still inconclusive. Moreover, at least in certain cases, there does seem to be a close link between EC and contingency awareness (e.g., Pleyers et al., 2007). Finally, because association formation is assumed to be a gradual process that is driven by the actual presence of stimuli, it is difﬁcult to see how the referential model can explain EC as the result of instructions. The Propositional Account

De Houwer (2007; De Houwer, Baeyens, & Field, 2005) put forward the suggestion that EC, like all other forms of conditioning (see De Houwer, 2009; Mitchell et al., 2009a), might result from the formation of propositions about the CS-US

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

relation. According to this propositional account, the liking of the CS will change only after participants have formed the conscious proposition that the CS is paired with a valenced US. Although the model does not always explain how this propositional knowledge results in a change in liking (see Mitchell, De Houwer, & Lovibond, 2009b), it does postulate that the formation of a proposition about the CS-US relation is a necessary mediating step. One possible way in which propositions can inﬂuence liking is that participants use propositional knowledge about the CS-US relation as a justiﬁcation for determining how much they like the CS. For instance, the fact that a CS is paired with a negative US can be seen as a justiﬁcation for disliking the CS (De Houwer et al., 2005). Because the formation of propositions is a conscious and effortful process, the propositional account predicts that EC should depend on awareness of the CS-US relation. It would thus not be able to account for convincing evidence for unaware EC. The model also predicts that other tasks that direct attention away from the CS-US pairings should hamper EC. The evidence on this issue is mixed (e.g., Fulcher & Hammerl, 2001; Pleyers et al., 2009). Although the propositional model does not make strong predictions about the statistical properties of the CS-US relation that determine EC, it is compatible with the observation that EC is driven primarily by co-occurrences of the CS and US (i.e., that contiguity rather than contingency seems to matter). Co-occurrences would be primary in those cases where EC depends not on the formation of propositions about the statistical contingency between the CS and US but on the formation of propositions about the co-occurrence of the CS and US. Because EC is assumed to depend on knowledge about the US, revaluation of the US should inﬂuence EC. Finally, because propositional knowledge can result either from experience or from instructions, the model can account for EC as the result of instructions.

CONCLUSIONS The research on EC that we reviewed in this chapter clearly shows that the pairing of stimuli

18-Schachtman-18.indd 411

411

can lead to changes in liking of those stimuli. We have also learned that EC is a general phenomenon that occurs with many different stimuli, inﬂuences many types of evaluative responses, can be found in many different organisms and contexts, and can result both from experience, observation, and instruction. We have also learned that EC seems to be driven mostly by the co-occurrence of the CS and the US, whereas contingency and redundancy seem to be less important. Nevertheless, there are still many uncertainties about the conditions under which EC occurs and the mental processes that underlie EC. First, although EC is a general phenomenon, there have also been genuine failures to ﬁnd EC (e.g., Rozin, Wrzesniewski, & Byrnes, 1998). This suggests that there are subtle but important boundary conditions that need to be fulﬁlled before the pairing of stimuli results in a change in liking. Second, the literature on EC is characterized by many conﬂicting results, including on important topics such as the relation between EC and awareness of the CS-US contingencies, the impact of US revaluation, the impact of CS postexposure trials (i.e., extinction), the impact of other tasks that direct attention toward or away from the CS-US contingencies, and the relation between EC effects that are due to experience versus instructions. Because of these conﬂicting results, progress regarding our understanding of the mental processes that underlie EC has been limited. Different theories make different predictions regarding the role of contingency awareness, US revaluation, extinction, attention, and instructions, but the conﬂicting results interfere with the selection between or reﬁnement of these theories. As was suggested by De Houwer et al. (2005; De Houwer, 2007), it is possible that EC effects can be due to different mental processes. The conﬂicting results in the literature could thus be due to the fact that different processes underlie EC in the different studies. For instance, it is possible that when EC is caused by propositional processes, it will depend on awareness, US revaluation, CS postexposures, attention, and instructions. In cases where EC is independent from contingency awareness, US revaluation, CS postexposures, attention, or instructions, it might be due to more automatic processes

2/16/2011 5:53:50 PM

412

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

such as the formation of holistic representations. As argued by De Houwer (2007), an important task for future research on EC should thus be to uncover the variables that determine the properties of EC, that is, whether EC depends on awareness, US revaluation, CS postexposures, attention, or instructions. Such an approach can lead to new insights in the important phenomenon of evaluative conditioning.

ACKNOWLEDGMENTS The preparation of this chapter was made possible by grants BOF/GOA2006/001 and BOF09/ 01M00209 of Ghent University. I thank Helena Matute, the editors of this book, and an anonymous reviewer for comments on an earlier draft of this chapter. Correspondence should be addressed to Jan De Houwer, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium. Electronic mail can be sent to Jan.DeHouwer@ UGent.be.

REFERENCES Arnould, E., Price, L., & Zinkhan, G. (2004). Consumers (2nd ed.). New York, NY: McGraw– Hill/Irwin. Baeyens, F., Crombez, G., De Houwer, J., & Eelen, P. (1996). No evidence for modulation of evaluative ﬂavor-ﬂavor associations in humans. Learning and Motivation, 27, 200–241. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact, always in contact: EC is resistant to extinction. Advances in Behaviour Research and Therapy, 10, 179–199. Baeyens, F., & De Houwer, J. (1995). Evaluative conditioning is a qualitatively distinct form of classical conditioning: A reply to Davey (1994). Behaviour Research and Therapy, 33, 825–831. Baeyens, F., Eelen, P., Crombez, G., & De Houwer, J. (2001). On the role of beliefs in observational ﬂavor conditioning. Current Psychology, 20, 183–203. Baeyens, F., Eelen, P., Crombez, G., & Van den Bergh, O. (1992a). Human EC: Acquisition trials, presentation schedule, evaluative style and contingency awareness. Behaviour Research and Therapy, 30, 133–142. Baeyens, F., Eelen, P., & Van den Bergh, O. (1990). Contingency awareness in EC: A case for unaware affective-evaluative learning. Cognition and Emotion, 4, 3–18.

18-Schachtman-18.indd 412

Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1989a). Acquired affectiveevaluative value: Conservative but not unchangeable. Behaviour Research and Therapy, 27, 279–287. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1989b). The inﬂuence of CS-UCS perceptual similarity/dissimilarity on human evaluative learning and signal learning. Learning and Motivation, 20, 322–333. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1990). Flavor-ﬂavor and colorﬂavor conditioning in humans. Learning and Motivation, 21, 434–455. Baeyens, F., Eelen, P., Van den Bergh, O., & Crombez, G. (1992b). The content of learning in human EC: Acquired valence is sensitive to US-revaluation. Learning and Motivation, 23, 200–224. Baeyens, F., Hendrickx, H., Crombez, G., & Hermans, D. (1998). Neither extended sequential nor simultaneous feature positive training result in modulation of evaluative ﬂavor conditioning in humans. Appetite, 31, 185–204. Baeyens, F., Hermans, R., & Eelen, P. (1993). The role of CS-US contingency in human evaluative conditioning. Behaviour Research and Therapy, 31, 731–737. Baeyens, F., Vanhouche, W., Crombez, G., & Eelen, P. (1998). Human evaluative ﬂavor-ﬂavor conditioning is not sensitive to post-acquisition US-inﬂation. Psychologica Belgica, 38, 83–108. Baeyens, F., Vansteenwegen, D., De Houwer, J., & Crombez, G. (1996). Observational conditioning of food valence in humans. Appetite, 27, 235–250. Baeyens, F., Wrzesniewski, A., De Houwer, J., & Eelen, P. (1996). Toilet rooms, body massages, and smells: Two ﬁeld studies on human evaluative odor conditioning. Current Psychology, 15, 77–96. Bar-Anan, Y., De Houwer, J., & Nosek, B. A. (in press). The role of contingency memory, intentional processes and number of pairings in evaluative conditioning. Quarterly Journal of Experimental Psychology. Bierley, C., McSweeney, F. K., & Vannieuwkerk, R. (1985). Classical conditioning of preferences for stimuli. Journal of Consumer Research, 12, 316–323. Blechert, J, Michael, T., Vriends, N., Margraf, J., & Wilhelm, F. H. (2007). Fear conditioning in posttraumatic stress disorder: Evidence for

2/16/2011 5:53:50 PM

EVALUATIVE CONDITIONING

delayed extinction of autonomic, experiential, and behavioural responses. Behavior Research and Therapy, 45, 2019–2033. Blechert, J., Michael, T., Williams, S. L., Purkis, H. M., & Wilhelm, F. H. (2008). When two paradigms meet: Does evaluative learning extinguish in differential fear conditioning? Learning and Motivation, 39, 58–70. Boakes, R. A., Albertella, L., & Harris, J. A. (2007). Expression of ﬂavor preference depends on type of test and on recent drinking history. Journal of Experimental Psychology: Animal Behavior Processes, 33, 327–338. Bornstein, R. F. (1989). Exposure and affect: Overview and meta-analysis of research, 1968-1987. Psychological Bulletin, 106, 265–289. Brown, P.L., & Jenkins, H.M. (1968). Autoshaping of the pigeon’s key-peck. Journal of Experimental Analysis of Behavior, 5, 325–337. Capaldi, E. D. (1992). Conditioned food preferences. The Psychology of Learning and Motivation, 28, 1–33. Coppens, E., Vansteenwegen, D., Baeyens, F., Vandenbulcke, M., Van Paesschen, W., & Eelen, P. (2006). Evaluative conditioning is intact after unilateral resection of the anterior temporal lobe in humans. Neuropsychologia, 44, 840–843. Corneille, O., Yzerbyt, V. Y., Pleyers, G., & Mussweiler, T. (2009). Beyond awareness and resources: Evaluative conditioning may be sensitive to processing goals. Journal of Experimental Social Psychology, 45, 279–282. Davey, G. C. L. (1994). Deﬁning the important questions to ask about evaluative conditioning: A reply to Martin and Levey (1994). Behaviour Research and Therapy, 32, 307–310. Dawson, M. E., Rissling, A. J., Schell, A., & Wilcox, R. (2007). Under what conditions can human affective conditioning occur without contingency awareness? Test of the evaluative conditioning paradigm. Emotion, 7, 755–766. Dawson, M. E., & Schell, A. M. (1987). Human autonomic and skeletal classical conditioning: The role of conscious cognitive factors. In G. Davey (Ed.) Cognitive processes and Pavlovian conditioning in humans (pp. 27–56). Chichester, England: Wiley. De Houwer, J. (1998). Leren is eenvoudig, doen is moeilijk: Een nieuwe visie op het onderscheid tussen evaluatieve en Pavloviaanse conditionering. [Learning is simple, performance is difﬁcult: A new view on the distinction between evaluative

18-Schachtman-18.indd 413

413

and Pavlovian conditioning]. Gedragstherapie, 31, 49–66. De Houwer, J. (2006). Using the implicit association test does not rule out an impact of conscious propositional knowledge on EC. Learning and Motivation, 37, 176–187. De Houwer, J. (2007). A conceptual and theoretical analysis of EC. The Spanish Journal of Psychology, 10, 230–241. De Houwer, J. (2009). Leerpsychologie [Learning Psychology]. Gent, Belgium: Academia Press. De Houwer, J. (in press). Why the cognitive approach in psychology would proﬁt from a functional approach and vice versa. Perspectives on Psychological Science. De Houwer, J., Baeyens, F., & Field, A. P. (2005). Associative learning of likes and dislikes: Some current controversies and possible ways forward. Cognition and Emotion, 19, 161–174. De Houwer, J., Baeyens, F., Vansteenwegen, D., & Eelen, P. (2000). Evaluative conditioning in the picture-picture paradigm with random assignment of CSs to USs. Journal of Experimental Psychology: Animal Behavior Processes, 26, 237–242. De Houwer, J., Hermans, D., & Eelen, P. (1998). Affective and identity priming with episodically associated stimuli. Cognition and Emotion, 12, 145–169. De Houwer, J., Thomas, S., & Baeyens, F. (2001) Associative learning of likes and dislikes: A review of 25 years of research on human EC. Psychological Bulletin, 127, 853B869. Delamater, A. R., Campese, V., LoLordo, V. M., & Sclafani, A. (2006). Unconditioned stimulus devaluation effects in nutrient-conditioned ﬂavor preferences. Journal of Experimental Psychology: Animal Behavior Processes, 32, 295–306. Diaz, E., Ruis, G., & Baeyens, F. (2005). Resistance to extinction of human evaluative conditioning using a between-subjects design. Cognition and Emotion, 19, 245–268. Dickinson, A., & Brown, K. J. (2007). Flavor evaluative conditioning is unaffected by contingency knowledge during training with color– ﬂavor compounds. Learning and Behavior, 35, 36–42. Dwyer, D. M., Jarratt, F., & Dick, K. (2007). Evaluative conditioning with foods as CSs and body shapes as USs: No evidence for sex differences, extinction, or overshadowing. Cognition and Emotion, 21, 281–199.

2/16/2011 5:53:51 PM

414

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Eelen, P. (1980). Klassieke conditioning: Klassiek en toch modern [Classical conditioning: Classic but nevertheless modern]. In J. R. Nuttin (Ed.), Gedrag, dynamische relatie en betekeniswereld [Behaviour, dynamic relation, and world of meaning] (pp. 321–343). Leuven, Belgium: Universitaire Pers Leuven. Evans, M., Jamal, A., & Foxall, G. R. (2006). Consumer behaviour. Chichester, England: Wiley. Field, A. P. (2006). I don’t like it because it eats sprouts: Conditioning preferences in children. Behavior Research and Therapy, 44, 439–455. Field, A. (2005). Learning to like (or dislike): Associative learning of preferences. In A. Wills (Ed.), New directions in human associative learning (pp. 221–252). Mahwah, NJ: Lawrence Erlbaum. Field, A. P., & Davey, G. C. L. (1999). Reevaluating EC: A nonassociative explanation of conditioning effects in the visual EC paradigm. Journal of Experimental Psychology: Animal Behaviour Processes, 25, 211–224. Field, A. P., & Moore, A. C. (2005). Dissociating the effects of attention and contingency awareness on evaluative conditioning effects in the visualparadigm. Cognition and Emotion, 19, 217–243. Fulcher, E. P., & Hammerl, M. (2001). When all is revealed: A dissociation between evaluative learning and contingency awareness. Consciousness and Cognition, 10, 524–549. Fulcher, E. P., Mathews, A., & Hammerl, M. (2008). Rapid acquisition of emotional information and attentional bias in anxious children. Journal of Behavior Therapy and Experimental Psychiatry, 39, 321–339. Garcia, J., & Koelling, R. A. (1966). The relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gregg, A. P., Seibt, B., & Banaji, M. R. (2006). Easier done than undone: Asymmetry in the malleability of implicit preferences. Journal of Personality and Social Psychology, 90, 1–20. Hammerl, M., Bloch, M., & Silverthorne, C. P. (1997). Effects of US-alone presentations on human evaluative conditioning. Learning and Motivation, 28, 491–509. Hammerl, M., & Grabitz, H-J. (1996). Human evaluative conditioning without experiencing a valued event. Learning and Motivation, 27, 278–293. Hammerl, M., & Grabitz, H. -J. (2000). Affectiveevaluative learning in humans: A form of

18-Schachtman-18.indd 414

associative learning or only an artifact? Learning and Motivation, 31, 345–363. Hardwick, S. A., & Lipp, O. V. (2000). Modulation of affective learning: An occasion for evaluative conditioning? Learning and Motivation, 31, 251–271. Hermans, D., Baeyens, F., Lamote, S., Spruyt, A., & Eelen, P. (2005). Affective priming as an indirect measure of food preferences acquired through odor conditioning. Experimental Psychology, 52, 180–186. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Johnsrude, I. S., Owen, A. M., White, N. M., Zhao, W. V., & Bohbot, V. (2000). Impaired preference conditioning after anterior temporal lobe resection in humans. Journal of Neuroscience, 20, 2649–2656. Jones, C. R., Fazio, R. H., & Olson, M. A. (2009). Implicit misattribution as a mechanism underlying evaluative conditioning. Journal of Personality and Social Psychology, 96, 933–948. Kerkhof, I., Goesaert, E., Dirikx, T., Vansteenwegen, D., Baeyens, F., D’Hooge, R., & Hermans, D. (2009). Assessing valence indirectly and online. Cognition and Emotion, 23, 1615–1629. Klucken, T., Kagerer, J., Schweckendiek, J., Tabbert, K., Vaitl, D., & Stark, R. (2009). Neural, electrodermal and behavioral response patterns in contingency aware and unaware subjects during a picture-picture conditioning paradigm. Neuroscience, 158, 721–731. Lamote, S., Baeyens, F., Hermans, D., & Eelen, P. (2004). An exploration of affective priming as an indirect measure of food attitudes. Appetite, 42, 279–286. Levey, A. B., & Martin, I. (1975). Classical conditioning of human > evaluative = responses. Behaviour Research and Therapy, 4, 205–207. Lipp, O. V., Neumann, D. L., & Mason, V. (2001). Stimulus competition in affective and relational learning. Learning and Motivation, 32, 306–331. Lipp, O. V., Oughton, N., & LeLievre, J. (2003). Evaluative learning in human Pavlovian conditioning: Extinct, but still there? Learning and Motivation, 34, 219–239.

2/16/2011 5:53:51 PM

EVALUATIVE CONDITIONING

Lipp, O. V., & Purkis, H. M. (2006). The effects of assessment type on verbal ratings of conditional stimulus valence and contingency judgement: Implications for the extinction of evaluative learning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 431–440. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Lubow, R. E., & Gewirtz, J. C. (1995). Latent inhibition in humans: Data, theory, and implications for schizophrenia. Psychological Bulletin, 117, 87–103. Martin, I., & Levey, A. B. (1978). Evaluative conditioning. Advances in Behaviour Research and Therapy, 1, 57–102. Martin, I., & Levey, A. B. (1987). Learning what will happen next: Conditioning, evaluation and cognitive processes. In G. Davey (Ed.), Cognitive processes and Pavlovian conditioning in humans (pp 57–82). New York, NY: John Wiley. Martin, I., & Levey, A. B. (1994). The evaluative response: Primitive but necessary. Behaviour Research and Therapy, 32, 301–305. Miller, R. R., & Matute, H. (1996). Biological signiﬁcance in forward and backward blocking: Resolution of a discrepancy between animal conditioning and human causal judgment. Journal of Experimental Psychology: General, 125, 370–386. Mitchell, C. J., Anderson, N. E., & Lovibond, P. F. (2003). Measuring EC using the implicit association test. Learning and Motivation, 34, 203–217. Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009a). The propositional nature of human associative learning. Behavioral and Brain Sciences, 32, 183–198. Mitchell, C. J., De Houwer, J., & Lovibond, P. F. (2009b). Link-based learning theory creates more problems than it solves. Behavioral and Brain Sciences, 32, 230–246. Olson, M., & Fazio, R. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 5, 413–417. Pavlov, I. (1927). Conditioned reﬂexes. London, England: Oxford University Press. Peeters, G., & Czapinski, J. (1990). Positive-negative asymmetry in evaluations: The distinction between affective and informational negativity effects. European Review of Social Psychology, 1, 33–60.

18-Schachtman-18.indd 415

415

Pleyers, G., Corneille, O., Luminet, O., & Yzerbyt, V. (2007). Aware and (dis)liking: Itembased analyses reveal that valence acquisition via EC emerges only when there is contingency awareness. Journal of Experimental Psychology: Learning, Memory and Cognition, 33, 130–144. Pleyers, G., Corneille, O., Yzerbyt, V., & Luminet, O. (2009). Evaluative conditioning incurs attentional costs. Journal of Experimental Psychology: Animal Behavior Processes, 35, 279–285. Rescorla, R. A. (1988) Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43, 151–160. Rozin, P., Wrzesniewski, A., & Byrnes, D. (1998). The elusiveness of EC. Learning and Motivation, 29, 397–415. Staats, A. W., & Staats, C. K. (1957). Meaning established by classical conditioning. Journal of Experimental Psychology, 54, 74–80. Stahl, C., & Unkelbach, C. (2009). Evaluative learning with single versus multiple USs: The role of contingency awareness. Journal of Experimental Psychology: Animal Behavior Processes, 35, 256–291. Stahl, C., Unkelbach, C., & Corneille, O. (2009). On the respective contributions of awareness for US valence and US identity in valence acquisition through evaluative conditioning. Journal of Personality and Social Psychology, 97, 404–420. Stevenson, R. J., Boakes, R. A., & Wilson, J. P. (2000). Resistance to extinction of conditioned odour perceptions: Evaluative conditioning is not unique. Journal of Experimental Psychology: Learning, Memory and Cognition, 26, 423–440. Stuart, E. W., Shimp, T. A., & Engle, R. W. (1987). Classical conditioning of consumer attitudes: Four experiments in an advertising context. Journal of Consumer Research, 14, 334–351. Todrank, J., Byrnes, D., Wrzesniewski, A., & Rozin, P. (1995). Odors can change preferences for people in photographs: A cross-modal evaluative conditioning study with olfactory USs and visual CSs. Learning and Motivation, 26, 116–140. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York, NY: MacMillan. Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (1998). Extinction in fear conditioning: Effects on startle modulation and evaluative self-reports. Psychophysiology, 35, 729–736. Vansteenwegen, D., Francken, G., Vervliet, B., De Clercq, A., & Eelen, P. (2006). Resistance to extinction in evaluative conditioning. Journal of

2/16/2011 5:53:51 PM

416

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Experimental Psychology: Animal Behavior Processes, 32, 71–79. Walther, E. (2002). Guilty by mere association: EC and the spreading attitude effect. Journal of Personality and Social Psychology, 82, 919–934. Walther, E., Gawronski, B., Blank, H., & Langer, T. (2009). Changing likes and dislikes through the back door: The US-revaluation effect. Cognition and Emotion, 23, 889–917. Wardle, S. G., Mitchell, C. J., & Lovibond, P. F. (2007) Flavor evaluative conditioning and

18-Schachtman-18.indd 416

contingency awareness. Learning and Behavior, 35, 233–241. Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177. Zellner, D. A., Rozin, P., Aron, M., & Kulish, C. (1983). Conditioned enhancement of human’s liking for ﬂavor by pairing with sweetness. Learning and Motivation, 14, 338–350.

2/16/2011 5:53:51 PM

CHAPTER 19 Instrumental and Pavlovian Conditioning Analogs of Familiar Social Processes Robert Ervin Cramer and Robert Frank Weiss

Participation in conversation is reinforced by the opportunity to speak in reply. People will learn an instrumental response, the sole reinforcement for which is the deliverance of another human being from suffering. Increasing or decreasing N-opponents in a competitive situation facilitates learning an instrumental response. Attitudinal agreements are less reinforcing from a person who, as a result of the agreements, is increasingly more attractive. And a supervisor will rate a new worker’s causal agency for high productivity lower if a consistently productive worker also is present. These fascinating relationships predicted and discovered in our speaking in reply, altruism, competition, interpersonal attraction, and causal relationship detection research, respectively, illustrate the power of learning theory for illuminating social process. A great body of research with roots in the work of Thorndike and Pavlov, and in the Hull-Miller-Spence tradition, informed and guided the social psychological experiments described in this chapter.

With human social behavior, psychology provides the principles of learning and of innate drives, cues, rewards, and responses. The other social sciences, such as sociology and social anthropology, describe the conditions of learning— or in other words, the location of the rewards, punishments, and other conditions of the social maze. One must know both the psychological principles and the social conditions in order to predict human behavior. —Neal Miller (1959)

INTRODUCTION Do learning principles apply only to rats and neurotics? Many psychologists continue to think so, despite the impressive gains of recent years in the learning-theoretical study of human behavior. Indeed, it is not so long since some rat runners, intimidated by the propagandists of (central processing unit) “cognitive revolution” could be heard to say, “I don’t care if it only applies to

animals and abnormal behavior, it’s what I do and I like it.” We ourselves ought not to accept a dismissive attitude toward learning-based clinical research, not only because of its practical importance but because what applies to abnormal behavior largely applies ipso facto to normal behavior. Together with others in this volume, we seek to convey a more expansive view of the reach of learning principles. Our own work extends conditioning principles to the realm of complex social behavior, and we will here show in detail how instrumental and Pavlovian principles govern social process in altruism, competition, interpersonal communication, social attraction, and the role of human agency in causal relationship detection. For each of these ﬁve social domains we have constructed a theory, employing a method common in physics by which a relatively wellknown body of knowledge (here Pavlovian or instrumental conditioning) is used by analogy as 417

19-Schachtman-19.indd 417

2/16/2011 6:50:45 PM

418

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

a model to predict one of our less-well-developed social domains. Analogy runs deep: For each theory we devise an artiﬁcial social structure (experimental apparatus and procedure) to the blueprint of instrumental or Pavlovian conditioning. To discover principles governing social cues we use the laws of Pavlovian conditioning (with particular emphasis on compound cues) jointly with Rescorla-Wagner theory as models to predict two of the ﬁve social domains. To similarly illuminate social motivation (social drives) and social reinforcement, we use discrete-trials instrumental conditioning as a model for three other domains. We now offer a brief example, just enough to give an impression of what is to come. Suppose that engaging in competition engenders an aversive drive: Then one good way to develop this supposition is by an instrumental escape conditioning model. Theoretical analogies can be developed easily if we diagram the structure of a reinforced trial, here (see Table 19.1) using the language of Miller and Dollard’s classic Social Learning and Imitation (1941). Because we are interested in the drive and reinforcement aspects of competition (if such things be), we have made the cue and response analogs as unproblematic as possible. Then, if engaging in competition is like electric shock, inducing aversive drive, the offset of competition should be reinforcing. Number of competition trials is the analog of number of (reinforced) escape conditioning trials. The time from the presentation of the (scoring-method “shift”) signal light until the participant makes the button-push provides an analog of escape response latency and its reciprocal, speed. Because the relationship between number of escape conditioning trials and speed is known to be a negatively accelerated increasing function in the conditioning model, it therefore follows that the same relationship is predicted

to hold between the analogous variables in competition. That is the relationship that our experiments show. A look at Table 19.1 invites us to examine the time between the button-push and the offset of competition as an analog of delay of reinforcement. Theory predicts that asymptotic speed of button-push should be a negatively accelerated decreasing function of delay of competition offset: That gradient was revealed by experiment. There is good reason to take N-opponents as an analog of drive intensity when participants compete as individuals (no coalitions or persecution of isolates, etc.), and speed was faster when competing against three opponents rather than one. In escape conditioning, shock can be reduced as reinforcement rather than terminated completely, and speed is then a negatively accelerated increasing function of the amount of shock reduction. Similarly, speed of the button-push is a negatively accelerated increasing function of the amount of reduction in N-opponents. To ﬁnd out how far our analogies hold true, and in hope of discovering social psychological facts that are not pieces to different jigsaw puzzles, we shall use Pavlovian compound-cue and instrumental conditioning models to provide more demanding analyses of ﬁve social domains than those offered in this brief illustration.

THEORETICAL METHOD Modeling

Using a general approach termed “extension of liberalized S-R theory” by Neal Miller (1959), we draw close analogies between familiar instrumental and Pavlovian conditioning variables and variables assumed to be important in the social process under investigation. Rules of

Table 19.1 Structure of a (Reinforced) Instrumental Escape Conditioning Trial and a Competition Trial: Analogous Sequences Drive

Cue

Response

Reinforcement

Competition-Onset

Signal Lights

Button-Push

Competition-Offset

Shock-Onset

Door Up

Hurdle Jump

Shock-Offset

19-Schachtman-19.indd 418

2/16/2011 6:50:45 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Correspondence or a Dictionary of Analogies relate the independent and dependent variables in the conditioning model to the corresponding (analogous) independent and dependent variables in the social process to be predicted and explained. Testable implications are feasible when analogies among conditioning and social process independent and dependent variables are speciﬁed, and when the conditioning model the social variables are analogous to is speciﬁed (i.e., instrumental reward conditioning, instrumental escape conditioning, Pavlovian conditioning). Upon this construction, the functional relations holding among the variables in the conditioning model must, theoretically, hold among the corresponding social process variables (Hesse, 1966, 1974, 1980; Masterman, 1980; Oppenheimer, 1956). By systematically using instrumental and Pavlovian models and procedures for investigating social processes, it is possible to use the known principles of conditioning to determine whether analogous principles operate in the social process under investigation.

419

Analogy and Reduction

Our experiments do not duplicate natural social situations in all their richness and complexity. Rather, artiﬁcial social situations that rigorously conformed to discrete-trials instrumental and Pavlovian conditioning paradigms were created in which the operation of our social analogs of conditioning could be observed clearly under special, “pure” circumstances. Nevertheless, social processes are not assumed to represent a “higher level” of phenomena, which can only be fully explained by reducing them to a “lower level” of phenomena such as instrumental and Pavlovian conditioning. Rather, the theoretical method involved the use of a relatively well-understood model for investigating that, which was presently less well understood. Our programmatic assumptions, in fact, are commonplaces of scientiﬁc method. That is, well-constructed models and analogies helped to stimulate and guide research, and to integrate broad ranges of knowledge through an underlying set of common principles. Boundary Conditions

Models and Translations

The systematic use of conditioning models to investigate social processes does not represent the “mere translation” of one behavioral science language into another. In fact, a “mere translation” requires that a hypothesis describing a relationship among the social process variables being investigated already exists in the research literature, and the relationship was merely translated into statements involving conditioning principles. Rather, it has been our experience that conditioning models quite often generate intriguing hypotheses about relationships not found in the social psychological literature. When Cramer, Lutz, Bartell, Dragna, and Helzer (1989), for example, reported social analogs of partial reinforcement, delay of reinforcement, and intermittent shock in interpersonal communication involving a female participant, a masculine male, and an androgynous male, these reinforcing and motivational relationships did not exist in the scientiﬁc literature on sex roles.

19-Schachtman-19.indd 419

As in all theoretical constructions, the speciﬁcation of analogies between conditioning variables and variables in the social process under investigation was intended to apply within a limited range of conditions (Logan, 1959). Because our social psychological research is informed by instrumental and Pavlovian conditioning models, many of the most essential boundary conditions of our research including the use of discrete-trials procedures, conditioning of a single response, controlling and manipulating temporal parameters, varying stimulus intensities, control of competing responses, and the speciﬁcation of conditioning dependent variables result from analogies with conditioning principles and procedures. Although the recognition of familiar boundary conditions may be unnecessary for learning researchers, the speciﬁcation of even an illustrative list is not a trivial matter to social psychologists who assume that human thoughts, feelings, and actions result from cognitive capacities not granted to rats.

2/16/2011 6:50:45 PM

420

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

INSTRUMENTAL CONDITIONING ANALOGS Instrumental conditioning principles and procedures have, in our laboratories, been extended to social processes, including altruism (e.g., Weiss, Buchanan, Altstatt, & Lombardo, 1971), competition (e.g., Steigleder, Weiss, Cramer, & Feinberg, 1978), interpersonal attraction (e.g., Lombardo, Libkuman, & Weiss, 1972), interpersonal communication (e.g., Weiss, Lombardo, Warren, & Kelley, 1971), nonconformity (Seybert & Weiss, 1974), sex roles (Cramer et al., 1989), and social facilitation (Weiss & Miller, 1971). The experimental paradigm corresponded to discrete-trials instrumental conditioning involving drive, cue, response, and reinforcement (Miller & Dollard, 1941). For example, in the Cramer et al. sex role experiments a communication trial began with the onset of a putative noxious social stimulus (drive): communication with a masculine male. The participant learned, upon presentation of a cue, to make an instrumental response (IR, switch-press) that was followed by the contingent opportunity to listen to an androgynous male (reinforcement). By exploiting analogs of instrumental conditioning principles and procedures it has been possible to demonstrate that a number of familiar social processes, in fact, exhibit known characteristics of learning-theoretical aversive stimuli (e.g., shock) and negative reinforcers, and therefore are analogous to known aversive drives and negative reinforcement. Rules of Correspondence (Instrumental Escape Conditioning)

General rules of correspondence relating instrumental escape conditioning variables to social process variables are provided in Table 19.2. These general rules, numbered here for later reference, are illustrative rather than exhaustive, but they are described in sufﬁcient detail to illuminate our interpersonal communication, altruism, and competition research. Interpersonal Communication: Speaking in Reply

For people who enjoy a spirited discussion with its normal “give and take,” it is reasonable to

19-Schachtman-19.indd 420

assume that an opportunity to put in their “two cents” when another person disagrees with their opinions is positively reinforcing. In such a case people ﬁnd a spirited discussion a pleasant pastime. This insight, together with the (intuitive) resemblance of the reply to a goal response (RG), was a key element in the stock of pretheoretical ideas that gave rise to the appetitive ﬁrst version of the theory. If disagreements are noxious, then the opportunity to reply could be, arguably, negatively reinforcing, and the reply might even intuitively be regarded as a coping-response that inhibits aversive drive (e.g., McAllister, McAllister, & Benton, 1983; Miller & Dollard, 1941, p. 60; Mowrer & Viek, 1948). Enjoyable spirited discussion in the lab led us to entertain both instrumental reward and escape models, and experiments resolved the question. General Method

On each interpersonal communication trial, participants ﬁrst listened to another person disagree with their opinion on a preselected topic of general interest, and then, upon presentation of a cue, threw a switch, the reinforcement for which was the opportunity to speak in reply to the other person. To mask the conditioning contingencies, the experiment was described as an opinion change study and opinion change was duly measured. Nonverbal communication effects were controlled by having the “other person,” participant, and experimenter sit in separate rooms, with the instructions and all comments taking place via an intercom system. Although the “other person” was, in fact, simulated by tape recordings, the participant could hear the experimenter giving the “other person” instructions and an occasional request to speak louder. An invariant cycle of events masked the experiment’s discrete-trials nature. A communication cycle began with participants listening to another person disagreeing with their opinion for approximately 20 s. When the “other person” ﬁnished commenting, the participant received the cue “throw switch if you wish to comment” and threw the “comment” switch (IR). The IR was measured using a latency timer. On the reinforced trials the participant received a “talk”

2/16/2011 6:50:45 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

421

Table 19.2 Rules of Correspondence Relating Instrumental Escape Conditioning Variables and Social Process Variables Rule

Escape Conditioning (Model)

Social Process (Analog)

I-1

N Reinforced trials

N trials instrumental response analog (IR) eliminates/reduces noxious social stimulus

I-2

N Nonreinforced trials

N trials IR does not eliminate/reduce noxious social stimulus

I-3

Partial reinforcement

IR sometimes does and sometimes does not eliminate/reduce noxious social stimulus

I-4

Extinction

N I-2 trials following series of I-1 trials

I-5

Omission of noxious stimulus (drive)

Omission of noxious social stimulus

I-6

Intermittent shock

Mixture of trials with and without noxious social stimulus

I-7

Delay of reinforcement

Time interval between IR and the reinforcing elimination/reduction of noxious social stimulus

I-8

Magnitude of reinforcement

Quantity of reduction of noxious social stimulus

I-9

Magnitude of reinforcement

Quality of reduction of noxious social stimulus

I-10

Correlated reinforcement

Elimination/reduction of noxious social stimulus contingent on slow IR speed

I-11

Intensity of noxious stimulus (drive)

Intensity of noxious social stimulus

I-12

Speed of instrumental response

Speed of instrumental response analog

signal and spoke in reply. Because interrupting was likely to be punishing (Mandler & Watson, 1966), the participants were allowed to exceed a recommended 20 s speaking time. On the nonreinforced trials the “talk” signal was not illuminated and the participant waited 20 s. Speaking in reply (reinforcement) was contingent on performing the IR, with response speed being the dependent variable. At the end of a conversation cycle, participants performed the masking task by registering their opinion change. Reinforcing Effects of Speaking in Reply Delay of Reinforcement Analog

In instrumental conditioning, response speed is faster with short delays of reinforcement than with long delays of reinforcement (Fowler &

19-Schachtman-19.indd 421

Trapold, 1962; see also McAllister & McAllister, 1992; Tarpy & Koster, 1970; Tarpy & Sawabini, 1974). If, after listening to another person disagree, the opportunity to speak in reply functions as reinforcement for the switch-throwing response, then the time interval between the response and the opportunity to reply should have functional properties of delay of reinforcement (Rules of Correspondence I-1, I-7, and I-12). Weiss, Lombardo et al. (1971) reported that response speeds increased as a function of the number of reinforced trials, that immediate reinforcement produced faster responding than delayed reinforcement, and that a multiplicative relationship existed between the delay of reinforcement and the number of conditioning trials. A six-group parametric study revealed a perfectly monotonic analog of a delay of reinforcement

2/16/2011 6:50:45 PM

422

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

gradient (Fig. 19.1; Weiss, Boyer, Colwick, & Moran, 1971). In a 2x2 study of delay shifts, speed was once again faster when the opportunity to speak in reply was immediate rather than delayed. Participants shifted from short-tolong delay matched speeds with the constant long-delay controls, while participants shifted from long-to-short delay matched speeds with constant short-delay controls (Weiss, Steigleder, Cramer, & Feinberg, 1977). Correlated Reinforcement Analog

In Logan’s correlated reinforcement procedure (discrete-trials DRL), reinforcement is made contingent upon the subject responding slower than an established cutoff value (Logan, 1960, 1961; escape conditioning, Bower, 1960). Response speed increases over conditioning trials until the cutoff is exceeded and reinforcement stops, following which speed decreases until it stabilizes just below the value of the cutoff. Figure 19.2 shows interpersonal communication effects that have an almost eerie similarity to correlated reinforcement effects in instrumental

conditioning (Rule I-10). A yoked control group receiving the same sequence of reinforcement as the correlated reinforcement group continued to improve after its experimental counterpart began to match the cutoff (Weiss, Boyer et al., 1971). We have replicated correlated reinforcement effects (Weiss, Cluts, Williams, & Miller, 1977). Observing correlated reinforcement effects in interpersonal communication, or in any other social process, is not a trivial matter because it shows that the effects were not due to the simple development of motor skill as a function of practice. Both the correlated reinforcement group and the yoked controls received the same amount of practice, the same number of reinforcements and nonreinforcements, and even the same sequence of reinforcements and nonreinforcements. The same design logic obtains in investigations of correlated delay of reinforcement. If the participant responds faster than the cutoff value, then the replyreinforcement is delayed (rather than omitted)

100

90

90 Speed (100/Latency)

85 80 Speed (100/Latency)

NON-CORRELATED

75 70

80

70 - - - - - - - - - - - - - - - - - - - - - - - CUTOFF - 60 CORRELATED

65 50 60 40

55 50

1 2 0

3

6

9 15 Delay (s)

21

2 3

3 4

4 5

5 6

6 7

7 8

8 9 10 11 9 10 11 12

Rolling Blocks of Two Trials

Figure 19.2 Correlated reinforcement analog: Figure 19.1 Delay

of reinforcement analog: Response speed as a function of delay of the reinforcing opportunity to speak in reply. (Redrawn from Weiss, Boyer, Colwick, & Moran, 1971).

19-Schachtman-19.indd 422

Acquisition curves of response speed under correlated (experimental group) and non-correlated (yoked control group) opportunity to speak in reply. (Redrawn from Weiss, Boyer et al., 1971).

2/16/2011 6:50:45 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

and participants learn to match the cutoff value just as do Logan’s rats (Logan, 1960; Weiss, Beck, & Stich, 1972). The yoked controls acquire the response under variable delay. The acquisition, delay of reinforcement, and correlated reinforcement effects found in our interpersonal communication experiments reported thus far are compatible with either an instrumental reward conditioning or an instrumental escape conditioning model. By developing additional analogs of instrumental conditioning principles and procedures, we were able to answer the question, “Is speaking in reply analogous to positive reinforcement or negative reinforcement?” Partial Reinforcement Analog

In instrumental reward conditioning, asymptotic response speed is faster under partial than it is under continuous reinforcement (e.g., Amsel, 1992; Capaldi, 1978; Goodrich, 1959). But in escape conditioning this pattern is reversed: partial slower than continuous (e.g., Bower, 1960; Gray, 1982; Woods, Markman, Lynch, & Stokely, 1972). In both paradigms, resistance to extinction is generally superior following acquisition with (say, 50%) partial reinforcement. If the opportunity to speak in reply occurs on only half the trials, it should then have the functional properties of partial reinforcement (Rule I-3). Weiss, Lombardo et al. (1971) obtained extinction effects typical of instrumental conditioning with gradually decreasing response speeds over extinction trials (Rule I-4) and greater resistance to extinction for the partial group. They discovered that acquisition speeds were analogous to those of escape conditioning: partial slower than continuous. These acquisition results ﬁrst led us to think that we were dealing with an analog of negative reinforcement, so we conducted an experiment with extended acquisition trials that replicated the slower speed of demonstrably asymptotic partials. Disagreement-Induced Drive Drive Intensity Analog

In escape conditioning, response speed is an increasing function of the intensity of the

19-Schachtman-19.indd 423

423

noxious stimulus (drive) when the drive is terminated completely after each escape (Franchina, 1969b; Trapold & Fowler, 1960). However, in their classic experiment, Campbell and Kraeling (1953) showed that when the amount of reinforcing drive reduction is held constant, speed is a decreasing function of the intensity of the noxious stimulus (see also Campbell, 1968; McAllister & McAllister, 1992; McAllister et al., 1983; Myers, 1969). An analog of Campbell and Kraeling effects was obtained by varying the formidability with which disagreements were set forth. While the formidable disagreements were superior to our standard disagreements in their logic, clarity, and factual support (drive intensity; Rule I-11; also Byrne, 1971), they did not generate differences in either the duration or quality of the participant’s reply (constant drive reduction). If the reply is viewed as a coping response that inhibits aversive drive (e.g., McAllister et al., 1983; Miller & Dollard, 1941, p. 60; Mowrer & Viek, 1948), then standard disagreements were all that our participants could cope with. As in conditioning, escape speed was actually faster for low drive (standard disagreement) than for high drive (formidable disagreement; Weiss, Lombardo et al., 1971). Energization Analog

Because attitudinal disagreements function as noxious stimuli, they should produce effects analogous to energization by irrelevant drive (Lombardo, Libkuman et al., 1972). To test this possibility, participants were given an opportunity to make the initial comment on topics of low and high interest. Participants, depending on treatment group, were then either disagreed with on high-interest topics and agreed with on low-interest topics (high drive analog; Rule I-11) or agreed with on high-interest topics and disagreed with on low-interest topics (low drive analog). Following the drive arousing interpersonal communication phase of the experiment, the participants immediately performed a pairedassociates task (Spence, Farber, & McFann, 1956). Verbal and nonverbal paired-associates tasks have been used because of their success in experiments studying the energization properties of aversive drives: manifest anxiety (Spence &

2/16/2011 6:50:45 PM

424

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Spence, 1966), frustration (Schmeck, 1970), conﬂict (Castaneda, 1965), time stress (Castaneda & Lipsitt, 1959), cognitive dissonance (Waterman, 1969), audience observation (Cottrell, 1968), and unsolvable-uncontrollable intellectual tasks (Feinberg, Miller, Weiss, Steigleder, & Lombardo, 1982, Exp. 6). As predicted from a modest expansion of our theory and Hull-Miller-Spence learning theory for aversive drives (Dollard & Miller, 1950; Hull, 1943; Logan, 1959; Spence, 1956), the energizing effects of disagreement-induced drive interacted with the competitive and noncompetitive paired-associates lists. In terms of the number of errors made and trials to criterion, the performance of the high-drive group compared to the low-drive group was enhanced on the noncompetitive list and impaired on the competitive list. In an unpublished study in Weiss’ laboratory, we conditioned a learned drive by pairing a conditioned stimulus (CS) analog with the unconditioned stimulus (US) of disagreement. The CS analog acquired the ability to energize both correct and erroneous associations in the verbal paired-associates lists, with effects on errors similar to those reported earlier. Because all known learned drives are based on aversive primary drives (US) such as pain, frustration, and nausea (Brown & Farber, 1968, indispensable for its review of numerous unpublished attempts to condition appetitive drive; Mineka, 1975; Mongeluzi, Rosellini, Caldarone, Stock, & Abrahamsen, 1996, and its escape companion, Van Sommers, 1963), these results imply that disagreement-induced drive is aversive. Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subjects continuing to respond with fear-motivated escape behavior on the nonshock trials. An analog of intermittent shock was achieved by having the “other person” state a disagreeing opinion on shock trials only, offering no opinion on nonshock trials (Rule I-1, I-5 and I-6). Results too were analogous to conditioning: Participants disagreed with at the outset of only half the trials still performed the IR, but more slowly than did participants disagreed with at the outset of all

19-Schachtman-19.indd 424

trials (Weiss, Miller, Steigleder, & Denton, 1977; Weiss, Williams, & Miller, 1972; escape conditioning, Franchina, 1966, 1969a). The manipulation permits us to specify the locus of the drive in the disagreeing opinion. There is no fully corresponding manipulation in instrumental reward conditioning, not least because of the fear-motivated escape on nonshock trials, and the fact that learned drives are aversive (e.g., Brown & Farber, 1968; Mineka, 1975). Implications Logic of Modeling

In physics, theoretical analogy quite often turns out to be logically or empirically imperfect. Such disanalogy characterizes even remarkably successful and long-lived theories. Thus, (a) radio and light waves are theoretically analogous to the once better known (b) sound and water waves, except for the fact that sound and water waves require a medium in which to propagate, while radio and light waves travel unabashed through a vacuum. This was no quibble to theoretical physicists, who invented several ingenious varieties of “ether” (a couple of them very ethereal indeed) in which radio and light waves might propagate with propriety. The logic of modeling leads to extensive testing of the escape conditioning model of speaking in reply because of the problem of disanalogy. Indeed, despite some effort, we have never been able to devise a satisfactory analog of magnitude of reinforcement. The logic of modeling also leads us to conduct a program of experiments because analogy is not identity, and we cannot conclude from an acquisition curve that all delay of reinforcement effects are guaranteed in advance of experiment. Functional Signiﬁcance of Speaking in Reply

A deﬁnitive difference between the monologue of mass communications and the dialogue of personal conversation is the opportunity to speak in reply. This fundamental dichotomy is reﬂected in laboratory research. In the laboratory monologue of persuasive communication or impression formation messages are typically directed by a communicator to a research participant

2/16/2011 6:50:45 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

who has no overt opportunity to reply; in the laboratory dialogue of experimentation on group discussion or interviews, however, the participant has that opportunity. The study of the reply already enjoys a position of cardinal importance in dialogue research as a dependent variable. We thought it likely that if the opportunity to speak in reply were a fundamental aspect of interpersonal communication, then it might be illuminating and enjoyable to turn the problem on its head by investigating the effects of replying on the person who makes the reply. With the problem thus inverted, theory and experiment reveal a convincingly coherent and demandingly detailed blueprint of the reinforcing function of speaking in reply. Speaking in reply can now be seen to be a deﬁnitive difference between the monologue of mass communication and the dialogue of personal conversation that possesses an equally fundamental functional signiﬁcance. Altruism

Much human behavior exhibits altruistic features, most dramatically in emergencies, warfare, and social movements, where group loyalties often take precedence over individual needs. An abundance of experiments on altruism delineates the circumstances under which people will help others who are in need: The focus is on the antecedents of altruistic behavior rather than on its consequences for the altruist (e.g., Staub, 1978, 1979). Developmental and some social psychologists have asked how socially constructive or altruistic behavior can be learned, generalized, and, perhaps, internalized through extrinsic reinforcement, thus viewing altruism as a response (e.g., Grusec, 1991; Grusec & Redler, 1980; Staub, Bar-Tal, Karylowski, & Reykowski, 1984). We thought it interesting to invert the question, standing the problem on its head, the better to view altruism as a reinforcer and examine the reinforcing consequences of altruism. This perspective, we hoped, might also give us a chance of further illuminating the extraordinary strength and prevalence of altruism, some of it in places that social psychologists did not think to look, including warfare and revolution.

19-Schachtman-19.indd 425

425

General Method

To study the predicted action of altruistic reinforcement, we devised an artiﬁcial social structure (experimental apparatus and procedure) to the blueprint of instrumental conditioning. The participants were to learn, upon presentation of a cue (“report” signal), to make an instrumental response (button-push), the reinforcement for which was the deliverance of another human being from suffering (simulated shock and also bright light). The cue and IR were concealed in the masking task: Participants observed and evaluated a trained confederate who ostensibly received continuous painful electric shock while performing an aviation-related tracking task. A participant sat in a darkened room in front of an array of visual signals, knobs, and buttons deployed so as to focus the participant’s attention on a window into an adjacent lighted booth there to see another person (the confederate) wearing a “shock bracelet.” Inside the booth were “controls” for tracking, a preprogrammed radar-like screen, a blower fan, houselights sufﬁcient for visibility, and a ﬂood lamp placed 0.30 m from the confederate’s face. To enhance the verisimilitude of the shock manipulation, the confederate exhibited agonized expressions, “nervous” genuine sweating, and occasional verbal expressions of pain and “reﬂexive” kicking of the wall. Were it not for the booth’s wall, the participant and confederate could easily have touched one another. When the shock-on signal was illuminated, the participant observed the confederate’s tracking, and, upon presentation of a series of evaluation signals, set dials to evaluate the confederate’s performance (masking only). Upon completion, participants then received the “report” signal (cue) and pressed, in sequence, six “report buttons.” A latency timer automatically measured the time from the cue to the ﬁrst buttonpush (IR). Altruistic reinforcement immediately followed the sixth button-push: shock-offset ([a] “shock-on” signal offsets and [b] shock-off signal onsets), (c) the confederate breathes a sigh of relief with the receipt of a 10 s break from the shock, (d) the sweat-inducing ﬂood lamp offsets,

2/16/2011 6:50:45 PM

426

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and (e) the efﬁcient blower fan onsets, visibly drying the “nervous” sweat. Because the houselights remained on in a darkened room, the reinforcing events were clearly visible. Altruistic Reinforcement Analogs of Acquisition, Partial and Delay of Reinforcement

People will learn an instrumental response, the reward for which is the deliverance of another human being from suffering. Moreover, the functional relationships found using altruistic reinforcement were analogous to those governing the effects of conventional reinforcement in instrumental conditioning. Speeds of responses delivering the confederate from suffering gradually increased over the course of our analog of trials (Rule I-1 and I-12); steadily diverged from no-reinforcement controls over trials (Rule I-2); were faster when deliverance from suffering was immediate rather than delayed (Rule I-7) and continuous rather than partial (Rule I-3; Weiss, Boyer, Lombardo, & Stich, 1973; Weiss, Buchanan et al., 1971). There is a decreasing negatively accelerated delay gradient. The delay gradient is steep, probably indicating that the threshold value for effective reinforcement (e.g., Campbell, 1955; Tarpy, 1969) is quite high (Weiss, Cecil, & Frank, 1973). Magnitude of Reinforcement Analog

If altruism is reinforcing, then the degree to which the participant is able to alleviate the other person’s suffering should have the functional properties of magnitude of reinforcement. In escape conditioning, for any given level of drive, response speed is an increasing function of magnitude of reinforcement (Rule I-8; e.g., Bower, Fowler, & Trapold, 1959; Campbell & Kraeling, 1953; McAllister, McAllister, Brooks, & Goldman, 1972; Woods & Holland, 1966). Weiss, Boyer et al. (1973) manipulated magnitude of altruistic reinforcement using three levels of shock and ﬂood lamp stress reduction. The high and zero magnitudes of altruistic reinforcement were complete reduction of shock and ﬂood lamp stress (reinforcement) and no reduction (no reinforcement). The medium

19-Schachtman-19.indd 426

magnitude consisted of reducing the shock the confederate experienced but not terminating it altogether, and dimming the ﬂood lamp but not turning it off. Pilot studies made clear that manipulating a medium magnitude of altruistic reinforcement would require extraordinary care because, to participants incomplete reduction in shock, ﬂood lamp effects and confederate’s suffering tend to resemble zero magnitude. Two boundary conditions were revealed: (1) there is a high threshold value for effective reinforcement, so medium reinforcement must be substantial (e.g., Campbell, 1955; Tarpy, 1969); (2) the reduction in suffering must be made very clearly discriminable. To meet the boundary conditions, participants ﬁrst witnessed a “shock calibration procedure” wherein the confederate’s predetermined responses established a reduced shock level deﬁned as “uncomfortable but not painful.” During medium reinforcement the confederate acted in accordance with experiencing a reduced shock level that, albeit uncomfortable, was not painful, and a dimmer ﬂood lamp permitted a slower and sometime incomplete drying of the confederate’s sweat. Second, the standard shock-on and shock-off signals were, for the ﬁrst time, buttressed by auditory feedback. Rigorous attention to boundary conditions enabled us to carve nature at the joint. Figure 19.3 shows striking correspondences between magnitude of reinforcement in instrumental escape conditioning and magnitude of altruistic reinforcement (Weiss, Boyer et al., 1973). Altruistic Drive Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subject continuing to respond with fear-motivated escape behavior on the shock-free trials. An analog of such a manipulation in our altruism experiments involved omitting shock and ﬂood lamp stress to the confederate on some trials (Rule I-6; Weiss, Boyer et al., 1973). In escape conditioning, response speed is a decreasing function of the percentage of trials on which shock is omitted (Franchina, 1966, 1969a). By analogy, the speed

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES HIGH 130

Speed (100/Latency)

120 110 100

MEDIUM

confederate’s shock- and ﬂood lamp-induced suffering. The present results are clearly aversive, without carrying any necessary implication that all altruistic behavior is aversively motivated: The experimental situation itself fairly reeks of pain and stress. Implications Coign of Vantage

90 80

ZERO

70 60 50 1 2 3 4 5 6 7 8 9 10 11 12 13 2 3 4 5 6 7 8 9 10 11 12 13 14 Rolling Blocks of Two Trials

Figure 19.3 Magnitude of reinforcement analog:

Acquisition curves of response speed under different magnitudes of altruistic reinforcement. High = complete reduction of shock and ﬂood lamp stress; Medium = reduction of shock but not terminating it, and dimming of ﬂood lamp but not turning it off; Zero = no reduction of shock and ﬂood lamp stress. (Redrawn from Weiss, Boyer, Lombardo, & Stich, 1973).

of altruistically reinforced responses should be a decreasing function of the percentage of trials on which “shock” and ﬂood lamp stress is omitted. Intermittent shock effects in altruistic learning were analogous to those found in escape conditioning, with response speeds being faster when shock was “administered” to the confederate on all of the trials than it was when shock was given on only one-third of the trials. Because the 33% shock group received altruistic reinforcement on one-third of the trials, participants in this group, as expected, responded faster than no-reinforcement controls. The rate of learning under 33% “shock” suggests that responding on shock-free trials was motivated by secondary altruistic drive (learned on the shock trials) and reinforced by reduction of the secondary altruistic drive. Our analog of intermittent shock pinpoints the source of altruistic drive in the

19-Schachtman-19.indd 427

427

Since the early days of social and developmental psychology, much interest has been shown in the question of how socially constructive or altruistic behavior can be learned and maintained through extrinsic reinforcement. In sharp contrast, our research demonstrates that instrumental behavior can be learned and maintained through the reinforcing function of altruism. This inversion-in-fact stems, in part, from the initial inversion-in-theory when ﬁrst we proposed to stand the question on its head and examine altruism as a reinforcer rather than as a response. If this theoretical curiosity is likely to hold some charm for learning psychologists, then so too may some curious aspects of experimental apparatus and procedure. For clarity’s sake one of the two experimenters running a participant has always been called the confederate, but it can now be seen that we have an experimenter (the confederate) in the box (booth) and the subject outside the box! The subject is motivated by “shock” administered to an experimenter and reinforced by “offset of shock” to that experimenter. This is, as they say, “no accident, comrade.” It was part of the fun in designing apparatus and procedure to ﬁt the theory. Failures of Altruism

Precisely because altruism is ubiquitous in emergencies, warfare, and social movements, these events afford some of the most vivid and haunting instances of its failure. Just as defection is of major concern in the social psychology of social movements (e.g., Weiss, 1963), so “leaving the scene” is one of the principal problems addressed in the experimental study of altruism (e.g., Piliavin, Dovidio, Gaertner, & Clark, 1981). Emergency intervention and leaving the scene

2/16/2011 6:50:46 PM

428

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

are typically studied in situations in which people choose between those two responses, thus violating the boundary conditions of conditioning and discouraging any analog of forced trials in a T-maze. Moreover, experiments on leaving the scene focus on a single, crucial, episode rather than on a series of such episodes or “trials.” At a more molecular level, it is difﬁcult to see how one might control an analog of (say) delay of reinforcement on each trial/episode. But if we stand the problem on its head so that the scene leaves the participant, then the problem simpliﬁes nicely as instrumental conditioning. In the leaving-the-scene condition, the buttonpush caused the temporary disappearance of the still-suffering confederate behind an opaque and soundproof shutter, thereby permitting a series of discrete trials with delay, magnitude, and schedule of reinforcement controlled. Leaving the scene proved to be weakly, almost trivially, reinforcing, reaching asymptote on the second block of trials, ending up just a little above the controls and far below the altruistic reinforcement group (Stich, Weiss, Cramer, & Feinberg, 1987). Replication

The delay effect has been replicated and the fundamental effect of acquisition under altruistic reinforcement very well so. Altruistic drive and reinforcement effects apply to males and females in equal measure. This is a typical ﬁnding in our research (Weiss, Weiss, Wenninger, & Balling, 1981). There are doubtless gender differences in socialization, but we study underlying processes (drive, cue, reinforcement) while conduct is both more targeted by parents and sometimes more malleable (e.g., than primary drives). The discovery of altruistic rewards reveals a profound similarity between altruistic and conventional, nonaltruistic drives and reinforcers. And it had never previously been demonstrated that the roots of altruism are so deep that people not only help others but ﬁnd doing so rewarding.

major social sciences according these processes considerable interest. If, like the sociologist Georg Simmel (1955), we view competition as a form of social conﬂict, then there is a consequent increase in the breadth and vigor of interest. Competition and cooperation have typically been viewed by psychologists as mere dependent variables, as behaviors, and even in the hands of some developmentalists, as responses to be learned. For the third and last time in this set of ﬁve theories, we will invert the question to examine the consequences of competition for the competitor: Engaging in competition engenders aversive drive. Turning the question on its head yielded a unique perspective in the case of functions of speaking in reply and of altruistic rewards. Other psychologists have attributed motivational properties to competition (e.g., Church, 1962; Cronbach, 1963; Shaw, 1958). This research began with a complimentary pair of instrumental reward and escape conditioning models, nicely separated by a clear boundary condition, but only the escape model survived and prospered. This report is written in light of what we learned by affording full opportunity for the logic of modeling and ideas from sociology to realize their full potential. General Method

Participants played a competitive game of Labyrinth with one or more opponents, and then, upon presentation of a cue performed the IR, the reinforcement for which was the termination (or reduction) of competition. The experiments were described as studies of the effects of different scoring methods on competitive behavior. This masking provided the procedural ﬂexibility necessary for testing theoretically exciting predictions regarding competitive drive and reinforcement by manipulating (a) the N-opponents the participant initially competed against, and (b) the N-opponents the participant continued to compete against after performing the IR.

Competition

Constant N-Opponents

Competition and cooperation have captivated social theorists for decades, with each of the

The cue and IR were concealed in the masking task: One of two competitors, the data-generating

19-Schachtman-19.indd 428

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

participant, made the IR that provided a measure of response speed by ostensibly resetting the machine performing “real-time scoring and analysis.” Because of the machine’s “limited capacity,” point tallying, and therefore competition, could not occur during the reset period (reinforcement). Each competitor played the game in a separate, partitioned section of the laboratory in front of a control panel that included an array of visual signals, the manipulandum, and headphones. Verbal feedback, and especially visual feedback consisting of control panel lights and different colored lights (e.g., white, red, blue) mounted beneath milk glass and the translucent game board, made the critical segments of a competition trial easily discernable. Without needing to take their eyes off the game, participants could effortlessly distinguish between the (a) aversive competition-on period (shock-box), (b) cue, and (c) reinforcing competition-free period (goal-box). Each trial began with a 45 s competition-on period, following which the participant received the “reset/end-tallying” signal (cue) and performed the “reset/end-tallying” IR. A latency timer automatically measured the time from the cue to the IR which (a) lit the “machine reset-no scoring” lights on the control panel and below the game board indicating that scoring was no longer occurring, and (b) initiated a reinforcing 20 s competition-free, no-scoring period. During the competition-free, no-scoring period, competitors were instructed to continue practice-playing the game: Competition offset was not confounded with task offset. Following reinforcement the next competition trial began. Varying N-Opponents

As in the constant N-opponents experiments, the experimenter kept the participant informed of the number of opponents also playing the game during the initial competition-on period (e.g., 45 s). The number of people sharing the waiting room, the apparatus, instructions, and procedure, and the physical arrangement of the laboratory, were carefully crafted to create, for the participant, the “reality” of a competitive situation with multiple opponents. For example,

19-Schachtman-19.indd 429

429

the data-generating participant was led past several partitioned sections of the laboratory each outﬁtted with the game, headphones, and a control panel. The masking task provided a plausible rationale for reducing the N-opponents the participants continued to compete against following the IR. To test different scoring methods, a signaled shift from one method to another would be necessary during the cycle of operation. The data-generating participant, upon presentation of the “shift” signal (cue), pushed the “removebutton” (IR), thereby providing a measure of response speed and reducing the N-opponents. Following the IR, verbal feedback indicated that scoring methods had shifted, and if competition continued, how many opponents the participant would continue to compete against. If all of the opponents were removed, participants experienced a reinforcing 20 s competition-free, practice-play period. If fewer than all of the opponents were removed, a 20 s competition-on period followed with a reduced N-opponents. All of the participants had a programmed 13 s rest period separating the competition trials. Again, verbal feedback speciﬁed N-opponents before and after pushing the remove-button. The verbal feedback assisted control panel lights and colored lights mounted beneath milk glass and the translucent game board in making each critical segment of a competition trial easily discernable. Without needing to take their eyes off the game, the (a) aversive competition-on period (shock-box), (b) cue, (c) reinforcing reduction in the N-opponents in the goal-box, and (d) intertrial interval were readily apparent to the participants as (say) blue lights softly illuminated the maze from below. Constant N-Opponents Delay of Reinforcement Analog

If the offset of competition is analogous to reinforcement, then delaying that offset is analogous to delay of reinforcement (Rule I-7). Figure 19.4 shows an elegant experimental analog of a delay of reinforcement gradient (Steigleder et al., 1978), negatively accelerated in shape just like rat-generated curves (Rule I-12; Fowler &

2/16/2011 6:50:46 PM

430

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

analog in the duration of the competition-free period (Rule I-8). After the IR, the participants were free from competition either for 10, 20, or 30 s before the next cycle began (Steigleder et al., 1978). The competitive and competition-free periods were very thoroughly demarcated, in a manner charmingly similar to Franchina’s hurdle-box. Response speeds were a monotonic increasing function of the duration of the competition-free period, as the escape model predicts. The longer shock-free period permits more relaxation and more conditioning of approach to the cues of the competition-free period. The effects of duration of the competition-free period replicate nicely.

Speed (100/Latency)

80

70

60

50

40

0

1

3

5

Delay (s)

Figure 19.4 Delay

of reinforcement analog: Response speed as a function of delay of the reinforcing offset of competition. (Redrawn from Steigleder, Weiss, Cramer, & Feinberg, 1978).

Trapold, 1962; see also McAllister & McAllister, 1992; Tarpy & Koster, 1970). Partial Reinforcement Analog

If the termination of competition functions as reinforcement, then the termination of competition on only half the trials should have the functional properties of partial reinforcement (Rule I-3). Partial reinforcement facilitates response speed in reward conditioning (e.g., Amsel, 1992; Capaldi, 1978; Goodrich, 1959) but impairs it in escape conditioning (e.g., Bower, 1960; Gray, 1982; Woods et al., 1972). Experimental results ﬁt the escape pattern, indicating that engaging in competition engenders an aversive drive (Steigleder et al., 1978). Analog of Duration of the Shock-Free Period

A curious variable, well researched in avoidance conditioning under the guidance of relaxation theory (Denny, 1971, 1991) but also studied in escape conditioning as part of Franchina’s valuable effort to isolate the escape component of avoidance (Franchina, Kash, Reeder, & Sheets, 1978; Franchina & Schindele, 1975), the duration of the shock-free period ﬁnds its social

19-Schachtman-19.indd 430

Intermittent Shock Analog

In intermittent shock experiments, the shock is omitted on some trials, with the subject continuing to respond with fear-motivated escape behavior on the nonshock trials. We had our participants compete either on 33%, 66%, or 100% of the trials (Rule I-6) and found that their speed was a monotonic increasing function of the percentage of competitive trials (Steigleder, Weiss, Balling, Wenninger, & Lombardo, 1980). The acquisition curves paralleled the intermittent shock data of Franchina (1966) in almost uncanny detail and provide further evidence for the aversiveness of competitive drive. Competence as a Drive Variable

Fear of situational stimuli is a decreasing function of mastery of a two-way avoidance task (McAllister & McAllister, 1991; McAllister et al., 1983). Similarly, audience-induced drive is a decreasing function of competence in humans (Cottrell, Rittle & Wack, 1967; Cramer, McMaster, Bartell, & Dragna, 1988). If competition engenders an aversive drive, it then seems likely that competent participants would ﬁnd competition less aversive. The analogy is reasonable and interesting, if not strictly entailed by the escape conditioning model. The experiment proceeded in two phases: competence induction followed by escape conditioning. Competence induction was not confounded with the experience of success or failure (Luginbuhl, 1972). Both competent and

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

incompetent participants gradually acquired the IR over the course of trials. The two curves gradually diverge, with the competents reaching the lower asymptote and the incompetents the higher, in a classic Drive x Trials interaction. We have replicated the drive effects of competence (Steigleder et al., 1978). Learned Drive

If engaging in competition engenders aversive drive, then stimuli appropriately associated with this aversive drive (US) should themselves acquire the capacity to elicit a learned drive. This holds, whether the drive induced by competing is primary or secondary, since there is ample evidence for second-order conditioning of drives (e.g., Anderson, Johnson, & Kempton, 1969; Marlin, 1983; McAllister & McAllister, 1964; Rescorla, 1980). We would have preferred a nonsocial CS such as a light or tone, but since this CS must appear in both the competitive phase and the testing phase the requirements of plausibility led to using a person as the CS and, in consequence, our design was fully controlled for audience effects. Drive conditioned to a stimulus associated with competition energized performance of associations on the Spence, Farber, and McFann paired-associates lists (Spence et al., 1956; Steigleder et al., 1980). Because all known learned drives are based on aversive primary drives (US) such as pain, frustration, and nausea (e.g., Brown & Farber, 1968; Mineka, 1975), these results imply that the drive induced by competition is aversive. Aversive or Appetitive

Let us brieﬂy gather the evidence now, the better to next focus on the strange phenomena and intriguing implications encountered when we vary the N-opponents. We will not need to rehearse the conditioning references already given at the individual variables and at “the functions of speaking in reply.” Asymptotic partial reinforcement curves followed the escape (continuous faster than partial) rather than reward (partial faster than continuous) pattern. Excellent intermittent shock (intermittent competition) results were obtained, there being no satisfying

19-Schachtman-19.indd 431

431

analog of this in reward conditioning especially because of the implied learned drive. We did not settle for an implied learned drive but conducted a separate experiment in which we conditioned an analog of a learned drive. All known learned drives are based on aversive primary drives (or on aversive learned drives via second-order conditioning). Those experiments provide the deﬁnitive evidence, but we have in addition the experiments on competence and the analog of the duration of the shock-free period. Competent participants had slower speeds but continued to escape from competition. We actually considered the possibility that a competence/incompetence manipulation might be a boundary condition separating a reward conditioning model from the escape conditioning model, but this potentially beautiful development did not come to pass. The predicted effect of duration of the competition-free period does not directly imply the aversiveness of competition-induced drive, but, given the evidence already provided, this analog has something of an amplifying effect, with our participants relaxing in the “shock-free goal-box.” In reaching a clear-headed conclusion, it is well to consider not only the experimental results but also (to some extent) the atmosphere of the experimental situation which, for the altruism research, led us to limit our conclusions because the experimental situation “fairly reeks of pain and stress.” We obviously do not have such a problem here, where the experimental task is the game Labyrinth, which is manufactured commercially and played recreationally and where participants do not experience failure. The drive induced by competition is aversive. Varying N-Opponents

The variable “number of people” is frequently manipulated when social processes such as social judgment (Knowles, 1983), collective action and social movements (Macy, 1990; Marwell & Oliver, 1993), diffusion of social impact (Latané & Bourgeois, 1996; Seta & Seta, 1996), family inﬂuence (Zajonc, 1976; Zajonc & Mullally, 1997), cognitive dissonance (Festinger, 1957), and social facilitation (Crisson, Seta, & Seta,

2/16/2011 6:50:46 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

1995; Weiss & Miller, 1971; Zajonc, 1965) are investigated. Social facilitation research suggests that number of people can function as a drive variable (Geen & Gange, 1977; Martens & Landers, 1972; Weiss & Miller, 1971). We take N-opponents as an analog of drive-intensity when participants compete as individuals, without alliances or power blocs. N-opponents, like number of trials, is a discrete variable that can mimic a continuous variable when N is reasonably large. In conditioning research, drive is usually manipulated as a continuous variable (in volts, milliamps, hours-of-deprivation, etc.) but is also certainly manipulated (e.g., shock + noise) as an inherently discrete variable (e.g., Campbell, 1968; McAllister & McAllister, 1992). Drive Intensity Analog

In escape conditioning, speed is an increasing function of drive when drive is terminated after each escape (e.g., Nation, Wrather, & Mellgren, 1974: Trapold & Fowler, 1960). Analogously, when participants compete as individuals and the IR terminates competition, the theory predicts that speed is an increasing function of the N-opponents (Rule I-11). There were two levels of drive: 1 versus 3 opponents. A more inclusive description of the two groups compares 1 opponent, reduced by 1 as reinforcement, = 0 opponents remaining in the goal-box (1 – 1 = 0) with 3 opponents, reduced by 3 as reinforcement, = 0 opponents remaining in the goal-box (3 – 3 = 0). High drive was indeed faster than low (Steigleder et al., 1980). At the inception of the second series of competition studies, we thought it essential to replicate the drive effect and to test out a second complete two-competitor apparatus system. Drive intensity effects replicate nicely as shown in Figure 19.5. Drive effects also replicate cleanly across the two apparatus systems, despite the fact that the competitors were close together in one system and widely separated for the other. (All data from the second series of competition experiments are as yet unpublished studies from Weiss’ laboratory.) Drive Downshifts

There were three drive conditions: a constant high drive, 3-opponent control (3 – 3 = 0), a constant

19-Schachtman-19.indd 432

80 3–3=0

Speed (100/Latency)

432

70

60 1–1=0

50

40

1

2

3

4

5

6

7

Trials

Figure 19.5 Shock-intensity

(Drive) analog: Acquisition curves of response speed as a function of N-opponents. Conditions: 1 – 1 = 0 (1 opponent, reduced by 1 as reinforcement, = 0 opponents remaining in the goal box) and 3 – 3 = 0 (3 opponents, reduced by 3 as reinforcement, = 0 opponents remaining in the goal box).

low drive, 1-opponent (1 – 1 = 0) control and a high-to-low drive downshift (3 – 3 = 0)/(1 – 1 = 0) condition. The high-drive controls were faster than the low-drive controls. The speeds of the downshift participants rapidly downshifted, meeting the speed of the low-drive controls within two trials, continuing down to show a negative contrast effect as in the escape conditioning model (Nation et al., 1974; Woods & Schutz, 1965). Campbell and Kraeling Analog

Campbell and Kraeling (1953) showed that when the amount of reinforcing shock reduction is held constant, speed is a decreasing function of shock intensity. Equivalent results have been shown with different drive manipulations (e.g., McAllister & McAllister, 1992; Myers, 1969). McAllister and McAllister do not directly address this question, but their virtuoso experimental manipulations produced this effect in several different ways and their data may be easily

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

regraphed to show it. McAllister and McAllister (personal communication) agree with this analysis. So now we need another analogy that has been present in the equation labeling all along. Magnitude of reinforcement is an increasing function of the reduction in the N-opponents. The most perspicuous way to present this Campbell and Kraeling experiment is via Figure 19.6. Reinforcement is held constant at reduction of two opponents: The remove-button removes two opponents. Drive is once more varied by N-opponents, 2, 3, or 4. The ﬁgure shows a clear Campbell and Kraeling effect with speed as a decreasing function of N-opponents (drive). Notice the crucial number after the equals sign,

90 2–2=0

Speed (100/Latency)

80

3–2=1

70

433

the 2-opponent group leaves no opponents waiting for the participant in the goal-box after the participant pushes the remove-button but the 4-opponent participants must face two opponents remaining in the goal-box after they make their instrumental response! If we recall the continuity between competition and social conﬂict in the sociology of Georg Simmel (1955), then this is not a happy outcome. Even in his brief treatment of games Simmel uses such words as ﬁght, victory, and kampf (which word in the very different sociology of Marx is commonly rendered as “struggle”). Acquisition Trials

The ﬁve acquisition curves in Figures 19.5 and 19.6 show gradual acquisition over the course of trials. Three of the curves show full reinforcement with no opponents remaining in the goal-box, while two others do have opponents remaining in the goal-box after the IR has been made. There are some 27 more acquisition curves not depicted here, but these should be sufﬁcient to convey the pattern and some variations. Magnitude of Reinforcement Analog

60 4–2=2 50

40

30

1 2 3

2 3 4

3 4 5

4 5 6

5 6 7

6 7 8

7 8 9

8 9 10

Rolling Blocks of Three Trials

Figure 19.6 Campbell

and Kraeling analog: “Paradoxical” drive effects when increasing initial N-opponents and holding the reinforcing reduction in N-opponents constant. Conditions: 2 – 2 = 0 (2 opponents, reduced by 2 as reinforcement, = 0 opponents remaining in the goal box), 3 – 2 = 1 (3 opponents, reduced by 2 as reinforcement, = 1 opponent remaining in the goal box), and 4 – 2 = 2 (4 opponents, reduced by 2 as reinforcement, = 2 opponents remaining in the goal box).

19-Schachtman-19.indd 433

Response speed is a negatively accelerated increasing function of magnitude of reinforcement, the amount of reinforcing reduction of aversive drive (e.g., Bower et al., 1959; McAllister et al., 1972; Woods & Holland, 1966). Since decreasing the N-opponents decreases drive intensity, the speed of an instrumental response that reduces the N-opponents is a negatively accelerated increasing function of the reinforcing reduction in the N-opponents (Rule I-8). After learning how to handle the variable, we essayed a 5-group parametric. Participants competed with an initial number of ﬁve opponents. Pushing the remove-button reduced the N-opponents by 1, 2, 3, 4, or 5 opponents. A description of just three cells will elucidate the experiment from a different perspective: “high” (5 – 5 = 0), 5 opponents, reduced by 5 as reinforcement, = 0 opponents remaining in the goalbox; “medium” (5 – 3 = 2), 5 opponents, reduced by 3 as reinforcement, = 2 opponents lurking in the goal-box; “low” (5 – 1 = 4), 5 opponents, reduced by 1 as reinforcement, = 4 opponents

2/16/2011 6:50:46 PM

434

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Speed (100/Latency)

85 80 75 70 65 60 1

2 3 4 5 Reduction in N-Opponents

Figure 19.7 Magnitude of reinforcement analog:

Asymptotic speed as a function of reduction (R) in N-opponents. Speed = M(1 – 10 -iR) + d, where M is the asymptotic speed of the magnitude of reinforcement gradient, i is the negatively accelerated rate of change, d is the intercept, and R is the reduction in N-opponents, as ﬁtted to individualparticipant mean speeds.

waiting in the goal-box. It was with a sense of wonder that we ﬁrst saw the asymptotic magnitude of reinforcement results graphed in Figure 19.7. The asymptotic response speed of competing human beings is a negatively accelerated increasing function of the reinforcing reduction in the N-opponents. Number of People and Models

This analogical theory does not merely translate the language of competition into the language of instrumental conditioning. Rather, the systematic use of an escape conditioning model generated predictions about competitive behavior likely to be overlooked by social psychologists. As in a variety of research areas in social psychology, it would stand to reason just to manipulate the variable “number of people.” But using escape conditioning as our model, we were able to ascertain that the variable “number of people” will have three different effects, depending upon where in the cycle of operation it is manipulated. Decreasing the variable by reducing the N-opponents (reinforcement) contingent

19-Schachtman-19.indd 434

upon the escape response will improve performance. “Paradoxically,” decreasing the initial N-opponents (drive intensity), whether between-groups or even within-subjects (from one trial to the next as in drive-downshifts), will impair performance provided that there is complete offset of competition as reinforcement. In sharp contrast, increasing the initial N-opponents (between groups) while keeping reinforcement constant will actually impair performance in a Campbell and Kraeling effect. The theory tells us where in the cycle of operation to manipulate “number of people” and to anticipate multiplicative effects in ways that did not “stand to reason” to social psychologists lacking this theoretical tool. Implications Galileo, Harvey, and Hobbes: A Nondigression

Hobbes is surprisingly comfortable reading for scientists like us, because he had an intimate knowledge of the best science of his day and used it in his own work. Hobbes acquired this knowledge not only from reading but also from his close friend Harvey, who discovered the circulation of the blood and who had been a student of Galileo at the University of Padua. All this is nicely brought out, and deepened, in a useful little book in the philosophy of science by Randall (1961). Hobbes tried to apply the theoretical method of Galileo, as it was taught to Harvey by Galileo and several other professors at the University of Padua, to society. Galileo analyzed the motions of the heavenly bodies into purely theoretical “elements” (intervening variables) in a purely theoretical space, and then, via appropriate composition rules recombined the theoretical elements so as to predict the actual motions of actual bodies in actual space. Hobbes in turn analyzed society into purely theoretical equal individuals (e.g., stripped of civilized restraint) in a theoretical space, which he called the State of Nature. “The fact that the State of Nature is a logical and not an historical hypothesis is generally understood” (Macpherson, 1962, p. 21). Hobbes then recombines these purely theoretical individuals via appropriate, but not

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

quantitative, composition rules to explain society empirically and normatively. Hobbes conceives humans as self-maintaining engines (we would say robots, automata) in motion, regardless of whether they are in or out of the State of Nature. The importance of motion is an insight carried over from Galileo (“nevertheless it moves”) and Harvey (the blood circulates) and Hobbes identiﬁes it with life itself. We too carry over such scientiﬁc insights, as when Hull captured the hard-won learning-performance distinction in the form of Drive x Habit, and when the Rescorla-Wagner equation appears in one form or another, not only in learning and social learning-analog theories, but even in cognitive theories. The War of All Against All

There are, of course, no peace ofﬁcers or any other protections peculiar to organized society in the State of Nature “wherein we suppose contention between men by nature equal, and able to destroy one another” (Hobbes, 1640/1984, chap. 14, No. 12, p. 73), but “warre of every man against every man” (Hobbes, 1651/1950, chap. 13, p. 105). Although we are testing our own theory, not Hobbes’s, it can now be seen that in our experiments in which N-opponents is varied, we have a laboratory analog of a war of all against all. An outstanding feature of the laboratory war is the elimination of opponents, also an outstanding feature of Hobbes’s State of Nature. “It may seem strange to some man, that has not well weighed these things; that Nature should thus disassociate, and render men apt to invade, and destroy one an other” (Hobbes, 1651/1950, chap. 13, p. 104). “From this equality of ability, ariseth equality of hope in the attaining of our Ends. And therefore if any two men desire the same thing, which neverthelesse they cannot both enjoy, they become enemies; and in the way to their End,… endeavor to destroy, or subdue one an other” (Hobbes, 1651/1950, chap. 13, p. 102). Hobbes certainly does not propose that competition induces aversive drive, but he places great emphasis upon it, even in his later works. Competition can be seen or inferred in some of the preceding quotes, but it is certainly available elsewhere. “Competition of

19-Schachtman-19.indd 435

435

Riches, Honour, and Command or other power, enclineth to Contention, Enmity and Warre; Because the way of one Competitor, to the attaining of his desire, is to kill, subdue, supplant, or repel the other” (Hobbes, 1651/1950, chap. 11, p. 80). “But the most frequent reason why men desire to hurt each other, ariseth hence, that many men at the same time have an Appetite to the same thing; which yet very often they can neither enjoy in common, nor yet divide it; whence it followes that the strongest must have it, and who is strongest must be decided by the Sword” (Hobbes, 1647/1983, chap. 1, No. 6, p. 46). Hobbes does not assume that men are innately competitive, but competition follows ineluctably from the social environment of the State of Nature (Gauthier, 1969, pp. 14-20, 208). We have found the juxtaposition of Hobbes to our competition theory to be provocative as well as intriguing, and we intend, in the future, to offer a learning-theoretical exit from the war of all against all in place of the social contract.

PAVLOVIAN CONDITIONING ANALOGS Analogs of Pavlovian conditioning principles and procedures have been extended to the attribution of liking (Cramer, Helzer, & Mone, 1986), persuasive communication (Weiss, 1968), interpersonal attraction (Cramer, Weiss, Steigleder, & Balling, 1985), and causal relationship detection (Cramer, Weiss, William, Reid, Nieri et al., 2002). Only the last two social processes will be discussed here because the use of compound social cues and Rescorla-Wagner theory yields research that is socially rich and surprising. Rules of Correspondence (Pavlovian Conditioning)

General rules of correspondence relating Pavlovian conditioning variables to social process variables are provided in Table 19.3. In our interpersonal attraction research, attitudinal agreements were assumed to elicit approach responses (person-directed actions), whereas in our causal relationship detection research

2/16/2011 6:50:46 PM

436

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Table 19.3 Rules of Correspondence Relating Pavlovian Conditioning Variables and Social Process Variables Rule

Pavlovian Conditioning (Model)

Social Process (Analog)

Fundamental Concepts P-1

Conditioned stimulus (CS; A, B, or X)

Discriminable social stimulus (a person)

P-2

Unconditioned stimulus (US)

Social stimulus reliably eliciting a response (e.g., attitudinal agreement in attraction)

P-3

Unconditioned response (UR)

Response elicited by US analog (e.g., approach in attraction)

P-4

Conditioned response (CR)

Conditioned form of UR analog

Variables P-5

N Forward trials (CS-US or A+)

N CS analog-US analog trials

P-6

N Nonreinforced/Extinction trials (A−)

N CS analog without US analog trials following series of P-5 trials

P-7

N US alone trials

N US analog without CS analog trials

P-8

N Backward trials (US-CS)

N US analog-CS analog trials

P-9

Compound CS (AX, BX)

Compound stimulus containing two or more social stimuli (two or more people)

P-10

N Forward compound CS trials (AX-US or AX+)

N Compound CS analog-US analog trials

P-11

N Backward compound CS trials (US-AX)

N US analog-compound CS analog trials

P-12

N Nonreinforced compound CS trials (BX−)

N Compound CS analog without US analog trials

P-13

CS intensity

CS analog intensity

P-14

US intensity

US analog power to elicit response (e.g., agreement strength)

P-15

Positive CS/US contingency

p(US analog/CS analog) > p(US analog/no CS analog)

P-16

Zero CS/US contingency

p(US analog/CS analog) = p(US analog/no CS analog)

P-17

CR speed

CR analog speed

P-18

CR strength/amplitude

CR analog strength/amplitude

an event or outcome such as production information (US analog) was assumed to elicit causality-seeking actions (UR analog). The power of agreements to elicit approach responses in attraction and the power of production information (e.g., level of production) to elicit

19-Schachtman-19.indd 436

causality-seeking actions are social analogs of US intensity. Again, the general rules, numbered here for later reference, do not exhaust the possibilities, but they are described in sufﬁcient detail to support our empirical work on interpersonal attraction and causal relationship detection.

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Interpersonal Attraction

In research guided by the neo-Hullian theory of Rescorla and Wagner (1972; Wagner & Rescorla, 1972) and by close analogs of Pavlovian compound-cue conditioning principles and procedures, we tested novel predictions regarding context effects in interpersonal attraction when a participant encountered more than one agreeable person (Cramer et al., 1985). Attitudinal agreements are reinforcing social stimuli (Byrne, 1971; Lombardo, Libkuman et al., 1972; Lombardo, Weiss, & Buchanan, 1972) that elicit person-directed actions such as approach responses or “striving for” behaviors (Domjan, Lyons, North, & Bruell, 1986; Ganesan & Pearce, 1988; Hearst & Jenkins, 1974; O’Connell & Rashotte, 1982; Staats, 1975). General Method

A social stimulus (a person or persons/CS analog) was repeatedly paired with social reinforcement (attitudinal agreement/US analog), and the participant’s attraction (approach response speed/CR analog) to the social stimulus measured across trials. From the participant’s perspective, the experiments, masked as studies of opinion change, involved a continuous cycle of opinion, discussion, feedback, and opinion change. After a group of people listened to and then brieﬂy discussed the participant’s opinion on a controversial topic, either one spokesperson or two spokespersons reported that a majority of the group members agreed with the opinion. The registration of opinion change immediately followed. Ostensibly, the “other people” in the study were in another room, and all communication would take place via an intercom. The “other people” were, in fact, simulated by tape recordings. For clarity of exposition we now use the theoretical labels A and X to describe the apparatus and procedure. The instructions and signals used colors (e.g., blue and orange) to refer to Person A and Person X in fully counterbalanced designs. A+ Attraction Conditioning

Conceptually, each single-stimulus (A+) cycle consisted of (1) a forward conditioning trial

19-Schachtman-19.indd 437

437

pairing a CS analog (Person A) with a US analog (agreement with the participant’s opinion), and (2) a nonreinforced CSA test trial. At the beginning of an A+ cycle, the experimenter gave the participant the topic to be discussed, followed by illumination of the “press switch to open intercom to Person A” (nonreinforced CSA test trial) signal. The attraction dependent variable was the speed of a response opening a line of communication to Person A by throwing a toggle switch toward a light that stood for Person A. The instructions to the participants, the apparatus conﬁguration, and the procedure maximized from the ﬁrst the occurrence of the to-be-learned approach response and minimized the occurrence of competing responses. After the participants gave their opinion, and following a brief group discussion, the “reporting” signal was illuminated while Person A reported that a majority of group members agreed with the opinion (CS-US acquisition trial). Participants then performed the masking task by registering their opinion change, and the apparatus automatically reset to begin a new cycle. AX+ Attraction Conditioning

Each compound-stimulus (AX+) cycle included (1) a forward conditioning trial pairing a compound CS analog (Person A and Person X) jointly with a US analog (agreement with the participant’s opinion), (2) a nonreinforced CSX test trial, and (3) a nonreinforced CSA test trial. The AX+ cycle differed from the A+ cycle in two ways. First, approach response speeds were measured on two nonreinforced test trials, CSX, “press switch to open intercom to Person X,” followed by CSA. Throwing a switch toward lights that stood for Person A or Person X opened a line of communication to Person A or to Person X. Again, the instructions, the apparatus, and the procedure maximized from the ﬁrst the occurrence of the to-be-learned approach response and minimized the occurrence of competing responses. Hence, the participants threw a single switch opening lines of communication to Person A and to Person X. Second, Person A and Person X were identiﬁed as spokespersons for a group of people listening

2/16/2011 6:50:46 PM

438

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

to and brieﬂy discussing the participant’s opinion. A “reporting” signal was illuminated while Person A and Person X in compound jointly reported that a majority of group members agreed with the opinion expressed. Having two spokespersons (AX), or as on the A+ cycles a single spokesperson (A), report that a majority of group members agreed with the participant served to control an analog of US intensity by having the feedback be a “single agreement” on each trial. Because agreement comes from the group, the number of people agreeing is not confounded with the number of social stimuli. Participants then performed the masking task by registering their opinion change, and the apparatus automatically reset to begin a new cycle. Acquisition and Blocking Analogs

Among the most frequently investigated stimulus selection or context problems in conditioning is the blocking effect. Reinforcing a novel target stimulus in the context of a stimulus paired separately with the US attenuates or blocks response acquisition to the target stimulus (e.g., Barnet, Grahame, & Miller, 1993; Kamin, 1968, 1969; Kremer, 1978; Wagner, 1969). Acquisition and blocking effects in interpersonal attraction were investigated using an analog of an interspersedtrials procedure (Wagner, 1969) to mix A+ and AX+ cycles of operation in the blocking group; an acquisition group received only AX+ cycles (Rules of Correspondence P-5 and P-10). Approach response speed to Person A (Rule P-17) was an increasing function of the number of A+ trials, with the reinforcing effects of agreement being less effective as the attractiveness of Person A increased. Acquisition effects to Person X were observed in the blocking and acquisition groups, but despite Person X’s consistent objective relationship with agreement in both groups, speed to Person X was attenuated in the blocking group (Cramer et al., 1985). Three additional experiments employing analogs of a discretephases conditioning procedure (Kamin, 1968, 1969) and modest variations in method replicated the acquisition and blocking effects (Cramer et al., 1985; Weiss, McDonald, Little, & Shull, 1991).

19-Schachtman-19.indd 438

Inhibition Analogs

Retardation of attraction acquisition and summation of inhibition of attraction were investigated in unpublished research in Weiss’ laboratory using an analog of a familiar procedure for producing conditioned inhibition, backward conditioning (Pavlov, 1927; Siegel & Domjan, 1974; Williams & Overmier, 1988; Zbroznya, 1958). These social analogs of retardation and summation (Pavlov, 1927; Rescorla, 1969) were investigated using a common procedure where testing is omitted in the initial phases of conditioning and is reserved for the ﬁnal phase of conditioning. Our standard cycle of operation was modiﬁed so that participants received a series of testless backward compound conditioning trials. During the training phase, spokespersons A and X in compound jointly delivered the group’s agreement with the participant’s previously expressed opinion, with the reporting signal indicating the two persons speaking lit only after the spokespersons delivered the group’s agreement (Rule P-11). Unlike our standard cycle of operation, participants did not open a line of communication to the spokespersons during the training phase. In a retardation of attraction acquisition study, two groups of participants received either zero or two backward compound conditioning trials in an initial training phase. The training phase was followed by six standard AX+ cycles of operation, with the participant’s approach response speeds to Person A and Person X measured separately (Rule P-10 and P-17). A third group of participants received six backward compound conditioning trials followed by six forward compound trials, with speed measured to the compound social stimulus containing persons A and X. Figure 19.8 shows increasing approach speeds across the AX+ trials. The combined speeds to the single inhibitors were slower for the group receiving two backward trials compared to zero backward trials in the initial phase of the experiment. Moreover, the effect of inhibition of attraction on the retardation of attraction acquisition was most pronounced when, following six backward compound trials, approach speeds to a double inhibitor social stimulus were measured.

2/16/2011 6:50:46 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

439

Zero Backward 90

90

Single Inhibitor 80

70

Two Backward

60 50 Six Backward Double Inhibitor

40

70 60 Double Inhibitor

50 40 30

30 20

Speed (100/Latency)

Speed (100/Latency)

80

20 1 2

2 3

3 4

4 5

5 6

Rolling Blocks of Two Trials

Figure 19.8 Inhibition of attraction: Retardation

of the acquisition of approach response speed to persons A and X as a function of number of backward conditioning trials to those people in the previous phase. In the Zero Backward and Two Backward conditions, speed was measured to Person A and Person X separately; in the Six Backward/Double Inhibitor condition, speed was measured to persons A and X in compound.

1 2

2 3

3 4

4 5

5 6

Rolling Blocks of Two Trials

Figure 19.9 Summation of two inhibitors of

attraction: Retardation test shows two inhibitors to be more potent than one. In the Single Inhibitor condition, approach response speed was measured to Person A and Person X separately; in the Double Inhibitor condition, speed was measured to persons A and X in compound.

Super-Conditioning Analog

Figure 19.9 shows an analog of summation of inhibition of attraction. Following six backward compound conditioning trials participants received six single inhibitor or double inhibitor nonreinforced test trials (Rule P-6 and P-12). In phase 2 approach speeds were measured either to a single inhibitor social stimulus, Person A and Person X separately, or to a double inhibitor stimulus, persons A and X in compound. The combined speeds in the single inhibitor test condition were faster than in the double inhibitor test condition. Despite the consistent objective relationship of Person A and Person X to social reinforcement, participants were more attracted to the spokespersons when they were tested as single inhibitors than when they were tested as a double inhibitor, twoperson group.

19-Schachtman-19.indd 439

Blocking effects in attraction occur when a target social stimulus, Person X, is paired with agreement in the context of another person participants ﬁnd attractive, Person A. By extending Pavlovian conditioning principles further, we observed, in unpublished research in Weiss’ laboratory, an effect in opposition to blocking, attraction super-conditioning. In animal learning, when a stimulus compound containing an inhibitory stimulus and a novel stimulus are reinforced, increased responding or superconditioning to the target stimulus was observed (e.g., Navarro, Hallam, Matzel, & Miller, 1989). By analogy, the use of backward conditioning in the ﬁrst phase of the experiment (Rule P-8) results in Person A becoming a conditioned inhibitor of attraction, which then results in super-conditioning of attraction to Person X in a second, compound stimulus conditioning phase

2/16/2011 6:50:46 PM

440

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

60 FIVE

Speed (100/Latency)

50 TWO

40

30

ZERO

20

10

0

1 2

2 3

3 4

4 5

Rolling Blocks of Two Trials

Figure 19.10 Super-conditioning of attraction:

Acquisition of approach response speed to Person X as a function of number of backward conditioning trials to Person A.

(Rule P-10). Figure 19.10 shows approach speeds to Person X were an increasing function of the number of ﬁrst phase, backward conditioning trials (0, 2, or 5) to Person A. The stronger the conditioned inhibition of attraction to Person A, the stronger the participant’s attraction to Person X when persons A and X jointly agreed. Implications Losses in Attraction Strength

Attitudinal agreements, like conventional reinforcers, do not always result in greater attraction. In fact, as an agreeable person’s attractiveness increases, an agreement become less and less effective as reinforcement, and provided agreement magnitude remains constant, attraction, like conventional conditioned responses, eventually reaches asymptote. Moreover, agreements may actually result in “losses in attraction strength.” It is now well supported in conditioning research and theory, that when highly conditioned stimuli are reinforced in compound, the response strength elicited by each stimulus

19-Schachtman-19.indd 440

is reduced (overexpectation effect; Ganesan & Pearce, 1988; Kremer, 1978; Lattal & Nakajima, 1998; Levitan, 1975; Rescorla, 1999; Rescorla & Wagner, 1972). Losses in response strength occur because the combined strength of the compounded stimuli exceeds the asymptote of conditioning supportable by the magnitude of reinforcement (Rescorla & Wagner, 1972). By analogy, if Person A and Person X are equally attractive and their combined attractiveness exceeds the asymptote supportable by the magnitude of agreement (Rule P-14), having persons A and X, in compound, jointly agree with the participant decreases by an equal amount their individual attraction strengths. If, on the other hand, Person A is liked more than Person X, and their combined attractiveness exceeds the asymptote supported by the magnitude of agreement, attraction to each person will be reduced, but with the attraction strength of Person A remaining higher than that of Person X. Consistently agreeable people, despite their objective relationship to social reinforcement, may not only fail to elicit attraction, but may actually contribute to a reduction in their attractiveness and the attractiveness of others. Social and Nonsocial Stimulus Interactions

The contexts in which attraction often develops are not composed only of individuals, but include nonsocial stimuli such as physical objects and conditions, setting, and ambience. Should any of these nonsocial stimuli be attractive, they will, in theory, block the acquisition of attraction among people who agree with one another. One strategy for avoiding attraction blocking is to reduce the salience of any competing nonsocial stimuli (Rule P-13). Reinforcing someone, hoping to engender attraction, therefore, is likely to be more effective in an environment with affectively neutral nonsocial stimuli than in one where the nonsocial stimuli are attractive. Dare we speculate that the development of interpersonal relationships will be facilitated in the sterile “stainless steel” environments often envisioned by futurists and science ﬁction writers? The familiar phrase “no one is more fanatical than a convert” succinctly summarizes a social analog of super-conditioning when a defector

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

from one social movement joins another (Weiss, 1963). Because few conversions are complete when the defection takes place, the new social movement provides the defector with numerous social rewards in the context of other social and nonsocial stimuli likely to continue inhibiting the defector’s attraction to the new movement. In theory, the defector’s experience, compared to that of a “raw recruit,” results in fanatical or super-conditioned attraction and devotion to the new movement. Attraction in Context

Attraction is no mere matter of repeated reinforcement. Our observations of blocking, retardation, summation, and super-conditioning analogs amply demonstrate the fundamental role social context plays in the acquisition of attraction. Further ampliﬁcation of this vital role is provided in the novel predictions of losses in attraction strength, and of blocking and superconditioning effects when social and nonsocial stimuli interact. The blocking and super-conditioning predictions are particularly instructive because they suggest that the effect of a social stimulus (person) becomes subject to a radical transformation by the action of nonsocial stimuli. Moreover, the role of nonsocial, situational stimuli becomes more interesting when they occur in compound with social stimuli. Our illustrative examples of attraction in context, either reported or predicted here, are outside the boundary conditions of simple contiguity models of attraction like the reinforcement-affect theory (e.g., Byrne, 1971; Clore & Byrne, 1974). Social psychologists should take note that these limitations do not constitute a “disproof” of Byrne’s law of attraction, one of the most robust results in social psychology. Human Agency and Causal Relationship Detection

Because experiments in selective learning normally begin with response tendencies of equal strength, associative accounts of causal relationship detection among nonsocial events and outcomes begin without a bias among the research participants for one putative cause over another

19-Schachtman-19.indd 441

441

(e.g., Dickinson & Shanks, 1985; Dickinson, Shanks, & Evenden, 1984; Shanks, 1991; Shanks & Dickinson, 1987). What social psychologists have long recognized is that when event-outcome relationships include the possibility of human agency, causal relationship detection is commonly affected by the correspondence bias (Gilbert & Malone, 1995; Jones, 1979; Jones & Harris, 1967), the general tendency to view people as particularly powerful “agents” or “at cause” for the outcomes they experience. In research guided once again by the theory of Rescorla and Wagner (1972; Wagner & Rescorla, 1972) and by close analogs of Pavlovian compound-cue conditioning principles and procedures, we investigated the implications for an associative account of causal relationship detection involving social stimuli when a possibility exists that a participant’s trial-by-trial sensitivity to the procedures is compromised, and a social stimulus evokes the target response prior to training (Cramer et al., 2002). General Method

Participants played the role of a production supervisor evaluating the effectiveness of parttime workers in a ﬁctional company. A computerized evaluation system presented graphic material representing a worker (event) and the company’s month-end level of production (outcome) across several reporting periods, and enabled participants to rate a worker’s causal relationship to production. For clarity of exposition, the theoretical labels A and X are used to describe the procedure. In the experiments, however, the instructions and computer labels used proper names when referring to the workers. A+ Conditioning

From the participant’s perspective, the experiment proceeded as a continuous cycle of evaluation. Conceptually, however, a single-worker evaluation cycle (A+) consisted of a forward conditioning trial pairing a CS analog (Worker A) and a US analog (company month-end level of production). In a social analog of delay conditioning, Worker A was displayed alone for 5 s before the worker was paired with graphic

2/16/2011 6:50:47 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

production information for an additional 10 s. Following the conditioning component of the cycle, participants rated the strength of the causal relationship (CR) of the worker to production using a 100-point scale anchored by 0 = Totally Ineffective and 100 = Totally Effective. AX+ Conditioning

A compound stimulus or multiple worker evaluation cycle (AX+) consisted of a forward conditioning trial on which two workers, Worker A and Worker X, were jointly paired with the company’s month-end production level. The compound CS analog was displayed for 5 s before the compound CS analog was paired with the US analog for an additional 10 s. Following the compound conditioning component of the cycle, participants rated the CR strength of a target worker to production, either Worker A or Worker X. Methodological Study

Our expectation that social stimuli are viewed as more “at cause” than nonsocial stimuli was conﬁrmed by participants asked to rank order three plausible causes for company productivity. In a brief survey, a worker was judged as more effective than either a production quota or quality control standards in causing company productivity. These results, and other related ﬁndings conﬁrming the agency of social stimuli (i.e., agency of a student and an athlete for examination and sport team performance, respectively), were consistent with the possibility that an associative account of causal relationship detection involving social stimuli may be constrained by the correspondence bias. CS/US Contingency Analogs

Despite possessing a priori agency, the acquisition of CR strength of a worker to production was determined by repeatedly pairing a worker with production information, and the contingency relationship between the worker and production. Figure 19.11 shows that the CR strength of a worker to production gradually increased across trials when a positive contingency (Rescorla, 1967) existed between the worker and production information (Rules

19-Schachtman-19.indd 442

85 CR Strength of Worker to Production

442

GROUP + 80 75 70 65

GROUP 0

60 55 50

1

2

3 4 5 6 7 Acquisition Trials

8

9

Figure 19.11 Human

agency conforms to Pavlovian CS/US contingency. CR = causal relationship; Group + = positive contingency; Group 0 = zero contingency. (Redrawn from Cramer, Weiss, William, Reid, Nieri et al., 2002).

of Correspondence P-15 and P-18). No reliable increase in CR strength was observed when a zero contingency existed between the worker and production information (Rule P-16). US Intensity Analog

Because ample evidence indicates that response strength increases as US intensity increases (e.g., Hoehler & Leonard, 1981; O’Connell & Rashotte, 1982; Prokasy, Grant, & Myers, 1958; Spence, 1956), we anticipated that workers paired with different production levels (Rule P-14; effect intensity = low, medium, and high) would not be uniformly evaluated as “effective” or “at cause.” Participants, as expected, were sensitive to the level of production paired with a worker, with the CR strength of the worker to production being an increasing function of US analog intensity. Blocking Analog

Blocking effects have been reported in studies of causal and contingency judgments involving nonsocial stimuli (e.g., Aitken, Larkin, & Dickinson, 2000; Kruschke & Blair, 2000; Shanks, 1985;

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Van Hamme, Kao, & Wasserman, 1993). And Van Overwalle and Van Rooy (1998, 2001) demonstrated that the delta rule of connectionist theory (McClelland & Rumelhart, 1988) reliably accounts for blocking effects using social stimuli. Blocking effects in causal relationship detection involving social stimuli were observed by manipulating, for the ﬁrst time, two close analogs of Pavlovian conditioning (Cramer et al., 2002). An analog of an interspersed-trials procedure (Wagner, 1969) mixed A+ and AX+ trials in a blocking group; an acquisition group received only AX+ trials (Rule P-5 and P-10). Figure 19.12 shows that the CR strength of Worker X to production measured on reinforced X+ test trials was lower in the blocking group than in the acquisition control group. Blocking effects replicated using an analog of discrete-phases conditioning (Fig. 19.12; Kamin, 1968, 1969). These results indicated that causal relationship learning involving social stimuli was no mere matter of repetition, or even reinforced repetition.

443

Rather, CR blocking occurred despite the participants’ a priori beliefs that workers are likely to be responsible or “at cause” for production, and despite Worker X’s consistent objective relationship to production. Super-Conditioning Analog

Aitken et al. (2000) observed super-conditioning in causality detection, manipulating an analog of the conditional procedure for creating a conditioned inhibitor (Pavlov, 1927; Rescorla, 1979) in the context of a food-allergic reaction prediction task. Super-conditioning of a causal relationship involving social stimuli was obtained in unpublished research in Cramer’s laboratory using close analogs of the conditional procedure. Participants in the experimental group received, in turn, ﬁve A+, AB–, and BX+ conditioning trials followed by ﬁve reinforced X+ test trials (Rule P-5, P-10, and P-12); the controls received only 10 reinforced X+ trials. Figure 19.13 shows two effects of particular interest to causal relationship researchers: a social analog of inhibition and of super-conditioning. Based on our laboratory

85

80 CR Strength of Worker to Production

CR Strength of Worker X to Production

85

75

70

65

60 Blocking (Interspersedtrials)

Blocking (Discretephases)

75

65

55

Acquisition

Experimental Groups

45 A

B X (Experimental)

X (Control)

Workers

Figure 19.12 Blocking of a causal relationship (CR)

in two kinds of experienced situations (discrete phases and interspersed trials) despite worker agency.

19-Schachtman-19.indd 443

Figure 19.13 Super-conditioning of causal rela-

tionship (CR) strength of Worker X to production via an analog of an A+/AB−/BX+ paradigm.

2/16/2011 6:50:47 PM

444

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

experience in measuring CRs and on the Group 0 ﬁndings reported in Figure 19.11, the subnormal CR strength of Worker B (Fig. 19.13) is indicative of an inhibition analog. Worker B’s CR strength is well below that of the zero contingency group on the initial trial and on the last two trials. Consequently, Figure 19.13 also shows that pairing workers B and X with production information on a series of reinforced compound trials in the BX+ phase resulted in super-conditioning of CR strength of Worker X to production. Implications Super-Conditioning and the Augmenting Effect

We join the connectionists Van Overwalle and Van Rooy (1998, 2001) in recognizing that when the events preceding an outcome include both excitatory/facilitatory and inhibitory stimuli, learning and social psychologists alike predict super-conditioning or augmenting, respectively. The explanation for augmenting preferred by social researchers, however, relies on the attributer’s causal schemata and logical inferences (e.g., Kelley, 1972a, 1972b) rather than on the operation of mechanical learning principles. Moreover, social researchers typically study augmenting in described situations (e.g., Hansen & Hall, 1985) rather than in experienced situations. Interestingly, a more complete understanding of causal agency is possible by studying experienced situations, where dynamic variables like the frequency of event-outcome pairings and the order of events and outcomes play a powerful role. For example, when a forward conditioning procedure is used to pair Worker A with production information, subsequent reinforcement of workers A and X in compound results in blocking of CR strength to Worker X. In sharp contrast, if production information is paired with Worker A using an analog of backward conditioning (Rule P-8), Worker A, in theory, becomes a conditioned inhibitor (e.g., Pavlov, 1927; Williams & Overmier, 1988), and we predict that subsequent reinforcement of workers A and X in compound results in augmenting or super-conditioning of CR strength to

19-Schachtman-19.indd 444

Worker X. Merely manipulating the order in which a worker is paired with production information provides a demonstration of that variable’s fundamental role in determining the strength of a supervisor’s evaluation of a worker’s causal agency. Multiplicity of Discounting

Worker A and Worker X are individually paired with production information until their individual CR strengths reach asymptote. What happens to their individual CR strengths if the two workers are subsequently paired in compound with the same analog of US intensity? Because the sum of the individual CR strengths is greater than the strength supportable by the US analog (Rule P-14), the CR strengths of Worker A and Worker X are predicted to decrease despite the workers’ continuing objective relation to production (e.g., Ganesan & Pearce, 1988; Kremer, 1978; Lattal & Nakajima, 1998; Levitan, 1975; Rescorla, 1999; Rescorla & Wagner, 1972). The anticipated losses in CR strength in conditioning are analogous to discounting in social psychology where, according to attribution theory, the mere presence of multiple plausible causes for an effect leads one “logically” to discount the agency of each cause (e.g., Jones & Davis, 1965; Kelley, 1972a, 1972b). The logic of modeling does not provide a challenge to the traditional explanations of discounting by merely translating the language of attribution into the language of conditioning. Rather, the logic of modeling provides for the speciﬁcation of a multiplicity of discounting rendered theoretically determinate by Pavlovian principles. What we mean by a multiplicity of discounting can be exempliﬁed by different types of unequal discounting, each with its own underlying causal mechanism. Consider an experiment in which the CR strengths of Worker A and Worker X are unequal (A > X) prior to being paired in compound with production information, and in which their combined CR strength is greater than the US analog can support. Pairing workers A and X in compound with production information is predicted to generate equal trialby-trial losses in the CR strength of each worker,

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

with the ﬁnal result being Worker A still stronger than Worker X (unequal discounting). Now turn to a different experiment in which the CS intensity of Worker A is greater than that of Worker X and in which their individual CR strengths are at asymptote before compound conditioning, the losses in CR strength will be related to their intensities (e.g., brightness or vividness of clothing color and pattern; Rule P-13). Pairing Worker A and Worker X in compound with production information is predicted to generate trial-by-trial losses in the CR strength of each worker, but with the more salient worker, A, losing more CR strength than the less salient worker, X (unequal discounting). Our term inverse overshadowing nicely captures the predicted effect in discounting and in conditioning (see Kamin & Gaioni, 1974; Kremer, 1978). When the underlying causal mechanism involves differences in the workers’ associative strengths, continued reinforcement of the workers in compound generates equal trial-bytrial losses in the CR strength of each worker and eventuates in a stronger Worker A than Worker X (unequal discounting). In sharp contrast, when the causal mechanism involves differences in the workers’ intensity, continued reinforcement generates unequal trial-by-trial losses in the CR strength of each worker, with both the trial-by-trial and ﬁnal loss in CR strength being greater for the “more intense” worker than the “less intense” worker (unequal discounting). These predictions of unequal discounting, which are guided by underlying causal mechanisms that determine where in the learning process unequal discounting is to be anticipated, provide powerful testimony indeed for the logic of modeling in social research. Because associative theory does not confuse the phenomenon of discounting with its underlying causal mechanisms, an Aristotelian error that Lewin (1935) long ago warned against, it effortlessly accounts for the richness and interest of a multiplicity of discounting.

SUMMARY The social psychological experiments reported in this chapter were informed and guided by

19-Schachtman-19.indd 445

445

research with roots in the work of Thorndike and Pavlov, and in the Hull-Miller-Spence tradition. Close analogs of instrumental escape and Pavlovian compound-cue conditioning were devised and manipulated to predict and observe a compelling number of correspondences between the learning models and ﬁve domains of social behavior. For convenience, Table 19.4 summarizes the correspondences (with particular emphasis on their replication) observed between variables in instrumental conditioning and in interpersonal communication, altruism, and competition, and between variables in Pavlovian conditioning and in attraction and causal relationship detection. The logic of modeling led to experimentally delineated analogical portraits of various social processes that are well ﬂeshed out and can be exquisite in their completeness. The beauty of the experimental delineation of each social process also comes, in no small part, from their origin as theoretical predictions: The experimental facts do ﬁt together to make theoretically coherent portraits. By continuing to construct artiﬁcial social structures to the blueprints of learningtheoretical models, the understanding of social behavior is enriched in ways both distinctive and fascinating.

THE EXCELLENCE OF CONDITIONING RESEARCH As scientists, we are accustomed to test the truth of a hypothesis by its consequences in experiment and to evaluate the goodness of the knowledge thus obtained by the degree to which these results are determinate, by p-values, by elegance of curves, by replicability, and so on. Replicability imposes an additional test of consequences, but there is a powerful further test of consequences about which we seldom speak: It is the ability of knowledge to serve as a tool in further inquiry (see Dewey, 1938). Such a tool may be used narrowly, as when Judson Brown’s ingenious startle-reﬂex technique is used once again as an index of conditioned fear. Broadly, a wellorganized body of knowledge acquires the ability to be used by analogy to predict a different domain. The special excellence of inquiry in

2/16/2011 6:50:47 PM

446

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Table 19.4 N-Experiments in Which Social Analogs of Instrumental and Pavlovian Conditioning Were Obtained Instrumental Analogs

Social Domains Communication

Altruism

16

6

12

Delay of reinforcement

3

2

1

Delay of reinforcement shifts

1

Partial reinforcement

2

1

1

1

1

Acquisition

Magnitude of reinforcement Duration of shock-free period

2

Correlated reinforcement

2

Correlated delay of reinforcement

1

Extinction

1

Extinction (partial reinforcement)

1

Drive intensity Campbell and Kraeling effects

Competition

3 1

1

Drive downshifts

1

Drive energization

2

1

Learned drive

1

1

Competence (drive variable) Intermittent shock Pavlovian Analogs Acquisition

2 2

1

1

Attraction

Human Agency

7

5

CS/US contingency

1

US intensity

1

Blocking (interspersed-trials)

1

1

Blocking (discrete-phases)

3

1

Forward conditioning

6

5

Backward conditioning

3

Acquisition of inhibition

3

Retardation test of inhibition

1

Summation of two inhibitors

1

Super-conditioning via backward conditioning

1

Super-conditioning via conditional procedure

19-Schachtman-19.indd 446

1

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

conditioning generates knowledge of such versatility that we could use it as a model to predict ﬁve different domains of social behavior. Nor was only one kind of conditioning able to serve as a model. In vivid contrast to the vast body of social psychology, we have been able to predict curve shapes. How was this possible? Because the conditioning models include the analogs of those curve shapes, and they often led us to continuous variables that varied on physical continua and on ratio scales or to approximations thereof. The scope widens to include “a war of all against all,” complete with the elimination of opponents, and still the conditioning model predicts. When we make a particularly satisfying discovery, we thank the rat runners, those excellent scientists who make our discoveries possible.

REFERENCES Aitken, M. R. F., Larkin, M. J. W., & Dickinson, A. (2000). Super-learning of causal judgments. Quarterly Journal of Experimental Psychology, 53B, 59–81. Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge, England: Cambridge University Press. Anderson, D. C., Johnson, D., & Kempton, H. (1969). Second-order fear conditioning as revealed through augmentation of a startle response: Part II. Psychonomic Science, 16, 7–9. Barnet, R. C., Grahame, N. J., & Miller, R. R. (1993). Temporal encoding as a determinant of blocking. Journal of Experimental Psychology: Animal Behavior Processes, 19, 327–341. Bower, G. H. (1960). Partial and correlated reward in escape conditioning. Journal of Experimental Psychology, 59, 126–130. Bower, G. H., Fowler, H., & Trapold, M. A. (1959). Escape learning as a function of amount of shock reduction. Journal of Experimental Psychology, 58, 482–484. Brown, J. S., & Farber, I. E. (1968). Secondary motivational systems. In P. R. Farnsworth (Ed.), Annual review of psychology (Vol. 19, pp. 99–134). Palo Alto, CA: Annual Reviews, Inc. Byrne, D. (1971). The attraction paradigm. New York, NY: Academic Press. Campbell, B. A. (1955). The fractional reduction in noxious stimulation required to produce “just noticeable” learning. Journal of Comparative and Physiological Psychology, 48, 141–148.

19-Schachtman-19.indd 447

447

Campbell, B. A. (1968). Interaction of aversive stimuli: Summation or inhibition? Journal of Experimental Psychology, 78, 181–190. Campbell, B. A., & Kraeling, D. (1953). Response strength as a function of drive level and amount of drive reduction. Journal of Experimental Psychology, 45, 97–101. Capaldi, E. J. (1978). Effects of schedule and delay of reinforcement on acquisition speed. Animal Learning & Behavior, 6, 330–334. Castaneda, A. (1965). The paired-associates method in the study of conﬂict. In L. P. Lipsitt & C. C. Spiker (Eds.), Advances in child development (Vol. 2, pp. 1–18). New York, NY: Academic Press. Castaneda, A., & Lipsitt, L. P. (1959). Relation of stress and differential position habits to performance in motor learning. Journal of Experimental Psychology, 57, 25–30. Church, R. M. (1962). The effects of competition on reaction time and palmar skin conductance. Journal of Abnormal and Social Psychology, 65, 32–40. Clore, G. L., & Byrne, D. (1974). A reinforcementaffect model of attraction. In T. L. Huston (Ed.), Foundations of interpersonal attraction (pp. 143– 170). New York, NY: Academic Press. Cottrell, N. B. (1968). Performance in the presence of other human beings: Mere presence, audience and afﬁliation effects. In E. C. Simmel, R. A. Hoppe, & G. A. Milton (Eds.), Social facilitation and imitative behavior (pp. 91–110). Boston, MA: Allyn & Bacon. Cottrell, N. B., Rittle, R. H., & Wack, D. L. (1967). The presence of an audience and list type (competitional or noncompetitional) as joint determinants of performance in paired-associates learning. Journal of Personality, 35, 425–434. Cramer, R. E., Helzer, K., & Mone, R. (1986). Variations in the conditions of reinforcement and the attribution of liking. Journal of General Psychology, 113, 341–349. Cramer, R. E., Lutz, D. J., Bartell, P. A., Dragna, M., & Helzer, K. (1989). Motivating and reinforcing functions of the male sex role: Social analogues of partial reinforcement, delay of reinforcement, and intermittent shock. Sex Roles, 20, 551–573. Cramer, R. E., McMaster, M. R., Bartell, P. A., & Dragna, M. (1988). Subject competence and minimization of the bystander effect. Journal of Applied Social Psychology, 18, 1133–1148. Cramer, R. E., Weiss, R. F., Steigleder, M. K., & Balling, S. S. (1985). Attraction in context: Acquisition and blocking of person-directed

2/16/2011 6:50:47 PM

448

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

action. Journal of Personality and Social Psychology, 49, 1221–1230. Cramer, R. E., Weiss, R. F., William, R., Reid, S., Nieri, L., & Manning-Ryan, B. (2002). Human agency and associative learning: Pavlovian principles govern social process in causal relationship detection. Quarterly Journal of Experimental Psychology, 55B, 241–266. Crisson, J. E., Seta, J. J., & Seta, C. E. (1995). The inﬂuence of expectations on task performance in audience and solitary settings. Basic and Applied Social Psychology, 17, 357–370. Cronbach, L. J. (1963). Educational psychology. New York, NY: Harcourt, Brace & World. Denny, M. R. (1971). Relaxation theory and experiments. In F. R. Brush (Ed.), Aversive conditioning and learning (pp. 235–295). New York, NY: Academic Press. Denny, M. R. (1991). Relaxation/relief: The effect of removing, postponing, or terminating aversive stimuli. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 199–230). Hillsdale, NJ: Erlbaum. Dewey, J. (1938). Logic: The theory of inquiry. New York, NY: Holt. Dickinson, A., & Shanks, D. R. (1985). Animal conditioning and human causality judgment. In L. G. Nilsson & T. Archer (Eds.), Perspectives on learning and memory (pp. 167–191). Hillsdale, NJ: Erlbaum. Dickinson, A., Shanks, D. R., & Evenden, J. L. (1984). Judgment of act-outcome contingency: The role of selective attribution. Quarterly Journal of Experimental Psychology, 36A, 29–50. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy. New York, NY: McGraw-Hill. Domjan, M., Lyons, R., North, N. C., & Bruell, J. (1986). Sexual Pavlovian conditioned approach behavior in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 413–421. Feinberg, R. A., Miller, F. G., Weiss, R. F., Steigleder, M. K., & Lombardo, J. P. (1982). Motivational aspects of learned helplessness. Journal of General Psychology, 106, 273–311. Festinger, L. (1957). A theory of cognitive dissonance. Evanston, IL: Row, Peterson. Fowler, H., & Trapold, M. A. (1962). Escape performance as a function of delay of reinforcement. Journal of Experimental Psychology, 63, 464–467. Franchina, J. J. (1966). Effects of shock schedules on the acquisition and extinction of escape behavior. Psychonomic Science, 4, 277–278.

19-Schachtman-19.indd 448

Franchina, J. J. (1969a). Intertrial intervals and shock schedules in escape conditioning. Journal of Comparative and Physiological Psychology, 67, 504–509. Franchina, J. J. (1969b). Escape behavior and shock intensity: Within-subject versus between-groups comparisons. Journal of Comparative and Physiological Psychology, 69, 241–245. Franchina, J. J., Kash, J. S., Reeder, J. R., & Sheets, C. T. (1978). Effects of exteroceptive feedback and safe-box conﬁnement durations on escape behavior in rats. Animal Learning & Behavior, 6, 423–428. Franchina, J. J., & Schindele, T. E. (1975). Nonshock conﬁnement duration and shock- and safebox similarity during escape training in rats. Animal Learning & Behavior, 3, 297–300. Ganesan, R., & Pearce, J. M. (1988). Interactions between conditioned stimuli for food and water in the rat. Quarterly Journal of Experimental Psychology, 40B, 229–242. Gauthier, D. P. (1969). The logic of Leviathan. Oxford, England: Clarendon Press. Geen, R. G., & Gange, J. J. (1977). Drive theory of social facilitation: Twelve years of theory and research. Psychological Bulletin, 84, 1267–1288. Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38. Goodrich, K. P. (1959). Performance in different segments of an instrumental response chain as a function of reinforcement schedule. Journal of Experimental Psychology, 57, 57–63. Gray, J. A. (1982). The neuropsychology of anxiety: An inquiry into the functions of the septohippocampal system. London, England: Oxford University Press. Grusec, J. E. (1991). The socialization of altruism. In M. S. Clark (Ed.), Review of personality and social psychology (Vol. 12, pp. 9–33). Newbury Park, CA: Sage. Grusec, J. E., & Redler, E. (1980). Attribution, reinforcement, and altruism: A developmental analysis. Developmental Psychology, 16, 525–534. Hansen, R. D., & Hall, C. A. (1985). Discounting and augmenting facilitative and inhibitory forces: The winner takes almost all. Journal of Personality and Social Psychology, 49, 1482–1493. Hearst, E., & Jenkins, H. M. (1974). Sign-tracking: The stimulus-reinforcer relation and directedaction. Austin, TX: The Psychonomic Society.

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Hesse, M. (1966). Models and analogies in science. Notre Dame, IN: University of Notre Dame Press. Hesse, M. (1974). The structure of scientiﬁc inference. Berkeley, CA: University of California Press. Hesse, M. (1980). Revolutions and reconstructions in the philosophy of science. Bloomington, IN: Indiana University Press. Hobbes, T. (1640/1984). The elements of law: Natural and politic (F. Tönnies, Ed., 2nd ed.). London, England: Cass. Hobbes, T. (1647/1983). De Cive. Oxford, England: Clarendon Press. Hobbes, T. (1651/1950). Leviathan. New York, NY: Dutton. Hoehler, F. K., & Leonard, D. W. (1981). Motivational vs. associative role of the US in classical conditioning of the rabbit’s nictitating membrane response. Animal Learning & Behavior, 9, 239–244. Hull, C. L. (1943). Principles of behavior. New York, NY: Appleton-Century-Crofts. Jones, E. E. (1979). The rocky road from acts to dispositions. American Psychologist, 34, 107–117. Jones, E. E., & Davis, K. E. (1965). From acts to dispositions: The attribution process in person perception. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 7, pp. 219– 266). New York, NY: Academic Press. Jones, E. E., & Harris, V. A. (1967). The attribution of attitudes. Journal of Experimental Social Psychology, 3, 1–24. Kamin, L. J. (1968). “Attention-like” processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior, 1967: Aversive stimulation (pp. 9–31). Coral Gables, FL: University of Miami Press. Kamin, L. J. (1969). Predictability, surprise, attention, and conditioning. In B. A. Campbell & R. M. Church (Eds.), Punishment and aversive behavior (pp. 279–296). New York, NY: Appleton-Century-Crofts. Kamin, L. J., & Gaioni, S. J. (1974). Compound conditioned emotional response conditioning with differentially salient elements in rats. Journal of Comparative and Physiological Psychology, 87, 591–597. Kelley, H. H. (1972a). Attribution in social interaction. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 1–26). Morristown, NJ: General Learning Press.

19-Schachtman-19.indd 449

449

Kelley, H. H. (1972b). Causal schemata and the attribution process. In E. E. Jones, D. E. Kanouse, H. H. Kelley, R. E. Nisbett, S. Valins, & B. Weiner (Eds.), Attribution: Perceiving the causes of behavior (pp. 151–174). Morristown, NJ: General Learning Press. Knowles, S. S. (1983). Social physics and the effects of others: Tests of the effects of audience size and distance on social judgments and behavior. Journal of Personality and Social Psychology, 45, 1263–1279. Kremer, E. F. (1978). The Rescorla-Wagner model: Losses in associative strength in compound conditioned stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 4, 22–36. Kruschke, J. K., & Blair, N. J. (2000). Blocking and backward blocking involve learned inattention. Psychonomic Bulletin & Review, 7, 636–645. Latané, B., & Bourgeois, M. J. (1996). Experimental evidence of dynamic social impact: The emergence of subcultures in electronic groups. Journal of Communication, 46, 35–47. Lattal, K. M., & Nakajima, S. (1998). Overexpectation in appetitive Pavlovian and instrumental conditioning. Animal Learning & Behavior, 26, 351–360. Levitan, L. (1975). Tests of the Rescorla-Wagner model of Pavlovian conditioning. Bulletin of the Psychonomic Society, 6, 265–268. Lewin, K. (1935). A dynamic theory of personality. New York, NY: McGraw-Hill. Logan, F. A. (1959). The Hull-Spence approach. In S. Koch (Ed.), Psychology: A study of a science (Vol. 2, pp. 293–358). New York, NY: McGraw-Hill. Logan, F. A. (1960). Incentive. New Haven, CT: Yale University Press. Logan, F. A. (1961). Discrete-trials DRL. Journal of the Experimental Analysis of Behavior, 4, 277–279. Lombardo, J. P., Libkuman, T. M., & Weiss, R. F. (1972). The energizing effects of disagreementinduced drive. Journal of Experimental Research in Personality, 6, 133–141. Lombardo, J. P., Weiss, R. F., & Buchanan, W. (1972). Reinforcing and attracting functions of yielding. Journal of Personality and Social Psychology, 21, 359–368. Luginbuhl, J. E. (1972). Role of choice and outcome on feelings of success and estimates of ability. Journal of Personality and Social Psychology, 22, 121–127. Macpherson, C. B. (1962). The political philosophy of possessive individualism: Hobbes to Locke. Oxford, England: Clarendon Press.

2/16/2011 6:50:47 PM

450

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Macy, M. W. (1990). Learning theory and the logic of critical mass. American Sociological Review, 55, 809–826. Mandler, G., & Watson, D. L. (1966). Anxiety and the interruption of behavior. In C. D. Spielberger (Ed.), Anxiety and behavior (pp. 263–288). New York, NY: Academic Press. Marlin, N. A. (1983). Second-order conditioning using contextual stimuli as S1. Learning & Behavior, 11, 290–294. Martens, R., & Landers, D. M. (1972). Evaluation potential as a determinant of coaction effects. Journal of Experimental Social Psychology, 8, 347–359. Marwell, G., & Oliver, P. (1993). The critical mass in collective action. Cambridge, England: Cambridge University Press. Masterman, M. (1980). Braithwaite and Kuhn: Analogy-clusters within and without hypothetico-deductive systems in science. In D. H. Mellor (Ed.), Science, belief, and behavior (pp. 61–86). Cambridge, England: Cambridge University Press. McAllister, D. E., & McAllister, W. R. (1964). Second-order conditioning of fear. Psychonomic Science, 1, 383–384. McAllister, D. E., & McAllister, W. R. (1991). Fear theory and aversively motivated behavior: Some controversial issues. In M. R. Denny (Ed.), Fear, avoidance, and phobias: A fundamental analysis (pp. 135–163). Hillsdale, NJ: Erlbaum. McAllister, D. E., McAllister, W. R., Brooks, C. I., & Goldman, J. A. (1972). Magnitude and shift of reward in instrumental aversive learning in rats. Journal of Comparative and Physiological Psychology, 80, 490–501. McAllister, W. R., & McAllister, D. E. (1992). Fear determines the effectiveness of a feedback stimulus in aversively motivated instrumental learning. Learning and Motivation, 23, 99–115. McAllister, W. R., McAllister, D. E., & Benton, M. M. (1983). Measurement of fear of the conditioned stimulus and of situational cues at several stages of two-way avoidance learning. Learning and Motivation, 14, 92–106. McClelland, J. M., & Rumelhart, D. E. (1988). Explorations in parallel distributed processing: A handbook of models, programs and exercises. Cambridge, MA: Bradford. Miller, N. E. (1959). Liberalization of basic S-R concepts: Extensions to conﬂict behavior, motivation and social learning. In S. Koch (Ed.),

19-Schachtman-19.indd 450

Psychology: A study of a science (Vol. 2, pp. 196–292). New York, NY: McGraw-Hill. Miller, N. E., & Dollard, J. (1941). Social learning and imitation. New Haven, CT: Yale University Press. Mineka, S. (1975). Some new perspectives on conditioned hunger. Journal of Experimental Psychology: Animal Behavior Processes, 104, 134–148. Mongeluzi, D. L., Rosellini, R. A., Caldarone, B. J., Stock, H. S., & Abrahamsen, G. C. (1996). Pavlovian aversive context conditioning using carbon dioxide as the unconditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 22, 244–257. Mowrer, O. H., & Viek, P. (1948). An experimental analogue of fear from a sense of helplessness. Journal of Abnormal and Social Psychology, 43, 193–200. Myers, A. K. (1969). Effects of continuous loud noise during instrumental shock-escape conditioning. Journal of Comparative and Physiological Psychology, 68, 617–622. Nation, J. R., Wrather, D. M., & Mellgren, R. L. (1974). Contrast effects in escape conditioning of rats. Journal of Comparative and Physiological Psychology, 86, 69–73. Navarro, J. I., Hallam, S. C., Matzel, L. D., & Miller, R. R. (1989). Superconditioning and overshadowing. Learning and Motivation, 20, 130–152. O’Connell, J. M., & Rashotte, M. E. (1982). Reinforcement magnitude effects in ﬁrst- and second-order conditioning of directed action. Learning and Motivation, 13, 1–25. Oppenheimer, R. (1956). Analogy in science. American Psychologist, 11, 127–135. Pavlov, I. (1927). Conditioned reﬂexes. Oxford, England: Oxford University Press. Piliavin, J. A., Dovidio, J. F., Gaertner, S. L., & Clark, R. D., III. (1981). Emergency intervention. New York, NY: Academic Press. Prokasy, W. F., Jr., Grant, D. A., & Myers, N. A. (1958). Eyelid conditioning as a function of unconditioned stimulus intensity and intertrial interval. Journal of Experimental Psychology, 55, 242–246. Randall, J. H. (1961). The school of Padua and the emergence of modern science. Padova, Italy: Editrice Antenore. Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological Review, 74, 71–80.

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

Rescorla, R. A. (1969). Conditioned inhibition of fear resulting from negative CS-US contingencies. Journal of Comparative and Physiological Psychology, 67, 504–509. Rescorla, R. A. (1979). Conditioned inhibition and extinction. In A. Dickinson & R. A. Boakes (Eds.), Mechanisms of learning and motivation: A memorial volume to Jerzy Konorski (pp. 83–110). Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1980). Pavlovian second-order conditioning. Hillsdale, NJ: Erlbaum. Rescorla, R. A. (1999). Summation and overexpectation with qualitatively different outcomes. Animal Learning & Behavior, 27, 50–62. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: AppletonCentury-Crofts. Schmeck, R. R. (1970). Error-produced frustration as a factor inﬂuencing the probability of occurrence of further errors. Journal of Experimental Psychology, 86, 153–156. Seta, C. E., & Seta, J. J. (1996). When more is less: An averaging/summation analysis of social anxiety. Journal of Research in Personality, 30, 496–509. Seybert, J. A., & Weiss, R. F. (1974). The negative reinforcing functions of nonconformity. Memory and Cognition, 2, 791–795. Shanks, D. R. (1985). Forward and backward blocking in human contingency judgment. Quarterly Journal of Experimental Psychology, 37B, 1–21. Shanks, D. R. (1991). On similarities between causal judgments in experienced and described situations. Psychological Science, 2, 341–350. Shanks, D. R., & Dickinson, A. (1987). Associative accounts of causality judgments. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 21, pp. 229–261). San Diego, CA: Academic Press. Shaw, M. E. (1958). Some motivational factors in cooperation and competition. Journal of Personality, 26, 155–169. Siegel, S., & Domjan, M. (1974). The inhibitory effect of backward conditioning as a function of the number of pairings. Bulletin of the Psychonomic Society, 4, 122–124. Simmel, G. (1955). Conﬂict and the web of group afﬁliations. Glencoe, IL: The Free Press.

19-Schachtman-19.indd 451

451

Spence, J. T., & Spence, K. W. (1966). The motivational components of manifest anxiety: Drive and drive stimuli. In C. D. Spielberger (Ed.), Anxiety and behavior (pp. 291–326). New York, NY: Academic Press. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press. Spence, K. W., Farber, I. E., & McFann, H. H. (1956). The relation of anxiety (drive) level to performance in competitional and noncompetitional paired-associates learning. Journal of Experimental Psychology, 52, 296–305. Staats, A. W. (1975). Social behaviorism. Homewood, IL: Dorsey Press. Staub, E. (1978). Positive social behavior and morality: Social and personal inﬂuences (Vol. 1). New York, NY: Academic Press. Staub, E. (1979). Positive social behavior and morality: Socialization and development (Vol. 2). New York, NY: Academic Press. Staub, E., Bar-Tal, D., Karylowski, J., & Reykowski, J. (1984). Development and maintenance of prosocial behavior. New York, NY: Plenum Press. Steigleder, M. K., Weiss, R. F., Balling, S. S., Wenninger, V. L., & Lombardo, J. P. (1980). Drivelike motivational properties of competitive behavior. Journal of Personality and Social Psychology, 38, 93–104. Steigleder, M. K., Weiss, R. F., Cramer, R. E., & Feinberg, R. A. (1978). Motivating and reinforcing functions of competitive behavior. Journal of Personality and Social Psychology, 36, 1291–1301. Stich, M. H., Weiss, R. F., Cramer, R. E., & Feinberg, R. A. (1987). Reinforcing functions of altruism and leaving the scene. The Journal of Psychology, 121, 459–473. Tarpy, R. M. (1969). Reinforcement difference limen (RDL) for delay in shock escape. Journal of Experimental Psychology, 79, 116–121. Tarpy, R. M., & Koster, E. D. (1970). Stimulus facilitation of delayed-reward learning in the rat. Journal of Comparative and Physiological Psychology, 71, 147–151. Tarpy, R. M., & Sawabini, F. L. (1974). Reinforcement delay: A selective review of the last decade. Psychological Bulletin, 81, 984–997. Trapold, M. A., & Fowler, H. (1960). Instrumental escape performance as a function of the intensity of noxious stimulation. Journal of Experimental Psychology, 60, 323–326. Van Hamme, L. J., Kao, S. F., & Wasserman, E. A. (1993). Judging interevent relations: From cause

2/16/2011 6:50:47 PM

452

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

to effect and from effect to cause. Memory & Cognition, 21, 802–808. Van Overwalle, F., & Van Rooy, D. (1998). A connectionist approach to causal attribution. In S. J. Read & L. C. Miller (Eds.), Connectionist models of social reasoning and social behavior (pp. 143–171). Mahwah, NJ: Erlbaum. Van Overwalle, F., & Van Rooy, D. (2001). How one cause discounts or augments another: A connectionist account of causal competition. Personality and Social Psychology Bulletin, 27, 1613–1626. Van Sommers, P. (1963). Carbon dioxide escape and avoidance behavior in the brown rat. Journal of Comparative and Physiological Psychology, 56, 584–589. Wagner, A. R. (1969). Stimulus selection and a “modiﬁed continuity theory.” In G. H. Bower & J. T. Spence (Eds.), Psychology of learning and motivation (Vol. 3, pp. 1–41). New York, NY: Academic Press. Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in Pavlovian conditioning: Application of a theory. In R. A. Boakes & M. S. Halliday (Eds.), Inhibition and learning (pp. 301–336). New York, NY: Academic Press. Waterman, C. K. (1969). The facilitating and interfering effects of cognitive dissonance on simple and complex paired associates learning tasks. Journal of Experimental Social Psychology, 5, 31–42. Weiss, R. F. (1963). Defection from social movements and subsequent recruitment to new movements. Sociometry, 26, 1–20. Weiss, R. F. (1968). An extension of Hullian learning theory to persuasive communication. In A. G. Greenwald, T. C. Brock, & T. M. Ostrom (Eds.), Psychological foundations of attitudes (pp. 109–145). New York, NY: Academic Press. Weiss, R. F., Beck, C. M., & Stich, M. H. (1972). Correlated delay of reinforcement in the instrumental conditioning of conversational behavior. Psychonomic Science, 28, 211–212. Weiss, R. F., Boyer, J. L., Colwick, J. T., & Moran, D. J. (1971). A delay of reinforcement gradient and correlated reinforcement in the instrumental conditioning of conversational behavior. Journal of Experimental Psychology, 90, 33–38. Weiss, R. F., Boyer, J. L., Lombardo, J. P., & Stich, M. G. (1973). Altruistic drive and altruistic reinforcement. Journal of Personality and Social Psychology, 25, 390–400.

19-Schachtman-19.indd 452

Weiss, R. F., Buchanan, W., Altstatt, L., & Lombardo, J. P. (1971). Altruism is rewarding. Science, 171, 1262–1263. Weiss, R. F., Cecil, J. S., & Frank, M. J. (1973). Steep delay of reinforcement gradient in escape conditioning with altruistic reinforcement. Bulletin of the Psychonomic Society, 2, 372–374. Weiss, R. F., Cluts, M. J., Williams, M. J., & Miller, F. G. (1977). Communication and conditioning: Correlated reinforcement. Bulletin of the Psychonomic Society, 10, 37–38. Weiss, R. F., Lombardo, J. P., Warren, D. R., & Kelley, K. A. (1971). Reinforcing effects of speaking in reply. Journal of Personality and Social Psychology, 20, 186–199. Weiss, R. F., McDonald, N. B., Little, C., & Shull, R. D. (1991, November). Pavlovian principles in human parallel distributed processing. Paper presented at the meeting of the Psychonomic Society, San Francisco, CA. Weiss, R. F., & Miller, F. G. (1971). The drive theory of social facilitation. Psychological Review, 78, 44–57. Weiss, R. F., Miller, F. G., Steigleder, M. K., & Denton, D. A. (1977). Drive effects on instrumental response speed induced by intermittent disagreement in conversation. Bulletin of the Psychonomic Society, 9, 5–7. Weiss, R. F., Steigleder, M. K., Cramer, R. E., & Feinberg, R. A. (1977). Delay of reinforcement and delay shifts in dyadic communication. Bulletin of the Psychonomic Society, 9, 193–196. Weiss, R. F., Weiss, J. J., Wenninger, V. L., & Balling, S. S. (1981). Sex differences in social inﬂuence: Social learning. Bulletin of the Psychonomic Society, 18, 233–236. Weiss, R. F., Williams, M. J., & Miller, C. R. (1972). Disagreement-induced drive in conversation: A social analog of intermittent shock in escape conditioning. Psychonomic Science, 29, 342–344. Williams, D. A., & Overmier, J. B. (1988). Backward inhibitory conditioning with signaled and unsignaled unconditioned stimuli: Distribution of trials across days and intertrial interval. Journal of Experimental Psychology: Animal Behavior Processes, 14, 26–35. Woods, P. J., & Holland, C. H. (1966). Instrumental escape conditioning in a water tank: Effects of constant reinforcement at different levels of drive stimulus intensity. Journal of Comparative and Physiological Psychology, 62, 403–408. Woods, P. J., Markman, B. S., Lynch, W. C., & Stokely, S. N. (1972). Partial reinforcement

2/16/2011 6:50:47 PM

CONDITIONING ANALOGS OF FAMILIAR SOCIAL PROCESSES

effects in instrumental escape conditioning. Learning and Motivation, 3, 279–292. Woods, P. J., & Schutz, L. J. (1965). Performance in instrumental escape conditioning following a shift in drive stimulus intensity. Proceedings of the 73rd Annual Conference of the American Psychological Association, 1965, 1, 23–24. Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274. Zajonc, R. B. (1976). Family conﬁguration and intelligence. Science, 192, 227–236.

19-Schachtman-19.indd 453

453

Zajonc, R. B., & Mullally, P. R. (1997). Birth order: Reconciling conﬂicting effects. American Psychologist, 52, 685–699. Zbroznya, A. W. (1958). On the conditioned reﬂex of the cessation of the act of eating. I. Establishment of the conditioned cessation reﬂex. Acta Biologiae Experimentalis, 18, 137–162.

2/16/2011 6:50:47 PM

CHAPTER 20 The Impact of Social Cognition on Emotional Learning A Cognitive Neuroscience Perspective Andreas Olsson

In this chapter, I discuss human emotional learning in social situations, bridging the research literatures on emotional learning and social cognition. I begin by relating basic research on social-emotional learning to everyday learning outside the laboratory and to clinical applications. Then, I discuss the mechanisms underlying classical fear conditioning, which has served as a model for our current understanding of the formation of emotional associations. Next, I review relevant ﬁndings from research on the neural bases of social cognition, the study of the perception and understanding of other individuals. These lines of research will show that the neural systems involved in emotional learning and social cognition are partly overlapping, highlighting important commonalities. This will lead to a discussion about how social cognition can affect two speciﬁc forms of learning; observational fear learning in humans and other animals, and instructed fear learning in our species. Next, I will review recent work on the impact of social cognition on the learning to fear others. I will end by discussing a recently proposed neural model of social fear learning that may help us to better understand how social interactions can shape the acquisition, expression, and modiﬁcation of emotional learning from and about others.

INTRODUCTION Imagine as a child you are observing your father arguing with the neighbor that just moved in next door. Pretend that the squabble escalates. Suddenly, the neighbour makes a swift move towards your parent, and you clearly see his fearful face and posture. You are freezing at the sight of your threatened relative facing the raging neighbor. Fortunately, the argument does not end in casualties. Nevertheless, the episode has a strong and long-lasting impression on you. A fear memory has been created. Although most neighborly interactions might be of the friendly and collaborative kind, such as praising the garden next door and borrowing baking powder, our close social environment is, and has been across our evolution as social

primates, a source of potentially lethal threat. Contemporary theorizing poses that threatening situations involving conspeciﬁcs have provided evolutionary pressures shaping the behavior and thus the brain of all social animals, enabling them to adaptively navigate their social and physical environment (Byrne & Whiten, 1988; Dunbar, 2003). Like many other social animals, humans are equipped with a fast and frugal ability to detect and react to conspeciﬁcs that are signaling fear, such as that in your father’s face, and aggression, such as that displayed by your neighbor. In addition to this automatic or “reﬂexive” ability, humans might be uniquely equipped to “read” the minds of others by means of attributing mental states, such as fear and the intention to harm, in a more controlled or “reﬂective” manner (Gilbert, 1998). Research on

454

20-Schachtman-20.indd 454

2/15/2011 7:53:52 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

reﬂexive and reﬂective evaluations of social cues, such as fearful and angry faces, suggests that these processes are supported by interacting, but partially dissociable, neural networks in the brain (Adolphs, 2009; Amodio & Frith, 2006; Lieberman, 2007; Olsson & Ochsner, 2008). Interestingly, these two ways of processing social information appear to be paralleled by emotional learning mechanisms operating on output from reﬂexive and reﬂective information processing, respectively, suggesting common bases for social cognition and emotional learning. Indeed, as we will see, this is becoming increasingly clear as research on the neural bases of social cognition ﬁnds itself investigating neural networks partially overlapping with those previously known from the literature on emotional learning. In this chapter, I am going to discuss human emotional learning in social situations, bridging the literatures on social cognition and emotional learning. As we saw in the earlier example, other individuals can be both the communicating source of a threatening event (your father) and the threat itself (the neighbor). In other words, we may learn both from and about others, reﬂecting two key functional characteristics of individuals around us. Here, the emphasis will be on research relevant to emotional learning from others. In particular, I will discuss the learning of fear, which is the kind of emotional learning currently most extensively investigated on both the behavioral and neural level. In addition, fear learning has several important implications for the development and maintenance of psychological disorders associated with fear and anxiety. I begin by highlighting the interrelatedness of knowledge from three perspectives on fear learning: the basic experimental; the adaptive, everyday learning outside the lab; and ﬁnally, its translation into clinical applications. Then, to better understand emotional learning from others, I review relevant ﬁndings in two separate domains of research—classical conditioning and social cognition—providing insights into the basic mechanisms underlying the formation of emotionally colored associations, and perceiving and understanding other individuals, respectively. To this aim, I ﬁrst summarize what is known about the mechanisms underlying classical

20-Schachtman-20.indd 455

455

conditioning, which has served as a model for our current understanding of the formation of emotional associations. Then, I review relevant ﬁndings in the research on the mechanisms of social cognition, which concerns itself with questions related to the perception and understanding of other individuals—the ones who we are learning from and about. As this discussion will clarify, the neural systems involved in emotional learning and social cognition share many functional regions, suggesting important commonalities. This will naturally lead the discussion into speciﬁc questions of social emotional learning that brings together the literatures on emotional learning and social cognition. In particular, I will review behavioral and neurobiological studies on two forms of socially mediated fear learning: observational fear learning in humans and other animals, and instructed fear learning in our species. I will end by discussing a neural model of social fear learning that may help us to understand how social interaction can shape the acquisition, expression, and eventually modiﬁcation, of emotional learning from and about others.

FROM CONDITIONING TO THE CLINIC Returning to the example in the beginning of the chapter, it becomes clear that watching someone else expressing strong fear can be a powerful learning episode shaping the observer’s future behavior. In the example, the child may acquire several bits of emotionally relevant information about the neighbor but also about the parent. First, even without experiencing the aversive qualities of the neighbor ﬁrsthand, the child is likely to develop negative evaluations, possibly also a fear, of the neighbor and the place where the episode took place. This fear might then lead to avoidance behavior, which could be adaptive. The neighbor might indeed be dangerous. However, if the child’s initial perceptions were mistaken and the neighbor in fact is friendly minded, avoidance would not be adaptive. In addition, if the fear is generalized to other, current and future, neighbors, it can cause problems. Even worse, if the fear is generalized to other grown-up males, looking like the neighbor,

2/15/2011 7:53:52 PM

456

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

or even creating anxiety in the presence of unknown individuals in general, it might be the beginning of a self-perpetuating vicious circle of dysfunctional anxiety and avoidance behaviors. Fears can also be acquired through verbal communication, such as hearing stories told by parents or peers. In fact, clinical research suggests that a sizable proportion of anxiety disorders, such as social anxiety and phobias, are related to fearful vicarious experiences or verbally communicated threat information (Askew & Field, 2008; Mineka & Zinbarg, 2006; Rachman, 1968). These claims are mainly based on retrospective self-reports about the origin of the fear and phobias. Self-reports are notoriously problematic because they are often impossible to verify, and research shows that fear learning can occur without the involvement of conscious awareness (Esteves, Parra, Dimberg, & Ohman, 1994). Also, the focus on etiology provides little insight into the proximate mechanisms of fear learning. Instead, much of our current understanding of the basic mechanisms believed to underlie maladaptive responses to traumatic and stressful events relies on research on classical conditioning. However, this research also has important limitations, such as its restricted ecological validity raising doubts about the applicability of conditioning research to clinical situations. Whereas most of our learned emotional responses acquired outside the research laboratory might be transmitted from fellow humans, often being about other individuals, classical conditioning in the lab is asocial, requiring ﬁrsthand experience of an aversive event, and often being about nonsocial stimuli, such as tones and colored squares. In other words, fear conditioning protocols are trading ecological validity to increase control, which reduces the generalizability of the results. However, new experimental research is trying to address these shortcomings of both clinical research and the traditional conditioning model. This research is greatly aided by the development of neuroimaging methods to study the mechanisms of the working brain, and improved techniques to present more ecological stimuli. Another important consideration related to the translation of basic research of emotional learning into more applied settings is the fact

20-Schachtman-20.indd 456

that individuals vary greatly in their responses to emotional stimuli. In particular, they differ in both their propensity to acquire emotional responses (e.g., conditionability), and in how they are reacting to social stressors with farreaching consequences for their well-being and the likelihood of developing related psychological disorders, such as anxiety disorders and depression. There is currently a growing understanding about how variations in genetic makeup (e.g., Canli & Lesch, 2007 Hettema, Annas, Neale, Kendler, & Fredrikson, 2003; Lonsdorf et al., 2009; Munafo, Brown, & Hariri, 2008) and personality (e.g., Lissek et al., 2008; Mineka & Zinbarg, 2006; Olsson, Carmona, Bolger, Downey, & Ochsner, 2007) are related to differences in emotional learning and social reactivity, and importantly, how such interactions may contribute to disorders. Furthermore, an individual’s personal learning history is in itself an important factor contributing to making the individual more or less vulnerable to respond maladaptively to future stressors and trauma implicated in the origins of anxiety disorders (Mineka & Zinbarg, 2006 See also chapter 3, this volume). To enhance the understanding of the underlying mechanisms mediating the impact of genetic vulnerabilities on dysfunctional responses to stressful events, there is a growing interest in research on neural responses to emotionally and socially evocative stimuli in individuals with speciﬁc genetic and personality markers (Bishop, 2008; Canli & Lesch, 2007; Munafo et al., 2008). On the whole, the consideration of the factors discussed here; the ecological validity of experimental manipulations and individual differences in learning, will likely lead to a deeper, as well as a more applicable, understanding of the etiology and maintenance of common psychological disorders.

DIFFERENT PROCEDURES, SAME UNDERLYING PROCESSES? One fundamental question related to social fear learning is whether the processes critical to classical conditioning are the same, or similar, to those supporting the acquisition of fear and anxiety from and about others. In other words, are

2/15/2011 7:53:52 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

these different learning procedures drawing on the same underlying learning processes? If the associative processes that are at the core of classical conditioning are also involved in forming social fear learning, then, much information can be gained about learned emotions outside the lab—be they normal or dysfunctional—by studying classical conditioning. For example, well-explored conditioning phenomena, such as latent inhibition, unconditioned stimulus (US) devaluation, and extinction (i.e., phenomena that can reduce the size of the conditioned response [CR]; see Chapter 1, this volume), would then be expected to constrain observational and instructed fear just as they do with conditioned fear. The extinction of learned fear responses is a particularly good illustration, because it is governed by principles that have been used to successfully guide behavioral therapies in treating phobias and other anxiety disorders for decades. In addition, recent efforts to develop extinction protocols both in combination with (Davis, Barad, Otto, & Southwick, 2006) and without (Monﬁls, Cowansage, Klann, & LeDoux, 2009) pharmacological treatment raise hopes for improved outcomes. Indeed, research over the last 40 years has shown that fear learning from others by observation display several commonalities with fear learning through direct personal experience, classical conditioning, providing a validation of the conditioning model (Askew & Field, 2008; Bandura, 1977; Bandura & Menlove, 1968; Hygge & Ohman, 1978; Mineka, Davidson, Cook, & Keir, 1984; Mineka & Zinbarg, 2006; Olsson & Phelps, 2007; also see Green & Osborne, 1985). However, some earlier work (Berber, 1962; see also Lanzetta & Englis, 1989) and a more recently emerging line of research on the neural aspects of observational fear learning (Hooker, Germine, Knight, & D’Esposito, 2006; Olsson, Nearing, & Phelps, 2007) suggests that in order to understand socially mediated fear learning, the conditioning model needs to be complemented by recent developments in social cognition and social cognitive neuroscience (Olsson & Ochsner, 2008; Olsson & Phelps, 2007). Also research on learning through verbal communication suggests that this form of

20-Schachtman-20.indd 457

457

learning draws on similar mechanisms as learning by observation and conditioning (Phelps et al., 2001). However, as will be discussed in greater detail in later sections, this form of symbolically mediated learning also displays important differences. The relevance of the conditioning model is also pertinent to learning about others. In the former example, the child gains emotionally relevant information about the individuals present in the situation, about the impulsive nature and possibly malicious intentions of the neighbor and the parent’s anxious disposition. These pieces of information might conﬁrm or update existing expectancies based on previous experiences of the parent or provide new diagnostic information about the neighbor. Recent research suggests that the principles guiding emotional learning about other individuals are echoing known principles of classical conditioning, such as dependence on prediction errors (Delgado, Li, Schiller, & Phelps, 2008). Just as in the case of learning from others, research has only begun to understand how emotional learning about others is affected by social cognition in normal, healthy individuals (Navarrete et al., 2009; Olsson, Ebert, Banaji, & Phelps, 2005; Singer et al., 2006; Todorov, Said, Engell & Oosterhof, 2008), and individuals showing deviations in social cognitive performance, such as an increased (Kross, Egner, Ochsner, Hirsch, & Downey, 2007; Lissek et al., 2008; Mineka & Zinbarg, 2006) or decreased (Blair, Colledge, Murray, & Mitchell, 2001) anxiety in social settings.

CLASSICAL CONDITIONING IN THE BRAIN Most of our knowledge about the neurobiological mechanisms of emotional learning comes from research on classical conditioning. In a traditional fear-conditioning procedure, a neutral conditioned stimulus (CS) is paired with a naturally aversive stimulus (US), leading to a conditioned fear response to the CS. The extensive use of this procedure since Pavlov (1927) has established classical conditioning as a model of fear learning (Phelps & LeDoux, 2005). Consistencies

2/15/2011 7:53:52 PM

458

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

in the physiological expression of conditioned fear elicited by the basic protocol indicate that mechanisms of emotional learning are analogous across species. The Role of the Amygdala in Classical Conditioning

Research on the neurobiology of fear conditioning has focused on the amygdala, a nut-shaped structure in the bilateral medial temporal lobes and a key structure in the brain’s fear circuitry (Figs. 20.1a, 20.1b, 20.2a). Although the mygdala processes a wide range of emotionally relevant information, much of its anatomy and functional role in fear conditioning has been conserved throughout evolution (LeDoux, 2000). The amygdala is composed of several subnuclei, some of which serves speciﬁc functions in fear conditioning. In short, sensory information is believed to arrive in the lateral nucleus from centers in the thalamus and sensory cortices (Amaral, 1986; LeDoux, Farb, & Ruggiero, 1990). The lateral nucleus also receives nocioceptive information speciﬁc for the US, providing a biological basis for the convergence of CS-US information (Fig. 20.2a). Supporting this conclusion, research suggests that this is the locus of synaptic plasticity shaping associations between representations of the CS and US (Blair, Schafe, Bauer, Rodrigues, & LeDoux, 2001; Quirk, Armony, & LeDoux, 1997; Romanski, Clugnet, Bordi, & LeDoux, 1993). The lateral nucleus then relays information to the central nucleus and basal nucleus that mediates the output to other regions that regulate the expression of fear and anxiety (LeDoux & Gorman, 2001). For example, projections to the hypothalamus are important for mediation of autonomic responses (Price & Amaral, 1981), which in humans can be measured through the skin conductance response (Davis & Whalen, 2001). Other areas of projection, such as the ventral tegmental area (Simon, Le Moal, & Calas, 1979) and the central gray (Hopkins & Holstege, 1978), serve roles in regulation of behavioral expressions of fear. Another fear-related behavior, avoidance, is mediated by input to the basal ganglia from the basal nucleus (Everitt & Robbins, 1992). The striatum, which is the input region of

20-Schachtman-20.indd 458

Figure 20.1 Amygdala activity during condi-

tioned and socially learned fear. (a) The outlined box contains the region of the medial temporal lobe that includes the bilateral amygdala. (b–d) Amygdala activation to the CS is seen bilaterally after fear conditioning (b) and observational fear learning (c), and unilaterally (d) in the left amygdala after instructed fear. Reprinted from Olsson, A. & Phelps, E. A. (2007). Social Learning of Fear. Nature Neuroscience, 10, 1095–1102.

the basal ganglia, is known for signaling prediction errors during reward learning (Schultz, Dayan, & Montague 1997), but it has also been increasingly implicated in aversive learning (Delgado et al., 2008). As will be discussed later, the interconnectivity both within and between the amygdala and other subcortical regions, such as the striatum, appears to have important implications for social-emotional learning.

2/15/2011 7:53:52 PM

a Conditioned fear

Cortically distributed (AI, ACC, hipp.) representation of the CS US CS-US/pairing

CE B

LA

Visual cortex CS

Primary (SI) and secondary (SII) somatosensory cortex

US

Visual thalamus Somatosensory thalamus CS

US

Autonomic output

b Observational fear Cortically distributed (AI, ACC, hipp.) representation of the CS-US/pairing CS

MPFC: mentalizing

ACC, AI Empathetic emotion

CE B

Visual cortex

Visual cortex

CS

Visual thalamus

Visual thalamus

Social US

CS

Autonomic output

c Instructed fear Left hemisphere: language representation

Threat

Left hemisphere:Cortically distributed (AI, ACC, hipp.) representation of the CS-US/pairing CS Threat

CE B

LA

Visual cortex CS

Visual thalamus CS

Autonomic output

Figure 20.2 A Neural Model of Social Fear Learning. The arrows describe the ﬂow of information between different functional brain regions. Although the arrows point only in one direction, the connectivity might be bidirectional. (a) Fear conditioning occurs by associating the visual representation of the CS with the somatosensory representation of the aversive US. The lateral nucleus (LA), in which sensory representations (Continued) 459

20-Schachtman-20.indd 459

2/15/2011 7:53:52 PM

460

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Dissociating Amygdala and Hippocampal Function in Classical Conditioning

As hinted at earlier, the role of the amygdala in classical fear conditioning is best understood together with other functional regions within a greater circuitry of fear learning. This neural network involves sensory input and motor output systems, as well as regions that contribute to explicit and conscious aspects of learning and expression of fear. For example, the hippocampus, a medial temporal lobe structure adjacent to the amygdala, is critical for coding contextual information about the fear learning situation, such as relationships between different features and the timing of events (Eichenbaum & Cohen, 2002). This indicates that whereas the amygdala is responsible for developing associations between somatosensory states and representations of individual stimuli (cue learning), the hippocampus appears to encode relations between the various cues that comprise the learning context (contextual learning). Patients with bilateral and unilateral amygdala damage can verbally report the CS-US contingency, but they lack the normally associated autonomic response (LaBar, LeDoux, Spencer, & Phelps, 1995), suggesting that the amygdala is necessary only for implicit, nonverbal processes underlying acquisition and expression of learned fear. In contrast, the hippocampus is essential for consolidation and retention of explicit or declarative memory of the CS-US contingency (Bechara et al., 1995) and the environmental contexts that regulate conditioned fear responses (LaBar & Phelps, 2005). These distinctions will prove to be of

importance in the discussion about different kinds of social-emotional learning. The Frontal Cortex in Classical Conditioning

The anterior insula (AI) and the anterior cingulate cortex (ACC) are two cortical regions consistently implied in aversive conditioning studies (Büchel, Morris, Dolan, & Friston, 1998). These two regions both receive ascending viscerosensory inputs (Decety & Jackson, 2004; Gallese, Keysers, & Rizzolatti, 2004; Iacoboni & Dapretto, 2006; Morrison et al., 2004; Singer et al., 2004; Zaki, Ochsner, Hanelin, Wager, & Mackey, 2007). The AI has a role in the awareness of an external threat (e.g., the US and the conditioned CS), and it is believed to support affective experience in part through interoceptive awareness of somatosensory inputs (Craig, 2009; Critchley, Wiens, Rotshtein, Ohman, & Dolan, 2004). The ACC is thought to code affective attributes of pain, such as the perceived unpleasantness (as opposed to sensory-discriminative properties, such as location and intensity) (Eisenberger, Lieberman, & Williams, 2003; Hutchison, Davis, Lozano, Tasker, & Dostrovsky, 1999) and motivate appropriate behavior, such as avoidance, through projections to motor and autonomic centers (Critchley et al., 2004). Across species, the prefrontal cortex (PFC) is a major player in the regulation of conditioned and other affective responses through its impact on the activation in subcortical regions, such as the amygdala (Ochsner & Gross, 2005; Robbins, 2005). In particular, the ventral (infralimbic)

Figure 20.2 (Continued)

of the CS and US converge, is believed to be the site of learning. The amygdala also receives input from the hippocampal memory system (hipp.), anterior insula (AI) and anterior cingulate cortex (ACC) containing secondary representations of the CS and US, information about the learning context and the internal state of the organism. (b) In observational fear learning, the visual representation of the CS is modiﬁed by its association with a representation of the distressed other, serving as the US. As in fear conditioning, it is hypothesized that representations of the CS and the US converge in the LA. The strength of the US may be modiﬁed by MPFC input related to the interpretation of the other’s mental state, as well as cortical representations of empathic pain through the ACC and AI. (c) Instructed fear learning occurs by modifying the processing of the visual representation of the CS through its association with an abstract representation of threat. Instead of being coded in the amygdala, the CS–’threat’ US contingency is likely to be represented in a cortically distributed network, critically depending on the hippocampal memory system. Reprinted from Olsson, A. & Phelps, E. A. (2007). Social Learning of Fear. Nature Neuroscience, 10, 1095–1102.

20-Schachtman-20.indd 460

2/15/2011 7:53:53 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

region of the medial prefrontal cortex (mPFC) is necessary for the retention of extinction of conditioned fear responses in rats (Quirk, Garcia, & Gonzalez-Lima, 2006), and the human homolog of this region is involved in extinction in humans (Phelps, Delgado, Nearing, & LeDoux, 2004). Strategic regulation of affective expressions by purposeful interpretation (appraisal) of the emotional meaning of a given situation involves more dorsal and lateral regions of the PFC. For example, the dorsolateral prefrontal cortex (dlPFC) has been assigned a key role in the up- and down-regulation of affective responses to images through appraisal strategies by their impact on amygdala activity (Ochsner & Gross, 2008; Ochsner, Ray, et al., 2004). Consistent with these ﬁndings, a recent study reported that subjects using appraisal strategies recruited the dlPFC to down-regulate their conditioned fear responses and the accompanying amygdala activation to the CS (Delgado et al., 2008). Interestingly, this study also showed activation in the mPFC that overlapped with regions previously implicated in studies of extinction of conditioned fear in humans, arguing that there are both similarities (mPFC) and differences (dlPFC) during passive (extinction) and active (reﬂective) regulation of learned emotional responses. In this section, I have discussed several functional brain regions that are involved in classical conditioning. Some of these (e.g., the amygdala) support the reﬂexive and implicit aspects of emotional learning. Other regions, such as the hippocampus and the PFC, support explicit and more controlled forms of emotional learning. As we will see in the next section, many of the regions highlighted here are also involved in the processing of social cognitions, hinting toward common functional bases.

SOCIAL COGNITION IN THE BRAIN Recent advances in brain imaging and work with brain-lesioned patients have dramatically increased our understanding about the social brain. For example, as will be discussed in greater detail later, the amygdala has been implicated in

20-Schachtman-20.indd 461

461

the initially fast and frugal processing of social cues (e.g., your parent’s fearful face) signaling the presence of a potential threat (e.g., the neighbor), thus allowing for a quick adaptive response. In contrast, the mPFC has been implicated in the reﬂective attribution of mental states (e.g., fearfulness to your parent and harmful intentions to your neighbor). The dual involvement of these and other regions in both nonsocial emotional learning and social cognition might be explained by the fact that the social cues triggers learning and/or that some more basic neural computations of motivational relevance contribute to both kinds of psychological functions. The Role of the Amygdala in Social Cognition

The amygdala has long been assigned a key role in social functioning. Since the seminal ﬁndings by Kluver and Bucy (1939), reporting severe impairments in a variety of social behaviors in monkeys following bilateral temporal lobectomy, this structure has become one of the most studied regions in a neural network now known to support the perception and evaluation of social stimuli. Although the amygdala has an ancient evolutionary past, its interconnectedness to neocortex has increased substantially in primates. The basolateral complex in the primate amygdala has strong reciprocal connections to visual cortex, in particular to the inferotemporal region that responds to face identity and to facial expression (Kanwisher & Yovel, 2006; Rolls, Tovée, Purcell, Stewart, & Azzopardi, 1994). Moreover, the basolateral complex has direct connections to the ventral part of the mPFC and indirectly with more dorsal regions of the mPFC (Barton, Aggleton, & Grenyer, 2003; Young, Scannell, Burns, & Blakemore, 1994). Interestingly, these observations corroborate the proposal that the primate amygdala may be particularly prone to form associations between more complex socioemotional stimuli, especially when they are visually represented. The chief role of the amygdala in social cognition appears to be the orchestration of quick responses to potentially dangerous stimuli and events in the social environment in order to

2/15/2011 7:53:53 PM

462

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

disambiguate its meaning (Adolphs, 2009; Adolphs et al., 2005; Whalen, 2007). Indeed, no aspect of our environment might be more ambiguous than its social domain. Through learning about our fellow humans, they may be become predictable to a certain degree, but never certain. For example, the swift alternation of facial expressions, signaling a change from benevolent to adversary intentions in a conspeciﬁc illustrates the rapidly unfolding of events that may have fatal implications for the individual if not responded to adaptively. In response to such environmental challenges, a neural system for rapid detection of potentially harmful cues in the environment, centered on the amygdala, has evolved (Adolphs, 2009; de Gelder, Snyder, Greve, Gerard, & Hadjikhani, 2004; LeDoux, 2000 Ohman & Mineka, 2001; Whalen et al., 1998). Consistent with the assumption that its role in social cognition is to rapidly respond to potentially dangerous and ambiguous stimuli, amygdala activation to faces and body postures can inﬂuences early visual and attentional processing (Anderson & Phelps, 2001; Morris, Ohman & Dolan, 1998a; Phelps, Ling, & Carrasco, 2006; Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004) and action representations (de Gelder et al., 2004). In addition to the earlier discussed role of the amygdala in the implicit responding to presentations of nonsocial conditioned stimuli, research shows that social stimuli may be especially suited to access this brain system, even in the absence of conscious awareness of their presence (see Chapter 18, this volume). For example, a series of studies by Ohman and colleagues has demonstrated conditioned responses as measured by physiological arousal to subliminally presented (and reportedly not seen) images of angry faces that were previously paired with a shock (Ohman & Mineka, 2001). Imaging studies employing a similar technique to subliminally present facial stimuli have highlighted the role of the amygdala (Morris, Ohman & Dolan, 1998b; Whalen et al., 1998). Many of the studies on the role of the amygdala in social cognition have suggested that this neural structure is particularly sensitive to facial stimuli expressing fear (Adolphs, 2009;

20-Schachtman-20.indd 462

Whalen et al., 1998), consistent with the assumption that fearful faces are particularly ambiguous relative to the location of the threat. However, it is now clear that the amygdala responds to a much wider range of stimuli and situations. For example, it has been shown to be interested in other facial expressions (Adolphs, 2009), untrustworthy versus trustworthy looking faces (Winston, Strange, O’Doherty, & Dolan, 2002) and to neutral faces belonging to a racial group other than the observer’s (Cunningham et al., 2004; Harris & Fiske, 2006; Hart et al., 2000; Lieberman, Hariri, Jarcho, Eisenberger, & Bookheimer, 2005; Phelps et al., 2000), and it is modulated by contextual information about social cues (Kim et al., 2003; Ochsner, Ray, et al., 2004). Based on a lesion study, Adolphs (2009) argues forcefully that the amygdala contributes to the generation of actions that can disambiguate the situation, such as guiding the direction of attention toward cues that are speciﬁcally diagnostic for the emotional signiﬁcance of a situation. Finally, some studies argue for a more elaborate role of the amygdala during development to enable the attributions of mental states, such as intentions, desires, and emotions to other individuals (Baron-Cohen et al., 2000). Taken together, existing research on the role of the amygdala in social cognition suggests that it may serve several functions of exploring salient cues in the inherently interactive social environment, which demands rapidly concerted responding of different social cognitive, emotional processes and motor actions. The fact that many of these social cues and events may be of critical importance for the individual to remember makes the amygdala a likely hub interconnecting neural circuitries supporting emotional learning and social cognition. The Role of the Frontal Cortex in Social Cognition

In addition to the amygdala, several other brain regions are implicated in a distributed functional network supporting social cognition. Here, the focus of the discussion will be on cortical regions that have already been reviewed in relation to

2/15/2011 7:53:53 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

emotional learning, in particular the anterior insula (AI), anterior cingulate cortex (ACC), and medial prefrontal cortex (mPFC). In addition, there are many other, more posteriorly located, cortical regions that have been consistently associated with social cognition and that are likely to serve various functions in socialemotional learning through their direct or indirect connectivity with the amygdala and other regions important for emotional learning, but less is known about their functions in such learning tasks. To this group of regions that will not be discussed here in any detail belongs the superior temporal sulcus (STS), which is known to be involved in the integration of information about body movements and higher levels of processing (Castelli, Frappe, Frith, & Frith, 2000), and the right temporal-parietal junction (rTPJ), which is implicated in mental state attributions about true or false beliefs to others (Saxe, Moran, Scholtz, & Gabrieli, 2006). Social cognition, including the understanding of emotional experiences of other individuals through the attribution of mental states (e.g., fear), is likely to be aided by the brain’s system for dual representations of one’s own and others’ experiences. This overlap between one’s own and others’ mental states enables an internal simulation in terms of ﬁrsthand experiential understanding of the others’ emotions and intentions, which aids the prediction of their actions. Evidence for such a system initially emerged from imaging studies showing that certain motor regions respond during both the execution and observation of speciﬁc movements (Gallese, Keysers, & Rizzolatti, 2004). It was argued that if motor regions code the intentions behind one’s own action, then if activated when observing another individual engaging in the same action, they might support an understanding of that individual’s intention through simulation of his or her experiences (Blakemore & Decety, 2001; Gallese, Keysers, & Rizzolatti, 2004). This “dual representation” logic has guided a host of studies on the direct experience and observation of pain or emotion that also show activation of overlapping neural systems, including most prominently the AI and the ACC.

20-Schachtman-20.indd 463

463

The engagement of these regions is thought to facilitate the automatic sharing of, and hence direct experiential understanding of, others’ affective states, thus providing a substrate for empathy (e.g., with your fearful parent) (Decety & Jackson, 2004; Singer et al., 2004). In turn, these processes are likely to affect the way we learn in the situation. In addition to the automatic sharing of experiences through simulation, such as that described in the dual-representation literature, more reﬂective attributions of mental states may be required to understand another individual’s emotional state. These controlled attributions allow us to purposefully take other peoples’ perspectives and make judgments about their emotions or diagnostic elements of their emotional dispositions, thereby changing empathic responding (Batson, Thompson, & Chen, 2002) and recruitment of the anterior insula and the ACC (Lamm, Batson, & Decety, 2007). These reﬂective processes have been shown to depend on a network of regions of the dorsal mPFC, including Brodmann area 10 (BA 10), and the right temporo-parietal junction (rTPJ; Mitchell, Macrae, & Banaji, 2006; Ochsner, Ray, et al., 2004; Saxe et al., 2006). Interestingly, some of the same regions involved in reﬂecting upon another individual’s emotional state are also involved in reﬂecting upon our own emotions (Ochsner, Ray, et al., 2004; Saxe et al., 2006), consistent with the conjecture that we sometimes treat ourselves as an “other” when making self-judgments (Ochsner, Ray, et al., 2004). Of course, the reverse might also be true. In other words, we might use information about our own mental states and traits when we reﬂect upon the mental states and traits of others, especially if these others seem to be similar to ourselves. Further supporting this idea, judgments about known or similar, as compared with less familiar or dissimilar, others draw on medial frontal regions overlapping with those used for self-referential processing (Mitchell et al., 2006; Ochsner, Ray, et al., 2004). Strikingly, recent research shows that taking others’ perspective can increase the self-other overlap by shifting the recruitment of the prefrontal activation more ventrally (Ames, Jenkins, Banaji, & Mitchell, 2008).

2/15/2011 7:53:53 PM

464

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Throughout the last two sections, I have discussed a range of brain regions belonging to the widely distributed neural network supporting both rapid and reﬂexive, and more reﬂective social cognitions. These regions are either directly overlapping with, or closely connected with, the neural network responsible for emotional learning that was reviewed earlier. Indeed, processes computed through the interaction between these two networks will be important for the understanding of social-emotional learning and its neural bases. This will be discussed next.

LEARNING FROM OTHERS As I have been arguing above, the individual neural regions involved in classical conditioning and social cognition can only be fully understood by recognizing their interactions with other functional regions in the greater neural circuitries to which they belong. Analogous to this, the full understanding of an individual’s learning experiences in a natural environment requires the understanding of the intricate social-interactive environment in which it occurs. Similar to classical conditioning, the transmission of fear signals through social channels is well documented in a range of species (Hauser, 1996). The ability to detect and respond appropriately to signs of fear and pain in a conspeciﬁc probably has conferred signiﬁcant selective advantage during evolution. However, these social cues not only alert the receiver about potential imminent dangers, as I have discussed earlier, but they also assign a threat value to cues or the context that are associated with the threat display. For example, a conspeciﬁc’s (e.g., your parent’s) fear expression may serve as a US, eliciting immediate reﬂexive aversive response in the observer that becomes associated with the paired stimuli (e.g., the neighbor and the location where the episode unfolds). However, observational learning may also be subserved by social inference, in which the conspeciﬁc’s fear expression is a CS that was previously associated with a directly experienced aversive event (US) and may act as a secondary punisher in future learning. To continue with the earlier example, you may have come to associate your parent’s

20-Schachtman-20.indd 464

fear face with the pain you felt when you cut yourself with a knife at an earlier time. Now, you are inferring the negative value of your neighbor through his association with your parent’s expression. Although these alternative views (i.e., emotional faces eliciting an unconditioned or conditioned response in the observer) are not mutually exclusive, existing research supports the former view. Our ultrasocial environment provides ample opportunities to watch and learn from others’ emotional responses to both social and nonsocial stimuli (Bandura, 1977; Rachman, 1977). In addition to this ability, which we share with other animals, our unique linguistic ability enables us to acquire information about emotional qualities through verbal communication. In 1968, Rachman proposed that observation and instruction, along with classical conditioning, constitute three main pathways to the development of fears (Rachman, 1968). Indeed, subsequent research has shown that they all can produce strong and persistent fear learning in humans and that they all can be related to the development of psychological disorders (Askew & Field, 2008; Mineka & Zinbarg, 2006; Olsson & Phelps, 2007). Next, I will ﬁrst review research on observational fear learning in humans and non-human animals, followed by the literature on instructed fear learning. Observational Fear Learning in Non-Human Animals

Observational fear learning has been studied in many species, including birds (Curio, 1988), mice (Kavaliers, Choleris, & Colwell, 2001), cats (John, Chesler, Bartlett, & Victor 1968), cows (Munksgaard, DePassille, Rushen, Herskin, & Kristensen, 2001), and primates (Berber, 1962; Hygge & Ohman, 1978; Mineka & Cook, 1993; Mineka, Davidson, Cook, & Keir, 1984; Olsson & Phelps, 2004; Olsson et al., 2007; Vaughan & Lanzetta, 1980). In a study on mice (Kavaliers et al., 2001), model mice were attacked by biting ﬂies while observer mice watched. When exposed 24 hours later to biting ﬂies, whose biting parts had been removed, the model and observer mice expressed conditioned analgesia

2/15/2011 7:53:53 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

and avoidance responses to similar degrees, implying that individual and social fear learning were equally effective. The strength of the model’s fear response during its own learning episode was not correlated with expressed fear learning in the observer at a later test. However, such a relationship was found during observational fear learning in rhesus monkey (Mineka & Cook, 1993), indicating that there may be a greater reliance on the model’s emotional expressions during the learning process in primates. Indeed, this conjecture is supported by the rich and ﬂexible musculature of the primate face, especially in humans, allowing us to produce a greater variety of emotional expressions, superior to that of other species (Ekman, 1982). The cortical areas dedicated to face processing are also relatively enlarged in primates (Rolls, 1999), implying an augmented reliance on facially transmitted emotional information. In monkeys (Mineka & Cook, 1993; Mineka et al., 1984) and humans (Gerull & Rapee, 2002; Olsson & Phelps, 2004; Olsson & Phelps, 2007; Vaughan & Lanzetta, 1980) facial fear expression appears to serve as a reliable US. For example, in an important series of studies by Mineka and colleagues, cage-reared monkeys were shown either live presentations or movies of model monkeys reacting fearfully to snakes (toy or real) or to non–fear-relevant objects (Mineka & Cook, 1993; Mineka et al., 1984). When fear-relevant objects were used, the relationships between the strength of a learning model’s expressed distress, the observer’s immediate emotional response to the model’s distress, and the learned fear in the observer as measured at a later time were similar to the known relationships between the US, unconditioned response (UR), and CR in classical fear conditioning (Mineka & Cook, 1993; Mineka et al. 1984). This single social encounter with a fearful fellow monkey produced a strong and robust fear response that was measured several months after the encounter (Mineka & Cook, 1993). These ﬁndings strongly indicate that observational fear learning draws on the same underlying processes as fear conditioning. Still, evidence for what neural processes are supporting observational fear learning in non-human animals is lacking.

20-Schachtman-20.indd 465

465

Observational Fear Learning in Humans

In a study inspired by the experimental work in monkeys by Mineka and colleagues (e.g., 1984), Gerull and Rapee (2002) showed children rubber snakes and rubber spiders together with their mothers’ facial expressions of either fear or happiness. Following the observation of the fear faces as compared to the happy face, the children showed behavioral expressions of fear and stimulus avoidance both at a direct test and at a later time, corroborating the ﬁndings of Mineka and colleagues in humans. Indeed, children with subclinical animal phobias or extreme fears toward certain situations, such as darkness, often report having observed parents fearful in the same or similar situations (Bandura & Menlove, 1968; Mineka & Zinbarg, 2006). Interestingly, a more recent follow-up study showed that exposure to positive maternal modeling, prior to fearful modeling as in the Gerull and Rapee study (2002), prevented the acquisition of fear during negative modeling (Egliston & Rapee, 2007; see also a similar “immunization” effect in monkeys, Mineka & Cook, 1986). Exposure to the stimulus alone did not have the same effect, suggesting that latent inhibition was not effective in this observational paradigm. In another line of studies, Field and colleagues have been using self-reported fear beliefs as an index of fear learning in children who were exposed to the combination of fearful faces and novel stimuli (Askew & Field, 2008; Field, Argyris, & Knowles, 2001). Although retrospective reports of emotional states and preferences are notoriously problematic (Johansson et al., 2005; Mineka & Zinbarg, 2006), these studies support the conclusion that observational fear learning in children is both strong and persistent (in some cases observed 3 months post observation) and appear to share various characteristics with classical conditioning. Also adults learn to fear through observing others. Indeed, over the last half a century, behavioral and psychophysiology research has produced consistent evidence of strong and persistent fear learning through social observation in adults (Bandura, 1977; Hygge & Ohman, 1978; Olsson & Phelps, 2004; Vaughan &

2/15/2011 7:53:54 PM

466

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Lanzetta, 1980, but see Green & Osborne, 1985). A study directly comparing observational and instructed fear learning with classical fear conditioning supported this conclusion. Olsson and Phelps (2004) demonstrated fear learning of equal magnitude following the three ways of learning when the CSs were presented subliminally (seen). Strikingly, when CSs were presented subliminally (and reportedly not seen), learned fear responses were only displayed in the conditioning and observational learning groups. This supported the idea that similar learning processes and representations underlie observational and conditioned fear. These claims have since been substantiated by imaging research. Olsson and colleagues (2007) asked subjects to watch a movie of another person expressing distress when receiving electric shocks paired with a CS. Later, subjects expected to receive shocks along with the same stimulus that was paired with the model’s distress in the movie they just had watched. Importantly, no shocks were administered to the subjects during the test stage to ensure that their representation of the US-CS pairing was based solely on indirect, vicarious experiences. The results showed that, similar to previous fear-conditioning studies, the bilateral amygdala was involved during both the learning (observation) and the subsequent expression (test) of learned fear, strongly supporting the assumption that similar associative mechanisms and their underlying neural processes support both conditioned and observational fear learning (Figs. 20.1a–c). Similar conclusions can be drawn from an experiment by Hooker and colleagues (2006). In this study, subjects watched images of emotional faces (fearful and happy) alone or together with arbitrary ﬁgures (learning objects). These results showed that the amygdala was more responsive to faces when their emotional expressions were associated with a learning object than presentations of the emotional expressions or objects alone, pointing toward the speciﬁc role of facial expressions as a source of emotional learning. These lines of evidence and the rich connectivity between the amygdala and visual and ventral parts of the mPFC as discussed earlier,

20-Schachtman-20.indd 466

indicates that, at least in primates, representation of fear learning through observation and classical conditioning may be rather similar within the amygdala. However, despite the many features shared between conditioned and observational fear, nonsocial and social forms of learning must differ in several fundamental ways, implying involvement of partially dissociable neural networks outside the amygdala. For example, a conspeciﬁc’s expression of distress may be naturally aversive and serve as a US eliciting an immediate unconditioned response in the observer that becomes associated with a CS. However, this response is also mediated by the observer’s perception of the model, which can be inﬂuenced by various social cognitive processes, such as emotional perspective taking and mental attributions. Indeed, in one of the ﬁrst studies of observational fear learning, Berber (1962) showed that another person’s arm movement in response to a shock acted as an US, but only when the observer believed that it was caused by a shock, not when the model’s arm moved without a shock or when a shock was delivered without arm movements. These ﬁndings support the conclusion that perceptual properties of the learning model interact with the observer’s understanding of the model’s (the demonstrator’s) mental states to instigate an unconditioned response. Similarly, information about another person’s spider phobia can induce an aversive response to a spider that is presented to the allegedly phobic model, even without any physical cues of distress (Hygge & Ohman, 1978), and the affective response in an observer can be modiﬁed by social context (Lanzetta & Englis, 1989; Singer et al., 2006). A recent psychophysiology study supports the idea that mental state attributions (e.g., the experience of pain) to the model during observation can causally impact the observer’s subsequent responses to the CS (Olsson, Ochsner, & Phelps, 2007). Mental state attributions may, in turn, be dependent on social factors, such as familiarity, relatedness, social status, and interpersonal learning history. Indeed, mice observing a familiar, but not an unfamiliar, mouse experiencing pain displayed enhanced sensitization to pain on a later test (Langford et al., 2006).

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

The intrinsic aversiveness of observing a conspeciﬁc in pain is evidenced by the willingness of monkeys to starve themselves if a shock is administered to a fellow monkey every time the observer attempts to eat (Masserman, Wechkin, & Terris, 1964); but again, this altruistic behavior is inﬂuenced by familiarity and past experience of the conspeciﬁc (Hauser et al., 2003; Masserman, Wechkin, & Terris, 1964). Relating back to the example at the inception of this chapter, the child might have had a different learning experience if it instead watched a fearful other, such as your neighbor, especially if the threat was your raging father. For sure, in such an alternative scenario, the child would have learned something quite differently both from and about the parent. Taken together, available research highlights both similarities and differences between conditioned and observational fear learning. Although some studies on observational learning in rats have failed to replicate various phenomena that are predicted from the principles guiding classical conditioning, including blocking, overshadowing, and latent inhibition (Bennet, Galef & Durlach, 1993), such phenomena have been replicated in primate observational learning (Bandura, 1968; Lanzetta & Orr, 1980; Mineka & Cook, 1993). It is possible that social-emotional learning shows greater interspecies variability than does classical conditioning. In addition, the greater reliance on well-developed systems for perceiving and signaling emotions through facial expressions in primates might make observational fear learning more similar to conditioning in primates as compared to other social species. Two Interacting Pathways in Observational Fear Learning

The studies reviewed so far suggest that fear learning through observation is supported by two interacting pathways mediating classical conditioning and social cognition, respectively. The role of conditioning is highlighted by work on observational fear learning in primates (Mineka & Cook, 1993; Olsson & Phelps, 2004; Olsson et al., 2007), showing that a conspeciﬁc’s

20-Schachtman-20.indd 467

467

expression of distress can be intrinsically aversive, which suggests that somatosensory representations may be reﬂexively triggered by mere observation of another individual’s emotional display without necessarily being accompanied by higher order social cognition. The same conclusion is supported by the demonstration of unconscious behavioral mimicry (Dimberg, Thunberg, & Elmehed, 2000) and physiological responses to subliminally presented faces (Ohman & Mineka, 2001; Whalen et al., 1998). Finally, dual-representation models of emotion perception and empathy in humans as described earlier (Dimberg, Thunberg, & Elmehed, 2000; Gallese, Keysers, & Rizzolatti, 2004; Preston & de Waal, 2002) provide additional support of the conditioning account. The importance of social cognition is underscored by the fact that factors related to the social context, shared representations of emotional states, and reﬂections about others’ mental states may moderate the ensuing learning. Indeed, as described earlier, affective responses to emotional faces and their recruitment of the amygdala depend on the context provided (Kim et al., 2004) and on cognitive appraisals by means of prefrontal brain systems (Lamm et al., 2007; Ochsner & Gross, 2005). Moreover, basic emotional responses to another’s distress are affected by interpersonal learning history and the goals of the observer. For example, an observer’s affective response to another’s distress depends on whether the other person is expected to cooperate or compete in a future interactive game (Lanzetta & Englis, 1989), and imaging work shows that brain regions, such as the AI and ACC, are modulated accordingly (Singer et al., 2006). These changes in affective response to another’s distress mediated by social cognition are also likely to inﬂuence the ensuing learning. Providing preliminary support of this conclusion, an imaging study on observational fear learning (Olsson et al., 2007, Figs. 20.3a, b) found activation in the AI and ACC both during observation of another person receiving shocks paired with a CS and in the later test stage when the person being imaged expected to receive shocks paired with the same stimulus, indicating that regions linked to empathy may be involved

2/15/2011 7:53:54 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS Conditioned response

468

3.0

b

3.0

2 r = 0.61

1

1.0 0.0 –2.0 0.0

2.0 1.0 0.0 –1.0

r = 0.67

2.0

0.0 1.0 Betas

2.0

Conditioned response

Conditioned response

a

2.0 4.0 Betas

6.0

3.0 r = 0.49 2.0 1.0 0.0 –2.0

0.0 2.0 Betas

4.0

Figure 20.3 Brain activation during observational fear learning predicts the level of expressed fear at a

subsequent time. Functional activation maps displaying brain activation during an observational fear learning task (Olsson, Nearing, & Phelps, 2007); (c) a coronal view of activation in the right AI when observing a learning model’s pain response to a shock. The adjacent graph shows that the magnitude of this activation predicts the strength of the conditioned response (indexed by the skin conductance response) at a later time to a cue associated with the learning model’s pain. (d) A saggital view of activation in the (1) mPFC and (2) ACC during the observation a learning model’s pain response to a shock. As in (c), adjacent graphs display the positive relationship between magnitude of activation during observation and subsequent conditioned response. Modiﬁed from Olsson, A & Ochsner, K. N. (2008). The Relationship Between Emotion and Social Cognition. Trends in Cognitive Science, 12, 65–71.

in observational fear learning. Important to this context, activation in both these regions (AI and ACC) during observation predicted learning as expressed in the subsequent test stage. In addition, another region of interest, the rostral mPFC, was only activated during the observation stage. Responses in this region marginally predicted the magnitude of subsequent learning, further indicating that social cognition is involved in observational learning of fear. The conjecture that mental state attributions can be causally related to the learning response following observation was strengthened by a recent study demonstrating that experimental manipulation of the observers’ empathic appraisals through increasing or decreasing empathy with a distressed learning model (demonstrator) affected later learning responses (Olsson, Brodbeck, Bolger, & Ochsner, 2008). To sum up, in accordance with research on non-human animals, the ﬁndings of observational fear learning in humans demonstrate, on the one hand, reﬂexive processes independent of conscious awareness and strategic regulation of affective responses and, on the other, reﬂective

20-Schachtman-20.indd 468

processes dependent on social-cognitive manipulations. It is also clear that these two different levels of learning interact. For example, the amygdala, supporting reﬂexive responses, interacts with frontal cortical mechanisms of shared affective representations and hippocampal representations about context and relevant social information about the learning model (such as social status and familiarity). In addition, these regions receive social-cognitive information from prefrontal cortices, such as the mPFC. Interestingly, although in rodents the ventral mPFC has a role in some social behaviors (Schneider & Koch, 2005), the primate mPFC is likely to be more important in social perception and learning, as shown by deﬁcits in social behavior after prefrontal lesions in both monkeys (Myers, Swett, & Miller, 1973) and humans (Beer, Heerey, Keltner, Scabini, & Knight, 2003). It is worth noting that the more anterior-rostral region of the mPFC is both quantitatively and qualitatively more developed in humans than other primates (Ongür, Ferry, & Price, 2003), implying the intriguing possibility of a unique neural substrate for the support of more complex

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

mental representations that might be involved in human observational learning. In fact, a metaanalysis of imaging studies reports that this region (speciﬁcally Brodmann’s area 10) is especially sensitive to experimental manipulations involving both social and emotional tasks (Gilbert et al., 2006). Instructed Fear Learning in Humans

Humans possess the unique ability to obtain emotional information through language. Whereas fear learning through observation involves visual representation of emotional properties of a stimulus, language is only arbitrarily related to, and thus detached from, its referent in the world. Language forces the receiver to rely on past experiences and internally generated imagery to form an emotional memory. In this process, brain regions involved in linguistic processing are likely to play an important role. In addition, imagery and self-projection into the future are thought to rely on neural systems similar to those involved in perception (Kosslyn & Thompson, 2003) and the construction of episodic memory (Tulving, 2002). Indeed, similar to the recollection of the past, self-projection into the future is impaired after hippocampal lesions (Hassabis, Kumaran, Vann, & Maguire, 2007). In addition, regions of the mPFC implicated in the simulation of future events (Buckner & Carroll, 2007; Schacter & Addis, 2007) overlap with those involved in thinking about others’ minds. These ﬁndings suggest that hippocampal and frontal regions involved in explicit and reﬂective processes are especially important during indirect, social forms of learning. Both clinical accounts that retrospectively target the etiology of phobic fears (King, Gullone, & Ollendick, 1998) and experimental studies on children involving fear provoked through storytelling (Askew & Field, 2008; Field, Argyris, & Knowles, 2001) reveal that verbal instructions can be a strong stimulus for fear learning. Indeed, adults instructed to expect a shock paired with a speciﬁc CS and later exposed to the same CS show learned responses similar to those seen after classical fear conditioning (Grillon, Ameli,

20-Schachtman-20.indd 469

469

Merikangas, Woods, & Davis, 1991; Hugdahl & Ohman, 1977; Olsson & Phelps, 2004; Phelps et al., 2001). These lines of research suggest that, at least partially, instructed fear learning draws on the same neural network as conditioned fear. To examine the neural mechanisms underlying expression of fears acquired through verbal instruction, Phelps and colleagues (Phelps et al., 2001) told subjects that they might receive a shock when shown a particular CS (“threat” stimulus), but not another CS (“safe” stimulus). Supporting extension of the fear-conditioning model to instructed fear, there was robust activation of the left amygdala, which correlated with the physiological expression of fear learning (Fig. 20.1d). Activation of the left insular cortex also correlated with expression of learning. As discussed earlier, the insular cortex is a critical component for conveying a cortical representation of pain to the amygdala (Shi & Davis, 1999) and for subjective awareness of physiological states (Critchley et al., 2004). The verbally mediated learning is likely to have resulted in an abstract cortical representation of the potentially painful shock, which may have been communicated to the amygdala through projections from the insular cortex (Fig. 20.2c). The left lateralization of the activation is consistent with the common view that the left hemisphere is more involved in language processing (Gazzaniga & LeDoux, 1978). However, brain imaging results cannot rule out involvement of the right amygdala, or indicate a critical role for the left amygdala in expression of fears learned through verbal instruction. Further support that the left amygdala mediates physiological expression of instructed fear learning was demonstrated in subjects with unilateral amygdala damage after a similar learning protocol. Those with damage to the left, but not right, amygdala showed an impaired expression of instructed fear (Funayama, Grillon, Davis, & Phelps, 2001). Taken together, these results suggest that there might be both common and unique mechanisms involved in instructed and conditioned fear learning. In an attempt to directly compare fears acquired through conditioning, observation, and verbal instruction, Olsson and Phelps (2004)

2/15/2011 7:53:54 PM

470

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

manipulated the learning procedure, keeping other factors constant. Conditioned stimuli acquired their threat value through being paired with either (1) a shock (conditioning group); (2) an observed fear expression in a learning model (observational group); or (3) the experimenter’s verbal instructions (instructed group). Fear responses to the CS were of comparable magnitude after the three kinds of learning. In addition, replicating previous ﬁndings (Ohman & Mineka, 2001), a subliminally presented (unperceived) CS triggered a response in the fear-conditioning group. The observational, but not the verbally instructed, group also showed a learning response to subliminal presentations of the CS, further supporting the common basis for conditioned and observational fear learning, and indicating that a different mechanism may support learning through language. These results support the notion that there are partially dissociable systems involved in different modes of social-emotional learning. Classical conditioning and observational learning, which humans share with many other species, might be supported by an evolutionarily old system that predates the emergence of language. In contrast, learning based on language is unique to humans and is likely to be, at least initially, dependent on representations in cortical areas that also support conscious processes. Indeed, these ﬁndings indicate that such cortically represented fear associations might depend on conscious awareness, which is in accordance with the observation that the manipulation of conscious awareness can be used to distinguish subdivisions of conditioning (e.g., context and trace conditioning, but not cue and delayed conditioning, are dependent on awareness).

LEARNING ABOUT OTHERS In this chapter, I have focused on the mechanisms underlying learning from others through observation and verbal communication. However, as announced at the outset, we are also frequently learning about others. Indeed, a good share of our waking hours are spent learning emotionally important information about others, either indirectly, for example, through

20-Schachtman-20.indd 470

gossiping, or directly through interaction (Dunbar, 2003). In the fearful episode at the outset of this chapter, the child acquired emotionally signiﬁcant information about both the parent (e.g., anxious disposition) and the neighbor (e.g., aggressive disposition). How does this kind of learning differ from learning about, for instance, a colored square or a tone? In light of the discussion about how social cognition affects our learning from someone else, it is likely that such cognitions also affect the way we develop likes or dislikes of our fellow humans. The two types of social cognitive processes (reﬂexive and reﬂective) that support learning from others also provide crucial diagnostic information about others’ stable social-emotional dispositions (e.g., aggressive personality) as well as their current intentions (e.g., the intention to harm). Consider a social interaction that unfolds over time. During the initial moments of contact, stimulus-driven systems might assess the affective value of social targets in a reﬂexive manner. For example, the rapid evaluation of either potentially threatening and untrustworthy or attractive faces activates either the amygdala and AI regions implicated in aversive learning, as described earlier (Adolphs, 2009; Cunningham et al., 2003; Todorov, Said, & Engell, 2008; Winston et al., 2002) or striatal regions implicated in reward and reinforcement learning (O’Doherty et al., 2004). Other social categories, such as gender, age, and racial belonging are also rapidly coded (Greenwald et al., 2002). This initial processing is followed by more reﬂective processing of the social target in terms of mental state attributions. These and other ways of categorizing other individuals based on social categories may also have an impact on how we learn about them. Interestingly, a line of research that has until recently not been connected to the study on social cognition, has shown that all CS are not created equal in the sense that certain stimuli are more easy to aversively condition than others. For example, studies on so-called prepared fear conditioning have found that conditioned responses to certain natural categories of fear-relevant stimuli (e.g., snakes and angry faces) as compared to fear-irrelevant stimuli (e.g., birds and happy faces) resist extinction

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

(Cook, Hodes, & Lang, 1986; Ohman & Mineka, 2001; see also Chapter 18, this volume). These observations, combined with the superior fear conditioning observed in non-human animals to certain types of ecologically relevant stimuli, has led researchers to posit that these particular stimuli may be prepared by evolution to engage in aversive associations (Ohman & Mineka, 2001; Seligman, 1971). These lines of work led Olsson and colleagues (Olsson et al., 2005) to ask whether similarly biased learning processes might support the acquisition of certain social group biases frequently reported in research on social cognition (Greenwald et al., 2002). Using classical conditioning, they showed that conditioned fear responses to an unknown, neutral looking male belonging to a racial outgroup versus ingroup was more resistant to extinction (Olsson et al., 2005). These results held true for both African American and Caucasian subjects who displayed similar results. A follow-up study showed that this conditioning bias is found only when male outgroup faces are used as CS (Navarrete et al., 2009). Of course, the affective value of a social target is also determined both by their dispositions and by situational variables (Gilbert, 1998). Turning back to the example at the outset of this chapter, the identical behavior of your neighbor might be taken as aggressive or playful depending on your assessment of his intent. In turn, such interpretations might affect the way the episode is learned from and later remembered. Supporting this prediction, a recent study using male faces demonstrated a similarly persistent conditioned fear response to facial images belonging to individuals believed to intentionally as compared to unintentionally inﬂict harm to the subject through delivering a mild electric shock (Olsson et al., 2008). This and other suggestive ﬁndings (Gray & Wegner, 2008; Young, Cushman, Hauser, & Saxe, 2007) indicate that attributions about intent can also impact how we develop fears of others. In many of the studies discussed thus far, the subjects were learning to fear others through the pairing with the direct presentations of an electro-tactile US (shock). However, when we learn about others, we often do this through the consequences of their behavior during an interaction.

20-Schachtman-20.indd 471

471

With repeated interaction with other individuals, imaging studies suggest that the striatum might encode which actions produce desired social outcomes, consistent with its general role in reinforcement learning (Delgado, 2007; O’Doherty et al., 2004). In such interactive games, activity in the striatum accompanies the development of both cooperation and trust (Delgado, 2007; Delgado, Frank, & Phelps, 2005; King-Casas et al., 2005; Sanfey, Loewenstein, McClure, & Cohen, 2006), but it is also involved in punishing a previously unfair partner (Quervain, Fischbacher, Treyer, Schellhammer, Schnyder, Buck, 2004) or learning that they are in pain (Singer et al., 2006). More recent work has begun to test learning models combining knowledge of reinforcement learning and mental state attributions to describe the development of strategic interactions (Hampton, Bossaerts, & O’Doherty, 2008). In summary, the amygdala is likely to be involved in the rapid, reﬂexive processing of stimuli-based categories, such as sex and race cues, followed by the involvement of ventral regions of the frontal cortex during mental state attributions (e.g., my neighbor is angry), and the striatal activation to regulate moment-to-moment behavior beyond the ﬁrst impression (e.g., I should avoid him). In contrast, more dorsal regions of the mPFC might support explicit reﬂections about the consequences of actions, as evidenced by its activation during strategic games (Quervain, Fischbacher, Treyer, Schellhammer, Schnyder, Buck, 2004; Gallagher, Jack, Roepstorff & Frith, 2002).

A NEURAL MODEL OF SOCIAL FEAR LEARNING Social-emotional learning offers the opportunity to study transmission of biologically relevant information between individuals. Indeed, some have argued that social learning at large may lie at the core of the forces that create and maintain culture (Plotkin & Odling-Smee, 1981; Whiten, Horner, & de Waal, 2005), which might then have an impact on biological evolution (Danchin, Giraldeau, Valone, & Wagner, 2004; Plotkin & Odling-Smee, 1981). A model of

2/15/2011 7:53:54 PM

472

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

social-emotional learning based on existing knowledge of the neurobiology of classical conditioning and social cognition might serve as an important bridge between biological principles of learning and cultural evolution. In addition, such a model might also provide an ecologically improved account to support the understanding of how clinical fears and anxiety develop and are maintained. Next, I will discuss a suggestive model of social-emotional learning originally proposed by Olsson and Phelps (2007). This framework outlines the relationship between neural mechanisms underlying fear conditioning and two forms of social learning—observational and instructed fear—and how these might be modulated by both reﬂexive and reﬂective processes. The model is centered on the amygdala, which is critical to physiological expression of learned fear, regardless of how learning is acquired. As outlined earlier, in classical fear conditioning (Fig. 20.2a), information about the CS is communicated to the lateral nucleus of the amygdala by way of the sensory cortices and thalamus; this information converges with US input from the somatosensory cortex and thalamus. Through synaptic plasticity in the lateral nucleus, the CS-US association is formed. In addition, a distributed cortical representation of the CS-US contingency is acquired through the hippocampal memory system and may be expressed in regions associated with pain processing, such as the ACC and insular cortex. In the presence of the CS, learned fear is expressed through projections from the lateral nucleus to the central nucleus, which in turn mediates autonomic expression. This might be the way the child’s parent in the example acquires a fear of the neighbor—through direct, ﬁrsthand aversive experiences. Other means of expression may depend on other pathways (LeDoux & Gorman, 2001). In addition, projections from the cortical representation of the CS-US contingency to the amygdala may contribute to autonomic expression of fear learning when there is subjective awareness of the CS-US contingency. The model further proposes that the mechanisms underlying learning through social observation (Fig. 20.2b) may be similar, with a few exceptions.

20-Schachtman-20.indd 472

For example, the US in observational fear learning may be the perceived fear expression of a conspeciﬁc, such as that in the attacked parent. A representation of the fearful face is conveyed to the lateral nucleus through the sensory cortices and possibly the sensory thalamus. Importantly, the representation of the strength of the US in the lateral nucleus may be modiﬁed by cortical representation of empathic pain through input from the ACC and insular cortex and the perception and interpretation of the learning model’s mental state during the observed painful experience as supported by the mPFC. The model suggests that, similar to classical fear conditioning, the lateral nucleus is a site of plasticity underlying memory for the CS-US association, in addition to a distributed cortical representation of the CS-US association acquired through the hippocampal memory system. The output mechanism for observational fear learning does not differ from that for fear conditioning. Finally, the model poses that fears that are acquired through verbal communication (Fig. 20.2c) rely on a somewhat different representation, because of its symbolic nature. It is unlikely that abstract representations of verbal threat are represented in subcortical structures, such as the amygdala. Although sensory information about the CS is conveyed to the lateral nucleus, the association between the CS and the verbal threat is likely to be represented solely in a distributed cortical network. Furthermore, this cortical representation is leftlateralized, reﬂecting the verbal nature of the threat. In accordance with the model, memory for this cortical association depends on the hippocampal complex for acquisition, and plasticity in the amygdala is not necessary. Still, autonomic expressions of instructed fears are modulated through communication of the cortical representation of the CS-US association and the potential for pain to the amygdala, perhaps by way of the insular cortex. As with other means of fear learning, the central nucleus is believed to mediate autonomic expression of instructed fear. Following the same logic as in the model outlined here, the representation of a social CS, such as the appearance of your neighbor in the initial

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

example, is likely to be affected by social cognitive processes. It is, for example, possible that brain regions involved in the rapid categorization of your neighbor in terms of sex, race, and trustworthiness assign an initial valence to him as a social CS. Then, these initial evaluations are likely to be modiﬁed trough further learning experiences by the inﬂuence of regions supporting reﬂective social cognition, such as the attribution of intent, and in the case of social interaction, brain regions supporting reinforcement learning. As highlighted by Olsson and Phelps (2007), the presented model remains speculative and should be viewed in light of some important caveats. First, the striatum is not highlighted in the model. Human brain-imaging studies on both conditioned (Büchel, Morris, Dolan, & Friston, 1998; LaBar, Gatenby, Gore, LeDoux, & Phelps, 1998) and social (Olsson et al., 2007; Phelps et al., 2001) fear learning report activation of the striatum. As we have seen, this region is also shown to be involved in learning through social interaction (Hampton et al., 2008; O’Doherty et al., 2004). Second, the model highlights unidirectional projections between brain regions, but most of the regions we have discussed have bidirectional connections with the amygdala. Third, this framework outlines how fear learning is initially expressed after social and nonsocial means of acquisition. Once a CS is experienced and a fear reaction occurs, further learning may result, which could change the nature of the representation further. For example, in instructed fear, co-occurrence of the CS and autonomic arousal may cause the CS to act as a secondary reinforcer, which projects its emotional salience to the lateral nucleus to facilitate an amygdala-dependent representation of the CS-threat association that was not present after initial verbal instruction. In this way, representation of verbally communicated fears may change over time and be experienced to be more similar to conditioned fears. Despite these caveats, the proposed framework represents a neural model that may help us to better understand the complexity and subtlety of human fear learning in a social and cultural environment.

20-Schachtman-20.indd 473

473

CONCLUDING REMARKS In this chapter, I have attempted to bring together relevant lines of work on the behavioral and neurobiological level from two domains of research: emotional (fear) learning and social cognition with the aim to better understand emotional learning in social situations in which we learn from and about others. As we have seen, brain regions in the two greater networks supporting emotional learning and social cognitions are partially overlapping, suggesting important commonalities. Investigations directly targeting social-emotional learning have mostly validated classical conditioning as a model for emotional learning through social means. However, in light of the existing literature, I have argued for an extension of this model to account for the socialcognitive aspects of social-emotional learning. To this aim, the model by Olsson and Phelps (2007) provides a ﬁrst, tentative step that needs further validation through continued work. Future research on social-emotional learning will also hopefully provide important knowledge about the underlying socio-emotional impairments that are hallmarks of many psychological disorders, such as phobias and anxiety disorders, characterized by dysfunctional assignment of emotional value to certain stimuli and situations. A next important step will be to improve our understanding of how dysfunctional learning can be changed. In other words, how can we help individuals who have developed maladaptive responses to their social environment due to adverse emotional experiences, be they caused by war, abusive parents, or raging neighbors? On a wider scale, a better understanding of the neural mechanisms supporting social-emotional learning is essential to integrate our current knowledge about the biological foundations of learning with an understanding of cultural change and evolution at large.

REFERENCES Adolphs, R. (2009). The social brain: Neural basis of social knowledge. Annual Review of Psychology, 60, 693–716. Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005).

2/15/2011 7:53:54 PM

474

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

A mechanism for impaired fear recognition after amygdala damage. Nature, 433, 68–72. Amaral, D. G. (1986). Amygdalohippocampal and amygdalocortical projections in the primate brain. Advances in Experimenta Medicine and Biology, 203, 3–17. Ames, D. L., Jenkins, A. C., Banaji, M. R., & Mitchell, J. P. (2008). Taking another person’s perspective increases self-referential neural processing. Psychological Science, 19, 642–644. Amodio, D. M., & Frith, C. D. (2006). Meeting of minds: The medial frontal cortex and social cognition. Nature Reviews Neuroscience, 7, 268–277. Anderson, A.K., & Phelps, E.A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411, 305–309. Askew, C., & Field A. P. (2008). The vicarious learning pathway to fear 40 years on. Clinical Psychology Review, 28, 1249–1265. Bandura, A. (1977). Social learning theory. New York, NY: General Learning Press. Bandura, A., & Menlove, F. L. (1968), Factors determining vicarious extinction of avoidance behavior through symbolic modeling. Journal of Personality and Social Psychology, 8, 99–108. Baron-Cohen, S., Ring, H. A., Bullmore, E. T., Wheelwright, S., Ashwin, C., & Williams, S. C. (2000). The amygdala theory of autism. Neuroscience and Biobehavioral Reviews, 24, 355–364. Barton, R. A., Aggleton, J. P., & Grenyer, R. (2003). Evolutionary coherence of the mammalian amygdala. Proceedings of the Royal Society of London B: Biological Sciences, 270, 539–543. Batson, C. D., Thompson, E. R., & Chen, H. (2002). Moral hypocrisy: Addressing some alternatives. Journal of Personality and Social Psychology, 83, 330–339. Bechara, A., Tranel, D., Damasio, H., Adolphs, R., Rockland, C., & Damasio, A. R. (1995). Double dissociation of conditioning and declarative knowledge relative to the amygdala and hippocampus in humans. Science, 269, 1115–1118. Beer, J. S., Heerey, E. A., Keltner, D., Scabini, D., & Knight, R. T. (2003). The regulatory function of self-conscious emotion: Insights from patients with orbitofrontal damage. Journal of Personality and Social Psychology, 85, 594–604. Bennet, G., Galef, B. G., & Durlach, P. J. (1993). Absence of blocking, overshadowing, latent inhibition in social enhancement of food

20-Schachtman-20.indd 474

preferences. Animal Learning and Behavior, 21, 214–220. Berber, S. M. (1962). Conditioning through vicarious instigation. Psychology Review, 69, 450–466. Bishop, S. J. (2008). Neural mechanisms underlying selective attention to threat. Annals of the New York Academy of Science, 1129, 141–152. Blair, H. T., Schafe, G. E., Bauer, E. P., Rodrigues, S. M., & LeDoux, J. E. (2001). Synaptic plasticity in the lateral amygdala: A cellular hypothesis of fear conditioning. Learning and Memory, 8, 229–242. Blair, R. J., Colledge, E., Murray, L., & Mitchell, D. G. (2001). A selective impairment in the processing of sad and fearful expressions in children with psychopathic tendencies. Journal of Abnormal Child Psychology, 29, 491–498. Büchel, C., Morris, J., Dolan, R. J., & Friston, K. J. (1998). Brain systems mediating aversive conditioning: An event-related fMRI study. Neuron, 20, 947–957. Buckner, R. L., & Carroll, D. C. (2007). Selfprojection and the brain. Trends in Cognitive Sciences, 11, 49–57. Byrne, R. W., & Whiten, A. (1988). Machiavellian intelligence. Oxford, England: Oxford University Press. Canli, T., & Lesch, K. P. (2007). Long story short: The serotonin transporter in emotion regulation and social cognition. Nature Neuroscience, 10, 1103–1109. Castelli, E., Happe, F., Frith, U., & Frith, C. (2000). Movement and mind: A functional imaging study of perception and interpretation of complex intentional movement. NeuroImage, 12, 314–325. Cook, E. W., Hodes, R. L., & Lang, P. J. (1986). Preparedness and phobia: Effects of stimulus content on human visceral conditioning. Journal of Abnormal Psychology, 95, 195–207. Craig, A. D. (2009). How do you feel now? The anterior insula and human awareness. Nature Reviews Neuroscience, 10, 59–70. Critchley, H. D., Wiens, S., Rotshtein, P., Ohman, A., & Dolan, R. J. (2004). Neural systems supporting interoceptive awareness. Nature Neuroscience, 7, 189–195. Cunningham, W. A., Johnson, M. K., Gatenby, J. C., Gore, J. C., & Banaji, M. R. (2003). Neural components of social evaluation. Journal of Personality and Social Psychology, 85, 639–649.

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

Cunningham, W. A., Johnson, M. K., Raye, C. L., Gatenby, J., Gore, J. C., & Banaji, M. R. (2004). Separable neural components in the processing of Black and White faces. Psychological Science, 15, 806–813. Curio, E. (1988). Cultural transmission of enemy recognition by birds. In T. R Zentall & B. G. Galef, Jr. (Eds), Social learning: Psychological and biological perspectives (pp. 75–97). Hillsdale, NJ: Erlbaum. Danchin, E., Giraldeau, L. A., Valone, T. J., & Wagner, R. H. (2004). Public information: From nosy neighbors to cultural evolution. Science, 305, 487–491. Davis, M., Barad, M., Otto, M., & Southwick, S. (2006). Combining pharmacotherapy with cognitive behavioral therapy: Traditional and new approaches. Journal of Traumatic Stress, 19, 571–581. Davis, M., & Whalen, P. J. (2001). The amygdala: Vigilance and emotion. Molecular Psychiatry, 6, 13–34. Decety, J., & Jackson, P. L. (2004). The functional architecture of human empathy. Behavioral and Cognitive Neuroscience Reviews, 3, 71–100. de Gelder, B., Snyder, J., Greve, D., Gerard, G., & Hadjikhani, N. (2004). Fear fosters ﬂight: A mechanism for fear contagion when perceiving emotion expressed by a whole body. Proceedings of the National Academy of Sciences USA, 101, 16701–16706. de Quervain, D. J., Fischbacher, U., Treyer, V., Schellhammer, M., Schnyder, U., Buck, A., et al. (2004). The neural basis of altruistic punishment. Science, 305, 1254–1258. Delgado, M. R., Frank, R. H., & Phelps, E. A. (2005). Perceptions of moral character modulate the neural systems of reward during the trust game. Nature Neuroscience, 8, 1611–1618. Delgado, M. R., Li, J., Schiller, D., & Phelps, E. A. (2008). The role of the striatum in aversive learning and aversive prediction errors. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 363, 3787–3800. Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconscious facial reactions to emotional facial expressions. Psychological Science, 11, 86–89. Dunbar, R. (2003). Psychology. Evolution of the social brain. Science, 302,1160–1161. Egliston KA & Rapee RM. (2007). Inhibition of fear acquisition in toddlers following positive modelling by their mothers. Behaviour research and therapy, 45, 1871–182.

20-Schachtman-20.indd 475

475

Ekman, P. (1982). Emotion in the human face. New York, NY: Cambridge University Press. Eichenbaum, H., & Cohen, N. J. (2002). From conditioning to conscious recollection: Memory systems of the brain. Oxford, England: Oxford University Press. Eisenberger, N. I., Lieberman, M. D., & Williams, K. D. (2003). Does rejection hurt? An FMRI study of social exclusion. Science, 302, 290–292. Esteves, F., Parra C., Dimberg U., & Ohman A. (1994). Nonconscious associative learning: Pavlovian conditioning of skin conductance responses to masked fear-relevant facial stimuli. Psychophysiology, 31, 375–385. Everitt, B. J., & Robbins, T. W. (1992). Amygdalaventral striatal interactions and reward-related processes. In J. P. Aggleton (Ed.), The amygdala: Neurobiological aspects of emotion, memory, and mental dysfunction (pp. 401–429). New York, NY: Wiley-Liss. Field, A. P., Argyris, N. G., & Knowles, K. A. (2001). Who’s afraid of the big bad wolf: A prospective paradigm to test Rachman’s indirect pathways in children. Behavioral Research Therapy, 39, 1259–1276. Funayama, E.S, Grillon, C., Davis, M., & Phelps, E.A. (2001). A double dissociation in the affective modulation of startle in humans: Effects of unilateral temporal lobectomy. Journal of Cognitive Neuroscience, 13, 721–729. Galef, B. G., & Laland, K. N. (2005). Social learning in animals: Empirical studies and theoretical models. BioScience, 55, 489–499. Gallese, V., Keysers, C., & Rizzolatti, G. (2004). A unifying view of the basis of social cognition. Trends in Cognitive Sciences, 8, 396–403. Gallagher, H. L., Jack, A. I., Roepstorff, A., & Frith, C. D. (2002). Imaging the intentional stance in a competitive game. Neuroimage, 16, 814–821. Gazzaniga, M. S., & LeDoux, J. E. (1978). The integrated mind. New York, NY: Plenum Press. Gerull, F. C., & Rapee, R. M. (2002). Mother knows best: Effects of maternal modeling on the acquisition of fear and avoidance behavior in toddlers. Behaviour Research and Therapy, 40, 279–287. Gilbert, D. T. (1998). Ordinary personology (4 ed.). New York: McGraw Hill. Gilbert, S. J., Spengler, S., Simons, J. S., Steele, J. D., Lawrie, S. M., Frith, C. D., & Burgess, P. W. (2006). Functional specialization within rostral

2/15/2011 7:53:54 PM

476

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

prefrontal cortex (area 10): A meta-analysis. Journal of Cognitive Neuroscience, 18, 932–948. Gray, K., & Wegner, D. M. (2008). The sting of intentional pain. Psychological Science, 19, 1260–1262. Green, G., & Osborne, J. G. (1985). Does vicarious instigation provide support for observational learning theories? A critical review. Psychological Bulletin, 97, 3–77. Greenwald, A. G., Banaji, M. R., Rudman, L. A., Farnham, S. D., Nosek, B. A., & Mellott, D. S. (2002). A uniﬁed theory of implicit attitudes, stereotypes, self-esteem, and self-concept. Psychaology Review 109. 3–25. Grillon, C., Ameli, R., Merikangas, K., Woods, S. W., & Davis, M. (1991). Fear-potentiated startle: Effects of anticipatory anxiety on the acoustic blink reﬂex. Psychophysiology, 28, 588–595. Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2008). Neural correlates of mentalizing-related computations during strategic interactions in humans. Proceedings of the National Academy of Sciences USA, 105, 6741–6746. Harris, L. T., & Fiske, S. T. (2006). Dehumanizing the lowest of the low: Neuroimaging responses to extreme out-groups. Psychological Science, 17, 847–853. Harris, L. T., Todorov, A., & Fiske, S. T. (2005). Attributions on the brain: Neuro-imaging dispositional inferences, beyond theory of mind. NeuroImage, 28, 763–769. Hassabis, D., Kumaran, D., Vann, S. D., & Maguire, E. A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proceedings of the National Academy of Sciences USA, 104, 1726–1735. Hauser, M. D. (1996). The evolution of communication. Cambridge, MA: MIT Press. Hauser, M. D., Chen M. K., Chen, F., & Chuang, E. (2003). Give unto others: Genetically unrelated cotton-top tamarin monkeys preferentially give food to those who altruistically give food back. Proceedings of the Royal Society of London B: Biological Sciences, 270, 2363–2370. Hettema, J. M., Annas, P., Neale, M. C., Kendler, K. S., & Fredrikson, M. (2003). A twin study of the genetics of fear conditioning. Archives of General Psychiatry, 60, 702–708. Heyes, C. M. (1993). Imitation without perspective-taking. Behavioral and Brain Sciences, 16, 524–525. Hooker, C. I., Germine, L. T., Knight, R. T., & D’Esposito, M. (2006). Amygdala response to

20-Schachtman-20.indd 476

facial expressions reﬂects emotional learning. Journal of Neuroscience, 26, 8915–8922. Hopkins, D. A., & Holstege, G. G. (1978). Amygdaloid projections to the mesencephalon, pons and medulla oblongata in the cat. Experimental Brain Research 32, 529–547. Hutchison, W. D., Davis, K. D., Lozano, A. M., Tasker, R. R., & Dostrovsky, J. O. (1999). Painrelated neurons in the human cingulated cortex. Nature Neuroscience, 2, 403–405. Hygge, S., & Ohman, A. (1978). Modeling processes in the acquisition of fears: Vicarious electrodermal conditioning to fear-relevant stimuli. Journal of Personality and Social Psychology, 36, 271–279. Iacoboni, M. Dapretto, M. (2006). The mirror neuron system and the consequences of its dysfunction. Nature Review Neuroscience, 7, 942–951. John, E. R., Chesler, P., Bartlett, F., & Victor, I. (1968). Observation learning in cats. Science, 59, 1489–1491. Johansson, P., Hall, L., Sikstrom, S., & Olsson, A. (2005). Failure to detect mismatches between intention and outcome in a simple decision task. Science, 310, 116–119. Kanwisher, N., & Yovel, G. (2006). The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society London B: Biological Sciences, 361, 2109–2128. Kavaliers, M., Choleris, E., & Colwell, D. D. (2001). Learning from others to cope with biting ﬂies: Social learning of fear-induced conditioned analgesia and active avoidance. Behavioral Neuroscience, 115, 661–674. Keller, M., Perrin, G., Meurisse, M., Ferreira, G., & Levy, F. (2004). Cortical and medial amygdala are both involved in the formation of olfactory offspring memory in sheep. European Journal of Neuroscience, 20, 3433–3441. Kim, H., Somerville, L. H., Johnstone, T., Polis, S., Alexander, A. L., Shin, L. M., & Whalen, P. J. (2004). Contextual modulation of amygdala responsivity to surprised faces. Journal of Cognitive Neuroscience, 16, 1730–1745. King, N. J., Gullone, E., & Ollendick, T. H. (1998). Etiology of childhood phobias: Current status of Rachman’s three pathways theory. Behaviour Research and Therapy 36, 297–309. King-Casas, B., Tomlin, D., Anen, C., Camerer, C. F., Quartz, S. R., & Montague, P. R. (2005). Getting to know you: Reputation and

2/15/2011 7:53:54 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

trust in a two-person economic exchange. Science, 308, 78–83. Klüver, H., & Bucy, P. (1939). Preliminary analysis of functioning of the temporal lobes in monkeys. Archives of Neurological Psychiatry, 42, 979–1000. Kosslyn, S. M., & Thompson, W. L. (2003). When is early visual cortex activated during visual mental imagery? Psychological Bulletin, 129, 723–746. Kross, E., Egner, T., Ochsner, K., Hirsch, J., & Downey, G. (2007). Neural dynamics of rejection sensitivity. Journal of Cognitive Neuroscience, 19, 945–956. LaBar, K. S., Gatenby, J. C., Gore, J. C., LeDoux, J. E., & Phelps, E. A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: A mixed-trial fMRI study. Neuron, 20, 937–945. LaBar, K. S., LeDoux, J. E., Spencer, D. D., & Phelps, E. A. (1995). Impaired fear conditioning following unilateral temporal lobectomy in humans. Journal of Neuroscience, 15, 6846–6855. LaBar, K. S., & Phelps, E. A. (2005). Reinstatement of conditioned fear in humans is context dependent and impaired in amnesia. Behavioral Neuroscience, 119, 677–686. Lamm, C., Batson, C. D., & Decety, J. (2007). The neural substrate of human empathy: Effects of perspective-taking and cognitive appraisal. Journal of Cogntive Neuroscience, 19, 42–58. Langford, D. J., Crager, S. E., Shehzad, Z., Smith, S. B., Sotocinal, S. G., Levenstadt, J. S., Chanda, M. L., Levitin, D. J., & Mogil J. S. (2006). Social modulation of pain as evidence for empathy in mice. Science, 312, 1967–1970. Lanzetta, J. T., & Englis, B. G. (1989). Expectations of cooperation and competition and their effects on observers’ vicarious emotional responses. Journal of Personality and Social Psychology 56, 543–554. Lanzetta, J. T., & Orr, S. P. (1980). Inﬂuence of facial expressions on the classical conditioning of fear. Journal of Personality and Social Psychology 39, 1081–1087. LeDoux, J. E. (2000). Emotion circuits in the brain. Annual Review of Neuroscience, 23, 155–184. LeDoux, J. E., & Gorman, J. M. (2001). A call to action: Overcoming anxiety through active coping. American Journal of Psychiatry, 158, 1953–1955. LeDoux, J. E., Farb, C., & Ruggiero, D. A. (1990). Topographic organization of neurons in the

20-Schachtman-20.indd 477

477

acoustic thalamus that project to the amygdala. Journal of Neuroscience, 10, 1043–1054. Lieberman, M. D. (2007). Social cognitive neuroscience: A review of core processes. Annual Review of Psychology, 58, 259–289. Lissek, S., Levenson, J., Biggs, A.L., Johnson, L.L., Ameli, R., Pine, D.S., & Grillon, C. (2008). Elevated fear conditioning to socially relevant unconditioned stimuli in social anxiety disorder. American Journal of Psychiatry, 165, 124–132. Lonsdorf, T. B., Weike, A. I., Nikamo, P., Schalling, M., Hamm, A. O., & Ohman, A. (2009). Genetic gating of human fear learning and extinction: Possible implications for geneenvironment interaction in anxiety disorder. Psychological Science, 20, 198–206. Masserman, J. H., Wechkin, S., & Terris, W. (1964). “Altruistic” behavior in rhesus monkeys. American Journal of Psychiatry, 121, 584–585. Mineka, S., & Cook, M. (1986). Immunization against the observational conditioning of snake fear in rhesus monkeys. Journal of Abnormal Psychology, 95, 307–318. Mineka, S., & Cook, M. (1993). Mechanisms involved in the observational conditioning of fear. Jounral of Experimental Psychology: General, 122, 23–38. Mineka, S., Davidson, M., Cook, M., & Keir, R. (1984). Observational conditioning of snake fear in rhesus monkey. Journal of Abnormal Psychology, 93, 355–372. Mineka, S., & Zinbarg, R. (2006). A contemporary learning theory perspective on the etiology of anxiety disorders: It’s not what you thought it was. American Psychologist, 61, 10–26. Mitchell, J.P., Heatherton, T.F., & Macrae, C.N. (2002). Distinct neural systems subserve person and object knowledge. Proceedings of the National Academy of Sciences USA, 99, 15238–15243. Mitchell, J. P., Macrae, C. N., & Banaji, M. R. (2006). Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron, 50, 655–663. Monﬁls, M. H., Cowansage, K. K., Klann, E., & LeDoux, J. E. (2009). Extinction-reconsolidation boundaries: Key to persistent attenuation of fear memories. Science, 324, 951–955. Morris, J. S., Ohman, A., & Dolan, R. J. (1998). Conscious and unconscious emotional learning in the amygdala. Nature, 393, 467–470. Morrison, I., Lloyd, D., di Pellegrino, G., & Roberts, N. (2004). Vicarious responses to pain

2/15/2011 7:53:55 PM

478

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

in anterior cingulate cortex: is empathy a multisensory issue? Cogn Affect Behav Neurosci, 4, 270–278. Munafo, M. R., Brown, S. M., & Hariri A. R. (2008). Serotonin transporter (5-HTTLPR) genotype and amygdala activation: A meta-analysis. Biological Psychiatry, 63, 852–857. Munksgaard, L., DePassille, A. M., Rushen, J., Herskin, M. S., & Kristensen, A. M. (2001). Dairy cows’ fear of people: Social learning, milk yield and behaviour at milking. Applied Animal Behaviour Science, 73, 15–26. Myers, R. E., Swett, C., & Miller, M. (1973). Loss of social group afﬁnity following prefrontal lesions in free-ranging macaques. Brain Research, 64, 257–269. Navarrete, C. D., Olsson, A., Ho, A. K., Mendes, W. B., Thomsen, L., & Sidanius, J. (2009). Fear extinction to an out-group face: The role of target gender. Psychological Science, 20,155–158. Ochsner, K. N., & Gross, J. J. (2005). The cognitive control of emotion. Trends in Cognitive Sciences, 9, 242–249. Ochsner, K. N., Knierim, K., Ludlow, D. H., Hanelin, J., Ramachandran, T., & Glover, G., Mackey, S. C. (2004). Reﬂecting upon feelings: An fMRI study of neural systems supporting the attribution of emotion to self and other. Journal of Cogntive Neuroscience, 16, 1746–1772. Ochsner, K. N., Ray, R. D., Cooper, J. C., Robertson, E. R., Chopra, S., Gabrieli, J. D. E., & Gross, J. J. (2004). For better or for worse: Neural systems supporting the cognitive downand up-regulation of negative emotion. NeuroImage, 23, 483–499. O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452–454. Ohman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychology Review, 108, 483–522. Ohman, A., & Soares, J. (1993). On the automatic nature of phobic fear: Conditioned electrodermal responses to masked fear-relevant stimuli. Journal of Abnormal Psychology, 102, 121–132. Olsson, A., Brodbeck, C., Bolger, N., & Ochsner, K. N. (2008, April 12–15). Attributions about others’ intentions affect emotional learning

20-Schachtman-20.indd 478

from their actions. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, San Francisco, CA. Olsson, A., Carmona, S., Katherine R., Downey, G., & Ochsner, K. N. (2007, May 5–8) Perceiving Threat and Learning to Fear: Linking Rejection Sensitivity to Biases in Social Perception and Learning. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, New York City, New York. Olsson, A., Ebert, J. P., Banaji, M. R., & Phelps, E. A. (2005). The role of social groups in the persistence of learned fear. Science, 309, 785–787. Olsson, A., Nearing, K. I., & Phelps, E. A. (2007). Learning fears by observing others: The neural systems of social fear transmission. Social Cognitive and Affective Neuroscience, 2, 3–11. Olsson, A., & Ochsner, K. N. (2008). The relationship between emotion and social cognition. Trends in Cognitive Sciences, 12, 65–71. Olsson, A., Ochsner, K. N., & Phelps, E. A. (2007, May 5–8). The role of the neural circuitry underlying empathy and mental attribution in observational fear learning. Abstract presented at the Annual Meeting of Cognitive Neuroscience Society, New York City, New York. Olsson, A., & Phelps, E.A. (2004). Learned fear of “unseen” faces after Pavlovian, observational, and instructed fear. Psychological Science, 15, 822–828. Olsson, A., & Phelps, E. A. (2007). Social learning of fear. Nature Neuroscience, 10, 1095–1102. Ongür, D., Ferry, A. T., & Price, J. L. (2003). Architectonic subdivision of the human orbital and medial prefrontal cortex. Journal of Comparitve Neurology, 460, 425–449. Pavlov, I. P. (1927). Conditioned reﬂexes. Oxford, England: Oxford University Press. Phelps, E. A., Delgado, M. R., Nearing, K. I., & LeDoux, J. E. (2004). Extinction learning in humans: role of the amygdala and vmPFC. Neuron, 43, 897–905. Phelps, E. A., & LeDoux, J. E. (2005). Contributions of the amygdala to emotion processing: From animal models to human behavior. Neuron, 48, 175–187. Phelps, E.A., Ling, S., Carrasco, M. (2006). Emotion Facilitates Perception and Potentiates the Perceptual Beneﬁts of Attention. Psychological Science,17, 292–299. Phelps, E. A., O’Connor, K. J., Cunningham, W. A., Funayma, E. S., Gatenby, J. C., Gore, J. C., &

2/15/2011 7:53:55 PM

THE IMPACT OF SOCIAL COGNITION ON EMOTIONAL LEARNING

Banaji, M. R. (2000). Performance on indirect measures of race evaluation predicts amygdala activity. Journal of Cogntive Neuroscience, 12, 1–10. Phelps, E. A., O’Connor, K. J., Gatenby, J. C., Gore, J. C., Grillon, C., & Davis, M. (2001). Activation of the left amygdala to a cognitive representation of fear. Nature Neuroscience 4, 437–441. Plotkin, H. C., & Odling-Smee, F. J. (1981). A multiple level model of evolution and its implications for sociobiology. Behavioral and Brain Sciences, 4, 225–268. Preston, S. D., & de Waal, F. B. (2002). Empathy: Its ultimate and proximate bases. Behavioral and Brain Sciences, 25, 1–20. Price, J. L., & Amaral, D. G. (1981). An autoradiographic study of the projections of the central nucleus of the monkey amygdala. Journal of Neuroscience, 1, 1242–1259. Quirk, G. J., Armony, J. L., & LeDoux, J. E. (1997). Fear conditioning enhances different temporal components of tone-evoked spike trains in auditory cortex and lateral amygdala. Neuron, 19, 613–624. Quirk, G. J., Garcia, R., & Gonzalez-Lima, F. (2006). Prefrontal mechanisms in extinction of conditioned fear. Biological Psychiatry, 60, 337–343. Rachman, S. (1968). Phobias: Their nature and control. Springﬁeld, IL: Charles C. Thomas. Rachman, S. (1977). The conditioning theory of fear acquisition: A critical examination. Behaviour Research and Therapy, 19, 439–447. Robbins, T. W. (2005). Chemistry of the mind: Neurochemical modulation of prefrontal cortical function. Journal of Comparative Neurology, 493, 140–146. Rolls, E. T. (1999). The brain and emotion. New York, NY: Oxford University Press. Rolls, E. T., Tovée, M. J., Purcell, D. G., Stewart, A. L., & Azzopardi, P. (1994). The responses of neurons in the temporal cortex of primates and face identiﬁcation and detection. Experimental Brain Research, 101, 473–484. Romanski, L. M., Clugnet, M. C., Bordi, F., & LeDoux, J. E. (1993). Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behavioral Neuroscience, 107, 444–450. Sanfey, A. G., Loewenstein, G., McClure, S. M., & Cohen, J. D. (2006). Neuroeconomics: Crosscurrents in research on decision-making. Trends in Cognitive Sciences, 10, 108–116.

20-Schachtman-20.indd 479

479

Saxe, R., Moran J. M., Scholz, J. K., & Gabrieli, J. D. E. (2006). Overlapping and non-overlapping brain regions for theory of mind and self reﬂection in individual subjects. Social Cognition and Affective Neuroscience, 1, 229–234. Schacter, D. L., & Addis, D. R. (2007). The cognitive neuroscience of constructive memory: Remembering the past and imagining the future. Philosophical Transactions of the Royal Society of London, 362, 773–786. Schneider, M., & Koch, M. (2005). Deﬁcient social and play behavior in juvenile and adult rats after neonatal cortical lesion: Effects of chronic pubertal cannabinoid treatment. Neuropsychopharmacology, 30, 944–957. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. Seligman, M. E. P. (1971). Phobias and preparedness. Behavior Therapy, 2, 307–321. Shi, C., & Davis, M. (1999). Pain pathways involved in fear conditioning measured with fearpotentiated startle: Lesion studies. Journal of Neuroscience, 19, 420–430. Simon, H., Le Moal, M., & Calas, A. (1979). Efferents and afferents of the ventral tegmentalA10 region studied after local injection of [3H] leucine and horseradish peroxidase. Brain Research, 178, 17–40. Singer, T., Seymour, B., O’Doherty, J., Kaube, H., Dolan, R. J., & Frith, C. D. (2004). Empathy for pain involves the affective but not sensory components of pain. Science, 303, 1157–1162. Singer, T., Seymour, B., O’Doherty, J., Stephan, K. E., Dolan, R. J., & Frith, C. D. (2006). Empathic neural responses are modulated by the perceived fairness of others. Nature, 439, 466–469. Todorov, A., Said, C. P., Engell, A. D., & Oosterhof, N. N. (2008). Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, 12, 455–460. Tulving, E. (2002). Episodic memory: From mind to brain. Annual Review of Psychology, 53, 1–25. Vaughan, K. B., & Lanzetta, J. T. (1980). Vicarious instigation and conditioning of facial expressive and autonomic responses to a model’s expressive display of pain. Journal of Personality and Social Psychology, 38, 909–923. Vuilleumier, P., Richardson, M. P., Armony, J. L., Driver, J., & Dolan, R. J. (2004). Nature Neuroscience, 7, 1271–1278. Whalen, P. J. (2007). The uncertainty of it all. Trends in Cognitive Sciences, 11, 499–500.

2/15/2011 7:53:55 PM

480

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998). Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. Journal of Neuroscience, 18, 411–418. White, D. J., & Galef, B. J. (1998). Social inﬂuence on avoidance of dangerous stimuli by rats. Animal Learning and Behavior, 26, 433–438. Whiten, A., Horner, V., & de Waal, F. B. (2005). Conformity to cultural norms of tool use in chimpanzees. Nature, 437, 737–740. Winston, J. S., Strange, B. A, O’Doherty, J., & Dolan, R. J. (2002). Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nature Neuroscience, 5, 277–283.

20-Schachtman-20.indd 480

Young, L., Cushman, F., Hauser, M., & Saxe, R. (2007). The neural basis of the interaction between theory of mind and moral judgment. Proceedings of the National Academy of Sciences USA, 104, 8235–8240. Young, M. P., Scannell, J. W., Burns, G. A., & Blakemore, C. (1994). Analysis of connectivity: Neural systems in the cerebral cortex. Nature Reviews Neuroscience, 5, 227–250. Zaki, J., Ochsner, K. N., Hanelin, H., Wager, T. D., & Mackey, S. (2007). Different circuits for different pain: Patterns of functional connectivity reveal distinct networks for processing pain in self and others. Social Neuroscience, 2, 276–291.

2/15/2011 7:53:55 PM

CHAPTER 21 Effects of Conditioning in Advertising Todd R. Schachtman, Jennifer Walker, and Stephanie Fowler

A large number of advertisements pair the presentation of a product or brand name with another stimulus that possesses affective value. The pairing of these two stimuli can result in a change in behavior (e.g., attitude, purchasing probability, attention to the product in the marketplace) toward the product or brand name. These pairings resemble the procedure of classical conditioning. This chapter discusses some of the research that has been done in the area of conditioning and advertising as well as some of the recent developments in conditioning theory and research that may assist in advertising research and its application. The chapter will address such topics as useful parameters for producing conditioning, the roles of affect and cognition, and the role of awareness; and many potentially relevant conditioning phenomena are discussed that might be of relevance to advertising.

INTRODUCTION Advertisements often pair two events together: the product or brand name with a pleasurable stimulus. The pairing of these two stimuli (sometimes called a “trial” when occurring in an experimental situation) results in a change in behavior (e.g., attitude, purchasing probability, attention to the product in the marketplace) toward the product or brand name. This pairing clearly resembles the procedure of classical conditioning (Pavlov, 1927). During classical conditioning, a neutral stimulus is paired with an event that typically has some affective value for the animal (typically something with biological signiﬁcance, such as food for a hungry animal or a painful event). The neutral stimulus is referred to as the conditioned stimulus (CS), and the event that already has affective value for the organism is the unconditioned stimulus (US). The pairing of these two events results in a change in the organism’s response to the CS.

Using Pavlov’s well-known experiments with dogs as an example (given that Pavlov developed this procedure), a tone might serve as the CS and food can serve as the US. The dogs would, of course, salivate to the food if food is placed in the dog’s mouth. This salivation is an unconditioned response and does not involve any learning. Before the tone and food were paired together, the dog had no tendency to salivate to the tone; but after the two events were paired together, the dog began to salivate to the tone. This latter response is the CR, and it is the measure of conditioning. If the organism makes a CR after such pairings (and if various control conditions rule out other possibilities), then it is assumed that the organism has formed an association between the tone and food. Returning to the case of paired events during advertisements, if the individual changes his or her behavior (attitude change, interest in purchasing the item) in the presence of the product or brand (the CS) as a function of pairings of

481

21-Schachtman-21.indd 481

2/16/2011 5:55:21 PM

482

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

this CS with an affective stimulus (the US), then this behavior can be said to be a conditioned response; and the pairing can reﬂect the development of an association between the product or brand (i.e., the CS) and the US. Note that even if the ad is an “informational” one rather than aiming at conditioning per se (pairing the ad with some pleasurable or attractive stimulus), it is very hard to avoid conditioning during exposure to the ad in that the individual(s) in the ad delivering the information is likely doing so with a pleasant voice; and various cues may be present that can promote conditioning. Gresham and Shimp (1985, p. 11) purported that classical conditioning “is the most widely discussed mechanism of [attitudes towards an ad] on consumers’ brand attitudes.” Interestingly, they go on to discuss the direction of causality of the inﬂuence between the ad and the brand; they mention that the attitude toward the brand can inﬂuence the attitude toward the ad, and the attitude toward the ad can inﬂuence the attitude toward the brand. The former may be more important for mature brands, and the latter may be important for new brands (Gresham & Shimp, 1985). Classical conditioning is a procedure in which two events (stimuli) are presented in the manner just described (e.g., Janiszewaki & Warlop, 1993: Kim, Lim, & Bhargava, 1998); and if a CR occurs, then one can state that classical conditioning as an effect has occurred. As a procedure and effect, classical conditioning is silent with respect to the underlying mechanisms (associative processes, cognitive processes, reﬂexive processes, etc.) that might be responsible for the change in the CR (see also Janiszewski & Warlop, 1993). Some recent work has used the expressions “classical conditioning” or “Pavlovian conditioning” to refer to a particular theoretical process (i.e., expectancy learning, see later section on “Evaluative Conditioning”); but we (and most researchers in the ﬁeld of conditioning) feel it is best to refer to classical or Pavlovian conditioning as an effect. Hence, all advertising that involves the pairing of events and results in a change in a learned response can be said to be instances of classical conditioning. However, one of many mechanisms may be responsible for this effect.

21-Schachtman-21.indd 482

The principles of operant conditioning have also been applied to advertising situations. Winters and Wallace (1970) discuss operant conditioning methodology, and how measures such as choice and giving the participant control over exposure to the ad can provide valuable assessment devices. Reed, McCarthy, Latif, and DeJongh (2002) show an interesting way to examine cues experimentally in the marketplace as a means of testing innovative conditioning phenomena in an assimilated natural environment. Given the role that classical conditioning plays in advertising, it is surprising how few review articles are available that speciﬁcally examine advertising with a focus on conditioning per se. The perusal of dozens of subject indices of various marketing and advertising textbooks and edited volumes produced few entries for “conditioning,” and even fewer chapters speciﬁcally devoted to that topic. McSweeney and Bierley (1984) and van Osselaer (2008) provided a valuable review of classical conditioning effects that could be of use to marketing researchers. Another chapter by Allen and Shimp (1990) provided a worthwhile overview of research on conditioning in advertising and some important methodological considerations (see also Cohen, 1990). Of course, it is quite possible, as Allen and Shimp stated about 20 years ago, that the relationship between conditioning and marketing is still in its early stages of development with respect to research efforts: “Classical conditioning research is in the introductory stage of a potentially gainful life cycle in consumer behavior” (p. 29). Moreover, the mechanisms of conditioning are still being investigated by learning and conditioning researchers, and its role in advertising is still being pursued. Indeed, Kim, Allen, and Kardes (1996, p. 318) noted that a “major reason why advertising researchers have failed to embrace knowledge products of the Pavlovian tradition is that no consensus has emerged about how or why conditioning procedures yield their effects on brand attitude.” The present chapter hopes to follow in the suit of these earlier reviews in providing a discussion of the following: (1) some of the research that has been done in the area of conditioning

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

and advertising; (2) some of the recent developments in conditioning theory and research that may assist in advertising research and its application; (3) some interesting issues that are relevant to the conﬂuence of the ﬁelds of conditioning and advertising. Since we come to this chapter as conditioning researchers rather than marketing researchers, our focus may strike the reader as ﬂavored in that way; and it will fall a bit short of providing an exhaustive scope of ﬁndings and issues in the ﬁeld of marketing. Nonetheless, we expect that some of the points made will be useful. Of course, before we begin with this review, we acknowledge that one can question the value of advertising per se. The effectiveness of advertising, and, therefore, of pairings of a product or brand with attractive stimuli, for a company has been challenged by Ehrenberg (1974, p. 32), who stated that advertising is not particularly effective although cutting it can lose sales for a company. He said that advertising is really used to “reinforce feelings of satisfaction for brands already being used.” D’Souza and Rao (1995, p. 32) similarly claimed that “advertising may be working to simply maintain the status quo [in sales].” If so, then maintaining the status quo for a product in high use may require advertising to keep this high use position. In this way, such a project may make use of the processes underlying conditioning that potentially occur during advertising.

ISSUES CONCERNING THE BEHAVIORIST-COGNITIVE DEBATE AND THE CURRENT STATUS OF CONDITIONING THEORY The putative debate between behaviorism and cognition has been discussed by many marketing researchers (e.g., Allen & Janiszewski, 1989; Allen & Shimp, 1990). The debate arose even within the ﬁeld of psychology because the initial decades of conditioning research began at a time when psychology was dominated by behaviorists. Many behaviorists steer clear of hypothetical constructs such as expectancies, memories, and associations. Later, in the 1970s, the ﬁeld of human learning and memory and that of animal conditioning began a “cognitive revolution”

21-Schachtman-21.indd 483

483

during which, among other things, researchers became interested in how acquired knowledge was organized in memory. Within the area of conditioning theory, the dramatic change toward cognitive theorizing occurred due to at least three factors. First, research in the late 1960s and early 1970s on compound conditioning (when two CSs are present on a conditioning trial) and a phenomenon known as “conditioned inhibition” (Pavlov, 1927; Rescorla, 1969) gave large emphasis to the concept of expectancy (and the interaction between CSs during the formation of such expectancies) in classical conditioning. Second, Rescorla’s work in the 1970s focused extensively on the content of associative learning—what are the representations of the events that are associated (i.e., “what is associated with what?” with respect to the events represented in memory). Finally, there was renewed focus on processes that inﬂuence conditioning besides that of acquisition (e.g., retrieval, rehearsal, motivation, and the reactivation of memories; see Lewis, 1979; Miller, Kasprow, & Schachtman, 1986; Spear, 1978). As discussed in more detail in the ﬁrst chapter to the present volume, all three of these factors or “directions” that the ﬁeld took in the 1970s are very cognitive in nature. Many researchers, indeed, a sizable number of psychologists, do not recognize that the ﬁeld of conditioning and learning has gone through very substantive changes, such as those just described, in the past 35 years. While classical conditioning remains an experimental procedure with a behavioral outcome, the theoretical discourse on the mechanisms of conditioning has taken on a very cognitive focus since the 1970s. The importance of parsimony, Morgan’s Canon, and Occam’s razor certainly remains as a scientiﬁc tool; but some research ﬁndings indicate that animal conditioning effects are best (or only) explained in terms of cognitive processes, such as the activation of representations of events and their interaction with associative mechanisms—as well as nonacquisition types of information processing mentioned earlier (rehearsal, reactivation, etc.). In other words, many conditioning phenomena are only explained by evoking cognitive processes.

2/16/2011 5:55:21 PM

484

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

During the viewing of an advertisement, there are a multitude of types of information processing that may take place. The person can experience a change in the emotional value of the CS (the brand or product) without any cognitive effort or awareness (i.e., implicitly). Alternatively, one can experience a change in the emotional value of the CS (or beliefs about it), and this change can come about with awareness on the part of the individual. The individual can also form an expectancy of the US when the CS occurs (e.g., Baeyens, Crombez, Van Den Bergh, & Eelen, 1988), and this learning may occur with or without awareness. The person can also be aware (or not aware) of the contingency between the CS and the US. All of these processes and/or others can give rise to a conditioned response (e.g., attitude toward a product, change in likelihood of purchasing a product). One valuable goal of conditioning research and advertising research is to design studies to examine the nature of the processes underlying the acquisition of information and the elicitation of the CR when exposed to a CS used in advertisements.

SOME OF THE EARLIER WORK EXAMINING CONDITIONING DURING ADVERTISING Allen and Shimp (1990) and Cohen (1990) summarized many of the research ﬁndings through to the date of their writing, and so interested readers can turn to those resources. However, some of those ﬁndings will be summarized brieﬂy here; and we mention a few valuable points about them. The present discussion will be a far cry from any kind of exhaustive presentation of the research in this area; but, rather, we will provide a description of a few studies and ﬁndings to “set the table” a little before we provide information about speciﬁc topics. A description of these early studies will highlight, albeit brieﬂy, examples of procedures as well as theoretical issues involved in such research. As Allen and Shimp (1990) point out in their review, Staats and Staats (1957, 1958) conducted a very early study and found that awareness is not necessary for learning an association. Since we will brieﬂy discuss the issue of awareness and

21-Schachtman-21.indd 484

conditioning later in this chapter, we will keep our comments about this topic even more brief for the time being. The study by Gorn (1982) was innovative in some respects in that it was a relatively early paper; and yet it discussed many of the issues that are critical to research examining the effects of conditioning in advertising. Speciﬁcally, the authors were interested in whether object preferences could be classically conditioned. Gorn (reported in the Allen and Shimp chapter as the ﬁrst experimental study on conditioning and marketing) had participants rate different kinds of music, and he used the most appealing music as the appetitive US (i.e., pleasurable) and the least attractive music as the aversive US (and only participants who rated this musical piece as attractive were included for the pairing of the CS with the appetitive US, and only those participants who rated the piece as unattractive were included for the pairing of the CS with the aversive US). Seventy-nine percent of subjects given a pairing of the colored pen with the attractive music chose this pen over a nonexposed pen when given a choice, and only 30% of the participants chose the pen paired with unattractive music if they had received a pairing of this pen with the aversive music (obviously a percentage of 50% would reﬂect indifference to the pens when given a choice). Since a single pairing was used, it shows that signiﬁcant conditioning can occur with one conditioning trial. Second, the CS and US presentations were simultaneous, participants heard one (of two) musical clips while viewing a slide image of one of the pens, thereby showing that such an arrangement of CS and US can produce appreciable conditioning. Finally, “mere exposure” (see section on “Mere Exposure”) cannot explain the subjects choosing the nonexposed pen over the one paired with the aversive music. The mere exposure effect refers to the increase in attraction to a stimulus simply because the individual has encountered it in the past. It is possible that mere exposure made the pen paired with attractive music more attractive (rather than the increase in attractiveness being due to the pairing with attractive music); however, the pairing of a pen with aversive music produced an aversion to this pen despite any

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

possible (and perhaps unlikely) mere exposure effect working in the opposite direction (making it more attractive). Regarding the Gorn study, there has been a discussion in the advertising literature about the role of demand characteristics in this research (see Darley & Lim, 1993; Shimp, Hyatt, & Snyder, 1993). (By the way, we also wish to point out that Kahle, Beatty, and Kennedy [1987] stated that proponents of conditioning theory discount or trivialize the issues of awareness and demand characteristics; but, to us, this claim seems unsubstantiated and contentious.) Overall, producing classical conditioning in the laboratory is not easily obtained given that some reports have found poor conditioning or the results have been mixed or subject to alternative interpretation (Allen & Madden, 1985; Bierley, McSweeney, & Vannieuwkerk, 1985; Gorn, 1982), whereas other studies have been more promising in showing conditioning (Shimp, Stuart, & Engle, 1987; see Cohen, 1990; Allen & Shimp, 1990 for reviews).

CONDITIONING PARAMETERS AND PROCEDURAL ISSUES IN ADVERTISING RESEARCH This section will discuss some of the procedural variables that have been (or are suggested) to be useful in conditioning research. Some variables (e.g., US or CS preexposure prior to conditioning) can be both “a conditioning phenomenon” and a “procedural variable”; we will reserve our discussion of these until a subsequent section on conditioning phenomena. We realize that the distinction between “What is a procedural variable?” and “What is a conditioning effect?” is arbitrary for some effects. For instance, trace conditioning refers to conditioning effect and a manipulation in the interstimulus interval between the CS and the US; but such instances will be placed in one section or the other, and we hope that these sections still provide some usefulness. Arrangement of the Conditioned Stimulus and Unconditioned Stimulus in Time

Stuart et al. (1987) compared backward conditioning with delayed conditioning using a brand

21-Schachtman-21.indd 485

485

as the CS and an attractive stimulus as the US. Backward conditioning is a classical conditioning procedure in which the US onset precedes the CS onset. Forward conditioning (often called delayed conditioning) is classical conditioning in which the CS precedes the onset of the US and the CS offset does not occur prior to US onset (since the latter would be “trace conditioning”). In one of the early experiments in that report, the authors used a forward conditioning procedure in which the CS not only preceded the onset of the US but also overlapped the US, and they found that forward conditioning was superior to backward conditioning. Many or most forward conditioning procedures do not have any overlap between the CS and the US (i.e., the CS onset precedes US onset but CS offset occurs at the same time as US onset). When the CS and US have simultaneous onsets and offsets (complete overlap), then this is referred to as “simultaneous conditioning.” Since Stuart et al.’s initial experiment (Exp. 1) used a forward procedure that contained this element of a simultaneous arrangement, they conducted another experiment in which a forward conditioning procedure was used, but the CS and US did not overlap. Forward conditioning continued to produce a better CR than backward conditioning. Macklin (1996) used school-aged children to compare forward and simultaneous conditioning in an advertising situation and found that the former produced better conditioning. As mentioned previously, Gorn (1982) obtained good conditioning with a simultaneous arrangement of the CS and US. Other conditioning arrangements have been used as well. Baker (1999) used trials (the product was paired with pleasant photographs) in which the CS was presented alone, followed by a presentation of the US alone, followed by the CS and US together; and conditioning resulted. Baker, Honea, and Russell (2004) examined the effectiveness of placing the brand name at the beginning of the ad or at the end; and, like Stuart et al. (1987), they found that conditioning effects were stronger when brands were placed at the beginning of the ad. Interestingly, they also included a group that received the brand

2/16/2011 5:55:21 PM

486

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

name at the beginning and the end of the ad, and found that this group exhibited conditioning that was as poor as the “end-only” condition. That is, this condition received the brand name at the beginning of the ad like the group that showed good conditioning; but the placement of the brand at the end caused poorer conditioning than the “beginning-only” group, suggesting that the placement at the end offset the positive effects of placement at the beginning. Baker et al. point out quite correctly that this was probably due to the fact that the total exposure time to the brand name during the ad was 5 seconds for all conditions such that the “beginning” and “end” group receive the brand name for 2.5 seconds on two occasions, and the other two conditions (end only and beginning only) received the brand exposure once for 5 seconds. Therefore, 2.5 seconds may not be long enough to be effective for brand name exposure. But other possibilities exist. The possibility exists that presenting the brand at the end produces some cognitive interference with the forward conditioning trial that just occurred at the start of the trial. Future research will likely enjoy teasing apart these alternatives as well as testing other possible explanations for this interesting effect. Allen and Shimp (1990) argued that simple contiguity is not responsible for conditioning (p. 30), but this point requires elaboration. Contiguity refers to the degree to which two events occur together in time or space (and only temporal contiguity is discussed here). Simple contiguity is neither necessary nor sufﬁcient for conditioning to occur. As mentioned, Blair and Shimp (1992) found second-order conditioning during an advertisement experiment (see section on “Second-Order Conditioning”) in which the target event (brand) was never paired with the US (an unpleasant, boring textbook experience) and, yet, conditioning occurred showing that contiguity is not necessary. The fact that particular values for various parameters (CS novelty, US novelty, and others) are needed or are important in order to obtain conditioning—despite contiguity between the CS and the US—reveals that contiguity is not sufﬁcient for conditioning.

21-Schachtman-21.indd 486

Relationship of the Stimulus Properties of the Conditioned Stimulus and Unconditioned Stimulus to Each Other

Thorndike’s concept of “belongingness” and, more recently and to much more acclaim, John Garcia’s discovery that some CSs condition more effectively if paired with certain USs (but not others); and some USs support conditioning more effectively if paired with certain CSs but not others (Garcia & Koelling, 1966) were important ﬁndings for the ﬁeld of conditioning (Freeman & Riley, 2009). Belongingness may have an important effect on conditioning in advertising (Allen & Shimp, 1990; McSweeney & Bierley, 1984; see also Kellaris, Cox, & Cox, 1993, as discussed later). Kim et al. (1998) found that if a CS and US have little or no preexperimental conceptually based relationship with each other, classical conditioning can still occur as long as the subject does not hold any beliefs about the stimulus that might preclude conditioning. Conditioning researchers have found that second-order conditioning effects (discussed in section on “Second-Order Conditioning”) are also sensitive to the modality of the two cues used in experiments with animal subjects (e.g., Nairne & Rescorla, 1981; Rescorla & Gillan, 1980). Kellaris, Cox, and Cox (1993) found that recall and recognition of brand name as well as the “point of the message” in the ad increases if attention-getting music is used; and recall and recognition are especially enhanced if there is a “congruency” between the meaning communicated nonverbally by the music and that verbally communicated by the ad; marketing researchers may wish to note the degree to which the thematic qualities of the background information matches that of the verbal message. Partial Reinforcement

Bierley, McSweeney, and Vannieuwkerk (1985), in a study relevant to advertising since music is often used as an unconditioned stimulus in ads, found conditioning of colors paired with attractive music. Partial reinforcement did not produce any conditioning of a color paired with the music. It can also be noted that the groups were

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

not equated for number of reinforcers, but rather, received the same number of trials as the continuous reinforcement condition because there are two ways to produce a partial reinforcement condition: equating reinforcers or trial number. Researchers need to decide which to manipulation or, rarely, to include various groups that manipulate both. Properties of the Conditioned Stimulus, Other Cues, and the Role of Attention

The intensity or salience of the CS can also have a considerable inﬂuence on conditioning during advertising (see, for example, Cohen, 1990, for a discussion). A more noticeable presentation of the brand name will likely help produce stronger conditioning. Additionally, the salience of cues may be determined, in part, by the characteristics of the individual. Gorn (1982) noted that those interested in purchasing a product may ﬁnd the product information (brand name, etc.) more salient than those not interested in a future purchase. To investigate the validity of this possibility, he examined participants who were in either a decision- or nondecisionmaking context with respect to their relative sensitivity to background cues (i.e., music in this case) and product information. Gorn discovered that non-decision-making participants (what might be considered “less involved” subjects) were most inﬂuenced by the music, whereas decision makers (more involved) were more inﬂuenced by the information provided. Hence, the impact of cues can be inﬂuenced by the person’s motivation. The duration of the CS can inﬂuence conditioning (Gibbon & Balsam, 1981; Miller & Schachtman, 1985; Miller & Matzel, 1988; a factor also noted by Allen and Shimp, 1990, p. 30). Conditioning theorists have discovered that extended CS durations can reduce the degree of conditioned responding. Of course, the stimulus must minimally be exposed long enough for the individual to detect or process it. The modalities of the cues presented in the ad can also play a vital role. Stammerjohan, Wood, Chang, and Thorson (2005) examined whether using multiple modalities for ads (visual

21-Schachtman-21.indd 487

487

and auditory rather than just visual or auditory) might inﬂuence processing of the ad. As noted by Stammerjohan et al., research on encoding variability (Tulving & Thomson, 1971) suggests that presenting information in more than one context or modality will improve memory and impact the degree of attitude change (and see Stemmerjohan et al. for a discussion of supporting ﬁndings and see also Cacioppo & Petty, 1985). Although the authors did not provide unequivocal support for these ideas, this important issue warrants more research. Vakratsas and Ambler (1999), when discussing persuasive hierarchy models, point out in their review that “varied ads” improve ad recall (Rao & Burnkrant, 1991; Zielske & Henry, 1980). Stammerjohan et al. mention that multiple modality input during ads also could include the subject-generated elaborations that occur during or following an ad, such that if the ad provides only auditory information (e.g., a radio ad) but the individual elaborates by imagining the product visually, then multiple-modality processing can be said to be occurring. Research can therefore determine whether elaboration-produced cues are comparable to having more than one modality present in the ad itself. Stammerjohan et al. cite Kahneman (1973) as having mentioned that a large amount of attention is given to items that are both complex and familiar and those that are both simple and novel, but this is not true for simple-familiar items nor for complex-novel stimuli. They also mention the “positivity effect” (that positive stimuli are processed more than negative stimuli). Hence, negative advertising should be less effective than positive advertising (see Cohen, 1990). However, Cohen points out that unpleasantness in an ad can sometimes generate attention and interest in a product such that the product may be expected to resolve this unpleasantness. Baker (1999) notes that high familiarly can result in less information being processed; hence, high brand familiarity means that providing product information can be less valuable since individuals will not process this information as well in an ad about a highly familiar product. Individuals already have opinions about a familiar product and do not always

2/16/2011 5:55:21 PM

488

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

process new information about it (see also section on “The Role of Prior Belief ”). In such situations, it is important to provide stimulating material to prevent boredom during an ad with a familiar product. Janiszewski and Warlop (1993) noted that attention to a stimulus often involves an orienting response to the stimulus (Hall & Channell, 1985; Hall & Schachtman, 1987; Sokolov, 1963). They reasoned that this orienting response can be very important to advertisers since one hopes for conditioning to the brand name (and attention is important for conditioning), but one also aims for a strong response to the brand later—at the time of purchase. Consistent with the ﬁndings of Hall and Channell (1985), Janiszewski and Warlop point out that if a brand (CS) (even a familiar one) is presented in a novel context, then the orienting response will be high in this context even if orienting had waned in the conditioning context (i.e., where the brand-US pairings occurred). Janiszewski and Warlop (1993) found that attention is increased to a CS as a function of conditioning (pairings of the brand with an attractive US). These researchers also said that if the brand is paired with the US in one context, but then the brand is seen in a store (new context) for the ﬁrst time, then orienting might be strong and this strong reaction may increase the chance of purchase. Janiszewski and Warlop (1993) found that if a brand is conditioned by pairing the brand with a US, then this brand will “pop out” perceptually when presented subsequently on a screen with other items (see Johnston & Hawley, 1994, for more information on such effects with various stimuli) revealing that conditioning, perhaps not surprisingly, can add to the attention-getting properties of a brand name. However, we can also imagine how novelty will promote attention to a product. The maturity (versus novelty) of a product can have a large inﬂuence on conditionability. It is also worth mentioning that conditioning researchers have posited that this orienting response is an index of associability (e.g., Swann & Pearce, 1988). Associability refers to the potential of a CS to enter into an association with a CS. Oxoby and Finnigan (2007) found that attention

21-Schachtman-21.indd 488

to one feature of a product (e.g., cost or brand quality information) caused poor attention to other—subsequently presented—information about the product. Gresham and Shimp (1985) suggested that mature (familiar) brands will be more inﬂuenced by the impact of the attitude of the brand on the attitude toward the ad, whereas newer products may experience the attitude toward the ad inﬂuencing the attitude toward the product. Research by Alpert and Kamins (1995) revealed that novel brands possess attention-getting properties that facilitate processes on some measures (attitude and purchase intention) but not others (recall or actual purchase behavior). Hence, attention to a CS can have an inﬂuence on conditioning, and conditioning can impact attention to a CS (Mackintosh, 1975; Pearce & Hall, 1980). Number of Trials

Kroeber-Riel (1984) stated that numerous trials are needed for conditioning, but this is not true. Kim, Lim, and Bhargava (1998) obtained conditioning with a single trial (see also Ehrenberg, 1974). Stuart et al. (1987) also obtained asymptotic conditioning with a single trial. Kim et al. examined the effect of the number of trials and found that affective conditioning requires fewer repetitions than cognitive belief acquisition. Vakratsas and Ambler (1999) mention that there may be an optimal number of trials to produce favorable advertising effects. A minimum number of trials is needed to get an effect (the “wear-in effect,” see, e.g., Blair, 1987) and the effect of advertising decreases after a certain number of exposures to an ad. There is an inverted-U shape to the effectiveness of advertising as a function of the number of conditioning trials. One valuable way to offset this inverted-U function—that is, to continue to get effects with additional trials—is to vary the ad somewhat so that individuals get exposed to a slightly different variation (Rao & Burnkrant, 1991; Zielske & Henry, 1980). As mentioned, conditioning can involve many processes besides the acquisition of an

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

association (e.g., the retrieval, retention, elaboration, rehearsal of information). So even if good learning is apparent after one or two conditioning trials, additional, beneﬁcial processing might occur if additional trials are given. Batra and Ray (1983) note that Krugman (1972) stated that there are only three truly effective trial types during an advertisement: an initial exposure that produces recognition, a second exposure that involves the subject’s processing, and the third and all subsequent exposures, which simply serve as reminders of what the viewer has already seen and thus maintain such processing. We note again that these later trials may produce rehearsal/retrieval-practice that can inﬂuence behavior on certain measures. One other point about the number of trials used can be made: Allen and Janiszewski (1989) found that more pairings resulted in more demand characteristics and so this concern should be addressed. Allen and Janiszewski provide extensive discussion of demand characteristics in their report (see also Kahle et al., 1987). The inﬂuence of the number of trials is also discussed in section on “Cognition and Affect.” Intertrial Interval

The intertrial interval can have a large effect on conditioning. Similar to the “spacing effect” in human learning (Crowder, 1976), classical conditioning is greater if a longer period of time occurs between trials during the experimental session. Such effects may also occur in an advertising situation (as mentioned by Allen & Janiszewski, 1989). Some conditioning theories have posited that such effects are due to the relative durations of the CS and the contextual cues that are exposed between trials (Gibbon & Balsam, 1981; Miller & Matzel, 1988; Miller & Schachtman, 1985). Different Behavioral Measures

Allen and Shimp (1990) discussed advertising research in which the experimental manipulation changes the participant’s attitude toward a brand or product, and then tests this change by giving the participant a preference between that item and a control item. They stated that

21-Schachtman-21.indd 489

489

preference is a “demanding measure” and may not be the best dependent variable. They note that Macklin (1998) used a “buy back measure” rather than a preference measure with children as subjects, and found that the former was a much more sensitive assessment. Another interesting assessment tool was used, as noted earlier, by Janiszewski and Warlop (1993), who, when using an eye-tracking device, found that conditioned brand names will receive a large amount of attention in a display. Rothschild and Hyun (1990) used electroencephalography (EEG) as a measure. Clearly many assessment tools are available to marketing researchers (see Cohen, 1990). Control Conditions for Conditioning

Many different kinds of control conditions can be used in an experiment to ensure that the conditioning is due to the contiguous pairing of the CS and US and the positive contingency between these events. McSweeney and Bierley (1984) discuss this issue in some detail and so readers may wish to refer to this resource. Some researchers have used random CS and US presentations (Bierley et al., 1985; Janiszewski & Warlop, 1993). Some researchers have used a procedure for the control condition in which the CS and US are presented randomly with respect to each other, except with the constraint that the two events not be paired together by chance (Grossman & Till, 1998; Priluck & Till, 2004; Stuart et al., 1987). One possible problem a group for which the CS and US never occur together (akin to “explicitly unpaired” CS and US presentations) is that explicitly unpairing the events produces a negative contingency such that one event comes to signal that the other event will not occur—a phenomenon known as conditioned inhibition (Rescorla, 1969). Rescorla proposed in the late 1960s that a reasonable control condition that will produce neither a positive contingency between the two the events, nor an unpaired arrangement which might produce conditioned inhibition, is to present the events randomly. Yet random presentations can produce their own problem as they can produce “learned irrelevance” between the CS and US (see Matzel,

2/16/2011 5:55:21 PM

490

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Schachtman, & Miller, 1988; but see Bonardi & Ong, 2003) and, as Grossman and Till (1998), Priluck and Till (2004), and Stuart et al. (1987) likely realize (since they avoided this problem), the chance pairings produce problems of their own. However, little such conditioning from chance pairings in a random condition seems to occur when this treatment is compared with a group that did not receive a US during conditioning (Stuart et al., 1987). Janiszewski and Warlop (1993) gave conditioning in which the trial consisted of three temporal phases: the CS period, the US period, and a posttrial period for the experimental condition. The control received these three phases in a random order on the trials—certainly a reasonable control condition (although perhaps too conservative in that some conditioning could occur in the control group since associations could be formed with a long interstimulus interval). McSweeney and Bierley (1984) mentioned that presenting the CS constantly and then presenting USs intermittently during the CS exposure does not produce a CR in animal research (Brown & Jenkins, 1968). This latter procedure is not too different from some of the procedures of advertising research in which several USs occur during a CS presentation in which CRs are produced. There is a need to examine various parameters (e.g., CS duration) to ﬁnd out why this difference in conditioning may exist. There are many different control conditions available, each with its advantages and disadvantages. Retention of Conditioning

Obviously, it is important for advertisers to know how long the effects of advertising might last, because the interval between viewing an ad (i.e., conditioning) and product purchase might be lengthy. Grossman and Till (1998), using six conditioning trials, found that attitudes toward experimentally conditioned brands lasted at least 3 weeks (the longest interval tested). They also discussed research by Mitchell (1993), who found that attitudes toward a brand decreased over a 2-week delay but intention to purchase did not decrease. Moore and

21-Schachtman-21.indd 490

Hutchinson (1985) found that a 7-day retention interval caused attitudes toward the brand to be reduced, but brand awareness (e.g., choosing the product category for a brand) increased. Kellaris et al. (1993) found that recall and recognition of brand name as well as the “point-ofthe-message” in the advertisement increases if attention-getting music is used. Gardiner, Mitchell, and Russo (1985) found that low involvement (deﬁned as viewing product information for its entertainment value; see section on “High Versus Low Involvement”) resulted in poorer memory (on four different measures) for product information, but resulted in a greater positive evaluation of the brand (when evaluating 22 attributes of the product, this condition had more positive judgments in 19 of them). Cohen (1990) pointed out that peripheral aspects of the message (voice quality of the ad, affective responses to the message) will play a signiﬁcant role in how much or whether elaboration occurs. Memory researchers know well that elaboration can improve retention with various types of information. Vakratsas and Ambler (1999) noted that “varied ads” improve ad recall (Rao & Burnkrant, 1991; Zielske & Henry, 1980). Burke and Srull (1988, p.65) examined interference and memory in advertising and found proactive and retroactive interference effects for ad information. They reported that “recall interference occurred when subjects rated the target ads on interest value but not when the advertised brands were evaluated for purchase”; and they went on to suggest that the greatest interference seems to occur for “consumers who are not in the market for a product, or who do not have the ability and/or motivation to process ads in a manner that will enhance information retrievability.” More research on retention and the effects of interference is needed. Use of Music in Advertising

As mentioned, Kellaris et al. (1993) found an increase in recall and recognition of the brand name and the point of the message in the ad if attention-getting music is used. Bruner (1990) provided an extensive review of music in

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

advertising, in which he suggested the use of music when products induce low cognitive involvement (e.g., jewelry, beer). Bruner discusses various kinds of music that can be used for ads. He points out the drawback of using familiar songs (overexposure to a song can render it less effective, as mentioned earlier) and suggests that advertisers use a song that has already been written but is not familiar or have one written for the purpose of the ad since it will obviously be novel. Hung (2001), in an advertising study, found that music can produce an image in the consumer’s mind (e.g., “successful,” “imaginative”) as well as an emotion (e.g., “calm,” “boring,” “annoying”). Hung found that music (classical versus hard rock) inﬂuenced (1) the estimated price of objects for sale at the advertised shopping mall; and (2) the perception that the store was darker or lighter with respect to its lighting. Some music also produced much greater within-group variability (i.e., the selection of classical music used produced much more consistent responses across subjects than the rock music). As mentioned earlier, Gorn (1982) experimentally conditioned products (pens) using liked and disliked music. Pitt and Abratt (1988) also used music in a classical conditioning experiment. Music has been used in many additional studies; unfortunately, an exhaustive review cannot be provided here, but this medium obviously impacts advertising in many ways.

Order of Ad Exposure and Experience with a Product

Vakratsas and Ambler (1999), when discussing low-involvement hierarchy models, point out that advertising is more effective when it precedes usage experience (see also the section on “The Role of Prior Belief” for the inﬂuence of past experience on future processing). In their review, they mention that Smith (1993) showed that if an individual experiences an ad prior to a negative experience with the product, then the ad can reduce the negative impact; but the ad has no effect if the person has had a positive

21-Schachtman-21.indd 491

491

experience. Ads that occur prior to experience have a much greater effect.

SOME CONDITIONING PHENOMENA AND THEORETICAL ISSUES INVOLVING ADVERTISING This section explores a variety of conditioning effects that can be useful for marketing researchers. Researchers may beneﬁt from an appreciation of the underlying mechanisms of the effects discussed herein. Alternatively, researchers may wish to explore these effects in an advertising experimental framework as many marketing researchers have done with a few of these effects. Mere Exposure

The mere exposure effect refers to the increase in attractiveness of a stimulus simply because it has been previously exposed to the individual. Batra and Ray (1983) stated that, for a person with high involvement, the affective changes that occur as a function of mere exposure are not the same kind of affective changes that can occur during CS-US pairings for a person with high involvement. In the latter case, cognitive processes such as awareness and comprehension will occur; and such stages are necessary for affective attitude change (although a deﬁnition of what is and what is not attitude change seems needed). As mentioned, Gorn ruled out mere exposure as the cause of a conditioned preference during a treatment of group in which a product was paired with an aversive US. Some theories of advertising state that simple exposure to an advertisement will increase liking due to familiarity such that this can happen independently of awareness or attention to the attributes of the product (Vakratsas & Ambler, 1999). Unconditioned Stimulus Preexposure

US preexposure involves the effects of administering US presentations prior to the pairings of the CS and US. Many marketing researchers (e.g., Bruner, 1990) have noted that earlier exposure to the US (i.e., an attractive US such as pleasurable music) can attenuate the effects of conditioning using that

2/16/2011 5:55:21 PM

492

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

US. The conditioning literature shows that US preexposure can reduce the effectiveness of the US in supporting conditioning (Gordon & Weaver, 1988; Randich & Ross, 1984; Tomie, Murphy, Fath, & Jackson, 1980). Allen and Shimp (1990), when discussing this effect, note that Bierley et al. (1985) required 28 pairings of a color with attractive music that was highly familiar (Star Wars theme) to the subjects in order to obtain conditioning. Allen and Shimp also discussed the tradeoff of using familiar celebrities and popular music (which has obviously been exposed to the individuals in the past), which has some advantages for conditioning versus the hindering effect such exposure might have. A few issues can be noted about these effects. First, these kinds of tradeoffs are not new to conditioning researchers. Many fear-conditioning researchers (using primarily animals, that is, non-humans) will give one or two exposures to the CS (in contrast to the US preexposure being discussed) prior to conditioning in order to remove the unwanted effects (e.g., startle) that a novel CS can have on the initial conditioning trials. Giving such an exposure or two prior to the conditioning trial will greatly decrease such unwanted responses on the conditioning trial, but event preexposure of this sort also often involves some price to pay. Although this effect involves an issue regarding prior CS exposures rather than US preexposures, it provides another example of a researcher dealing with the tradeoff among various factors. Conditioning researchers often weigh these various tradeoffs when using procedures in which the stimuli are novel or familiar. Conditioning theorists claim that the poor conditioning that results from US preexposure occurs by one of two processes. First, this effect may be the result of habituation to the US. Habituation is the loss of responding to a stimulus that has been presented repeatedly. This repeated presentation can cause the individual to stop responding to the US; a poor response to a stimulus can also be indicative of poor ability to support conditioning to a CS. A second process is that the US preexposures cause an association between the contextual cues (e.g., the environment that the individual is in) and the US. This association “blocks” the learning of

21-Schachtman-21.indd 492

an association between the CS (e.g., the brand/ product) and the US (e.g., Gordon & Weaver, 1988). Blocking (and overshadowing, a very related conditioning effect) will be reviewed next. The multifaceted effects of US preexposure make the decision to use a familiar US versus a novel one a challenging decision. Latent Inhibition or the Conditioned Stimulus Preexposure Effect

Latent inhibition (Lubow & Moore, 1959) is the poor conditioning that occurs to a CS (a product or brand) if this stimulus is presented many times by itself (i.e., without the US) prior to the conditioning trials (i.e., being paired with the US). This poor learning is compared to a conditioning that did not receive the CS-alone exposures (and this latter group shows normal, strong conditioning). Stuart et al. (1987, Exp. 2) examined latent inhibition in an advertising experiment by exposing participants to a particular brand name on either 8 or 20 occasions prior to pairing it with an attractive US on the conditioning trials. Participants in these conditions showed poorer conditioning to the brand name than those in a control condition who received conditioning without any CS (i.e., brand name) preexposure. It is valuable for marketing professionals to know that brand exposure prior to conditioning can hinder conditioning during advertisements. Not surprisingly, advertising researchers have noted that ads are much more effective for new products with names that have not received much or any exposure prior to the ads (e.g., Baker, 1999; Gresham & Shimp, 1985; Vakratsas & Ambler, 1999). Baker (1999) noted that that high familiarly can reduce the amount of information processed when later exposures occur; in other words, high brand familiarity can mean that providing product information will be less valuable since individuals will not use this information in an ad about a highly familiar product. Overshadowing and Blocking

Blocking and overshadowing involve competition among CSs for processing (or competition between the context cues and a CS) as described

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

in the preceding section. McSweeney and Bierley (1984) and van Osselaer (2008) also discuss these phenomena in their reviews. Overshadowing refers to the poor learning that occurs to a CS if it is conditioned (i.e., paired with the US) in the presence of a second CS relative to a condition that receives this CS paired with the US in the absence of a second CS. Hence, if we use a symbol for the “target” CS (e.g., the brand or product that one is interested in assessing for the degree of CR); “CSX”; and we use the symbol “CSA” to refer to the second CS, and we use the symbol “+” to refer to the US, then overshadowing refers to the poor conditioning that occurs to CSX when it is paired with the US when CSA is also present (hence, CSXCSA+ trials). This group’s performance is compared to a control group that simply receives CSX+ trials. Simply put, CSX is learned about much more if it is paired with the US alone rather than in the presence of a second CS (CSA). When an overshadowing effect is obtained, researchers will often say that CSA overshadowed learning to CSX; often CSA is a very salient stimulus and CSX is relatively less salient, explaining why CSA obtains learning at the expense of CSX, that is, CSA beneﬁts from the competition between the cues. For instance, a product name may be presented during an ad along with another salient item and these two stimuli will compete for becoming associated with the US. Salience of the CSs will inﬂuence the degree of overshadowing. As mentioned earlier, researchers have noted that some cues will be more salient depending on certain factors such as the amount of involvement (Gorn, 1982). Those interested in purchasing a product may ﬁnd the product information more salient than those not interested in purchasing the product, and the latter individuals may ﬁnd the music, if one can think of the music as a cue that may compete with the product for processing, in the ad more salient. Conditioning theorists have known about overshadowing for over 100 years (since the work of Pavlov), but a more recently discovered phenomenon is “potentiation.” Potentiation is the opposite of overshadowing, but it uses the same procedure. Potentiation refers to the increased conditioning to a CS (e.g., a brand or

21-Schachtman-21.indd 493

493

product) due to its pairing with a US in the presence of a second CS. This group’s performance is compared to that of a control group in which the CS was paired with the US in the absence of a second CS to show the inﬂuence of the second CS in the other (experimental) group. (The treatment conditions are the same as in an overshadowing experiment.) One major interpretation of potentiation is a mechanism like that of second-order conditioning (described later). It is valuable for marketing researchers to know that having a second stimulus (perhaps a different product) present during an advertisement could potentially promote the conditioning to the target product, although competition for conditioning between the stimuli (overshadowing) may be the more likely outcome in most conditioning situations (i.e., overshadowing is likely more common that potentiation). Blocking (Kamin, 1969) refers to the poor conditioning that occurs to a target CS (a brand) when it is paired with the US in the presence of a second CS, when that second CS was previously paired with the US. That is, if CSA is paired with the US in an initial phase of the experiment (CSA+ trials) and then CSA and CSX are both paired with the US in a second phase of the experiment (CSACSX+), then CSX is poorly learned about. CSA is said to “block” learning about CSX. CSA has an advantage in the competition for learning since it already predicts the US because of its initial training (CSA-US pairings). The performance by CSX for this group is compared to a condition that received the same treatment to the group described except no CSA+ trials occurred in the initial phase; this control group only receives the CSACSX+ trials. The control condition will show a stronger CR to CSX because CSA was not pretrained. Both CSs will still compete for learning on these trials for the control condition, but CSA will not beneﬁt from the great advantage of having been previously paired with the US in the initial phase. The control group produces a greater CR to CSX than the blocking condition. Blocking can be an important phenomenon for marketing professionals interested in producing an effective ad. If the ad involves pairing a product (e.g., sunglasses) with an attractive US

2/16/2011 5:55:21 PM

494

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

(people having fun on the beach), and this ad contains other cues that already predict fun at the beach, then those latter cues may block the product from being learned about. One troubling issue concerns how to discern which stimuli serve as the US in such a circumstance and what cues are stimuli that might compete with the target CS (the product). For instance, if the ad contains people playing sand volleyball, is this cue part of the US—people having fun at the beach? Or is it a competing CS—a cue that already predicts fun at the beach which will then prevent learning about the product? The answer is not clear to us, but marketing researchers may appreciate knowing about the different possible outcomes and the conditioning processes believed to underlie these effects. In support of ﬁnding competition between cues in an advertising situation, Van Osselaer and Alba (2000) found that learning about one characteristic of a brand (e.g., that the brand is high quality) will result in poor subsequent learning about more reliable information (see Oxoby & Finnigan, 2007 for a discussion). Oxoby and Finnigan (2007) point out that such an advantage of ﬁrst-learned information means that companies should be careful about the initial messages that are delivered to consumers since subsequent information may not be adequately processed. They also address how “brand extentions” may not receive adequate processing for similar reasons. Indeed, Oxoby and Finnigan found blocking not just for subsequently exposed attributes about that same product but also for related products. Janiszewski and van Osselaer (2000) examined interactions among brand names and found that such interactions between two different brand names can occur during advertising. That is, when a product has two brand names, a regular brand and a “subbrand,” associated with it (e.g., a certain brand of ice cream with Hershey’s chocolate mixed in). These researchers mention that such interactions are consistent with connectionist, least-mean-squares models (e.g., the Rescorla-Wagner model, Rescorla & Wagner, 1972) of conditioning rather than models in which associations are formed independently of each other.

21-Schachtman-21.indd 494

Van Osselaer and Janiszewski (2001, as reviewed by Oxoby & Finnigan, 2007) examined the learning processes that underlie consumers’ processing according to the human associative memory (HAM) model (Anderson & Bower, 1973) and adaptive network (AN) models. The former model involves associations being acquired independently, whereas AN models allow for associations to compete (they are not necessarily learned independently). Van Osselaer and Janiszewski conclude that HAM models describe performance when the participants do not have a speciﬁc processing goal, whereas AN models describe performance when the subject does have a processing goal. Given their import in conditioning theory, the effects of competition may receive more empirical attention in the future. Second-Order Conditioning

Second-order conditioning refers to the conditioning that occurs to a target CS because that CS was paired with another (nontarget) CS, and this latter CS had been previously paired with the US. That is, the target CS (CSA) is paired with the US in the initial phase of the experiment (CSA+ trials). Then, CSX is paired with CSA. Note that CSX is never paired with the US, yet CSX produces a CR. Second-order conditioning is held as evidence that contiguity between the CS (CSX) and the US is not necessary to produce conditioning. A similar procedure, sensory preconditioning, is essentially the same as secondorder conditioning except that the two phases of conditioning are reversed: CSA and CSX are paired together ﬁrst (CSACSX trials) and then CSA is paired with the US (CSA+). As with second-order conditioning, a CR occurs to CSX as a result of this procedure. Second-order conditioning can be said to occur if one product is paired with an attractive US (and the product now becomes attractive). A second product is then paired with the ﬁrst product and the second product is now attractive because of its association with the ﬁrst product. Blair and Shimp (1992) obtained secondorder conditioning during an advertisement experiment in which music (music served as the

2/16/2011 5:55:21 PM

CONDITIONING AND ADVERTISING

CSA, which was perceived as very neutral to the participants at the start of the experiment) was paired with an aversive US (an unpleasant, boring experience while reading a selection from a certain textbook). Then CSX (a ﬁctitious sports brand) was paired with the music (CSA). Firstorder conditioning to the music was found, and second-order conditioning to the brand was also obtained. That is, when subjects were tested on their liking of the music (CSA) after it had been paired with the aversive event (the US), they found the music aversive. The subjects also found the brand (CSX) aversive after it had been paired with the music (CSA) a second-order conditioning effect. Note that the brand itself was never paired with the US (the aversive event). One major explanation for second-order conditioning (and sensory preconditioning) is that the subject forms an association between the two CSs. CSA becomes associated directly with the US. Cognitive conditioning theorists will claim that, if an association between CSA and the US is formed, the presentation of CSA will activate a representation of CSA, in the individual’s memory network. The activation of this representation (CSA) will cause, via the association between CSA and the US, the US representation in memory to become activated. Since CSA and CSX are associated, when CSX is presented after all phases of conditioning are completed, it will be able to activate a representation of CSA due to the association between the CSs. The activation of CSA will cause activation of the US. This indirect activation of the US when CSX is exposed (i.e., via the CSA representation) is the reason for the CR to CSX (see Pearce, 2008 for a discussion). The implications of second-order conditioning is that marketing professionals may ﬁnd that associations are formed between stimuli during ads and changing the value of one of these stimuli may inﬂuence the value of the other stimulus. Effects of Contextual Cues

Many marketing researchers mention the impact of contextual factors (e.g., Allen & Shimp, 1990; Cohen, 1990), but what is meant by that term

21-Schachtman-21.indd 495

495

varies greatly. We will view contextual cues the way they are often viewed in conditioning experiments: environmental cues, including mood and circadian cues. Allen and Shimp (1990) pointed out that contextual cues can be critical as determinants of the CR; and, given its import in many conditioning theories and phenomena (e.g., Balsam & Tomie, 1985), more experimental work needs to be done on contextual factors and advertising. Janiszewski and Warlop (1993) noted, as mentioned earlier, that changes in the attentional response or orienting response to a brand or product may be context dependent such that, even if the response has dissipated for a certain product or brand, presentation of the item in a new context (at the point of purchase) may increase that orienting or attentional response. Stammerjohan et al. (2005), as noted earlier, mentioned that presenting information in more than one context or modality would improve memory and attitude change. One interesting context effect in the conditioning literature should be noted before we move to the next section of this review. “Comparator theories” of classical conditioning (Gibbon & Balsam, 1981; Miller & Matzel, 1988) argue that the reason that long intertrial intervals and short CS durations enhance conditioning is due to a comparator process in which the durations of CS exposure and context exposure are compared. Conditioned responding is greater to the extent that the CS duration is short and the context duration (the intertrial interval) is long; and it is poorer to the extent that the CS duration is long and the contextual period is short (Miller & Schachtman, 1985). So these theories would predict that conditioning to an ad will be greater if the session period (the “period” in which the brand name is paired with the US, which might be the entire ad itself) is lengthened. Shorter CS (brand name) exposure will also help conditioning. In sum, if the entire commercial happens to serve as a “context” for the CS (the presentation of the brand name) then longer commercials might facilitate conditioning. For instance, let’s assume that the ad shows a person using a lawn mower of a particular brand (CS) and the commercial shows that

2/16/2011 5:55:22 PM

496

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

using the mower leads to some happy outcome (the US); that is, the person on the mower is happy and his or her neighbors wave (they like him or her, in part, because of the lawn mower). If this “trial” is contained within a commercial (context) and the trial is longer rather than shorter, then conditioning to the brand may be enhanced. Conversely, if the ad simply gives the CS-US pairing in 10 seconds and that is the extent of the commercial, then conditioning may be poor (meaning many trials might be needed to get conditioning), according to comparator theory. Hence, it could turn out to be more lucrative to have a longer, albeit more effective, commercial than a shorter commercial that requires many exposures. As another issue pertaining to the role of context in advertising, Gordon (2001) mentions “need states” as a context for the consumption or purchase of products (e.g., coffee). Clearly, researchers and marketing specialists need to take contextual cues into consideration when assessing the efﬁcacy of advertisements. Cognition and Affect

Vakratsas and Ambler (1999) as well as many other researchers (e.g., Stout & Rust, 1993) have discussed the complex relationship between cognitive processes, including beliefs and affective processes, during advertising. They refer to “affect” and “cognition” as “intermediate effects” in modeling the processing that occurs during the viewing of an ad. The issue of “which comes ﬁrst: emotion or cognition?” is as old as the famous Cannon-Bard/James-Lang debate of a century ago and other early debates in the literature exist (e.g., Lazarus, 1981; Zajonc, 1980). Cohen (1990) assumes that affective traces must be interpreted by the cognitive system before they can become manifest in behavior. Vakratsas and Ambler’s ﬁne review article on models of advertising show that there are many different ways of conceptualizing the relationship between affect and cognition (see also Cohen, 1990). Among the many potential processes by which conditioning might change behavior following advertising, “affect transfer” and a “change in beliefs” are two such mechanisms.

21-Schachtman-21.indd 496

Kim et al. (1998) investigated the role of forming beliefs as well as acquiring affective properties of a product during advertising. They reported that beliefs are not the entire story to successful conditioning in that affective properties can also be transferred from the US to the CS. The two effects are not mutually exclusive during an advertising experience (see also Kim et al., 1996). Lutz (1985) claimed that affect transfer is more likely when low involvement occurs. Kim et al. (1998) used a single conditioning trial and obtained conditioning; they concluded that only affect (not belief) could have been responsible for the conditioning effect they observed because, they claimed, the US they used did not provide any belief-related information. However, one could say that the participants generated belief information on their own through elaboration. Nonetheless, Kim et al. concluded that multiple conditioning trials produce belief information in addition to the previously produced affect, and they stated that both affect and belief can occur during advertising. When they used multiple pairings, they found that the size of the effects of these two processes (affective and belief formation) were found to be statistically indistinguishable from each other. They concluded that the learning of affective properties was stronger than the forming of cognitive beliefs with a single trial but both processes are equally inﬂuential with multiple trials. Kim et al. discusses an article by Pechmann and Stewart (1988) in which affective conditioning occurred with fewer trials than cognitive-based ads. Allen and Madden (1985) argue against affect transfer as a mechanism of classical conditioning effects during advertising effects. Kroeber-Riel (1984) believed that classical conditioning occurs without cognition, while stating that limiting cognitive processes during classical conditioning is important. He may have been assuming that conditioning would be worse if such cognitive processes (such as awareness) occurred, which we know to not be true because cognition/awareness can enhance conditioning (see next section). Kroeber-Riel also said that cognitive responses are always accompanied with emotional responses.

2/16/2011 5:55:22 PM

CONDITIONING AND ADVERTISING

Many researchers agree that noncognitive processing of advertisements (suggesting less involvement as will be discussed later) lead to more positive evaluation of the ad than more in-depth cognitive processing. Vakratsas and Ambler (1999), when discussing advertising, also pointed out that, according to some models, the order in which cognitions, affective responses, and memories for past experiences occur during ad viewing depends on the level of involvement. They also mention that “affect is relatively more important in low involvement and nonelaborative situations” and that “cognitive and affective beliefs may occur independently in these circumstances” (see also Janiszewski, 1988). They point out how hard it is to dissociate cognition and affect; for instance, asking about feelings will give rise to cognitive processes. This topic of modeling of the processes that occur during ad viewing is so complex that this terse sketch of the issue hardly does it justice; but viewers are encouraged to turn to the Vakratsas and Ambler (1999) review for much more detail about the subject. Evaluative Conditioning (as a Type of Referential Learning) Versus Classical Conditioning (Expectancy Learning) and the Role of Awareness During Advertisements

It is useful to distinguish between the implicit and explicit processes that can underlie classical conditioning, and the degree that conscious (or nonconscious) and effortful (or noneffortful) processing is involved (e.g., Baeyens, Crombez, Van Bergh, & Esten, 1988; Dawson, Beers, & Kelly, 1982; Gordon, 2001). Many possibilities exist. A learning experience can involve awareness or lack of awareness and may be effortful or noneffortful. These types of processes can be applied to the experience of association formation, as well as to the time of performance when this learned information is used for behavior. A stimulus can acquire an association with another event automatically (i.e., without any cognitive effort or awareness of the processes involved) or explicitly (with awareness). This stimulus can evoke the conditioned response

21-Schachtman-21.indd 497

497

after such learning through an automatic, implicit process or does so explicitly (i.e., the subject is consciously aware that this stimulus predicts an outcome stimulus and this awareness promotes the CR). Allen and Shimp (1990) correctly point out that classical conditioning is often described as a theoretical explanation of the changes in behavior that can occur as a result of advertising (rather than as a procedure). Classical conditioning is the mechanism that is usually associated with the processing of ads with low-involvement products. This theoretical explanation usually makes assumptions about the processes that underlie such learning; that is, they assume the conditioning is a noncognitive process and it might be assumed that the process is implicit and automatic. However, these authors (p. 22) also point out that awareness during classical conditioning is possible and can even be expected during such conditioning. Kahle et al. (1987) claim that conditioning theory requires that classical conditioning in adults occurs without awareness. Although, as Kahle et al. point out, conditioning was discovered and developed by early researchers with such a view in mind, it seems bold and erroneous to make such a claim in the 1980s. Kahle et al. also mention that awareness during conditioning “implies that participants grasp the nature of the hypotheses of the study” [italics added]. Although awareness can give rise to knowledge of the hypothesis, this implication is a bold assumption. Other researchers (e.g., Brewer, 1974) have claimed that awareness is necessary for conditioning. Allen and Janiszewski (1989) found that contingency awareness existed when conditioning occurred in an advertising situation, suggesting that conditioning may not occur without awareness. The nonaware participants did show a small conditioning effect in their initial experiment, and so their data are not unequivocal with respect to the role of awareness on conditioning. Priluck and Till (2004) found that awareness of the contingency in an ad increased the degree of conditioning. Bierley et al. (1985) found conditioning of colors paired with attractive music; and awareness increased conditioning

2/16/2011 5:55:22 PM

498

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

but conditioning did not require it. Shimp, Stuart, and Engle (1991) found that awareness increased conditioning as well. Baeyens and colleagues, albeit not using an advertising situation, have found that evaluative conditioning in humans, a type of conditioning that seems quite related to advertising effects since outcomes are used to modify the attitude toward a stimulus, is not dependent on contingency awareness (see Chapter 18, this volume). Baeyens et al. (1988) provided evidence that evaluative conditioning is retained for at least a 2-month period and is resistant to extinction. Extinction refers to the typical loss in conditioned responding when the CS is now presented without the CS following the original conditioning (pairings of the CS and the US). Baeyens and De Houwer and colleagues have discussed the difference between signal learning and evaluative conditioning (the latter is also assumed to be a form of referential learning). Expectancy learning involves a cue that predicts or signals the presence or absence of an outcome, and an expectancy regarding this outcome is produced. This theoretically deﬁned form of learning corresponds to Pavlovian conditioning or classical conditioning. In referential learning, the CS makes one think (consciously or unconsciously) of the outcome without activating an expectancy of the US (see De Houwer et al., 2001). It may be valuable to map various advertising effects onto these processes; that is, is the processing that occurs during ad exposure more like expectancy learning or referential learning? The current and published work on contingency awareness during ad viewing may begin to answer these questions. It is easy to imagine a larger wave of assent for a referential learning view of ad processing than a signal learning view. Nonetheless, I can imagine many young adults claiming quite consciously that they expect to have a good time when they are drinking Bud Lite. Bud Lite signals a good time. These conclusions could be the results of experience with Bud Lite or the result of viewing ads, or both. But referential learning also seems quite common in low-involvement advertising effects, and many would claim that explicit awareness of the contingency is not needed for such effects. Baeyens

21-Schachtman-21.indd 498

et al. (1988) also dissociated expectancy learning from referential learning in that signal learning may be less resistant to extinction. Reﬂecting on Brewer’s conclusion that explicit knowledge about the relationship between a CS and a US can inﬂuence the CR, we agree with the conclusion of McSweeney and Bierley (1984) that just because manipulation of awareness can inﬂuence conditioning does not necessarily mean that awareness is necessary. Awareness of the contingency appears to enhance conditioning, but it is likely not necessary for conditioning to occur. It is our suggestion that the referential-signal learning distinction is an extremely important one (see also the Introduction to Reilly & Schachtman, 2009), but that implicit and explicit processes can apply to both of them (depending on the circumstances). One very promising approach that mirrors much of the research in animal conditioning and human cognition is work that attempts to empirically dissociate different processes that might be producing an effect. Such research makes predictions that one outcome of the experiment will occur if one theoretical process is at work while another outcome will occur if the alternative process is inﬂuencing the subjects. This approach was pursued by Janiszewski and Warlop (1993).

PERSONALITY AND INDIVIDUAL DIFFERENCE VARIABLES This section will review a few characteristics of the individual that can affect the inﬂuence of conditioning during advertising. Many advertising researchers mention in their reports that they acknowledge individual differences among people with respect to such effects; and, rather than approaching an advertisement with a tabula rasa, people are exposed to a trial while possessing a history of experience as well as personality differences and acquired biases and heuristics. High Versus Low Involvement

Many marketing researchers have examined the role of involvement during advertising. Involvement is a concept initiated by Krugman

2/16/2011 5:55:22 PM

CONDITIONING AND ADVERTISING

(1967), but there has been little agreement on the deﬁnition of involvement (see, for example, Burnkrant & Sawyer, 1983). Involvement can be said to refer to the connections a person has with the stimulus prior to the target experience of interest. High involvement, of course, suggests a stronger and qualitatively different set of connections with the stimulus, which often means that such a person might be interested in purchasing the item. The deﬁnition of involvement provided by Celsi and Olsen (1988) is the degree of perceived personal relevance. Cohen (1990) describes high involvement as perceiving the product as having a large number of perceived beneﬁts. Some individuals may have low involvement for a particular product, brand, stimulus, or US, whereas others may have high involvement. Involvement can have a large impact on conditioning. Batra and Ray (1983) discovered that different processes may result from exposure to advertisements with low versus high involvement. Speciﬁcally, low involvement results in simple awareness (a low level of cognitive process), which may result in action, but little affective attitude change will result from the process; whereas high involvement can results in awareness, comprehension, action, and then affective attitude change. Lutz (1985) stated that affect transfer is higher when low involvement occurs. Gardiner et al. (1985) provide an extensive discussion of involvement. As mentioned earlier, Gardiner et al. found that memory differences occur for low- versus high-involvement processing. Grossman (1996) found more conditioning for highly involved participants. Priluck and Till (2004) noted that highly involved subjects will use belief information, whereas those low in involvement may only be subject to affective transfer. This point appears to conﬂict with the conclusions of Batra and Ray, who suggest that beliefs are needed for affective transfer to occur. Celsi and Olsen (1988) found that participants devote more attention to stimuli (brands) if they have high involvement (i.e., described as situational factors of the person’s environment that contribute to personal relevance). A high amount of a second type of involvement, “intrinsic involvement,” also resulted in a large amount

21-Schachtman-21.indd 499

499

of time attending to the stimulus, but it did not produce an independent source of attention (based on other measures they looked at, see Celsi & Olsen, 1988). Note that Celci and Olsen found that greater involvement produced more time attending to the information, whereas Gardner et al. (1985) found that the noninvolved group spent a longer time looking at each ad. It should be pointed out that Gardner et al. had the noninvolvement condition engaged in a mundane but potentially demanding task in which they had to look for grammatical, word-sound, and conceptual features, which can explain why this group looked at the ad such a long time. Gardner et al. (1985) distinguish between stages of processing based on low and high involvement such that low involvement while viewing advertisements involves basic, minimal comprehension in which the basic meaning of the elements in an ad are recognized (see also Vakratsas & Ambler, 1999), whereas high involvement produces elaboration such that internally generated information occurs. They discussed the issue of distraction during ad presentation, noting it can produce more favorable brand attitudes because distraction requires attention that disrupts elaborative processes (such as the production of counterarguments). If you are not interested in purchasing the product, then you may not engage in elaboration, but purchase-oriented individuals will elaborate and make inferences, and associate these inferences with the product (and counterarguments may arise). Gorn (1982, Experiment 2) examined subjects that were in a non-decision-making context (e.g., what can be considered eliciting low involvement) and those in a decision-making context with respect to their relative sensitivity to background cues (i.e., music) and to product information. Gorn found that non-decisionmaking participants were most inﬂuenced by the music, whereas decision makers (more involved) were most inﬂuenced by the information provided. Some additional points about involvement will be made in the following text, but we will not have printed space to be able to elaborate on these issues at this time. Bruner (1990) discusses

2/16/2011 5:55:22 PM

500

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

a fact likely well known to marketing researchers— some items elicit more cognitive involvement than others for most people. For instance, items with high cognitive involvement include appliances and vehicles, whereas low cognitive involvement occurs for other products such as jewelry and beer (and Vakratsas & Ambler, 1999 noted that frequently purchased packaged goods often induce low involvement). Complex ads require inferences that use cognitive elaborative analysis and high-involvement occurs (see Vakratsas & Ambler, 1999 for a discussion). Alternatively, it has been said that simple conditioning effects stem from less complex ads, such ads will be processed without elaborative cognitive evaluation, and lower involvement will occur (Petty, Cacioppo, & Schumann, 1983). Vakratsas and Ambler stated that with low involvement, “advertising merely serves to reinforce behavior rather than causing it” and that “the ‘weak theory’ of advertising (Jones, 1990) … is similar to operant, or instrumental, conditioning … .” This article mentions that, according to one model (the IIRM model), ads inﬂuence low-involvement situations by increasing awareness via lower order beliefs and introducing uncertainty, such that experience with the product will resolve the uncertainty and allow the expectations to be conﬁrmed or not. Higher order beliefs occur with high-involvement products or after many purchases of a product (but note that frequent purchases may produce lowinvolvement interactions with the product even if higher order beliefs exist). Need for Cognition

Priluck and Till (2004) examined the role of “need for cognition” on advertising. Need for cognition (Cacioppo & Petty, 1982) is a personality construct in which individuals enjoy thinking about events and engaging in difﬁcult cognitive processes. Priluck and Till found that individuals with high need for cognition showed the greatest conditioning during advertisements, and they recalled the information better later. They stated that these participants were more likely, due to their extra thought processing, to be aware of the pairing of the events. Priluck and Trill also suggested ways that one can inspire

21-Schachtman-21.indd 500

other individuals to engage in such extra processing: (1) pay them for paying attention; (2) tell them that they will be questioned about the material later; or (3) make the information relevant to their lives so they are motivated to process the information. Perhaps related to high need for cognition, Cohen (1990) discussed the dimension of “cognitive complexity” and research ﬁndings by Zinkhan and Martin (1983) that those individuals that are high in cognitive complexity preferred more complex ads (and those low in cognitive complexity prefer simple ads) and so the adage that “simple ads are always better” may not always hold true (see Cohen, 1990). The Role of Prior Belief

Many marketing researchers (e.g., Kim et al., 1998; Stammerjohan et al., 2005) note that an individual possesses prior beliefs about a product and this can greatly inﬂuence the individual’s current assessment of the product when viewing an ad. Vakratsas and Ambler (1999, p.27) remind us that: “… the consumer’s mind is not a blank sheet awaiting advertising but rather already contains conscious and unconscious memories of product purchasing and usage. Thus, behavior feeds back to experience …” Stammerjohan et al. (2005) point out that individuals that already have opinions about a familiar product do not always process new information very effectively. When analyzing the relationship between two types of information—the current “situational data” in the present advertisement and the data from past experience—we cannot help but allude to Alloy and Tabachnik’s (1984) article on the relationship between these types of information. Like the conclusions of many marketing researchers, this article points out how inﬂuential previously acquired knowledge can be for the processing of current information; they discuss many of the learning phenomena mentioned in this review, including latent inhibition, blocking, and the US preexposure effect. However, we wish to point two opposing processes that highlight this interaction between past and current information. On the one hand, there are a lot of data showing that individuals

2/16/2011 5:55:22 PM

CONDITIONING AND ADVERTISING

(relatively speaking) disregard current data because of knowledge that has already been acquired. On the other hand, effects such as the “hindsight bias” (see Hawkins & Hastie, 1990) show that subjects exposed to contemporaneous events (e.g., the outcome of an election) will bias their past processing greatly in favor of making that event ﬁt into their existing schema. Hence, if Candidate A wins the election, people will distort their own history to convince themselves (erroneously) that they predicted Candidate A would win “all along.” Future research should isolate which variables determine when current data are weighed so heavily that they result in the transformation of existing data in memory (e.g., hindsight bias) and when current data are more or less neglected because of the strong inﬂuence of previously acquired knowledge (e.g., blocking, latent inhibition).

CONCLUSION As most marketing researchers know, certain factors have been found to have a large impact on the effectiveness of advertising, including (1) the degree of involvement; (2) the temporal placement of the brand name during the ad; (3) the use of music; (4) the relationship between product information and affective qualities; and (5) the extent that the processing during an ad is implicit (occurs without awareness) or explicit (occurs with awareness), to name just a few. Given that many (or most ads) involve a classical conditioning procedure, it is not surprising that these processes are also critical in the ﬁeld of conditioning theory per se and they also illustrate how conditioning theory can shed light on the processing of ads. As a ﬁnal note, we wish to point out three additional issues in conditioning research that might offer interest for marketing researchers. These three conditioning phenomena are, in our opinion, among the more recent and fascinating areas of such work. First, marketing researchers should be aware that conditioning can occur for cues that are not present on a particular conditioning trial. For example, if two CSs are associated (CSA and CSB) and two other events are associated (CSC and the US), if CSA and CSC are

21-Schachtman-21.indd 501

501

presented, then CSB and the US can become associated since CSA will activate the representation of CSB and CSC will activate a representation of the US. Hence, the representations of CSB and the US will be contiguously active in memory. Research has found many instances in which CSs that are not present on a trial are changed in their associative strength (Dickinson & Burke, 1996; Holland & Wheeler, 2009; Van Hamme & Wasserman, 1994). Such outcomes, sometimes called “representation-mediated conditioning” are an important and exciting area of conditioning research and may apply to a brand that is not even present during an ad. Secondly, many instances in which an individual shows no evidence of having acquired information are cases in which the information has been acquired but is not retrieved or performed due to lack of motivation or poor retrievability of the information (Lewis, 1979; Miller et al., 1986; Tolman & Honzik, 1930; Warrington & Weiskrantz, 1968). Many of the phenomena discussed in this chapter (blocking, latent inhibition, overshadowing) in which a relatively poor CR is observed have been shown to be due to a retrieval problem rather than a lack of acquisition of the association (Miller et al., 1986). Hence, marketing researchers may value knowing that if few trials are used (or a brief stimulus presentation) and that causes the individual to show little evidence of a change in attitude, acquisition of such a change in attitude or affect might have occurred although it is not expressed. Certain tests can be used to show that processing did occur in the past even though it is not presently manifest in behavior. The effects reported in this chapter illustrate the ways in which conditioning and marketing can beneﬁt from the interdisciplinary conﬂuence of ﬁndings and theories. As mentioned in this chapter, D’Souza and Rao (1995) describe two different models of information processing: an accumulation model and a replacement model (see Stewart, 1989). The accumulation model claims that new information is acquired alongside of the old information, and it is the relative strength of the new and old responses that determine which will be expressed. A view of performance deﬁcits that focuses on retrievability will

2/16/2011 5:55:22 PM

502

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

likely claim that new information is acquired along with old information, but the less seemingly adaptive information will be poorly retrieved and other, more adaptive information will be better retrieved (Miller et al., 1986). Indeed, newly acquired information can compete for retrievability with the earlier learned information. Hence, associations compete for retrievability and the relative retrievability of the cues/brands will determine which are expressed in behavior. Finally, as noted earlier, brand competition effects during an ad have begun to be explored by advertising researchers (Janiszewski & van Osselaer, 2000; van Osselaer & Alba, 2000; van Osselaer & Janiszewski, 2001). Moreover, van Osselaer and Janiszewski (2001) also examined a conditioning phenomenon often referred to in the conditioning literature as “retrospective revaluation” or, as van Osselaer and Janiszewski called it, “backlooking learning.” This phenomenon involves competition between features or brands (CSA and CSB) such that one brand or product (CSA) is dominant in getting control over the participant’s behavior (e.g., purchasing power or attention or memory) as a result of the ad. Then, subsequently, this brand or feature has its status changed such that it loses value (the feature or product becomes less credible or less interesting). When this happens, even without any additional presentations of the other brand (CSB), this alternative brand or product is increased in its status. That is, CSB now is improved in its ability to inﬂuence the person’s performance, even though it had not been presented between the viewing of the ad and the ﬁnal assessment of this behavioral control (i.e., only CSA was manipulated). The competition between cues can be inﬂuenced by the later change in status of one cue, and this will inﬂuence the other cue. This ﬁnding, retrospective revaluation, has been very impactful in the human contingency judgment literature as well as in animal conditioning; and van Osselaer and Janiszewski (2001) were the ﬁrst to examine its potential in an advertising situation (see also Chapters 1 and 8, this volume, for additional discussion or examples of this effect).

21-Schachtman-21.indd 502

There are many ways that the ﬁelds of conditioning and marketing potentiate each others’ ﬁndings and theories. Many additional conditioning phenomena have been and will continue to be applied to an advertising setting (e.g., Till & Priluck, 2000 and its application to brand extensions) for the mutual beneﬁt of both ﬁelds.

REFERENCES Allen, C. T., & Janiszewski, C. A. (1989). Assessing the role of contingency awareness and attitudinal conditioning with implications for advertising research. Journal of Marketing Research, 26, 30–43. Allen, C. T., & Madden, T. J. (1985). A closer look at classical conditioning. Journal of Consumer Research, 12, 301–315. Allen, C. T., & Shimp, T. A. (1990). On using classical conditioning methods for researching the impact of ad-evoked feelings. In S. J. Agras, J. A. Edell, & T. M. Dubitsky (Eds.), Emotion in advertising: Theoretical and practical explorations. Westport, CT: Quorum Books. Alloy, L., & Tabachnik, N. (1984). Assessment of covariation by humans and animals: The joint inﬂuence of prior expectations and current situational information. Psychological Review, 91, 112–149. Alpert, F. H., & Kamins, M. A. (1995). An empirical investigation of consumer memory, attitude and perceptions toward pioneer and follower brands. Journal of Marketing, 59, 34–45. Anderson, N., & Bower, G. H. (1973). Human associative memory. Oxford, England: Winston & Sons. Baeyens, F., Crombez, G., Van Den Bergh, O., & Eelen, P. (1988). Once in contact always in contact: Evaluative conditioning is resistant to extinction. Advances in Behavior Research and Therapy, 10, 179–199. Baker, W. E. (1999). When can affective conditioning and mere exposure directly inﬂuence brand choice? Journal of Advertising, 28, 31–46. Baker, W. E., Honea, H., & Russell, C. A. (2004). Do not wait to reveal the brand name: The effect of brand-name placement on television advertising effectiveness. Journal of Advertising, 33, 77–85. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact always in

2/16/2011 5:55:22 PM

CONDITIONING AND ADVERTISING

contact: Evaluative conditioning is resistant to extinction. Advances in Behavior Research and Therapy, 10, 179–199. Balsam, P. D., & Tomie. A. (1985). Context and learning. Hillsdale, NJ: Erlbaum. Batra, R., & Ray, M. L. (1983). Advertising situations: The implications of differential involvement and accompanying affective responses. In R. J. Harris (Ed.), Information processing research in advertising (pp. 127–151). Hillsdale, NJ: Erlbaum. Bierley, C., McSweeney, F. K., & Vannieuwkerk, R. (1985). Classical conditioning of preferences for stimuli. Journal of Consumer Research, 12, 316–323. Blair, M. (1987). An empirical investigation of advertising wearin and wearout. Journal of Advertising Research, 27, 45–50. Blair, M. E., & Shimp, T. A. (1992). Consequences of an unpleasant experience with music: A second-order negative conditioning perspective. Journal of Advertising, 21, 35–43. Bonardi, C., & Ong, S. Y. (2003). Learned irrelevance: A contemporary overview. Quarterly Journal of Experimental Psychology, 56B, 80–89. Brewer, W. F. (1974). There is no convincing evidence for operant or classical conditioning in adult humans. In W. B. Weimer, & D. S. Palermo (Eds.), Cognition and the symbolic processes. Hillsdale, NJ: Lawrence Erlbaum. Brown, P. L., & Jenkins, H. M. (1968). Autoshaping of the pigeon’s key-peck. Journal of the Experimental Analysis of Behavior, 11, 1–8. Bruner, G. C., II. (1990, October). Music, mood and marketing. Journal of Marketing, 54(4), 94–104. Burke, R. R., & Srull, T. K. (1988). Competitive interference and consumer memory for advertising. Journal of Consumer Research, 15, 55–68. Burnkrant, R. E., & Sawyer, A. G. (1983). Effects of involvement and message content on information-processing intensity. In R. J. Harris (Ed.), Information processing research in advertising (pp. 43–64). Hillsdale, NJ: Erlbaum. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131. Cacioppo, J. T., & Petty, R. E. (1985). Central and peripheral routes to persuasion: The role of message repetition. In A. A. Mitchell & L. F. Alwitt (Eds.), Psychological processes and advertising effects (pp. 91–112). Hillsdale, NJ: Erlbaum.

21-Schachtman-21.indd 503

503

Celsi, R. L., & Olson, J. C. (1988). The role of involvement in attention and comprehension processes. Journal of Consumer Research, 15, 210–224. Cohen, J. (1990). Attitude, affect, and consumer behavior. In B. M. Moore & A. M. Isen (Eds.), Affect and social behavior (pp. 152–206). New York, NY: Cambridge University Press. Crowder, R. G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum. Darley, W. K., & Lim, J-S. (1993). Assessing demand artifacts in consumer research: An alternative perspective. Journal of Consumer Research, 20, 489–495. Dawson, M. E., Beers, J. R., Schell, A. M. & Kelly, A. (1982). Allocation of cognitive processing capacity during human autonomic classical conditioning. Journal of Experimental Psychology: General, 111, 273–295. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dickinson, A., & Burke, J. (1996). Withincompound associations mediate the retrospective revaluation of causality judgments. Quarterly Journal of Experimental Psychology, 49B, 60–80. D’Souza, G., & Rao, R. C. (1995). Can repeating an advertisement more frequently than the competition affect brand preference in a mature market? Journal of Marketing, 59, 32–42. Ehrenberg, A. S. C. (1974). Repetitive advertising and the consumer. Journal of Advertising Research, 14, 25–34. Freeman, K. B., & Riley, A. L. (2009). The origins of conditioned taste aversion learning: A historical analysis. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Behavioral and neural processes (pp. 9–36). New York, NY: Oxford University Press. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gardiner, M. P., Mitchell, A. A., & Russo, J. E. (1985). Low involvement strategies for processing advertisements. Journal of Advertising, 14, 4–12, 56. Gibbon, J., & Balsam, P. D. (1981). Spreading association in time. In C. M. LoCurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York, NY: Academic Press.

2/16/2011 5:55:22 PM

504

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Gordon, W. (2001). The darkroom of the mind–What does neuropsychology now tell us about brands? Journal of Consumer Behavior, 1, 280–292. Gordon, W. C., & Weaver, M. S. (1988). The role of contextual stimuli in the blocking paradigm. Psychobiology, 16, 59–66. Gorn, G. J. (1982). The effects of music in advertising on choice behavior: A classical conditioning approach. Journal of Marketing, 46, 94–101. Gresham, L. G., & Shimp, T. A. (1985). Attitude toward the advertisement and brand attitudes: A classical conditioning perspective. Journal of Advertising, 14, 10–17. Grossman, R. P. (1996). Neo-classical conditioning and the elaboration likelihood model: A new perspective on the role of involvement. Dissertation Abstracts International, 56, 4859. Grossman, R. P., & Till, B. D. (1998). The persistence of classically conditioned brand attitudes. Journal of Advertising, 27, 23–31. Hall, G., & Channell, S. (1985). Differential effects of contextual change on latent inhibition and on the habituation of the orienting response. Journal of Experimental Psychology: Animal Behavior Processes, 11, 470–481. Hall, G., & Schachtman, T. R. (1987). Differential effects of a retention interval on latent inhibition and habituation of an orienting response. Animal Learning and Behavior, 15, 76–82. Hawkins, S. A., & Hastie, R. (1990). Hindsight: Biased judgments of past events after the outcomes are known. Psychological Bulletin, 107, 311–327. Holland, P. C., & Wheeler, D. S. (2009). Representationmediated food aversions. In S. Reilly & T. R. Schachtman (Eds.), Conditioned taste aversion: Neural and behavioral processes (pp.196–225). New York, NY: Oxford University Press. Hung, K. (2001). Framing meaning perceptions with music: The case of teaser ads. Journal of Advertising, 30(3), 39–49. Janiszewski, C. (1988). Implications of hemispheric processing theory for consumer research. Dissertation Abstracts International, 48, 2107. Janiszewski, C., & van Osselaer, S. M. J. (2000). A connectionist model of brand-quality associations. Journal of Marketing Research, 37, 331–350. Janiszewski, C., & Warlop, L. (1993). The inﬂuence of classical conditioning procedures on subsequent attention to the conditioned brand. Journal of Consumer Research, 20, 171–189. Johnston, W. A., & Hawley, K. J. (1994). Perceptual inhibition of expected inputs: The key that opens closed minds. Psychonomic Bulletin and Review, 1, 56–72.

21-Schachtman-21.indd 504

Jones, J. P. (1990). Advertising: Strong force or weak force? Two views an ocean apart. International Journal of Advertising, 9, 233–246. Kahle, L. R., Beatty, S. E., & Kennedy, P. (1987). Comment on classically music on ad processing: A contingency explanation. Journal of Marketing, 57(4), 114–125. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall. Kamin, L. J. (1969). Selective association and conditioning. In N. J. Mackintosh & W. K. Honig (Eds.), Fundamental issues in associative learning (pp. 42–64). Halifax, Nova Scotia: Dalhousie University Press. Kellaris, J. J., Cox, A. D., & Cox, D. (1993). The effect of background music on ad processing: A contingency explanation. Journal of Marketing, 57, 114–125. Kim, J., Allen, C. T., & Kardes, F. R. (1996). An investigation of the meditational mechanisms underlying attitudinal conditioning. Journal of Marketing Research, 33, 318–328. Kim, J., Lim, J. S., & Bhargara, M. (1998). The role of affect in attitude formation: A classical conditioning approach. Journal of the Academy of Marketing Science, 26, 143–152. Kroeber-Riel, W. (1984). Emotional product differentiation by classical conditioning (with consequences for the “low involvement hierarchy”). Advances in Consumer Research, 11, 538–543. Krugman, H. E. (1967). The measurement of advertising involvement. Public Opinion Quarterly, 30, 583–596. Krugman, H. E. (1972). Why three exposures may be enough. Journal of Advertising Research, 12, 11–14. Lazarus, R. S. (1981). A cognitivist’s reply to Zajonc on emotion and cognition. American Psychologist, 36, 222–223. Lewis, D. J. (1979). Psychobiology of active and inactive memory. Psychological Bulletin, 86, 1054–1083. Lubow, R. E., & Moore, A. U. (1959). Latent inhibition: The effect of nonreinforced preexposure to the conditioned stimulus. Journal of Comparative and Physiological Psychology, 52, 415–419. Lutz, R.J. (1985). Affective and cognitive antecedents of attitude toward the ad: A conceptual framework. In L. F. Alwitt & A. A. Mitchell (Eds.), Psychological processes and advertising effects: Theory, Research and Application. Hillsdale, NJ: Erlbaum. Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.

2/16/2011 5:55:22 PM

CONDITIONING AND ADVERTISING

Macklin, M. C. (1996). Preschoolers’ learning of brand names from visual cues. Journal of Consumer Research, 23, 251–261. Matzel, L. D., Schachtman, T. R., & Miller, R. R. (1988). Learned irrelevance exceeds the sum of CS-preexposure and US-preexposure deﬁcits. Journal of Experimental Psychology: Animal Behavior Processes, 14, 311–319. McSweeney, F. K., & Bierley, C. (1984). Recent developments in classical conditioning. Journal of Consumer Research, 11, 619–631. Miller, R. R., Kasprow, W. J., & Schachtman, T. R. (1986). Retrieval variability: Psychobiological sources and consequences. American Journal of Psychology, 99, 145–218. Miller, R. R., & Matzel, L. D. (1988). The comparator hypothesis: A response rule for the expression of associations. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 22, pp. 51–92). San Diego, CA: Academic Press. Miller, R. R., & Schachtman, T. R. (1985). Conditioning context as an associative baseline: Implications for response generation and the nature of conditioned inhibition. In R. R. Miller & N. E. Spear (Eds.), Information processing in animals: Conditioned inhibition (pp. 51–88). Hillsdale, NJ: Erlbaum. Mitchell, A. A. (1993). Attitude toward the advertising effects over time and in attitude change situations. In A. A. Mitchell (Ed.), Advertising exposure, memory and choice. Hillsdale, NJ: Erlbaum. Moore, D. L., & Hutchinson, J. W. (1985). The inﬂuence of affective reactions to advertising: Direct and indirect mechanisms of attitude change. In L. F. Alwitt & A. A. Mitchell (Eds.), Psychological processes and advertising effects: Theory, research and application (pp. 65–87). Hillsdale, NJ: Erlbaum. Nairne, J. S., & Rescorla, R. A. (1981). Secondorder conditioning with diffuse auditory reinforcers in the pigeon. Learning and Motivation, 12, 65–91. Oxoby, R. J., & Finnigan, H. (2007). Developing heuristic-based quality judgments: Blocking in consumer choice. Psychology and Marketing, 24, 295–313. Pavlov, I. P. (1927). Conditioned reﬂexes (G. V. Anrep, Trans.). London, England: Oxford University Press. Pearce, J. M. (2008). Animal learning and cognition: An introduction (3rd ed.). New York, NY: Psychology Press.

21-Schachtman-21.indd 505

505

Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not unconditioned stimuli. Psychological Review, 87, 532–552. Pechmann, C., & Stewart, D. W. (1988). Advertising repetition: A critical review of wearin and wearout. Current Issues and Research in Advertising, 11, 285–329. Petty, R. E., Cacioppo, J. T., & Schumann, D. (1983). Central and peripheral routes to advertising effectiveness: The moderating role of involvement. Journal of Consumer Research, 10, 135–146. Pitt, L. F., & Abratt, R. (1988). Music in advertisement for unmentional products–A classical conditioning experiment. International Journal of Advertising, 7, 130–137. Priluck, R. & Till, B. D. (2004). The role of contingency awareness, involvement, and need for cognition in attitude formation. Journal of the Academy of Marketing Science, 32, 329–344. Randich, A., & Ross, R. T. (1984). Mechanisms of blocking by context. Learning and Motivation, 15, 106–117. Rao, U. H., & Burnkrant, R. E. (1991). Effects of repeating varied ad executions on brand name memory. Journal of Marketing Research, 28, 406–426. Reed, P., McCarthy, J., Latif, N., & DeJongh, J. (2002). The role of stimuli in a virtual shopping environment: A test of predictions derived from conditioning models of marketing ﬁrms. Journal of Economic Psychology, 23, 449–467. Reilly, S., & Schachtman, T. R. (Eds.). (2009). Conditioned taste aversion: Neural and behavioral processes. New York, NY: Oxford University Press. Rescorla, R. A. (1969). Pavlovian conditioned inhibition. Psychological Bulletin, 72, 77–94. Rescorla, R. A., & Gillan, D. J. (1980). An analysis of the facilitative effect of similarity on second-order conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 6, 339–351. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: AppletonCentury-Crofts. Rothschild, M. L., & Hyun, Y. J. (1990). Predicting memory for components of TV commercials from EEG. Journal of Consumer Research, 16, 472–478.

2/16/2011 5:55:22 PM

506

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Shimp, T. A. (1991). The role of subject awareness in classical conditioning: A cause of opposing ontologies and conﬂicting evidence. Advances in Consumer Research, 18, 158–163. Shimp, T. A., Hyatt, E. M., & Snyder, D. J. (1993). A critique of Darley and Lim’s “Alternative Perspectives.” Journal of Consumer Research, 20, 496–501. Shimp, T. A., Stuart, E. W., & Engle, R. W. (1991). A program of classical conditioning experiments testing variations in the conditioned stimulus and context. Journal of Consumer Research, 18, 1–12. Smith, R. E. (1993). Integrating information from advertising and trial: Processes and effects on consumer response to product information. Journal of Marketing Research, 30, 204–219. Sokolov, Y. N. (1963). Higher nervous functions: The orienting reﬂex. Annual Review of Physiology, 25, 545–580. Spear, N. E. (1978). The processing of memories: Forgetting and retention. Hillsdale, NJ: Erlbaum. Staats, A. W., & Staats, C. K. (1958). Attitudes established through classical conditioning. Journal of Abnormal and Social Psychology, 57, 37–40. Staats, C. K., & Staats, A. W. (1957). Meaing established through classical conditioning. Journal of Experimental Psychology, 54, 74–80. Stewart, D. W. (1989). Measures, methods, and models in advertising research. Journal of Advertising Research, 29, 54–60. Stammerjohan, C., Wood, C. M., Chang, Y., & Thorson, E. (2005). An empirical investigation of the interaction between publicity, advertising, and previous brand attitudes and knowledge. Journal of Advertising, 34, 55–67. Stout, P. A., & Rust, R. T. (1993). Emotional feelings and evaluative dimensions of advertising: Are they related? Journal of Advertising, 22, 61–71. Stuart, E. W., Shimp, T. A., & Randall, W. (1987). Classical conditioning of consumer attitudes: Four experiments in an advertising context. Journal of Consumer Research, 13, 334–339. Swan, J. A., & Pearce, J. M. (1988). The orienting response as an index of associability in rats. Journal of Experimental Psychology: Animal Behavior Processes, 14, 292–301. Till, B. D., & Priluck, R. L., (2000). Stimulus generalization in classical conditioning: An initial investigation and extension. Psychology and Marketing, 17, 55–72.

21-Schachtman-21.indd 506

Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward, and maze performance in rats. University of California Publications in Psychology, 4, 257–275. Tomie, A., Murphy, A. L., Fath, S., & Jackson, R. L. (1980). Retardation of autoshaping following pretraining with unpredictable food: Effects of changing the context between pretraining and testing. Learning and Motivation, 11, 117–134. Tulving, E., & Thomson, D. M. (1971). Retrieval processes in recognition memory: Effects of associative context. Journal of Experimental Psychology, 87, 116–124. Vakratsas, D., & Ambler, T. (1999). How advertising works: What do we really know? Journal of Marketing, 63, 26–43. Van Hamme, L. J., & Wasserman, E. A. (1994). Cue competition in causality judgments: The role of nonpresentation of compound stimulus elements. Learning and Motivation, 25, 127–151. Van Osselaer, S. M. J. (2008). Associative learning and consumer decisions. In C. P. Haugtvedt, P. M. Herr, & K. R. Kardes (Eds.), Handbook of consumer psychology (pp. 699–729). New York, NY: Erlbaum. Van Osselaer, S. M. J., & Alba, J. W. (2000). Consumer learning and brand equity. Journal of Consumer Research, 27, 1–16. Van Osselaer, S. M. J., & Janiszewski, C. (2001). Two ways of learning brand associations. Journal of Consumer Research, 38, 202–223. Winters, L. C., & Wallace, W. H. (1970). On operant conditioning techniques. Journal of Advertising Research, 10, 39–45. Warrington, E. K., & Weizkrantz, L. (1968). A new method of testing long-term retention with special reference to amnesic patients. Neuropsychologia, 6, 283–291. Winters, L. C., & Wallace, W. H. (1970). On operant conditioning techniques. Journal of Advertising Research, 10, 39–45. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inference. American Psychologist, 35, 151–175. Zielske, H., & Henry, W. (1980). Remembering and forgetting television advertisements. Journal of Advertising Research, 20, 7–13. Zinkhan, G.M., & Martin, C.R., Jr. (1983). Message characteristics and audience characteristics: Predictors of advertising response. Advances in Consumer Research, 10, 27–31.

2/16/2011 5:55:22 PM

CHAPTER 22 Applications of Pavlovian Conditioning to Sexual Behavior and Reproduction Michael Domjan and Chana K. Akins

Pavlovian conditioning procedures can be readily adapted to the sexual behavior system. In reviewing studies of sexual conditioning, we focus on how conditioning modiﬁes sexual behavior and how this contributes to reproductive success. We ﬁrst consider how principles of learning that have been developed in conventional laboratory settings generalize to the sexual behavior system. We then discuss how studies of sexual conditioning have contributed to the understanding of sexual functioning. Finally, we examine how the results of sexual conditioning studies with laboratory animals (mostly domesticated quail and rats) might be used in the design and interpretation of human studies of sexual conditioning. Our review focuses on studies with male participants because much more research has been conducted with males than with females. However, we have included studies of female sexual conditioning in both human and non-human animals to the extent that they were available and relevant.

INTRODUCTION The basic Pavlovian or classical conditioning paradigm is highly familiar. It is commonly described as pairing an initially “neutral” or ineffective stimulus (the conditioned stimulus [CS]) with a biologically signiﬁcant event (the unconditioned stimulus [US]). After a sufﬁcient number of pairings, the CS is no longer behaviorally neutral and comes to elicit a conditioned response. Pavlov presented food as the unconditioned stimulus to the dogs in his experiments and measured salivation as the conditioned response. Since salivation was initially elicited as a reﬂex response to food, Pavlov’s procedure was characterized as the conditioning of a reﬂex. Food continues to be used as the unconditioned stimulus in many contemporary studies of Pavlovian conditioning but more often with rats and pigeons as experimental participants than dogs. Other common conditioning

preparations include fear conditioning, in which a brief shock to the feet of laboratory rats serves as the US, and eyeblink conditioning, in which irritation of the skin near the eye serves as the US. The objective of many of these experiments is to elucidate general mechanisms of learning rather than examine how Pavlovian conditioning modiﬁes the operations of the feeding or defensive behavior systems or how Pavlovian conditioning contributes to the adaptive functioning of the organism. Our goal in this chapter is to review research on the applications of Pavlovian conditioning to sexual behavior. Our focus is not on the conditioning of a reﬂex. Rather, we are interested in how conditioning modiﬁes the sexual behavior system and how this contributes to reproductive success. Like feeding, sexual behavior and reproduction are critical to the evolutionary survival and success of a species. In fact, one might argue that sexual behavior is so important that it cannot be left to the uncertainties of conditioning 507

22-Schachtman-22.indd 507

2/16/2011 6:52:29 PM

508

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

and learning, which typically requires multiple training trials. Perhaps because of this perspective, most prior research on sexual behavior has focused on its neuroendocrine mechanisms and the emergence of those mechanisms in ontogenetic development (e.g., Ball & Balthazart, 2004; McCarthy & Bell, 2008). However, research in the last 25 years has shown that sexual behavior is no less susceptive to learning than feeding or defensive behavior. Furthermore, sexual conditioning can lead not only to anticipatory conditioned responses (analogous to anticipatory salivation) but also increases in the efﬁciency of the unconditioned response (copulation) and increases in the number of offspring that are produced by a copulation episode. We will concentrate on studies of sexual conditioning involving the Japanese or domesticated quail, Coturnix japonica, because this species has been studied most extensively in this type of research and because we are most familiar with sexual conditioning in this species. However, we will mention ﬁndings with other species, including Homo sapiens, when appropriate. Coturnix japonica was domesticated in the 12th century in Japan and is a common species in poultry science research. Domesticated quail have been popular in studies of sexual conditioning because of their small size and ready adaptation to laboratory housing. They are seasonal breeders in the wild, but the breeding season can be extended in the laboratory by maintaining the birds on a long photoperiod (16 hr light and 8 hr darkness daily). Under these conditions, females lay an egg nearly every day, and male-female pairs readily copulate when brought together. The male typically initiates copulation by grabbing the back of the female’s neck, mounting on top of the female with both feet, and then making a series of cloacal thrusts, juxtaposing its cloaca against that of the female for sperm transfer. Unlike unconditioned salivation to food or an eyeblink response to irritation of the eye, copulation between a male and a female is followed by a substantial refractory period. This limits the frequency with which copulation can be used as a US. In most of the research described in this chapter, conditioning trials were spaced at least 24 hours apart.

22-Schachtman-22.indd 508

SEXUAL CONDITIONING AND THE GENERALITY OF THE PRINCIPLES OF LEARNING The ﬁrst study of the conditioning of sexual behavior in domesticated quail was conducted by Howard Farris in 1964, as a Ph.D. dissertation project at Michigan State University, part of which was published (Farris, 1967). Conditioning trials consisted of presenting a soft buzzer as the CS for 10 seconds to male quail, followed by access to a female (the US). Three males received the CS paired with the US and two males received the CS and US unpaired. Conditioning was evident in components of male courtship behavior being elicited by the CS. These included increased body tonus, increased stiffening of the legs, toe walking, vocalization, and feather pufﬁng. Farris’s original experiment generated considerable interest but no empirical replications for about 20 years. The next study of sexual conditioning in domesticated quail reported several experiments in which a light served as the CS preceding access to a female quail (Domjan, Lyons, North, & Bruell, 1986). In addition to recording the courtship and vocalization responses that were described by Farris, Domjan et al. measured approach to the CS, hoping to observe the development of sign tracking as the conditioned response (Hearst & Jenkins, 1974). Eight males received paired presentations of the CS and sexual reinforcement and eight males received an explicitly unpaired control procedure. The ﬂoor area of the experimental chamber measured 121 cm x 91 cm, with the CS positioned in the middle of one wall. The subjects were considered to have approached the CS if they were in a small area (40 cm x 30 cm) in front of the CS light. As predicted, sexual conditioning resulted in rapid acquisition of approach to the CS or sign tracking (see Fig. 22.1). By the sixth trial, acquisition was close to asymptote for the paired subjects, who spent signiﬁcantly more time near the CS than the unpaired control group. Interestingly, in this experiment the action patterns that Farris identiﬁed as conditioned courtship responses did not occur in response to the CS or in anticipation of copulation. Rather, these responses

2/16/2011 6:52:29 PM

SEXUAL BEHAVIOR AND REPRODUCTION 10

Seconds Near CS Light

CS-US Paired 8 6 4 CS/US Unpaired 2 0 0

5

10

15

20

25

Trials

Figure 22.1 Acquisition of sexual conditioned approach behavior to the conditioned stimulus (CS) in male quail. The solid line represents data for subjects that received paired presentations of the CS with sexual reinforcement. The dotted line represents data for unpaired control subjects.

occurred predominantly after copulation with the female. Because approach to the CS or sign tracking was the predominant conditioned response in the study by Domjan et al. (1986), numerous subsequent experiments employed this response measure. However, the interpretation of those studies requires resolving two major questions. First, was the CS–approach response controlled by a Pavlovian CS-US relationship, or was it controlled by instrumental reinforcement? Second, were the subjects approaching the CS or approaching the door where the female was to be released? Since the light that served as the CS was close to the door from which the female was released, approaching the female door (goal tracking) would have been measured as CS approach. The omission control procedure was developed to determine whether responses that are acquired in a Pavlovian conditioning experiment reﬂect a Pavlovian CS-US relation or instrumental reinforcement of the conditioned response (Shefﬁeld, 1965). In the omission control procedure, the CS is followed by the US if the subject does not make the CR. However, if the CR occurs on a particular trial, the US is omitted on that trial. Thus, the omission contingency insures

22-Schachtman-22.indd 509

509

that the CR is not reinforced by presentation of the US. Crawford and Domjan (1993) showed that sexually conditioned CS approach behavior is acquired by male domesticated quail even if an omission control procedure is implemented, thus ruling out instrumental conditioning as the mechanism of the learning. Let us next turn to whether sexually conditioned quail are approaching the CS (sign tracking) or approaching the location where the female US is released (goal tracking). Goal tracking is a common measure of appetitive Pavlovian conditioning. Rats conditioned with food as the US, for example, will often nose the food cup during the CS in anticipation of the US (e.g., Boakes, 1977; Nelson, 2009). To help decide whether a conditioned approach response reﬂects sign tracking rather than goal tracking, the recommended strategy is to position the CS some distance from the goal object (Hearst & Jenkins, 1974). Figure 22.2 shows the ﬂoor plan of the apparatus in one such experiment (Burns & Domjan, 1996, Experiment 2). The CS was a small wood block lowered from the ceiling. For independent groups, the CS was presented in the front of the experimental chamber (where the female was released) or in the middle or the back (91 cm from the door to the female compartment). In addition, after 15 conditioning trials, each group was tested with the CS in each of the three locations. The subjects

+

122 cm

Figure 22.2 Floor plan for measuring sign track-

ing and goal tracking. The conditioned stimulus (CS), represented by star, was presented at different distances from the door that provided access to the female for independent groups of males.

2/16/2011 6:52:30 PM

510

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

approached the CS regardless of the group to which they were assigned. Furthermore, the quail continued to approach the CS even if it was moved from the training location to one of the other locations during test trials. Experiments in which the CS is located some distance from the location of the US have been called “long-box” experiments, because the original study of this phenomenon employed a Skinner box for pigeons in which the food cup was located about 90 cm from a key light that served as the CS. To test the limits of sexually conditioned CS approach behavior, Burns and Domjan (2000) substantially increased the length of the experimental chamber they previously used, with the CS now located 233 cm from the location of the female. Such a spatial separation between CS and US is unprecedented in the conditioning literature. However, it did not prevent sexual approach conditioning. Even with such a long distance between the CS and the US, the conditioned behavior that developed was approach to the CS rather than approach to the goal location. Using CS approach as a measure of sexual conditioning, studies have shown that virtually any phenomenon that has been found in more familiar appetitive and fear-conditioning situations can be replicated in sexual conditioning. So far we have described acquisition, omission control, and “long-box” sign tracking. Other effects that have been documented include retention of conditioned behavior (Domjan et al., 1986), extinction (Domjan et al., 1986; Krause, Cusato, & Domjan, 2003), CS-US interval effects (Akins, Domjan, & Gutierrez, 1994), trace conditioning (Akins & Domjan, 1996), simple and conditional discrimination learning (e.g., Domjan, Akins, & Vandergriff, 1992), conditioned inhibition (Crawford & Domjan, 1996), context conditioning (Domjan, Greene, & North, 1989; Hilliard, Nguyen, & Domjan, 1997), US devaluation effects (Holloway & Domjan, 1993a), blocking (Köksal, Domjan, & Weisman, 1994), second-order conditioning (Crawford & Domjan, 1995), and observational conditioning (Köksal & Domjan, 1998). This impressive range of learning effects clearly proves that Pavlovian conditioning can be

22-Schachtman-22.indd 510

extended to sexual behavior. If the goal of the application is simply to demonstrate the generality of learning, one could declare victory and consider the job ﬁnished. However, the extension of learning to a new situation should also provide novel insights into the nature of the target system. We turn to that issue next.

WHAT SEXUAL CONDITIONING TELLS US ABOUT SEXUAL BEHAVIOR Unconditioned Stimulus Factors in Sexual Conditioning

So far we have described sexual conditioning as involving the presentation of a CS paired with a US that consists of the opportunity for a male to copulate with a female. Copulation is a highly effective reinforcer for sexual learning. However, if sexual conditioning only occurred in situations in which copulation took place, it would be of limited applicability because copulation is a relatively rare event in the lives of many animals. In an early experiment involving a straight alley runway, Shefﬁeld, Wulff, and Backer (1951) found that male rats did not have to copulate to ejaculation to learn to run down the alley for access to a female. Mounting and intromission were sufﬁcient to reinforce the instrumental running response. In a subsequent experiment, Zamble, Hadad, Mitchell, and Cutmore (1985) provided evidence of sexual Pavlovian conditioning in male rats using a procedure in which exposure to a female behind a barrier was the US. Noncopulatory exposure to a female also served as the US in studies of sexual conditioning with blue gourami ﬁsh (Hollis, Cadieux, & Colbert, 1989; Hollis, Pharr, Dumas, Britton, & Field, 1997). Visual exposure to sexual scenes often serves as the US in human studies of sexual conditioning (e.g., Hoffmann, Jannsen, & Turner, 2004; Lalumière & Quinsey, 1998; Langevin & Martin, 1975). Although the preceding evidence clearly indicates that copulation is not necessary for sexual conditioning, copulation may be more effective in producing learning than exposure to a potential sexual partner on the other side of a barrier.

2/16/2011 6:52:30 PM

SEXUAL BEHAVIOR AND REPRODUCTION

This question was addressed by Holloway and Domjan (1993b), who compared sexual approach conditioning in two groups of male quail. For one group the 30-sec CS was followed by the release of a female with whom the male could copulate. For a second group, the female was presented for an equal period of time, but behind a wire-mesh screen that allowed the transmission of female visual, olfactory, and auditory cues but limited physical contact. A third group received the CS followed by no US. Acquisition of CS approach behavior is presented in Figure 22.3 for each group of subjects. Clearly the strongest level of responding occurred when the subjects were permitted to copulate with the female. However, signiﬁcant CS approach also developed with exposure to female cues in the absence of copulation. Furthermore, a subsequent experiment conﬁrmed that this CS approach was an associative effect, because it did not develop if the CS was presented unpaired with exposure to a female behind the wire screen. The fact that the visual and other features of a female quail can be reinforcing for males is also evident if the male is permitted to watch a female through a small window, as illustrated in Figure 22.4. Male quail spend 75%–80% of their daylight hours near a window through which they can observe a female, provided that they

previously received opportunities to copulate with a female (Domjan & Hall, 1986). This social proximity behavior persists without decline for at least 2 weeks with continual window exposure in the absence of physical access to a female. It appears that copulation with a female provides sexual reinforcement, which then increases the reinforcing value of visual and other female cues that are encountered when a male sees a female through a window. If those female features are not paired with copulation at least once, the female cues are much less reinforcing and social proximity and looking behavior are not maintained (Domjan, 1998).

Exposure Only CS Only

20 Seconds

Figure 22.4 A male quail looking through a narrow vertical window at a female quail, after visual exposure to the female was paired with copulation.

Conditioned Stimulus Factors in Sexual Conditioning

Copulation 30

511

10

0 0

2

4

6

8

10

Trials

Figure 22.3 Acquisition of sexual conditioned approach behavior to a conditioned stimulus (CS) that was paired with copulation, exposure to a female behind a wire screen, or presented alone.

22-Schachtman-22.indd 511

Early studies of Pavlovian sexual conditioning followed the Pavlovian tradition of using a CS that is convenient and initially “neutral” or unrelated to the US. Domjan et al. (1986) used a light as the CS in their ﬁrst studies with domesticated quail. Hollis et al. (1989) also used a light CS in their study of sexual conditioning in the blue gourami. Arbitrary olfactory cues were used in studies of sexual conditioning with rodents (Graham & Desjardins, 1980; Kippin & Pfaus, 2001; Zamble et al., 1985). These experiments were successful in demonstrating the applicability of Pavlovian procedures to sexual behavior. However, given the arbitrary nature of the conditioned stimuli employed, it is not entirely clear

2/16/2011 6:52:30 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Conditioning of Contextual Cues with Sexual Reinforcement

Sexual behavior in the wild does not occur in unpredictable and unfamiliar places. Rather, sexual behavior most likely occurs in speciﬁc locations or territories that are defended by the male or occupied by the female. Contextual cues present during a sexual encounter may be additionally limited by the time of day, time of the month, or the time of year. For example, sexual behavior may occur only in areas with particular plants and food sources that are characteristic of the breeding season. Thus, contextual cues are one set of stimuli that may serve as CSs in sexual conditioning outside the laboratory. Research has shown that contextual cues are readily associated with sexual reinforcement. In addition, contextual sexual conditioning has two important behavioral consequences: spatial preference and potentiation of responding to female cues. The spatial preference outcome of sexual context conditioning was demonstrated in an experiment by Akins (1998). In this experiment, male quail showed an initial preference for a distinctive context where they had been housed for 2 weeks. They then received brief exposure (5 min) to an alternate nonpreferred distinctive context (CS) followed by copulation with a receptive female quail (US) in that context. One such context conditioning trial was conducted each day. An unpaired group of males received similar stimulus exposures except that they received the US in their colony cage 2 hr before placement into the CS context. Preference tests,

22-Schachtman-22.indd 512

which involved giving free access to the initially preferred context and the CS context, were given after the ﬁrst ﬁve conditioning trials and again after ﬁve more conditioning trials. Figure 22.5 (top panel) shows that male quail shifted their preference to the context that had been paired with the sexual US after ﬁve CS-US pairings and this shift was maintained after ﬁve additional pairings. Quail in the paired group also demonstrated increased locomotor activity during the 5 min prior to the introduction of the female quail, thus demonstrating anticipatory responding to the introduction of the female (Fig. 22.5, bottom panel). Experiments conducted with rodents have demonstrated similar effects of contextual cues in sexual conditioning.

120 % Time in Trained Context

how the sexual learning that was observed in these laboratory experiments would occur in the natural environment of the animals. One would have to argue that some arbitrary stimulus that an animal happened to experience before a sexual encounter would become sexually conditioned to elicit the conditioned response. But, for that scenario to be viable, the arbitrary CS would have to appear with several sexual encounters and not without them. That is, there would have to be a naturally occurring positive contingency between the CS and the US. What cues could have that kind of relationship outside the laboratory?

Group-Paired Group-Unpaired

100 80 60 40 20 0 1

2 Preference Tests

3

200 Line Cross Frequency

512

150

100

50

0 0

1

2

3

4

5

6

7

8

9 10

Conditioning Trials

Figure 22.5 Mean (± SE) percentage of time that

Group Paired and Group Unpaired spent in the trained context across preference tests (top) and their line cross-frequency as a function of conditioning trials (bottom).

2/16/2011 6:52:31 PM

SEXUAL BEHAVIOR AND REPRODUCTION

For example, male rats increase their movements from one level to another in a bilevel chamber in anticipation of the introduction of a sexually receptive female rat (Mendelson & Pfaus, 1989; Pfaus, Mendelson, & Phillips, 1990). Male rats also develop a conditioned place preference for a context that has become associated with copulation with a receptive female (Agmo & Berenfeld, 1990; Hughes, Everitt, & Herbert, 1990; Mehrara & Baum, 1990; Paredes & Alonso, 1997). The importance of contextual cues during a sexual encounter is also evident in studies with females. For example, female rodents show a place preference for a distinct context in which copulation with a male took place (Oldenburger, Everitt, & de Jonge, 1992). Furthermore, female rodents demonstrate a more robust place preference for a context in which they are allowed to pace the rate of copulation by controlling the speed and frequency of male copulatory responses (Jenkins & Becker, 2003; Paredes & Alonzo, 1997; Paredes & Vazquez, 1999). The spatial preference that results from context conditioning should increase preference for these areas during the breeding season. Thus, contextual conditioning can determine the location where sexual activity is likely to take place. Further facilitating that process is the fact that sexually conditioned contextual cues enhance the effectiveness of female cues in generating male approach and copulatory behavior. The ﬁrst evidence of this effect was obtained by Domjan et al. (1989), who tested male quail with

Frequency

Grab

513

a model that had some features of a female quail but not enough to elicit much behavior without learning. The model consisted of a taxidermically prepared head and about 5 cm of neck feathers of a female quail mounted on a vertical dowel in front of a gray foam block that the males could use to make mount and cloacal thrusting responses. The model had so few of the cues of a live female that it did not stimulate copulatory behavior as an unconditioned response. To determine whether context conditioning would enhance the effectiveness of limited female cues, Domjan et al. (1989) permitted one group of male quail to copulate with a female on 15 trials in a distinctive experimental chamber. A second group received an equal number of copulation trials and was equally familiar with the experimental chambers but for them the copulatory experiences occurred in their home cages. All of the subjects were then tested with the female head+neck model in the experimental chamber, and the frequency of copulatory responses directed toward the model was recorded. Note that this was the ﬁrst time any of the subjects encountered the taxidermic head+neck. The results are presented in Figure 22.6. Subjects for whom the experimental chamber was paired with sexual reinforcement made signiﬁcantly more grab, mount, and cloacal contact responses directed at the head+neck model than subjects that received copulatory

Mount

10

10

8

8

6

6

4

4

2

2

0

0 Paired

Cloacal Contact 10 4 2 0

Unpaired

Figure 22.6 Mean (± SE) frequency of grabs, mounts, and cloacal contact responses directed toward a model with female head and neck features, for subjects who had the test context paired or unpaired with sexual reinforcement.

22-Schachtman-22.indd 513

2/16/2011 6:52:32 PM

514

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

experiences in the home cage. These ﬁndings show that context conditioning increased the effectiveness of the limited female cues in eliciting copulatory responses on the part of the male subjects. Similar evidence for the enhancement of copulatory responses by contextual cues has been found in rodents. Sachs and Garinello (1978), for example, found that the penile reﬂex of male rats was enhanced when males were placed in a test cage where they had previously copulated. Domjan et al. (1989) conducted 15 conditioning trials. However, subsequent research has shown that a single context conditioning trial is sufﬁcient to produce enhanced reactivity to female cues in male subjects (Hilliard et al., 1997). This makes context conditioning a powerful mediator of sexual behavior. Males prefer areas where they previously encountered females and are more likely to respond to limited female stimuli in a sexually conditioned context. Outside the laboratory, some of the contextual cues experienced during copulation, such as the male’s territory or seasonal cues, may be familiar to the copulation partners. The extent to which such familiarity produces a latent inhibition effect and attenuates learning by one or both participants has not been examined so far. However, the fact that sexual conditioning of contextual cues is robust enough to occur in one trial suggests that latent inhibition may not have a major impact on this type of learning. Conditioning of Body Adornments

Contextual cues provide one way in which Pavlovian mechanisms could modify sexual behavior in the wild. Another category of stimuli that could be conditioned under natural mating circumstances are unique features of a sexual partner. All copulatory encounters are preceded by telereceptive cues of the sexual partner that are initially encountered at a distance. As the sexual partners come closer together, these telereceptive cues are followed by more proximal visual, olfactory, tactile, and other cues. If a potential sexual partner is identiﬁed by a unique body feature (height, pattern of coloration, or a distinctive posture or call), that feature may become associated with sexual

22-Schachtman-22.indd 514

reinforcement and thereby elicit future sexual responses. The possibility that a unique body feature may become sexually conditioned was ﬁrst examined in male domesticated quail by adding artiﬁcial orange feathers to a female quail (Domjan, O’Vary, & Greene, 1988). On conditioning trials, the adorned female was presented for 30 seconds behind a wire screen as the CS. A normal unadorned female was then introduced to provide the US (copulatory reinforcement). A control group received the CS and US females in an unpaired fashion. Initially, the males did not approach the adorned female, but after four to ﬁve pairings with copulation they came to spend nearly all of the 30-second CS period near her. After the conditioning phase, the wire screen restraining the adorned female was removed to see whether the males would copulate with her. Signiﬁcantly more grab, mount, and cloacal contact responses occurred with the adorned female in the conditioned group than in the unpaired control group. Thus, the CS-US pairings produced not only CS approach responses but also conditioned copulatory responses. In research with laboratory rats, adornment of female rats is typically accomplished by adding a CS odor to the female. In one study, for example, Kippin and colleagues (Kippin, Talianakis, Schattmann, Bartholomew, & Pfaus, 1998) gave male rats access to female rats that were either scented with an almond odor (CS) or not scented. Copulatory preferences were tested subsequently by allowing males to copulate with either a scented or an unscented female. Males that were previously conditioned with scented females displayed a copulatory preference for the scented female, indicating that the unique features of the sexual partner facilitated copulatory behavior through Pavlovian conditioning. Conditioning of Natural Female Features

Experimental studies with body adornments demonstrate that unique features of a sexual partner can become conditioned, but the manipulations in the aforementioned experiments were admittedly exaggerations of the type of body features that might occur in nature. Females quail

2/16/2011 6:52:32 PM

SEXUAL BEHAVIOR AND REPRODUCTION

22-Schachtman-22.indd 515

Head CS

No Head CS

Figure 22.7 Terrycloth objects used as condi-

tioned stimuli in studies of sexual conditioning. The object on the left includes taxidermically prepared features of a female quail, making that a more ecologically relevant conditioned stimulus (CS) object.

a signal for the opportunity to copulate with a female quail. In one experiment, Köksal et al. (1994) tested two groups of male quail. For one group, the female head model was paired with the opportunity to copulate with a live female. For a control group, exposure to the CS and copulation occurred in an unpaired fashion. Every third trial was a nonreinforced test trial with just the CS. Males that received the paired CS-US procedure quickly came to approach and remain near the CS object. Learning was evident after the ﬁrst two conditioning trials and reached asymptote by the fourth trial (see Fig. 22.8).

30

Seconds Near CS

do not sport bright orange feathers in the wild nor do female rodents typically smell of almonds. This raises the question of whether body features that are within the range of normal variation for a species can serve as a basis for Pavlovian sexual conditioning. As we noted earlier, all sexual encounters begin with exposure to cues from the male and female that are encountered at a distance. These cues serve to identify a potential sexual partner. When the male and female are far apart, these cues are likely to be partially occluded by other objects and will be smaller, of lower intensity, and less distinctive than they are when the male and female are close to each other. Perhaps one of the things that brings sexual partners together is having learned to respond to incomplete and low-intensity distal cues of a sexual partner as signals for a more intimate social interaction. The aforementioned scenario is highly likely for Japanese quail. Japanese quail are grounddwelling birds that live in grassy areas in the wild (Schwartz & Schwartz, 1949). When a male ﬁrst encounters a female, she is likely to be mostly hidden by the grass. Only as the two birds come together will they fully see each other. Thus, in nature, intimate social interactions are likely to be preceded by partial cues of the potential sexual partner. These partial cues might come to serve as a signal or CS for the ensuing intimate social interaction. This sequence of events may be modeled in the laboratory by presenting partial female cues as a CS preceding unhindered access to a female quail, which would serve as the US. The ﬁrst studies of sexual conditioning with partial female cues as the CS were performed by Köksal et al. (1994). The left panel of Figure 22.7 shows a drawing of a CS object similar to that used by Köksal et al. (1994). The object was made of terrycloth stuffed with soft polyester ﬁber and had a vertical and a horizontal section to permit the male to grab and mount the object. The vertical section of the CS object included a taxidermically prepared head of a female quail and limited neck feathers. As we noted earlier, such an object lacks sufﬁcient female cues to elicit sexual behavior unconditionally. However, it can rapidly become conditioned if it is used as

515

Paired Unpaired

20

10

0 1

2

T1

3

4

T2

5

6

T3

Figure 22.8 Acquisition of conditioned approach

behavior to a conditioned stimulus (CS) object that includes cues of a female’s head in subjects that had this CS paired or unpaired with sexual reinforcement.

2/16/2011 6:52:33 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Signiﬁcantly less approach behavior was evident in the unpaired control group. The acquisition of the conditioned approach response occurred much more quickly in the experiment by Köksal et al. (1994) than in the earlier study in which a light served as a CS (see Fig. 22.1). Thus, including partial female features in a CS object facilitated the acquisition of sexual approach responding. Other experiments have shown that sexual learning is much more robust in many ways if the CS object includes partial cues of a female quail (see Domjan, Cusato, & Krause, 2004, for a review). In these investigations, the CS object was either the terrycloth model that included a taxidermic female head or a terrycloth model that was the same size and shape that did not include the partial female cues (see Fig. 22.7). Akins (2000) examined the effects of the CS-US interval using independent groups of male quail that received either the head CS object or the no-head CS object paired or unpaired with copulatory opportunity. For one set of groups, the CS-US interval was 1 minute. For another set of groups, the CS-US interval was 20 minutes. (With both intervals, the CS remained visible until the presentation of the US.) The results of the experiment are summarized in Figure 22.9. When the CS object did not include female features, only subjects conditioned with the 1-minute CS-US interval showed acquisition of approach behavior. In contrast, when the CS object included a taxidemic female head, substantial conditioned approach behavior developed with both the 1-minute and the 20-minute CS-US interval, although responding developed more slowly with the 20-minute interval. These results show that including female features in a CS object makes the ensuing learning of the approach response more resistant to increases in the CS-US interval. 1 Other experiments have shown that sexual conditioning with a CS object that includes partial female cues is resistant to the blocking effect (Köksal et al., 1994). That is, if such a CS object is presented at the same time as a second CS that was previously well conditioned, the presence of the previously conditioned cue does not disrupt the conditioning of the head CS. A CS object

22-Schachtman-22.indd 516

100

75 % Time in Zone 0

516

50

25

0 0

5

10

15

Trails

HN-Short Unpaired HN-Long Paired

T-Short Paired T-Short Unpaired T-Long Paired

HN-Long Unpaired

T-Long Unpaired

HN-Short Paired

Figure 22.9 Conditioned approach behavior to a

conditioned stimulus (CS) object that includes cues of a female head and neck (HN) or is made entirely of terrycloth (T), when this CS is paired or unpaired with sexual reinforcement. The CS was presented for either 1 min (Short) or 20 min (Long) during the conditioning trials. The data were obtained during the ﬁrst minute of each trial.

that includes partial female cues is also less likely to undergo extinction when the CS is repeatedly presented without the US (Krause et al., 2003), and such a CS object also supports stronger second-order conditioning (Domjan et al., 2004). Range of Sexually Conditioned Responses

So far we have emphasized approaching the CS as the primary behavioral manifestation of sexual conditioning. Although this is a commonly observed conditioned response, it is not the only type of behavior that develops with sexual conditioning. The nature of the conditioned response depends on the nature of the CS as well as the CS-US interval. CS approach behavior is most likely to develop if the CS is not diffuse so that the subject has a clear location to approach and if the CS-US interval is relatively short (1 minute or less). If a long CS-US interval is used (e.g., 20 minutes), the predominant conditioned response that emerges is an increase in

2/16/2011 6:52:33 PM

SEXUAL BEHAVIOR AND REPRODUCTION

nondirected locomotor behavior (Akins, 2000; Akins et al., 1994). Conditioned approach to a CS with a 20-minute CS-US interval develops as a conditioned response only if the CS includes partial female cues (Akins, 2000). Investigators have also been interested in conditioning components of copulatory behavior. The cloacal gland of adult male quail contains a foamy substance that facilitates sperm transport and fertilization of eggs in the female’s oviduct. Males engage in rhythmic contractions of the cloacal sphincter muscle when they are in the presence of a female. These cloacal sphincter contractions facilitate production of cloacal foam in the visual presence of a female. Holloway, Balthazart, and Cornil (2005) demonstrated that cloacal sphincter contractions can also occur as a Pavlovian conditioned response to a CS that has been paired with visual access to a female. Thus, sexual conditioning serves to elicit cloacal contractions in anticipation of copulation. In a related study, Domjan, Blesbois, and Williams (1998) demonstrated that sexual conditioning also increases the quantity of sperm that is released into the cloaca in anticipation of copulation. As we described earlier in the section on conditioning of body adornments, a sexually conditioned stimulus can also come to elicit components of copulatory behavior (grab, mount, and cloacal contact responses). In those experiments CS body adornments were attached to a female quail. Therefore, the copulation that occurred was with a live female that had been altered by the adornments. The power of Pavlovian conditioning in modifying sexual behavior is more dramatically illustrated by cases in which conditioned copulatory behavior is directed toward an artiﬁcial inanimate object. Conditioned copulation with an artiﬁcial object is of interest because such behavior may provide an animal model of sexual fetishism. For sexual conditioning to generate copulatory responses directed toward the CS, the CS has to have a shape and texture that support grabs, mount, and cloacal thrusts. In the study of body adornments by Domjan et al. (1988), one group of subjects was conditioned with a CS that was a small stuffed toy in the shape of a dog.

22-Schachtman-22.indd 517

517

Conditioning resulted in approach behavior, but this object evidently did not have the required conﬁguration to elicit copulatory responses. Köksal and his associates (Köksal et al., 2004) were more successful in conditioning copulatory responses to a CS using a terrycloth object similar to that shown in the right panel of Figure 22.7. However, even after 30 conditioning trials, the CS elicited grab, mount, and cloacal thrust responses in only about 50% of the subjects (see also Çetinkaya & Domjan, 2006). Interestingly, these subjects showed much more resistance to extinction when the CS was subsequently presented without sexual reinforcement. More consistent conditioned copulatory responses occur if the CS object has partial cues of a female quail, such as a taxidermically prepared head and a bit of neck feathers (see Fig. 22.7, left panel). As we noted earlier, such a CS object does not elicit male sexual behavior unconditionally. However, if the CS is paired with sexual reinforcement, it quickly comes to elicit conditioned approach responses and also elicits grabs, mount, and cloacal contact responses (Cusato & Domjan, 1998). In general, the more of a female’s features that are included in a CS object, the more quickly copulatory responses come to be directed toward the CS. In addition, once the copulatory responses develop, they persist, even if the female features are subsequently gradually covered with terrycloth (Domjan, Huber-McDonald, & Holloway, 1992). These experiments leave no doubt that sexual conditioning can result in copulatory responses being elicited by an inanimate CS object. However, such conditioned behavior critically depends on the speciﬁc conﬁguration of the CS object. Effects of Psychostimulants on Conditioned Sexual Behavior

A fairly recent approach to the study of sexual conditioning has been to investigate the effects of psychostimulants. This approach is of great interest because psychostimulant use has been linked to increased sexual desire (Volkaw et al., 2007) and high-risk sexual behaviors (e.g., Zule, Costenbader, Meyer, & Wechsberg, 2007)

2/16/2011 6:52:33 PM

518

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

in humans. Therefore, studies of the effects of psychostimulants on sexual conditioning may have special clinical relevance. Research in both quail and rodents provides evidence for the modiﬁcation of conditioned sexual behavior by prior psychostimulant exposure. In one experiment, Levens and Akins (2004) utilized the sexual conditioning paradigm to determine the effects of a history of cocaine treatment on subsequent conditioned sexual approach and copulatory responses. Male quail received administration of cocaine (10 mg/kg ip) or saline once a day for 6 days. After a 10-day withdrawal period, conditioning trials were given that consisted of presentation of a CS followed by copulation with a female quail (US). Unpaired control groups (preexposed to either chronic cocaine or saline) received the same treatment except that the US was given 3 hr prior to the CS. Figure 22.10 shows that male quail that received paired CS-US trials showed greater approach to the CS across trials compared to unpaired groups. Of most importance, subjects with a history of chronic cocaine administration showed considerably more CS approach than any of the other groups.

The cocaine paired group also had shorter latencies to copulate with the female partner (see Fig. 22.11), made more cloacal contact responses, and was more efﬁcient at copulating than any of the other groups (see Fig. 22.12). In a related study, Fiorino and Phillips (1999) administered chronic preexposure of amphetamine 3 weeks before sexual conditioning in male rats. During sexual conditioning, the male rats showed increased frequency of level changing behavior in bilevel chambers in anticipation of female presentation and also showed shorter latencies to mount and intromit. These studies demonstrate that a history of psychostimulant administration may enhance sexual conditioning, even after a period of withdrawal. It should be noted that in the study by Levens and Akins, withdrawal was probably complete within an hour after each cocaine administration (though no metabolic data are available in quail). Conducting the sexual conditioning trials after 10 days of withdrawal insured that motor and other physiological effects resulting from acute drug administration would not inﬂuence the results. The 10-day withdrawal period was also employed because the purpose of the study

350

25 20 15 10

250 200 150 100 50

5

0

0 2

3

4

5

6

7

8

9

10

Trial

Figure 22.10 Mean time (s) spent in the condi-

tioned stimulus (CS) zone (±SE) during the 30 s presentation of the CS across sexual conditioning trials.

22-Schachtman-22.indd 518

Paired Cocaine Paired Saline

300

Paired Cocaine Paired Saline Unpaired Cocaine Unpaired Saline

Latency to Copulate (S)

Time in Goal Zone (s)

30

1

2

3

4

5 6 Trial

7

8

9

10

Figure 22.11 Mean latency (s) to copulate (±SE)

during a 5-min test for paired subjects that received a conditioned stimulus (CS) followed by copulatory opportunity across sexual conditioning trials.

2/16/2011 6:52:33 PM

SEXUAL BEHAVIOR AND REPRODUCTION

Copulation Efficiency Ratio

25

20

that predicted copulation with a female. Another group of males were sexually deprived prior to testing. Male quail displayed signiﬁcantly less approach to the CS when sexually satiated than when sexually deprived.

Paired Cocaine Paired Saline Unpaired Cocaine Unpaired Saline

15

10

Conditioned Modiﬁcations of Interactions with a Sexual Partner

5

0 1

2

3 4 Female Trials

5

Figure 22.12 Mean copulatory efﬁciency ratio (±SE) calculated as number of cloacal contact movements divided by the number of grabs plus mounts plus cloacal contact movements. This number is multiplied by 100. Perfect efﬁciency is 33.

was to investigate relatively long-term changes in the brain due to prior cocaine administration rather than acute drug effects on sexual conditioning. In contrast to the literature that demonstrates how psychostimulants enhance sexual motivation and consummation, a recent experiment conducted with Japanese quail indicated that chronic preexposure to methamphetamine impaired sexual motivation as indicated by a slower running time down a runway toward a female quail compared with saline controls (Bolin & Akins, 2009). Similarly, some nondrug manipulations have been shown to reduce sexual motivation. Sexual motivation of male quail conditioned to approach an arbitrary stimulus in a Pavlovian sexual conditioning paradigm was reduced by exposing them to a short photoperiod (Holloway & Domjan, 1993a). Responding was restored when males were returned to a long photoperiod and when exogenous testosterone was administered. In another experiment, Hilliard and Domjan (1995) sexually satiated male quail by allowing them to copulate repeatedly with receptive female quail prior to testing for Pavlovian conditioned approach to a visual CS

22-Schachtman-22.indd 519

519

The behavioral consequence of Pavlovian conditioning is commonly described as the development of a new response to the initially ineffective CS. In fact, the conditioned response is typically deﬁned as a learned response to the CS. In contrast to this view, Domjan (2005) has argued that from a functional perspective the most important thing that is learned in Pavlovian conditioning is not a new response to the CS but a more effective manner of responding to the US. After all, the US is the biologically signiﬁcant event that is a challenge for the organism. How the organism deals with the US is critical to its survival and biological success. Conditioned responses made in the absence of the US are merely false starts that have no functional signiﬁcance. In the sexual behavior system, the US is a sexual partner. If sexual conditioning is of functional signiﬁcance, it should improve how the organism interacts with its sexual partner. Numerous studies have shown that this is indeed the case. In an early study of sexual conditioning (Graham & Desjardins, 1980), male rats presented with a CS that reliably predicted access to a sexually receptive female rat showed increases in serum levels of testosterone and luteinizing hormone after CS presentation. In another experiment, Zamble and colleagues (Zamble et al., 1985) found a decrease in latency of conditioned rats to ejaculate when permitted to copulate with a female rat after exposure to the CS. Subsequent more detailed studies with rodents have provided more evidence for the functional signiﬁcance of sexual conditioning by demonstrating enhanced conditioned partner preferences (e.g., Coria-Avila, Ouimet, Pacheco, Manzo, & Pfaus, 2005; Coria-Avila et al., 2006).

2/16/2011 6:52:34 PM

520

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

An especially dramatic example of conditioned modiﬁcations in how a male interacts with a female was provided by a study with blue gourami ﬁsh (Hollis et al., 1997). After 18 conditioning trials in which the presentation of a light for 10 seconds was followed by visual access to a female, the CS was presented for 10 seconds and the barrier separating the male from the female was removed, allowing the two ﬁsh to interact for the next several days. Presentation of the CS for 10 seconds prior to this extended social interaction had profound effects. In comparison to an unpaired control group, conditioned males showed less aggression toward the female, more nest-building behavior, more clasping behavior, and shorter latencies to spawn. As a result of these enhanced sexual responses, conditioned males also ended up with about 40 times more offspring than males in the control group. It is remarkable that all of these behavioral and physiological effects occurred because of a 10-second sexually conditioned light presented before the extended social interaction. A sexually conditioned stimulus also improves the ways in which male quail interact with a sexually receptive female. As is the case with rats and gouramis, a sexually conditioned stimulus decreases the latency of male quail to initiate copulation with a female (Domjan et al., 1986). Furthermore, this decrease in copulatory latency provides conditioned males with an advantage when two males compete for access to the same female (Gutiérrez & Domjan, 1996). In such sexual competition, the male who receives a Pavlovian CS before the mating opportunity is the one who copulates with the female ﬁrst. Sexual conditioning also increases the efﬁciency of copulatory behavior. In Japanese quail, the copulatory response sequence begins with the male grabbing the back of the female’s neck. The female then squats, allowing the male to mount and make cloacal contact responses. If the female is not sexually receptive, she will attempt to run away or throw the male off when he tries to mount. A persistent male will then reinitiate the grab and mount responses. The efﬁciency of the sexual encounter can be quantiﬁed by assessing how many grab and

22-Schachtman-22.indd 520

mount responses a male bird has to make before he can achieve cloacal contact with the female. Copulatory efﬁciency is related to female genetic characteristics and is positively correlated with female squatting behavior (Domjan, Mahometa, & Mills, 2003). Interestingly, the efﬁciency of copulatory interactions increases as a function of Pavlovian conditioning trials (Mahometa & Domjan, 2005). However, this effect requires that both the male and the female receive the Pavlovian CS that signals the forthcoming mating opportunity. If only the male or only the female receives the signal, copulatory behavior is no more efﬁcient than if the CS is omitted for both subjects (Mahometa & Domjan, 2005). This last ﬁnding is a bit puzzling because Gutiérrez and Domjan (1997) found that sexually conditioned female quail are more apt to squat in the presence of a male than females in an unpaired control group. Evidently, if a male is not also conditioned, he is unable to take full advantage of the increased receptivity of a sexually conditioned female. Sexual Conditioning and Fertilization Success

As described in the preceding section, exposure to a sexually conditioned stimulus changes how a male and female interact. In quail, the latency to copulate decreases and the efﬁciency of copulation increases. In the blue gourami, presentation of a sexually conditioned CS prior to the sexual interaction decreases aggression and the latency to spawn and increases nest-building and clasping behavior. These behavioral changes appear to be improvements in the quality of the sexual interaction. Biologically, the ultimate measure of the quality of a sexual interaction is fertilization success or the number of offspring that are produced. Therefore, one might predict that presentation of a sexually conditioned stimulus will increase the number of offspring that result from the sexual interaction. Hollis et al. (1997) were the ﬁrst to measure the reproductive consequences of sexual Pavlovian conditioning in their study of the blue gourami. They found that the sexual interactions

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

of gourami that received paired presentations of a light and a potential sexual partner resulted in more than 1,000 offspring, whereas the number of offspring produced by the unpaired control group was about 25. Thus, the shorter latencies to spawn, the decreased aggression, and the increased nest-building and clasping behavior that were evident in the Pavlovian conditioned group served to increase the effectiveness of their sexual interactions by the measure that is ultimately the critical outcome sexual activity— namely, reproductive success. In Japanese quail, Domjan et al. (2003) found that shorter latencies to copulate, increased durations of female squatting, and increased copulatory efﬁciency were all positively correlated with the proportion of fertilized eggs the female laid in the next 10 days. Since these behavioral changes can be elicited by a sexually conditioned stimulus, rates of fertilization also should be increased by a sexual CS. That direct link between Pavlovian conditioning and fertilization success was demonstrated by Adkins-Regan and MacKillop (2003), who ﬁrst conditioned male and female subjects by pairing exposure to distinctive contextual cues with a potential sexual partner. The quail were then permitted to copulate with a novel (and experimentally naïve) sexual partner either in the sexually conditioned context or an equally familiar control context. The eggs subsequently laid by the females were assessed for successful fertilization. Exposure to a sexually conditioned context increased the rate of fertilization whether the contextual cues had been conditioned for the male or the female subjects. Increased fertilization rates were also found by Mahometa and Domjan (2005) in a study that employed a light rather than contextual cues as the sexually conditioned CS. However, Mahometa and Domjan (2005) found increased fertilization rates only if both the male and the female received the Pavlovian CS prior to the test copulation. Presenting the conditioned light CS to just the male or just the female did not have a signiﬁcant effect. The requirement of signaling both the male and the female in the study by Mahometa and Domjan (2005) may have been related to the fact that increased copulatory efﬁciency occurred

22-Schachtman-22.indd 521

521

only when the Pavlovian signal was presented to both of the participants. Pavlovian conditioning can also increase fertilization success in sperm competition. Sperm competition occurs when two males inseminate the same female. Various mechanisms have evolved (e.g., dislodging a sperm plug) that give one of the males the upper hand in fathering the offspring under these conditions. Pavlovian conditioning is one of those mechanisms. This was demonstrated by Matthews, Domjan, Ramsey, and Crews (2007), who used genetic ﬁngerprinting to determine paternity in Japanese quail after females copulated with two males in succession. During the initial conditioning phase of the study by Matthews et al. (2007), both male and female quail received exposures to two distinctively different experimental chambers. For female subjects, exposure to both contexts was paired with copulation. In contrast, for the males one context was paired with sexual reinforcement and the other was unpaired or not reinforced. On the critical sperm competition day, each female copulated with two males in succession, one in each of the experimental contexts. One of the males was in its paired context and the other male was in its unpaired context. Thus, the test copulation was signaled for one of the males but not the other. Order of exposure to the signaled and unsignaled male was counterbalanced across females. Eggs were collected from each female for 10 days starting 2 days after the sperm competition copulations. The eggs were incubated for 5 days and subjected to genetic analysis to determine paternity. Of the 78 eggs laid by the females, 39 were fertilized. Genetic analysis indicated that among the eggs that were fertilized, 28 (72%) had been fertilized by the signaled male and 11 (28%) were fertilized by the unsignaled male. Whether the female copulated ﬁrst with the signaled male or the unsignaled male did not matter. These results show that sexual conditioning provides a signiﬁcant paternity advantage in a situation where a female copulates with more than one male and sperm competition ensues. In a subsequent previously unpublished study, Matthews and Domjan examined the role of Pavlovian conditioning in a sperm allocation

2/16/2011 6:52:34 PM

522

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

situation. Here each male was allowed to copulate with two females in succession, and the question was how many of the eggs produced by each female turned out to be fertilized. The two copulations were separated by 15 minutes. In a control group, both of the females were presented to the male subjects in the absence of a sexually conditioned CS. Under these conditions, 24% of the eggs produced by the ﬁrst female were fertilized but only 6% of the eggs laid by the second female were fertilized (see Table 22.1). This reﬂects a common sperm depletion effect. Having copulated with the ﬁrst female, the males had less sperm available to inseminate the second female. What if access to the second female were signaled by a Pavlovian CS? This was evaluated in another group of male quail. Signaling the second female did not change the proportion of fertilized eggs that were laid by the ﬁrst female with whom the male copulated (23% of the eggs in this group were fertilized as compared to 24% in the control group). However, signaling the second female signiﬁcantly increased the proportion of fertilized eggs that were laid by the second female (27% in this group were fertilized as contrasted to 6% in the control group; see Table 22.1). Matthews and Domjan also examined the effects of signaling the ﬁrst female but not the second. Under those conditions, 40% of the eggs laid by the ﬁrst female were fertilized, an increase of 16% from the control group. As expected, having fertilized so many of the eggs of the ﬁrst female, the males managed to fertilize only 5% of the eggs of the second female. These experiments show that Pavlovian conditioning also inﬂuences sperm allocation, with a greater

effect on fertilization rates of the second female the male encounters. The aforementioned studies are important because they demonstrate the adaptive signiﬁcance of sexual conditioning in particular and of Pavlovian mechanisms in general. The modiﬁcations of behavior and physiology that result from Pavlovian signaling facilitate reproduction in disparate species (blue gourami and quail) and in a variety of situations that include not only one-to-one male/female interactions but also sperm competition and sperm allocation paradigms. The mechanisms of these effects remain to be worked out. In Japanese quail, increased fertility related to the sexual conditioning of the male participants is probably mediated by conditioned cloacal sphincter contractions (Holloway et al., 2005) that help produce cloacal foam which facilitates sperm transport. An additional mechanism is the increased sperm release that occurs in male quail exposed to a sexually conditioned stimulus (Domjan et al., 1998). How sexual conditioning of females facilitates fertilization is a bit less clear, but it might be related to the fact that conditioned females spend more time squatting in the presence of a male (Gutiérrez and Domjan, 1997) and this increases the copulatory efﬁciency of the male (Domjan et al., 2003).

LESSONS FROM SEXUAL CONDITIONING FOR THE STUDY OF LEARNING Ideally a project of application is not just a oneway street, exporting knowledge from one area to another but an enterprise that brings new

Table 22.1 Pavlovian Signaling When One Male Copulates with Two Females in Succession Percent Fertilized Eggs Female Signaled

1 Female

2 nd Female

Neither

24%

6%

1 st Female

40%

5%

23%

27%

nd

2 Female

22-Schachtman-22.indd 522

st

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

insights and perspectives to the source of the application. We next turn to considering the implications of research on sexual conditioning for the study of basic learning processes. Attending to Various Manifestations of Conditioned Behavior

In embarking on an application of Pavlovian conditioning, there are numerous decisions to make. One that has stood out in our experience is deciding what aspect of behavior to measure to obtain evidence of learning. As we described, sexual conditioning can result in changes in a variety of different aspects of appetitive and consummatory sexual behavior and can also alter how sexual partners interact in what is perhaps best described as modiﬁcations of unconditioned or instinctive behavior. Success in studies of sexual conditioning would have been severely limited had investigators adopted rigid preconceptions of what a sexually conditioned response should be. However, adopting a broader conception of conditioned behavior yielded beneﬁts beyond providing more sensitive behavioral indices of learning. By measuring a broader range of behaviors, the experiments revealed learning processes that otherwise would not have come to light. Here we describe two of those effects, one involving studies of the CS-US interval and the other involving studies of the C/T ratio in conditioning. Effects of the CS-US Interval

One of the ﬁrst unexpected ﬁndings that emerged from using multiple response measures occurred during studies of the CS-US interval in sexual conditioning. Akins et al. (1994) conditioned independent groups of male quail with CS-US intervals of 0.5, 2.5, 5, 10, 15, and 20 minutes using a gray foam block as the CS and copulation with a female as the US. As expected, less conditioned approach behavior developed in groups that were conditioned with longer CS-US intervals. In fact, no conditioned approach occurred with the 20-minute CS-US interval, suggesting that these subjects did not learn. However, the investigators noticed that quail conditioned with the longer CS-US intervals

22-Schachtman-22.indd 523

523

seemed “restless” and paced from one side of the experimental chamber to the other. Subsequent follow-up experiments indicated that this increased locomotor behavior was a result of the sexual conditioning procedure, since unpaired control groups did not show the effect (Akins et al., 1994, Experiment 2; Akins, 2000). The ﬁndings mentioned earlier clearly indicate that the decreases in conditioned behavior that are commonly observed with longer CS-US intervals should not be automatically interpreted as reﬂecting decrements in learning. Rather, longer CS-US intervals may support learning that is manifest in different behavioral changes. This interpretation is squarely in line with behavior systems theory, which assumes that learning occurs within the context of the behavior system that is activated by a conditioning procedure (Timberlake, 2001). Components of a behavior system are organized in terms of their temporal and spatial proximity to the primary reinforcer. In the feeding system, for example, general search behaviors predominate when the subject is hungry but food has not yet been found and is unavailable. Focal search responses predominate when a potential food source has been located or food is about to become available. Finally, consummatory responses occur when the food is actually encountered. Behavior systems theory assumes that the CS-US interval determines which behavioral component comes to be elicited by the CS. Longer CS-US intervals come to elicit general search responses and short CS-US intervals elicit focal search or consummatory responses (Silva & Timberlake, 1997, 1998). The pacing behavior that developed with a 20-min CS-US interval in the experiments by Akins et al. (1994) and Akins (2000) was probably reﬂective of “general search” for a sexual partner, whereas the CS approach behavior that is evident with shorter CS-US intervals is reﬂective of “focal search” behavior. Effects of the C/T Ratio on Conditioning

The aforementioned studies illustrate the importance of the CS-US interval in learning. However, some investigators have argued that the critical factor for learning is not the duration of the CS

2/16/2011 6:52:34 PM

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

or the CS-US interval but the ratio between the CS duration or trial time (T) and how long the subject spends in the experimental context (C) overall (e.g., Gallistel & Gibbon, 2000). According to this idea, learning will be evident only if the duration of the conditioning trial (T) is substantially shorter than the time that subjects spend in the experimental context (C). This relation can be quantiﬁed by dividing time spent in the experimental context (C) by the duration of the conditioning trial (T) to form the C/T ratio. Learning is predicted to occur if the C/T ratio exceeds a critical or threshold value. Burns and Domjan (2001) examined the effects of the C/T ratio in studies of sexual conditioning using a “long box” in which the CS (a wood block) was presented 112 cm from the door that provided access to a female quail. Male quail received one conditioning trial per day for 15 days and were returned to their home cages after each trial. The CS was always presented for 30 seconds, immediately followed by access to the female. To vary the C/T ratio, independent groups of male quail were permitted to remain in the experimental chamber for different lengths of time before the CS was presented. These manipulations created C/T ratios of 1.5, 4.5, 45, and 180. Based on previous theories such as scalar expectancy theory (Gibbon & Balsam, 1981) and rate estimation theory (Gallistel & Gibbon, 2000), learning was expected when the C/T ratio was 45 or 180 but not when it was 1.5 or 4.5. To measure learning, Burns and Domjan (2001) recorded not only approach to the CS area (sign tracking) but also approach to the door from which the female was to be released (goal tracking) on the opposite side of the experimental chamber. The results of these two response measures are summarized in Figure 22.13. Data on sign tracking were as predicted by previous claims that longer C/T ratios are necessary for learning. Sign tracking did not occur with a C/T ratio of 1.5 but was prominent at C/T ratios of 45 and 180. In contrast, the opposite results were obtained when goal tracking served as the response measure. Goal tracking only occurred at the low C/T ratios of 1.5 and 4.5. Additional evidence indicated that the goal tracking that occurred with the low C/T ratios was an associate

22-Schachtman-22.indd 524

60 Sign Tracking Goal Tracking

50 Percentage of Time

524

40 30 20 10 0 1.5

4.5

45

180

C/T Ratio

Figure 22.13 Sign tracking and goal tracking as a function of the C/T ratio.

effect but reﬂected context conditioning rather than conditioning of the target CS. (For related evidence and discussion, see Domjan, 2003.) These ﬁndings provide additional evidence that multiple responses have to be measured in learning experiments to fully appreciate what is learned as a result of a particular training procedure. Conditioning Effects and the Nature of the Conditioned Stimulus

We previously discussed the major impact that the nature of the CS has on the sexual conditioned response. Learning investigators have frequently taken advantage of the fact that different conditioned stimuli generate different conditioned responses. For example, a light conditioned with food comes to elicit rearing behavior in rats, whereas a tone comes to elicit a head-jerk response (Holland, 1977). Using these contrasting conditioned responses, one can determine whether a tone or a light is the controlling stimulus when the two are presented in the same situation. The CS effects that have been discovered in sexual conditioning go beyond providing such methodological advantages. As we noted, use of a CS that includes partial cues of a female not only leads to more rapid learning and a wider range of conditioned responses but also leads to novel learning effects, namely resistance to blocking, resistance to increases in the CS-US interval, and resistance

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

to extinction. These ﬁndings are reminiscent of examples of adaptive specializations in learning that were prominently discussed some years ago (e.g., Rozin & Schull, 1988). Studies of sexual conditioning bring a new perspective to those theoretical issues. Earlier “adaptive specializations” in learning were discovered by chance and were identiﬁed primarily by their inconsistency with prevailing views of learning at the time (Domjan & Galef, 1983). The present ﬁndings suggest that natural precursors of a sexual interaction have a privileged status in becoming associated with sexual reinforcement. This suggests a method of discovery of learning specializations. Namely, facilitated learning is predicted to occur whenever the CS used is a precursor of the US in the natural habitat of the organism. This prediction is based on the evidence we reviewed as well as on the basis of the assumption that evolution has shaped how animals learn about cues in their natural habitat rather than how they learn about “arbitrary” cues. Thus, one would expect more robust learning about natural CSs than arbitrary cues. In the present experiments, the primary natural precursor of a copulatory interaction was provided by including partial cues of a female quail in the CS. Including such female cues did not turn the CS into an unconditioned stimulus but allowed the CS to become associated with sexual reinforcement more readily. This suggests that a continuum of sexual conditioning effects may be obtained by varying the extent to which the CS includes components of the US. Increasing those components should facilitate learning and decreasing them should make the CS more like an arbitrary cue. This continuum is reminiscent of the concept of evolutionary preparedness for learning originally proposed by Seligman (1970). However, the current conception lacks the circularity of the original formulation. (For a more detailed discussion of these issues, see Domjan, 2008.)

IMPLICATIONS FOR THE STUDY OF SEXUAL CONDITIONING IN PEOPLE The insights into sexual conditioning that have been provided by research with quail and other non-human species have important implications

22-Schachtman-22.indd 525

525

for the study of sexual learning in Homo sapiens. In contrast to the numerous investigations of sexual conditioning in non-human animals, empirical evidence of sexual conditioning in humans is scant (see Akins, 2004, for a review). In addition, studies in which sexual conditioning may be evident sometimes lack appropriate control groups, thereby making the results difﬁcult to interpret. Hoffmann and colleagues (Hoffmann et al., 2004; see Chapter 23, this volume for further discussion) demonstrated increased genital arousal in men and women to a sexually relevant picture that was presented subliminally and paired with an erotic ﬁlm. Both and colleagues (Both et al., 2008) found similar results in women when they paired an erotic picture (presented subliminally) with genital vibrotactile stimulation. Thus, there appears to be increasing evidence for sexual conditioning in humans. However, one might argue that sexual conditioning studies with humans could beneﬁt from greater efforts to take advantage of factors that have been found to facilitate sexual conditioning in animal experiments. The animal research suggests that human studies of sexual conditioning should not focus on a single response measure, such as penile tumescence or vaginal lubrication, to detect conditioned responses to a CS. While a few human studies have observed sexual conditioning using measures other than genital responses (Both et al., 2008; Letourneau & Donohue, 1997; see Chapter 23, this volume), the majority of human studies suggest that genital responses may not show strong conditioning effects. Genital responses are close to consummatory responses at the end of the sexual behavior sequence. Research with non-human species has shown that appetitive components of the sexual behavior sequence (e.g., CS approach or increased locomotion in a bilevel chamber) are most easily conditioned. Research with quail suggests that conditioning of genital components of sexual behavior would require special conditioned stimuli, namely CSs that are part of the causal sequence of events that lead to copulation outside the laboratory. Furthermore, conditioning may be more successful if the CSs included some of stimulus elements of a sexual unconditioned stimulus. Indeed, successful

2/16/2011 6:52:34 PM

526

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

demonstrations of sexual conditioning in humans have utilized erotic pictures and arousing videotapes of heterosexual sexual interactions as the CS (Both et al., 2008; Kantorowitz, 1978; Lalumière & Quinsey, 1998). In contrast to these recommendations, most studies of human sexual conditioning have typically employed entirely arbitrary CSs (e.g., pictures of squares and triangles) that have no inherent relation to normal sexual activity. The animal research also indicates that copulation is much more effective as a US than visual exposure to a potential sexual partner. Yet visual exposure to sexual pictures or ﬁlm clips is typically employed in human research (e.g., Hoffmann et al., 2004; Langevin & Martin, 1975; Rachman, 1966) with the exception of a few (e.g., Kantorowitz, 1978; see also Chapter 23, this volume). Furthermore, the individual pictures cannot be characterized as potential sexual partners because no actual sexual activity takes place in the human studies. Another major issue is the duration of the intertrial interval. Pavlovian conditioning in conventional laboratory preparations (e.g., eyeblink conditioning or appetitive conditioning with food) involves multiple conditioning trials in each session, with the trials spaced no more than a minute or two apart. The use of such massed trials has been incorporated in human studies of sexual conditioning (e.g., Letourneau & O’Donohue, 1997; Plaud & Martini, 1999). This contrasts with studies of sexual conditioning in non-human subjects, which typically provide no more than one conditioning trial per day. The rationale for long intertrial intervals is that sexual behavior has a substantial refractory period and is therefore only reinforcing if a long intertrial interval is used. Success in human research on sexual learning is likely to involve the same fundamental considerations that have facilitated the study of such learning in non-human species. In particular, investigators have to be cognizant of the behavioral and temporal organization of human sexual behavior under natural conditions and then take this information into account in

22-Schachtman-22.indd 526

making decisions about the temporal organization of the conditioning procedure, the CS and US that are used, and the responses that are expected to change as a result of the conditioning procedure.

SUMMARY AND CONCLUSION Research on sexual conditioning has clearly shown that the sexual behavior system is readily susceptible to learning effects. Given the range of learning effects that have been documented, it is safe to say that there is probably no conditioning effect that occurs in conventional learning preparations that cannot be also found in the sexual behavior system. Thus, the research has amply demonstrated that the generality of learning extends to sexual behavior. Research on sexual conditioning has also provided new insights into the nature of sexual behavior. It has told us that the stimulus control of various components of sexual behavior is not limited to stimuli that elicit the behavior unconditionally. Rather, the stimulus control of various components of sexual behavior can be extended to a wide range of new stimuli that include visual and olfactory cues, as well as various three-dimensional objects. There appear to be few limitations on the types of stimuli that can come to control appetitive components of sexual behavior (e.g., approach responses). However, consummatory sexual responses (grabs, mount, and cloacal contact responses in quail) are most easily conditioned if the CS object includes at least limited species typical features or stimulus elements that are part of the courtship → copulation sequence in the natural environment of the species. Studies have also demonstrated that conditioning can modify not only behavioral aspects of sex but also the preparation of cloacal foam, the quantity of sperm that are released, and the number and paternity of the offspring that are produced. Thus, the impact of Pavlovian learning on sexual behavior is broad and of substantial evolutionary signiﬁcance. Studies of sexual conditioning have also provided important lessons about how to investigate learning and how learning operates in

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

situations that more closely resemble the natural environment than common laboratory situations. In particular, studies of sexual conditioning illustrate the importance of taking multiple measures of behavior in a learning experiment rather than focusing on the detection and measurement of a single index of conditioning. They also illustrate that rather than just examining conditioned responses elicited by the CS, it is important to consider how Pavlovian conditioning may modify the organism’s interactions with the unconditioned stimulus. These learned modiﬁcations on the unconditioned response may be of greater adaptive signiﬁcance than the development of a new response to the CS. Finally, studies of sexual conditioning have demonstrated that the nature of the CS contributes to much more than just the topography of the conditioned response. The nature of the CS may also modify the learning effects that are observed, especially if the CS is part of the natural causal chain that leads to the US in the wild and includes species typical features. Studies of sexual conditioning with human participants has been far more limited than with laboratory animals. Future studies with human participants may beneﬁt from greater attention to the factors that have been identiﬁed as important in the animal experiments. These factors include the use of more naturalistic conditioned and unconditioned stimuli, the measurement of various components of appetitive sexual behavior rather than just focusing on genital responses, and the use of more widely spaced conditioning trials. Overall, the application of conditioning to sexual behavior has provided a wealth of information relevant to learning theory, biological ﬁtness, and reproductive behavior. More recently, neurochemical and neurobiological aspects of sexual conditioning have been also explored (e.g., Coria-Avila & Pfaus, 2007; CoriaAvila et al., 2008; Taziaux, Kahn, Moore, Balthazart, & Holloway, 2008). The application of learning to sexual behavior also continues to be used to investigate the causal relationship between sexual arousal/motivation and other clinically related behaviors such as substance abuse (e.g., Troisi & Akins, 2004).

22-Schachtman-22.indd 527

527

NOTE 1. The CS was present during the entire duration of the CS-US interval in this experiment so that all paired subjects would receive a delayed conditioning procedure. If the CS duration had been kept constant at 1 minute, subjects conditioned with a 20-minute CS-US interval would have experienced a trace conditioning procedure with a long trace interval. For a study of trace conditioning in the sexual behavior system, see Akins and Domjan (1996).

REFERENCES Adkins-Regan, E., & MacKillop, E. A. (2003). Japanese quail (Coturnix japonica) inseminations are more likely to fertilize eggs in a context predicting mating opportunities. Proceedings of the Royal Society of London, Series B, 270, 1685–1689. Agmo, A., & Berenfeld, R. (1990). Reinforcing properties of ejaculation in the male rat: The role of opioids and dopamine. Behavioral Neuroscience, 107, 177–182. Akins, C. K. (1998). Context excitation and modulation of conditioned sexual behavior. Animal Learning and Behavior, 26, 416–426. Akins, C. K. (2000). Effects of species-speciﬁc cues and the CS-US interval on the topography of the sexually conditioned response. Learning and Motivation, 31, 211–235. Akins, C. K. (2004). The role of Pavlovian conditioning in sexual behavior: A comparative analysis of human and non-human animals. International Journal of Comparative Psychology, 17, 241–262. Akins, C. K., & Domjan, M. (1996). The topography of sexually conditioned behaviour: Effects of a trace interval. Quarterly Journal of Experimental Psychology, 49B, 346–356. Akins, C. K., Domjan, M., & Gutiérrez, G. (1994). Topography of sexually conditioned behavior in male Japanese quail (Coturnix japonica) depends on the CS-US interval. Journal of Experimental Psychology: Animal Behavior Processes, 20, 199–209. Ball, G. F., & Balthazart, J. (2004). Hormonal regulation of brain circuits mediating male sexual behavior in birds. Physiology and Behavior, 83(2), 329–346. Boakes, R. A. (1977). Performance on learning to associate a stimulus with positive reinforcement.

2/16/2011 6:52:34 PM

528

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

In H. Davis & H. M. B. Hurwitz (Eds.), OperantPavlovian interactions (pp. 67–101). Hillsdale, NJ: Erlbaum. Bolin, B. L., & Akins, C. K. (2009). Methamphetamine impairs sexual motivation but not sexual performance in male Japanese quail. Experimental and Clinical Psychopharmacology, 17, 10–20. Both, S., Spiering, M., Laan, E., Belcome, S., van den Heuvel, B., & Everaerd, W. (2008). Unconscious classical conditioning of sexual arousal: Evidence for the conditioning of female genital arousal to subliminally presented sexual stimuli. Journal of Sexual Mediciine, 5, 100–109. Burns, M., & Domjan, M. (1996). Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica). Journal of Experimental Psychology: Animal Behavior Processes, 22, 297–306. Burns, M., & Domjan, M. (2000). Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters. Animal Learning and Behavior, 28, 109–119. Burns, M., & Domjan, M. (2001). Topography of spatially directed conditioned responding: Effects of context and trial duration. Journal of Experimental Psychology: Animal Behavior Processes, 27, 269–278. Çetinkaya, H., & Domjan, M. (2006). Sexual fetishism in a quail (Coturnix japonica) model system: Test of reproductive success. Journal of Comparative Psychology, 120, 427–432. Coria-Avila, G. A., Gavrila, A. M., Boulard, B., Charron, N., Stanley, G, & Pfaus, J. G. (2008). Neurochemical basis of conditioned partner preference in the female rat: II. Disruption by ﬂupenthixol. Behavioral Neuroscience, 122(2), 396–406. Coria-Avila, G. A., Jones, S. L., Solomon, C. E., Gavrila, A. M., Jordan, G. J., & Pfaus, J. G. (2006). Conditioned partner preference in female rats for strain of male. Physiology and Behavior, 88, 529–537. Coria-Avila, G. A., Ouimet, A. J., Pacheco, P., Manzo, J., & Pfaus, J. G. (2005). Olfactory conditioned partner preference in the female rat. Behavioral Neuroscience, 119(3), 716–725. Coria-Avila, G. A., & Pfaus, J. G. (2007). Neuronal activation by stimuli that predict sexual reward in female rats. Neuroscience, 148(3), 623–632. Crawford, L. L., & Domjan, M. (1993). Sexual approach conditioning: Omission contingency tests. Animal Learning and Behavior, 21, 42–50.

22-Schachtman-22.indd 528

Crawford, L. L., & Domjan, M. (1995). Secondorder sexual conditioning in male Japanese quail (Coturnix japonica). Animal Learning and Behavior, 23, 327–334. Crawford, L. L., & Domjan, M. (1996). Conditioned inhibition of social approach in male Japanese quail (Coturnix japonica) using visual exposure to a female. Behavioural Processes, 36, 163–169. Cusato, B., & Domjan, M. (1998). Special efﬁcacy of sexual conditioned stimuli that include species typical cues: Tests with a CS preexposure design. Learning and Motivation, 29, 152–167. Domjan, M. (1998). Going wild in the laboratory: Learning about species typical cues. In D. L. Medin (Ed.), The psychology of learning and motivation (pp. 155–186). San Diego, CA: Academic Press. Domjan, M. (2003). Stepping outside the box in considering the C/T ratio. Behavioural Processes, 62, 103–114. Domjan, M. (2005). Pavlovian conditioning: A functional perspective. Annual Review of Psychology, 56, 179–206. Domjan, M. (2008). Adaptive specializations and generality of the laws of classical and instrumental conditioning. In J. Byrne (Series Ed.) & R. Menzel (Vol. Ed.), Learning and memory: A comprehensive reference: Vol. 1. Learning and behavior theory (pp. 327–340) Oxford, England: Elsevier. Domjan, M., Akins, C., & Vandergriff, D. H. (1992). Increased responding to female stimuli as a result of sexual experience: Tests of mechanisms of learning. Quarterly Journal of Experimental Psychology, 45B, 139–157. Domjan, M., Blesbois, E., & Williams, J. (1998). The adaptive signiﬁcance of sexual conditioning: Pavlovian control of sperm release. Psychological Science, 9, 411–415. Domjan, M., Cusato, B., & Krause, M. (2004). Learning with arbitrary versus ecological conditioned stimuli: Evidence from sexual conditioning. Psychonomic Bulletin and Review, 11, 232–246. Domjan, M., & Galef, B. G., Jr. (1983). Biological constraints on instrumental and classical conditioning: Retrospect and prospect. Animal Learning and Behavior, 11, 151–161. Domjan, M., Greene, P., & North, N. C. (1989). Contextual conditioning and the control of copulatory behavior by species-speciﬁc sign stimuli in male Japanese quail. Journal of Experimental Psychology: Animal Behavior Processes, 15, 147–153.

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

Domjan, M., & Hall, S. (1986). Determinants of social proximity in Japanese quail (Coturnix coturnix japonica): Male behavior. Journal of Comparative Psychology, 100, 59–67. Domjan, M., Huber-McDonald, M., & Holloway, K. S. (1992). Conditioning copulatory behavior to an artiﬁcial object: Efﬁcacy of stimulus fading. Animal Learning and Behavior, 20, 350–362. Domjan, M., Lyons, R., North, N. C., & Bruell, J. (1986). Sexual Pavlovian conditioned approach behavior in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 413–421. Domjan, M., Mahometa, M., & Mills, A. D. (2003). Relative contributions of the male and female to sexual behavior and reproductive success in the Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 117, 391–399. Domjan, M., O’Vary, D., & Greene, P. (1988). Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. Journal of the Experimental Analysis of Behavior, 50, 505–519. Farris, H. E. (1967). Classical conditioning of courting behavior in the Japanese quail, Coturnix coturnix japonica. Journal of the Experimental Analysis of Behavior, 10, 213–217. Fiorino, D. F., & Phillips, A. G. (1999). Faciliation of sexual behavior in male rats following d-amphetamine-induced behavioral sensitization. Psychopharmacology (Berlin), 142(2), 200–208. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289–344. Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York, NY: Academic Press. Graham, J. M., & Desjardins, C. (1980). Pavlovian conditioning: Induction of luteinizing hormone and testosterone secretion in anticipation of sexual activity. Science, 210, 1039–1041. Gutiérrez, G., & Domjan, M. (1996). Learning and male-male sexual competition in Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 110, 170–175. Gutiérrez, G., & Domjan, M. (1997). Differences in the sexual conditioned behavior of male and female Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 110, 135–142.

22-Schachtman-22.indd 529

529

Hearst, E., & Jenkins, H. M. (1974). Sign tracking: The stimulus-reinforcer relation and directed action. Austin, TX: Psychonomic Society. Hilliard, S., & Domjan, M. (1995). Effects of sexual conditioning of devaluing the US through satiation. Quarterly Journal of Experimental Psychology, 48B, 84–92. Hilliard, S., Nguyen, M., & Domjan, M. (1997). One-trial appetitive conditioning in the sexual behavior system. Psychonomic Bulletin and Review, 4, 237–241. Hoffmann, H., Janssen, E., & Turner, S. L. (2004). Classical conditioning in the effects of sexual arousal of women and men: Effects of varying awareness and biological relevance of the conditioned stimulus. Archives of Sexual Behavior, 33(1), 43–53. Holland, P. C. (1977). Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. Journal of Experimental Psychology: Animal Behavior Processes, 3, 77–104. Hollis, K. L., Cadieux, E. L., & Colbert, M. M. (1989). The biological function of Pavlovian conditioning: A mechanism for mating success in the blue gourami (Trichogaster trichopterus). Journal of Comparative Psychology, 103, 115–121. Hollis, K. L., Pharr, V. L., Dumas, M. J., Britton, G. B., & Field, J. (1997). Pavlovian conditioning provides paternity advantage for territorial male blue gouramis. (Trichogaster trichopterus). Journal of Comparative Psychology, 111, 219–225. Holloway, K. S., Balthazart, J., & Cornil, C. A. (2005). Androgen mediation of conditioned rhythmic cloacal sphincter movements in Japanese Quail (Coturnix japonica). Journal of Comparative Psychology, 119(1), 49–57. Holloway, K. S., & Domjan, M. (1993a). Sexual approach conditioning: Tests of unconditioned stimulus devaluation using hormone manipulations. Journal of Experimental Psychology: Animal Behavior Processes, 19, 47–55. Holloway, K. S., & Domjan, M. (1993b). Sexual approach conditioning: Unconditioned stimulus factors. Journal of Experimental Psychology: Animal Behavior Processes, 19, 38–46. Hughes, A. M., Everitt, B. J., & Herbert, J. H. (1990). Comparative effects of preoptic area infusions of opioid peptides, lesions, and castration on sexual behavior in male rats: Studies of instrumental behavior, conditioned place preference, and partner preference. Psychopharmacology, 102, 243–256.

2/16/2011 6:52:34 PM

530

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Jenkins, W. J., & Becker, J. B. (2003). Female rats develop conditioned place preference for sex at their preferred interval. Hormones and Behavior, 43, 503–507. Kantorowitz, D. A. (1978). Personality and conditioning of tumescence and detumescence. Behaviour Research and Therapy, 16, 117–123. Kippin, T. E., Talianakis, S., Schattmann, L., Bartholomew, S., & Pfaus, J. G. (1998). Olfactory conditioning of sexual behavior in the male rat (Rattus norvegicus). Journal of Comparative Psychology, 112, 389–399. Kippin, T. E., & Pfaus, J. G. (2001). Nature of the conditioned response mediating olfactory conditioned ejaculatory preference in the male rat. Behavioral Brain Research, 122, 11–24. Köksal, F., & Domjan, M. (1998). Observational conditioning of sexual behavior in the domesticated quail. Animal Learning and Behavior, 26, 427–432. Köksal, F., Domjan, M., Kurt, A., Sertel, Ö., Sabiha, Ö., Bowers, R., & Kumru, G. (2004). An animal model of fetishism. Behavioural Research and Therapy, 42, 1421–1434. Köksal, F., Domjan, M., & Weisman, G. (1994). Blocking of the sexual conditioning of differentially effective conditioned stimulus objects. Animal Learning and Behavior, 22, 103–111. Krause, M. A., Cusato, B., & Domjan, M. (2003). Extinction of conditioned sexual responses in male Japanese quail (Coturnix japonica): Role of species typical cues. Journal of Comparative Psychology, 117, 76–86. Lalumière, M. L., & Quinsey, V. L. (1998). Pavlovian conditioning of sexual interests in human males. Archives of Sexual Behavior, 27, 241–252. Langevin, R., & Martin, M. (1975). Can erotic response be classically conditioned? Behavior Therapy, 6, 350–355. Letourneau, E. J., & O’Donohue, W. (1997). Pavlovian conditioning of female sexual arousal. Archives of Sexual Behavior, 26, 63–78. Levens, N., & Akins, C. K. (2004). Chronic cocaine pretreatment facilitates Pavlovian sexual conditioning in male Japanese quail. Pharmacology, Biochemistry, and Behavior, 79(3), 451–457. Mehrara, B. J., & Baum, M. J. (1990). Naloxone disrupts the expression but not the acquisition by male rats of a conditioned place preference response for an oestrous female. Psychopharmacology, 101, 118–125. Mahometa, M. J., & Domjan, M. (2005). Classical conditioning increases reproductive success in

22-Schachtman-22.indd 530

Japanese quail, Coturnix japonica. Animal Behaviour, 69, 983–989. Matthews, R. N., Domjan, M., Ramsey, M., & Crews, D. (2007). Learning effects on sperm competition and reproductive ﬁtness. Psychological Science, 18, 758–762. McCarthy, M. M., & Bell, G. F. (2008). The neuroendocrine control of sex-speciﬁc behavior in vertebrates: Lessons from mammals and birds. Current Topics in Developmental Biology, 83, 213–248. Mendelson, S. D., & Pfaus, J. G. (1989). Level searching: A new assay of sexual motivation in the male rat. Physiology and Behavior, 45, 337–341. Nelson, J. B. (2009). Contextual control of ﬁrstand second-learned excitation and inhibition in equally ambiguous stimuli. Learning and Behavior, 37, 95–106. Oldenburger, W. P., Everitt, B. J., & de Jonge, F. H. (1992). Conditioned place preference produced by sexual interaction in female rats. Hormones and Behavior, 26, 214–228. Paredes, R. G., & Alonso, A. (1997). Sexual behavior regulated (paced) by the female induces conditioned place preference. Behavioral Neuroscience, 111, 123–128. Paredes, R. G., & Vazquez, B. (1999). What do females like about sex? Paced mating. Behavioural Brain Research, 105, 117–127. Pfaus, J. G., Mendelson, S. D., & Phillips, A. G. (1990). A correlational and factor analysis of anticipatory and consummatory measures of sexual behavior in the male rat. Psychoneuroendocrinology, 15, 329–340. Plaud, J. J., & Martini, J. R. (1999). The respondent conditioning of male sexual arousal. Behavior Modiﬁcation, 23, 254–269. Rachman, S. (1966). Sexual fetishism: An experimental analogue. Psychological Record, 16, 293–296. Rozin, P., & Schull, J. (1988). The adaptiveevolutionary point of view in experimental psychology. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. Duncan Luce (Eds.), Stevens’ handbook of experimental psychology (2nd ed., Vol. 1, pp. 503–546). New York, NY: Wiley. Sachs, B. D., & Garinello, L. D. (1978). Interaction between penile reﬂexes and copulation in male rats. Journal of Comparative Physiological Psychology, 92, 759–678. Seligman, R. E. P. (1970). On the generality of the laws of learning. Psychological Review, 77, 406–418.

2/16/2011 6:52:34 PM

SEXUAL BEHAVIOR AND REPRODUCTION

Shefﬁeld, F. D., Wulff, J. J., & Backer, R. (1951). Reward value of copulation without sex drive reduction. Journal of Comparative Physiological Psychology, 44, 3–8. Schwartz, C. W., & Schwartz, E. R. (1949). A reconnaissance of the game birds in Hawaii. Hilo, Hawaii: Hawaii Board of Commissioners of Agriculture and Forestry. Shefﬁeld, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical conditioning (pp. 302–322). New York, NY: AppletonCentury-Crofts. Silva, K. M., & Timberlake, W. (1997). A behavior systems view of conditioned states during long and short CS-US intervals. Learning and Motivation, 28, 465–490. Silva, K. M., & Timberlake, W. (1998). A behavior systems view of responding to probe stimuli during an interfood clock. Animal Learning and Behavior, 26, 313–325. Taziaux, M., Kahn, A., Moore, J., Balthazart, J., & Holloway, K. S. (2008). Enhanced neural activation in brain regions mediating sexual responses following exposure to a CS that predicts copulation. Neuroscience, 151(3), 644–658.

22-Schachtman-22.indd 531

531

Timberlake, W. (2001). Motivational modes in behavior systems. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 155–209). Mahwah, NJ: Erlbaum. Troisi, J. R., & Akins, C. K. (2004). The discriminative stimulus effects of cocaine in a Pavlovian sexual approach paradigm in male Japanese quail. Experimental and Clinical Psychopharmacology, 12(4), 237–242. Volkow, N. D, Wang, G. J., Fowler, J. S., Telang, F., Jayne, M., & Wong, C. (2007). Stimulantinduced enhanced sexual desire as a potential contributing factor in HIV transmission. American Journal of Psychiatry, 164(1), 157–160. Zamble, E., Hadad, G. M., Mitchell, J. B., & Cutmore, T. R. H. (1985). Pavlovian conditioning of sexual arousal: First- and second-order effects. Journal of Experimental Psychology: Animal Behavior Processes, 11, 598–610. Zule, W. A., Costenbader, E. C., Meyer, W. J., Jr., & Wechsberg, W. M. (2007). Methamphetamine use and risky sexual behaviors during heterosexual encounters. Sexually Transmitted Diseases, 34(9), 689–694.

2/16/2011 6:52:34 PM

CHAPTER 23 Hot and Bothered Classical Conditioning of Sexual Incentives in Humans Heather Hoffmann

Cross-cultural and individual variation in erotic taste indicate that what we ﬁnd sexually attractive depends on experience. Partner and other environmental cues can acquire sexually arousing (or inhibiting) properties through a variety of different types of learning processes, including imprinting, mere exposure, social learning, verbal relational learning, and operant conditioning. Most laboratory research on sexual learning, however, has employed classical conditioning procedures. Numerous studies demonstrate the impact of such conditioning on a wide range of sexual behaviors in non-humans, yet relatively few studies have shown such effects in humans. The present chapter reviews the experimental research on classical conditioning of sexual arousal in humans, highlighting newer studies that use women participants and more diverse paradigms. Individual differences in conditionability and a distinction between signal versus evaluative learning are also considered. Such research has the potential to contribute to the literature on (human) learning theory as well as to enhance learning-based therapies used to alter problematic sexual responding, for example, in the case of sexual risk taking and/or sexual compulsivity.

The idea that learning plays an important role in sexual responding is not new (e.g., Binet, 1888; Craig, 1918), and it is now commonly assumed that conditioning processes affect the development of normative as well as atypical (e.g., fetishism) sexual arousal patterns (e.g., Ågmo, 1999; Gaither, Rosenkranz, & Plaud, 1998; Hardy, 1964; McConaghy, 1987; Pfaus, Kippin, & Centeno, 2001; Roche & Barnes, 1998; Woodson, 2002). Indeed, numerous experimental studies have demonstrated the impact that learning has on a wide range of sexual behaviors across a variety of species (see Akins, 2004; Chapter 2, this volume; Domjan & Holloway, 1998; Pfaus, Kippin, & Centeno, 2001, for reviews). However, there is still relatively little empirical evidence of sexual conditioning from studies using human and/or female subjects. Furthermore, the animal

and human sexual conditioning literatures are not clearly integrated (Akins, 2004), and precisely how conditioning processes affect what we ﬁnd erotic remains unclear. Only a narrow range of stimuli can be regarded as primary or “inherent” sexual incentives (i.e., sexually attractive cues). Stimuli typically acquired sexually arousing properties through experience. While a few studies have shown that partner and other environmental stimuli can become sexually arousing through imprinting (e.g., Bateson, 1978; Kendrick, Hinton, Atkins, Haupt, & Skinner, 1998), mere exposure (e.g., Dewsbury, 1981; Lisk & Baron, 1982), observational learning (e.g., Köksal & Domjan, 1998; White, 2004), and verbal learning (Gavin, Roche, & Ruiz, 2008; Roche & Barnes, 1998), acquired preferences are more

532

23-Schachtman-23.indd 532

2/16/2011 6:05:41 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

commonly attributed to classical (Pavlovian) conditioning and/or operant (instrumental) conditioning (e.g., Akins, 2004; Laws & Marshall, 1991; McConaghy, 1987; Pfaus et al., 2001; Woodson, 2002). Classical conditioning consists of learning about the relationship between an initially innocuous cue (the conditioned stimulus [CS]) and a biologically signiﬁcant one (the unconditioned stimulus [US]). This type of conditioning, in contrast to operant conditioning, appears most directly related to how cues can acquire arousing properties and therefore is the focus of the present chapter. Nonetheless, operant procedures have been shown to inﬂuence human sexual arousal (Rosen, Shapiro, & Schwartz, 1975), and both classical and instrumental processes/procedures most likely interact in the development of sexual preferences (e.g., Junginger, 1997; McGuire, Carlisle, & Young, 1965; also see Chapter 1, this volume).

THE NATURE OF SEXUAL CONDITIONING Pavlovian procedures have been employed in most animal and human studies aimed at examining the role of learning in sexual arousal. Although viewed as a reﬂexive process with limited applicability to higher level human behavior, we now recognize that classical conditioning can serve a number of important functions. For example, it can prepare organisms for interaction with biologically signiﬁcant cues or events (signal or expectancy learning), and it can alter the preference for stimuli associated with such cues or events (evaluative conditioning). The outcome of classical conditioning is typically regarded as signal learning in which the CS comes to predict the occurrence of the US and hence prepares the organism for it (e.g., Rescorla, 1988). Extensive laboratory data using animals support this conceptualization, and recent research deriving from a functional approach illustrates how classical conditioning leads to adaptive behaviors in the naturalistic environment (Domjan; 2005, Hollis, 1997; Timberlake, 2001). More speciﬁcally, there is evidence that sexual conditioning involves S-S rather than merely S-R relations (e.g., Hilliard &

23-Schachtman-23.indd 533

533

Domjan, 1995; Hilliard, Domjan, Nguyen, & Cusato, 1998; Holloway & Domjan, 1993) and that classical conditioning increases reproductive ﬁtness (Adkins-Regan & MacKillop, 2003; Hollis, Pharr, Dumas, Britton, & Field, 1997; Mahometa & Domjan, 2005; Matthews, Domjan, Ramsey, & Crews, 2007). Evaluative conditioning (EC) is a more recently recognized form of classical conditioning that involves a (associative) transfer of affective value, or valence, as a result of exposure to CS-US pairings (De Houwer, Thomas, & Baeyens, 2001; Chapter 18, this volume). In contrast to signal learning, EC has been almost exclusively researched in humans (largely because the subjective experience of liking or disliking is difﬁcult to measure in animals). The prototypical EC paradigm involves pairing of a neutral CS (e.g., a neutrally rated face) with pictures of liked or disliked stimuli, and it results in a change in liking for the CS. Evaluative conditioning appears to be robust in some instances (yet fails to appear in others; see De Houwer, Baeyens, & Field, 2005; Rozin, Wrzesniewski & Barnes, 1998) and has been demonstrated in a variety of paradigms employing diverse cues, including CSs from a range of sensory modalities and more biologically relevant USs as well as with direct and indirect measures (De Houwer et al., 2001). EC appears to be involved in the development of likes and dislikes (e.g., DeHouwer et al., 2005) as well as attitudes (e.g., Baccus, Baldwin, & Packer, 2004; Dijksterhuis, 2004; Gawronski & Bodenhausen, 2006; Karpinski & Hilton, 2001; Livingston & Drwecki, 2007), and recent research suggests it may also occur during sexual conditioning (Both et al., 2008; Hoffmann, 2007; Hoffmann & Janssen, 2006). Signal and evaluative conditioning can be dissociated within the same paradigm (e.g., Hermans, Vansteenwegen, Crombez, Baeyens, & Eelen, 2002), suggesting they are distinct, although not necessarily independent processes. Furthermore, EC appears less sensitive to CS-US contingency and modulation (e.g., Baeyens, Crombez, De Houwer, & Eelen, 1996; Baeyens, Hendrickx, Crombez, & Hermans, 1998; Olson & Fazio, 2001; but see Hardwick & Lipp, 2000; Lipp & Purkis, 2005) and more resistant to

2/16/2011 6:05:41 PM

534

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

extinction (e.g., Baeyens, Crombez, Van den Bergh, & Eelen, 1988; Hermans, Crombez, Vansteenwegen, Baeyens, & Eelen, 2002; but see Lipp & Purkis, 2005, 2006). In addition, there is evidence that EC can be acquired without awareness of the CS-US contingency (e.g., De Houwer, Hendrickx, & Baeyens, 1997; Dickinson & Brown, 2007; Walther & Nagengast, 2006) and that its expression may be more automatic (Neumann, Forster, & Strack, 2003; Öhman & Mineka, 2001; Yin & Knowlton, 2006). It has been proposed that classical conditioning may incorporate at least two different types of processes that may employ distinct algorithms for association (e.g., Baeyens, Vansteenwegen, Hermans, & Eelens, 2001; DeHouwer et al., 2001) and/or expression of learning (Field, 2005). Baeyens et al. (2001) proposed that signal learning is governed by an expectancy system that requires more cognitive resources to process or translate complex information resulting in anticipation of an object or event. On the other hand, EC is mediated by a more “primitive” referential system that employs more rudimentary learning or performance rules resulting in changes in affective value that can inﬂuence the direction of behavior (approach/avoid) and modulate (facilitate/suppress) responses generated by the expectancy system. It is possible that sexual conditioning involves both signal and evaluative learning. It is also possible that the nature of the association(s) acquired in a sexual setting may depend on factors such as type of cue, learning context, or individual personality. It was with these ideas that I began to explore what was known about the speciﬁc conditions under which classical conditioning affects (human) sexual responding and to plan and execute studies to better understand the nature of classical associations acquired in sexual situations.

SEXUAL CONDITIONING EXPERIMENTS: PAST AND PRESENT Non-Humans

The majority of studies demonstrating a role for classical conditioning in sexual behavior have

23-Schachtman-23.indd 534

been conducted in animals (see Akins, 2004; Chapter 22, this volume; Domjan & Holloway, 1998; Pfaus et al., 2001, for review). For example, Domjan and colleagues have found conditioned approach, conditioned courtship, and conditioned copulatory behaviors in male quail in the presence of CSs (e.g., colored lights, orange feathers, bird models, and contextual cues) that were previously paired with either visual exposure to a female or the opportunity to copulate with a female (for review, see Chapter 22, this volume; Domjan & Holloway, 1998). In male rats, Zamble, Hadad, Mitchell, and Cutmore (1985) found conditioned decreases in ejaculatory latency in the presence of a CS (plastic tub) that had previously been paired with exposure to a female, and Pfaus and colleagues found conditioned ejaculatory preference for females scented with an odor that was previously associated with the opportunity to copulate (Kippin, Talianakas, & Pfaus, 1997; Kippin, Talinakis, Schattmann, Bartholomew, & Pfaus, 1998). Sexual conditioning in males has also been evidenced using physiological and other nonbehavioral measures. For example, cues paired with access to a receptive female can increase serum lutenizing hormone (LH) and testosterone levels in male rats (Graham & Desjardins, 1980), sperm volume and concentration in quail (Domjan, Blesbois, & Williams, 1998), the number of offspring in quail (Adkins-Regan & MacKillop, 2003; Mahometa & Domjan, 2005; Matthews, Domjan, Ramsey, & Crews, 2007), and the number of offspring in blue gourami ﬁsh (Hollis, Pharr, Dumas, Britton, & Field, 1997). Although there is less research conducted using female non-humans, there are a few studies showing that they also can be sexually conditioned. Gutiérrez and Domjan (1997) found increased squatting behavior, an index of sexual receptivity, in female quail following the paired presentation of a particular compartment (CS) and copulatory opportunity. Coria-Avila, Ouimet, Pachero, Manzo, and Pfaus (2005) and Coria-Avila, Jones, Solomon, Garvrila, Jordan, and Pfaus (2006) showed a conditioned partner preference in female rats (increased solicitations, higher magnitude lordosis) for males bearing cues paired with the ability to pace copulation,

2/16/2011 6:05:41 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

which is rewarding for female rats (Paredes & Alonso, 1997). Furthermore, female hamsters show conditioned placed preference for an environment paired with sexual interaction (Meisel & Joppa, 1994) and an environment paired with vaginal stimulation (Kohlert & Olexa, 2005). Humans

O’Donohue and Plaud (1994) and Akins (2004) provide the most recent reviews of the conditioning of human sexual arousal. Most studies have tested male subjects using visual stimuli and have measured learning via changes in genital responding. Speciﬁcally, CSs have been nonsexual or sexual images, most often photographs, presented in slides or on a computer screen. Several studies, particularly the more recent ones, have employed a differential conditioning design. including a CS+ and a CS– (i.e., a cue presented during acquisition but not explicitly un/paired with the US nor explicitly un/paired with the CS+). The most commonly used USs have been sexually explicit photographs or ﬁlms. Genital responding (i.e., increased blood ﬂow) has been monitored by use of a penile plethysmograph (e.g., electromechanical strain

535

gauge’ Barlow, Becker, Leitenberg, & Agras, 1970; see Fig. 23.1) or vaginal photoplethysmograph (Hoon, Wincze, & Hoon, 1976; Sintchak & Geer 1975; see Fig. 23.2). Participants place these monitors on themselves in private with instruction from researchers. The strain gauge, placed approximately 1 cm from the base of the penis, detects changes in penile circumference. The photoplethysmograph resembles an acrylic tampon and its depth and position can be controlled by a perspex plate placement device (Laan, Everaerd, & Evers, 1995). This vaginal device monitors light reﬂection off vaginal walls, with greater back-scattered light representing increased blood ﬂow. It yields two analyzable signals, yet the AC-coupled signal known as vaginal pulse amplitude (VPA), which appears to reﬂect phasic changes in pressure in vaginal blood vessels, is the most commonly used measure. Although there is still some uncertainty about what changes in VPA represent (e.g., Levin, 1998), they appear to be speciﬁc to sexual as opposed to other types of stimuli (Laan et al., 1995). Due to their ease of use, both of these plethysmographs are standard in sexual psychophysiological research (Geer & Janssen, 2000) However, care should be taken in generalizing from human sexual psychophysiological

Figure 23.1 Penile plethysmograph. Courtesy Nikki Prause.

23-Schachtman-23.indd 535

2/16/2011 6:05:41 PM

536

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Figure 23.2 Vaginal photoplethysmograph. Courtesy Jorge Ponseti.

studies as a volunteer bias has been documented (e.g., participants have more as well as more varied sexual experience, higher rates of masturbation, lower sexual guilt and inhibition, higher sensation-seeking tendencies, and are lower in conformity; Bogaert, 1996; Morokoff, 1986; Plaud, Gaither, Hegstad, Rowan, & Devitt, 1999; Strassberg & Lowe, 1995). Although numerous case studies indicate that behavior or response modiﬁcation techniques can alter patterns of sexual arousal/ behavior (e.g., see Akins, 2004; Gaither et al., 1998 for review), relatively few experiments have shown that Pavlovian procedures can be used to condition arousal in a nonclinical sample. Rachman’s (1966) study was among the ﬁrst attempts. After 24–65 pairings of a slide of a color photograph depicting a pair of women’s knee-high black boots immediately followed by one of six slides of attractive nude women, the three male participants showed enhanced genital responding to the slide of the boots during extinction testing. However, methodological concerns (e.g., that lack of a control group) in this study as well as several other experiments conducted in the 1970s (e.g., Kantorowitz, 1978; Langevin & Martin, 1975; Rachman & Hodgson, 1968) precluded a convincing demonstration of classical conditioning of human sexual arousal.

23-Schachtman-23.indd 536

More carefully controlled experiments carried out in the late 1990s revealed unequivocal evidence of sexual conditioning, at least in men. Lalumière and Quinsey (1998), using a CS-only control in addition to an experimental group, found enhanced genital responding to slides of partially nude females after they had been paired with sexually explicit videotapes of heterosexual interactions. Plaud and Martini (1999), using a backward and a random control in addition to an experimental group, found conditioned increases in penile circumference to a penny jar after it was paired with slides of nude or partially nude females. Recent Research

More recent sexual conditioning research includes women and more diverse CSs (e.g., olfactory cues), USs (e.g., vibrogenital stimulation), and measures of conditioned responding (e.g., subjective arousal, general affective ratings, skin conductance). Our laboratory has also shown conditioning of sexual arousal in men outside of the laboratory (i.e., in their residence). The ﬁrst published study to examine the role of classical conditioning in women’s sexual response was conducted by Letourneau and O’Donohue (1997), but they failed to show

2/16/2011 6:05:41 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

conditioned genital or subjective responses to a CS (an amber light) that had been repeatedly paired with an erotic ﬁlm. The ﬁrst clear demonstration of sexual conditioning in women was found in our laboratory (Hoffmann, Janssen, & Turner, 2004). Like Letourneau and O’Donohue, we also used a visual stimulus as the CS and erotic ﬁlm clips as USs, yet our US and intertrial interval durations were shorter (paralleling those used in studies ﬁnding conditioned genital arousal in men) and our USs were more effective than theirs in inducing genital arousal. However, we did not directly test whether these variables were critical for effective conditioning. In fact, Both, Spiering, Laan, Belcome, Heuvel, and Everaerd (2008) and Both, Laan, Spiering, Nilsson, Oomens, and Everaerd (2008) have shown conditioned sexual arousal in women who reported that the US was moderately as opposed to highly sexually arousing (these latter studies are discussed more fully later in this chapter). Our original study was aimed at more than simply showing that women’s sexual arousal could be inﬂuenced by conditioning. We also tested gender differences by comparing males and females, as well as examined selective associations (e.g., whether particular CSs were more readily associated with sexual USs) and the role of conscious awareness in human sexual conditioning. We used a differential conditioning (CS+/CS–) design, an explicitly unpaired control group, and the same procedures to examine the conditioning of genital arousal in both women and men—using erotic ﬁlm clips (of heterosexual interactions) that had been rated as arousing by both men and women (Janssen, Carpenter, & Graham, 2003) as USs. Domjan and Hollis (1988) proposed that males might show conditioned sexual arousal more readily, and to a wider range of cues, than females. It has also been suggested that women’s sexual arousal may not be as readily conditionable as men’s (Bancroft, 1989; Kinsey, Pomeroy, Martin, & Gebhard, 1953). However, Baumeister’s (2000) proposal that women are more erotically plastic (e.g., ﬂexible in what they ﬁnd sexually attractive) suggests that they could be more sensitive to conditioning than men.

23-Schachtman-23.indd 537

537

We also explored the notion of biological preparedness (Garcia & Koelling, 1966; Seligman, 1970) since it appears that sexual incentives are not randomly distributed across the range of cues found in sexual situations. Stimuli associated with potential partners (e.g., feathers or more complete models of partners in birds) appear more effective than arbitrary ones in sexual conditioning in non-humans (Akins, 2000; Cusato & Domjan, 1998), and selective associations have been shown in human fear conditioning (e.g., Öhman, Esteves, & Soares, 1995). Unpublished studies examining cue-toconsequence speciﬁcity in sexual conditioning in men (Clement, 1989; De Gagne, 1988) yielded equivocal results. We proposed that a sexually relevant CS (photograph of an abdomen of the opposite sex) would be more effective as a conditionable cue than a sexually irrelevant CS (photograph of a gun). Finally we also manipulated the awareness of CS presentation because we assumed subjects would be less likely to alter their expression of arousal if they did not realize they were being conditioned. Speciﬁcally, we presented CSs either “subliminally” (i.e., for 30 msec followed immediately and hence backward masked by the ﬁlm US) or “consciously” (i.e., for 10 sec). Many argue that humans do not learn classical associations unless they are aware of the CS-US contingency (Lovibond & Shanks, 2002); however, examples of Pavlovian learning without awareness in humans exists (e.g., Bechara et al., 1995; Clark & Squire, 1998; Esteves, Parra, Dimberg, & Öhman, 1994; Öhman et al., 1995; Öhman & Mineka, 2001; Öhman & Soares, 1994; Öhman & Soares, 1998; Soares & Öhman, 1993), including cases of such learning when evaluative conditioning paradigms are used (e.g., De Houwer et al, 1997; Dickinson & Brown, 2007; Walther & Nagengast, 2006; but see, e.g., Dawson, Rissling, Schell, & Wilcox, 2007; Pleyers, Corneille, Luminet, & Yzerbyt, 2007). Our results showed that when stimuli were presented outside the subjects’ awareness, both women and men showed conditioned genital arousal to the abdomen CS but not to the gun CS. These results are similar to those found by Öhman, Esteves, and Soares (1995) using a

2/16/2011 6:05:43 PM

538

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

fear-conditioning paradigm; that is, they found conditioned increases in skin conductance to fear-relevant but not fear-irrelevant stimuli that were presented outside the subject awareness and paired with a mild shock to the ﬁngers. Even though our manipulation and measurement of awareness were rather crude, our results suggest a prepared link between sexually relevant stimuli and genital responses and support an independent role for automatic processing in sexual responding consistent with some models of sexual arousal (Janssen, Everaerd, Spiering, & Janssen, 2000). When consciously perceived CSs were used, however, men again showed conditioned increases in penile tumescence to the abdomen but not the gun CS, whereas women showed the opposite effect (i.e., conditioning to the gun but not the abdomen stimulus). The latter result was unexpected. Perhaps the gunarousal associations in women may have been facilitated by increased attention (Beylin & Shors, 1998; Shors & Matzel, 1997) or excitation transfer (Hoon, Wincze, & Hoon, 1977; Meston & Gorzalka, 1995; 1996; Meston & Heiman, 1998) as women (but not men) showed increased skin conductance responses to the gun (but not the abdomen) CS and a few women (anecdotally) reported that the picture of the gun made them slightly anxious. Few studies have concurrently examined sexual conditioning in men and women (also see Klucken et al., 2009) Across a range of studies detailed later, our work has shown that men are more readily and consistently conditionable; however, since our main measure of learning is physiological and we use different genital measures of arousal, it prevents a direct gender comparison. It could be that VPA is not as sensitive to conditioning effects as changes in penile circumference. A subsequent unpublished study conducted in our laboratory found that if the subjects were explicitly told that they were being conditioned they were more likely to show learning. In particular, men showed conditioned genital arousal to a cartoon sketch of a mason jar after it was paired with erotic ﬁlms clips but only when they were aware of being conditioned, although there was a nonsigniﬁcant trend for learning in

23-Schachtman-23.indd 538

unaware subjects. The effects appeared weaker in women, with only aware subjects showing a nonsigniﬁcant trend for learning. Perhaps awareness of the contingency may facilitate conditioning to sexually irrelevant CSs, but learning about sexually relevant stimuli may occur without such knowledge. Both, Spiering, Laan, Belcome, Heuvel, and Everaerd (2008) have provided stronger evidence of unconscious learning about (relevant) sexual stimuli in women. They paired an erotic photograph as the CS+ with clitoral vibrostimulation over 24 trials. A CS– (a different erotic photo) was also presented during acquisition. Both cues were displayed for 30 ms followed immediately by a 100 ms masking stimulus. A forced-choice recognition test veriﬁed lack of awareness of CS-US contingency. During training as well as the ﬁrst nine extinction trials, women showed higher VPA to the CS+ versus the CS–. This differential conditioning was impressive considering the similarity between CS+ and CS– stimuli (i.e., both depicted heterosexual intercourse, differing only by actor and position). Both, Laan, Spiering, Nilsson, Oomens, and Everaerd (2008) showed another demonstration of appetitive sexual conditioning in women as well as providing the ﬁrst demonstration of attenuation of sexual response by aversive conditioning in women. For appetitive conditioning they used a consciously displayed black and white drawing of a neutral male face as the CS+ paired with vibrogenital stimulation (and another face as the CS– presented during 10 conditioning trials. They observed increased VPA to the CS+ relative to the CS–. Furthermore, they tested for evaluative conditioning, ﬁnding a marginally signiﬁcant preference for the CS+ over the CS– face. However, it is unclear if this represented an increase in preference for the CS+ or a decrease in preference for the CS–, or both. Unpublished results from our laboratory suggest that the affective value of both CSs can change (in opposite directions) during sexual conditioning (see later discussion). Nonetheless, Both et al. found genital and the evaluative CRs to be positively correlated. For aversive conditioning an erotic photograph was paired with a 50 ms wrist shock during aversive conditioning.

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

Decreased VPA to CS+ relative to the CS– during acquisition and extinction testing as well as lower affective preference for CS+ versus CS– were found. However even though subjects liked the CS+ picture less than the CS– picture, they found them equally sexually arousing (We also have not found changes in subjective arousal to the CS after conditioning). Evidence from Both and her colleagues as well as from our laboratory corroborate the existence of sexual conditioning in women. Yet, as with men, the learning has not been robust. Hence, we investigated whether changing the CS (to an olfactory cue) or the conditioning context (to the “real world” as opposed to the laboratory) could increase the strength of learning. Furthermore, we have begun to explore individual differences in sexual conditionability. Olfactory Conditioned Stimuli

Olfactory cues play a large role in sexual arousal in other mammals, and odors have been used as effective CSs in the conditioning of sexual arousal in a variety of non-human species (Domjan & Holloway, 1998; Pfaus et al., 2001). While noxious odor stimuli have been used as USs to decrease sexual arousal in clinical settings (e.g., Colson, 1972; Earls & Castonguay, 1989; Junginger, 1997), until recently olfactory stimuli have not been used as CSs in the conditioning of human sexual arousal. Studies indicate that smell plays a signiﬁcant role in human sexual attraction. Herz and Cahill (1997) found that odor is an important guide for mate selection in women and men, and that women valued odor more so than men. Herz and Inzlicht (2002) showed that, for women, body odor was more important than looks (the reverse was true for men); and that, for women, smell was more valuable than all but one social factor (i.e., pleasantness). Because people report that smell is important in attraction and since the majority of fetishes are related to olfactory or tactile stimuli (Money, 1988), we reasoned that humans, and in particular women, may be likely to associate sexual arousal with odor cues. Hoffmann and Janssen (2006) used discrete odor cues as CSs and 30-sec erotic ﬁlm clips as USs. Subjects received 28 pairings of either

23-Schachtman-23.indd 539

539

lemon or strawberry odor delivered via an olfactometer with a different ﬁlm clip over a 2-day period. A control group received explicitly unpaired presentations of the CS and US. Genital and subjective arousal to the CS, as well as preference for the odor that served as the CS, were assessed before and after training. Subsequently, Hoffmann (2007) used a similar procedure except that there were fewer conditioning trials (22), a randomized control as opposed to explicitly unpaired control group was used, and most important, the US was more primary (vibrotacile stimulation of the genitals; 30 sec per trial). Results for each of these studies were nearly identical. In both of these studies men showed appreciable learning. Although genital conditioned responses (CRs) were not robust, there was a signiﬁcant difference in conditioned responding between the experimental and control groups and the CRs were comparable in strength to those obtained with visual CSs (e.g., Hoffmann et al., 2004; Lalumière & Quinsey, 1998). There was some suggestion that women also showed learning, but it was much less convincing. Neither men nor women reported a change in subjective arousal to the CS after conditioning; however, there was a nonsigniﬁcant trend for men to prefer the odor CS more after conditioning with the ﬁlm clip US, whereas a trend for both men and women to show an increased preference for the odor CS after training occurred with the vibrogenital US. Hence, it appears that evaluative in addition to signal or predictive learning may occur in certain sexual conditioning situations. That is, in addition to showing increased genital responding to the CS+, participants showed an increased preference for the odor CS. Field Conditioning

Human sexual conditioning effects are less robust compared with those obtained in other animals. This may be because of the artiﬁciality of the laboratory environment and/or because of the choice of US (watching erotic ﬁlm clips as opposed to participating in sexual activity). Although a ﬁeld study is not as controlled as a laboratory experiment, it offers the potential of a

2/16/2011 6:05:43 PM

540

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

more appropriate context for sexual arousal and a more effective US. Furthermore, affective learning may be stronger in real-world settings (Baeyens, Wrzesniewski, De Houwer, & Eelen, 1996; Öhman & Mineka, 2001; Rozin, Wrzesniewski, & Byrnes, 1998). We predicted that enhanced sexual conditioning would occur outside the laboratory. Heterosexual couples were instructed to include a novel scent during sexual interaction (CS+) and another novel scent during nonsexual interaction (CS–). Control couples used both scents during nonsexual interaction. Conducted over a 2-week period, both experimental and control couples had three sexual interactions (oral sex and/or intercourse) during this time. In addition, experimental couples had three, while the controls had six, nonsexual interactions (e.g., studying together, watching a movie, playing video games) that also involved the presence of a novel (CS–) odor. Genital responding to and affective preference for the odors were assessed in the laboratory before and after the experience in the men. We only tested men for several reasons. Based on our previous studies men were more likely to show conditioning, women are less likely to agree to use the genital monitors, and we needed the assistance of one of the partners to “conduct the conditioning”; hence, men knew they were in a study on odors and sexual arousal but did not know they were being conditioned. We found evidence of conditioning using both the genital and affective measures. We observed signiﬁcantly increased genital responding to the CS+ in the experimental group relative to the control group; however, CRs were not much stronger than those obtained in the laboratory conditioning. One reason for the relatively weak conditioning (the conditioning in the laboratory, as mentioned earlier, was also weak) may have been that the males, although instructed to contact the experimenter as soon as they ﬁnished the study, did not return to the laboratory until several (as many as 11) days after completion. However there was no correlation between the length of the “retention interval” and the strength of the CR, but our sample was small (n = 7 in each group). Affective learning, that is, the increased preference for the CS+

23-Schachtman-23.indd 540

odor and decreased preference for the CS– odor, was evident and stronger than in laboratory studies. However, we used more neutrally rated odors as CS in the ﬁeld study, potentially increasing the sensitivity of affective preference test. It was interesting to see learning about the CS– odor, and in fact the decreased preference for the CS– was greater than the increased preference for the CS+, although we did not explicitly test whether the CS– became a conditioned inhibitor. We have seen a trend for this effect in other studies, and the demonstration of conditioned inhibition in a sexual situation has potential for enhancing learning-based therapies to be used to alter problematic sexual responding. Individual Differences

Individual differences in classical conditionability exist in humans (e.g., Martin, 1997). Certain types of people appear more susceptible to expectancy conditioning (e.g., Cook, Hodes, & Lang, 1986; Hamm & Vaitl, 1996; Hodes, Cook, & Lang, 1985; Kvale, Psychol, & Hugdahl, 1994; Mineka & Zinbarg, 2006) as well as evaluative conditioning (Baeyens et al., 1996; Yeomans, Mobini, Elliman, Walker, & Stevenson, 2006). A few studies report individual differences in sexual conditionability. In an early study Kantorowitz (1978) showed a signiﬁcant correlation between extraversion and conditioning of preorgasmic sexual arousal and a signiﬁcant correlation between introversion and postorgasmic sexual arousal, indicating that certain personality types are more susceptible to sexual conditioning. Our studies have found that conditionability did not appear to be related to the amount of experience subjects had with erotic ﬁlm; however, there was not much variation among our participants on this amount of experience. We also collected survey data from participants in some of our studies and correlated it with the strength of the CR. Speciﬁcally, we measured introversion/extraversion and Sexual Sensation Seeking (SSS; Kalichman et al. 1994), and we administered the Sexual Experience Scales (SES) subscale for Psychosexual Stimulation (Frenken, 1981) and the Sexual Inhibition

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

Scales/Sexual Excitation Scales (SIS/SES) (Janssen, Vorst, Finn, & Bancroft, 2002). The SES measures the extent that someone seeks sexual stimuli of an auditory-visual or imaginary kind and the SISSES measures sexual excitation (SE) or the propensity for sexual arousal and two forms of sexual inhibition (SIS-1 and SIS-2). SIS-1 is inhibition related to performance failure and SIS-2 is inhibition related to other negative consequences. For men we have not found a clear relationship between the strength of the CR and our survey measures, yet our male participants appeared to be a somewhat homogeneous group. For women we found that the strength of the CR tended to be inversely related (near signiﬁcant correlation coefﬁcients) to their scores on both inhibition scales (SIS-1 and SIS-2), so women who are low in sexual inhibition may be more likely to show conditioned sexual arousal. Both et al. (2008) found that women higher in sexual functioning showed stronger genital CRs and those higher in sexual arousability showed stronger genital along with affective CRs. Perhaps women who are more comfortable with sexuality are more conditionable. Exploring individual differences in sexual conditionablity may yield insights into sexual problems such as sexual dysfunction and the development of paraphilias, for example, fetishism. We recently have begun to consider the role of conditioning and conditionablity in sexual risk taking and sexual compulsion. Application to Sexual Risk Taking and Sexual Compulsion

The term sexual compulsivity refers to a preoccupation with and/or engagement in a range of sexual behaviors (e.g., masturbation, partnered sex, use of erotic/pornographic imagery) that are experienced as being excessive or out of control and that may have a negative impact on physical, emotional, and behavioral functioning of the individual. It has been estimated, although not studied in nationally representative samples, to occur in up to 10% of the population. It appears to be more common in men, and even more common in men who have sex with men (MSM; Black, Gates, Sanders, & Taylor, 2000; Black,

23-Schachtman-23.indd 541

541

Kehrberg, Flumerfelt, & Schlosser, 1997; Parsons et al., 2008). While a number of hypotheses exist on the origins of sexual compulsivity, little empirical research has examined its etiology. Sexual compulsivity and other forms of “out-ofcontrol” behaviors tend to be subjectively experienced as involving uncontrollable internal desires or “urges”; however, most models of sexual response and sexual motivation emphasize the role of external cues as triggers for sexual arousal and associated behavior (Barlow, 1986; Hardy, 1964; Pfaus, 1999; Toates, 2009; Whalen, 1966). Hence, sexual behavior is not seen as primarily “pushed” from within but rather starts with and depends on the effectiveness of sexual cues and incentives. Parsons, Kelly, Bimbi, Muench, and Morgenstern (2007) interviewed MSM who reported experiencing compulsive sexual behavior about the triggers for such behavior. They reported two main types: (1) event triggers such as relationship turmoil and other catastrophes and (2) contextual triggers such as particular locations, past partners, substance use, and erotic material. The acknowledgment that external cues can precipitate or activate sexual compulsivity suggests that learning and in particular conditioning may play a role in the etiology and maintenance of out-of-control sexual behavior (Goodman, 1998; Parsons et al., 2007; Putnam, 2000). Thus, a way to conceptualize compulsive sexual behavior that leads to testable hypotheses on etiology involves the idea that the opposing tendencies of appetitive (sexual) motivation and restraint are not balanced to yield appropriate (safe) sexual choices in certain individuals and/ or under certain conditions (Orford, 2001). Speciﬁcally, people who engage in compulsive sexual behaviors may be, or may have learned to be, more responsive to sexual stimuli (e.g., show stronger excitatory responses) and/or they may be less able to control their response to such cues (e.g., show weaker regulatory or inhibitory responses). I am particularly interested in whether sexually compulsive individuals could be more responsive to sexual cues because they are more (sexually) conditionable. Support for the idea that classical conditioning is involved in compulsive behavior can be

2/16/2011 6:05:43 PM

542

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

found in the drug addiction literature. Incentive sensitization theory is an experimentally validated neurobiological theory of addiction (Berridge & Robinson, 2003; Robinson & Berridge, 1993, 2000, 2003) implicating classical conditioning as the main mechanism by which stimuli paired with abused drugs acquire incentive properties. This theory proposes that drugs of abuse can produce long-lasting changes in the mesolimbic dopamine (i.e., brain reward) system. Once a drug (US) is predicted by environmental cues (CSs), increased dopamine in the reward circuit does not occur to the drug itself but rather to its predictive cues (Schultz, Dayan, & Montague, 1997). More speciﬁcally, drug-induced changes in the reward system contribute to sensitization or a heightened responsiveness to the drug itself as well as to the stimuli associated with drug use. Drugpaired cues are said to become “motivational magnets” for inducing drug-seeking behavior. The phenomenon of sign tracking also links classical conditioning to compulsive behavior. Sign tracking involves continued approach to a CS that previously predicted an appetitive (e.g., food or water) US, even when such a response delays or eliminates presentation of the appetitive US (e.g., Brown & Jenkins, 1968; Costa & Broakes, 2007; Davey & Cleland, 1982; Jenkins & Moore, 1973; Peterson, Ackil, Frommer, & Hearst, 1972; Williams & Williams, 1969). Signtracking behavior is seen as being maladaptive, and it has been incorporated in models of addiction (Luenberger, 1979; Newlin, 2002; Tomie, Grimes, & Pohorecky, 2008). Recent evidence from animal studies shows that sign tracking occurs with ethanol- (Cunningham & Patel, 2007) and cocaine- (Uslaner, Acerbo, Jones, & Robinson, 2006) related cues. Furthermore, Flagel, Watson, Robinson, and Akil (2007) found individual differences in the propensity for sign tracking and proposed that such differences may be due to differences in mesolimbic reward circuitry. Although sexual behavior does not involve “exogenous” alteration of the mesolimbic dopamine system (as is the case with drug use), sexual compulsivity is often associated with substance abuse and has been conceptualized as an

23-Schachtman-23.indd 542

addiction itself. Furthermore, there is good evidence that the mesolimbic dopamine system is involved with sexual reward (Balfour, Yu, & Coolen, 2004; Bradley & Meisel, 2001; Fiorino, Coury & Phillips, 1997; Hull, Du, Lorrian, & Matuszewich, 1995; Kohlert & Meisel, 1999; Mermelstein & Becker, 1995; Parades & Ågmo, 2004; Pfaus, Damsma, Wenkstern, & Fibigar, 1995), and cross-sensitization between drug and sexual reward has been demonstrated (Levens & Akins, 2004; Mitchell & Stewart, 1990; Nocjar & Panksepp, 2002). Sexual compulsion could result from stronger arousal and/or reward system responding to sexual cues and/or from being aroused/motivated by a wider range of cues, either of which could derive from being more sexually conditionable. It is also possible that in sexually compulsive individuals conditioning processes lead to changes in sexual cue valence (evaluative conditioning), possibly leading to stronger and more enduring conditioned sexual responding. A few studies provide evidence for a link between sexual conditioning and compulsive behavior in the form of sexual sign tracking. Burns and Domjan (1996, 2000) found that male Japanese quail would approach and remain close to a wooden block CS that predicted copulatory access to a female, even when the block was located a distance from the goal door in which the female would appear. In addition, Kimura, Fukui, and Inaki (1990) also showed that men and women came to ﬁxate on and press a button, whose lighting preceded the showing of an erotic ﬁlm clip, despite the fact that such behavior was unrelated to presentation of the erotic US. We are currently examining the role of sexual conditionability in sexual risk taking and sexual compulsion.

CONCLUSION Although it was and perhaps still is commonly assumed that classical conditioning plays a role in what we ﬁnd sexually arousing, particularly in cases of deviant arousal patterns such as fetishes, a strict learning interpretation of the development of sexual preferences has fallen out of favor. Yet with modern developments in learning theory it seems appropriate to renew the

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

investigation of contributions and limitations of conditioning processes in explaining how cues acquire erotic meaning. Further research into human sexual conditioning and the relative contributions of expectancy versus evaluative conditioning may contribute to the literatures on human learning theory and in particular evaluative conditioning. Moreover, such research may help us to better understanding the impact that erotic stimuli have on sexual arousal and subsequent behavior, potentially allowing us to alter such responses to improve sexual functioning. Such information could have direct application to managing sexual risk taking, sexual compulsion, and paraphilic (e.g., fetishistic) behavior as well as in attempting to restore sexual functioning after rape or other sexual trauma.

REFERENCES Adkins-Regan, E., & MacKillop, E. A. (2003). Japanese quail (Coturnix japonica) inseminations are more likely to fertilize eggs in a context predicting mating opportunities. Proceedings of the Royal Society of London B: Biological Sciences, 270, 1685–1689. Ågmo, A. (1999). Sexual motivation—an inquiry into events determining the occurrence of sexual behavior. Behavioural Brain Research, 105, 129–150. Akins, C. K. (2000). Effects of species speciﬁc cues and the CS-US interval on the topography of the sexually conditioned response. Learning and Motivation, 31, 211–235. Akins, C. K. (2004). The role of Pavlovian conditioning in sexual behavior: A comparative analysis of human and nonhuman animals. International Journal of Comparative Psychology, 17, 241–262. Baccus, J. R., Baldwin, M. W., & Packer, D. J. (2004). Increasing implicit self-esteem through classical conditioning. Psychological Science, 15, 498–502. Baeyens, F., Crombez, G., De Houwer, J., & Eelen, P. (1996). No evidence for modulation of evaluative ﬂavor-ﬂavor associations in humans. Learning and Motivation, 27, 200–241. Baeyens, F., Crombez, G., Van den Bergh, O., & Eelen, P. (1988). Once in contact always in contact: Evaluative conditioning is resistant to

23-Schachtman-23.indd 543

543

extinction. Advances in Behaviour Research and Therapy, 10, 179–199. Baeyens, F., Hendrickx, H., Crombez, G., & Hermans, D. (1998). Neither extended sequential nor simultaneous feature positive training result in modulation of evaluative ﬂavor-ﬂavor conditioning in humans. Appetite, 31, 185–204. Baeyens, F., Vansteenwegen, D., Hermans, D., & Eelen, P. (2001). Chilled white wine, when all of a sudden the doorbell rings: Mere reference and evaluation versus expectancy and preparation in human Pavlovian learning. In F. Columbus (Ed.), Advances in psychology research (Vol. 4, pp. 241–277). Hauppauge, NY: Nova Science Publishers. Baeyens, F., Wrzesniewski, A., De Houwer, J., & Eelen, P. (1996). Toilet rooms, body massages, and smells: Two ﬁeld studies on human evaluative odor conditioning. Current Psychology, 15, 77–96. Balfour, M. E., Yu, L., & Coolen, L. M. (2004). Sexual behavior and sex-associated environmental cues activate the mesolimbic system in male rats. Neuropsychopharmacology, 29, 718–730. Bancroft, J. (1989). Human sexuality and its problems (2nd ed.). Edinburgh, Scotland: Churchill Livingstone. Barlow, D. H. (1986). Causes of sexual dysfunction: The role of anxiety and cognitive interference. Journal of Consulting and Clinical Psychology, 54, 140–148. Barlow, D. H., Becker, R., Leitenberg, H., & Agras, W. S. (1970). A mechanical strain gauge for recording penile circumference change. Journal of Applied Behavioral Analysis, 3, 73–76. Bateson, P. P. G. (1978). Sexual imprinting and optimal outbreeding. Nature, 273, 659–660. Baumeister, R. (2000). Gender differences in erotic plasticity: The female sex drive as socially ﬂexible and responsive. Psychological Bulletin, 126, 347–374. Bechara, A., Tranel, D., Damasio, H., Adolphs, R., Rockland, C., & Damasio, A. R. (1995). Double dissociation of conditioning and declarative knowledge relative to the amygdale and hippocampus in humans. Science, 269, 1115–1118. Berridge, K. C., & Robinson, T. E. (2003). Passing reward. Trends in Neurosciences, 26, 507–513. Beylin, A. V., & Shors, T. J. (1998). Stress enhances excitatory trace eyeblink conditioning and opposes acquisition of inhibitory conditioning. Behavioral Neuroscience, 112, 1327–1338.

2/16/2011 6:05:43 PM

544

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Binet, A. (1888). Etudes de psychologie experimentale. Paris, France: Octave Doin. Black, D., Gates, G., Sanders, S., & Taylor, L. (2000). Demographics of the gay and lesbian population in the United States: Evidence from available systematic data sources. Demography, 37, 139–154. Black, D. W., Kehrberg, L. L. D., Flumerfelt, D. L., & Schlosser, S. S. (1997). Characteristics of 36 subjects reporting compulsive sexual behavior. American Journal of Psychiatry, 154, 243–249. Bogaert, A. F. (1996). Volunteer bias in human sexuality research: Evidence for both sexuality and personality differences in males. Archives of Sexual Behavior, 25, 125–140. Both, S., Laan, E., Spiering, M., Nilsson, T., Oomens, S., & Everaerd, W. (2008). Appetitive and aversive classical conditioning of female sexual response. Journal of Sexual Medicine, 5, 1386–1401. Both, S., Spiering, M., Laan, E., Belcome, S., Van den Heuvel, B., & Everaerd, W. (2008). Unconscious classical conditioning of sexual arousal: Evidence for the conditioning of female genital arousal to subliminally presented sexual stimuli. Journal of Sexual Medicine, 5, 100–109. Bradley, K. C., & Meisel, R. L. (2001). Sexual behavior induction of c-Fos in the nucleus accumbens and amphetamine-stimulated locomotor activity are sensitized by previous sexual experience in female Syrian hamsters. Journal of Neuroscience, 21, 2123–2130. Brown, P. L., & Jenkins, H. M. (1968). Auto-shaping the pigeon’s key peck. Journal of the Experimental Analysis of Behavior, 11, 1–8. Burns, M., & Domjan, M. (1996). Sign tracking versus goal tracking in the sexual conditioning of male Japanese quail (Coturnix japonica). Journal of Experimental Psychology: Animal Behavior Processes, 22, 297–306. Burns, M., & Domjan, M. (2000). Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters. Animal Learning and Behavior, 28, 109–119. Clark, R. E., & Squire, L. R. (1998). Classical conditioning and brain systems: The role of awareneness. Science, 280, 77–81. Clement, D. (1989). Le conditionnement classique et la réponse sexuelle masculine. Unpublished master’s thesis, Univeriste de Montreal, Montreal, Montreal, Quebec.

23-Schachtman-23.indd 544

Colson, C. E. (1972). Olfactory aversion therapy for homosexual behavior. Journal of Behavior Therapy and Experimental Psychiatry, 3, 185–187. Cook, E. W., Hodes, R. L., & Lang, P. J. (1986). Preparedness and phobia: Effects of stimulus content on human visceral conditioning. Journal of Abnormal Psychology, 95, 195–207. Coria-Avila, G. A., Jones, S. L., Solomon, C. E., Gavrila, A. M., Jordan, G. J., & Pfaus, J. G. (2006). Conditioned partner preference in female rats for strain of male. Physiology and Behavior, 88, 529–537. Coria-Avila, G. A., Ouimet, A. J., Pachero, P., Manzo, J., & Pfaus, J. G. (2005). Olfactory conditioned partner preference in the female rat. Behavioral Neuroscience, 119, 716–725. Costa, D. S. J., & Broakes, R. A. (2007). Maintenance of responding when reinforcement becomes delayed. Learning and Behavior, 35, 95–105. Craig, W. (1918). Appetites and aversions as constituents of instincts. Biological Bulletin, 34, 91–107. Cunningham, C. L., & Patel, P. (2007). Rapid induction of Pavlovian approach to an ethanolpaired visual cue in mice. Psychopharmacology, 192, 231–241. Cusato, B., & Domjan, M. (1998). Special efﬁcacy of sexual conditioned stimuli that include species typical cues: Tests with a conditioned stimuli preexposure design. Learning and Motivation, 29, 152–167. Davey, G. C. L., & Cleland, G. G. (1982). Topography of signal-centered behavior in the rat: Effects of deprivation state and reinforcer type. Journal of the Experimental Analysis of Behavior, 38, 291–304. Dawson, M. E., Rissling, A. J., Schell, A. M., & Wilcox, R. (2007). Under what conditions can human affective conditioning occur without contingency awareness? Test of the evaluative conditioning paradigm. Emotion, 7, 755–766. De Gagne, R. (1988). Le conditionnement classique et la réponse sexualle humaine. Unpublished manuscript, Universite de Montreal, Montreal, Montreal, Quebec. De Houwer, J., Baeyens, F., & Field, A. P. (2005). Associative learning of likes and dislikes: Some current controversies and possible ways forward. Cognition and Emotion, 19, 161–174. De Houwer, J., Hendrickx, H., & Baeyens, F. (1997). Evaluative conditioning with “subliminally”

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

presented stimuli. Consciousness and Cognition, 6, 87–107. De Houwer, J., Thomas, S., & Baeyens, F. (2001). Associative learning of likes and dislikes: A review of 25 years of research on human evaluative conditioning. Psychological Bulletin, 127, 853–869. Dewsbury, D. A. (1981). Effects of novelty of copulatory behavior: The Coolidge effect and related phenomena. Psychological Bulletin, 89, 464–482. Dickinson, A., & Brown, K. J. (2007). Flavorevaluative conditioning is unaffected by contingency knowledge during training with color-ﬂavor compounds. Learning and Behavior, 35, 36–42. Dijksterhuis, A. (2004). I like myself but I don’t know why: Enhancing implicit self-esteem by subliminal evaluative conditioning. Journal of Personality and Social Psychology, 86, 345–355. Domjan, M. (2005). Pavlovian conditioning: A functional perspective. Annual Review of Psychology, 56, 179–206. Domjan, M., Blesbois, E., & Williams, J. (1998). The adaptive signiﬁcance of sexual conditioning: Pavlovian control of sperm release. Psychological Science, 9, 411–415. Domjan, M., & Hollis, K. L. (1988). Reproductive behavior: A potential model system for adaptive specializations in learning. In R. C. Bolles & M. D. Beecher (Eds.) Evolution and learning (pp. 213–232). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Domjan, M., & Holloway, K. S. (1998). Sexual learning. In G. Greenberg & M. M. Haraway (Eds.), Comparative psychology: A handbook (pp. 602–613). New York, NY: Garland. Earls, C. M., & Castonguay, L. G. (1989). The evaluation of olfactory aversion for a bisexual pedophile with a single-case multiple baseline design. Behavior Therapy, 20, 137–146. Esteves, F., Parra, C., Dimberg, U., & Öhman, A. (1994). Nonconscious associative learning: Pavlovian conditioning of skin conductance responses to masked fear-relevance facial stimuli. Psychophysiology, 31, 375–385. Field, A. P. (2005). Learning to like (or dislike): Associative learning of preferences. In A. J. Wills (Ed.), New directions in human associative learning (pp. 221–252). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Fiorino, D. F., Coury, A., & Phillips, A. G. (1997). Dynamic changes in nucleus accumbens

23-Schachtman-23.indd 545

545

dopamine efﬂux during the Coolidge effect in male rats. Journal of Neuroscience, 17, 4849–4855. Flagel, S. B., Watson, S. J., Robinson, T. E., & Akil, H. (2007). Individual differences in the propensity to approach signals vs. goals promote different adaptations in the dopamine system of rats. Psychopharmacology, 191, 599–607. Frenken, J. (1981). Sexual experience scale manual. Zeitst, Netherlands: Swets & Zeitlinger BV. Gaither, G. A., Rosenkranz, R. R., & Plaud, J. J. (1998). Sexual disorders. In J. J. Plaud & G. H. Eifert (Eds.) From behavior theory to behavior therapy (pp. 152–177). Needham Heights, MA: Allyn & Bacon. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124. Gavin, A., Roche, B., & Ruiz, M. R. (2008). Competing contingencies over derived relational responding: A behavioral model of the implicit association test. Psychological Record, 58, 427–441. Gawronski, B., & Bodenhausen, G. V. (2006). Associative and propositional processes in evaluation: Conceptual, empirical, and metatheoretical issues. Psychological Bulletin, 132, 745–750. Geer, J. H., & Janssen, E. (2000). The sexual response system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed., pp. 315–341). New York, NY: Cambridge University Press. Graham, J. M., & Desjardins, C. (1980). Classical conditioning: Induction of luteinizing hormone and testosterone secretion in anticipation of sexual activity. Science, 210, 1039–1041. Gutiérrez, G., & Domjan, M. (1997). Differences in the sexual conditioned behavior of male and female Japanese quail (Coturnix japonica). Journal of Comparative Psychology, 111, 135–142. Goodman, A. (1998). Sexual addiction: An integrated approach. Madison, CT: International Universities Press, Inc. Hamm, A. O., & Vaitl, D. (1996). Affective learning: Awareness and aversion. Psychophysiology, 33, 698–710. Hardy, K. R. (1964). An appetitional theory of sexual motivation. Psychological Review, 71, 1–18. Hardwick, S. A., & Lipp, O. V. (2000). Modulation of affective learning: An occasion for evaluative

2/16/2011 6:05:43 PM

546

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

conditioning? Learning and Motivation, 31, 251–271. Hermans, D., Crombez, G., Vansteenwegen, D., Baeyens, F., & Eelen, P. (2002). Expectancylearning and evaluative learning in human classical conditioning: Differential effects of extinction. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 12, pp. 17–40). Hauppauge, NY: Nova Science Publishers. Hermans, D., Vansteenwegen, D., Crombez, G., Baeyens, F., & Eelen, P. (2002). Expectancy learning and evaluative learning in human classical conditioning: Affective priming as an indirect and unobtrusive measure of conditioned stimulus valence. Behaviour Research and Therapy, 40, 217–234. Herz, R. S., & Cahill, E. D. (1997). Differential use of sensory information in sexual behavior as a function of gender. Human Nature, 8, 275–286. Herz, R. S., & Inzlicht, M. (2002). Sex differences in response to physical and social factors involved in human mate selection: The importance of smell for women. Evolution and Human Behavior, 23, 359–364. Hilliard, S., & Domjan, M. (1995). Effects of sexual conditioning of devaluing the US through satiation. The Quarterly Journal of Experimental Psychology B: Comparative and Physiological Psychology, 48, 84–92. Hilliard, S., Domjan, M., Nguyen, M., & Cusato, B. (1998). Dissociation of conditioned appetitive and consummatory sexual behavior: Satiation and extinction tests. Animal Learning and Behavior, 26, 20–33. Hodes, R. L., Cook, E. W., & Lang, P. J. (1985). Individual differences in autonomic response: Conditioned association or conditioned fear? Psychophysiology, 22, 545–560. Hoffmann, H. (2007, August). Classical conditioning of sexual arousal in men and women with an olfactory CS and vibrotactile US: Evidence for signal and affective learning. Paper presented at the meeting of the International Academy of Sex Research, Vancouver, Canada. Hoffmann, H., & Janssen, E. (2006, July). Classical conditioning of sexual arousal to an olfactory cue in women and men: Who learns and what is learned? Paper presented at the meeting of the International Academy of Sex Research, Amsterdam, Netherlands. Hoffmann, H., Janssen, E., & Turner, S. (2004). Classical conditioning of sexual arousal in

23-Schachtman-23.indd 546

women and men: Effects of varying awareness and biological relevance of the conditioned stimulus. Archives of Sexual Behavior, 33, 43–53. Hollis, K. L. (1997). Contemporary research on Pavlovian conditioning: A “new” functional analysis. American Psychologist, 52, 956–965. Hollis, K. L., Pharr, V. L., Dumas, M. J., Britton, G. B., & Field, J. (1997). Classical conditioning provides paternity advantage for territorial male blue gouramis (Trichogaster trichopterus). Journal of Comparative Psychology, 111, 219–225. Holloway, K. S., & Domjan, M. (1993). Sexual approach conditioning: Tests of unconditioned stimulus devaluation using hormone manipulations. Journal of Experimental Psychology: Animal Behavior Processes, 19, 47–55. Hoon, P. W., Wincze, J. P., & Hoon, E. F. (1976). Physiological assessment of sexual arousal in women. Psychophysiology, 13, 196–204. Hoon, P. W., Wincze, J. P., & Hoon, E. F. (1977). A test of reciprocal inhibition: Are anxiety and sexual arousal in women mutually inhibitory? Journal of Abnormal Psychology, 86, 65–67. Hull, E. M., Du, J., Lorrian, D. S., & Matuszewich, L. (1995). Extracellular dopamine in the medial preoptic area: Implications for sexual motivation and hormonal control of copulation. Journal of Neuroscience, 15, 7465–7471. Janssen, E., Carpenter, D., & Graham, C. A. (2003). Selecting ﬁlms for sex research: Gender differences in erotic ﬁlm preference. Archives of Sexual Behavior, 32, 243–251. Janssen, E., Everaerd, W., Spiering, M., & Janssen, J. (2000). Automatic processes and the appraisal of sexual stimuli: Towards an information processing model of sexual arousal. Journal of Sex Research, 37, 8–23. Janssen, E., Vorst, H., Finn, P., & Bancroft, J. (2002). The Sexual Inhibition (SIS) and Sexual Excitation (SES) Scales: I. Measuring sexual inhibition and excitation proneness in men. Journal of Sex Research, 39, 114–126. Jenkins, H. M., & Moore, B. R. (1973). The form of the auto-shaped response with food or water reinforcers. Journal of the Experimental Analysis of Behavior, 20, 163–181. Junginger, J. (1997). Fetishism: Assessment and treatment. In D. R. Laws & W. O’Donohue (Eds.), Sexual deviance: Theory, assessment and treatment (pp. 92–110). New York, NY: Guilford Press.

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

Kalichman, S. C., Johnson, J. R., Adair, V., Rompa, D., Multhauf, K., & Kelly, J. A. (1994). Sexual sensation seeking: Scale development and predicting AIDS-risk behavior among homosexually active men. Journal of Personality Assessment, 62, 385–397. Kantorowitz, D. A. (1978). Personality and conditioning of tumescence and detumescence. Behaviour Research and Therapy, 16, 117–123. Karpinski, A., & Hilton, J. L. (2001). Attitudes and the implicit association test. Journal of Personality and Social Psychology, 81, 774–788. Kendrick, K. M., Hinton, M. R., Atkins, K., Haupt, M. A., & Skinner, J. D. (1998). Mothers determine sexual preference. Nature, 395, 229–230. Kimura, H., Fukui, I., & Inaki, K. (1990). Autoshaping of a button-push response and eye movement in human subjects. Japanese Journal of Psychology, 61, 351–355. Kinsey, A. C., Pomeroy, W. B., Martin, C. E., & Gebhard, P. H. (1953). Sexual behavior in the human female. Oxford, England: Saunders. Kippin, T. E., Talianakis, S., & Pfaus, J. G. (1997). The role of ejaculation in the development of conditioned sexual behaviors in the male rat. Social Behavioral Neuroendocrinology Abstracts, 1, 38. Kippin, T. E., Talianakis, S., Schattmann, L., Bartholomew, S., & Pfaus, J. G. (1998). Olfactory conditioning of sexual behavior in the male rat (Rattus norvegicus). Journal of Comparative Psychology, 112, 389–399. Köksal, F., & Domjan, M. (1998). Observational conditioning of sexual behavior in the domesticated quail. Animal Learning and Behavior, 26, 427–432. Kohlert, J. G., & Meisel, R. L. (1999). Sexual experience sensitizes mating-related nucleus accumbens dopamine response of female Syrian hamsters. Behavioural Brain Research, 99, 45–52. Kohlert, J. G., & Olexa, N. (2005). The role of vaginal stimulation for the acquisition of conditioned place preference in female Syrian hamsters. Physiology and Behavior, 84, 135–139. Kvale, G., Psychol, C., & Hugdahl, K. (1994). Cardiovascular conditioning and anticipatory nausea and vomiting in cancer patients. Behavioral Medicine, 20, 78–85. Laan, E., Everaerd, W., & Evers, A. (1995). Assessment of female sexual arousal: Response

23-Schachtman-23.indd 547

547

speciﬁcityandconstructvalidity.Psychophysiology, 32, 476–485. Lalumière, M. L., & Quinsey, V. L. (1998). Pavlovian conditioning of sexual interests in human males. Archives of Sexual Behavior, 27, 241–252. Langevin, R., & Martin, M. (1975). Can erotic response be classically conditioned? Behavioral Therapy, 6, 350–355. Laws, D. R., & Marshall, W. L. (1991). Masturbatory reconditioning with sexual deviates: An evaluative review. Advances in Behaviour Research and Therapy, 13, 13–25. Letourneau, E. J., & O’Donohue, W. (1997). Classical conditioning of female sexual arousal. Archives of Sexual Behavior, 26, 63–78. Levens, N., & Akins, C. K. (2004). Chronic cocaine pretreatment facilitates Pavlovian sexual conditioning in male Japanese quail. Pharmacology, Biochemistry and Behavior, 79, 451–457. Levin, R. J. (1998). Assessing human female sexual arousal by vaginal photoplethysmography: A critical examination. Sexologies, 6, 26–31. Lipp, O. V., & Purkis, H. M. (2005). No support for dual process accounts of human affective learning in simple Pavlovian conditioning. Cognition and Emotion, 19, 269–282. Lipp, O. V., & Purkis, H. M. (2006). The effects of assessment type on verbal ratings of conditional stimulus valence and contingency judgments: Implications for the extinction of evaluative learning. Journal of Experimental Psychology: Animal Behavior Processes, 32, 431–440. Lisk, R. D., & Baron, G. (1982). Female regulation of mating location and acceptance of new mating partners following mating to sexual satiety: The Coolidge effect demonstrated in the female golden hamster. Behavioral and Neural Biology, 46, 416–421. Livingston, R. W., & Drwecki, B. B. (2007). Why are some individuals not racially biased? Susceptibility to affective conditioning predicts nonprejudice towards Blacks. Psychological Science, 18, 816–823. Lovibond, P. F., & Shanks, D. R. (2002). The role of awareness in Pavlovian conditioning: Empirical evidence and theoretical implications. Journal of Experimental Psychology: Animal Behavior Processes, 28, 3–26. Luenberger, D. G. (1979). Introduction to dynamic systems: Theory, models, and applications. New York, NY: Wiley. Mahometa, M. J., & Domjan, M. (2005). Classical conditioning increases reproductive success in

2/16/2011 6:05:43 PM

548

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Japanese quail (Coturnix japonica). Animal Behavior, 69, 983–989. Martin, I. (1997). Classical conditioning and the role of personality. In H. Nyborg (Ed.), The scientiﬁc study of human nature: Tribute to Hans J. Eysenck at eighty (pp. 339–363). Amsterdam, Netherlands: Pergamon/Elsevier Sciences, Inc. Matthews, R. N., Domjan, M., Ramsey, M., & Crews, D. (2007). Learning effects on sperm competition and reproductive ﬁtness. Psychological Science, 18, 758–762. McConaghy, N. (1987). A learning approach. In J. Geer & W. O’Donohue (Eds.), Theories of human sexuality (pp. 287–334). New York, NY: Plenum Press. McGuire, R. J., Carlisle, J. M., & Young, B. G. (1965). Sexual deviations as conditioned behaviour: A hypothesis. Behavioral Research and Therapy, 3, 185–190. Meisel, R. L., & Joppa, M.A. (1994). Conditioned place preference in female hamsters following aggressive or sexual encounters. Physiology and Behavior, 56, 1115–1118. Meston, C. M., & Gorzalka, B. B. (1995). The effect of sympathetic activation on physiological and subjective sexual arousal in women. Behaviour Research and Therapy, 33, 651–664. Meston, C. M., & Gorzalka, B. B. (1996). The effects of immediate, delayed, and residual sympathetic activation on sexual arousal in women. Behaviour Research and Therapy, 34, 143–148. Meston, C. M., & Heiman, J. R. (1998). Ephedrineactivated physiological arousal in women. Archives of General Psychiatry, 55, 652–656. Mermelstein, P. G., & Becker, J. B. (1995). Increased extracellular dopamine in the nucleus accumbens and striatum of the female rat during paced copulatory behavior. Behavioral Neuroscience, 109, 354–365. Mineka, S., & Zinbarg, R. (2006). A contemporary learning theory perspective on the etiology of anxiety disorders: It’s not what you thought it was. American Psychologist, 61, 10–26. Mitchell, J. B., & Stewart, J. (1990). Facilitation of sexual behaviors in the male rat in the presence of stimuli previously paired with systemic injections of morphine. Pharmacology, Biochemistry and Behavior, 35, 367–372. Money, J. (1988). Gay, straight and in-between: The sexology of erotic orientation. New York, NY: Oxford University Press.

23-Schachtman-23.indd 548

Morokoff, P. J. (1986). Volunteer bias in the psychophysiological study of female sexuality. Journal of Sex Research, 22, 35–51. Newlin, D. B. (2002). The self-perceived survival ability and reproductive ﬁtness (SPFit) theory of substance use disorders. Addiction, 97, 427–445. Neuman, R., Forster, J., & Strack, F. (2003). Motor compatibility: The bidirectional link between behavior and evaluation. In J. Musch & K. Klaver (Eds.), The psychology of evaluation: Affective processes in cognition and emotion. Mahwah, NJ: Lawrence Erlbaum Associates. Nocjar, C., & Panksepp, J. (2002). Chronic intermittent amphetamine pretreatment enhances future appetitive behavior for drug- and natural-reward: Interaction with environmental variables. Behavioural Brain Research, 128, 189–203. O’Donohue, W., & Plaud, J. J. (1994). The conditioning of human sexual arousal. Archives of Sexual Behavior, 23, 321–344. Öhman, A., Esteves, F., & Soares, J. J. F. (1995). Preparedness and preattentive associative learning: Electrodermal conditioning to masked stimuli. Journal of Psychophysiology, 9, 99–108. Öhman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychological Review, 108, 483–522. Öhman, A., & Soares, J. J. F. (1994). “Unconscious anxiety”: Phobic responses to masked stimuli. Journal of Abnormal Psychology, 103, 231–240. Öhman, A., & Soares, J. J. F. (1998). Emotional conditioning to masked stimuli: Expectancies for aversive outcomes following nonrecognized fear-relevant stimuli. Journal of Experimental Psychology: General, 127, 69–82. Olson, M. A., & Fazio, R. H. (2001). Implicit attitude formation through classical conditioning. Psychological Science, 12, 413–417. Orford, J. (2001). Addiction as excessive appetite. Addiction, 96, 15–31. Paredes, R. G., & Ågmo, A. (2004). Has dopamine a physiological role in the control of sexual behavior? A critical review of the evidence. Progress in Neurobiology, 73, 179–226. Paredes, R. G., & Alonso, A. (1997). Sexual behavior regulated (paced) by the female induces conditioned place preference. Behavioral Neuroscience, 111, 123–128. Parsons, J. T., Kelly, B. C., Bimbi, D. S., DiMaria, L., Wainberg, M. L., & Morgenstern, J. (2008).

2/16/2011 6:05:43 PM

CLASSICAL CONDITIONING OF SEXUAL INCENTIVES IN HUMANS

Explanations for the origins of sexual compulsivity among gay and bisexual men. Archives of Sexual Behavior, 37, 817–826. Parsons, J. T., Kelly, B. C., Bimbi, D. S., Muench, F., & Morgenstern, J. (2007). Accounting for the social triggers of sexual compulsivity. Journal of Addictive Diseases, 26, 5–16. Peterson, G. B., Ackil, J. E., Frommer, G. P., & Hearst, E. S. (1972). Conditioned approach and contact behavior toward signals for food and brain-stimulation reinforcement. Science, 177, 1009–1011. Pfaus, J. G. (1999). Neurobiology of sexual behavior. Current Opinion in Neurobiology, 9, 751–758. Pfaus, J. G., Damsma, G., Wenkstern, D., & Fibigar, H. C. (1995). Sexual activity increases dopamine transmission in the nucleus accumbens and striatum of female rats. Brain Research, 693, 21–30. Pfaus, J. G., Kippin, T. E., & Centeno, S. (2001). Conditioning and sexual behavior: A review. Hormones and Behavior, 40, 291–321. Plaud, J. J., Gaither, G. A., Hegstad, H. J., Rowan, L., & Devitt, M. K. (1999). Volunteer bias in human psychophysiological sexual arousal research: To whom do our research results apply? Journal of Sex Research, 36, 171–179. Plaud, J. J., & Martini, J. R. (1999). The respondent conditioning of male sexual arousal. Behavior Modiﬁcation, 23, 254–268. Pleyers, G., Corneille, O., Luminet, O., & Yzerbyt, V. (2007). Aware and (dis)liking: Item-based analyses reveal that valence acquisition via evaluative conditioning emerges only when there is contingency awareness. Journal of Experimental Psychology: Learning, Memory and Cognition, 33, 130–144. Putnam, D. E. (2000). Initiation and maintenance of online sexual compulsivity: Implications for assessment and treatment. CyberPsychology and Behavior, 3, 553–563. Rachman, S. (1966). Sexual fetishism: An experimental analogue. Psychological Record, 16, 293–296. Rachman, S., & Hodgson, R. J. (1968). Experimentallyinduced “Sexual Fetishism” replication and development. Psychological Record, 18, 25–27. Rescorla, R. A. (1988). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43, 151–160. Robinson, T. E., & Berridge, K. C. (1993). The neural basis of drug craving: An incentivesensitization theory of addiction. Brain Research Reviews, 18, 247–291.

23-Schachtman-23.indd 549

549

Robinson, T. E., & Berridge, K. C. (2000). The psychology and neurobiology of addiction: An incentive-sensitization view. Addiction, 95, S91–S117. Robinson, T. E., & Berridge, K. C. (2003). Addiction. Annual Review of Psychology, 54, 25–53. Roche, B., & Barnes, D. (1998). The experimental analysis of human sexual arousal: Some recent developments. The Behavior Analyst, 21, 37–52. Rosen, R. C., Shapiro, D., & Schwartz, G. (1975). Voluntary control of penile tumescence. Psychosomatic Medicine, 37, 479–483. Rozin, P., Wrzesniewski, A., & Byrnes, D. (1998). The elusiveness of evaluative conditioning. Learning and Motivation, 29, 397–415. Schultz, W., Dayan, P., Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. Seligman, M. E. P. (1970). On the generality of the laws of learning. Psychological Review, 77, 406–418. Shors, T. J., & Matzel, L. D. (1997). Long-term potentiation: What’s learning got to do with it? Brain and Behavioral Sciences, 20, 597–655. Sintchak, G., & Geer, J. H. (1975). A vaginal plethysmograph system. Psychophysiology, 12, 113–115. Soares, J. J. F., & Öhman, A. (1993). Backward masking and skin conductance responses after conditioning to nonfeared but fear-relevant stimuli in fearful subjects. Psychophysiology, 30, 460–466. Strassberg, D. S., & Lowe, K. (1995). Volunteer bias in sexuality research. Archives of Sexual Behavior, 24, 369–382. Timberlake, W. (2001). Motivational modes in behavior systems. In R. R. Mowrer & S. B. Klein (Eds.), Handbook of contemporary learning theories (pp. 155–209). Mahwah, NJ: Erlbaum. Tomie, A., Grimes, K. L., & Pohorecky, L. A. (2008). Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse. Brain Research Reviews, 58, 121–135. Uslaner, J. M., Acerbo, M. J., Jones, S. A., & Robinson, T. E. (2006). The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behavioral Brain Research, 169, 320–324. Walther, E., & Nagengast, B. (2006). Evaluative conditioning and the awareness issue: Assessing contingency awareness with the four-picture recognition test. Journal of Experimental Psychology: Animal Behavior Processes, 32, 454–459.

2/16/2011 6:05:43 PM

550

LEARNING: HUMAN AND NON-HUMAN APPLICATIONS

Whalen, R. E. (1966). Sexual motivation. Psychological Review, 73, 151–163. White, D. J. (2004). Inﬂuences of social learning on mate-choice decisions. Learning and Behavior, 32, 105–113. Williams, D. R., & Williams, H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511–520. Woodson, J. C. (2002). Including “learned sexuality” in the organization of sexual behavior. Neuroscience and Biobehavioral Reviews, 26, 69–80.

23-Schachtman-23.indd 550

Yeomans, M. R., Mobini, S., Elliman, T. D., Walker, H. C., & Stevenson, R. J. (2006). Hedonic and sensory characteristics of odors conditioned by pairing with tastants in humans. Journal of Experimental Psychology: Animal Behavior Processes, 32, 215–228. Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7, 464–476. Zamble, E., Hadad, G. M., Mitchell, G. B., & Cutmore, T. R. (1985). Pavlovian conditioning of sexual arousal: First- and second-order effects. Journal Experimental Psychology: Animal Behavior Processes, 11, 598–610.

2/16/2011 6:05:43 PM

INDEX

Note: Page numbers followed by “f” or “t” indicate ﬁgures and tables, respectively. abstinence alcohol, 219–20 bupropion and, 254 cocaine, 324 through drug replacement therapy, 251 positive attention and, 256 related expectancies, 230 ACC. See anterior cingulate cortex acceptance, 257 acquisition, 49–50, 349f advertising and, 501 altruistic reinforcement and, 426 analogs, 423 attraction conditioning, 438 Campbell and Kraeling analog and, 433 conditional analgesia and, 312 context, 94 CS processing and, 313 emergent properties of, 348–51 with natural female features, 515f reacquisition in contrast to, 246 renewal and, 81 sample size and, 348–49 with sexual conditioning, 508, 509f social judgments and, 351– 58 verbal instruction and, 105 acquisition-extinction interval, 95–96 ACTH. See adrenocorticotropic hormone activation amygdala, 52–53 of behavioral control, 222–25 external, 348 factors for, 31–32 fear, 30–33 of fear structure, 37 inappropriate expectancies and, 226–27 internal, 348 psychomotor, 236 spreading, 346 adaptation, connectionist learning and, 346–48

adaptive specializations, 525 addiction associative bases for, 235 associative learning and, 285 cocaine, 274 context in, 249–50 cost of, 235 cue exposure and, 255–56 drug effects in contrast to, 222 drugs and, 13 habit learning and, 240 integrated treatment for, 261–62 interdrug associations and, 280–81 interoceptive conditioning and, 278–85 learning theory and, 261 reinforcement schedules and, 243 sexual compulsivity and, 541–42 S-R associations and, 240 substance abuse treatment and, 250–61 through substitution, 283 ADHD. See attention deﬁcit/hyperactivity disorder adrenergic binding sites, 192 adrenergic system, 31 adrenocorticotropic hormone (ACTH) HPA axis and, 191, 193f inescapable shocks and, 129 learned helplessness and, 125 advertising, 481–83 acquisition and, 501 affect in, 496–97 attention in, 487–88 behavioral measures, 489 blocking in, 492–94 brand placement within, 485–86 cognition and, 496–97 conditioning during, 484–85 conditioning parameters, 485–91 contextual cues in, 495–96 contingency awareness during, 497–98 control conditions, 489–90

551

24-Schachtman-Index.indd 551

2/16/2011 9:31:21 AM

552 advertising (Cont’d) evaluative conditioning in contrast to classical conditioning, 497–98 exposure in, 489 food preferences and, 291 individual difference and, 498–501 intertrial interval in, 489 involvement in, 498–500 latent inhibition in, 492 music in, 490–91 need for cognition in, 500 overshadowing in, 492–94 partial reinforcement in, 486–87 personality variables and, 498–501 positivity effect in, 487 prior belief and, 500–501 second-order conditioning in, 494–95 US preexposure in, 491–92 affect in advertising, 496–97 body size and, 388 eating disorders and, 385 men’s perceptions of women and, 382–84 perceived salience of, in women, 394 transfer, 496 aggression, 454 disinhibiting effects of alcohol and, 227–28 sexual, 376, 382–84 stop-signal model of behavioral control and, 223 AI. See anterior insula alcohol antidepressants and, 217 approach-avoidance behavior and, 227 binge drinking, 229 comorbidity with smoking, 281 compensatory reactions and, 224 conﬂicting expectancies and, 227–28 cue-exposure therapy for, 281–82 disinhibiting effects of, 227–28 effects, 214 evaluative conditioning and, 90 extinction and recovery and, 84 impairment, 216–18 inhibitory control and, 224–25 as learned behavior, 235 mediating expectancies with, 221 motor skills and, 218–19, 219f Pavlovian features of, 271 placebo in contrast to, 215–16 related expectancies, 224–25 response-appropriate expectancies and, 225–27 resurgence and, 247–48 S*d-Rd expectancies and, 216–18 social drinking and, 216–17 stop-signal model of behavioral control and, 223 tolerance, 213, 218–21 use and abuse, 229 allergic reactions, 199

24-Schachtman-Index.indd 552

INDEX altruism, 417, 425–28 delay effect in, 428 failures of, 427–28 replication of, 428 Alzheimer’s disease, 200 AM404, 261 ambiguity, extinction learning and, 82, 249 amnesia circuit selection and, 316 infantile, 112 amphetamine, 227 blocking and, 314 dopamine neurons and, 315 between drug conditioning and, 272–73 LI and, 154 overshadowing and, 314 amygdala, 32–33 activation, 52–53 activity, 458f in classical conditioning, 458 fear and anxiety behavior and, 47 observational fear learning and, 466 in social cognition, 461–62 social fear learning and, 472 startle reﬂex and, 49 threat responsiveness and, 63 trait anxiety and, 63 analgesia, 308–12 animals aversive stimuli and, 200 evaluative conditioning and, 406 expectancy in, 214 interference effect and, 122–23 learned fear in, 464–65 PTSD in, 29 R-O associations in, 9 sexual conditioning, 534–35 single-system propositional model and, 107 S-R associations in, 8–9 SSDRs and, 307 anterior cingulate cortex (ACC), 460, 463, 467 anterior insula (AI), 460, 463, 467 anticipatory behavior, 107 antidepressants, 217 antigenic challenge, 192 antipain mechanisms, 306 antisocial personality, 223 anxiety. See also pathological anxiety; trait anxiety attenuated LI and, 159 brain lesions and, 49 CBT for, 113–14 classical conditioning and, 45 clinical, 59 clinical evidence for, 47 conditioning, 53 conditioning paradigms, 49–50 as CR, 49 exposure therapy and, 84, 113

2/16/2011 9:31:21 AM

INDEX fear in contrast to, 46 generalized fear and, 456 genetic variation and, 52–53 individual differences in cognitive processing and, 376 inescapable shocks and, 129 irrational, 109–10 learned, 113 manifest, 423–24 neuroanatomical evidence in, 47–49 neuroimaging variation and, 52–53 observational learning and, 114 other pathway for acquisition of, 54–56 panic attacks and, 47 pathology, 60 physiology of, 46 reactions, 110 routes to, 110–13 single-system propositional model and, 107–16 treatment implications, 113–15 unpredictability and, 58–59 anxiety disorders attentional processes in models of, 44–45 conditioning abnormalities in, 49 differences in learning, 54–59 emotional processing theory and, 28 etiology of, 80 individual differences in fear conditioning and, 52 single-system propositional model and, 107–16 trait anxiety in contrast to, 69 apomorphine, 194 appetitive outcomes aversive in contrast to, 10t, 431 avoidance behavior in contrast to, 243 drug states and, 279 extinction and, 245 within-session fear reduction and, 34 applied behavior analytics, 171–72 approach-avoidance behavior, 227 approach behavior, 508–9 to CS, with copulation, 511f aroma, 295–96 ASD. See autism spectrum disorder Asperger’s syndrome, 168 assimilation, 360–61, 361f associability in advertising, 488 associative learning and, 159 CS, 313–14 associations algorithms, 534 in fear learning, 56–57 forming, 400 interdrug, 280–81 intra-administration, 273–74 learned, 238f types of, 3 within-compound, 14–15 associative learning

24-Schachtman-Index.indd 553

553 addiction and, 285 anxiety reactions and, 110 associability and, 159 circuit selection and, 316–17 cognitive processes in, 105–6 dual-system model of, 105–6 expectancy and, 104 nonreinforcement and, 281 single-system propositional model of, 106–7 social processes and, 345–46 in women’s learning study, 387 associative structures, 3, 15. See also speciﬁc associative structures attention in advertising, 487–88 dimensional, 381 error correction and, 312–14 negative, 256 positive, 256 reallocating, 383–84 selective, 379 attentional biases assessment, 60–61 automaticity of, 66–68 conditioning in, 68 CS+ in contrast to CS-, 64–65 depression-related, 61–62 fear conditioning and, 64–69 index, 66f theories, 61–64 threat-relevant, 44–45, 59–64 time course of, 62 trait anxiety and, 59, 68–69 attentional deﬁcits, 152, 169 attentional shifting, 65–66 artiﬁcial stimuli and, 396 blocking and, 395 category learning and, 381 hypothesis, 130 in women, 394 attention deﬁcit/hyperactivity disorder (ADHD), 223 attitude change, 359t, 365–67 formation, 367f attraction conditioning, 437–41 in context, 441 interpersonal, 437–41 losses in strength of, 440 social, 417 superconditioning, 439–40 augmentation, 349–50 attribution ratings after, 357f in causal attribution, 355–56 of causal relationship detection, 444 in dispositional attribution, 356–58 of human agency, 444

2/16/2011 9:31:21 AM

554 autism spectrum disorder (ASD), 168–69 central coherence theory and, 170–71 cognitive approaches to, 169–72 comparator theory of, 178–82 conceptualization of, 171 diagnosing, 168 discrimination learning and, 172–78 discriminations and, 169 executive dysfunction and, 170 extinction treatment for, 183–84 high-functioning in contrast to low-functioning, 183–84 implications for interventions, 182–84 importance of discrimination learning for, 173–74 language development and, 176–77 learning theory and, 171–72 MTS procedure and, 175–76 overselective responding and, 177–78 sensitive comparator and, 181–82 theory of mind and, 170 within-stimulus learning and, 177 automatic behavior, 107 automatic processing, 160 autoshaping, 274–75, 509f. See also sign tracking C/T ratio and, 524f sexual compulsivity and, 542 sexual conditioning and, 509–10 aversive conditioning energization properties of, 423–24 response learning and, 244 smoking and, 252–53 aversive drive, 428, 431 aversive outcomes, 10t, 11, 431 avoidance, 3, 10 appetitive outcomes in contrast to, 243 attentional bias and, 62 aversive conditioning and, 244 counseling and, 257 escape in contrast to, 11 extinction and, 245 generalized fear and, 456 instrumental, 107, 109 interference effect and, 122 negative contingencies and, 11 passive, 11–12 PTSD and, 139 punishment in contrast to, 11 punishment of, 245 situational, 110 of trauma-related stimuli, 29 two-factor theory and, 8 awareness, 497–98

basal ganglia, 458 basolateral amygdala (BLA) circuit selection and, 316 dorsal raphe nucleus and, 133

24-Schachtman-Index.indd 554

INDEX drug abuse and, 203–4 fear conditioning and, 47 bed nucleus of the stria terminalis (BNST) fear and anxiety behavior and, 47–49 fear reinstatement and, 58 learned helplessness and, 135 behavial measures, 489 behavioral control, 221–28, 222f behavioral economics, 321–22, 327–28 behavioral mimicry, unconscious, 467 behavioral therapy, 113–14 behaviorism, 483–84 behaviorist-cognitive debate, 483–84 benzodiazepine, 30, 133 benzodiazepine chlorodiazepoxide (CDP), 133 benzodiazepine inverse agonists, 133–34 between-session habituation, 28 cognitive processes in, 36 fear reduction and, 34–36 implications of, 37 biases group, 359t, 361–65 hindsight, 501 modiﬁcation, 59 simulations of social, 358–70 unit, 298 binge drinking, 229 biological preparedness, 537 BLA. See basolateral amygdala blocking, 6–7 in advertising, 492–94 amphetamine and, 314 attentional shifting and, 395 attraction conditioning, 438 causal relationship detection and, 442–43, 443f compound conditioning and, 17 discounting and, 349–50 human agency and, 442–43 negative feedback and, 310–12 one-trial, 313 perceptual organization and, 387, 392–94, 392f, 393f R-O associations, with drug antagonists, 253–54 BNST. See bed nucleus of the stria terminalis body adornments, 514 body size, 385, 388 boundary conditions, 419 brain. See also neuroanatomical evidence dispositional attributions in, 367–71 imaging, 359t instrumental learning and, 9 mechanisms underlying retention of fear extinction, 35–36 brain lesions anxiety and, 49 conditioned immunomodulation and, 198 fear and, 132 brand names competition effects and, 502

2/16/2011 9:31:21 AM

INDEX contexts for, 488 as CS, 482–83 familiarity with, 487–88 placement of, 485–86, 501 bupropion, 254, 284

caffeine, 217 calculating circuits, 314 Campbell and Kraeling analog, 432–34, 433f cannabanoid reuptake inhibitors, 261 carbon monoxide (CO), 324 catechol-O-methyltransferase gene (COMT), 52 heritability, 54 polymorphisms, 53, 59 threat attention and, 63 category learning, 377 dimensional salience and, 384 of men, about women, 382–83 in men’s learning study, 383f perceptual organizing and, 377–81 role of, 396 category structure, 392–93, 392f causal attributions, 351 discounting and augmentation in, 355–56 inferred from covariation information, 351–52 ratings, 353f ratings, after discounting and augmentation, 357f sample size in, 352–54 causal relationship detection, 417, 441–45 CBT. See cognitive behavioral therapy CDP. See benzodiazepine chlorodiazepoxide central coherence theory, 170–71 central nervous system (CNS), 222 cephalic-phase responses, to food cues, 293 cessation signals, 131. See also smoking cessation childhood disintegrative disorder, 168 children food neophobia and, 290 food preferences of, 290–91 overcoming satiety in, 293 portion size and, 297 responsiveness to food cues of, 292 chlorpromazine, 154 circuit selection, 315–17 classical conditioning, 3. See also Pavlovian conditioning advertising and, 481–83, 497–98 anxiety and, 45 in anxiety disorders, 46–59 fear and, 45 features of, 6t frontal cortex in, 460–61 neuroanatomical evidence, 457–61 occasion setters and, 7 schizophrenia and, 153 sexual behavior and, 533 S-S association and, 5–6 clinical science, 376–77

24-Schachtman-Index.indd 555

555 CNS. See central nervous system CO. See carbon monoxide cocaine, 202–3 abstinence, 324 addictive liability of, 274 drug-related expectancies, 229 sexual conditioning and, 518 sign tracking and, 275 Cochrane Collaboration, 324 cognition, 500 cognitive behavioral therapy (CBT), 113–14 cognitive control behavior and, 222–23 counseling and, 256–57 cognitive disorganization, 152 cognitive dissonance, 365–67 energization and, 424 N-opponents and, 430–31 cognitive enhancers, pharmaceutical, 85 cognitive facilitation, 238 cognitive load, 105 cognitive processes in advertising, 496–97 alcohol and, 44 in anxiety disorders, 44 behaviorism in contrast to, 483–84 between-session habituation in contrast to withinsession habituation, 36 drug effects and, 222 individual differences in, 376–77 in learning, 105–6 need for, in advertising, 500 cognitive restructuring, for preventing relapse, 84 cognitive revolution, 417, 483 cognitive science, 376–77 cognitive therapy, 113 coign of vantage, 427 commitment, to counseling, 257 communication interpersonal, 417, 420–21 persuasive, 366–67 Community Reinforcement Approach (CRA) programs, 328 community support, 330 comorbidity, between alcohol and smoking, 281 comparator theory of autism spectrum disorder, 178–82 intertrial intervals and, 495 compensatory reactions, 224 competence, 430 competition, 349f, 417, 428–31 brand names and, 502 connectionist learning and, 356 dispositional attribution and, 357–58 emergent properties of, 348–51 Hobbes and, 435 N-opponents in, 429–30 person impression formation and, 358–60

2/16/2011 9:31:21 AM

556 competition (Cont’d) response, 130 social judgments and, 351–58 subtyping and, 364–65 complex social behavior, 417–18 COMT. See catechol-O-methyltransferase gene COMT Val158 Met polymorphism, 53, 63–64 conceptual-categorization account, of evaluative conditioning, 408–9 conditioned drug tolerance, 213, 218–21, 273, 283 conditioned fear backward CS and, 131 escape behavior and, 126 fear in contrast to, 115 learned helplessness and, 130–31 conditioned immunomodulation, 191–92 aversive stimuli and, 199–201 drug abuse and, 201–5 historical perspectives, 193–95 immunostimulatory agents and, 198–99 with immunosuppressive agents, 195–98 conditioned responses (CR) anxiety as, 49 conscious propositional knowledge and, 116 contextual control of, 57–58 contingency awareness in contrast to, 106 impairments to, 105 recovery of extinguished, 81, 83–84 varying, 15 verbally expressed knowledge in contrast to, 104 conditioned stimuli (CS) arrangement of, in time, 485–86 associability, 313–14 backward, 131 body adornments as, 514 brand names as, 482–83 competition between, 17 conditioning effects of, 524–25 distinctions among procedures with, 18–19 drug state as, 281 factors, in sexual conditioning, 509f, 511–12 for fear activation, 31–32 fear response to, 309 gustatory, 197, 199 intrinsic relation of, with US, 405–6 modality and semantic category of, 404 natural female features as, 514–16 olfactory, 197, 539 only, in evaluative conditioning, 403 in Pavlovian conditioning, 399 preexposure effect, 492 presentation of, in evaluative conditioning, 406 processing, 313 properties of, in advertising, 487–88 relationship to US of, 486 salience of, with overshadowing, 493 as second excitor, 90–92 second-order conditioning and, 494–95

24-Schachtman-Index.indd 556

INDEX sexually relevant, 537 within-compound associations and, 14–15 conditioned stimulus-no-outcome unconditioned stimulus (CS-noUS) associations, 7, 16 conditioned stimulus-unconditioned stimulus (CS-US) associations, 4–5, 400 awareness of, 411 causal relationship detection, 442 communication of, 408 conditioning and, 16 contextual cues and, 16 evaluative conditioning and, 402 event-memory model and, 13–14 experimental extinction in, 80–81 human agency, 442 implemented in evaluative conditioning, 404–8 interval, 523 misattributions in, 409–10 omission training and, 8 other effects of, 407–8 statistical properties of, 402–3 conditioned suppression paradigms, 58 conditioning. See also classical conditioning; evaluative conditioning; fear conditioning; instrumental conditioning; instrumental escape conditioning; interoceptive conditioning; Pavlovian conditioning; sexual conditioning abnormalities, anxiety disorders and, 49 during advertising, 484–85 anxiety, 53 in attentional biases, 68 attraction, 437–38 causal relationship detection, 441–42 compound, 17–18 context, 57–58, 512–14 contextual cues during, 16–17 CS-US associations and, 16 developments in, 4–5 discriminative, 50 between drug, 272–74 escape, 426–27 evaluative, 35 excitatory in contrast to inhibitory, 51 expectancy, 35 ﬁeld, 539–40 higher order, 15 human agency, 441–42 number of trials for, 488–89 observational, 57 overlap between types of, 5–6 paradigms, 49–51 representation-mediated, 501 representations in, 13–15 research, 445–47 retention, in advertising, 490 second-order, 15, 17, 180, 494–95 simple, 49–50 symmetry of, 5–6

2/16/2011 9:31:21 AM

INDEX theory, 483–84 trace, 252, 485 conditioning-extinction theories, of discrimination learning, 172 conﬂict, 424 response, 259 connectionist learning, 346–48 causal attribution and, 354 cognitive dissonance and, 365–66 dispositional attribution and, 355 persuasive communication and, 366–67 simulations, 358–70 conscious belief, 104 consensus, 352 coding schemes, 369t–370t manipulating, 353–54 neuroimaging of, 368–71 consistency, 368–71, 369t–370t constraint satisfaction models, 346 contests, 322, 325–26 context acquisition, 94 in addiction, 249–50 for attraction, 441 for brand names, 488 conditioned drug tolerance and, 273 conditioning, 57–58, 512–14 of CR, 57–58 enhancing, 96 for evaluative conditioning, 407–8 of extinction, 94 extinction in contrast to acquisition, 94 extinction retention and, 36 incentives and, 330 manipulations, 15 multiple, in extinction, 86–88, 248–49 multiple, in massive extinction, 88–89 reducing, 253, 260 renewal and, 247 temporal, 83, 96 test, 96 unpredictability and, 58–59 contextual cues in advertising, 495–96 during conditioning, 16–17 sexual conditioning, 512–14, 512f US preexposure and, 492 contiguity, 486 contingencies in CS-US associations, 402–3 deposit, 325 escape, 131 instrumental, 10–13, 10t management, 254–55 masking, 420 negative, 10–11 positive, 10–11 punishment, 12

24-Schachtman-Index.indd 557

557 smoking cessation and, 322 stimulus-outcome, 106 contingency awareness, 67 in advertising, 497–98 CR in contrast to, 106 contrast, 360–61, 361f contrast effects, to stimuli, 243–44 controllability, 128f, 130 controlled processing, 160 coping, 257 anxiety and, 108 appraisal, 108 copulation, 510, 511f with artiﬁcial objects, 517 efﬁciency, 520 correlation, illusory, 362–64, 362f, 363f correlation, positive, 10 corticosterones, 129–30, 192 corticotrophin-releasing hormone (CRH), 134–35, 191, 193f cortisol, 193f Coturnix japonica, 508 Coturnix quail, 508 counseling, 256–58 counterconditioning, 15 aversive, 260 aversive conditioning and, 253 methadone and, 252 response conﬂict and, 259 covariation, 348, 351–52 CR. See conditioned responses craving, 256–57 CRH. See corticotrophin-releasing hormone CS. See conditioned stimuli CS+. See positive conditioned stimulus CS-. See negative conditioned stimulus CS-0. See nominal target stimulus CsA. See cyclosporine-A CS-noUS associations. See conditioned stimulus-nooutcome unconditioned stimulus associations CS-US associations. See conditioned stimulusunconditioned stimulus associations C/T ratio, 523–24, 524f cued go/no-go task, 225 cue-exposure therapy, 281–82 cues. See also food cues drug, 201, 203, 215, 241 exposure, 255–56 extinction, 89–90, 90f reminder, 128–29, 140–41 retraining of extinguished, 81 retrieval, 89–90, 249 salience of, 182 second CS, accompanying a CS during extinction, 90–92 social, 454–55 CY. See cyclophosphamide cyclophilins, 197

2/16/2011 9:31:22 AM

558 cyclophosphamide (CY), 193, 195–97 cyclosporine-A (CsA), 197 cytokines, 192, 197, 202

data preparation, 390–91 DCS. See D-cycloserine D-cycloserine (DCS), 32 in exposure therapy, 38 pharmacoptherapy, 260–61 for preventing relapse, 85 defensive behavior, 307–9 delta learning algorithm, 347–48 delusions, 159 depression attentional bias and, 61–62 hopelessness, 139 individual differences in cognitive processing and, 376 individual differences in fear conditioning and, 52 learned helplessness and, 138–39 desensitizing treatment, 16 dexamethasone, 125 DGT. See discriminated goal tracking DHHS. See Department of Health and Human Services Diagnostic and Statistical Manual of Mental Disorder, fourth edition (DSM-IV), 29, 139, 152, 168 diazepam, 135, 279 differential responding, 178 direct experience, 114 disagreement-induced drive, 423–24 discounting, 349–50 attribution ratings after, 357f in causal attribution, 355–56 in dispositional attribution, 356–58 multiplicity of, 444–45 discriminated goal tracking (DGT), 275–78, 277f, 281 discriminated taste aversion (DTA), 275 discrimination, 6–7, 169 autism spectrum disorder and, 172–78 conditional, 175–77 conditioning-extinction theories of, 172 drug, 278 enhanced, 177 simple, 174–75 stimulus prompts and, 173–74 two-card, 181f within-compound, 180 disengagement attentional bias and, 62 in nonanxious individuals, 65–66 threat, 61 dishabituation, 3, 18 disinhibited behavior, 222 disinhibition alcohol abuse and, 229 drug-related expectancies and, 228 of restrained eating, 299–300 single-stimulus presentation and, 18–19

24-Schachtman-Index.indd 558

INDEX disordered eating, 376, 384–85 dispositional attribution, 351 connectionist learning and, 355 discounting and augmentation in, 356–58 inferred from covariation information, 351–52 neuroimaging of, 367–71 ratings, 355f, 357f sample size in, 354–55 dissociations to avoid renewal, 247 between CS-US associations and evaluative conditioning, 407–8 dual-system model of learning and, 106 dissonance. See cognitive dissonance distinctiveness, 352–54 distractibility in healthy individuals, 156 schizophrenia and, 154 distraction, 84, 105 distress tolerance, 257 disturbance, locus of, 110 disuse, theory of, 248 dlPFC. See dorsolateral prefrontal cortex DMCM, 133–34 dopamine attention and, 314 in BLA, 204 bupropion and, 254 cocaine addiction and, 274 drug-induced neural plasticity and, 235 immunostimulatory agents and, 199 neurons, 314–15 schizophrenia and, 162 dopamine-receptor antagonists, 154 dorsal periaqueductal gray (DPAG), 132–33, 138f dorsal raphe nucleus (DRN), 132–34, 137f, 142, 200 dorsolateral prefrontal cortex (dlPFC), 461 dot-probe task, 60–63, 66 Down syndrome, 174 DPAG. See dorsal periaqueductal gray drive intensity, 423, 432 DRN. See dorsal raphe nucleus drug antagonists, 253–54, 259, 311 drug replacement therapy, 250–52. See also substitution drugs, 13. See also addiction; alcohol; smoking abuse, effects of, 229 acute effects of, 221–28 altering stimulant effects of, 283–84 AM404, 261 conditioned immunomodulation and, 201–5 conditioned tolerance, 213, 218–21, 273, 283 conditioned tolerance and, 273 cues, 201, 203, 215, 241 deprivation, 240 discrimination, 278 disinhibited behavior and, 222 extinction and, 245–50 identifying triggers for, 257

2/16/2011 9:31:22 AM

INDEX incentive learning and, 236 incentive salience and, 239 incentives for, 323–24 integrated treatment for, 261–62 intransigence of use of, 251 as learned behavior, 235 long-term memory and, 238 metabolites of, 272 model situation, 215f motivation, 241 negative attention and, 256 neural plasticity induced by, 235 polydrug abuse, 280–81 psychoactive, 13 reinstatement and, 246 related expectancy, 214–21 self-administration of, 244, 259 smoking in contrast to, 243 S-R associations and, 9 substance abuse treatment and, 250–61 substitutability of, 327–28 drug states appetitive outcomes and, 279 as CS, 281 between drug conditioning with, 272–74 factors impacting, 272 as Pavlovian stimuli, 271–78 substitutions for, 282–83 transfer of occasion setting, 283f DSM-IV. See Diagnostic and Statistical Manual of Mental Disorder, fourth edition dual-system model of associative learning, 105–6 dysphoria, 59, 292

eating, 290. See also food; restrained eating disinhibited, 299–300 memory and, 295 motivations for, 293–94 quantity of, 295–300 restrained, 292 of snacks, 295 social facilitation of, 297 timing of, 292–95 eating disorders cognitive factors in, 384–85 individual differences in cognitive processing and, 376 symptomatic differences in, 385 ego threats, 300 emergent properties, 350–51 emotional learning, 454–55 procedures in contrast to processes, 456–57 research, 455–56 emotional processing indicators, 28 theory, 27–37, 31 within-session fear reduction and, 34 emotional reactions, innate in contrast to learned, 116

24-Schachtman-Index.indd 559

559 emotional regulation, 385 encoding, 169 endorphins, 125, 307, 308–9 energization, 423–24 energy, 5 environmental factors for anxiety, 111 in evaluative conditioning, 407 induction in contrast to test, 127–28 in learned helplessness, 127–28 epinephrine, 193f error correction, 305–6 attention and, 312–14 calculating circuits, 314 circuit selection and, 315–17 decremental, 312 dopamine neurons and, 314–15 perceptual-defensive-recuperative model of, 306–8 errors minimization, 348 negative, 306 signals, 315 escape. See also instrumental escape conditioning aversive conditioning and, 244 avoidance in contrast to, 11 conditioning, 426–27 contingencies, 131 dorsal raphe nucleus and, 132–33 learned helplessness and, 126–27 learning, 131 PTSD and, 141 ethanol. See alcohol evaluative conditioning, 35, 67, 401–2 advertising and, 497–98 alcohol and, 90 communication of CS-US association in, 408 conceptual-categorization account of, 408–11 context for, 407–8 CS-only in, 403 CS-US associations implemented in, 404–8 CS-US CO-occurrence in, 402–3 holistic account of, 409 intrinsic relation between CS and US in, 405–6 mechanisms of, 411–12 mental process theories, 408–11 misattribution account of, 409–10 occasion setting in, 403–4 organisms experiencing, 406–7 presentation of CS and US in, 406 propositional learning account of, 410–11 referential account of, 410 sexual conditioning and, 533–34 stimuli in, 404–6 US-only in, 403 valence of US in, 404–5 evaluative learning, 534 event-memory model, 13–14 excitatory properties, acquired, 278–80

2/16/2011 9:31:22 AM

560 excitors, second, 90–92 executive function autism spectrum disorder and, 170 PTSD and, 143 schizophrenia and, 152 expectancies, 4, 104 abstinence-related, 230 acute drug effects and, 221–28 alcohol and, 224–25, 229 conditioning, 35 conﬂicting, 227–28 drug-related, 214–21, 228–29 inappropriate, 226 inhibitory control and, 224–25 learned, 228, 230 learned anxiety and, 113 mediating, 221 outcome differences and, 220 perceptual-defensive-recuperative model and, 307 persuasive communication and, 367 placebos and, 215–16 Rd-S*, 218–21, 224 response-appropriate, 225–27, 226f R-O associations and, 239 S*d-Rd, 216–18 signals, 315 single-system propositional model and, 107–9 S-O associations and, 239 sources of, 214 S-S*d, 216 stimulus-outcome contingencies and, 106 experience-based interventions, 114. See also behavioral therapy experiential differences, 56 experimental design, in anxiety, 49–50 experimental neurosis, 79 exposure in advertising, 489 cues, 255–56 food preferences and, 291 men’s perceptions of women and, 382–84 mere, 484, 491 exposure therapy anxiety and, 84, 113 between-session fear reduction and, 34–35 beyond habituation, 84 changing environments for, 84 D-cycloserine in, 38 experimental extinction as model of, 80–81 extinction paradigms in contrast to, 27 fear reactions and, 115 fear stimuli during, 84 increasing sessions of, 84 massed, 84 nonpharmacologic tools for, 38 over time, 115 relapse models after, 81–84 safety behaviors and, 108

24-Schachtman-Index.indd 560

INDEX within-session fear reduction and, 34 extinction, 3, 49–50, 195, 245–50. See also fear extinction; massive extinction ambiguity and, 82, 249 attentional bias and, 66f aversive conditioning in contrast to, 253 avoidance in contrast to appetitive outcomes, 245 as clinical tool, 183–84 to combat overselectivity, 183–84 context of, 94 CS-US associations and, 7, 95 cue exposure and, 255–56 cues, 89–90, 90f deepening, 96 event-memory model and, 14 experimental, 80–81, 84 exposure therapy in contrast to, 27 fear activation during, 30–33 habituation in contrast to, 39n1 implications of emotional processing theory in, 37 interval after acquisition, 95 of learned fear, 457 learning, 82 moderate in contrast to massive, 89f in multiple contexts, 86–88, 248–49 neurons, 33 optimizing, 248–50 overselectivity and, 179 pharmacoptherapeutic, 260–61 protection from, 109 punishment and, 245 recovery and, 83–84 reducing recovery after, 84–96 retention, 35–36 retrieval cues from, 89–90 R-O associations, 260 salient retrieval cues for, 249 with second excitor, 90–92 sessions, spaced, 93–94 single-stimulus presentation and, 18–19 S-O associations and, 260 spaced training in, 92–93, 93f S-R associations and, 258 US and, 15–16, 94–95, 95f verbal instruction and, 105

facial expressions amygdala and, 32, 64 in animals, 465 fear and, 462 fading procedures, 334–35 families, anxiety acquisition in, 55–56 fear. See also conditioned fear; social fear learning activation, during extinction, 30–33 anxiety in contrast to, 46 between-session reduction of, 34–36 brain lesions and, 132

2/16/2011 9:31:22 AM

INDEX classical conditioning and, 45 clinical evidence for, 47 conditioned fear in contrast to, 115 conditioning abnormalities, 49 CS-induced, 309 disinhibited eating and, 300 emotional learning and, 455 etiology of, 80 facial expressions and, 462 freeze behavior as measure of, 126 generalized, 455–56 inhibition of, 29–30 latent, 112 measures of, 28 memory, 32 neural studies of, 32–33 neuroanatomical evidence in, 47–49 neurons, 33 other pathway for acquisition of, 54–56 PDR and, 307–8 processing pathways, 66 PTSD in contrast to, 29 reaction mechanisms, 115–16 reduction, 30 reinstatement of, 57–58 selective associations in learning, 56–57 signals, transmission of, 464 stimuli, 84 structures, 28, 30, 37 threat imminence and, 47 within-session reduction of, 33–34 yohimbine-induced, 31 fear conditioning, 507 analgesia and, 309 attentional biases and, 64–69 circuit selection and, 316 empirical status of, 51 with exteroceptive stimuli, 274 genetics and, 53–54 individual differences in, 51–52 learning about others and, 471 negative feedback in, 309–12, 310f nonanxious individuals and, 65–66 personality and temperament in, 53 fear extinction, 27, 29–30 brain mechanisms underlying retention of, 35–36 consolidation of, 38–39 efﬁcacy ceiling in, 37 fear structure activation in, 37 implications of, 36–39 fear learning, 456–57. See also social fear learning in animals, 464–65 instructed, 469–70 interacting pathways in, 467–69 observational, 464–69, 469f pain and, 466–67 feedback, 377. See also negative feedback eating disorders and, 385

24-Schachtman-Index.indd 561

561 feedforward networks, 347f fertility, sexually conditioned, 520–22, 522t fever, 192 FGF-2. See ﬁbroblast growth factor 2 ﬁbroblast growth factor 2 (FGF-2), 143 ﬁeld conditioning, 539–40 5-HT neurons, 132–34, 136–37 5-HTT. See serotonin transporter gene ﬂashbacks, 129 ﬂight behaviors, 306 ﬂight-or-ﬂight response, 46 fMRI. See functional magnetic resonance imaging food. See also eating memories of, 299 neophobia, 290 palatable in contrast to unpalatable, 296 portion size of, 297–98 preferences, 290–92 presentation of, 294 restricted access to, 291–92 as US, 507 variety of, 298–99 words, 291 food cues, 290 aroma, 295–96 auditory, 296–97 cephalic-phase responses, 293 food variety as, 298–99 mealtime as, 294–95 memories as, 299 normative, 297–98 palatability, 295–96 portion size as, 298 responsiveness to, 291–92 restrained eating and, 292 sensory, 295–97 social, 292 timing of eating and, 292–95 verbal, 292 visual, 292, 296 forced-choice recognition test, 66 forgetting, extinction in contrast to, 245 formalin, 309 freezing behavior, 126f, 127f fear and, 46–47 fear extinction and, 31 as measure of fear, 126 perceptual-defensive-recuperative model and, 306 frontal cortex, 460–64 functional knowledge, 401–8 functional magnetic resonance imaging (fMRI), 64, 368, 368f

GABA neurons, 133–34, 142 GAD. See generalized anxiety disorder Galileo, 434–35 gambler’s fallacy effect, 106

2/16/2011 9:31:22 AM

562 generalization, 351 ABA renewal and, 81 decrement, 91 stimulus learning and, 241 generalized anxiety disorder (GAD), 45, 58 genetics anxiety and, 52–53 emotional learning and, 456 evaluative conditioning and, 407 fear conditioning and, 51–52, 53–54 as route to anxiety, 111 threat attention and, 63–64 genital responding, 535 glucocorticoids, 193f glutamate, 235 glutamatergic output, 136, 142 goal tracking, 276, 509f, 524f. See also discriminated goal tracking group biases, 359t, 361–65 group differentiation, 361 group homogeneity, 361 group impression formation, 361

habitual behavior, 9 habituation, 3. See also between-session habituation; within-session habituation exposure therapy beyond, 84 extinction in contrast to, 39n1 protection from, 116 single-stimulus presentation and, 18 US preexposure and, 492 Hall-Pearce negative transfer, 313f hallucinations, 159 halperidol, 154 HAM model. See human associative memory model health promotion, 322–23, 335 Hebbian algorithm, 350 helplessness. See learned helplessness heritability, fear conditioning and, 53–54 heroin, 201–4, 282. See also opioids Heterosocial Perception Survey, 382 higher order conditioning, 15, 494–95 high-priority behaviors, 332–33 hippocampus circuit selection and, 316 COMT and, 53 dissociating, 460 extinction retention and, 36 learned helplessness and, 142–43 PTSD and, 142–43 histamine, 199 Hobbes, Thomas, 434–35 holistic account, of evaluative conditioning, 409 homophones, 170–71 hopelessness, 139 HPA axis. See hypothalamic-pituitary-adrenal axis

24-Schachtman-Index.indd 562

INDEX human agency, 417, 441–45, 442f human associative memory (HAM) model, 494 hyperarousal, 29, 139 hypervigilance, 142 hypothalamic-pituitary-adrenal (HPA) axis, 191–92, 193f hypothalamus, 192, 200

ICD-10. See International Classiﬁcation of Diseases, 10th Edition IL-1. See interleukin-1 illusory correlation, 362–64, 362f, 363f IL region. See infralimbic region immune system, 191–92. See also conditioned immunomodulation immunization effect interference effect and, 124 mPFCv and, 136 training, 132f immunizations, 322–23 immunostimulatory agents, 198–99 immunosuppression, 196 immunosuppressive agents, 195–98 Implicit Association Task, 67 implicit misattribution, 409 impulsivity, 223, 229 inattention, 154–55 incentive learning, 239 during drug replacement treatment, 251–52 drugs and, 236 incentives carry-over effects of, 333 clinical impact of, 324–25 community support as, 330 context and, 330 directly applied to behavior, 322–27 for drug-use problems, 323–24 factors for, 329–33 with fading procedures, 334–35 future research for, 333–34 in health promotion, 322–23, 335 for high-priority behaviors, 332–33 indirect inﬂuence of, 327–29 lasting effect of, 332 monetary, 323, 326–28 monetary in contrast to nonmonetary, 329–30 public policy, 335 reﬁning, for smoking cessation, 334–35 salience, 239 sensitization, 238, 240–41 sexual, 532–33 smoking cessation and, 322, 324–25 social, 326–27 individual difference variables, 498–501 induction, 127–28 inescapable shocks (IS), 124–26, 129, 133

2/16/2011 9:31:22 AM

INDEX infantile amnesia, 112 inﬂation effect, 56 information processing, 501–2 information representations, 350–51 infralimbic (IL) region, 135 inhibition. See also latent inhibition attraction conditioning, 438–39, 439f of behavioral control, 53, 222–25 conditioned, 259, 284–85, 483 control, 224–25 inappropriate expectancies and, 226–27 of return, 61 instrumental conditioning, 3 advertising and, 482–83 analogs, 420–35, 446t complex social behavior and, 417–18 features of, 6t Pavlovian-instrumental transfer and, 240–41 response learning and, 242–43 R-S association and, 6 instrumental escape conditioning, 418, 418t boundary conditions, 419 modeling, 418–19 reduction, 419 rules of correspondence and, 420, 421t translations, 419 instrumental learning, 105 brain system and, 9 goal-directed, 9 interdrug associations, 280–81 interference effect generality of, 123–24 learned helplessness hypothesis and, 122–23 response competition and, 130 interleukin-1 (IL-1), 192 intermittent shock, 424, 426–27, 430 internal activation, 348 International Classiﬁcation of Diseases, 10th Edition (ICD-10), 168 interoceptive conditioning acquired excitatory properties in, 278–80 drug addiction and, 278–85 interdrug associations and, 280–81 interoceptive stimuli, 270–71 feature-positive in contrast to feature negative, 271f Pavlovian conditioning with, 285 interpersonal communication, 417, 420–21 intertrial interval, 489, 495 sexual conditioning and, 526 interval schedules, 12, 12t intrinsic motivation, 331–32 introversion/extraversion measurements, 540–41 inverse overshadowing, 445 involvement, in advertising, 498–500 irrational anxiety, 109–10

24-Schachtman-Index.indd 563

563 keyhole limpet hemocyanin (KLH), 198, 200 KLH. See keyhole limpet hemocyanin knowledge associative, 106 conscious contingency, 105 conscious propositional, 116 expression of, 4 functional, 401–8 propositional, 411 verbally expressed, 104

language, 104 in anxiety interventions, 114 anxiety reactions and, 110 autism spectrum disorder and, 171 development, autism spectrum disorder and, 176–77 instructed fear learning and, 469 latencies, in response learning, 243 latent inhibition (LI), 16–17, 195 abnormal, 159 accounting for, 159–62 in advertising, 492 attenuation, 159, 161–62 basic procedure, 154–55 CS processing and, 313 dependent variables in research of, 157t–158t experiential differences and, 56 experiments, 153, 155t in healthy individuals, 156–59 masking tasks and, 155–56 naloxone and, 313f pharmacological effects on, 154 potentiated, 161–62 pre-exposure and, 241–42 rationale for exploring, 154 schizophrenia and, 154–59 single-stimulus presentation and, 18–19 theories of, 159–60 lateral nucleus, 458 learned helplessness, 108, 121–22, 134–35 alternative perspectives on, 129–31 duration of, 124–27 escape behavior and, 126–27 escape contingencies, 131 fear and, 130–31 hypothesis, 122–23 neural bases of, 141–43 neural mechanisms of, 131–38 prefrontal cortex and, 135–38 psychopathological applications of, 138–43 PTSD and, 121, 139–40 reminder cues and, 128–29 stress and, 130 learned irrelevance, 123

2/16/2011 9:31:22 AM

564 learning. See also associative learning; category learning; conditioning; connectionist learning; discrimination learning; fear learning; incentive learning; instrumental learning; social fear learning; social learning about others, 470–71 attentional, 387 autism spectrum disorder and, 169 automaticity of, 66–68 between/within-stimulus, 173 content of, 5 discrimination-reversal, 229 effects of experiential differences on, 56 evaluative, 534 extinction, 82 gradual, 348 habit, 239–40, 251 hierarchy, 174, 175t incentive, 236 latent, 4 measurements, 524 observational, 114 from others, 464–70 performance in contrast to, 258 propositional, 410–11 as reasoning, 106 referential, 410 response, 240, 242–45 reward, 458 second, 82–83 sexual conditioning and, 522–25 signal, 410 social-emotional, 472–73 S-R associations and, 111 stimulus, 236–42, 255 structure of, 5–9 types of, 236–45 unconscious, 106 within-stimulus, 177 learning acquisition phase, 31 learning data, 390–91 learning disabilities, 173–74 learning theory addiction and, 261 autism spectrum disorder and, 171–72 aversive conditioning and, 253 contemporary, 44–45 interference effect and, 123 learned helplessness and, 152 LH. See lutenizing hormone LI. See latent inhibition limbic area, 63 lipopolysaccharide (LPS), 192 lithium chloride, 194 localist representations, 350–51 locus coeruleus (LC), 133, 200 logistic-regression techniques, 379, 390 lotteries, 322, 325–26

24-Schachtman-Index.indd 564

INDEX LPS. See lipopolysaccharide lupus erythmatosus, 197 lutenizing hormone (LH), 535 lymphocytes, 197, 200

maladaptive beliefs, 110 masking tasks, 155–56 altruism and, 425 automatic processing of CS-0 and, 160 competition and, 429 load, 160–61 massive extinction, 85 moderate extinction in contrast to, 89f in multiple contexts, 88–89 sessions, 34, 38 smoking and, 248 spaced trials, 93 match-to-sample (MTS) procedure, 175–76, 176f MDS model. See multidimensional spatial model mecamylamine, 253–54 medial prefrontal cortex (mPFC) in classical conditioning, 461 dispositional attributions and, 368 extinction retention and, 35–36 PTSD and, 141–42 SKF 38393 and, 314 in social cognition, 461, 463 memory consolidation, 38–39 declarative, 240 eating and, 295 emotional, 32 episodic, 107, 469 fear, 32 fear as cognitive structure in, 28 of food, 299 illusory correlation and, 363f inaccessible, 4 long-term, 108, 238 unavailable, 4 verbal, 142–43 men perceptions of women, 381–84 sexual aggression of, 382–84 sexual arousal in, 537 mental attribution, 466 mental process theories, of evaluative conditioning, 408–11 mental state attributions, 466–67 mere exposure, 484, 491 Met allele, 53 methadone, 250–52. See also opioids as incentive, 323–24 sign tracking with, 275 methamphetamine altering stimulant effects of, 283–84 discriminated goal tracking and, 276

2/16/2011 9:31:22 AM

INDEX Pavlovian features of, 271 mind, theory of, 170 mindfulness, 257 Minnesota Heart Disease Prevention Program, 330 misattribution account, of evaluative conditioning, 409–10 modeling, logic of, 424 monetary incentives, 323, 326–30 mood disorders, 59 morphine, 125–26. See also opioids conditioned drug tolerance and, 283 conditioned immunomodulation and, 201–4 DTA and, 275 learned helplessness and, 136 motor learning systems, 315 motor skills, 218–20, 219f mPFC. See medial prefrontal cortex mPFCv. See ventral medial prefrontal cortex MTS procedure. See match-to-sample procedure multidimensional spatial (MDS) model, 378–79, 378f muscimol, 136–37 music, 490–91, 494–95, 501

nadolol, 205 naloxone, 309, 311, 311f, 313f naltrexone, 201, 311–12 natural female features, 514–16, 515f naturalistic conditioned stimuli, 527 natural killer (NK) cells, 196 aversive stimuli and, 200 poly I:C and, 199 NE. See norepinephrine “near-miss” appraisals, 114 need for cognition, 500 negative conditioned stimulus (CS-), 50 attentional bias and, 64 PTSD and, 140 negative feedback decremental error correction and, 312 in fear conditioning, 309–12 model, of fear conditioning, 310f Pavlovian conditioning and, 306 neophobia, 141, 290 nervous system, 192 neural endophenotype, 63 neural plasticity, drug-induced, 235 neuroanatomical evidence anxiety and, 47–49 for classical conditioning, 457–61 of fear extinction, 32–33 PTSD in contrast to learned helplessness, 141–43 for social cognition, 461–64 threat attention and, 63–64 neurobiological processes in anxiety disorders, 44

24-Schachtman-Index.indd 565

565 behavioral control and, 222 for evaluative conditioning, 407 fear and, 47–49 in learned helplessness effects, 131–38 of social fear learning, 471–73 neuroimaging anxiety and, 52–53 dispositional attributions and, 367–71 drug cues and, 203 fear conditioning and, 51–52 fear extinction and, 33 instructed fear learning and, 469 social judgments and, 372 neuroimmune interactions, 192–93 neurons dopamine, 314–15 extinction, 33 fear, 33 neurotransmitter release, conditioned, 274 neutralizing behavior, 109 nicotine, 239 acquired excitatory properties and, 279–80 antagonists, 253–54 discriminated goal tracking and, 276–77 between drug conditioning and, 273 learned expectancies and, 230 patch, 251 Pavlovian features of, 271 replacement therapy, 250–52, 330 social support as substitute for, 330–31 nitric oxide, 202–4 NK cells. See natural killer cells NMDA. See N-methyl-D-asparate N-methyl-D-asparate (NMDA), 32–33, 35 N-methylnaltrexone, 201 nominal target stimulus (CS-0), 155 automatic processing of, 160 LI theory and, 159–60 nonanxious individuals, 65–66 nonreinforcement, 281–82 nonresponses, 10–11 N-opponents, 429–31, 434 noradrenergic system, 131 norepinephrine (NE) bupropion and, 254 drug-induced neural plasticity and, 235 immunostimulatory agents and, 199 learned helplessness and, 134–35 in PTSD, 31 normal behavior, 152 North Karelia program, 330 nucleus accumbens, 203, 221

obesity eating motivations and, 294 portion size and, 297 responsiveness to food cues and, 291–92

2/16/2011 9:31:22 AM

566 object recognition, 107 observation anxiety reactions and, 110 audience, 424 fear learning, in animals, 464–65 fear learning, in humans, 465–67 food preferences and, 291 functional knowledge and, 401–2 learning and, 114 observational conditioning, 57 observing response procedures, 182–83 obsessive compulsive disorder learned expectancies and, 230 neutralizing behaviors and, 109 stop-signal model of behavioral control and, 223 occasion setting, 7 in evaluative conditioning, 403–4 transfer of, 283f O-LIFE. See Oxford-Liverpool Inventory of Feelings and Experiences omission training, 3, 8 as negative contingency, 11 with sexual conditioning, 509 operant conditioning. See instrumental conditioning opioids antagonists, 311 antipain mechanisms and, 306 aversive stimuli and, 200–201 conditioned immunomodulation and, 201–5 contingency management for, 254–55 learned helplessness and, 134–35 withdrawal, 252 orientation attentional bias and, 62 spatial-cueing task and, 61 outcomes, 236. See also appetitive outcomes; aversive outcomes devaluation, 9, 13 expectancies and, 220 learned associations and, 238f modulating, 242 preexposure effects, 7 in response learning, 242–43 reward, 220–21 value of, 9 OVA. See ovalbumin ovalbumin (OVA), 199 overeating, 295, 298 overgeneralization, 28 overjustiﬁcation effect, 331–32 overlearning, 248 overselectivity, 173 autism spectrum disorder and, 177–78 in comparator model, 179 extinction and, 179 extinction to combat, 183–84 interventions, 182–83 MTS procedure and, 176

24-Schachtman-Index.indd 566

INDEX sensitive comparator and, 181–82 subastymptotic performance and, 180–81 overshadowing, 7, 20n4 in advertising, 492–94 amphetamine and, 314 circuit selection and, 316 discounting and, 350 inverse, 445 negative feedback and, 311–12 one-trial, 314 overselectivity and, 180 reciprocal, 180–81 Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE), 158–59

PAG. See periaqueductal gray pain observational fear learning and, 466–67 perceptual-defensive-recuperative model and, 306–8 pain-inhibiting peptides, 306 palatability, 295–96, 298 panic, 46–47 panic disorder, 29, 58 parental modeling, 55 parsimony, 106, 483 pathological anxiety, 27 emotional processing theory and, 28 treatment of, 37 Pavlovian conditioning, 5, 79, 105, 399–401. See also classical conditioning analogs, 435–45, 446t fertilization success and, 522t with interoceptive stimuli, 285 interoceptive stimuli and, 270–71 negative-feedback and, 306 neuroimmune interactions and, 192–93 rules of correspondence and, 435–36, 436t US for, 508 Pavlovian features, 271 Pavlovian-instrumental transfer, 240–41 penile plethysmograph, 535, 535f pentobarbital, 272–73 perception of CS and US, in evaluative conditioning, 405–6 single-system propositional model and, 107 social, 468 of women, by men, 381–84 perceptual-defensive-recuperative (PDR) model, 306–8, 307f perceptual organizing blocking and, 387, 392–94, 392f, 393f category learning and, 377–81 congruence of, with category structure, 391–92, 392f in men’s learning study, 383f prototype-classiﬁcation task data and, 390 spatial representation of, 378f

2/16/2011 9:31:22 AM

INDEX of women, by women, 384–95 performance comparator model and, 179 expectancy and, 104 learning in contrast to, 258 utilizing associative knowledge, 106 periaqueductal gray (PAG), 200, 308 peripheral neuropeptide Y, 205 personality advertising and, 498–501 anxiety conditioning and, 53 emotional learning and, 456 evaluative conditioning and, 406–7 fear conditioning and, 51–53 person impression formation, 351–52, 358–61, 359t assimilation and contrast and, 360–61, 361f recurrent network model of, 360f persuasive communication, 366–67 pervasive developmental disorder not otherwise speciﬁed, 168 PET. See positron emission tomography PFC. See prefrontal cortex pharmacokinetics, 250–51 pharmacological effects of cognitive enhancers, 85 LI and, 154 pharmacotherapy, 260–61 phased-learning task, 385–87, 386f, 386t prototype-classiﬁcation task and, 388–89 phobias. See also speciﬁc phobias contextual cues and, 16 differences in learning, 54–59 nonassociative model of, 112 preparedness theory of, 56–57, 111 sensitization model of, 112 simple, 29 social, 45, 175 phobic objects, 80 photo stimuli, 388 picrotoxin, 136–37 placebos alcohol impairment and, 217–18 expectancy and, 215–16 R-O associations and, 259–60 place-conditioning apparatus, 280f plaque-forming cells, 196 PL region. See prelimbic region polydrug abuse, 280–81 poly I:C. See polyinosinic:polycytidylic polyinosinic:polycytidylic (poly I:C), 199 portion size, of food, 297–98 positive conditioned stimulus (CS+), 49–50 attentional bias and, 64 contextual conditioning, 57 PTSD and, 140 positivity effect, 487

24-Schachtman-Index.indd 567

567 positron emission tomography (PET), 141–42 posttraumatic stress disorder (PTSD), 29 amygdala and, 32 attentional biases and, 45 conditioning paradigms, 50 criteria for, 139 escape and, 141 evaluative conditioning and, 406–7 hippocampus and, 142–43 learned helplessness and, 121, 139–40 neophobia and, 141 neural bases of, 141–43 norepinephrine in, 31 preparedness theory of phobias and, 111–12 psychopathological applications of learned helplessness, 139 reminder cues, 129 single-system propositional model and, 110–11 sources of, 141 unpredictability and, 58 yohimbine in treatment of, 38 potentiation, 7 predatory imminence, 46 predictability, 130 prediction, 4 preexposure. See also latent inhibition CS, 492 effects, 7 in stimulus learning, 241–42 US, 491–92 prefrontal area, 53, 63 prefrontal cortex (PFC) in classical conditioning, 460–61 COMT and, 53 learned helpless effects and, 135–38 pregnancy, 332–33 prelimbic (PL) region, 135, 203 preparedness theory, of phobias, 56–57, 111–12 prior belief, 500–501 problem solving, 257, 385 product experience, 491 prohibition, 365, 366f prompting techniques, 173–74 propositional learning, 410–11. See also single-system propositional model prototype-classiﬁcation task, 388–90 pseudoconditioning, 3, 18–19 psychoeducation, 258 psychological space, 378, 381 psychopathology early associative accounts, 79–80 learned helplessness, 138–43 psychostimulants, 517–19 PTSD. See posttraumatic stress disorder public policy incentives, 335

2/16/2011 9:31:22 AM

568 punishment, 3 of avoidance, 245 avoidance in contrast to, 11 contingencies, 12 instrumental contingencies of, 10t passive avoidance as, 11–12

quality of life, 335

radioligand binding, 192 Rape Justiﬁability Score, 382 RASHNL model, 380–81 sexual aggression of men and, 382–84 in women’s learning study, 387 ratio schedules, 12t, 13 reacquisition, 246, 277 reaction times (RTs), 60 reasoning associative learning and, 105 faulty, 110 learning as, 106 Wason selection task and, 110 recovery evaluative conditioning and, 89–90 of extinguished CR, 81, 83–84 reducing, after extinction, 84–96 spaced extinction sessions and, 94 recuperative behavior, 308–9 recurrent networks, 347f, 358, 360f reduction, 419 redundancy, in CS-US associations, 403 reexperiencing, 139 referential account, of evaluative conditioning, 410 reﬂexive theory, 111 reinforcement, 11 altruistic, 425 amount of, 329 in competition, 429–30 correlated analogies for, 422–23, 422f delay of, 329, 421–22, 422f, 429–30, 430f drug antagonists and, 253–54 drugs and, 235–36 habit learning and, 239 instrumental contingencies of, 10t intrinsic motivation and, 331–32 negative, 8, 10 nicotine replacement and, 250–51 of non-drug behaviors, 327–29 partial, 423, 430, 486–87 positive, 10 schedules, 12, 12t, 243, 323 self-administration of drugs and, 244 of speaking in reply, 421–23 reinforcers, 305, 322 reinstatement, 246 attentional bias and, 66f

24-Schachtman-Index.indd 568

INDEX of fear, 57–58 mental, 90 of treatment context, 84 relapse associative models of, 81–84 attenuating, 97 drug cues and, 201 experimental extinction and, 84 preventing, 84–85 single-system propositional model and, 115 through substitution, 283 triggers, 241 types of, 246–48 reminder cues, 128–29, 140–41 renewal, 247 AAC, 81–83, 85 ABA, 81–83 evaluative conditioning and, 90f massive extinction and, 85 preventing, 88 ABC, 81–83 extinction in multiple contexts and, 86–88 massive extinction and, 85 types of, 81–84 representation-mediated conditioning, 501 representations, 13–15, 28, 350–51 resampling approach, to statistical analysis, 391 Rescorla-Wagner model, 4 ABC renewal and, 82 anxiety and, 109 complex social behavior and, 418 compound conditioning, 17 delta learning algorithm and, 347 one-trial blocking and, 313 US processing and, 310 response-no-outcome (R-noO) associations, 7 response-outcome (R-O) associations, 3, 6 in animals, 9 drug antagonists in blocking, 253–54 expectancies about, 239 extinction, 260 smoking and, 236–38 treatment for, 259–60 responses conﬂict, 259 evaluative, 406 learned associations and, 238f positive contingencies and, 10–11 self-administration, 240 sexually conditioned, 516–17 response-stimulus (R-S) associations, 6 restrained eating, 292, 299–300 restricted access, to food, 291–92 resurgence, 247–48 retraining, of extinguished cues, 81 retrieval cues, 89–90, 249 discriminations and, 169

2/16/2011 9:31:22 AM

INDEX facilitation procedure, 5 second learning and, 82 Rett’s disorder, 168 return-of-deposit programs, 325 rewards. See also incentives centers, 221 dopamine neurons and, 315 learning, 458 outcomes, 220–21 smoking cessation and, 322 risk taking disinhibiting effects of alcohol and, 227–28 sexual, 541–42 R-noO associations. See response-no-outcome associations R-O associations. See response-outcome associations R-S associations. See response-stimulus associations RTs. See reaction times rules of correspondence, 418–19 instrumental escape conditioning and, 420, 421t Pavlovian conditioning and, 435–36, 436f

safety behaviors, 108–10 safety signals, 58, 244 salience category learning and, 381 of CS, with overshadowing, 493 of cues, 182 dimensional, 384 incentive, 239 perceived, of affect, in women, 394 perceived dimensional, 379 weights, 379–80 “Sally-Anne” task, 170 sample size acquisition and, 348–49 in causal attribution, 352–54 in dispositional attribution, 354–55 effect, 348 manipulating, 356 satiety, 293, 298 schizophrenia, 152–54 abnormal latent inhibition and, 159 distractibility and, 154 dopamine function and, 162 individual differences in cognitive processing and, 376 irrelevant stimuli and, 160 latent inhibition effects with, 156 LI and, 154–59 LI attenuation and, 161–62 LI potentiation and, 161–62 normal behavior in contrast to, 152 positive and negative symptoms, 152, 158–59, 161–62 smoking and, 324 schizotypality, 156–59 Schizotypal Personality Questionnaire (SPQ), 158–59 SCR. See skin conductance

24-Schachtman-Index.indd 569

569 S∆ , 7 S delta. See S∆ second excitor, 90–92 second-order conditioning, 15, 17, 180, 494–95 self-control disinhibited eating and, 299 modifying, 300 self-efﬁcacy, 108 self-esteem, 139 self-injurious behavior, 183 self-judgments, 463 self-learning, 346 self-organization, 346 self-regulation, 228 self-reports, 456 sensitive comparator, 181–82 sensitization incentive, 238, 240–41 in learned helplessness, 140 model, for phobias, 112 PTSD, 140 single-stimulus presentation and, 18–19 tolerance in contrast to, 221 sensory preconditioning, 4 compound conditioning and, 17–18 within-compound associations and, 15 serotonin, 52–53, 133 serotonin transporter gene (5-HTT), 52–54 polymorphisms, 59 threat attention and, 63 SES. See Sexual Experience Scales sexual aggression individual differences in cognitive processing and, 376 of men, 382–84 sexual arousal, 535–36 in men, 537 models, 538 in women, 536–37 sexual behavior, 507–8 disinhibiting effects of alcohol and, 227–28 human, 532–33 psychostimulants and, 517–19 sexual conditioning and, 510–22 sexual compulsivity, 541–42 sexual conditioning, 275, 508–10 animal, 534–35 animal in contrast to human, 527 appetitive, 538 of approach behavior, 509f with artiﬁcial objects, 517 of body adornments, 514 contextual cues, 512–14, 512f CS factors in, 509f, 511–12 CS-US interval and, 523 effect of psychostimulants on, 517–19 fertilization success and, 520–22 ﬁeld studies of, 539–40 human, 525–26, 535–36

2/16/2011 9:31:22 AM

570 sexual conditioning (Cont’d) individual differences in, 540–41 intertrial interval and, 526 learning and, 522–25 manifestations of behavior after, 523 modiﬁcations on interactions with sexual partner, 519–20 of natural female features, 514 nature of, 533–34 olfactory CS in, 539 range of responses to, 516–17 research, 534–42, 543 sexual behavior and, 510–22 signal learning and, 533–34 US factors in, 510–11 Sexual Experience Scales (SES), 540–41 Sexual Inhibition Scales /Sexual Excitation Scales (SIS/ SES), 540–41 sexual learning, 522–25 sexual partners, 519–20 sexual risk taking, 541–42 Sexual Sensation Seeking (SSS), 540–41 sheep red blood cells (SRBC), 194 shortcut strategies, 110 sickness behaviors, 192 signal learning, 410, 533–34 sign tracking. See autoshaping similarity-ratings paradigm, 377 single-stimulus presentation, 18–19 single-system propositional model, 106–16 SKF 38393, 314 skin conductance (SCR), 53 expectancy and, 107 Implicit Association Task and, 67 Skinner box, 510 sleep, 192 smokeless tobacco, 326 smoking aversive conditioning and, 252–53 combination treatment for, 251 comorbidity with alcohol, 281 coping and, 257 decline in, 321 denicotized, 260, 278–79 drugs in contrast to, 243 extinction and, 245–50 learned expectancies and, 230 massive extinction and, 248 with nicotine patch, 251 pregnancy and, 332–33 prevalence of, 321 rapid, 252, 260 reacquisition and, 246 reducing contexts for, 253, 260 R-O associations and, 236–38 scheduled reduction of, 253 schizophrenia and, 324 smokeless tobacco and, 326

24-Schachtman-Index.indd 570

INDEX spontaneous recovery and, 247 substance abuse treatment and, 250–61 smoking cessation, 236 aversive conditioning and, 252–53 bupropion for, 254 contests and lotteries for, 325–26 counseling, 257 deposit contingencies for, 325 effectiveness of, 321–22 incentives for, 324–25 programs, 333 reﬁning incentives for, 334–35 taxes and, 326 treatment, 247 worksite interventions, 326–27 snacking, 295 S-noO associations. See stimulus-no-outcome associations S-O associations. See stimulus-outcome associations sobriety, 214. See also abstinence social analogs, 418–19 acquisition, 423 altruistic drive, 426–27 Campbell and Kraeling, 432–34, 433f causal relationship detection, 441–45 competition, 429–30 correlated reinforcement, 422–23, 422f CS-US association, 442 delayed reinforcement, 421–22, 422f dictionary of, 418–19 drive intensity, 423, 432 energization, 423–24 human agency, 441–45 instrumental conditioning, 420–35, 446t intermittent shock, 424 interpersonal attraction, 437–41 partial reinforcement, 423 Pavlovian conditioning, 435–45, 446t social cognition amygdala in, 461–62 emotional learning and, 455 frontal cortex in, 462–64 neuroanatomical bases for, 461–64 reﬂective in contrast to reﬂexive, 470 topics, 359t social connectionism, 346 social drinking, 216–17 social facilitation of eating, 297 N-opponents and, 430–31 social fear learning, 456–57 amygdala and, 472 neural model of, 459f–460f, 471–73 social judgments future research in, 371 neuroimaging and, 372 N-opponents and, 430–31 simulations, 358–70

2/16/2011 9:31:22 AM

INDEX studies, 351–58 social learning, 345–46, 371 artiﬁcial stimuli and, 396 sexual aggression and, 384 social motivation, 418 social neuroscience, 346 social perception, 468 social processes, 345–46, 371, 455 social reinforcement, 418 social skills, 171 social support, 330–31 societal status, 406 spaced training advertising and, 489 in extinction, 92–93, 93f spatial-cueing task, 61, 65 speaking in reply, 421–25 species-speciﬁc defense reactions (SSDRs), 307 speciﬁc phobias attentional biases and, 45 experimental extinction and, 80 infantile amnesia and, 112 preparedness theory of, 111 sperm depletion effect, 522 spleen denervation, 198 spontaneous recovery, 3, 7, 247 with second excitor, 91 theoretical accounts of, 83 SPQ. See Schizotypal Personality Questionnaire spreading activation models, 346 S-R associations. See stimulus-response associations SRBC. See sheep red blood cells S-S associations. See stimulus-stimulus associations SSDRs. See species-speciﬁc defense reactions SSRT. See stop signal reaction time SSS. See Sexual Sensation Seeking startle reﬂex, 47–49 contextually modulated, 59 fear-potentiated, 47, 57 potentiation of, 55f sensitization and, 140 statistical analysis, resampling approach to, 391 stereotype-inconsistent information, 364–65, 364f stereotyping, 361 stimulant effects, 283–84 stimuli. See also interoceptive stimuli artiﬁcial, 396 aversive, 199–201 category learning and, 380–81 contrast effects to, 243–44 differentiation, 173 dimensional metrics for, 378–79 drug states as, 271–78 environmental, 8 in evaluative conditioning, 404–6 exteroceptive, 274 fear, 84 irrelevant, 160

24-Schachtman-Index.indd 571

571 learned associations and, 238f local in contrast to global processing of, 177 naturalistic conditioned, 527 nondrug, during withdrawal, 251 Pavlovian, 271–78 perceptual organization of, 378 phobic, 111 photo, 388 pre-exposure, 241–42 prompts, 173–74 social in contrast to nonsocial, 440–41 threat-relevant, 44–45 trauma-related, 29 unattended, 162n3 stimulus-no-outcome (S-noO) associations, 7 stimulus-outcome (S-O) associations, 3, 6 cue exposure and, 255 expectancies about, 239 treatment for, 260 stimulus-response (S-R) associations, 3 addiction and, 240 in animals, 8–9 cue exposure and, 255 drugs and, 9, 215 learning and, 111 treatment and, 258–59 stimulus-stimulus (S-S) associations classical conditioning, 5–6 implicit misattribution in, 409 stop-signal model, of behavioral control, 222–23 stop signal reaction time (SSRT), 223 stress attenuated LI and, 159 controllability of, 128f corticosteroids and, 130 disinhibited eating and, 299–300 emotional learning and, 456 induced analgesia, 308 learned helplessness and, 130 responsiveness to food cues and, 292 time, 424 wound healing and, 200 stress-induced ulceration paradigms, 58 striatum, 458, 473 Stroop task, emotional, 60 substance abuse. See also addiction; alcohol; drugs memory consolidation and, 38–39 sexual compulsivity and, 542 treatment, 250–61 substitutability, 327–28 substitution. See also drug replacement therapy drugs, 327–28 functional in contrast to pharmacological, 282–83 nicotine replacement as, 330 social support as, 330–31 subtyping, 364–65 summation, 82 summation and retardation test, 284

2/16/2011 9:31:22 AM

572 superconditioning, 350 attraction, 439–40, 440f causal relationship detection and, 443–44, 443f human agency and, 443–44 support groups, 327 surprise, 310 symbolic representations, 350 sympathetic nervous system, 46

taste aversion, 193–94, 275 taxes, for smoking cessation, 326 teaching, 5 team competition, 327 temperament, 53 temporal weighting, 83 temporo-parietal junction (TPJ), 368 terminology, 48t, 237t testability, of dual-system model of learning, 105–6 test environments, 127–28 testosterone, 535 thalamus, 458 theory of mind, 170 threat appraisal, 108, 115 attentional biases for, 44–45, 59–64 cognitive dissonance and, 365 detection, 67 disengagement, 61 ego, 300 emotional learning and, 454 imminence, 47 prohibition and, 366f responsiveness, 63 stimuli, 44–45 threat beliefs, 104 reducing, 116 single-system propositional model and, 107–9 time stress, 424 tobacco. See smoking tolerance alcohol, 213, 218–21 conditioned drug, 273–74 distress, 257 drug, 221 trace conditioning, 252, 485 training interval in contrast to ratio, 243 reinforcement schedules and, 243 trait anxiety, 53 amygdala and, 63 anxiety disorders in contrast to, 69 attentional biases and, 59, 68–69 trait-implying behavior, 360 transfer affect, 496 data, 390 of occasion setting, 283f

24-Schachtman-Index.indd 572

INDEX Pavlovian-instrumental, 240–41 positive, 4 rules, 123 trauma coping with, 144 PTSD and, 29 treatment anxiety, 37, 113–15 autism spectrum disorder, 183–84 combination, 251 context, reinstatement of, 84 desensitizing, 16 drug replacement, 250–52 integrated addiction, 261–62 novel, 258 pharmacotherapy, 260–61 psychosocial, 256–58 PTSD, 38 for R-O associations, 259–60 smoking cessation, 247 for S-O associations, 260 spacing of, 247 for S-R associations, 258–59 substance abuse, 250–61 Treatment Episode Data Set, 281 triadic design, 122t, 125 triggers identifying, 257 relapse, 241 trinitrophenyl (TNP), 196 two-card discrimination procedure, 175f two-factor theory, 7–8

unconditioned stimuli (US) arrangement of, in time, 485–86 causal relationship detection and, 442 during extinction, 94–95, 95 extinction and, 15–16 factors, in sexual conditioning, 510–11 food as, 507 human agency and, 442 intrinsic relation of, with CS, 405–6 modality and semantic category of, 404 only, in evaluative conditioning, 403 in Pavlovian conditioning, 399, 507 postconditioning presentations of, 15–16 postextinction presentations of, 81 preexposure, in advertising, 491–92 presentation of, in evaluative conditioning, 406 processing, 310, 313–14 reinstatement, 16 relationship to CS of, 486 valence of, 404–5 unconscious processes, 107 unpredictability, 58–59 US. See unconditioned stimuli utilization coefﬁcients, 380

2/16/2011 9:31:22 AM

INDEX vaginal photoplethysmograph, 535, 536f vaginal pulse amplitude (VPA), 535 Val158 allele, 53 valence, of US, 404–5 varenicline, 254 variety effect, 298–99 ventral medial prefrontal cortex (mPFCv), 135–36, 138f, 141–42 verbal instruction, 105 verbal interventions, 114. See also cognitive therapy veterans, 141–42 virtual reality exposure (VRE), 38 visual search tasks, 61, 67 vouchers, 326–28 VPA. See vaginal pulse amplitude VRE. See virtual reality exposure

Wason selection task, 110 withdrawal from drugs, 235–36

24-Schachtman-Index.indd 573

573 nicotine, 239 nondrug stimuli during, 251 opioids, 252 within-compound associations, 14–15 within-session habituation, 28 cognitive processes in, 36 efﬁcacy of, 38 fear reduction and, 33–34 implications of, 37 women appetitive sexual conditioning in, 538 attentional shifts in, 394 eating disorders and, 384–85 men’s perceptions of, 381–84 perceived salience of affect in, 394 perception of women by, 384–95 sexual arousal in, 536–37

yohimbine, 31, 38, 261

2/16/2011 9:31:22 AM